This project has moved. For the latest updates, please go here.

Compiling into assembly (dll)

Apr 14, 2012 at 8:26 PM

Hi,

I little explore this parser/compiler and I think it is very nice component. Only think I miss is/was compiling into assembly which I can save to disk and later reuse.

So... I added like this functionality. If someone will need same, see below  (I did from jurassic-39608d99435c.zip\jurassic version).

 

Have a nice day

Marek

 

Compiler/Expressions/FunctionExpression.cs
***************
*** 76,82 ****
              }
 
              // Store the generated method in the cache.
!             long generatedMethodID = GeneratedMethod.Save(this.context.GeneratedMethod);
 
              // Create a UserDefinedFunction.
 
--- 76,82 ----
              }
 
              // Store the generated method in the cache.
!                 long generatedMethodID=this.context.Engine.SaveMethod(this.context.GeneratedMethod);
 
              // Create a UserDefinedFunction.
 
***************
*** 106,111 ****
--- 106,112 ----
              generator.LoadString(this.BodyText);
 
              // body
+                 EmitHelpers.LoadScriptEngine(generator);
              generator.LoadInt64(generatedMethodID);
              generator.Call(ReflectionHelpers.GeneratedMethod_Load);
 
=====================================================
Compiler/MethodGenerator/CompilerOptions.cs
***************
*** 35,40 ****
--- 35,42 ----
              set;
          }
 
+           public string Path { get; set; }
+
          /// <summary>
          /// Performs a shallow clone of this instance.
          /// </summary>
=====================================================
Compiler/MethodGenerator/GeneratedMethod.cs
***************
*** 47,120 ****
              get;
              set;
          }
-
-
-
-         private static Dictionary<long, WeakReference> generatedMethodCache;
-         private static object cacheLock = new object();
-         private static long generatedMethodID;
-         private const int compactGeneratedCacheCount = 100;
-
-         /// <summary>
-         /// Retrieves the code for a generated method, given the ID.  For internal use only.
-         /// </summary>
-         /// <param name="id"> The ID of the generated method. </param>
-         /// <returns> A <c>GeneratedMethodInfo</c> instance. </returns>
-         public static GeneratedMethod Load(long id)
-         {
-             lock (cacheLock)
-             {
-                 if (generatedMethodCache == null)
-                     throw new InvalidOperationException("Internal error: no generated method cache available.");
-                 WeakReference generatedMethodReference;
-                 if (generatedMethodCache.TryGetValue(id, out generatedMethodReference) == false)
-                     throw new InvalidOperationException(string.Format("Internal error: generated method {0} was garbage collected.", id));
-                 var generatedMethod = (GeneratedMethod)generatedMethodReference.Target;
-                 if (generatedMethod == null)
-                     throw new InvalidOperationException(string.Format("Internal error: generated method {0} was garbage collected.", id));
-                 return generatedMethod;
-             }
-         }
-
-         /// <summary>
-         /// Saves the given generated method and returns an ID.  For internal use only.
-         /// </summary>
-         /// <param name="generatedMethod"> The generated method to save. </param>
-         /// <returns> The ID that was associated with the generated method. </returns>
-         public static long Save(GeneratedMethod generatedMethod)
-         {
-             if (generatedMethod == null)
-                 throw new ArgumentNullException("generatedMethod");
-             lock (cacheLock)
-             {
-                 // Create a cache (if it hasn't already been created).
-                 if (generatedMethodCache == null)
-                     generatedMethodCache = new Dictionary<long, WeakReference>();
-
-                 // Create a weak reference to the generated method and add it to the cache.
-                 long id = generatedMethodID;
-                 var weakReference = new WeakReference(generatedMethod);
-                 generatedMethodCache.Add(id, weakReference);
-
-                 // Increment the ID for next time.
-                 generatedMethodID++;
-
-                 // Every X calls to this method, compact the cache by removing any weak references that
-                 // point to objects that have been collected.
-                 if (generatedMethodID % compactGeneratedCacheCount == 0)
-                 {
-                     // Remove any weak references that have expired.
-                     var expiredIDs = new List<long>();
-                     foreach (var pair in generatedMethodCache)
-                         if (pair.Value.Target == null)
-                             expiredIDs.Add(pair.Key);
-                     foreach (int expiredID in expiredIDs)
-                         generatedMethodCache.Remove(expiredID);
-                 }
-
-                 // Return the ID that was allocated.
-                 return id;
-             }
-         }
      }
  }
--- 47,51 ----
=====================================================
Compiler/MethodGenerator/MethodGenerator.cs
***************
*** 1,5 ****
--- 1,10 ----
  using System;
  using System.Collections.Generic;
+ using System.IO;
+ using System.Reflection;
+ using System.Reflection.Emit;
+ using Jurassic;
+ using sysEmit=System.Reflection.Emit;
 
  namespace Jurassic.Compiler
  {
***************
*** 167,173 ****
              optimizationInfo.MethodOptimizationHints = this.MethodOptimizationHints;
 
              ILGenerator generator;
!             if (this.Options.EnableDebugging == false)
              {
                  // DynamicMethod requires full trust because of generator.LoadMethodPointer in the
                  // FunctionExpression class.
--- 172,178 ----
              optimizationInfo.MethodOptimizationHints = this.MethodOptimizationHints;
 
              ILGenerator generator;
!             if ((this.Options.EnableDebugging == false)&&(this.Options.Path==null))
              {
                  // DynamicMethod requires full trust because of generator.LoadMethodPointer in the
                  // FunctionExpression class.
***************
*** 222,236 ****
  #if WINDOWS_PHONE
                  throw new NotImplementedException();
  #else
!                 // Debugging or low trust path.
                  ScriptEngine.ReflectionEmitModuleInfo reflectionEmitInfo = this.Engine.ReflectionEmitInfo;
                  if (reflectionEmitInfo == null)
                  {
                      reflectionEmitInfo = new ScriptEngine.ReflectionEmitModuleInfo();
 
                      // Create a dynamic assembly and module.
!                     reflectionEmitInfo.AssemblyBuilder = System.Threading.Thread.GetDomain().DefineDynamicAssembly(
!                         new System.Reflection.AssemblyName("Jurassic Dynamic Assembly"), System.Reflection.Emit.AssemblyBuilderAccess.Run);
 
                      // Mark the assembly as debuggable.  This must be done before the module is created.
                      var debuggableAttributeConstructor = typeof(System.Diagnostics.DebuggableAttribute).GetConstructor(
--- 227,248 ----
  #if WINDOWS_PHONE
                  throw new NotImplementedException();
  #else
!                     string filepath=this.Options.Path;
!                     string f=Path.GetFileName(filepath);
!
!                     // Debugging or low trust path.
                  ScriptEngine.ReflectionEmitModuleInfo reflectionEmitInfo = this.Engine.ReflectionEmitInfo;
+                      bool rootLvl=reflectionEmitInfo==null;
                  if (reflectionEmitInfo == null)
                  {
                      reflectionEmitInfo = new ScriptEngine.ReflectionEmitModuleInfo();
 
                      // Create a dynamic assembly and module.
!                         reflectionEmitInfo.AssemblyBuilder = filepath==null
!                             ?System.Threading.Thread.GetDomain().DefineDynamicAssembly(
!                                 new System.Reflection.AssemblyName("Jurassic Dynamic Assembly"),System.Reflection.Emit.AssemblyBuilderAccess.Run)
!                             :System.Threading.Thread.GetDomain().DefineDynamicAssembly(
!                                 new System.Reflection.AssemblyName("Jurassic Dynamic Assembly"+"-"+GetMethodName()),System.Reflection.Emit.AssemblyBuilderAccess.RunAndSave,Path.GetDirectoryName(filepath));
 
                      // Mark the assembly as debuggable.  This must be done before the module is created.
                      var debuggableAttributeConstructor = typeof(System.Diagnostics.DebuggableAttribute).GetConstructor(
***************
*** 242,248 ****
                                  System.Diagnostics.DebuggableAttribute.DebuggingModes.Default }));
 
                      // Create a dynamic module.
!                     reflectionEmitInfo.ModuleBuilder = reflectionEmitInfo.AssemblyBuilder.DefineDynamicModule("Module", this.Options.EnableDebugging);
 
                      this.Engine.ReflectionEmitInfo = reflectionEmitInfo;
                  }
--- 254,262 ----
                                  System.Diagnostics.DebuggableAttribute.DebuggingModes.Default }));
 
                      // Create a dynamic module.
!                     reflectionEmitInfo.ModuleBuilder = filepath==null
!                               ?reflectionEmitInfo.AssemblyBuilder.DefineDynamicModule("Module")
!                               :reflectionEmitInfo.AssemblyBuilder.DefineDynamicModule("Module",f, false);
 
                      this.Engine.ReflectionEmitInfo = reflectionEmitInfo;
                  }
***************
*** 275,286 ****
                      generator.MarkSequencePoint(optimizationInfo.DebugDocument, new SourceCodeSpan(1, 1, 1, 1));
                  }
                  GenerateCode(generator, optimizationInfo);
!                 generator.Complete();
 
                  // Bake it.
                  var type = typeBuilder.CreateType();
                  var methodInfo = type.GetMethod(this.GetMethodName());
                  this.GeneratedMethod = new GeneratedMethod(Delegate.CreateDelegate(GetDelegate(), methodInfo), optimizationInfo.NestedFunctions);
  #endif //WINDOWS_PHONE
              }
 
--- 289,306 ----
                      generator.MarkSequencePoint(optimizationInfo.DebugDocument, new SourceCodeSpan(1, 1, 1, 1));
                  }
                  GenerateCode(generator, optimizationInfo);
!                      generator.Complete();
 
                  // Bake it.
                  var type = typeBuilder.CreateType();
                  var methodInfo = type.GetMethod(this.GetMethodName());
                  this.GeneratedMethod = new GeneratedMethod(Delegate.CreateDelegate(GetDelegate(), methodInfo), optimizationInfo.NestedFunctions);
+
+                      if ((rootLvl)&&(filepath!=null))
+                      {
+                          CreateAssemblyExecutor(reflectionEmitInfo);
+                          reflectionEmitInfo.AssemblyBuilder.Save(f,PortableExecutableKinds.ILOnly,ImageFileMachine.I386);
+                      }
  #endif //WINDOWS_PHONE
              }
 
***************
*** 350,355 ****
          {
              return typeof(Func<ScriptEngine, Scope, object, object>);
          }
-     }
 
  }
\ No newline at end of file
--- 370,395 ----
          {
              return typeof(Func<ScriptEngine, Scope, object, object>);
          }
 
+         void CreateAssemblyExecutor(ScriptEngine.ReflectionEmitModuleInfo emitInfo)
+         {
+             Type assemblyExecutorType=typeof(AssemblyExecutor);
+
+             TypeBuilder typeBuilder=emitInfo.ModuleBuilder.DefineType("AssemblyExecutorSingle",TypeAttributes.Public|TypeAttributes.Class);
+             FieldBuilder fieldBuilder=typeBuilder.DefineField("_ae",assemblyExecutorType,FieldAttributes.Private|FieldAttributes.Static);
+
+             ConstructorBuilder constructorBuilder=typeBuilder.DefineConstructor(MethodAttributes.Public|MethodAttributes.Static,CallingConventions.Standard,Type.EmptyTypes);
+             sysEmit.ILGenerator gen = constructorBuilder.GetILGenerator();
+             gen.Emit(OpCodes.Newobj, assemblyExecutorType.GetConstructor(Type.EmptyTypes));
+             gen.Emit(OpCodes.Stsfld, fieldBuilder);
+             gen.Emit(OpCodes.Ret);
+
+             MethodBuilder methodBuilder = typeBuilder.DefineMethod("GetExecutor", MethodAttributes.Public|MethodAttributes.Static,typeof(AssemblyExecutor),Type.EmptyTypes);
+             gen = methodBuilder.GetILGenerator();
+             gen.Emit(OpCodes.Ldsfld, fieldBuilder);
+             gen.Emit(OpCodes.Ret);
+
+             typeBuilder.CreateType();
+         }
+     }
  }
\ No newline at end of file
=====================================================
Core/ReflectionHelpers.cs
***************
*** 238,244 ****
              ClrInstanceWrapper_Constructor = GetConstructor(typeof(ClrInstanceWrapper), typeof(ScriptEngine), typeof(object));
              Decimal_Constructor_Double = GetConstructor(typeof(decimal), typeof(double));
 
!             GeneratedMethod_Load = GetStaticMethod(typeof(GeneratedMethod), "Load", typeof(long));
              ClrInstanceWrapper_GetWrappedInstance = GetInstanceMethod(typeof(ClrInstanceWrapper), "get_WrappedInstance");
              Decimal_ToDouble = GetStaticMethod(typeof(decimal), "ToDouble", typeof(decimal));
              BinderUtilities_ResolveOverloads = GetStaticMethod(typeof(BinderUtilities), "ResolveOverloads", typeof(RuntimeMethodHandle[]), typeof(ScriptEngine), typeof(object), typeof(object[]));
--- 238,244 ----
              ClrInstanceWrapper_Constructor = GetConstructor(typeof(ClrInstanceWrapper), typeof(ScriptEngine), typeof(object));
              Decimal_Constructor_Double = GetConstructor(typeof(decimal), typeof(double));
 
!                 GeneratedMethod_Load=GetGenericInstanceMethod(typeof(ScriptEngine),"LoadMethod");
              ClrInstanceWrapper_GetWrappedInstance = GetInstanceMethod(typeof(ClrInstanceWrapper), "get_WrappedInstance");
              Decimal_ToDouble = GetStaticMethod(typeof(decimal), "ToDouble", typeof(decimal));
              BinderUtilities_ResolveOverloads = GetStaticMethod(typeof(BinderUtilities), "ResolveOverloads", typeof(RuntimeMethodHandle[]), typeof(ScriptEngine), typeof(object), typeof(object[]));
=====================================================
Core/ScriptEngine.cs
***************
*** 1,6 ****
--- 1,9 ----
  using System;
  using System.Collections.Generic;
  using Jurassic.Library;
+ using System.CodeDom.Compiler;
+ using Jurassic.Compiler;
+ using reflection=System.Reflection;
 
  namespace Jurassic
  {
***************
*** 660,665 ****
--- 663,704 ----
              methodGen.Execute();
          }
 
+         public CompilerResults Compile(string code,string path)
+         {
+             return Compile(new StringScriptSource(code),path);
+         }
+
+         CompilerResults Compile(ScriptSource source,string path)
+         {
+             CompilerResults result = new CompilerResults(new TempFileCollection());
+
+             CompilerOptions options=this.CreateOptions();
+             options.Path=path;
+             try
+             {
+                 var methodGen=new Jurassic.Compiler.GlobalMethodGenerator(this,source,options);
+                 methodGen.Parse();
+                 methodGen.Optimize();
+                 methodGen.GenerateCode();
+             }
+             catch (JavaScriptException ex)
+             {
+                 result.Errors.Add(new CompilerError(ex.SourcePath,ex.LineNumber,0,ex.Name,ex.Message));
+             }
+             return result;
+         }
+
+         public static AssemblyExecutor GetAssemblyExecutor(reflection.Assembly assembly)
+         {
+             return (AssemblyExecutor)assembly.GetType("AssemblyExecutorSingle").GetMethod("GetExecutor", reflection.BindingFlags.Public | reflection.BindingFlags.Static).Invoke(null, new object[0]);
+         }
+
+         public static object Execute(reflection.Assembly assembly)
+         {
+             return GetAssemblyExecutor(assembly).Execute();
+         }
+
+
          /// <summary>
          /// Verifies the generated byte code.
          /// </summary>
***************
*** 1072,1076 ****
--- 1111,1176 ----
                  return this.staticTypeWrapperCache;
              }
          }
+
+           private Dictionary<long,WeakReference> generatedMethodCache;
+           private long generatedMethodID;
+           private const int compactGeneratedCacheCount=100;
+
+           /// <summary>
+           /// Retrieves the code for a generated method, given the ID.  For internal use only.
+           /// </summary>
+           /// <param name="id"> The ID of the generated method. </param>
+           /// <returns> A <c>GeneratedMethodInfo</c> instance. </returns>
+           public GeneratedMethod LoadMethod(long id)
+           {
+                 if (generatedMethodCache==null)
+                     throw new InvalidOperationException("Internal error: no generated method cache available.");
+                 WeakReference generatedMethodReference;
+                 if (generatedMethodCache.TryGetValue(id,out generatedMethodReference)==false)
+                     throw new InvalidOperationException(string.Format("Internal error: generated method {0} was garbage collected.",id));
+                 var generatedMethod=(GeneratedMethod)generatedMethodReference.Target;
+                 if (generatedMethod==null)
+                     throw new InvalidOperationException(string.Format("Internal error: generated method {0} was garbage collected.",id));
+                 return generatedMethod;
+           }
+
+           /// <summary>
+           /// Saves the given generated method and returns an ID.  For internal use only.
+           /// </summary>
+           /// <param name="generatedMethod"> The generated method to save. </param>
+           /// <returns> The ID that was associated with the generated method. </returns>
+           public long SaveMethod(GeneratedMethod generatedMethod)
+           {
+               if (generatedMethod==null)
+                   throw new ArgumentNullException("generatedMethod");
+
+               // Create a cache (if it hasn't already been created).
+                 if (generatedMethodCache==null)
+                     generatedMethodCache=new Dictionary<long,WeakReference>();
+
+                 // Create a weak reference to the generated method and add it to the cache.
+                 long id=generatedMethodID;
+                 var weakReference=new WeakReference(generatedMethod);
+                 generatedMethodCache.Add(id,weakReference);
+
+                 // Increment the ID for next time.
+                 generatedMethodID++;
+
+                 // Every X calls to this method, compact the cache by removing any weak references that
+                 // point to objects that have been collected.
+                 if (generatedMethodID%compactGeneratedCacheCount==0)
+                 {
+                     // Remove any weak references that have expired.
+                     var expiredIDs=new List<long>();
+                     foreach (var pair in generatedMethodCache)
+                         if (pair.Value.Target==null)
+                             expiredIDs.Add(pair.Key);
+                     foreach (int expiredID in expiredIDs)
+                         generatedMethodCache.Remove(expiredID);
+                 }
+
+                 // Return the ID that was allocated.
+                 return id;
+           }
      }
  }
=====================================================
Jurassic.csproj
***************
*** 109,114 ****
--- 111,117 ----
      <Compile Include="Compiler\Statements\ForStatement.cs" />
      <Compile Include="Compiler\Statements\Statement.cs" />
      <Compile Include="Compiler\Debugging\SourceCodeSpan.cs" />
+     <Compile Include="Core\AssemblyExecutor.cs" />
      <Compile Include="Core\BigInteger.cs" />
      <Compile Include="Core\CompatibilityMode.cs" />
      <Compile Include="Core\ConcatenatedString.cs" />
=====================================================
+Core\AssemblyExecutor.cs
***************
using System;
using System.Reflection;
using System.Collections.Generic;
using Jurassic.Compiler;
using Jurassic.Library;

namespace Jurassic
{
    public class AssemblyExecutor
    {
        ScriptEngine _se;
        MethodInfo _globalMethodInfo;
        object[] _globalMethodInfoParms;

        public AssemblyExecutor():this(Assembly.GetCallingAssembly())
        {}

        public AssemblyExecutor(Assembly assembly)
        {
            Type[] types=assembly.GetTypes();
            Array.Resize(ref types, types.Length - 1);
            Array.Sort(types,delegate(Type x,Type y) { return int.Parse(x.Name.Substring(15)).CompareTo(int.Parse(y.Name.Substring(15))); });

            _se=new ScriptEngine();
            _se.EnableDebugging=false;

            bool first=true;
            foreach (Type item in types)
                if (first)
                    first=false;
                else
                    _se.SaveMethod(new GeneratedMethod(Delegate.CreateDelegate(typeof(FunctionDelegate),item.GetMethods(BindingFlags.Static|BindingFlags.Public)[0]),new List<GeneratedMethod>()));

            _globalMethodInfo=types[0].GetMethod("global_");
            _globalMethodInfoParms=new object[] { _se,_se.GetType().GetMethod("CreateGlobalScope",BindingFlags.Instance|BindingFlags.NonPublic).Invoke(_se,new object[0]),null };
        }

        public object Execute()
        {
            //this is very slow
            return _globalMethodInfo.Invoke(null,_globalMethodInfoParms);
        }

        public ScriptEngine ScriptEngine
        {
            get { return _se; }
        }
    }
}

Coordinator
Apr 19, 2012 at 1:29 PM

Thanks for your efforts!  A couple of people have requested this functionality, but as I think you've discovered, it's not an easy thing to get right :-)

I will look at integrating your code into Jurassic the next chance I get.  Sadly I have quite a few big projects on the go at the moment, so it may take me quite a while.

May 6, 2012 at 5:58 PM

Hi,

I got private mail, some1 was interested in save assembly to disk. First people and... first bug. I explored exception and I think problem is in methods cache. Try write script which have 200 (exactly: more then 100) - if you really need so many functions 1) you are masochist 2) you get exception.

To solve this, replace LoadMethod/SaveMethod methods with this (see below) code:

          private Dictionary<long, GeneratedMethod> generatedMethodCache;
          private long generatedMethodID;

          public GeneratedMethod LoadMethod(long id)
          {
              GeneratedMethod generatedMethod;
              if ((generatedMethodCache==null)||(!generatedMethodCache.TryGetValue(id, out generatedMethod)))
                  throw new InvalidOperationException(string.Format("Internal error: generated method {0} not saved.", id));
              return generatedMethod;
          }

          public long SaveMethod(GeneratedMethod generatedMethod)
          {
              if (generatedMethod == null)
                  throw new ArgumentNullException("generatedMethod");

              // Create a cache (if it hasn't already been created).
              if (generatedMethodCache == null)
                  generatedMethodCache = new Dictionary<long, GeneratedMethod>();

              // Create a weak reference to the generated method and add it to the cache.
              long id = generatedMethodID;
              generatedMethodCache.Add(id, generatedMethod);

              // Increment the ID for next time.
              generatedMethodID++;

              // Return the ID that was allocated.
              return id;
          }

Jul 16, 2013 at 8:35 AM
Using above code how can I generate library targeting to WIndows Phone Devices? I wanted to compile JavaScript at build time and then wanted to use this library from Windows Phone Devices.