dinsdag 4 september 2018

Template Compiler Update

Hi everybody. After samcv++ released MoarVM again, it was finally time to introduce some updates to the expression template compiler.

As you may or may not know, the expression JIT backend maps MoarVM opcodes to its internal representation via expression templates. At runtime (during the JIT compilation phase), these templates are combined to form an expression tree. The expression template language remains somewhat underdocumented, but quite a few brave developers have still ventured to add templates. With the new template compiler, a few things will change:
  • Template syntax and type checking is much stricter. This means that, for example, it is no longer possible to place a C-macro expression (which evaluates to a constant value) where an expression operand (i.e. code) would be expected.
  • Templates can contain references to MoarVM opcode operands (written as $0, $1, $2 etc.) At runtime, read operand references are substituted with the value of the operand, and write operands with the address of the register they refer to. This was a source of confusion, so to reduce this all write operands must now be written with a '\' prefix, like \$0.
  • At the same time, the '!' suffix on a template name (like slice!) indicate that the template code does not return the value of the output, but overwrite the register where it is stored. (It tells the expression compiler that the value in that output register must be loaded from memory the next time it is used). Many templates that do not yield an output (like unshift)  had this suffix, but although that is harmless at runtime, it is nevertheless confusing, so the template compiler will now raise an error if it finds this.
  • Last but not least, macro application is now hygienic, meaning that a name declared in a macro will never conflict with a name declared in a template. So something like this should now work, even though it is nonsense:

(macro: ^foo (,bar)
  (let (($obj ...))
    (add $obj ,bar)))
(template: foobar
  (let (($obj ...))
    (^foo $obj))

Prior to these patches, this would expand to something like:

(template: foobar
  (let (($obj ...))
    (let (($obj ...)) # name conflict is here
      (add $obj $obj))) # which $obj?

Although this code would previously fail during compilation, having to deal with such conflicts in remote code segments is annoying. But now named declarations are resolved prior to expanding the inner macro, and this means that the $obj for macro ^foo is resolved to the let: within that macro, and the outer $obj in the template refers to the outer declaration, as if you'd written:

(template: foobar
  (let (($obj1 ...))
    (let (($obj2 ...))
      (add $obj2 $obj1)))

I believe that this makes writing templates safer and simpler, catching more errors at compile time and fewer at runtime. Happy hacking!