fengari

daurnimator

October 2017

What is fengari?

  • Lua VM for the browser
  • Not a transpiler
  • Port of PUC-Rio Lua 5.3 implementation to ES6

Why

  • Javascript is not the best language to work in
  • Lua coroutines
  • Browser is most common platform

Other projects

  • brozula
  • lua.vm.js
  • moonshine
  • starlight
  • https://github.com/logiceditor-com/lua5.1.js
  • https://github.com/fiatjaf/glua
  • http://code.matthewwild.co.uk/ljs
  • https://github.com/mherkender/lua.js

Why another?

Other projects tend to have one (or two) missing pieces:

  • Imperfect DOM interoperability
  • Lack of coroutine support

JS interop

Lua is quite close to Javascript in terms of language features:

  • data types
  • same strings are equal
  • first class functions/closures
  • a single object type (sort of)
  • commonly use prototypical inheritance

GC problem

  • If you give an object to the JS DOM, you never know when the DOM is done with it.
  • Even with finalisers you aren't safe: a garbage collector needs to sweep all objects to prevent cycles.
  • For ease of use, we don't want DOM interaction to require manual annotations

Architecture

  • Built so that JS garbage collector could act as lua GC
  • No transpilation
  • VM loop in Javascript
  • Same code organisation as PUC-Rio Lua
    • core
    • low level C API
    • auxlib
    • standard library

Components

  • fengari: the core, largely a port of the PUC-Rio C implementation to lua
    Mostly bug-for-bug compatible with Lua 5.3
  • fengari-interop: a library for interacting between Lua and JS Written just like a Lua C library: uses fengari's "C api"
  • fengari-web: Write Lua in your browser just like Javascript
  • fengari-node-cli: Lua CLI but running on top of node.js

Difficulties

  • Strings
    • Lua strings are 8-bit clean
    • Javascript strings are UCS2
  • Integers
    • Lua 5.3 (usually) has 64 integers
    • Javascript only has doubles
    • In Fengari, integers are 32bit while numbers are doubles.
      As if lua was compiled with -DLUA_INT_TYPE=LUA_INT_INT

Difficulties: Tables

  • Lua table keys can be anything
  • JS objects only allow string and Symbol keys
  • JS WeakMaps only allow objects (and functions) as keys (excluding null)
  • JS WeakMaps are not iterable
  • What to use as string key?
  • lightuserdata can be any JS object.

Difficulties: next

  • Given just table and a key, need to get next key
  • Has to work when given removed ("dead") keys!

next

Tables are implemented as not just a key => value map, but also as a doubly linked list. When you remove an item from a table we:

  1. "hash" key and lookup in main part of table
  2. Remove it from linked list
  3. Delete references to key, value, previous list item. Just keep next list item. Mark pair as 'dead'
  4. Delete from main part of table
  5. Add pair to collection of "dead" pairs

next

To iterate (next), we:

  1. hash key provided
  2. lookup in main part of table
    if found, return pair's next field
  3. lookup in "dead" pairs
    if found, search through linked list until we find non-dead pair

Exposing Lua to JS

  • JS functions return a single value
  • Intercepting operations
  • fengari-interop by default exposes object with JS Map-like methods
    • .get, .set, .has, etc.
    • object is actually a function, so can be called (reduced to one return value)
    • .invoke if you want variable return: returned as Array

Testing

  • Wrote own tests against fengari "C API"
  • Lua test suite
  • Also ported ltests.c

Extensions

  • lua_atnativeerror
    Allows setting a function to intercept errors that originated outside of fengari.
  • "proxies" (lua_toproxy and lua_isproxy)
    Allows user to take reference to object on stack and keep in JS variable

Demo

https://fengari.io/

Limitations

  • Primarily around GC
    • No __gc metamethod
    • No weak tables (__mode is ignored)
    • collectgarbage and lua_gc mostly unimplementable
  • Some functions are only available in Node:
    • The io lib
    • os.remove, os.rename, os.tmpname, os.execute

Performance

  • Hasn't been optimized much yet
  • Preliminary benchmarks have it running at ~20th the speed of native lua
  • Always have option of writing hot loops in JS or WASM

FAQs

  • Why not WASM?
    • WASM doesn't allow directly interacting with the DOM
    • JS GC doesn't sweep WASM structures
  • I really want __gc

Questions?