The Underworld

The Underworld

A computer language may have all of the purity and elegance in the world, but it can never be better than the platform underneath, or the ecosystem it inhabits. NaaN leverages JavaScript to provide these benefits:

• Runs on almost every desktop, phone, tablet, and server in the world
• Has an excellent set of core libraries and support functions
• With HTML5, the browser environment offers amazing UI capabilities
• With NodeJS, servers offer comprehensive OS interfaces
• JIT compilation brings solid performance
• NPM offers two million library packages

These are not just nice to have, they are essential.

Libraries are one place Common Lisp falls short. There are only rudimentary libraries for manipulating strings, and almost none for talking to the operating system.
    — Paul Graham, Being Popular

Any sophisticated programming project requires writing a lot of code, so it is economically infeasible to rewrite substantial functionality that already exists. Instead you have to be very good at utilizing available resources. The NaaN strategy for utilizing JavaScript is to have clean, translucent interoperability. Not transparent, because improvements require differences, but clean and simple. The remainder of this article describes how NaaN connects to JavaScript and inhabits the environment.

Common Datatypes

NaaN strings, floats, and regular expressions are nearly identical to their JavaScript counterparts. The global classes String, Number, Math, RegExp, and Date are available and have the same methods. The underlying data is identical. For example:

Play-lingo> String.fromCodePoint(0x1FAD3)  
$: "🫓"

Play-lingo> $.length
$: 2

The largest difference is that integers and floats are distinct datatypes in NaaN, and NaaN integers have arbitrary precision. There are also some additional instance methods on these datatypes in NaaN. For example, the interned member function on strings reports a matching symbol if it exists:

Play-lingo> "car".interned
$: car

Native Datatypes

NaaN has its own datatypes that are distinct from those in the JavaScript environment. These include:

• symbol, as in Lisp, with annotations and separate value and procedure cells
• namespace, contains symbols and delineates write protection boundaries
• tuple, i.e. cons cells in Lisp used for lists and dotted pairs
• array, similar to JavaScript arrays but capable of holding NaaN types
• dictionary, supporting key/data pairs
• object, supporting OOP with classes, metaclasses, instances, etc.
• xobject, external objects, i.e. all other JavaScript datatypes

When passing native datatypes to JavaScript, NaaN tries to do the right thing although some fidelity can be lost. Symbols become strings, tuples become arrays, array, dictionary, and object contents are converted, and xobjects are unboxed and used unmodified.

External Datatypes

As noted earlier, JavaScript strings, numerics, and regular expressions are identical in both environments and can be exchanged freely. When JavaScript numbers are imported they are automatically made NaaN integers when the fractional value is zero, and otherwise remain floats.

NaaN supports all other JavaScript datatypes as external objects, which can be freely passed, assigned, invoked, and otherwise used within NaaN. When imported these datatypes are boxed as xobjects, and when exported back to JavaScript they are unboxed and used natively.

JavaScript objects cannot hold references to NaaN datatypes, so these are converted on assignment to a JavaScript field or array element. Alternatively JavaScript arrays and objects can be explicitly imported to their NaaN counterparts using the new() builtin, eliminating any need for further conversion.

Play-lingo> bob = Array(1,2,3)
$: [Array 1,2,3]           // JavaScript array

Play-lingo> bob.1 = `bob
$: bob                     // assigned a symbol to element 1

Play-lingo> bob
$: [Array 1,"bob",3]       // symbol converted to string

Play-lingo> naanbob = new(Array(1,2,3))
$: [1, 2, 3]               // NaaN array

Play-lingo> naanbob.1 = `naanbob
$: naanbob                 // assigned a symbol to element 1

Play-lingo> naanbob
$: [1, naanbob, 3]         // element 1 remains a symbol (no quotes)

JavaScript globals

JavaScript uses global objects for class methods, and the following subset are mirrored within NaaN: String, RegExp, Date, Number, Math, Array, and Promise. These are used to create useful external objects, such as:

Play-lingo> rx = RegExp("ab+c")
$: [RegExp /ab+c/]

Play-lingo> "abbbc".match(rx)
$: [Array "abbbc"]

Play-lingo> "acccb".match(rx)
$: null

JavaScript has a wealth of additional globals that depend upon the execution environment. For example, in browsers window holds the global namespace. In NodeJS the equivalent is global. One can access these within NaaN using the inbuilt js global. For example, in a browser:

Play-lingo> js.w
$: [object Window]

Play-lingo> js.w.location.href
$: "http://localhost:8008/nide.html"

And in NodeJS:

Play-lingo> js.g
$: [object global]

Play-lingo> js.g.process.version
$: "v20.10.0"

One might legitimately complain that the js syntax is excessively terse. The goal is to avoid always polluting the namespace with a global variable like window when it's not needed. The best practice is to explicitly declare such global variables as required. For example:

Play-lingo> window = js.w
$: [object Window]

Play-lingo> document = window.document
$: [object HTMLDocument]

Converting external objects

Sometimes it's desirable to convert JavaScript external objects to their NaaN equivalents. For example, one cannot store a tuple in a JavaScript object, which must be converted to a NaaN dictionary first. Conversion is done with the new() builtin, which uses deep recursion and handles cyclic references.

Play-lingo> js.w.location     
$: [Location http://localhost:8008/nide.html]

Play-lingo> js.w.location.host
$: "localhost:8008"

Play-lingo> new(js.w.location)
$: {
    valueOf        : [Function valueOf],
    ancestorOrigins: [object DOMStringList],
    href           : "http://localhost:8008/nide.html",
    origin         : "http://localhost:8008",
    protocol       : "http:",
    host           : "localhost:8008",
    hostname       : "localhost",
    port           : "8008",
    pathname       : "/nide.html",
    search         : "",
    hash           : "",
    assign         : [Function assign],
    reload         : [Function reload],
    replace        : [Function replace],
    toString       : [Function toString] }

As can be seen above, NaaN briefly summarizes xobjects in square brackets, while native arrays and dictionaries are fully enumerated.^[1]

JavaScript callbacks to NaaN

NaaN functions and closures can be called from JavaScript, with the arguments and return value converted automatically. In the following example the JavaScript setTimeout function delays execution for 1 second and then displays "done" in the REPL.

Play-lingo> js.w.setTimeout(function() { debuglog("done") }, 1000)
$: 587529

Play-lingo> done   // <= displayed after 1 second

When specifying a named function as a callback, it's important to pass the procedure and not the name of the procedure:

Play-lingo> function finish() { debuglog("done") }
$: finish

Play-lingo> js.w.setTimeout(finish, 1000)
$: 587723

Play-lingo>        // <= nothing displayed

Play-lingo> js.w.setTimeout(finish.proc, 1000)
$: 587768

Play-lingo> done   // <= finish got called due to ".proc"

The reason finish did not get called the first time is because the symbol finish was passed to setTimeout, which was converted to the string "finish", and there was nothing to call. For the second call, the expression finish.proc retrieves the tuple-based definition of the procedure. NaaN recognizes this as executable and passes a function to setTimeout.

The .proc method was not needed in the previous example in this section because an anonymous function evaluates to a tuple, which is automatically converted to a JavaScript function.

Concurrency Complications

A potential challenge arrises because NaaN can suspend execution while JavaScript cannot. For example, the Array.map method takes a NaaN callback:

Play-lingo> Array("a", "bb", "ccc").map(function(elem) { elem.length })
$: [Array 1,2,3]

However if NaaN suspends execution in the callback this does not suspend the JavaScript caller:

Play-lingo> Array("a", "bb", "ccc").map(function(elem) {
    sleep(1000)
    elem.length
})

$: [Array true,true,true]

In the transcript above, the 1000 millisecond delay had no visible effect, and JavaScript received a return value of true from the suspended fiber. The callback continues to execute after the sleep, but the later returned value is ignored.

If the JavaScript code is structured to expect an async callback function then NaaN can use its concurrency freely by returning a promise. However most JavaScript callbacks are simple function calls.

NaaN Environment

NaaN executes using two elements, the JavaScript library and the environment file. The JavaScript library is the same everywhere, while the environment file contains JavaScript code that loads, initializes, and runs NaaN differently for browsers and NodeJS.

The standard NaaN distribution has four environment files:

• env_web.js - browser main thread
• env_webworker.js - browser worker thread
• env_node.js - NodeJS main thread
• env_nodeworker.js - NodeJS worker thread

These can be customized for specific application requirements.

Browsers

In browsers the code to load NaaN is typically:

    <script src="code/naan.min.js"></script>
    <script src="code/env_web.js"></script>

The HTML above starts the NaaN interpreter and loads a file in the root of the page called naan_init.nlg, which contains the actual NaaN webapp code to be executed.

If the webapp starts a worker then that loads the env_webworker.js file, which connects a message receiver interface to the webapp. Workers and their tasks are typically managed through NaaN frameworks, such as browser and execution.

NodeJS

In NodeJS one adds the NPM package name @naanlang/naan to the dependencies in the package.json file. Actual execution begins with JavaScript code that loads and initializes the NaaN package and then executes the appropriate NaaN code. For example, the following code is extracted from index.js in the NaaN command line tool:

// interpreter startup options:
var naanOptions = { };

// load the environment (which loads NaaN):
var NaanNodeREPL = require('../lib/env_node.js');

// create the interpreter instance with NodeJS REPL adapter:
var naanrepl = new NaanNodeREPL(naanOptions);

// tell NaaN to find additional modules in the lib subfolder:
naanrepl.setDirectory(path.join(__dirname, "../lib"));

// load the application code and begin:
naanrepl.textCommand("nodeparse('node_repl_init.nlg');;\n");

Next Steps

As always, if you wish to investigate further, the NaaN interpreter and the NaanIDE IDE are available under the MIT license.