Architecture of NaaN

Architecture of NaaN

This article describes how NaaN is structured. The overall goals are to provide code portability across a variety of environments, extensibility to simplify adding new features, uniform debugging and logging access for reliability, and above all simplicity wherever possible.

Description of the Diagram

The layer diagram above shows that NaaN runs on top of JavaScript and provides services for implementing Applications. Within NaaN there are two major sections:

• NaaN Core: the foundation that interfaces with the environment to provide a uniform execution context, and
• NaaN Runtime: which organizes the code at runtime with a module/component structure, and provides inbuilt functionality via libraries and frameworks.

This article describes the components in the diagram from the bottom up, because everything is defined in terms of the elements defined below it.

NaaN Core

The Environment block initializes the interpreter, obtains execution time, loads and saves state when requested, receives lifecycle events such as unload warnings, relays debugger messages, and manages the REPL both local and remote. Different environment files enable NaaN to run in NodeJS, browsers, and background workers for both.

The I/O block performs terminal buffering, low-level tokenizing, and redirection using in-memory string storage.

The JS Interop block handles conversions between NaaN and JavaScript environments. This allows NaaN to freely interoperate with NPM libraries, e.g. using callbacks, promises, async/await, and both CommonJS and ES6 modules.

Namespaces are an inbuilt NaaN datatype for segregating data among different parts of the codebase. Symbols, and optionally dictionaries and objects, are assigned to a namespace and can only be modified by code running from that namespace. This provides low-level support for modules described below.

The NaaN Interpreter operates in all environments, using hooks out to the Environment block for any platform variations. This implements the fundamental datatypes with some or all of the associated functionality.

The Builtins block implements supporting procedures that establish the base runtime environment. Builtins become the procedure binding when a symbol of the same name is first mentioned in a namespace. For example:

Start-lingo> length.proc
$: [Builtin length]
Start-lingo> length("abc")
$: 3
Start-lingo> "abc".length
$: 3

Because NaaN is Lisp-2^[1], one can still use the symbol length as a variable, or even redefine its procedure binding. A dictionary of all available builtins is available by evaluating the inbuilt global Naan.

NaaN Runtime

The NaaN Runtime is implemented using NaaN code, which executes using the JavaScript-based services of Core described above.

Module Manager

The Module Manager is a prominent divider between the upper and lower sections of the architecture. All NaaN code executes within a module in the Runtime, the upper section of the diagram. The goals for modules are:

• provide a customizable execution context
• enforce isolation between unrelated implementations
• allow load-on-demand so that only required functionality occupies memory

A NaaN module is a named unit of deployment for code, similar to a library. A module comprises one or more load-on-demand components. The default component, which has the same name as the module, is responsible for initialization. Ordinarily a module corresponds to a single namespace, so the components are expected to coordinate with each other on global names, etc.

An important characteristic of NaaN modules is that they are write-protected from each other. One module can call procedures in another module, and reference globals, but it cannot modify them directly. Mutations are only allowed by procedures defined within the module. This both enforces isolation and makes it easier to reason about how changes evolve within a running system.

The Boot modules: CLI / Util / Lib

The Boot modules are loaded automatically when NaaN starts up. Their composition and purpose are as follows.

Lib comprises three components:

• RuntimeLib has a number of support procedures for higher-level functions that are easier to implement in NaaN than in low-level JavaScript. For example, iterators, mappers, and async support functions are defined here.
• Languages is the language and dialect manager, which allows a single running instance of NaaN to support a variety of coding styles.
• LangLib has procedures that can be used across a variety of language implementions. For example, there are parse functions for brace-based structures, etc.

Util comprises four components:

• Module is the module manager, and is itself a module of course. Initializing this requires a little bit of footwork.
• Packager implements some of the several different schemes for encoding and decoding native NaaN data. The NaanPackage data format is capable of representing most data that can occupy a single namespace, and can be loaded in one place and restored in another.
• Errors defines standard NaaN Errors. Much of the system takes advantage of the uniformity provided by this single, extensible datatype.
• Debug implements a low-level debugging controller that interfaces between debugger UIs and the interpreter builtins that supply debuggin hooks.

CLI comprises three components:

• Driver implements NaaN's REPL command-line iterface, including the executive functions and /help system.
• Debugger implements NDB, the NaaN command-line debugger. This is just the text-based UI, which utilizes the debug controller in Util to actually perform debug operations.
• CliUtils are support functions used by both the Driver and Debugger. For example, it provides utilities to convert between a fully-scoped procedure specification and a reference to the procedure tuple structure.

The Lingo module

Lingo is the default language used by NaaN, though one can build upon it with Dialects or add a parallel language. This module defines the essential componnents that give Lingo its syntax:

• The Lingo component implements the parser for the language, which converts textual input into S-expressions with the help of the Lib:LangLib component and the tokenizer in the the interpreter's I/O block.
• The Unparse component implements the unparser for the language, which is required. The CLI and debuggers use this unparser to generate "source code" on the fly, making it possible to investigate and debug NaaN instances where no source files are available.

Frameworks

NaaN frameworks are just inbuilt modules that are guaranteed to be available in any standard installation of the software. Here is a high level description of the current functionality:

• browser is a framework supporting running in the browser. It has components for http[s] requests, low-level access for PouchDB databases, terminal interfaces to NaaN instances, workers, service workers, and websockets.
• client is a framework for clients of NaanServers, which provide host remote control and filesystem access from the browser.
• common provides uniform utility functions across all execution environments. For example, it provides functions for computing standard hash types in both browser and NodeJS where the implementations are very different. Common also has utility components, such as REPL support tools, generic utilities, etc.
• node is analogous to the Browser component above, but for NodeJS. It also contains local filesystem access code and the NaanServer implementation.
• project contains code for managing NaanIDE projects, including opening, modifying, and building them. NaanIDE uses this module for its own operation.
• running has components for remote evaluation, local and remote GUI debugging, worker pools, and other aspects of execution.
• storage contains PSM (Platform Storage Manager), which is an abstract storage layer that works across browsers and NodeJS environments. It supports direct filesystems as well as filesystem-on-blob filesystem emulation within databases.

Plugins

Plugins are modules, like frameworks, but provide interoperability with external services. There are a number of prototype-level plugins in the repository, but the production quality plugins are:

• gitClient uses the git command line tool to access version control information on a NodeJS filesystem. It can also be invoked on behalf of a browser via the PSM abstraction layer.
• openSearch provides a simple, but extensible subset of OpenSearch functionality adapted to standard NaaN datatypes.
• sericeAws provides a simple, but extensible subset of AWS functionality including CloudWatch, DynamoDB, S3, SQS, Lambdas, and PSM adapters on top of these services. This makes AWS much easier to use from NaaN. For example, DyanmoDB datatypes are automatically mapped to the corresponding NaaN types.

Summary

The NaaN architecture is designed to provide code portability across a variety of environments, extensibility to simplify adding new features, uniform debugging and logging access for reliability, and above all simplicity wherever possible.