Remote Evaluation

Remote Evaluation

Modern software involves an intricate dance among connected systems. Often the communication mechanism is an RPC or API, but NaaN also provides the more versatile and dynamic approach of Remote Evaluation.

Terminology can be tricky, so this article uses client to refer to the system that makes a request, and server to refer to the system that responds.

APIs

APIs are a good way to offer functionality, because they solve two problems simultaneously:

• An API connects systems so that the requests can be made across a communication link, and
• An API models services to define the specific data types and operations available to clients.

APIs can be challenging to design and maintain. Changing an API would break existing clients, so they tend to expand and become redundant over time. Many APIs have an SDK that also requires coding and maintenance.

Engineers often use an API methodology between related systems even when no public access is required, just to achieve connectivity. Remote evaluation offers a simpler and more flexible approach.

Remote Evaluation

Remote Evaluation uses the server as a programmable processor^[1]. Code is copied to the server, evaluated there, and then the results are returned to the client. This has several advantages:

• Elements of the client's business logic can reside on the server, reducing the cost of communications latency.
• Because the code on each side of the link is effectively talking to itself, it can communicate using private data formats that change as frequently as desired. Neither API documentation nor SDKs are required.
• Remote evaluation abstracts the location differences between systems. For example, NaaN running in a browser can perform remote evaluation in an associated worker thread, in a different process on the same computer, or across the Internet. Timing varies across systems, but the code works the same.

How NaaN Uses Remote Evaluation

NaaN provides remote evaluation via its inbuilt frameworks. Available services include the following:

• Track available worker threads and notify as they come and go
• Spawn worker threads running NaaN in the browser and NodeJS
• Connect to a main or worker thread in the browser or NodeJS and perform remote evaluation
• Connect to remote NaaN Servers running in NodeJS
• NaaN code running on NodeJS can perform remote evaluation in NaanIDE browsers connected to that NodeJS instance.
• NaaN code running in a browser can perform remote evaluation in other browser tabs connected to the same NaanIDE server

These capabilities are the basis for several NaanIDE features:

• The NaanIDE GUI debugger operates using remote evaluation, allowing it to debug worker threads, remote systems, etc.
• The NaanIDE command line has an extensible command system to configure the programmer environment. For example, naanide workspace opens multiple browser tabs on selected projects, naanide make builds projects, and naanide idecon_debug provides a test environment for creating additional commands.
• The App.clone command splits a running instance of NaanIDE so that one instance can debug the other.
• Build, the NaaN make system, uses concurrency for efficient i/o as well as simultaneous execution of the build steps using multiprocessing.

Multiprocessing

Although NaaN offers concurrent programming, it does not automatically uses multiple processing cores because it is built on JavaScript, which is not a multithreaded system. One processor is enough for most i/o bound tasks, but some workloads really benefit from having a lot of CPU bandwidth available.

Remote Evaluation allows NaaN to execute multiple CPU-bound tasks simultaneously, taking advantage of available CPU cores without requiring programming effort to send data between the workers.

Communications

NaaN remote evaluation automatically transfers most NaaN data types across the communication link, preserving NaaN semantics. For example, numerics, strings, dictionaries, arrays, tuples, symbols, and procedures are all transferred automatically. Datatypes specific to the environment cannot be transferred, such as nonces, promises, and weak maps. Namespaces and RegExps are also not transferred.

Objects are not transferred by design. Instead they are intended to be used as proxies for object-based communications, where methods can be executed remotely. The arguments to and results from these method calls are automatically transferred.

Example - Heavy CPU Processing

Many programming tasks are i/o bound, where a single CPU core is more than enough to keep data flowing. However for compute-heavy tasks multiprocessing can be essential. This example uses Remote Evaluation to take advantage of multiple CPU cores in the same computer.

A time-honored benchmark for computing performance is the Tak function using integer math. NaaN numerics are quite slow due to their non-native bignum implementation, but multiprocessing speeds things up by an order of magnitude.

The code below is for a simple object called TakTester with methods to open (create a worker pool), close (release the worker pool), and perform various benchmarks.

/*
 * TakTester
 *
 * Create a tak testing framework with local and remote execution.
 *
 */

closure TakTester(local tester) {
    tester = new(object, TakTester)

    // tak
    //
    // Tak benchmark function
    // See: https://en.wikipedia.org/wiki/Tak_(function)
    //
    function tak(x,y,z) {
        if y < x
            tak(tak(x-1, y, z), tak(y-1, z, x), tak(z-1, x, y))
        else
            z
    }

    // bench_tak
    //
    // Benchmark measurement for tak that tests for the specified duration.
    //
    function bench_tak(duration, local ms, count) {
        ms = milliseconds()
        count = 0
        while milliseconds() - ms < duration {
            tak(18, 12, 6)
            ++count
        }
        list(count, duration)
    }

The functions above implement the benchmark and a "test bench" for it, that repeats execution until the duration has elapsed. This is in lieu of a fixed number of iterations, which can vary quite a bit between systems.

Note that tak and bench_tak are defined within TakTester but execute remotely, effectively extending the context to a foreign environment. This stands in contrast to typical distributed computing where the abstractions are strewn widely.

The open and close functions below create a worker pool and close it again. There is a bit of overhead to creating a worker and starting an instance of NaaN, so the worker pool keeps its instances available for subsequent work. To release these resources you execute the close method.

    // open
    //
    // Open the tester with a maximum worker count.
    //
    tester.open = function open(maxworker) {
        close()
        tester.max = maxworker
        tester.pex = WorkerPool(maxworker)
    }
    
    // close
    //
    // Close the tester if open, releasing the worker pool.
    //
    tester.close = function close() {
        if tester.pex
            tester.pex.destroy()
        tester.pex = false
    }
    
    // localTak
    //
    // Run a local Tak test, returning the iterations per second.
    //
    tester.localTak = function localTak() {
        bench_tak(1000).0
    }

    // remoteTak
    //
    // Run a remote Tak test in a worker, returning the iterations per second.
    //
    tester.remoteTak = function remoteTak() {
        if tester.pex                               // run if open, else false
            return (tester.pex.evalq(bench_tak(1000), `(tak, bench_tak)).1).0
    }

The rather arcane expression

return (tester.pex.evalq(bench_tak(1000), `(tak, bench_tak)).1).0

above executes evalq on the worker pool, which does the following:

• Creates a worker if needed,
• copies the tak and bench_tak member functions to it, and
• remotely evalutes the expression bench_tak(1000), returning the results.

Here is the resulting performance:

In the chart above, ideal performance would be a horizontal line, indicating perfect linear scalability. However perforance drops with more than 8 workers because they are competing for the fixed amount of available CPU cycles.

Live Demo

Please try this out for yourself. The terminal is preloaded with the code above, and runs the remoteTak() test in a worker. If you're feeling ambitious you might try running future(function(){printline(tt.remoteTak())},0) multiple times simultaneously and see the benefit of parallel operation.

This is just a simple embedded REPL terminal, so the worker is hidden in the background. NaanIDE displays all of the available contexts, including local and remote workers, which makes it clear what is executing where.