Procedures & Bindings & Scopes, Oh My!

Procedures and Bindings and Scopes, Oh My!

This article introduces NaaN procedures, symbol bindings, and scopes over the course of ten examples. At the end of this article is a summary of the procedure forms and their characteristics.

Example 1 Elementary functions

The simplest procedure form in NaaN is the function, which can be named or anonymous. The value of evaluating a named function definition is the symbol that names the function. The value of evaluating an anonymous function definition is the function definition itself.

Play-lingo> function(x) { 2 * x }
$: function (x) {
    2 * x
}

Play-lingo> function(x) { 2 * x }(3)    // immediate execution
$: 6

Play-lingo> function double(x) { 2 * x }
$: double

Play-lingo> double(3)
$: 6

Closures and macros have the same syntax as functions.

Example 2 Nested procedures

Nested procedures are useful for associating related logic in one place. In the following example, a function takes an expression and recursively substitutes symbols as directed.

Play-lingo> 

// subst
//
// In tuple expr, make substitutions using the rep array.
//
function subst(expr, rep) {

    function sub1(expr, key, data, local out) {
        debuglog("    sub1 ${expr} ${key} ${data}")
        while tuple(expr) {
            item = pop(expr)
            if tuple(item)
                item = sub1(item, key, data)
            else if item == key
                item = data
            push(item, out)
        }
        reverse(out)
    }

    let (pair) {
        for pair in rep
            expr = sub1(expr, car(pair), cadr(pair))
    }()
    expr
};
$: subst

Play-lingo> subst(`(+ a (+ b c)), `((+, -)))
    sub1 (+ a (+ b c)) + -
    sub1 (+ b c) + -
$: a - (b-c)

Play-lingo> subst(`(+ a (+ b c)), `((+, *), (b,d)))
    sub1 (+ a (+ b c)) + *
    sub1 (+ b c) + *
    sub1 (* a (* b c)) b d
    sub1 (* b c) b d
$: a * (d*c)

In the transcript above, the function subst is defined with a recursive helper called sub1 that performs one substitution operation. A call to debuglog displays the arguments to illustrate its operation. These are written in a Lisp-oriented coding style with tuples (lists), but can just as easily use JavaScript idioms with arrays.

Instead of let one could use an anonymous function with immediate execution, but the advantage of let is that return ignores it and returns from the enclosing procedure.

In NaaN local variables are established by adding parameters to the procedure. The local keyword stops argument processing and guarantees that the locals default to the value false even if the caller specifies extra arguments.

Example 3 Macros

NaaN does not implement macros in the traditional Lisp manner because it has a more structured approach called Dialects. However it's still important to be able to invoke procedures without evaluating all of the arguments. NaaN macros provide this ability.

NaaN macros are hygienic in that their internal symbols cannot interfere with any arguments passed to them. Also, macros execute in the namespace of the calling procedure instead of the namespace where they were defined. This gives the macro the right to modify data passed to it by the owner.

Building on the previous example, the following code shows how to take a NaaN expression as an argument and pass it to another function unevaluated. This gives the subst function a cleaner interface for expressions.

Please note that the debuglog message is removed from this and the following examples.

Play-lingo> macro subex(expr, rep) {
    subst(expr, eval(rep))           // eval only second arg
};
$: subex

Play-lingo> subex(a*b+c*d, `((*, /), (b, e)))
$: a/e + c/d

Example 4 Variable arguments

The examples above utilize procedures with fixed arguments, but variable arguments are often useful. If NaaN procedures are defined with a symbol instead of an argument list then the actual arguments to the procedure are bound to the symbol.

Nested procedures can be used to implement any desired argument protocol, such as having a fixed argument followed by variable number, optional options dictionaries, argument terminators, etc.

Building on the previous examples, the following code allows each substitution to be expressed as an argument.

Play-lingo> macro subexv args {
    function(expr, reps) {
        subst(expr, eval(reps))
    } (pop(args), args)
};
$: subexv

Play-lingo> subexv(a*b+c*d, `(*, /), `(b, e), `(a, z))
$: z/e + c/d

The backquote expressions like `(a, b) above are tuple literals. Lisp generally uses a single quote character for list literals like '(a, b) however JavaScript uses single quote as a string delimiter and it was a constant annoyance to have to update all occurrences when copying JavaScript code into NaaN.

Example 5 Procedure Factories

This pattern returns a procedure generated according to a certain criteria. The following DiscountFactory returns a function that discounts a numeric quantity by a specified percentage, specified in the range 0 to 1.0. If the discount cannot be calculated then the original argument is returned.

Play-lingo> closure DiscountFactory(discount) {
    function(amount) {
        if amount * (1-discount)
            return
        amount
    }
};
$: DiscountFactory

Play-lingo> d10 = DiscountFactory(0.1)     // 10% discount
$: function (amount) {
    if amount * (1 - discount)
        return
    amount
}

Play-lingo> d10(30)
$: 27.0

Play-lingo> d10("hello")
$: "hello"

In the transcript above DiscountFactory is defined as a closure, which creates a new copy with the specified parameters each time it is executed. Everything inside the closure -- variables, functions, etc. -- persist past the end of closure execution.

Evaluating DiscountFactory(0.1) creates a new closure where the symbol discount has the value 0.1, and then returns the enclosed function which is assigned as the value of the symbol d10. Applying d10() shows that the closure context is being used.

Example 6 Earle's Lo-Cost Objects

The procedure factory pattern is very useful for one-off functionality, but sometimes you need the versatility of multiple methods. NaaN objects have the capabilities for classes, metaclasses, inheritance, etc. but here is a simple pattern when a few methods sharing a common state is enough.

The code below calls Receipt to make and return a receipt object for a specific customer. This receipt object offers methods to add items to the receipt and to print out the results.

Play-lingo> 

// Receipt
//
// Create a receipt for a set of operations.
//
closure Receipt(customer, local rcpt, taxrate) {
    rcpt = new(object, Receipt)
    taxrate = 0.105                // 10.5 % (private constant)
    rcpt.notax = 0
    rcpt.taxable = 0
    rcpt.items = []

    // add - public method to add an item
    //
    rcpt.add = function add(desc, amount, exempt,
            local item)
    {
        item = {
            desc: desc
            amount: amount
            exempt: exempt || undefined
        }
        if exempt
            rcpt.notax += amount
        else
            rcpt.taxable += amount
        rcpt.items.push(item)
        item
    }
    
    // print1 - print receipt item (private function)
    //
    function print1(item, suffix) {
        if string(item)
            printline(prepad("", 20, item))
        else
            printline(prepad(item. desc, 10),
                prepad(item.amount.toFixed(2), 6),
                prepad(suffix || "", 4))
    }
    
    // print - public method to print the receipt
    //
    rcpt.print = function print(local total, item) {
        total = rcpt.notax + rcpt.taxable + rcpt.taxable*taxrate
        printline(cr, "Receipt for ${customer}", cr)
        for item in rcpt.items
            print1(item, if item.exempt "EX" else "")
        print1("-")
        print1({ desc: "Taxable:", amount: rcpt.taxable })
        print1({ desc: "Exempt:", amount: rcpt.notax })
        print1("=")
        print1({ desc: "Tax:", amount: rcpt.taxable*taxrate })
        print1({ desc: "Total:", amount: total })
        printline()
        total
    }

    // return the new object

    rcpt
};
$: Receipt

Play-lingo> receipt = Receipt("Richard")
$: Object{5}

Play-lingo> receipt.add("Coffee", 5.30)
$: { desc: "Coffee", amount: 5.3 }

Play-lingo> receipt.add("Candy", 3.00)
$: { desc: "Candy", amount: 3.0 }

Play-lingo> receipt.add("Aspirin", 1.10, true)
$: { desc: "Aspirin", amount: 1.1, exempt: true }

Play-lingo> receipt.print()

Receipt for Richard

    Coffee  5.30    
     Candy  3.00    
   Aspirin  1.10  EX
--------------------
  Taxable:  8.30    
   Exempt:  1.10    
====================
      Tax:  0.87    
    Total: 10.27    

$: 10.2715

Example 7 Basic dynamic binding

NaaN has dynamic binding because occasionally it is very useful.^[1] The function& and let& procedure forms perform dynamic binding on their arguments. This binds argument values to the procedure's formal parameters for the duration of the call. The formal parameters can be globals or lexically bound symbols in a parent procedure.

Play-lingo> function& bindx(x, p) { p() }
$: bindx

Play-lingo> function printx() { printline("x: ${x}") }
$: printx

Play-lingo> x=2
$: 2

Play-lingo> bindx(3, printx)
x: 3
$: "x: 3"

Play-lingo> x
$: 2

In the transcript above, bindx dynamically binds symbol x to the value of the first argument and then calls the second argument. When printx emits the current value it finds that x is 3, but upon return x is back to 2 again.

Normally it is difficult to ascertain the value of global variables in complex code. But because dynamic binding correlates to the pattern of function calls in the code (the "dynamic call graph") it's far more straightforward to analyze.

Example 8 Advanced dynamic binding

Dynamic binding usually applies to global variables, but in NaaN a dynamically bound function can be embedded inside a lexically bound one. Side effects outside of the parent procedure are impossible when no global variables are involved.

One can use this approach to change the context for other functions inside the lexical parent, allowing procedure arguments to be inherited instead of passing them directly. The NaaN language libraries use this for optimization and error detection on S-expressions. The code for the post-parse S-expression error checker is too long to reproduce here, but the following will list it in the REPL:

Play-lingo> /list Lib::langErrorCheck
   1  function langErrorCheck(tree, local xproc, proc_parent, xloop, loop_parent,
   2      xbody, blocknest) {
   3      (global, langDebuglog)
   4      function debugErrorCheck args {
   5          if false
   6              apply(langDebuglog, args)
   7      }
   8  
   9      function& inbody(lbody, xbody, local item, more) {
  10          if atom(lbody)
  11              lbody
  12          else {
  13              debugErrorCheck("inbody:", lbody)
  14              xbody = { live: true }
   [...]

Example 9 Dynamic Binding and Concurrency

A NaaN fiber saves and restores its entire continuation, making dynamically bound variables similar to threaded local storage in other languages.

Play-lingo> closure run(local f1, f2, tls, stop) {

    // printloop - print the current tls every second until stop
    function printloop() {
        while !stop {
            printline("fiber: ${tls}")
            sleep(1000)
        }
    }

    f1 = future(function&(tls) {    // tls dynamically bound in fiber f1
        tls = "f1"
        printloop()
    }, 0)
    
    f2 = future(function&(tls) {    // tls dynamically bound in fiber f2
        tls = "f2"
        printloop()
    }, 500)

    sleep(4000)
    stop = true
};
$: run

Play-lingo> run()
fiber: f1
fiber: f2
fiber: f1
fiber: f2
fiber: f1
fiber: f2
fiber: f1
fiber: f2
$: true

Play-lingo> 

The procedure run is declared as a closure so that the stop variable persists past the execution of the function. The lifetime of closure contents is also discussed in Example 5 - Procedure Factories

Example 10 Closures and Concurrency

Closures must be used instead of functions under two circumstances:

• When a local variable lifetime must persist after the procedure completes, or
• When a procedure is shared by multiple concurrent fibers.

NaaN uses explicit closures because this provides full control over variable scope and lifetime, making clear what is captured when the closure is created. In other languages closures are created implicitly and it can be challenging to trace where closures are created and what they contain.

Knowing where to use closures requires some thought, and at this writing NaaN does not perform error checking on proper use. The classic JavaScript example of a mistake:

Play-lingo> function printBad(arr, local ff, item, index) {
    ff = []
    for `(item, index) in arr
        ff.push(function() { printline("item ${index} is ${item}") })
    while ff.length > 0
        call(ff.shift())
};
$: printBad

Play-lingo> printBad([a,b,c])
item 2 is c
item 2 is c
item 2 is c
$: true

This did not print the items in the array properly because the same variables item and index were shared by each of the future calls to print. What is needed is a closure for each iteration:

Play-lingo> function printGood(arr, local ff, item, index) {
    ff = []
    for `(item, index) in arr
        ff.push(closure(item, index) {
            function() { printline("item ${index} is ${item}") }
        }(item, index))
    while ff.length > 0
        call(ff.shift())
};
$: printGood

Play-lingo> printGood([a,b,c])
item 0 is a
item 1 is b
item 2 is c
$: true

A similar challenge arises when multiple fibers share the same function. In the code below, "end request 2" is printed twice because the second call to request overrides the value of req parameter from the first call.

Play-lingo> closure badness() {

    function mockio(req, cb) {
        sleep(10)
    }
    
    function request(req) {
        debuglog("start request ${req}")
        mockio(req)
        debuglog("end request ${req}")
    }

    future(function() { request(1) }, 0)
    future(function() { request(2) }, 0)
    sleep(20)
};
$: badness

Play-lingo> badness()
start request 1
start request 2
end request 2
end request 2
$: true

The solution to this challenge is to make request a closure, so that it has a private copy of the req parameter for each fiber.

Here is the corrected version:

Play-lingo> closure goodness() {

    closure mockio(req, cb) {
        sleep(10)
    }
    
    closure request(req) {
        debuglog("start request ${req}")
        mockio(req)
        debuglog("end request ${req}")
    }

    future(function() { request(1) }, 0)
    future(function() { request(2) }, 0)
    sleep(20)
};
$: goodness

Play-lingo> goodness()
start request 1
start request 2
end request 1
end request 2
$: true

An alternative solution is to declare request as function& so that req is dynamically bound and therefore thread-specific. That works here, and it works when awaiting async Javascript functions that return promises.

However dynamically bound variables do not work with asynchronous JavaScript callback functions. Callbacks are executed while the initiating fiber is suspended, so its dynamically bound variables are inaccessible. JavaScript callbacks work fine with NaaN if you ensure that each JavaScript request and its resulting callback occur in the same closure, because the closure variables are always available.

Summary

NaaN provides six procedure forms with variables having different scope and extent. Scope refers to where in the textual representation the binding can be accessed. Extent refers to when the binding exists during execution. NaaN procedures can be named or anonymous, with fixed or variable arguments.

Variables defined in lexical scope procedures are accessible anywhere within the procedure or its transitive children. They are:

• A function has variables with fixed extent, while the function is executing.

• A closure has variables with indefinite extent. Evaluating a closure makes a new copy of all variables within it, and they extend until garbage collected.

• A macro has variables with fixed extent, while the macro is executing.

• A let block has variables with fixed extent, while the block is executing. A let block is always anonymous and always takes a fixed number of arguments.

The procedure forms with dynamically bound variables are:

• A function& has variables with both dynamic scope and dynamic extent. If the function& is defined at the global level the variables are bound globally. If it is defined within a procedure the variables are bound within the lexical scope of the parent procedure.

• A let& block has variables with both dynamic scope and dynamic extent. If defined at the global level the variables are bound globally. If the let& is defined within a procedure the variables are bound within the lexical scope of the parent procedure. A let& block is always anonymous and always takes a fixed number of arguments.

Playpen

All of the examples above are preloaded into this terminal.

As always, if you wish to investigate further, the NaaN interpreter and the NaanIDE IDE are available under the MIT license.