NaaN Tutorial: Library Integration

NaaN Tutorial: Library Integration

One of the most important software trends of the 21st century has been the rise of Open Source Software. Startup companies that were going to market with a couple hundred thousand lines of proprietary code gave way to those having several million, with 90% or more being open source. This allows modern teams to deliver vastly more functionality for the same development effort.

Libraries have always been important to the viability of a computer language and its ecosystem, but now they are essential. A key strength of NaaN is that it can readily integrate libraries from the NPM registry of over two million packages.

An Email Sender

This tutorial describes the process for for sending email using an NPM library. The tasks required are:

1. Select an appropriate library
2. Make a project folder and install the prerequisites
3. Experiment and evaluate
4. Create an interface component

The process of adopting external libraries is more than a coding exercise because you are taking responsibility for the library in your application. If it misbehaves then so does your application. It's therefore important to perform some due diligence to guard against problems.

1. Selecting the Library

A short search with NPM yields emailjs, which has several positive points:

• It's quite popular at 28,000 weekly downloads. 28 per week would be questionable.
• It was last updated 9 months ago, which is fairly recent but not raw. Software updated in the last few weeks might be unstable, or if five years ago might be abandoned. The sweet spot is typically in the range of two months to two years.
• The ReadMe document provides clear documentation. We're trying to save time, not solve puzzles. Also, clearly-written documentation often correlates with well-written code.
• It has no prerequisites. A quick look at the package.json file tells you what else the package brings along. If a 2000 line library includes 2 megabytes of support packages one might question the author's judgement.

On face value emailjs looks good enough to install and evaluate.

When selecting a library that will require a long term commitment it's important to scrutinize the library and its support ecosystem carefully to avoid lock-in on a failing project. For example it could be quite painful to replace a database that uses a proprietary data format. Searching stackoverflow and reddit can help identify problems that others have faced. Email is pretty easy to replace, so deep analysis isn't as important here.

2. Making a Project

We need a project folder because NodeJS expects its dependencies to be structured that way on disk. NaaN uses the same module structures as JavaScript in NodeJS, accommodating both the older CommonJS and newer ES6^[1] modules. EmailJS uses ES6.

The following transcript shows how to create the project from a blank slate. It starts off in zsh and proceeds into NaaN where the emailjs module is loaded.

% mkdir email
% cd email
% npm install @naanlang/naan emailjs

added 2 packages in 3s
% npx @naanlang/naan
Welcome to Naan v1.2.0+1
Naan REPL - type "/help" for more information

Start-lingo> await(js.i("emailjs"))
$: (false, [object Module])      // (error, data) return tuple

Start-lingo> emailjs = $.1       // $.1 is the literal form of $[1]
$: [object Module]

In the transcript above, js.i is NaaN's access to the dynamic import function for ES6 modules, which returns a promise. The await() function suspends execution until the promise completes, returning an (error, value) result tuple. This NaaN idiom is similar to golang, but the error is ordered first to follow the pattern established by callbacks in NodeJS.

Using JavaScript async/await requires setting a try...catch handler on every single await, which seems to needlessly obscure the logic of the code. It's good if you're paid by the line though. Alternatively one can establish a single try/catch higher up on the stack and report generic errors for anything that shows up. There are many defenders of this practice, however my experience in evaluating tens of millions of lines of code is that this is really saying "I don't want to think about error handling."

We have passed the first hurdle in our evaluation, which is successfully loading the library.

3. Experimenting and Evaluation

Experimenting is the fun part because the library is actually doing useful things. If not, then it's time to reject this one and move on to another option.

The EmailJS ReadMe file is quite good, so we know how it should work, but I always prefer to see it in operation. This proves that the necessary functions work, but also clarifies understanding of the details of the library's API. One can also probe the actual data structures at each stage of operation to validate proper operation.

The following transcript uses NaaN's new() builtin to decode the emailjs module. One can glean a lot from this, such as determining the available options and auth techniques.

Start-lingo> new(emailjs)       // import to decode the structure
$: {
    AUTH_METHODS       : {
        PLAIN                  : "PLAIN",
        CRAM-MD5               : "CRAM-MD5",
        LOGIN                  : "LOGIN",
        XOAUTH2                : "XOAUTH2" },
    BUFFERSIZE         : 12768,
    DEFAULT_TIMEOUT    : 5000,
    MIME64CHUNK        : 456,
    MIMECHUNK          : 76,
    Message            : [Function Message],
    SMTPClient         : [Function SMTPClient],
    SMTPConnection     : [Function SMTPConnection],
    SMTPError          : [Function SMTPError],
    SMTPErrorStates    : {
        COULDNOTCONNECT        : 1,
        BADRESPONSE            : 2,
        AUTHFAILED             : 3,
        TIMEDOUT               : 4,
        ERROR                  : 5,
        NOCONNECTION           : 6,
        AUTHNOTSUPPORTED       : 7,
        CONNECTIONCLOSED       : 8,
        CONNECTIONENDED        : 9,
        CONNECTIONAUTH         : 10 },
    SMTPResponseMonitor: [Function SMTPResponseMonitor],
    SMTPState          : { NOTCONNECTED: 0, CONNECTING: 1, CONNECTED: 2 },
    addressparser      : [Function addressparser],
    getRFC2822Date     : [Function getRFC2822Date],
    getRFC2822DateUTC  : [Function getRFC2822DateUTC],
    isRFC2822Date      : [Function isRFC2822Date],
    mimeEncode         : [Function mimeEncode],
    mimeWordEncode     : [Function mimeWordEncode] }

To actually try out the library we can copy sample code almost verbatim from the ReadMe document:

Start-lingo> client = xnew(emailjs.SMTPClient, {
    user: 'richard@naanlang.org'
    password: 'hunter2'
    host: 'smtp.naanlang.org'
    ssl: true,
})
$: [object Object]

In the transcript above we create an emailjs client for the mail server that will acct as an SMTP gateway. The NaaN builtin xnew, i.e. "new for external objects" enables NaaN to create new JavaScript objects.

Now we can actually send an email:

Start-lingo> `(error, message) = await(client.sendAsync({
    text: 'i hope this works'
    from: 'me <richard@naanlang.org>'
    to: 'you <tanya@naanlang.org>'
    subject: 'testing emailjs'
}))
$: (false, [object Object])     // result of sending the message

Start-lingo> new(message)
$: {                            // the decoded message result...
    attachments: [],
    header     : {
        message-id     : "<1718350098767.1.81342@ip-192-168-1-9.ec3.internal>",
        date           : "Fri, 14 Jun 2024 10:28:17 -0800",
        from           : "=?UTF-8?Q?me?= <richard@naanlang.org>",
        to             : "=?UTF-8?Q?you?= <tanya@naanlang.org>",
        subject        : "=?UTF-8?Q?testing_emailjs?=" },
    content    : "text/plain; charset=utf-8",
    alternative: null,
    text       : "i hope this works" }
    
Start-lingo> 

In the transcript above, the line `(error, message) = await(client.sendAsync({ performs a destructuring assignment of the (error, value) tuple returned by await. One of the advantages of using tuples for multiple return values is that they can be kept as a tuple or destructured into separate variables.

We now have a proof of concept for emailjs. The next steps are to try out any additional APIs we might need, like enclosures or HTML messages. When everything seems to be working, it's time to start integrating the library into our application.

4. Create an interface component

While not strictly required, creating an interface component is a strongly recommended practice for the following reasons:

• An interface component matches the application needs to the library capabilities. Libraries tend to be general purpose with many options, while application requirements tend to be more narrow and can include fixed choices that would be awkward for a library.
• If future library versions have breaking changes then these can be addressed in one place, instead of hunting around in the application code.
• An interface component makes it much simpler if you ever need to switch to a different library, because the changes are localized to one place.
• It provides a convenient place to document problems and particularities with the library.

The best time to write code for an interface component is immediately after completing the proof of concept.

In the code below, an EmailSender constructor accepts the secret credentials as a single dictionary argument and returns an object for using the functionality. The public methods are open and send.

Less experienced programmers are often tempted to combine object construction and initialization, but this is almost always a mistake. A service pattern that has separate open (and close, if applicable) methods has several advantages:

• It's hard to report useful errors from a constructor, and the resulting state of the object is ambiguous. An open call can typically be retried after failure. For example, after network connectivity is restored.
• Initializing after the rest of the application is constructed simplifies logging and/or reporting errors.
• Deferring slow operations after startup improves the user experience. The open operation, which can take time for a network request, may never be needed.

Here is a rough outline of our new EmailSender:

closure EmailSender(creds, local email) {
    email = new(object, this)

    email.open = closure open(local error, ejs) {
        // retrieve credentials
        // load email module
        // create email client
    }

    email.send = closure send(params ...) {
        // error checking
        // optional HTML processing
        // send email
    }

    // finis
    email
};

Based on the documentation and our recent experience with the API, the following is a reasonable implementation of the open method:

/*
 * EmailSender
 *
 * An EmailSender object is constructed with appropriate credentials.
 *
 * Credentials:
 *  {
 *      user:       <string>                    // e.g. noreply@naanlang.org
 *      pswd:       <string>
 *      smtp:       <string>                    // e.g. smtp.naanlang.org
 *      ssl:        <boolean>                   // true for SSL
 *  }
 *
 */

closure EmailSender(creds local email) {
    global(js)
    email = new(object, this)

    // open
    //
    // Open the email sender, returning a result tuple.
    // This loads the SMTP library and creates a client.
    //
    email.open = closure open(local error, ejs) {
        if email.client
            return (list(false, { open: true }))
        if !string(creds.user, creds.pswd, creds.smtp)
            return (list(Error("invalid credentials", creds)))
        `(error, ejs) = await(js.i("emailjs"))
        if error
            return (list(error))
        email.ejs = ejs
        email.client = xnew(ejs.SMTPClient, {
            user: creds.user
            password: creds.pswd
            host: creds.smtp
            ssl: creds.ssl
        })
        return (list(false, { ok: true }))
    }

The open method above ensures that emails are ready to be sent and then returns a standard (error, value) tuple. If already open this does nothing.

Continuing to the send method:

    // send
    //
    // Send an email using the following parameters,
    // returning a result tuple when finished.
    //
    //  {
    //      from:       <string>
    //      to:         <string>
    //      subject:    <string>
    //      text:       <string>                // body of email
    //  }
    //
    email.send = closure send(params) {
        if !email.client
            return (Error("email not open"))
        if !params.from
            return (Error("from address missing"))
        if !params.to
            return (Error("to address missing"))
        await(email.client.sendAsync({
            from: params.from
            to: params.to
            text: params.text || ""
            subject: params.subject || ""
        }))
    }

    // finis
    email
};

This code is very straightforward. While it's possible to rely on error checking within the library for missing from and to addresses, one has to be careful to avoid defaults. Since we definitely want an error in these cases we check explicitly.

Summary

In this tutorial we performed four tasks to adopt an NPM library for sending emails:

1. Selected an appropriate library
2. Created a project folder and installed the prerequisites
3. Experimented with and evaluated the library
4. Created an interface component

Please stay tuned for a future post that describes NaaN and JavaScript integration in more detail.

As always, if you wish to investigate further, the NaaN interpreter and the NaanIDE IDE are available under the MIT license.

Special thanks to Gabriel Sollmann for the Stuttgart Library photo