Building a Conversational Bot with JavaScript and Node.js

Let's admit it – speaking to other people is so last century. Today, we might talk to our machines (Siri, Alexa, etc.), but, if we want to communicate with people, we use text. We SMS rather than call or chat via Slack rather than video via Skype. Even things like one-on-one customer support seem to be increasingly chat rather than call.

This desire to text rather than talk has led to the increasing utility of bots to the degree that banks can now understand emoji. That's right – who needs words even!

However, creating bots isn't necessarily easy. You have to be able to receive text (or in some cases, convert voice commands to text), parse it and match it to a response. This is complicated by the fact that there are a hundred ways to say any one thing. In some cases, you may even want your bot to understand the nature of a conversation, so that every input isn't looked at in a silo, completely disconnected with what has previously been communicated.

This is where a tool like SuperScript comes into play. SuperScript is an engine for creating bots using JavaScript and Node.js that helps overcome some of these difficulties by offering a way to do things like normalize user inputs and build complex conversations. It was something that personally fascinated me and, in this article, I hope to introduce you to how SuperScript works by discussing a sample project I created with it.

The MUTHUR Demo

A conversational bot is arguably a simplistic form of artificial intelligence in that it mimics a human behavior while appearing to understand both language and context within a conversation. Of course, AI has been the subject of endless science fiction, which is where I took the inspiration for my demo project.

MUTHUR is the AI that runs the Nostromo in the movie Alien. One of the interesting things about MUTHUR compared to other computer AI in science fiction movies is that every interaction with her on screen is done via text.

If you watch the scene above, the MUTHUR AI clearly understands the context of conversation. For instance, she knows that the "enhancement request" is related to the question about special order 937 and so on.

Alien being one of my favorite movies, this inspired me to see if I could recreate MUTHUR as a chat AI. Admittedly, there is limited direct interaction with MUTHUR in the movie (most occurring in the scene above), but there is quite a bit of implied interaction. So, much of the chat bot I ended up creating is more "inspired by" rather than direct recreation of the movie.

Keep in mind that the bot I created is still a work in progress – there were a lot of interactions I had planned that I simply haven't been able to complete yet and improvements to existing interactions that I'd like to see. You can find the project on GitHub here. Let's take a deeper look at some of the features and how SuperScript makes them possible.

Most of the conversational ideas in my MUTHUR bot and the script quotes in this article are taken from the Alien script on The Daily Script.

As a side note, there is apparently some sort of MUTHUR bot that was created as part of some promotional material for the upcoming movie, although it does not respond to some of the basic queries from the original film when I tried it.

Setup

I won't regurgitate the nicely done SuperScript installation guide here other than to reiterate that you will need both Node.js and Mongo installed to get started.

Once everything is installed, you can create a new bot project via:

bot-init muthurBot --clients telnet
cd muthurBot
npm install

This will create a default project and install the necessary dependencies. For the purposes of my MUTHER bot demo, I have only created a telnet client. However, SuperScript does support other clients, including Slack.

If you want to follow along with the code, feel free to clone the repo. Once you have, you'll need to run through the parsing of the text, building the project and starting the telnet client.

Before we can do that though, we need to make sure Mongo is running. In a Terminal/Command Prompt tab, start Mongo.

mongod

Now in a separate Terminal/Command Prompt tab, let's get the project ready to run and run it.

npm run parse
npm run build
npm run start-telnet

Below you can see the result of my running these commands on the project.

Our telnet server is now running. So, in another Terminal/Command Prompt tab, we can connect to it.

telnet localhost 2000

You can see what it looks like if this connects properly below.

The welcome message is obviously customizable ("interface 2037" is straight out of the Alien movie after all and not a SuperScript default). This is set within script that runs the telnet server. For the most part I've used the default script, with one exception we'll get into later, but it is important to note that you can customize how the telnet server functions.

Here is the code from the newSocket method where the welcome message is set:

// Send a welcome message.
socket.write(`HELLO '${socket.name}'\n`);
socket.write('INTERFACE 2037 READY FOR INQUIRY\n\n');

Basic Interaction

Now that we have our basic telnet interface set, let's start by building a simple command-and-response type interaction.

RIPLEY: Have you tried putting the transmission through ECIU.

ASH (voice over): Mother hasn't identified it as yet. It's not a language.

We define our communication with the bot within the chat/main.ss file. Within that file, any line starting with a + is a statement by the user (aka trigger), and the lines beginning with - are a response from the bot (aka reply).

So a super simple trigger and reply might look like…

+ hello
- hi

However, there are a lot of ways that you can customize how even simple trigger/replies are handled. Let's look at an example from MUTHUR.

+ {ordered} decode transmission
- UNIDENTIFIED.
- NOT A LANGUAGE.
- INITIAL READINGS INDICATE THAT COMMUNICATION IS A WARNING

I should note here that replies are in all caps solely to remain consistent with the interface in the Alien movie and not due to any requirement of SuperScript.

In this case, I have specified three replies to the trigger. Under normal circumstances, SuperScript would just randomly choose one, but I have used the {ordered} flag to indicate that I would like it to use all three responses in order. The result looks like this:

It's important to point out that in the normal course of conversation with the bot, it will throw out duplicate triggers. This means that if I were to keep throwing the decode transmission trigger at the bot, it would stop responding once it had run out of replies. SuperScript keeps the last ten triggers sent from a user and is designed to stop replying to repeated use of the same trigger. However, you can override this behavior by using the {keep} flag.

There are plenty of other ways to make triggers more flexible, including things like wildcards and optional text. We'll explore these a bit as we go on. You can learn more about basic replies in the documentation.

Complex Interactions

Let's look at how to do some slightly more advanced conversations using SuperScript.

Conversations

A typical conversation would involve multiple levels of discussion that build upon one another. For example, in the scene from Alien where Ripley has a conversation with MUTHUR about how to destroy the alien shown earlier.

Ripley: Request Clarification on Science Inability to Neutralize Alien

MUTHUR: Unable to Clarify

Special Order 937 – Science Officer Eyes Only

Ripley: Emergency Command Override 100375

MUTHUR: Special Order 937 Nostromo rerouted to new co-ordinates. investigate life form. Gather specimen. Priority One. Insure return of organism for analysis. All other considerations secondary. Crew expendable

Using this example, when Ripley enters the emergency command override, MUTHUR knows that it has to do with Special Order 937, which was from the prior trigger and response. Let's see how we can recreate this conversation within SuperScript.

+ request clarification on science inability to neutralize alien
- UNABLE TO CLARIFY

    + request enhancement
    % UNABLE TO CLARIFY
    - NO FURTHER ENHANCEMENT SPECIAL ORDER 937 SCIENCE OFFICER EYES ONLY

        + emergency command override 100375
        % NO FURTHER ENHANCEMENT SPECIAL ORDER 937 SCIENCE OFFICER EYES ONLY
        - ACKNOWLEDGED

            + what is special order 937
            % ACKNOWLEDGED
            - NOSTROMO REROUTED TO INVESTIGATE LIFE FORM\nINVESTIGATE LIFE FORM. GATHER SPECIMEN.\nPRIORITY ONE\nINSURE RETURN OF ORGANISM FOR ANALYSIS\nALL OTHER CONSIDERATIONS SECONDARY\nCREW EXPENDABLE
        
        + *
        % NO FURTHER ENHANCEMENT SPECIAL ORDER 937 SCIENCE OFFICER EYES ONLY
        - COMMAND NOT RECOGNIZED

Note that the conversation is indented for readability purposes, but this has no effect on the functionality.

As you can see, we can tie one trigger to a prior reply by using % and specifying the reply that the trigger is connected to. Multiple triggers can be tied to a single reply. For example, the last trigger was set as a catch all using * in the case someone enters the wrong emergency command override.

I should state that, in this case, my conversation was very fixed to specific statements in order to recreate the exact movie dialogue, but you'd likely want to use SuperScript's wildcards, alternates optionals and other tools to make the triggers much more flexible.

Learning About a User

At times, as you learn things about the user that are pertinent to the conversation, you'll want to retain that information. For instance, if the bot asked my name and I answered, I'd expect it to know my name from that point on rather than need to ask again. This can be done easily within SuperScript.

Legend on the screen…

DALLAS: What's my God damn key.

Print-out from computer answers…

01335 on the binary side.

DALLAS: Thank you Mother.

SuperScript has the concept of user knowledge, which will allow us to store and retrieve information about them. First, let's look at storing the information.

+ access code *1
-  COMMAND PRIORITY ACCESS ONLY. YOUR ACCESS CODE HAS BEEN AUTHORIZED. ^save("accesscode",<cap>)

The *1 in the trigger indicates that I expect one "word" to follow the access code statement. This wildcard is then captured by SuperScript. In my reply I save that captured item (i.e. <cap>) using the save method, and I've given the saved item a key of accesscode.

Technically, I could retrieve this key as easily as follows.

+ what is my [god damn] (access code|key)
- {keep} ^get("accesscode")

Note that I am using the {keep} flag here so that if the person asks again, they will get a response rather than have the trigger ignored as a duplicate.

First, I am using some new things in the trigger: alternates and optionals. This means that this specific trigger could be one of the following:

  • what is my god damn access code?
  • what is my god damn key?
  • what is my access code?
  • what is my key?

With a few simple changes, I've made the trigger way more flexible. Meanwhile the response is simply using the get function and passing the key that I used to save the access code. Simple.

Of course, I wanted to make it even more flexible. What if the bot doesn't know your access code yet? I wanted it to respond differently in this case. Doing this requires using a SuperScript plugin. So, let's look at how to do that.

Creating Plugins

A SuperScript plugin is essentially just functions living within a JavaScript module in the plugins folder of your project. Let's look at how we can use this to create a more advanced version of our access code response.

First, let's call the plugin in our reply.

+ what is my [god damn] (access code|key)
- {keep} ^accessCode()

In this case, I've replaced the reply text with a call to a method called accessCode that lives within a plugin (accesscode.js) in the plugins directory.

Here is the code for my plugin.

exports.accessCode = function accessCode(cb) {
    // if the access code has been set, return it, if not, respond differently
    this.user.getVar('accesscode', (e, ac) => {
    if (ac !== null) {
        cb(null, 'ACCESS CODE IS ' + ac);
        } else {
        cb(null, 'NOT YET AUTHORIZED');
        }
    });
}

The result looks like this:

Now, because it is accessing an internal function (getVar) within SuperScript, this plugin is slightly more complicated than typical. Under most circumstances you would simply call the callback function (i.e. the cb value that was passed in) with the text you want returned to the user. For example, you might call an external service. Let's look at another example that does just that.

For more details on accessing the internals of SuperScript from within a plugin, check the documentation.

Calling External APIs from Plugins

As part of my MUTHUR bot, I created a plugin that actually calls two external APIs to try to a) figure out your location based upon your IP and then b) get the sunrise based upon your location.

Dallas stares at the dark screens.

KANE: We can't go anywhere in this.

ASH: Mother says the sun's coming up in about twenty minutes.

The trigger and reply are simple.

+ (when|what time) is the sun coming up
- ^getSunrise()

Now let's look at the getSunrise method (I've placed this in a file named sunrise.js).

const http = require('http');
exports.getSunrise = function getSunrise(cb) {
    getLocation(cb);
}

Whao! That's it?! Well…no. All I'm doing here is passing along the callback function to another internal function in our plugin called getLocation. This method will use the FreeGeoIP API to try to determine the location (of the telnet server in this case, which is running locally anyway). This uses the http module in Node, which was imported in the earlier code.

function getLocation(cb) {
    var options = {
        host: 'freegeoip.net',
        port: 80,
        path: '/json/',
        method: 'GET'
    }

    http.request(options, function(res){
        var body = "";

        res.on('data', function(chunk) {
            body += chunk;
        });
        res.on('end', function(){
            var result = JSON.parse(body),
                lat,
                long;
            lat = result.latitude;
            long = result.longitude;
            getSunriseAPI(cb, lat, long);
        });
    }).end();
}

Now that I have the latitude and longitude, I am passing that information, along with the callback, to the next method: getSunriseAPI. This method uses the Sunrise/Sunset API to try to determine the time of the sunrise based upon the latitude and longitude.

function getSunriseAPI(cb, lat, long) {
    var options = {
        host: 'api.sunrise-sunset.org',
        port: 80,
        path: '/json?lat='+lat+'&lng='+long,
        method: 'GET'
    }

    http.request(options, function(res){
        var body = "";

        res.on('data', function(chunk) {
            body += chunk;
        });
        res.on('end', function(){
            var result = JSON.parse(body);
            cb(null, 'SUNRISE IS AT ' + result.results.sunrise + ' UTC');
        });
    }).end();
}

Once I get a response, I finally send a message back to the user via the callback function. Here's what it looks like in use.

An Advanced Plugin Example

Ok. I know I've covered a lot of ground here, but bear with me for one more fun (and slightly experimental) idea for what you can do with a plugin.

The sirens continue sounding.

MOTHER'S VOICE: Attention. Engines will overload in three minutes.

Ripley pushes a button and speaks into it.

RIPLEY: Mother, I've turned all the cooling units back on.

MOTHER'S VOICE: Too late for remedial action. The core has begun to melt. Engines will overload in two minutes, thirty-five seconds.

Under normal circumstances, the bot is designed to speak only when spoken too. However, to recreate the countdown timer, I needed to have the bot speak on it's own once the countdown was triggered.

Calling the plugin is no different than previous examples.

> topic selfdestruct
    + set timer *1
    - ^setDestruct(<cap>)

    + I've turned all the cooling units back on
    - TOO LATE FOR REMEDIAL ACTION. THE CORE HAS BEGUN TO MELT.
< topic

To create the countdown itself, I used a module called node-timers. The countdown would trigger an event on a customizable interval. The issue is that we can only use the callback once.

To overcome this, (with advice Rob Ellis, the creator of SuperScript) I tweaked the botHandle method within my telnet server.

bot.reply(socket.remoteAddress, message.trim(), (err, reply) => {
  // Find the right socket
  const i = sockets.indexOf(socket);
  const soc = sockets[i];

  soc.write(`\n> ${reply.string}\n`);
  soc.write('> ');
}, {ws: socket});

The key here is the {ws: socket} parameter to the reply method. Basically, this passes the socket being used as a data parameter so that I can then access it within my plugin using the extraScope property of this.

var countdown = require('node-timers/countdown');
exports.setDestruct= function setDestruct(tm, cb) {
    var timer = countdown({
        startTime:tm*60000,
        pollInterval:60000
    }),
    that = this;
    timer.on('poll', function(time) {
        var response = '';
        if (time > 0) {
            response = 'ATTENTION. ENGINES WILL OVERLOAD IN '+millisToMinutesAndSeconds(time)+' MINUTES.';
            // handle error if the user session doesn't exist anymore
            try {
                that.extraScope.ws.write(`\n> ${response}\n`);
                that.extraScope.ws.write('> ');
            }
            catch (e) {
                console.log('Message not posted. User has disconnected.\n');
            }
        }
        else {
            response = 'ENGINES OVERLOADED.'
            that.extraScope.ws.write(`\n> ${response}\n`);
        }
    });
    timer.start();
    cb(null,'THE EMERGENCY DESTRUCT SYSTEM IS NOW ACTIVATED. THE SHIP WILL DETONATE IN T MINUS ' + tm + ':00 MINUTES.')
}

In my selfDestruct method, I create an event listener based upon the timer, which is set to the time that the user passed in their trigger. At each poll interval, MUTHUR replies with the time remaining using the websocket connection until the timer runs out and she responds with "engines overloaded."

For the purposes of illustration, I've sped the above GIF up to 10 seconds between each reply.

I should also note that Rob Ellis warned that there could be potential side effects of forking the conversation in this way – so a method of this sort in a production application would require some deeper testing.

Tips

Before I leave you to start off creating your version of WOPR from WarGames, I wanted to share a few helpful tips to overcome some stumbling blocks I ran into while building a bot with SuperScript.

  • Open up the log in the logs folder in a log viewer. It offers valuable information about how SuperScript is trying to match a gambit (i.e. a command from the user) including the normalized input. SuperScript is smart enough to normalize things like, for example, fixing "what's my key?" to "what is my key."
  • Keep in mind that if your plugin fails in some way, SuperScript will simply act as though it did not match a gambit. You'll need to check your console tab that is running the telnet server to get details on the error it encountered.
  • The Slack channel for SuperScript is immensely helpful. The people behind the project, Rob Ellis and Ben James are incredibly open to helping and, quite honestly, a lot of what I was able to do here was thanks to their willingness to assist.

Conclusion

I covered a lot here (and you made it all the way to the end, congrats!), but, honestly, there is a lot more to SuperScript than I show here. The project's documentation details many of these other topics.

Obviously, recreating classic movie AI, while critically important, isn't the only use for bots. Personally, one future I see for bots is as a way to simplify input and interaction. For instance, forms can sometimes be overwhelming, with multiple pathways all represented in a single, complex form page. But bots potentially present a more comfortable way of data collection that could potentially hide the complexity of a form via a naturally branching conversation. That's just one idea, I know there are many more…so go out and have some fun creating your own.

The GitHub project for this article is available at github.com/remotesynth/muthurBot

Comments