How to Install a Self-Hosted CSML Engine on Ubuntu 18.04
Tags: csml, ubuntu, opensource
Published: 2021-01-25 16:16:33

CSML is a compelling open-source language and conversational engine to create rich chatbots. Did you know that you can also install and host it by yourself for free as a standalone web server on your own server? Let's find out how in this article!

Installation
------------

On [CSML Engine's github repository](https://github.com/CSML-by-Clevy/csml-engine), we compile pre-built binaries for MacOS and Linux of the CSML Engine web server, called **CSML Server**, at [every release](https://github.com/CSML-by-Clevy/csml-engine/releases), under the _assets_ section ([we also maintain docker images if you prefer](https://hub.docker.com/r/clevy/csml-engine)). As in this example, we are going to install CSML Server on Ubuntu 18.04 (or most recent linux distributions such as Debian, CentOS, Amazon Linux 2...), we will use the **linux-based 64-bit binary**, called `csml-server-linux-amd64`. The latest [release to date is v1.4.0](https://github.com/CSML-by-Clevy/csml-engine/releases/tag/v1.4.0): let's [copy the address of the corresponding asset](https://github.com/CSML-by-Clevy/csml-engine/releases/download/v1.4.0/csml-server-linux-amd64).

![](https://blog.csml.dev/content/images/2021/01/image-2.png)

Right Click > Copy Link Address

To download the CSML Server executable on your own server, type the following command in your terminal:

    $ wget https://github.com/CSML-by-Clevy/csml-engine/releases/download/v1.4.0/csml-server-linux-amd64

![](https://blog.csml.dev/content/images/2021/01/image-4.png)

After the file is downloaded, you can simply make it executable, rename it and run it directly:

    $ chmod +x csml-server-linux-amd64
    $ mv csml-server-linux-amd64 csml-server
    $ ./csml-server
    > CSML Server listening on port 5000

Congratulations! It runs 🎉. Now go visit your server's IP on port 5000, which is in my case [http://34.241.38.167:5000/](http://34.241.38.167:5000/) (but it will very likely be a different IP for you).

![](https://blog.csml.dev/content/images/2021/01/image-5.png)

CSML Server's default homepage

If you can see this page, CSML is correctly installed and runs properly on your machine. But we are not there yet! We do have a bit of configuration to do. You can for now simply hit `ctrl+C` to kill the process.

Configuration
-------------

CSML Server can easily be configured with environment variables. In fact, the environment variables are documented on this very landing page, so it should be easy to do!

The CSML Engine requires a database, and works either with MongoDB or Amazon DynamoDB. There are advantages and drawbacks to each of these offerings; in this example, we are going to use MongoDB, which is a popular choice.

We will not cover how to install MongoDB in this article, but you can find instructions on [how to setup MongoDB](https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/) in their documentation, or you can also use one of the many ready-to-use SaaS offerings by some hosting providers. For example, MongoDB themselves have [Atlas](https://www.mongodb.com/cloud/atlas), a managed, cloud-based MongoDB service.

There are many ways to load environment variables in your shell. Let's simply create a simple `.env` file next to the server executable and copy-paste the default environment variables in this file for your setup:

    ENGINE_DB_TYPE=mongo
    
    MONGODB_HOST=localhost
    MONGODB_PORT=27017
    MONGODB_DATABASE=csml
    MONGODB_USERNAME=
    MONGODB_PASSWORD=
    
    ENGINE_SERVER_PORT=5000
    
    ENCRYPTION_SECRET=some-secret-string
    DISABLE_SSL_VERIFY=false
    DEBUG=true

Of course, you are free to change these variables to accommodate your own setup. Among other things, you should also obviously pick your own `ENCRYPTION_SECRET` and not leave the default value!

Then, you can run this command to export the variables from this file into your shell:

    $ export $(cat .env | xargs)

To check that the environment is correctly set, you can use the `env` command and verify that everything looks right.

Then, re-run the `./csml-server` command. The result will be the same, but now your server should be correctly connected to your database and ready to use!

Using your Self-Hosted CSML Server
----------------------------------

To test if your CSML Server installation works properly, you can run a simple query, for example by saying "Hi!" to a sample bot:

    curl -X "POST" "http://YOUR_SERVER_IP:ENGINE_PORT/run" \
         -H 'content-type: application/json' \
         -d $'{
      "bot": {
        "default_flow": "Default",
        "id": "mybot",
        "name": "MySuperBot"
        "flows": [
          {
            "id": "e0a13373-2037-4590-8018-ab14e74b27a1",
            "content": "start:\\n\\tsay \\"Hello from CSML Server!\\"\\ngoto end",
            "commands": [
              "/default"
            ],
            "name": "Default",
            "description": "Default custom flow"
          }
        ]
      },
      "event": {
        "metadata": {
          "some": "info",
          "about": "the current user"
        },
        "payload": {
          "content": {
            "text": "Hi!"
          },
          "content_type": "text"
        },
        "request_id": "67d283bd-35d5-4744-be61-2f063573022f",
        "client": {
          "user_id": "myuser",
          "channel_id": "mychan",
          "bot_id": "mybot"
        }
      }
    }'

A sample request to a CSML Server instance

The server should respond with something like:

    {
      "messages": [
        {
          "payload": {
            "content_type": "text",
            "content": {
              "text": "Hello from CSML Server!"
            }
          },
          "interaction_order": 0,
          "conversation_id": "555416aa-c033-4676-ad64-eaf5c6783617",
          "direction": "SEND"
        }
      ],
      "conversation_end": true,
      "request_id": "67d283bd-35d5-4744-be61-2f063573022f",
      "received_at": "2021-01-22T20:58:40.154Z",
      "interaction_id": "591c2ad1-b540-47b0-bd5c-9a5a3907cd0f",
      "client": {
        "bot_id": "mybot",
        "user_id": "myuser",
        "channel_id": "mychan"
      }
    }

A sample response from CSML Server

You can find the full REST API documentation in OpenAPI v3 format [on our Github repository](https://github.com/CSML-by-Clevy/csml-engine/blob/master/csml_server/swagger.yaml).

Next Steps: Prepare For Production
----------------------------------

This article gives you an easy starting point to install the CSML Engine on your own server. The instructions are similar if you plan to use your own development machine as well. (By the way, we also provide a MacOS (intel) binary.)

However, if you need to run your own CSML Server for a production payload, there are probably a few extra steps that you should take (or just use [CSML Studio](https://studio.csml.dev), the online CSML development environment that does not need any installation). You will probably need to:

*   add a reverse-proxy such as nginx and enable HTTPS for secure exchanges
*   add some redundancy to make sure that it is always available
*   make sure that your service is resilient and handles the loss of one or more processes gracefully
*   add some auto-scalability into the mix, to make sure that you are ready to serve the right number of simultaneous conversations for your use case

Depending on how you plan to distribute your bot, there are many good ways to do it. If you are looking for architecture advice, please [reach out on Slack](https://csml.dev/slack)! We are running millions of conversations securely on CSML Studio and can probably point you in the right direction.