Building a Terminal-based Chatbot in Rust from Scratch
Tags: chatbot, rust, openai, gpt
Published: 2024-02-24 13:52:00

## Introduction

This is going to be a learning experience for me, and I hope it will be for you too. It's easy to get stuck in your comfort zone, and I find it helpful (and refreshing!) to break out occasionally.

I'm a JavaScript/TypeScript developer mostly. It's my day job, and it's what I find fastest and easiest for side projects. But I'm getting that itch to learn something new, so this project is designed as an intro to Rust.

Rust is a quite low-level programming language with some really interesting concepts surrounding memory management. This post will serve as a light introduction to the language in a practical way. We're going to build a functional chatbot together, that you can use in your terminal.

You can find the [full code here](https://github.com/andyjessop/rust-terminal-chatbot), so just in case you miss something during the post, you will always have a full working reference.

Right, let's dive in!

## Setting up the Rust toolchain

**NB: If you already have Rust setup, you can skip this section.**

The Rust toolchain is managed by `rustup`. Let's get that installed:

```bash
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
```

The toolchain includes these three binaries:

- `rustup` - The Rust toolchain installer and version management tool.
- `rustc` - The Rust compiler, used to compile Rust code into executable binaries.
- `cargo` - The Rust package manager and build system, used for managing Rust projects and dependencies.

They should be installed now at `~/.cargo/bin`, and we'll add this to our `PATH` as a convenience when running them. In a bash environment, you can append the following line to your .bashrc or .bash_profile file:

```bash
export PATH="$HOME/.cargo/bin:$PATH"
```

To get that applied, either restart your terminal client or run:

```bash
source ~/.bashrc
```

Now, confirm that you have everything installed correctly:

```bash
rustup -V
# rustup 1.26.0 (5af9b9484 2023-04-05)

rustc -V
# rustc 1.76.0 (07dca489a 2024-02-04)

cargo -V
#cargo 1.76.0 (c84b36747 2024-01-18)
```

If you don't see something like the above, please check the official docs to get your toolchain setup correctly.

If you do, congratulations, you're now a Rustacean (well...nearly).

## Creating the Project

In Rust, `cargo` is used to manage projects their builds. We're going to use it to create the project, which we'll call `ask`, and then `cd` into the project folder.

```bash
cargo new ask --bin
cd ask
```

Notice we used the `--bin` flag, this is because we want to setup the project to build a binary file instead of a library.

If you inspect your project now, you will see that `cargo` has kindly created the following files for us.

```bash
cargo.toml
.gitignore
- src
  main.rs
```

The `cargo.toml` is our project's definition. Inside here, we can define what dependencies it uses and some other package information. At the moment, it should look like this:

```toml
[package]
name = "ask"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
```

This is all quite obvious, so I won't go further into it. The `main.rs` file is similarly uninteresting at the moment.

```rust
fn main() {
    println!("Hello, world!");
}
```

Let's just check everything's working properly before we move onto the meat of this post - the chatbot itself. In order to build the project and run it, we use `cargo` once again. The command `cargo run` will build and run the `main.rs` file for us. I wonder if you can guess what will be output to the terminal...I know, it's a tough one...

```bash
cargo run
   Compiling ask v0.1.0 (/Users/andy/dev/ask)
    Finished dev [unoptimized + debuginfo] target(s) in 0.82s
     Running `target/debug/ask`
Hello, world!
```

This is a momentous occasion. We've setup Rust correctly, and built our first Rust binary.

We could also have just run `cargo build`, but that wouldn't have run it too, and we wanted to see the output in this case. You can therefore think of `cargo run` as `cargo build` plus executing the binary that is produced. In this post, we'll mostly be using `cargo run`.

## Scaffolding Out the Chatbot: Dependencies

For the initial scaffolding, we're going to bring in a few packages, or "crates". I'll explain why we need each of them, but first, let's add them to the dependencies section in our `cargo.toml`:

```toml
reqwest = { version = "0.11", features = ["json"] }
tokio = { version = "1", features = ["full"] }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
dotenv = "0.15.0"
```
- `reqwest` - This is an asynchronous HTTP client used for making HTTP requests. The `"json"` feature is enabled to facilitate the handling of JSON data, which we'll need for the OpenAI API.

- `tokio` - An asynchronous runtime for Rust, providing features for writing asynchronous applications. The Rust standard library provides async/await, but it doesn't provide the runtime itself. I suppose this will come in a later version, but for now we need to use `tokio`. Don't worry, it's just one easy declaration.

- `serde` - A serialisation and deserialisation framework. The `"derive"` feature allows us to apply serialisation and deserialisation to our rust `struct`s, so that they can be converted - for example -  to and from JSON without us having to manually implement this formatting.

- `serde_json` - An extension of `serde` for JSON data handling, providing utilities to convert between JSON and Rust data structures. Often used in conjunction with `reqwest` and `serde` for web API interactions.

- `dotenv` - A library for loading environment variables from a `.env` file, where we'll be storing the OpenAI API key.

Ok, we've added our dependencies, now let's start building out the `main.rs` file.

## Scaffolding Out the Chatbot: Boilerplate

We'll add some boilerplate first to handle things like:
- loading the API key from .env
- setting up the `reqwest` client
- declaring the use of the `tokio` async runtime

Replace the existing contents of your `main.rs` file with this:

```rs
use reqwest::Client;
use dotenv::dotenv;

#[tokio::main]
async fn main() -> Result<(), String> {
    dotenv().ok();

    let client = Client::new();

    Ok(())
}
```

I love the way Rust has borrowed some of the best features from other programming languages. Here you'll see parts that remind you of PHP, JS, C, and others.

But what is `#[tokio::main]`? This is called an attribute, and is used to apply metadata to functions, structs or other items. Here it's telling the Rust compiler to enable asynchronous functionality in the function immediately following - in this case it's the `main` function.

`dotenv().ok();` is fairly self-explanatory, we're pulling in the environment variables from `.env` (which currently doesn't exist, but we'll change that in a second).

Then we're instantiating an HTTP client for use later.

Finally, we'll return `Ok(())`. Note that in Rust you can do this: `return Ok(());`, but you can also return just by removing the semicolon and the `return` keyword, which is what we're doing here.

Ok, moving on! We're finally going to start doing some real programming.

## Implementing the Chat Interface

### Adding Structs for OpenAI Communication

Firstly, we need to define some structures (structs) that will help us manage the data we send to and receive from the OpenAI API. These structs will represent the request and response formats expected by the API.

Insert the following code below your imports and above your `main` function:

```rust
#[derive(Serialize)]
struct OpenAIChatRequest {
    model: String,
    messages: Vec<Message>,
}

#[derive(Serialize, Deserialize, Clone)]
struct Message {
    role: String,
    content: String,
}

#[derive(Deserialize)]
struct OpenAIChatResponse {
    choices: Vec<ChatChoice>,
}

#[derive(Serialize, Deserialize)]
struct Conversation {
    messages: Vec<Message>,
}

#[derive(Deserialize)]
struct ChatChoice {
    message: ChatMessage,
}

#[derive(Deserialize)]
struct ChatMessage {
    content: String,
}
```

Here’s what each struct represents:

- `OpenAIChatRequest` - The data structure for sending a request to OpenAI, containing the model you want to use and the messages of the conversation.
- `Message` - Represents a single message in the conversation, with a role (user or assistant) and the message content.
- `OpenAIChatResponse` - The structure expected in the response from OpenAI, primarily containing a list of choices.
`Conversation` - Used to keep track of the ongoing conversation in memory.
- `ChatChoice` and `ChatMessage` - Nested within OpenAIChatResponse, representing individual responses from the AI.

### Implementing the Conversation Logic

Let’s build the core interaction loop where the user can input questions, and the program adds them to the conversation and requests responses from OpenAI.

Replace the `main` function with the following:

```rust
#[tokio::main]
async fn main() -> Result<(), String> {
    dotenv().ok();
    let client = Client::new();
    let mut conversation = Conversation {
        messages: Vec::new(),
    };

    println!("\nWhat do you want to talk about today?\n");

    loop {
        println!("You:");
        let mut question = String::new();
        stdin().read_line(&mut question).map_err(|e| e.to_string())?;
        let question = question.trim();

        if question.eq_ignore_ascii_case("exit") {
            break;
        }

        conversation.messages.push(Message {
            role: "user".to_string(),
            content: question.to_string(),
        });

        let response = ask_openai(&client, &mut conversation).await?;
        println!("\nAI: {}", response);
    }

    Ok(())
}
```

In this updated `main` function, we've created a loop that prompts the user to input their question. If the user types "exit", the loop breaks, and the program ends. Otherwise, the question is added to the conversation and sent to the OpenAI API for a response.

I won't go into all the details here, because there are already a lot of things going on here, and [The Rust Programming Language book](https://doc.rust-lang.org/book/ch00-00-introduction.html) is _far_ better than me at explaining. I highly recommend giving it a read - it's really a work of art. And frankly, it needs to be because some of these memory management concepts take a fair amount of brain power initially. However, I suspect that once you "get it", Rust will become very natural.

```rust
async fn main() -> Result<(), String>: {
```

The `async` keyword makes main an asynchronous function. This basically allows you to use `.await` on "futures", which are like `Promises` from JS.

`Result<(), String>` is the return type of the function, and is a really common Rust pattern for error handling. In this case, `main` either returns `Ok(())` indicating success, or an `Err(String)` containing an error message. This is a simplification for (my) educational purposes; in larger applications, you might use more specific error types.

```rust
let mut question = String::new();
```

Here, Rust's ownership rules ensure that `question` is properly allocated and deallocated, preventing memory leaks. The `question` instance is owned by the loop block and is dropped (freed) when it goes out of scope. This is a core feature of Rust's memory safety and frees us from having to handle memory release explicitly.

Using `.map_err(|e| e.to_string())?` converts errors from one type to another (`reqwest::Error` to `String`). The `?` operator is syntactic sugar for early returns in case of an `Err`.

### Crafting the `ask_openai` Function

Now, we need to define the `ask_openai` function, which sends the current state of the conversation to OpenAI and retrieves the AI's response.

Insert this function below your `main` function:

```rust
async fn ask_openai(client: &Client, conversation: &mut Conversation) -> Result<String, String> {
    let request_body = OpenAIChatRequest {
        model: "gpt-3.5-turbo".to_string(), // Specify the OpenAI model you want here
        messages: conversation.messages.clone(),
    };

    let response = client
        .post("https://api.openai.com/v1/chat/completions")
        .header("Content-Type", "application/json")
        .header(
            "Authorization",
            format!("Bearer {}", env::var("OPENAI_API_KEY").expect("OPENAI_API_KEY not set")),
        )
        .json(&request_body)
        .send()
        .await
        .map_err(|e| e.to_string())?;

    let response_body = response
        .json::<OpenAIChatResponse>()
        .await
        .map_err(|e| e.to_string())?;

    if let Some(choice) = response_body.choices.last() {
        conversation.messages.push(Message {
            role: "assistant".to_string(),
            content: choice.message.content.clone(),
        });
        Ok(choice.message.content)
    } else {
        Err("No response from AI".to_string())
    }
}
```

In `ask_openai`, we construct the request with the current conversation and send it to OpenAI using the `reqwest` client. We then parse the response and add the AI's reply back into the conversation. If the API call is successful, the function returns the AI's response; otherwise, it returns an error.

The function takes `&Client` (a reference to `Client`) and `&mut Conversation` (a mutable reference to `Conversation`). Rust's borrow checker ensures that while `client` is immutably borrowed (you can't modify it within `ask_openai`), conversation can be modified.

Rust's clever borrow checker automatically infers the lifetimes here, ensuring that the references to client and conversation do not outlive the data they refer to. This prevents dangling references, a common source of bugs in low-level languages like C/C++.

```rust
let response_body = response
    .json::<OpenAIChatResponse>()
    .await
    .map_err(|e| e.to_string())?
```

The `.await` pauses the function's execution until the network request completes, without blocking the entire thread. Notice that we have to map the error to a `String` at the end. This is because we're only returning simple strings as errors. For more complex error handling, I've heard good things about the [`anyhow` crate](https://docs.rs/anyhow/latest/anyhow/), which as I understand it will simplify handling more complex error structures and will avoid the need to cast the response to a string as I'm doing here.

Oh, I nearly forgot, you need to get an OpenAI API key if you haven't already, and store it in a `.env` file at the root of the project. Head over to [OpenAI](https://openai.com/) and login there to get hold of your key.

```bash
OPENAI_API_KEY=your_open_ai_key
```

**Remember to add `.env` to the `.gitignore` because you don't want to commit your API key otherwise you could wake up to a nasty financial surprise!**

Right, I think we're ready to build and run this thing!

## Building and Running the Project

Let's test it out:

```text
cargo run
   Compiling ask v0.1.0 (/Users/andy/dev/ask)
    Finished dev [unoptimized + debuginfo] target(s) in 2.49s
     Running `target/debug/ask`

What do you want to talk about today?

You:
Let's talk about you!

AI:
Sure, what would you like to know about me?

You:
What's your name?

AI:
I am a language model AI created by a team of developers. You can call me Assistanto.

You:
Haha, you're so funny Assistanto.

AI:
Thank you! I'm here to help and hopefully bring a smile to your face as well.

You:
Bye!

AI:
Goodbye! Feel free to come back anytime if you have more questions or just want to chat. Have a great day!
```

Well, it looks like Assistanto is working. Congratulations if you've made it this far!

There's so much we haven't touched on here. Testing, publishing, etc. And we haven't had to use [lifetimes](https://doc.rust-lang.org/book/ch10-00-generics.html) at all either. Rust is a rich and expressive language that is also extremely powerful and actually quite fun too.

I hope you enjoyed this little tutorial, I certainly enjoyed learning some Rust in the process. Next up, I'm not sure whether to dive deeper into Rust or perhaps write a chatbot in Go by way of comparison. Maybe you can suggest something in the comments?