A web app built with Rust and Python
Tags: rust, python
Published: 2020-04-25 21:25:36

When I started learning Rust I took a few of my Python projects and rewrote the code with this new language I was checking but then I found out that there wasn't a real alternative for some of the libraries I was using.

After searching on [crates.io](https://crates.io), two crates got my attention, [CPython](https://github.com/dgrunwald/rust-cpython) and [PyO3](https://pyo3.rs), you can use both for writing Python native modules and running and interacting with Python.

While you can find some examples in their corresponding GitHub repository on how to use Python within your Rust projects, I didn't find so much information on how to call a Python function (in an external library, on a `.py` file) from Rust, that returns a value or a set of values to be processed.

An answer on a [StackOverflow's post](https://stackoverflow.com/questions/51409744/modulenotfounderror-when-importing-a-python-module-using-the-cpython-crate#51415918) helped me preparing a practical example I presented at a local Google I/O Extended event, a Rust web app that connects to Firebase through Python, retrieve data and print it on a HTML document.

By the time I proposed the talk "Web Apps with Rust & Firebase" for Google I/O Extended last year, I didn't know that there isn't a crate to connect to Firebase that actually works. So I started to think about the possibility to build the web app with Rust and only use Python for connecting to Firebase.

Let me explain how I built the project.

## Configuration
First of all I created a new Rust project using Cargo:

```bash
cargo new rust-python-demo
```

The above command will create a new directory with the name `rust-python-demo`, inside this directory you will find a `src` dir that contains a `main.rs` file that will later be modified for writing the code of the app, and the file `Cargo.toml` that is the manifest of the project.

Don't forget to change to the new directory:

```bash
cd rust-python-demo
```

### Cargo.toml
**Cargo.toml** must be modified to look as follows:

```toml
[package]
name = "rust-python-demo"
version = "0.1.0"
authors = ["mattdark"]
edition = "2018"

[dependencies]
serde = "1.0.99"
serde_derive = "1.0.99"
serde_json = "1.0.40"
rocket = "0.4.2"

[dependencies.cpython]
version = "0.4"
features = ["python-3-7"]

[dependencies.rocket_contrib]
version = "0.4"
features = ["handlebars_templates"]

[[bin]]
name = "rust-python-demo"
path = "src/main.rs"
```

The section `[package]` contains name and version of the app, information of the developer and edition of Rust being used, 2015 or 2018.

`[dependencies]` section has the list of dependencies required for the project, I'm using [Rocket](https://rocket.rs), a web framework for Rust, and [Serde](https://serde.rs), a framework for serializing and deserializing Rust data structures. For CPython you can specify the version of Python you'll be using, and as Rocket has support for both Handlebars and Tera (template engines), you must indicate the one being used for the project, in this case Handlebars.

To assign a specific name to the binary of the app, you can do it in the `[[bin]]` section, indicating the name of the source code file.

### pyproject.toml
**pyproject.toml** is the file that Poetry uses to manage a Python project and its dependencies. It contains information about the project like name, version, description and authors, in the `[tool.poetry]` section. The version of Python and list of dependencies must be specified in the `[tool.poetry.dependencies]` section, for this project I'm using any Python version greater than or equal to `3.7.3` and less than `3.8`. To connect to Firebase I'm using the `firebase` library and the rest of packages are dependencies of it.

The file should look as follows:

```toml
[tool.poetry]
name = "rust-python-demo"
version = "0.1.0"
description = ""
authors = ["Mario Garcia <iscmariog@gmail.com>"]

[tool.poetry.dependencies]
python = "^3.7.3"
firebase = "*"
python-jwt = "*"
gcloud = "*"
sseclient = "*"
pycrypto = "*"
requests-toolbelt = "*"

[tool.poetry.dev-dependencies]
pytest = "^5.2"

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"
```

### Python
Why using a specific Python version other than the one available on your system or the one that you installed from the repositories? Some Linux distributions, specially those rolling release ones, will offer the latest stable version of Python available, 3.8.2 at the moment of writing this article.

Other distros will have the latest release of Python 3.6 or Python 3.7 but you will probably need a different version depending on the compatibility and support of the libraries you're using for the project.

So this is the moment when pyenv can help, as it makes possible to have other versions of Python installed on your system in a friendly way.

To list the versions of Python available through pyenv, run the following command:

```bash
pyenv install --list
```

For this project I'm using Python 3.7.7. Before installing the version of Python you'll be using for the project bear in mind that if you want to embed Python in Rust, shared libraries must be built for Python. Run the following command to install Python properly:

```bash
env PYTHON_CONFIGURE_OPTS="--enable-shared" pyenv install 3.7.7
```

After running the above command add the shared libraries directory to the LD_LIBRARY_PATH environment variable as follows:

```bash
export LD_LIBRARY_PATH=~/.pyenv/versions/3.7.7/lib/
```

### Poetry
Before installing the dependencies of the project you must specify the version of Python that will be used for Poetry to create the virtual environment of the project, run the following command:

```bash
poetry env use 3.7.7
```

Then install the dependencies of the project by running:

```bash
poetry install
```

This command will also create a virtual environment.

Note: If you don't require a virtual environment for your project, you can run the following command before `poetry install`:

```bash
poetry config virtualenvs.create false
```

### Rust
CPython and PyO3 works with the Nightly version of Rust, so before writing the code of your app change the toolchain of the project by running:

```bash
rustup override set nightly
```

## Understanding the code
### Directory structure
After configuring the app you will have `Cargo.toml`, `pyproject.toml` and the `main.rs` file in the `src` directory that it will be modified later to put the code of the app.

In the `src` directory a `python` dir must be created to put the Python module that will make the connection to Firebase and retrieve the data from it, named `pyrebase.py`.

In the root directory the `templates` and `static` dirs should be added. The Handlebars templates and the static files (e.g. CSS, JavaScript, Pictures) will be stored in those dirs.

### Firebase
Go to [console.firebase.google.com](https://console.firebase.google.com) and add a new project, I will name it **Rust Python Demo**.

Then add Firebase to a web app, you will get the following configuration details that you will need to configure the Python module:
```javascript
var firebaseConfig = {
    apiKey: "APIKEY",
    authDomain: "rust-python-demo.firebaseapp.com",
    databaseURL: "https://rust-python-demo.firebaseio.com",
    projectId: "rust-python-demo",
    storageBucket: "rust-python-demo.appspot.com",
    messagingSenderId: "SENDERID",
    appId: "APPID"
  };
```

And finally go to **Database** from the left sidebar and **Create database**. For this demo I will store a list of random names generated through [listofrandomnames.com](http://listofrandomnames.com/).

The structure of the database will look as follows, you can import the following JSON to the database for testing:

#### firebase.json
```json
{
  "speakers" : {
    "speaker1" : {
      "id" : "1",
      "last_name" : "Moore",
      "name" : "Madie"
    },
    "speaker2" : {
      "id" : "2",
      "last_name" : "Nathanson",
      "name" : "Norbert"
    },
    "speaker3" : {
      "id" : "3",
      "last_name" : "Mcconelle",
      "name" : "Major"
    },
    "speaker4" : {
      "id" : "4",
      "last_name" : "Schalk",
      "name" : "Sophia"
    },
    "speaker5" : {
      "id" : "5",
      "last_name" : "Bertrand",
      "name" : "Benny"
    }
  }
}
```

The name of the database is **speakers** as this demo is part of a web app that I'm working on.

Don't forget to configure the rules of your database as follows:

```
{
  "rules": {
    ".read": true,
    ".write": false
  }
}
```

### Python
#### src/python/pyrebase.py
```bash
import json
from firebase import Firebase
def read_data(self):
    config = {
        "apiKey": "APIKEY",
        "authDomain": "rust-python-demo.firebaseapp.com",
        "databaseURL": "https://rust-python-demo.firebaseio.com",
        "projectId": "rust-python-demo",
        "storageBucket": "rust-python-demo.appspot.com",
        "messagingSenderId": "MESSAGINGSENDERID"
    }
    firebase = Firebase(config)

    speaker = list()
    db = firebase.database()
    all_speakers = db.child("speakers").get()
    for x in all_speakers.each():
        speaker.append(x.val())
    s = json.dumps(speaker)
    return s
```

For this module I'm using the `json` and `firebase` libraries. It only has a function named **read_data()** that will connect to Firebase and retrieve the data from the database.

Inside the function I defined the **config** variable that has the configuration details required to connect to Firebase.

After connecting to Firebase I get all the registries stored in the database and append it to the **speaker** list and convert that list to a JSON structure for Rust to import this data properly.

### Rust
#### src/main.rs
```rust
#![feature(proc_macro_hygiene, decl_macro)]

#[macro_use] extern crate rocket;

extern crate rocket_contrib;
extern crate cpython;

use cpython::{Python, PyResult, PyModule};

#[macro_use]
extern crate serde_derive;
extern crate serde_json;

use std::collections::HashMap;
use std::path::{Path, PathBuf};
use crate::handlebars::{to_json};

use rocket::response::NamedFile;
use rocket_contrib::templates::{Template, handlebars};

#[derive(Serialize, Deserialize, Debug)]
pub struct Speakers {
    pub id: String,
    pub name: String,
    pub last_name: String,
}

const FIRE_PY: &'static str = include_str!("./python/pyrebase.py");

#[get("/")]
fn index() -> Template {
    let gil = Python::acquire_gil();
    let py = gil.python();
    let s = run_python(py).unwrap();
    let mut data = HashMap::new();
    data.insert("speakers".to_string(), to_json(&s));
    Template::render("index", &data)
}

fn run_python(py: Python<'_>) -> PyResult<Vec<Speakers>> {
    let m = module_from_str(py, "pyrebase", FIRE_PY)?;
    let out: String = m.call(py, "read_data", (2,), None)?.extract(py)?;
    let speakers: Vec<Speakers> = serde_json::from_str(&out).unwrap();
    Ok(speakers)
}

fn module_from_str(py: Python<'_>, name: &str, source: &str) -> PyResult<PyModule> {
    let m = PyModule::new(py, name)?;
    m.add(py, "__builtins__", py.import("builtins")?)?;

    let m_locals = m.get(py, "__dict__")?.extract(py)?;
    py.run(source, Some(&m_locals), None)?;
    Ok(m)
}

#[get("/<file..>", rank=3)]
fn files(file: PathBuf) -> Option<NamedFile> {
    NamedFile::open(Path::new("static/").join(file)).ok()
}

fn rocket() -> rocket::Rocket {
    rocket::ignite().mount("/", routes![index, files])
    .attach(Template::fairing())
}

fn main() {
    rocket().launch();
}
```

At first, in the `main()` function the Rocket app is run through the `rocket()` function where the routes of the app are mounted and the template Fairing is attached to the app.

As it only loads a HTML document, it has two routes, `index` and `files`, the first one points to the `index()` function where the root of the app is loaded and the last one pointing to the `files()` function where the static files in the `static` directory is loaded.

In the `index()` function, the Python interpreter is embedded and the reference to the Python binary is assigned to the `py` variable, in the first two lines.

Then the function `run_python()` is called. In the first line the `module_from_str()` function is called, where the `pyrebase` module is loaded.

After that the `read_data()` Python method is called and the value returned by it is assign to the `out` variable. Then the `out` variable is deserialize and the values are converted into a vector using the struct `Speakers`.

The vector `speakers` is returned to the `index()` function where it was called and converted into a HashMap to pass the values to the `index` template.

### Template
#### templates/index.html.hbs
```html
<body>
	<div class="limiter">
		<div class="container-table100">
			<div class="wrap-table100">
				<div class="table100">
					<table>
						<thead>
							<tr class="table100-head">
								<th class="column1">ID</th>
								<th class="column2">Name</th>
								<th class="column3">Last Name</th>
							</tr>
						</thead>
						<tbody>
							{{ #each speakers }}
								<tr>
									<td class="column1">{{ id }}</td>
									<td class="column2">{{ name }}</td>
									<td class="column3">{{ last_name }}</td>
								</tr>
							{{ /each }}
						</tbody>
					</table>
				</div>
			</div>
		</div>
	</div>
...
</body>
```

The `index` HTML document will only display the data retrieved from the database in a table.

### Running the app
The code of the app is written so it's time to run the demo built.

The project must be built first, run the following command from the terminal:

```bash
poetry run cargo build --release
```

As the app is being run through a virtual env created by poetry, the `poetry run` command must be executed first and then the project is built for production with `cargo build --release`.

Cargo created a binary name `rust-python-demo` in the `target/release` directory. To run the app write the following command in the terminal:

```bash
poetry run ./target/release/rust-python-demo
```

Now go to `localhost:8000` in your browser. You will see the following screen.
![Rust Python Demo](https://dev-to-uploads.s3.amazonaws.com/i/zes6sy36ejz252qrswsu.png)

The demo of the web app built with Rust and Python is up and running, to get the code go to [gitlab.com/mattdark/rust-python-demo](https://gitlab.com/mattdark/rust-python-demo).