TensorFlow Model Deployment using FastAPI & Docker
Tags: machinelearning, deeplearning, fastapi, docker
Published: 2021-04-02 17:10:27

<img alt="TensorFlow" src="https://img.shields.io/badge/TensorFlow%20-%23FF6F00.svg?&style=for-the-badge&logo=TensorFlow&logoColor=white" />

<img alt="Docker" src="https://img.shields.io/badge/docker%20-%230db7ed.svg?&style=for-the-badge&logo=docker&logoColor=white"/>

<img alt="Python" src="https://img.shields.io/badge/python%20-%2314354C.svg?&style=for-the-badge&logo=python&logoColor=white"/>


__TL; DR:__ 
In this article, we are going to build a _TensorFlow model (v2)_ and using FastAPI create REST API calls to predict from the model, and finally containerize it using _Docker_ :smiley:


I want to emphasize the usage of __FastAPI__ and how rapidly this framework is a game-changer for building easy to go and much faster API calls for a machine learning pipeline.
Traditionally, we have been using Flask Microservices for building REST API calls but the process involves a bit of nitty-gritty to understand the framework and implement it.
On the other end, I found FastAPI to be pretty user-friendly and very easy to pick up and implement type of framework.

And finally from one game-changer to another, __Docker__ 
As a data scientist: our role is vague and it keeps on changing year in year out. Some skillset gets added, some get extinct and obsolete, but Docker has made its mark as one of the very important and most sought out skills in the market. _Docker_ gives us the ability to containerize a solution with all its binding software and requirements.

__The Data__

We have used a text classification problem : [IMDb Dataset](https://www.kaggle.com/lakshmi25npathi/imdb-dataset-of-50k-movie-reviews) for the purpose of building the model.

The dataset comprises 50,000 reviews of movies and is a binary classification problem with the target variable being a _sentiment_: positive or negative.

__Preprocessing__

We use Tensorflow's _TextVectorization_ layer which tidies things up and outputs a layer which we will be using in the process of creating a graph on a Sequential or Functional Model.


```python
VOCAB_SIZE = 5000
encoder = tf.keras.layers.experimental.preprocessing.TextVectorization(
    max_tokens = VOCAB_SIZE, standardize = 'lower_and_strip_punctuation',
    output_mode = 'int', output_sequence_length = 200,
)

encoder.adapt(train_dataset.map(lambda text, label: text))
```

We can go for _custom standardization_ by curating a function for our own use case but there are some bugs in tf:2.4.1 which create trouble whilst creating REST API call for the model.


__Model__

As we can see below, we are using the _encoder_ layer on the top of _Embedding_ that outputs us with a 256 dimension vector.
The rest of the graph is self-explanatory, although we are giving a _probabilistic output instead of a 2-class softmax layer_: the closer the probability to 1 meaning a positive sentiment for the review and vice-versa. 

```python

# Creating the model
model = tf.keras.Sequential([
    encoder,
    Embedding(input_dim=len(encoder.get_vocabulary()), output_dim= 256, mask_zero=True),
    GlobalAveragePooling1D(),
    Dropout(0.2),
    Dense(128, activation = 'relu'),
    Dropout(0.2),
    Dense(1, activation = 'sigmoid')])

```

After initialising the graph, we compile and fit the model:

```python
# Compiling the model
model.compile(optimizer= tf.keras.optimizers.Adam(learning_rate= 0.001), 
              loss = tf.keras.losses.BinaryCrossentropy(from_logits= False), metrics = ['accuracy'])


# Training the model
history = model.fit(train_dataset, epochs=10,
                    validation_data=test_dataset, 
                    validation_steps=5)
```


__Evaluation__

After model training, we evaluate the model on the test dataset and get a reasonably satisfactory test accuracy of 86.2%
(Although our major focus is the API & Docker and not extending our virtues in model building for this scenario)

```python
# Evaluating the model on test dataset
loss, accuracy = model.evaluate(test_dataset)
print("Loss: ", loss)
print("Accuracy: ", accuracy)

# Output:
Loss:  0.3457462191581726
Accuracy:  0.8626000285148621


# Saving the model
model.save('tf_keras_imdb/')
```

In TensorFlow, we can save the model in two ways: 'tf' or '.h5' format. Our model cannot be saved in '.h5' format since we are using the _TextVectorization_ layer

__FastAPI__
Before we start creating APIs, we need a particular directory structure that will be utilized for creating a Docker image.

_tf_keras_imdb/_ : SavedModel from TensorFlow
main.py : Python file for creating REST APIs using FastAPI framework 

```bash
|
|--- model
|    |______ tf_keras_imdb/
|
|--- app
|    |_______ main.py
|
|--- Dockerfile
```

Whenever we are building an API using FastAPI, we use _pydantic_ to set the type of input our API expects. For eg, a list, dictionary, JSON, string, integer, float.

To create an object using pydantic, we use _BaseModel_ that defines our type of inputs.

One of the reasons why FastAPI is faster and more efficient is its usage of ASGI - Asynchronous Server Gateway Interface, instead of traditional WSGI - Web Server Gateway Interface (which is used in Flask, Django)

POST request is assigned to our prediction API, since it requires us to _post_ the data and fetch back the results.

Uvicorn is a lightning-fast ASGI server implementation, which creates a server on our host machine and lets our API host the model on.


{% gist
https://gist.github.com/kushalvala/098bbe7f5ff2056ab708936781ae2fe0 
%}


We can test our API on SwaggerUI:

![SwaggerUI](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2774y8yzfl8yo2eoct8n.png)


__Docker__
Finally, to wrap it all up, we create a _Dockerfile_

```docker
FROM tiangolo/uvicorn-gunicorn-fastapi:python3.7

RUN pip install tensorflow==2.4.1

COPY ./model /model/

COPY ./app /app

EXPOSE 8000

CMD ["python", "main.py"]
```

We have attached a docker container (tiangolo/uvicorn-gunicorn-fastapi) which is made public on docker-hub, which makes quick work of creating a docker image on our own functionalities.

To create a docker image and deploy it, we run the following commands, and voila! 

```bash
docker build -t api .

docker run -d -p 8000:8000 api
```



__Conclusion__

After going through the process of working around FastAPI and Docker, I feel this skillset is a necessary repertoire in a data scientist's toolkit. The process of building around our model and deploying it has become easier and much more accessible than it was before.

Github Link: https://github.com/kushalvala/fastapi-nlp

Kushal Vala
Data Scientist