Deploying a Simple Machine Learning Model into a WebApp using TensorFlow.js | by Carlos Aguayo | Towards Data Science

Go ahead and try it in your browser or phone:
https://carlos-aguayo.github.io/tfjs.html
The beauty in TensorFlow.js is that you can train a Machine Learning Model in Python using Keras or Tensorflow and deploy it on a browser using TensorFlow.js. No need for an external service to run your queries.
This fun and simple app lets you draw a single-digit number, and it recognizes it using simple machine learning tools.
Here’s the complete code if you want to jump straight at it:
Google Colab Notebook to generate the Machine Learning Model
HTML file
Let’s get started; these are the ingredients and steps needed to build this demo.
Ingredients and steps:
Training Data
Training Environment
Pre-processing Data
Machine Learning
Convert a Keras model to Tensorflow.js
HTML5 Canvas
Hooking everything up
Any Machine Learning model needs quality data. We are going to use the MNIST dataset, which is a dataset of handwritten digits.
In this dataset, you have 60,000 images, all of them are of size 28 x 28 pixels in grayscale, with pixel values from 0 to 255.
See the snippet below for loading and viewing the MNIST dataset.
%tensorflow_version 2.x
from tensorflow.keras.datasets import mnist
import matplotlib.pyplot as plt

(X_train, y_train), (X_test, y_test) = mnist.load_data()
print ("X_train.shape: {}".format(X_train.shape))
print ("y_train.shape: {}".format(y_train.shape))
print ("X_test.shape: {}".format(X_test.shape))
print ("y_test.shape: {}".format(y_test.shape))

X_train.shape: (60000, 28, 28)
y_train.shape: (60000,)
X_test.shape: (10000, 28, 28)
y_test.shape: (10000,)

plt.subplot(161)
plt.imshow(X_train[3], cmap=plt.get_cmap('gray'))
plt.subplot(162)
plt.imshow(X_train[5], cmap=plt.get_cmap('gray'))
plt.subplot(163)
plt.imshow(X_train[7], cmap=plt.get_cmap('gray'))
plt.subplot(164)
plt.imshow(X_train[2], cmap=plt.get_cmap('gray'))
plt.subplot(165)
plt.imshow(X_train[0], cmap=plt.get_cmap('gray'))
plt.subplot(166)
plt.imshow(X_train[13], cmap=plt.get_cmap('gray'))

plt.show()

Google Colab allows you to write and execute Python in your browser.
Colab is a very convenient Jupyter Notebook Python Platform pre-loaded with most of the Python Machine Learning libraries that you need, making it a no-hassle approach to get you quickly up and running with an ML project.
Plus, it gives you free access to GPU/CPU.
There’s a small preprocessing that needs to happen in the data from MNIST:
Normalize inputs: The data comes with values from 0 to 255, we should normalize them to a scale from 0 to 1.One-Hot Encode the outputs.
Normalize inputs: The data comes with values from 0 to 255, we should normalize them to a scale from 0 to 1.
One-Hot Encode the outputs.
# Normalize Inputs from 0–255 to 0–1x_train = x_train / 255x_test = x_test / 255# One-Hot Encode outputsy_train = np_utils.to_categorical(y_train)y_test = np_utils.to_categorical(y_test)num_classes = 10
We are finally ready to do some Machine Learning. We can start with a very simple model. We will use a simple Neural Network with just one Hidden layer. This simple model is enough to get a 98% accuracy.
x_train_simple = x_train.reshape(60000, 28 * 28).astype(‘float32’)x_test_simple = x_test.reshape(10000, 28 * 28).astype(‘float32’)model = Sequential()model.add(Dense(28 * 28, input_dim=28 * 28, activation=’relu’))model.add(Dense(num_classes, activation=’softmax’))model.compile(loss=’categorical_crossentropy’, optimizer=’adam’, metrics=[‘accuracy’])model.fit(x_train_simple, y_train, validation_data=(x_test_simple, y_test), epochs=30, batch_size=200, verbose=2)
If you want to be fancy, you can try a Deep Learning Model. With it, you can improve the accuracy to 99%.
x_train_deep_model = x_train.reshape((60000, 28, 28, 1)).astype(‘float32’)x_test_deep_model = x_test.reshape((10000, 28, 28, 1)).astype(‘float32’)deep_model = Sequential()deep_model.add(Conv2D(30, (5, 5), input_shape=(28, 28, 1), activation=’relu’))deep_model.add(MaxPooling2D())deep_model.add(Conv2D(15, (3, 3), activation=’relu’))deep_model.add(MaxPooling2D())deep_model.add(Dropout(0.2))deep_model.add(Flatten())deep_model.add(Dense(128, activation=’relu’))deep_model.add(Dense(50, activation=’relu’))deep_model.add(Dense(num_classes, activation=’softmax’))deep_model.compile(loss=’categorical_crossentropy’, optimizer=’adam’, metrics=[‘accuracy’])deep_model.fit(x_train_deep_model, y_train, validation_data=(x_test_deep_model, y_test), epochs=30, batch_size=200, verbose=2)
Now that we have a trained model, we need to convert it so that we can use it with TensorFlow.js.
First, we need to save the model into an HDF5 model.
model.save(“model.h5”)
Afterward, you can access the files saved by clicking on the folder icon in the left nav.
There are a few ways to convert the model. A simple one within a Notebook is like this:
!pip install tensorflowjs!tensorflowjs_converter --input_format keras ‘/content/model.h5’ ‘/content/model’
/content/model.h5 is the input and the output is saved into /content/model folder.
TensorFlow.js needs to be pointed at the JSON file (model.json) and expects a sibling file called “group1-shard1of1.bin”. You need these two files. Download both files.
Let’s have a simple HTML page that uses the HTML5 Canvas component that lets us draw on it. Let’s call this file “tfjs.html”.
The core drawing code comes from this website:
Using the HTML5 Canvas component, we can hook mouse events to draw into the Canvas.
canvas.addEventListener('mousedown', function(e) {  context.moveTo(mouse.x, mouse.y);  context.beginPath();  canvas.addEventListener('mousemove', onPaint, false);}, false);var onPaint = function() {  context.lineTo(mouse.x, mouse.y);  context.stroke();};
Then we add touch events so that it works on mobile.
Add the touch action to disable scrolling. The code for that is inspired by this site.
Once we can draw, let’s fetch the image upon mouse up. We will scale it down to 28 by 28 pixels so that it matches the trained model.
canvas.addEventListener('mouseup', function() {  $('#number').html('<img id="spinner" src="spinner.gif"/>');  canvas.removeEventListener('mousemove', onPaint, false);  var img = new Image();  img.onload = function() {    context.drawImage(img, 0, 0, 28, 28);    data = context.getImageData(0, 0, 28, 28).data;    var input = [];    for(var i = 0; i < data.length; i += 4) {      input.push(data[i + 2] / 255);    }    predict(input);  };  img.src = canvas.toDataURL('image/png');}, false);
We then get the data, keep it into an “input” array and pass it to a predict function that we will define later.
canvas.addEventListener('mouseup', function() {  $('#number').html('<img id="spinner" src="spinner.gif"/>');  canvas.removeEventListener('mousemove', onPaint, false);  var img = new Image();  img.onload = function() {    context.drawImage(img, 0, 0, 28, 28);    data = context.getImageData(0, 0, 28, 28).data;    var input = [];    for(var i = 0; i < data.length; i += 4) {      input.push(data[i + 2] / 255);    }    predict(input);  };  img.src = canvas.toDataURL('image/png');}, false);
data is a 1D array with values RGBA values. Our model only takes 0 to 1 values (or 0 from 255 thinking in grayscale). Given that we are drawing Blue into the canvas, we can slice the array in chunks of four and take every second element.
RGBARGBARG...0123456789...
Finally, let’s load TensorFlow.js and run predictions.
<script src=”https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@1.5.2/dist/tf.min.js"></script>
You should have downloaded the files model.json and group1-shard1of1.bin and save them into a folder called model in the same folder where you have your HTML file.
Once loaded, we can load the trained model by simply doing:
tf.loadLayersModel(‘model/model.json’).then(function(model) { window.model = model;});
And upon mouseup, once we have the data, we can just feed it into the model:
window.model.predict([tf.tensor(input).reshape([1, 28, 28, 1])]).array().then(function(scores){  scores = scores[0];  predicted = scores.indexOf(Math.max(...scores));  $('#number').html(predicted);});
It’s straightforward to test things locally, and you can easily set up an HTTP server for testing with Python:
python3 -m http.server 8080
If then you want to test things using your phone, you can take advantage of this nifty tool called ngrok.
$ ngrok http 8080
That opens a tunnel to a URL that you can access from through phone.
Once you are happy with the result, you can deploy your HTML into a web hosting site. A simple place is Github. If you have never created a static website with Github, you need to create a repository named “{username}.github.io”. For example, my repository is:
https://github.com/carlos-aguayo/carlos-aguayo.github.io
And then you can access it via:
https://carlos-aguayo.github.io/tfjs.html
Notice how easily we can train a model in Google Colab, export the trained model, and query it within JavaScript without ever leaving the browser!
I’m Director of Software Development and Machine Learning Engineer at Appian. I’ve worked at Appian for the last 15 years, and I keep having a blast. Send me a message if you would like to know how we craft software, please our customers and have fun!