Deploy your Machine Learning Model with ml5.js | by Natassha Selvaraj | Towards Data Science

As data scientists, most of our work is done in a Jupyter notebook. Whether it is building a machine learning model, training it, or using it to make predictions, our work usually doesn’t make it out of iPython notebooks.
While this is a great environment to create and test our model, what happens when we want to get our model into the hands of another person?
Let’s say we want a potential employer or colleague to use our model. Wouldn’t it be nicer to have a functional application for them to play around with?
An interactive environment that anybody can use easily is a lot more interesting than showing people a bunch of code lying in your Jupyter notebook.
In this article, I’m going to show you how to do that.
By the end of this tutorial, you will be familiar with the following concepts:
What is a client-side neural network?
What are the advantages of implementing a browser based model?
How to deploy and run your model in the browser?
Creating an object detection model and deploying it to make predictions using JavaScript.
This tutorial is going to walk you through creating a client-side neural network like the one above.
When deploying neural networks or machine learning models, we usually have two things — a front-end application and a back-end application.
The front-end application will take the image uploaded by the user, and make a call to the back-end application. It passes the image to the back-end, and the model in the back-end then makes a prediction.
The prediction is then passed back to the front-end application, and shown to the user.
However, client-side neural networks don’t have a back-end.
Both the neural network and the data are in the front-end, and the model is running in the browser.
It doesn’t have to make calls to the back-end, since the model is embedded with the data in the front-end application.
Popularity: Developers who are used to programming in JavaScript will find it easy to create and deploy AI and machine learning models.
Security: Users get to keep data on their local devices, which minimizes security risk.
The neural network or machine learning model can be created in real time while you are training the model.
In fact, Google’s Teachable Machine trains machine learning models directly on the web browser.
Before I move on to creating and deploying the model, I would like to briefly walk you through the tools and libraries we will be using in this tutorial.
If you are someone who has been in the field of machine learning for a while now, you’ve probably heard about libraries like Tensorflow and Keras.
Tensorflow is a library that allows you to create deep learning models in Python. Keras is a higher level library that is built on top of Tensorflow.
Although it uses Tensorflow in the back-end, it is a lot easier to code out neural networks in Keras than in Tensorflow.
When using Tensorflow to develop applications, you can use either one of these two languages — Python or C++.
However, when we talk about browser based machine learning models, we want to use a library that can create neural networks in JavaScript itself.
This is where Tensorflow.js comes in.
Tensorflow.js is the equivalent of the Tensorflow library that you can use in Python. It allows you to create and train neural networks in JavaScript — in the front-end.
To make things even easier, we have something called ml5.js.
Just as Keras is built on top of Tensorflow and can be used to easily create neural networks, ml5.js is built on top of Tensorflow.js.
It is a high-level library that makes it easier for you to program neural networks in the browser. With ml5.js, you can even gain access to pre-trained models, which you can easily load and train.
I will link some resources below to both Tensorflow.js and ml5.js, both of which you can use to create browser-based machine learning models.
p5.js is a JavaScript library that was built for creative coding. It allows you to do things like draw on a canvas, design, and create visual pieces.
In this tutorial, we will be using p5.js to do things like display the image uploaded by the user onto the screen.
It will also be used to draw the object detection bounding box on the canvas, as seen in the image above. (Yes, its funny the model thinks that the golden retriever is a cow).
CocoSSD is a pre-trained object detection model. This model is able to identify different objects in one image, and come up with a prediction for each of those objects.
As seen in the example above, the cocoSSD model identified two objects in the image — a cow and a dog.
It did misclassify the golden retriever in the image, which was probably because it was larger than the dogs in the dataset used to train the model.
CocoSSD uses a dataset called Coco, which has labels of 80 different object classes. You can read more about the CocoSSD model here.
This isn’t a library or a framework, but is a pre-requisite to starting this tutorial.
If you are someone who works predominantly in an IDE like a Jupyter notebook, you might find the transition to a text-based IDE a little strange.
You will need to download a JavaScript IDE. Some popular ones include Visual Studio Code, Atom, and Sublime Text.
I personally like to use Atom because it comes with a live-server that allows you to see all your changes without having to refresh the page. I will link installation guides to all these IDEs below. If you don’t have any of them already, you can pick one and set it up.
Now that you’re familiar with the technology stack we will be using, we can start to create the app.
If you would like to skip the rest of the tutorial and go straight to the code, I have you covered. The code is uploaded onto my GitHub repository here.
As seen in the image above, getting started with the project is pretty simple. You only need to create three files — one HTML file, one CSS file, and one JavaScript file.
This is our HTML file, where all the necessary libraries are loaded. Keep in mind that I will only be walking through code snippets here, and you will have to visit my GitHub repo for the complete code.
To import the p5.js and ml5.js library, the following lines of code will be used:
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.6.1/p5.min.js"></script>        <script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.6.1/addons/p5.dom.min.js"></script>        <script src="https://unpkg.com/ml5@latest/dist/ml5.min.js"></script>
Then, the CocoSSD model will be imported. This is the model that will be loaded later to make predictions on our images.
<script src="https://cdn.jsdelivr.net/npm/@tensorflow-models/coco-ssd"> </script>
The app is pretty simple, and I didn’t spend too much time with CSS styling except for the heading text.
h1 {    text-align: center;    color: black !important;    padding-bottom: 10px;    font-family: Helvetica, Arial, sans-serif;}h1::after {    content: '';    background: #800000;    display: block;    height: 3px;    width: 190px;    margin: 20px auto 5px;}
This is the most important part of creating the application. The following will be done in JavaScript:
Loading the pre-trained model
Taking in input image from the user
Using the model to detect objects in the image
Displaying the model’s output (bounding boxes and prediction) to the user
The best part about using pre-trained models is that a huge portion of the work is already done.
There is no need to create, train and fine-tune your model. Just load it, and you are ready to go! We will be using the cocoSSD model for object detection.
This can be done very easily using the built-in functions in ml5.js:
function preload() { //load model before anything else happens  detector = ml5.objectDetector('cocossd');}
In case you were wondering, ‘preload()’ is a p5.js function. It allows you to load external files before running the rest of the code.
In this case, it waits for the object detection model to load before the next function is executed.
You can check out the p5.js documentation here to learn more about the preload() function.
p5.js allows you to do this quite easily:
input = createFileInput(handleFile);
After the user uploads their file, you will need to run the following lines of code to create the image and store it in a variable. Later, you will need to access this image to make predictions.
function handleFile(file) {   print(file);  if (file.type === 'image') {    img = createImg(file.data, '');    img.hide();  } else {    img = null;  }}
After storing the uploaded image into the “img” variable, we can proceed to detect objects in the image with the model loaded earlier.
The object detector model can then be used to detect objects in the image. Keep in mind that the cocoSSD model was only trained on around 80 classes of objects, which you can find out more about here.
This means that the model won’t be able to detect objects out of these 80 classes.
If you want to create your project with an object detection model, make sure to choose a pre-trained model that is aligned with the kind of task you want to perform.
Otherwise, you can consider using transfer learning approaches — this will allow you further train a pre-trained model with your own training examples.
function detectImage() {  detector.detect(img, function(error,results) {    if(error){         console.error(error);       }     for(let i = 0; i<results.length; i++){       let object = results[i];       stroke(0,255,0);       strokeWeight(4);       noFill();       rect(object.x+350,object.y,object.width,object.height);       noStroke();       fill(255);       textSize(34);       text(object.label,object.x+350,object.y);       console.log(object);     }     detectImage();  });}
With the following lines of code, the model will detect all the objects within the image, and draw a bounding box around the object.
The model prediction will also be drawn onto the canvas.
After you’re done, go to your application and upload an image of your choice. Click on the predict button, and wait for the results. Your output should look similar to this:
We have successfully loaded a pre-trained model into the browser, and made predictions based on user input.
The ability to train models in the browser and deploy them opens up a wide array of possibilities; you can easily create a front-end application to showcase your machine learning models, and even do things like make predictions in real-time.
I hope you learnt something new from this tutorial! If you want to learn more about model deployment, do check out the learning resources below.
Source CodeText Editors: Atom, VSCode, Sublime TextTensorflow.js tutorialFlask model deployment tutorialp5.js tutorial
Source Code
Text Editors: Atom, VSCode, Sublime Text
Tensorflow.js tutorial
Flask model deployment tutorial
p5.js tutorial
We do these things not because they are easy, but because they are hard — John F.Kennedy