Building Web App for Computer Vision Model & Deploying to Production in 10 Minutes*: A Detailed Guide | by Pankaj Mathur | Towards Data Science

After downloading starter packs & performing one-time setup for one of the Docker hosted web app providers, You can quickly deploy your Computer Vision model created in fast.ai or keras libraries as a responsive web app, all in less than 10–15 minutes.
UPDATES on August 23rd, 2020: The world has changed a lot, since i updated this article, so does fastai and keras libraries, Fastai has released version 2 which is complete rewrite of fastai, so does keras, which is now part of tensorflow2. I have also released updated starter pack for both fastai and keras, please download them here:
Fastai version 2:
git clone https://github.com/psmathur/fastai2-vision-app
Keras (under tensorflow2)
git clone https://github.com/psmathur/keras-vision-app
UPDATES on January 4th, 2019: Added one more deployment option: “Render” a new simplest yet powerful Docker-based web service hosting providers & much more.
UPDATES on Dec 4th, 2018: I am not sure, I want to continue using Zeit services, as Zeit has recently put a lot of changes to restrict 100 MB limits. I was not able to resolve 100 MB limit issue even when I have unlimited paid plan and even when I try to explicitly use version 1 of the platform via CLI with “now.json”. IMHO, They are definitely missing huge business opportunities here by restricting themselves with 100 MB Limits!
********************************************************
So, you have built an awesome Image Classification model using the famous deep learning libraries like Fast.AI or Keras, then wrote an article about it, while explaining how your model can classify various types of blue jay birds, plant diseases, Italian food, french fries, Swiss watches or can even predict composer of the classical music playing around you, using spectrogram of audio files (and you thought Image Classification was over-hyped for nothing).
Now, What next?
Well, more than just writing a technical blog post on how you have built an amazing Computer Vision model, you also want to share your work easily with your co-workers, friends, families, or anyone who has a device with a web browser. You want to share your work with them because they all can quickly test & provide valuable feedback on your state of the art deep learning based model results, which is an important step towards evaluating your final DL models.
You need a Demo Web App!
Let’s say you have built an Image Classification algorithm, and now needs a simple web app which can handle your image or pictures upload from a web browser in an asynchronous way, then it also needs to download your trained model, hosted some other place, in an asynchronous fashion.
Deep Learning models are heavy, you know a huge size multi-dimensional activations matrixes, so they need to be uploaded independently on some cloud storage services, plus you want to have an option to switch to a better-upgraded model, especially based upon the feedback you received, without changing & re-deploying underlying web app.
Then this web app also should be able to load your models in the memory of the machine, using your preferred deep learning libraries, and then it can predict labels on your uploaded image.
In the end, all this stuff need to be configured, deployed and hosted on some cloud hosting provider. Sound fun, right...
Well, we all know it’s not, any of those steps mentioned above, can end up taking a huge amount of time, if you don’t have an established process, especially during bug fixing or deployment configuration. So, to cut down all these steps to a minimum, I wrote this guide. Once, you followed this article, you will have a 3 (or 2) step process, by which you can quickly build a web app for your computer vision models & can deploy them to production in < 10 Minutes.
Here is the link of a sample web app: https://keras-vision.app.render.com/
To build and deploy this kind of web app, First, we are going to download or clone starter packs hosted on my GitHub repo, currently, these web app starter packs are for build only for computer vision models build with Keras and Fast.AI.
These starter packs contain a simple responsive web app which is built on top of Starlette.io & Uvicorn ASGI server. After downloading one of these starter packages, you only need to change a few lines of code to make your deep learning model, up and running on a local machine.
Second, we are going to use one of the Docker hosted services. We will take our static web app directory with docker configuration in it, and then deploy it to production to one of these fully managed dockers hosted services.
In this article, I am covering, AWS BeanStalk, Azure Web App and Now Version 1. However, you can use similar steps to deploy to any of the other Docker hosted services.
So, now, without further ado, let’s start with step 1.
For your keras model deploymentGo to my Github and download the keras-vision-app repo.
wget https://github.com/pankymathur/keras-vision-app/archive/master.zip
For your fast.ai model deploymentGo to my Github and download the fastai-vision-app repo.
wget https://github.com/pankymathur/fastai-vision-app/archive/master.zip
unzip the zip file, rename the directory so, it matches your app name
unzip fastai-vision-app.zipmv fastai-vision-app YOUR-APP-NAME-HEREcd YOUR-APP-NAME-HERE
Note: You can also, leave everything like the way it is now in the directory of Keras or Fast.AI starter pack app and just move to Step 2 & Step 3 for a quick trial deployment, In that case, you will be deploying my X-Finder App or Clothing-Finder App. If you don’t and want to perfect everything before deploying, then read along.
Upload your Trained ModelUpload your trained model to any public cloud hosted services. Usually trained model has a file type of “.h5” in keras and can be created using method model.save() and file type of “.pth” in fast.ai and can be created using method learn.save().
For example, for my article and fast.ai project “Beating Fine Grained Clothing Classification Benchmark”, my final model file name is “stage-1_sz-150.pth”. It has a size < 100 MB and it is publicly hosted on my public GitHub project repo.
If your model file is > 100 MB you can use any other cloud service like Google Drive or Dropbox to upload your model file and then copy the raw download link for the file.
Note: The raw download link is the one which starts the file download directly and is normally different than the share link which presents to you. For example, the dropbox raw download link looks like this:
'https://www.dropbox.com/s/y4kl2gv1akv7y4i/stage-2.pth?raw=1'
Customized Starter Web App for Keras Models
Open up the file "server.py”which is inside the “app” directory and then update the line with "model_file_url” variable with the raw download URL of your hosted model file. For example:
model_file_url = 'https://github.com/pankymathur/Fine-Grained-Clothing-Classification/blob/master/data/cloth_categories/models/model.h5?raw=true'
Leave everything else unchanged, If you are using something like me, for example, I have created a simple image classification model using Transfer Learning process in Keras with Resnet50 Architecture and Imagenet pre-trained weights for 46 different types of clothing categories. Of course, you can completely customize the code to suit your needs.
Then I have uploaded my trained Keras model < 100 MB on my GitHub public repo, so here is how my final “server.py” look like, just before Step 2.
Customized Starter Web App for Fast.AI Models
Open up the file "server.py”which is inside the “app” directory and then update the line with "model_file_url” variable with the raw download URL of your hosted model file. For example:
model_file_url = 'https://github.com/pankymathur/Fine-Grained-Clothing-Classification/blob/master/data/cloth_categories/models/stage-1_sz-150.pth?raw=true'
In the same file, update the below line with the classes of labels you are expecting from your model to predict.
classes = ['Blouse', 'Blazer', 'Button-Down', 'Bomber', 'Anorak', 'Tee', 'Tank', 'Top', 'Sweater', 'Flannel', 'Hoodie', 'Cardigan', 'Jacket', 'Henley', 'Poncho', 'Jersey', 'Turtleneck', 'Parka', 'Peacoat', 'Halter', 'Skirt', 'Shorts', 'Jeans', 'Joggers', 'Sweatpants', 'Jeggings', 'Cutoffs', 'Sweatshorts', 'Leggings', 'Culottes', 'Chinos', 'Trunks', 'Sarong', 'Gauchos', 'Jodhpurs', 'Capris', 'Dress', 'Romper', 'Coat', 'Kimono', 'Jumpsuit', 'Robe', 'Caftan', 'Kaftan', 'Coverup', 'Onesie']
Leave everything else unchanged, and your app is ready for quick deployment (Of course, you can completely customize the code to suit your needs). For some of the deployment options, you need to push your app to GitHub repo, which could be public and private depending upon choice.
Render is a simplest yet powerful cloud provider services for all of your apps and sites. Sign up at this link => https://dashboard.render.com/register?i=fastai-v3 to receive $25 in credits. Render web services cost $5 per month and are prorated by the second.
After signing up go to your Render Dashboard, and then create a new Web Service. Here use the GitHub repo for your app which you created in Step 1 above.
Please note, You will need to grant Render permission to access your GitHub repo in this step.
After this, On the deployment screen, pick a name for your service and use Docker for the Environment. Also, if you have multiple branches in GitHub repo, choose branch appropriately or leave master branch selection as it is.
Just click Save Web Service and That’s it! Your docker based web app service will begin building and should be live in around ~10 minutes. You can follow its progress in the deploy logs. In the end, you can check the web URL displayed in your Render dashboard
https://keras-vision.app.render.com/
AWS Elastic Beanstalk is an easy to use AWS services for deploying a web app for Docker applications, plus AWS does offer a free trial for new accounts. Especially since our app is not going to use any DB instance, the final pricing will only include EC2 instance, load balancer, and S3 pricing. Read more about AWS Beanstalk Pricing here.
To begin, Open AWS Dashboard, Search for Elastic Beanstalk services, and click getting started with “Create New Application”. You will be shown following screen.
Add details for your demo application name & description and click “create”. You will be presented with a new screen for creating a new environment.
Choose “Create one now” and then you will be presented with below screen.
Select Web Server Environment here and you will be presented with create web server environment details screen.
Add your “Environment name”, Choose your “Domain” name, see if it’s available, and then add “Description”.
IMPORTANT: Under Base configuration Section, Choose Preconfigured Platform and then choose “Docker” generic option.
Scroll down below on the same page since you need to add more details.
Choose “Upload our code” option and click upload, then you will be presented with the following screen. Before uploading your starter pack Please read instructions below!
IMPORTANT: For some weird reason you can just upload a direct zipped file of our web app folder, you have to zipped contents under your web app folder directly, I guess because AWS Beanstalk needs to see Dockerfile found directly inside the root of the zipped file, not under web app folder. So, to do do this, just open web app in your local machine and zipped content directly. This is how I did in Mac OS X.
Optional: If you have run “server.py ” locally on your machine, then you may have downloaded model files already under directory app > static > model > “model.h5” or “model.pth”, in order to keep this compress zipped file smaller in size, you can delete those model file. However, do note, AWS Beanstalk does allow uploading file size up to 512 MB.
After uploading your source bundle of a web app, “Create a web server Environment page” will look like below.
IMPORTANT: Don’t press “Create Environment” now, we have to use “Configure More Options”!
These are essential steps, In order, to make our Web App deploy smoothly on AWS Beanstalk, Click “Configure More Options”, It will open a new page, where we have to change “Configuration presets” section to “Custom Configuration” option, It will look like below
Then go down to the first Section > “Software”, click “Modify”, It will open a page like below, Change “Proxy server” from “Nginx” to “None”. Leave everything else unchanged and press save.
After “Save” You will be back to the Configuration page, Here go down to the second Section > “Instances”, click “Modify”, It will open a page like below, Change “Instance type” from “t1.micro” to “t3.medium”. Leave everything else unchanged and press save.
That’s it for custom configuration, now press “Create Environment” and go grab a coffee or something, it will take 10–20 minutes for AWS Beanstalk to deploy your source code to create a Docker image, then deploy that docker image to your choice of ec2 instance, create application load balancer, create appropriate security groups with inbound and outbound network settings and many other things, which are beyond the scope of this guide. You will see a screen showing log files entries of AWS BeanStalk deployment.
Please be patience, After 8-10 minutes, you will see Dashboard like this, which will show Health as “Green” and detail logs for each stage.
Click on URL, shown on the dashboard, and you will be redirected to your new demo app, up and running on AWS.
NOTE: If you see on the Dashboard , Health not “Green” but “Red”, please look at the log files in details. You can find detail logs under “Logs” section on Left side, and by pressing Request Logs > Full logs or Last 100 lines. Reading will logs will show you the main reason of Health state “Red” and why your deployment failed. Usually, it is the type of small ec2 instances you choose which can’t handle the deployment or memory issues while running your app or try applying other custom settings, which I didn’t mention above.
Google App Engine Custom runtimes let you build apps that run in an environment defined by a Dockerfile. Plus Google Cloud do offer $300 credit in a 12-month free trial for new accounts. Especially since our app is not going to use any DB instance, the final pricing will only include Google Compute instance, load balancer, and Google cloud storage pricing. Read more about Google App Engine Pricing here.
To begin, Open Google Cloud Dashboard, Click Click Create Project and then name your new GCP project. Enable billing in your new GCP project by creating a new billing account or setting an existing one. You will be shown following screen.
After creating a new project, you will be presented with GCP Dashboard page, Go to the far right corner of the page and click “Activate Cloud Shell”. A terminal window will open in the same page.
At this point onwards you can continue in this same terminal window or open this terminal window in a new browser page.
Run the following command to create an App Engine application and specify in which geographical region that you want your app to run.
gcloud app create
I choose 13 and press enter, after few minutes, it will show something like “Success! the app is now created.”
Now, download your starter pack app repository hosted on Github or any other git repository management services by typing below command. Make sure that your starter pack has same app.yaml and Dockerfile from my repo.
git clone https://github.com/pankymathur/fastai-vision-app
After download of your repository, navigate to your app directory.
cd fastai-vision-app
Deploy your app to Google App Engine
gcloud app deploy
You will be presented with a screen showing “Services to deploy”, enter Y
Then go grab a quick coffee, as it will take 8~10 minutes for app engine to deploy your docker based app and present to your final URL. To see your app running at http://YOUR_PROJECT_ID.appspot.com, run the following command to launch your browser:
gcloud app browse
Here is my default fastai-vision-app starter pack app running on GCP.
Now Version 1 has a free plan that has unlimited deployments & domains for your static web apps, can handle storage & bandwidth up to 1 GB, & many other features, as long as you have max file size under 5 MB.
For Mac OSx and Windows
Go to https://zeit.co/download and download now.sh desktop app for quick drag & drop deployment.
On Mac OS X
# Intall https://brew.sh, if not already installed# a mac without brew is just another empty glass/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"# Install npm, if not already installedbrew install npm# Install now.sh clisudo npm i -g --unsafe-perm now
On Linux
# if not already installedsudo apt install npm# Install now.sh clisudo npm i -g --unsafe-perm now
On Windows
Download NPM Windows Installer https://nodejs.org/en/download/
Use Powershell:
# Install now.sh clisudo npm i -g --unsafe-perm now
Use Linux App on Windows 10:
Choose your choice of Linux app: https://docs.microsoft.com/en-us/windows/wsl/install-win10 After installing Linux app, Go to Terminal and follow Linux instructions above to install now.sh CLI
via Now Desktop
Deploy by dragging and dropping or uploading your responsive-web-app directory to “Now Desktop”.
It will show all the deployment log details in a separate browser window. You may need to login to zeit.co before deployment begins.
It will create a unique deployment URL for your static app and it will be shown that in the first few lines of logs during deployment. Usually, It has a format of https://YOUR-APP-NAME-HERE-xxxxxxxxxx.now.sh
When the deployment finishes and it will show “Success! Deployment-ready” in the browser window.
Optional: If you are like me, you want to use some pretty URL name for your app, instead of default one like https://YOURAPPNAMEHERE-xxxxxxxxxx.now.sh
you can add the alias to your app by going to dashboard on zeit.co and using create alias option for your current deployment
via Now CLI
On the terminal, make sure you are in the keras-vision-app or fastai-vision-app directory, then type:
now
Every time you deploy with “Now CLI” it’ll create a unique deployment URL for your static app and it will be shown that in the first few lines of logs during deployment. Usually, It has a format of https://YOURAPPNAMEHERE-xxxxxxxxxx.now.sh
When the deployment finishes and it will show “Success! Deployment-ready” on the terminal.
Optional: If you are like me, you want to use some pretty URL name for your app, instead of default one like https://YOURAPPNAMEHERE-xxxxxxxxxx.now.sh
you can change the alias of your app by using the following command:
now alias PRETTY_URL_NAME YOURAPPNAMEHERE-xxxxxxxxxx.now.sh
It will show something:
> Success! YOUR-APP-NAME-HERE.now.sh now points to YOURAPPNAMEHERE-rhainypwfk.now.sh
By default, all now deployment on now.sh goes to sleep after some inactive time. This is not good if you want your app to be active during sharing or demo. So scale up your app instance:
# You only need to do this once.now scale YOUR-APP-NAME-HERE.now.sh sfo 1
It will show something:
> ✔ Scaled 1 instance in sfo1 [923ms]> Success! Scale state verified
Now you can sharehttps://YOUR-APP-NAME-HERE.now.sh with anyone for quick feedback and demo.
For example, here is link to my default demo app https://x-finder.now.sh
You will need a free Docker hub account. Docker Hub is the container hosting repository which we will use to supply docker image to one of the Docker hosted app services provider like “Azure Website for Containers” so these services know, where to pull your Docker image from.
There are many-many container hosting services, giving the option of hosting your docker image public or private, for example, these are common container registry in industry Docker Hub, Azure Container Registry, Amazon Elastic Container Registry, or Google Container Registry. For our scenario, a public container repository on Docker Hub is a great option.
There are two ways to create your docker repository on Docker Hub, First one is with Docker CLI, which comes packaged with Docker Desktop App for Mac OSx or Windows.
After installing Docker Desktop App, and log in to Docker Hub account, here are steps to build and upload your docker image to Docker Hub container registry:
Create a Docker image
docker build -t <imageName> .#For ex: docker build -t keras-vision-app .
Run the Image locally to see your app in action
docker run -p 5042:5042 <imageName>#For ex: docker run -p 5042:5042 keras-vision-app
Tag the Image with your Docker Hub username
docker tag image username/repository:tag#For ex: docker tag keras-vision-app pankajmathur/keras-vision-app
Push the Image to Docker Hub
docker push username/repository:tag#For ex: docker push pankajmathur/keras-vision-app
After a successful push, you can see your local docker image in Docker Hub account.
The second way is using GitHub integration with Docker Hub account. This is the fastest way to from 0 to Docker container registry in no time.
Link Docker Hub with your GitHub Profile
Log into Docker Hub, then Navigate to Profile > Settings > Linked Accounts & Services then click the GitHub service. A pop up prompts for you to enter your GitHub credentials. After you grant access to your code repositories. Your DockerHub Linked Accounts & Services page will look like:
Create an automated build
Go to your DockerHub Dashboard, then select Create > Create Automated Build from the drop down, upper right. It will ask you to choose GitHub linked account:
A system prompt will display all your GitHub repositories, type to filter the repository list, then pick the project to build.
A system prompt will display the Create Automated Build dialog. The dialog assumes some defaults which you can customize. By default, Docker Hub builds images for each branch in your repository. It assumes the Dockerfile lives at the root of your source repository. When it builds an image, Docker Hub tags it with the branch name. If you want to customize the default options, then Click here to customize this behavior link. (For our use case, leaving default is good)
Click Create, The system displays the home page for your AUTOMATED BUILD.
That’s it, you have your docker hosted container registry copy the name of your repository, in my case, it is pankajmathur/fastai-vision-app and now you are ready for Step 2 (Option 4): Azure Web App for Container Service.
Azure Web App for Containers one click docker app provides very easy to deploy and host docker static apps mechanism. There is a good article written by fellow fast.ai student Nissan https://redditech.blog/2018/11/04/hosting-fastai-app-in-azure-websites-for-containers/.
Please follow the article that shows step by step screenshots of Azure Portal App Services for Container Services process. I was able to read along with the article and was able to set up my web app in < 10 minutes***.
*** Azure takes an initial long time to deploy the whole container and propagate DNS for almost additional 10 minutes.
This part was not covered by Nissan, so here it is, By default, all azure web app deployment goes to sleep after some inactive time. This is not good if you want your app to be active during sharing or demo. So scale up your app instance by going to Azure App Services Settings > Application Settingsand by turning App Always => On and by turning ARR Affinity => Off
Currently Azure offers a free trial credit of $200 for the first 30 days to try out their services, so you can easily sign up for a new account and try out all the steps.
Here is our Azure Web App =>https://keras-vision-app.azurewebsites.net
Well, both of my keras and fast.ai starter packs repositories are open sources on GitHub, so you can just check all the code before downloading or cloning it. The main meat of the starter pack is “server.py” which is a tiny Starlette.io & Uvicorn powered ASGI server.
This tiny server is actually not at all tiny but a mighty one, this tiny server with the help of Asyncio Libraries handles your image or pictures upload in a concurrent fashion using “ upload(request)” method, then it has “download_file(url, dest)” method which gracefully downloads your trained model, hosted on the public cloud, to local drive, then this tiny server, with help of Event loop, Tasks, and Coroutines mechanism loads your downloaded model file in hosting machine memory using your keras or fast.ai Deep Learning libraries, and in the end this tiny server predicts labels on your uploaded image and spit it back to prediction.txt file.
The frontend statics are a simple HTML5 responsive web app with few neat JS hacks to handle big image files upload, especially useful for those awesome raw cameras or selfie pictures.
In the end, the tiny ASGI server & responsive statics are put inside a docker using a docker configuration that will install correct python version with Keras or Fast.ai, starlette.io, uvicorn and all other libraries dependencies with the help of requirement.txt, then this docker configuration will expose the port 5042 and kick off the server. All this so you can easily host your Docker image on top of docker cloud machine...
Caution: Most of the modern browser works perfectly well with these JS hacks and HTML5 files. If you are still stuck in the early 2000s ignore this whole post.
That’s it, Hopefully, you enjoyed this guide, and was able to Build your Web App for your Computer Vision Models & was able to Deploy them to Production and while doing that established a 3 Step process, so you can repeat all of them in no time.If not, Please do let me know your questions under the comments section. If yes, I would really appreciate getting some feedback on this article & ideas to improve it.
In the meanwhile, Happy Deep Learning...