Develop and sell a Machine Learning app — from start to end tutorial | by Daniel Deutsch | Towards Data Science

After developing and selling a Python API, I now want to expand the idea with a machine learning solution. So I decided to quickly write a COVID-19 prediction algorithm, deploy it, and make it sellable. If you want to see how I did it, check out the post for a step by step tutorial.
About this article
Disclaimer
Stack used
1. Create project formalities
2. Develop a solution for a problem
Install packages and track jupyter files properly
Develop solution to problem
Build server to execute function with REST
BONUS: Make reproducible with Docker
3. Deploy to AWS
Set up zappa
Set up AWS
4. Set up Rapidapi
End result
Inspiration
Final links
About
In this article, I take the ideas from my previous article “How to sell a Python API from start to end” further and build a machine learning application. If the steps described here are too rough consider reading my previous article first.
There are a number of new and more complicated issues to cover in this project:
Machine Learning content. The application takes basic steps of building a Machine Learning model. This covers the preparation, but also the prediction.In time evaluation (not in time training) of the prediction. This means that the dataset is freshly fetched and the prediction is performed on the latest data.Deployment. Deploying a Machine Learning app has various challenges. In this article, we met and solved the issue of outsourcing the trained model on AWS.It is not only an API but also has a minor frontend.
Machine Learning content. The application takes basic steps of building a Machine Learning model. This covers the preparation, but also the prediction.
In time evaluation (not in time training) of the prediction. This means that the dataset is freshly fetched and the prediction is performed on the latest data.
Deployment. Deploying a Machine Learning app has various challenges. In this article, we met and solved the issue of outsourcing the trained model on AWS.
It is not only an API but also has a minor frontend.
It paints a picture for developing a Python API from start to finish and provides help in more difficult areas like the setup with AWS Lambda.
There were various difficulties, which allowed me to learn more about the deployment and building process. It is also a great way to build side projects and maybe even make some money.
As the Table of content shows, it consists of 4 major parts, namely:
Setting up the environmentCreating a problem solution with PythonSetting up AWSSetting up Rapidapi
Setting up the environment
Creating a problem solution with Python
Setting up AWS
Setting up Rapidapi
You will find all my code open on Github:
https://github.com/Createdd/ml_api_covid
You will find the end result here on Rapidapi:
https://rapidapi.com/Createdd/api/covid_new_cases_prediction
I am not associated with any of the services I use in this article.
I do not consider myself an expert. If you have the feeling that I am missing important steps or neglected something, consider pointing it out in the comment section or get in touch with me. Also, always make sure to monitor your AWS costs to not pay for things you do not know about.
I am always happy for constructive input and how to improve.
There are numerous things to improve and build upon. For example, the machine learning part has a very low effort. The preparation was very rough and many steps are missing. From my professional work, I am aware of this fact. However, I cannot cover every detail in one article. Nevertheless, I am curious to hear your suggestions on improvement in the comments. :)
If you need more information on certain parts, feel free to point it out in the comments.
I consider this as a step by step tutorial. However, as I am already too long working as a developer, I assume some knowledge of certain tools. This makes the tutorial probably an intermediate/advanced app.
I assume knowledge of:
Python
Git
Jupyter Notebook
Terminal/Shell/Unix commands
We will use
Github (Code hosting),
Anaconda (Dependency and environment management),
Docker (for possible further usage in microservices)
Jupyter Notebook (code development and documentation),
Python (programming language),
AWS, especially AWS Lambda and S3(for deployment),
Rapidapi (market to sell)
It’s always the same but necessary. I do it along with these steps:
Create a local folder mkdir NAMECreate a new repository on Github with NAMECreate conda environment conda create --name NAME python=3.7Activate conda environment conda activate PATH_TO_ENVIRONMENTCreate git repo git initConnect to Github repo. Add Readme file, commit it and
Create a local folder mkdir NAME
Create a new repository on Github with NAME
Create conda environment conda create --name NAME python=3.7
Activate conda environment conda activate PATH_TO_ENVIRONMENT
Create git repo git init
Connect to Github repo. Add Readme file, commit it and
git remote add origin URL_TO_GIT_REPOgit push -u origin master
Install packages and track jupyter files properly
Develop solution to problem
Build server to execute function with REST
BONUS: Make reproducible with Docker
As we will develop a Machine Learning solution, a Jupyter Notebook will be very useful.
Install jupyter notebook and jupytext:
pip install notebook jupytext
Register new environment in jupyter ipython kernel install --name NAME--user
Set a hook in .git/hooks/pre-commit for tracking the notebook changes in git properly:
touch .git/hooks/pre-commitcode  .git/hooks/pre-commit
copy this in the file
#!/bin/sh# For every ipynb file in the git index, add a Python representationjupytext --from ipynb --to py:light --pre-commit
afterward for making the hook executable (on mac)
chmod +x .git/hooks/pre-commit
As currently, the world is in a pandemic I thought I use one of the multiple datasets for Covid-19 cases. Given the structure of the dataset, we want to predict the new cases of infections per day for a country.
pip install -r requirements.txt
This will install all packages we need. Have a look in the /development/predict_covid.ipynb notebook to see what libraries are used.
Most important are the libraries
pandas for transforming the dataset and
sklearn for machine learning
For the following subheadings please check out the Jupyter notebook for more details:
https://github.com/Createdd/ml_api_covid/blob/master/development/predict_covid.ipynb
We will use the dataset from https://ourworldindata.org/coronavirus-source-data in csv format.
The license of data is Attribution 4.0 International (CC BY 4.0)
Source code available on Github
In short, I did:
Check for missing dataRemove columns with more than 50% missing dataRemove rows with remaining missing content like continent or isocode. (Not useful for my app solution which requires a country)Encode categorical data with labelsFill in the remaining numerical missing data with the mean of the columnSplit into training and test set
Check for missing data
Remove columns with more than 50% missing data
Remove rows with remaining missing content like continent or isocode. (Not useful for my app solution which requires a country)
Encode categorical data with labels
Fill in the remaining numerical missing data with the mean of the column
Split into training and test set
Create a Random Forest RegressorTrain it on the data and evaluatePerform hyperparameter tuning with RandomizedSearchCVSave the trained modelPredict the new cases by providing a country name
Create a Random Forest Regressor
Train it on the data and evaluate
Perform hyperparameter tuning with RandomizedSearchCV
Save the trained model
Predict the new cases by providing a country name
For the API functionality, we will use a Flask server (in app.py)
https://github.com/Createdd/ml_api_covid/blob/master/app.py
@app.route('/')def home():    return render_template("home.html")
Which serves a basic HTML and CSS file.
This is a little more complex.
The key route is this:
But before we can return the prediction result we need to get the latest data and pre-process it again. This is done with
Pre-process again transforms the downloaded dataset for machine learning purposes, whereas get_prediction_params takes the input value (which is the country to be predicted) and the URL to the latest dataset.
Those processes make the prediction true for the latest data but also slows down the app.
You might wonder why we do rf = load_model(BUCKET_NAME, MODEL_FILE_NAME, MODEL_LOCAL_PATH). The reason for this is that we need to load the pre-trained model from an AWS S3 bucket to save memory when executing everything with AWS Lambda. Scroll down for more details.
But if we do not want to deploy it in the cloud we can simply do something like joblib.load(PATH_TO_YOUR_EXPORTED_MODEL). In the notebook, we export the model with joblib.dump. More info on model exports in the sklearn docs
But that is the mere functionality of the FLAK server. Providing a route for serving the HTML template and a route for prediction. Quite simple!
Running now
env FLASK_APP=app.py FLASK_ENV=development flask run
will start the server.
Maybe you want to scale the app or allow other people to test it more easily. For this, we can create a Docker container. I will not explain in detail how it works but if you are interested check one of the links in my “Inspiration” section.
Building a Docker container is not necessary for making this application work!
FROM python:3.7ENV PYTHONDONTWRITEBYTECODE=1ENV PYTHONUNBUFFERED=1ENV FLASK_APP=app.pyENV FLASK_ENV=development# install system dependenciesRUN apt-get update \    && apt-get -y install gcc make \    && rm -rf /var/lib/apt/lists/*sRUN python3 --versionRUN pip3 --versionRUN pip install --no-cache-dir --upgrade pipWORKDIR /appCOPY ./requirements.txt /app/requirements.txtRUN pip3 install --no-cache-dir -r requirements.txtCOPY . .EXPOSE 8080CMD ["gunicorn", "--bind", "0.0.0.0:8080", "app:app"]
Note: the last line is for starting the Flask server
After creating the Dockerfile run
docker build -t YOUR_APP_NAME .
and afterward
docker run -d -p 80:8080 YOUR_APP_NAME
Afterward, you will see your app running on http://localhost/
Set up zappa
Set up AWS
AWS credentials
Deploy
AWS API Gateway — restrict access
Until now this was a rather easy path. Nothing too complicated, nothing too fancy. Now that we come to deployment it gets interesting and challenging.
Again, I would strongly encourage you to check out my previous article https://towardsdatascience.com/develop-and-sell-a-python-api-from-start-to-end-tutorial-9a038e433966 if you have any issues with Zappa and AWS.
I will not go so much in detail here anymore but rather point out pain points.
After we created the app locally we need to start setting up the hosting on a real server. We will use zappa.
Zappa makes it super easy to build and deploy server-less, event-driven Python applications (including, but not limited to, WSGI web apps) on AWS Lambda + API Gateway. Think of it as “serverless” web hosting for your Python apps. That means infinite scaling, zero downtime, zero maintenance — and at a fraction of the cost of your current deployments!
pip install zappa
As we are using a conda environment we need to specify it:
which python
will give you /Users/XXX/opt/anaconda3/envs/XXXX/bin/python (for Mac)
remove the bin/python/ and export
export VIRTUAL_ENV=/Users/XXXX/opt/anaconda3/envs/XXXXX/
Now we can do
zappa init
to set up the config.
Just click through everything and you will have a zappa_settings.json like
{    "dev": {        "app_function": "app.app",        "aws_region": "eu-central-1",        "profile_name": "default",        "project_name": "ml-api-covid",        "runtime": "python3.7",        "s3_bucket": "zappa-eyy4wkd2l",        "slim_handler": true,        "exclude": [            "*.joblib", "development", "models"        ]    }}
NOTA BENE! Do not enter a name for the s3 bucket as it cannot be found. I really don’t know what the problem with naming your s3 bucket is, but it never worked. There were multiple error statements and I could not resolve this. Just leave the suggested one and everything works fine. ;)
Note that we are NOT yet ready to deploy. First, we need to get some AWS credentials.
Note: This takes quite some effort. Do not be discouraged by the complexity of AWS and its policy management.
First, you need te get an AWS access key id and access key
I break it down as simple as possible:
Within the AWS Console, type IAM into the search box. IAM is the AWS user and permissions dashboard.Create a groupGive your group a name (for example zappa_group)Create our own specific inline policy for your groupIn the Permissions tab, under the Inline Policies section, choose the link to create a new Inline PolicyIn the Set Permissions screen, click the Custom Policy radio button and click the “Select” button on the right.Create a Custom Policy written in json formatRead through and copy a policy discussed here: https://github.com/Miserlou/Zappa/issues/244Scroll down to “My Custom policy” see a snippet of my policy.After pasting and modifying the json with your AWS Account Number, click the “Validate Policy” button to ensure you copied valid json. Then click the “Apply Policy” button to attach the inline policy to the group.Create a user and add the user to the groupBack at the IAM Dashboard, create a new user with the “Users” left-hand menu option and the “Add User” button.In the Add user screen, give your new user a name and select the Access Type for Programmatic access. Then click the “Next: Permissions” button.In the Set permissions screen, select the group you created earlier in the Add user to group section and click “Next: Tags”.Tags are optional. Add tags if you want, then click “Next: Review”.Review the user details and click “Create user”Copy the user’s keysDon’t close the AWS IAM window yet. In the next step, you will copy and paste these keys into a file. At this point, it’s not a bad idea to copy and save these keys into a text file in a secure location. Make sure you don’t save keys under version control.
Within the AWS Console, type IAM into the search box. IAM is the AWS user and permissions dashboard.
Create a group
Give your group a name (for example zappa_group)
Create our own specific inline policy for your group
In the Permissions tab, under the Inline Policies section, choose the link to create a new Inline Policy
In the Set Permissions screen, click the Custom Policy radio button and click the “Select” button on the right.
Create a Custom Policy written in json format
Read through and copy a policy discussed here: https://github.com/Miserlou/Zappa/issues/244
Scroll down to “My Custom policy” see a snippet of my policy.
After pasting and modifying the json with your AWS Account Number, click the “Validate Policy” button to ensure you copied valid json. Then click the “Apply Policy” button to attach the inline policy to the group.
Create a user and add the user to the group
Back at the IAM Dashboard, create a new user with the “Users” left-hand menu option and the “Add User” button.
In the Add user screen, give your new user a name and select the Access Type for Programmatic access. Then click the “Next: Permissions” button.
In the Set permissions screen, select the group you created earlier in the Add user to group section and click “Next: Tags”.
Tags are optional. Add tags if you want, then click “Next: Review”.
Review the user details and click “Create user”
Copy the user’s keys
Don’t close the AWS IAM window yet. In the next step, you will copy and paste these keys into a file. At this point, it’s not a bad idea to copy and save these keys into a text file in a secure location. Make sure you don’t save keys under version control.
My Custom policy:
As you can see in the policy, I added S3 related policies. This is because we want to download our pre-trained model from S3. More Infos on that later.
Create a .aws/credentials folder in your root with
mkdir ~/.awscode ~/.aws/credentials
and paste your credentials from AWS
[dev]aws_access_key_id = YOUR_KEYaws_secret_access_key = YOUR_KEY
Same with the config
code ~/.aws/config# and add:[default]region = YOUR_REGION (eg. eu-central-1)
Note that code is for opening a folder with vscode, my editor of choice.
Save the AWS access key id and secret access key assigned to the user you created in the file ~/.aws/credentials. Note the .aws/ directory needs to be in your home directory and the credentials file has no file extension.
Now you can do deploy your API with
zappa deploy dev
However, there are a few things to consider:
Zappa will pack your entire environment and whole root content. This will be quite large.There is an upload limit for AWS Lambda
Zappa will pack your entire environment and whole root content. This will be quite large.
There is an upload limit for AWS Lambda
There are several discussions on how to reduce the upload size with zappa. Check out the Inspiration section for links.
First, we need to reduce the package size for the upload.
We will put all exploratory content into an own folder. I named it “development”. Afterwards, you can specify excluded files and folder in zappa_settings.json with exclude:
{    "dev": {        ...        "slim_handler": true,        "exclude": [            "*.ipynb", "*.joblib", "jupytext_conversion/", ".ipynb_checkpoints/",            "predict_covid.ipynb", "development", "models"        ]    }}
You can add everything that doesn’t need to be packaged for deployment.
Another issue is the environment dependencies. In our case, we have multiple dependencies, which we don’t need for deployment. To solve this I created a new “requirements_prod.txt” file. This shall only have dependencies which are needed on AWS.
Make sure to export your current packages with
pip freeze > requirements.txt
Afterward, uninstall all packages
pip uninstall -r requirements.txt -y
Install new packages for deployment and save them in the file
pip install Flask pandas boto3 sklearn zappapip freeze > requirements_prod.txt
When you hit zappa deploy dev there should be considerably less size to package.
You will note that I also set slim_handler=true. This allows us to upload more than 50MB. Behind the scenes, zappa already puts content into an own S3 bucket. Read the zappa docs for more info.
Since we excluded our model from the AWS Lambda upload we need to get the model from somewhere else. We will use a AWS S3 Bucket.
During the development process, I tried to upload it there programmatically as well but I just uploaded it by hand as it was just faster now. (But you can still try to upload it — I still have an outcommented file in the repo)
Go to https://console.aws.amazon.com/s3/
“create bucket”give a name and leave rest as default. check for sufficient permissions.“create bucket”
“create bucket”
give a name and leave rest as default. check for sufficient permissions.
“create bucket”
check if you have a sufficient policy for interacting with the bucket and boto3. You should have something similar to
{      "Effect": "Allow",      "Action": [        "s3:CreateBucket",        "s3:ListBucket",        "s3:ListBucketMultipartUploads",        "s3:ListAllMyBuckets",        "s3:GetObject"      ],      "Resource": [        "arn:aws:s3:::zappa-*",        "arn:aws:s3:::*"      ]    }
Finally, there shouldn’t be any errors anymore. However, if there are still some, you can debug with:
zappa status# andzappa tail
The most common errors are permission related (then check your permission policy) or about python libraries that are incompatible. Either way, zappa will provide good enough error messages for debugging. Top list of errors from my experience are:
Policy issues with your user from IAMZappa and size issuesBoto3 and permission/location of files issues
Policy issues with your user from IAM
Zappa and size issues
Boto3 and permission/location of files issues
If you update your code don’t forget to update the deployment as well with
zappa update dev
To set up the API on a market we need to first restrict its usage with an API-key and then set it up on the market platform.
To break it down:
go to your AWS Console and go to API gatewayclick on your APIwe want to create an x-api-key to restrict undesired access to the API and also have a metered usagecreate a Usage plan for the API, with the desired throttle and quota limitscreate an associated API stageadd an API keyin the API key overview section, click “show” at the API key and copy itthen associate the API with the key and discard all requests that come without the keygo back to the API overview. under resources, click the “/ any” go to the “method request”. then in settings, set “API key required” to truedo the same for the “/{proxy+} Methods”
go to your AWS Console and go to API gateway
click on your API
we want to create an x-api-key to restrict undesired access to the API and also have a metered usage
create a Usage plan for the API, with the desired throttle and quota limits
create an associated API stage
add an API key
in the API key overview section, click “show” at the API key and copy it
then associate the API with the key and discard all requests that come without the key
go back to the API overview. under resources, click the “/ any” go to the “method request”. then in settings, set “API key required” to true
do the same for the “/{proxy+} Methods”
Now you have restricted access to your API.
Add new API
Test endpoint with rapidapi
Create code to consume API
I will not go into detail in this article anymore. Again, check my previous https://towardsdatascience.com/develop-and-sell-a-python-api-from-start-to-end-tutorial-9a038e433966 for setting everything up. There is no big difference in my new machine learning model.
https://rapidapi.com/Createdd/api/covid_new_cases_prediction
My main motivation this time came from Moez Ali, who provides great articles on deploying machine learning systems. I also enjoy following him on social media. I can recommend his articles:
https://towardsdatascience.com/build-and-deploy-your-first-machine-learning-web-app-e020db344a99
https://towardsdatascience.com/deploy-machine-learning-pipeline-on-aws-fargate-eb6e1c50507
Also François Marceau with
https://towardsdatascience.com/how-to-deploy-a-machine-learning-model-on-aws-lambda-24c36dcaed20
https://github.com/Miserlou/Zappa/issues/1927 Package Error: python-dateutil
https://stackabuse.com/file-management-with-aws-s3-python-and-flask/
https://ianwhitestone.work/zappa-zip-callbacks/ remove unnecessary files in zappa
https://stackoverflow.com/questions/62941174/how-to-write-load-machine-learning-model-to-from-s3-bucket-through-joblib
https://www.freecodecamp.org/news/what-we-learned-by-serving-machine-learning-models-using-aws-lambda-c70b303404a1/
https://ianwhitestone.work/slides/serverless-meetup-feb-2020.html
https://read.iopipe.com/the-right-way-to-do-serverless-in-python-e99535574454
https://www.bluematador.com/blog/serverless-in-aws-lambda-vs-fargate aws lambda vs fargate
https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/monitor_estimated_charges_with_cloudwatch.html billing alarm
Open source code:
https://github.com/Createdd/ml_api_covid
On Rapidapi:
https://rapidapi.com/Createdd/api/covid_new_cases_prediction
Daniel is an entrepreneur, software developer, and business law graduate. He has worked at various IT companies, tax advisory, management consulting, and at the Austrian court.
His knowledge and interests currently revolve around programming machine learning applications and all their related aspects. To the core, he considers himself a problem solver of complex environments, which is reflected in his various projects.
Don’t hesitate to get in touch if you have ideas, projects, or problems.