An end-to-end machine learning project with Python Pandas, Keras, Flask, Docker and Heroku | by Ryan Lamb | Towards Data Science

At the time of the Rugby World Cup in 2019 I did a small data science project to try and predict rugby match results, which I wrote about here. I’ve expanded this into an example end-to-end machine learning project to demonstrate how to deploy a machine learning model as an interactive web app.
To provide a high-level overview of the key steps needed in going from raw data to a live deployed machine learning app.
Once you’ve gone through this — pick a topic that you’re interested in, find some data, get your hands dirty and aim to build your own machine learning app, from data preparation to deployment.
Data wrangling with Pandas & data storage with SQLiteMachine learning (Neural Network) with KerasWeb app with Flask (and a bit of CSS & HTML)App deployment with Docker and Heroku
Data wrangling with Pandas & data storage with SQLite
Machine learning (Neural Network) with Keras
Web app with Flask (and a bit of CSS & HTML)
App deployment with Docker and Heroku
The code for this is available on GitHub here and the live app can be viewed here. (Note that this code isn’t necessarily production level, but meant to show what can be done as a starting point. The live app uses a snapshot of data at a specific point in time).
Data scientists spend about 45% of their time simply getting data ready for model development
Data wrangling includes cleaning, structuring and enriching your data to get it ready for your machine learning model. For example, dealing with blank data or inconsistent formats.
Unless you’re using an already clean dataset (which seldom happens in the real world), you’d need to perform Exploratory Data Analysis (EDA) to better understand the data profile.
Pandas-profiling is an excellent module for EDA. It generates profile reports of DataFrames including quantile and descriptive statistics, correlations, missing values and histograms.
For this project, I needed data from ESPN Scrum. You can find detailed cleaning steps in the data_prep.py file, for example defining the Match Date format with the to_datetime() method. I also enriched the data with the World Rugby Rankings and a measure of skill using Microsoft’s TrueSkill rating system.
If you’re looking for datasets to get started, UC Irvine’s Machine Learning repository and Kaggle are good sources to explore. Alternatively you can get data from scraping using BeautifulSoup.
I stored my DataFrames as tables in a SQLite database. This is a lightweight database and the mostly widely deployed in the world. Plain csv files can of course be used, but this provides an opportunity to learn about SQLite. The code below connects to the match_results database and writes the latest_stats DataFrame to it as a table.
conn = sqlite3.connect('match_results.db')latest_stats.to_sql('latest_stats', conn, if_exists='replace')
Now we’ve got the data - time to test and train the machine learning model.
Mean Squared Error (MSE) was used as a loss function to minimize. Hyperparameter tuning was performed (Jason Brownlee has a tutorial here), and I experimented with both wide (one layer with a lot of neurons) and deep (more layers but fewer neurons per layer) networks until the MSE didn’t improve significantly.
You want the model to be able to perform well with unseen data. If you overfit your model, you may have a low error on your training data but a high error on your test (unseen) data. When this happens, the model has learned the training data almost too well and cannot generalize to new data. There are various techniques to avoid overfitting.
Below you can see the MSE decreasing rapidly up until ~25 epochs and then remaining relatively stable for both training and testing data.
The final model had a first layer of 15 neurons and a second layer of 8 neurons, both with the Rectified Linear Activation Unit (ReLU).
It was compiled with the Adam optimization algorithm, which is an efficient gradient descent algorithm.
model = Sequential()model.add(Dense(15, input_dim=5, kernel_initializer='normal', activation='relu'))model.add(Dense(8, kernel_initializer='normal', activation='relu'))model.add(Dense(2, kernel_initializer='normal'))model.compile(loss='mse', optimizer='adam')
The save() function is used to save the final Keras model. This saves the model architecture and weights, and will allow us to load the model later on in the app to make predictions on new data.
Flask is a web framework that can be used for developing web applications relatively quickly. You can find a walk through for quick development here.
The code below in the app.py file essentially sets up the home page and serves the index.html to the user:
WTForms is used for the form so that the user can select their two rugby teams. When the button is clicked, the inputs are sent to the saved ML model via the model_predict function. The resulting model predictions are then sent as input variables to the prediction.html template.
@app.route("/", methods=['GET', 'POST'])def home():    form = TeamForm(request.form)    # Get team names and send to model    if request.method == 'POST' and form.validate():        team1 = request.form['team1']        team2 = request.form['team2']        return render_template('prediction.html', win_prob=model_predict(team1,team2)[0],                               score=model_predict(team1, team2)[1])    # Send template information to index.html    return render_template('index.html', form=form)
The prediction.html template does three key things:
Gets the win probability and score sent from the output of the model to display to the end user (via the win_prob and score variables)Provides a button for the user to return to the home pageReferences the main.css file to provide some styling
Gets the win probability and score sent from the output of the model to display to the end user (via the win_prob and score variables)
Provides a button for the user to return to the home page
References the main.css file to provide some styling
<!DOCTYPE html><head>  <title>Prediction</title>  <link rel="stylesheet" href="/static/css/main.css"></head><body>  <div class="container">    <h1>      <center>Predictions</center>    </h1>    {% block content %}    <div class="flash">{{win_prob}}</div>    <div class="flash">{{score}}</div>    <form action="/button" method = "POST">    <p><input type = "submit" value = "Return" /></p>    </form>    {% endblock %}  </div></body></html>
And that’s it! In summary, you have:
Two templates, index.html and prediction.html for the home page and prediction results page
CSS files to provide styling
Your saved model file to make predictions
The app.py Flask code to define your pages and their functionality and to run your app
Now to deploy your app and share it with the world.
I won’t go into much detail about Docker but the key thing to know is that it allows you to package your application in a self-sufficient container that can be used across systems i.e. it creates a portable container that makes it easier to deploy your app. You can dig into the details in the official Docker documentation here.
Download Docker Desktop here.
Create a Dockerfile in your app directory. This contains the instructions for building your Docker image. Best practices for Dockerfiles can be viewed here and here. Below you’ll see I’ve included instructions to run the requirements.txt file that contains the app dependencies.
FROM python:3.7.3-stretchRUN mkdir /appWORKDIR /app#Copy all filesCOPY . .# Install dependenciesRUN pip install --upgrade pipRUN pip install -r requirements.txt# RunCMD ["python","./main.py"]
Open a terminal and go to the directory containing your Dockerfile and app. Use the build command to build your image: docker build -t rugby_prediction . This builds the Docker image from the Dockerfile.
You can run the app locally using the runcommand which creates a container: docker run --name rugby -p 5000:5000 -d rugby_prediction. Here, the -p publishes the container’s port to the host and -d makes the container run in the background. Once this is running, you should be able to view your app running in your browser (in Docker Desktop, click on the open in browser button).
If you experience any errors when running your app, you can use the docker logs command to view the logs and debug.
I used Heroku for hosting because they have a good free tier for non-commercial apps.
Once you’ve created a Heroku account, create a new app and select Container Registry as the deployment method
Download the Heroku CLI and log in to your Heroku account:heroku login
Sign in to the Container Registry: heroku container:login
Go to your app working directory and push the Docker image: heroku container:push web --app [YOUR APP NAME]
Deploy your app: heroku container:release web --app [YOUR APP NAME]
Done! You are ready to share your live machine learning web app.
Hope this has been helpful to get you started. Feel free to post links to your own ML apps in the comments - would be great to see them.
If you’d like to collaborate on rugby (or sport) analytics, drop me a message on LinkedIn!