Bringing Your ML Models to Life With Flask
Tags: Deployment, Mlops, Flask, Data Science, Machine Learning

These predictions, as well as the input features used to make them, will no longer come in a CSV format, but rather as a JSON object. Why a JSON object? JSON objects are easy to pass between different applications, and can also easily be read into a pandas dataframe, just like the data we originally used to train our model.

We will also need to change how predictions from our model are returned. In the code we used to train the model, we returned all the predictions from the training and validation sets and used them to calculate performance metrics like accuracy. In the operational phase, we will need to return each individual prediction in real time. We will also no longer need to calculate scores like accuracy.

The following code shows the .py file for doing the main steps needed in the operational phase:

Note that in this prediction script, we load the pickled models and transformers and apply them to new, unseen data in JSON format.

We now know that we can apply our trained model to a new datapoint in JSON format and get a prediction!

Step 3: Creating a prediction web service with Flask

Now that we have updated the prediction input and return formats to JSON and incorporated loading in the saved model and transformer, the next step in our deployment process is to make our predictor available. This means both available to receive new datapoints with which to make predictions, and available to supply these predictions to some other application that will use them. This is where we will use Flask to transform our prediction script from a local script that can only run on our machine, to a kind of prediction factory. When we use Flask to transform our prediction script from a script to a web service, it’s like opening the factory doors. Now, we no longer simply make predictions on local data inside our closed factory. Rather, we now accept requests for new predictions from outside “customers”, and then provide them with predictions.

The most famous example of a web service is a web server. A web server can be thought of as a webpage-producing factory that takes requests all day long for different web pages (in the form of a URL), and then returns those requests (in the form of HTML code that renders in your browser as a pretty webpage). Our web service will work much the same way, but instead of taking requests in the form of different URLs, it will take requests as a JSON, and instead of returning the HTML needed to show a webpage, it will return a JSON of predictions.

So how do we open the doors to our factory? As the name suggests, transforming our script into a web service involves making it available over the internet. The code below shows the changes that were made to the prediction script to turn it into a web service:

The first thing we notice is that we instantiated a Flask app, by making an instance of the Flask class. This is the first step for opening up our “factory” — you can think of it like getting the necessary business permits. The next step is to set a URL that will trigger our prediction function. Keeping with our factory example, this is like the address to which our customers will send their requests for predictions. We do this with the app.route() decorator from flask, which is placed just above our prediction function in the code.

If we want our factory to produce different products, we can specify different addresses for the different types of “orders”. The link we specify in the app.route() decorator placed just above the function defines the address where the requests for the product produced by that particular function should go.

Finally, we change how we get the input data, and how we return our predictions. Before, in our predictions script, we just loaded in a JSON file from our local machine. Now, we want to accept JSONS that are inside a request. We have to use the requests package from Python to parse the incoming JSON our of the incoming request. We then also return a JSON again on the way out. You can note these changes in the generate_predictions() function.

The final change is to the last part of the script. It now simply runs the app, rather than calling the generate_predictions() function directly. In doing so, it still calls the generate_predictions function, since this is the function that is triggered when the app starts.

Now that we have our server script, we can test it on our local machine by sending it a test request.

The first step is simply to run our new python web service script from the command line. When we run it, it starts up the Flask app, calling our web service on a local port, meaning it just runs on our local machine.