Web App development for Data Scientists | by Harsh Rana | Towards Data Science

I know some of you are probably thinking: why should a data scientist care about building a web application? The answer to your question is fairly simple: while analyzing datasets, finding patterns and building models are all important skills for a data scientist, it is equally important to communicate ideas and build interactive layers for your models. This can be especially important if you’re thinking about building a consumer facing product or even starting your own business.
The only person more valuable than a good data scientist, is a good data scientist who also understands engineering and deployment.
My goal with this project was to build a basic sentiment analysis machine learning model+web application and deploy it in under two weeks to learn three new technologies- natural language processing (NLP), flask and incremental/ongoing learning. In this article, I’ll cover building a flask web application to help users interact with a sentiment analysis model. You can use your own model, or feel free to look utilize this repository for one that I built previously. All my code for this post can be found on my Github so feel free to follow along.
Web applications can be as simple or as complex as you want them to be. While the architecture of complex applications can confuse even the best of developers, a simple web application isn’t very complicated. In a nutshell, it includes the following pieces:
HTML pages: these are the different webpages that your users can interact with. Many simple applications utilize a single webpage, hence the name single page application (SPA).Cascading Style Sheets (CSS): CSS is a style sheet language used to alter the appearance of an HTML page. Everything from text color to background is described in CSS.JavaScript: this is an object-oriented programming language which has taken over the web development community. Javascript is primarily used for introducing complex features on a webpage, such as interactivity, request handling etc.Web Server: this is the most confusing part of a web application. A web server can consist of hardware and/or software and its job is to “serve” resources to a user. Thus, when a user wants to see a webpage, the browser requests a resource from the web server over HTTP and receives a response. For our project, we’ll be using a flask server for development.
HTML pages: these are the different webpages that your users can interact with. Many simple applications utilize a single webpage, hence the name single page application (SPA).
Cascading Style Sheets (CSS): CSS is a style sheet language used to alter the appearance of an HTML page. Everything from text color to background is described in CSS.
JavaScript: this is an object-oriented programming language which has taken over the web development community. Javascript is primarily used for introducing complex features on a webpage, such as interactivity, request handling etc.
Web Server: this is the most confusing part of a web application. A web server can consist of hardware and/or software and its job is to “serve” resources to a user. Thus, when a user wants to see a webpage, the browser requests a resource from the web server over HTTP and receives a response. For our project, we’ll be using a flask server for development.
Now that you know the different parts of a web application, let me introduce you to Flask which will be the web application framework we’ll be using.
Flask is a lightweight and extremely versatile web application framework. It is built with flexibility and speed in mind, which makes it perfect for smaller applications with little-to-no backend needs. What makes Flask an especially useful tool in the data scientist’s toolkit is that it’s built in Python; which many data scientists are already comfortable with. So what does Flask do?
Flask takes care of a lot of the environment and project setup involved in a web application. What this essentially means is that you, the developer, can focus on your application while Flask takes care of HTTP, routing, assets etc. I promise all this will become easier to understand as we progress along this article.
Our Flask application will consist of the following pieces:
Based on the diagram above, you can clearly see the different functional pieces of our application. The app.py file will serve as the main server logic and help serve files and run any ML logic as needed. The static folder will contain any assets required for our app, from images to our pickeled models. Lastly, the template folder will contain the single HTML web page used in our app.
Now that you know what makes up a web application and why we’re using flask, let’s talk about how we’ll achieve all this.
Throughout this project, I’ve tried to follow the following mindset:
I wanted to do all this in a rapid building/learning mode with a focus on functionality, not on aesthetic.
Given this, I want to share the application development life cycle that I followed. This will help you understand how to value functionality over everything else and build rapid prototypes.
As you can see the main functionality will be built iteratively. This’ll help us always have a working application in place, while we keep adding features. We’ll outline the features needed in our application in the requirement gathering phase. Then, we’ll design the UI/UX for the entire application to avoid worrying about aesthetics later in the development period. After that, we’ll prioritize features and build them in 24 hour sprints. Finally, I’ll be deploying this application to an AWS EC2 instance, which will be covered entirely in another article. Let’s begin!
Our project involves the user interacting with a machine learning model, so we have to really understand how these interactions will occur. We have an NLP classifier that can predict whether a user is feeling positively or negatively about a subject. So, we need an input field where the user can enter a review/thoughts and an output location where the prediction will be displayed. Additionally, we also built a functionality to incrementally train our model. So, we need to provide the user with a way to provide feedback (correct/incorrect prediction). Those are all the interactions in our basic application.
Given the simplicity of our application, it makes sense to built this as a single page application. So, another requirement for us it that the input, output and feedback functionalities need to all be in the same page. This will help keep our project lean and minimal and reduce development time.
I like drawing UI designs on a piece of paper before writing any code because we can make changes faster, revamp entire sections and have a concrete design in place for rapid building. Additionally, I also believe that starting out with all different functionalities on different screens, helps putting everything together into a single page application (SPA) without missing any pieces. My UI/UX design progression can be seen here.
I started out with three different screens (design #1) each with it’s own functionality. The first screen would be where a user can enter an input and click predict. The second would be where the user sees the output and has the option to mark it as a correct or incorrect prediction. Finally, the last screen would be a confirmation screen so the user knows his feedback was taken and an option to start again.
These three screens molded together for my first SPA design (design #2). This was a static design with placeholders where the individual pieces (output, feedback buttons etc.) would become active when needed. I considered it active because none of the elements would hide at any point; they were all always visible.
My final design was a dynamic version of the static SPA (design #3), where output and feedback elements would initially be hidden (left side screen) and then appear when the user clicked the prediction button (right side screen). I was pretty happy with this design and comfortable moving forward at this point with the actual development.
Before starting development, I created a priority queue of my features needed to finish this project. They can be seen (in order) below:
Build out the static user interface for the entire application (vanilla HTML+CSS)Capture input from user, run it through our model and come up with a prediction, which is displayed in the frontendCapture user feedback and appropriately teach the model using the incremental learning functionality built beforeChange the UI from static to dynamic: hide unnecessary elements on startup and show them when required
Build out the static user interface for the entire application (vanilla HTML+CSS)
Capture input from user, run it through our model and come up with a prediction, which is displayed in the frontend
Capture user feedback and appropriately teach the model using the incremental learning functionality built before
Change the UI from static to dynamic: hide unnecessary elements on startup and show them when required
At the end of each iteration, I would have a working application with limited functionality. This would help test and debug new features rapidly. Let me share my code and progress through each of these features.
Building out a static user interface was fairly simple. If you’ve never developed for the web before, I would highly recommend checking out MDN Web Docs for any and all questions you have. Given the simplicity of our application, our HTML code was minimal and while I do not want to waste your time describing this in depth, I’ll cover it from a high-level. Our UI was broken up into two main <div> tags: one for the top half which contained the instructions, input text-box and prediction button, and the bottom half which contained the prediction output and feedback buttons. Most of the HTML associated with this project should make sense to anyone who’s done web development before, but I do want to point out one important flask specific code:
<form id="input-form" class="input-form"    action="{{ url_for('predict')}}" method="post">     ..     ..     ..</form>
The {{url_for(‘predict’)}} attribute in the form tag is a flask method which points to the “predict” route. This association helps us capture user input from the form and process it inside our app.py file where our model is available. You’ll see this in action when we cover flask in the next section.
Lastly, To style this HTML page, I primarily used CSS’s flexbox display mode to create a minimal layout. So even though none of the buttons worked yet, and the image was unchanging, my static layout at the end of this iteration can be seen below:
I won’t be covering CSS in this article given the audience’s interest, but if you’re interested you can find all my code here.
At the end of the previous section, we only had an HTML and CSS file. Now, we’ll start using the app.py file to build our application’s logic and also move from directly displaying the HTML page to serving it using the Flask server. To start serving the HTML page using Flask, we only need a few lines of code:
from flask import Flask, render_template, requestapp = Flask(__name__, template_folder='../templates', static_folder='../static')@app.route('/')def home():    return render_template('index.html', image_filename="img/happy.webp", display_mode="none")@app.route('/predict', methods=['POST'])def predict():    return render_template('index.html', image_filename="img/happy.webp", display_mode="none")@app.route('/save_pred', methods=['POST'])def save_pred():    return render_template('index.html', image_filename="img/happy.webp", display_mode="none")if __name__ == "__main__":    app.run(debug=True)
Note: Flask doesn’t require the template_folder and static_folder to be explicitly mentioned, but I like doing this in case I mess up names/paths. Also, at this point, our project folder structure should look like this:
This way all our assets (images, models etc.) are in different sub-folders inside the same main folder static , our HTML page is inside the templates folder and our app.py file is in the python folder.
At this point, we’re ready to run our application using the Flask server. To do so: navigate to the python folder using your command line and run the command python app.py . Remember that you should’ve already install Flask using pip at this point. More information about that here. Congrats, your application is up and running! Now lets add the prediction logic.
Notice how we’ve put placeholders for both the predict and the save_pred routes. This is done to ensure that the Flask app deploys without errors. To make our predict function work, we need to capture the user’s input from the textbox, convert it into a bag-of-words vector using our NLP vectorizer (input_transformer.pkl) and run that through our classifier (review_sentiment.pkl). Also note that I’ll be using global variables to store the model input and output, to persist them across the prediction and incremental learning functions. The code for this functionality can be seen below:
# module needed to load modelsfrom sklearn.externals import joblib# load models pickeled earlierinput_transformer = joblib.load(open('../static/models/input_transformer.pkl', 'rb'))model = joblib.load(open('../static/models/review_sentiment.pkl', 'rb'))# Global variables for persistence across methods (and requests)model_input=""model_output=""@app.route('/predict', methods=['POST'])def predict():    # retrieve global variables to store input and output    global model_input    global model_output        # get text from the incoming request (submitted on predict button click)    text = request.form['input_text']        # convert text to model input vector    final_features = input_transformer.transform([text])        # use classifier's predict method to get prediction    prediction = model.predict(final_features)        # store model input and output    model_input = text    model_output = prediction[0]    return model_output
There you have it! Now our app can convert user-inputted text into a vector which can be classified by our pre-trained model.
The steps we take to make this happen are the following: vectorize input text using the global variable, get the button user clicked on (correct/incorrect), assert “correct” prediction based on the user’s button choice, incrementally train model till it relearns, update model pickle. The code for this can be seen below:
@app.route('/save_pred', methods=['POST'])def save_pred():    # retrieve global variables    global model_input    global model_output    # vectorize user input    final_features = input_transformer.transform([model_input])        # get user's button choice (correct/incorrect)    save_type = request.form["save_type"]        # modify global variable if user selected "incorrect"    if(save_type == 'incorrect'):        if(model_output == 'p'):            model_output = 'n'        elif(model_output == 'n'):            model_output = 'p'        else:            print("Error: Model output was neither N nor P")    # Strengthen weight of particular connection    max_iter = 100    counter = 0    for i in range (0,max_iter):        model.partial_fit(final_features, [model_output])        if(model.predict(final_features) == [model_output]):            counter = i            break        # Save trained model pickle    joblib.dump(model, '../../review_sentiment.pkl')           # return placeholder (will be updated later)    return ""
While the code above achieves all the functionality that we need, it does not provide the user with any sort of feedback once the retraining is complete. To address this, we’ll add a return text variable to either return “the weights were changed, thank you for correcting me!” or “the weights were strengthened, thank you for teaching me!” based on the situation. Additionally, I also believe that it is very important to capture the data that the model is being retrained on to prevent misuse of this functionality. A few lines of code can help us store the input, the corrected output and the number of cycles it took to retrain in a csv file:
fields = [model_input, model_output, counter]    with open('user_teaching_data.csv', 'a') as file:        writer = csv.writer(file)        writer.writerow(fields)
Iteration 4: Dynamic Content
With our static app complete, this iteration was all about making it dynamic and more user-friendly. You must’ve noticed that all the elements were always visible in our app and the page was reloading each time. While this isn’t a big deal, we can easily use jQuery and AJAX to avoid reloading and also hide elements as needed. This will help give our application a polished look and also transform to positive user experience. Going in-depth into either jQuery or Ajax is beyond the scope of this article, but if you’re interested I highly suggest checking out my code, and this awesome article about AJAX.
At the end of this iteration, our app was working as expected: it accepted user information, displayed the results from our NLP models, allowed the user to retrain our model and did all this in a minimal and elegant manner. Our fully functioning web app demo can be seen again below:
Congrats on getting this far into our two week challenge! Hopefully, by now you’re not only capable of building an NLP model, but also have the necessary skills to build web applications to make those models interactive.
So far we’ve achieved a lot of cool things: we took our previously trained data science model and built a fully functional web application around it. We spent around six days building our web application by focusing on functionality over aesthetics. This mindset allowed us to build our model and web application all within ~10 days.
With four more days left in our two week period, we’ll tackle our biggest challenge yet: model and application deployment on AWS using a vanilla EC2 instance. I’ll be writing another post to guide you through the deployment process, so feel free to follow my column on Towards Data Science to get a notification when it’s published!
I hope you enjoyed reading this article just as much as I enjoyed putting it together. If you have any questions, feedback or comments, feel free to get in touch with me through LinkedIn, or my website.