A Full Stack Machine Learning Project | by Natassha Selvaraj | Towards Data Science

When I was a kid, I used to be obsessed with Harry Potter (the books, not the movie). I was a huge fan of Hermione Granger, and was deeply intrigued by the character of Severus Snape.
So when I found an MBTI personality prediction dataset, I decided that there was no better way to use it than create a Harry Potter character prediction model.
As you can see, the interface takes in some text as input, calls the back-end model, and generates a prediction.
As (aspiring) data scientists, our focus is mainly on the data and building models. Most of our machine learning projects lie in a carefully formatted Jupyter Notebook, and will probably stay there forever.
However, creating a product that people can interact with is an important skill.
Recruiters go through hundreds of profiles each day.
Will they have the time to look up all your blog posts and read all your code to see what your project is about?
Probably not.
However, if you create an environment they can interact with and actually use, they will be far more interested in your project.
Now, I will go through all the steps I took to complete this machine learning project.
I used this MBTI personality type dataset to create and train the model. I used the fastai library to do this.
Building a classifier from scratch to work with text data takes a lot of time, and requires a lot of data cleaning and pre-processing. Instead, I used a pre-trained model from fastai’s text module.
If you want to create a similar text classification model, I suggest following their guide here.
All you need to do is install the required libraries, and use your dataset to train the pre-existing model.
I used Google Colab to train the model on my dataset, since it allows users to connect to a GPU back-end for free. This makes training deep learning models a lot faster.
If you haven’t used Colab before, here is a guide on getting started.
After creating and training the model in Colab, I exported it to my Jupyter Notebook, and used it to make predictions with random sentences.
The predictions I got were tensors:
pred = learn.predict('Happiness is not something ready-made.')
I had to convert them to string:
print(str(pred[0]))
The output: ‘ENFJ’
Now, all that was left to do was create a front-end interface, and output Harry Potter character names for each personality type.
A Dash app is a software that allows you to create a front-end interface from within your work environment. It is easy to use without having to know front-end languages such as HTML, CSS, and JavaScript.
As data scientists, most of our work is within a Jupyter Notebook. It is the place we are most comfortable with, and creating separate files with different extensions can be time consuming.
The solution?
JupyterDash!
JupyterDash is a library that allows you to create Dash apps directly from your Jupyter Notebook.
It is very simple to use, and here is some code to help you get started:
# importsfrom jupyter_dash import JupyterDashimport dashfrom dash.dependencies import Input, Output, Stateimport dash_core_components as dccimport dash_html_components as htmlfrom dash import no_updateexternal_stylesheets = ['https://codepen.io/chriddyp/pen/bWLwgP.css']app = JupyterDash(__name__, external_stylesheets=external_stylesheets)app.layout = html.Div([    html.H1(children="Personality prediction: Which Harry Potter character are you?", style={'textAlign': 'center'}),    html.Img(src='data:image/png;base64,{}'.format(encoded_image.decode()),style={'height':'350px','width':'550px'}),    dcc.Markdown('''                ###### Step 1: Type a sentence that you feel represents your personality.                ###### Step 2: Click submit                ###### Step 3: Wait for your prediction to appear!    ''',style={'textAlign': 'center'}),        dcc.Input(id='username',type='text'),    html.Button(id='submit-button', type='submit', children='Submit'),    html.Div(id='output_div')    ],style={'textAlign': 'center','justify':'center','align':'middle','verticalAlign':'middle'})learn = load_learner("C:/Users/natassha selvaraj/Desktop")@app.callback(Output('output_div', 'children'),                [Input('submit-button', 'n_clicks')],                [State('username', 'value')],                )def update_output(clicks, input_value):    answ = learn.predict(input_value)    if clicks is not None:        if input_value is not None:                if(str(answ[0])=='ISTP'):                    return(html.H2(children='Harry Potter (ISTP)', style={'textAlign': 'center'}))                elif(str(answ[0])=='ENFP'):                    return(html.H2(children='Ron Weasley (ENFP)', style={'textAlign': 'center'}))                elif(str(answ[0])=='INTP'):                    return(html.H2(children='Hermione Granger (INTP)', style={'textAlign': 'center'}))                elif(str(answ[0])=='ENFJ'):                    return(html.H2(children='Albus Dumbledore (ENFJ)', style={'textAlign': 'center'}))                elif(str(answ[0])=='ESTP'):                    return(html.H2(children='Ginny Weasley (ESTP)', style={'textAlign': 'center'}))                elif(str(answ[0])=='ISFJ'):                    return(html.H2(children='Neville Longbottom (ISFJ)', style={'textAlign': 'center'}))                elif(str(answ[0])=='INFP'):                    return(html.H2(children='Luna Lovegood (INFP)', style={'textAlign': 'center'}))                elif(str(answ[0])=='ESFP'):                    return(html.H2(children='Fred and George Weasley (ESFP)', style={'textAlign': 'center'}))                 elif(str(answ[0])=='ISFP'):                    return(html.H2(children='Rubeus Hagrid (ISFP)', style={'textAlign': 'center'}))                elif(str(answ[0])=='ISTJ'):                    return(html.H2(children='Severus Snape (ISTJ)', style={'textAlign': 'center'}))                elif(str(answ[0])=='ENTJ'):                    return(html.H2(children='Minerva McGonagall (ENTJ)', style={'textAlign': 'center'}))                elif(str(answ[0])=='ESFJ'):                    return(html.H2(children='Fleur Delacour (ESFJ)', style={'textAlign': 'center'}))                elif(str(answ[0])=='INFJ'):                    return(html.H2(children='Remus Lupin (INFJ)', style={'textAlign': 'center'}))        return('Please enter a valid sentence')            app.run_server(mode='external')
The output cell looks like this:
Just click on the URL, and you will find your Dash app running:
If you want a more in-depth Jupyter-Dash tutorial, I suggest the following resources:
Introducing JupyterDash — Medium ArticleJupyterDash Tutorial — YouTube VideoSetting up Dash Plotly in Jupyter — YouTube Video
Introducing JupyterDash — Medium Article
JupyterDash Tutorial — YouTube Video
Setting up Dash Plotly in Jupyter — YouTube Video
Now, your Dash app is running on your local server. You will need to host it somewhere for other people to be able to access it.
One popular option is Heroku. Heroku allows you to host your app directly from your remote server. You don’t need to pay for hosting, and the setup is very quick and easy to follow. Here is a guide on deploying your app with Heroku.
For this project, I used ngrok. Ngrok is an application that allows you to expose a development server to the Internet with minimal effort.
It is generally used to test sites before deployment, instead of actually hosting a web application. Also, if you use the free tier, the URL can change every time you run the app.
However, ngrok is very easy to set up, and is great if you want to share some quick applications or dashboards with friends or potential employers.
Here’s how you can use ngrok:
! pip install pyngrok
Then, visit their website and log in (or sign up, if its your first time).
You will see a page like this, with a token in number 2:
Copy the token. Run the following lines of code in your Notebook and insert the token in place of ‘xxxxx’:
! ngrok authtoken xxxxxx
Add the following lines of code:
from pyngrok import ngrok# Open a HTTP tunnel on the default port 80public_url = ngrok.connect(port='8050')print(public_url)
Copy-paste the URL that appears into your web browser, and your site will run!
Here is the Dash app I deployed on ngrok (It will take some time to load).
Here is a comprehensive video tutorial on ngrok.
Never stop learning. Life never stops teaching.