Building a data dashboard for housing prices using Plotly-Dash in Python
By: Martin Palkovic
Tags: Machine Learning, Data Visualization, Python, Dash, Data Science

For this project, I’ll be using Plotly-Dash, which is a Python library for creating analytical data web apps. Plotly is a data visualization library, similar to matplotlib, but allows you to make interactive graphs, and integrate them in to Dash.

I began by heading to Zillow [1] to download data on housing prices — I used Zillow’s Home Value Index (ZHVI) dataset for all homes (single family residences and condos) as raw time series data, which looked at homes in the ‘mid tier’ price range. Zillow defines the ‘mid tier’ as homes with values in the 35th to 65th percentile range, and the median is captured within this range. I downloaded income data [2] from Kaggle — while some consider Kaggle datasets too ‘clean’ for a data science project, I opted to go for it since I’m actually combining a few different datasets to create my own dataset for the dashboard. Lastly, I called the Geonames API to download latitude, longitude and population data for each city. Once I dove in to the Zillow dataset, I discovered numerous cities that I needed to rename to work with the Geonames API. Cleaning data isn’t the most glamorous part of the job, but it’s an essential one that doesn’t get enough attention, in my opinion. The income data also had many entries for some cities, so I grouped them together by the mean value of each city:

After a bit of cleaning, I put the cities in to a Python list and looped through them with the Geonames API to iteratively download latitude, longitude, and population for each city. Note that you’ll need to register on Geonames.org to use their Python API, and the key is your username:

With these results in hand, I added the population data to a Python dictionary with the city name as the key, and the population as the value, so that I could easily determine which cities had a zero value for the population (i.e, the API didn’t work). I ran some quick statistics on the success rate of the API for pulling population data, and discovered that it successfully downloaded population data for ~ 94% of the cities. I manually added populations for the other 6% from wikipedia. Remember: coding will never get you all of the way there. But in this case, it got me 94% of the way there, and that saves me a lot of time. Coding does NOT negate the need to perform manual quality control on your results.

Next, I performed a left outer join on the income dataset to merge it with the Zillow dataset. This is a SQL concept meaning that the size of the Zillow dataset will stay the same. Income data only gets added to the Zillow dataset if it exists for a city. If no income data exists for a city, it fills the column with a NaN (not a number) value. I also finalized the dataset for usage in Dash as a data table:

Lastly, I also finalized a dataframe for use in a home price time series (‘df_h’ below) on the dashboard, and looped through each city to forecast housing prices for the next two years. I used the autoregressive integrated moving average (ARIMA) model from the statsmodels library. This is a great go-to machine learning algorithm for time series data. If you’re curious as to why I didn’t use Scikit-Learn for this task, check out this example on Github.

Alright! We’ve finally got all of the data ready to build our dashboard. I’m a big fan of Jupyter Lab for most Python development, as I love to be able to easily run my code as I’m writing it. However, You’ll likely want to switch to a text editor of some sort for deploying your app in Dash — I opted for Spyder. You can run Dash in Jupyter Lab, but I ended up preferring a text editor.

To begin the app creation, you’ll want to make a directory on your computer to store all of the files associated with the app, and you’ll need to initialize a virtual environment in that directory. Virtual environments are a standalone instance of Python in the folder you create, and you’ll have to re-install any libraries you need that are not included in the Python standard libraries. Input the following code in to your Terminal (Mac):

$ mkdir housing_dash && cd housing_dash

$ python3 -m venv venv

$ source venv/bin/activate (venv) $ python -m pip install dash

(venv) $ python -m pip install plotly

(venv) $ python -m pip install pandas

(venv) $ python -m pip install gunicorn

Add the dataframes you’ll need for the app to this folder, and create a python file (simple is the convention here, I opted for ‘app.py’) in this same folder. Additionally, you’ll need to make the following files in your app folder (bolded text = file name):

‘requirements.txt’ with the versions of the following libraries your virtual environment uses (adjust version accordingly if you are running a different version):

dash==1.19.0

pandas==1.2.3

gunicorn==20.0.4

‘runtime.txt’ with the version of Python running on your virtual environment:

python-3.9.0

‘.gitignore’:

venv

*.pyc

.DS_Store

.env

‘Procfile’:

web: gunicorn app:server

If you need to check the version of Python or your Python libraries, you can call the following in your terminal:

#check Python version

(venv) $ python3 --version

Python 3.9.0 #check Pandas version

$ python3

>>>import pandas as pd

>>>pd.__version__

'1.2.3' #check Dash version

$ python3

>>>import dash

>>>dash.__version__

'1.19.0'

Alright, that’s it for folder set up. Below is my full Dash app code. I imported the csv files we created above, laid out the app, and created ‘app callbacks’ to add interactivity to the figures. App callbacks use a function that you define, and any time one of the input arguments changes (ex. dropdown, the user queries a table, etc), the callback calls the function on the output you specify. Note that you can’t have any white space between the function and the callback — the function must be placed on the next line immediately below the callback, or it won’t work.

Dash uses dictionaries and lists extensively in its keyword arguments, so getting familiar with these Python concepts is definitely helpful when building a Dash app.

To deploy the app, it’s recommended to use Git and Heroku, although there are other ways to deploy your app so that anyone can view it. I wont go into great detail on this part, but if you’ve followed along up to this point, you’re most of the way there. Dash has excellent documentation on how to do this and there are plenty of resources on the web to help you deploy your app. Even if you’re experienced with Python, deploying your app can be a maddening process. If your app successfully deploys to Heroku, you’ll probably get an error message the first time you load the app in a browser, and feel just like this:

Credit: Big Bang Theory.

A few tips I can offer:

Double check your requirements.txt and runtime.txt to ensure the versions you used to develop your code exactly match what are in your .txt files. Your Python version also has to be written exactly like this: python-3.x.x’,

and to ensure the versions you used to develop your code exactly match what are in your .txt files. Your Python version also has to be written exactly like this: python-3.x.x’, Triple check your code, and run this in your terminal:

heroku --tail

Take a deep breath and be patient. You’ve got this 😃

As I was working on this project, I couldn’t stop thinking about the episode of Rick and Morty where aliens try to take over earth by tricking Jerry into developing an app with them 😆

Credit: Giphy/RickandMorty.com

Below is a video demo of me using the app and some screenshots. The time series has a dropdown that will show the housing prices vs year for each city in the dataframe (913 in total), with two year housing price forecasts for approximately half of those cities. Additionally, the map and graph will update when you filter the datatable. You can enter things like ‘>300000’, or ‘<100000’, and the table will accept multiple queries: