COVID-19: Impact on Housing Security Across the U.S.
Tags: Data Science, Python, Housing, Coronavirus, Covid 19

COVID-19: Impact on Housing Security Across the U.S. Jbochenek Follow Dec 10 · 14 min read

Housing is essential, but not guaranteed. This has never been more obvious than since the start of the COVID-19 lockdowns stranded Americans from their jobs, and thus their incomes. Without income, paying for routine and necessary bills such as food and housing can become a struggle. Housing insecurity is certainly not a new addition to America, but for the first time, we have week by week data on how it has impacted households across America.

Starting in April, the U.S. Census Bureau began a new project, the Household Pulse Survey, with the goal of determining the social and economic impacts of COVID-19 on the American populace. Phase one lasted from April 23rd to July 21st, and this analysis examines those 12 weeks (calendar savvy will notice that this is in fact 13 weeks, but that will be discussed below).

The Household Pulse Survey phase one results are available as Public Use Files (PUF), where each row is a response. However, due to privacy reasons, the PUF does not include location indicators, which was desired for this analysis. Instead, we used the summarized data which was slightly edited due to nested headers. The file we used is available here.

For this, we also worked in Google Colab for easier code sharing across the team. First we imported the necessary packages.

from google.colab import drive

import pandas as pd

import matplotlib.pyplot as plt

import seaborn as sns

import plotly.express as px

import plotly

from sklearn import preprocessing

from urllib.request import urlopen

import json # This will prompt for authorization.

drive.mount(‘/content/drive’)

Then we imported the data:

Household = ‘/content/drive/My Drive/Data/Housing/Household Pulse Survey/phase-one-household-pulse-survey-tool overall.xlsx’ Phase1 = pd.read_excel(Household, sheet_name=’Data’)

The data has three different levels of location, nationwide, state level, and the top 15 largest metro areas. It was important to separate these out, as we wanted to make comparisons within these location groups, not between these location groups. We grabbed only the rows we wanted into three different datasets:

State=[‘Alabama’, ‘Alaska’, ‘Arizona’, ‘Arkansas’, ‘California’, ‘Colorado’, ‘Connecticut’, ‘Delaware’, ‘District of Columbia’, ‘Florida’, ‘Georgia’, ‘Hawaii’, ‘Idaho’, ‘Illinois’, ‘Indiana’, ‘Iowa’, ‘Kansas’, ‘Kentucky’, ‘Louisiana’, ‘Maine’, ‘Maryland’, ‘Massachusetts’, ‘Michigan’, ‘Minnesota’, ‘Mississippi’, ‘Missouri’, ‘Montana’, ‘Nebraska’, ‘Nevada’, ‘New Hampshire’, ‘New Jersey’, ‘New Mexico’, ‘New York’, ‘North Carolina’, ‘North Dakota’, ‘Ohio’, ‘Oklahoma’, ‘Oregon’, ‘Pennsylvania’, ‘Rhode Island’, ‘South Carolina’, ‘South Dakota’, ‘Tennessee’, ‘Texas’, ‘Utah’, ‘Vermont’, ‘Virginia’, ‘Washington’, ‘West Virginia’, ‘Wisconsin’, ‘Wyoming’] US = [‘United States’] Metros=[‘Atlanta-Sandy Springs-Alpharetta, GA Metro Area’, ‘Boston-Cambridge-Newton, MA-NH Metro Area’, ‘Chicago-Naperville-Elgin, IL-IN-WI Metro Area’, ‘Dallas-Fort Worth-Arlington, TX Metro Area’, ‘Detroit-Warren-Dearborn, MI Metro Area’, ‘Houston-The Woodlands-Sugar Land, TX Metro Area’, ‘Los Angeles-Long Beach-Anaheim, CA Metro Area’, ‘Miami-Fort Lauderdale-Pompano Beach, FL Metro Area’, ‘New York-Newark-Jersey City, NY-NJ-PA Metro Area’, ‘Philadelphia-Camden-Wilmington, PA-NJ-DE-MD Metro Area’, ‘Phoenix-Mesa-Chandler, AZ Metro Area’, ‘Riverside-San Bernardino-Ontario, CA Metro Area’, ‘San Francisco-Oakland-Berkeley, CA Metro Area’, ‘Seattle-Tacoma-Bellevue, WA Metro Area’, ‘Washington-Arlington-Alexandria, DC-VA-MD-WV Metro Area’] StatesP1 = Phase1[Phase1[‘Geography (State or Metropolitan Area)’].isin(State)] USP1 = Phase1[Phase1[‘Geography (State or Metropolitan Area)’].isin(US)] MetroP1 = Phase1[Phase1[‘Geography (State or Metropolitan Area)’].isin(Metros)]

It soon became obvious that 50 states was a large number to handle, in visualization, so we added another level to the state data — Divisions. The U.S. Census defines the US by several location levels, one of the most familiar is Regions: Midwest, Northeast, South, and West. There are also Divisions, which split the regions up even smaller. Figure 1 below shows the breakdown of Regions into Divisions.

Figure 1. Regions and Divisions of the United States

We used a data dictionary to add that to the data. I’m including this here so maybe no-one else has to write this code again.

Divisions = {‘Alabama’: ‘East South Central’,

‘Alaska’: ‘Pacific’,

‘Arizona’: ‘Mountain’,

‘Arkansas’: ‘West South Central’,

‘California’: ‘Pacific’,

‘Colorado’: ‘Mountain’,

‘Connecticut’: ‘New England’,

‘Delaware’: ‘South Atlantic’,

‘District of Columbia’: ‘South Atlantic’,

‘Florida’: ‘South Atlantic’,

‘Georgia’: ‘South Atlantic’,

‘Hawaii’: ‘Pacific’,

‘Idaho’: ‘Mountain’,

‘Illinois’: ‘East North Central’,

‘Indiana’: ‘East North Central’,

‘Iowa’: ‘West North Central’,

‘Kansas’: ‘West North Central’,

‘Kentucky’: ‘East South Central’,

‘Louisiana’: ‘West South Central’,

‘Maine’: ‘New England’,

‘Maryland’: ‘South Atlantic’,

‘Massachusetts’: ‘New England’,

‘Michigan’: ‘East North Central’,

‘Minnesota’: ‘West North Central’,

‘Mississippi’: ‘East South Central’,

‘Missouri’: ‘West North Central’,

‘Montana’: ‘Mountain’,

‘Nebraska’: ‘West North Central’,

‘Nevada’: ‘Mountain’,

‘New Hampshire’: ‘New England’,

‘New Jersey’: ‘Middle Atlantic’,

‘New Mexico’: ‘Mountain’,

‘New York’: ‘Middle Atlantic’,

‘North Carolina’: ‘South Atlantic’,

‘North Dakota’: ‘West North Central’,

‘Ohio’: ‘East North Central’,

‘Oklahoma’: ‘West South Central’,

‘Oregon’: ‘Pacific’,

‘Pennsylvania’: ‘Middle Atlantic’,

‘Rhode Island’: ‘New England’,

‘South Carolina’: ‘South Atlantic’,

‘South Dakota’: ‘West North Central’,

‘Tennessee’: ‘East South Central’ ,

‘Texas’: ‘West South Central’,

‘Utah’: ‘Mountain’,

‘Vermont’: ‘New England’,

‘Virginia’: ‘South Atlantic’,

‘Washington’: ‘Pacific’,

‘West Virginia’: ‘South Atlantic’,

‘Wisconsin’: ‘East North Central’,

‘Wyoming’: ‘Mountain’} StatesP1[“State”] = StatesP1[“Geography (State or Metropolitan Area)”].astype(‘category’) StatesP1[‘Division’] = StatesP1[‘State’].map(Divisions)

We needed to do some exploratory data analysis to determine the quality of the data and any adjustments that would need to be made.

sns.displot(StatesP1, x=”Housing Insecurity Percent”, element=”step”, col=”Division”, col_wrap=3) g = sns.boxplot(x="Division", y="Housing Insecurity Percent", #hue="Selected Horizontal Dimension", data=StatesP1, palette="Set3")

g.set(xlabel='Division', ylabel='Housing Insecurity (%)')

g.set_xticklabels(g.get_xticklabels(),rotation=45,ha="right")

Figure 2. Histogram of Housing Insecurity Percent from the Household Pulse Survey by Census division from April 2020 — July 2020

Figure 3. Boxplot of Housing Insecurity Percent from the Household Pulse Survey by Census division from April 2020 — July 2020

Overall, we were very pleased with the distribution of data, in the histograms it shows as relatively normal and in the boxplots we only see one true outlier. For the purposes of this analysis, we kept that outlier as it was important trend data.

We also wanted to get a first look at the actual data, how has the housing security changed over the 12 week period across the US?

g = sns.relplot(kind = ‘line’, data=StatesP1, y =’Housing Insecurity Percent’, x= ‘Week Number’) g = sns.relplot(kind = ‘line’, col=’Division’, col_wrap=5, col_order =[‘Pacific’, ‘West North Central’, ‘East North Central’, ‘Middle Atlantic’, ‘New England’, ’Mountain’, ‘West South Central’, ‘East South Central’, ‘South Atlantic’ ], data=StatesP1, y =’Housing Insecurity Percent’, x= ‘Week Number’)