Exploring the US Cars Dataset
By: Sadrach Pierre
Tags: Software Development, Programming, Python, Data Science, Technology

The US Cars Dataset contains scraped data from the online North American Car auction. It contains information about 28 car brands for sale in the US. In this post, we will perform exploratory data analysis on the US Cars Dataset. The data can be found here.

Let’s get started!

First, let’s import the Pandas library

import pandas as pd

Next, let’s remove the default display limits for Pandas data frames:

pd.set_option('display.max_columns', None)

Now, let’s read the data into a data frame:

df = pd.read_csv("USA_cars_datasets.csv")

Let’s print the list of columns in the data:

print(list(df.columns))

We can also take a look at the number of rows in the data:

print("Number of rows: ", len(df))

Next, let’s print the first five rows of data:

print(df.head())

We can see that there are several categorical columns. Let’s define a function that takes as input a data frame, column name, and limit. When called, it prints a dictionary of categorical values and how frequently they appear:

from collections import Counter

def return_counter(data_frame, column_name, limit):

print(dict(Counter(data_frame[column_name]

.values).most_common(limit)))

Let’s apply our function to the ‘brand’ column and limit our results to the five most common values:

return_counter(df, 'brand', 5)

As we can see, we have 1,235 Fords, 432 Dodges , 312 Nissans, 297 Chevrolets, and 42 GMCs.

Let’s apply our function to the ‘color’ column:

return_counter(df, 'color', 5)

Now, let’s look at the brands of white cars :

df_d1 = df[df['color'] =='white']

print(set(df_d1['brand']))

We can also look at the most common brands for white cars:

print(dict(Counter(df_d1['brand']).most_common(5)))

We see that most of the white cars are Fords, Dodges, and Chevrolets.

We can also look at the most common states where white cars are being sold:

print(dict(Counter(df_d1['state']).most_common(5)))

Next, it would be useful to generate summary statistics from numerical columns like ‘duration’. Let’s define a function that takes a data frame, a categorical column, and a numerical column. The mean and standard deviation of the numerical column for each category is stored in a data frame and the data frame is sorted in descending order according to the mean. This is useful if you want to quickly see if certain categories have higher or lower mean and/or standard deviation values for a particular numerical column.

def return_statistics(data_frame, categorical_column, numerical_column):

mean = []

std = []

field = []

for i in set(list(data_frame[categorical_column].values)):

new_data = data_frame[data_frame[categorical_column] == i]

field.append(i)

mean.append(new_data[numerical_column].mean())

std.append(new_data[numerical_column].std())

df = pd.DataFrame({'{}'.format(categorical_column): field, 'mean {}'.format(numerical_column): mean, 'std in {}'.format(numerical_column): std})

df.sort_values('mean {}'.format(numerical_column), inplace = True, ascending = False)

df.dropna(inplace = True)

return df

Let’s call our function with categorical column ‘brand’ and numerical column ‘price’:

stats = return_statistics(df, 'brand', 'price')

print(stats.head(15))

Next, we will use boxplots to visualize the distribution in numeric values based on the minimum, maximum, median, first quartile, and third quartile.

Similar to the summary statistics function, this function takes a data frame, categorical column, and numerical column and displays boxplots for the most common categories based on the limit:

import matplotlib.pyplot as plt

def get_boxplot_of_categories(data_frame, categorical_column, numerical_column, limit):

import seaborn as sns

from collections import Counter

keys = []

for i in dict(Counter(df[categorical_column].values).most_common(limit)):

keys.append(i)

print(keys)

df_new = df[df[categorical_column].isin(keys)]

sns.set()

sns.boxplot(x = df_new[categorical_column], y = df_new[numerical_column])

plt.show()

Let’s generate boxplots for ‘price’ in the 5 most commonly occurring ‘brand’ categories:

get_boxplot_of_categories(df, 'listed_in', 'duration', 5)

Finally, let’s define a function that takes a data frame and a numerical column as input and displays a histogram:

def get_histogram(data_frame, numerical_column):

df_new = data_frame

df_new[numerical_column].hist(bins=100)

plt.title('{} histogram'.format(numerical_column))

plt.show()

Let’s call the function with the data frame and generate a histogram from ‘price’:

get_histogram(df, 'price')

I will stop here but please feel free to play around with the data and code yourself.

CONCLUSIONS

To summarize, in this post we went over several methods for analyzing the US Cars Dataset. This included defining functions for generating summary statistics like the mean, standard deviation, and counts for categorical values. We also defined functions for visualizing data with boxplots and histograms. I hope this post was useful/interesting. The code from this post is available on GitHub. Thank you for reading!