Practical Pandas: Synthetic Store Sales Analysis | by Soner Yıldırım | Towards Data Science

I’m skipping the part where I mention the great features and dominance of Pandas in the field of data science.
This post is more like a practical guide that demonstrates how Pandas can be used in data analysis. I will also try to provide a semi-structured approach to a data analysis task.
I call it “synthetic” because the data will be created randomly. Let’s start with creating our store sales dataset.
import numpy as npimport pandas as pdstores = pd.Series(['A','B','C','D']*125).sample(500)cities = pd.Series(['Rome','Madrid','Houston']*200).sample(500). reset_index(drop = True)
We have created two random series of stores and cities using the sample function of Pandas. The sample size must be equal to or less than the size of the original series. That is the reason we extended the lists by multiplying with constants.
It is time to put them together in a dataframe along with dates and sales.
df = pd.DataFrame({'day':np.random.randint(1,30, size=500),    'month':np.random.randint(1,12, size=500),'year':2020,'store':stores,'city':cities,'product1':np.random.randint(50, size=500),'product2':np.random.randint(40, size=500),'product3':np.random.randint(30, size=500)})df.head()
The sale quantities of 3 products are listed based on store-city-date combinations. It is better to also have a “date” column by combining separate day, month, and year. We will locate the new “date” column right after the “year” column which can be done with the insert function.
df.insert(3, 'date', pd.to_datetime(df[['day','month','year']]))df.head()
The first parameter of the index function indicates the location of the new column.
Note: Since we randomly generate the data, it can result in a date of February 30th which does not exist. In that case, the to_datetime function will raise an error saying that “day is out of bound for the month”. You can either limit your days at 29 or re-run the random functions.
We now have our synthetic data. One way to start the analysis is to check if there is any seasonality in sales quantities. We can use the groupby function to group sales by month and then plot the result.
df[['month','product1','product2','product3']].groupby('month')\.mean().plot(figsize=(10,6), fontsize=12, title='Monthly Sales of Products')
The product 1 and 2 seem to have a seasonality whereas product 3 is more stable. We have checked the sales in terms of monthly averages.
The sale quantities at different store-city combinations may provide valuable insight which can be achieved by the groupby function as well. Instead of plotting the results, we will use the Style property of pandas to make the results more appealing than plain numbers.
df[['store','city','product1','product2','product3']]\.groupby(['store','city'])\.mean().style.highlight_max(color='lightgreen', axis=0)\.highlight_min(color='orange', axis=0)
In addition to store-city average sales, the minimum and maximum values in each column are highlighted. It is definitely more appealing to use in reports than plain numbers.
There are other styling options available. Let’s do another one.
df[['store','city','product1','product2','product3']]\.groupby(['store','city'])\.mean().style.bar(color='lightgreen')
The sizes of the bars are proportional to the values in the column.
The original dataset had “day”, “month”, and “year” as separate columns. We combined them into a “date” column with an appropriate data type.
In a typical analysis, we are likely to have lots of columns so it is a good practice to eliminate redundant or repetitive ones. Thus, we can drop “day”, “month”, and “year” columns since the data provided by them is store in the “date” column.
df.drop(['day','month','year'], axis=1, inplace=True)df.head()
We have used the “month” column in our analysis. We can still use the individual parts of the dates by using the dt accessor.
df['date'].dt.month[:5]0     8 1    11 2     8 3     4 4    11
In the beginning, we checked the monthly averages of sales quantities. The monthly averages may not be detailed enough in some cases. Pandas is highly flexible in terms of selecting the frequency.
The following code will plot 10-day sales averages of product1.
df[['date','product1']].groupby('date').mean()\.resample('10D').mean().plot(figsize=(10,6), fontsize=12,title="10-Day Average Sales")
We first grouped the sales by date which gives us the daily averages. Then resample function is used to downsample dates to 10-day periods. The mean applied to take the average of 10-day periods. Finally, the results are plotted.
We can also create density plots with Pandas. For instance, we can use the kde function to plot density distributions of the sale quantities. KDE (kernel density estimation) is a non-parametric way to estimate the probability density function (PDF) of a random variable.
subset = df[['store','city','product1','product2','product3']]subset.plot.kde(figsize=(12,6), alpha=1, fontsize=12)
Please note that KDE is an estimation, not the actual probability distribution. This is the reason why we see negative values in the plots.
Let’s get our analysis a little more specific. For instance, you may want to see the largest increase or decrease between the average sales in two consecutive days. As an example, we will check in store A.
df[df.store == 'A'].groupby('date').mean()
These are the daily averages of sales quantities in store A. The diff function returns the differences between two consecutive rows.
df[df.store == 'A']\.groupby('date').mean().diff().sort_values(by='product1')[:5]
We have sorted the values in ascending order by product1 to see the largest decreases in the sale quantities of product1. The sale quantity of product1 was decreased by 37 on 2020–10–28 compared to the previous day.
Similarly, we can find the largest increases in product2.
df[df.store == 'A'].groupby('date').mean().diff().\sort_values(by='product2', ascending=False)[:5]
We have covered some techniques that can be used in a typical data analysis task. Pandas' capabilities extend far beyond the techniques discussed here. You will discover more as you need to complete the tasks.
Thank you for reading. Please let me know if you have any feedback.