Python Classes for Statistics with Pandas | by Sadrach Pierre, Ph.D. | Towards Data Science

In computer programming, a class is a blueprint for a user-defined data type. Classes are defined in terms of attributes (data) and methods (functions). These data structures are a great way to organize data and methods such that they are easy to reuse and extend in the future. In this post, we will define a python class that will allow us to generate simple summary statistics and perform some EDA on data.
Let’s get started!
For our purposes we will be working with the FIFA 19 data set which can be found here.
To start, let’s import the pandas package:
import pandas as pd
Next, let’s set the maximum number of display columns and rows to ‘None’:
pd.set_option('display.max_columns', None)pd.set_option('display.max_rows', None)
Now, let’s read in our data:
df = pd.read_csv('fifa_data.csv')
Next we will print the first five rows of data to get an idea of the column types and their values (column results are truncated):
print(df.head())
To start, let’s define a simple class called ‘Summary’ that reads in the data and displays the firsts five rows upon initialization:
class Summary:    def __init__(self, data):        pd.set_option('display.max_columns', None)        pd.set_option('display.max_rows', None)        self.df = pd.read_csv(data)        print(self.df.head())
If we call an instance of our Summary class with data = ‘fifa_data.csv’ we get:
Summary('fifa_data.csv')
Let’s modify this class such that we have a method that we can call to display the first five rows:
class Summary:    def __init__(self, data):        pd.set_option('display.max_columns', None)        pd.set_option('display.max_rows', None)        self.df = pd.read_csv(data)    def print_head(self):        print(self.df.head())
Now if we define an instance, we can call the the ‘print_head’ method on our instance and it will display the first five rows:
data = Summary('fifa_data.csv')data.print_head()
We can also modify our ‘print_head’ method so that it can print more than the default five rows.
class Summary:    def __init__(self, data):        pd.set_option('display.max_columns', None)        pd.set_option('display.max_rows', None)        self.df = pd.read_csv(data)    def print_head(self, rows):        print(self.df.head(rows))
Now if we call print head with ‘rows’=10 get (I truncated the number of columns shown:):
data = Summary('fifa_data.csv')data.print_head(10)
Let’s also define a function that gets the column names for the data:
class Summary:    ...    def get_columns(self):        print(list(self.df.columns))
And if we call ‘get_columns’ we get:
data.get_columns()
We can also define a method that prints the number of columns and rows:
class Summary:    ...    def get_dim(self):        print('Rows:', len(self.df))        print('Columns:', len(list(self.df.columns)))
And if we call ‘get_dim’ on ‘data’:
data.get_dim()
We see that our data has 18,207 rows and 88 columns. Now let’s define some functions that can give us some statistical insights. The Pandas data frame has an ‘describe()’ method that gives us some basic statistical info. This includes count, mean, standard deviation, min, max, and quartiles:
class Summary:    ...    def get_stats(self):        print(self.df.describe())
Now let’s call ‘get_stats’. Below the stats for ‘Special’, ‘International Reputation’, ‘Weak Foot’, ‘Skill Moves’ are shown:
data. get_stats()
Since the data contains 88 columns it can be a bit daunting to inspect the summary statistics for all of these columns at once. We can define a function that calculates the mean for a specific column:
class Summary:    ...    def get_mean(self, column):        print(f"Mean {column}:", self.df[column].mean())
Let’s call this method on the ‘Age’ column:
data.get_mean('Age')
We can do the same for the standard deviation:
class Summary:    ...    def get_standard_dev(self, column):        print(f"STD {column}:", self.df[column].std())...data.get_standard_dev('Age')
For categorical columns we can define a method that displays the distribution in categorical values, using the ‘Counter’ method from the collections module. Let’s apply this to the ‘Nationality’ column:
...from collections import Counterclass Summary:    ...    def get_counter(self, column):        print(dict(Counter(self.df[column])))...data.get_counter('Nationality')
We can define a function that displays a histogram for a specific numerical column. Let’s apply this to the ‘Age’ column:
...import matplotlib.pyplot as pltimport seaborn as snsclass Summary:    ...    def get_hist(self, column):        sns.set()        self.df[column].hist(bins=100)        plt.title(f"{column} Histogram")        plt.show()...data.get_hist('Age')
Next, we can use boxplots to visualize the distribution in numeric values across different categorical values in a categorical column:
class Summary:    ...    def get_boxplot_of_categories(self, categorical_column, numerical_column, limit):        keys = []        for i in dict(Counter(self.df[categorical_column].values).most_common(limit)):            keys.append(i)        df_new = self.df[self.df[categorical_column].isin(keys)]        sns.set()        sns.boxplot(x = df_new[categorical_column], y = df_new[numerical_column])plt.show()
Let’s generate boxplots for ‘Age’ in the 5 most commonly occurring Nationalities :
data.get_boxplot_of_categories(df, 'Nationality', 'Age', 5)
Finally, we can define a function that displays a heat map for a specific numerical column:
class Summary:    ...    def get_heatmap(self, columns):        df_new = self.df[columns]        sns.heatmap(df_new.corr())        plt.title(f"Heatmap of {columns}")
Let’s apply this to ‘Age’, ‘Overall’, ‘Potential’, ‘Special’:
data.get_heatmap(['Age', 'Overall', 'Potential', 'Special'])
The nice thing about this class is that it is a general blueprint that can be reapplied to any ‘csv’ file. All you would need to change is the initiation of the instance with the file name:
data = Summary('new_file.csv')
Upon using ‘get_head’ and ‘get_column’ you will learn the column names and types and you will be able to start applying the statistics and visualization methods we used in this post. I’ll stop here but I encourage you to play around with the data and code yourself.
To summarize, in this post we showed how to use python classes to generate simple summary statistics using the Pandas library. First we showed basic operations like how to get the first set of rows, column and row lengths, and simple statistics using the describe method. These methods allow you to quickly get a rough understanding of the data types and their values. Further we built custom statistic functions that allow us to analyze specific columns. This is useful when we have too many columns to analyze quickly. We also showed some methods for visualizing the distribution in numerical values in our data which gives us insights into which values fall close to the mean and which are outliers. Finally, we discussed how to define a method that allows you to generate a heat-map of correlation values. This helps us visualize how variables depend on each other. This can be very useful for feature engineering and model building in machine learning. I hope you found this post useful/interesting. The code in this post is available on Github. Thank you for reading!