Pywedge: A complete package for EDA, Data Preprocessing and Modelling
By: Himanshu Sharma
Tags: Machine Learning, Data Science, Python, Data Analysis, Data Visualization

Pywedge: A complete package for EDA, Data Preprocessing and Modelling

Pywedge helps in visualizing the data, preprocessing, and creating baseline models

Pywedge(Source: By Author)

What is Pywedge?

Pywedge is an open-source python library which is a complete package that helps you in Visualizing the data, Pre-process the data and also create some baseline models which can be further tuned to make the best machine learning model for the data.

It is a hassle-free package that saves the time and effort of creating different types of plots to analyze the data. It contains 8 different types of visualization which can be used using the user-friendly GUI of pywedge. It takes care of all the pre-processing that the data may require whether it is cleaning the data, normalization, or handling the class imbalance pywedge covers it all. It works on both classification and regression problems.

In this article, we will see what all we can do using Pywedge and explore its features.

Before starting kindly follow me on medium by clicking here and stay updated about my new articles in the field of data science.

Installing pywedge

Pywedge is pip installable so we will install it using the command prompt by running the below-given command.

pip install pywedge

Importing the required libraries

We are exploring pywedge so we will start by importing it. Also, we will import pandas for loading our dataset.

import pandas as pd

import pywedge as pw

Loading the Dataset

We will use different datasets for exploring pywedge. Let’s start by importing the Iris dataset from seaborn for the visualization part.

import seaborn as sns

df = sns.load_dataset("iris")

df

Iris Dataset(Source: By Author)

Visualization using Pywedge

Now we will use pywedge functions to visualize the dataset. Here we will use the pywedge_charts function which requires three inputs namely the data, “C” which is a redundant column in the dataset that we want to remove, and the dependant variable according to the problem i.e classification or regression.

mc = pw.Pywedge_Charts(df, c=None, y="species" )

# For Visualization

chart = mc.make_charts()

Visualization(Source: By Author)

In the output above we can clearly see visualize the interface which is created by pywedge. We can select different types of charts and use different variables to visualize the data.

Preprocessing the data

Next, we will see how easily we can preprocess the data using pywedge. For this, I have used the Boston dataset which I have split into the test and train dataset. We will see how pywedge works on finding missing values, normalization, class imbalance(for classification problem).

#Loading the dataset

train = pd.read_csv('bos_test.csv')

test = pd.read_csv('bos_train.csv') #We are handling regression so we will use type as Regression

ppd = pw.Pre_process_data(train, test, c=None, y='medv', type='Regression') #Storing the processed data into new dataframe

new_X, new_y, new_test = ppd.dataframe_clean()

Preprocessing(Source: By Author)

In the output above we can clearly see how pywedge identifies missing data and provides options to the users(highlighted in image) for standardization and converting categorical to numerical. Similarly, if we will solve a classification problem it will show whether the data is imbalanced or not.

Baseline Models

Pywedge also creates some baseline models according to our dataset, so that we can select the best performing model and tune it to reach the maximum accuracy. It also helps us in finding out the importance of features.

blm = pw.baseline_model(new_X,new_y) #We will use regression_summary, for classification we will use #classification_summary

blm.Regression_summary()

Baseline Models(Source: By Author)

Here we can clearly analyze the different types of models and their performance, we can select the best performing model and further tune it.

Go ahead try this interesting and useful library on different datasets and in case you face any difficulty reach out to me in the response section.

This post is in collaboration with Piyush Ingale

Before You Go

Thanks for reading! If you want to get in touch with me, feel free to reach me on hmix13@gmail.com or my LinkedIn Profile. You can view my Github profile for different data science projects and packages tutorials. Also, feel free to explore my profile and read different articles I have written related to Data Science.