Speed up EDA With the Intelligent Lux
By: Pranavi Duvva
Tags: Data Science, Python, Jupyter Notebook, Exploratory Data Analysis, Data Visualization

Case Study

Let’s consider an example dataset to explore the features of the Lux library.

I would be using the Graduate Admission dataset taken from the Kaggle data repository.

This dataset contains several parameters which are considered important during the application for Masters Programs.

Data Dictionary

GRE Scores ( out of 340 ) TOEFL Scores ( out of 120 ) University Rating ( out of 5 ) Statement of Purpose and Letter of Recommendation Strength ( out of 5 ) Undergraduate GPA ( out of 10 ) Research Experience ( either 0 or 1 ) Chance of Admit ( ranging from 0 to 1 )

1. Importing all the necessary libraries

Now that the package has been successfully installed. We just have to import the lux library into our jupyter notebook.

2. Loading the data set and checking the concise summary

Let’s load the dataset and check the top 5 rows.

Source: Image by Author

Checking the shape of the data set.

(400, 9)

There are a total of 400 rows and 9 columns.

Removing the first column Serial No. and checking the concise summary of the data set with the info()

Source: Image by Author

We observe that the data type of all the 8 columns in the dataset is numeric.

3. Visual Data Exploration with Lux 💡

Let’s now display the data frame and explore the Lux widget.

Source: Image by Author

When the data frame is displayed Lux by default provides us with 3 tabs which are Correlation, Distribution, and Occurrence.

Let’s get to know each of these

Correlation

Source: Image by Author

The correlation tab displays the relationship between the quantitative variables present in the dataset.

The order in which it displays is the most correlated ones to the least correlated ones.

Source: Image by Author

2. Distribution

Source: Image by Author

The distribution tab displays the histograms of the quantitative variables in the dataset.

The order in which it displays is the highly skewed ones to the least skewed ones.

Source: Image by Author

3. Occurrence

Source: Image by Author

The occurrence tab displays the bar charts of the categorical attributes.

The order it follows is the most uneven distribution to the even distribution.

Although our dataset did not contain any features with the categorical data type. It did recommend bar charts for those features which it thinks might be useful for our analysis.

4. Visualizations and Recommendations based on user intent.

Let’s say you want to know more about a specific feature or multiple features together. You can get all the visualizations related to those attributes with the help of intent

The lux widget not only displays the visualization for that feature intended. But will also provide you with additional recommendations for further analysis with the help of Filter and Enhance options.

Enhance

The Enhance feature of lux adds an additional attribute to the intended attributes specified by the user for visualization.

It lets the user compare the effect of the added attribute to the intended visualization. This is similar to adding a hue.

2. Filter

The filter lets the user visualize the intended attributes for different subsets of the data.

Let’s understand better with the following examples.

Consider one attribute CGPA,

df.intent=[“CGPA”]

df

1.Enhance Recommendations for one attribute

Enhance Tab recommendations when the intended attribute is CGPA, Source: Image by Author

The Enhance tab when the given input is one feature “CGPA” fixes the intended variable “CGPA” on the x-axis and gives us recommendations by comparing it with different attributes.

2. Filter Recommendations for one attribute

Filter Tab recommendations when the intended attribute is CGPA, Source: Image by Author

The Filter tab fixes the intended variable “CGPA” on the x-axis and gives us recommendations by comparing it with different subparts of the data set.

Consider two attributes “TOEFL Score” and “GRE Score”,

df.intent=[“TOEFL Score”,”GRE Score”]

df

1.Enhance Recommendations for two attributes

Enhance Tab recommendations when the intended attributes are TOEFEL Score and GRE Score, Source: Image by Author

The Enhance tab when the given input is two attributes “TOEFL Score”, “GRE Score”. It fixes the intended variables “TOEFL Score” on the x-axis and the “GRE Score” on the y-axis. It then gives us recommendations by comparing with different attributes.

2. Filter Recommendations for two attributes

Filter Tab recommendations when the intended attributes are TOEFEL Score and GRE Score, Source: Image by Author

The Filter tab when the given input is two attributes “TOEFL Score”, “GRE Score”. It fixes the intended variable “TOEFL Score” on the x-axis and the “GRE Score” on the y-axis. It then gives us recommendations by comparing both together with different subparts of the data.

Source: Image by Author

5. Exporting Visualizations.

Lux makes it very easy to share the visualizations. To export visualizations into a static HTML the following command has to be used.