Intelligent Visual Data Discovery with Lux — A Python library
By: Parul Pandey
Tags: Exploratory Data Analysis, Python, Data Visualization, Data Analysis, Lux

EDA with Lux: Supporting a Visual dataframe workflow

Image from the presentation with permission from the author

df

When we print out the data frame, we see the default pandas table display. We can toggle it to get a set of recommendations generated automatically by Lux.

Image by Author

The recommendations in lux are organized by three different tabs, which represent potential next steps that users can take in their exploration.

The Correlation Tab: shows a set of pairwise relationships between quantitative attributes ranked by the most correlated to the least correlated one.

Image by Author

We can see that the penguin flipper length and body mass show a positive correlation. Penguins’ culmen length and depth also show some interesting patterns, and it appears that there is a negative correlation. To be specific, the culmen is the upper ridge of a bird’s bill.

Image by Author

The Distribution Tab shows a set of univariate distributions ranked by the most skewed to the least skewed.

Image by Author

The Occurrence Tab shows a set of bar charts that can be generated from the data set.

Image by Author

This tab shows there are three different species of penguins — Adelie, Chinstrap, and Gentoo. There are also three different islands — Torgersen, Biscoe, and Dream; and both male and female species have been included in the dataset.

Intent-based recommendations

Beyond the basic recommendations, we can also specify our analysis intent. Let's say that we want to find out how the culmen length varies with the species. We can set the intent here as [‘culmen_length_mm’,’species’]. When we print out the data frame again, we can see that the recommendations are steered to what is relevant to the intent that we’ve specified.

df.intent = ['culmen_length_mm','species']

df

On the left-hand side in the image below, what we see is Current Visualization corresponding to the attributes that we have selected. On the right-hand side, we have Enhance i.e. what happens when we add an attribute to the current selection. We also have the Filter tab which adds filter while fixing the selected variable.

Image by Author

If you closely look at the correlations within species, culmen length and depth are positively correlated. This is a classic example of Simpson’s paradox.

Image by Author

Finally, you can get a pretty clear separation between all three species by looking at flipper length versus culmen length.

Image by Author

Exporting visualizations from Widget

Lux also makes it pretty easy to export and share the generated visualizations. The visualizations can be exported into a static HTML as follows:

df.save_as_html('file.html')

We can also access the set of recommendations generated for the data frames via the properties recommendation. The output is a dictionary, keyed by the name of the recommendation category.

df.recommendation

Image by Author

Exporting Visualizations as Code

Not only can we export visualization as HTML but also as code. The GIF below shows how you can view the first bar chart's code in the Occurrence tab. The visualizations can then be exported to code in Altair for further edits or as Vega-Lite specification. More details can be found in the documentation.