Fast EDA in Jupyter & Colab notebooks using Sweetviz 2.0
By: Francois Bertrand
Tags: Colab, Sweetviz, Jupyter, Eda, Pandas

Fast EDA in Jupyter & Colab notebooks using Sweetviz 2.0

Image by author

Sweetviz is an open-source Python library that generates beautiful, high-density visualizations to kickstart EDA (Exploratory Data Analysis) with just two lines of code!

In addition to detailed feature analysis, it stands out from other EDA packages by providing:

Target analysis: shows how a target value (e.g. “Survived” in the Titanic dataset) relates to other features Dataset comparisons: e.g. “Train vs Test” and intra-set “Male vs Female” Correlation/associations: full integration of numerical and categorical data correlations and associations, all in one graph and table

For an in-depth look at its insights & features, check out this Medium article.

Sweetviz can now be integrated in notebooks

New with version 2.0 is that these visualizations can now be embedded in Colab/Jupyter-style notebooks (in addition to the original standalone HTML reports). This is a long-requested feature that improve its usefulness and usability for many users.

This article will go over the details of this integration, as well as the other main new features of the package: report scaling and vertical layout.

👀 Companion notebook & documentation

Note that all code in this article, as well as extensive usage notes, are contained in an example Colab notebook that can be found here.

📡 Installing & importing into a notebook

Just like from a regular IDE, unless sweetviz is already present in the environment, it must be installed. This is carried out using pip with the line !pip install sweetviz . Then simply import it as usual using something like import sweetviz as sv .

📰 Generating an integrated report

The major new feature of sweetviz 2.0 is that reports can now be embedded inside any notebook, instead of being restricted to an external HTML file.

To generate an integrated notebook once a report object has been created, use the show_notebook() command instead of the previous show_html() . Please note that show_html() is still available, and has been upgraded as we will see below.

Width, height & scale parameters

Since any report must be made to fit inside an existing notebook, and each person has their own preferences and hardware setups, show_notebook() has multiple options to both scale the report and the window through which it is displayed.

Note that the size of the window (given by the w and h parameters) is independent of the size of the report itself (given by the scale parameter), so scrollbars will be present if the window is not as large as the report. This is generally the case since reports are mostly much taller than a single window.

The parameters to show_notebook() are thus:

w: this is the width of the window through which the report is displayed. It can either be in pixels w=900 or as a percentage of the width of the notebook w="100%" .

this is the width of the through which the report is displayed. It can either be in pixels or as a percentage of the width of the notebook . h: this is the height of the window through which the report is displayed. It can also be in pixels, or h="FULL" , to stretch the window to the full tallness of the report. Note that the “full” option is often clamped by the notebook software and may not work.

this is the height of the window through which the report is displayed. It can also be in pixels, or , to stretch the window to the full tallness of the report. Note that the “full” option is often clamped by the notebook software and may not work. scale: this parameter drives the scale of the report itself. It is given using a fractional number, e.g. scale=0.8 . Note that this new parameter is also available with show_html() for HTML reports .

this parameter drives the scale of the report itself. It is given using a fractional number, e.g. . Note that this new parameter is with for . filepath: this optional parameter will cause an HTML report to also be generated in the current working folder. However, contrary to show_html() , a browser will not pop up to display it immediately.

Note that if these parameters are not present, the defaults will be used (more on these below).

# Some examples!

comparison_report.show_notebook()

comparison_report.show_notebook(w=1500, h=300, scale=0.8)

comparison_report.show_html(scale=0.9)

The “Layout” parameter: going vertical!

Two variations of the expanded detail information for the target variable “Survived”: in the Widescreen and Vertical layouts. Image by author.

Another significant customization feature is the addition of a new “vertical” layout. The previous default was “widescreen” which took a lot of horizontal real estate, even with the new “scale” parameter. This was especially an issue in notebooks, which often have less horizontal room to work with.

To remedy this, an additional layout parameter was added both to show_notebook() and show_html() . To use it, simply add the parameter layout='vertical' or layout='widescreen' to either command.

🧬Specifying your own default presentation: .ini override files

Once you find a scale and layout that work for you, it is easy to set up the defaults for your future notebooks. All default parameters are contained in the sweetviz_defaults.ini file that comes with the installation of the package. Although editing that file itself is possible, it is a bit more tricky and gets overwritten every time the package is reinstalled, which happens more often when working with some notebook environments in the cloud.

An easier way is to set up one (or multiple) “override” INI files and load up the relevant one in the beginning of your notebook, right after your imports. To load an override INI, simply activate it using the following:

sv.config_parser.read("My_Override.ini")

From the sweetviz_defaults.ini file, the most useful possible values to override are:

[General]

; Set this to 1 to support Chinese, Japanese, Korean characters

use_cjk_font = 0



[Output_Defaults]

html_layout = widescreen

html_scale = 1.0

notebook_layout = vertical

notebook_scale = 1.0

notebook_width = 100%%

notebook_height = 750



[Layout]

show_logo = 1



[Type_Detection]

; Numeric columns will be considered CATEGORICAL if fewer than this many distinct

max_numeric_distinct_to_be_categorical = 10

; Text columns will be considered TEXT if more than this many distinct (CATEGORICAL otherwise)

max_text_distinct_to_be_categorical = 101

; Text columns will be considered TEXT if more than this fraction are distinct

max_text_fraction_distinct_to_be_categorical = 0.33

Note the [Output_Defaults] section containing the default layout, scaling and positioning for the HTML and notebook reports.

INI override notes & example

It is best to load overrides before any other command, as many of the INI options are used in the report generation.

Values in the INI are separated with a header (e.g. [General] ). Always set the header before the value, otherwise there will be an error.

). set the header before the value, otherwise there will be an error. Note the double ‘%’ if specifying a percentage string (see notebook_width ).

The override INI does not need to contain every value, and could be something like:

[General]

use_cjk_font = 1



[Output_Defaults]

notebook_layout = widescreen

notebook_scale = 0.8

Conclusion

Through notebook integration, multiple layouts and report scaling, sweetviz can now generate insightful reports that can be used in a much wider variety of environments. As I mentioned in the beginning, for an in-depth look at the various analysis possible with the package, you can find a Medium article here. I hope you find it as useful as I do!