Repository: hpi-bp1819-naumann/shift-detector

Issue:
Simplecheck Numerical Metrics
I think the metric `num_distinct` should not be used for numerical values as it does make any sense if they are not categorical.

However this means that the metric would be useful for categorical columns.
The metric `completeness` could be used for categorical columns, too.

This method in `SimpleCheck` does not do what it should for `uniqueness`, `completeness`.
If `completeness` is 0 in df1 and 1 in df2 it will return 0 and not the difference.

```python
    def relative_metric_difference(self, column, metric_name):
        metric_in_df1 = self.data['numerical_comparison'][column][metric_name]['df1']
        metric_in_df2 = self.data['numerical_comparison'][column][metric_name]['df2']

        if metric_in_df1 == 0 and metric_in_df2 == 0:
            return 0
        if metric_in_df1 == 0:
            logger.warning('column {} \t \t {}: no comparison of distance possible, division by zero'
                           .format(column, metric_name))
            return 0

        relative_difference = (metric_in_df2 / metric_in_df1 - 1)
        if metric_name in ['uniqueness', 'completeness', 'completeness']:
            relative_difference = metric_in_df2 - metric_in_df1

        return relative_difference
```

In addition, why is there a duplication in
```python
if metric_name in ['uniqueness', 'completeness', 'completeness']:
```
?

Discussion:
pep8speaks: Hello @PaulZGit! Thanks for opening this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:

* In the file [`shift_detector/checks/dq_metrics_check.py`](https://github.com/hpi-bp1819-naumann/shift-detector/blob/7957605ded9be50443ddfd7fe7d061d44131f80f/shift_detector/checks/dq_metrics_check.py):

> [Line 102:121](https://github.com/hpi-bp1819-naumann/shift-detector/blob/7957605ded9be50443ddfd7fe7d061d44131f80f/shift_detector/checks/dq_metrics_check.py#L102): [E501](https://duckduckgo.com/?q=pep8%20E501) line too long (139 > 120 characters)













Patch:
diff --git a/docs/source/checks/dq_metrics.rst b/docs/source/checks/dq_metrics.rst
index 0e368df..3bf66c7 100644
--- a/docs/source/checks/dq_metrics.rst
+++ b/docs/source/checks/dq_metrics.rst
@@ -7,14 +7,14 @@ Description
 -----------
 
 The :ref:`dq_metrics` gives you basic information about your datasets and detects the most obvious of datashifts. The
-'dq' stands for data quality, the metrics are in the style of the 'Unit Test for data'-framework Amazon Deequ.
+'DQ' stands for data quality, the metrics are in the style of the 'Unit tests for data'-libreary Deequ by amazon.
 For more information on Deequ see [SLSCB18]_.
 
-Some shifts result in the change of basic metrics, that are commonly used for the analysis and exploration of datasets.
-Those metrics and statistics are calculated on both datasets and compared afterwards.
+Some data shifts result in the change of basic metrics, that are commonly used for the analysis and exploration of
+datasets. This check calculates those metrics and statistics on both datasets and compares them.
 
-The check should be used in most analyses, its metrics are commonly used to describe datasets.
-It works directly on categorical and numerical data and, with additional Precalculations, also on text data.
+The :ref:`dq_metrics` works directly on categorical and numerical data and also optionally on
+text metadata.
 
 
 Example
@@ -54,21 +54,26 @@ The :ref:`dq_metrics` produces this report:
 
 ::
 
-    Numerical Columns:
+    Numerical Columns
 
-    Column 'payment':
-    Metric: mean with Diff: -29.4 %, threshold: 20%
-    Metric: std with Diff: +80.2 %, threshold: 50%
+    Column 'amount':
+        Metric  Val in DS1  Val in DS2   Threshold     Relative Diff
+    0   mean    1000.0      706.0        +/- 20.0 %    -29.4 %
+    1   std     20          36.04        +/- 50.0 %    +80.02 %
 
+    Column 'coord_x':
     (...)
 
 
+
     Categorical Columns
 
-    Column 'payment_option':
-    Attribute: 'debit' with Diff: +6.0 %, categorical threshold: 5%
-    Attribute: 'cash' with Diff: -6.5 %, categorical threshold: 5%
+    Attribute 'payment_option':
+        Attribute Value   Val in DS1    Val in DS2    Threshold   Relative Diff
+    0   'debit'           3.0           9.0           +6.0 %      +/- 5.0 %
+    1   'cash'            13.0          6.5           -6.5 %      +/- 5.0 %
 
+    Attribute 'customer_status':
     (...)
 
 
@@ -76,37 +81,51 @@ The :ref:`dq_metrics` produces this report:
 Interpretation
 ++++++++++++++
 
-The results show column-wise wich shifts appear in your dataset. First the reports of your numerical columns are shown,
-after that the reports of your categorical columns.
+The results show column-wise wich shifts appear in your dataset. First, the reports of all shifted numerical columns
+are shown, after that follow the reports of all shifted categorical columns.
 
 Numerical
 ~~~~~~~~~
 
-In the example, below the heading 'numerical columns', you see that the column with the name 'payment' is detected
-as shifted. Below the columnname, you can see that two numerical metrics detected a shift, *mean* and *std*
+In the example, below the headline 'numerical columns', you see that the column with the name 'amount' is detected
+as shifted. Below the column name, you can see that two numerical metrics detected a shift, *mean* and *std*
 (standard deviation).
 
-*Diff* describes the relative value-distance of the metrics that are calculated on your two datasets. If the Diff
+*Relative Diff* describes the relative value-distance of the metrics that are calculated on your two datasets, so the
+relative difference between the value of dataset1 minus the value of the dataset2. If this Relative Diff
 exceeds a metric-specific threshold, the dataset differ that much, that the check calls shift on this column.
 
-In this case, the mean-value of 'payment' could be 1000.0 in your first, but 706.0 in your second dataset. This difference is
-results in an difference of -294 and an relative difference of -29.4%. The threshold set here is 20% which is exceeded
-by the absolute value of -29.4%. All thresholds are customizable through the API of the
-:class:`~Morpheus.checks.dq_metrics_check.DQMetricsCheck`. All the metrics are listed in the section
+In this case, the mean-value of 'amount' is 1000.0 in your first and 706.0 in the second dataset. This
+results in a difference of -294 and an relative difference of -29.4%. The threshold set here is 20% which is exceeded
+by the absolute amount of -29.4%. All thresholds are customizable through the API of the
+:class:`~Morpheus.checks.dq_metrics_check.DQMetricsCheck`. The used metrics are listed in the section
 :ref:`dq_metrics_check_parameters`.
 
 Categorical
 ~~~~~~~~~~~
 
-Using the :ref:`dq_metrics` shifts can also be detected on categorical columns. Here the histograms over the attribute-values are
-compared. If the difference between those values exceeds the *categorical_threshold* the check calls shift on this
+Using the :ref:`dq_metrics`, shifts can also be detected on categorical columns. Most of the numerical metrics, with
+exception of *uniqueness* and *completeness* are not applicable for non-numerical data.
+
+For the categorical columns, the histograms over the attribute-values are compared. If the difference between those
+values exceeds the *categorical_threshold* the check calls shift on this
 column.
 
 The example above shows a shift in the categorical column 'payment_option'. There are different
-attribute-values in this column, i.e. 'debit and 'cash'. In those, the differences between the dataset are 6% and
+attribute-values in this column: 'debit and 'cash'. In those, the differences between the dataset are 6% and
 -6.5% which both exceed the threshold of 5%. This indicates that in dataset2 more people use
-cash as a payment-option and fewer use debit, the check calls shift on the column.
+debit as a payment option and fewer use cash, the check calls shift on the column.
+
+In addition, the metric *num_distinct*, which counts the number of distinct values in a column, is applied on
+categorical columns.
+
+
+Text Metadata
+~~~~~~~~~~~~~
 
+With the optional parameter *check_text_metadata=True*, the :ref:`dq_metrics` also uses the text metadata from the
+:ref:`text_metadata` as input parameters.
+Those metadata are then handled like other numerical or  categorical attributes in the dataset.
 
 .. _dq_metrics_check_parameters:
 
@@ -141,7 +160,8 @@ All parameters expect float values larger than 0.0. The smallest value of 0.0 me
 100% difference is recommended.
 
 The 9 threshold parameters for numerical metrics are accessable through their name *'metric_name'_threshold*,
-the threshold parameter for the categorical columns is called *categorical_threshold*.
+the threshold parameter for the categorical columns is called *categorical_threshold*. As explained above, the parameter
+*check_text_metadata* expects a boolean value. If it is set to True, the check also uses text metadata as input columns.
 
 Example
 ::
diff --git a/shift_detector/checks/dq_metrics_check.py b/shift_detector/checks/dq_metrics_check.py
index f5e1410..ccaeb9c 100644
--- a/shift_detector/checks/dq_metrics_check.py
+++ b/shift_detector/checks/dq_metrics_check.py
@@ -1,22 +1,25 @@
 import logging as logger
-from collections import defaultdict
+from collections import defaultdict, namedtuple
 
 import matplotlib.pyplot as plt
 import numpy as np
-from matplotlib import gridspec
+import pandas as pd
 
 from shift_detector.checks.check import Check, Report
 from shift_detector.precalculations.dq_metrics_precalculation import DQMetricsPrecalculation
 from shift_detector.utils.column_management import ColumnType
-from shift_detector.utils.custom_print import nprint
-from shift_detector.utils.visualization import plot_title, LEGEND_1, LEGEND_2, PLOT_GRID_WIDTH, PLOT_ROW_HEIGHT
+from shift_detector.utils.custom_print import nprint, diagram_title
+from shift_detector.utils.visualization import LEGEND_1, LEGEND_2
+
+ReportRow = namedtuple('ReportRow', 'metric_name val1 val2 threshold diff')
 
 
 class DQMetricsCheck(Check):
 
     def __init__(self, categorical_threshold=0.05, mean_threshold=0.15, median_threshold=0.15,
                  value_range_threshold=0.5, quartile_1_threshold=0.2, quartile_3_threshold=0.2,
-                 uniqueness_threshold=0.1, num_distinct_threshold=0.2, completeness_threshold=0.1, std_threshold=0.25):
+                 uniqueness_threshold=0.1, num_distinct_threshold=0.2,  std_threshold=0.25, completeness_threshold=0.1,
+                 check_text_metadata=False):
 
         threshold_names_values = {'mean': mean_threshold, 'median': median_threshold,
                                   'value_range': value_range_threshold, 'quartile_1': quartile_1_threshold,
@@ -24,85 +27,118 @@ class DQMetricsCheck(Check):
                                   'num_distinct': num_distinct_threshold, 'std': std_threshold, 'completeness':
                                   completeness_threshold}
 
-        if categorical_threshold < 0:
+        if categorical_threshold <= 0.0:
             raise ValueError("The categorical threshold of {} is not correct. It has to be larger than 0.0"
                              .format(categorical_threshold))
 
         for t_name, t_value in threshold_names_values.items():
-            if t_value < 0:
+            if t_value <= 0.0:
                 raise ValueError("The {}_threshold of {} is not correct. It has to be larger than 0.0"
                                  .format(t_name, t_value))
-
-        self.data = None
         self.categorical_threshold = categorical_threshold
         self.metrics_thresholds_percentage = threshold_names_values
 
+        self.data = None
+        self.text_metadata = None
+        self.check_language_metadata = check_text_metadata
+
     def run(self, store):
         df1_numerical, df2_numerical = store[ColumnType.numerical]
-        self.data = store[DQMetricsPrecalculation()]
-        numerical_report = self.numerical_report(df1_numerical, df2_numerical)
-        categorical_report = self.categorical_report()
+        self.data = store[DQMetricsPrecalculation(text_metadata=self.check_language_metadata)]
 
-        return numerical_report + categorical_report
+        numerical_report = self.numerical_categorical_report(df1_numerical, df2_numerical)
+        attribute_val_report = self.attribute_val_report()
 
-    def relative_metric_difference(self, column, metric_name):
-        metric_in_df1 = self.data['numerical_comparison'][column][metric_name]['df1']
-        metric_in_df2 = self.data['numerical_comparison'][column][metric_name]['df2']
+        if self.check_language_metadata:
+            metadata_report = self.text_metadata_report()
+        else:
+            metadata_report = DQMetricsReport()
 
-        if metric_in_df1 == 0 and metric_in_df2 == 0:
-            return 0
-        if metric_in_df1 == 0:
-            logger.warning("column {} \t \t {}: no comparison of distance possible, division by zero"
-                           .format(column, metric_name))
-            return 0
+        agg_examined_columns = set(numerical_report.examined_columns + attribute_val_report.examined_columns +
+                                   metadata_report.examined_columns)
+        agg_shifted_columns = set(numerical_report.shifted_columns + attribute_val_report.shifted_columns +
+                                  metadata_report.shifted_columns)
 
-        relative_difference = (metric_in_df2 / metric_in_df1 - 1)
-        if metric_name in ['uniqueness', 'completeness', 'completeness']:
+        agg_explanations = numerical_report.explanation.copy()
+        agg_explanations.update(attribute_val_report.explanation)
+        agg_explanations.update(metadata_report.explanation)
+
+        agg_figures = numerical_report.figures + attribute_val_report.figures
+
+        agg_reports = DQMetricsReport(agg_examined_columns, agg_shifted_columns, agg_explanations, {}, agg_figures)
+        return agg_reports
+
+    def relative_metric_difference(self, column, metric_name, comparison='numerical_comparison'):
+        metric_in_df1 = self.data[comparison][column][metric_name]['df1']
+        metric_in_df2 = self.data[comparison][column][metric_name]['df2']
+        relative_difference = 0
+
+        if metric_name in ['uniqueness', 'completeness']:
             relative_difference = metric_in_df2 - metric_in_df1
 
-        return relative_difference
+        elif metric_in_df1 == 0:
+            logger.warning("Column {} \t \t {}: no comparison of distance possible, division by zero"
+                           .format(column, metric_name))
+        else:
+            relative_difference = metric_in_df2 / metric_in_df1 - 1
 
-    @staticmethod
-    def difference_to_string(metrics_difference, print_plus_minus=False):
-        metrics_difference = round(metrics_difference*100, 2)
-        metrics_difference_string = str(metrics_difference) + ' %'
+        return metric_in_df1, metric_in_df2, relative_difference
 
-        if print_plus_minus:
-            metrics_difference_string = '+/- ' + metrics_difference_string
-        elif metrics_difference > 0:
-            metrics_difference_string = '+' + metrics_difference_string
+    def text_metadata_report(self):
+        metadata_comparison = self.data['metadata_comparison']
+        examined_columns = set()
+        shifted_columns = set()
+        explanation = defaultdict(list)
 
-        return metrics_difference_string
+        for column_tuple, metrics in sorted(metadata_comparison.items()):
+            examined_columns.add(column_tuple[0])
+
+            for metric_name in metrics:
+                val1, val2, diff = self.relative_metric_difference(column_tuple, metric_name,
+                                                                   comparison='metadata_comparison')
+                if abs(diff) > self.metrics_thresholds_percentage[metric_name]:
+                    shifted_columns.add(column_tuple[0])
+
+                    explanation[column_tuple[0]].append(
+                        ReportRow(metric_name + ', ' + column_tuple[1], val1, val2,
+                                  self.metrics_thresholds_percentage[metric_name], diff))
 
-    def numerical_report(self, df1, df2):
+        return DQMetricsReport(examined_columns, shifted_columns, explanation={'text_metadata': explanation},
+                               figures=[])
+
+    def numerical_categorical_report(self, df1, df2):
         numerical_comparison = self.data['numerical_comparison']
+        categorical_comparison = self.data['categorical_comparison']
+
         examined_columns = set()
         shifted_columns = set()
-        explanation = defaultdict(str)
+        explanation = defaultdict(list)
 
-        for column_name, metrics in numerical_comparison.items():
-            examined_columns.add(column_name)
+        for comparison, comparison_name in [(numerical_comparison, 'numerical_comparison'), (categorical_comparison,
+                                                                                             'categorical_comparison')]:
+            for column_name, metrics in sorted(comparison.items()):
+                examined_columns.add(column_name)
 
-            for metric in metrics:
-                diff = self.relative_metric_difference(column_name, metric)
-                if abs(diff) > self.metrics_thresholds_percentage[metric]:
-                    shifted_columns.add(column_name)
-                    explanation[column_name] += "Metric: {}, Diff: {}, threshold: {}\n".\
-                        format(metric, self.difference_to_string(diff),
-                               self.difference_to_string(self.metrics_thresholds_percentage[metric]),
-                               print_plus_minus=True)
+                for metric_name in metrics:
+                    val1, val2, diff = self.relative_metric_difference(column_name, metric_name,
+                                                                       comparison=comparison_name)
+                    if abs(diff) > self.metrics_thresholds_percentage[metric_name]:
+                        shifted_columns.add(column_name)
+
+                        explanation[column_name].append(
+                            ReportRow(metric_name, val1, val2, self.metrics_thresholds_percentage[metric_name], diff))
 
-        return DQMetricsReport(examined_columns, shifted_columns, dict(explanation),
+        return DQMetricsReport(examined_columns, shifted_columns, explanation={'numerical_categorical': explanation},
                                figures=[DQMetricsReport.numerical_plot(df1, df2)])
 
-    def categorical_report(self):
-        categorical_comparison = self.data['categorical_comparison']
+    def attribute_val_report(self):
+        attribute_val_comparison = self.data['attribute_val_comparison']
         examined_columns = set()
         shifted_columns = set()
-        explanation = defaultdict(str)
+        explanation = defaultdict(list)
         plot_infos = []
 
-        for column_name, attribute in categorical_comparison.items():
+        for column_name, attribute in sorted(attribute_val_comparison.items()):
             examined_columns.add(column_name)
 
             bar_df1 = []
@@ -110,84 +146,115 @@ class DQMetricsCheck(Check):
             attribute_names = []
 
             for attribute_name, attribute_values in attribute.items():
+                val1 = attribute_values['df1']
+                val2 = attribute_values['df2']
+                diff = val1 - val2
 
-                diff = attribute_values['df1'] - attribute_values['df2']
-
-                bar_df1.append(attribute_values['df1'])
-                bar_df2.append(attribute_values['df2'])
+                bar_df1.append(val1)
+                bar_df2.append(val2)
                 attribute_names.append(attribute_name)
 
                 if diff > self.categorical_threshold:
                     shifted_columns.add(column_name)
-                    explanation[column_name] += "Attribute: '{}' with Diff: {}, categorical threshold: {}\n"\
-                        .format(attribute_name, self.difference_to_string(diff),
-                                self.difference_to_string(self.categorical_threshold, print_plus_minus=True))
+                    explanation[column_name].append(
+                        ReportRow(attribute_name, val1, val2, self.categorical_threshold, diff))
 
             plot_infos.append((bar_df1, bar_df2, attribute_names, column_name))
 
-        return DQMetricsReport(examined_columns, shifted_columns, dict(explanation),
-                               figures=[DQMetricsReport.categorical_plot(plot_infos)])
+        return DQMetricsReport(examined_columns, shifted_columns, explanation={'attribute_val': explanation},
+                               information={}, figures=[DQMetricsReport.attribute_val_plot(plot_infos)])
 
 
 class DQMetricsReport(Report):
 
-    def __init__(self, examined_columns, shifted_columns, information={}, explanation={}, figures=[]):
-        super().__init__("DQ Metrics Check", examined_columns, shifted_columns, information, explanation, figures)
+    def __init__(self, examined_columns=set(), shifted_columns=set(), explanation=[], information={}, figures=[]):
+        super().__init__("DQ Metrics Check", examined_columns, shifted_columns, explanation, information, figures)
 
     def print_explanation(self):
-        categorical_found = False
-        nprint("Numerical columns", text_formatting='h3')
 
-        for column, explanation in self.explanation.items():
-            if not categorical_found:
-                if 'categorical' in explanation:
-                    categorical_found = True
-                    nprint("Categorical columns", text_formatting='h3')
+        for metric_name, name, sub_name, heading in \
+                [('numerical_categorical', 'Metric', 'Column', 'Numerical Columns'),
+                 ('attribute_val', 'Attribute Value', 'Attribute', 'Categorical Columns'),
+                 ('text_metadata', 'Text Metadata', 'Column', 'Text Metadata')]:
+
+            if metric_name not in self.explanation:
+                return
+
+            nprint(heading, text_formatting='h3')
+            for column_name, data in self.explanation[metric_name].items():
+                names = []
+                val1s = []
+                val2s = []
+                thresholds = []
+                diffs = []
+
+                for row in data:
+                    names.append(row.metric_name)
+                    val1s.append(round(row.val1, 2))
+                    val2s.append(round(row.val2, 2))
+                    thresholds.append(self.difference_to_string(row.threshold, print_plus_minus=True))
+                    diffs.append(self.difference_to_string(row.diff))
+
+                table_data = {name: names, 'Val in DS1': val1s, 'Val in DS2': val2s, 'Threshold': thresholds,
+                              'Relative Diff': diffs}
+                table = pd.DataFrame(table_data)
+
+                print("{} '{}':".format(sub_name, column_name))
+                nprint(table)
+                print("\n")
+
+    @staticmethod
+    def difference_to_string(metrics_difference, print_plus_minus=False):
+        metrics_difference = round(metrics_difference * 100, 2)
+        metrics_difference_string = str(metrics_difference) + ' %'
+
+        if print_plus_minus:
+            metrics_difference_string = '+/- ' + metrics_difference_string
+        elif metrics_difference > 0:
+            metrics_difference_string = '+' + metrics_difference_string
 
-            print("Column '{}':\n{}\n".format(column, explanation))
+        return metrics_difference_string
 
     @staticmethod
     def numerical_plot(df1, df2):
         def custom_plot():
-            num_columns = len(list(df1.columns))
-            grid_cols = 5
-            grid_rows = int(np.ceil(num_columns / grid_cols))
-            f = plt.figure(figsize=(PLOT_GRID_WIDTH, PLOT_ROW_HEIGHT * grid_rows))
-            grid = gridspec.GridSpec(grid_rows, grid_cols, wspace=0.3)
-            for column, tile in zip(df1.columns, grid):
-                a, b = df1[column], df2[column]
-                ax = plt.Subplot(f, tile)
-
-                ax.boxplot([a, b])
-                ax.set_title(plot_title(column))
-                ax.set_xticklabels([LEGEND_1, LEGEND_2])
-                f.add_subplot(ax)
+            num_figures = len(list(df1.columns))
+            num_cols = 5
+            f = plt.figure(figsize=(20, 6 * (num_figures/num_cols)))
+
+            for num, column in enumerate(sorted(list(df1.columns))):
+                a, b = df1[column].dropna(), df2[column].dropna()
+                ax = f.add_subplot(num_figures/num_cols + 1, num_cols, num + 1)
+
+                ax.boxplot([a, b], labels=[LEGEND_1, LEGEND_2])
+                ax.set_title(diagram_title(column))
 
             plt.show()
 
         return custom_plot
 
     @staticmethod
-    def categorical_plot(plot_infos):
-
+    def attribute_val_plot(plot_infos):
         def custom_plot():
-            f = plt.figure(figsize=(20, 7))
-            num_columns = len(list(plot_infos))
+            num_figures = len(list(plot_infos))
+            num_cols = 3
+
+            f = plt.figure(figsize=(18, 8 * (num_figures/num_cols)))
 
             for i, plot_info in enumerate(list(plot_infos)):
                 bars1, bars2, attribute_names, column_name = plot_info[0], plot_info[1], plot_info[2], plot_info[3]
 
-                subplot = f.add_subplot(1, num_columns, i + 1)
+                subplot = f.add_subplot(num_figures/3 + 1, 3, i + 1)
                 bar_width = 0.25
 
                 ind = np.arange(len(bars1))
-                subplot.barh(ind, bars1, bar_width, label=LEGEND_1)
-                subplot.barh(ind + bar_width, bars2, bar_width, label=LEGEND_2)
+                subplot.barh(ind + bar_width, bars1, bar_width, label=LEGEND_1)
+                subplot.barh(ind, bars2, bar_width, label=LEGEND_2)
 
                 subplot.set_yticklabels(attribute_names)
                 subplot.set_yticks(np.arange(len(attribute_names)) + bar_width / 2)
 
-                subplot.title.set_text(plot_title(column_name))
+                subplot.title.set_text(diagram_title(column_name))
                 subplot.legend()
 
             plt.show()
diff --git a/shift_detector/detector.py b/shift_detector/detector.py
index 6600045..8d9fbac 100644
--- a/shift_detector/detector.py
+++ b/shift_detector/detector.py
@@ -82,7 +82,7 @@ class Detector:
         Evaluate the reports.
         """
         nprint("OVERVIEW", text_formatting='h1')
-        nprint("Executed {} checks".format(len(self.check_reports)))
+        nprint("Executed {} check{}".format(len(self.check_reports), 's' if len(self.check_reports) > 1 else ''))
 
         detected = defaultdict(int)
         examined = defaultdict(int)
diff --git a/shift_detector/precalculations/dq_metrics_precalculation.py b/shift_detector/precalculations/dq_metrics_precalculation.py
index 18e15bd..10d72a0 100644
--- a/shift_detector/precalculations/dq_metrics_precalculation.py
+++ b/shift_detector/precalculations/dq_metrics_precalculation.py
@@ -1,10 +1,16 @@
 from copy import deepcopy
+
+from shift_detector.precalculations.low_cardinality_precalculation import LowCardinalityPrecalculation
 from shift_detector.precalculations.precalculation import Precalculation
+from shift_detector.precalculations.text_metadata import TextMetadata
 from shift_detector.utils.column_management import ColumnType
 
 
 class DQMetricsPrecalculation(Precalculation):
 
+    def __init__(self, text_metadata=False):
+        self.text_metadata = text_metadata
+
     def __eq__(self, other):
         return isinstance(other, self.__class__)
 
@@ -12,20 +18,28 @@ class DQMetricsPrecalculation(Precalculation):
         return hash(self.__class__)
 
     def process(self, store):
+        if self.text_metadata:
+            df1_metadata, df2_metadata = store[TextMetadata()]
+            metadata_comparison = self.compare_numerical_columns(df1_metadata, df2_metadata)
+        else:
+            metadata_comparison = None
+
         df1_numerical, df2_numerical = store[ColumnType.numerical]
-        df1_categorical, df2_categorical = store[ColumnType.categorical]
+        df1_categorical, df2_categorical, _ = store[LowCardinalityPrecalculation()]
 
         numerical_comparison = self.compare_numerical_columns(df1_numerical, df2_numerical)
         categorical_comparison = self.compare_categorical_columns(df1_categorical, df2_categorical)
-        combined_comparisons = {'categorical_comparison': categorical_comparison,
-                                'numerical_comparison': numerical_comparison}
+        attribute_val_comparison = self.compare_categorical_attribute_vals(df1_categorical, df2_categorical)
+        combined_comparisons = {'attribute_val_comparison': attribute_val_comparison,
+                                'numerical_comparison': numerical_comparison, 'categorical_comparison':
+                                categorical_comparison, 'metadata_comparison': metadata_comparison}
         return combined_comparisons
 
     @staticmethod
     def compare_numerical_columns(df1, df2):
         numerical_comparison = dict()
         empty_metrics_dict = {'mean': {}, 'median': {}, 'value_range': {}, 'quartile_1': {}, 'quartile_3': {},
-                              'uniqueness': {}, 'num_distinct': {}, 'completeness': {}, 'std': {}}
+                              'uniqueness': {}, 'completeness': {}, 'std': {}}
 
         for df_name, df in [('df1', df1), ('df2', df2)]:
             for column in df.columns:
@@ -43,18 +57,32 @@ class DQMetricsPrecalculation(Precalculation):
 
                 column_droppedna = df[column].dropna()
                 numerical_comparison[column]['std'][df_name] = column_droppedna.std()
-
-                numerical_comparison[column]['num_distinct'][df_name] = column_droppedna.nunique()
-
                 numerical_comparison[column]['completeness'][df_name] = len(column_droppedna) / len(df[column])
-
-                numerical_comparison[column]['uniqueness'][df_name] = len(df.groupby(column)
-                                                                          .filter(lambda x: len(x) == 1)) / \
-                                                                          len(column_droppedna)
+                numerical_comparison[column]['uniqueness'][df_name] = \
+                    len(df.groupby(column).filter(lambda x: len(x) == 1)) / len(column_droppedna)
         return numerical_comparison
 
     @staticmethod
     def compare_categorical_columns(df1, df2):
+        categorical_comparison = dict()
+        empty_metrics_dict = {'num_distinct': {}, 'completeness': {}, 'uniqueness': {}}
+
+        for df_name, df in [('df1', df1), ('df2', df2)]:
+            for column in df.columns:
+                if df_name == 'df1':
+                    categorical_comparison[column] = deepcopy(empty_metrics_dict)
+                elif not categorical_comparison.get(column):
+                    categorical_comparison[column] = deepcopy(empty_metrics_dict)
+
+                column_droppedna = df[column].dropna()
+                categorical_comparison[column]['num_distinct'][df_name] = column_droppedna.nunique()
+                categorical_comparison[column]['completeness'][df_name] = len(column_droppedna) / len(df[column])
+                categorical_comparison[column]['uniqueness'][df_name] = \
+                    len(df.groupby(column).filter(lambda x: len(x) == 1)) / len(column_droppedna)
+        return categorical_comparison
+
+    @staticmethod
+    def compare_categorical_attribute_vals(df1, df2):
         category_comparison = {}
 
         for column in list(df1.columns):
diff --git a/shift_detector/utils/custom_print.py b/shift_detector/utils/custom_print.py
index a78802d..04d7e0c 100644
--- a/shift_detector/utils/custom_print.py
+++ b/shift_detector/utils/custom_print.py
@@ -7,20 +7,23 @@ def nprint(input_str, num_tabs=0, text_formatting='normal'):
     for i in range(num_tabs):
         input_str = '\t' + input_str
 
-    if not is_in_jupyter() or text_formatting == 'normal':
+    if not is_in_jupyter():
         print(input_str)
+    else:
+        if text_formatting == 'h1':
+            display(Markdown('# {}'.format(input_str)))
 
-    if text_formatting == 'h1':
-        display(Markdown('# {}'.format(input_str)))
+        elif text_formatting == 'h2':
+            display(Markdown('## {}'.format(input_str)))
 
-    elif text_formatting == 'h2':
-        display(Markdown('## {}'.format(input_str)))
+        elif text_formatting == 'h3':
+            display(Markdown('### {}'.format(input_str)))
 
-    elif text_formatting == 'h3':
-        display(Markdown('### {}'.format(input_str)))
+        elif text_formatting == 'h4':
+            display(Markdown('#### {}'.format(input_str)))
 
-    elif text_formatting == 'h4':
-        display(Markdown('#### {}'.format(input_str)))
+        else:
+            display(input_str)
 
 
 # prints inline markdown
@@ -38,3 +41,7 @@ def is_in_jupyter():
 def lprint(string, log_print):
     if log_print:
         print(string)
+
+
+def diagram_title(name):
+    return "Column '{}'".format(name)


Test patch:
diff --git a/tests/checks/test_dq_metrics_check.py b/tests/checks/test_dq_metrics_check.py
index cd5b95a..656b72c 100644
--- a/tests/checks/test_dq_metrics_check.py
+++ b/tests/checks/test_dq_metrics_check.py
@@ -30,24 +30,36 @@ class TestDQMetricsCheck(TestCase):
             self.check = DQMetricsCheck()
             report = self.check.run(self.store_num)
 
-            explanation_string = report.explanation['cool_numbers']
-            formatted = re.sub('Metric: (.*), Diff: (.*) %, threshold: (.*)%', r"\1 \2", explanation_string)
-            formatted_list = re.split(' |\n', formatted)
-            self.assertCountEqual(formatted_list, ['mean', '+20.0', 'value_range', '+67.33', 'std', '+66.67', ''])
+            explanations = report.explanation['numerical_categorical']['cool_numbers']
+
+            self.assertEqual(explanations[0].val1, 2.5)
+            self.assertEqual(explanations[0].val2, 3.0)
+            self.assertEqual(round(explanations[0].diff, 2), 0.2)
+
+            self.assertEqual(explanations[1].val1, 3.0)
+            self.assertEqual(explanations[1].val2, 5.0)
+            self.assertEqual(round(explanations[1].diff, 2), 0.67)
+
+            self.assertEqual(round(explanations[2].val1, 2), 1.13)
+            self.assertEqual(round(explanations[2].val2, 2), 1.89)
+            self.assertEqual(round(explanations[2].diff, 2), 0.67)
 
         with self.subTest('no analyzer should detect shift'):
-            self.check = DQMetricsCheck(mean_threshold=.3, value_range_threshold=.68, std_threshold=.7)
+            self.check = DQMetricsCheck(mean_threshold=.3, value_range_threshold=.68, std_threshold=.7,
+                                        categorical_threshold=.25)
             report = self.check.run(self.store_num)
             self.assertEqual(report.shifted_columns, [])
 
         with self.subTest('only std should detect shift'):
-            self.check = DQMetricsCheck(mean_threshold=.3, value_range_threshold=.68, std_threshold=.5)
-            report = self.check.run(self.store_num)
+            self.check = DQMetricsCheck(mean_threshold=.3, value_range_threshold=.68, std_threshold=.5,
+                                        categorical_threshold=.25)
 
-            explanation_string = report.explanation['cool_numbers']
-            formatted = re.sub('Metric: (.*), Diff: (.*) %, threshold: (.*)%', r"\1 \2", explanation_string)
-            formatted_list = re.split(' |\n', formatted)
-            self.assertCountEqual(formatted_list, ['std', '+67.33', ''])
+            report = self.check.run(self.store_num)
+            explanations = report.explanation['numerical_categorical']['cool_numbers']
+            self.assertEqual(len(explanations), 1)
+            self.assertEqual(round(explanations[0].val1, 2), 1.13)
+            self.assertEqual(round(explanations[0].val2, 2), 1.89)
+            self.assertEqual(round(explanations[0].diff, 2), 0.67)
 
     def test_run_categorical(self):
         with self.subTest("Test precalculation"):
@@ -56,39 +68,27 @@ class TestDQMetricsCheck(TestCase):
         with self.subTest("Test shifted categorical columns"):
             self.assertEqual(report.shifted_columns, ['shift'])
             self.assertCountEqual(report.examined_columns, ['shift', 'no_shift'])
-            self.assertEqual(report.explanation['shift'], "Attribute: 'A' with Diff: +100.0 %, "
-                                                          "categorical threshold: +/- 5.0 %\n")
-
-    def test_run_numerical(self):
-        with self.subTest("Test precalculation"):
-            report = self.check.run(self.store_num)
-
-        with self.subTest('Test shifted numerical columns'):
-            self.assertEqual(report.shifted_columns, ['cool_numbers'])
-            self.assertCountEqual(report.examined_columns, ['cool_numbers'])
 
-        with self.subTest('Test explanations'):
-            explanation_string = report.explanation['cool_numbers']
-            formatted = re.sub('Metric: (.*), Diff: (.*) %, threshold: (.*)%', r"\1 \2", explanation_string)
-            formatted_list = re.split(' |\n', formatted)
-            self.assertCountEqual(formatted_list, ['mean', '+20.0', 'value_range', '+66.67', 'std',
-                                                   '+67.33', ''])
+            shift_explanation = report.explanation['attribute_val']['shift'][0]
+            self.assertEqual(shift_explanation.val1, 1.0)
+            self.assertEqual(shift_explanation.val2, 0)
+            self.assertEqual(shift_explanation.diff, 1.0)
 
     def test_relative_metric_difference(self):
         with self.subTest('Normal case'):
             self.check.data = {'numerical_comparison': {'column': {'metric_name': {'df1': 10.0, 'df2': 5.0}}}}
-            self.assertEqual(self.check.relative_metric_difference('column', 'metric_name'), -.5)
+            self.assertEqual(self.check.relative_metric_difference('column', 'metric_name')[2], -.5)
 
             self.check.data = {'numerical_comparison': {'column': {'metric_name': {'df1': 1.2, 'df2': 12.0}}}}
-            self.assertEqual(self.check.relative_metric_difference('column', 'metric_name'), 9.0)
+            self.assertEqual(self.check.relative_metric_difference('column', 'metric_name')[2], 9.0)
 
         with self.subTest('Both values zero'):
             self.check.data = {'numerical_comparison': {'column': {'metric_name': {'df1': 0, 'df2': 0}}}}
-            self.assertEqual(self.check.relative_metric_difference('column', 'metric_name'), 0)
+            self.assertEqual(self.check.relative_metric_difference('column', 'metric_name')[2], 0)
 
         with self.subTest('Value in df1 zero'):
             self.check.data = {'numerical_comparison': {'column': {'metric_name': {'df1': 0, 'df2': 12.4}}}}
-            self.assertEqual(self.check.relative_metric_difference('column', 'metric_name'), 0)
+            self.assertEqual(self.check.relative_metric_difference('column', 'metric_name')[2], 0)
 
     @mock.patch('shift_detector.checks.dq_metrics_check.plt')
     def test_numerical_plots_work(self, mock_plt):
@@ -107,15 +107,10 @@ class TestDQMetricsCheck(TestCase):
         self.assertFalse(mock_plt.figure.called)
 
         report = self.check.run(self.store)
-        custom_plot_categorical = report.categorical_plot([([1, 2, 3], [2, 4, 6], ['Heinz', 'Peter', 'Rudolf'],
+        custom_plot_categorical = report.attribute_val_plot([([1, 2, 3], [2, 4, 6], ['Heinz', 'Peter', 'Rudolf'],
                                                             'A very important plot')])
         custom_plot_categorical()
 
         self.assertTrue(mock_plt.figure.called)
         self.assertTrue(mock_plt.figure().add_subplot.called)
         self.assertTrue(mock_plt.show.called)
-
-
-
-
-
diff --git a/tests/precalculations/test_simple_precalculation.py b/tests/precalculations/test_dq_metrics_precalculation.py
similarity index 82%
rename from tests/precalculations/test_simple_precalculation.py
rename to tests/precalculations/test_dq_metrics_precalculation.py
index ed3e173..84bf883 100644
--- a/tests/precalculations/test_simple_precalculation.py
+++ b/tests/precalculations/test_dq_metrics_precalculation.py
@@ -8,7 +8,7 @@ from shift_detector.precalculations.dq_metrics_precalculation import DQMetricsPr
 from shift_detector.precalculations.store import Store
 
 
-class TestSimplePrecalculation(unittest.TestCase):
+class TestDQMetricsPrecalculation(unittest.TestCase):
 
     def setUp(self) -> None:
         self.precalculation = DQMetricsPrecalculation()
@@ -73,16 +73,6 @@ class TestSimplePrecalculation(unittest.TestCase):
         self.assertEqual(comparison_numeric['col_2']['uniqueness']['df2'], 1.0)
         self.assertEqual(comparison_numeric['col_3']['uniqueness']['df2'], 0)
 
-    def test_num_distinct(self):
-        comparison_numeric = self.precalculation.process(self.store_numerical)['numerical_comparison']
-
-        self.assertEqual(comparison_numeric['col_1']['num_distinct']['df1'], 100.0)
-        self.assertEqual(comparison_numeric['col_2']['num_distinct']['df1'], 50.0)
-        self.assertEqual(comparison_numeric['col_3']['num_distinct']['df1'], 100.0)
-        self.assertEqual(comparison_numeric['col_1']['num_distinct']['df2'], 100.0)
-        self.assertEqual(comparison_numeric['col_2']['num_distinct']['df2'], 1.0)
-        self.assertEqual(comparison_numeric['col_3']['num_distinct']['df2'], 50.0)
-
     def test_completeness(self):
         comparison_numeric = self.precalculation.process(self.store_numerical)['numerical_comparison']
 
@@ -94,10 +84,12 @@ class TestSimplePrecalculation(unittest.TestCase):
         self.assertEqual(comparison_numeric['col_3']['completeness']['df2'], 1.00)
 
     def test_categorical_values(self):
-        comparison_categorical = self.precalculation.process(self.store_categorical)['categorical_comparison']
-
-        self.assertEqual(comparison_categorical[0]['blue']['df1'], 1/3)
-        self.assertEqual(comparison_categorical[0]['green']['df1'], 0.5)
-        self.assertEqual(comparison_categorical[0]['green']['df2'], 5/6)
-        self.assertEqual(comparison_categorical[0]['red']['df1'], 1/6)
-        self.assertEqual(comparison_categorical[0]['red']['df2'], 1/6)
+        comparison_categorical = self.precalculation.process(self.store_categorical)['categorical_comparison'][0]
+
+        print(comparison_categorical)
+        self.assertEqual(comparison_categorical['num_distinct']['df1'], 3)
+        self.assertEqual(comparison_categorical['num_distinct']['df2'], 2)
+        self.assertEqual(comparison_categorical['completeness']['df1'], 1)
+        self.assertEqual(comparison_categorical['completeness']['df2'], 1)
+        self.assertEqual(comparison_categorical['uniqueness']['df1'], 0.0)
+        self.assertEqual(comparison_categorical['uniqueness']['df2'], 0.0)