Repository: mindsdb/lightwood

Issue:
:wrench: Implement a SHAP analysis block
## Introduction

Lightwood is an AutoML library that aims at abstracting common procedures in the Machine Learning workflow (e.g. data preparation, feature engineering, model building, model analysis) with an approach that can be fully automated but easily customizable and extended thanks to its flexibility. 

To analyze a model, a series of "analysis blocks" run after the ensemble step and when generating predictions. Each of these blocks can (1) do some type of analysis after training has finished, (2) do computations at inference time to give additional insights about predictions (e.g. a confidence score), or both (1) and (2).

[SHAP](https://github.com/slundberg/shap) (SHapley Additive exPlanations) is a very popular method for explaining predictions of a ML model. For each prediction, features are assigned an importance value. Taking the sum of them all gets you from the model "base" output to that individual problem instance's output.

## Task

In this task, we seek to add an analysis block that will output feature attribution for a model's predictions.

## Steps :male_detective: :female_detective: 

- Fork the Lightwood repository, checkout the `staging` branch and from it create a new one.
- Add an analysis block (inherit from `lightwood/analysis/base/BaseAnalysisBlock`) in `lightwood/analysis/helpers/` that computes shapley values in its `.explain()` method. This would require instancing a `shap.Explainer` object and calling its `.shap_values()` method. The output should be added to the input insights in additional columns with a naming scheme like `f"feature_{i}_impact"`, or similar. _Ideally_, you will also add the base response of the model (tip: you can derive this value by subtracting all feature impacts to the predicted value).
> Note: Unless you deem it necessary, an `.analysis()` implementation is not needed.
- Create (or modify an existing) integration test that uses your SHAP analysis block and checks the output has the expected properties. Alternatively, you can create a tutorial in the `docssrc/source/tutorial/custom_explainer` folder, explaining how to use this block and what insights can be made from an example predictive task.
- Make the PR and address any comments that reviewers might make.

## Additional rewards :1st_place_medal: 

Accepted PRs for model analysis bring :five: points for entry into the draw for a :computer:  Deep Learning Laptop powered by the NVIDIA RTX 3080 Max-Q GPU or other swag :shirt: :bear: .  For more info check out https://mindsdb.com/hacktoberfest/


Discussion:

Resolving patch:
diff --git a/lightwood/analysis/__init__.py b/lightwood/analysis/__init__.py
--- a/lightwood/analysis/__init__.py
+++ b/lightwood/analysis/__init__.py
@@ -8,5 +8,11 @@
 from lightwood.analysis.helpers.acc_stats import AccStats
 from lightwood.analysis.helpers.feature_importance import GlobalFeatureImportance
 
+try:
+    from lightwood.analysis.helpers.shap import ShapleyValues
+except Exception:
+    ShapleyValues = None
 
-__all__ = ['model_analyzer', 'explain', 'ICP', 'AccStats', 'GlobalFeatureImportance', 'BaseAnalysisBlock']
+
+__all__ = ['model_analyzer', 'explain', 'ICP', 'AccStats',
+           'GlobalFeatureImportance', 'BaseAnalysisBlock', 'ShapleyValues']
diff --git a/lightwood/analysis/helpers/shap.py b/lightwood/analysis/helpers/shap.py
new file mode 100644
--- /dev/null
+++ b/lightwood/analysis/helpers/shap.py
@@ -0,0 +1,94 @@
+import warnings
+from types import SimpleNamespace
+from typing import Dict, Optional, Tuple
+
+import numpy as np
+import pandas as pd
+from lightwood.analysis.base import BaseAnalysisBlock
+from lightwood.api import dtype
+from lightwood.api.types import PredictionArguments
+from lightwood.data.encoded_ds import EncodedDs
+from lightwood.helpers.log import log
+from sklearn.preprocessing import LabelEncoder
+
+import shap
+
+
+class ShapleyValues(BaseAnalysisBlock):
+    """
+    Analysis block that estimates column importance with SHAP (SHapley Additive exPlanations), a game theoretic approach
+    to explain the ouput of any machine learning model. SHAP assigns each feature an importance value for a particular
+    prediction.
+
+    Reference:
+        https://shap.readthedocs.io/en/stable/
+        https://proceedings.neurips.cc/paper/2017/file/8a20a8621978632d76c43dfd28b67767-Paper.pdf
+    """
+    label_encoder: LabelEncoder
+
+    def __init__(self, deps: Optional[Tuple] = ...):
+        super().__init__(deps=deps)
+        self.label_encoder = LabelEncoder()
+
+    def analyze(self, info: Dict[str, object], **kwargs) -> Dict[str, object]:
+        log.info('Preparing to compute feature importance values with SHAP')
+        ns = SimpleNamespace(**kwargs)
+
+        output_dtype = ns.dtype_dict[ns.target]
+        train_data: EncodedDs = ns.train_data
+
+        if output_dtype in (dtype.integer, dtype.float, dtype.quantity):
+            pass
+        elif output_dtype in (dtype.binary, dtype.categorical, dtype.tags):
+            self.label_encoder.fit(train_data.data_frame[ns.target].values)
+        else:
+            log.warning(f'ShapleyValues analyzers not supported for type: {output_dtype}')
+            return info
+
+        def model(x: np.ndarray) -> np.ndarray:
+            assert(isinstance(x, np.ndarray))
+            df = pd.DataFrame(data=x, columns=train_data.data_frame.columns)
+            ds = EncodedDs(encoders=train_data.encoders, data_frame=df, target=train_data.target)
+
+            decoded_predictions = ns.predictor(ds=ds, args=PredictionArguments())
+            if output_dtype in (dtype.integer, dtype.float, dtype.quantity):
+                encoded_predictions = decoded_predictions['prediction'].values
+            elif output_dtype in (dtype.binary, dtype.categorical, dtype.tags):
+                encoded_predictions = self.label_encoder.transform(decoded_predictions['prediction'].values)
+
+            return encoded_predictions
+
+        info['shap_explainer'] = shap.KernelExplainer(model=model, data=train_data.data_frame)
+
+        return info
+
+    def explain(self,
+                row_insights: pd.DataFrame,
+                global_insights: Dict[str, object],
+                **kwargs
+                ) -> Tuple[pd.DataFrame, Dict[str, object]]:
+        log.info('Computing feature importance values with Kernel SHAP method')
+        ns = SimpleNamespace(**kwargs)
+
+        shap_explainer = ns.analysis.get('shap_explainer', None)
+        if shap_explainer is None:
+            return row_insights, global_insights
+
+        with warnings.catch_warnings():
+            warnings.filterwarnings("ignore", category=DeprecationWarning)
+            shap_values = shap_explainer.shap_values(ns.data, silent=True)
+
+        shap_values_df = pd.DataFrame(shap_values).rename(
+            mapper=lambda i: f"feature_{i}_impact", axis='columns')
+
+        if kwargs.get('target_dtype', None) in (dtype.binary, dtype.categorical, dtype.tags):
+            predictions = self.label_encoder.transform(row_insights['prediction'])
+        else:
+            predictions = row_insights['prediction']
+
+        base_response = (predictions - shap_values_df.sum(axis='columns')).mean()
+        global_insights['base_response'] = base_response
+
+        row_insights = pd.concat([row_insights, shap_values_df], axis='columns')
+
+        return row_insights, global_insights


Test patch:
diff --git a/tests/unit_tests/analysis/helpers/test_shap.py b/tests/unit_tests/analysis/helpers/test_shap.py
new file mode 100644
--- /dev/null
+++ b/tests/unit_tests/analysis/helpers/test_shap.py
@@ -0,0 +1,35 @@
+import unittest
+import pandas as pd
+from lightwood.analysis import ShapleyValues
+
+from lightwood.api.high_level import ProblemDefinition, json_ai_from_problem
+from lightwood.api.high_level import code_from_json_ai, predictor_from_code
+
+
+class TestBasic(unittest.TestCase):
+    def test_0_shap_analysis(self):
+        if ShapleyValues is None:
+            print('Skipping this test since the Shapley values library is not installed')
+            return
+
+        df = pd.read_csv('tests/data/hdi.csv')
+        target = 'Development Index'
+
+        pdef = ProblemDefinition.from_dict({'target': target, 'time_aim': 5})
+        json_ai = json_ai_from_problem(df, problem_definition=pdef)
+
+        json_ai.analysis_blocks = [{
+            'module': 'lightwood.analysis.ShapleyValues',
+            'args': {}
+        }]
+
+        code = code_from_json_ai(json_ai)
+        predictor = predictor_from_code(code)
+
+        predictor.learn(df)
+        predictions = predictor.predict(df.head())
+
+        self.assertIn('shap_explainer', predictor.runtime_analyzer)
+        self.assertIn('feature_4_impact', predictions.columns)
+        # TODO: once global_insights is exposed, check that all feature impacts
+        # plus base_response sum up to the prediction value