# Making Sense of Machine Learning: A Review of Interpretation Techniques and Their Applications
## 3 Techniques for Explainable AI
### Post Hoc Interpretation Techniques  
Post hoc interpretation techniques generate explanations for models that are not inherently explainable, making it possible to understand more of the logic behind black-box or complex ML models. These techniques include LIME [16], SHAP [15], and Grad-CAM [23], which generate local explanations by highlighting the features that contribute most to a model's decision. Some methods, such as LIME and Kernel SHAP [15], are model-agnostic and can be used on any ML models, while others, such as Tree SHAP [21], Deep SHAP [15], DeepLIFT [22] and Grad-CAM [23], are model-specific by leveraging extra knowledge of the specific models.  
[PERSON] et al. proposed Local interpretable model-agnostic explanations (LIMEs) to explain the predictions of any classifier by learning an interpretable model locally around the prediction [16]. To explain the prediction for an instance, a LIME generates new fake instances around it by sampling its neighborhood, applies the original complex or black-box model to those new instances to produce predictions, and trains an interpretable model (e.g., a linear model) that captures the behaviors of the complex model in that neighborhood. Based on that, a representative set of individual instances can be selected and explained to achieve a global understanding of the model. LIMEs are claimed to be capable of explaining the predictions of any classifier in an interpretable and faithful manner.  
[PERSON] et al. presented a unified framework named Shapley additive explanations (SHAPs) for interpreting model predictions [15]. Shapley values [57] are a concept for measuring each player's contribution to the game in cooperative game theory and are comparable to the importance of features in ML model prediction. The Shapley value of a feature is calculated as a weighted average of the model prediction differences between models trained with and without the feature under all possible combinations of the other features. Formally, it is calculated as  
\[\phi_{i}=\sum_{S\subseteq F\setminus\{i\}}\frac{|S|!\;(|F|-|S|-1)!}{|F|!} \Big{[}f_{S^{\prime}}(x_{S^{\prime}})-f_{S}(x_{S})\Big{]}, \tag{2}\]  
where \(\phi_{i}\) is the Shapley value of feature \(i\), \(F\) is the set of all features, \(S\) is a subset of \(F\setminus\{i\}\), \(x_{S}\) is the values of input features in set \(S\), \(f_{S}\) is a model trained with feature set \(S\), \(f_{S}(x_{S})\) is the model prediction with set \(S\), and \(S^{\prime}=S\cup\{i\}\). The Shapley values are very challenging to calculate, in that there are \(2^{|F|}\) differences to be computed. [PERSON] et al. proposed Kernel SHAP and Deep SHAP to approximate them efficiently. Kernel SHAP is a model-agnostic method that can be used on any model, and Deep SHAP is a model-specific method for deep neural networks models. [PERSON] et al. also proposed Tree SHAP [21] for estimating the SHAP values of tree ensembles, such as random forests and XGBoost, in polynomial time.  
[PERSON] et al. [58] compared LIMEs and SHAPs in generating explanations for a deep learning model trained on a medical dataset. The results showed that both techniques were useful in generating explanations for the model's decision-making process.