## 2 Methods
### Model Interpretation and Shapley Additive Explanations Method (SHAP)  
The authors of [34] define interpretability as the degree to which a human can understand the cause of a decision. One can describe a model as interpretable if he or she can comprehend the entire model at once. Typically, the concept of global interpretation of model-agnostic feature importance means that we measure a feature's importance by calculating the increase in the model's prediction error after perturbating the feature's value. The permutation feature's importance is defined as the decrease in a model's score when a single feature value is randomly shuffled.  
If BCM were an inherently interpretable model that imposes some kind of interpretability constraints and presents explanations, for the NN, we would need to search for model agnostic methods that are also post hoc since they are decoupled from the black box [35]. We use the Shapley additive explanations (SHAP) method to explain how each input feature affects a prediction. The goal of SHAP is to explain the prediction of an instance by computing the contribution of each feature to the prediction. As [PERSON] [34] explains, the SHAP explanation method computes Shapley values using coalitional game theory. The feature values of a data instance act as players in a coalition. Shapley values tell us how to fairly distribute the \"payout\" (i.e., the prediction) among the features. A player can be an individual feature value, e.g., for tabular data. A player can also be a group of feature values. The Shapley value of a feature value is its contribution to the payout, weighted and summed over all possible feature value combinations. In other words, the Shapley value is the average of all the marginal contributions to all possible coalitions. The Shapley value of a feature value is not the difference in the predicted value after removing the feature from the model training, as in permutation feature importance. Given the current set of feature values, the estimated Shapley value is the contribution of a feature value to the difference between the actual prediction and the mean prediction. SHAP feature importance may offer an alternative to permutation feature importance, but they are not the same: permutation feature importance is based on the decrease in model performance, while SHAP is based on the magnitude of feature attributions [34].  
Shapley values can be combined into global explanations. Using the SHAP method for every instance, we can get a matrix of Shapley values (one row per data instance and one column per feature). We can interpret the entire model by analyzing the Shapley values in this matrix [34].  
When applying SHAP, we often assume independence between the different input features. It must be noted that Shapley values may be potentially misleading when predictors are highly correlated. As explained by [PERSON] [34], \"to simulate that a feature value is missing from a coalition, we marginalize the feature. This is achieved by sampling values from the feature's marginal distribution. This is fine, as long as the features are independent, but when features are dependent, we might sample feature values that do not make sense for the instance\". For cases with correlated input data features, several options have been considered in the paper: discarding one of the two correlated features, grouping using Shapley cohort refinement [36], using an extended-kernel SHAP [33], or applying other interpretation methods--influential instances, adversarial examples, etc. In cases of multi-collinearity, where several independent variables in a model are correlated, we can perform hierarchical clustering on the features' Spearman rank-order correlations, pick a threshold, and keep a single feature from each cluster, or use the multi-collinearity correction method presented in [37].