# _Applied Sciences_
## 3 Materials and Methods
### Comparison Methods and Data Sets  
gbt-HIPS produces CR-based explanations. Direct comparisons are possible against other methods that either output a single CR as an explanation, or a rule list from which a single CR can be extracted. Readers that have some familiarity with XAI may question the omission of LIME [19] and SHAP [20] from this study since they are two of the most discussed explanation methods to date. However, as the authors of [20] make clear, these are AFAM and, therefore, of an entirely different class. There is no straightforward way to compare explanations from different classes as prior works have demonstrated [21; 22]. For example, there is no way to measure the coverage of an AFAM explanation over a test set, whereas for a CR the coverage is unambiguous. Fortunately, Anchors [22] has been developed by the same research group that contributed LIME. Anchors can be viewed as a CR-based extension of LIME and its inclusion into this study provides a useful comparison to AFAM research. In addition to Anchors, LORE [21] is included, as another per-instance, CR-based explanation method. These are the only such methods that are freely available as open-source libraries for Python and R development environments.  
We also included three leading CRL-based interpretable machine learning (IML) methods into the study design. When using CRL models, the first covering (or firing) rule is used to classify the instance and, thus, is also the stand-alone explanation. If there is no covering rule, the default rule is fired. This _null_ rule simply classifies using the prior class majority. For the purposes of measuring rule length, a firing null rule has a length of zero. All selected methods are detailed in Table 2.  
The nine data sets selected for this study have been carefully selected to represent a mix of binary and multi-class problems, to exhibit different levels of class imbalance (no artificial balancing will be applied), to be a mixture of discrete and continuous features, and to be a contextual fit for XAI (i.e., credit and personal data) where possible. The data sets are detailed in Table 3. All are publicly available and taken from the UCI Machine Learning Repository [36] except lending (Kaggle) and rcdv (ICPSR; [[https://tinyurl.com/y8](https://tinyurl.com/y8) qvcgwu]([https://tinyurl.com/y8](https://tinyurl.com/y8) qvcgwu) (30 October 2019)). Three of these data sets (adult, lending and rcdv) are those used in [22] and, therefore, align with precedents and provide direct comparisons to state-of-the-art methods Anchors and LIME. The exceptionally large lending data set has been subsampled. The number of instances in the original data set is 842,000 and was downsampled to \(N=2105\) for these experiments. Training and test data sets were sampled without replacement into partitions of size 70 and 30% of the original data set.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Method** & **IML/Local** & **Ref.** \\ \hline inTrees & IML & [17] \\ defragTrees & IML & [16] \\ BRL & IML & [18] \\ Anchors & Local & [22] \\ LORE & Local & [21] \\ \hline \hline \end{tabular}
\end{table}
Table 2: Methods used in the experiments.