## 3 A New Approach for Bias Detection Combining Rule Mining and Regression Analysis
### Model Comparison and Selection  
Once the hierarchical structure is in place, we apply a model selection procedure by comparing each parent with all its child models. Our pruning strategy consists of checking whether the addition of variables to a child model adds information that leads to a better description of the data, or whether the simpler model is preferable (in that it describes the data indistinguishably well with respect to the more complex model). This is operationalized through the ANOVA test. Alternatives to ANOVA for model comparison exist, such as AIC, BIC, or maximum likelihood; in this work, we adopt ANOVA for model comparison, but any other method could be used. The ANOVA test is a widely used statistical test that allows one to compare the fits of two regression models, one nested in the other, by comparing the (sum of squares of the) residuals (i.e., the errors) of the respective model predictions [14]. If the output of the ANOVA test indicates that the more complex model provides a significantly better explanation of the variance in the prediction, the child model is marked as preferable compared to the more general, simpler parent model. Otherwise, the parent model is marked as preferable. Formally, given a set \(\mathcal{M}\) of regression models to test, we aim to derive the set \(\mathcal{M}_{sel}=\{M_{i}\in\mathcal{M}\mid\
exists M_{j}\in\mathcal{M}\) s.t. \(M_{j}\sqsubset M_{i}\wedge ANOVA(M_{i},M_{j})\leq\rho\}\), where \(\rho\) represents a threshold to determine whether \(ANOVA(M_{i},M_{j})\) shows a statistically significant difference in the sum of the squared residuals between the two models. Operationally speaking, the set \(\mathcal{M}_{sel}\) was derived using the procedure shown in Algorithm 1. ```
Input: Model hierarchy \((\mathcal{M},\sqsubset)\) and significance threshold \(\rho\) Output: \(\mathcal{M}_{sel}\)
1\(\mathcal{M}_{sel}\leftarrow\emptyset\) ;
2\(\mathcal{M}_{p}\leftarrow\{M_{i}\in\mathcal{M}\mid\
exists M_{j}\text{ s.t. }M_{j}\subset M_{i}\ \}\) ;
3 while\(\mathcal{M}_{p}\
eq\emptyset\)do
4\(M_{i}\leftarrow\mathcal{M}_{p}.pop()\);
5\(\mathcal{M}_{\text{acc}}\leftarrow\{M_{j}\mid M_{j}\sqsubset M_{i}\wedge ANOVA (M_{i},M_{j})\leq\rho\}\) ;
6 if\(\mathcal{M}_{\text{acc}}=\emptyset\)then
7\(\mathcal{M}_{sel}\leftarrow\mathcal{M}_{sel}\cup\{M_{i}\}\);
8
9 else
10\(\mathcal{M}_{p}\leftarrow\mathcal{M}_{p}\cup\mathcal{M}_{\text{acc}}\);
11 return\(\mathcal{M}_{sel}\)
```  
**Algorithm 1**Model Selection. This procedure takes as input _(i)_ a model hierarchy \((\mathcal{M},\sqsubset)\), where \(\mathcal{M}\) is the set of regression models and \(\sqsubset\) is the directly nested relation on \(\mathcal{M}\), and _(ii)_ a threshold \(\rho\) determining whether the result of the ANOVA test is significant, and iteratively checks whether nested models provide more significant explanation of biases. At the beginning, the output set \(\mathcal{M}_{sel}\) is initialized to the empty set, while the set of regression models to be analyzed  
Figure 2: Example of a hierarchy of models. is stored in a stack, \(\mathcal{M}_{p}\), which is initialized to the root models (i.e., the models that are not nested in any other model) (lines 1-2). The models in \(\mathcal{M}_{p}\) are iteratively extracted from \(\mathcal{M}_{p}\) and compared against their direct nested models using the ANOVA test (lines 4-5). Every child that is preferable compared to its parent(s) based on the ANOVA test is added to the set of _winner children_\(\mathcal{M}_{wc}\) (line 5). If the set \(\mathcal{M}_{wc}\) for a given parent \(M_{i}\) is empty, i.e., there are no better performing children models than \(M_{i}\) according to the ANOVA test, then \(M_{i}\) is added to the output set \(\mathcal{M}_{sel}\) (lines 6-7). Otherwise, the models in \(\mathcal{M}_{wc}\) are added to \(\mathcal{M}_{p}\) in order to be analyzed in the next iterations of the algorithm, i.e., they are compared against their children (line 9). The procedure terminates when \(\mathcal{M}_{p}\) is empty.