## 3 A New Approach for Bias Detection Combining Rule Mining and Regression Analysis
### Model Derivation  
To determine whether the candidate hypotheses extracted using rule mining corresponds to biases in the decisional process, we derive regression models from the set of relevant rules \(R_{rel}\). Specifically, for each rule \(r:\textit{Var}_{1}=v_{1},\ldots,\textit{Var}_{N}=v_{N}\rightarrow\texttt{ Class}=Y\) in \(R_{rel}\), we consider the set of explanatory variables occurring in the antecedent of the rule \(V_{i}=\{\textit{Var}_{1},\ldots,\textit{Var}_{N}\}\subseteq\mathcal{V}\) and build a corresponding regression model \(M_{i}\) of the form of:  
\[M_{i}:\texttt{Class}=c+\sum_{i=1}^{N}\beta_{i}\textit{Var}_{i}+ \sum_{1\leq i<j\leq N}\beta_{ij}\textit{Var}_{i}\cdot\textit{Var}_{j}\\ +\sum_{1\leq i<j<k\leq N}\beta_{ijk}\textit{Var}_{i}\cdot\textit {Var}_{j}\cdot\textit{Var}_{k}+\ldots+\beta_{1\ldots N}\textit{Var}_{1}\cdot \ldots\cdot\textit{Var}_{N} \tag{4}\]  
**Example 2**.: _From the set of rules in Example 1, we can extract three models:_  
\[M_{1}:\quad\texttt{HighRisk} =c_{1}+\beta_{1,1}\texttt{Income}+\beta_{1,2}\texttt{EthnicGroup} +\beta_{1,3}\texttt{Income}\cdot\texttt{EthnicGroup}\] \[M_{2}:\quad\texttt{HighRisk} =c_{2}+\beta_{2,1}\texttt{Gender}\] \[M_{3}:\quad\texttt{HighRisk} =c_{3}+\beta_{3,1}\texttt{Gender}+\beta_{3,2}\texttt{Employed}+ \beta_{3,3}\texttt{Gender}\cdot\texttt{Employed}\]  
_Note that some rules collapse in a single model as they contain exactly the same set of variables. In our example, this is the case for rules \(r_{1}\) and \(r_{4}\), which are both represented by model \(M_{1}\)._  
To efficiently compare the 'credibility' of the obtained models (next step), we organize the derived models in a hierarchical structure. To this end, we introduce a partial order relation over regression models which resembles the subrule relation. Given two regression models \(M_{i}\) and \(M_{j}\) defined over the sets of explanatory variables \(V_{i}\) and \(V_{j}\), respectively, we say that \(M_{j}\) is _nested_ in \(M_{i}\), denoted as \(M_{j}\subset M_{i}\), if and only if \(V_{i}\subset V_{j}\). Whereas in the econometrics literature the term _nested_ usually refers to the more general model (i.e., \(M_{j}\) is _nested_ in \(M_{i}\) if \(V_{j}\subset V_{i}\)), here we adopt the opposite definition to remain consistent with the definition of _subrule_ provided in Section 3.1. Moreover, we say that \(M_{j}\) is _directly nested_ in \(M_{i}\), denoted as \(M_{j}\sqsubset M_{i}\), if and only if \(M_{j}\subset M_{i}\) and there does not exist a model \(M_{k}\) such that \(M_{k}\subset M_{i}\) and \(M_{j}\subset M_{k}\). Based on the direct nesting relation, we construct a forest of models whereby the model at the root of each tree is the simplest model (i.e., with the smallest number of variables on the right hand side), and each child is a direct nested model of its parent(s).  
**Example 3**.: _The three models in Example 2 along with other hypothetical regression models can be represented in a hierarchy, as shown Figure 2. The hierarchy has two root nodes, i.e., \(M_{5}\) and \(M_{2}\), each with two children (\(M_{1}\) and \(M_{4}\) for \(M_{5}\), \(M_{3}\) and \(M_{4}\) for \(M_{2}\)) among which one is in common (i.e., \(M_{4}\)). \(M_{4}\) is further extended by \(M_{6}\), which is also a child of \(M_{3}\)._