## 4 Experiments
### Approach Validation
#### 4.1.3 Results  
**Models vs. rules. ** Table 5 reports descriptive statistics of the results over all experiment runs. The first set of rows reports the statistics for association rule mining, whereas the second set reports the statistics for our approach. We first observe that the number of rules is significantly larger than both the number of total models (i.e., the models derived from the set of rules) and that of selected models (i.e., the models returned in output by our approach). Furthermore, the number of the selected models is, on average, four times smaller than the overall number of models. The average experimental run produces approximately 12.87 rules, with a maximum of 139. The relatively high standard deviation (with respect to the mean) indicates that the number of rules in output can vary by large amounts across experimental setups. In contrast, the number of total (selected) models per experimental setup is, on average, more than two (twelve) times smaller, similarly to what can be observed for the maximum. The low standard deviation indicates a relatively stable output across experiments, especially for the selected models. Overall, this indicates that the model selection procedure appears to be removing a large number of rules but says little about the _correctness_ of this process. A first indication of the correctness of this process can be derived by evaluating of the number of _exact_, _too general_, _too specific_, _partial_, and _off target_ rules/models in output of our method. Considering the obtained rules, we observed that association rule mining never returns _off target_ recommendations. Moreover, it is able to identify, on average, at least one correct recommendation, either in terms of _exact_ or _too general_ recommendations. However, comparing these numbers with the overall average number of rules returned, these recommendations are likely to be hidden in a multitude of misleading recommendations. In fact, the results show that rule mining tends to return a much higher number of undesirable recommendations; on average, we obtain 3.52 _too specific_ and 6.58 _partial_ recommendations. On the other hand, we observe that our approach returns a higher number of max _exact_ recommendations. This is because a regressor (matching the ground truth) can be significant in multiple models. In general, we observe a similar distribution in the first and second quartile, even though the mean and median values show in general a lower overall capability of our approach to identify relevant groups under most circumstances (the median of _Exact_ is 0). However, this minimal loss in detection is compensated by a large reduction in false positives to investigate. Moving to higher quartiles, we observe that our approach never returns _partial_ or _off target_ results, and generates much less _too specific_ and _too general_ recommendations. Overall, the results suggest that our approach is able to generate a more accurate output. However, this clearly depends on the number of discriminated groups, and results may vary significantly depending on the noise and overlap introduced in the synthetic datasets. Figure 3 shows the density of 'useful' recommendations provided by rules mining and our approach, respectively, across our experimental conditions.