# Attribute Relevance Score: A Novel Measure for Identifying Attribute Importance
## 2 Proposal: Attribute Relevance Score
The ARS algorithm for classification is detailed in Algorithm 1. The primary objective of our measure is to answer two fundamental questions:  
1. Is there a significant dependency between an attribute \(\mathbf{x}\) and another attribute or target variable \(\mathbf{y}\), independent of factors such as cardinality, specific distributions, or artifacts from the data or model configuration? A score of \(\text{ARS}(\mathbf{x})>0\) would indicate such a genuine dependency. 2. How effectively can \(\mathbf{y}\) be approximated based on \(\mathbf{x}\) using a specific error measure and a chosen algorithm? The attribute relevance score (ARS) quantifies the practical capability of \(\mathbf{x}\) to predict or approximate \(\mathbf{y}\), capturing not only the existence of a statistical dependency but also the magnitude of its predictive power in the context of the selected model and performance metric. This ensures that any detected relevance corresponds to a genuine and meaningful predictive capacity, rather than an artifact of the statistical properties of \(\mathbf{x}\) or models based on chance. To illustrate the behavior of the ARS and build intuition, consider the simple equation \(\mathbf{y}=\mathbf{x}_{1}+\mathbf{x}_{2}+\mathbf{x}_{3}\), where \(\mathbf{x}_{i}\) are three independent, nonredundant predictor attributes within the same range. Suppose that, for the first attribute, we obtain \(\overline{F}(\mathcal{M}_{s})+\epsilon=0.5\) and \(\overline{F}(\mathcal{M})=0.75\); for the second attribute, \(\overline{F}(\mathcal{M}_{s})+\epsilon=0.5\) and \(\overline{F}(\mathcal{M})=0.65\); and for the third attribute, \(\overline{F}(\mathcal{M}_{s})+\epsilon=0.5\) and \(\overline{F}(\mathcal{M})=0.6\). Calculating the attribute relevance score (ARS) for each attribute yields scores of 0.5, 0.3, and 0.2, respectively. Notably, in the ideal case, the sum of these scores is 1.0, which suggests that these three independent variables, collectively, have the capacity to fully represent the target variable. This example illustrates how the ARS quantifies the cumulative relevance of independent attributes, capturing the extent to which a set of predictors can collectively account for the target variable. ```
1:Set up model: (e.g., DecisionTreeClassifier)
2:Set up performance metric \(F\): (e.g., \(f_{1}\)-score)
3:Set up acceptable error margin: \(\epsilon\)
4:Set significance level: \(\alpha\) (e.g., \(\alpha=0.05\))
5:Initialize lists to store performance metrics of the original and shadow models
6:for iteration from 1 to \(N\)do
7: Split the data into training and testing sets: \(\mathbf{X}_{\text{train}},\mathbf{X}_{\text{test}},\mathbf{y}_{\text{train}}, \mathbf{y}_{\text{test}}\)
8: Train the original model \(\mathcal{M}\) on \(\mathbf{X}_{\text{train}}\) and \(\mathbf{y}_{\text{train}}\)
9: Generate shadow attributes by permuting the original attributes to obtain \(\mathbf{X}_{\text{shadow}}\)
10: Train the shadow model \(\mathcal{M}_{s}\) on \(\mathbf{X}_{\text{shadow}}\) and \(\mathbf{y}_{\text{train}}\)
11: Evaluate and store the performance metric \(F\) of both models \(\mathcal{M}\) and \(\mathcal{M}_{s}\) on \(\mathbf{X}_{\text{test}},\mathbf{y}_{\text{test}}\)
12:endfor
13: Compute the median performances \(\overline{F}(\mathcal{M})\) and \(\overline{F}(\mathcal{M}_{s})\)
14:Conduct statistical test: Perform a test of means (e.g., a \(t\)-test) on the performance metrics collected from \(\mathcal{M}\) and \(\mathcal{M}_{s}\) to obtain a \(p\)-value
15:if\(p\)-value \(<\alpha\)then
16: Compute the Attribute Relevance Score (ARS) using the following formula: \[\text{ARS}=\text{max}\Bigg{(}0,\frac{\overline{F}(\mathcal{M})-\left( \overline{F}(\mathcal{M}_{s})+\epsilon\right)}{1-\left(\overline{F}(\mathcal{M }_{s})+\epsilon\right)}\Bigg{)}\]
17:else
18: Set \(\text{ARS}=0\)
19:endif
20:return \(\text{ARS}\) and the \(p\)-value from the statistical test
```  
**Algorithm 1** Attribute relevance score for classification.