Original: Cover Letter for SaTML Submission

Original: We thank the reviewers for the time spent in assessing our submission and the numerous, useful comments they provided.

Original: The revised version highlights in red the main changes with respect to the original submission.

Original: This cover letter explains how the changes address the reviewers’ comments.

Original: Reviewer

Original: (Questions For Authors):

Original: In Definition 3, there is no minimality condition on U and thus one could set U= R^d

Original: =

Original: (-infty,+infty)\times…\times(-infty,+infty)_(d times), making the algorithm output a trivial U. Could you please explain why that is?

Original: In our algorithm we just assume that U over-approximates the portion of the feature space where the classifier f is unstable.

Original: The reason is that finding the minimal U may be too computationally expensive, e.g., the algorithm that we use as a black box in our tool returns an over-approximated solution rather than the minimal solution.

Original: We expanded the discussion under Definition 3 to clarify this point.

Original: (Questions For Authors):

Original: In your experiments, how is D_rand generated?

Original: Each instance in D_rand is generated just by assigning a random value to all features, restricting categorical features to a set of plausible values observed in the dataset.

Original: We clarified this point in Section IV.A.

Original: (Requested Changes): Please include the description of GEN_ITEMSETS and discuss the complexity of finding U (I understand that you use the algorithm to find U as a black-box, but it is still helpful to understand its complexity).

Original: We revised the paper to include the definition of GEN_ITEMSETS (see Algorithm 1) and we now mention the computational cost of finding U (in the Implementation section).

Original: Please note that we also included empirical performance estimates of the cost of finding U, as you requested in the next comment.

Original: (Requested Changes): Please include at least one set of experiments evaluating the time required to compute U and how it is affected by the different parameters.

Original: We revised the paper to include the new Appendix C with the requested experiments.

Original: Reviewer

Original: What exactly is the novel contribution in the paper?

Original: A key novel contribution of the paper is the proposal of a global fairness verification method for decision tree ensembles.

Original: As discussed in the introduction and in Section V.B, most of the fairness verification methods in the literature focus on neural networks and prior work by

Original: Ranzato et al.

Original: [9] on decision trees only verifies the weaker notion of local fairness, which predicates just on a specific set of test instances rather than the entire feature space.

Original: Our work does not just allow to verify global fairness (rather than local fairness) for tree-based models, which is already an important contribution in its own rights, but it also includes the explainability dimension, because our algorithm characterizes global fairness in terms of traditional logic formulas ensuring lack of discrimination.

Original: We think that the contribution is already clear from the introduction, however we would certainly be happy to incorporate any additional feedback in the final version of the paper.

Original: How does the approach differ from post-hoc analysis of numerous fairness notions in a possible outcome space?

Original: Post-hoc analysis starts from the decision given on a specific input and tries to explain it, e.g., using factual or counterfactual examples.

Original: Our global fairness analysis instead verifies the classifier itself rather than its behavior on a specific input, synthesizing sufficient fairness conditions predicating over the space of all the possible inputs of the classifier.

Original: This way, our analysis predicates even over unseen inputs, as opposed to post-hoc analyses applied just to specific inputs; in a sense, this boils down to the previous discussion on global fairness vs local fairness.

Original: We extended Section V.C to discuss related work on post-hoc analysis.

Original: Although explainability is introduced as a contribution, it is not discussed in detail in the results.

Original: We respectfully note that our paper already presented a rather detailed discussion on explainability: the experimental evaluation includes a dedicated section “Explainability of the results” (Section IV.C) with several quantitative experiments and a more qualitative case study in Appendix B.

Original: We acknowledge that the evaluation criteria for explainability were not very apparent in the original submission and we expanded Section IV.C to clarify upfront how we assess explainability.

Original: In particular, we claim that our approach is explainable because it is based on logical formulas and only a small number of short formulas is enough to provide useful information.

Original: This is in line with different contributions in the literature that exploit rules based on logical formulas to extract explainable information from classifiers.

Original: In these proposals, the complexity of the logical formula is measured by its length, defined as the number of atoms, and short logical formulas are preferred because of their understandability.

Original: Section IV.C now clarifies this point, including the appropriate bibliographic references.

Original: How would you evaluate the approach if there were numerous sensitive attributes?

Original: Our approach does not make any assumptions about the number of sensitive features, so it can be used also when the set S contains more than one sensitive feature.

Original: This is also supported by our implementation.

Original: We just consider the case |S| = 1 in our experiments for simplicity and because lack of discrimination w.r.t.

Original: gender is a clear and intuitive property for the considered datasets.

Original: We clarified this point in the paper (Section IV.A).

Original: The related work is missing significant directions and areas of research in fairness in machine learning.

Original: There are various explainability methods as well as different types of fairness analysis approaches, especially in representation learning settings.

Original: Given the numerous contributions in the explainability field, we have just included the most closely related work about “fairness explanations” in

Original: Section V.C.

Original: We extended the related work section with other recent papers on the topic and a discussion of post-hoc analysis, as mentioned in a previous point.

Original: Could you maybe give concrete examples with the logic formulas?

Original: How does one assume that the explainability part is human understandable?

Original: Please note that concrete examples of logic formulas for one of the considered dataset were already presented in Appendix B (see Table III).

Original: The appendix also presents qualitative examples of how a human would interpret the results of our tool.

Original: Moreover, we now revised

Original: Section IV.C to clarify why we argue that the formulas are human understandable, i.e., because important formulas are short and limited in number.

Original: The sections that detail the code implementation might be redundant.

Original: Focusing on how conceptually the approach related to discrimination, which categories are feature types of considered and how they would affect the outcomes would be more relevant to the field.

Original: We moved a significant part of the implementation section to the appendix.

Original: This allowed us to recover space to address additional concerns from the reviewers.

Original: We expanded Section

Original: IV.A to motivate the relevance of the considered datasets for discrimination.

Original: Moreover, we extended Section II.B to better position our fairness notion w.r.t.

Original: the existing literature.

Original: We hope that this sufficiently addresses this comment.

Original: Please include more precise empirical evidence to present the contributions in fairness and explainability space.

Original: If the reviewer has additional concrete suggestions on how to improve these aspects, we would be happy to take them into account in the final version of the paper.

Original: Reviewer

Original: Can you provide more detail on your evaluative criteria, what indicates success (and why), and the representativeness of your selected datasets?

Original: The three considered datasets have been used in prior work on fairness verification [9] and they are all associated with classification tasks where fairness matters.

Original: Adult requires predicting yearly income (above or below $50K) and German assigns credit scores (good or bad), hence they could be used to train classifiers deployed, e.g., to assess loan requests.

Original: Health, instead, requires predicting ten-year mortality (above or below the median Charlson index), hence it could be used to train classifiers deployed in the health insurance setting.

Original: In all cases, we do not want to discriminate customers based on their gender, which has been considered as a sensitive feature also in [9].

Original: We now revised Section IV.A to include this information.

Original: We also acknowledge that our evaluation criteria for explainability were not completely apparent and a little bit intermingled with the evaluation itself: thanks for raising the issue, we apologize for that.

Original: Our evaluation criteria for explainability are in line with different contributions in the literature that exploit rules based on logical formulas to extract explainable information from classifiers.

Original: In these proposals, the complexity of the logical formula is measured by their length, defined as the number of atoms, and short logical formulas are preferred because of their understandability.

Original: We take the same route and assess explainability based on the number and complexity of the formulas synthesized by our algorithm.

Original: We revised Section IV.C to clarify this upfront.

Original: Can you specify what fairness definition your work aims to operationalize?

Original: The considered fairness definition is lack of causal discrimination (Definition 1), which was introduced in a seminal paper on fairness testing [14].

Original: This definition is well known and has been considered in prominent surveys on fairness definitions, including [22] and [23].

Original: We acknowledge that fairness is a difficult concept to define and many different notions have been proposed in the literature, so it is easy to get lost.

Original: To better contextualize our work, we clarified in Section II.B that lack of causal discrimination is an “individual fairness” definition (as opposed to “group fairness” definitions), i.e., it provides guarantees for every individual of a sub-population, rather allowing individually unfair behavior to compensate each other across a group of individuals.

Original: Individual fairness and group fairness are the two big families of fairness definitions: they are both prominent and useful concepts, yet may be conflicting and are studied orthogonally (see [23], Section 7.3).

Original: Since individual fairness generally captures the idea that similar individual should have similar predictions, lack of causal discrimination (Definition 1) is a possible, representative instantiation of this intuition that was considered in the literature.

Original: We clarified this point in Section II.B.

Original: Explainable Global Fairness Verification of Tree-Based Classifiers
Revised: Explainable Global Fairness Veriﬁcation of Tree-Based Classiﬁers

Original: Anonymous Authors

Original: Abstract —We present a new approach to the global fairness verification of tree-based classifiers.
Revised: Abstract —We present a new approach to the global fairness veriﬁcation of tree-based classiﬁers.

Original: Given a tree-based classifier and a set of sensitive features potentially leading to discrimina- tion, our analysis synthesizes sufficient conditions for fairness, expressed as a set of traditional propositional logic formulas, which are readily understandable by human experts.
Revised: Given a tree-based classiﬁer and a set of sensitive features potentially leading to discrimina- tion, our analysis synthesizes sufﬁcient conditions for fairness, expressed as a set of traditional propositional logic formulas, which are readily understandable by human experts.

Original: The verified fairness guarantees are global, in that the formulas predicate over all the possible inputs of the classifier, rather than just a few specific test instances.
Revised: The veriﬁed fairness guarantees are global, in that the formulas predicate over all the possible inputs of the classiﬁer, rather than just a few speciﬁc test instances.

Original: Our analysis is formally proved both sound and complete.
Revised: Our analysis is formally proved both sound and complete.

Original: Experimental results on public datasets show that the analysis is precise, explainable to human experts and efficient enough for practical adoption.
Revised: Experimental results on public datasets show that the analysis is precise, explainable to human experts and efﬁcient enough for practical adoption.

Original: I. I NTRODUCTION
Revised: I. I NTRODUCTION

Original: The ever-increasing success of Machine Learning (ML) led to its massive deployment in a range of different settings over the last years.
Revised: The ever-increasing success of Machine Learning (ML) led to its massive deployment in a range of different settings over the last years.

Original: Unfortunately, classic approaches to assess the performance of ML models do not always provide a reliable picture of their effectiveness in so many varied practical deployments.
Revised: Unfortunately, classic approaches to assess the performance of ML models do not always provide a reliable picture of their effectiveness in so many varied practical deployments.

Original: For example, traditional deep learning models with state-of-the-art accuracy may be vulnerable to evasion attacks, i.e., small perturbations of test inputs designed to force prediction errors, thus making their deployment in adversarial settings unfeasible [1], [2].
Revised: For example, traditional deep learning models with state-of-the-art accuracy may be vulnerable to evasion attacks, i e., small perturbations of test inputs designed to force prediction errors, thus making their deployment in adversarial settings unfeasible [1], [2].

Original: Similarly, ML models may turn out to be unfair for automated decision-making.
Revised: Similarly, ML models may turn out to be unfair for automated decision-making.

Original: For example, a commercial recidivism-risk assessment algorithm was found to be racially biased [3] and an existing algorithm adopted in the US falsely determined that black patients were healthier than other patients with similar conditions [4].
Revised: For example, a commercial recidivism-risk assessment algorithm was found to be racially biased [3] and an existing algorithm adopted in the US falsely determined that black patients were healthier than other patients with similar conditions [4].

Original: These incidents led to a proliferation of research on fair ML in recent times, as summarized in different surveys [5], [6], [7].
Revised: These incidents led to a proliferation of research on fair ML in recent times, as summarized in different surveys [5], [6], [7].

Original: Fairness in ML has been analyzed from different angles and can be broadly categorized in two main research lines.
Revised: Fairness in ML has been analyzed from different angles and can be broadly categorized in two main research lines.

Original: The first one includes the development of new ML algorithms that are able to mitigate the bias that is directly or indirectly present in the training data [8], [9], [10].
Revised: The ﬁrst one includes the development of new ML algorithms that are able to mitigate the bias that is directly or indirectly present in the training data [8], [9], [10].

Original: The second, complementary research line investigates techniques to estimate or even formally verify the fairness guarantees provided by existing ML models [11], [12], [13].
Revised: The second, complementary research line investigates techniques to estimate or even formally verify the fairness guarantees provided by existing ML models [11], [12], [13].

Original: This paper contributes to the latter line of work, which is still at an early stage of development and suffers from relevant shortcomings.
Revised: This paper contributes to the latter line of work, which is still at an early stage of development and suffers from relevant shortcomings.

Original: A very popular approach to assess the fairness guarantees of ML models is based on testing [14], [15], [16], [17].
Revised: A very popular approach to assess the fairness guarantees of ML models is based on testing [14], [15], [16], [17].

Original: The key common intuition underlying any fairness testing strategy is straightforward: generating a number of test inputs designed to automatically identify individuals who may suffer from discrimination by the ML model.
Revised: The key common intuition underlying any fairness testing strategy is straightforward: generating a number of test inputs designed to automatically identify individuals who may suffer from discrimination by the ML model.

Original: Unfortunately, as for any testing approach, this type of analysis is under-approximated: these proposals can identify room for unfair treatment, but cannot establish formal fairness proofs.
Revised: Unfortunately, as for any testing approach, this type of analysis is under-approximated: these proposals can identify room for unfair treatment, but cannot establish formal fairness proofs.

Original: This is sub-optimal because it does not allow one to prove that unfair behavior can never affect specific classes of individuals.
Revised: This is sub-optimal because it does not allow one to prove that unfair behavior can never affect speciﬁc classes of individuals.

Original: For this reason, recent papers advocated the adoption of formal fairness verification techniques to prove lack of discrimination [10], [11], [12], [13], [18].
Revised: For this reason, recent papers advocated the adoption of formal fairness veriﬁcation techniques to prove lack of discrimination [10], [11], [12], [13], [18].

Original: Most of the work in the area, however, just focuses on deep neural networks and disregards tree-based ML models, such as decision trees [19] and random forests [20], which are still exceptionally popular, in particular for nonperceptual classification tasks.
Revised: Most of the work in the area, however, just focuses on deep neural networks and disregards tree-based ML models, such as decision trees [19] and random forests [20], which are still exceptionally popular, in particular for nonperceptual classiﬁcation tasks.

Original: The only notable fairness verification approach designed for tree-based classifiers leverages abstract interpretation to verify local fairness properties [9].
Revised: The only notable fairness veriﬁcation approach designed for tree-based classiﬁers leverages abstract interpretation to verify local fairness properties [9].

Original: Unfortunately, local fairness is now recognized as a rather weak property predicating just on specific test instances, while global fairness predicates over all the possible inputs of the classifier and is thus more reliable to assess the actual fairness guarantees that it provides [18].
Revised: Unfortunately, local fairness is now recognized as a rather weak property predicating just on speciﬁc test instances, while global fairness predicates over all the possible inputs of the classiﬁer and is thus more reliable to assess the actual fairness guarantees that it provides [18].

Original: A. Contributions
Revised: A. Contributions

Original: We present a new approach to the global fairness verification of tree-based classifiers.
Revised: We present a new approach to the global fairness veriﬁcation of tree-based classiﬁers.

Original: Given a tree-based classifier and a set of sensitive features potentially leading to discrimination, our analysis synthesizes sufficient conditions for fairness, expressed as a set of traditional propositional logic formulas
Revised: Given a tree-based classiﬁer and a set of sensitive features potentially leading to discrimination, our analysis synthesizes sufﬁcient conditions for fairness, expressed as a set of traditional propositional logic formulas

Original: F predicating over the entire feature space, rather than just on a specific test set, thus providing global fairness guarantees.
Revised: F predicating over the entire feature space, rather than just on a speciﬁc test set, thus providing global fairness guarantees.

Original: Our fairness verification approach is formally proved both sound and complete , i.e., fairness is certified for any instance satisfying some formula in F , and the formulas in F can characterize all the instances where the classifier is fair.
Revised: Our fairness veriﬁcation approach is formally proved both sound and complete , i.e., fairness is certiﬁed for any instance satisfying some formula in F , and the formulas in F can characterize all the instances where the classiﬁer is fair.

Original: Moreover, our approach is explainable , i.e., it is readily understandable by human experts, being based on traditional logic formulas.
Revised: Moreover, our approach is explainable , i.e., it is readily understandable by human experts, being based on traditional logic formulas.

Original: In particular, we empirically show that a small set of simple logic formulas suffices to largely characterize the fairness guarantees provided by the classifier in practice.
Revised: In particular, we empirically show that a small set of simple logic formulas sufﬁces to largely characterize the fairness guarantees provided by the classiﬁer in practice.

Original: This makes our approach particularly appealing for problems like algorithmic hiring, where automated decisions must be carefully audited [21].
Revised: This makes our approach particularly appealing for problems like algorithmic hiring, where automated decisions must be carefully audited [21].

Original: B. Structure of the Paper
Revised: B. Structure of the Paper

Original: The paper is organized as follows:
Revised: The paper is organized as follows:

Original: • Section II reviews the background and introduces the necessary ingredients to appreciate the technical contribution; • Section III presents our fairness verification approach and establishes the formal guarantees provided by our analysis.
Revised: • Section II reviews the background and introduces the necessary ingredients to appreciate the technical contribution; • Section III presents our fairness veriﬁcation approach and establishes the formal guarantees provided by our analysis.

Original: Moreover, it describes the implementation of the analysis in C++.
Revised: Moreover, it describes the implementation of the analysis in C++.

Original: We plan to release the developed software upon paper acceptance; [Figure] • Section IV reports on our experimental evaluation on public datasets.
Revised: The source code of our analyzer is publicly available on Github 1 ; • Section IV reports on our experimental evaluation on public datasets.

Original: Our experiments assess the precision of the analysis, its explainability, and its performance; • Section V presents and compares with related work, thus clarifying the distinctive features of our proposal; • Section VI briefly concludes the paper and describes possible future work in the area.
Revised: Our experiments assess the precision of the analysis, its explainability, and its performance; • Section V presents and compares with related work, thus clarifying the distinctive features of our proposal; • Section VI brieﬂy concludes the paper and describes possible future work in the area.

Original: II. B ACKGROUND
Revised: II. B ACKGROUND

Original: We here introduce the key technical ingredients required to appreciate the contribution of the paper.
Revised: We here introduce the key technical ingredients required to appreciate the contribution of the paper.

Original: A. Tree-Based Classifiers
Revised: A. Tree-Based Classiﬁers

Original: Let X ⊆
Revised: Let X ⊆

Original: R
Revised: R

Original: d be a d -dimensional vector space of features and let Y be a finite set of class labels.
Revised: d be a d -dimensional vector space of features and let Y be a ﬁnite set of class labels.

Original: We assume each element of the feature space ⃗x = ⟨ x 1 , . . .
Revised: We assume each element of the feature space (cid:126)x = (cid:104) x 1 , . . .

Original: , x d ⟩ , called instance , to be assigned a correct class y by an unknown target function g : X → Y .
Revised: , x d (cid:105) , called instance , to be assigned a correct class y by an unknown target function g : X → Y .

Original: A classifier is a function f : X → Y intended to approximate the target function g as accurately as possible.
Revised: A classiﬁer is a function f : X → Y intended to approximate the target function g as accurately as possible.

Original: Normally, f is automatically trained by a supervised learning algorithm, using a training set D train = { ( ⃗x i , g ( ⃗x i )) } i of correctly labeled instances.
Revised: Normally, f is automatically trained by a supervised learning algorithm, using a training set D train = { ( (cid:126)x i , g ( (cid:126)x i )) } i of correctly labeled instances.

Original: The performance of f is then assessed on a separate test set D test =
Revised: The performance of f is then assessed on a separate test set D test =

Original: { ( ⃗z i , g ( ⃗z i )) } i , sampled from the same distribution of the training set.
Revised: { ( (cid:126)z i , g ( (cid:126)z i )) } i , sampled from the same distribution of the training set.

Original: In this paper, we focus on decision tree classifiers [19].
Revised: In this paper, we focus on decision tree classiﬁers [19].

Original: Decision trees can be inductively defined as follows: a decision tree t is either a leaf λ ( y ) for some label y ∈ Y or an internal node σ ( f, v, t l , t r ) , where f ∈ { 1 , . . .
Revised: Decision trees can be inductively deﬁned as follows: a decision tree t is either a leaf λ ( y ) for some label y ∈ Y or an internal node σ ( f, v, t l , t r ) , where f ∈ { 1 , . . .

Original: , d } identifies a feature, v ∈ R is a threshold for the feature, and t l , t r are decision trees (left and right respectively).
Revised: , d } identiﬁes a feature, v ∈ R is a threshold for the feature, and t l , t r are decision trees (left and right respectively).

Original: At test time, the instance ⃗x traverses the tree t until it reaches a leaf λ ( y ) , which returns the prediction y , denoted by t ( ⃗x ) =
Revised: At test time, the instance (cid:126)x traverses the tree t until it reaches a leaf λ ( y ) , which returns the prediction y , denoted by t ( (cid:126)x ) =

Original: y .
Revised: y .

Original: Specifically, for each traversed tree node σ ( f, v, t l , t r ) , ⃗x falls into the left subtree t l if x f ≤ v , and into the right sub-tree t r otherwise.
Revised: Speciﬁcally, for each traversed tree node σ ( f, v, t l , t r ) , (cid:126)x falls into the left subtree t l if x f ≤ v , and into the right sub-tree t r otherwise.

Original: Figure 1 represents an example decision tree of depth two, which assigns the label +1 to the instance ⟨ 10 , 6 ⟩ and the label − 1 to the instance ⟨ 6 , 9 ⟩ .
Revised: Figure 1 represents an example decision tree of depth two, which assigns the label +1 to the instance (cid:104) 10 , 6 (cid:105) and the label − 1 to the instance (cid:104) 6 , 9 (cid:105) .

Original: Decision trees are normally combined into an ensemble T = { t 1 , . . .
Revised: Decision trees are normally combined into an ensemble T = { t 1 , . . .

Original: , t n } to improve their predictive power [20]: in this case, the ensemble prediction
Revised: , t n } to improve their predictive power [20]: in this case, the ensemble prediction

Original: T ( ⃗x ) is computed by combining together the individual tree predictions t i ( ⃗x ) , e.g., by performing majority voting on the individually predicted classes.
Revised: T ( (cid:126)x ) is computed by combining together the individual tree predictions t i ( (cid:126)x ) , e.g., by performing majority voting on the individually predicted classes.

Original: B. Fairness in ML
Revised: B. Fairness in ML

Original: Many definitions of fairness have been proposed in the literature, each with its pros and cons [22].
Revised: Many deﬁnitions of fairness have been proposed in the literature, each with its pros and cons [22].

Original: Existing fairness definitions can be broadly categorized in two main classes: individual fairness definitions, requiring that similar inputs must result in similar outputs, and group fairness definitions, requiring that a particular group of inputs taken as a whole must be treated like a different group.
Revised: Existing fairness deﬁnitions can be broadly categorized in two main classes: individual fairness deﬁnitions, requiring that similar inputs must result in similar outputs, and group fairness deﬁnitions, [Figure] [Figure] requiring that a particular group of inputs taken as a whole must be treated like a different group.

Original: It is now acknowledged that individual fairness and group fairness are both prominent and useful concepts, however they may be conflicting and are normally studied separately [23].
Revised: It is now acknowledged that individual fairness and group fairness are both prominent and useful concepts, however, they may be conﬂicting and are normally studied separately [23].

Original: In this paper, we focus on a specific individual fairness definition known as lack of causal discrimination , which was introduced in a seminal paper on fairness testing [14].
Revised: In this paper, we focus on a speciﬁc individual fairness deﬁnition known as lack of causal discrimination , which was introduced in a seminal paper on fairness testing [14].

Original: We consider this property for several reasons: its intuitive flavour, its popularity in the literature, and its independence from the distribution of the class labels (i.e., the target function), which simplifies its practical application.
Revised: We consider this property for several reasons: its intuitive ﬂavour, its popularity in the literature, and its independence from the distribution of the class labels (i.e., the target function), which simpliﬁes its practical application.

Original: Moreover, lack of causal discrimination is a powerful foundation for fairness, because it does not rely on the choice of a specific test set, but rather predicates over all the possible instances in (a subset of) the feature space X , much in line with recent proposals on the verification of global robustness properties of machine learning models [24], [25], [26].
Revised: Moreover, lack of causal discrimination is a powerful foundation for fairness, because it does not rely on the choice of a speciﬁc test set, but rather predicates over all the possible instances in (a subset of) the feature space X , much in line with recent proposals on the veriﬁcation of global robustness properties of machine learning models [24], [25], [26].

Original: This is important in the fairness setting, because fairness is particularly relevant for minorities for which it might be hard to collect representative data in the test set.
Revised: This is important in the fairness setting, because fairness is particularly relevant for minorities for which it might be hard to collect representative data in the test set.

Original: Indeed, the need for global fairness verification has been recently advocated for neural networks [18].
Revised: Indeed, the need for global fairness veriﬁcation has been recently advocated for neural networks [18].

Original: Intuitively, lack of causal discrimination requires that any two similar inputs must lead the classifier to the same outputs, thus capturing the intuition of individual fairness; more specifically, lack of causal discrimination requires that the classifier returns the same prediction on any two instances differing just for the value of a set of sensitive features S ⊆ { 1 , . . .
Revised: Intuitively, lack of causal discrimination requires that any two similar inputs must lead the classiﬁer to the same outputs, thus capturing the intuition of individual fairness; more specifically, lack of causal discrimination requires that the classiﬁer returns the same prediction on any two instances differing just for the value of a set of sensitive features S ⊆ { 1 , . . .

Original: , d } .
Revised: , d } .

Original: Given an instance ⃗x , we let δ ( ⃗x, S ) stand for the set of the instances differing from ⃗x just for a (possibly empty) subset of the sensitive features S .
Revised: Given an instance (cid:126)x , we let δ ( (cid:126)x, S ) stand for the set of the instances differing from (cid:126)x just for a (possibly empty) subset of the sensitive features S .

Original: For example, if S included just two binary features, then δ ( ⃗x, S ) would include the four instances obtained from ⃗x by setting each of the sensitive features in S to one of their two possible values, while keeping the other features unchanged.
Revised: For example, if S included just two binary features, then δ ( (cid:126)x, S ) would include the four instances obtained from (cid:126)x by setting each of the sensitive features in S to one of their two possible values, while keeping the other features unchanged.

Original: Formally, lack of causal discrimination is then defined as follows.
Revised: Formally, lack of causal discrimination is then deﬁned as follows.

Original: Definition 1 (Causal Discrimination) .
Revised: Deﬁnition 1 (Causal Discrimination) .

Original: Let f : X → Y be a classifier and let S be a set of sensitive features.
Revised: Let f : X → Y be a classiﬁer and let S be a set of sensitive features.

Original: We say that f does not perform causal discrimination on X ′ ⊆
Revised: We say that f does not perform causal discrimination on X (cid:48) ⊆

Original: X if and only if, for every instance ⃗x ∈ X ′ , we have that ∀ ⃗z ∈ δ ( ⃗x, S ) : f ( ⃗z ) =
Revised: X if and only if, for every instance (cid:126)x ∈ X (cid:48) , we have that ∀ (cid:126)z ∈ δ ( (cid:126)x, S ) : f ( (cid:126)z ) =

Original: f ( ⃗x ) .
Revised: f ( (cid:126)x ) .

Original: To exemplify the definition at work, suppose a classifier is used to evaluate the legitimacy of loan requests and that the set of sensitive features S just includes the customer’s gender.
Revised: To exemplify the deﬁnition at work, suppose a classiﬁer is used to evaluate the legitimacy of loan requests and that the set of sensitive features S just includes the customer’s gender.

Original: Lack of causal discrimination on X requires that any two identical customers differing just for their gender are guaranteed to get the same response to their loan requests.
Revised: Lack of causal discrimination on X requires that any two identical customers differing just for their gender are guaranteed to get the same response to their loan requests.

Original: Focusing on a specific X ′ ⊆
Revised: Focusing on a speciﬁc X (cid:48) ⊆

Original: X allows one to make the fairness guarantees conditional and thus more practically useful, e.g., by requiring that any two identical customers with a monthly salary higher than $4,000 are guaranteed to receive the same response to their loan requests, irrespective of their gender.
Revised: X allows one to make the fairness guarantees conditional and thus more practically useful, e.g., by requiring that any two identical customers with a monthly salary higher than $4,000 are guaranteed to receive the same response to their loan requests, irrespective of their gender.

Original: III. G LOBAL F AIRNESS V ERIFICATION
Revised: III. G LOBAL F AIRNESS V ERIFICATION

Original: We here present our new approach to the global fairness verification of tree-based classifiers.
Revised: We here present our new approach to the global fairness veriﬁcation of tree-based classiﬁers.

Original: A. Overview
Revised: A. Overview

Original: Verifying the lack of causal discrimination is challenging, because it involves a universal quantification over a set of instances, possibly drawn from a continuous, unbounded feature space.
Revised: Verifying the lack of causal discrimination is challenging, because it involves a universal quantiﬁcation over a set of instances, possibly drawn from a continuous, unbounded feature space.

Original: Prior work on causal discrimination circumvented this problem by restricting its focus to a finite feature space and assessing fairness by means of a testing approach [14].
Revised: Prior work on causal discrimination circumvented this problem by restricting its focus to a ﬁnite feature space and assessing fairness by means of a testing approach [14].

Original: In particular, this restriction enables the computation of a causal discrimination score , defined as the fraction of instances in the feature space suffering from causal discrimination, a measure which is only meaningful as long as the feature space is finite and instances therein can be exhaustively enumerated.
Revised: In particular, this restriction enables the computation of a causal discrimination score , deﬁned as the fraction of instances in the feature space suffering from causal discrimination, a measure which is only meaningful as long as the feature space is ﬁnite and instances therein can be exhaustively enumerated.

Original: While one can play around this limitation by using binning to discretize the feature space, the testing approach in [14] is not exhaustive for scalability reasons, hence it can only identify counter-examples suffering from causal discrimination, but it cannot prove a lack of causal discrimination.
Revised: While one can play around this limitation by using binning to discretize the feature space, the testing approach in [14] is not exhaustive for scalability reasons, hence it can only identify counter-examples suffering from causal discrimination, but it cannot prove a lack of causal discrimination.

Original: Similar criticisms apply to other more recent proposals on fairness testing [15], [16], [17].
Revised: Similar criticisms apply to other more recent proposals on fairness testing [15], [16], [17].

Original: In the present work, we improve over the state of the art by proposing a new verification technique to formally verify the fairness guarantees supported by tree-based models.
Revised: In the present work, we improve over the state of the art by proposing a new veriﬁcation technique to formally verify the fairness guarantees supported by tree-based models.

Original: In particular, our technique allows one to automatically identify subsets of the feature space where lack of causal discrimination is ensured, rather than just counter-examples suffering from causal discrimination.
Revised: In particular, our technique allows one to automatically identify subsets of the feature space where lack of causal discrimination is ensured, rather than just counter-examples suffering from causal discrimination.

Original: Concretely, we first discuss how one can verify lack of causal discrimination for tree ensembles given a continuous, unbounded subset of the feature space
Revised: Concretely, we ﬁrst discuss how one can verify lack of causal discrimination for tree ensembles given a continuous, unbounded subset of the feature space

Original: X ′ ⊆
Revised: X (cid:48) ⊆

Original: X as input (Section III-B).
Revised: X as input (Section III-B).

Original: We then build on this idea to design an effective algorithm to automatically synthesize sufficient conditions ensuring lack of causal discrimination, expressed as traditional propositional logic formulas, which are readily understandable by human experts (Section III-C).
Revised: We then build on this idea to design an effective algorithm to automatically synthesize sufﬁcient conditions ensuring lack of causal discrimination, expressed as traditional propositional logic formulas, which are readily understandable by human experts (Section III-C).

Original: Our algorithm is iterative and its execution can be safely stopped before convergence to improve performance, without sacrificing soundness, i.e., only sufficient conditions are returned by the algorithm.
Revised: Our algorithm is iterative and its execution can be safely stopped before convergence to improve performance, without sacriﬁcing soundness, i.e., only sufﬁcient conditions are returned by the algorithm.

Original: Moreover, the algorithm is proved complete upon termination, i.e., the logical formulas returned by the algorithm may eventually provide a complete characterization of the fairness guarantees given by the classifier.
Revised: Moreover, the algorithm is proved complete upon termination, i.e., the logic formulas returned by the algorithm may eventually provide a complete characterization of the fairness guarantees given by the classiﬁer.

Original: B. Verification Algorithm
Revised: B. Veriﬁcation Algorithm

Original: The first problem we investigate can be formulated as follows: given a decision tree ensemble T , a set of sensitive features S and a subset of the feature space X ′ ⊆
Revised: The ﬁrst problem we investigate can be formulated as follows: given a decision tree ensemble T , a set of sensitive features S and a subset of the feature space X (cid:48) ⊆

Original: X , verify that T does not perform causal discrimination on X ′ .
Revised: X , verify that T does not perform causal discrimination on X (cid:48) .

Original: To answer this question, we leverage the stability notion from the adversarial machine learning literature [27].
Revised: To answer this question, we leverage the stability notion from the adversarial machine learning literature [27].

Original: The idea underlying stability is that classifiers deployed in adversarial settings should not be fooled by adversarial manipulations of test instances, designed to force the classifier to return wrong predictions (so-called evasion attacks).
Revised: The idea underlying stability is that classiﬁers deployed in adversarial settings should not be fooled by adversarial manipulations of test instances, designed to force the classiﬁer to return wrong predictions (so-called evasion attacks).

Original: This can be enforced by requiring the classifier to be “stable” with respect to adversarial manipulations, i.e., to stick to its original predictions despite such manipulations.
Revised: This can be enforced by requiring the classiﬁer to be “stable” with respect to adversarial manipulations, i.e., to stick to its original predictions despite such manipulations.

Original: For example, malware detectors must not change their predictions as the result of semanticallypreserving adversarial manipulations of malicious software, e.g., if malware replaces a given API call with an equivalent set of instructions, it should still be classified as malware.
Revised: For example, malware detectors must not change their predictions as the result of semanticallypreserving adversarial manipulations of malicious software, e.g., if malware replaces a given API call with an equivalent set of instructions, it should still be classiﬁed as malware.

Original: Formally, stability is defined as follows.
Revised: Formally, stability is deﬁned as follows.

Original: Definition 2 (Stability) .
Revised: Deﬁnition 2 (Stability) .

Original: Given a classifier f : X → Y , an instance ⃗x ∈ X and a set of adversarial manipulations A ( ⃗x ) , f is stable on ⃗x if and only if ∀ 
Revised: Given a classiﬁer f : X → Y , an instance (cid:126)x ∈ X and a set of adversarial manipulations A ( (cid:126)x ) , f is stable on (cid:126)x if and only if ∀ (

Original: ⃗z ∈ A ( ⃗x ) : f ( ⃗z ) =
Revised: cid:126)z ∈ A [Equation]

Original: f ( ⃗x ) .
Revised: [Equation]

Original: The idea of stability generalizes from the adversarial setting to the fairness setting, because lack of causal discrimination requires that arbitrary changes to the sensitive features S must not affect the classifier predictions.
Revised: The idea of stability generalizes from the adversarial setting to the fairness setting, because lack of causal discrimination requires that arbitrary changes to the sensitive features S must not affect the classiﬁer predictions.

Original: A similar observation for a different definition of fairness has been performed in recent independent work [9].
Revised: A similar observation for a different deﬁnition of fairness has been performed in recent independent work [9].

Original: The important point to note, though, is that while stability predicates on a specific instance ⃗x ∈ X , lack of causal discrimination predicates over a potentially unbounded set of instances X ′ ⊆
Revised: The important point to note, though, is that while stability predicates on a speciﬁc instance (cid:126)x ∈ X , lack of causal discrimination predicates over a potentially unbounded set of instances X (cid:48) ⊆

Original: X , hence traditional approaches to stability/fairness verification such as [27] cannot be directly applied to verify lack of causal discrimination on X ′ .
Revised: X , hence traditional approaches to stability/fairness veriﬁcation such as [27] cannot be directly applied to verify lack of causal discrimination on X (cid:48) .

Original: We thus propose to leverage recent solutions for the dataindependent verification of decision tree ensembles [26], [28] to verify lack of causal discrimination in a continuous, unbounded subset of the feature space X ′ .
Revised: We thus propose to leverage recent solutions for the dataindependent veriﬁcation of decision tree ensembles [26], [28] to verify lack of causal discrimination in a continuous, unbounded subset of the feature space X (cid:48) .

Original: These verification approaches are data-independent because they do not analyze the behavior of the ensemble on specific test instances, but they rather analyze the structure of the ensemble to allow the verification of properties predicating over all the instances in (a subset of) the feature space.
Revised: These veriﬁcation approaches are data-independent because they do not analyze the behavior of the ensemble on speciﬁc test instances, but they rather analyze the structure of the ensemble to allow the veriﬁcation of properties predicating over all the instances in (a subset of) the feature space.

Original: Concretely, data-independent analyses operate in terms of a set of hyper-rectangles H ⊆
Revised: Concretely, data-independent analyses operate in terms of a set of hyper-rectangles H ⊆

Original: X such that all instances ⃗x ∈ H satisfy a property of interest, e.g., lead to the same prediction or enjoy stability.
Revised: X such that all instances (cid:126)x ∈ H satisfy a property of interest, e.g., lead to the same prediction or enjoy stability.

Original: In particular, we build on the following definition of data-independent stability analysis for our verification purposes.
Revised: In particular, we build on the following deﬁnition of data-independent stability analysis for our veriﬁcation purposes.

Original: Definition 3 (Data-Independent Stability Analysis) .
Revised: Deﬁnition 3 (Data-Independent Stability Analysis) .

Original: Let f :
Revised: Let f :

Original: X → Y be a classifier and let S be a set of sensitive features.
Revised: X → Y be a classiﬁer and let S be a set of sensitive features.

Original: A data-independent stability analysis is an algorithm which takes as input f and S to return as output a set of hyper-rectangles
Revised: A data-independent stability analysis is an algorithm which takes as input f and S to return as output a set of hyper-rectangles

Original: U satisfying the following property: for every instance ⃗x ∈ X , if there exists ⃗z ∈ δ ( ⃗x, S ) such that f ( ⃗z ) ̸
Revised: U satisfying the following property: for every instance (cid:126)x ∈ X , if there exists (cid:126)z ∈ δ ( (cid:126)x, S ) such that f ( (cid:126)z ) (cid:54) = f

Original: = f ( ⃗x ) , then there exists H ∈ U such that ⃗x ∈ H .
Revised: ( (cid:126)x ) , then there exists H ∈ U such that (cid:126)x ∈ H .

Original: Intuitively, the output of a data-independent stability analysis over-approximates the subset of the feature space where the classifier violates stability for some instances therein, i.e., if ⃗x suffers from causal discrimination, then the result of the analysis must include a hyper-rectangle H such that ⃗x ∈ H .
Revised: Intuitively, the output of a data-independent stability analysis over-approximates the subset of the feature space where the classiﬁer violates stability for some instances therein, i.e., if (cid:126)x suffers from causal discrimination, then the result of the analysis must include a hyper-rectangle H such that (cid:126)x ∈ H .

Original: In the fairness setting, this means that the union of the hyperrectangles returned by the analysis over-approximates the set of the counter-examples suffering from causal discrimination.
Revised: In the fairness setting, this means that the union of the hyperrectangles returned by the analysis over-approximates the set of the counter-examples suffering from causal discrimination.

Original: Note that we only require an over-approximation rather than an exact characterization, because this is a minimal assumption for sound fairness verification and data-independent analyses might be approximated by design for scalability reasons.
Revised: Note that we only require an over-approximation rather than an exact characterization, because this is a minimal assumption for sound fairness veriﬁcation and data-independent analyses might be approximated by design for scalability reasons.

Original: For example, the stability analysis in [26] identifies a superset of the portion of the feature space where the classifier is unstable, because robustness verification for tree ensembles (an NP-hard problem [29]) can be reduced to stability analysis.
Revised: For example, the stability analysis in [26] identiﬁes a superset of the portion of the feature space where the classiﬁer is unstable, because robustness veriﬁcation for tree ensembles (an NP-hard problem [29]) can be reduced to stability analysis.

Original: If a data-independent stability analysis is available, solving our problem of interest is straightforward.
Revised: If a data-independent stability analysis is available, solving our problem of interest is straightforward.

Original: In particular, given a decision tree ensemble T and a set of sensitive features S , we can run the data-independent stability analysis on T and
Revised: In particular, given a decision tree ensemble T and a set of sensitive features S , we can run the data-independent stability analysis on T and

Original: S to produce the set of hyper-rectangles U .
Revised: S to produce the set of hyper-rectangles U .

Original: To prove that T does not perform causal discrimination on X ′ ⊆
Revised: To prove that T does not perform causal discrimination on X (cid:48) ⊆

Original: X , it suffices to show that there exists
Revised: X , it sufﬁces to show that there exists

Original: no H ∈
Revised: no H ∈

Original: U such that H ∩ X ′ ̸ = ∅ .
Revised: U such that H ∩ X (cid:48) (cid:54) = ∅ .

Original: We now discuss how the verification approaches in [26], [28] can be used to implement a data-independent stability analysis as required by our definition, thus allowing us to take advantage of existing state-of-the-art verification approaches.
Revised: We now discuss how the veriﬁcation approaches in [26], [28] can be used to implement a data-independent stability analysis as required by our deﬁnition, thus allowing us to take advantage of existing state-of-the-art veriﬁcation approaches.

Original: 1) Resilience Verifier [26]: Recent work on adversarial machine learning introduced the resilience notion to characterize the security of classifiers deployed in adversarial settings.
Revised: 1) Resilience Veriﬁer [26]: Recent work on adversarial machine learning introduced the resilience notion to characterize the security of classiﬁers deployed in adversarial settings.

Original: At the core of its resilience verification algorithm, there is a data-independent stability analysis for decision tree ensembles, which can be readily leveraged for our purposes by modeling an attacker such that, for all instances ⃗x ∈ X , A ( ⃗x ) =
Revised: At the core of its resilience veriﬁcation algorithm, there is a data-independent stability analysis for decision tree ensembles, which can be readily leveraged for our purposes by modeling an attacker such that, for all instances (cid:126)x ∈ X , A ( (cid:126)x ) =

Original: δ ( ⃗x, S ) .
Revised: δ ( (cid:126)x, S ) .

Original: Intuitively, this attacker has full control of the value of the sensitive features S , so stability against such an attacker ensures that causal discrimination is not possible.
Revised: Intuitively, this attacker has full control of the value of the sensitive features S , so stability against such an attacker ensures that causal discrimination is not possible.

Original: The output of the analysis is a set of hyper-rectangles over-approximating the subset of the feature space where the decision tree ensemble is unstable, thus satisfying the conditions of Definition 3.
Revised: The output of the analysis is a set of hyper-rectangles over-approximating the subset of the feature space where the decision tree ensemble is unstable, thus satisfying the conditions of Deﬁnition 3.

Original: 2) VoTE Checker [28]: The VoTE Checker takes as input a decision tree ensemble and produces as output a partitioning of the feature space X in terms of the equivalence classes induced by the ensemble.
Revised: 2) VoTE Checker [28]: The VoTE Checker takes as input a decision tree ensemble and produces as output a partitioning of the feature space X in terms of the equivalence classes induced by the ensemble.

Original: Each equivalence class is represented as a pair ( H, y ) , meaning that all the instances ⃗x ∈ H traverse the same path combination in the ensemble leading to prediction y .
Revised: Each equivalence class is represented as a pair ( H, y ) , meaning that all the instances (cid:126)x ∈ H traverse the same path combination in the ensemble leading to prediction y .

Original: If there exist two equivalence classes ( H 1 , y 1 ) , ( H 2 , y 2 ) such that: ( i ) y 1 ̸
Revised: If there exist two equivalence classes ( H 1 , y 1 ) , ( H 2 , y 2 ) such that: ( i ) y 1 (cid:54) =

Original: = y 2 , ( ii ) H 1 , H 2 have a non-empty intersection for all the features j ̸∈ S , and ( iii ) H 1 , H 2 have different intervals for some feature k ∈ S , then H 1 ∪ H 2 includes a counter-example suffering from causal discrimination, because it contains two instances differing just for some sensitive feature k where the classifier returns two different predictions.
Revised: y 2 , ( ii ) H 1 , H 2 have a non-empty intersection for all the features j (cid:54)∈ S , and ( iii ) H 1 , H 2 have different intervals for some feature k ∈ S , then H 1 ∪ H 2 includes a counter-example suffering from causal discrimination, because it contains two instances differing just for some sensitive feature k where the classiﬁer returns two different predictions.

Original: It is thus possible to add both H 1 and H 2 to the set of hyperrectangles to be returned by the analysis, thus satisfying the conditions of Definition 3.C. Synthesis Algorithm
Revised: It is thus possible to add both H 1 and H 2 to the set of hyperrectangles to be returned by the analysis, thus satisfying the conditions of Deﬁnition 3.

Original: We now consider a different, more interesting problem: given a decision tree ensemble T and a set of sensitive features
Revised: We now consider a different, more interesting problem: given a decision tree ensemble T and a set of sensitive features

Original: S , characterize the subset of the feature space X ′ ⊆
Revised: S , characterize the subset of the feature space X (cid:48) ⊆

Original: X such that T does not perform causal discrimination on X ′ .
Revised: X such that T does not perform causal discrimination on X (cid:48) .

Original: Intuitively, this problem can be conservatively solved by subtracting from X
Revised: Intuitively, this problem can be conservatively solved by subtracting from X

Original: all the hyper-rectangles H ∈ U returned by the data-independent stability analysis assumed in the previous section, thus under-approximating the subset of the feature space where T is stable.
Revised: all the hyper-rectangles H ∈ U returned by the data-independent stability analysis assumed in the previous section, thus under-approximating the subset of the feature space where T is stable.

Original: This approach would be sound, but also computationally inefficient, because subtracting hyper-rectangles suffers from an exponential blowup with respect to the dimensionality of the feature space.
Revised: This approach would be sound, but also computationally inefﬁcient, because subtracting hyper-rectangles suffers from an exponential blowup with respect to the dimensionality of the feature space.

Original: Indeed, given two hyper-rectangles with d features, their subtraction might generate O ( d ) hyper-rectangles in the general case, leading to O ( d | U | ) hyper-rectangles in the worst case at the end of the subtraction process.
Revised: Indeed, given two hyper-rectangles with d features, their subtraction might generate O ( d ) hyper-rectangles in the general case, leading to O ( d | U | ) hyper-rectangles in the worst case at the end of the subtraction process.

Original: This limitation can be circumvented by avoiding the computation of the subtraction and by directly reasoning in terms of instances falling out from the hyper-rectangles U .
Revised: This limitation can be circumvented by avoiding the computation of the subtraction and by directly reasoning in terms of instances falling out from the hyper-rectangles U .

Original: However, this would make it hard to characterize X ′ in a human-understandable way, due to both the sheer number of the hyper-rectangles and the potentially large dimensionality of the feature space.
Revised: However, this would make it hard to characterize X (cid:48) in a human-understandable way, due to both the sheer number of the hyper-rectangles and the potentially large dimensionality of the feature space.

Original: We thus propose an iterative algorithm designed to incrementally generate increasingly complex sufficient conditions ensuring lack of causal discrimination, expressed as traditional logic formulas.
Revised: We thus propose an iterative algorithm designed to incrementally generate increasingly complex sufﬁcient conditions ensuring lack of causal discrimination, expressed as traditional logic formulas.

Original: This way, the first iterations of the algorithm can efficiently generate conditions which are short and easy to understand for human experts, while being arguably the most useful to analysts; the more analysis time and computational resources are available, the more complex conditions can be identified, thus detecting other subsets of the feature space where lack of causal discrimination is ensured.
Revised: This way, the ﬁrst iterations of the algorithm can efﬁciently generate conditions that are short and easy to understand for human experts, while being arguably the most useful to analysts; the more analysis time and computational resources are available, the more complex conditions can be identiﬁed, thus detecting other subsets of the feature space where lack of causal discrimination is ensured.

Original: In other words, each iteration of the algorithm extends the sound approximation identified by the previous iteration by accounting for more complicated sufficient conditions for fairness, until the whole relevant subset of the feature space is covered by the conditions (or an early stopping criterion is met).
Revised: In other words, each iteration of the algorithm extends the sound approximation identiﬁed by the previous iteration by accounting for more complicated sufﬁcient conditions for fairness, until the whole relevant subset of the feature space is covered by the conditions (or an early stopping criterion is met).

Original: 1) Overview:
Revised: 1) Overview:

Original: Our algorithm is inspired by the classic
Revised: Our algorithm is inspired by the classic

Original: Apriori algorithm for frequent itemset mining [30].
Revised: Apriori algorithm for frequent itemset mining [30].

Original: We define an item i as a formula of the form x f ≤ v or x f > v , where f ∈ { 1 , . . .
Revised: We deﬁne an item i as a formula of the form x f ≤ v or x f > v , where f ∈ { 1 , . . .

Original: , d } identifies a feature and v ∈ R .
Revised: , d } identiﬁes a feature and v ∈ R .

Original: An itemset
Revised: An itemset

Original: I is a set of formulas { i 1 , . .
Revised: I is a set of formulas { i 1 , . .

Original: , i n } , interpreted in conjunctive form.
Revised: , i n } , interpreted in conjunctive form.

Original: For example, the itemset I = { x 1 > 1 , x 1 ≤ 3 , x 2 > 5 } identifies all the instances such that the first feature is in the interval (1 , 3] and the second feature is in the interval (5 , + ∞ ) .
Revised: For example, the itemset I = { x 1 > 1 , x 1 ≤ 3 , x 2 > 5 } identiﬁes all the instances such that the ﬁrst feature is in the interval (1 , 3] and the second feature is in the interval (5 , + ∞ ) .

Original: Formally, we write (cid:74) i (cid:75) for the set of the instances identified by item i and we let (cid:74) I (cid:75)
Revised: Formally, we write (cid:74) i (cid:75) for the set of the instances identiﬁed by item i and we let (cid:74) I (cid:75)

Original: = (cid:84)
Revised: = (cid:84)

Original: i ∈ I (cid:74) i (cid:75)
Revised: i ∈ I (cid:74) i (cid:75)

Original: stand for the set of the instances identified by the itemset I .
Revised: stand for the set of the instances identiﬁed by the itemset I .

Original: Given a hyper-rectangle
Revised: Given a hyper-rectangle

Original: H , we denote with to itemset ( H ) its itemset representation, i.e., the itemset I such that (cid:74) I (cid:75)
Revised: H , we denote with to itemset ( H ) its itemset representation, i.e., the itemset I such that (cid:74) I (cid:75)

Original: =
Revised: =

Original: H .
Revised: H .

Original: An itemset identifies a subset of the feature space which is assessed for lack of causal discrimination, thus allowing us to leverage the following monotonicity property to prune the search space: if I enjoys lack of causal discrimination , then any other I ′ ⊇
Revised: An itemset identiﬁes a subset of the feature space which is assessed for lack of causal discrimination, thus allowing us to leverage the following monotonicity property to prune the search space: if I enjoys lack of causal discrimination, then any other I (cid:48) ⊇

Original: I does that as well, because (cid:74) I ′ (cid:75) ⊆
Revised: I does that as well, because (cid:74) I (cid:48) (cid:75) ⊆

Original: (cid:74) I (cid:75) ; hence, I ′ itself does not need to be analyzed.
Revised: (cid:74) I (cid:75) ; hence, I (cid:48) itself does not need to be analyzed.

Original: This property allows one to assess itemsets for lack of causal discrimination by starting from those with smaller cardinality.
Revised: This property allows one to assess itemsets for lack of causal discrimination by starting from those with smaller cardinality.

Original: To exemplify how the algorithm works at a high level, consider a two-dimensional feature space and assume U contains just two hyper-rectangles H 1 = ⟨ (1 , 5] , (3 , 8] ⟩ and
Revised: To exemplify how the algorithm works at a high level, consider a two-dimensional feature space and assume U contains just two hyper-rectangles H 1 = (cid:104) (1 , 5] , (3 , 8] (cid:105) and

Original: H 2 = ⟨ (4 , 7] , (2 , 6] ⟩ .
Revised: H 2 = (cid:104) (4 , 7] , (2 , 6] (cid:105) .

Original: Our goal is characterizing those instances falling outside both H 1 and H 2 .
Revised: Our goal is characterizing those instances falling outside both H 1 and H 2 .

Original: Instances laying outside Hcan be described as the union of the following four hyperrectangles:
Revised: Instances laying outside Hcan be described as the union of the following four hyperrectangles:

Original: • H 11 = ⟨ ( −∞ , 1] , ( −∞ , + ∞ ) ⟩ , represented as the itemset I 11 = to itemset ( H 11 ) = { x 1 ≤ 1 } • H 12 = ⟨ (5 , + ∞ ) , ( −∞ , + ∞ ) ⟩ , represented as the itemset I 12 = to itemset ( H 12 ) = { x 1 > 5 } • H 13 = ⟨ ( −∞ , + ∞ ) , ( −∞ , 3] ⟩ , represented as the itemset I 13 = to itemset ( H 13 ) = { x 2 ≤ 3 } • H 14 = ⟨ ( −∞ , + ∞ ) , (8 , + ∞ ) ⟩ , represented as the itemset I 14 = to itemset ( H 14 ) = { x 2 > 8 }
Revised: • H 11 = (cid:104) ( −∞ , 1] , ( −∞ , + ∞ ) (cid:105) , represented as the itemset I 11 = to itemset ( H 11 ) = { x 1 ≤ 1 } • H 12 = (cid:104) (5 , + ∞ ) , ( −∞ , + ∞ ) (cid:105) , represented as the itemset I 12 = to itemset ( H 12 ) = { x 1 > 5 } • H 13 = (cid:104) ( −∞ , + ∞ ) , ( −∞ , 3] (cid:105) , represented as the itemset I 13 = to itemset ( H 13 ) = { x 2 ≤ 3 } • H 14 = (cid:104) ( −∞ , + ∞ ) , (8 , + ∞ ) (cid:105) , represented as the itemset I 14 = to itemset ( H 14 ) = { x 2 > 8 }

Original: Instances laying outside H 2 can instead be described as the union of the following four hyper-rectangles:
Revised: Instances laying outside H 2 can instead be described as the union of the following four hyper-rectangles:

Original: • H 21 = ⟨ ( −∞ , 4] , ( −∞ , + ∞ ) ⟩ , represented as the itemset I 21 = to itemset ( H 21 ) = { x 1 ≤ 4 } • H 22 = ⟨ (7 , + ∞ ) , ( −∞ , + ∞ ) ⟩ , represented as the itemset I 22 = to itemset ( H 22 ) = { x 1 > 7 } • H 23 = ⟨ ( −∞ , + ∞ ) , ( −∞ , 2] ⟩ , represented as the itemset I 23 = to itemset ( H 23 ) =
Revised: • H 21 = (cid:104) ( −∞ , 4] , ( −∞ , + ∞ ) (cid:105) , represented as the itemset I 21 = to itemset ( H 21 ) = { x 1 ≤ 4 } • H 22 = (cid:104) (7 , + ∞ ) , ( −∞ , + ∞ ) (cid:105) , represented as the itemset I 22 = to itemset ( H 22 ) = { x 1 > 7 } • H 23 = (cid:104) ( −∞ , + ∞ ) , ( −∞ , 2] (cid:105) , represented as the itemset I 23 = to itemset ( H 23 ) =

Original: { x 2 ≤ 2 } • H 24 = ⟨ ( −∞ , + ∞ ) , (6 , + ∞ ) ⟩ , represented as the itemset I 24 = to itemset ( H 24 )=
Revised: { x 2 ≤ 2 } • H 24 = (cid:104) ( −∞ , + ∞ ) , (6 , + ∞ ) (cid:105) , represented as the itemset I 24 = to itemset ( H 24 )

Original:  { x 2 > 6 }
Revised: = { x 2 > 6 }

Original: To identify instances laying outside both H 1 and H 2 , we first inspect all the itemsets above and we check whether they identify subsets of the feature space intersecting H 1 or H 2 .
Revised: To identify instances laying outside both H 1 and H 2 , we ﬁrst inspect all the itemsets above and we check whether they identify subsets of the feature space intersecting H 1 or H 2 .

Original: If we do not find overlaps, the itemsets already represent instances falling out both H 1 and H 2 , hence causal discrimination cannot happen there.
Revised: If we do not ﬁnd overlaps, the itemsets already represent instances falling out both H 1 and H 2 , hence causal discrimination cannot happen there.

Original: In our example, we identify the itemsets I 11 , I 14 , I 

Revised: In our example, we identify the itemsets I 11 , I 14 , I

Original: and I 23 as sufficient conditions for lack of causal discrimination; note that the first and the third itemsets only involve the feature x 1 , while the second and the fourth itemsets only involve the feature x 2 .
Revised: and I 23 as sufﬁcient conditions for lack of causal discrimination; note that the ﬁrst and the third itemsets only involve the feature x 1 , while the second and the fourth itemsets only involve the feature x 2 .

Original: The other itemsets I 12 ,
Revised: The other itemsets I 12 ,

Original: I 13 , I 21 and I 24 , instead, identify subsets of the feature space where causal discrimination might potentially happen.
Revised: I 13 , I 21 and I 24 , instead, identify subsets of the feature space where causal discrimination might potentially happen.

Original: These itemsets are thus combined together to generate additional itemsets to check, each possibly using both features x 1 and x 2 , leading to conditions of higher complexity.
Revised: These itemsets are thus combined together to generate additional itemsets to check, each possibly using both features x 1 and x 2 , leading to conditions of higher complexity.

Original: For example,
Revised: For example,

Original: I 

Revised: I

Original: and I 24 generate the new itemset { x 1 > 5 , x 2 > 6 } , which represents again instances falling out both H 1 and
Revised: and I 24 generate the new itemset { x 1 > 5 , x 2 > 6 } , which represents again instances falling out both H 1 and

Original: H 2 , thus identifying sufficient conditions for lack of causal discrimination.
Revised: H 2 , thus identifying sufﬁcient conditions for lack of causal discrimination.

Original: Instead, I 

Revised: Instead, I

Original: and I 13 generate the new itemset { x 1 > 5 , x 2 ≤ 3 } , which overlaps with H 2 , hence it identifies a subset of the feature space where causal discrimination may happen.
Revised: and I 13 generate the new itemset { x 1 > 5 , x 2 ≤ 3 } , which overlaps with H 2 , hence it identiﬁes a subset of the feature space where causal discrimination may happen.

Original: This itemset may thus be combined with other itemsets to undergo further refinements over x 1 and x 2 , leading to smaller intervals on them, possibly identifying additional sufficient conditions for proving lack of causal discrimination.
Revised: This itemset may thus be combined with other itemsets to undergo further reﬁnements over x 1 and x 2 , leading to smaller intervals on them, possibly identifying additional sufﬁcient conditions for proving lack of causal discrimination.

Original: 2) Algorithm: Having defined the key intuitions of our proposal, we are now ready to present the details of the synthesis algorithm
Revised: 2) Algorithm: Having deﬁned the key intuitions of our proposal, we are now ready to present the details of the synthesis algorithm

Original: (Algorithm 1).
Revised: (Algorithm 1).

Original: The algorithm takes as input a tree ensemble T and a set of sensitive features S to return as output a set of sufficient conditions on the feature space ensuring lack of causal discrimination.
Revised: The algorithm takes as input a tree ensemble T and a set of sensitive features S to return as output a set of sufﬁcient conditions on the feature space ensuring lack of causal discrimination.

Original: The algorithm starts by invoking the A NALYZE function over T and S , which implements a data-independent stability analysis (cf.
Revised: The algorithm starts by invoking the A NALYZE function over T and S , which implements a data-independent stability analysis (cf.

Original: Definition 3) returning a set of hyper-rectangles U where T may be unstable (line 2).
Revised: Deﬁnition 3) returning a set of hyper-rectangles U where T may be unstable (line 2).

Original: The algorithm then initializes an empty set of fairness conditions F and generates a set of candidates C , initialized with itemsets involving just a single feature, as described in our example; this step is implemented by the G EN I TEMSETS function (lines 3-4).
Revised: The algorithm then initializes an empty set of fairness conditions F and generates a set of candidates C , initialized with itemsets involving just a single feature, as described in our example; this step is implemented by the G EN I TEMSETS function (lines 3-4).

Original: The definition of G EN I TEMSETS is also reported in Algorithm 1; it implements the simple intuition from our previous example to first construct a hyper-rectangle
Revised: The deﬁnition of G EN I TEMSETS is also reported in Algorithm 1; it implements the simple intuition from our previous example to ﬁrst construct a hyper-rectangle

Original: H ′ constraining just a single feature i and then convert it into an equivalent itemset.
Revised: H (cid:48) constraining just a single feature i and then convert it into an equivalent itemset.

Original: In the first loop of the algorithm, each itemset I ∈ C is then checked against U : if (cid:74) I (cid:75) does not intersect any hyper-rectangle H ∈ U , the itemset identifies a sufficient condition for lack of causal discrimination, hence it is added to the set of fairness conditions F (lines 5-7).
Revised: In the ﬁrst loop of the algorithm, each itemset I ∈ C is then checked against U : if (cid:74) I (cid:75) does not intersect any hyper-rectangle H ∈ U , the itemset identiﬁes a sufﬁcient condition for lack of causal discrimination, hence it is added to the set of fairness conditions F (lines 5-7).

Original: The itemsets which do not immediately contribute to extending F are instead used in the main loop of the algorithm (lines 8-19).
Revised: The itemsets which do not immediately contribute to extending F are instead used in the main loop of the algorithm (lines 8-19).

Original: In particular, all such itemsets are combined with each other through a meet operator ⊓ to produce new itemsets to analyze; such itemsets can either be proved fair or undergo additional refinements at later iterations, as long as there are candidates to process.
Revised: In particular, all such itemsets are combined with each other through a meet operator (cid:117) to produce new itemsets to analyze; such itemsets can either be proved fair or undergo additional reﬁnements at later iterations, as long as there are candidates to process.

Original: The meet operator is defined and commented below.
Revised: The meet operator is deﬁned and commented below.

Original: Definition 4 (Meet Operator) .
Revised: Deﬁnition 4 (Meet Operator) .

Original: Given two itemsets I 1 , I 2 such that | I 1 | = | I 2 | =
Revised: Given two itemsets I 1 , I 2 such that | I 1 | = | I 2 | =

Original: k and | I 1 ∩ I 2 | = k − 1 , we define their meet I 1 ⊓ I 2 as the itemset I 
Revised: k and | I 1 ∩ I 2 | = k − 1 , we deﬁne their meet I (cid:117) I 2 as the itemset I =

Original: = I 1 ∪ I 2 , provided that the following conditions hold:
Revised: I 1 ∪ I 2 , provided that the following conditions hold:

Original: 1) (cid:74) I (cid:75) ̸ = ∅ , i .e
Revised: 1) (cid:74) I (cid:75) (cid:54) = ∅ , i.e., the itemset

Original: ., the itemset I identifies a non-empty subset of instances; 2) (cid:74) I (cid:75)
Revised: I identiﬁes a non-empty subset of instances; 2) (cid:74) I (cid:75)

Original: ⊂ (cid:74)
Revised: ⊂ (cid:74)

Original: I 1 (cid:75) and (cid:74) I (cid:75)
Revised: I 1 (cid:75) and (cid:74) I (cid:75)

Original: ⊂ (cid:74)
Revised: ⊂ (cid:74)

Original: I 2 (cid:75) , i.e., the itemset I identifies less instances than both I 

Revised: I 2 (cid:75) , i e., the itemset I identiﬁes less instances than both I

Original: and I 2 .
Revised: and I 2 .

Original: If any of the aforementioned conditions do not hold, the meet I does not exist.
Revised: If any of the aforementioned conditions do not hold, the meet I does not exist.

Original: Observe that, given two itemsets I 1 , I 2 of cardinality k sharing k − 1 elements, their meet I 1 ⊓ I 2 produces a new itemset I 1 ∪ I 2 of cardinality k +1 , i.e., itemsets are generated in increasing order of cardinality to leverage the discussed monotonicity property.
Revised: Observe that, given two itemsets I 1 , I 2 of cardinality k sharing k − 1 elements, their meet I 1 (cid:117) I 2 produces a new itemset I 1 ∪ I 2 of cardinality k +1 , i.e., itemsets are generated in increasing order of cardinality to leverage the discussed monotonicity property.

Original: The two technical conditions of Definition 4 just ensure that testing the newly generated itemset might be useful, i.e., the new itemset is non-empty and differs from the previously generated itemsets I 1 , I 2 .
Revised: The two technical conditions of Definition 4 just ensure that testing the newly generated itemset might be useful, i.e., the new itemset is non-empty and differs from the previously generated itemsets I 1 , I 2 .

Original: Although these conditions are formulated in a declarative style to simplify their understanding, the meet operator ⊓ is straightforward to implement in practice.
Revised: Although these conditions are formulated in a declarative style to simplify their understanding, the meet operator (cid:117) is straightforward to implement in practice.

Original: In particular, let i ∗ be the (only) item such that i ∗ ∈
Revised: In particular, let i ∗ be the (only) item such that i ∗ ∈

Original: I 2 \ I 1 and let f ∗ be the feature predicated upon by i ∗ , then we let I 1 [-⊓ I 
-]
Revised: I 2 \ I 1 and let f ∗ be the feature predicated upon by i ∗ , then we let I 1 [-⊓ I 
-](cid:117)

Original: ={+I 
+}
Revised: {+I 
+}

Original:  I 1 ∪ { i ∗ } provided that the two conditions of the definition are satisfied.
Revised: = I 1 ∪ { i ∗ } provided that the two conditions of the deﬁnition are satisﬁed.

Original: The implementation of the first condition checks whether all the items i ∈Ipredicating on f ∗ have a non-empty intersection with i ∗ .
Revised: The implementation of the ﬁrst condition checks whether all the items i ∈

Original: The implementation of the second condition, instead, amounts to verifying that adding i ∗ to I1 identifies a smaller interval for the feature f ∗ . There is just one point left to discuss, which is the key observation that not all the itemsets must undergo the po-
Revised: I

Original: 1: function S YNTHESIZE ( T, S ) 2:
Revised: 1: function S YNTHESIZE ( T, S ) 2:

Original: U ← A NALYZE ( T, S ) 3:
Revised: U ← A NALYZE ( T, S ) 3:

Original: F ← ∅ 4:
Revised: F ← ∅ 4:

Original: C ← G EN I TEMSETS ( U ) 5: for I ∈ C do 6:
Revised: C ← G EN I TEMSETS ( U ) 5: for I ∈ C do 6:

Original: if ∀ H ∈ U : (cid:74) I (cid:75)
Revised: if ∀ H ∈ U : (cid:74) I (cid:75)

Original: ∩ H = ∅
Revised: ∩ H = ∅

Original: then 7:
Revised: then 7:

Original: F ← F ∪ { I } 8:
Revised: F ← F ∪ { I } 8:

Original: C ← C \ F 9: while C ̸ = ∅ do 10:
Revised: C ← C \ F 9: while C (cid:54) = ∅ do 10:

Original: C ′ ← ∅ 11: for I 1 ∈ C do 12: for I 2 ∈ C do 13:
Revised: C (cid:48) ← ∅ 11: for I 1 ∈ C do 12: for I 2 ∈ C do 13:

Original: I ← I 1 ⊓ I 2 14: if I exists ∧ ̸ ∃ I ′ ∈ F : (cid:74) I (cid:75) ⊆
Revised: I ← I 1 (cid:117) I 2 14: if I exists ∧ (cid:54) ∃ I (cid:48) ∈ F : (cid:74) I (cid:75) ⊆

Original: (cid:74) I ′ (cid:75) then 15: if ∀ H ∈
Revised: (cid:74) I (cid:48) (cid:75) then 15: if ∀ H ∈

Original: U : (cid:74) I (cid:75)
Revised: U : (cid:74) I (cid:75)

Original: ∩ H = ∅
Revised: ∩ H = ∅

Original: then 16:
Revised: then 16:

Original: F ← F ∪ { I } 17: else 18:
Revised: F ← F ∪ { I } 17: else 18:

Original: C ′ ← C ′ ∪ { I } 19:
Revised: C (cid:48) ← C (cid:48) ∪ { I } 19:

Original: C ← C ′ 20: return F 21: 22: function G EN I TEMSETS ( U ) 23:
Revised: C ← C (cid:48) 20: return F 21: 22: function G EN I TEMSETS ( U ) 23:

Original: C ← ∅ 24: for H = ⟨ H 1 , . ..
Revised: C ← ∅ 24: for H = (cid:104) H 1 , . .

Original: , H d ⟩ ∈ U do 25: for i ∈ { 1 , . . .
Revised: , H d (cid:105) ∈ U do 25: for i ∈ { 1 , . . .

Original: , d } do 26:
Revised: , d } do 26:

Original: H i ← ( −∞ , + ∞ ) \ H i 27: for intv ∈ H i do 28:
Revised: H i ← ( −∞ , + ∞ ) \ H i 27: for intv ∈ H i do 28:

Original: H ′ ← ⟨ ( −∞ , + ∞ ) i − 1 , intv, ( −∞ , + ∞ )
Revised: H (cid:48) ← (cid:104) ( −∞ , + ∞ ) i − 1 , intv, ( −∞ , + ∞ )

Original: d − i ⟩ 29:
Revised: d − i (cid:105) 29:

Original: C ← C ∪ { to itemset ( H ′ ) } 30: return C tentially expensive intersection test against U .
Revised: C ← C ∪ { to itemset ( H (cid:48) ) } 30: return C predicating on f ∗ have a non-empty intersection with i ∗ .

Revised: The implementation of the second condition amounts to verifying that adding i ∗ to I 1 identiﬁes a smaller interval for feature f ∗ .

Revised: Finally, we report the key insight that not all the itemsets must undergo the potentially expensive intersection test against

Original:  In particular, if (cid:74) I (cid:75) ⊆(cid:74)
Revised: U . In particular, if (cid:74) I (cid:75) ⊆

Original:  I ′ (cid:75) for some I ′ which we already proved fair, I can be ignored, because it does not identify new sufficient conditions for fairness.
Revised: (cid:74) I (cid:48) (cid:75) for some I (cid:48) which we already proved fair, I can be ignored, because it does not identify new sufﬁcient conditions for fairness (cf.

Original: The check at line 14 implements this optimization.
Revised: the check at line 14).

Original: Note that it is easy to move from the declarative style of this check to its implementation, because checking (cid:74) I (cid:75) ⊆
Revised: Note that it is easy to move from the declarative style of this check to its implementation, because checking (cid:74) I (cid:75) ⊆

Original: (cid:74) I ′ (cid:75) amounts to checking that, for all features f , the interval on f identified by I is included in the interval on f identified by I ′ .
Revised: (cid:74) I (cid:48) (cid:75) amounts to checking that, for all features f , the interval on f identiﬁed by I is included in the interval on f identiﬁed by I (cid:48) .

Original: 3) Formal Results: We can prove that our algorithm is both sound and complete, as formalized below.
Revised: 3) Formal Results: We can prove that our algorithm is both sound and complete, as formalized below.

Original: Soundness ensures that any itemset I returned by the synthesis algorithm is a sufficient condition for fairness, i.e., instances in (cid:74) I (cid:75) cannot suffer from causal discrimination.
Revised: Soundness ensures that any itemset I returned by the synthesis algorithm is a sufﬁcient condition for fairness, i.e., instances in (cid:74) I (cid:75) cannot suffer from causal discrimination.

Original: Theorem 1 (Soundness) .
Revised: Theorem 1 (Soundness) .

Original: For any decision tree ensemble T and set of sensitive features S , the call S YNTHESIZE ( T, S ) returns a set of itemsets F such that, for every I ∈ F , T does not perform causal discrimination on (cid:74) I (cid:75) .
Revised: For any decision tree ensemble T and set of sensitive features S , the call S YNTHESIZE ( T, S ) returns a set of itemsets F such that, for every I ∈ F , T does not perform causal discrimination on (cid:74) I (cid:75) .

Original: Proof.
Revised: Proof.

Original: The call A NALYZE ( T, S ) returns a set of hyperrectangles U satisfying the following property: for every instance ⃗x ∈ X , if there exists ⃗z ∈ δ ( ⃗x, S ) such that [Equation]
Revised: The call A NALYZE ( T, S ) returns a set of hyperrectangles U satisfying the following property: for every instance (cid:126)x ∈ X , if there exists (cid:126)z ∈ δ ( (cid:126)x, S ) such that f ( (cid:126)z ) (cid:54) =

Original:  , then there exists H ∈ [Equation] such that ⃗x ∈ H .
Revised: f ( (cid:126)x ) , then there exists H ∈ U such that (cid:126)x ∈ H .

Original: This means that T cannot perform causal discrimination on any X ′ ⊆
Revised: This means that T cannot perform causal discrimination on any X (cid:48) ⊆

Original: X such that ∀
Revised: X such that ∀

Original: H ∈
Revised: H ∈

Original: U : X ′ ∩ H = ∅ .
Revised: U : X (cid:48) ∩ H = ∅ .

Original: The conclusion follows by observing that any itemset I which is added to F must satisfy this property (cf.
Revised: The conclusion follows by observing that any itemset I which is added to F must satisfy this property (cf.

Original: lines 6-7 and lines 15-16).
Revised: lines 6-7 and lines 15-16).

Original: Completeness, instead, ensures that the combination of all the itemsets F returned by the synthesis algorithm coincides with the subset of the feature space disjoint from the result of the data-independent stability analysis, i.e., it represents the ideal outcome of the synthesis algorithm.
Revised: Completeness, instead, ensures that the combination of all the itemsets F returned by the synthesis algorithm coincides with the subset of the feature space disjoint from the result of the data-independent stability analysis, i.e., it represents the ideal outcome of the synthesis algorithm.

Original: Note that, if the data-independent stability analysis is not over-approximated but exact, this ensures that F covers all the instances where the classifier is fair.
Revised: Note that, if the data-independent stability analysis is not over-approximated but exact, this ensures that F covers all the instances where the classiﬁer is fair.

Original: Theorem 2 (Completeness) .
Revised: Theorem 2 (Completeness) .

Original: For any decision tree ensemble
Revised: For any decision tree ensemble

Original: T and set of sensitive features S , the call S YNTHESIZE ( T, S ) returns a set of itemsets F such that (cid:83) I ∈ F (cid:74) [Equation]
Revised: T and set of sensitive features S , the call S YNTHESIZE ( T, S ) returns a set of itemsets F such that (cid:83) I ∈ F (cid:74) I (cid:75)

Original: =
Revised: =

Original: [Equation]

Original:  \ (cid:83)
Revised: X \ (cid:83)

Original: H ∈
Revised: H ∈

Original: U H , where U is the output of A NALYZE ( T, S ) .
Revised: U H , where U is the output of A NALYZE ( T, S ) .

Original: Proof.
Revised: Proof.

Original: We prove the equality of the two sets by showing that the first is included in the second and vice-versa.
Revised: We prove the equality of the two sets by showing that the ﬁrst is included in the second and vice-versa.

Original: Specifically:
Revised: Speciﬁcally:

Original: • Consider any instance ⃗x ∈ (cid:83) I ∈ F (cid:74) I (cid:75) , we show that for all H ∈ U we have ⃗x ̸∈ H , which implies ⃗x ∈ X \ (cid:83)
Revised: • Consider any instance (cid:126)x ∈ (cid:83) I ∈ F (cid:74) I (cid:75) , we show that for all H ∈ U we have (cid:126)x (cid:54)∈ H , which implies (cid:126)x ∈ X \ (cid:83)

Original: H ∈
Revised: H ∈

Original: U H .
Revised: U H .

Original: Indeed, the algorithm ensures that for all I ∈ F we have that ∀ H ∈
Revised: Indeed, the algorithm ensures that for all I ∈ F we have that ∀ H ∈

Original: U : (cid:74) I (cid:75)
Revised: U : (cid:74) I (cid:75)

Original: ∩ H = ∅ (cf.
Revised: ∩ H = ∅ (cf.

Original: lines 6-7 and lines 15-16).
Revised: lines 6-7 and lines 15-16).

Original: • Consider any instance ⃗x ∈ X \ (cid:83)
Revised: • Consider any instance (cid:126)x ∈ X \ (cid:83)

Original: H ∈
Revised: H ∈

Original: U H , we show that there exists I ∈ F such that ⃗x ∈ (cid:74) I (cid:75) , which implies ⃗x ∈ (cid:83)
Revised: U H , we show that there exists I ∈ F such that (cid:126)x ∈ (cid:74) I (cid:75) , which implies (cid:126)x ∈ (cid:83)

Original: I ∈ F (cid:74) I (cid:75).
Revised: I ∈ F (cid:74) I (cid:75)

Original: We first observe that, for all H j ∈ U,
Revised: We ﬁrst observe that, for all H j ∈ U

Original:  the call G EN I TEMSETS ( U ) returns a set of at most 2 d itemsets:
Revised: , the call G EN I TEMSETS ( U ) returns a set of at most 2 d itemsets:

Original: [Equation]
Revised: [Equation]

Original:  such that ⃗x ∈ (cid:74) i k (cid:75) for some
Revised: . , { i 2 d }} , such that (cid:126)x ∈ (cid:74) i k (cid:75) for some

Original: i k ; we refer to such item
Revised: i k ; we refer to such item

Original: i k as the witness for H j .
Revised: i k as the witness for H j .

Original: The itemset I includes all the witnesses for each H j ∈ U , thus ensures that ⃗x ∈ (cid:74) I (cid:75) 
Revised: The itemset I includes all the witnesses for each H j ∈ U , thus ensures that (cid:126)x ∈ (cid:74) I (cid:75) =

Original: = (cid:84)
Revised: (cid:84)

Original: i ∈ I (cid:74) i (cid:75) .
Revised: i ∈ I (cid:74) i (cid:75) .

Original: The conclusion follows by observing that either the itemset I or another itemset I ′ such that
Revised: The conclusion follows by observing that either the itemset I or another itemset I (cid:48) such that

Original: [Equation]
Revised: [Equation]

Original: is eventually enumerated by the algorithm.
Revised: is eventually enumerated by the algorithm.

Original: 4) Implementation:
Revised: 4) Implementation:

Original: We implemented the synthesis algorithm presented in this section in C++.
Revised: We implemented the synthesis algorithm presented in this section in C++.

Original: Our implementation leverages the data-independent stability analysis used by the resilience verifier presented in [26], which we simply leverage as a black box to implement the A NALYZE function.
Revised: Our implementation leverages the data-independent stability analysis used by the resilience veriﬁer presented in [26], which we simply leverage as a black box to implement the A NALYZE function.

Original: The stability analysis is based on an iterative algorithm, whose computational cost may show an exponential trend, but can be constrained by fixing the number of iterations, without significantly affecting the precision of the results [26].
Revised: The stability analysis is based on an iterative algorithm, whose computational cost may show an exponential trend, but can be constrained by ﬁxing the number of iterations, without signiﬁcantly affecting the precision of the results [26].

Original: Although the rest of the implementation is a rather direct translation of the pseudocode in Algorithm 1, a few important details are worth discussing.
Revised: Although the rest of the implementation is a rather direct translation of the pseudocode in Algorithm 1, a few important details are worth discussing.

Original: A first point to note is that our implementation supports a user-specified early stopping criterion in terms ofa maximum number of iterations of the algorithm.
Revised: A ﬁrst point to note is that our implementation supports a user-speciﬁed early stopping criterion in terms of

Original: This is useful because, like Apriori, our algorithm has an exponential time complexity with respect to the number of items in

Original: [Table]
Revised: [Table]

Original:  the worst case [30].
Revised: a maximum number of iterations of the algorithm. This is useful because, like Apriori, our algorithm has an exponential time complexity with respect to the number of items in the worst case [30].

Original: However, we empirically observed in our experimental evaluation that a small number of short conditions already allow one to largely characterize the fairness guarantees of tree-based classifiers, hence restricting the number of iterations saves analysis time, while leading to just a small loss in precision in practice.
Revised: However, we empirically observed in our experimental evaluation that a small number of short conditions already allow one to largely characterize the fairness guarantees of tree-based classiﬁers, hence restricting the number of iterations saves analysis time, while leading to just a small loss in precision in practice.

Original: Note that early stopping preserves the soundness of the analysis, while obviously sacrificing its completeness.
Revised: Note that early stopping preserves the soundness of the analysis, while obviously sacriﬁcing its completeness.

Original: For space reasons, other interesting implementation details are presented in Appendix A.
Revised: For space reasons, other interesting implementation details are presented in Appendix A.

Original: IV. E XPERIMENTAL E VALUATION
Revised: IV. E XPERIMENTAL E VALUATION

Original: In this section, we experimentally assess the effectiveness of our global fairness verification approach along different axes.
Revised: In this section, we experimentally assess the effectiveness of our global fairness veriﬁcation approach along different axes.

Original: A. Methodology
Revised: A. Methodology

Original: We evaluate our proposal on decision tree ensembles trained over three public datasets used in the fairness literature [9], each associated with a binary classification task where fairness matters: Adult 1 , German 2 and Health 3 .
Revised: We evaluate our proposal on decision tree ensembles trained over three public datasets used in the fairness literature [9], each associated with a binary classiﬁcation task where fairness matters: Adult 2 , German 3 and Health 4 .

Original: The classification task for Adult requires predicting yearly income (above or below $50K), while the classification task for German requires assigning credit scores (good or bad), hence these datasets could be used to train classifiers deployed, e.g., to assess loan requests.
Revised: The classiﬁcation task for Adult requires predicting yearly income (above or below $50K), while the classiﬁcation task for German requires assigning credit scores (good or bad), hence these datasets could be used to train classiﬁers deployed, e.g., to assess loan requests.

Original: The classification task for Health, instead, requires predicting ten-year mortality (above or below the median
Revised: The classiﬁcation task for Health, instead, requires predicting ten-year mortality (above or below the median

Original: Charlson index), hence it could be used to train classifiers deployed in the health insurance setting.
Revised: Charlson index), hence it could be used to train classiﬁers deployed in the health insurance setting.

Original: Since in all cases we do not want to discriminate customers based on their gender, we use the attribute sex as the binary sensitive feature.
Revised: Since in all cases we do not want to discriminate customers based on their gender, we use the attribute sex as the binary sensitive feature.

Original: For simplicity, we only consider a single sensitive feature leading to a clear understanding of the related fairness issues, however, our proposal supports an arbitrary number of sensitive features.
Revised: For simplicity, we only consider a single sensitive feature leading to a clear understanding of the related fairness issues, however, our proposal supports an arbitrary number of sensitive features.

Original: We pre-process the datasets by following three steps: (i) we normalize all the numerical features in the interval [0 , 1] ; (ii) we perform the one-hot-encoding of all the categorical features; (iii) we remove the features and instances containing missing values.
Revised: We pre-process the datasets by following three steps: (i) we normalize all the numerical features in the interval [0 , 1] ; (ii) we perform the one-hot-encoding of all the categorical features; (iii) we remove the features and instances containing missing values.

Original: The characteristics of the datasets are in
Revised: The characteristics of the datasets are in

Original: Table I, along with the number of categorical features before and after step (ii).
Revised: Table I, along with the number of categorical features before and after step (ii).

Original: We partition datasets into a training set D train and a test set D test , using 80-20 stratified random sampling.
Revised: We partition datasets into a training set D train and a test set D test , using 80-20 stratiﬁed random sampling.

Original: For each dataset, we use D train to train standard Random Forest (RF) models as available in sklearn [31].
Revised: For each dataset, we use D train to train standard Random Forest (RF) models as available in sklearn [31].

Original: Experiments are then conducted on both D test and a synthetic dataset D rand including  100k random instances, generated by assigning a random value to all their features, while restricting categorical features to a set of plausible values observed in the original dataset.
Revised: Experiments are then conducted on both D test and a synthetic dataset D rand including 2 https://archive.ics.uci.edu/ml/datasets/adult 3 https://archive.ics.uci.edu/ml/datasets/statlog+(german+credit+data) 4 https://www.kaggle.com/c/hhp 100k random instances, generated by assigning a random value to all their features, while restricting categorical features to a set of plausible values observed in the original dataset.

Original: The rationale for having these two sets is that D test is supposed to follow the same distribution of D train , hence it represents expected inputs at test time, while D rand provides a much larger view of the entire feature space, which is interesting because our fairness guarantees are global and generalize beyond the test set.
Revised: The rationale for having these two sets is that D test is supposed to follow the same distribution of D train , hence it represents expected inputs at test time, while D rand provides a much larger view of the entire feature space, which is interesting because our fairness guarantees are global and generalize beyond the test set.

Original: Both D test and D rand are thus useful to investigate different aspects of our proposal.
Revised: Both D test and D rand are thus useful to investigate different aspects of our proposal.

Original: We write Din formulas to stand for any between D test and D rand , using the notation ⃗x ∈ D to identify instances in D while ignoring their class label when D = D test .
Revised: We write D to stand for any between D test and D rand , using the notation (cid:126)x ∈ D to identify instances in D while ignoring their class label when D = D test .

Original: The experiments are designed to answer three key research questions:
Revised: The experiments are designed to answer three key research questions:

Original: 1) What is the precision of the analysis performed by the synthesis algorithm?
Revised: 1) What is the precision of the analysis performed by the synthesis algorithm?

Original: In other words, do the synthesized fairness conditions cover well the portions of the feature space where lack of causal discrimination is ensured?
Revised: In other words, do the synthesized fairness conditions cover well the portions of the feature space where lack of causal discrimination is ensured?

Original: What is the explainability of the results returned by the synthesis algorithm?
Revised: What is the explainability of the results returned by the synthesis algorithm?

Original: Can we effectively characterize the fairness guarantees of classifiers in terms of a small number of conditions of limited complexity?
Revised: Can we effectively characterize the fairness guarantees of classiﬁers in terms of a small number of conditions of limited complexity?

Original: What is the performance of the synthesis algorithm?
Revised: What is the performance of the synthesis algorithm?

Original: How is the analysis running time influenced by the size and complexity of the classifiers?
Revised: How is the analysis running time inﬂuenced by the size and complexity of the classiﬁers?

Original: B. Precision of the Analysis
Revised: B. Precision of the Analysis

Original: In our first experiment, we estimate the precision of our verification approach by comparing the set of hyper-rectangles
Revised: In our ﬁrst experiment, we estimate the precision of our veriﬁcation approach by comparing the set of hyper-rectangles

Original: U returned by the data-independent stability analysis with the formulas F returned by the synthesis algorithm.
Revised: U returned by the data-independent stability analysis with the formulas F returned by the synthesis algorithm.

Original: The completeness theorem ensures that F identifies the subset of the feature space disjoint from all the hyper-rectangles in U , provided that the synthesis algorithm runs up to completion.
Revised: The completeness theorem ensures that F identiﬁes the subset of the feature space disjoint from all the hyper-rectangles in U , provided that the synthesis algorithm runs up to completion.

Original: In practice, however, we expect the synthesis algorithm to be normally subject to early stopping, due to the exponential blow-up in the number of candidates to be analyzed at the different iterations and the increasing complexity of the synthesized formulas.
Revised: In practice, however, we expect the synthesis algorithm to be normally subject to early stopping, due to the exponential blow-up in the number of candidates to be analyzed at the different iterations and the increasing complexity of the synthesized formulas.

Original: We thus estimate the precision of a partial output obtained when the synthesis algorithm is forcefully stopped after six iterations, because we are interested in formulas with a small number of items, amenable for human understanding.
Revised: We thus estimate the precision of a partial output obtained when the synthesis algorithm is forcefully stopped after six iterations, because we are interested in formulas with a small number of items, amenable for human understanding.

Original: Later experiments assess the impact of the number of iterations on the analysis results and running times.
Revised: Later experiments assess the impact of the number of iterations on the analysis results and running times.

Original: We first compute the casual discrimination score on the dataset D based on the set of hyper-rectangles U .
Revised: We ﬁrst compute the casual discrimination score on the dataset D based on the set of hyper-rectangles U .

Original: In particular, we observe that an instance ⃗x can only suffer from causal discrimination when it belongs to some H ∈ U , so the casual discrimination score on D can be conservatively approximated as follows:
Revised: In particular, we observe that an instance (cid:126)x can only suffer from causal discrimination when it belongs to some H ∈ U , so the casual discrimination score on D can be conservatively approximated as follows:

Original: [Equation]
Revised: [Equation]

Original: |D| Similarly, we can use the formulas in F to reason about fairness by observing that any instance satisfying some formula  in F cannot suffer from causal discrimination.
Revised: Similarly, we can use the formulas in F to reason about fairness by observing that any instance satisfying some formula [Table] in F cannot suffer from causal discrimination.

Original: This allows us to compute the following over-approximation of d :
Revised: This allows us to compute the following over-approximation of d :

Original: [Equation]
Revised: [Equation]

Original: [Table] Observe that ˜ d is always an upper bound of d , because our analysis is sound.
Revised: Observe that ˜ d is always an upper bound of d , because our analysis is sound.

Original: Table II reports the values of d and ˜ d computed for different datasets and models.
Revised: Table II reports the values of d and ˜ d computed for different datasets and models.

Original: We observe that d and ˜ d coincide for the very large majority of the cases, meaning that the formulas in F accurately characterize the subset of the feature space which is disjoint from all the hyper-rectangles in U , even when enforcing early stopping after six iterations.
Revised: We observe that d and ˜ d coincide for the very large majority of the cases, meaning that the formulas in F accurately characterize the subset of the feature space which is disjoint from all the hyper-rectangles in U , even when enforcing early stopping after six iterations.

Original: Note that ˜ d does not coincide with d just in three cases, all associated with the German dataset, but we experimentally assessed that the two scores coincide also there when increasing the number of iterations of the synthesizer to seven.
Revised: Note that ˜ d does not coincide with d just in three cases, all associated with the German dataset, but we experimentally assessed that the two scores coincide also there when increasing the number of iterations of the synthesizer to seven.

Original: The table also reports the accuracy a on the test set
Revised: The table also reports the accuracy a on the test set

Original: D test to show that all the analyzed models perform reasonably well in practice.
Revised: D test to show that all the analyzed models perform reasonably well in practice.

Original: C. Explainability of the Results
Revised: C. Explainability of the Results

Original: To assess to what extent the formulas F are explainable, we carry out selected experiments on the most complex models that we trained on the considered datasets, i.e., ensembles oftrees with maximum depth six, because they are expected to be the most challenging to explain.
Revised: To assess to what extent the formulas F are explainable, we carry out selected experiments on the most complex models that we trained on the considered datasets, i.e., ensembles oftrees with maximum depth six, because they are expected to be the most challenging to explain.

Original: We follow the methodology used in previous work on the use of rules based on logical formulas to extract explainable information from classifiers, like [32], [33], [34].
Revised: We follow the methodology used in previous work on the use of rules based on logic formulas to extract explainable information from classiﬁers, like [32], [33], [34].

Original: In these proposals, the complexity of logical formulas is measured by their length (number of atoms) and short logical formulas are preferred because of their clearer understanding.
Revised: In these proposals, the complexity of logic formulas is measured by their length (number of atoms) and short logic formulas are preferred because of their clearer understanding.

Original: Of course, having a small number of formulas is also important for explainability.
Revised: Of course, having a small number of formulas is also important for explainability.

Original: In the first experiment, we assess how the percentage of instances for which F is able to provide a proof of fairness grows when varying the number of iterations of the synthesis algorithm.
Revised: In the ﬁrst experiment, we assess how the percentage of instances for which F is able to provide a proof of fairness grows when varying the number of iterations of the synthesis algorithm.

Original: Of course, we compute this percentage with respect to the number of instances which do not suffer from causal discrimination, i.e., which might actually admit a proof of fairness.
Revised: Of course, we compute this percentage with respect to the number of instances which do not suffer from causal discrimination, i.e., which might actually admit a proof of fairness.

Original: Since a run with k iterations can only produce formulas involving at most k items, this experiment provides insights on how complex sufficient conditions for fairness turn out to be in practice.
Revised: Since a run with k iterations can only produce formulas involving at most k items, this experiment provides insights on how complex sufﬁcient conditions for fairness turn out to be in practice.

Original: Figure 2 plots the observed trend.
Revised: Figure 2 plots the observed trend.

Original: The experimental results for the Adult dataset show that just four iterations of the algorithm suffice to establish fairness proofs for more than 90% of the instances of both D test and D rand , while just two iterations are enough to cover basically all the instances of the two sets for the Health dataset.
Revised: The experimental results for the Adult dataset show that just four iterations of the algorithm sufﬁce to establish fairness proofs for more than 90% of the instances of both D test and D rand , while just two iterations are enough to cover basically all the instances of the two sets for the Health dataset.

Original: The German dataset is the most challenging, since five iterations of the algorithm are needed to cover around 80% of the instances in the two sets.
Revised: The German dataset is the most challenging, since ﬁve iterations of the algorithm are needed to cover around 80% of the instances in the two sets.

Original: In the end, the experiment shows that short logical formulas including at most five items are expressive enough to establish useful fairness proofs in practice, while being small enough to be easily understandable by human experts.
Revised: In the end, the experiment shows that short logic formulas including at most ﬁve items are expressive enough to establish useful fairness proofs in practice, while being small enough to be easily understandable by human experts.

Original: Clearly, however, our first experiment provides just a partial picture of explainability, because it captures information about the complexity of formulas, but it does not tell how many formulas should be taken into consideration by human analysts to draw useful conclusions.
Revised: Clearly, however, our ﬁrst experiment provides just a partial picture of explainability, because it captures information about the complexity of formulas, but it does not tell how many formulas should be taken into consideration by human analysts to draw useful conclusions.

Original: Indeed, we observe that the amount of formulas can significantly grow as the number of iterations of the synthesis algorithm increases.
Revised: Indeed, we observe that the amount of formulas can signiﬁcantly grow as the number of iterations of the synthesis algorithm increases.

Original: Figure 3 plots how the number of synthesized formulas grows when varying the number of iterations of the synthesis algorithm, showing an exponential trend for the Adult and German datasets.
Revised: Figure 3 plots how the number of synthesized formulas grows when varying the number of iterations of the synthesis algorithm, showing an exponential trend for the Adult and German datasets.

Original: We see an interesting trend in the results for the Health dataset: the number of formulas does not necessarily increase from one iteration to another, since in some cases the synthesizer is not able to produce longer formulas that provide proof of fairness.
Revised: We see an interesting trend in the results for the Health dataset: the number of formulas does not necessarily increase from one iteration to another, since in some cases the synthesizer is not able to produce longer formulas that provide proof of fairness.

Original: However, the figure shows a significant increase in the number of formulas when five iterations are performed, revealing that later iterations could mine longer formulas even when, during an iteration, new formulas are not discovered.
Revised: However, the ﬁgure shows a signiﬁcant increase in the number of formulas when ﬁve iterations are performed, revealing that later iterations could mine longer formulas even when, during an iteration, new formulas are not discovered.

Original: Nevertheless, we can show that the number of important formulas for human analysts is relatively small in practice.
Revised: Nevertheless, we can show that the number of important formulas for human analysts is relatively small in practice.

Original: We estimate the importance of a formula by counting the number of instances in D train which are covered by the formula: the intuition is that the more instances are covered by the formula, the more the formula is expressive to prove fairness according to the training data.
Revised: We estimate the importance of a formula by counting the number of instances in D train which are covered by the formula: the intuition is that the more instances are covered by the formula, the more the formula is expressive to prove fairness according to the training data.

Original: We identify the set of the top k most important formulas by means of a greedy strategy.
Revised: We identify the set of the top k most important formulas by means of a greedy strategy.

Original: We first select the most important formula in terms of number of covered instances, we then remove the covered instances from
Revised: We ﬁrst select the most important formula in terms of number of covered instances, we then remove the covered instances from

Original: D train before selecting the second most important formula and so on, until k formulas have been selected.
Revised: D train before selecting the second most important formula and so on, until k formulas have been selected.

Original: Thus, by fixing the number of iterations of the synthesis algorithm and varying the number k , we can assess how the percentage of instances proved fair by the top k formulas grows.
Revised: Thus, by ﬁxing the number of iterations of the synthesis algorithm and varying the number k , we can assess how the percentage of instances proved fair by the top k formulas grows.

Original: Figureplots the observed trend for six iterations of the synthesis algorithm.
Revised: Figureplots the observed trend for six iterations of the synthesis algorithm.

Original: The figure shows that for Adult and Health just the top 10 formulas suffice to cover around 90% of the D test instances, while for German the top 20 formulas allow one to establish a proof of fairness for 80% of the instances in D test .
Revised: The ﬁgure shows that for Adult and Health just the top 10 formulas sufﬁce to cover around 90% of the D test instances, while for German the top 20 formulas allow one to establish a proof of fairness for 80% of the instances in D test .

Original: This shows that a small number of formulas is sufficient to  characterize the fairness guarantees on the test data for all the considered datasets.
Revised: This shows that a small number of formulas is sufﬁcient to [Figure] [Figure] characterize the fairness guarantees on the test data for all the considered datasets.

Original: In general, more formulas are needed to cover synthetic instances in D rand .
Revised: In general, more formulas are needed to cover synthetic instances in D rand .

Original: Indeed, the figure shows that, while the top 20 formulas still allow one to cover more than 90% of the instances in D rand for Health, they cover just[Figure] [Figure] around 30% of the instances of Adult and around 60% of the instances of German.
Revised: Indeed, the ﬁgure shows that, while the top 20 formulas still allow one to cover more than 90% of the instances in D rand for Health, they cover just around 30% of the instances of Adult and around 60% of the instances of German.

Original: The difference between the results on D test and D rand can be explained by observing that the synthesized conditions depend on the thresholds learned from
Revised: The difference between the results on D test and D rand can be explained by observing that the synthesized conditions depend on the thresholds learned from

Original: D train , which is the same set used to rank the conditions, hence the top conditions generalize better on a test set with the same distribution of D train than on a random set.
Revised: D train , which is the same set used to rank the conditions, hence the top conditions generalize better on a test set with the same distribution of D train than on a set of randomly generated instances.

Original: The good news is that a small number of conditions still suffice to cover a nonnegligible share of D rand in all cases.
Revised: The good news is that a small number of conditions still sufﬁce to cover a non-negligible share of D rand in all cases.

Original: We finally perform a more qualitative evaluation of the explainability of the  synthesis algorithm by considering the German dataset as a case study.
Revised: We ﬁnally perform a more qualitative evaluation of the explainability of the conditions generated by the synthesis algorithm by considering the German dataset as a case study.

Original: For space reasons, this is discussed in Appendix B.
Revised: For space reasons, this is discussed in Appendix B.

Original: D. Performance Evaluation
Revised: D. Performance Evaluation

Original: We finally analyze the performance of the synthesis algorithm when varying different dimensions of our experimental setting.
Revised: We ﬁnally analyze the performance of the synthesis algorithm when varying different dimensions of our experimental setting.

Original: Recall that the synthesis algorithm invokes the dataindependent stability analysis in [26] as a black box, however any other analysis like [28] could be used in its place, so we do not include the cost of this preliminary operation in our performance evaluation.
Revised: Recall that the synthesis algorithm invokes the dataindependent stability analysis in [26] as a black box, however any other analysis like [28] could be used in its place, so we do not include the cost of this preliminary operation in our performance evaluation.

Original: For the sake of completeness, we separately assess the computational cost of the data-independent stability analysis in Appendix C. Times are computed for our sequential implementation of the synthesis algorithm, running on a virtual machine with 20 cores, 98.8GB of RAM and Ubuntu 20.04.
Revised: For the sake of completeness, we separately assess the computational cost of the data-independent stability analysis in Appendix C. Times are computed for our sequential implementation of the synthesis algorithm, running on a virtual machine with 20 cores, 98.8GB of RAM and Ubuntu 20.04.

Original: LTS on a server with an Intel Xeon Gold 6148 2.40GHz.
Revised: LTS on a server with an Intel Xeon Gold 6148 2.40GHz.

Original: First, we plot how the analysis times change when increasing the number of analysis iterations from one to seven.
Revised: First, we plot how the analysis times change when increasing the number of analysis iterations from one to seven.

Original: For the sake of readability, we only focus on ensembles of 13 trees with maximum depth six, which are the most complex models in our experimental evaluation and likely the most challenging models to analyze.
Revised: For the sake of readability, we only focus on ensembles of 13 trees with maximum depth six, which are the most complex models in our experimental evaluation and likely the most challenging models to analyze.

Original: The results are shown in Figure 5.
Revised: The results are shown in Figure 5.

Original: The first observation is that all the models can be analyzed in a matter of minutes when performing six iterations of the algorithm, like in our previous experiments: the analysis takes around 30 minutes on the Adult dataset, five minutes on the German dataset and just 13 seconds on the Health dataset.
Revised: The ﬁrst observation is that all the models can be analyzed in a matter of minutes when performing six iterations of the algorithm, like in our previous experiments: the analysis takes around 30 minutes on the Adult dataset, ﬁve minutes on the German dataset and just 13 seconds on the Health dataset.

Original: The gap in the third dataset can be explained by the smaller number of categorical features therein.
Revised: The gap of the analysis time on the third dataset can be explained by the smaller number of categorical features therein.

Original: In general, the plots show an exponential growth of the analysis time as the number of iterations increases.
Revised: In general, the plots show an exponential growth of the analysis time as the number of iterations increases.

Original: Luckily, previous results show that conditions containing at most five items(i.e., generated after at most five iterations) are already enough to cover a large part of the feature space for all the analyzed models, hence further increasing the number of analysis iterations for them is unimportant in practice.
Revised: Luckily, previous results show that conditions containing at most ﬁve items are already enough to cover a large part of the feature space for all the analyzed models, hence further increasing the number of analysis iterations for them is unimportant in practice.

Original: We then show how the running times of the synthesisalgorithm are affected by the number of trees in the ensemble,
Revised: We then show how the running times of the synthesis

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original:  as well as by their depth.
Revised: algorithm are affected by the number of trees in the ensemble, as well as by their depth.

Original: We first run the algorithm over ensembles including an increasing number of trees of at most depth six (Figure 6) and then over ensembles includingtrees of increasing maximum depth (Figure 7), stopping the analysis after six iterations.
Revised: We ﬁrst run the algorithm over ensembles including an increasing number of trees of at most depth six (Figure 6) and then over ensembles includingtrees of increasing maximum depth (Figure 7), stopping the analysis after six iterations.

Original: All the settings involving tree ensembles with at most 13 trees or trees with maximum depth six like in our previous experiments terminate in a matter of minutes, thus showing the practicality of our proposal.
Revised: All the settings involving tree ensembles with at most 13 trees or trees with maximum depth six like in our previous experiments terminate in a matter of minutes, thus showing the practicality of our proposal.

Original: Nevertheless, both plots show that the analysis times exhibit an exponential growth with respect to the complexity of the model, since the complexity of the analysis algorithm is exponential with respect to the number of items, which in turn depends on the number of features and thresholds occurring in the model.
Revised: Nevertheless, both plots show that the analysis times exhibit an exponential growth with respect to the complexity of the model, since the complexity of the analysis algorithm is exponential with respect to the number of items, which in turn depends on the number of features and thresholds occurring in the model.

Original: Thus, the analysis time for more complex models, e.g., the tree ensembles with maximum depth seven trained on the Adult dataset, could significantly increase.
Revised: Thus, the analysis time for more complex models, e.g., the tree ensembles with maximum depth seven trained on the Adult dataset, could signiﬁcantly increase.

Original: This further motivates the benefits of an iterative analysis approach as the proposed one, which allows one to leverage an early stopping criterion to collect partial (yet empirically precise) results even for cases where analysis convergence may turn out to be too expensive.
Revised: This further motivates the beneﬁts of an iterative analysis approach as the proposed one, which allows one to leverage an early stopping criterion to collect partial (yet empirically precise) results even for cases where analysis convergence may turn out to be too expensive.

Original: E. Summary
Revised: E. Summary

Original: Our previous experiments showed that a small number of iterations of the synthesis algorithm are sufficient to provide a precise and human-understandable characterization of the fairness guarantees provided by the analyzed classifiers.
Revised: Our previous experiments showed that a small number of iterations of the synthesis algorithm are sufﬁcient to provide a precise and human-understandable characterization of the fairness guarantees provided by the analyzed classiﬁers.

Original: Specifically, for all datasets and models we were consistently able to obtain practically useful analysis results when terminating the analysis after six iterations.
Revised: Speciﬁcally, for all datasets and models we were consistently able to obtain practically useful analysis results when terminating the analysis after six iterations.

Original: The results were precise , because they matched the fairness guarantees provided by the classifier on both the test set and a set of synthetic instances randomly created to provide a more extensive picture of the entire feature space.
Revised: The results were precise , because they matched the fairness guarantees provided by the classiﬁer on both the test set and a set of synthetic instances randomly created to provide a more extensive picture of the entire feature space.

Original: The results were explainable , because a small number of conditions of limited complexity turned out to be sufficient to largely characterize the fairness guarantees provided by the classifier, in particular over the test set.
Revised: The results were explainable , because a small number of conditions of limited complexity turned out to be sufﬁcient to largely characterize the fairness guarantees provided by the classiﬁer, in particular over the test set.

Original: Finally, our algorithm is efficient enough for practical adoption on the analyzed models, because all the experiments terminated in a matter of minutes when limiting the number of analysis iterations.
Revised: Finally, our algorithm is efﬁcient enough for practical adoption on the analyzed models, because all the experiments terminated in a matter of minutes when limiting the number of analysis iterations.

Original: Yet, we notice that our algorithm shows an exponential complexity with respect to the number of items, like the Apriori algorithm it is inspired from [30].
Revised: Yet, we notice that our algorithm shows an exponential complexity with respect to the number of items, like the Apriori algorithm it is inspired from [30].

Original: In our application setting, the number of items is correlated with the number of features and thresholds occurring in the model, hence we observe that large models in terms of depth or number of trees may eventually pose challenges to scalability.
Revised: In our application setting, the number of items is correlated with the number of features and thresholds occurring in the model, hence we observe that large models in terms of depth or number of trees may eventually pose challenges to scalability.

Original: Nevertheless, we showed that limiting the number of analysis iterations is useful to collect meaningful results even for cases 
Revised: Nevertheless, we showed that limiting the number of analysis iterations is useful to collect meaningful results even for cases that may be too complicated to analyze up to convergence, which might be enough to mitigate scalability problems. We

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Revised: t i m e ( s )

Original: [Figure]
Revised: [Figure]

Original: German depth=6, #iters=
Revised: depth [Figure] 6, #iters

Original: [Figure]
Revised: [Figure]

Original: [Figure]

Revised: (a)     15 #

Original: [Figure]
Revised: [Figure]

Original:  Health 
Revised: t i m e ( s ) German depth=6, #iters= (b)     15 #trees     t i m e ( s ) Health depth=6, #iters= (c)

Original: [Figure]
Revised: [Figure]

Original: [Figure]

Original: [Figure]
Revised: [Figure]

Original: that may be too complicated to analyze up to convergence, which might be enough to mitigate scalability problems.

Original: We leave a more extensive evaluation on larger models and an additional optimization of our implementation todeal with them to future work.
Revised: (c) leave a more extensive evaluation on larger models and an additional optimization of our implementation to future work.

Original: V. R ELATED W ORK
Revised: V. R ELATED W ORK

Original: We categorize related work in two broad research areas: fairness testing and fairness verification.
Revised: We categorize related work in two broad research areas: fairness testing and fairness veriﬁcation.

Original: We also briefly discuss prior work on fairness and explainability, which is a very recent research area that is getting traction.
Revised: We also brieﬂy discuss prior work on fairness and explainability, which is a very recent research area that is getting traction.

Original: A. Fairness Testing
Revised: A. Fairness Testing

Original: Fairness testing estimates the fairness guarantees of a classifier by means of the automated generation of a number of test instances, which are fed to the model to identify potential room for discrimination.
Revised: Fairness testing estimates the fairness guarantees of a classiﬁer by means of the automated generation of a number of test instances, which are fed to the model to identify potential room for discrimination.

Original: The first work on fairness testing we are aware of led to the development of the Themis tool [14], [35].
Revised: The ﬁrst work on fairness testing we are aware of led to the development of the Themis tool [14], [35].

Original: Themis uses a testing approach to identify instances which may suffer from causal discrimination, which is the same notion of fairness used in this paper.
Revised: Themis uses a testing approach to identify instances which may suffer from causal discrimination, which is the same notion of fairness used in this paper.

Original: Its approach is based on a random generation of test instances, whose number is determined by a simple statistical test.
Revised: Its approach is based on a random generation of test instances, whose number is determined by a simple statistical test.

Original: Later work in the area like Aequitas improved the test instance generation approach by integrating the global random search of Themis with a perturbation-based local search [16].
Revised: Later work in the area like Aequitas improved the test instance generation approach by integrating the global random search of Themis with a perturbation-based local search [16].

Original: Even more recent work combined symbolic execution with local explanation techniques to further improve the effectiveness of fairness testing [15], proposed more advanced statistical tests [36] and improved the explainability of the testing results [17].
Revised: Even more recent work combined symbolic execution with local explanation techniques to further improve the effectiveness of fairness testing [15], proposed more advanced statistical tests [36] and improved the explainability of the testing results [17].

Original: Contrary to our proposal, fairness testing performs an underapproximated analysis, i.e., it can find counter-examples to fairness, but it cannot prove fairness for specific classes of individuals.
Revised: Contrary to our proposal, fairness testing performs an underapproximated analysis, i.e., it can ﬁnd counter-examples to fairness, but it cannot prove fairness for speciﬁc classes of individuals.

Original: This is similar to what happens in software verification, where testing can be used to prove the presence of bugs, but not their absence.
Revised: This is similar to what happens in software veriﬁcation, where testing can be used to prove the presence of bugs, but not their absence.

Original: Testing thus plays an orthogonal role with respect to verification: when fairness guarantees cannot be proved for specific classes of individuals, one can rely on testing to identify counter-examples.
Revised: Testing thus plays an orthogonal role with respect to veriﬁcation: when fairness guarantees cannot be proved for speciﬁc classes of individuals, one can rely on testing to identify counter-examples.

Original: Indeed, it would be interesting to study how our fairness verification approach could boost the effectiveness of fairness testing, in particular by directing it towards portions of the feature space where discrimination might potentially happen.
Revised: Indeed, it would be interesting to study how our fairness veriﬁcation approach could boost the effectiveness of fairness testing, in particular by directing it towards portions of the feature space where discrimination might potentially happen.

Original: We leave the investigation of this research idea to future work.
Revised: We leave the investigation of this research idea to future work.

Original: B. Fairness Verification
Revised: B. Fairness Veriﬁcation

Original: Fairness verification of ML models attracted a lot of attention from the community, as shown by the emergence of several recent surveys on the topic [5], [6], [7].
Revised: Fairness veriﬁcation of ML models attracted a lot of attention from the community, as shown by the emergence of several recent surveys on the topic [5], [6], [7].

Original: Fairness is indeed a broad research area and several definitions of fairness have been proposed in the literature, each with its pros and cons: see [37], [22], [23] for a critical comparison of different fairness definitions.
Revised: Fairness is indeed a broad research area and several deﬁnitions of fairness have been proposed in the literature, each with its pros and cons: see [37], [22], [23] for a critical comparison of different fairness deﬁnitions.

Original: Of course, when pursuing formal verification, one has to stick to a specific notion of fairness, which is lack of causal discrimination in our case.
Revised: Of course, when pursuing formal veriﬁcation, one has to stick to a speciﬁc notion of fairness, which is lack of causal discrimination in our case.

Original: This property belongs to the class of individual fairness and we choose it for different reasons: it is intuitive, popular, and provides global fairness guarantees beyond the test data.
Revised: This property belongs to the class of individual fairness and we choose it for different reasons: it is intuitive, popular, and provides global fairness guarantees beyond the test data.

Original: Since individual fairness and group fairness deal with different concerns and rely on rather different technical tools, we consider verification of group fairness as complementary to our work [38], [39], [40], [41].
Revised: Since individual fairness and group fairness deal with different concerns and rely on rather different technical tools, we consider veriﬁcation of group fairness as complementary to our work [38], [39], [40], [41].

Original: A key difference of our work with respect to the state of the art is its focus on tree-based models, which have been largely ignored by prior work.
Revised: A key difference of our work with respect to the state of the art is its focus on tree-based models, which have been largely ignored by prior work.

Original: Indeed, the first work on the verification of individual fairness focused on linear models such as logistic regression and support vector machines [11].
Revised: Indeed, the ﬁrst work on the veriﬁcation of individual fairness focused on linear models such as logistic regression and support vector machines [11].

Original: Most later work, instead, focused on neural networks and cannot be applied to decision tree ensembles [18], [12], [10].
Revised: Most later work, instead, focused on neural networks and cannot be applied to decision tree ensembles [18], [12], [10].

Original: A general approach to fairness verification based on SMT solving was presented in [13].
Revised: A general approach to fairness veriﬁcation based on SMT solving was presented in [13].

Original: The approach can be applied to tree-based models by encoding them into SMT, however, it treats fairness just as a binary, unconditional property, i.e., a model is only considered to be fair when sensitive features never play any role in classification.
Revised: The approach can be applied to tree-based models by encoding them into SMT, however, it treats fairness just as a binary, unconditional property, i.e., a model is only considered to be fair when sensitive features never play any role in classiﬁcation.

Original: This is too restrictive in practice, as shown by their experimental evaluation which only verified two out of 40 analyzed models as fair.
Revised: This is too restrictive in practice, as shown by their experimental evaluation which only veriﬁed two out of 40 analyzed models as fair.

Original: Our verification approach is more expressive, because it automatically identifies sufficient conditions for fairness, defining specific classes of individuals where discrimination cannot happen; these classes may be small or large depending on the actual fairness guarantees of the model.
Revised: Our veriﬁcation approach is more expressive, because it automatically identiﬁes sufﬁcient conditions for fairness, deﬁning speciﬁc classes of individuals where discrimination cannot happen; these classes may be small or large depending on the actual fairness guarantees of the model.

Original: Indeed, observe that the trivially true condition subsumes the simple verification problem considered in [13].
Revised: Indeed, observe that the trivially true condition subsumes the simple veriﬁcation problem considered in [13].

Original: The only fairness verification approach designed for treebased models we are aware of is presented in [9].
Revised: The only fairness veriﬁcation approach designed for treebased models we are aware of is presented in [9].

Original: Their approach allows the verification of a local fairness property, which just predicates on a specific set of test instances, thus providing weaker fairness guarantees than the global fairness proofs offered by lack of causal discrimination.
Revised: Their approach allows the veriﬁcation of a local fairness property, which just predicates on a speciﬁc set of test instances, thus providing weaker fairness guarantees than the global fairness proofs offered by lack of causal discrimination.

Original: Also, their proposed approach just computes the causal discrimination score of the classifier over the test set, without providing any explanation of which individuals cannot suffer from discrimination.
Revised: Also, their proposed approach just computes the causal discrimination score of the classiﬁer over the test set, without providing any explanation of which individuals cannot suffer from discrimination.

Original: On the other hand, their approach was integrated into a fair training algorithm called FATT, which is an interesting avenue for future work we would like to pursue.
Revised: On the other hand, their approach was integrated into a fair training algorithm called FATT, which is an interesting avenue for future work we would like to pursue.

Original: C. Fairness and Explainability
Revised: C. Fairness and Explainability

Original: Reconciling fairness and explainability has been recognized as an important problem for specific application scenarios, such as algorithmic hiring [21].
Revised: Reconciling fairness and explainability has been recognized as an important problem for speciﬁc application scenarios, such as algorithmic hiring [21].

Original: However, the area is still in its infancy [42] and there has been limited work on the topic so far.
Revised: However, the area is still in its infancy [42] and there has been limited work on the topic so far.

Original: Recent work focused on designing post-hoc analysis for fairness that provide feature-level explanations, i.e., estimate the influence of each feature on the bias of the model w.r.t.
Revised: Recent work focused on designing post-hoc analyses for fairness that provide feature-level explanations, i.e., estimate the inﬂuence of each feature on the bias of the model w.r.t.

Original: a fairness definition and a specific set of instances [43], [44].
Revised: a fairness deﬁnition and a speciﬁc set of instances [43], [44].

Original: An alternative type of post-hoc fairness analysis is based on counterfactual explanations computed on specific inputs to explain the unfair behavior of the ML model [45], [46].
Revised: An alternative type of post-hoc fairness analysis is based on counterfactual explanations computed on speciﬁc inputs to explain the unfair behavior of the ML model [45], [46].

Original: Contrary to post-hoc fairness analysis methods, which allow one to derive explanations about the fair behaviour of the ML model on a specific set of instances, our verification method analyzes the classifier itself without relying on a specific set of instances and provides explanations about the fair behavior of the classifier over the entire feature space, i.e., all the possible inputs of the classifier, including unseen ones.
Revised: Contrary to post-hoc fairness analysis methods, which allow one to derive explanations about the fair behavior of the ML model on a speciﬁc set of instances, our veriﬁcation method analyzes the classiﬁer itself without relying on a speciﬁc set of instances and provides explanations about the fair behavior of the classiﬁer over the entire feature space, i.e., all the possible inputs of the classiﬁer, including unseen ones.

Original: Moreover, recent research also investigated how to train ML models that are both explainable and fair [47], [48], [49] and explainable fairness for recommender systems [50], [51].
Revised: Moreover, recent research also investigated how to train ML models that are both explainable and fair [47], [48], [49] and explainable fairness for recommender systems [50], [51].

Original: The use of logical formulas for explainability purposes has also been investigated by the community.
Revised: The use of logic formulas for explainability purposes has also been investigated by the community.

Original: Prior work proposed approaches to use logical formulas as building blocks of logicrelated models, i.e., rule lists [52] and decision sets [53], that exhibit high explainability and accuracy.
Revised: Prior work proposed approaches to use logic formulas as building blocks of logicrelated models, i.e., rule lists [52] and decision sets [53], that exhibit high explainability and accuracy.

Original: Moreover, since it is difficult to explain the logic behind decision tree ensembles because of their large number of trees, several contributions in the literature propose methods to extract a set of decision rules that describe the tree-based model in order to provide explanations at a global level about the outcome of its predictions [32], [33], [34], [54], [55], [56].
Revised: Moreover, since it is difﬁcult to explain the logic behind decision tree ensembles because of their large number of trees, several contributions in the literature proposed methods to extract a set of decision rules that describe the tree-based model in order to provide explanations at a global level about the outcome of its predictions [32], [33], [34], [54], [55], [56].

Original: However, the idea of using logic formulas to explain the fairness guarantees provided by a ML model is novel to the best of our knowledge.
Revised: However, the idea of using logic formulas to explain the fairness guarantees provided by a ML model is novel to the best of our knowledge.

Original: In other words, a key difference of our work with respect to the state of the art is the target of the explanations, because our proposal aims to explain the fairness guarantees of a tree ensemble, not the outcome of its predictions.
Revised: In other words, a key difference of our work with respect to the state of the art is the target of the explanations, because our proposal aims to explain the fairness guarantees of a tree ensemble, not the outcome of its predictions.

Original: VI. C ONCLUSION
Revised: VI. C ONCLUSION

Original: In this paper, we presented a new global fairness verification approach for tree-based classifiers.
Revised: In this paper, we presented a new global fairness veriﬁcation approach for tree-based classiﬁers.

Revised: Our approach synthesizes sufﬁcient conditions for fairness that provide global fairness guarantees, since they predicate over the entire feature space rather than on a test set of instances.

Original: Our approach synthesizes sufficient conditions for fairness, expressed asa set of traditional propositional logic formulas, which are readily understandable by human experts.
Revised: The generated conditions are expressed as traditional propositional logic formulas that are readily understandable by human experts, i.e., they are explainable.

Original: The analysis is proved to be sound and complete.
Revised: The analysis is proved to be sound and complete.

Original: Extensive experimental results on public datasets show that the analysis is precise, easily understandable by human experts and efficient enough for practical adoption.
Revised: Extensive experimental results on public datasets show that the analysis is precise, easily understandable by human experts and efﬁcient enough for practical adoption.

Original: We foresee a few relevant directions for future work.
Revised: We foresee a few relevant directions for future work.

Original: First, we would like to leverage our verification approach as a powerful foundation to train tree ensembles satisfying global fairness properties.
Revised: First, we would like to leverage our veriﬁcation approach as a powerful foundation to train tree ensembles satisfying global fairness properties.

Original: This seems feasible because prior work showed how a local fairness verifier can be used to train locally fair models [9].
Revised: This seems feasible because prior work showed how a local fairness veriﬁer can be used to train locally fair models [9].

Original: Moreover, we plan to integrate fairness verification and fairness testing by using the conditions generated by our analysis and SMT solving to effectively find counterexamples suffering from causal discrimination.
Revised: Moreover, we plan to integrate fairness veriﬁcation and fairness testing by using the conditions generated by our analysis and SMT solving to effectively ﬁnd counterexamples suffering from causal discrimination.

Original: Indeed, the conditions returned by our synthesis algorithm identify portions of the feature space that cannot include such counterexamples, so fairness testing can be made more effective by sampling only from different areas.
Revised: Indeed, the conditions returned by our synthesis algorithm identify portions of the feature space that cannot include such counterexamples, so fairness testing can be made more effective by sampling only from different areas.

Original: Finally, we would like to explore the generalization of our approach to capture group fairness properties of tree-based models [40].
Revised: Finally, we would like to explore the generalization of our approach to capture group fairness properties of tree-based models [40].

Revised: A CKNOWLEDGEMENT

Revised: This work was partially supported by iNEST (Interconnected NordEst Innovation Ecosystem, Project ID: ECS 00000043), funded by PNRR (Mission 4.2, Investment 1.5)

Revised: NextGeneration EU.

Original: A PPENDIX A I MPLEMENTATION D ETAILS
Revised: A PPENDIX A I MPLEMENTATION D ETAILS

Original: Another important aspect of the implementation is the treatment of categorical features, i.e., features which do not take arbitrary values in R , but may only take values from a set of unordered options, e.g., gender or ethnicity.
Revised: Another important aspect of the implementation is the treatment of categorical features, i.e., features which do not take arbitrary values in R , but may only take values from a set of unordered options, e.g., gender or ethnicity.

Original: Categorical features are not part of our model for simplicity, however, they are not difficult to handle in practice.
Revised: Categorical features are not part of our model for simplicity, however, they are not difﬁcult to handle in practice.

Original: In particular, we handle binary features just like numerical features, since the threshold 0.5 can be used to tell apart the two possible values of the feature.
Revised: In particular, we handle binary features just like numerical features, since the threshold 0.5 can be used to tell apart the two possible values of the feature.

Original: As for other categorical features, we deal with them by using one-hot-encoding, i.e., we generate a new feature f for each possible value of the original feature, with the idea that just one such feature will have a value greater than 0.5.
Revised: As for other categorical features, we deal with them by using one-hot-encoding, i.e., we generate a new feature f for each possible value of the original feature, with the idea that just one such feature will have a value greater than 0.5.

Original: We then enforce that an itemset cannot contain two or more items of the form x f > 0 .
Revised: We then enforce that an itemset cannot contain two or more items of the form x f > 0 .

Original: 5 that predicate on features resulting by one-hot-encoding the same categorical feature.
Revised: 5 that predicate on features resulting by one-hot-encoding the same categorical feature.

Original: Since the categorical features and the result of their one-hot-encoding are known, the implementation of the meet operator checks whether the generated itemsets respect the described integrity condition and the meet is considered undefined in case of violations.
Revised: Since the categorical features and the result of their one-hot-encoding are known, the implementation of the meet operator checks whether the generated itemsets respect the described integrity condition and the meet is considered undeﬁned in case of violations.

Original: The resulting itemsets in the result F are postprocessed in order to improve their readability.
Revised: The resulting itemsets in the result F are postprocessed in order to improve their readability.

Original: In particular, we collapse itemsets that differ just for formulas predicating on different values of the same categorical feature by merging these formulas in a single formula using a disjunction.
Revised: In particular, we collapse itemsets that differ just for formulas predicating on different values of the same categorical feature by merging these formulas in a single formula using a disjunction.

Original: Finally, we mention a few optimizations of the implementation that we borrowed and adapted from Apriori [30].
Revised: Finally, we mention a few optimizations of the implementation that we borrowed and adapted from Apriori [30].

Original: The items within an itemset and the itemsets themselves are kept ordered to reduce the number of executions of the meet operator at line 13 and optimize its checks.
Revised: The items within an itemset and the itemsets themselves are kept ordered to reduce the number of executions of the meet operator at line 13 and optimize its checks.

Original: The items within an itemset are ordered as follows: first, we follow the lexicographic order of the feature involved in the item; then, for any two items involving the same feature, the formulas with predicate ≤ precede the ones with predicate > ; finally, the items with predicate ≤ are ordered by decreasing value of the threshold, while we do the opposite for the other items.
Revised: The items within an itemset are ordered as follows: ﬁrst, we follow the lexicographic order of the feature involved in the item; then, for any two items involving the same feature, the formulas with predicate ≤ precede the ones with predicate > ; ﬁnally, the items with predicate ≤ are ordered by decreasing value of the threshold, while we do the opposite for the other items.

Original: The order of the items makes it possible to define the prefix of an itemset of size k as the ordered sequence of its first k − 1 items.
Revised: The order of the items makes it possible to deﬁne the preﬁx of an itemset of size k as the ordered sequence of its ﬁrst k − 1 items.

Original: Itemsets are ordered to examine itemsets sharing the same prefix consecutively, since only itemsets sharing the same prefix satisfy the preconditions for performing the meet operation.
Revised: Itemsets are ordered to examine itemsets sharing the same preﬁx consecutively, since only itemsets sharing the same preﬁx satisfy the preconditions for performing the meet operation.

Original: This allows one to reduce the number of times that the meet operation is attempted between two itemsets that do not satisfy the preconditions of the meetdefinition.
Revised: This allows one to reduce the number of times that the meet operation is attempted between two itemsets that do not satisfy the preconditions of the meet

Original: In particular, two ordered itemsets of size k are lexicographically ordered using the established ordering of the items: at first, their prefixes are compared starting from the first item to

Original: [Table]
Revised: [Table]

Original:  the ( k − 1) -th item; then, if the two itemsets share the same prefix, they are ordered by comparing their last, different item.
Revised: deﬁnition. In particular, two ordered itemsets of size k are lexicographically sorted using the established order of the items: at ﬁrst, their preﬁxes are compared starting from the ﬁrst item to the ( k − 1) -th item; then, if the two itemsets share the same preﬁx, they are ordered by comparing their last, different item.

Original: Moreover, the order of the items followed by the itemsets sharing the same prefix enables an optimization of the checks of the meet operation.
Revised: Moreover, the order of the items followed by the itemsets sharing the same preﬁx enables an optimization of the checks of the meet operation.

Original: Indeed, for any itemset I ′ listed after an itemset I sharing the same prefix we have (cid:74) I (cid:75)
Revised: Indeed, for any itemset I (cid:48) listed after an itemset I sharing the same preﬁx we have (cid:74) I (cid:75)

Original: ̸⊂ (cid:74) I ′ (cid:75) .
Revised: (cid:54)⊂ (cid:74) I (cid:48) (cid:75) .

Original: As a result, the implementation of the second condition of the meet I ⊓ I ′ just needs to check (cid:74) I ′ (cid:75) ⊂ (cid:74) I (cid:75) .
Revised: As a result, the implementation of the second condition of the meet I (cid:117) I (cid:48) just needs to check (cid:74) I (cid:48) (cid:75) ⊂ (cid:74) I (cid:75) .

Original: Another optimization involves the condition at line 15, which would require scanning the entire set U for every generated itemset.
Revised: Another optimization involves the condition at line 15, which would require scanning the entire set U for every generated itemset.

Original: We optimize this step by assigning to each H ∈ U an identifier id H ∈ N and associating to each itemset I ∈ C a set ids I = { id H : H ∈ U ∧ H ∩ (cid:74) I (cid:75)
Revised: We optimize this step by assigning to each H ∈ U an identiﬁer id H ∈ N and associating to each itemset I ∈ C a set ids I = { id H : H ∈ U ∧ H ∩ (cid:74) I (cid:75)

Original: = ∅} .
Revised: = ∅} .

Original: Then, the set of the identifiers associated to the meet I = I⊓ I 2 of two itemset I 

Revised: Then, the set of the identiﬁers associated to the meet I = I(cid:117) I 2 of two itemset I

Original: and I 2 is ids
Revised: and I 2 is ids

Original: I
Revised: I

Original: =
Revised: =

Original: ids I 1 ∪ ids
Revised: ids I 1 ∪ ids

Original: I 2 , since I identifies less instances than both I 

Revised: I 2 , since I identiﬁes less instances than both I

Original: and I 2 by definition.
Revised: and I 2 by deﬁnition.

Original: This information is useful, because the condition at line 15 amounts just to checking whether | ids I | = | U | , rather than scanning U .
Revised: This information is useful, because the condition at line 15 amounts just to checking whether | ids I | = | U | , rather than scanning U .

Original: A PPENDIX B C ASE S TUDY : G ERMAN D ATASET
Revised: A PPENDIX B C ASE S TUDY : G ERMAN D ATASET

Original: In this section, we present some examples of logic formulas generated by our synthesis algorithm to show that they provide effective explanations about the fairness of the model.
Revised: In this section, we present some examples of logic formulas generated by our synthesis algorithm to show that they provide effective explanations about the fairness of the model.

Original: We consider logic formulas synthesized for a tree ensemble with 13 trees of maximum depth six, trained on the German dataset.
Revised: We consider logic formulas synthesized for a tree ensemble with 13 trees of maximum depth six, trained on the German dataset.

Original: This dataset consists of 1,000 people who would like to get credit from a bank and the corresponding classification task consists in classifying their requests as high or low risk.
Revised: This dataset consists of 1,000 people who would like to get credit from a bank and the corresponding classiﬁcation task consists in classifying their requests as high or low risk.

Original: We consider sex as the sensitive feature, so the logic formulas provide us the description of subsets of people whose credit risk prediction does not change by flipping their sex.
Revised: We consider sex as the sensitive feature, so the logic formulas provide us the description of subsets of people whose credit risk prediction does not change by ﬂipping their sex.

Original: We present the top 5 most important formulas returned by the synthesis algorithm in Table III, in decreasing order of importance.
Revised: We present the top 5 most important formulas generated after six iterations of the synthesis algorithm in Table III, in decreasing order of importance.

Original: We immediately observe that the formulas are explainable, since they predicate on at most four features.
Revised: We immediately observe that the formulas are explainable, since they predicate on at most four features.

Original: We then comment three representative formulas below.
Revised: We then comment three representative formulas below.

Original: The first-ranked logic formula is interesting to examine, since it covers a subset of people that we expect to be highly represented also in the test set, because the formulas are ranked by the importance computed on the training set (see
Revised: The ﬁrst-ranked logic formula is interesting to examine, since it covers a subset of people that we expect to be highly represented also in the test set, because the formulas are ranked by the importance computed on the training set (see

Original: Section IV).
Revised: Section IV).

Original: In particular, it indicates that an individual that has no checking account, possesses a saving account whose amount is registered with the bank and has no installment plans will not receive a different credit risk depending on their sex.
Revised: In particular, it indicates that an individual that has no checking account, possesses a saving account whose amount is registered with the bank and has no installment plans will not receive a different credit risk depending on their sex.

Original: The formula suggests that the person’s savings represent enough information for assigning the credit risk irrespective of the sex of the requester, at least when no information about the individual’s checking account or other installment plans is available.
Revised: The formula suggests that the person’s savings represent enough information for assigning the credit risk irrespective of the sex of the requester, at least when no information about the individual’s checking account or other installment plans is available.

Original: The second formula is interesting too.
Revised: The second formula is interesting too.

Original: In particular, the formula indicates that the model does not discriminate by sex individuals who request a small credit amount (betweenand ∼ 7,500 DM), do not have a registered checking account and do not present a critical credit history.
Revised: In particular, the formula indicates that the model does not discriminate by sex individuals who request a small credit amount (betweenand ∼ 7,500 DM), do not have a registered checking account and do not present a critical credit history.

Original: The formula highlights a reasonable behavior of the ML classifier: when requesting small credit amounts, the presence of a good credit history already suffices for taking a decision, without relying also on the individual’s sex.
Revised: The formula highlights a reasonable behavior of the ML classiﬁer: when requesting small credit amounts, the presence of a good credit history already sufﬁces for taking a decision, without relying also on the individual’s sex.

Original: Finally, the fourth formula is the only one in the topformulas that predicates on individuals with a bad credit history.
Revised: Finally, the fourth formula is the only one in the topformulas that predicates on individuals with a bad credit history.

Original: In particular, it explains that an individual who requests a small credit amount, has no checking account and other installment plans, but presents a critical credit history, is not discriminated by the classifier based on his sex.
Revised: In particular, it explains that an individual who requests a small credit amount, has no checking account and other installment plans, but presents a critical credit history, is not discriminated by the classiﬁer based on his sex.

Original: This is interesting because bad credit history should be the most important information when assessing a loan request.
Revised: This is interesting because bad credit history should be the most important information when assessing a loan request.

Original: The reason why this only emerges for small loan requests might be twofold.
Revised: The reason why this only emerges for small loan requests might be twofold.

Original: First, assessing requests for small loans is expected to be easier, hence the corresponding proof of fairness is also easier and is established with a small number of analysis iterations.
Revised: First, assessing requests for small loans is expected to be easier, hence the corresponding proof of fairness is also easier and is established with a small number of analysis iterations.

Original: Moreover, the dataset includes many such requests, hence the importance of the formula increases and leads to its inclusion in the top formulas.
Revised: Moreover, the dataset includes many such requests, hence the importance of the formula increases and leads to its inclusion in the top formulas.

Original: In conclusion, the synthesized logic formulas are useful to the analyst to conclude whether fairness guarantees can be provided for particular subgroups of instances in the domain of interest, not just for specific instances in the test set.
Revised: In conclusion, the synthesized logic formulas are useful to the analyst to conclude whether fairness guarantees can be provided for particular subgroups of instances in the domain of interest, not just for speciﬁc instances in the test set.

Original: A PPENDIX C P ERFORMANCE OF THE S TABILITY A NALYSIS
Revised: A PPENDIX C P ERFORMANCE OF THE S TABILITY A NALYSIS

Original: We present some additional results about the time required by the stability analysis proposed in [26].
Revised: We present some additional results about the time required by the stability analysis proposed in [26].

Original: The analysis is based on an iterative algorithm that supports iterative refinements in order to deal with the exponential complexity characterizing the problem of verifying decision tree ensembles [29].
Revised: The analysis is based on an iterative algorithm that supports iterative reﬁnements in order to deal with the exponential complexity characterizing the problem of verifying decision tree ensembles [29].

Original: In our experiments, we fix the number of analysis iterations to 100, because a limited number of iterations already suffices to obtain a reasonably accurate over-approximation of the subset of the feature space where the ensemble is unstable [26].
Revised: In our experiments, we ﬁx the number of analysis iterations to 100, since a limited number of iterations already sufﬁces to obtain a reasonably accurate over-approximation of the subset of the feature space where the ensemble is unstable [26].

Original: We evaluate the performance of the stability analysis when varying the number of trees in the ensemble, as well as their depth, on the three datasets used in Section IV to assess our proposal.
Revised: We evaluate the performance of the stability analysis when varying the number of trees in the ensemble, as well as their depth, on the three datasets used in Section IV to assess our proposal.

Original: The experimental results are shown in Figure 8 and Figure 9, which show how the running time of the stability analysis changes when increasing the number of trees and their maximum depth respectively.
Revised: The experimental results are shown in Figure 8 and Figure 9, which show how the running time of the stability analysis changes when increasing the number of trees and their maximum depth respectively.

Original: For all the considered datasets and models, the analysis for 100 iterations terminates in a matter of seconds, thus showing the practicality of the stability analysis for the considered cases.
Revised: For all the considered datasets and models, the analysis for 100 iterations terminates in a matter of seconds, thus showing the practicality of the stability analysis for the considered cases.

Original: Even though the analysis of models trained on the Adult dataset requires more time than analyzing the same models trained on the other two datasets, the stability analysis takes at most 16 seconds on the ensemble of 13 trees with maximum depth seven.
Revised: Even though the analysis of models trained on the Adult dataset requires more time than analyzing the same models trained on the other two datasets, the stability analysis takes at most 16 seconds on the ensemble of 13 trees with maximum depth seven.

Original: Note that the time required by the stability analysis normally represents a small amount of the time required by the synthesis algorithm developed in our work.
Revised: Note that the time required by the stability analysis normally represents a small amount of the time required by the synthesis algorithm developed in our work.

Original: For example, consider the case of an ensemble of 13 trees with maximum depth six, trained on the Adult dataset: the stability analysis just takes six seconds, while the synthesis algorithm takes more thanminutes (to generate conditions of length six).
Revised: For example, consider the case of an ensemble of 13 trees with maximum depth six, trained on the Adult dataset: the stability analysis just takes six seconds, while the synthesis algorithm takes more thanminutes (to generate conditions of length six).

Original: We do not try to further improve the precision of the stability analysis, since the main goal of our experimental evaluation is assessing the effectiveness of the synthesis algorithm, however, these results show that it would be certainly possible to replace the current implementation of the stability analysis with a more expensive, yet more precise alternative.
Revised: We do not try to further improve the precision of the stability analysis, since the main goal of our experimental evaluation is to assess the effectiveness of the synthesis algorithm. However, these results show that it would be certainly possible to replace the current implementation of the stability analysis with a more expensive, yet more precise alternative.

Original: 15 #trees       t
Revised: 15 #trees       t

Original: i m e ( s )
Revised: i m e ( s )

Original: Adult depth=6, #iters= (a)     15 #trees     t i m e ( s )
Revised: Adult depth=6, #iters= (a)     15 #trees     t i m e ( s )

Original: German depth=6, #iters= (b)     15 #trees     t
Revised: German depth=6, #iters= (b)     15 #trees     t

Original: i m e ( s )
Revised: i m e ( s )

Original: Health depth=6, #iters= (c)
Revised: Health depth=6, #iters= (c)

Original: [Figure]

Original: [Figure]
Revised: [Figure]

Original: 7 Depth         t i m e ( s )
Revised: Depth         t i m e ( s )

Original: Adult #trees=13, #iters= (a)   7 Depth        t i m e ( s )
Revised: Adult #trees=13, #iters= (a)   7 Depth        t i m e ( s )

Original: German #trees=13, #iters= (b)   7 Depth       t i m e ( s )
Revised: German #trees=13, #iters= (b)   7 Depth       t i m e ( s )

Original: Health #trees=13, #iters= (c)
Revised: Health #trees=13, #iters= (c)