Paper: Enhancing Group Fairness in Online Settings Using Oblique Decision Forests (ICLR-2024)

Reviewer comment (Theory):
Utilizing the aggregate of past statistics to adjust the model parameters poses a challenge. As the model parameters shift with each time step, past statistics, derived from earlier model parameters, might diverge significantly from what would be obtained if the model gradients were calculated using all the previously stored data. I'm curious if the authors could elaborate on the conditions under which using aggregated past gradients to update the current model would be successful or not.

Author response:
> Conditions for online learning using stale gradients

Our proposed method, Aranyani, is a tree structure specialized and related learning algorithm designed to effectively use such aggregated computations. The tree structure is parameterized using a logistic network at each node. We compute fairness gradients by taking the derivative of node outputs. Crucially, this structure allows us to prove that the error in computing the fairness gradients cannot diverge in an unbounded manner. 

The theoretical results we propose rely solely on the assumptions of bounded inputs, a compact parameter set, and smooth activation functions, as well as a smooth task loss. This allows us to derive the explicit bound in Lemma 3 (Page 6), which shows that the error is a function of the $\delta$ parameter (associated with the Huber loss) and input bound $B$.

Reviewer comment (Writing):
[Major] Problem setting should be stated more clearly. For instance, in this paper, considers an online setting where only one single instance can be processed. Typical online learning indeed process one data point at a time but also privy to the feedback from previous data points, but this paper seems to assume that the previous data points are not available. A more clearly setup should be stated before getting into the gradients used in the algorithm (5).

Author response:
> Fairness-aware online learning setting and the significance of storing past samples within this context.

As mentioned by the reviewer, to achieve group fairness in online settings, we assume that we can only store aggregate feedback from previous data points (e.g., aggregate of previous gradients). We assume that we cannot store the actual data points that contain sensitive demographic information. 

Unlike typical online learning setups, it is not possible to obtain a fairness regularizer gradients update using a single instance. This is because group fairness regularizers are defined using a group of instances. This poses an interesting challenge of updating the model at each step without accessing previous instances.

This is the major challenge that we overcome in this paper. We have updated Section 3.1 to articulate the setup and our contribution better.

Reviewer comment (Presentation):
Where is the FAHT’s result in the rightmost plot in Figure 2?

Author response:
> Missing FAHT results on COMPAS

In the updated draft, we have incorporated the results of FAHT for the COMPAS dataset (rightmost plot of Figure 2). 

Before submission, we had difficulty re-running and reproducing the Java-based FAHT implementation on our own and reported the FAHT results directly from their paper (on the datasets the authors evaluated: Adult & Census). With additional work, we were able to modify the FAHT codebase to run on the remaining datasets. Updated results are reported in Figure 2. We observe that Aranyani consistently outperformed FAHT. For example, on the Adult dataset, Aranyani can significantly reduce the DP score (0.19 $\rightarrow$ 0.11) while achieving similar levels of accuracy.

Reviewer comment (Evaluation):
Why is FAHT worse than the adaptive HT in terms of DP?

Author response:
> Comparison of the performance between FAHT and adaptive HT

We have replicated the FAHT baseline and updated the results in Figure 2. The updated results show that FAHT slightly lags behind adaptive HT in DP only in the Adult dataset. The underlying implementations of Hoeffding Trees differ between FAHT and adaptive HT. FAHT is implemented in Weka ([FAHT github](https://github.com/vanbanTruong/FAHT)) and Adaptive HT in sklearn ([Adaptive HT github](https://github.com/scikit-multiflow/scikit-multiflow/blob/a7e316d1cc79988a6df40da35312e00f6c4eabb2/src/skmultiflow/trees/hoeffding_adaptive_tree.py)). The scikit-learn implementation of Adaptive HT exhibited numerical instabilities at times that could have affected its convergence. We hypothesize that this issue might have influenced the results on the Adult dataset.

Reviewer comment (Reproducibility):
The manuscript would be enhanced with a more detailed breakdown of the concluding stages of the training process. It remains uncertain whether the oblique decision tree is designed as an end-to-end neural network or if Aranyani follows the training pattern of standard tree models while adopting a neural configuration.

Author response:
> Details of Aranyani’s training procedure.

We have now provided a detailed outline of Aranyani’s training procedure in Appendix C.3 (Algorithm 1). 

Yes, Aranyani can be trained as an end-to-end neural network. The task loss gradients can be computed directly using standard autograd libraries. Also, for each input gradient w.r.t. node outputs can be efficiently computed. Then, these node outputs and gradients are aggregated over time to form the final gradients shown in Equation 5. We have implemented Aranyani using TensorFlow.

Reviewer comment (Evaluation):
It would be beneficial to have deeper insights regarding Aranyani's subdued performance on the COMPAS dataset. Could there be challenges in using Aranyani for datasets with less than 7k entries?

Author response:
> Impact of dataset size on Aranyani’s performance

We have performed additional experiments to evaluate the robustness of Aranyani when the dataset size is reduced. We report the performance in Appendix D (Subsection: Dataset Size Ablation). We observe that the tradeoff between fairness and utility gradually deteriorates with a decrease in the dataset size. This happens primarily because it is difficult to achieve higher accuracies with lesser data.

Reviewer comment (Experiments):
While the paper indicates that Aranyani is more suited to trees of limited height, it would be useful to explore alternative approaches for instances where greater tree height is needed.

Author response:
> Impact of tree depth on Aranyani’s performance 

In ablation experiments (Section 5.3), we experimented with deep trees up to a height of 8 and investigated its impact on the performance. Figure 4, shows the tradeoff between accuracy (improves with height) and DP (worsens with height) with tree height. 

In practice, we have observed that a small tree depth is generally adequate for the majority of tasks. This aligns with findings from prior studies, such as TEL [1] (which employed a tree depth of 4) and DGT [2] (which utilized tree depths up to 10). 

[1] Tree Ensemble Layer, Hazimeh et al. 2020.
[2] Learning Accurate Decision Trees with Bandit Feedback via Quantized Gradient Descent, Karthikeyan et al., 2022.

Reviewer comment (Experiments):
To further solidify the paper's comparative analysis, including more recent fairness representation learning models from the related works — modified for online environments — as baselines would be valuable.

Author response:
> Comparison of Aranyani with recent batch learning algorithms

Most representation learning methods are trained in an incremental setting where they require access to task identities. Even within a task, these algorithms need access to a batch with samples from different protected groups. Such labeled instances are not available in the online learning scenario.

Due to its stochastic batch optimization procedure, we found that only FERMI [4] can be adapted to function in the online setting. We report the results in Appendix D (Subsection: Comparison with Batch Techniques). We observe that FERMI struggles to perform with batch size=1, and is unable to improve fairness in this setting.

[4] A Stochastic Optimization Framework for Fair Risk Minimization, Lowy et al. 2022

Reviewer comment (Evaluation):
The evaluations only show demographic parity results. While the authors mention how to formulate Aranyani for other group-fairness notions, they provide no results to highlight how it would perform for notions also based on the true label, e.g., equalized odds or equal opportunity.

Author response:
> Efficacy of Aranyani using other notions of fairness

We have added the results for the equality of opportunity fairness measure in Appendix D (Subsection: Equality of Opportunity). Even in this setup, we find that Aranyani achieves a strong accuracy-EO tradeoff and outperforms the majority baseline significantly.

Reviewer comment (Evaluation):
While the work measures and reports the expectation of the fairness measure at each time step, it is unclear what the fairness dynamic looks like for different methods across time as the training proceeds. In online settings, the variation of the fairness measure over time may tell a story and be essential to see. Specifically, it is not clear how the fairness measure varies across different time steps. It is not clear if the training process is relatively stable or not, and how Aranyani compares to the baselines in this respect.

Author response:
> Variance in accuracy and DP scores during the online training process.

We have performed additional experiments to examine this scenario in Appendix D (Subsection: Performance Variance). We report the results from 5 different runs on the Adult dataset. We observe that the variation in the metrics: accuracy and DP, exist only during the initial phase of the online learning process (please note that the $x$-axis is in log-scale for better visibility). After as few as 1000 samples, both the accuracy and DP scores stabilize with minimal variation.

Reviewer comment (Evaluation):
As per the appendix results, the fairness result variance seems to increase significantly as we move from a single tree to a forest. The authors motivate the need for a forest instead of a tree with a promised reduction of variance. However, in some sense, it seems that variance can increase as well. This aspect does not seem to be discussed well.

Author response:
> Variance in results with an increase in the number of trees in the forest.

Thank you for pointing this out. In the initial draft, we reported the minimum and maximum scores out of 3 runs by the error bars. We have fixed that now and report the standard deviation across 5 different runs for each tree count configuration. We have also expanded our evaluation to investigate up to 15 trees in the forest. In the updated figure, we observe a decreasing trend in the standard deviation of the reported metrics with an increase in tree count, as hypothesized in the main paper (Section 2).

Reviewer comment (Evaluation):
Figure 4 (right) includes error bars, but it is not clear how the variance is measured in this case. Is it the variance of a measure across time steps? It is not clear why the tree depth impacts the variance of fairness so greatly.

Author response:
> The influence of tree depth on the variability in Aranyani's performance.

We have updated the results in Figure 4 (right) to include the standard deviation computed across 5 runs (with different random initialization). We observe that the variance is high when the tree depth is very small, depth=2. Here the complexity of the model is low, and it is neither overfitting nor underfitting showcasing lower accuracy (high bias) and high variance. As the depth increases, the model starts to converge better and the variance decreases. At much higher depths, the variance increases again showcasing that the model may have overfit the data.

Reviewer comment (Theory):
The fairness performance seems to degrade considerably when the tree depth is higher. This follows from the theoretical finding, where the fairness bound becomes looser as we increase the tree depth. However, this may limit the expressivity of the model in some situations.

Author response:
> The influence of tree depth on the expressivity of Aranyani.

The empirical results for accuracy and fairness follow that of our theoretical results. Apart from the tree depth, there are several ways to increase the expressivity of the model: (a) using more trees, and (b) using stronger classifiers at the node level, among others. In practice, we have observed that a small tree depth is generally adequate for the majority of tasks. This aligns with findings from prior studies, such as TEL [1] (which employed a tree depth of 4) and DGT [2] (which utilized tree depths up to 10). 

[1] Tree Ensemble Layer, Hazimeh et al. 2020.
[2] Learning Accurate Decision Trees with Bandit Feedback via Quantized Gradient Descent, Karthikeyan et al., 2022.

Reviewer comment (Experiments):
There are no ablation studies that show the impact of the particular loss that is used to measure fairness, i.e., using Huber loss over $\ell_2$ loss or hinge loss.

Author response:
> Ablation with different regularization loss functions. 

In this work, we focus on the Huber loss as the regularization measure. We choose Huber as it provides us with better theoretical guarantees while using stale gradients. In general, it is possible to explore other divergence measures like L2 loss.  If we use L2 loss instead the fairness gradient in Equation 5 will have the $F_{ij} \Delta F_{ij}$ term only. The approximation error of $F_{ij}$ may increase over time leading to a worse convergence bound of the gradients. For Huber loss, it is possible to theoretically bound the error using the $\delta$ parameter.  Hinge loss on the other hand is used to measure the classification performance and we could not think of how to apply it directly to regularize $F_{ij}$. 

We have also added the results and discussion for this experiment in Appendix D (Subsection: Regularizer Ablation).

Reviewer comment (Presentation):
Figure 4 (right) does not have any legend. It is difficult to understand which line/symbol corresponds to which metric.

Author response:
> Legends in Figure 4 (right)

We have updated the figure and included legends to improve its clarity.

Reviewer comment (Presentation):
The Aranyani triangle symbol in the text does not match with the plots.

Author response:
> Typo in Aranyani’s symbol

We have fixed the symbol in the current draft.