Paper: Procedural Fairness Through Decoupling Objectionable Data Generating Components (ICLR-2024)

Reviewer comment (Experiments):
There are only two experiments, one of which is a synthetic dataset.

Author response:
Thanks for the comment. In addition to the two sets of experiments on simulated data and the UCI Adult data set, we have included additional experimental results on the real-world Folktables data set (Ding et al., 2021). In particular, we consider the public health coverage prediction task where there are 17 features and 1 target variable. In light of the comment, we have included the additional experimental results in Appendix D.4, along with the side note `Re: C1 by Reviewer R8GC` on page 23.

Reviewer comment (Theory):
Does your framework require a causal model between attributes?

Author response:
Thanks for the question on the causal model. Yes, our framework utilizes the causal model between attributes. Our assumption on the availability of the causal model (e.g., in terms of a causal graph) is consistent with previous literature on causal fairness. For the convenience of locating the related material, we provide the side note `Re: Q2 by Reviewer R8GC` on page 5.

Reviewer comment (Theory):
In practice, what does it mean to constrain the data generating process (DGP)? Are we essentially filtering out data based on constraints before learning a model?

Author response:
Thanks for the question. By "constraint" on DGP we are referring to the fairness constraints on certain causal quantities proposed in previous causal fairness literature. We are not filtering out data based on such constraints before learning a model. Instead, previous causal fairness notions propose that the causal effect of interest should be bounded. For completeness, we presented the literature review on causal fairness notions in Appendix A.1. We provide the side note `Re: Q4 by Reviewer R8GC` on page 17.

Reviewer comment (Experiments):
It's unclear how broadly the framework can be applied across different domains and whether there are any limitations to its scalability. In particular, the evaluation seems to be limited, with the only "real-world" dataset used is the UCI adult dataset where only 6 features are used.

Author response:
Thanks for considering the applicability and scalability of our approach! Our framework can be applied to linear and nonlinear data generating processes, and scales well with the number of variables. To demonstrate the scalability of our framework, we have included a quantitative illustration of the computational cost (with more than 1,000 variables) in Appendix D.1.2. The computational cost of our approach is comparable to that of the vanilla regression model (implemented with a two-hidden-layer neural network). For the convenience of locating the related material, we provide the side note `Re: C1 by Reviewer 2K3E` on page 24.

Reviewer comment (Reproducibility):
I also did not find a discussion regarding the practicality of implementing the proposed framework.

Author response:
Thanks for carefully considering the practicality of implementing our framework! In addition to our code implementation in the supplementary material (we provided [the anonymous link](https://anonymous.4open.science/r/DecouplingObjectionable) in the paper), we have also included descriptions of our implementation details in Appendix D.1.1, along with the side note `Re: C2 by Reviewer 2K3E` on page 24.

Reviewer comment (Experiments):
Have you tried to use the proposed framework to more complex datasets (more variables and larger domains)? How does it scale? What happens when you have multiple confounding variables?

Author response:
Thanks for the question on experiments with additional data. In addition to the two sets of experiments on simulated data and UCI Adult data set, we have included additional experimental results on the real-world Folktables data set (Ding et al., 2021). In particular, we consider the public health coverage prediction task where there are 17 features and 1 target variable. In light of the comment, we have included the additional experimental results in Appendix D.4, along with the side note `Re: Q3 by Reviewer 2K3E` on page 28.

Reviewer comment (Evaluation):
As a follow-up from the previous question; What are the computational costs associated with implementing the framework, and how do they compare to existing methods?

Author response:
Thanks for the question about computational costs. The decoupling of objectionable data generating components does not require retraining the modeling for local causal modules. The computational overhead only comes from the inference phase of the model during the simulated annealing process (to optimize reference point configurations). The computation cost of our approach is comparable to that of existing methods based on constrained optimization. In light of the question, we have included the quantitative illustration as well as the above discussion on computational costs in Appendix D.1.2, along with the side note `Re: Q5 by Reviewer 2K3E` on page 24.