Venue: NeurIPS 2022 Workshop SyntheticData4ML

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: The paper studies the impact of synthetic, fair data on classifiers. The authors propose using a TabFairGAN to train GANs  under demographic parity assumption. While this helps generate data with low demographic parity, these gains are not transferred when learning models.  Experiments are performed to investigate demographic parity of downstream models.  However, no insights into this phenomenon or explanations have been offered. Perhaps one reason for this might be as follows:

While demographic parity might help ensure low correlation of a certain variable (say S) with the label, there could be information leakage via other certain variables that are correlated with S. These correlations, when apriori not controlled might lead to disappearing demographic parity at the model level. 

Could the authors provide some intuition?
review_title: Interesting paper  but without any insights

final_score: rating: 7: Good paper, accept
review_content: The paper is clearly written and experimental setup is thoroughly explained. Explaining mode collapse in separate section might increase readability for the reader. Additional paper could have been referenced to set proper context for novelty of the paper (one example below). 

https://ojs.aaai.org/index.php/AAAI/article/view/16965

The results could have been compared with other approaches on same datasets and evaluation setup. 





review_title:  Needs additional references

final_score: rating: 6: Marginally above acceptance threshold
review_content: I thank the reviewers for submitting a paper such as this one. I believe this is important work and should receive more attention. However, I have two minor remarks:
- I would have enjoyed a more intuitive explanation/conclusion for the behaviour we observe in the experiments. In fact, it seems almost expected that DP would have the same performance as the data when lambda=0? Is this a bad thing? 
- As the authors compare using only one fair synth data generator, I wonder whether the observed experiments are merely an artifact of that specific method, or whether there is a bigger point to be made here. Of course, I understand that this being a workshop, we cannot expect a full sweep of all the existing methods. However, one or two additional ones may actually result in an entirely different conclusion.


review_title: Interesting setup!

Decision: Accept