Are Two Heads the Same as One? Identifying Disparate Treatment in Fair Neural Networks
Abstract
We show that deep networks that are trained to satisfy demographic parity fairnessdo so through a form of race or gender awareness, and that the more we forcea network to be fair, the more accurately we can recover race or gender fromthe internal state of the network.
Based on this observation, we investigate analternative fairness approach: we add a second classiﬁcation head to the networkto explicitly predict the protected attribute (such as race or gender) alongside theoriginal task.
After training the two-headed network, we enforce demographicparity by merging the two heads, creating a network with the same architecture asthe original network.
We establish a close relationship between existing approachesand our approach by showing (1) that the decisions of a fair classiﬁer are wellapproximated by our approach, and (2) that an unfair and optimally accurateclassiﬁer can be recovered from a fair classiﬁer and our second head predictingthe protected attribute.
We use our explicit formulation to argue that the existingfairness approaches, just as ours, demonstrate disparate treatment and that they arelikely to be unlawful in a wide range of scenarios under the US law.
Introduction
Autonomous systems that make substantive decisions about people must conform to relevant antidiscrimination legislation.
Within the US legal system, two common tests of anti-discriminationlegislation are referred to as disparate treatment and disparate impact [37].
Disparate treatmentcorresponds to the idea that people should not be treated differently because of their membership of aprotected group (such as race or gender) while disparate impact refers to the idea that seeminglyneutral practices should not cause a substantial adverse impact to a protected group.
Importantly, ithas been argued [7] that there are a large range of scenarios where disparate treatment is unlawfuleven when performed as a remedy to disparate impact.
This is in departure from the EU where morelatitude exists when rectifying indirect discrimination (analogous to disparate impact) [67].
Consequentially, disparate treatment doctrine prevents a wide range of actions intended to addresssustained inequality [9].
Of particular relevance to our work is a 1991 amendment to Title VII[60].
This amendment explicitly prohibits the “use [of] different cutoff scores for . . .
employmentrelated tests on the basis of race, color, religion, sex, or national origin”, even if done for reasons ofafﬁrmative action.
In this paper, we examine the relationship between existing methods for enforcingdemographic parity (a deﬁnition closely related to avoiding disparate impact) that are commonlyclaimed to not exhibit disparate treatment and methods for demographic parity that alter the cutoffscore on the basis of inferred race or gender.
In doing so, we raise fundamental questions about thelegality of existing approaches for enforcing demographic parity.
In particular, we examine the behavior of deep neural networks trained to satisfy demographicparity either with a fairness regularizer [72] or by preprocessing [33].
Such models could learn
[Figure]
an internal representation that either (i) is independent of the protected attribute (as methods fromfair representation learning aim to do—see Appendix B for related work), or (ii) that distinguishesbetween groups so as to tune “separate” classiﬁers for each group in a way that results in thedemographic parity of the network.
In the context of US law, it is vital to understand which caseoccurs in practice.
If the learned algorithms treat people differently on the basis of their race or gender,this may correspond to disparate treatment .
We ﬁnd that networks trained to satisfy demographicparity via a fairness regularizer or preprocessing fall into the second case.
Figure 1 provides anillustration of this ﬁnding.
Moreover, we ﬁnd that the more strongly demographic parity is enforced,the more predictive the internal representation is of the protected attribute.
Building on the observation that the protected attribute is implicitly learned, we train a neural networkwith a second classiﬁcation head that explicitly predicts the protected attribute.
After training, we usethe second head response to directly reduce demographic disparity.
On certain tasks we show that ourapproach is tightly related to existing regularizer and preprocessing approaches and conclude thatthese approaches, just as ours, demonstrate disparate treatment.
To summarize, we make the following contributions:
The internal state of a fair network trained with a fairness regularizer or preprocessing is strongly predictive of the protected attribute.
As the fairness is more strongly enforced, the accuracy of predicting the protected attribute increases (Section 3).
Merging a second classiﬁcation head with the network is an effective approach for creating
demographic parity-fair classiﬁers.
We show that the performance of our approach closely tracks
the optimal accuracy-fairness trade-offs of Lipton et al.
[42]; however, in contrast to their method, ours does not require explicit access to the protected attribute (Section 4).
In our main contribution , we show cases where the decisions of a fair model (via regularizer orpreprocessing) are well approximated by our approach.
Similarly, the decisions of an unconstrainedclassiﬁer can be approximated by a weighted sum of the fair classiﬁer and our second head(Section 5.1).
Using the close relationship between a demographic parity-fair classiﬁer and our explicit approach
we are able to identify individuals who are systematically disadvantaged by the fair classiﬁer, and who would receive a positive decision if their apparent race or gender were different.
This allows us to conclude that the fair method exhibits disparate treatment (Section 5.2).
Preliminaries
A binary classiﬁer is a function h : X → Y that, given a datapoint x from feature space X , aimsto accurately predict the datapoint’s assigned label y ∈ Y = { 0 , 1 } .
We consider thresholded classiﬁers of the form h ( x )
= 1 ( f ( x ) > 0) , where f is a continuous function f : X → R .
Wewant h to be as accurate as possible, while at the same time being fair with respect to a protectedattribute s ∈ S
= { 0 , 1 } according to the fairness notion of demographic parity.
Deﬁnition 2.1 (Demographic Parity)
A classiﬁer h : X → { 0 , 1 } satisﬁes demographicparity under distribution P if its prediction h ( X ) is independent of the protected attribute S , i.e.,
P ( h ( X ) = 1 | S = 0) =
P ( h ( X ) = 1 | S = 1) .
We measure the violation of demographic parity by the demographic disparity (DDP) , given by
[Equation]
In this paper, we examine two popular fairness methods for training deep neural networks to satisfydemographic parity:
Regularized Fair Classiﬁcation One common approach for learning a fair classiﬁer is to add aregularizer to the standard training objective [e.g., 11, 72, 47, 44, 40, 56, 8].
Such regularizersare used to enforce numerous fairness deﬁnitions and typically impose a continuous relaxation ofa discrete fairness measure such as the DDP (1).
The regularizer used in [72] is a sigmoid-basedrelaxation of the squared value of (1), evaluated on a dataset ( x i , y i , s i ) ni
=1 with i.i.d.
samples from P :(cid:98) R DP ( f ) := (cid:16) |{ i : s i = 1 }| (cid:88) i ∈
[ n ]: s i =
σ ( f ( x i ))
− |{ i : s i = 0 }| (cid:88) i ∈ [ n ]: s i =
σ ( f (
[Equation]
The fairness regularizer is added to the overall loss and trades-off fairness versus accuracy via amultiplicative hyperparameter λ , where higher values of lambda encourage an increase in fairness atthe cost of accuracy.
In the main paper, we report experimental results for (cid:98) R DP ( f ) .
In Appendix D wepresent the results for a related regularizer.
Note that we follow [8, 72] in applying the regularizer (2)to the sigmoid output of the networks.
This differs from approaches such as [76, 16] that enforcerelaxed fairness constraints on real-valued margin distances and have recently been criticized for theirineffectiveness [44].
Preprocessing: Massaging the Dataset We also examine the preprocessing method of [33], whichalters target labels prior to training.
Massaging ‘promotes’ negative points from the disadvantagedgroup to the positive label if they have a (comparatively) high positive class probability as calculatedby an unconstrained classiﬁer and ‘demotes’ positive points from the advantaged group to the negativelabel if they have a low positive class probability.
The number of changed labels is controlled by aparameter λ ∈ [0 , M ] , where λ
=
M results in the same fraction of positive points for both groups.
To set the ground for our paper, we discuss the most closely related work of Lipton et al.
Wediscuss further related work, including methods for fair representation learning, in Appendix A and B.
Implicit Disparate Treatment Lipton et al.
[42] examine the popular claim that machine learningmodels that do not use protected information at test time do not exhibit disparate treatment [24, 77,76, 16, 74, 47, 26].
Lipton et al.
recommend caution and observe that if the protected attribute sis a deterministic function s = g ( x ) of the non-protected features x , any sufﬁciently powerful MLmodel can learn a function f
( x, s ) =
˜ f
( x ) with ˜ f
( x ) =
f ( x, g ( x )) .
They argue that even thoughthe protected attribute is not provided at test time, such a model would constitute a case of disparatetreatment since it makes decisions based on the implicitly reconstructed protected attribute.
However,beyond a synthetic experiment in which a classiﬁer discriminates based on hair length (as a proxy forgender), they do not study whether—or how —implicit disparate treatment happens in practice.
Wecomplement their work by providing strong evidence that deep neural networks that are enforced tosatisfy demographic parity by means of a regularizer or preprocessing suffer from disparate treatment,even when not explicitly using the protected attribute at test time—and that they do so by separatinglast-layer representations based on protected attributes.
A second contribution of Lipton et al.
[42] is to prove that the most accurate classiﬁer among allclassiﬁers satisfying a bound on the demographic disparity utilizes group-speciﬁc thresholds (a similarresult has also been shown by Menon and Williamson [49]).
They then propose a postprocessingmethod that greedily chooses per-group decision thresholds for a classiﬁer that has been learnedwithout fairness constraints; this requires access to protected attributes at test time.
In Section 4 wecompare our proposed two-head approach for demographic parity-fairness to their approach.
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table] [Figure] [Table] [Figure]
[Table]
[Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
Young
Smiling
Male
High_Cheekbones
Eyeglasses
Black_Hair
Bangs
Bags
[Figure]
Under_Eyes
Attractive
[Figure]
ec
[Figure]
d
[Figure]
tt
[Figure]
Protected Attribute Awareness in Fair Networks
Here, we provide evidence that deep networks that use a fairness regularizer or preprocessing tosatisfy demographic parity, learn an internal representation that separates groups, thus allowing eachgroup to be treated differently.
To measure how well a fair network separates the protected groups,we examine how accurately a linear classiﬁer can recover the protected attribute from the output ofthe last layer.
We present results for the fairness regularizer (cid:98) R DP ( f ) deﬁned in (2) on one dataset;see Appendix D.1 for the results with Massaging [33], another regularizer and on a second dataset.
Experimental Setup From the CelebA image dataset [43] and nine of its 40 binary attributes, wechoose a target attribute and a protected attribute, such as S MILING and Y OUNG .
We use [55] asreference for restricting our study to only nine of the 40 attributes.
For each distinct pair of target andprotected attribute, we train ResNet50 models for twelve different values of the fairness parameter λ .
For each model, we then train a linear classiﬁer using logistic regression to predict the protectedattribute from the model’s frozen last-layer representation.
We refer to these classiﬁers as the groupclassiﬁers.
See Appendix C.1 for technical details and descriptions of the datasets.
Evaluation For each pair of target and protected attribute, we evaluate if increasing λ increasesthe accuracy of the group classiﬁer.
We test for a monotonic relationship using the Kendall-taucorrelation τ [35] on 12 datapoint pairs (consisting of fairness parameter λ
and the accuracy ofthe group classiﬁer).
Additionally, we compute a two-sided p-value for the null hypothesis ofindependence between λ and the group classiﬁer’s accuracy.
Since the regularized approach cancollapse to a trivial near-constant classiﬁer when λ is too large, we discard models when their accuracyis too close to the one of the constant classiﬁer (concretely, if their accuracy is not at least the constantclassiﬁer’s accuracy plus 25% of the additional accuracy that the unconstrained classiﬁer achieves).
Results Figure 2 summarizes our results.
For 71 out of 72 experiments the Kendall-tau correlationshows a positive correlation between λ and the protected attribute accuracy, with p < 0 .
05 for
[Figure]
Classification Task Attractive with Protected Attribute Male
62 experiments.
Those experiments show a very strong monotonic relationship with a correlationhigher than 0 . 49 .
For nine experiments, we observe lower but still positive values of the correlationcoefﬁcient τ .
Only for one experiment we observe τ < 0 , but with insigniﬁcant p = 0 . 64 .
Conclusion For both fairness regularizers and the Massaging preprocessing method we ﬁnd thatincreasing the fairness parameter increases the ability to recover the protected attribute fromthe last layer.
This increase in accuracy is a cause for concern since disparate treatment can occur ifa system is able to infer the protected attribute (cf.
Section 2).
We build on this initial analysis in
Sections 4 and 5 and show that disparate treatment occurs in practice.
Explicit Group Awareness for Enforcing Demographic Parity
In this section, we evaluate a novel post-processing approach explicitly designed to exhibit disparate treatment while using the same single network architecture used by preprocessing andregularized approaches.
In short, a network with two fully-connected output heads on the last-layerrepresentation is trained: one head f : X → R is trained to minimize a logistic loss over the targetvariable y , and the other head g : X → R to minimize a squared loss over the protected attribute s .
We use the squared loss to encourage g to take values close to zero or one.
Overall, we minimize thefollowing objective:
[Equation]
A weighted sum of the two heads results in a classiﬁer F that satisﬁes (approximate) demographicparity.
The function F takes the form F ( x ) =
f ( x ) + a 1 g ( x ) + a 2 for coefﬁcients a 1 , a 2 ∈ R .
Even though we train two heads, we can compress them into a single head and generate a singleheaded architecture that makes the same decisions .
More precisely if f ( x ) =
w f · z ( x ) + b f andg ( x ) =
w g · z ( x ) + b g , with z ( x ) being the last-layer representation, then F ( x ) =
( w f + a 1 w g ) ·z ( x ) + ( b f + a 1 b g + a 2 ) .
This results in a single-headed network with identical architecture to thepreprocessing and regularized approaches.
We consider our approach a post-processing method as itdoesn’t induce a fair classiﬁer until we compress the two heads.
As we show, there are substantial advantages to post-processing, both in terms of stability andefﬁciency of training.
However, the key limitation of existing methods is that they either requireknowledge of the protected attribute [42, 3] or more complex systems at evaluation time that must [Table]
Regularizer (cid:98) R DP
[Table]
also infer the protected attribute [73, 51].
We remove this limitation, by compressing our two-headedapproach into the same architecture used by the preprocessing and regularized approaches.
Thismatching architecture is key to the legal argument in the following sections: our approach explicitlyexhibits disparate treatment while sharing a common architecture and making similar decisions toother approaches.
Model Construction We consider a standard network backbone (e.g., ResNet50), with two heads.
We present results with a ResNet50 in the main body of the paper, and include further results with the
MobileNetV3-Small architecture in Appendix D.2.
To ﬁnd a 1 and a 2 such that the predictions of thethresholding rule 1 ( F ( x ) > 0) are fair and maximally accurate, we perform a grid search on validation data and select the most accurate classiﬁer that does not exceed a given demographic disparity.
We compare our approach to Lipton et al.
[42] who’s similar approach provides optimal per-groupthresholds, but requires explicit knowledge of the protected attributes at test time.
If the groupclassiﬁer g is perfect, our approach and Lipton’s approach coincide.
If g does not predict the protectedattribute well, our procedure can only perform worse than Lipton’s
(cf.
Theorem 4 in their paper).
Inour computer vision setting, however, the accuracy of predicting the protected attribute is typicallyvery high (e.g., on M ALE from CelebA, we achieve an accuracy of around 98% ), and, as we show,the performance of our approach is very close to that of [42].
Results Figure 3 compares the accuracy-fairness trade-off of our approach, the regularizer approach,and Lipton et al.
Since the computational cost of fully training 12 regularized models (with thesame 12 fairness parameter values as in Section 3) is much higher than training one model for ourapproach and Lipton’s, we compute 20 solutions with our grid search or Lipton’s greedy search.
Allthree methods offer similar accuracy for a particular level of demographic disparity.
However, for theregularized model, it is difﬁcult to control this trade-off or to ﬁnd very fair solutions.
In contrast, ourapproach and [42] allow easy selection of a model with a particular demographic disparity.
While theperformance of our approach and [42] is similar, we do not require the protected attribute at test time.
Table 1 reports the accuracy obtained under strict fairness constraints.
With respect to the accuracyfairness trade-off all methods perform comparably.
However, if we require a substantial reduction ofunfairness (second block of rows in Table 1), the regularizer approach and preprocessing often failto ﬁnd a valid solution.
In contrast, Lipton et al.
[42] and our approach always ﬁnd sufﬁciently fairsolutions due to their direct search for per-group thresholds.
[Figure]
Conclusion Our method has several advantages: (i) Other approaches require training a new modelfor every fairness parameter λ , which might make tuning λ very expensive until a desirable level offairness is reached.
Our approach only requires a single explicit model and the output scores of thetwo heads.
(ii)
Due to the simple weighted sum, we make the inﬂuence of the group classiﬁer inthe ﬁnal decision explicit and transparent.
In summary, our approach reliably ﬁnds high accuracysolutions for a given demographic disparity without requiring the protected attribute at test time.
Disparate Treatment in Fair Networks
In this section we examine the tight relationship between our explicit approach and the behavior offair neural networks.
First, we reconstruct the fair network with our group-aware method by buildinga weighted sum of the two heads and merging them into one (Section 4).
Second, from a given fairneural network, we recover the corresponding unconstrained model using only the fair network andthe group classiﬁer from our approach.
The relationship between fair model and our approach allowsus to identify individuals who are treated differently based on inferred group membership and todemonstrate disparate treatment.
5.1 Fair Networks Behave like the Explicit Approach
We recover the predictions of a fair model r λ by using the target task head f and group classiﬁer g .
Weuse logistic regression to ﬁnd parameters a 1 , a 2 ∈ R such that 1 ( f ( x )+ a 1 g ( x )+ a 2 > 0) accuratelyreplicates 1 ( r λ ( x ) > 0) .
Since we can merge the weighted sum into a single output head, ourtwo-headed approach has thus found a model of the same original architecture with one output headthat predicts equivalently to the fair model.
Next, we show that it is possible to recover the predictionsof the unconstrained model h with a weighted sum of the fair classiﬁer r λ and the group classiﬁerg .
We run logistic regression to ﬁnd b 1 , b 2 ∈ R such that 1 ( r λ ( x ) − b 1 g ( x ) − b 2 > 0) recovers theprediction y
=
( h ( x ) > 0) .
Coefﬁcients a 1 and a 2 , or b 1 and b 2 are found using validation data.
Error Bars and Random Baseline For both experiments, a substantial challenge lies in the randombehavior of training a deep network.
The solution found highly depends on its initial seed.
To takethis instability into account, we provide baselines that measure how decisions vary when retrainingnetworks.
In the ﬁrst experiment, we compute error bars by repeating the process for ﬁve differentinitial random seeds of r λ .
We compute a random baseline by comparing the ﬁve models to asixth, differently initialized model r λ .
In the second experiment, we recover ﬁve unconstrained models trained with different initial seeds (for the error bars) and we compute a random baseline bycomparing the predictions to a sixth unconstrained model.
Results In Figure 4 we show regularized ResNet50 models trained to predict A TTRACTIVE ; theprotected attribute is M ALE (see Appendix D.3 for other tasks, models, and preprocessing results).
The left panel evaluates how accurately our explicit approach recovers the predictions of the regularized model r λ .
We ﬁnd that most of the error in recovering predictions can be attributed toclassiﬁer instability, and that retraining a classiﬁer from scratch with a new random seed givessimilar disagreement to using our reconstructed classiﬁer.
In the right panel of Figure 4, we plotthe reconstruction accuracy of r λ − b 1 g − b 2 with respect to the unconstrained classiﬁer.
By simplyadding the group classiﬁer response g ( x ) to r λ ( x ) , we obtain the predictions of the unconstrainedclassiﬁer.
Compared to the baseline, we recover the unconstrained classiﬁer responses for allr λ with similar ﬁdelity to simply retraining the target classiﬁer from scratch.
5.2 Identifying Disparate Treatment in Deep Networks
We can now quantify the disparate treatment of a neural network.
By exploiting our explicit estimationof the protected attribute, we can ask the counterfactual question: how would the decision havechanged if the individual had belonged to the other group?
Experimental Setup As described in the previous subsection, we ﬁnd the closest weighted sumf + a 1 g + a 2 of the two heads that best replicates the decisions of a given model r λ .
Then, for everyindividual x in the test set, we replace the group classiﬁer response g ( x ) with the median output ofthe group that x does not belong to.
We evaluate how many times the prediction changes when thesecond head output is replaced by this counterfactual.
Results Figure 5 shows the proportion of individuals for whom their prediction changes.
For thefairest models in the left panel, up to 30 % of all individuals receive a different outcome when theirsecond head output g ( x ) is replaced by the median output of the other group.
This is substantiallymore than the number of changed predictions, which we obtain when retraining with a differentrandom seed (roughly 7 % of the points).
As expected, the proportion of changed predictions linearlyincreases with model fairness (governed by the parameter λ ).
Similar behavior occurs for both theregularized approach (Figure 5, left) and preprocessing (right).
While the behavior of our system is difﬁcult to distinguish from that of a retrained fair classiﬁer,the disagreement between retrained classiﬁers means that we cannot point to an individual andconclude that they received a different decision because of their protected attribute.
Nonetheless, inscenarios where changing the protected attribute alters a much greater proportion of decisions thanthe proportion of decisions where the classiﬁers disagree (see Figure 5 left and right, in contrast tocenter) we can conclude that it is likely particular individuals suffered disparate treatment.
Conclusion When fair networks show the same behavior as our explicit awareness model, we cananalyze the inﬂuence of group membership.
Using our explicit approach, we can evaluate how fairnetworks systematically treat individuals differently on the basis of their protected attribute.
Legal Implications of our Analysis
We provide an extensive discussion of the legal implications of our analysis in Appendix A; here weprovide an overview of our argument, which can be decomposed into three parts:
Disparate treatment may occur even if the treatment is based on inferred attributes (such as race orsex) rather than explicitly provided attributes.
By construction, our explicit formulation (Section 4) exhibits disparate treatment.3.
Other fairness approaches considered exhibit the same behavior as our explicit approach, and asthe relevant tests for disparate treatment are based on systematic behavior, these approaches alsoexhibit disparate treatment.
Finally, we explicitly identify the individuals likely disadvantaged by enforcing fairness, and considercircumstances where the use of such systems may be acceptable, and look at existing legal arguments.
[Figure]
Conclusion
In our main contribution we have shown how existing methods for enforcing demographic parity indeep networks learn a latent representation that is more predictive of the protected attribute.
The closecoupling between the behavior of these approaches to our explicit model provides a tool to identifyindividuals who are likely to be systematically favored or disfavored by virtue of their protectedattribute and to conclude that existing methods for enforcing fairness do so through disparate treatment.
We have shown that the use of some fairness methods on some datasets exhibits disparate treatment.
Our ﬁndings do not imply that all methods for enforcing demographic parity suffer from disparatetreatment, merely that some can, and that caution should be used when deploying such methods.
Furthermore, our approach for enforcing fairness offers a more efﬁcient and reliable way of enforcinga chosen degree of demographic parity.
Unlike regularization-based approaches that require multipletraining runs with different regularization parameters to ﬁnd a desired trade-off, our approach allowsfor joint training of both heads once, and then a search for the desired trade-off by tuning weightsusing precomputed network responses on a validation set, and then ﬁnally merging the two heads.
Assuch, it is possible to efﬁciently ﬁnd a family of classiﬁers from the original architecture, of varyingfairness and accuracy for little more compute than training an unfair classiﬁer in the ﬁrst place.
While our analysis presents several challenges to deploying fair ML systems in the US, it is consistentwith other rulings on discrimination in US law.
In general, the US requires that considerations ofequity and afﬁrmative action are satisﬁed by shaping an entire process to be more inclusive, and notsimply by imposing race or gender-based quotas on outcomes [32, 21].
However, the opaque natureof ML makes it extremely challenging to deﬁne fair algorithms without formulating the deﬁnitions interms of outcomes 1 .
For this reason, we believe that legal reform is needed to explicitly allow the useof fair ML techniques as a tool to reduce disparate impact and increase equity.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
contributions and scope?
[Yes](b) Did you describe the limitations of your work?
[Yes] In the conclusion we emphasize that wehave only found disparate treatment for some fairness methods on some datasets.(c)
Did you discuss any potential negative societal impacts of your work?
[Yes] Our analysis has
legal implications which we discuss shortly in Section 6 and in more depth in Appendix A.(d)
Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental
results (either in the supplemental material or as a URL)?
[Yes] We do not provide code as
we simply train PyTorch implementations of popular neural networks.
We provide all details
on the data and experimental details in Appendix C.
(b) Did you specify all the training details (e
g., data splits, hyperparameters, how they were
chosen)?
[Yes] See answer above
(c) Did you report error bars (e.g., with respect to the random seed after running experimentsmultiple times)?
[Yes]
[Table]
(d) Did you include the total amount of compute and the type of resources used (e.g., type ofGPUs, internal cluster, or cloud provider)?
[No] [Table]
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
[Table]
(a) If your work uses existing assets, did you cite the creators?
[Yes] We use pretrained PyTorch [Table] models and publicly available data as discussed in Appendix C.1.
[Table]
(b) Did you mention the license of the assets?
[Yes]
[Table]
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
[Table]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
[Table]
using/curating?
[No]
[Table]
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable [Table] information or offensive content?
[No] We use standard image datasets.
[Table]
If you used crowdsourcing or conducted research with human subjects...
[Table]
(a) Did you include the full text of instructions given to participants and screenshots, if applica-
[Table]
ble?
[N/A]
[Table]
(b) Did you describe any potential participant risks, with links to Institutional Review Board
[Table]
(IRB) approvals, if applicable?
[N/A]
[Table]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on
[Table]
participant compensation?
[N/A]
[Table]