O VERPARAMETERISATION AND WORST - CASE GENERALISATION : FRIEND OR FOE ?
Anonymous authors
Paper under double-blind review
A BSTRACT
Overparameterised neural networks have demonstrated the remarkable ability to perfectly ﬁt training samples, while still generalising to unseen test samples.
However, several recent works have revealed that such models’ good average performance does not always translate to good worst-case performance: in particular, they may perform poorly on subgroups that are under-represented in the training set.
In this paper, we show that in certain settings, overparameterised models’ performance on under-represented subgroups may be improved via post-hoc processing.
Speciﬁcally, such models’ bias can be restricted to their classiﬁcation layers, and manifest as structured prediction shifts for rare subgroups.
We detail two post-hoc correction techniques to mitigate this bias, which operate purely on the outputs of standard model training.
We empirically verify that with such post-hoc correction, overparameterisation can improve average and worst-case performance.
I NTRODUCTION
Overparameterised neural networks have demonstrated the remarkable ability to perfectly ﬁt training samples, while still generalising to unseen test samples (Zhang et al., 2017; Neyshabur et al., 2019;
Nakkiran et al., 2020).
However, several recent works have revealed that overparameterised models’ good average performance does not translate to good worst-case performance (Buolamwini & Gebru, 2018; Hashimoto et al., 2018; Sagawa et al., 2020a;b).
In particular, the test performance of such models may be poor on certain subgroups that are under-represented in the training data.
Worse still, such degradation can be exacerbated as model complexity increases.
This indicates the unsuitability of such models in ensuring fairness across subgroups, a topical concern given the growing societal uses of machine learning (Dwork et al., 2012;
Hardt et al., 2016; Buolamwini & Gebru, 2018).
Why does overparameterisation induce such unfavourable bias, and how can one correct for it?
Sagawa et al.
(2020a) demonstrated how such models may ﬁt to spurious correlations that explain under-represented samples, which can generalise poorly.
Sagawa et al.
(2020b) further posited that overparameterised models have an inductive bias towards memorising labels for as few samples as possible, which are invariably those from under-represented subgroups.
To mitigate such bias, existing approaches include subsampling majority subgroups (Sagawa et al., 2020b), and modifying the training objective (Sagawa et al., 2020a;
Nam et al., 2020; Zhang et al., 2020; Goel et al., 2020).
This suggests two important points regarding overparameterised models’ performance:
(a) with standard training, increasing model complexity exacerbates degradation on rare subgroups; (b) controlling this degradation may require alternate training objectives or procedures
In this paper, we establish that while overparameterised models are biased against under-represented examples, in certain settings, such bias may be easily corrected via post-hoc processing of the model outputs.
Speciﬁcally, such models’ bias can be largely restricted to their classiﬁcation layers, and manifest as structured shifts in predictions for rare subgroups.
We thus show how two simple techniques applied to the model outputs — classiﬁer retraining based on the learned representations, and correction of the classiﬁcation threshold — can help overparameterised models improve worstsubgroup performance over underparameterised counterparts.
Consequently, even with standard training, overparameterised models can learn sufﬁcient information to model rare subgroups.
To make the above concrete, Figure 1 plots a histogram of model predictions for a synthetic dataset from Sagawa et al.
(2020b)
(cf.
The data comprises four subgroups generated from combinations
[Figure]
(a) Distribution of overparameterised
[Figure]
Effect of post-hoc correction
[Figure]
( y, a ( x )) of labels y ∈ {± 1 } and a feature a ( x ) ∈ {± 1 } .
Most samples ( x, y ) have y = a ( x ) , and so these comprise two dominant subgroups within the positive and negative samples.
We train an overparameterised linear model, yielding logits f ± 1 ( x ) .
We then plot the decision scores f +1 ( x ) − f − 1 ( x ) , which are expected to be >
iff y
=
+1 .
Strikingly, there is a distinct separation amongst the subgroup scores: e.g., samples with y = +1 , a ( x ) = − 1 have systematically lower scores than those with y
= +1 , a ( x ) =
+1 .
Consequently, the model incurs a signiﬁcant error rate on rare subgroups.
The structured nature of the separation implies suggests to post-hoc shift the scores to align the distributions; this markedly improves performance on the rare subgroups (Figure 1b).
Scope and contributions .
The primary aim of this work is furthering the understanding of the behaviour of overparameterised models, rather than proposing new techniques.
Indeed, the post-hoc correction techniques we employ have been well-studied in the related problem setting of longtail learning or learning under class imbalance (He & Garcia, 2009; Buda et al., 2017; Van Horn & Perona, 2017).
Several works have demonstrated that the representations learned by standard networks contain sufﬁcient information to distinguish between dominant and rare labels (Liu et al., 2019; Zhang et al., 2019;
Kang et al., 2020; Menon et al., 2020).
Similar techniques are also common the fairness literature (Hardt et al., 2016;
Chzhen et al., 2019).
However, it is not a-priori clear whether such techniques are effective for overparameterised models, whose ability to perfectly ﬁt the training labels can thwart otherwise effective approaches (Sagawa et al., 2020a).
Existing techniques for improving the worst-subgroup error of overparameterised models involve altering the inputs to the model (Sagawa et al., 2020b), or the training objective (Sagawa et al., 2020a).
By contrast, the techniques we study alter the outputs of a standard network, trained to minimise the softmax cross-entropy on the entire data.
Our ﬁndings illustrate that such models do not necessarily require bespoke training modiﬁcations to perform well on rare subgroups: even with standard training, overparameterised models can (in certain settings) learn useful information about rare subgroups.
In summary, our contributions are:
(i) we demonstrate that, in certain settings, overparameterised models’ poor performance on underrepresented subgroups is the result of a structured bias in the classiﬁcation layer (cf. §3); (ii) we show that two simple post-hoc correction procedures (cf. §4) can mitigate the above bias, and thus signiﬁcantly reduce their worst-subgroup error (cf.
B ACKGROUND AND SETTING
Suppose we have a labelled training sample S
= { ( x i , y i ) } ni =1 ∈ ( X × Y ) n , for instance space
X ⊂ R d and label space Y .
One typically assumes S is an i.i.d.
draw from some unknown distribution
P ( x, y ) .
Further, suppose each ( x, y ) has an associated group membership g ( x, y ) ∈ G , with
G .
= | G | .
This induces G data subgroups , with a prior P ( g ) and conditional distributions P ( x, y | g ) .
Following Sagawa et al.
(2020a;b), we consider groups g ( x, y )
=
( y, a ( x )) , where a ( x ) ∈ R is some attribute within x .
We assume a ( x ) is fully speciﬁed during train and test time; while not always realistic, such an assumption has precedent in the fairness literature (Lipton et al., 2018).
The standard goal in classiﬁcation is to learn a classiﬁer h : X → Y that minimises the average error
[Equation]
where ‘ 01 ( y, h ( x ))
=
J y
= h ( x )
K is the 0-1 loss.
Typically, one constructs h ( x ) =
argmax y f y ( x ) , where f ( x ) ∈ R Y comprises real-valued logits , as learned by empirical risk minimisation (ERM): min
[Equation]
P ni
=1 ‘ ( y i , f ( x i )) .
Here, ‘ is a surrogate loss such as the softmax cross-entropy, and F is a function class, such as a neural networks with a ﬁxed architecture.
A network is overparametrised if it can perfectly ﬁt the training labels, and thus drive the training error to zero.
Remarkably — and in apparent contrast to orthodox statistical wisdom — this does not come at the expense of generalisation on test samples (Zhang et al., 2017; Belkin et al., 2019; Nakkiran et al., 2020).
This apparent power comes at a price, however.
Let us deﬁne the worst-subgroup error as
[Equation]
i.e., the worst-case error over all data subgroups.
Prior work (Sagawa et al., 2020a;b) established that for overparameterised models, the worst-subgroup training error can go to zero (since the model can ﬁt all samples), but the worst-subgroup test error can devolve to that of random guessing (since the model can ﬁt spurious correlations for rare subgroups).
Further, the degree of degradation can increase with the model complexity.
This indicates that the naïve use of overparametrised models may be at odds with ensuring fairness across data subgroups, a core concern in moden applications of machine learning (Calders & Verwer, 2010; Dwork et al., 2012;
Hardt et al., 2016; Zafar et al., 2017).
There are several potential strategies to cope with this.
One is to perform distributionally robust optimisation (Hashimoto et al., 2018;
Mohri et al., 2019; Sagawa et al., 2020a), and minimise:
[Equation]
where Ω g is some per-group regulariser.
In settings where P ( g ) is non-uniform, Sagawa et al.
(2020a) proposed to set Ω g ( f ) ≡ 1 √ n g , where n g is the number of training samples with group g .
Alternatively, one can reweight samples to upweight the contribution of rarer groups and minimise:
[Equation]
where, e.g., w g = P ( g ) leads to the standard average error, while w g = 1 implicitly upweights rare subgroups.
While intuitive, Sagawa et al.
(2020b) established that such an approach is also subject to poor worst-subgroup performance, owing to a broader issue with using importance weighting in conjunction with neural networks (Wen et al., 2014; Byrd & Lipton, 2019).
Sagawa et al.
(2020b) established that one can achieve good performance by instead subsampling dominant groups, an operation equivalent in expectation to minimising L RW ( h ) with w g = 1 .
Recent developments in the mitigation of worst-subgroup errors include Nam et al.
; Zhang et al.
; Goel et al.
In the sequel, we shall make extensive use of three datasets from Sagawa et al.
(2020a;b), each of which involve binary labels y ∈ Y and a binary attribute a ( x ) ∈ A :
(i) synth , a synthetic dataset where X ⊂ R 200 , Y = {± 1 } , and A ∈ {± 1 } .
(ii) waterbirds , a dataset of bird images with Y =
{ land bird , water bird } corresponding to the bird type, and A = { land background , water background } corresponding to the background.
(iii) celebA , a dataset of celebrity images with Y =
{ blond , dark } corresponding to individuals’ hair colour, and A = { male , female } .
For each dataset, we construct four subgroups g ( x, y ) =
( y, a ( x )) , with two such subgroups being under-represented.
On synth and waterbirds , these correspond to subgroups with y 6 = a ( x ) , while on celebA , these corespond to the subgroups { ( blond , male ) } and { ( dark , female ) } .
Owing to the rarity of certain subgroups, it is intuitively easy for an overparameterised network to learn to predict a ( x ) rather than y , and memorise spurious patterns to predict the rare subgroups.
[Figure]
This suggests that the representations learned by the models contain sufﬁcient information to help distinguish be traced to issues with layer.
To train overparameterised models, we follow the setup of Sagawa et al.
(2020a;b), which we brieﬂy summarise.
For celebA and waterbirds , we use a ResNet-50, which can attain perfect training accuracy.
For synth , we train a weakly regularised ( λ = 10 − 16 ) logistic regression model on a ﬁxed representation Φ constructed as follows: for ﬁxed m , we construct Φ( x ) =
ReLU( V x ) , where V ∈ R m × 200 is a random Gaussian matrix with normalised rows.
Overparameterised models consistently demonstrate a signiﬁcant gap between the average and worst-subgroup error: e.g. (see
Figure 4), on synth , the model achieves 91% average accuracy, but 36% worst-subgroup accuracy.
T HE DOMINANT SUBGROUP BIAS OF OVERPARAMETERISED MODELS
We study the nature of overparameterised models’ poor performance on rare subgroups more closely.
We make two observations: ﬁrst, this under-performance is largely owing to a bias in the classiﬁcation layer.
Second, this bias manifests in the form of a distribution shift in the model scores for rare subgroups.
This shall subsequently motivate post-hoc correction procedures.
3. C AN THE LEARNED REPRESENTATIONS DISTINGUISH BETWEEN SUBGROUPS ?
Neural models make predictions based on logits
f y ( x ) = w > y Φ( x ) , for classiﬁcation weights w y ∈ R K and representations Φ( x ) ∈ R K .
Suppose such a model performs poorly on a subgroup (¯ y, ¯ a ) .
This implies that for a sample ( x, ¯ y ) in this subgroup, f ¯ y ( x ) (cid:28) f
y 0 ( x ) for some competing y 0 .
Why do overparameterised models underperform on rare subgroups?
The factorised nature of the logits suggests this arises from issues with the representations, the classiﬁcation weights, or both.
To determine which of these is likely, we begin by inspecting the representations.
Recall that we train a ResNet-50 on celebA and waterbirds , which produce K = 2048 dimensional instance embeddings.
We may thus embed test instances from each of the four data subgroups, and study their geometric structure.
Intuitively, if instances from rare and dominant subgroups with the same label have limited similarity, the representations for rare samples are insufﬁciently rich.
The high dimensionality of the embeddings prohibits an exact inspection, but as a rough surrogate, we employ a two-dimensional tSNE (Maaten & Hinton, 2008) visualisation.
As tSNE attempts to preserve neighbourhood information amongst samples, we hope to evince the relative geometries of samples from the various subgroups.
Figure 2 reveals that for celebA and waterbirds , samples from the rare subgroups tend to be closely clustered with those belonging to the same class.
The above suggests that the representations learned by the models contain information to (at least partly) help distinguish samples from rare versus dominant subgroups.
1 This suggests the poor worst-subgroup performance of such models may result from issues with the classiﬁcation layer.
[Figure]
3. H OW DOES BIAS MANIFEST IN THE CLASSIFICATION LAYER ?
To study the potential issues in the classiﬁcation layer
[Figure]
we continue our strategy of visualising the model outputs.
Since the datasets we consider involve binary labels, we may simply study the distribution of the decision scores f +1 ( x ) − f − 1 ( x ) .
Since our predicted label is the highest scoring logit, we desire this score to be >
iff the sample has a positive label.
As in the previous section
[Figure]
we may break down these scores for each of the subgroups induced by label y and attribute a ( x )
[Figure]
We earlier illustrated this distribution for synth in Figure 1; Figure 3 further provides distributions for celebA and waterbirds .
On both datasets, within each label, there is a shift in the score distributions for one or both of the rare subgroups.
This is in keeping with the poor performance of the model on these subgroups: e.g., on waterbirds , rare samples in the positive class systematically have negative decision scores, implying the model incurs a high false-negative rate on these samples.
In light of the above ﬁndings, we may revisit ablations from prior work that tease apart the factors causing poor worst-subgroup performance.
For example, Sagawa et al.
(2020b) showed that increasing model complexity on synth can degrade worst-subgroup performance beyond a certain critical point, and that this can be mitigated with a combination of strong regularisation and subsampling.
The same conclusions largely hold (see Appendix C) for the varying decision scores amongst subgroups: e.g., the difference in rare and dominant subgroup scores is exacerbated as we increase model complexity.
3. D ISCUSSION AND IMPLICATIONS
The systematic under-prediction of scores for rare subgroups can be seen as a particular manifestation of neural networks producing uncalibrated probability estimates (Guo et al., 2017): from Figure 3, the model will systematically under- or over-estimate the probability of rare samples being positive.
We emphasise here that our illustrations above are for test samples not observed during training.
Training samples exhibit qualitatively different trends, reﬂective of overparameterised models’ ability to perfectly ﬁt them: e.g., the decision scores for all samples are consistently on the correct side of the decision boundary (see Appendix C).
The fact that the scores on unseen samples exhibit a distinction amongst subgroups suggest the network encodes an implicit bias against such samples.
At the same time, this bias largely manifests as a translation of the scores.
This suggests a simple post-hoc correction of the scores may sufﬁce to improve performance; e.g., bumping up scores for samples with a ( x ) = land background in waterbirds can make the error on the rare subgroup ( water bird , land background ) more equitable.
It remains now to more carefully describe such post-hoc procedures, and study their performance.
C ORRECTING THE SUBGROUP BIAS OF OVERPARAMETERISED MODELS
Drawing from the literature on long-tail learning (Zhang et al., 2017; Kang et al., 2020;
Ye et al., 2020) and fairness (Hardt et al., 2016;
Chzhen et al., 2019), we now detail two post-hoc correction techniques to mitigate overparameterised models’ bias against rare subgroups.
4. C LASSIFIER RETRAINING
Given that §3.1 demonstrates that learned representations Φ( x ) appear meaningful across subgroups, a natural thought is to ﬁt a linear classiﬁer on top of them; i.e., we treat { (Φ( x i ) , y i ) } ni =1 as a new training set for a linear model.
Overparameterisation introduces a challenge, however: since the original network can ﬁnd a classiﬁer with the lowest possible (i.e., zero) training error, simple modiﬁcations to the loss (e.g., reweighting samples) will result in learning the same classiﬁer.
Fortunately, there are several options to ﬁnd a distinct classiﬁer than the original network: e.g., one can subsample elements from the majority subgroups, per Sagawa et al.
(2020b).
We emphasise an important difference between employing such techniques in standard training, and in classiﬁer retraining: the latter uses representations learned from a standard network, as opposed to changing the network objective itself.
The success of the latter shall thus demonstrate that the standard network representations are rich enough to reasonably distinguish between subgroups.
4. T HRESHOLD CORRECTION
The illustrations in Figures 3 suggest a simple approach to improving classiﬁcation performance: rather than using an identical classiﬁcation threshold for all samples, one can employ per-subgroup thresholds.
In detail, observe that in the case of binary labels, we predict h ( x ) =
+ ⇐⇒
f +1 ( x ) − f − 1 ( x ) > 0 .
Instead, we can predict
[Equation]
where { t a ∈ R : a ∈ A } are per-attribute thresholds.
Equivalently, this translates the scores for all samples with a given attribute a ( x ) so that the decision boundary is at 0 .
This can compensate for the distribution shifts observed in Figures 1 and 3: intuitively, by enforcing a lower threshold for samples with a ( x ) = − 1 , we can account for the fact that most of these samples obtain a low model score.
It remains to specify how to choose the thresholds t a .
One simple option is to perform a parameter sweep, and employ the thresholds that minimise the worst-subgroup error on a holdout set.
This is feasible in settings where | A | and | Y | are small, and can directly target the performance measure of interest.
For more complex problems, one may suitably parameterise the thresholds, or attempt to learn so as to minimise a suitable objective (see discussion below).
4. D ISCUSSION AND RELATED WORK
The long-tail learning literature has demonstrated the value of both classiﬁer retraining (Zhang et al., 2017; Kang et al., 2020) and threshold correction (Zhou & Liu, 2006;
Collell et al., 2016; Menon et al., 2020).
Similarly, in the fairness literature, post processing of classiﬁer outputs based on per-subgroup thresholds is a similarly well-established technique (Hardt et al., 2016).
Our aim here is to investigate the effectiveness of such techniques in the overparameterised setting.
Given such models can perfectly ﬁt training labels, it is less clear whether the representations learned from standard training are sufﬁciently useful to learn a good classiﬁer, and whether their outputs can be easily corrected post-hoc; e.g., a model that merely memorised certain training labels could not be meaningfully corrected to perform well on test samples.
To extend threshold correction to multi-class settings, one could tie the thresholds to the frequencies
P (( y, a ( x ))) of various subgroups, rather than tune them.
This is akin to class prior or logit correction techniques from long-tail learning (Collell et al., 2016; Menon et al., 2020).
In the fairness literature, relevant techniques include Hardt et al.
(2016), who propose an objective to select optimal thresholds for ensuring a particular notion of fairness; and recent techniques that learn a post-processing of model scores (Chzhen et al., 2019; Jiang et al., 2020;
Wei et al., 2020) to de-bias predictions.
Exploring such techniques in the overparameterised setting is an interesting direction for future work.
The contemporaneous work of Sohoni et al.
(2020) also considers the viability of improving worstsubgroup performance using the learned representations.
Speciﬁcally, they propose to cluster these embeddings, and train a new distributionally robust classiﬁer to predict cluster assignments.
While very much in the spirit of our classiﬁer retraining proposal, their study does not consider the efﬁcacy of such a procedure under varying model complexity, nor correction techniques that operate purely on the classiﬁcation outputs.
[Table]
E XPERIMENTS : HOW EFFECTIVE IS POST - HOC CORRECTION ?
We now show that the above post-hoc correction techniques can signiﬁcantly improve overparameterised models’ performance on rare subgroups.
This indicates that even when trained in the usual manner, such models can learn useful information about rare subgroups.
5. E XPERIMENTAL SETUP
We follow the same basic setup as Sagawa et al.
(2020a;b).
We instantiate overparameterised models on each dataset: a ResNet-50 on celebA and waterbirds ; a linear logistic regression on synth , using ﬁxed features as described in §2; and a linear logistic regression on waterbirds , using the embeddings from a ResNet-18 pre-trained on ImageNet.
We measure both the average and worst-subgroup errors on both the train and test set, repeating each experiment 5 times.
We apply post-hoc correction to these learned models, via classiﬁer retraining ( CRT ) on the learned representations, using a linear logistic regression model with subsampling of the dominant subgroups per Sagawa et al.
(2020b); and threshold correction ( THR ) on the decision scores, using a holdout set to estimate thresholds { t a : a ∈ {± 1 }} that minimise the worst-subgroup error.
For waterbirds , we use the holdout set from Sagawa et al.
(2020a); for celebA , we use the standard holdout set; and for synth , we construct a holdout set using 20% of the training samples.
We also report the results of the regularised distributionally robust optimisation (DRO) procedure of Sagawa et al.
(2020a).
This involves strongly regularising the model, and modifying the training objective to target the worst-subgroup error.
Our aim is not to improve on the performance of this method, which directly trains to minimise the worst-subgroup error; rather, our goal is to understand and quantify how much useful information standard model training learns about rare subgroups.
5. R ESULTS AND DISCUSSION
Table 1 summarises the test set results on all datasets.
We highlight several key points.
ERM has poor worst-subgroup error .
In keeping with Sagawa et al.
(2020a;b), standard ERM performs poorly in terms of worst-subgroup error.
This may be strongly mitigated by modifying the training objective per DRO, which incurs a mild penalty in terms of the average error.
Post-hoc correction improves worst-subgroup error .
Encouragingly, post-hoc techniques consistently and signiﬁcantly reduce the worst-subgroup error of ERM.
For example, on celebA , THR reduces the worst-subgroup error from 56.94% to 12.10%, with only a mild increase in the average error.
Further, these techniques generally compare favourably with DRO, particular on waterbirds .
While DRO is notably superior on synth , this gap appears to be in part due to the challenge of tuning hyperparameters given holdout data with limited subgroup representation.
For example, increasing the holdout set size to 80% of the training set improves the performance of THR on synth to 24% . [Figure] [Figure]
[Figure] The superior overall performance of DRO is in keeping with ﬁndings about the efﬁcacy of training modiﬁcation for fairness (Agarwal et al., 2018).
Nonetheless, the generally competitive performance of CRT and THR suggests the latter can extract non-trivial gains from overparameterised models.
Overparameterisation + post-hoc correction can improve worst-subgroup error .
As the synth and waterbirds datasets with ﬁxed features allow for modifying the feature dimensionality m — which controls the degree of overparameterisation – we perform an experiment where we vary m between 10 1 and 10 4 .
Figure 4 shows the average and worst-subgroup error on both the train and test set, over 5 independent trials.
The overall and worst-case training error approaches 0% as m increases (i.e., the model becomes overparameterised).
Further, the average test error appears reasonable, approaching 10%; however, the worst-case error devolves as m increases, exceeding 60%.
The plots also show the results of THR as m is varied.
Here, both average and worst-case error steadily improve: indeed, overparameterisation aids worst-subgroup performance.
Further, the gains in the worst-subgroup test error are dramatic compared to ERM.
Thus, while ERM is susceptible to a tradeoff between overparameterisation and worst group accuracy, this tension can be mitigated with simple mechanisms to encourage equitable performance.
This general trend is largely robust to distributional properties such as the fraction of majority samples in synth , though signiﬁcantly increasing this expectedly degrades the best achievable performance; see Appendix C.
Illustration of classiﬁcation thresholds .
Figure 5 studies the effect of tuning the classiﬁcation thresholds t a on each of the per-subgroup errors for synth ( m = 10 4 ).
Here, we plot the errors for each of the four subgroups as we sweep over the thresholds.
The case t a = 0 corresponds to the baseline.
Modifying t a trades off performance on the rare subgroups compared to the dominant ones.
One may then pick optimal thresholds for a = ± 1 , which correspond to cross-over points of the error curves for each value of a : for example, when t +1 ∼ 10 , the errors on the subgroups y
= +1 , a = +
and y = − 1 , a = +1 are equitable, and thus the maximum of the two errors is minimised.
Choosing such thresholds aligns the scores distributions for rare and dominant subgroups (right).
D ISCUSSION AND FUTURE WORK
Post-hoc correction relies on knowledge of the data subgroups at train and test time.
An important practical challenge is extending this to settings with unknown subgroups.
One natural strategy is to attempt to unearth these subgroups via clustering of the model outputs, but careful study is needed to inform design choices (e g., choosing the number of clusters).
For a contemporaneous study of a similar technique, see Sohoni et al.
Exploring the viability of post-hoc correction in overparameterised settings with multi-class labels and multiple subgroups — e g., through adaptations of techniques noted in §4.3 — is also of interest.
More broadly, the study of learning bias-free representations has received signiﬁcant interest (Bahng et al., 2020;
Kim et al., 2019; Li & Vasconcelos, 2019; Arjovsky et al., 2020; Nam et al., 2020).
Exploring the efﬁcacy of such approaches in overparameterised settings would be of interest