Are We Viewing the Problem of Robust Generalisation through the Appropriate Lens?
Abstract
We discuss different approaches to the challenge of robust object recognition under distribution shifts.
We advocate a view of this challenge that is more closely informed by the problem of visual recognition, and which emphasizes dynamic model behaviour as opposed to centering the statistical properties of training and test distributions.
We introduce an experimental setting geared towards developing models that can exhibit robust behaviour in a reliable and scalable manner.
We refer to this requirement as “systematic robustness”, which involves excluding certain combinations of classes and image attributes systematically during training.
Unlike prior work which studies systematic generalisation in DNNs or their susceptibility to spurious correlations, we use synthetic operations and data sampling to scale such experiments up to large naturalistic datasets.
1. Introduction
Automating visual perception is immensely hard – not least because the world is complex and ever-changing (Raji et al., 2020), constantly throwing up new and unlikely objects and scenes to recognise and handle.
The problem of recognising objects with improbable properties – such as an unusual location (Alcorn et al., 2019) or style of appearance (Hendrycks et al., 2021) – is often subsumed under the more general problem of making predictive models robust to “distribution shifts” or to “out-of-distribution” inputs.
This framing foregrounds the difference in input statistics between a model’s training and deployment conditions.
Variants of this general problem, such as domain generalisation and subpopulation shift (Gulrajani & Lopez-Paz, 2021; Santurkar et al., 2021; et al., 2021), are then distinguished by the difference – in statistical terms – that arises after training is completed.
For example, we are faced with a domain generalisation problem when the respective marginal distributions of some property measured over training and evaluation sets have non-overlapping mass.
If we consider, say, the property “depiction style”, objects might be depicted using natural images or paintings during training, and during evaluation using line drawings (Peng et al., 2019).
The goal is then to learn representations from the set of training “domains” or “environments” that are useful in distinct environments held-out for evaluation.
Often, which property changes from one environment to the other is of secondary importance; it is merely assumed that some property changes.
Recent benchmarks reflect this abstract statistical view, collating various datasets that fit a specific problem template but which target diverse challenges in visual recognition (Gulrajani & Lopez-Paz, 2021) and even in other modalities beyond the visual (Koh et al., 2021).
Methods for handling certain types of distribution shifts are expected to be agnostic to what exactly changes between environments, whether it’s the viewpoint (Ghifary et al., 2015), the imaging sensor and illumination condition (Beery et al., 2018), or the time and location (David et al., 2020).
Accordingly, such methods typically rely on some general objective or loss function designed to encourage robustness, e.g. via learning an invariant predictor (Arjovsky et al., 2019) or via aligning the feature statistics across environments (Li et al., 2018).
In this short paper, which describes ongoing work, we seek to question this problem framing and its underlying assumptions: that we should primarily view problems of robust prediction through an abstract statistical lens, and – in the case of domain generalisation specifically – that we can learn representations from arbitrary environments that will usefully carry over to others.
Instead, we propose an alternate view of robust recognition and an accompanying experimental setting that we believe can help overcome the limitations of the problem-agnostic approach.
We will also argue that it is not sufficient to merely design methods and benchmarks that address specific types of robustness, e.g. to image corruptions (Hendrycks & Dietterich, 2019) or adversarial perturbations (Croce et al., 2021), but that we
[Figure] need to rethink current training and evaluation protocols intended for such purposes to ensure that they result in a reliable and scalable form of robustness.
2. What Does Robust Recognition Entail?
Before introducing our proposed setting, it is useful to consider the task of recognition in computational terms (Marr & Poggio, 1976).
We can view recognition as requiring us to establish the similarity between a novel input and past experience (Ullman, 1996).
Given (i) a closed set of classes
C , (ii) a model trained to recognise objects belonging to any class c ∈ C , and (iii) a novel input image x , said model needs to extract some representation of x as a first step.
If this representation is “closest” to the stored representation(s) of some class c k than to those of other classes, then we assign to x the label c .
Consider the images in Figure 1(a).
These are images that a ResNet-50 model trained on ImageNet has no trouble recognising as depicting elephants.
Rows 1(b)–1(d) contain mis-classified versions of the same images that are transformed in some manner: by applying Gaussian noise, by rotating the image, or by distorting the colours of the image.
Evidently, the model extracts a representation from these inputs that is more similar to internal representations of other object classes.
What then would we have to change about our model to obtain correct predictions?
We could simply treat this as a data problem.
The training distribution does not contain images of purple elephants, or noisy images, and these are transformations that can be simulated, so why not modify the training distribution?
Many successful approaches to robust recognition in fact rely on reshaping the training distribution, e.g by including difficult examples we might encounter during deployment through augmentation (Madry et al., 2018;
Hendrycks et al., 2020; Cubuk et al., 2020), or by rebalancing the training set to emphasize under-represented examples (Sagawa et al., 2020; Idrissi et al., 2021).
Is this scalable though?
Can we anticipate every possible test-time image?
What about transformations that cannot be easily simulated, or examples that are not represented at all in the training data?
Furthermore, data augmentation schemes – whether hand-crafted or learned – are applied with equal probability to every image and thus every class.
There are limitations to how much robustness this can foster to nuisance factors (Bouchacourt et al., 2021), but we also can’t balance every relevant factor in the dataset, and will inevitably have factors that are spuriously correlated.
We could alternatively focus on learning representations with desirable properties, e.g. representations that are invariant across environments: in this case invariant to noise, rotations, and colour.
Many approaches to domain generalisation rely on training schemes or loss terms that seek to encourage such invariances (Li et al., 2018; Arjovsky et al., 2019).
Much work in self-supervised learning also aims to learn representations invariant to a pre-defined set of image transformations (Bardes et al., 2021).
Is this however a sensible goal?
To recognise the images in Figure 1(d) we might want to ignore the misleading colour, but do we want to discount colour cues in general?
What information about shape would be preserved by a rotation-invariant representation?
Instead, we think it is beneficial to view recognition as a necessarily dynamic process that is contingent on the characteristics of any given input.
For the images in Figure 1(b), removing the noise would most likely result in correct predictions.
On the other hand, in an image of an old broken
TV set displaying static, Gaussian noise might be useful signal.
To recognise the next set of images (1(c)), we would benefit from compensating for a specific image rotation ( 180 ◦ ).
This could bring these images into alignment with a canonical representation of shape, but would be superfluous or even harmful for the first set of examples.
For the images in Figure 1(d), we would have to correct for the unusual colours, colours that in other cases might help us distinguish objects for which colour is the salient feature.
This view of recognition that is centered around how an input image needs to be processed has a few useful implications for robustness.
First of all, it suggests an approach to evaluation that focuses on desirable model behaviours:
Can a model learn to compensate reliably for things like noise, rotation, or colour?
Do these behaviours generalise in a non-trivial manner?
Secondly, it suggests a role for causal inference in recognition – not merely during training (Zhang et al., 2021) – but at evaluation time: We need to infer the transformations that would bring some input image into alignment with its class representation and apply these transformations accordingly.
In this work, we focus on the first point: How do we ensure that robust behaviour can generalise reliably in a non-trivial manner?
3. Systematic Robustness: Problem Setting
There is widespread interest in developing robust recognition models, that can recognise a variety of object classes despite the presence of unlikely object, scene, or image properties.
This is reflected by the proliferation of challenging benchmarks that focus on various kinds of robust recognition behaviour, e.g. robustness to image corruptions (Hendrycks & Dietterich, 2019), to spatial transformations (Engstrom et al., 2019), to harsh weather conditions (Sakaridis et al., 2018), to imperceptible perturbations (Gilmer et al., 2018), and to abstract depictions of objects (Rusak et al., 2021).
While it is hard to formulate a compact definition of robustness that covers these diverse requirements, we can nonetheless impose a useful metarequirement that we should pursue in general: For robust behaviour to be scalable , it should transfer flexibly across familiar object classes, and not be separately learned for every class of interest.
We refer to this problem setting as systematic robustness .
As argued in the previous section, in any realistic training set, certain scene factors will inevitably be correlated.
Some of these correlations will be salient – e.g. elephants are typically grey, but some of them will be spurious: We typically encounter and thus photograph elephants during the day but they nontheless continue to exist at night.
We need models that can learn from correlated data and ignore correlations as needed – whether salient or spurious.
A self-driving car for example needs to reliably recognise unlikely obstacles, independently of their distribution in the training data.
While there is extensive work on related problems, e.g. systematic generalisation (Bahdanau et al., 2019; Ruis et al., 2020; Montero et al., 2021; Schott et al., 2021), spurious correlations (Geirhos et al., 2020), experiments are typically conducted on small datasets: Either synthetic ones where we have full control over the data generation process and can correspondingly work with tightly-controlled train/test splits, or naturalistic datasets with limited variability but which nonetheless admit some form of control.
Here we aim to bridge the gap between work on controlled systematic generalisation and work on large-scale recognition models, in which models are trained on large naturalistic datasets with an “almost anything goes” approach.
We also depart from related work on domain generalisation (Arjovsky et al., 2019; Gulrajani & Lopez-Paz, 2021), by acknowledging the need to learn specific robust behaviours from data: We are for example unlikely to be able to learn how to handle blurry images by only looking at clean and noisy images.
These are perturbations with very different frequency characteristics, and it’s not obvious that this should work without the appropriate evaluation.
Our experiments also complement work on domain adaptation under label shift (Johansson et al., 2019; Zhao et al., 2019; Ben-David et al., 2010).
We formulate a variety of challenges probing systematic robustness w.r.t.
(1) photometric transforms, (2) geometric transforms, (3) rendition styles.
Our proposed benchmarks are a compromise between a tightly-controlled but overly simplistic setups and large-scale datasets without meaningful experimental controls, and they build on top of existing data (Russakovsky et al., 2015; Hendrycks & Dietterich, 2019; Peng et al., 2019).
See Figure 2 for an example.
We now introduce the general problem setting of systematic robustness, describe a simple metric for measuring it, and then describe different instantiations of the problem that we examine in this paper.
[Figure]
clean noise blur contrast
O B J E
C T
C L A SS c a t d og b i r d
DOMAIN
[Figure]
3.1. Preliminaries
In this paper, we focus on the task of image classification.
Our goal is to train a classifier to recognise object classes from some closed set C
= { c i } Ni =1 given a training set of images annotated with class labels.
The training data is drawn from a set of “environments” or “domains”
E = { E 0 , . . .
, E M } .
Each environment E k
= T k × C k can be characterised by a set of classes C k and some property
T k that applies only to images from that environment, e.g.
“contain Gaussian noise”, “ are rotated by 90 ◦ ”, and “objects are depicted with line drawings”.
E 0 is the default environment in which all classes C are present.
The remaining training environments are restricted to disjoint subsets of classes { C 1 , . .
, C M } , s.t.: (1) (cid:83) Mk =1 C k ⊆ C, and (2) ∀ ( i, j ) ∈ { 1 , . . .
, M } : i ̸ =
j ⇒ C i ∩ C j = Ø.
At test time, we consider all possible environments: { T 0 ×
C } ∪ { T i ×
C j | i, j ∈ { 1 , . . .
, M }} .
Thus for any valid i, j , if i ̸
=
j then T i × C j has not been seen during training.
This allows us to contrast performance in seen and unseen environments, and determine how well robust behaviour learned for a subset of available classes transfers to the rest.
See Figure 2 for an illustration.
3.2. Measuring Systematic Robustness
To measure systematic robustness of a model f w.r.t.
some scene factor T k , we first need to measure classification accuracy acc f ( · ) in two environments: (1) the control environment T k × C k , i.e. the set of classes C k for which robustness to T k was learned during training, and (2) the experimental environment T k × C k , i.e. the environment in which the property T k holds for classes C k
=
C \ C k .
We can then simply normalise the accuracy in the experimental environment acc f
( T k × C k ) by the accuracy in the control environment acc f
( T k × C k ) .
This represents the degree to which robust behaviour learned in a seen environment transfers to an unseen one.
Sometimes it will make sense to compensate for some baseline robustness, e.g. if we are fine-tuning a pretrained model f 0 that already shows some degree of robust accuracy.
We simply subtract average robust accuracy of the baseline model acc f
( T k × C ) from the accuracy measured in both environments.
We refer to the resulting quantity as ρ k , i.e. systematic robustness w.r.t.
T scenefac , and compute it as follows while limiting the range to [0 , 1] :
[Equation]
It should be noted that a value of 1 is trivial to achieve if f is not much more robust than the reference classifier f 0 .
Thus on its own, this measure of robustness is insufficient and has to be paired with the other metrics that measure absolute robust accuracy.
A related metric is that of effective robustness (Taori et al., 2020).
3.3. Problem Instantiations
We opt to study three instantiations of this problem: systematic robustness w.r.t.
(1) image corruptions (e.g. noise, blur) (Hendrycks & Dietterich, 2019), (2) in-plane image rotations (Engstrom et al., 2019), and (3) rendition styles (e.g. natural images, line drawings) (Hendrycks et al., 2021;
Rusak et al., 2021).
These represent basic challenges in recognition that robust models should arguably handle in a flexible manner.
They also differ from one another qualitatively: The first are photometric perturbations that can drastically affect image statistics, whereas recognising objects from different viewpoints can require reasoning of a more geometric nature.
While the way an object is depicted can also heavily affect the statistics of the corresponding image, successfully managing this challenge requires non-trivial abstract reasoning about the essence of an object’s texture and shape.
Most importantly, we can study this problems in the large-scale setting building on prior efforts to collect and generate the relevant data.
We can study systematic robustness w.r.t.
corruptions and rotations by applying selective data augmentation to ImageNet (Russakovsky et al., 2015;
Hendrycks & Dietterich, 2019), and for rendition styles, we can sample images from DomainNet (Peng et al., 2019) as required.
For the image corruptions we select four out of the 19 suggested for the ImageNet-C benchmark, each representing one of four categories: noise ( N ), blur ( B ), digital ( D ), and ( W ).
More details can be found in the appendix.
4. Summary of Ongoing Experiments
Our findings so far can be summarised as follows (see Appendix A for selected results): We train a standard CNN (He et al., 2016) from scratch on ImageNet with the default “clean” data as well as data that is selectively transformed to exclude certain combinations of class and image corruption – or alternatively class and in-plane image rotation.
We find that the network picks up on image corruptions as a spurious feature resulting in a huge gap between seen and unseen combinations (Tab. 1).
The difference when it comes to image rotations is much less dramatic (Figure 4).
We find that the learning rate is a critical hyperparameter for avoiding reliance on spurious correlations, which is why fine-tuning a network pretrained on clean data results in more – albeit unstable – systematic robustness (Figure 3).
We find that the class composition of the different environments matters for systematic robustness (Tab. 2), i.e. whether these contain groups of fine-grained classes or not.
We also find that popular domain generalisation methods result in improved systematic robustness for corruptions but in limited improvements for rotations (Appendix A.3) and non-existent improvements for alternate renditions (not included).
Aligning a subset of the available environments only results in robustness to those environments Tab. 3.
A. Appendix A.1. Motivating Experiment
As a first exploration of this setting we conduct the following experiment:
We define five environments { E 0 , E N , E B , E D , E W } .
The default environment E
= T 0 × C includes all 1000 ImageNet classes and consists of the original, untransformed images.
All classes are equally distributed among the remaining environments resulting in class sets { C N , C B , C W , C D } and corresponding image corruptions { T N , T B , T W , T D } .
Images from all environments additionally undergo simple geometric transformations during training including resizing, cropping, and horizontal flipping
We train ResNet-50 models with cross-entropy loss for 90 epochs with four different configurations: a baseline with data only from the default environment, and three other models with data from either one, two, or four additional environment(s).
In all cases, we sample data from the default environment with a probability of p = 0 .
75 , and otherwise sample uniformly from the rest.
This is a simple baseline, often referred to in the domain generalisation literature as ERM (empirical risk minimisation), where risk is taken to mean the classification loss.
Data from all environments is effectively treated as a single unified training set.
This baseline has been shown to be as resilient to domain shifts as most specialised methods (Gulrajani &
Lopez-Paz, 2021; Koh et al., 2021) and thus is a good starting point for our analysis.
We report top-1 accuracy for these different configurations in Tab. 1.
We observe the following: Relative to the baseline, corruption robustness tends to improve on average.
However, the large improvement on seen environments (e.g. T N × C N ) comes at the cost of reduced accuracy on unseen environments (e.g. T N × C N ).
Does starting from a pre-trained model help?
We start from a ResNet-50 pre-trained on data from the default environment
E
= T 0 × C. We fine-tune this model with cross-entropy loss for 25 epochs using data from two environments: { E 0 , E N } .
Since the model has already been trained on uncorrupted data, we sample images from the default environment with a probability of p = 0 .
E N contains data for 25% of the classes and with a probability of p = 0 .
5 we apply Gaussian noise at one of five severity levels (Hendrycks & Dietterich, 2019).
This is very similar to the Gaussian Noise Training baseline of (Rusak et al., 2020).
During our initial search over the space of hyperparameters, we observed that the learning rate plays a key role above all others.
We report the results of a learning rate sweep in Figure 3 together with two reference results: [I] the ImageNet -pretrained model, [II] a pre-trained ResNet-50 fine-tuned using unrestricted data augmentation with all available corruptions.
We focus on accuracy on clean and noisy data and observe the following: Robustness to seen environments is at odds with performance on unseen environments.
There is also a trade-off between achieving a small systematic robustness gap and good robust accuracy that doesn’t exist when we apply noise augmentation to all classes.
We conducted an identical learning rate sweep for the latter case, and found that performance is much less sensitive to the learning rate and monotonically increases with it for the entirety of the range we examined.
Corrupting an image introduces spurious but salient visual features that can easily be picked up by the model to distinguish classes from different environments.
What happens we consider a transformation that does not affect the image statistics as dramatically?
To this end, we fine-tune pre-trained ResNet-50 models with different learning rates on data from two environments: { E 0 , E R } .
Images from the second environment are rotated with a probability of p = 0 .
5 by an angle sampled from the set { 18 ◦ , 36 ◦ , 54 ◦ 72 ◦ , 90 ◦ } .
Since rotating an image by a right angle does not introduce any artifacts from interpolation, we measure robustness rotation only for 90 ◦ .
We plot the validation accuracy throughout training for three such models together with three models trained with selective noise augmentation.
The results can be seen in Figure 4.
We see a significant difference in behaviour between models trained for noise-robustness vs. models trained for rotation robustness.
A.2. Comparing across Class Sets
So far we have examined a single model, and split classes randomly among different environments.
(1) What happens when we consider different class splits, especially in light of the numerous fine-grained classes in ImageNet ?
(2) Do we observe different behaviour when considering a different class of models?
The size and diversity of ImageNet classes allows us to analyse the behaviour of the ERM baseline under various conditions, which we summarise in Tab.
We both increase and decrease the size of C 1 from 25% to 50% and 10% respectively.
We also consider the smaller subsets 200adv and 150det .
Finally, we can leverage the class structure of ImageNet , specifically the availability of a large number of fine-grained classes.
This echoes the recommendation of Shankar et al.
(2020), who recommend distinguishing between animate and inanimate objects for evaluation.
We go further and examine the case where the training conditions for (mostly) animate objects depart from other classes.
We find that for example when all the fine-grained classes are in the corruption set, that the robustness gap increases, also w.r.t.
to the opposite case: when no super-classes are in the corruption set.
For more details on how we construct these splits please consult the supplementary material.
A.3.
: Corruptions vs. Rotations
We experimented with several domain adaptation and generalisation methods from the literature, but found that one class of methods was able to improve over the simple ERM baseline.
These methods are based on aligning features across environments or instances by means of an appropriate loss term.
These are also relevant in “self-supervised” learning, where the goal is to extract representations that are invariant to a predefined set transformations that normally preserve label semantics, such as changes in scale or illumination.
We experiment with three different schemes to construct image pairs depicted in Appendix A.3: (1) pair view results in pairs that differ only in a single property T k e.g. clean vs. noisy, (2) pair instance samples two different crops from the same image before applying T k to one of them, and (3) pair class samples [Figure] [Figure] two images from the same class.
This give us a more fine-grained picture of any improvements to systematic robustness.
In addition to a simple classification loss ( CLS ) we consider three different loss terms: (1) COS , a simple cosine similarity term applied to the final layer embeddings of an image pair, (2)
VICReg (Bardes et al., 2021), which is a recent loss term used for self-supervised learning that adds variance and covariance terms to an invariance loss similar to COS , and (3) MMD (Li et al., 2018) which is a domain adaptation method for aligning the statistics across different environments.
Unlike the first two, the latter does not require aligned image pairs.
The results can be seen in Appendix A.3.
What we can observe is that the different objective functions do not improve significantly over the baseline trained with just classification loss.
We should note that we reduced the learning rate for CLS in keeping with our findings from earlier sections.
What also sticks out is that the alignment losses only improve over CLS in the pair view setting, i.e. when the images are perfectly aligned, suggesting that the network perhaps is learning to compensate for specific corruptions.
To follow up on this, we trained ResNet-18 models on a subset of ImageNet , applying the cosine similarity loss to certain environments (see Tab. 3).
Perhaps surprisingly, even when we align three out of four environments to the default one, there is no benefit to the remaining one. [Figure]
.s
y s t
e m a t
i c r o bu s t n e ss[-67  75  
-]
[Figure]
[Figure]
CLSCOSMMDVICRegCLSCOSMMDVICReg
[Figure]
[Table]
[Table]