Revised: Averaged over 10 seeds, reweighting-NuRD produced a model with 81% test accuracy which is similar to the accuracy achieved by N U RD using a 28-pixel border as the nuisance.

Original: the training data; as ρ increases, the nuisance-label relationship changes and becomes more different from the training data.
Revised: In comparison, ERM’s achieves an accuracy of 68% .

Original: We plot the average and standard error of accuracies aggregated over 10 seeds for different test
Revised: N U RD without a nuisance and no nuisance-induced spurious correlations in classifying Waterbirds.

Original: ρ ∈ { 0 . 5 , 0 . 7 , 0 . 0 } in ﬁg. 4.
Revised: We performed an additional experiment on classifying Waterbirds where N U RD is given a constant nuisance which is equivalent to not using the nuisance.

Original: Nuisance speciﬁcation with different borders For Waterbirds, we ran NuRD with  the pixels outside the central 168x168 patch (a 56 pixel border) as the nuisance.
Revised: We generated training and test data with independence between the nuisance and the label; the nuisance-label relationship does not change between training and test. Averaged over 10 seeds, ERM achieved a test accuracy of 89 ± 0 .

Original: Averaged over 10 seeds, reweighting-NuRD produced amodel with 73% test accuracy compared to ERM’s 58% .
Revised: and N U RD achieved a test accuracy of 88 ± 1% .

Original: This drop in accuracy compared to what was in section 5 could be because this nuisance is a 56 pixel border which may capture a few semantic features of the bird and N U RDwould build a model that avoids these features. Downsampling to balance labels in classifying Pneumonia We regenerated both training and test datasets where the healthy cases are downsampled (instead of pneumonia cases being upsampled) but with the same nuisance-label relationships in training and test data as in section 5. The resulting training data had 19000 samples compared to 34200 in section 5. The average accuracy over 10 seeds of the models produced by N U RD is 56% compared to ERM’s 37% .
Revised: Reweighting-N U RD with different λ .

Original: N U RD without a nuisance and no nuisance-induced spurious correlations in classifying Waterbirds
Revised: Large λ s may make optimization unstable by penalizing even small violations of joint independence.

Original: We performed an additional experiment on classifying Waterbirds where N U RDis given a constant nuisance which is equivalent to  not using the nuisance.
Revised: Such can instabilities lead N U RD to build predictive models that do not do better than marginal prediction, resulting in large distillation loss (loglikelihood + information loss) on the validation subset of the training data.

Original: We generated training and test data that have the same nuisance-label relationship as the training data in the Waterbirds experiment in section 5; the nuisance-label relationship does not change between training and test.
Revised: However, a small λ may result in N U RD learning non-uncorrelating representations which can also perform worse than chance.

Original: Averaged   
Revised: We ran the waterbirds and class-conditional Gaussians experiments with λ = 5 (instead of λ = 1 like in section 5) and found that, on a few seeds, N U RD produces models with close to 50% accuracy (which is the same as majority prediction) or large information loss or both.

Original:      $ Y J  D FF X U D F\ 1X5'YV(50RQFODVVLI\LQJ:DWHUELUGV 1X5' (5        $ Y J  D FF X U D
Revised: Excluding seeds with large validation loss, reweighting-N U RD achieves an average test accuracy of 76% on waterbirds and 61% on class-conditional Gaussians.

Original: F\ 1X5'YV(50RQ3QHXPRQLDGHWHFWLRQ 1X5' (5 over 10 seeds, ERM achieved a test accuracy of 88 ± 0 .
Revised: Annealing the λ during training could help stabilize optimization.