[18] Robert Geirhos, Jorn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A. Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2:665-673, 2020. Cited on pages 1 and 3.
[19] Karan Goel, Albert Gu, Yixuan Li, and Christopher Ré. Model patching: Closing the subgroup performance gap with data augmentation. In International Conference on Learning Representations (ICLR), 2021. Cited on page 3.
[20] Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. Annotation artifacts in natural language inference data. In North American Association for Computational Linguistics (NAACL), 2018. Cited on page 3.
[21] Steve Hanneke. Theory of Disagreement-Based Active Learning. Foundations and Trends in Machine Learning, 2014. Cited on page 3.
[22] Charles R. Harris, K. Jarrod Millman, Stefan J. van der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J. Smith, Robert Kern, Matti Picus, Stephan Hoyer, Marten H. van Kerkwijk, Matthew Brett, Allan Haldane, Jaime Fernandez del Rio, Mark Wiebe, Pearu Peterson, Pierre Gerard-Marchant, Kevin Sheppard, Tyler Reddy, Warren Weckesser, Hameer Abbasi, Christoph Gohlke, and Travis E. Oliphant. Array programming with NumPy. Nature, 585(1):357-362, 2020. Cited on page 25.
[23] Tatsunori B. Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. Fairness without demographics in repeated loss minimization. In International Conference on Machine Learning (ICML), 2018. Cited on page 3.
[24] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Conference on Computer Vision and Pattern Recognition (CVPR), 2016. Cited on pages 5 and 25.
[25] Dirk Hovy and Anders Søgaard. Tagging performance correlates with author age. In Association for Computational Linguistics (ACL), 2015. Cited on page 3.
[26] John D. Hunter. Matplotlib: A 2D graphics environment. Computing in Science & Engineering, 9(3):90-95, 2007. Cited on page 25.
[27] Badr Youbi Idrissi, Martin Arjovsky, Mohammad Pezeshki, and David Lopez-Paz. Simple data balancing achieves competitive worst-group-accuracy. In Conference on Causal Learning and Reasoning (CLeaR), 2022. Cited on pages 2, 3, 4, 5, 7, and 25.
[28] Pavel Izmailov, Polina Kirichenko, Nate Gruver, and Andrew Gordon Wilson. On feature learning in the presence of spurious correlations. In Conference on Neural Information Processing Systems (NeurIPS), 2022. Cited on pages 2, 3, 4, 5, 6, 8, 18, 25, and 28.
[29] Stanisław Jastrzebski, Maciej Szymczak, Stanislav Fort, Devansh Arpit, Jacek Tabor, Kyunghyun Cho, and Krzysztof Geras. The break-even point on optimization trajectories of deep neural networks. In International Conference on Learning Representations (ICLR), 2020. Cited on page 23.
[30] Yiding Jiang, Vaishnavh Nagarajan, Christina Baek, and J. Zico Kolter. Assessing generalization of SGD via disagreement. In International Conference on Learning Representations (ICLR), 2022. Cited on page 4.
[31] Eungyeup Kim, Jihyeon Lee, and Jaegul Choo. BiaSwap: removing dataset bias with biastailored swapping augmentation. In International Conference on Computer Vision (ICCV), 2021. Cited on page 3.
[32] Nayeong Kim, Sehyun Hwang, Sungsoo Ahn, Jaesik Park, and Suha Kwak. Learning debiased classifier with biased committee. In Conference on Neural Information Processing Systems (NeurIPS), 2022. Cited on page 3.
[33] Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson. Last layer re-training is sufficient for robustness to spurious correlations. In International Conference on Learning Representations (ICLR), 2023. Cited on pages 1, 2, 3, 4, 5, 6, 8, 18, 25, and 28.