SlowGuess/NAACL_2022_OCR huggingface dataset:
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
Geoff Pleiss, Tianyi Zhang, Ethan R. Elenberg, and Kilian Q. Weinberger. 2020. Identifying mislabeled data using the area under the margin ranking. In Advances in Neural Information Processing Systems, volume 33.
Lichao Sun, Congying Xia, Wenpeng Yin, Tingting Liang, Philip Yu, and Lifang He. 2020. Mixup-transformer: Dynamic data augmentation for NLP tasks. In Proceedings of the 28th International Conference on Computational Linguistics, pages 3436-3440, Barcelona, Spain (Online). International Committee on Computational Linguistics.
Swabha Swayamdipta, Roy Schwartz, Nicholas Lourie, Yizhong Wang, Hannaneh Hajishirzi, Noah A. Smith, and Yejin Choi. 2020. Dataset cartography: Mapping and diagnosing datasets with training dynamics. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9275-9293, Online. Association for Computational Linguistics.
Sunil Thulasidasan, Gopinath Chennupati, Jeff A Bilmes, Tanmoy Bhattacharya, and Sarah Michalak. 2019. On mixup training: Improved calibration and predictive uncertainty for deep neural networks. In Advances in Neural Information Processing Systems, volume 32.
Vikas Verma, Alex Lamb, Christopher Beckham, Amir Najafi, Ioannis Mitliagkas, David Lopez-Paz, and Yoshua Bengio. 2019. *Manifold mixup: Better representations by interpolating hidden states*. In *International Conference on Machine Learning*, pages 6438-6447. PMLR.
Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1112-1122. Association for Computational Linguistics.
Wenpeng Yin, Huan Wang, Jin Qu, and Caiming Xiong. 2021. BatchMixup: Improving training by interpolating hidden states of the entire mini-batch. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4908-4912, Online. Association for Computational Linguistics.
Rowan Zellers, Yonatan Bisk, Roy Schwartz, and Yejin Choi. 2018. SWAG: A large-scale adversarial dataset for grounded commonsense inference. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 93-104, Brussels, Belgium. Association for Computational Linguistics.
Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. 2018. mixup: Beyond empirical risk minimization. In Proceedings of the 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada.
In our experiments, we use bert-base-uncased classification model on top of a task-specific fully-connected layer. The model is fine-tuned with a maximum of 3 epochs, batch size of 16 for SNLI and QQP, batch size 4 for SWAG, a learning rate of 1e-5, gradient clip of 1.0, and no weight decay. We use the hyper-parameter of MixUp  $\alpha$  as 0.4. All hyper-parameters are estimated on the validation set of each task. For all results, we report averaged results across 5 fine-tuning runs with random starts. Finally, all experiments are conducted on a single NVIDIA RTX A6000 48G GPU with the total time for fine-tuning all models being under 24 hours. For each dataset, we follow the published train/validation/test split by Desai and Durrett (2020) and show the statistics of the datasets in Table 4.
Dataset	Train	Dev	Test
SNLI	549,368	4,922	4,923
MNLI	392,702	4,908	4,907
QQP	363,871	20,216	20,217
TwitterPPDB	46,667	5,060	5,060
SWAG	73,547	10,004	10,004
HellaSWAG	39,905	5,021	5,021
Table 4: The statistics of all used datasets.
A.2 Data Maps
In this section, we provide data maps (Swayamdipta et al., 2020) of our in-domain datasets on bert-base-uncased model in Figure 1. These data maps are used to identify the characteristics of each training sample (i.e., easy-to-learn, ambiguous, and hard-to-learn).
Figure 1: Data Maps of SNLI, QQP and SWAG on bert-bert-uncased model.