SlowGuess/NAACL_2021_OCR huggingface dataset:
answer triggering using hard negatives. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5911-5917, Hong Kong, China. Association for Computational Linguistics.
Ivano Lauriola and Alessandro Moschitti. 2020. Context-based transformer models for answer sentence selection. arXiv preprint arXiv:2006.01285.
Yitong Li, Trevor Cohn, and Timothy Baldwin. 2017. Robust training under linguistic adversity. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 21-27, Valencia, Spain. Association for Computational Linguistics.
Rishabh Misra. 2018. Huffman news category dataset.
Maali Mnasri, Gael de Chameleon, and Olivier Ferret. 2017. Taking into account inter-sentence similarity for update summarization. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 204-209, Taipei, Taiwan. Asian Federation of Natural Language Processing.
Siyuan Qiu, Binxia Xu, Jie Zhang, Yafang Wang, Xiaoyu Shen, Gerard de Melo, Chong Long, and Xiaolong Li. 2020. Easyaug: An automatic textual data augmentation platform for classification tasks. In Companion Proceedings of the Web Conference 2020, WWW '20, page 249-252, New York, NY, USA. Association for Computing Machinery.
F. Schroff, D. Kalenichenko, and J. Philbin. 2015. Facenet: A unified embedding for face recognition and clustering. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 815-823.
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016a. Edinburgh neural machine translation systems for WMT 16. In Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers, pages 371-376, Berlin, Germany. Association for Computational Linguistics.
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016b. Improving neural machine translation models with monolingual data. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 86-96, Berlin, Germany. Association for Computational Linguistics.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing,
pages 1631-1642, Seattle, Washington, USA. Association for Computational Linguistics.
H. O. Song, Y. Xiang, S. Jegelka, and S. Savarese. 2016. Deep metric learning via lifted structured feature embedding. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4004-4012.
Yulia Tsvetkov, Manaal Faruqui, Wang Ling, Brian MacWhinney, and Chris Dyer. 2016. Learning the curriculum with Bayesian optimization for task-specific word representation learning. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 130-139. Association for Computational Linguistics.
Xinyi Wang, Hieu Pham, Zihang Dai, and Graham Neubig. 2018. SwitchOut: an efficient data augmentation algorithm for neural machine translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 856-861, Brussels, Belgium. Association for Computational Linguistics.
Jason Wei and Kai Zou. 2019. EDA: Easy data augmentation techniques for boosting performance on text classification tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 6382-6388, Hong Kong, China. Association for Computational Linguistics.
Jerry Wei, Chengyu Huang, Soroush Vosoughi, and Jason Wei. 2020. What Are People Asking About COVID-19? A Question Classification Dataset. arXiv preprint arXiv:2005.12522.
Daphna Weinshall, Gad Cohen, and Dan Amir. 2018. Curriculum learning by transfer learning: Theory and experiments with deep networks. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 5238-5246. PMLR.
Qizhe Xie, Zihang Dai, Eduard Hovy, Minh-Thang Luong, and Quoc V. Le. 2020. Unsupervised data augmentation for consistency training.
Kashnitsky Yury. 2020. Hierarchical text classification of Amazon product reviews.
Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems 28, pages 649-657. Curran Associates, Inc.
HUFF
c = 41	FEWREL
c = 64	COV-C
c = 87	AMZN
c = 318
SINGLE-STAGE TRAINING
Updates until convergence, no aug. (approx)	4,000	8,000	4,000	15,000
Update until convergence, aug. (approx)	10,000	10,000	8,000	20,000
Total updates	15,000	15,000	15,000	25,000
CURRICULUM: TWO-STAGE
Stage 1 updates	4,000	6,000	4,000	8,000
Stage 2 updates	11,000	9,000	11,000	17,000
Total updates	15,000	15,000	15,000	25,000
CURRICULUM: GRADUAL
Stage 1 updates	6,000	6,000	6,000	10,000
Updates per stage in stages 2-6	6,000	6,000	4,000	8,000
Total updates	36,000	36,000	26,000	50,000
Table 3: Training schedules for single-stage training, two-stage curriculum training, and gradual curriculum training.
Table 3 shows the training schedules for single-stage, two-stage curriculum, and gradual curriculum training.
All models in standard single-stage training (with and without augmentation) for the same dataset trained for the same number of updates; convergence typically took longer with augmentation compared to without augmentation.
Curriculum two-stage training employs a first stage of only original data and a second stage of augmented data, using the same number of updates as single-stage training in total. We determined the number of updates in the first stage based on when training loss plateaued in the training plot for training with no augmentation.
The gradual curriculum starts with one stage of training with original data only and then increases the augmentation temperature by 0.1 in each of the following five stages. To determine the number of updates in each stage, we examined training plots in preliminary experiments and increased the augmentation temperature (i.e., begun the next stage) whenever training loss plateaued. Since our preliminary experiments already showed relatively strong performance improvements, we did not perform an extensive hyperparameter search or experiment with automatic scheduling, which could further improve performance. As the gradual curriculum trains on more diverse set of augmented data, more updates are needed than in the single-stage and two-stage schedules.
For evaluation, we evaluate our models every 200 updates for COV-C and every 300 updates for HUFF, FEWREL, and AMZN, reporting the highest validation accuracy achieved during training.
In all models, we include  $20\%$  original data whenever augmented data is used, in order to prevent catastrophic forgetting.