SlowGuess/NAACL_2022_OCR huggingface dataset:
Shuo Wang, Yang Liu, Chao Wang, Huanbo Luan, and Maosong Sun. 2019. Improving back-translation with uncertainty-based confidence estimation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 791-802.
Xinyi Wang, Hieu Pham, Zihang Dai, and Graham Neubig. 2018. Swithout: an efficient data augmentation algorithm for neural machine translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 856-861.
Roman Werpachowski, András György, and Csaba Szepesvári. 2019. Detecting overfitting via adversarial examples. Advances in Neural Information Processing Systems, 32.
Lijun Wu, Yiren Wang, Yingce Xia, Tao Qin, Jianhuang Lai, and Tie-Yan Liu. 2019. Exploiting monolingual data at scale for neural machine translation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4207-4216.
Jiajun Zhang and Chengqing Zong. 2016. Exploiting source-side monolingual data in neural machine translation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1535-1545.
Zhirui Zhang, Shuo Ren, Shujie Liu, Jianyong Wang, Peng Chen, Mu Li, Ming Zhou, and Enhong Chen 2018. Style transfer as unsupervised machine translation. arXiv preprint arXiv:1808.07894.
Jinhua Zhu, Yingce Xia, Lijun Wu, Di He, Tao Qin, Wengang Zhou, Houqiang Li, and Tieyan Liu. 2020. Incorporating bert into neural machine translation. In International Conference on Learning Representations.
The models are optimized using Adam optimizer (Kingma and Ba, 2015), with  $\beta_{1} = 0.9$ ,  $\beta_{2} = 0.98$ . We use the same learning rate scheduler as (Vaswani et al., 2017) with maximum learning rate  $7 \times 10^{-4}$ , and 4000 warmup updates. We use the fairseq 10.2 as the framework and the training command as well as the model hyperparameters are listed below,
And the GPT model we employ is only trained on source side of bitext corpus, without extra datasets. The training command line and core settings are listed below.