SlowGuess/NAACL_2021_OCR huggingface dataset:
John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, C. Clopath, D. Kumaran, and Raia Hadsell. 2017. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114:3521 - 3526.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. 2016. Adversarial examples in the physical world. arXiv preprint arXiv:1607.02533.
Stefan Larson, Anish Mahendran, Joseph Peper, Christopher Clarke, Andrew Lee, Parker Hill, Jonathan K. Kummerfeld, Kevin Leach, Michael Laurenzano, Lingjia Tang, and Jason Mars. 2019. An evaluation dataset for intent classification and out-of-scope prediction. In EMNLP/IJCNLP.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. 2018. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. ArXiv, abs/1807.03888.
Shiyu Liang, Yixuan Li, and R. Srikant. 2017. Principled detection of out-of-distribution examples in neural networks. ArXiv, abs/1706.02690.
Shiyu Liang, Yixuan Li, and R. Srikant. 2018. Enhancing the reliability of out-of-distribution image detection in neural networks. arXiv: Learning.
Ting-En Lin and Hua Xu. 2019. Deep unknown intent detection with margin loss. In ACL.
Takeru Miyato, Andrew M Dai, and Ian Goodfellow. 2016. Adversarial training methods for semi-supervised text classification. arXiv preprint arXiv:1605.07725.
Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532-1543.
Jie Ren, Peter J. Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark A. DePristo, Joshua V. Dillon, and Balaji Lakshminarayanan. 2019a. Likelihood ratios for out-of-distribution detection. ArXiv, abs/1906.02845.
Shuhuai Ren, Yihe Deng, Kun He, and Wanxiang Che. 2019b. Generating natural language adversarial examples through probability weighted word saliency. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1085-1097.
Andras Rozsa, Ethan M Rudd, and Terrance E Boult. 2016. Adversarial diversity and hard positive generation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 25-32.
Ruhi Sarikaya. 2017. The technology behind personal digital assistants: An overview of the system architecture and key components. IEEE Signal Processing Magazine, 34:67-81.
Walter J. Scheirer, Anderson Rocha, Archana Sapkota, and Terrance E. Boult. 2013. Toward open set recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35:1757-1772.
Lei Shu, Hu Xu, and Bing Liu. 2017. Doc: Deep open classification of text documents. In EMNLP.
H. Shum, X. He, and Di Li. 2018. From eliza to xiaoice: challenges and opportunities with social chatbots. Frontiers of Information Technology Electronic Engineering, 19:10-26.
J. Snell, Kevin Swersky, and R. Zemel. 2017. Prototypical networks for few-shot learning. In NIPS.
Amrita S. Tulshan and S. Dhage. 2018. Survey on virtual assistant: Google assistant, siri, cortana, alexa.
J. Winkens, R. Bunel, Abhijit Guha Roy, Robert Stanforth, V. Natarajan, Joseph R. Ledsam, Patricia MacWilliams, Pushmeet Kohli, Alan Karthikesalingam, Simon Kohl, T. cemgil, S. Eslami, and O. Ronneberger. 2020. Contrastive training for improved out-of-distribution detection. ArXiv, abs/2007.05566.
H. Xu, Keqing He, Yuanmeng Yan, Si hong Liu, Z. Liu, and Weiran Xu. 2020. A deep generative distance-based classifier for out-of-domain detection with mahalanobis space. In COLING.
Huangzhao Zhang, Hao Zhou, Ning Miao, and Lei Li. 2019. Generating fluent adversarial examples for natural languages. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5564-5569.
Yinhe Zheng, Guanyi Chen, and Minlie Huang. 2020. Out-of-domain detection for natural language understanding in dialog systems. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 28:1198-1209.
We sample the augmentation data proportional to the size of the training set of each dataset. We use public APIs from multiple platforms to complete the back-translation process. Considering the availability of augmented data obtained through the back-translation process, we only sample back translated sequences that are more than  $70\%$  and less than  $90\%$  similar to the origin text in words overlapping. The total amount of data we sampled is equal to  $10\%$  of the volume of in-domain training data. We use the pre-trained GloVe embeddings (Pennington et al., 2014) as the word embedding matrix. For the BiLSTM encoder, we set the dimension of hidden states to 128 and use a dropout rate of 0.5. We use Adam optimizer (Kingma and Ba, 2014) to train our model and use a learning rate of 0.001. In the training stage, 20 epochs of supervised training are first conducted on in-domain labeled data, and then 200 epochs of alternate training are conducted by adding the process of contrastive learning on unlabeled data. The alternate training stage has an early stop setting with patience equalling 20. The algorithm of our proposed training process can be found in the Algorithm.1. We use the best F1-scores on the validation set to calculate the GDA threshold adaptively. Each result of the experiments is tested 5 times under the same setting and gets the average value. The amplitude of adversarial perturbation is obtained by the heuristic method in the range of 0 to 1E-2 (2.5/250), in which MSP and LOF are 4E-3 (1.0/250) and GDA is 6E-3 (1.5/250). In order to fairly compare with other settings, we set the weights of two losses equal in DS ( $\alpha = 1$  in Algorithm.1) and S2A ( $\beta = 0.5$  in Algorithm.1). The training stage of our model lasts about 15 minutes on a single Tesla T4 GPU(16 Gb of memory). The average value of the model parameters is 2.52M.