SlowGuess/NAACL_2022_OCR huggingface dataset:
5. TREC-Fine. (Li and Roth, 2002) contains the same set of questions as TREC-Coarse grouped into 47 fine-grained classes.
6. AHS. (Albadi et al., 2018) is an Arabic hate speech classification dataset focusing mainly on Saudi Twittersphere.
7. RTE. (Bentivogli et al., 2009) The Recognizing Textual Entailment (RTE) datasets come from a series of textual entailment challenges. Data from RTE1, RTE2, RTE3 and RTE5 is combined. Examples are constructed based on news and Wikipedia text. Metric used here is accuracy
8. MRPC.(Dolan and Brockett, 2005) The Microsoft Research Paraphrase Corpus is a corpus of sentence pairs automatically extracted from online news sources, with human annotations for whether the sentences in the pair are semantically equivalent. Metric used here is accuracy
9. CONLL. (Tjong Kim Sang and De Meulder, 2003) It is a named entity recognition dataset released as a part of CoNLL-2003 shared task: language-independent named entity recognition. The data consists of eight files covering two languages: English and German.
We provide a detailed explanation of our experimental setup in table 6
Parameter	Value
Optimizer	BERTAdam 2020
Learning Rate	2e-5
Batch Size	8
β1, β2, ε	0.9, 0.999, 1e-6
# Epochs	10
Evaluation Metric	F1 Score
Base Model	BERT-base-uncased, BERT-base-multilingual-uncased
Classifier (over architecture)	Linear layer
Hardware	Nvidia V100
Table 6: Model and training setup for CIAug.
Mixup Layer Set	AHS	MRPC
{3,4}	71.12	82.84
{1,2}	72.21	85.62
{6,7,9}	72.57	83.08
{7,9,12}	73.77	84.06
{3,4,6,7,9,12}	74.14	87
Table 7: Layer-wise ablation score (F1) when performing interpolative augmentation.
E.1 Layer-wise Ablation
We compare the performance of CIAug on different set of mixup layers in table 7. TMix attains best performance on layer set 7,9,12 is used because layers 6,7,9,12 contains the most amount of syntactic and semantic information (Chen et al., 2020a). CIAug achieves best performance on layer set 3,4,6,7,9,12, this suggests other than syntactic and semantic information, curriculum learning based approach in CIAug helps to capture the surface-level information in layer 3 and 4 (Jawahar et al., 2019).
Even though CIAug converges faster and performs better than other baseline interpolative techniques, the computational power required is not so fairly available in many devices. We plan to work on improving the efficiency of the parameterized matrices involved in the computation, such as using sparse matrices.