SlowGuess/NAACL_2022_OCR huggingface dataset:
Jeanine Treffers-Daller. 2020. Turkish-German code-switching patterns revisited: What naturalistic data can(not) tell us. Advances in Contact Linguistics: In honour of Pieter Muysken, 57:237.
Jeanine Treffers-Daller and Ozlem Çetinoğlu. 2022. TuGeBiC: A Turkish German bilingual code-switching corpus. arXiv, abs/2205.00868.
Leonoor Van der Beek, Gosse Bouma, Rob Malouf, and Gertjan Van Noord. 2002. The Alpine dependency treebank. In Computational linguistics in the Netherlands 2001, pages 8-22. Brill Rodopi.
Rob van der Goot, Ahmet Üstun, Alan Ramponi, Ibrahim Sharaf, and Barbara Plank. 2021. Massive choice, ample tasks (MaChAmp): A toolkit for multitask learning in NLP. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, pages 176-197, Online. Association for Computational Linguistics.
Emre Yilmaz, Maaike Andringa, Sigrid Kingma, Jelske Dijkstra, Frits van der Kuip, Hans Van de Velde, Frederik Kampstra, Jouke Algra, Henk van den Heuvel, and David van Leeuwen. 2016. A longitudinal bilingual Frisian-Dutch radio broadcast database designed for code-switching research. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16), pages 4666-4669, Porto Roz, Slovenia. European Language Resources Association (ELRA).
Xiang Yu, Ngoc Thang Vu, and Jonas Kuhn. 2020. Ensemble self-training for low-resource languages: Grapheme-to-phoneme conversion and morphological inflection. In Proceedings of the 17th SIGMOR-PHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 70-78, Online. Association for Computational Linguistics.
A.1 Model Configuration and Hyper-parameters
We provide the configuration and hyper-parameters of the parser and sequence labeler models presented in Section 4.3.
$\mathsf{Base}_{\mathsf{LSTM}}$  We use Adam optimizer (Kingma and Ba, 2015) with a learning rate of 0.002, batch size of 16, and all dropout probabilities are set to 0.33 for the parser and the sequence labeler models. We train the parser for 150 epochs and sequence labeling tasks for 100 epochs.
We use 300-dimensional FastText embeddings (Grave et al., 2018) as pretrained word vectors. Since these embeddings are monolingual, we choose Russian FastText embeddings for Kpv-Ru, Hindi embeddings for Hi-En, Dutch embeddings for Fy-Nl, and Turkish embeddings for Tr-De treebanks. The model also uses 100-dimensional character embeddings and POS tag embeddings which are randomly initialized. The 3-layer BiLSTM modules of the parser and the sequence labeler have hidden layer size of 512 on each side. The decoder of the parser includes an arc MLP of size 512
Figure 4: Comparison of BaseLSTM, Self-training, and our best model for Hi-En and Fy-Nl in terms of attachment scores.
and a label MLP of size 128. The decoder of the sequence labeler consists of two fully connected layers with size 128 and 64, respectively.
$\mathrm{Base}_{\mathrm{XLMR}}$  Due to computational efficiency, we choose the 768-dimensional XLM-R base language model as the word representation module of the  $\mathrm{Base}_{\mathrm{XLMR}}$  architecture. For the parser, the arc MLP of the biaffine classifier has the same size with XLM-R model and the label MLP has the size of 256. Dropout for the classifier is set to 0.33. For the sequence labeler, we use a single-layer feed-forward neural network to extract logit vectors. We use AdamW optimizer (Loshchilov and Hutter, 2019) with a learning rate of 0.00004 and set batch size to 16. The number of epochs for the parser is 300 with an early stop of 50 epochs. For the sequence labeler, we train the models for 100 epochs with an early stop of 15 epochs.
A.2 Effect of Gold Labeled Data on the Parsing Performance
In our main experiments the gold training data size differs among the four datasets. While the gold labeled data used for training of Kpv-Ru and TrDe includes approximately 500 sentences, Hi-En has 1,448 gold labeled training CS data and for Fy-NI we used the training set of the Dutch Alpino UD Treebank which consists of 12,289 gold la
beled Dutch sentences. In order to observe how the amount of gold labeled training data affects the models' performance, we did a set of experiments on each of Hi-En and Fy-Nl datasets by incrementally increasing the size of labeled training data from 500 to the original training data size as used in the main experiments. Figure 4 shows results of these experiments.
We observe that our best model on these datasets  $(+NOC, +DTR, +RPE$  for Hi-En and  $+NOC, +RPE, +LIH$  for Fy-NI) always surpasses Self-training and BaseLSTM regardless of the available gold training data. Increasing the labeled training data has always a positive effect on the performance of all models for Hi-En but causes fluctuations in the performance for the case of Fy-NI. The reason for this difference might be that the training data of Hi-En is in-domain and includes CS sentences, while the training data we use for Fy-NI is out-of-domain and includes monolingual Dutch sentences.