SlowGuess/ACL_2020_OCR huggingface dataset:
Once all the layers have been unfrozen, we update all the parameters together. This technique allows us get the most of our model moving from English to a code-switching setting. We train our fine-tuned models for 200 epochs and a initial learning rate of 0.01 that gets modified during training.
Additionally, we use this fine-tuning process for the downstream NLP task presented in the paper (i.e., NER and POS tagging).
E Visualization of Attention Weights for Hindi-English
Figure 4 shows the attention behavior for tri-grams on the Hindi-English dataset. Similar to the cases discussed for Spanish-English in the main content, we observe that the model learns tri-grams like -ing, -ian for English and iye, isi for Hindi.
8044