Update README.md

README.md

@@ -30,17 +30,17 @@ For training, we used all Icelandic data that was present in the monolingual Ice
 
 # Benchmark performance
 
-We tested the performance of **XLMR-MaCoCu-is** on benchmarks of XPOS, UPOS and NER. For UPOS and XPOS, we used the data from the [Universal Dependencies](https://universaldependencies.org/) project. For NER, we used the data from the MIM-GOLD-NER data set. We compare performance to the strong multi-lingual models XLMR-base and XLMR-large, but also the monolingual [IceBERT](https://huggingface.co/vesteinn/IceBERT) model. For details regarding the fine-tuning procedure you can checkout our [Github](https://github.com/macocu/LanguageModels).
+We tested the performance of **XLMR-MaCoCu-is** on benchmarks of XPOS, UPOS, NER and COPA. For UPOS and XPOS, we used the data from the [Universal Dependencies](https://universaldependencies.org/) project. For NER, we used the data from the MIM-GOLD-NER data set. For COPA, we automatically translated the English data set by using Google Translate. For details please see our [Github repo](https://github.com/RikVN/COPA). We compare performance to the strong multi-lingual models XLMR-base and XLMR-large, but also the monolingual [IceBERT](https://huggingface.co/vesteinn/IceBERT) model. For details regarding the XPOS/UPOS/NER fine-tuning procedure you can checkout our [Github](https://github.com/macocu/LanguageModels).
 
-Scores are averages of three runs. We use the same hyperparameter settings for all models.
+Scores are averages of three runs, except for COPA, for which we use 10 runs. We use the same hyperparameter settings for all models.
 
-|                    | **UPOS** | **UPOS** | **XPOS** | **XPOS** | **NER** | **NER**  |
-|--------------------|:--------:|:--------:|:--------:|:--------:|---------|----------|
-|                    |  **Dev** | **Test** |  **Dev** | **Test** | **Dev** | **Test** |
-| **XLM-R-base**     |   96.8   |   96.5   |   94.6   |   94.3   |   85.3  |   89.7   |
-| **XLM-R-large**    |   97.0   |   96.7   |   94.9   |   94.7   |   88.5  |   91.7   |
-| **IceBERT**        |   96.4   |   96.0   |   94.0   |   93.7   |   83.8  |   89.7   |
-| **XLMR-MaCoCu-is** |   **97.3**   |    **97.0**    |   **95.4**   |   **95.1**  |   **90.8**  |   **93.2**   |
+|                    | **UPOS** | **UPOS** | **XPOS** | **XPOS** | **NER** | **NER**  | **COPA**  |
+|--------------------|:--------:|:--------:|:--------:|:--------:|---------|----------| ----------|
+|                    |  **Dev** | **Test** |  **Dev** | **Test** | **Dev** | **Test** | **Test** |
+| **XLM-R-base**     |   96.8   |   96.5   |   94.6   |   94.3   |   85.3  |   89.7   | 55.2 |
+| **XLM-R-large**    |   97.0   |   96.7   |   94.9   |   94.7   |   88.5  |   91.7   | 54.3 |
+| **IceBERT**        |   96.4   |   96.0   |   94.0   |   93.7   |   83.8  |   89.7   | 54.6 | 
+| **XLMR-MaCoCu-is** |   **97.3**   |    **97.0**    |   **95.4**   |   **95.1**  |   **90.8**  |   **93.2**   | **59.6** |
 
 # Acknowledgements