Update README.md

README.md

@@ -12,8 +12,8 @@ widget:
 - text: 'translate spanish to nahuatl: México lindo y querido.'
 ---
 
-# t5-small-spanish-nahuatl
-Nahuatl is the most widely spoken indigenous language in Mexico. However, training a neural network for the neural machine translation task is challenging due to the lack of structured data. The most popular datasets, such as the Axolot and bible-corpus, only consist of ~16,000 and ~7,000 samples, respectively. Moreover, there are multiple variants of Nahuatl, which makes this task even more difficult. For example, it is possible to find a single word from the Axolot dataset written in more than three different ways. Therefore, we leverage the T5 text-to-text prefix training strategy to compensate for the lack of data. We first train the multilingual model to learn Spanish and then adapt it to Nahuatl. The resulting T5 Transformer successfully translates short sentences. Finally, we report Chrf and BLEU results.
+# mt5-large-spanish-nahuatl
+Nahuatl is the most widely spoken indigenous language in Mexico, yet training a neural network for machine translation presents significant challenges due to insufficient structured data. Popular datasets, like Axolot and the Bible corpus, contain only approximately 16,000 and 7,000 samples, respectively. Complicating matters further, Nahuatl has multiple dialects, and a single word in the Axolot dataset can appear in over three different forms. I conclude with evaluations of the model's performance using the Chrf and BLEU metrics.
 
 
 ## Model description
@@ -38,7 +38,8 @@ outputs = tokenizer.batch_decode(outputs, skip_special_tokens=True)[0]
 
 ## Approach
 ### Dataset
-Since the Axolotl corpus contains misalignments, we select the best samples (12,207). We also use the [bible-corpus](https://github.com/christos-c/bible-corpus) (7,821). 
+Since the Axolotl corpus contains misalignments, the best samples were selected (12,207). 
+This in addition to the [bible-corpus](https://github.com/christos-c/bible-corpus) (7,821). 
 
 | Axolotl best aligned books                            | 
 |:-----------------------------------------------------:|
@@ -57,10 +58,10 @@ Since the Axolotl corpus contains misalignments, we select the best samples (12,
 | Una tortillita nomás - Se taxkaltsin saj              |
 | Vida económica de Tenochtitlan                        |
 
-Also, we collected 3,000 extra samples from the web to increase the data.
+Also, additional 30,000 samples were collected from the web to enhance the data.
 
 ### Model and training
-We employ two training stages using a multilingual T5-small. The advantage of this model is that it can handle different vocabularies and prefixes. T5-small is pre-trained on different tasks and languages (French, Romanian, English, German).
+The employed method uses a single training stage using the mt5. This model was leveraged given that it can handle different vocabularies and prefixes.
 
 ### Training-stage 1 (learning Spanish)
 In training stage 1, we first introduce Spanish to the model. The goal is to learn a new language rich in data (Spanish) and not lose the previous knowledge. We use the English-Spanish [Anki](https://www.manythings.org/anki/) dataset, which consists of 118.964 text pairs. The model is trained till convergence, adding the prefix "Translate Spanish to English: "
@@ -73,15 +74,16 @@ We train the models on the same datasets for 660k steps using batch size = 16 an
 
 
 ## Evaluation results
-We evaluate the models on the same 505 validation Nahuatl sentences for a fair comparison. Finally, we report the results using chrf and sacrebleu hugging face metrics:
-
-| English-Spanish pretraining  | Validation loss | BLEU | Chrf   |
-|:----------------------------:|:---------------:|:-----|-------:|
-| False                        | 1.34            | 6.17 | 26.96  | 
-| True                         | 1.31            | 6.18 | 28.21  | 
+The models are evaluated on 2 different datasets: 
+1. First on the test sentences similar to the evaluation ones.
+2. Then, on zero-shot sentences obtained from the test sentences of AmericasNLP2021
 
+The results are reported using CHRF++ and BLEU:
 
-The English-Spanish pretraining improves BLEU and Chrf and leads to faster convergence. The evaluation is available on the [eval.ipynb](https://github.com/milmor/spanish-nahuatl-translation/blob/main/eval.ipynb) notebook. 
+| Nahuatl-Spanish Bidirectional Training  | Set | BLEU | CHRF++   |
+|:----------------------------:|:---------------:|:-----|-------:|
+| True                        | Test            | 18.01 | 54.15  | 
+| True                        | Zero-shot            | 5.24 | 25.7  | 
 
 ## References
 - Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2019. Exploring the limits