Update README.md

README.md

@@ -22,6 +22,7 @@ tags:
 - foundation model
 datasets:
 - PORTULAN/glue-ptpt
+- PORTULAN/extraglue
 ---
 </br>
 </br>
@@ -82,7 +83,7 @@ Gervásio-7B-PTPT-Decoder is distributed under an [MIT license](https://huggingf
 
 # Training Data
 
-**Gervásio 7B PT-PT** over standard supervised fine-tuning, and to keep some alignment with mainstream benchmarks for English, we resorted to tasks and respective datasets in the GLUE and the SuperGLUE collections.
+**Gervásio 7B PT-PT** was trained over standard supervised fine-tuning, and to keep some alignment with mainstream benchmarks for English, we resorted to tasks and respective datasets in the GLUE and the SuperGLUE collections.
 
 
 We selected those datasets where the outcome of their machine translation into Portuguese could preserve, in the target language, the linguistic properties at stake.
@@ -102,11 +103,11 @@ And from SuperGLUE, we included these other four tasks:
 
 Instruction templates have been manually crafted for each task.
 These take the various fields in the dataset and arrange them into a prompt.
-These templates are listed in full detail in TODO.
+These templates are listed in full detail in the [Extraglue dataset](https://huggingface.co/datasets/PORTULAN/extraglue).
 
 # Training Details
 
-We applied supervised fine-tuning with causal language modeling (CLM) training objective with a zero-out technique during the fine-tuning process.
+We applied supervised fine-tuning with a causal language modeling (CLM) training objective following a zero-out technique during the fine-tuning process.
 Specifically, while the entire prompt received attention during fine-tuning, only the response tokens were subjected to back-propagation.
 
 In terms of hyper-parameters, both models were trained with a learning rate of 2 * 10^-5, a weight decay of 0.1, a two-epoch training regime without warm-up, and to ensure the same number of tokens back-propagated per step, we employed an input sequence of 512 tokens with a batch size of 16 and 16 accumulation steps.
@@ -139,7 +140,7 @@ You can use this model directly with a pipeline for causal language modeling (CL
 
 ```python3
 >>> from transformers import pipeline
->>> generator = pipeline(model='PORTULAN/gervasio-ptpt-decoder')
+>>> generator = pipeline(model='PORTULAN/gervasio-7b-portuguese-ptpt-decoder')
 >>> generator("A música portuguesa é", max_new_tokens=10)
 [{'generated_text': 'A música portuguesa é uma das mais ricas do mundo'}]
 
@@ -156,4 +157,4 @@ grant PINFRA/22117/2016; research project GPT-PT - Transformer-based Decoder for
 grant CPCA-IAC/AV/478395/2022; innovation project 
 ACCELERAT.AI - Multilingual Intelligent Contact Centers, funded by IAPMEI, I.P. - Agência para a Competitividade e Inovação 
 under the grant C625734525-00462629, of Plano de Recuperação e Resiliência, 
-call RE-C05-i01.01 – Agendas/Alianças Mobilizadoras para a Reindustrialização. 
+call RE-C05-i01.01 – Agendas/Alianças Mobilizadoras para a Reindustrialização.