Thacio Garcia Scandaroli commited on
Commit
fedb6f2
1 Parent(s): 54fbe35

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -5
README.md CHANGED
@@ -18,10 +18,10 @@ library_name: transformers
18
 
19
  <!-- Provide a longer summary of what this model is. -->
20
 
21
- ULT5-pt é um modelo de arquitetura T5-v1.1 treinado com o framework UL2 - [Unifying Language Learning Paradigms](https://arxiv.org/abs/2205.05131v1), que utiliza Mixture-of-Denoisers (MoD), o qual combina o objetivo de Causal Language Modeling (CLM) com Span Corruption.
22
-
23
  *ULT5-pt is a T5-v1.1 architecture model trained using the UL2 - Unifying Language Learning Paradigms framework, which uses Mixture-of-Denoisers (MoD), combining Causal Language Modeling (CLM) objective with Span Corruption.*
24
 
 
 
25
  | Model | Parameters |
26
  | :-: | :-: |
27
  | [thacio/ult5-pt-small](https://huggingface.co/thacio/ult5-pt-small) | 82.4M |
@@ -54,11 +54,11 @@ Os modos de *span corruption* podem ser acionados adicionado ao início do text
54
  Os autores do UL2 apontam uma possivel diferença no resultado do fine-tune dependendo do modo ativado.
55
  Porém, para o ult5-pt, não se notou diferença nos testes de benchmark.
56
 
57
- *Fine-tunning is the recommended use for the model.
58
 
59
- A tutorial (in portuguse) in notebook format for decoder and encoder-decoder (T5) model fine-tuning was provided: [Fine-tune Large Language Models](link here).
60
 
61
- Span corruption modes can be activated by adding the prefixes '<|NLU|>' and '<|NLG|>' to the beginning of the text. The UL2 authors point out a possible difference in the fine-tuning result depending on the activated mode. However, for ult5-pt, no difference was noticed in benchmark tests.*
62
 
63
  ### Direct Use
64
 
 
18
 
19
  <!-- Provide a longer summary of what this model is. -->
20
 
 
 
21
  *ULT5-pt is a T5-v1.1 architecture model trained using the UL2 - Unifying Language Learning Paradigms framework, which uses Mixture-of-Denoisers (MoD), combining Causal Language Modeling (CLM) objective with Span Corruption.*
22
 
23
+ ULT5-pt é um modelo de arquitetura T5-v1.1 treinado com o framework UL2 - [Unifying Language Learning Paradigms](https://arxiv.org/abs/2205.05131v1), que utiliza Mixture-of-Denoisers (MoD), o qual combina o objetivo de Causal Language Modeling (CLM) com Span Corruption.
24
+
25
  | Model | Parameters |
26
  | :-: | :-: |
27
  | [thacio/ult5-pt-small](https://huggingface.co/thacio/ult5-pt-small) | 82.4M |
 
54
  Os autores do UL2 apontam uma possivel diferença no resultado do fine-tune dependendo do modo ativado.
55
  Porém, para o ult5-pt, não se notou diferença nos testes de benchmark.
56
 
57
+ *Fine-tunning is the recommended use for the model.*
58
 
59
+ *A tutorial (in portuguse) in notebook format for decoder and encoder-decoder (T5) model fine-tuning was provided: [Fine-tune Large Language Models](link here).*
60
 
61
+ *Span corruption modes can be activated by adding the prefixes '<|NLU|>' and '<|NLG|>' to the beginning of the text. The UL2 authors point out a possible difference in the fine-tuning result depending on the activated mode. However, for ult5-pt, no difference was noticed in benchmark tests.*
62
 
63
  ### Direct Use
64