Files changed (1) hide show
  1. README.md +3 -1
README.md CHANGED
@@ -11,7 +11,9 @@ license: apache-2.0
11
 
12
  ## Version 1.1
13
 
14
- [T5 Version 1.1](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/released_checkpoints.md#t511) includes the following improvements compared to the original T5 model- GEGLU activation in feed-forward hidden layer, rather than ReLU - see [here](https://arxiv.org/abs/2002.05202).
 
 
15
 
16
  - Dropout was turned off in pre-training (quality win). Dropout should be re-enabled during fine-tuning.
17
 
 
11
 
12
  ## Version 1.1
13
 
14
+ [T5 Version 1.1](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/released_checkpoints.md#t511) includes the following improvements compared to the original T5 model
15
+
16
+ - GEGLU activation in feed-forward hidden layer, rather than ReLU - see [here](https://arxiv.org/abs/2002.05202).
17
 
18
  - Dropout was turned off in pre-training (quality win). Dropout should be re-enabled during fine-tuning.
19