uer commited on
Commit
dbd1394
1 Parent(s): d38b6b8

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -1
README.md CHANGED
@@ -12,7 +12,7 @@ widget:
12
 
13
  ## Model description
14
 
15
- This is the set of Chinese T5 models pre-trained by [UER-py](https://github.com/dbiir/UER-py/), which is introduced in [this paper](https://arxiv.org/abs/1909.05658).
16
 
17
  The Text-to-Text Transfer Transformer (T5) leverages a unified text-to-text format and attains state-of-the-art results on a wide variety of English-language NLP tasks. Following their work, we released a series of Chinese T5 models.
18
 
@@ -120,6 +120,13 @@ python3 scripts/convert_t5_from_uer_to_huggingface.py --input_model_path cluecor
120
  pages={241},
121
  year={2019}
122
  }
 
 
 
 
 
 
 
123
  ```
124
 
125
  [small]:https://huggingface.co/uer/t5-small-chinese-cluecorpussmall
 
12
 
13
  ## Model description
14
 
15
+ This is the set of Chinese T5 models pre-trained by [UER-py](https://github.com/dbiir/UER-py/), which is introduced in [this paper](https://arxiv.org/abs/1909.05658). Besides, the models could also be pre-trained by [TencentPretrain](https://github.com/Tencent/TencentPretrain) introduced in [this paper](https://arxiv.org/abs/2212.06385), which inherits UER-py to support models with parameters above one billion, and extends it to a multimodal pre-training framework.
16
 
17
  The Text-to-Text Transfer Transformer (T5) leverages a unified text-to-text format and attains state-of-the-art results on a wide variety of English-language NLP tasks. Following their work, we released a series of Chinese T5 models.
18
 
 
120
  pages={241},
121
  year={2019}
122
  }
123
+
124
+ @article{zhao2023tencentpretrain,
125
+ title={TencentPretrain: A Scalable and Flexible Toolkit for Pre-training Models of Different Modalities},
126
+ author={Zhao, Zhe and Li, Yudong and Hou, Cheng and Zhao, Jing and others},
127
+ journal={ACL 2023},
128
+ pages={217},
129
+ year={2023}
130
  ```
131
 
132
  [small]:https://huggingface.co/uer/t5-small-chinese-cluecorpussmall