hackyon commited on
Commit
c0285b0
1 Parent(s): 0e378f2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +16 -6
README.md CHANGED
@@ -1,5 +1,13 @@
1
  ---
 
 
 
 
 
 
 
2
  library_name: transformers
 
3
  license: apache-2.0
4
  ---
5
 
@@ -21,15 +29,15 @@ to EncT5:
21
  1. There are less decoder layers (a single decoder layer by default), and so has fewer parameters/resources than the
22
  standard T5.
23
  3. There is a separate decoder word embedding, with the decoder input ids being predefined constants. During
24
- fine-tuning, these constants are trained to effectively "prompt" the encoder to perform the necessary
25
  classification/regression tasks.
26
- 4. There is a classification head on top of the decoder output.
27
 
28
  Research has shown that this model can be more efficient and usable over T5 and BERT for non-autoregressive
29
  tasks such as classification and regression.
30
 
31
- - **Developed by:** Frederick Liu, Terry Huang, Shihang Lyu, Siamak Shakeri, Hongkun Yu, Jing Li. See
32
- [associated paper](https://arxiv.org/abs/2110.08426)
33
  - **Model type:** Language Model
34
  - **Language(s) (NLP):** English, French, Romanian, German
35
  - **License:** Apache 2.0
@@ -41,8 +49,10 @@ tasks such as classification and regression.
41
 
42
  Use the code below to get started with the model.
43
 
44
- model = AutoModelForSequenceClassification.from_pretrained("hackyon/enct5-base", trust_remote_code=True)
45
- # Fine-tune the model before use.
 
 
46
 
47
  See the [github repro](https://github.com/hackyon/EncT5) for a more comprehensive guide.
48
 
 
1
  ---
2
+ language:
3
+ - en
4
+ - fr
5
+ - ro
6
+ - de
7
+ datasets:
8
+ - c4
9
  library_name: transformers
10
+
11
  license: apache-2.0
12
  ---
13
 
 
29
  1. There are less decoder layers (a single decoder layer by default), and so has fewer parameters/resources than the
30
  standard T5.
31
  3. There is a separate decoder word embedding, with the decoder input ids being predefined constants. During
32
+ fine-tuning, the decoder embedding learns to use these constants as "prompts" to the encoder for the corresponding
33
  classification/regression tasks.
34
+ 5. There is a classification head on top of the decoder output.
35
 
36
  Research has shown that this model can be more efficient and usable over T5 and BERT for non-autoregressive
37
  tasks such as classification and regression.
38
 
39
+ - **Developed by:** Frederick Liu, Terry Huang, Shihang Lyu, Siamak Shakeri, Hongkun Yu, Jing Li. See the
40
+ [associated paper](https://arxiv.org/abs/2110.08426).
41
  - **Model type:** Language Model
42
  - **Language(s) (NLP):** English, French, Romanian, German
43
  - **License:** Apache 2.0
 
49
 
50
  Use the code below to get started with the model.
51
 
52
+ ```python
53
+ model = AutoModelForSequenceClassification.from_pretrained("hackyon/enct5-base", trust_remote_code=True)
54
+ # Fine-tune the model before use.
55
+ ```
56
 
57
  See the [github repro](https://github.com/hackyon/EncT5) for a more comprehensive guide.
58