Update README.md
Browse files
README.md
CHANGED
@@ -1,5 +1,13 @@
|
|
1 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2 |
library_name: transformers
|
|
|
3 |
license: apache-2.0
|
4 |
---
|
5 |
|
@@ -21,15 +29,15 @@ to EncT5:
|
|
21 |
1. There are less decoder layers (a single decoder layer by default), and so has fewer parameters/resources than the
|
22 |
standard T5.
|
23 |
3. There is a separate decoder word embedding, with the decoder input ids being predefined constants. During
|
24 |
-
fine-tuning,
|
25 |
classification/regression tasks.
|
26 |
-
|
27 |
|
28 |
Research has shown that this model can be more efficient and usable over T5 and BERT for non-autoregressive
|
29 |
tasks such as classification and regression.
|
30 |
|
31 |
-
- **Developed by:** Frederick Liu, Terry Huang, Shihang Lyu, Siamak Shakeri, Hongkun Yu, Jing Li. See
|
32 |
-
[associated paper](https://arxiv.org/abs/2110.08426)
|
33 |
- **Model type:** Language Model
|
34 |
- **Language(s) (NLP):** English, French, Romanian, German
|
35 |
- **License:** Apache 2.0
|
@@ -41,8 +49,10 @@ tasks such as classification and regression.
|
|
41 |
|
42 |
Use the code below to get started with the model.
|
43 |
|
44 |
-
|
45 |
-
|
|
|
|
|
46 |
|
47 |
See the [github repro](https://github.com/hackyon/EncT5) for a more comprehensive guide.
|
48 |
|
|
|
1 |
---
|
2 |
+
language:
|
3 |
+
- en
|
4 |
+
- fr
|
5 |
+
- ro
|
6 |
+
- de
|
7 |
+
datasets:
|
8 |
+
- c4
|
9 |
library_name: transformers
|
10 |
+
|
11 |
license: apache-2.0
|
12 |
---
|
13 |
|
|
|
29 |
1. There are less decoder layers (a single decoder layer by default), and so has fewer parameters/resources than the
|
30 |
standard T5.
|
31 |
3. There is a separate decoder word embedding, with the decoder input ids being predefined constants. During
|
32 |
+
fine-tuning, the decoder embedding learns to use these constants as "prompts" to the encoder for the corresponding
|
33 |
classification/regression tasks.
|
34 |
+
5. There is a classification head on top of the decoder output.
|
35 |
|
36 |
Research has shown that this model can be more efficient and usable over T5 and BERT for non-autoregressive
|
37 |
tasks such as classification and regression.
|
38 |
|
39 |
+
- **Developed by:** Frederick Liu, Terry Huang, Shihang Lyu, Siamak Shakeri, Hongkun Yu, Jing Li. See the
|
40 |
+
[associated paper](https://arxiv.org/abs/2110.08426).
|
41 |
- **Model type:** Language Model
|
42 |
- **Language(s) (NLP):** English, French, Romanian, German
|
43 |
- **License:** Apache 2.0
|
|
|
49 |
|
50 |
Use the code below to get started with the model.
|
51 |
|
52 |
+
```python
|
53 |
+
model = AutoModelForSequenceClassification.from_pretrained("hackyon/enct5-base", trust_remote_code=True)
|
54 |
+
# Fine-tune the model before use.
|
55 |
+
```
|
56 |
|
57 |
See the [github repro](https://github.com/hackyon/EncT5) for a more comprehensive guide.
|
58 |
|