Update README.md
Browse files
README.md
CHANGED
@@ -10,6 +10,7 @@ datasets:
|
|
10 |
Tk-Instruct is a series of encoder-decoder Transformer models that are trained to solve various NLP tasks by following in-context instructions (plain language task definitions, k-shot examples, explanations, etc). Built upon the pre-trained [T5 models](https://arxiv.org/abs/1910.10683), they are fine-tuned on a large number of tasks & instructions that are collected in the [Natural Instructions benchmark](https://github.com/allenai/natural-instructions), which contains 1600+ tasks in 70+ broach categories in total. This enables the model to not only process the training tasks, but also generalize to many unseen tasks without further parameter update.
|
11 |
|
12 |
More resources for using the model:
|
|
|
13 |
- **Code repository**: [Tk-Instruct](https://github.com/yizhongw/Tk-Instruct)
|
14 |
- **Official Website**: [Natural Instructions](https://instructions.apps.allenai.org/)
|
15 |
- **All released models**: [allenai/tk-instruct](https://huggingface.co/models?search=allenai/tk-instruct)
|
@@ -52,7 +53,9 @@ If you find serious issues or any interesting result, you are welcome to share w
|
|
52 |
|
53 |
## Training data
|
54 |
|
55 |
-
Tk-Instruct is trained using the tasks & instructions in [Natural Instructions benchmark](https://github.com/allenai/natural-instructions), which contains 1600+ tasks in 70+ broach categories in total. We follow the official train/test split. Tk-Instruct model series were trained using 757 tasks, and mTk-Instruct series were trained using 1271 tasks (including some non-English tasks).
|
|
|
|
|
56 |
|
57 |
|
58 |
## Training procedure
|
@@ -73,6 +76,4 @@ Although they are trained with only one type of instruction encodings, we found
|
|
73 |
eprint={2204.07705},
|
74 |
primaryClass={cs.CL},
|
75 |
}
|
76 |
-
```
|
77 |
-
|
78 |
-
|
|
|
10 |
Tk-Instruct is a series of encoder-decoder Transformer models that are trained to solve various NLP tasks by following in-context instructions (plain language task definitions, k-shot examples, explanations, etc). Built upon the pre-trained [T5 models](https://arxiv.org/abs/1910.10683), they are fine-tuned on a large number of tasks & instructions that are collected in the [Natural Instructions benchmark](https://github.com/allenai/natural-instructions), which contains 1600+ tasks in 70+ broach categories in total. This enables the model to not only process the training tasks, but also generalize to many unseen tasks without further parameter update.
|
11 |
|
12 |
More resources for using the model:
|
13 |
+
- **Paper**: [link](https://arxiv.org/abs/2204.07705)
|
14 |
- **Code repository**: [Tk-Instruct](https://github.com/yizhongw/Tk-Instruct)
|
15 |
- **Official Website**: [Natural Instructions](https://instructions.apps.allenai.org/)
|
16 |
- **All released models**: [allenai/tk-instruct](https://huggingface.co/models?search=allenai/tk-instruct)
|
|
|
53 |
|
54 |
## Training data
|
55 |
|
56 |
+
Tk-Instruct is trained using the tasks & instructions in [Natural Instructions benchmark](https://github.com/allenai/natural-instructions), which contains 1600+ tasks in 70+ broach categories in total. We follow the official train/test split. Tk-Instruct model series were trained using 757 tasks, and mTk-Instruct series were trained using 1271 tasks (including some non-English tasks).
|
57 |
+
|
58 |
+
The training tasks are in 64 broad categories, such as text categorization / question answering / sentiment analysis / summarization / grammar error detection / dialogue generation / etc. The other 12 categories are selected for evaluation.
|
59 |
|
60 |
|
61 |
## Training procedure
|
|
|
76 |
eprint={2204.07705},
|
77 |
primaryClass={cs.CL},
|
78 |
}
|
79 |
+
```
|
|
|
|