File size: 6,329 Bytes
d585ab6 2956d3f b5cbf93 d585ab6 2956d3f 30893e8 d585ab6 0e4b9bc 2956d3f 77cfc00 2956d3f 77cfc00 2956d3f |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 |
---
tags:
- text2text-generation
datasets:
- kaist-ai/CoT-Collection
- SirNeural/flan_v2
license: apache-2.0
language:
- en
pipeline_tag: text2text-generation
library_name: transformers
---
## Links for Reference
- **Homepage:https://github.com/kaistAI/CoT-Collection**
- **Repository:https://github.com/kaistAI/CoT-Collection**
- **Paper:https://arxiv.org/abs/2305.14045**
- **Point of Contact:seungone@kaist.ac.kr**
# TL;DR
CoT-T5 is a language model using [Flan-T5](https://huggingface.co/google/flan-t5-xxl) as a base model, and CoT fine-tuned on 1.84 million rationales across 1,060 tasks from the [CoT Collection](https://huggingface.co/datasets/kaist-ai/CoT-Collection).
Since it was CoT fine-tuned on a large amount of rationales, it shows superior performance with CoT compared to Flan-T5.
One could use CoT-T5 for (1) Solving unseen tasks in zero-shot setting, and (2) Adapting to new tasks with CoT fine-tuning.
# Model Details
## Model Description
- **Model type:** Language model
- **Language(s) (NLP):** English
- **License:** Apache 2.0
- **Related Models:** [All CoT-T5 Checkpoints](https://huggingface.co/models?search=cot-t5)
- **Resources for more information:**
- [Research paper](https://arxiv.org/abs/2305.14045)
- [GitHub Repo](https://github.com/kaistAI/CoT-Collection)
CoT-T5 is trained with two different sizes (3B and 11B).
You could check the 3B sized LM on [this page](https://huggingface.co/kaist-ai/CoT-T5-3B).
Also, check out our dataset as well on [this page](https://huggingface.co/datasets/kaist-ai/CoT-Collection).
## License
CoT Collection and CoT-T5 is subject to OpenAI's Terms of Use for the generated data. If you suspect any violations, please reach out to us.
# Usage
Find below some example scripts on how to use the model in `transformers`:
## Using the Pytorch model
### Running the model on a CPU
<details>
<summary> Click to expand </summary>
```python
from transformers import T5Tokenizer, T5ForConditionalGeneration
tokenizer = T5Tokenizer.from_pretrained("kaist-ai/CoT-T5-11B")
model = T5ForConditionalGeneration.from_pretrained("kaist-ai/CoT-T5-11B")
input_text = "Read the Directions and try to pick among A,B,C,D.\n\nDirecitons: A good way to figure out the relationship in a given question is to make up a sentence that describes the relationship between the first two words. Then, try to use the same sentence to find out which of the answer choices completes the same relationship with the third word.\nQuestion: Odometer is to mileage as compass is to?\nOptions: (A) speed, (B) hiking, (C) needle, (D) direction.\nLet's think step by step.\n"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids
outputs = model.generate(input_ids)
print(tokenizer.decode(outputs[0]))
```
</details>
### Running the model on a GPU
<details>
<summary> Click to expand </summary>
```python
# pip install accelerate
from transformers import T5Tokenizer, T5ForConditionalGeneration
tokenizer = T5Tokenizer.from_pretrained("kaist-ai/CoT-T5-11B")
model = T5ForConditionalGeneration.from_pretrained("kaist-ai/CoT-T5-11B", device_map="auto")
input_text = "Read the Directions and try to pick among A,B,C,D.\n\nDirecitons: A good way to figure out the relationship in a given question is to make up a sentence that describes the relationship between the first two words. Then, try to use the same sentence to find out which of the answer choices completes the same relationship with the third word.\nQuestion: Odometer is to mileage as compass is to?\nOptions: (A) speed, (B) hiking, (C) needle, (D) direction.\nLet's think step by step.\n"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")
outputs = model.generate(input_ids)
print(tokenizer.decode(outputs[0]))
```
</details>
### Running the model on a GPU using different precisions
#### FP16
<details>
<summary> Click to expand </summary>
```python
# pip install accelerate
import torch
from transformers import T5Tokenizer, T5ForConditionalGeneration
tokenizer = T5Tokenizer.from_pretrained("kaist-ai/CoT-T5-11B")
model = T5ForConditionalGeneration.from_pretrained("kaist-ai/CoT-T5-11B", device_map="auto", torch_dtype=torch.float16)
input_text = "Read the Directions and try to pick among A,B,C,D.\n\nDirecitons: A good way to figure out the relationship in a given question is to make up a sentence that describes the relationship between the first two words. Then, try to use the same sentence to find out which of the answer choices completes the same relationship with the third word.\nQuestion: Odometer is to mileage as compass is to?\nOptions: (A) speed, (B) hiking, (C) needle, (D) direction.\nLet's think step by step.\n"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")
outputs = model.generate(input_ids)
print(tokenizer.decode(outputs[0]))
```
</details>
#### INT8
<details>
<summary> Click to expand </summary>
```python
# pip install bitsandbytes accelerate
from transformers import T5Tokenizer, T5ForConditionalGeneration
tokenizer = T5Tokenizer.from_pretrained("kaist-ai/CoT-T5-11B")
model = T5ForConditionalGeneration.from_pretrained("kaist-ai/CoT-T5-11B", device_map="auto", load_in_8bit=True)
input_text = "Read the Directions and try to pick among A,B,C,D.\n\nDirecitons: A good way to figure out the relationship in a given question is to make up a sentence that describes the relationship between the first two words. Then, try to use the same sentence to find out which of the answer choices completes the same relationship with the third word.\nQuestion: Odometer is to mileage as compass is to?\nOptions: (A) speed, (B) hiking, (C) needle, (D) direction.\nLet's think step by step.\n"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")
outputs = model.generate(input_ids)
print(tokenizer.decode(outputs[0]))
```
</details>
# Citation
If you find the following model helpful, please considering citing our paper!
**BibTeX:**
```bibtex
@article{kim2023cot,
title={The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning},
author={Kim, Seungone and Joo, Se June and Kim, Doyoung and Jang, Joel and Ye, Seonghyeon and Shin, Jamin and Seo, Minjoon},
journal={arXiv preprint arXiv:2305.14045},
year={2023}
}
``` |