draft readme
Browse files
README.md
CHANGED
@@ -4,18 +4,35 @@ tags:
|
|
4 |
- axolotl
|
5 |
- generated_from_trainer
|
6 |
base_model: openchat/openchat-3.5-0106
|
7 |
-
|
8 |
-
-
|
9 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
10 |
---
|
|
|
11 |
|
12 |
-
|
13 |
-
|
|
|
|
|
|
|
|
|
|
|
14 |
|
15 |
-
[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
|
16 |
<details><summary>See axolotl config</summary>
|
17 |
|
18 |
-
axolotl version: `0.
|
19 |
```yaml
|
20 |
base_model: openchat/openchat-3.5-0106
|
21 |
model_type: MistralForCausalLM
|
@@ -117,14 +134,75 @@ special_tokens:
|
|
117 |
tokens:
|
118 |
- "<|end_of_turn|>"
|
119 |
- "<|pad_0|>"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
120 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
121 |
|
|
|
|
|
122 |
```
|
123 |
|
124 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
125 |
|
126 |
-
|
127 |
|
128 |
-
|
129 |
-
It achieves the following results on the evaluation set:
|
130 |
-
- Loss: 0.0800
|
|
|
4 |
- axolotl
|
5 |
- generated_from_trainer
|
6 |
base_model: openchat/openchat-3.5-0106
|
7 |
+
datasets:
|
8 |
+
- hendrycks/competition_math
|
9 |
+
- allenai/ai2_arc
|
10 |
+
- camel-ai/physics
|
11 |
+
- camel-ai/chemistry
|
12 |
+
- camel-ai/biology
|
13 |
+
- camel-ai/math
|
14 |
+
- STEM-AI-mtl/Electrical-engineering
|
15 |
+
- openbookqa
|
16 |
+
- piqa
|
17 |
+
- metaeval/reclor
|
18 |
+
- mandyyyyii/scibench
|
19 |
+
- derek-thomas/ScienceQA
|
20 |
+
- sciq
|
21 |
+
- TIGER-Lab/ScienceEval
|
22 |
---
|
23 |
+
(logo)
|
24 |
|
25 |
+
# 🔬👩🔬 Newton-7B
|
26 |
+
|
27 |
+
This model is a fine-tuned version of [openchat/openchat-3.5-0106](https://huggingface.co/openchat/openchat-3.5-0106) on datasets related to science.
|
28 |
+
|
29 |
+
This model is fine-tuned using [QLoRa](https://arxiv.org/abs/2305.14314) and [axolotl](https://github.com/OpenAccess-AI-Collective/axolotl).
|
30 |
+
|
31 |
+
This model's training was sponsored by [sablo.ai](https://sablo.ai).
|
32 |
|
|
|
33 |
<details><summary>See axolotl config</summary>
|
34 |
|
35 |
+
axolotl version: `0.3.0`
|
36 |
```yaml
|
37 |
base_model: openchat/openchat-3.5-0106
|
38 |
model_type: MistralForCausalLM
|
|
|
134 |
tokens:
|
135 |
- "<|end_of_turn|>"
|
136 |
- "<|pad_0|>"
|
137 |
+
```
|
138 |
+
|
139 |
+
</details><br>
|
140 |
+
|
141 |
+
# 📊 Datasets
|
142 |
+
|
143 |
+
Following datasets were used in this model:
|
144 |
+
|
145 |
+
- [MATH](https://huggingface.co/datasets/dahendrycks/competition_math)
|
146 |
+
|
147 |
+
- [ARC](https://huggingface.co/datasets/allenai/ai2_arc) (Note: Only **train** part)
|
148 |
+
|
149 |
+
- [camel-ai/physics](https://huggingface.co/datasets/camel-ai/physics)
|
150 |
+
|
151 |
+
- [camel-ai/chemistry](https://huggingface.co/datasets/camel-ai/chemistry)
|
152 |
+
|
153 |
+
- [camel-ai/biology](https://huggingface.co/datasets/camel-ai/biology)
|
154 |
+
|
155 |
+
- [camel-ai/math](https://huggingface.co/datasets/camel-ai/math)
|
156 |
+
|
157 |
+
- [STEM-AI-mtl/Electrical-engineering](https://huggingface.co/datasets/STEM-AI-mtl/Electrical-engineering)
|
158 |
+
|
159 |
+
- [openbookqa](https://huggingface.co/datasets/openbookqa)
|
160 |
+
|
161 |
+
- [piqa](https://huggingface.co/datasets/piqa)
|
162 |
|
163 |
+
- [reclor](https://huggingface.co/datasets/metaeval/reclor)
|
164 |
+
|
165 |
+
- [scibench](https://github.com/mandyyyyii/scibench)
|
166 |
+
|
167 |
+
- [ScienceQA](https://huggingface.co/datasets/derek-thomas/ScienceQA)
|
168 |
+
|
169 |
+
- [sciq](https://huggingface.co/datasets/sciq)
|
170 |
+
|
171 |
+
- [ScienceEval](https://huggingface.co/datasets/TIGER-Lab/ScienceEval)
|
172 |
+
|
173 |
+
# 💬 Prompt Template
|
174 |
+
|
175 |
+
You can use this prompt template while using the model:
|
176 |
+
|
177 |
+
### GPT4 Correct [(Openchat)](https://huggingface.co/openchat/openchat-3.5-0106#conversation-templates)
|
178 |
|
179 |
+
```
|
180 |
+
GPT4 Correct User: {user}<|end_of_turn|>GPT4 Correct Assistant: {asistant}<|end_of_turn|>GPT4 Correct User: {user}<|end_of_turn|>GPT4 Correct Assistant:
|
181 |
```
|
182 |
|
183 |
+
You can also utilize the chat template method from the tokenizer config like here:
|
184 |
+
|
185 |
+
```python
|
186 |
+
messages = [
|
187 |
+
{"role": "user", "content": "Hello"},
|
188 |
+
{"role": "assistant", "content": "Hi"},
|
189 |
+
{"role": "user", "content": "How are you today?"}
|
190 |
+
]
|
191 |
+
tokens = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
|
192 |
+
```
|
193 |
+
|
194 |
+
# 🤝 Acknowledgments
|
195 |
+
|
196 |
+
Thanks to [@jondurbin](https://hf.co/jondurbin) for reformatting codes for some datasets: [bagel/data_sources](https://github.com/jondurbin/bagel/tree/main/bagel/data_sources)
|
197 |
+
|
198 |
+
Thanks to [Together AI](https://www.together.ai) for providing everyone with free credits, which I used to generate a dataset in multiple choice to explanations format.
|
199 |
+
|
200 |
+
Thanks to all the dataset authors mentioned in the datasets section.
|
201 |
+
|
202 |
+
Thanks to [axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) for making the repository I used to make this model.
|
203 |
+
|
204 |
+
[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
|
205 |
|
206 |
+
If you would like to support me:
|
207 |
|
208 |
+
[☕ Buy Me a Coffee](https://www.buymeacoffee.com/weyaxi)
|
|
|
|