Update README.md
Browse files
README.md
CHANGED
@@ -116,4 +116,27 @@ As I expected, it improves GSM8K, but doesn't do much to ARC.
|
|
116 |
- Epochs: 6
|
117 |
- Learning rate: 1e-5
|
118 |
- Learning rate schedule: One Cycle, cosine, no cycle_momentum
|
119 |
-
- Regularization weight: 0.1
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
116 |
- Epochs: 6
|
117 |
- Learning rate: 1e-5
|
118 |
- Learning rate schedule: One Cycle, cosine, no cycle_momentum
|
119 |
+
- Regularization weight: 0.1
|
120 |
+
|
121 |
+
## Prompt format
|
122 |
+
|
123 |
+
The format for reddit-instruct and oasst2 was:
|
124 |
+
|
125 |
+
```
|
126 |
+
<|user|>
|
127 |
+
[insert instruction here]
|
128 |
+
<|assistant|>
|
129 |
+
[insert response here]
|
130 |
+
<|user|>
|
131 |
+
...
|
132 |
+
```
|
133 |
+
|
134 |
+
The format for TinyCoT was:
|
135 |
+
```
|
136 |
+
<|user|>
|
137 |
+
[insert instruction here]
|
138 |
+
<|rationale|>
|
139 |
+
[insert reasoning here]
|
140 |
+
<|answer|>
|
141 |
+
[insert direct answer here]
|
142 |
+
```
|