migtissera
commited on
Commit
•
4e110f2
1
Parent(s):
8511385
Update README.md
Browse files
README.md
CHANGED
@@ -38,7 +38,10 @@ The system message *must* be the following:
|
|
38 |
|
39 |
# Inference
|
40 |
|
41 |
-
The model was trained mostly with Chain-of-Thought reasoning data, including the XML tags. However, to generalize model generations, some single-turn and multi-turn data without XML tags were also included. Due to this, in some instances the model does not produce XML tags and does not fully utilize test-time compute capabilities.
|
|
|
|
|
|
|
42 |
|
43 |
I have included a sample Python script below.
|
44 |
|
|
|
38 |
|
39 |
# Inference
|
40 |
|
41 |
+
The model was trained mostly with Chain-of-Thought reasoning data, including the XML tags. However, to generalize model generations, some single-turn and multi-turn data without XML tags were also included. Due to this, in some instances the model does not produce XML tags and does not fully utilize test-time compute capabilities. There is two ways to get around this:
|
42 |
+
|
43 |
+
- Include a try/catch statement in your inference script, and only pass on the contents between the `<output>` `</output>` tags if it's available.
|
44 |
+
- Use the `<thinking>` tag as the seed in the generation. i.e: `f"{conversation}{user_input}<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n<thinking>"`
|
45 |
|
46 |
I have included a sample Python script below.
|
47 |
|