migtissera commited on
Commit
4e110f2
1 Parent(s): 8511385

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -1
README.md CHANGED
@@ -38,7 +38,10 @@ The system message *must* be the following:
38
 
39
  # Inference
40
 
41
- The model was trained mostly with Chain-of-Thought reasoning data, including the XML tags. However, to generalize model generations, some single-turn and multi-turn data without XML tags were also included. Due to this, in some instances the model does not produce XML tags and does not fully utilize test-time compute capabilities. Therefore you should include a try statement in your inference script, and only pass on the contents between the `<output>` `</output>` tags if it's available.
 
 
 
42
 
43
  I have included a sample Python script below.
44
 
 
38
 
39
  # Inference
40
 
41
+ The model was trained mostly with Chain-of-Thought reasoning data, including the XML tags. However, to generalize model generations, some single-turn and multi-turn data without XML tags were also included. Due to this, in some instances the model does not produce XML tags and does not fully utilize test-time compute capabilities. There is two ways to get around this:
42
+
43
+ - Include a try/catch statement in your inference script, and only pass on the contents between the `<output>` `</output>` tags if it's available.
44
+ - Use the `<thinking>` tag as the seed in the generation. i.e: `f"{conversation}{user_input}<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n<thinking>"`
45
 
46
  I have included a sample Python script below.
47