athirdpath commited on
Commit
8db6409
1 Parent(s): da937ee

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -1
README.md CHANGED
@@ -9,4 +9,6 @@ This model is a fine-tuned version of microsoft/Orca-2-13b on a subset of the Ve
9
 
10
  Only the q_proj and k_proj modules were targeted and a low rank (8) was used, in hopes of containing the adjustments to the prompt format and alignment. This is promising on paper, with the training's per-step loss averaging <0.9 for the last third of the run.
11
 
12
- I'll test it tomorrow.
 
 
 
9
 
10
  Only the q_proj and k_proj modules were targeted and a low rank (8) was used, in hopes of containing the adjustments to the prompt format and alignment. This is promising on paper, with the training's per-step loss averaging <0.9 for the last third of the run.
11
 
12
+ Reasoning stayed solid (for a 13b model) and I consider this a success. Performance is slighty worse than OG Orca-2 in Ooba's chat mode, comparable in Alpaca chat-instruct mode to the OG in ChatLM chat-instruct mode.
13
+
14
+ May still reject some shocking prompts, but can easily be overcome with author's note or character card.