Update README.md

README.md

@@ -4,41 +4,43 @@ language:
 - en
 tags:
 - merge
-- not-for-all-audiences
-- nsfw
 ---
 
 <div style="display: flex; justify-content: center; align-items: center">
-  <img src="https://huggingface.co/SanjiWatsuki/Silicon-Maid-7B/resolve/main/assets/cybermaid.png">
+  <img src="https://huggingface.co/SanjiWatsuki/Sonya-7B/resolve/main/assets/Sonya.jpg">
 </div
 >
 
 <p align="center">
-  <big><b>Top 1 RP Performer on MT-bench 🤪</b
-></big>
-</p>
-
-<p align="center">
-  <strong>Next Gen Silicon-Based RP Maid</strong>
+  <big><b>Top 1 Performer MT-bench 🤪</b></big>
 </p>
 
 ## WTF is This?
 
-Silicon-Maid-7B is another model targeted at being both strong at RP **and** being a smart cookie that can follow character cards very well. As of right now, Silicon-Maid-7B outscores both of my previous 7B RP models in my RP benchmark and I have been impressed by this model's creativity. It is suitable for RP/ERP and general use.
+Sonya-7B is, at the time of writing, the **#1 performing model in MT-Bench first turn**, ahead of GPT-4, and overall the #2 model in MT-Bench, to the best of my knowledge. Sonya-7B should be a good all-purpose model for all tasks  including assistant, RP, etc.
+
+Sonya-7B has a similar structure to my previous model, Silicon-Maid-7B, and uses a very similar structure. It's a merge of [xDAN-AI/xDAN-L1-Chat-RL-v1](https://huggingface.co/xDAN-AI/xDAN-L1-Chat-RL-v1), [Jan-Ai's Stealth v1.2](https://huggingface.co/jan-hq/stealth-v1.2), [chargoddard/piano-medley-7b](https://huggingface.co/chargoddard/piano-medley-7b), [NeverSleep/Noromaid-7B-v0.2](https://huggingface.co/NeverSleep/Noromaid-7b-v0.2), and [athirdpath/NSFW_DPO_vmgb-7b](athirdpath/NSFW_DPO_vmgb-7b). Sauce is below. Somehow, by combining these pieces, it substantially outscores any of its parents on MT-Bench.
+
+I picked these models because:
+* MT-Bench normally correlates well with real world model quality and xDAN performs well on it.
+* Almost all models in the mix were Alpaca prompt formatted which gives prompt consistency.
+* Stealth v1.2 has been a magic sprinkle that seems to increase my MT-Bench scores.
+* I added RP models because it boosted the Writing and Roleplay benchmarks 👀
+
+Based on the parent models, I expect this model to be used with an 8192 context window. Please use NTK scaling alpha of 2.6 to experimentally try out 16383 context.
 
-It's built on [xDAN-AI/xDAN-L1-Chat-RL-v1](https://huggingface.co/xDAN-AI/xDAN-L1-Chat-RL-v1), a 7B model which scores unusually high on MT-Bench, and chargoddard/loyal-piano-m7, an Alpaca format 7B model with surprisingly creative outputs. I was excited to see this model for two main reasons:
-* MT-Bench normally correlates well with real world model quality
-* It was an Alpaca prompt model with high benches which meant I could try swapping out my Marcoroni frankenmerge used in my previous model.
+**Let me be candid:** Despite the test scores, I do not believe this model is a GPT killer. I think it's a very sharp model, it probably punches way above its weight, but it's still a 7B model. Keep your expectations in check 😉
 
 **MT-Bench Average Turn**
 | model              | score     | size
 |--------------------|-----------|--------
 | gpt-4              | 8.99      |  -
-| *xDAN-L1-Chat-RL-v1* | 8.24^1      |  7b
+| **Sonya-7B**         | **8.52**      |  **7b**
+| xDAN-L1-Chat-RL-v1 | 8.34      |  7b
 | Starling-7B        | 8.09      |  7b
 | Claude-2           | 8.06      |  -
-| **Silicon-Maid**   | **7.96**  |  **7b**
-| *Loyal-Macaroni-Maid*| 7.95      |  7b
+| *Silicon-Maid*   | *7.96*  |  *7b*
+| *Loyal-Macaroni-Maid*| *7.95*      |  *7b*
 | gpt-3.5-turbo      | 7.94      |  20b?
 | Claude-1           | 7.90      |  -
 | OpenChat-3.5       | 7.81      |  -
@@ -46,55 +48,40 @@ It's built on [xDAN-AI/xDAN-L1-Chat-RL-v1](https://huggingface.co/xDAN-AI/xDAN-L
 | wizardlm-30b       | 7.01      |  30b
 | Llama-2-70b-chat   | 6.86      |  70b
 
-^1 xDAN's testing placed it 8.35 - this number is from my independent MT-Bench run.
+<img src="https://huggingface.co/SanjiWatsuki/Sonya-7B/resolve/main/assets/mt-bench-gpt.png">
 
-<img src="https://huggingface.co/SanjiWatsuki/Silicon-Maid-7B/resolve/main/assets/fig-silicon-loyal.png">
-
-It's unclear to me if xDAN-L1-Chat-RL-v1 is overtly benchmaxxing but it seemed like a solid 7B from my limited testing (although nothing that screams 2nd best model behind GPT-4). Amusingly, the model lost a lot of Reasoning and Coding skills in the merger. This was a much greater MT-Bench dropoff than I expected, perhaps suggesting the Math/Reasoning ability in the original model was rather dense and susceptible to being lost to a DARE TIE merger?
-
-Besides that, the merger is almost identical to the Loyal-Macaroni-Maid merger with a new base "smart cookie" model. If you liked any of my previous RP models, give this one a shot and let me know in the Community tab what you think!
+<img src="https://huggingface.co/SanjiWatsuki/Sonya-7B/resolve/main/assets/mt-bench-comparison.png">
 
 ### The Sauce
 
 ```
-models: # Top-Loyal-Bruins-Maid-DARE-7B
-  - model: mistralai/Mistral-7B-v0.1
-    # no parameters necessary for base model
+models:
   - model: xDAN-AI/xDAN-L1-Chat-RL-v1
     parameters:
-      weight: 0.4
-      density: 0.8
-  - model: chargoddard/loyal-piano-m7
+      weight: 1
+      density: 1
+  - model: chargoddard/piano-medley-7b
     parameters:
       weight: 0.3
-      density: 0.8
-  - model: Undi95/Toppy-M-7B
+  - model: jan-hq/stealth-v1.2
     parameters:
       weight: 0.2
-      density: 0.4
   - model: NeverSleep/Noromaid-7b-v0.2
     parameters:
       weight: 0.2
-      density: 0.4
   - model: athirdpath/NSFW_DPO_vmgb-7b
     parameters:
       weight: 0.2
-      density: 0.4
-merge_method: dare_ties
+merge_method: ties
 base_model: mistralai/Mistral-7B-v0.1
 parameters:
+  density: 0.4
   int8_mask: true
+  normalize: true
 dtype: bfloat16
 ```
 
-For more information about why I use this merger, see the [Loyal-Macaroni-Maid repo](https://huggingface.co/SanjiWatsuki/Loyal-Macaroni-Maid-7B#the-sauce-all-you-need-is-dare)
-
 ### Prompt Template (Alpaca)
-I found the best SillyTavern results from using the Noromaid template but please try other templates! Let me know if you find anything good.
-
-SillyTavern config files: [Context](https://files.catbox.moe/ifmhai.json), [Instruct](https://files.catbox.moe/ttw1l9.json).
-
-Additionally, here is my highly recommended [Text Completion preset](https://huggingface.co/SanjiWatsuki/Loyal-Macaroni-Maid-7B/blob/main/Characters/MinP.json). You can tweak this by adjusting temperature up or dropping min p to boost creativity or raise min p to increase stability. You shouldn't need to touch anything else!
 
 ```
 Below is an instruction that describes a task. Write a response that appropriately completes the request.
@@ -105,4 +92,34 @@ Below is an instruction that describes a task. Write a response that appropriate
 ### Response:
 ```
 
-
+I found that this model **performed worse** with the xDAN prompt format so, despite the heavy weight of xDAN in this merger, I recommeend *against* its use.
+
+### Other Benchmark Stuff
+
+**########## First turn ##########**
+| model              | turn | score    | size
+|--------------------|------|----------|--------
+| **Sonya-7B** | 1    | **9.06875**  |  **7b**
+| gpt-4              | 1    | 8.95625  |  -
+| xDAN-L1-Chat-RL-v1 | 1    | *8.87500*  |  *7b*
+| xDAN-L2-Chat-RL-v2 | 1    | 8.78750  |  30b
+| claude-v1          | 1    | 8.15000  |  -
+| gpt-3.5-turbo      | 1    | 8.07500  |  20b
+| vicuna-33b-v1.3    | 1    | 7.45625  |  33b
+| wizardlm-30b       | 1    | 7.13125  |  30b
+| oasst-sft-7-llama-30b | 1 | 7.10625  |  30b
+| Llama-2-70b-chat   | 1    | 6.98750  |  70b
+
+
+########## Second turn ##########
+| model              | turn | score     | size
+|--------------------|------|-----------|--------
+| gpt-4              | 2    | 9.025000  |  -
+| xDAN-L2-Chat-RL-v2 | 2    | 8.087500  |  30b
+| **Sonya-7B**       | 2    | **7.962500**  |  **7b**
+| xDAN-L1-Chat-RL-v1 | 2   | 7.825000  |   7b
+| gpt-3.5-turbo      | 2    | 7.812500  |  20b
+| claude-v1          | 2    | 7.650000  |  -
+| wizardlm-30b       | 2    | 6.887500  |  30b
+| vicuna-33b-v1.3    | 2    | 6.787500  |  33b
+| Llama-2-70b-chat   | 2    | 6.725000  |  70b