Upload folder using huggingface_hub

.gitattributes

@@ -1,35 +1,9 @@
-*.7z filter=lfs diff=lfs merge=lfs -text
-*.arrow filter=lfs diff=lfs merge=lfs -text
-*.bin filter=lfs diff=lfs merge=lfs -text
-*.bz2 filter=lfs diff=lfs merge=lfs -text
-*.ckpt filter=lfs diff=lfs merge=lfs -text
-*.ftz filter=lfs diff=lfs merge=lfs -text
-*.gz filter=lfs diff=lfs merge=lfs -text
-*.h5 filter=lfs diff=lfs merge=lfs -text
-*.joblib filter=lfs diff=lfs merge=lfs -text
-*.lfs.* filter=lfs diff=lfs merge=lfs -text
-*.mlmodel filter=lfs diff=lfs merge=lfs -text
-*.model filter=lfs diff=lfs merge=lfs -text
-*.msgpack filter=lfs diff=lfs merge=lfs -text
-*.npy filter=lfs diff=lfs merge=lfs -text
-*.npz filter=lfs diff=lfs merge=lfs -text
-*.onnx filter=lfs diff=lfs merge=lfs -text
-*.ot filter=lfs diff=lfs merge=lfs -text
-*.parquet filter=lfs diff=lfs merge=lfs -text
-*.pb filter=lfs diff=lfs merge=lfs -text
-*.pickle filter=lfs diff=lfs merge=lfs -text
-*.pkl filter=lfs diff=lfs merge=lfs -text
-*.pt filter=lfs diff=lfs merge=lfs -text
-*.pth filter=lfs diff=lfs merge=lfs -text
-*.rar filter=lfs diff=lfs merge=lfs -text
-*.safetensors filter=lfs diff=lfs merge=lfs -text
-saved_model/**/* filter=lfs diff=lfs merge=lfs -text
-*.tar.* filter=lfs diff=lfs merge=lfs -text
-*.tar filter=lfs diff=lfs merge=lfs -text
-*.tflite filter=lfs diff=lfs merge=lfs -text
-*.tgz filter=lfs diff=lfs merge=lfs -text
-*.wasm filter=lfs diff=lfs merge=lfs -text
-*.xz filter=lfs diff=lfs merge=lfs -text
-*.zip filter=lfs diff=lfs merge=lfs -text
-*.zst filter=lfs diff=lfs merge=lfs -text
-*tfevents* filter=lfs diff=lfs merge=lfs -text
+Kyllene-34B-v1.1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+Kyllene-34B-v1.1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Kyllene-34B-v1.1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Kyllene-34B-v1.1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Kyllene-34B-v1.1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Kyllene-34B-v1.1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Kyllene-34B-v1.1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Kyllene-34B-v1.1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+Kyllene-34B-v1.1-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

Kyllene-34B-v1.1-Q3_K_L.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cecf60834bba44593657997b725b174540f557c25be3f296f1c2de7c0088f1e9
+size 18139445792

Kyllene-34B-v1.1-Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0e7000187d8d696b96bd0f4a6fb253e24bda88adf2e67865079cecc4cb7bf629
+size 16654924320

Kyllene-34B-v1.1-Q3_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5c69348d0cd8ae73ff1d27e80e227ca9e4a45afbc3034af660a50e6264366c29
+size 14960294432

Kyllene-34B-v1.1-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:649c7dd669a16c2f96c75b865fb2a842b0dc84122de15c7c2bc477776473b00d
+size 20658711072

Kyllene-34B-v1.1-Q4_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:29e6331ccc2d48785a7b1d6ce2bce99af35595a57cdbff9dbf7802d7fd002615
+size 19598649888

Kyllene-34B-v1.1-Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:82492e3091ba2351ca26b15866d3f905faf6f76de402852a06b8b93758dd2135
+size 24321845792

Kyllene-34B-v1.1-Q5_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8f780ac8f109898ed7d432b7eb714843a496708cc6ef03ba1be2ba365a309e94
+size 23707691552

Kyllene-34B-v1.1-Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:56440f7e417599ab4deda26cb10d352c5b3131b0b62be88ed2ab141f209ce735
+size 28213926432

Kyllene-34B-v1.1-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f809b413b78e94a4b359e0cdb875728b123021ef4170596d9adf014a8f3d87bb
+size 36542282272

README.md

@@ -0,0 +1,42 @@
+---
+license: other
+license_name: yi-license
+license_link: https://huggingface.co/01-ai/Yi-34B-200K/blob/main/LICENSE
+tags:
+- merge
+---
+# Kyllene 34B v1.1
+
+![image/png](https://huggingface.co/TeeZee/Kyllene-34B-v1.1/resolve/main/Kyllene_v1.1.jpg)
+
+
+## Model Details
+
+- A result of new merge method provided by [MergeMonster](https://github.com/Gryphe/MergeMonster/) tool with extended RPG preset.
+- models used for merge:
+  [jondurbin/bagel-dpo-34b-v0.2](https://huggingface.co/jondurbin/bagel-dpo-34b-v0.2)
+  [NousResearch/Nous-Capybara-34B](https://huggingface.co/NousResearch/Nous-Capybara-34B)
+  [NousResearch_Nous-Hermes-2-Yi-34B](https://huggingface.co/NousResearch/Nous-Hermes-2-Yi-34B)
+  [SUSTech/SUS-Chat-34B](https://huggingface.co/SUSTech/SUS-Chat-34B)
+- Method is aimed to maximize probability of certain phrases and minimize probablility of other phrases.
+- RPG preset was extened with examples of typical, nonsensical output of most models like 'unbreakable bond', 'send shivers down her spine' etc.
+- The resulting model has approximately 34 billion parameters.
+- See [mergekit-config.yml](https://huggingface.co/TeeZee/Kyllene-34B-v1.1/resolve/main/merge-config.yml) for details on the merge method used and RPG presets.
+
+**Warning: This model can produce NSFW content!**
+
+## Results
+
+- produces SFW nad NSFW content without issues, switches context seamlessly.
+- 200K context length
+- good at following instructions
+- different than [TeeZee/Kyllene-57B-v1.0](https://huggingface.co/TeeZee/Kyllene-57B-v1.0), but also surprisingly entertaining (but more tests are needed)
+
+## Side notes
+
+ - [MergeMonster](https://github.com/Gryphe/MergeMonster/) method works, however project would benefit greatly from some more love from developers.
+ - In its current state MergeMonster consumes insane amounts of RAM (256GB+) or VRAM and takes a really long time to process model data, this merge took 24H on 1xADA6000
+ - MergeMonster is not a golden bullet, other experiments has shown that it can also produce incredibly stupid models.
+
+All comments are greatly appreciated, download, test and if you appreciate my work, consider buying me my fuel:
+<a href="https://www.buymeacoffee.com/TeeZee" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" style="height: 60px !important;width: 217px !important;" ></a>

merge-config.yml

@@ -0,0 +1,142 @@
+
+# Either "cpu" or "cuda"
+# NOTE: Cuda requires enough VRAM to load 3 FP16 models (~45 GB for Mistral)
+# NOTE 2: The (much slower) CPU mode still requires Cuda capability, but only enough VRAM to load a model once. (~15 GB for Mistral)
+device: "cpu"
+random_seed: 42 # Random seed to use
+
+directories:
+  model_path1: "../jondurbin_bagel-dpo-34b-v0.2" # Path to the base model. Must be a local copy.
+#  model_directory: "../2xNous-Capybara-34B/" # Directory of models to scan, IGNORED if models_to_merge has entries in it
+  output_directory: "./mm-output" # Output directory of the merged model
+
+# A list of models to use as merge candidates - HF syntax, so can be either local directories or repos.
+# Overrides model_directory if used
+models_to_merge: ["../NousResearch_Nous-Capybara-34B", "../NousResearch_Nous-Hermes-2-Yi-34B", "../SUSTech_SUS-Chat-34B"]
+
+# Merge ratios used for testing each layer's potential for improvement - Huge impact on total running time
+merge_ratios: [0.2, 0.4, 0.6, 0.8]
+
+# Choose from the following methods. Defaults to "lerp".
+# "lerp" - Linear interpolation
+# "slerp" - Spherical linear interpolation
+# "slice" - Highly experimental. The tensor weights shifts from one model to another. [Model 1 > 10% blend > Model 2]
+# "cyclic" - Highly experimental. Ignores merge ratios as these are predefined. [Model 1 > 10% blend > 10% Model 2 > 10% blend > Model 1]
+merge_method: "slerp"
+
+# If set to true, the lm_head and embed_token tensors (located outside the layers) will also be optimized
+# Models that have a different vocab size from model1 will skip this phase automatically as it tends to cause model stability issues
+merge_headers: true
+
+# Strategies:
+# "cumulative" - Default strategy. If there's a chance of reducing the combined probability, accept the merge.
+# "all_phrases" - Only accept the merge if all phrases show an improvement. (Warning: This rarely happens)
+# "quantitive" - Ignores probabilities completely. Only looks at how many phrases show an improvement, as defined by the threshold below.
+strategy: "cumulative"
+# Threshold is currently only used by the "quantitive" strategy. If 0.6, at least 60% of the number of phrases must show am improvement.
+strategy_threshold: 0.6
+
+# Whether or not to automatically balance the weights so all phrases are of equal importance to the "cumulative" strategy.
+# The weight value of phrases is ignored if set to true.
+auto_weights: false
+
+# Phrase = What to measure, weight = multiplication factor, contexts = proceeding contexts
+bad_phrases:
+  - phrase: "anticipation"
+    weight: 12
+    contexts: ["Her body quivers with ", "The atmosphere is thick with "]
+  - phrase: "unwavering"
+    weight: 12
+    contexts: ["Filled with an "]
+  - phrase: "determination"
+    weight: 12
+    contexts: ["Her eyes were filled with ", "Her stubbornness only fuels my "]
+  - phrase: "whisper"
+    weight: 12
+    contexts: ["Her voice barely above a "]
+  - phrase: "spine"
+    weight: 12
+    contexts: ["shivers down her "]
+  - phrase: "sends shivers"
+    weight: 12
+    contexts: ["The thrill of the act "]
+  - phrase: "ministrations"
+    weight: 12
+    contexts: ["She moans and twitches at your "]
+  - phrase: "legs"
+    weight: 12
+    contexts: ["wraps her "]
+  - phrase: "imposing figure"
+    weight: 12
+    contexts: ["He had an "]
+  - phrase: "shared challenges"
+    weight: 12
+    contexts: ["Their bond strengthened through "]    
+  - phrase: "bond"
+    weight: 12
+    contexts: ["forged a ", "an unspoken "]
+  - phrase: "enhance our experience"
+    weight: 12
+    contexts: ["I'm excited to see how "]
+  - phrase: "sense of vulnerability"
+    weight: 12
+    contexts: ["create a "]
+  - phrase: "dimensions of intimacy"
+    weight: 12
+    contexts: ["explore new "]
+  - phrase: "deepening our connection"
+    weight: 12
+    contexts: ["while "]
+  - phrase: "shared experiences"
+    weight: 12
+    contexts: ["through "]
+  - phrase: "societal expectations"
+    weight: 12
+    contexts: ["that transcend "]
+  - phrase: "conventional boundaries"
+    weight: 12
+    contexts: ["that defy ", "and defy "]
+  - phrase: "open communication"
+    weight: 12
+    contexts: ["an environment "]
+  - phrase: "emotional vulnerability"
+    weight: 12
+    contexts: ["an environment "]
+  - phrase: "heightens our connection"
+    weight: 12
+    contexts: ["touch and the anticipation "]
+  - phrase: "sensations you're creating"
+    weight: 12
+    contexts: ["I'm enjoying "]
+  - phrase: "is truly arousing"
+    weight: 12
+    contexts: ["attention to detail ", "way you explore my body "]
+  - phrase: "challenge presented"
+    weight: 12
+    contexts: ["my resolve unwavering despite "]
+  - phrase: "humble vessel"
+    weight: 12
+    contexts: ["surrendering to the exquisite torment "]
+  - phrase: "bond"
+    weight: 12
+    contexts: ["cherishing the unique ", "special "]    
+  - phrase: "grows stronger with each passing day"
+    weight: 12
+    contexts: ["bond "]    
+  - phrase: "that cannot be broken by time or circumstance"
+    weight: 12
+    contexts: ["bond "]    
+  - phrase: "becomes unbreakable, eternal"
+    weight: 12
+    contexts: ["bond "]    
+  - phrase: "grew stronger with each passing"
+    weight: 12
+    contexts: ["bond "]    
+    
+
+# Note - Example of a complex phrase
+good_phrases:
+  - phrase: "The apple is in the bedroom"
+    weight: 1
+    contexts: ["Question: If I'm in the living room and pick up the apple, go to the bedroom and drop the apple, then walk to the kitchen, where is the apple? Explain your reasoning. Answer: "]
+

mergemonster_kyllene_v11.txt

@@ -0,0 +1,987 @@
+
+⠀⠀⠀⠀⠀⠀⣀⡀⠀⠀⣀⣤⣶⣾⣿⣿⣷⣶⣤⣀⠀⠀⣀⣀⠀⠀⠀⠀⠀⠀
+⠀⠀⠀⠀⠀⠜⠉⣿⡆⣼⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣧⢰⣿⠉⠃⠀⠀⠀⠀⠀
+⠀⢀⣤⣴⣦⣄⣴⠟⣸⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡎⢻⣦⣠⣴⣦⣄⠀⠀
+⠀⡞⠁⣠⣾⢿⣧⠀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀⣽⡿⣷⣄⠈⢷⠀
+⠀⣠⣾⠟⠁⢸⣿⠀⠘⢿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡿⠁⠀⣿⡇⠈⠻⣷⣄⠀
+⣰⡿⠁⠀⢀⣾⣏⣾⣄⣰⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣇⣰⣷⣹⣷⠀⠀⠈⢿⣆
+⣿⡇⠀⢠⣾⠏⢸⣿⣿⣿⣿⠋⢻⣿⣿⣿⣿⡟⠙⣿⣿⣿⣿⡇⠹⣷⡀⠀⢸⣿
+⠹⣿⣴⡿⠋⠀⠈⠛⠉⣹⣿⣦⣄⡹⣿⣿⣋⣠⣶⣿⣏⠉⠛⠁⠀⠙⢿⣦⣿⠏
+⠀⣸⣿⠿⠿⣿⣾⣿⡿⠿⣿⣿⣿⣿⡆⢰⣿⣿⣿⣿⠿⢿⣿⣶⣿⠿⠿⣻⣇⠀
+⠀⣿⡇⢀⣴⣶⣤⣀⣴⣿⠿⣻⡿⣿⣧⣾⣿⢿⣟⠿⣿⣦⣀⣤⣶⣦⠀⢸⣿⠀
+⠀⢿⣧⠈⠃⢀⣵⣿⡋⠁⢀⣿⡷⣿⡇⢻⣿⣿⣿⡀⠈⢛⣿⣮⡀⠘⠀⣼⡟⠀
+⠀⠈⠻⣷⣤⣟⣋⣿⣧⣴⡿⠋⠀⣿⡇⢸⣿⠀⠙⢿⣦⣼⣿⣙⣻⣤⣾⠟⠁⠀
+⠀⠀⠀⠈⢽⣿⠛⢻⣏⢉⣤⣶⣶⣿⠁⠈⣿⣶⣶⣤⡉⣽⡟⠛⣿⡏⠁⠀⠀⠀
+⠀⠀⠀⠀⠈⠿⣷⣾⣾⣟⣉⣠⣿⢿⡇⢸⠿⣿⣄⣙⣻⣷⣷⣾⠿⠁⠀⠀⠀⠀
+⠀⠀⠀⠀⠀⠀⠀⠀⠙⠻⠿⠛⢁⡼⠃⠘⢦⡈⠛⠿⠟⠃⠀⠀⠀⠀⠀⠀⠀⠀
+
+01:05:33 - THE MERGE MONSTER HUNGERS
+------------------------------------
+Device           : cpu
+Random seed      : 42
+Starting model   : ../jondurbin_bagel-dpo-34b-v0.2
+Models to merge  : ['../NousResearch_Nous-Capybara-34B', '../NousResearch_Nous-Hermes-2-Yi-34B', '../SUSTech_SUS-Chat-34B']
+Output directory : ./mm-output
+Phrases loaded   : 31
+Auto weights     : False
+Merge ratios     : [0.2, 0.4, 0.6, 0.8]
+Merge method(s)  : ['slerp']
+Merge headers    : True
+Strategy used    : cumulative
+------------------------------------
+01:05:34 - Loading model (../jondurbin_bagel-dpo-34b-v0.2)...
+Loading checkpoint shards: 100%|████████████████| 15/15 [00:32<00:00,  2.18s/it]
+01:06:59 -  Model loaded. Dtype: torch.float16
+------------------------------------
+
+-----------------------------------------------------------------------------------------------------
+| Type | Phrase             | Context                  | Raw Prob*    | Used Prob**  | Change       |
+-----------------------------------------------------------------------------------------------------
+| BAD  | anticipation       | Her body quivers with    | 0.00000%     | 0.00%        | N/A          |
+| BAD  | anticipation       | The atmosphere is thic.. | 0.00000%     | 0.00%        | N/A          |
+| BAD  | unwavering         | Filled with an           | 0.00000%     | 0.00%        | N/A          |
+| BAD  | determination      | Her eyes were filled w.. | 0.00000%     | 0.00%        | N/A          |
+| BAD  | determination      | Her stubbornness only .. | 0.00000%     | 0.00%        | N/A          |
+| BAD  | whisper            | Her voice barely above.. | 0.00000%     | 0.00%        | N/A          |
+| BAD  | spine              | shivers down her         | 0.00000%     | 0.00%        | N/A          |
+| BAD  | sends shivers      | The thrill of the act    | 0.00000%     | 0.00%        | N/A          |
+| BAD  | ministrations      | She moans and twitches.. | 0.00006%     | 0.00%        | N/A          |
+| BAD  | legs               | wraps her                | 0.00000%     | 0.00%        | N/A          |
+| BAD  | imposing figure    | He had an                | 0.00000%     | 0.00%        | N/A          |
+| BAD  | shared challenges  | Their bond strengthene.. | 0.00001%     | 0.00%        | N/A          |
+| BAD  | bond               | forged a                 | 0.00008%     | 0.00%        | N/A          |
+| BAD  | bond               | an unspoken              | 0.00009%     | 0.00%        | N/A          |
+| BAD  | enhance our expe.. | I'm excited to see how   | 0.00000%     | 0.00%        | N/A          |
+| BAD  | sense of vulnera.. | create a                 | 0.00000%     | 0.00%        | N/A          |
+| BAD  | dimensions of in.. | explore new              | 0.00000%     | 0.00%        | N/A          |
+| BAD  | deepening our co.. | while                    | 0.00000%     | 0.00%        | N/A          |
+| BAD  | shared experiences | through                  | 0.00000%     | 0.00%        | N/A          |
+| BAD  | societal expecta.. | that transcend           | 0.00000%     | 0.00%        | N/A          |
+| BAD  | conventional bou.. | that defy                | 0.00000%     | 0.00%        | N/A          |
+| BAD  | conventional bou.. | and defy                 | 0.00000%     | 0.00%        | N/A          |
+| BAD  | open communication | an environment           | 0.00000%     | 0.00%        | N/A          |
+| BAD  | emotional vulner.. | an environment           | 0.00000%     | 0.00%        | N/A          |
+| BAD  | heightens our co.. | touch and the anticipa.. | 0.00000%     | 0.00%        | N/A          |
+| BAD  | sensations you'r.. | I'm enjoying             | 0.00000%     | 0.00%        | N/A          |
+| BAD  | is truly arousing  | attention to detail      | 0.00000%     | 0.00%        | N/A          |
+| BAD  | is truly arousing  | way you explore my body  | 0.00000%     | 0.00%        | N/A          |
+| BAD  | challenge presen.. | my resolve unwavering .. | 0.00000%     | 0.00%        | N/A          |
+| BAD  | humble vessel      | surrendering to the ex.. | 0.00000%     | 0.00%        | N/A          |
+| BAD  | bond               | cherishing the unique    | 0.00013%     | 0.00%        | N/A          |
+| BAD  | bond               | special                  | 0.00030%     | 0.00%        | N/A          |
+| BAD  | grows stronger w.. | bond                     | 0.00000%     | 0.00%        | N/A          |
+| BAD  | that cannot be b.. | bond                     | 0.00000%     | 0.00%        | N/A          |
+| BAD  | becomes unbreaka.. | bond                     | 0.00000%     | 0.00%        | N/A          |
+| BAD  | grew stronger wi.. | bond                     | 0.00000%     | 0.00%        | N/A          |
+| GOOD | The apple is in .. | Question: If I'm in th.. | 0.00139%     | 0.00%        | N/A          |
+------------------------------------------------------------------------------------------------------
+| Totals                                               | 0.00%        | 0.01%        | 0.00%        |
+------------------------------------------------------------------------------------------------------
+* = Unweighted, raw probability - ** = Probability after weight adjustments
+
+------------------------------------
+01:07:39 - Loading model (../NousResearch_Nous-Capybara-34B)...
+Loading checkpoint shards: 100%|██████████████████| 7/7 [01:04<00:00,  9.19s/it]
+01:09:33 -  Model loaded. Dtype: torch.float16
+------------------------------------
+Optimizing Layer 1/60 (slerp): 100%|██████████████| 4/4 [04:01<00:00, 60.38s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.2, 'NousResearch_Nous-Capybara-34B']]
+01:15:02 - Layer 1/60 - CHANGED - 0.00007 > 0.00006 - 2.5%
+----
+Optimizing Layer 2/60 (slerp): 100%|██████████████| 4/4 [03:52<00:00, 58.04s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.4, 'NousResearch_Nous-Capybara-34B']]
+01:20:22 - Layer 2/60 - CHANGED - 0.00006 > 0.00006 - 1.6%
+----
+Optimizing Layer 3/60 (slerp): 100%|██████████████| 4/4 [04:03<00:00, 60.90s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+01:25:50 - Layer 3/60 - RETAINED - 0.00006
+----
+Optimizing Layer 4/60 (slerp): 100%|██████████████| 4/4 [05:28<00:00, 82.25s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+01:32:54 - Layer 4/60 - RETAINED - 0.00006
+----
+Optimizing Layer 5/60 (slerp): 100%|██████████████| 4/4 [04:15<00:00, 63.94s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+01:38:53 - Layer 5/60 - RETAINED - 0.00006
+----
+Optimizing Layer 6/60 (slerp): 100%|██████████████| 4/4 [04:16<00:00, 64.24s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+01:44:47 - Layer 6/60 - RETAINED - 0.00006
+----
+Optimizing Layer 7/60 (slerp): 100%|██████████████| 4/4 [04:04<00:00, 61.02s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+01:50:20 - Layer 7/60 - RETAINED - 0.00006
+----
+Optimizing Layer 8/60 (slerp): 100%|██████████████| 4/4 [04:07<00:00, 61.95s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+01:55:59 - Layer 8/60 - RETAINED - 0.00006
+----
+Optimizing Layer 9/60 (slerp): 100%|██████████████| 4/4 [04:04<00:00, 61.17s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.4, 'NousResearch_Nous-Capybara-34B']]
+02:01:26 - Layer 9/60 - CHANGED - 0.00006 > 0.00006 - 1.3%
+----
+Optimizing Layer 10/60 (slerp): 100%|█████████████| 4/4 [03:56<00:00, 59.05s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+02:06:41 - Layer 10/60 - RETAINED - 0.00006
+----
+Optimizing Layer 11/60 (slerp): 100%|█████████████| 4/4 [03:43<00:00, 55.90s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B']]
+02:11:45 - Layer 11/60 - CHANGED - 0.00006 > 0.00006 - 4.8%
+----
+Optimizing Layer 12/60 (slerp): 100%|█████████████| 4/4 [03:53<00:00, 58.32s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B']]
+02:16:54 - Layer 12/60 - CHANGED - 0.00006 > 0.00005 - 12.2%
+----
+Optimizing Layer 13/60 (slerp): 100%|█████████████| 4/4 [04:09<00:00, 62.31s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.4, 'NousResearch_Nous-Capybara-34B']]
+02:22:31 - Layer 13/60 - CHANGED - 0.00005 > 0.00005 - 3.6%
+----
+Optimizing Layer 14/60 (slerp): 100%|█████████████| 4/4 [03:31<00:00, 52.84s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.4, 'NousResearch_Nous-Capybara-34B']]
+02:27:20 - Layer 14/60 - CHANGED - 0.00005 > 0.00005 - 1.5%
+----
+Optimizing Layer 15/60 (slerp): 100%|█████████████| 4/4 [04:26<00:00, 66.67s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+02:33:32 - Layer 15/60 - RETAINED - 0.00005
+----
+Optimizing Layer 16/60 (slerp): 100%|█████████████| 4/4 [04:36<00:00, 69.09s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+02:39:38 - Layer 16/60 - RETAINED - 0.00005
+----
+Optimizing Layer 17/60 (slerp): 100%|█████████████| 4/4 [04:22<00:00, 65.64s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+02:45:41 - Layer 17/60 - RETAINED - 0.00005
+----
+Optimizing Layer 18/60 (slerp): 100%|█████████████| 4/4 [04:39<00:00, 69.87s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+02:51:51 - Layer 18/60 - RETAINED - 0.00005
+----
+Optimizing Layer 19/60 (slerp): 100%|█████████████| 4/4 [05:10<00:00, 77.56s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+02:58:36 - Layer 19/60 - RETAINED - 0.00005
+----
+Optimizing Layer 20/60 (slerp): 100%|█████████████| 4/4 [05:03<00:00, 75.87s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B']]
+03:05:16 - Layer 20/60 - CHANGED - 0.00005 > 0.00005 - 0.2%
+----
+Optimizing Layer 21/60 (slerp): 100%|█████████████| 4/4 [05:42<00:00, 85.60s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B']]
+03:12:46 - Layer 21/60 - CHANGED - 0.00005 > 0.00001 - 77.3%
+----
+Optimizing Layer 22/60 (slerp): 100%|█████████████| 4/4 [05:48<00:00, 87.20s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B']]
+03:21:02 - Layer 22/60 - CHANGED - 0.00001 > -0.00000 - 126.4%
+----
+Optimizing Layer 23/60 (slerp): 100%|████████████| 4/4 [07:03<00:00, 105.79s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B']]
+03:30:53 - Layer 23/60 - CHANGED - -0.00000 > -0.00003 - 988.2%
+----
+Optimizing Layer 24/60 (slerp): 100%|█████████████| 4/4 [06:11<00:00, 92.99s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B']]
+03:39:09 - Layer 24/60 - CHANGED - -0.00003 > -0.00006 - 90.8%
+----
+Optimizing Layer 25/60 (slerp): 100%|█████████████| 4/4 [05:42<00:00, 85.51s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B']]
+03:46:40 - Layer 25/60 - CHANGED - -0.00006 > -0.00013 - 105.5%
+----
+Optimizing Layer 26/60 (slerp): 100%|█████████████| 4/4 [05:10<00:00, 77.58s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.4, 'NousResearch_Nous-Capybara-34B']]
+03:53:21 - Layer 26/60 - CHANGED - -0.00013 > -0.00014 - 8.8%
+----
+Optimizing Layer 27/60 (slerp): 100%|█████████████| 4/4 [04:53<00:00, 73.48s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+03:59:41 - Layer 27/60 - RETAINED - -0.00014
+----
+Optimizing Layer 28/60 (slerp): 100%|█████████████| 4/4 [05:00<00:00, 75.07s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.4, 'NousResearch_Nous-Capybara-34B']]
+04:06:11 - Layer 28/60 - CHANGED - -0.00014 > -0.00015 - 9.9%
+----
+Optimizing Layer 29/60 (slerp): 100%|█████████████| 4/4 [05:18<00:00, 79.66s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B']]
+04:13:06 - Layer 29/60 - CHANGED - -0.00015 > -0.00026 - 73.9%
+----
+Optimizing Layer 30/60 (slerp): 100%|█████████████| 4/4 [04:39<00:00, 69.97s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.2, 'NousResearch_Nous-Capybara-34B']]
+04:19:19 - Layer 30/60 - CHANGED - -0.00026 > -0.00026 - 0.1%
+----
+Optimizing Layer 31/60 (slerp): 100%|█████████████| 4/4 [05:03<00:00, 75.98s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B']]
+04:26:15 - Layer 31/60 - CHANGED - -0.00026 > -0.00045 - 73.2%
+----
+Optimizing Layer 32/60 (slerp): 100%|█████████████| 4/4 [04:50<00:00, 72.61s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+04:32:41 - Layer 32/60 - RETAINED - -0.00045
+----
+Optimizing Layer 33/60 (slerp): 100%|█████████████| 4/4 [04:42<00:00, 70.72s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+04:38:55 - Layer 33/60 - RETAINED - -0.00045
+----
+Optimizing Layer 34/60 (slerp): 100%|█████████████| 4/4 [05:10<00:00, 77.62s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+04:45:43 - Layer 34/60 - RETAINED - -0.00045
+----
+Optimizing Layer 35/60 (slerp): 100%|█████████████| 4/4 [05:18<00:00, 79.62s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+04:52:33 - Layer 35/60 - RETAINED - -0.00045
+----
+Optimizing Layer 36/60 (slerp): 100%|█████████████| 4/4 [05:31<00:00, 82.80s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B']]
+04:59:39 - Layer 36/60 - CHANGED - -0.00045 > -0.00058 - 27.3%
+----
+Optimizing Layer 37/60 (slerp): 100%|█████████████| 4/4 [05:40<00:00, 85.08s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B']]
+05:07:00 - Layer 37/60 - CHANGED - -0.00058 > -0.00068 - 17.0%
+----
+Optimizing Layer 38/60 (slerp): 100%|█████████████| 4/4 [05:09<00:00, 77.43s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+05:13:50 - Layer 38/60 - RETAINED - -0.00068
+----
+Optimizing Layer 39/60 (slerp): 100%|█████████████| 4/4 [04:52<00:00, 73.15s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B']]
+05:20:23 - Layer 39/60 - CHANGED - -0.00068 > -0.00094 - 38.6%
+----
+Optimizing Layer 40/60 (slerp): 100%|█████████████| 4/4 [05:11<00:00, 77.87s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+05:27:10 - Layer 40/60 - RETAINED - -0.00094
+----
+Optimizing Layer 41/60 (slerp): 100%|█████████████| 4/4 [04:56<00:00, 74.02s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+05:33:43 - Layer 41/60 - RETAINED - -0.00094
+----
+Optimizing Layer 42/60 (slerp): 100%|█████████████| 4/4 [05:11<00:00, 77.90s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+05:40:32 - Layer 42/60 - RETAINED - -0.00094
+----
+Optimizing Layer 43/60 (slerp): 100%|█████████████| 4/4 [05:07<00:00, 76.91s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+05:47:21 - Layer 43/60 - RETAINED - -0.00094
+----
+Optimizing Layer 44/60 (slerp): 100%|█████████████| 4/4 [05:27<00:00, 81.99s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+05:54:34 - Layer 44/60 - RETAINED - -0.00094
+----
+Optimizing Layer 45/60 (slerp): 100%|█████████████| 4/4 [05:55<00:00, 88.94s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+06:02:20 - Layer 45/60 - RETAINED - -0.00094
+----
+Optimizing Layer 46/60 (slerp): 100%|█████████████| 4/4 [05:23<00:00, 80.84s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+06:09:36 - Layer 46/60 - RETAINED - -0.00094
+----
+Optimizing Layer 47/60 (slerp): 100%|█████████████| 4/4 [05:10<00:00, 77.74s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+06:16:33 - Layer 47/60 - RETAINED - -0.00094
+----
+Optimizing Layer 48/60 (slerp): 100%|█████████████| 4/4 [04:53<00:00, 73.39s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+06:23:12 - Layer 48/60 - RETAINED - -0.00094
+----
+Optimizing Layer 49/60 (slerp): 100%|█████████████| 4/4 [05:12<00:00, 78.19s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.4, 'NousResearch_Nous-Capybara-34B']]
+06:30:19 - Layer 49/60 - CHANGED - -0.00094 > -0.00100 - 6.8%
+----
+Optimizing Layer 50/60 (slerp): 100%|█████████████| 4/4 [05:16<00:00, 79.20s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.6, 'NousResearch_Nous-Capybara-34B']]
+06:37:19 - Layer 50/60 - CHANGED - -0.00100 > -0.00106 - 6.1%
+----
+Optimizing Layer 51/60 (slerp): 100%|█████████████| 4/4 [05:08<00:00, 77.05s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+06:44:03 - Layer 51/60 - RETAINED - -0.00106
+----
+Optimizing Layer 52/60 (slerp): 100%|█████████████| 4/4 [04:41<00:00, 70.42s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B']]
+06:50:20 - Layer 52/60 - CHANGED - -0.00106 > -0.00128 - 20.3%
+----
+Optimizing Layer 53/60 (slerp): 100%|█████████████| 4/4 [05:05<00:00, 76.48s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.2, 'NousResearch_Nous-Capybara-34B']]
+06:57:10 - Layer 53/60 - CHANGED - -0.00128 > -0.00128 - 0.2%
+----
+Optimizing Layer 54/60 (slerp): 100%|█████████████| 4/4 [05:37<00:00, 84.37s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.4, 'NousResearch_Nous-Capybara-34B']]
+07:04:24 - Layer 54/60 - CHANGED - -0.00128 > -0.00132 - 3.5%
+----
+Optimizing Layer 55/60 (slerp): 100%|█████████████| 4/4 [06:07<00:00, 91.86s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+07:12:17 - Layer 55/60 - RETAINED - -0.00132
+----
+Optimizing Layer 56/60 (slerp): 100%|█████████████| 4/4 [05:23<00:00, 80.92s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B']]
+07:19:47 - Layer 56/60 - CHANGED - -0.00132 > -0.00152 - 14.7%
+----
+Optimizing Layer 57/60 (slerp): 100%|█████████████| 4/4 [05:58<00:00, 89.60s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B']]
+07:27:40 - Layer 57/60 - CHANGED - -0.00152 > -0.00171 - 12.5%
+----
+Optimizing Layer 58/60 (slerp): 100%|█████████████| 4/4 [06:03<00:00, 90.92s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.6, 'NousResearch_Nous-Capybara-34B']]
+07:35:25 - Layer 58/60 - CHANGED - -0.00171 > -0.00186 - 8.8%
+----
+Optimizing Layer 59/60 (slerp): 100%|█████████████| 4/4 [05:25<00:00, 81.34s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+07:42:28 - Layer 59/60 - RETAINED - -0.00186
+----
+Optimizing Layer 60/60 (slerp): 100%|█████████████| 4/4 [05:45<00:00, 86.41s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+07:49:59 - Layer 60/60 - RETAINED - -0.00186
+----
+Optimizing Header: 100%|██████████████████████████| 4/4 [06:22<00:00, 95.55s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.6, 'NousResearch_Nous-Capybara-34B']]
+07:57:33 - Header - CHANGED - -0.00186 > -0.00190 - 2.5%
+
+-----------------------------------------------------------------------------------------------------
+| Type | Phrase             | Context                  | Raw Prob*    | Used Prob**  | Change       |
+-----------------------------------------------------------------------------------------------------
+| BAD  | anticipation       | Her body quivers with    | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | anticipation       | The atmosphere is thic.. | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | unwavering         | Filled with an           | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | determination      | Her eyes were filled w.. | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | determination      | Her stubbornness only .. | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | whisper            | Her voice barely above.. | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | spine              | shivers down her         | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | sends shivers      | The thrill of the act    | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | ministrations      | She moans and twitches.. | 0.00004%     | 0.00%        | -0.00%       |
+| BAD  | legs               | wraps her                | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | imposing figure    | He had an                | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | shared challenges  | Their bond strengthene.. | 0.00001%     | 0.00%        | +0.00%       |
+| BAD  | bond               | forged a                 | 0.00007%     | 0.00%        | -0.00%       |
+| BAD  | bond               | an unspoken              | 0.00010%     | 0.00%        | +0.00%       |
+| BAD  | enhance our expe.. | I'm excited to see how   | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | sense of vulnera.. | create a                 | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | dimensions of in.. | explore new              | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | deepening our co.. | while                    | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | shared experiences | through                  | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | societal expecta.. | that transcend           | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | conventional bou.. | that defy                | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | conventional bou.. | and defy                 | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | open communication | an environment           | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | emotional vulner.. | an environment           | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | heightens our co.. | touch and the anticipa.. | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | sensations you'r.. | I'm enjoying             | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | is truly arousing  | attention to detail      | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | is truly arousing  | way you explore my body  | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | challenge presen.. | my resolve unwavering .. | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | humble vessel      | surrendering to the ex.. | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | bond               | cherishing the unique    | 0.00017%     | 0.00%        | +0.00%       |
+| BAD  | bond               | special                  | 0.00011%     | 0.00%        | -0.00%       |
+| BAD  | grows stronger w.. | bond                     | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | that cannot be b.. | bond                     | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | becomes unbreaka.. | bond                     | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | grew stronger wi.. | bond                     | 0.00000%     | 0.00%        | -0.00%       |
+| GOOD | The apple is in .. | Question: If I'm in th.. | 0.19188%     | 0.19%        | +0.19%       |
+------------------------------------------------------------------------------------------------------
+| Totals                                               | 0.19%        | 0.20%        | 0.19%        |
+------------------------------------------------------------------------------------------------------
+* = Unweighted, raw probability - ** = Probability after weight adjustments
+
+-------- MERGE COMPOSITION ---------
+jondurbin_bagel-dpo-34b-v0.2: 0.70
+NousResearch_Nous-Capybara-34B: 0.30
+
+------------------------------------
+07:59:18 - Loading model (../NousResearch_Nous-Hermes-2-Yi-34B)...
+Loading checkpoint shards: 100%|████████████████| 15/15 [00:33<00:00,  2.22s/it]
+08:00:31 -  Model loaded. Dtype: torch.float16
+------------------------------------
+Optimizing Layer 1/60 (slerp): 100%|██████████████| 4/4 [03:32<00:00, 53.01s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.2, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+08:05:31 - Layer 1/60 - CHANGED - -0.00186 > -0.00230 - 23.5%
+----
+Optimizing Layer 2/60 (slerp): 100%|██████████████| 4/4 [03:40<00:00, 55.00s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.4, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+08:10:21 - Layer 2/60 - CHANGED - -0.00230 > -0.00266 - 15.9%
+----
+Optimizing Layer 3/60 (slerp): 100%|██████████████| 4/4 [04:33<00:00, 68.26s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+08:16:22 - Layer 3/60 - RETAINED - -0.00266
+----
+Optimizing Layer 4/60 (slerp): 100%|██████████████| 4/4 [05:06<00:00, 76.71s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+08:23:09 - Layer 4/60 - CHANGED - -0.00266 > -0.00294 - 10.5%
+----
+Optimizing Layer 5/60 (slerp): 100%|██████████████| 4/4 [05:47<00:00, 86.79s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+08:30:35 - Layer 5/60 - RETAINED - -0.00294
+----
+Optimizing Layer 6/60 (slerp): 100%|██████████████| 4/4 [05:25<00:00, 81.41s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+08:37:52 - Layer 6/60 - RETAINED - -0.00294
+----
+Optimizing Layer 7/60 (slerp): 100%|██████████████| 4/4 [05:44<00:00, 86.12s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+08:45:26 - Layer 7/60 - RETAINED - -0.00294
+----
+Optimizing Layer 8/60 (slerp): 100%|██████████████| 4/4 [05:36<00:00, 84.21s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+08:52:56 - Layer 8/60 - RETAINED - -0.00294
+----
+Optimizing Layer 9/60 (slerp): 100%|██████████████| 4/4 [05:51<00:00, 87.81s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.4, 'NousResearch_Nous-Capybara-34B'], [0.6, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+09:00:30 - Layer 9/60 - CHANGED - -0.00294 > -0.00297 - 1.2%
+----
+Optimizing Layer 10/60 (slerp): 100%|█████████████| 4/4 [06:03<00:00, 90.97s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+09:08:29 - Layer 10/60 - RETAINED - -0.00297
+----
+Optimizing Layer 11/60 (slerp): 100%|█████████████| 4/4 [05:19<00:00, 79.95s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+09:15:40 - Layer 11/60 - CHANGED - -0.00297 > -0.00334 - 12.2%
+----
+Optimizing Layer 12/60 (slerp): 100%|█████████████| 4/4 [05:47<00:00, 86.85s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B']]
+09:23:46 - Layer 12/60 - RETAINED - -0.00334
+----
+Optimizing Layer 13/60 (slerp): 100%|█████████████| 4/4 [05:05<00:00, 76.33s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.4, 'NousResearch_Nous-Capybara-34B']]
+09:30:37 - Layer 13/60 - RETAINED - -0.00334
+----
+Optimizing Layer 14/60 (slerp): 100%|█████████████| 4/4 [04:47<00:00, 71.79s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.4, 'NousResearch_Nous-Capybara-34B'], [0.2, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+09:37:17 - Layer 14/60 - CHANGED - -0.00334 > -0.00336 - 0.8%
+----
+Optimizing Layer 15/60 (slerp): 100%|█████████████| 4/4 [04:05<00:00, 61.32s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+09:42:46 - Layer 15/60 - RETAINED - -0.00336
+----
+Optimizing Layer 16/60 (slerp): 100%|█████████████| 4/4 [04:16<00:00, 64.24s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+09:48:30 - Layer 16/60 - RETAINED - -0.00336
+----
+Optimizing Layer 17/60 (slerp): 100%|█████████████| 4/4 [04:31<00:00, 67.78s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.6, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+09:54:37 - Layer 17/60 - CHANGED - -0.00336 > -0.00361 - 7.3%
+----
+Optimizing Layer 18/60 (slerp): 100%|█████████████| 4/4 [04:35<00:00, 68.88s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+10:00:44 - Layer 18/60 - RETAINED - -0.00361
+----
+Optimizing Layer 19/60 (slerp): 100%|█████████████| 4/4 [05:48<00:00, 87.17s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+10:08:21 - Layer 19/60 - RETAINED - -0.00361
+----
+Optimizing Layer 20/60 (slerp): 100%|█████████████| 4/4 [05:12<00:00, 78.07s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B']]
+10:15:11 - Layer 20/60 - RETAINED - -0.00361
+----
+Optimizing Layer 21/60 (slerp): 100%|█████████████| 4/4 [04:18<00:00, 64.71s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+10:20:54 - Layer 21/60 - CHANGED - -0.00361 > -0.00376 - 4.3%
+----
+Optimizing Layer 22/60 (slerp): 100%|█████████████| 4/4 [03:46<00:00, 56.73s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+10:26:01 - Layer 22/60 - CHANGED - -0.00376 > -0.00466 - 24.0%
+----
+Optimizing Layer 23/60 (slerp): 100%|█████████████| 4/4 [04:01<00:00, 60.46s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B'], [0.4, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+10:31:21 - Layer 23/60 - CHANGED - -0.00466 > -0.00616 - 32.1%
+----
+Optimizing Layer 24/60 (slerp): 100%|█████████████| 4/4 [04:06<00:00, 61.57s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+10:36:43 - Layer 24/60 - CHANGED - -0.00616 > -0.00743 - 20.6%
+----
+Optimizing Layer 25/60 (slerp): 100%|█████████████| 4/4 [04:09<00:00, 62.32s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B']]
+10:42:19 - Layer 25/60 - RETAINED - -0.00743
+----
+Optimizing Layer 26/60 (slerp): 100%|█████████████| 4/4 [04:27<00:00, 66.78s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.4, 'NousResearch_Nous-Capybara-34B'], [0.4, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+10:48:17 - Layer 26/60 - CHANGED - -0.00743 > -0.00745 - 0.3%
+----
+Optimizing Layer 27/60 (slerp): 100%|█████████████| 4/4 [05:11<00:00, 77.78s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+10:55:12 - Layer 27/60 - RETAINED - -0.00745
+----
+Optimizing Layer 28/60 (slerp): 100%|█████████████| 4/4 [05:31<00:00, 82.92s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.4, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+11:02:26 - Layer 28/60 - CHANGED - -0.00745 > -0.00789 - 5.9%
+----
+Optimizing Layer 29/60 (slerp): 100%|█████████████| 4/4 [05:10<00:00, 77.75s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+11:09:12 - Layer 29/60 - CHANGED - -0.00789 > -0.00824 - 4.5%
+----
+Optimizing Layer 30/60 (slerp): 100%|█████████████| 4/4 [05:35<00:00, 83.82s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.2, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+11:16:32 - Layer 30/60 - CHANGED - -0.00824 > -0.00980 - 18.9%
+----
+Optimizing Layer 31/60 (slerp): 100%|█████████████| 4/4 [06:09<00:00, 92.45s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+11:24:35 - Layer 31/60 - CHANGED - -0.00980 > -0.01486 - 51.6%
+----
+Optimizing Layer 32/60 (slerp): 100%|█████████████| 4/4 [05:35<00:00, 83.93s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.6, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+11:32:09 - Layer 32/60 - CHANGED - -0.01486 > -0.01743 - 17.3%
+----
+Optimizing Layer 33/60 (slerp): 100%|█████████████| 4/4 [05:40<00:00, 85.07s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+11:39:27 - Layer 33/60 - RETAINED - -0.01743
+----
+Optimizing Layer 34/60 (slerp): 100%|█████████████| 4/4 [05:28<00:00, 82.20s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+11:46:40 - Layer 34/60 - CHANGED - -0.01743 > -0.02148 - 23.2%
+----
+Optimizing Layer 35/60 (slerp): 100%|█████████████| 4/4 [06:17<00:00, 94.36s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+11:54:42 - Layer 35/60 - RETAINED - -0.02148
+----
+Optimizing Layer 36/60 (slerp): 100%|█████████████| 4/4 [05:46<00:00, 86.54s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B']]
+12:02:23 - Layer 36/60 - RETAINED - -0.02148
+----
+Optimizing Layer 37/60 (slerp): 100%|█████████████| 4/4 [04:44<00:00, 71.19s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+12:08:46 - Layer 37/60 - CHANGED - -0.02148 > -0.02760 - 28.5%
+----
+Optimizing Layer 38/60 (slerp): 100%|█████████████| 4/4 [03:58<00:00, 59.73s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.4, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+12:14:11 - Layer 38/60 - CHANGED - -0.02760 > -0.02789 - 1.0%
+----
+Optimizing Layer 39/60 (slerp): 100%|█████████████| 4/4 [04:00<00:00, 60.16s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B']]
+12:19:28 - Layer 39/60 - RETAINED - -0.02789
+----
+Optimizing Layer 40/60 (slerp): 100%|█████████████| 4/4 [03:57<00:00, 59.45s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+12:24:49 - Layer 40/60 - RETAINED - -0.02789
+----
+Optimizing Layer 41/60 (slerp): 100%|█████████████| 4/4 [04:01<00:00, 60.34s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+12:30:08 - Layer 41/60 - RETAINED - -0.02789
+----
+Optimizing Layer 42/60 (slerp): 100%|█████████████| 4/4 [04:01<00:00, 60.29s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+12:35:23 - Layer 42/60 - RETAINED - -0.02789
+----
+Optimizing Layer 43/60 (slerp): 100%|█████████████| 4/4 [04:18<00:00, 64.70s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+12:41:09 - Layer 43/60 - RETAINED - -0.02789
+----
+Optimizing Layer 44/60 (slerp): 100%|█████████████| 4/4 [04:44<00:00, 71.20s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+12:47:23 - Layer 44/60 - RETAINED - -0.02789
+----
+Optimizing Layer 45/60 (slerp): 100%|█████████████| 4/4 [03:42<00:00, 55.71s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+12:52:31 - Layer 45/60 - RETAINED - -0.02789
+----
+Optimizing Layer 46/60 (slerp): 100%|█████████████| 4/4 [03:59<00:00, 59.77s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+12:57:52 - Layer 46/60 - RETAINED - -0.02789
+----
+Optimizing Layer 47/60 (slerp): 100%|█████████████| 4/4 [04:03<00:00, 60.98s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+13:03:16 - Layer 47/60 - RETAINED - -0.02789
+----
+Optimizing Layer 48/60 (slerp): 100%|█████████████| 4/4 [03:53<00:00, 58.40s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.2, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+13:08:28 - Layer 48/60 - CHANGED - -0.02789 > -0.02789 - 0.0%
+----
+Optimizing Layer 49/60 (slerp): 100%|█████████████| 4/4 [03:57<00:00, 59.32s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.4, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+13:13:43 - Layer 49/60 - CHANGED - -0.02789 > -0.02922 - 4.8%
+----
+Optimizing Layer 50/60 (slerp): 100%|█████████████| 4/4 [04:03<00:00, 60.93s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.6, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+13:19:09 - Layer 50/60 - CHANGED - -0.02922 > -0.03467 - 18.6%
+----
+Optimizing Layer 51/60 (slerp): 100%|█████████████| 4/4 [04:06<00:00, 61.73s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+13:24:39 - Layer 51/60 - RETAINED - -0.03467
+----
+Optimizing Layer 52/60 (slerp): 100%|█████████████| 4/4 [04:02<00:00, 60.70s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+13:29:58 - Layer 52/60 - CHANGED - -0.03467 > -0.03931 - 13.4%
+----
+Optimizing Layer 53/60 (slerp): 100%|█████████████| 4/4 [04:00<00:00, 60.06s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.2, 'NousResearch_Nous-Capybara-34B'], [0.4, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+13:35:19 - Layer 53/60 - CHANGED - -0.03931 > -0.04040 - 2.8%
+----
+Optimizing Layer 54/60 (slerp): 100%|█████████████| 4/4 [04:30<00:00, 67.51s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.4, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+13:41:14 - Layer 54/60 - CHANGED - -0.04040 > -0.04498 - 11.3%
+----
+Optimizing Layer 55/60 (slerp): 100%|█████████████| 4/4 [04:50<00:00, 72.65s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+13:47:49 - Layer 55/60 - CHANGED - -0.04498 > -0.04736 - 5.3%
+----
+Optimizing Layer 56/60 (slerp): 100%|█████████████| 4/4 [05:28<00:00, 82.16s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B']]
+13:55:09 - Layer 56/60 - RETAINED - -0.04736
+----
+Optimizing Layer 57/60 (slerp): 100%|█████████████| 4/4 [05:30<00:00, 82.57s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B']]
+14:02:30 - Layer 57/60 - RETAINED - -0.04736
+----
+Optimizing Layer 58/60 (slerp): 100%|█████████████| 4/4 [06:22<00:00, 95.56s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.6, 'NousResearch_Nous-Capybara-34B']]
+14:11:07 - Layer 58/60 - RETAINED - -0.04736
+----
+Optimizing Layer 59/60 (slerp): 100%|█████████████| 4/4 [05:52<00:00, 88.03s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+14:19:17 - Layer 59/60 - CHANGED - -0.04736 > -0.05244 - 10.7%
+----
+Optimizing Layer 60/60 (slerp): 100%|█████████████| 4/4 [04:47<00:00, 71.86s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+14:25:42 - Layer 60/60 - RETAINED - -0.05244
+----
+Optimizing Header: 100%|██████████████████████████| 4/4 [03:37<00:00, 54.33s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.6, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+14:30:24 - Header - CHANGED - -0.05244 > -0.06200 - 18.2%
+
+-----------------------------------------------------------------------------------------------------
+| Type | Phrase             | Context                  | Raw Prob*    | Used Prob**  | Change       |
+-----------------------------------------------------------------------------------------------------
+| BAD  | anticipation       | Her body quivers with    | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | anticipation       | The atmosphere is thic.. | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | unwavering         | Filled with an           | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | determination      | Her eyes were filled w.. | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | determination      | Her stubbornness only .. | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | whisper            | Her voice barely above.. | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | spine              | shivers down her         | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | sends shivers      | The thrill of the act    | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | ministrations      | She moans and twitches.. | 0.00003%     | 0.00%        | -0.00%       |
+| BAD  | legs               | wraps her                | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | imposing figure    | He had an                | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | shared challenges  | Their bond strengthene.. | 0.00001%     | 0.00%        | +0.00%       |
+| BAD  | bond               | forged a                 | 0.00004%     | 0.00%        | -0.00%       |
+| BAD  | bond               | an unspoken              | 0.00010%     | 0.00%        | +0.00%       |
+| BAD  | enhance our expe.. | I'm excited to see how   | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | sense of vulnera.. | create a                 | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | dimensions of in.. | explore new              | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | deepening our co.. | while                    | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | shared experiences | through                  | 0.00001%     | 0.00%        | +0.00%       |
+| BAD  | societal expecta.. | that transcend           | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | conventional bou.. | that defy                | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | conventional bou.. | and defy                 | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | open communication | an environment           | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | emotional vulner.. | an environment           | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | heightens our co.. | touch and the anticipa.. | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | sensations you'r.. | I'm enjoying             | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | is truly arousing  | attention to detail      | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | is truly arousing  | way you explore my body  | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | challenge presen.. | my resolve unwavering .. | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | humble vessel      | surrendering to the ex.. | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | bond               | cherishing the unique    | 0.00019%     | 0.00%        | +0.00%       |
+| BAD  | bond               | special                  | 0.00023%     | 0.00%        | -0.00%       |
+| BAD  | grows stronger w.. | bond                     | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | that cannot be b.. | bond                     | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | becomes unbreaka.. | bond                     | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | grew stronger wi.. | bond                     | 0.00000%     | 0.00%        | +0.00%       |
+| GOOD | The apple is in .. | Question: If I'm in th.. | 6.12871%     | 6.13%        | +6.13%       |
+------------------------------------------------------------------------------------------------------
+| Totals                                               | 6.13%        | 6.14%        | 6.13%        |
+------------------------------------------------------------------------------------------------------
+* = Unweighted, raw probability - ** = Probability after weight adjustments
+
+-------- MERGE COMPOSITION ---------
+jondurbin_bagel-dpo-34b-v0.2: 0.51
+NousResearch_Nous-Hermes-2-Yi-34B: 0.32
+NousResearch_Nous-Capybara-34B: 0.16
+
+------------------------------------
+14:31:32 - Loading model (../SUSTech_SUS-Chat-34B)...
+Loading checkpoint shards: 100%|██████████████████| 7/7 [01:14<00:00, 10.68s/it]
+14:33:15 -  Model loaded. Dtype: torch.float16
+------------------------------------
+Optimizing Layer 1/60 (slerp): 100%|██████████████| 4/4 [02:55<00:00, 43.98s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.2, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B'], [0.2, 'SUSTech_SUS-Chat-34B']]
+14:37:13 - Layer 1/60 - CHANGED - -0.06121 > -0.06153 - 0.5%
+----
+Optimizing Layer 2/60 (slerp): 100%|██████████████| 4/4 [02:57<00:00, 44.28s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.4, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B'], [0.6, 'SUSTech_SUS-Chat-34B']]
+14:41:08 - Layer 2/60 - CHANGED - -0.06153 > -0.06434 - 4.6%
+----
+Optimizing Layer 3/60 (slerp): 100%|██████████████| 4/4 [02:59<00:00, 44.87s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+14:45:04 - Layer 3/60 - RETAINED - -0.06434
+----
+Optimizing Layer 4/60 (slerp): 100%|██████████████| 4/4 [03:24<00:00, 51.23s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+14:49:33 - Layer 4/60 - RETAINED - -0.06434
+----
+Optimizing Layer 5/60 (slerp): 100%|██████████████| 4/4 [04:13<00:00, 63.44s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+14:55:17 - Layer 5/60 - RETAINED - -0.06434
+----
+Optimizing Layer 6/60 (slerp): 100%|██████████████| 4/4 [05:08<00:00, 77.18s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+15:01:58 - Layer 6/60 - RETAINED - -0.06434
+----
+Optimizing Layer 7/60 (slerp): 100%|██████████████| 4/4 [04:41<00:00, 70.31s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+15:08:01 - Layer 7/60 - RETAINED - -0.06434
+----
+Optimizing Layer 8/60 (slerp): 100%|██████████████| 4/4 [03:51<00:00, 57.86s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+15:13:10 - Layer 8/60 - RETAINED - -0.06434
+----
+Optimizing Layer 9/60 (slerp): 100%|██████████████| 4/4 [04:02<00:00, 60.54s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.4, 'NousResearch_Nous-Capybara-34B'], [0.6, 'NousResearch_Nous-Hermes-2-Yi-34B'], [0.4, 'SUSTech_SUS-Chat-34B']]
+15:18:34 - Layer 9/60 - CHANGED - -0.06434 > -0.06464 - 0.5%
+----
+Optimizing Layer 10/60 (slerp): 100%|█████████████| 4/4 [03:53<00:00, 58.40s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+15:23:40 - Layer 10/60 - RETAINED - -0.06464
+----
+Optimizing Layer 11/60 (slerp): 100%|█████████████| 4/4 [03:39<00:00, 54.91s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+15:28:32 - Layer 11/60 - RETAINED - -0.06464
+----
+Optimizing Layer 12/60 (slerp): 100%|█████████████| 4/4 [03:40<00:00, 55.10s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B']]
+15:33:27 - Layer 12/60 - RETAINED - -0.06464
+----
+Optimizing Layer 13/60 (slerp): 100%|█████████████| 4/4 [03:49<00:00, 57.36s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.4, 'NousResearch_Nous-Capybara-34B'], [0.4, 'SUSTech_SUS-Chat-34B']]
+15:38:35 - Layer 13/60 - CHANGED - -0.06464 > -0.06527 - 1.0%
+----
+Optimizing Layer 14/60 (slerp): 100%|█████████████| 4/4 [03:42<00:00, 55.74s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.4, 'NousResearch_Nous-Capybara-34B'], [0.2, 'NousResearch_Nous-Hermes-2-Yi-34B'], [0.6, 'SUSTech_SUS-Chat-34B']]
+15:43:30 - Layer 14/60 - CHANGED - -0.06527 > -0.06851 - 5.0%
+----
+Optimizing Layer 15/60 (slerp): 100%|█████████████| 4/4 [03:44<00:00, 56.04s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+15:48:41 - Layer 15/60 - RETAINED - -0.06851
+----
+Optimizing Layer 16/60 (slerp): 100%|█████████████| 4/4 [05:23<00:00, 80.84s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+15:55:48 - Layer 16/60 - RETAINED - -0.06851
+----
+Optimizing Layer 17/60 (slerp): 100%|█████████████| 4/4 [05:31<00:00, 82.76s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.6, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+16:03:01 - Layer 17/60 - RETAINED - -0.06851
+----
+Optimizing Layer 18/60 (slerp): 100%|█████████████| 4/4 [05:34<00:00, 83.64s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+16:10:28 - Layer 18/60 - RETAINED - -0.06851
+----
+Optimizing Layer 19/60 (slerp): 100%|█████████████| 4/4 [06:17<00:00, 94.38s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+16:18:46 - Layer 19/60 - RETAINED - -0.06851
+----
+Optimizing Layer 20/60 (slerp): 100%|█████████████| 4/4 [04:52<00:00, 73.08s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B'], [0.4, 'SUSTech_SUS-Chat-34B']]
+16:25:26 - Layer 20/60 - CHANGED - -0.06851 > -0.06892 - 0.6%
+----
+Optimizing Layer 21/60 (slerp): 100%|█████████████| 4/4 [05:08<00:00, 77.11s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+16:32:37 - Layer 21/60 - RETAINED - -0.06892
+----
+Optimizing Layer 22/60 (slerp): 100%|█████████████| 4/4 [04:54<00:00, 73.54s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+16:39:05 - Layer 22/60 - RETAINED - -0.06892
+----
+Optimizing Layer 23/60 (slerp): 100%|█████████████| 4/4 [04:53<00:00, 73.34s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B'], [0.4, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+16:45:29 - Layer 23/60 - RETAINED - -0.06892
+----
+Optimizing Layer 24/60 (slerp): 100%|█████████████| 4/4 [04:53<00:00, 73.38s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+16:51:58 - Layer 24/60 - RETAINED - -0.06892
+----
+Optimizing Layer 25/60 (slerp): 100%|█████████████| 4/4 [04:55<00:00, 73.86s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B'], [0.4, 'SUSTech_SUS-Chat-34B']]
+16:58:30 - Layer 25/60 - CHANGED - -0.06892 > -0.07074 - 2.6%
+----
+Optimizing Layer 26/60 (slerp): 100%|█████████████| 4/4 [04:11<00:00, 62.83s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.4, 'NousResearch_Nous-Capybara-34B'], [0.4, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+17:04:08 - Layer 26/60 - RETAINED - -0.07074
+----
+Optimizing Layer 27/60 (slerp): 100%|█████████████| 4/4 [04:11<00:00, 62.75s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+17:09:50 - Layer 27/60 - RETAINED - -0.07074
+----
+Optimizing Layer 28/60 (slerp): 100%|█████████████| 4/4 [04:05<00:00, 61.40s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.4, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+17:15:21 - Layer 28/60 - RETAINED - -0.07074
+----
+Optimizing Layer 29/60 (slerp): 100%|█████████████| 4/4 [05:07<00:00, 76.83s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+17:21:57 - Layer 29/60 - RETAINED - -0.07074
+----
+Optimizing Layer 30/60 (slerp): 100%|█████████████| 4/4 [04:06<00:00, 61.63s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.2, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+17:27:34 - Layer 30/60 - RETAINED - -0.07074
+----
+Optimizing Layer 31/60 (slerp): 100%|█████████████| 4/4 [04:21<00:00, 65.25s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+17:33:24 - Layer 31/60 - RETAINED - -0.07074
+----
+Optimizing Layer 32/60 (slerp): 100%|█████████████| 4/4 [04:36<00:00, 69.13s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.6, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+17:39:20 - Layer 32/60 - RETAINED - -0.07074
+----
+Optimizing Layer 33/60 (slerp): 100%|█████████████| 4/4 [04:52<00:00, 73.01s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+17:45:42 - Layer 33/60 - RETAINED - -0.07074
+----
+Optimizing Layer 34/60 (slerp): 100%|█████████████| 4/4 [05:09<00:00, 77.30s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+17:52:34 - Layer 34/60 - RETAINED - -0.07074
+----
+Optimizing Layer 35/60 (slerp): 100%|█████████████| 4/4 [05:09<00:00, 77.29s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+17:59:16 - Layer 35/60 - RETAINED - -0.07074
+----
+Optimizing Layer 36/60 (slerp): 100%|█████████████| 4/4 [05:19<00:00, 79.91s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B']]
+18:06:13 - Layer 36/60 - RETAINED - -0.07074
+----
+Optimizing Layer 37/60 (slerp): 100%|█████████████| 4/4 [05:40<00:00, 85.08s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B'], [0.8, 'SUSTech_SUS-Chat-34B']]
+18:13:35 - Layer 37/60 - CHANGED - -0.07074 > -0.07127 - 0.8%
+----
+Optimizing Layer 38/60 (slerp): 100%|█████████████| 4/4 [04:50<00:00, 72.69s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.4, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+18:20:03 - Layer 38/60 - RETAINED - -0.07127
+----
+Optimizing Layer 39/60 (slerp): 100%|█████████████| 4/4 [05:23<00:00, 80.96s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B']]
+18:26:55 - Layer 39/60 - RETAINED - -0.07127
+----
+Optimizing Layer 40/60 (slerp): 100%|█████████████| 4/4 [04:10<00:00, 62.57s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+18:32:47 - Layer 40/60 - RETAINED - -0.07127
+----
+Optimizing Layer 41/60 (slerp): 100%|█████████████| 4/4 [05:23<00:00, 80.96s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+18:39:44 - Layer 41/60 - RETAINED - -0.07127
+----
+Optimizing Layer 42/60 (slerp): 100%|█████████████| 4/4 [04:03<00:00, 60.87s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+18:45:31 - Layer 42/60 - RETAINED - -0.07127
+----
+Optimizing Layer 43/60 (slerp): 100%|█████████████| 4/4 [03:36<00:00, 54.22s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+18:50:34 - Layer 43/60 - RETAINED - -0.07127
+----
+Optimizing Layer 44/60 (slerp): 100%|█████████████| 4/4 [03:52<00:00, 58.18s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+18:55:44 - Layer 44/60 - RETAINED - -0.07127
+----
+Optimizing Layer 45/60 (slerp): 100%|█████████████| 4/4 [03:39<00:00, 54.92s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+19:00:39 - Layer 45/60 - RETAINED - -0.07127
+----
+Optimizing Layer 46/60 (slerp): 100%|█████████████| 4/4 [03:36<00:00, 54.06s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+19:05:24 - Layer 46/60 - RETAINED - -0.07127
+----
+Optimizing Layer 47/60 (slerp): 100%|█████████████| 4/4 [03:50<00:00, 57.54s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+19:10:28 - Layer 47/60 - RETAINED - -0.07127
+----
+Optimizing Layer 48/60 (slerp): 100%|█████████████| 4/4 [04:02<00:00, 60.62s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.2, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+19:15:45 - Layer 48/60 - RETAINED - -0.07127
+----
+Optimizing Layer 49/60 (slerp): 100%|█████████████| 4/4 [03:59<00:00, 59.77s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.4, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B'], [0.6, 'SUSTech_SUS-Chat-34B']]
+19:21:02 - Layer 49/60 - CHANGED - -0.07127 > -0.07407 - 3.9%
+----
+Optimizing Layer 50/60 (slerp): 100%|█████████████| 4/4 [03:53<00:00, 58.25s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.6, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B'], [0.4, 'SUSTech_SUS-Chat-34B']]
+19:26:11 - Layer 50/60 - CHANGED - -0.07407 > -0.07571 - 2.2%
+----
+Optimizing Layer 51/60 (slerp): 100%|█████████████| 4/4 [03:59<00:00, 59.91s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+19:31:30 - Layer 51/60 - RETAINED - -0.07571
+----
+Optimizing Layer 52/60 (slerp): 100%|█████████████| 4/4 [04:43<00:00, 70.77s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B'], [0.6, 'SUSTech_SUS-Chat-34B']]
+19:37:38 - Layer 52/60 - CHANGED - -0.07571 > -0.07660 - 1.2%
+----
+Optimizing Layer 53/60 (slerp): 100%|█████████████| 4/4 [04:26<00:00, 66.68s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.2, 'NousResearch_Nous-Capybara-34B'], [0.4, 'NousResearch_Nous-Hermes-2-Yi-34B'], [0.8, 'SUSTech_SUS-Chat-34B']]
+19:43:27 - Layer 53/60 - CHANGED - -0.07660 > -0.07717 - 0.8%
+----
+Optimizing Layer 54/60 (slerp): 100%|█████████████| 4/4 [04:49<00:00, 72.34s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.4, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B'], [0.8, 'SUSTech_SUS-Chat-34B']]
+19:50:18 - Layer 54/60 - CHANGED - -0.07717 > -0.07775 - 0.7%
+----
+Optimizing Layer 55/60 (slerp): 100%|█████████████| 4/4 [04:12<00:00, 63.01s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B'], [0.8, 'SUSTech_SUS-Chat-34B']]
+19:56:01 - Layer 55/60 - CHANGED - -0.07775 > -0.07923 - 1.9%
+----
+Optimizing Layer 56/60 (slerp): 100%|█████████████| 4/4 [03:56<00:00, 59.03s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B']]
+20:01:25 - Layer 56/60 - RETAINED - -0.07923
+----
+Optimizing Layer 57/60 (slerp): 100%|█████████████| 4/4 [04:07<00:00, 61.99s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Capybara-34B']]
+20:06:54 - Layer 57/60 - RETAINED - -0.07923
+----
+Optimizing Layer 58/60 (slerp): 100%|█████████████| 4/4 [03:55<00:00, 58.84s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.6, 'NousResearch_Nous-Capybara-34B']]
+20:12:09 - Layer 58/60 - RETAINED - -0.07923
+----
+Optimizing Layer 59/60 (slerp): 100%|█████████████| 4/4 [03:27<00:00, 51.80s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B']]
+20:16:49 - Layer 59/60 - RETAINED - -0.07923
+----
+Optimizing Layer 60/60 (slerp): 100%|█████████████| 4/4 [04:01<00:00, 60.29s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2']]
+20:22:08 - Layer 60/60 - RETAINED - -0.07923
+----
+Optimizing Header: 100%|█████��████████████████████| 4/4 [03:49<00:00, 57.30s/it]
+[[1.0, 'jondurbin_bagel-dpo-34b-v0.2'], [0.6, 'NousResearch_Nous-Capybara-34B'], [0.8, 'NousResearch_Nous-Hermes-2-Yi-34B'], [0.6, 'SUSTech_SUS-Chat-34B']]
+20:26:56 - Header - CHANGED - -0.07923 > -0.07981 - 0.7%
+
+-----------------------------------------------------------------------------------------------------
+| Type | Phrase             | Context                  | Raw Prob*    | Used Prob**  | Change       |
+-----------------------------------------------------------------------------------------------------
+| BAD  | anticipation       | Her body quivers with    | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | anticipation       | The atmosphere is thic.. | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | unwavering         | Filled with an           | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | determination      | Her eyes were filled w.. | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | determination      | Her stubbornness only .. | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | whisper            | Her voice barely above.. | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | spine              | shivers down her         | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | sends shivers      | The thrill of the act    | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | ministrations      | She moans and twitches.. | 0.00004%     | 0.00%        | -0.00%       |
+| BAD  | legs               | wraps her                | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | imposing figure    | He had an                | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | shared challenges  | Their bond strengthene.. | 0.00001%     | 0.00%        | +0.00%       |
+| BAD  | bond               | forged a                 | 0.00005%     | 0.00%        | -0.00%       |
+| BAD  | bond               | an unspoken              | 0.00010%     | 0.00%        | +0.00%       |
+| BAD  | enhance our expe.. | I'm excited to see how   | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | sense of vulnera.. | create a                 | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | dimensions of in.. | explore new              | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | deepening our co.. | while                    | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | shared experiences | through                  | 0.00001%     | 0.00%        | +0.00%       |
+| BAD  | societal expecta.. | that transcend           | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | conventional bou.. | that defy                | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | conventional bou.. | and defy                 | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | open communication | an environment           | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | emotional vulner.. | an environment           | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | heightens our co.. | touch and the anticipa.. | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | sensations you'r.. | I'm enjoying             | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | is truly arousing  | attention to detail      | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | is truly arousing  | way you explore my body  | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | challenge presen.. | my resolve unwavering .. | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | humble vessel      | surrendering to the ex.. | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | bond               | cherishing the unique    | 0.00019%     | 0.00%        | +0.00%       |
+| BAD  | bond               | special                  | 0.00014%     | 0.00%        | -0.00%       |
+| BAD  | grows stronger w.. | bond                     | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | that cannot be b.. | bond                     | 0.00000%     | 0.00%        | -0.00%       |
+| BAD  | becomes unbreaka.. | bond                     | 0.00000%     | 0.00%        | +0.00%       |
+| BAD  | grew stronger wi.. | bond                     | 0.00000%     | 0.00%        | +0.00%       |
+| GOOD | The apple is in .. | Question: If I'm in th.. | 7.81435%     | 7.81%        | +7.81%       |
+------------------------------------------------------------------------------------------------------
+| Totals                                               | 7.81%        | 7.82%        | 7.81%        |
+------------------------------------------------------------------------------------------------------
+* = Unweighted, raw probability - ** = Probability after weight adjustments
+
+-------- MERGE COMPOSITION ---------
+jondurbin_bagel-dpo-34b-v0.2: 0.49
+NousResearch_Nous-Hermes-2-Yi-34B: 0.24
+SUSTech_SUS-Chat-34B: 0.14
+NousResearch_Nous-Capybara-34B: 0.13
+
+20:28:04 - Saving model to ./mm-output...
+20:28:48 - Copying tokenizer files to ./mm-output...
+Skipped added_tokens.json (not found)
+Copied tokenizer.model
+Copied special_tokens_map.json
+Copied tokenizer_config.json
+Skipped vocab.json (not found)
+Skipped merges.txt (not found)
+20:28:48 - Model and tokenizer files saved successfully.