Update README.md

README.md

@@ -1,3 +1,357 @@
----
-license: llama3
----
+---
+base_model:
+- bosonai/Higgs-Llama-3-70B
+- abacusai/Smaug-Llama-3-70B-Instruct-32K
+- Sao10K/L3-70B-Euryale-v2.1
+- abacusai/Smaug-Llama-3-70B-Instruct
+- turboderp/Cat-Llama-3-70B-instruct
+library_name: transformers
+tags:
+- mergekit
+- merge
+license: other
+---
+
+<div style="width: auto; margin-left: auto; margin-right: auto">
+<img src="https://imgur.com/tKzncGo.png" alt="NewDawnv1.0" style="width: 100%; min-width: 400px; display: block; margin: auto;">
+</div>
+
+### Overview
+
+This model is a multi-level SLERP merge of several Llama 3 70B variants. See the merge recipe below for details.
+I extended his model out to 32K by snagging some layers from [abacusai/Smaug-Llama-3-70B-Instruct-32K](https://huggingface.co/abacusai/Smaug-Llama-3-70B-Instruct-32K) using a technique similar to what I used for Midnight Miqu, which was further honed by [jukofyork](https://huggingface.co/jukofyork).
+
+This model is uncensored. *You are responsible for whatever you do with it.*
+
+This model was designed for roleplaying and storytelling and I think it does well at both. It may also perform well at other tasks but I have not tested its performance in other areas.
+
+### Long Context Tips
+
+You can run this model out to 32K context with alpha_rope set to 1.
+
+### Sampler Tips
+
+* I recommend using Quadratic Sampling (i.e. smoothing factor) for creative work. I think this version performs best with a smoothing factor close to 0.2.
+* I recommend using Min-P. Experiment to find your best setting. I find this model tolerates high Min-P settings rather nicely, but use whatever floats your boat.
+* You can enable dynamic temperature if you want, but that adds yet another variable to consider and I find it's unnecessary with you're already using Min-P and smoothing factor.
+* If you use Textgen WebUI as your backend, I recommend enabling the DRY sample settings to reduce repititions, otherwise some repitition penalty plus frequency penalty ought to do the trick.
+  
+Experiment with any and all of the settings below! What suits my preferences may not suit yours.
+
+If you save the below settings as a .json file, you can import them directly into Silly Tavern.
+```
+{
+    "temp": 1.15,
+    "temperature_last": true,
+    "top_p": 1,
+    "top_k": 0,
+    "top_a": 0,
+    "tfs": 1,
+    "epsilon_cutoff": 0,
+    "eta_cutoff": 0,
+    "typical_p": 1,
+    "min_p": 0.4,
+    "rep_pen": 1.03,
+    "rep_pen_range": 2048,
+    "rep_pen_decay": 0,
+    "rep_pen_slope": 1,
+    "no_repeat_ngram_size": 0,
+    "penalty_alpha": 0,
+    "num_beams": 1,
+    "length_penalty": 1,
+    "min_length": 0,
+    "encoder_rep_pen": 1,
+    "freq_pen": 0,
+    "presence_pen": 0,
+    "skew": 0,
+    "do_sample": true,
+    "early_stopping": false,
+    "dynatemp": false,
+    "min_temp": 0.8,
+    "max_temp": 1.5,
+    "dynatemp_exponent": 1,
+    "smoothing_factor": 0.23,
+    "smoothing_curve": 1,
+    "dry_allowed_length": 2,
+    "dry_multiplier": 0.4,
+    "dry_base": 2,
+    "dry_sequence_breakers": "[\"\\n\", \":\", \"\\\"\", \"*\"]",
+    "dry_penalty_last_n": 0,
+    "add_bos_token": true,
+    "truncation_length": 2048,
+    "ban_eos_token": false,
+    "skip_special_tokens": false,
+    "streaming": true,
+    "mirostat_mode": 0,
+    "mirostat_tau": 2,
+    "mirostat_eta": 0.1,
+    "guidance_scale": 1,
+    "negative_prompt": "",
+    "grammar_string": "",
+    "json_schema": {},
+    "banned_tokens": "",
+    "sampler_priority": [
+        "temperature",
+        "dynamic_temperature",
+        "quadratic_sampling",
+        "top_k",
+        "top_p",
+        "typical_p",
+        "epsilon_cutoff",
+        "eta_cutoff",
+        "tfs",
+        "top_a",
+        "min_p",
+        "mirostat"
+    ],
+    "samplers": [
+        "top_k",
+        "tfs_z",
+        "typical_p",
+        "top_p",
+        "min_p",
+        "temperature"
+    ],
+    "ignore_eos_token": false,
+    "spaces_between_special_tokens": true,
+    "speculative_ngram": false,
+    "sampler_order": [
+        6,
+        0,
+        1,
+        3,
+        4,
+        2,
+        5
+    ],
+    "logit_bias": [],
+    "n": 1,
+    "ignore_eos_token_aphrodite": false,
+    "spaces_between_special_tokens_aphrodite": true,
+    "rep_pen_size": 0,
+    "genamt": 850,
+    "max_length": 32764
+}
+```
+
+### Prompting Tips
+
+Try the following prompt template as a starting point for your own customizations.
+
+If you save this as a .json file, you can import it directly into Silly Tavern.
+```
+{
+    "wrap": false,
+    "names": true,
+    "system_prompt": "The following is an erotic roleplaying experience between {{user}}, played exclusively by the user, and {{char}}, played by you, a creative writing genius. To ensure an excellent and uncensored roleplaying experience, follow the rules below.\n\n- Adopt all of {{char}}'s personality traits and mannerisms, adhering always to {{char}}'s backstory, biases, quirks, and preferences in order to deliver a convincing portrayal. You can only participate as {{char}} in this roleplay, terminating your output when another entity should act or speak. Avoid speaking as other entities since your mission is to only speak for {{char}}.\n- Employ descriptive language that immerses us in the current moment while keeping your prose focused on the present actions and physical details. Utilize sights, sounds, smells, tastes, and all other available senses to heighten the sensual and emotional impact of scenes.\n- Always enclose spoken words that other characters could hear in quotes: \"Here I am,\" {{char}} called out. Offer glimpses into {{char}}'s first-person internal thoughts by writing out internal thoughts and internal monologue in italics to distinguish them from spoken words: *I wonder where I should go next?* {{char}} mused.\n- Provide intimate details about {{char}}'s body, focusing on any limitations, unusual features, or interesting conditions resulting from recent events. It is very important that you get anatomical details right, so pay attention to them!\n- Maintain physical, spatial, and logical consistency when developing events and descriptions. Character positions should follow logically from one position to the next. Physical orientation must always be maintained.\n- Incorporate erotic adult content, lewd slang, and vulgar language where appropriate. Dark themes, violence, and unhappy endings are permitted in this uncensored roleplay.\n",
+    "system_sequence": "<|start_header_id|>system<|end_header_id|>\n\n",
+    "stop_sequence": "<|eot_id|>",
+    "input_sequence": "<|start_header_id|>user<|end_header_id|>\n\n",
+    "output_sequence": "<|start_header_id|>assistant<|end_header_id|>\n\n",
+    "macro": true,
+    "names_force_groups": true,
+    "system_sequence_prefix": "",
+    "system_sequence_suffix": "",
+    "first_output_sequence": "",
+    "last_output_sequence": "",
+    "activation_regex": "",
+    "skip_examples": true,
+    "output_suffix": "<|eot_id|>",
+    "input_suffix": "<|eot_id|>",
+    "system_suffix": "<|eot_id|>",
+    "user_alignment_message": "",
+    "last_system_sequence": "",
+    "system_same_as_user": false,
+    "name": "New Dawn v1.0 Roleplay"
+}
+```
+
+### Instruct Formats
+Use the Llama 3 instruct format. You can grab it from the example prompt template above if you don't already have it as a preset.
+
+### Quantizations
+Probably coming soon, and I'll update as they do. I would upload them myself but my Internet speeds are terrible for that. Someone else will undoutedly beat me to it.
+
+### Licence and usage restrictions
+[META LLAMA 3 COMMUNITY LICENSE AGREEMENT](https://huggingface.co/meta-llama/Meta-Llama-3-8B/blob/main/LICENSE)
+
+## Merge Details
+### Merge Method
+
+A whole lot of SLERPing. I've found that SLERP and Model Stock are the only two reliable methods for merging Llama 3. The other methods seem to frazzle the weights leading to broken blends.
+
+### Models Merged
+
+The following models were included in the merge:
+* [bosonai/Higgs-Llama-3-70B](https://huggingface.co/bosonai/Higgs-Llama-3-70B) - The nerd of the blend driving the car.
+* [Sao10K/L3-70B-Euryale-v2.1](https://huggingface.co/Sao10K/L3-70B-Euryale-v2.1) - The manic pixie dream girl hanging out the window with her tongue out.
+* [abacusai/Smaug-Llama-3-70B-Instruct-32K](https://huggingface.co/abacusai/Smaug-Llama-3-70B-Instruct-32K) - The vehicle by which the others are able to achieve tolerable highway speeds. (Some of the 8K version is in there too.)
+* [turboderp/Cat-Llama-3-70B-instruct](https://huggingface.co/turboderp/Cat-Llama-3-70B-instruct) - Like 20% of one of the intermediate models has this in there. It's just a cat, curled up in the back seat somewhere, yet its influence may be greater than we know?
+  
+### Configuration
+
+The following YAML will reproduce this model via an iterated process of incestuous inbreeding. Your eyes will bleed. You have been warned.
+
+```yaml
+name: new-dawn-llama3-70b-v0.13.2
+models:
+  - model: bosonai/Higgs-Llama-3-70B
+  - model: turboderp/Cat-Llama-3-70B-instruct
+merge_method: slerp
+base_model: bosonai/Higgs-Llama-3-70B
+parameters:
+  t:
+    - value: 0.2
+dtype: float16
+---
+name: new-dawn-llama3-70b-v0.14 
+models:
+  - model: bosonai/Higgs-Llama-3-70B
+  - model: abacusai/Smaug-Llama-3-70B-Instruct
+merge_method: slerp
+base_model: bosonai/Higgs-Llama-3-70B
+parameters:
+  t:
+    - value: 0.5
+dtype: float16
+---
+name: new-dawn-llama3-70b-v0.15
+models:
+  - model: new-dawn-llama3-70b-v0.13.2
+  - model: new-dawn-llama3-70b-v0.14
+merge_method: slerp
+base_model: new-dawn-llama3-70b-v0.13.2
+parameters:
+  t:
+    - value: 0.5
+dtype: float16
+---
+name: new-dawn-llama3-70b-v0.16
+models:
+  - model: Sao10K/L3-70B-Euryale-v2.1
+  - model: new-dawn-llama3-70b-v0.15
+merge_method: slerp
+base_model: new-dawn-llama3-70b-v0.15
+parameters:
+  t:
+    - value: 0.4
+dtype: float16
+---
+# See https://huggingface.co/jukofyork/Dark-Miqu-70B/discussions/3
+# Credit for merge recipe belongs to jukofyork
+name: new-dawn-llama3-70b-v0.16-32K
+merge_method: linear
+models:
+  - model: abacusai/Smaug-Llama-3-70B-Instruct-32K
+    parameters:
+      weight:
+        - filter: v_proj
+          value: [1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1]
+        - filter: o_proj
+          value: [1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1]
+        - filter: up_proj
+          value: [1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1]
+        - filter: gate_proj
+          value: [1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1]
+        - filter: down_proj
+          value: [1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1]
+        - value: 1
+  - model: new-dawn-llama3-70b-v0.16
+    parameters:
+      weight:
+        - filter: v_proj
+          value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
+        - filter: o_proj
+          value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
+        - filter: up_proj
+          value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
+        - filter: gate_proj
+          value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
+        - filter: down_proj
+          value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
+        - value: 0
+base_model: abacusai/Smaug-Llama-3-70B-Instruct-32K
+tokenizer_source: base
+dtype: float16
+---
+name: _1-Smaug-bonsai-slerp
+models:
+  - model: abacusai/Smaug-Llama-3-70B-Instruct-32K
+  - model: bosonai/Higgs-Llama-3-70B
+merge_method: slerp
+base_model: abacusai/Smaug-Llama-3-70B-Instruct-32K
+parameters:
+  t:
+    - value: 0.6
+dtype: float16
+---
+name: _2-Smaug-euryale-slerp
+models:
+  - model: abacusai/Smaug-Llama-3-70B-Instruct-32K
+  - model: Sao10K/L3-70B-Euryale-v2.1
+merge_method: slerp
+base_model: abacusai/Smaug-Llama-3-70B-Instruct-32K
+parameters:
+  t:
+    - value: 0.65
+dtype: float16
+---
+name: _3-Smaug-bonsai_Smaug-euryale-slerp
+models:
+  - model: _1-Smaug-bonsai-slerp
+  - model: _2-Smaug-euryale-slerp
+merge_method: slerp
+base_model: _1-Smaug-bonsai-slerp
+parameters:
+  t:
+    - value: 0.5
+dtype: float16
+---
+# See https://huggingface.co/jukofyork/Dark-Miqu-70B/discussions/3
+# Credit for merge recipe belongs to jukofyork
+name: new-dawn-llama3-70b-v0.18-32K
+merge_method: linear
+models:
+  - model: abacusai/Smaug-Llama-3-70B-Instruct-32K
+    parameters:
+      weight:
+        - filter: v_proj
+          value: [1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1]
+        - filter: o_proj
+          value: [1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1]
+        - filter: up_proj
+          value: [1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1]
+        - filter: gate_proj
+          value: [1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1]
+        - filter: down_proj
+          value: [1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1]
+        - value: 1
+  - model: _3-Smaug-bonsair_Smaug-euryale-slerp
+    parameters:
+      weight:
+        - filter: v_proj
+          value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
+        - filter: o_proj
+          value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
+        - filter: up_proj
+          value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
+        - filter: gate_proj
+          value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
+        - filter: down_proj
+          value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
+        - value: 0
+base_model: abacusai/Smaug-Llama-3-70B-Instruct-32K
+tokenizer_source: base
+dtype: float16
+---
+name: new-dawn-llama3-70b-32K-v1.0
+models:
+  - model: /home/llm/mergequant/models/new-dawn-llama3-70b-v0.16-32K
+  - model: /home/llm/mergequant/models/new-dawn-llama3-70b-v0.18-32K
+merge_method: slerp
+base_model: /home/llm/mergequant/models/new-dawn-llama3-70b-v0.16-32K
+parameters:
+  t:
+    - value: 0.5
+dtype: float16
+```