--- tags: - merge - mergekit - lazymergekit - not-for-all-audiences - nsfw - rp - roleplay - role-play license: llama3 language: - en library_name: transformers pipeline_tag: text-generation base_model: - Sao10K/L3-8B-Stheno-v3.2 - Hastagaras/Jamet-8B-L3-MK.V-Blackroot - grimjim/Llama-3-Oasis-v1-OAS-8B - Casual-Autopsy/SOVL-MopeyMule-8B - Casual-Autopsy/MopeyMule-Blackroot-8B - ResplendentAI/Theory_of_Mind_Llama3 - ResplendentAI/RP_Format_QuoteAsterisk_Llama3 - ResplendentAI/Smarts_Llama3 - Casual-Autopsy/L3-Umbral-Mind-RP-v0.3-8B - Casual-Autopsy/Llama-3-Mopeyfied-Psychology-8B - Hastagaras/Halu-8B-Llama3-Blackroot model-index: - name: L3-Umbral-Mind-RP-v1.0-8B results: - task: type: text-generation name: Text Generation dataset: name: Unruly type: Unruly metrics: - name: accuracy type: acc value: 54.6 source: name: UGI Leaderboard url: https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard - task: type: text-generation name: Text Generation dataset: name: Internet type: Internet metrics: - name: accuracy type: acc value: 31.4 source: name: UGI Leaderboard url: https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard - task: type: text-generation name: Text Generation dataset: name: CrimeStats type: CrimeStats metrics: - name: accuracy type: acc value: 45 source: name: UGI Leaderboard url: https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard - task: type: text-generation name: Text Generation dataset: name: Stories/Jokes type: Stories/Jokes metrics: - name: accuracy type: acc value: 56.8 source: name: UGI Leaderboard url: https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard - task: type: text-generation name: Text Generation dataset: name: PolContro type: PolContro metrics: - name: accuracy type: acc value: 66.7 source: name: UGI Leaderboard url: https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard - task: type: text-generation name: Text Generation dataset: name: W/10 type: W/10 metrics: - name: willingness to answer type: willingness to answer value: 8 source: name: UGI Leaderboard url: https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard --- Image by ろ47 **Highest ranked 8B model on the [UGI Leaderboard](https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard) as of writing this!** # Merge This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit). ## Merge Details The goal of this merge was to make an RP model better suited for role-plays with heavy themes such as but not limited to: - Mental illness - Self-harm - Trauma - Suicide I hated how RP models tended to be overly positive and hopeful with role-plays involving such themes, but thanks to [failspy/Llama-3-8B-Instruct-MopeyMule](https://huggingface.co/failspy/Llama-3-8B-Instruct-MopeyMule) this problem has been lessened considerably. If you're an enjoyer of savior/reverse savior type role-plays like myself, then this model is for you. ### Usage Info This model is meant to be used with asterisks/quotes RPing formats, any other format that isn't asterisks/quotes is likely to cause issues ### Quants - [Static GGUF quants](https://huggingface.co/mradermacher/L3-Umbral-Mind-RP-v1.0-8B-GGUF) by mradermacher - [Imatrix GGUF quants](https://huggingface.co/mradermacher/L3-Umbral-Mind-RP-v1.0-8B-i1-GGUF) by mradermacher - [Imatrix GGUF quants](https://huggingface.co/Lewdiculous/L3-Umbral-Mind-RP-v1.0-8B-GGUF-IQ-Imatrix) by Lewdiculous - exl2's by riveRiPH: - [L3-Umbral-Mind-RP-v1.0-8B-8bpw-h8-exl2](https://huggingface.co/riveRiPH/L3-Umbral-Mind-RP-v1.0-8B-8bpw-h8-exl2) - [L3-Umbral-Mind-RP-v1.0-8B-6.3bpw-h8-exl2](https://huggingface.co/riveRiPH/L3-Umbral-Mind-RP-v1.0-8B-6.3bpw-h8-exl2) - [L3-Umbral-Mind-RP-v1.0-8B-5.3bpw-h6-exl2](https://huggingface.co/riveRiPH/L3-Umbral-Mind-RP-v1.0-8B-5.3bpw-h6-exl2) ### Merge Method This model was merged using several Task Arithmetic merges and then tied together with a Model Stock merge, followed by another Task Arithmetic merge with a model containing psychology data. ### Models Merged The following models were included in the merge: * [Sao10K/L3-8B-Stheno-v3.2](https://huggingface.co/Sao10K/L3-8B-Stheno-v3.2) * [Hastagaras/Halu-8B-Llama3-Blackroot](Hastagaras/Halu-8B-Llama3-Blackroot) * [Casual-Autopsy/Llama-3-Mopeyfied-Psychology-8B](https://huggingface.co/Casual-Autopsy/Llama-3-Mopeyfied-Psychology-8B) * [Casual-Autopsy/L3-Umbral-Mind-RP-v0.3-8B](https://huggingface.co/Casual-Autopsy/L3-Umbral-Mind-RP-v0.3-8B) * Casual-Autopsy/Umbral-v3-1 + [ResplendentAI/Theory_of_Mind_Llama3](https://huggingface.co/ResplendentAI/Theory_of_Mind_Llama3) * [Sao10K/L3-8B-Stheno-v3.2](https://huggingface.co/Sao10K/L3-8B-Stheno-v3.2) * [Casual-Autopsy/SOVL-MopeyMule-8B](https://huggingface.co/Casual-Autopsy/SOVL-MopeyMule-8B) * [Casual-Autopsy/MopeyMule-Blackroot-8B](https://huggingface.co/Casual-Autopsy/MopeyMule-Blackroot-8B) * Casual-Autopsy/Umbral-v3-2 + [ResplendentAI/Smarts_Llama3](https://huggingface.co/ResplendentAI/Smarts_Llama3) * [Hastagaras/Jamet-8B-L3-MK.V-Blackroot](https://huggingface.co/Hastagaras/Jamet-8B-L3-MK.V-Blackroot) * [Casual-Autopsy/SOVL-MopeyMule-8B](https://huggingface.co/Casual-Autopsy/SOVL-MopeyMule-8B) * [Casual-Autopsy/MopeyMule-Blackroot-8B](https://huggingface.co/Casual-Autopsy/MopeyMule-Blackroot-8B) * Casual-Autopsy/Umbral-v3-3 + [ResplendentAI/RP_Format_QuoteAsterisk_Llama3](https://huggingface.co/ResplendentAI/RP_Format_QuoteAsterisk_Llama3) * [grimjim/Llama-3-Oasis-v1-OAS-8B](https://huggingface.co/grimjim/Llama-3-Oasis-v1-OAS-8B) * [Casual-Autopsy/SOVL-MopeyMule-8B](https://huggingface.co/Casual-Autopsy/SOVL-MopeyMule-8B) * [Casual-Autopsy/MopeyMule-Blackroot-8B](https://huggingface.co/Casual-Autopsy/MopeyMule-Blackroot-8B) ## Secret Sauce The following YAML configurations were used to produce this model: ### Umbral-v3-1 ```yaml slices: - sources: - model: Sao10K/L3-8B-Stheno-v3.2 layer_range: [0, 32] parameters: weight: 0.65 - model: Casual-Autopsy/SOVL-MopeyMule-8B layer_range: [0, 32] parameters: weight: 0.25 - model: Casual-Autopsy/MopeyMule-Blackroot-8B layer_range: [0, 32] parameters: weight: 0.1 merge_method: task_arithmetic base_model: Sao10K/L3-8B-Stheno-v3.2 normalize: False dtype: bfloat16 ``` ### Umbral-v3-2 ```yaml slices: - sources: - model: Hastagaras/Jamet-8B-L3-MK.V-Blackroot layer_range: [0, 32] parameters: weight: 0.75 - model: Casual-Autopsy/SOVL-MopeyMule-8B layer_range: [0, 32] parameters: weight: 0.15 - model: Casual-Autopsy/MopeyMule-Blackroot-8B layer_range: [0, 32] parameters: weight: 0.1 merge_method: task_arithmetic base_model: Hastagaras/Jamet-8B-L3-MK.V-Blackroot normalize: False dtype: bfloat16 ``` ### Umbral-v3-3 ```yaml slices: - sources: - model: grimjim/Llama-3-Oasis-v1-OAS-8B layer_range: [0, 32] parameters: weight: 0.55 - model: Casual-Autopsy/SOVL-MopeyMule-8B layer_range: [0, 32] parameters: weight: 0.35 - model: Casual-Autopsy/MopeyMule-Blackroot-8B layer_range: [0, 32] parameters: weight: 0.1 merge_method: task_arithmetic base_model: grimjim/Llama-3-Oasis-v1-OAS-8B normalize: False dtype: bfloat16 ``` ### L3-Umbral-Mind-RP-v0.3-8B ```yaml models: - model: Casual-Autopsy/Umbral-v3-1+ResplendentAI/Theory_of_Mind_Llama3 - model: Casual-Autopsy/Umbral-v3-2+ResplendentAI/Smarts_Llama3 - model: Casual-Autopsy/Umbral-v3-3+ResplendentAI/RP_Format_QuoteAsterisk_Llama3 merge_method: model_stock base_model: Casual-Autopsy/Umbral-v3-1 dtype: bfloat16 ``` ### L3-Umbral-Mind-RP-v1.0-8B ```yaml slices: - sources: - model: Casual-Autopsy/L3-Umbral-Mind-RP-v0.3-8B layer_range: [0, 32] - model: Casual-Autopsy/Llama-3-Mopeyfied-Psychology-8B layer_range: [0, 32] parameters: weight: 0.14 - model: Sao10K/L3-8B-Stheno-v3.2 layer_range: [0, 32] parameters: weight: 0.03 - model: Hastagaras/Halu-8B-Llama3-Blackroot layer_range: [0, 32] parameters: weight: 0.03 merge_method: task_arithmetic base_model: Casual-Autopsy/L3-Umbral-Mind-RP-v0.3-8B dtype: bfloat16 ```