Cas-Archive/L3-Penumbral-Mind-RP-8B

merge

This is a merge of pre-trained language models created using mergekit.

Merge Details

This version uses Halu Blackroot which has fewer GPTisms than Jamet Blackroot as well as being more easily tamed in terms of leaning towards romantic/NSFW scenarios compared to Jamet Blackroot, making this model more suitable for slow-burns.

Note: Personally, I don't think this model feels quite right, MopeyMule doesn't seem to synergize as well with Halu as I had hoped so it still feels a bit too positive, but I'll leave this up for people that prefer Halu over Jamet

The goal of this merge was to make an RP model better suited for role-plays with heavy themes such as but not limited to:

• Mental illness
• Self-harm
• Trauma
• Suicide

I hated how RP models tended to be overly positive and hopeful with role-plays involving such themes, but thanks to failspy/Llama-3-8B-Instruct-MopeyMule this problem has been lessened considerably.

I haven't done extensive testing yet, but the bit of testing I did do has shown noticeably good results in lessening how positive the bot is in its writing style.

If you're an enjoyer of savior/reverse savior type role-plays like myself, then this bot is for you.

Merge Method

This model was merged using several Task Arithmetic merges and then tied together with a Model Stock merge.

Models Merged

The following models were included in the merge:

• Casual-Autopsy/Umbral-1

  • Sao10K/L3-8B-Stheno-v3.2
  • failspy/Llama-3-8B-Instruct-MopeyMule

• Casual-Autopsy/Penumbral

  • Hastagaras/Halu-8B-Llama3-Blackroot
  • failspy/Llama-3-8B-Instruct-MopeyMule

• Casual-Autopsy/Umbral-3

  • grimjim/Llama-3-Oasis-v1-OAS-8B
  • failspy/Llama-3-8B-Instruct-MopeyMule

Secret Sauce

The following YAML configurations were used to produce this model:

Umbral-1

slices:
- sources:
  - model: Sao10K/L3-8B-Stheno-v3.2
    layer_range: [0, 32]
  - model: failspy/Llama-3-8B-Instruct-MopeyMule
    layer_range: [0, 32]
    parameters:
      weight: 0.25
merge_method: task_arithmetic
base_model: Sao10K/L3-8B-Stheno-v3.2
dtype: bfloat16

Penumbral

slices:
- sources:
  - model: Hastagaras/Halu-8B-Llama3-Blackroot
    layer_range: [0, 32]
  - model: failspy/Llama-3-8B-Instruct-MopeyMule
    layer_range: [0, 32]
    parameters:
      weight: 0.15
merge_method: task_arithmetic
base_model: Hastagaras/Halu-8B-Llama3-Blackroot
dtype: bfloat16

Umbral-3

slices:
- sources:
  - model: grimjim/Llama-3-Oasis-v1-OAS-8B
    layer_range: [0, 32]
  - model: failspy/Llama-3-8B-Instruct-MopeyMule
    layer_range: [0, 32]
    parameters:
      weight: 0.35
merge_method: task_arithmetic
base_model: grimjim/Llama-3-Oasis-v1-OAS-8B
dtype: bfloat16

Penumbral-Mind-RP-8B

models:
  - model: Casual-Autopsy/Umbral-1
  - model: Casual-Autopsy/Penumbral
  - model: Casual-Autopsy/Umbral-3
merge_method: model_stock
base_model: Casual-Autopsy/Umbral-1
dtype: bfloat16