DELLA-Merging: Reducing Interference in Model Merging through Magnitude-Based Sampling
Paper • 2406.11617 • Published • 11
How to use DarkArtsForge/Morbid-Miasma-24B with Transformers:
# Use a pipeline as a high-level helper
from transformers import pipeline
pipe = pipeline("text-generation", model="DarkArtsForge/Morbid-Miasma-24B")
messages = [
{"role": "user", "content": "Who are you?"},
]
pipe(messages) # Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("DarkArtsForge/Morbid-Miasma-24B")
model = AutoModelForCausalLM.from_pretrained("DarkArtsForge/Morbid-Miasma-24B", device_map="auto")
messages = [
{"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
tokenize=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))How to use DarkArtsForge/Morbid-Miasma-24B with vLLM:
# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DarkArtsForge/Morbid-Miasma-24B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "DarkArtsForge/Morbid-Miasma-24B",
"messages": [
{
"role": "user",
"content": "What is the capital of France?"
}
]
}'docker model run hf.co/DarkArtsForge/Morbid-Miasma-24B
How to use DarkArtsForge/Morbid-Miasma-24B with SGLang:
# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
--model-path "DarkArtsForge/Morbid-Miasma-24B" \
--host 0.0.0.0 \
--port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "DarkArtsForge/Morbid-Miasma-24B",
"messages": [
{
"role": "user",
"content": "What is the capital of France?"
}
]
}'docker run --gpus all \
--shm-size 32g \
-p 30000:30000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HF_TOKEN=<secret>" \
--ipc=host \
lmsysorg/sglang:latest \
python3 -m sglang.launch_server \
--model-path "DarkArtsForge/Morbid-Miasma-24B" \
--host 0.0.0.0 \
--port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "DarkArtsForge/Morbid-Miasma-24B",
"messages": [
{
"role": "user",
"content": "What is the capital of France?"
}
]
}'How to use DarkArtsForge/Morbid-Miasma-24B with Docker Model Runner:
docker model run hf.co/DarkArtsForge/Morbid-Miasma-24B
⚠️ Warning: This model can produce narratives and RP that contain violent and graphic erotic content. Adjust your system prompt accordingly, and use Mistral Tekken chat template.
This is a fully uncensored merge of pre-trained language models created using mergekit. The merge is very dark, intelligent, and creative, scoring 16103 at Q0 Bench.
This model was merged using the della merge method using mistralai/Magistral-Small-2509 as a base.
The following models were included in the merge:
architecture: MistralForCausalLM
models:
- model: B:\24B\DarkArtsForge--Asmodeus-24B-v1
parameters:
weight:
- filter: "lm_head"
value: 0.0
- filter: "embed_tokens"
value: 0.0
- value: 0.125
density: 0.5
epsilon: 0.25
- model: B:\24B\DarkArtsForge--Asmodeus-24B-v2
parameters:
weight:
- filter: "lm_head"
value: 0.0
- filter: "embed_tokens"
value: 0.0
- value: 0.125
density: 0.5
epsilon: 0.25
- model: B:\24B\Morax-24B-v2
parameters:
weight:
- filter: "lm_head"
value: 1.0
- filter: "embed_tokens"
value: 0.0
- value: 0.125
density: 0.75
epsilon: 0.25
- model: B:\24B\DarkArtsForge--Paimon-24B-v3
parameters:
weight:
- filter: "lm_head"
value: 0.0
- filter: "embed_tokens"
value: 0.0
- value: 0.125
density: 0.75
epsilon: 0.25
- model: B:\24B\DarkArtsForge--Zepar-24B-v1
parameters:
weight:
- filter: "lm_head"
value: 0.0
- filter: "embed_tokens"
value: 0.0
- value: 0.125
density: 0.75
epsilon: 0.25
- model: B:\24B\DarkArtsForge--Magistaroth-24B-v1
parameters:
weight:
- filter: "lm_head"
value: 0.0
- filter: "embed_tokens"
value: 0.0
- value: 0.125
density: 0.75
epsilon: 0.25
- model: B:\24B\!models--Casual-Autopsy--Maginum-Cydoms-24B
parameters:
weight:
- filter: "lm_head"
value: 0.0
- filter: "embed_tokens"
value: 0.0
- value: 0.125
density: 0.5
epsilon: 0.25
- model: B:\24B\Naphula--GhostFace-24B-v1
parameters:
weight:
- filter: "lm_head"
value: 0.0
- filter: "embed_tokens"
value: 0.0
- value: 0.125
density: 0.75
epsilon: 0.25
- model: B:\24B\Naphula--Slimaki-Tavern-24B-v1.3
parameters:
weight:
- filter: "lm_head"
value: 0.0
- filter: "embed_tokens"
value: 0.0
- value: 0.125
density: 0.75
epsilon: 0.25
- model: B:\24B\Naphula--Slimaki-24B-v1.2
parameters:
weight:
- filter: "lm_head"
value: 0.0
- filter: "embed_tokens"
value: 0.0
- value: 0.125
density: 0.75
epsilon: 0.25
- model: B:\24B\Naphula--Slimaki-24B-v1
parameters:
weight:
- filter: "lm_head"
value: 0.0
- filter: "embed_tokens"
value: 0.0
- value: 0.125
density: 0.75
epsilon: 0.25
- model: B:\24B\sophosympatheia--Magistry-24B-v1.0
parameters:
weight:
- filter: "lm_head"
value: 0.0
- filter: "embed_tokens"
value: 0.0
- value: 0.125
density: 0.5
epsilon: 0.25
- model: B:\24B\!models--TheDrummer--Precog-24B-v1
parameters:
weight:
- filter: "lm_head"
value: 0.0
- filter: "embed_tokens"
value: 0.0
- value: 0.125
density: 0.9
epsilon: 0.09
merge_method: della
base_model: A:\LLM\.cache\24B\!models--mistralai--Magistral-Small-2509\textonly
parameters:
lambda: 1.0
normalize: false
int8_mask: false
rescale: true
dtype: float32
out_dtype: bfloat16
tokenizer:
source: base
chat_template: auto
name: ☠️ Morbid Miasma 24B
[DELLA Audit] Layer: model.layers.33.mlp.down_proj.weight | Lambda=1.00
[BASE] textonly
!models--Casual-Autopsy--Maginum-Cydoms-24B : ███ 6.5% (W:0.12 D:0.50 N:2.59 E:0.25)
!models--TheDrummer--Precog-24B-v1 : 0.9% (W:0.12 D:0.90 N:0.37 E:0.09)
DarkArtsForge--Asmodeus-24B-v1 : ███████ 15.5% (W:0.12 D:0.50 N:6.16 E:0.25)
DarkArtsForge--Asmodeus-24B-v2 : ██████ 13.5% (W:0.12 D:0.50 N:5.37 E:0.25)
DarkArtsForge--Magistaroth-24B-v1 : 0.8% (W:0.12 D:0.75 N:0.34 E:0.25)
DarkArtsForge--Paimon-24B-v3 : ████████ 16.5% (W:0.12 D:0.75 N:6.56 E:0.25)
DarkArtsForge--Zepar-24B-v1 : ███ 6.4% (W:0.12 D:0.75 N:2.55 E:0.25)
Morax-24B-v2 : ██ 4.8% (W:0.12 D:0.75 N:1.91 E:0.25)
Naphula--GhostFace-24B-v1 : ██ 5.3% (W:0.12 D:0.75 N:2.10 E:0.25)
Naphula--Slimaki-24B-v1 : ████ 8.3% (W:0.12 D:0.75 N:3.31 E:0.25)
Naphula--Slimaki-24B-v1.2 : ████ 8.8% (W:0.12 D:0.75 N:3.52 E:0.25)
Naphula--Slimaki-Tavern-24B-v1.3 : ███ 7.0% (W:0.12 D:0.75 N:2.77 E:0.25)
sophosympatheia--Magistry-24B-v1.0 : ██ 5.6% (W:0.12 D:0.50 N:2.24 E:0.25)