Instructions to use genevera/Muse-Glimmer-30B-heretic-exl3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use genevera/Muse-Glimmer-30B-heretic-exl3 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="genevera/Muse-Glimmer-30B-heretic-exl3")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("genevera/Muse-Glimmer-30B-heretic-exl3")
model = AutoModelForMultimodalLM.from_pretrained("genevera/Muse-Glimmer-30B-heretic-exl3", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use genevera/Muse-Glimmer-30B-heretic-exl3 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "genevera/Muse-Glimmer-30B-heretic-exl3"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "genevera/Muse-Glimmer-30B-heretic-exl3",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/genevera/Muse-Glimmer-30B-heretic-exl3

SGLang

How to use genevera/Muse-Glimmer-30B-heretic-exl3 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "genevera/Muse-Glimmer-30B-heretic-exl3" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "genevera/Muse-Glimmer-30B-heretic-exl3",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "genevera/Muse-Glimmer-30B-heretic-exl3" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "genevera/Muse-Glimmer-30B-heretic-exl3",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use genevera/Muse-Glimmer-30B-heretic-exl3 with Docker Model Runner:
```
docker model run hf.co/genevera/Muse-Glimmer-30B-heretic-exl3
```

Muse-Glimmer-30B-heretic EXL3 (6.0 bpw)

EXL3 quantized build of coder3101/Muse-Glimmer-30B-heretic, an ARA-abliterated (uncensored) version of meta-models/Muse-Glimmer-30B, converted with exllamav3 v1.4.2.

Model details

Parameter	Value
Architecture	`MuseGlimmerForConditionalGeneration`
Model type	`muse_glimmer` (VLM — image-text-to-text)
Hidden layers	52 (39 sliding-attention + 13 full-attention, interleaved every 4th layer)
Hidden size	6656
KV heads	2 (GQA)
Head dim	128
Vocab size	202,048
Context (native)	131,072 tokens
Sliding window	2,048

Quantization details

Parameter	Value
Quant method	exl3
exllamav3 version	1.4.2
Decoder bits per weight	6.0
Head bits per weight	6.0
Codebook	mul1
Calibration rows	250
Calibration seq length	2048
Output scales	always
Shards	4
Total size	~24.6 GB

Usage — TabbyAPI

hf download genevera/Muse-Glimmer-30B-heretic-exl3 --local-dir models/Muse-Glimmer-30B-heretic-exl3

# config.yml
model:
  model_name: Muse-Glimmer-30B-heretic-exl3
  max_seq_len: 131072
  cache_mode: 8,8
  vision: true
  reasoning: true

Multimodal (image) input

The model supports image input via the OAI chat completions API with image_url content blocks:

{
  "messages": [{
    "role": "user",
    "content": [
      {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}},
      {"type": "text", "text": "Describe this image."}
    ]
  }]
}

Speculative decoding

Pair with turboderp/Muse-Glimmer-30B-assistant-exl3 (DFlash draft model) for 2-3× speedup:

draft_model:
  draft_mode: model
  draft_model_name: Muse-Glimmer-30B-assistant-exl3-6.0bpw

Heretic abliteration

This model is a decensored version of meta-models/Muse-Glimmer-30B, made using Heretic v1.2.0 with the Arbitrary-Rank Ablation (ARA) method (with row-norm preservation).

Parameter	Value
start_layer_index	10
end_layer_index	50
preserve_good_behavior_weight	0.5826
steer_bad_behavior_weight	0.0008
overcorrect_relative_weight	0.9762
neighbor_count	5