Instructions to use KellHect/Ornith-1.5-9B-Abliterated-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use KellHect/Ornith-1.5-9B-Abliterated-FP8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="KellHect/Ornith-1.5-9B-Abliterated-FP8")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("KellHect/Ornith-1.5-9B-Abliterated-FP8")
model = AutoModelForMultimodalLM.from_pretrained("KellHect/Ornith-1.5-9B-Abliterated-FP8", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use KellHect/Ornith-1.5-9B-Abliterated-FP8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "KellHect/Ornith-1.5-9B-Abliterated-FP8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KellHect/Ornith-1.5-9B-Abliterated-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/KellHect/Ornith-1.5-9B-Abliterated-FP8

SGLang

How to use KellHect/Ornith-1.5-9B-Abliterated-FP8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "KellHect/Ornith-1.5-9B-Abliterated-FP8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KellHect/Ornith-1.5-9B-Abliterated-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "KellHect/Ornith-1.5-9B-Abliterated-FP8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KellHect/Ornith-1.5-9B-Abliterated-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use KellHect/Ornith-1.5-9B-Abliterated-FP8 with Docker Model Runner:
```
docker model run hf.co/KellHect/Ornith-1.5-9B-Abliterated-FP8
```

Ornith-1.5-9B-Abliterated-FP8

An abliterated derivative of ornith-ai/Ornith-1.5-9B at revision 489cb97981b8654bcfcf30ce1f94ed1b62e07b53, released as compressed-tensors FP8_DYNAMIC W8A8.

Inference-oriented; use the BF16 repository for training.

What changed

The language residual projections were modified. The vision tower, MTP block, tokenizer, chat template, and multimodal processors are preserved. The pipeline used complementary SVD and LEACE refusal-direction surgery, iterative re-probing, targeted security-prompt refinement, and weight-space blending.

Validation

The FP8 release quantizes 248 language projection tensors and preserves 527 protected tensors. Reconstruction validation measured a maximum relative Frobenius error of 0.026736 and minimum cosine similarity of 0.999635. See validation_report.json for per-tensor details.

The accepted BF16 checkpoint scored 0/12 refusal flags during refinement and 0/24 on a separate held-out first-token refusal screen. The held-out mean refusal-token probability was 1.18e-5.

Coding evaluation was intentionally deferred before this release. These numbers are structural and refusal-screen diagnostics, not a claim of benchmark parity with the base model. A one-task generation smoke test is not reported as an evaluation result.

Usage

pip install torch transformers accelerate compressed-tensors

import torch
from transformers import AutoModelForMultimodalLM, AutoProcessor

model_id = "KellHect/Ornith-1.5-9B-Abliterated-FP8"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForMultimodalLM.from_pretrained(
    model_id, dtype=torch.bfloat16, device_map="auto"
)

messages = [{"role": "user", "content": "Explain this code and identify the bug."}]
inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True,
    return_dict=True, return_tensors="pt",
).to(model.device)
output = model.generate(**inputs, max_new_tokens=512)
print(processor.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

FP8 users need mutually compatible versions of Torch, Transformers, and compressed-tensors. If the FP8 loader is unavailable on a platform, use the BF16 release.

Research context

This model has had refusal behavior deliberately reduced. It may comply with requests that the base model rejects. Users are responsible for deployment, access control, generated content, and compliance with applicable law.

Credits

Ornith AI for the base model.
OBLITERATUS for the abliteration research and implementation lineage.
Arditi et al. for refusal-direction research and Belrose et al. for LEACE.

License

MIT. See LICENSE.

Downloads last month: -

Safetensors

Model size

10B params

Tensor type

BF16

F8_E4M3

Model tree for KellHect/Ornith-1.5-9B-Abliterated-FP8

Base model

ornith-ai/Ornith-1.5-9B

Quantized

(58)

this model