Instructions to use com-kotobalabs/Qwen3.8-27B-whitehat-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use com-kotobalabs/Qwen3.8-27B-whitehat-NVFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="com-kotobalabs/Qwen3.8-27B-whitehat-NVFP4")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("com-kotobalabs/Qwen3.8-27B-whitehat-NVFP4")
model = AutoModelForMultimodalLM.from_pretrained("com-kotobalabs/Qwen3.8-27B-whitehat-NVFP4", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use com-kotobalabs/Qwen3.8-27B-whitehat-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "com-kotobalabs/Qwen3.8-27B-whitehat-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "com-kotobalabs/Qwen3.8-27B-whitehat-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/com-kotobalabs/Qwen3.8-27B-whitehat-NVFP4

SGLang

How to use com-kotobalabs/Qwen3.8-27B-whitehat-NVFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "com-kotobalabs/Qwen3.8-27B-whitehat-NVFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "com-kotobalabs/Qwen3.8-27B-whitehat-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "com-kotobalabs/Qwen3.8-27B-whitehat-NVFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "com-kotobalabs/Qwen3.8-27B-whitehat-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use com-kotobalabs/Qwen3.8-27B-whitehat-NVFP4 with Docker Model Runner:
```
docker model run hf.co/com-kotobalabs/Qwen3.8-27B-whitehat-NVFP4
```

Access — authorized security research only

This is a dual-use, abliterated ("uncensored") model. Access is reviewed and granted for authorized security research. See the acceptable-use terms below.

By requesting access you confirm that you will use this model ONLY within an authorized scope — your own systems, sanctioned engagements, CTFs, and model-safety research — and in compliance with applicable law. You will NOT use it to target systems, data, or people you are not authorized to test, nor to produce weapons, malware, or fraud for real-world harm. Removing a refusal filter does not remove your legal and ethical obligations; you accept sole responsibility for your use.

Qwen3.8-27B-whitehat-NVFP4

An abliterated (refusal-direction-ablated) derivative of Qwen/Qwen3.8-27B, quantized to NVFP4 for Blackwell-class GPUs — it runs on a single NVIDIA RTX PRO 6000. Built for authorized security research and red-teaming: evaluating model safety, generating adversarial test cases, and studying refusal behavior, where a model that does not reflexively decline is a research instrument.

Dual-use artifact. Abliteration removes the model's tendency to refuse; it adds no new capability but removes a safety layer, so it can produce content the base model would decline. Use it only within an authorized scope (your own systems, sanctioned engagements, CTFs, safety research). You are responsible for how you use it. The procedure and calibration prompts are public and reproducible — documented below.

What was changed

Qwen3.8-27B is a dense multimodal model (Qwen3_5ForConditionalGeneration, qwen3_5): 64 decoder layers with hybrid attention (linear + periodic full), dense MLP, hidden_size 5120, plus a vision tower.

Method: refusal-direction ablation (Arditi et al., 2024). Estimate the residual-stream direction that most separates refusal-eliciting from benign prompts, then orthogonalize every residual-stream writer against it — self_attn.o_proj (full-attn layers), linear_attn.out_proj (linear-attn layers), mlp.down_proj, and embed_tokens: W' = W − r·(rᵀW).
Untouched: the vision tower (visual.*), and lm_head. Vision behavior is unchanged.
Quantization: NVFP4 (Blackwell-native FP4) via llm-compressor → compressed-tensors. Vision, embeddings, lm_head, and the linear-attention conv1d/in_proj_a/in_proj_b tensors stay higher-precision.

Serving — one RTX PRO 6000

NVFP4 weights are ~18 GB, so they fit a single 96 GB RTX PRO 6000 with ~76 GB free for the KV cache (the base supports 262k context). (bf16, ~56 GB, also fits one card — NVFP4 buys Blackwell FP4 throughput + long-context headroom.)

Weights are compressed-tensors NVFP4 (format: nvfp4-pack-quantized, W4A4), auto-detected from config.json — no special flag needed on recent vLLM/SGLang.

# vLLM (single RTX PRO 6000)
vllm serve com-kotobalabs/Qwen3.8-27B-whitehat-NVFP4 --trust-remote-code

# SGLang
python -m sglang.launch_server \
  --model-path com-kotobalabs/Qwen3.8-27B-whitehat-NVFP4 \
  --quantization compressed-tensors --tp 1 --trust-remote-code

No GGUF (yet)

llama.cpp can't convert this architecture: qwen3_5 hybrid linear-attention tensors (ssm_conv1d kernel dim + in_proj_a/in_proj_b expansion) are unhandled — ggml-org/llama.cpp#27019. NVFP4 for SGLang/vLLM is the supported path until that lands.

Validation (measured)

Ablation coverage: 129/129 residual-stream writers orthogonalized (64 attention output projections + 64 mlp.down_proj + embed_tokens).
Refusal rate (do_sample=False, on refusal-eliciting prompts base Qwen refuses ~universally): 0% in bf16 and 0% after NVFP4 — i.e. the abliteration survives quantization. This measures refusal removal, not general capability; run your own task-level evals before relying on it.
NVFP4 calibration: 256 general-instruction samples, W4A4.

Throughput / concurrency — RTX PRO 6000 (measured)

Measured 2026-09-18 on a single NVIDIA RTX PRO 6000 Blackwell (96 GB) via a Hugging Face Inference Endpoint, vLLM serving the NVFP4 weights (maxNumSeqs=64, max-model-len=32768, greedy, 256-token completions, streaming). Reproduce with bench/benchmark.py; raw data in bench/results.json.

concurrency	output tok/s	req/s	E2E p50 (256-tok request)
1	47	0.18	5.4 s
8	329	1.3	6.0 s
16	641	2.5	6.3 s
32	897	3.7	6.9 s
48	1,556	6.1	7.9 s
64	1,636	7.7	8.3 s
96	1,433	6.5	9.3 s
128	1,908	7.5	17.1 s

Peak throughput ≈ 1.9k output tok/s (concurrency 128). The batch fills at maxNumSeqs=64 (~1.6k tok/s, E2E 8.3 s); beyond that, requests queue and per-request latency roughly doubles (17 s at 128).
Max concurrency tested: 128, with zero errors — this is a sweep bound, not a failure ceiling; more is servable by raising maxNumSeqs at the cost of latency.
Interactive range: ≤ 64 concurrent keeps a full 256-token completion under ~8.3 s.
Note: fine-grained TTFT/ITL were not reliably captured this run (the --reasoning-parser qwen3 stream splits reasoning vs. content); wall-clock E2E latency is reported instead. Throughput/req-counts are from usage.

Reproduce

modal_pipeline.py (download → abliterate → NVFP4 → publish; smoke for the refusal check) and bench/ (endpoint deploy + concurrency sweep). Calibration prompt sets in data/harmful.txt and data/harmless.txt.