Instructions to use HarimxChoi/WarpQuant-Qwen3.8-27B-R16E4H4-Text with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use HarimxChoi/WarpQuant-Qwen3.8-27B-R16E4H4-Text with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="HarimxChoi/WarpQuant-Qwen3.8-27B-R16E4H4-Text")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("HarimxChoi/WarpQuant-Qwen3.8-27B-R16E4H4-Text")
model = AutoModelForCausalLM.from_pretrained("HarimxChoi/WarpQuant-Qwen3.8-27B-R16E4H4-Text", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use HarimxChoi/WarpQuant-Qwen3.8-27B-R16E4H4-Text with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "HarimxChoi/WarpQuant-Qwen3.8-27B-R16E4H4-Text"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HarimxChoi/WarpQuant-Qwen3.8-27B-R16E4H4-Text",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/HarimxChoi/WarpQuant-Qwen3.8-27B-R16E4H4-Text

SGLang

How to use HarimxChoi/WarpQuant-Qwen3.8-27B-R16E4H4-Text with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "HarimxChoi/WarpQuant-Qwen3.8-27B-R16E4H4-Text" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HarimxChoi/WarpQuant-Qwen3.8-27B-R16E4H4-Text",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "HarimxChoi/WarpQuant-Qwen3.8-27B-R16E4H4-Text" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HarimxChoi/WarpQuant-Qwen3.8-27B-R16E4H4-Text",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use HarimxChoi/WarpQuant-Qwen3.8-27B-R16E4H4-Text with Docker Model Runner:
```
docker model run hf.co/HarimxChoi/WarpQuant-Qwen3.8-27B-R16E4H4-Text
```

WarpQuant Qwen3.8-27B R16E4H4 Text

This is the text-only Qwen3.8-27B checkpoint produced with WarpQuant. It contains the causal language model, tokenizer, and chat template. The vision tower and MTP layer are not included.

Technical report · Code · Multimodal model

Format

WarpQuant applies a signed Hadamard rotation and 3-bit group quantization to 400 projection tensors. An Output-Fisher score selects a 0.05 bpw weak-column recovery budget in the original activation coordinates. The token embedding and output head use grouped INT4.

Scope	Value
Text parameters	26,895,998,464
Analytical payload	3.616506953 bpw
Analytical payload size	11.323668 GiB
Calibration	512 sequences × 512 tokens
Group / tile size	128 / 128

The first release checkpoint stores reconstructed tensors in BF16 for direct Transformers evaluation. Its physical safetensor size is therefore larger than the analytical packed payload.

The text-only export was compared against the source conditional-generation checkpoint on the same tokenized prompt. The last-token logits matched exactly (max_abs_logit_error = 0, mean_abs_logit_error = 0, identical top-1), followed by a deterministic generation smoke test.

Evaluation

Format	Text bpw	Payload	WT2 PPL ↓	ARC-299 ↑	MMLU-13,943 ↑	Commonsense ↑	GSM8K-500 flex ↑
BF16	16.00	50.11 GiB	6.9548	52.17	43.07	79.23	70.40
Q4_K_M	4.92	15.41 GiB	6.9656	50.84	42.90	79.23	75.20
IQ3_S	3.6940	11.57 GiB	7.1820	52.17	42.97	78.83	59.40
WarpQuant R16E4H4	3.6165	11.32 GiB	7.4737	56.86	42.72	78.83	61.00

Commonsense is the macro average of fixed 1,000-example HellaSwag, WinoGrande, and PIQA screens. GSM8K uses the same first 500 examples, 5-shot prompts, and flexible-extract accuracy for all four models.

Use

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "HarimxChoi/WarpQuant-Qwen3.8-27B-R16E4H4-Text"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

Citation

@misc{choi2026warpquant,
  author = {Harim Choi},
  title = {WarpQuant: Dual-Domain LLM Quantization via Hadamard Rotation and Output-Fisher Sensitivity},
  year = {2026},
  url = {https://github.com/HarimxChoi/WarpQuant}
}