Instructions to use JC1DA/Qwopus3.8-27B-Flash-INT4-W4A16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use JC1DA/Qwopus3.8-27B-Flash-INT4-W4A16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="JC1DA/Qwopus3.8-27B-Flash-INT4-W4A16")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("JC1DA/Qwopus3.8-27B-Flash-INT4-W4A16")
model = AutoModelForMultimodalLM.from_pretrained("JC1DA/Qwopus3.8-27B-Flash-INT4-W4A16", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use JC1DA/Qwopus3.8-27B-Flash-INT4-W4A16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "JC1DA/Qwopus3.8-27B-Flash-INT4-W4A16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JC1DA/Qwopus3.8-27B-Flash-INT4-W4A16",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/JC1DA/Qwopus3.8-27B-Flash-INT4-W4A16

SGLang

How to use JC1DA/Qwopus3.8-27B-Flash-INT4-W4A16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "JC1DA/Qwopus3.8-27B-Flash-INT4-W4A16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JC1DA/Qwopus3.8-27B-Flash-INT4-W4A16",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "JC1DA/Qwopus3.8-27B-Flash-INT4-W4A16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JC1DA/Qwopus3.8-27B-Flash-INT4-W4A16",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use JC1DA/Qwopus3.8-27B-Flash-INT4-W4A16 with Docker Model Runner:
```
docker model run hf.co/JC1DA/Qwopus3.8-27B-Flash-INT4-W4A16
```

🪐 Qwopus3.8-27B-Flash (W4A16)

Qwopus3.8-27B-Flash W4A16 is a 4-bit weight / 16-bit activation quantized version of Qwopus3.8-27B-Flash, using AutoRound with GPTQ group size 128. It preserves the Flash model's efficiency characteristics while reducing memory footprint.

W4A16 Quantization · ~65% Smaller · Same Efficiency Characteristics · FP16 Parity on ARC-Easy

📊 Benchmark Results

ARC-Easy

Model	Accuracy	Acc Norm
Qwopus3.8-27B-Flash (FP16)	83.71%	77.06%
Qwopus3.8-27B-Flash (W4A16)	84.13%	78.54%

The W4A16 quantized version matches or slightly exceeds the FP16 original on ARC-Easy.

Additional benchmarks (MMLU, HumanEval, BBH, WikiText2 PPL) are in progress. Results will be added to this README when complete.

💾 Quantization Details

Parameter	Value
Algorithm	AutoRound + GPTQ
Weight bits	4
Activation bits	16
Group size	128
Version	AutoRound 0.12.3

Model size: ~19GB (vs ~54GB FP16) — 65% reduction

The quantization preserves the following layers in FP16 for quality:

Linear attention projection matrices (in_proj_a, in_proj_b) across all 64 layers
MTP (Multi-Token Prediction) module
Visual encoder

🚀 Usage

Works with any framework supporting GPTQ/AutoRound checkpoints:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Jackrong/Qwopus3.8-27B-Flash-W4A16/Jackrong_Qwopus3.8-27B-Flash-w4g128"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype="auto"
)

Or via vLLM for high-throughput serving:

vllm serve Jackrong/Qwopus3.8-27B-Flash-W4A16/Jackrong_Qwopus3.8-27B-Flash-w4g128 \
    --dtype auto --tensor-parallel-size 1

🎯 Recommended Use Cases

Local MTP speculative decoding where decoder throughput and draft acceptance matter.
Long-running agent workflows with repeated tool calls and iterative edits.
Resource-conscious workloads where wall-clock time and generated-token cost are operational constraints.
Memory-constrained environments where the 65% size reduction enables local deployment on smaller GPUs.

⚠️ Limitations

Accuracy–Efficiency Trade-off: The underlying Flash fine-tuning prioritizes practical inference efficiency. On the reported 2,500-question MMLU-Pro mixed set, Qwopus scores 91.28% versus 92.73% for the Qwen3.8 base comparison.
Reasoning Stability: Although the long-tail behavior is improved in the reported evaluation, edge cases may still exhibit reasoning drift, loops, or inefficient trajectories.
Agent Dependence: Tool-use quality and task completion remain dependent on the prompt, tool environment, orchestration layer, and feedback loop.
Experimental Release: This is an independent experimental release for research, local evaluation, and technical exploration.

🙏 Acknowledgements

Special thanks to:

Qwen for the Qwen3.8-27B base-model foundation.
Jackrong for the Qwopus3.8-27B-Flash fine-tuning.
Unsloth for practical, efficient large-model fine-tuning tooling.
The open-source community for training tools, evaluation methods, and technical discussion.

Downloads last month: -

Safetensors

Model size

6B params

Tensor type

I32

BF16

F16

Model tree for JC1DA/Qwopus3.8-27B-Flash-INT4-W4A16

Base model

Qwen/Qwen3.8-27B

Finetuned

Jackrong/Qwopus3.8-27B-Flash

Quantized

(17)

this model