Instructions to use eevqd/Crystal-31B-RL-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use eevqd/Crystal-31B-RL-NVFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="eevqd/Crystal-31B-RL-NVFP4")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForCausalLM

processor = AutoProcessor.from_pretrained("eevqd/Crystal-31B-RL-NVFP4")
model = AutoModelForCausalLM.from_pretrained("eevqd/Crystal-31B-RL-NVFP4", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use eevqd/Crystal-31B-RL-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "eevqd/Crystal-31B-RL-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "eevqd/Crystal-31B-RL-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/eevqd/Crystal-31B-RL-NVFP4

SGLang

How to use eevqd/Crystal-31B-RL-NVFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "eevqd/Crystal-31B-RL-NVFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "eevqd/Crystal-31B-RL-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "eevqd/Crystal-31B-RL-NVFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "eevqd/Crystal-31B-RL-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use eevqd/Crystal-31B-RL-NVFP4 with Docker Model Runner:
```
docker model run hf.co/eevqd/Crystal-31B-RL-NVFP4
```

Crystal-31B-RL NVFP4

NVFP4 (W4A4) quantization of hiwaifu-research/Crystal-31B-RL, a dense Gemma 4 31B role-play model. All credit for the model goes to its authors; this repo only changes the precision.

Format: NVIDIA ModelOpt 0.47 export, nvfp4_default with kv_fp8_cast (FP8 KV cache, no KV scales). Every Linear of the text stack is NVFP4; embeddings and norms stay bf16. Weight block scales chosen by MSE search (mse).
Text only: the vision tower is removed and the config is Gemma4ForCausalLM (text config as in Google's release). About 20 GB on disk instead of 62 GB.
Calibration: 256 multi-turn role-play conversations (mostly Russian, up to 16k tokens each) plus short service prompts. The calibration data is not released.
Chat template and generation_config.json (temperature 0.8, top-p 0.95, top-k 64) are Crystal's own.

Serving with SGLang

Tested with SGLang 0.5.20 on RTX PRO Blackwell cards. A text-only Gemma 4 checkpoint needs --language-model-only, and SGLang 0.5.20 additionally needs a two-line patch to accept Gemma4ForCausalLM with that flag (allowlist the flag and pick text_config in the text class).

python3 -m sglang.launch_server --model-path <this repo> --language-model-only \
  --context-length 32768 --kv-cache-dtype fp8_e4m3 --swa-full-tokens-ratio 0.15

Use the long role-play system prompt and sampling advice from the original model card.

License

Apache 2.0, as the original model and Gemma 4.

Downloads last month: 57

Safetensors

Model size

16B params

Tensor type

BF16

Model tree for eevqd/Crystal-31B-RL-NVFP4

Base model

google/gemma-4-31B

Finetuned

google/gemma-4-31B-it

Finetuned

hiwaifu-research/Crystal-31B-RL

Quantized

(3)

this model