Instructions to use 8times4/apex-flash-1-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use 8times4/apex-flash-1-FP8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="8times4/apex-flash-1-FP8")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("8times4/apex-flash-1-FP8")
model = AutoModelForMultimodalLM.from_pretrained("8times4/apex-flash-1-FP8", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use 8times4/apex-flash-1-FP8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "8times4/apex-flash-1-FP8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "8times4/apex-flash-1-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/8times4/apex-flash-1-FP8

SGLang

How to use 8times4/apex-flash-1-FP8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "8times4/apex-flash-1-FP8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "8times4/apex-flash-1-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "8times4/apex-flash-1-FP8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "8times4/apex-flash-1-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use 8times4/apex-flash-1-FP8 with Docker Model Runner:
```
docker model run hf.co/8times4/apex-flash-1-FP8
```

apex-flash-1-FP8

Community FP8 quantization of cantina-security/apex-flash-1, a 321B MoE model based on GLM-5.3-Flash. The checkpoint is approximately 328 GB, down from approximately 643 GB for the BF16 source.

Quantization

Uses the block FP8 format of the official zai-org/GLM-5.3-Flash release:

quant_method: fp8, fmt: e4m3, activation_scheme: dynamic, weight_block_size: [128, 128].

E4M3FN weights with 128×128 block scales, stored in F32 as weight_scale_inv.
Per-block absmax scaling: scale = amax(abs(W)) / 448, followed by round-to-nearest FP8 conversion. Weights are reconstructed as q * scale.
The official FP8 index determines which tensors are quantized: routed and shared experts, dense MLPs and MLA projections. Other tensors retain their source BF16/F32 values.
The official quantization_config is used, including dynamic activation scaling.

Conversion was performed on CPU without calibration data or additional training. All 62 output shards were verified with SHA-256 checksums; unquantized tensors were checked for byte-identical preservation.

Provenance

Weights, tokenizer, chat template and processor files come from standard Apex Flash 1, at source revision 28c647a6bd0444973a7ef3d940c24c67e64f52fe. The tokenizer, chat template and processor files are copied unchanged.

FP8 tensor selection and quantization configuration come from GLM-5.3-Flash revision eb9eb208eb0d988989d07a6a12d0fdeb5f52574a.

The conversion approach was inspired by pqhaz/apex-flash-1-abliterated-FP8 and its conversion script. This checkpoint was converted independently from standard Apex Flash 1.

Testing

Successfully loaded and tested with vLLM on 8× NVIDIA H100 80 GB GPUs.

A separate task-quality comparison against the BF16 source has not been performed. Upstream benchmark results have not been reproduced for this quantization.