Docker image and serving guide

Raon-1-mini-preview-dflash2-nvfp4

An NVFP4 W4A4 quantized version of KRAFTON's Raon-1-mini DFlash2 draft model.

Quantized by using NVIDIA Model Optimizer. No additional fine-tuning was performed.

Quantization

Component	Format
Feature projection (fc)	NVFP4 W4A4
Attention Q/K/V/O projections	NVFP4 W4A4
MLP gate/up/down projections	NVFP4 W4A4
Draft output head (lm_head)	NVFP4 W4A4

A total of 36 body Linear projections and one output head are quantized. Selector, convolution and normalization tensors retain their original values.

The output head was converted from the MXFP8 head in Raon-1-mini-preview-nvfp4.

Usage

This is a DFlash2 draft model for speculative decoding. It must be used with the Raon-1-mini target model.

See the serving guide for DGX Spark Docker commands.

License

Apache License 2.0. The original model is by KRAFTON AI. This is a community quantization by Pilcothink.