osoleve/Ripple-65B-Base-Preview-NVFP4

Ripple-65B Base Preview NVFP4

Community post-training quantization of Ricursive Intelligence's Ripple-65B Base Preview, pinned to revision 6187d1108a25fcc31681ec942ec3c581721b9fe1.

This is the pretrained base checkpoint. No instruction tuning, training updates, or instruction-model weights were used. Use raw text completions. The tokenizer and model configuration retain the base checkpoint's 8,192-token context limit.

Quantization

The recipe follows the numerical layout of Ricursive's experimental instruction-model NVFP4 release: NVFP4 W4A4 for all MLP gate, up, and down projections; BF16 attention, embeddings, output head, normalization, and KV cache. That upstream NVFP4 release supplies the serving adapter, not the weights for this conversion.

NVIDIA Model Optimizer 0.46.0 calibrates 128 samples of 1,024 tokens from the WikiText-2 raw training split, using seed 42. NVFP4 uses 16-element blocks, E2M1 packed weights, E4M3 block scales, and calibrated per-tensor activation scales. Gate/up scales are synchronized for SGLang's fused projection. All 240 MLP linear layers are quantized; all 400 attention projections remain BF16.

The conversion processes one decoder layer at a time, propagates quantized calibration activations, and writes standard Hugging Face safetensors with ModelOpt quantization metadata. This makes conversion possible without holding the entire BF16 model in memory. quantization-provenance.json records source and dataset revisions, sampled token offsets and hashes, software versions, scale statistics, and timings.

Validation

The exported checkpoint is 57.21 GB (53.28 GiB) of safetensors. The complete audit checked all 1,923 exported tensors, including the packed shapes and scales of all 240 quantized projections. All 963 retained source tensors match the BF16 source exactly. SHA256SUMS lists every weight file.

The packed checkpoint was loaded and served on one NVIDIA GB10 DGX Spark with SGLang 0.5.19, CUDA 13, and FlashInfer CUTLASS NVFP4 GEMMs. Strict loading rejects unconsumed checkpoint tensors and uninitialized model parameters.

Paired WikiText-2 raw test subset: sixteen independent 1,024-token blocks, 16,368 scored next tokens. This is not full-test-set WikiText perplexity.

Inference	Subset perplexity
BF16, SGLang TP=2 on two Sparks	2.963662
NVFP4 MLP / BF16 attention, SGLang TP=1 on one Spark	3.210088

Perplexity changed by +8.31% on this subset. Independent HF layerwise BF16 and ModelOpt fake-quant comparisons yielded 2.964162 and 3.206985, respectively.

Additional finite-log-probability checks passed with 2,048, 4,096, and 8,180 prompt tokens, exercising attention beyond the sliding-window boundary and near the native 8,192-token context limit. These additional windows overlap and do not form an independent benchmark. Raw prose and Python-code completions also ran successfully. Detailed results, sample hashes, and smoke completions are in validation.json.

These results cover a small language-modeling subset held out from calibration, plus basic inference checks. They do not establish broad downstream accuracy, coding benchmark retention, or long-context performance. This is an experimental PTQ release.

Run on a DGX Spark

Use the supplied SGLang adapter. The first start can take several minutes to compile native GPU kernels; later starts reuse the cache. The Ricursive architecture requires this custom implementation; generic vLLM, TensorRT-LLM, and Transformers inference compatibility has not been established for this package.

hf download osoleve/Ripple-65B-Base-Preview-NVFP4 --local-dir ripple-base-nvfp4
cd ripple-base-nvfp4
# Install the versions in requirements-inference.txt using CUDA 13 PyTorch wheels.
export CUDA_HOME=/usr/local/cuda-13.0
export PATH="$CUDA_HOME/bin:$PATH"
export MAX_JOBS=2 NVCC_THREADS=1
python inference/serve.py \
  --model-path . --trust-remote-code \
  --served-model-name ripple-65b-base-preview-nvfp4 \
  --dtype bfloat16 --kv-cache-dtype bfloat16 --tp-size 1 \
  --attention-backend flashinfer --fp4-gemm-backend flashinfer_cutlass \
  --context-length 8192 --max-total-tokens 16384 \
  --mem-fraction-static 0.72 --max-running-requests 2 \
  --chunked-prefill-size 1024 --cuda-graph-bs-decode 1 2 \
  --cuda-graph-backend-prefill disabled \
  --weight-loader-disable-mmap --weight-loader-drop-cache-after-load \
  --model-loader-extra-config '{"enable_multithread_load": false}' \
  --host 127.0.0.1 --port 8006

from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:8006/v1", api_key="local")
response = client.completions.create(
    model="ripple-65b-base-preview-nvfp4",
    prompt="The old observatory stood at the edge of the forest. Every evening,",
    max_tokens=128,
    temperature=0.7,
)
print(response.choices[0].text)

The included tokenizer carries upstream chat-template files for provenance. Their presence does not make this an instruction-tuned model. The base checkpoint has no matching speculative draft configured here.

Reproduce

See reproduce/README.md and reproduce/quantize.py. Calibration uses the training split; evaluation uses separate test data. No private cluster prompts, conversation history, access credentials, or private datasets are included.

Sources and license

The model weights, tokenizer, configuration, and Hugging Face architecture code originate from Ricursive Intelligence's Apache-2.0 base release. The SGLang adapter originates from the Apache-2.0 NVFP4 instruction release at revision 99eea4477a001a41b86914e41feb4b2ed86105e5, with stricter tensor-loading coverage checks and explicit BF16 KV handling added for this package. LICENSE preserves the upstream license, and NOTICE describes these modifications.

Quantization uses NVIDIA Model Optimizer, licensed under Apache 2.0. WikiText is attributed to Salesforce's WikiText dataset; only provenance and aggregate evaluation results are distributed here, not dataset text.