UIST Labs

A small, independent AI/ML lab. We build tooling and publish model checkpoints - and we hold them to a stated bar before they ship.

What we do

Certified quantization. We quantize open models and publish a checkpoint only after it clears a hard, stated accuracy gate against its full-precision baseline - measured on the identical evaluation harness, with the real deltas published alongside. A checkpoint that misses the bar doesn't ship. The name of the pipeline behind it says the idea plainly: an assay is the test that certifies a refined metal's purity.

Runs on constrained hardware. Our checkpoints are weight-only NVFP4: they run via vLLM's Marlin FP4 kernel with no Blackwell GPU required, validated down to Ada-generation (sm_89) cards like the L4. Certified accuracy that travels to the hardware you already have, not the hardware you would have to buy.

Models

• Qwen2.5-7B-Instruct-NVFP4A16

  • weight-only NVFP4: ~100% mean accuracy retention vs the bf16 baseline, perplexity within the stated bar. Runs via vLLM's weight-only Marlin FP4 kernel - no Blackwell required; the ~2.5x smaller weights free GPU memory for a much larger KV cache. Validated on Ada (sm_89); Ampere/Hopper are expected but not yet independently validated here, and Turing (sm_75) is excluded pending a known vLLM NVFP4-Marlin correctness fix. Loads natively in vLLM (compressed-tensors auto-detected).

• DeepSeek-R1-Distill-Qwen-7B-NVFP4A16

  • weight-only NVFP4 of the R1-Distill reasoning model: mean accuracy retention within noise of the bf16 baseline (gate: PASS), perplexity within the bar. Same weight-only Marlin FP4 path - no Blackwell required, validated on Ada (sm_89) (Ampere/Hopper expected, Turing excluded, as above). Loads natively in vLLM.

Tooling

• assay - the open-source (Apache-2.0) NVFP4 quantize -> benchmark -> gate -> publish pipeline behind our releases. Every checkpoint we publish was produced by it, and passed its gate.

How we work

• Security first. We don't trade security for convenience without saying so.
• Trust, but verify. Vendor claims are a starting point; the measured numbers are the product.
• Standard tools, honest docs. We prefer well-understood approaches, and we document the trade-offs - including what a release deliberately doesn't do.