kiruluta

/

SPECTRA-K3-HF-Scaling-Benchmark

Like 0

We are looking for high-compute collaborators to extend the SPECTRA-K3 Hugging Face Scaling Benchmark beyond the current single-GPU DGX Spark baseline.

The project studies high-dimensional compression primitives, expert-block scaling, distributed execution, and eventually real Kimi K3 checkpoint structure for the SPECTRA-MoE compression program.

Current reference baseline

Initial measurements were completed on a single NVIDIA DGX Spark / GB10 with PyTorch 2.14.0+cu130 and CUDA 13.0.

Key measured points:

• 32768 × 32768, rank 64, 100 steps
  41.27B coefficients/s, 13.21 approximate TFLOP/s, 10.04 GiB peak accelerator memory

• 8192 × 8192, rank 64, 16 experts, 100 steps
  15.30B coefficients/s, 4.90 approximate TFLOP/s, 10.05 GiB peak accelerator memory

• At fixed dimension 16384, relative residual energy improved from 0.2469 at rank 32 to 0.1555 at rank 256, showing the expected quality/compute tradeoff.

These are synthetic compression-primitive scaling measurements, not end-to-end Kimi K3 quality results.

Contributors wanted

We are especially interested in results from:

• H100 / H200
• B100 / B200
• GB200 / GB300
• MI300X-class accelerators
• 2 / 4 / 8 / 16+ GPU servers
• multi-node systems

High-priority experiments

1. Larger single-GPU high-dimensional sweeps
2. Multi-GPU weak scaling
3. Multi-GPU strong scaling
4. Real Kimi K3 checkpoint inventory / shard audits
5. OLMoE → K2/K2.5 → K3 model-quality stages

How to contribute

No Git workflow is required.

Run the benchmark, then post:

• the generated result JSON
• exact command used
• accelerator model and count
• single-node or multi-node
• CUDA/ROCm/PyTorch versions
• any benchmark modifications
• OOMs, unsupported dtypes, scaling failures, or negative results

Negative results are welcome. This is a falsification-oriented benchmark.

Suggested result discussion title:

[RESULT] <GPU x count> | <weak/strong/single> | dim=<D> rank=<R>

Example:

[RESULT] 8x H200 | weak | dim=16384 rank=128

Repository:

https://huggingface.co/kiruluta/SPECTRA-K3-HF-Scaling-Benchmark

See COLLABORATION.md and BENCHMARK_PROTOCOL.md in the repository for the full protocol.