Hugging Face's logo

malaiwah
/
kimi-k3-tiny-random-bf16

Image-Text-to-Text
Transformers
Safetensors
kimi_k3
text-generation
qfs
fixture
random-init
cpu
reproducibility
random-architecture-fixture
tiny
custom_code
Model card Files
xet
 Community

Kimi K3: a tiny toolchain fixture

A small, independently initialized random-weight reference model for learning, debugging, and reproducing QFS capture workflows. It is a base/root fixture, not a quantized child.

These weights are untrained, not an assistant, and not a language-quality benchmark. Generated text has no useful semantic quality. No upstream trained weights or training data are implied by an architecture name.

Weight lineage

This checkpoint was initialized directly for an architecture test. It was not fine-tuned from a production model and does not inherit that model's trained weights. Architecture/configuration lineage is documented separately from weight lineage.

Browse the Random Architecture Fixtures collection. A random base may also serve as the common source for an aligned quantization family.

At a glance

Property Observed value
Artifact role Base/reference model; capture role root
Serialized top-level weight files 558,904 bytes (0.533 MiB); metadata, tokenizer and evidence excluded
Generated parameters 269,688
Included vision parameters 9,184; inclusion is not vision-quality evidence
Saved semantic tensors 154 (parameters and buffers are not interchangeable)
Fixture vocabulary 272 tokens; independent byte tokenizer, not upstream vocabulary
CPU stack Python 3.12; Torch 2.11.0+cpu; Transformers 5.16.1; two Torch threads in the recorded capture workflow
Remote model code Explicit immutable consent required; see below

Size is serialized artifact size, not runtime RAM or a capacity/performance guarantee. Packed-array element counts are not model parameter counts.

What is actually included

Complete upstream KimiK3ForConditionalGeneration wrapper with real 1-layer MoonViT-V2 and patchmergerv2; 4-layer KimiLinear text, 3 KDA + 1 gated full MLA, attention-residual block size2 (crosses block boundary), SITU(beta4,linear_beta25), latent MoE dimension32 with normalization/down/up projections, 4 routed/top2 and 2 shared experts, complete untied head. No GLM substitution.

The repository file inventory includes config.json, generation_config.json, tokenizer.json, tokenizer_config.json, build-manifest.json, support.json, requirements-cpu.txt, upstream-code-audit.json, runtime-patches.json. Exact build/runtime/license inventories are linked below. A file may describe historical provenance without being an executable entry point.

Good community uses—and boundaries

  • Learn how to download pinned artifacts, seal a small synthetic token panel, capture hidden states and replay a full vocabulary head.
  • Debug model-family adapters, strict tensor loading, storage decoders, or reproducibility tooling without downloading a production-sized checkpoint.
  • Reproduce a narrowly scoped result, report an adapter/reader regression, and retain the source, panel and runtime identities needed to explain it.

Not established: trained-model accuracy; useful instruction following; quantizer optimization quality; original production-weight compatibility; GPU/NPU/serving-kernel parity; cross-hardware determinism; long-context behavior outside the recorded panel; throughput or paid-compute admission.

Original-release limitations

Published source uses compressed-tensors mxfp4-pack-quantized weight format, floating4 group32 with uint8 scales; advertised serving includes MXFP8 activations. Original runtime needs a compressed-tensors version with MXFP4 pack support and matching supported accelerator kernels/serving engine; neither installed CPU Torch nor this BF16 reference fork implements packed MXFP4 or MXFP8. Fork preserves config/weight names for ordinary unquantized checkpoints only. Do not delete quantization metadata or reinterpret packed arrays as BF16; obtain an explicitly complete standard BF16/FP32 checkpoint for CPU capture.

Additional build caveats:

  • Random initialization, not trained or distilled.
  • Not evidence for upstream compressed-tensors INT4 or MXFP4 runtime compatibility, model quality, or production serving performance.
  • Generation and cache observations are recorded separately by verify_native.py.

Model family, root dataset and evidence

This model is the base/root of its own random fixture family, not a reproduction of the trained upstream model. The fidelity-root dataset repository is the family reference location. A link is not a claim that registration or publication has completed.

The historical CPU evidence bundle retains two independent captures at first/ and repeat/, plus comparison/. Its repository root is a receipt bundle, not a single canonical QFS dataset. Those recorded same-machine/same-stack forced comparisons reported 0.0 nats on 252 synthetic scored positions; that is not a trained-quality result or a transferable hardware floor. The original detailed receipt and caveats remain authoritative.

Runtime requirements and safe local reproduction

Replay the published evidence without loading a model

This uses QFS's existing NumPy FP64 comparator in a Torch-free environment. It reads the stored hidden states and each side's own head. No model forward, remote model code, GPU, upload or registry mutation is involved. Runtime receipts name the actual backend; last-bit differences from another FP64 implementation are not a new quality claim. Synthetic exact controls remain zero.

Set QFS to a reviewed Quant Fidelity Suite checkout and use Bash:

: "${QFS:?Set QFS to your reviewed QFS checkout}"
WORK=$(mktemp -d)
export OMP_NUM_THREADS=2 MKL_NUM_THREADS=2 OPENBLAS_NUM_THREADS=2
python3.12 -m venv "$WORK/replay-env"
"$WORK/replay-env/bin/pip" install 'numpy==2.5.3' 'huggingface-hub==1.30.0'
"$WORK/replay-env/bin/hf" download malaiwah/qfs-fixture-root-captures-v1 --repo-type dataset \
  --revision f53b204091c988ce4a2161af81886f3018745559 --include 'roots/kimi-k3/**' --local-dir "$WORK/evidence"
"$WORK/replay-env/bin/python" "$QFS/bin/fidelity_dataset.py" verify \
  "$WORK/evidence/roots/kimi-k3/first" --verify-tensors
"$WORK/replay-env/bin/python" "$QFS/bin/fidelity_dataset.py" verify \
  "$WORK/evidence/roots/kimi-k3/repeat" --verify-tensors
"$WORK/replay-env/bin/python" "$QFS/bin/fidelity_dataset.py" compare \
  --reference "$WORK/evidence/roots/kimi-k3/first" \
  --candidate "$WORK/evidence/roots/kimi-k3/repeat" \
  --out "$WORK/replayed" --device cpu --replay-device numpy --replay-dtype float32 \
  --vocab-chunk 8192 --verify-tensors --self-compare --force-compute

Reconstructed format comparisons are intentionally advisory and normally return exit code 2 while writing a valid receipt. Inspect that receipt; do not silence refusals or interpret an advisory result as a production-quality ranking.

Capture the actual checkpoint

Capture uses a separate pinned Torch CPU environment. The input below is the original token-panel format, not a sealed capture's internal panel/ folder. The native source, tokenizer and model revisions remain explicit. Set AUTHOR to your own HF handle; the dataset repository argument is attribution only and nothing is uploaded by these commands.

: "${AUTHOR:?Set AUTHOR to your Hugging Face handle}"
"$WORK/replay-env/bin/hf" download malaiwah/kimi-k3-tiny-random-bf16 --revision 3c1e2f22e5ccfe37addc9906bfb0fdea6ad00e16 --local-dir "$WORK/model"
"$WORK/replay-env/bin/hf" download malaiwah/kimi-k3-tiny-random-bf16 --revision 3c1e2f22e5ccfe37addc9906bfb0fdea6ad00e16 --local-dir "$WORK/source"
"$WORK/replay-env/bin/hf" download malaiwah/qfs-fixture-root-captures-v1 --repo-type dataset \
  --revision f53b204091c988ce4a2161af81886f3018745559 --include 'requirements-capture.txt' --include 'roots/kimi-k3/input-panel/**' \
  --local-dir "$WORK/inputs"
python3.12 -m venv "$WORK/capture-env"
"$WORK/capture-env/bin/pip" install -r "$WORK/inputs/requirements-capture.txt"
"$WORK/capture-env/bin/python" "$QFS/bin/fidelity_dataset.py" architectures prepare \
  --architecture kimi-k3 --model-dir "$WORK/model" \
  --model-repository malaiwah/kimi-k3-tiny-random-bf16 --model-revision 3c1e2f22e5ccfe37addc9906bfb0fdea6ad00e16 \
  --panel "$WORK/inputs/roots/kimi-k3/input-panel" --tokenizer-root "$WORK/source" \
  --author "$AUTHOR" --dataset-repository "$AUTHOR/kimi-k3-tiny-random-bf16-capture" \
  --dataset-id "fidelity--$AUTHOR.kimi-k3-tiny-random-bf16" --out "$WORK/workflow" \
  --trust-remote-code --code-repository malaiwah/kimi-k3-tiny-random-bf16 --code-revision 3a12212e4692e70b531f541e20f6237efb79637e

Inspect workflow.json. This launcher executes its exact capture/verification commands and selects the Torch-free interpreter only for the final comparison:

export OMP_NUM_THREADS=2 MKL_NUM_THREADS=2 OPENBLAS_NUM_THREADS=2
"$WORK/replay-env/bin/python" - "$WORK/workflow/workflow.json" <<'PY'
import json, subprocess, sys
workflow = json.load(open(sys.argv[1]))
for step in workflow["commands"]:
    argv = list(step["argv"])
    if step["step"] == "compare":
        argv[0] = sys.executable
    result = subprocess.run(argv)
    if result.returncode:
        raise SystemExit(result.returncode)
PY

Use the actual root at malaiwah/kimi-k3-tiny-fidelity-root-v1@74f15b550582c96fb26c40c87a839298b914721b. The community collection groups models and captures. Custom code, where required above, is explicitly pinned and executed only after your consent; hash verification is provenance, not a sandbox.

Licensing and detailed provenance

Retain Kimi K3 License (commercial Model-as-a-Service >$20M aggregate12-month revenue requires separate agreement; UI attribution thresholds apply), Apache-2.0 adapted DeepSeek/Transformers portions, MIT FLA reference code. Random independent weights do not relicense copied runtime code.

Immutable provenance, historical cards, source inventories and full caveats

Selected original provenance fields (full tensor/component evidence remains in the linked inventories):

{
  "generation": "Untouched FP32 initialization serialized temporarily; actual BF16 loader retains required FP32 tensors. Every converted tensor equals original.to(target_dtype); final mixed-dtype save/reload is exact.",
  "generator_sha256": "adfbd5489e3baecab66d3ba7a391f218d1877d0531675096786d454723601812",
  "limitations": [
    "Random initialization, not trained or distilled.",
    "Not evidence for upstream compressed-tensors INT4 or MXFP4 runtime compatibility, model quality, or production serving performance.",
    "Generation and cache observations are recorded separately by verify_native.py."
  ],
  "lineage": {
    "relationship": "architecture lineage only; no upstream weights",
    "repository": "moonshotai/Kimi-K3",
    "revision": "f831ab66814297da540d832a5235f8e904f29d06"
  },
  "native_fp32_exceptions": [
    "language_model.model.layers.0.self_attn.A_log",
    "language_model.model.layers.0.self_attn.dt_bias",
    "language_model.model.layers.1.self_attn.A_log",
    "language_model.model.layers.1.self_attn.dt_bias",
    "language_model.model.layers.2.self_attn.A_log",
    "language_model.model.layers.2.self_attn.dt_bias"
  ],
  "parameter_count": 269688,
  "scope": "Text-panel CPU capture with a complete real small vision encoder and projector. verify_native.py separately exercises synthetic vision and cache boundaries. No production processor or quality claim.",
  "seed": 20260907,
  "versions": {
    "safetensors": "0.8.0",
    "tokenizers": "0.23.2",
    "torch": "2.11.0+cpu",
    "transformers": "5.16.1"
  },
  "vision_parameter_count": 9184,
  "reviewed_code_pin": {
    "repository": "malaiwah/kimi-k3-tiny-random-bf16",
    "revision": "3a12212e4692e70b531f541e20f6237efb79637e"
  }
}
Downloads last month
32
Safetensors
Model size
270k params
Tensor type
F32
·
BF16
·
Inference Providers NEW
Image-Text-to-Text
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collections including malaiwah/kimi-k3-tiny-random-bf16