malaiwah/deepseek-v4-tiny-random-bf16

DeepSeek V4: a tiny toolchain fixture

A small, independently initialized random-weight reference model for learning, debugging, and reproducing QFS capture workflows. It is a base/root fixture, not a quantized child.

These weights are untrained, not an assistant, and not a language-quality benchmark. Generated text has no useful semantic quality. No upstream trained weights or training data are implied by an architecture name.

Weight lineage

This checkpoint was initialized directly for an architecture test. It was not fine-tuned from a production model and does not inherit that model's trained weights. Architecture/configuration lineage is documented separately from weight lineage.

Browse the Random Architecture Fixtures collection. A random base may also serve as the common source for an aligned quantization family.

At a glance

Property	Observed value
Artifact role	Base/reference model; capture role root
Serialized top-level weight files	814,660 bytes (0.777 MiB); metadata, tokenizer and evidence excluded
Generated parameters	339,037
Included vision parameters	0; inclusion is not vision-quality evidence
Saved semantic tensors	112 (parameters and buffers are not interchangeable)
Fixture vocabulary	272 tokens; independent byte tokenizer, not upstream vocabulary
CPU stack	Python 3.12; Torch 2.11.0+cpu; Transformers 5.16.1; two Torch threads in the recorded capture workflow
Remote model code	Explicit immutable consent required; see below

Size is serialized artifact size, not runtime RAM or a capacity/performance guarantee. Packed-array element counts are not model parameter counts.

What is actually included

Complete native-primitives text class with explicit reviewed per-instance RMSNorm arithmetic correction: weighted normalization in FP32 followed by final cast to input dtype. This is not bit-identical native Transformers execution or the complete upstream release. No vision encoder/aligner, DSpark, MTP, original FP8/FP4 storage execution, full-size numerical alignment, or million-token claim. HCA rate is intentionally 16 rather than production 128; CSA rate remains 4 and sliding window is 8 rather than 128. Random weights are not a quality benchmark.

The repository file inventory includes config.json, generation_config.json, tokenizer.json, tokenizer_config.json, build-manifest.json, support.json, requirements-cpu.txt. Exact build/runtime/license inventories are linked below. A file may describe historical provenance without being an executable entry point.

Good community uses—and boundaries

• Learn how to download pinned artifacts, seal a small synthetic token panel, capture hidden states and replay a full vocabulary head.
• Debug model-family adapters, strict tensor loading, storage decoders, or reproducibility tooling without downloading a production-sized checkpoint.
• Reproduce a narrowly scoped result, report an adapter/reader regression, and retain the source, panel and runtime identities needed to explain it.

Not established: trained-model accuracy; useful instruction following; quantizer optimization quality; original production-weight compatibility; GPU/NPU/serving-kernel parity; cross-hardware determinism; long-context behavior outside the recorded panel; throughput or paid-compute admission.

Original-release limitations

• Forbidden workaround: Never remove quantization_config/expert_dtype from real original-format checkpoint bytes, suppress conversion errors, or allow random initialization/mismatched tensors to obtain a nominal pass.
• Native bf16 route: Use explicit pinned corrected-runtime override for complete BF16 text checkpoints; unmodified native5.16.1 fails at FP32-retained RMSNorm/BF16Linear. For an original config without auto_map, selected code repository supplies authenticated auto_map ONLY; original config/quantization/weight bytes remain unchanged. A BF16 derivative must retain converter/version/input/output hashes and is not original-format execution proof.
• Native source coverage: Installed finegrained_fp8 integration includes packed FP4 and FP8 experts/linear dispatch through CUDA DeepGEMM or Triton; this CPU BF16 workflow does not measure those paths.
• Published storage: - Activation scheme: dynamic
• Expert dtype: fp4
• Fmt: e4m3
• Quant method: fp8
• Scale fmt: ue8m0
• Weight block size: 128; 128
• Qualification required: Actual pinned full-checkpoint loading without missing/mismatched/conversion errors, exact scope for unused speculative/vision tensors, supported original-format GPU kernels and independent captured evidence. No original large weights are downloaded or rented compute used here.
• Quantizer plan bug: hf_capture --drop-parallel-plan neutralizes unused single-process TP/EP plans to bypass the HF5.16.1 quantizer plan rewrite defect. This changes in-memory plans, not checkpoint quantization metadata/bytes, and is recorded by QFS.

Model family, root dataset and evidence

This model is the base/root of its own random fixture family, not a reproduction of the trained upstream model. The fidelity-root dataset repository is the family reference location. A link is not a claim that registration or publication has completed.

The historical CPU evidence bundle retains two independent captures at first/ and repeat/, plus comparison/. Its repository root is a receipt bundle, not a single canonical QFS dataset. Those recorded same-machine/same-stack forced comparisons reported 0.0 nats on 252 synthetic scored positions; that is not a trained-quality result or a transferable hardware floor. The original detailed receipt and caveats remain authoritative.

Runtime requirements and safe local reproduction

Replay the published evidence without loading a model

This uses QFS's existing NumPy FP64 comparator in a Torch-free environment. It reads the stored hidden states and each side's own head. No model forward, remote model code, GPU, upload or registry mutation is involved. Runtime receipts name the actual backend; last-bit differences from another FP64 implementation are not a new quality claim. Synthetic exact controls remain zero.

Set QFS to a reviewed Quant Fidelity Suite checkout and use Bash:

: "${QFS:?Set QFS to your reviewed QFS checkout}"
WORK=$(mktemp -d)
export OMP_NUM_THREADS=2 MKL_NUM_THREADS=2 OPENBLAS_NUM_THREADS=2
python3.12 -m venv "$WORK/replay-env"
"$WORK/replay-env/bin/pip" install 'numpy==2.5.3' 'huggingface-hub==1.30.0'
"$WORK/replay-env/bin/hf" download malaiwah/qfs-fixture-root-captures-v1 --repo-type dataset \
  --revision f53b204091c988ce4a2161af81886f3018745559 --include 'roots/deepseek-v4/**' --local-dir "$WORK/evidence"
"$WORK/replay-env/bin/python" "$QFS/bin/fidelity_dataset.py" verify \
  "$WORK/evidence/roots/deepseek-v4/first" --verify-tensors
"$WORK/replay-env/bin/python" "$QFS/bin/fidelity_dataset.py" verify \
  "$WORK/evidence/roots/deepseek-v4/repeat" --verify-tensors
"$WORK/replay-env/bin/python" "$QFS/bin/fidelity_dataset.py" compare \
  --reference "$WORK/evidence/roots/deepseek-v4/first" \
  --candidate "$WORK/evidence/roots/deepseek-v4/repeat" \
  --out "$WORK/replayed" --device cpu --replay-device numpy --replay-dtype float32 \
  --vocab-chunk 8192 --verify-tensors --self-compare --force-compute

Reconstructed format comparisons are intentionally advisory and normally return exit code 2 while writing a valid receipt. Inspect that receipt; do not silence refusals or interpret an advisory result as a production-quality ranking.

Capture the actual checkpoint

Capture uses a separate pinned Torch CPU environment. The input below is the original token-panel format, not a sealed capture's internal panel/ folder. The native source, tokenizer and model revisions remain explicit. Set AUTHOR to your own HF handle; the dataset repository argument is attribution only and nothing is uploaded by these commands.

: "${AUTHOR:?Set AUTHOR to your Hugging Face handle}"
"$WORK/replay-env/bin/hf" download malaiwah/deepseek-v4-tiny-random-bf16 --revision 185723bec10ee57db86e8c269b3c6a811122a0ca --local-dir "$WORK/model"
"$WORK/replay-env/bin/hf" download malaiwah/deepseek-v4-tiny-random-bf16 --revision 185723bec10ee57db86e8c269b3c6a811122a0ca --local-dir "$WORK/source"
"$WORK/replay-env/bin/hf" download malaiwah/qfs-fixture-root-captures-v1 --repo-type dataset \
  --revision f53b204091c988ce4a2161af81886f3018745559 --include 'requirements-capture.txt' --include 'roots/deepseek-v4/input-panel/**' \
  --local-dir "$WORK/inputs"
python3.12 -m venv "$WORK/capture-env"
"$WORK/capture-env/bin/pip" install -r "$WORK/inputs/requirements-capture.txt"
"$WORK/capture-env/bin/python" "$QFS/bin/fidelity_dataset.py" architectures prepare \
  --architecture deepseek-v4 --model-dir "$WORK/model" \
  --model-repository malaiwah/deepseek-v4-tiny-random-bf16 --model-revision 185723bec10ee57db86e8c269b3c6a811122a0ca \
  --panel "$WORK/inputs/roots/deepseek-v4/input-panel" --tokenizer-root "$WORK/source" \
  --author "$AUTHOR" --dataset-repository "$AUTHOR/deepseek-v4-tiny-random-bf16-capture" \
  --dataset-id "fidelity--$AUTHOR.deepseek-v4-tiny-random-bf16" --out "$WORK/workflow" \
  --trust-remote-code --code-repository malaiwah/deepseek-v4-tiny-random-bf16 --code-revision 4f9dcfcbabdbe336f5f83e836dd8fac7138dda23

Inspect workflow.json. This launcher executes its exact capture/verification commands and selects the Torch-free interpreter only for the final comparison:

export OMP_NUM_THREADS=2 MKL_NUM_THREADS=2 OPENBLAS_NUM_THREADS=2
"$WORK/replay-env/bin/python" - "$WORK/workflow/workflow.json" <<'PY'
import json, subprocess, sys
workflow = json.load(open(sys.argv[1]))
for step in workflow["commands"]:
    argv = list(step["argv"])
    if step["step"] == "compare":
        argv[0] = sys.executable
    result = subprocess.run(argv)
    if result.returncode:
        raise SystemExit(result.returncode)
PY

Use the actual root at malaiwah/deepseek-v4-tiny-fidelity-root-v1@c86180f1844f932b2e685eeeed96b633e4c54cfb. The community collection groups models and captures. Custom code, where required above, is explicitly pinned and executed only after your consent; hash verification is provenance, not a sandbox.

Licensing and detailed provenance

• Fixture weights and workflow: MIT
• Native runtime: Transformers5.16.1, Apache-2.0 dependency; native config/model source SHA256 recorded during build. Derived deepseek_v4_qfs.py retains Apache notices and LICENSE-APACHE-2.0, not MIT.
• Remote code: Fixture captures explicitly opt in to reviewed deepseek_v4_qfs.py from model repository at immutable code SHA; code closure verified by QFS before execution. Generator may import the reviewed local wrapper directly. Weights/tokenizer remain local; HF network permitted for pinned code verification, not entirely offline capture.
• Synthetic panel: MIT; authored text, no upstream training data
• Upstream text and vision: MIT; Copyright (c)2023 DeepSeek; pinned originals retained at stage as lineage-LICENSE and vision-lineage-LICENSE

Immutable provenance, historical cards, source inventories and full caveats

• Model/card snapshot used while authoring: malaiwah/deepseek-v4-tiny-random-bf16@185723bec10ee57db86e8c269b3c6a811122a0ca.
• Full preserved earlier model card, SHA-256 d034249beb37f995c2d33769830f0ffd5a2331fd795d5f07ceb6d85360117cfa. This is an immutable historical record, including original build pins, measured resource tables, command logs, limitations and failed-attempt provenance. Historical publisher commands are not part of the local-use recipe above.
• build-manifest.json.
• support.json.
• qfs-source.json.
• evidence-summary.json.
• native-verification.json.
• regeneration-verification.json.
• LICENSE-APACHE-2.0.

Selected original provenance fields (full tensor/component evidence remains in the linked inventories):

{
  "compress_rates": {
    "compressed_sparse_attention": 4,
    "heavily_compressed_attention": 16
  },
  "compress_ratio_selectors": [
    0,
    0,
    4,
    128
  ],
  "generation": "Complete native FP32 random initialization with seeded hash-router buffers; native BF16 conversion retaining native-required FP32 state.",
  "generator_sha256": "0801acbd80e6eab6d7891c384c8fd7e3778f7e4e92ad75a7fff79831adb237ab",
  "layer_types": [
    "sliding_attention",
    "sliding_attention",
    "compressed_sparse_attention",
    "heavily_compressed_attention"
  ],
  "lineage": {
    "relationship": "text architecture lineage only; no upstream weights copied",
    "repository": "deepseek-ai/DeepSeek-V4-Flash-0731",
    "revision": "7872f01b1d1fe23eabc4c98b48bffcef5a386062"
  },
  "native_fp32_exceptions": [
    "model.layers.0.self_attn.sinks",
    "model.layers.0.self_attn.q_a_norm.weight",
    "model.layers.0.self_attn.kv_norm.weight",
    "model.layers.0.input_layernorm.weight",
    "model.layers.0.post_attention_layernorm.weight",
    "model.layers.0.attn_hc.fn",
    "model.layers.0.attn_hc.base",
    "model.layers.0.attn_hc.scale",
    "model.layers.0.ffn_hc.fn",
    "model.layers.0.ffn_hc.base",
    "model.layers.0.ffn_hc.scale",
    "model.layers.1.self_attn.sinks",
    "model.layers.1.self_attn.q_a_norm.weight",
    "model.layers.1.self_attn.kv_norm.weight",
    "model.layers.1.input_layernorm.weight",
    "model.layers.1.post_attention_layernorm.weight",
    "model.layers.1.attn_hc.fn",
    "model.layers.1.attn_hc.base",
    "model.layers.1.attn_hc.scale",
    "model.layers.1.ffn_hc.fn",
    "model.layers.1.ffn_hc.base",
    "model.layers.1.ffn_hc.scale",
    "model.layers.2.self_attn.sinks",
    "model.layers.2.self_attn.q_a_norm.weight",
    "model.layers.2.self_attn.kv_norm.weight",
    "model.layers.2.self_attn.compressor.position_bias",
    "model.layers.2.self_attn.compressor.kv_norm.weight",
    "model.layers.2.self_attn.compressor.indexer.position_bias",
    "model.layers.2.self_attn.compressor.indexer.kv_norm.weight",
    "model.layers.2.input_layernorm.weight",
    "model.layers.2.post_attention_layernorm.weight",
    "model.layers.2.attn_hc.fn",
    "model.layers.2.attn_hc.base",
    "model.layers.2.attn_hc.scale",
    "model.layers.2.ffn_hc.fn",
    "model.layers.2.ffn_hc.base",
    "model.layers.2.ffn_hc.scale",
    "model.layers.3.self_attn.sinks",
    "model.layers.3.self_attn.q_a_norm.weight",
    "model.layers.3.self_attn.kv_norm.weight",
    "model.layers.3.self_attn.compressor.position_bias",
    "model.layers.3.self_attn.compressor.kv_norm.weight",
    "model.layers.3.mlp.gate.e_score_correction_bias",
    "model.layers.3.input_layernorm.weight",
    "model.layers.3.post_attention_layernorm.weight",
    "model.layers.3.attn_hc.fn",
    "model.layers.3.attn_hc.base",
    "model.layers.3.attn_hc.scale",
    "model.layers.3.ffn_hc.fn",
    "model.layers.3.ffn_hc.base",
    "model.layers.3.ffn_hc.scale",
    "model.norm.weight",
    "model.hc_head.hc_fn",
    "model.hc_head.hc_base",
    "model.hc_head.hc_scale"
  ],
  "parameter_count": 339037,
  "scope": "Complete native-primitives text class with explicit reviewed per-instance RMSNorm arithmetic correction: weighted normalization in FP32 followed by final cast to input dtype. This is not bit-identical native Transformers execution or the complete upstream release. No vision encoder/aligner, DSpark, MTP, original FP8/FP4 storage execution, full-size numerical alignment, or million-token claim. HCA rate is intentionally 16 rather than production 128; CSA rate remains 4 and sliding window is 8 rather than 128. Random weights are not a quality benchmark.",
  "seed": 20260907,
  "versions": {
    "safetensors": "0.8.0",
    "tokenizers": "0.23.2",
    "torch": "2.11.0+cpu",
    "transformers": "5.16.1"
  },
  "vision_parameter_count": 0,
  "reviewed_code_pin": {
    "repository": "malaiwah/deepseek-v4-tiny-random-bf16",
    "revision": "4f9dcfcbabdbe336f5f83e836dd8fac7138dda23"
  }
}