yujiepan/flint-tiny-random

This tiny model is intended for debugging. It is randomly initialized using the configuration adapted from RecursiveSI/flint.

Structural comparison with RecursiveSI/flint:

Feature	Source	Tiny
Loading implementation	source SGLang plugin	same source plugin wheel
Stored precision	BF16	BF16
Vocabulary / embedding and LM-head ties	150,144 / untied	same
Layers / dense prefix	49 / layers 0-1	8 / layers 0-1
Attention cadence / RoPE-free layers	49-layer source schedule	first 8 source layers
Hidden size / Q and KV heads / head dimension	2,048 / 32 and 4 / 128	32 / 1 and 1 / 32
Routed experts / active experts	256 / 6	64 / 6
MoE / shared-expert widths	512 / 1,024	32 / 64
Sigmoid router, top-k renormalization, learned sinks, QK norms	present	same checkpoint schema

Direct checkpoint generation is required because Transformers does not implement Flint. Tensor names and shapes come from the pinned source safetensors headers and the source-provided flint_sglang loader wheel. The source plugin explicitly validates TP=1 only, so this model carries no TP=4 claim.

File path	Size
model.safetensors	21.9MB

Example usage:

• SGLang (unverified)

• The source Flint plugin validates one GPU per replica only; it explicitly marks tensor/expert/pipeline parallelism as unvalidated. The tiny checkpoint therefore does not claim TP=4 support.

• Install the source-provided plugin wheel before this command. Its learned attention sinks, sigmoid-router top-k, shared expert, tool grammar and reasoning parser are not implemented by stock SGLang.

pip install ./sglang/flint_sglang-0.1.4-py3-none-any.whl
SGLANG_EXTERNAL_MODEL_PACKAGE=flint_sglang FLASHINFER_USE_CUDA_NORM=1 \
python -m sglang.launch_server \
  --model-path tiny-random/flint --served-model-name flint --tp-size 1 \
  --attention-backend fa3 --moe-runner-backend triton --enable-fp32-lm-head \
  --tool-call-parser flint --reasoning-parser flint --grammar-backend flint

Codes to create this repo:

Click to expand

import json
from pathlib import Path
from shutil import copyfile

import torch
from huggingface_hub import hf_hub_download
from safetensors.torch import save_file

source_model_id = "RecursiveSI/flint"
save_folder = "/tmp/tiny-random/flint"
source_revision = "2a247be0c0aec895ccb0ac7ca92229ec6df37288"
folder = Path(save_folder)
folder.mkdir(parents=True, exist_ok=True)
raw = json.loads(Path(hf_hub_download(source_model_id, "config.json", revision=source_revision)).read_text())
# Transformers has no Flint model class.  The source SGLang plugin is the
# authoritative executable schema, so create its public checkpoint tensors directly.
# The fixed 150,144-row untied embedding and LM head make hidden_size=32 the
# smallest practical BF16 geometry; one 32-wide attention head retains the
# required LLM head-dimension floor.
layers = 8
raw.update(
    hidden_size=32, head_dim=32, num_attention_heads=1, num_key_value_heads=1,
    intermediate_size=128, moe_intermediate_size=32, shared_expert_intermediate_size=64,
    num_experts=64, num_experts_per_tok=6, max_position_embeddings=4096,
    num_hidden_layers=layers,
    layer_types=raw["layer_types"][:layers],
    no_rope_layer_indices=[index for index in raw["no_rope_layer_indices"] if index < layers],
)
(folder / "config.json").write_text(json.dumps(raw, indent=2) + "\n", encoding="utf-8")
generator = torch.Generator(device="cpu").manual_seed(42)

def random_tensor(*shape: int) -> torch.Tensor:
    return torch.randn(shape, generator=generator, dtype=torch.bfloat16).mul_(0.02)

hidden = raw["hidden_size"]
head_dim = raw["head_dim"]
experts = raw["num_experts"]
state: dict[str, torch.Tensor] = {
    "model.embed_tokens.weight": random_tensor(raw["vocab_size"], hidden),
    "lm_head.weight": random_tensor(raw["vocab_size"], hidden),
    "model.norm.weight": torch.ones(hidden, dtype=torch.bfloat16),
}
dense_layers = set(raw["mlp_only_layers"])
for layer in range(raw["num_hidden_layers"]):
    prefix = f"model.layers.{layer}"
    state[f"{prefix}.input_layernorm.weight"] = torch.ones(hidden, dtype=torch.bfloat16)
    state[f"{prefix}.post_attention_layernorm.weight"] = torch.ones(hidden, dtype=torch.bfloat16)
    state[f"{prefix}.self_attn.q_proj.weight"] = random_tensor(hidden, hidden)
    state[f"{prefix}.self_attn.k_proj.weight"] = random_tensor(head_dim, hidden)
    state[f"{prefix}.self_attn.v_proj.weight"] = random_tensor(head_dim, hidden)
    state[f"{prefix}.self_attn.o_proj.weight"] = random_tensor(hidden, hidden)
    state[f"{prefix}.self_attn.q_norm.weight"] = torch.ones(head_dim, dtype=torch.bfloat16)
    state[f"{prefix}.self_attn.k_norm.weight"] = torch.ones(head_dim, dtype=torch.bfloat16)
    state[f"{prefix}.self_attn.softmax_offset"] = random_tensor(raw["num_attention_heads"])
    if layer in dense_layers:
        for projection in ("gate_proj", "up_proj"):
            state[f"{prefix}.mlp.{projection}.weight"] = random_tensor(raw["intermediate_size"], hidden)
        state[f"{prefix}.mlp.down_proj.weight"] = random_tensor(hidden, raw["intermediate_size"])
        continue
    state[f"{prefix}.mlp.gate.weight"] = random_tensor(experts, hidden)
    for projection in ("gate_proj", "up_proj"):
        state[f"{prefix}.mlp.shared_expert.{projection}.weight"] = random_tensor(
            raw["shared_expert_intermediate_size"], hidden
        )
    state[f"{prefix}.mlp.shared_expert.down_proj.weight"] = random_tensor(
        hidden, raw["shared_expert_intermediate_size"]
    )
    for expert in range(experts):
        expert_prefix = f"{prefix}.mlp.experts.{expert}"
        for projection in ("gate_proj", "up_proj"):
            state[f"{expert_prefix}.{projection}.weight"] = random_tensor(raw["moe_intermediate_size"], hidden)
        state[f"{expert_prefix}.down_proj.weight"] = random_tensor(hidden, raw["moe_intermediate_size"])
save_file(state, folder / "model.safetensors", metadata={"format": "pt"})
for filename in (
    "LICENSE", "chat_template.jinja", "generation_config.json", "tokenizer.json", "tokenizer_config.json",
    "sglang/flint_sglang-0.1.4-py3-none-any.whl", "sglang/serve_h200.sh", "sglang/README.md",
):
    destination = folder / filename
    destination.parent.mkdir(parents=True, exist_ok=True)
    copyfile(hf_hub_download(source_model_id, filename, revision=source_revision), destination)

Test environment:

• huggingface_hub: 1.27.0
• safetensors: 0.8.0
• tokenizers: 0.23.0rc0
• torch: 2.13.0+cu126
• transformers: 5.15.0.dev0

Change log:

• 2026-10-23: Initial version.

This repo is generated by GPT-6.1-Sol