Kimi-K3-4layer (CI slice)
A 4-layer bf16 slice of moonshotai/Kimi-K3, built for
SkyRL's Megatron <> vLLM roundtrip logprob tests with
examples/train/kimi_k3/make_slice.py.
Not a usable language model.
- Layers 0-3 of the release: KDA (dense MLP), KDA (MoE) x2, NoPE-MLA (MoE).
- First 128 of 896 routed experts per MoE layer (router rows truncated to match; top-16 routing).
- Routed experts dequantized from MXFP4 to bf16 (exact);
quantization_configremoved. attn_res_block_size= 2 (release: 12) so the attention-residual bank spans two blocks.- Everything else (attention, shared experts, embeddings, LM head, vision tower) copied unchanged.
Subject to the license of the original model.
- Downloads last month
- 463
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐ Ask for provider support
Model tree for eatang/Kimi-K3-4layer
Base model
moonshotai/Kimi-K3