YAML Metadata Warning:empty or missing yaml metadata in repo card

Check out the documentation for more information.

Standard-W2

Configuration Value
Block allocation 14 sequential blocks
Outer d / m / f 1152 / 576 / 576
Branch d / m / f โ€”
Outer Q / K / V heads 18 / 9 / 9
Branch Q / K / V heads โ€”
Head dimension 64
MoE 64 experts, top 8
Context 4,096 tokens

Megatron-native model-only safetensors checkpoint.

Downloads last month
44
Safetensors
Model size
1B params
Tensor type
F32
ยท
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Collection including gaotang/Standard-W2