mistralai

/

Mistral-7B-v0.1

like 3.44k

Follow

Mistral AI_ 2,584

Closing the PR as this seems to be the correct one:

more mistral-7B-v0.1/params.json
>>> {
    "dim": 4096,
    "n_layers": 32,
    "head_dim": 128,
    "hidden_dim": 14336,
    "n_heads": 32,
    "n_kv_heads": 8,
    "norm_eps": 1e-05,
    "sliding_window": 4096,
    "vocab_size": 32000
}