LlamaFinetuneBase commited on
Commit
5860462
1 Parent(s): 2130f9d

Upload params.json with huggingface_hub

Browse files
Files changed (1) hide show
  1. params.json +11 -0
params.json ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "dim": 5120,
3
+ "n_layers": 40,
4
+ "head_dim": 128,
5
+ "hidden_dim": 14336,
6
+ "n_heads": 32,
7
+ "n_kv_heads": 8,
8
+ "norm_eps": 1e-05,
9
+ "vocab_size": 131072,
10
+ "rope_theta": 1000000.0
11
+ }