jetmoe
/

jetmoe-8b

YikangS commited on Apr 3

Commit

8de3db1

•

1 Parent(s): 243d98f

restore config

Files changed (1) hide show

config.json CHANGED Viewed

@@ -15,7 +15,7 @@
   "n_embd": 2048,
   "n_layer": 24,
   "n_positions": 4096,
-  "num_attention_heads": 32,
   "num_key_value_heads": 16,
   "num_layers": 24,
   "rms_norm_eps": 1e-05,

   "n_embd": 2048,
   "n_layer": 24,
   "n_positions": 4096,
+  "n_head": 32,
   "num_key_value_heads": 16,
   "num_layers": 24,
   "rms_norm_eps": 1e-05,