Hugging Face's logo

reaperdoesntknow
/
MoA-150M

Text Generation
Transformers
PyTorch
TensorBoard
English
moa_metric
trl
sft
conversational
Model card Files
xet
Metrics Community
reaperdoesntknow commited on
Commit
63361ab
·
verified ·
1 Parent(s): f47e179

Upload MoAMetricLM

Browse files
Files changed (2) hide show
  1. config.json +1 -1
  2. pytorch_model.bin +1 -1
config.json CHANGED
@@ -42,7 +42,7 @@
42
  "router_bias_heads": 4,
43
  "router_dropout": 0.1,
44
  "router_hidden": 1536,
45
- "router_init_temperature": 2.0,
46
  "router_temperature": 2.0,
47
  "router_topk": 3,
48
  "shared_kv_ratio": 0.55,
 
42
  "router_bias_heads": 4,
43
  "router_dropout": 0.1,
44
  "router_hidden": 1536,
45
+ "router_init_temperature": 1.5,
46
  "router_temperature": 2.0,
47
  "router_topk": 3,
48
  "shared_kv_ratio": 0.55,
pytorch_model.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:53f48a3e9d845304bda043526f8caaa4af8d9798feb2fb6dd37d843922579218
3
  size 623899323
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2d3cadbb015b877d1c4d2abeb15cf1d20242f748749ec38052c88effd9e3789e
3
  size 623899323