Qwen
/

Qwen1.5-MoE-A2.7B

Text Generation

Mixture of Experts

Inference Endpoints

Model card Files Files and versions Community

bzheng commited on Mar 21

Commit

40b8b73

•

1 Parent(s): ff81856

Update config.json

Files changed (1) hide show

config.json +1 -1

config.json CHANGED Viewed

@@ -24,7 +24,7 @@
   "use_cache": true,
   "use_sliding_window": false,
   "vocab_size": 151936,
-  "expert_interval": 1,
   "moe_intermediate_size": 1408,
   "shared_expert_intermediate_size": 5632,
   "num_experts_per_tok": 4,

   "use_cache": true,
   "use_sliding_window": false,
   "vocab_size": 151936,
+  "decoder_sparse_step": 1,
   "moe_intermediate_size": 1408,
   "shared_expert_intermediate_size": 5632,
   "num_experts_per_tok": 4,