xverse
/

XVERSE-MoE-A4.2B

Text Generation

Model card Files Files and versions Community

underspirit commited on Apr 1

Commit

ddaa64a

•

1 Parent(s): a6d67b5

Update configuration_xverse.py

Files changed (1) hide show

configuration_xverse.py +1 -1

configuration_xverse.py CHANGED Viewed

@@ -97,7 +97,7 @@ class XverseConfig(PretrainedConfig):
             Number of shared experts, None for no shared experts.
         output_router_logits (`bool`, optional):
             Whether or not to return the router logits.
-        router_aux_loss_coef (`float`, *optional*, defaults to 0.001):
             The aux loss factor for the total loss.
         Example:

             Number of shared experts, None for no shared experts.
         output_router_logits (`bool`, optional):
             Whether or not to return the router logits.
+        router_aux_loss_coef (`float`, *optional*, defaults to 0.01):
             The aux loss factor for the total loss.
         Example: