Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models
Paper
•
2501.11873
•
Published
•
63
Yixin Song
yixinsong
AI & ML interests
None yet
Recent Activity
updated
a model
about 1 month ago
PowerInfer/SmallThinker-3B-Preview
new activity
about 1 month ago
PowerInfer/SmallThinker-3B-Preview:Eval script
Organizations
yixinsong's activity
upvoted
a
paper
26 days ago
updated
a
model
about 1 month ago
Eval script
4
#9 opened about 1 month ago
by
rawsh
liked
a
dataset
about 1 month ago
About the training details
5
#5 opened about 1 month ago
by
hiyouga
How to Pair with Larger Models
4
#7 opened about 1 month ago
by
windkkk
Prompt/token adjust to stop "Overthinking" in unnescissary cases
2
#6 opened about 1 month ago
by
fuzzy-mittenz
example use colab?
8
#3 opened about 2 months ago
by
NickyNicky
Update README.md
#4 opened about 2 months ago
by
AISafety
liked
a
dataset
about 2 months ago
Training: Second Phase
6
#2 opened about 2 months ago
by
tugstugi
[bot] Conversion to Parquet
#1 opened 2 months ago
by
parquet-converter
[bot] Conversion to Parquet
#1 opened about 2 months ago
by
parquet-converter
Librarian Bot: Add language metadata for dataset
#2 opened about 2 months ago
by
librarian-bot
liked
a
model
about 2 months ago
Training: Second Phase
6
#2 opened about 2 months ago
by
tugstugi
updated
a
model
about 2 months ago
Evaluation
6
#1 opened about 2 months ago
by
tugstugi
updated
a
dataset
about 2 months ago
updated
a
model
about 2 months ago