Article
makeMoE: Implement a Sparse Mixture of Experts Language Model from Scratch
AviSoori1x
• • 125
Nikhil Dubey
dummydubey
·
AI & ML interests
None yet
Recent Activity
liked a dataset about 16 hours ago
abisee/cnn_dailymail liked a model about 16 hours ago
Qwen/Qwen2.5-1.5B-Instruct liked a model about 19 hours ago
microsoft/deberta-largeOrganizations
None yet
upvoted an article about 1 month ago
upvoted an article 3 months ago
Article
You could have designed state of the art positional encoding
FL33TW00D-HF
• • 500