Hugging Face model: NamrataThakur/Small_Language_Model_MOE_127M_Pretrained
Author: NamrataThakur
Created: 2026-03-08 02:30:12
Last modified: 2026-03-17 11:16:02
Pipeline: text-generation
Likes: 3 | Downloads: 10
Tags: pytorch, customGPT_pretrain, Pretrain, SLM, Group_Query_Attention, Mixture-Of-Experts, Top K Routing, Custom_Model, From_Scratch, tiny-model, research, text-generation, en, dataset:roneneldan/TinyStories, arxiv:1910.09700, license:mit, region:us
Card metadata:
license: mit
datasets: roneneldan/TinyStories
language: en
pipeline_tag: text-generation
tags: Pretrain, SLM, Group_Query_Attention, Mixture-Of-Experts, Top K Routing, Custom_Model, From_Scratch, tiny-model, research