Decoder-Only Transformer (AIL861 Assignment)
This repository contains a decoder-only transformer model trained from scratch for the AIL861 Advanced LLMs assignment.
Model Details
- Architecture: Decoder-Only Transformer (3 layers)
- Dataset: TinyStories
- Tokenization: Word-level (Vocab size: 20000)
- Embeddings: 300-dim (Initialized with FastText)
- Context Length: 64
Trained by
reaper-113
- Downloads last month
- 8
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support