|
--- |
|
license: apache-2.0 |
|
tags: |
|
- generated_from_trainer |
|
- storytelling |
|
- fiction |
|
- tiny-stories |
|
pipeline_tag: text-generation |
|
library_name: transformers |
|
--- |
|
|
|
# Athspi LLM |
|
|
|
🧠 A small but capable language model for creative story generation, trained on the TinyStories dataset. |
|
|
|
 <!-- Add your banner image URL --> |
|
|
|
## Model Details |
|
|
|
### Architecture |
|
- **Model Type**: Transformer-based language model |
|
- **Layers**: 4 |
|
- **Embedding Dim**: 384 |
|
- **Heads**: 6 |
|
- **Sequence Length**: 128 tokens |
|
- **Parameters**: ~28M |
|
|
|
### Training Data |
|
- **Dataset**: [TinyStories](https://huggingface.co/datasets/roneneldan/TinyStories) |
|
- **Training Coverage**: 5% of dataset (~100k samples) |
|
|
|
## Usage |
|
|
|
### Installation |
|
```bash |
|
pip install torch transformers sentencepiece |
