metadata

title: Gpt2 Tinystories Generator
emoji: 🏢
colorFrom: yellow
colorTo: red
sdk: gradio
sdk_version: 5.13.2
app_file: app.py
pinned: false
license: apache-2.0
short_description: Gradio

GPT-2 TinyStories Generator 📖 FableWeaver AI

Overview

This project fine-tunes a GPT-2 model on the TinyStories dataset to generate structured, coherent, and engaging short narratives. The model is hosted on Hugging Face Spaces and provides a user-friendly interface for story generation.

Features

✅ Story Generation: Produces structured, child-friendly short stories.
✅ Bias Monitoring: Ensures balanced gender and cultural representation.
✅ Efficient Training: Fine-tuned on 200,000 training samples and 20,000 test samples.
✅ Grammar & Readability Enhancements: Integrated grammar-checking tools and text refinement.
✅ Optimized Performance: Uses loss tracking, sampling techniques, and bias mitigation strategies.

🚀 System Architecture

The model is designed for easy interaction via Hugging Face Spaces and follows this workflow:

Data Preprocessing & Cleaning
- Tokenization, formatting, and encoding normalization.
- Bias mitigation and balanced data preparation.
Model Fine-Tuning
- Fine-tuned GPT-2 (124M parameters) using Hugging Face Transformers.
- Hyperparameter optimization (batch size, learning rate, weight decay).
Story Generation Pipeline
- Uses top-k filtering (k=50), top-p nucleus sampling (p=0.9), and temperature adjustments.
Post-Processing & Bias Mitigation
- Contextual reinforcement and diversity-aware storytelling.
- No-repeat n-gram settings and logical scene transitions.
Evaluation & Performance Monitoring
- Tracked using Weights & Biases (W&B) and TensorBoard.
- Analyzed validation loss and coherence checks.

🔹 Getting Started

Accessing the Model

The model is available on Hugging Face Spaces: GPT-2 TinyStories Generator

Usage Instructions

Visit the Hugging Face Space linked above.
Enter a prompt (e.g., "Once upon a time...") in the input field.
Click Generate to receive an AI-generated short story.
Modify the prompt and settings (temperature, top-k, top-p) for different results.

📊 Training Details

Model: GPT-2 (124M)
Dataset: RonenEldan/TinyStories
Training: 3 epochs on Google Colab GPU (T4)
Loss Metrics:
- Training Loss: 3.08 → 2.86
- Validation Loss: 1.46 → 1.40

📝 Evaluation & Observations

Perplexity improved from 8.12 → 2.09, indicating better text fluency.
Validation loss decreased consistently, suggesting effective generalization.
Human evaluation highlighted minor inconsistencies, such as abrupt scene shifts and simplistic narratives.

⚖️ Ethical Considerations

Bias Monitoring: Pronoun analysis and diversity checks to ensure fairness.
Harmful Content Mitigation: Manually reviewed outputs for stereotypes.
Text Processing Issues: UTF-8 encoding applied to prevent character errors.

🔮 Future Improvements

Enhancing Creativity: Fine-tune temperature and randomness settings.
Genre-Specific Training: Introduce theme-based datasets.
Larger Model Training: Experiment with GPT-2 (355M) for richer storytelling.

🤝 Contributors

Charla Pia Vella (Project Developer)
Affiliation: ARI3333 Generative AI

📜 License

This project is released under the Apache-2.0 License.

🎓 Acknowledgments

OpenAI for GPT-2
Hugging Face for the fine-tuning framework
Ronen Eldan for the TinyStories dataset

📌 For more details, visit the Hugging Face Space.

Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference