tidelganesh/gpt2-astro-finetuned

GPT-2 Astro Horoscope

GPT-2 (small) fine-tuned on the karthiksagarn/astro_horoscope dataset to generate horoscope-style text.

Model description

This model is a causal language model fine-tuned via full fine-tuning on ~22K horoscope text samples. It learns the general style, tone, and phrasing patterns of horoscope writing, rather than being conditioned on a specific zodiac sign, date, or category.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 3
• mixed_precision_training: Native AMP

Training details

• Base model: gpt2 (124M parameters)
• Dataset: karthiksagarn/astro_horoscope (21,959 rows, ~9.6MB)
• Epochs: 3
• Effective batch size: 16 (batch_size=2 × gradient_accumulation_steps=8)
• Learning rate: 2e-5
• Precision: fp16
• Hardware: Google Colab (T4 GPU)

Epoch	Training Loss	Validation Loss
1	2.888	2.765
2	2.693	2.628
3	2.664	2.582

Usage

from transformers import pipeline

generator = pipeline("text-generation", model="tidelganesh/gpt2-astro-finetuned")
output = generator("Today your career", max_length=100, do_sample=True, temperature=0.8)
print(output[0]["generated_text"])

Limitations

• Not conditioned on zodiac sign, date, or category — generates general horoscope-style text.
• Output should be treated as generative text, not real astrological guidance.
• Inherits any biases present in GPT-2's original pretraining and in the fine-tuning dataset.

Intended use

Educational / experimental — this was a first fine-tuning project to learn the end-to-end training workflow (tokenization, Trainer API, Hub push).

Framework versions

• Transformers 5.13.1
• Pytorch 2.11.0+cu128
• Datasets 4.0.0
• Tokenizers 0.22.2

Google Colab File

https://colab.research.google.com/drive/1r6b5xyY_5mlb6-LksoUWjWuxa4R1HQGp?usp=sharing