uzlnee/t5-small-custom

Model Card for t5_small Summarization Model

Model Details

• Model Architecture: T5-small (Text-to-Text Transfer Transformer)
• Hyperparameters:
  • Learning Rate: 5e-5
  • Batch Size: 16
  • Epochs: 1
  • Max Input Length: 512
  • Max Output Length: 150

Training Data

• Dataset: CNN/DailyMail
• Description: This dataset contains articles from CNN and the Daily Mail, with corresponding summaries.
• Preprocessing Steps:
  • Text normalization
  • Tokenization using the T5 tokenizer
  • Input-target pair creation for summarization

Training Procedure

• Framework: Hugging Face Transformers and PyTorch
• Environment: Python 3.8, CUDA 11.1
• Training Script: Used the Trainer API from Hugging Face.
• Evaluation Metrics: ROUGE score

How to Use

1. Install the required libraries:

   pip install transformers torch
   

Evaluation

Limitations

Ethical Considerations