t5-base-medium-title-generation

This model was trained from scratch on Medium articles.

Training and evaluation data

Dataset

Dataset used was Medium articles: https://www.kaggle.com/datasets/fabiochiusano/medium-articles.

The following are the columns in the dataset:

• title [string]: The title of the article.
• text [string]: The text content of the article.
• url [string]: The URL associated to the article.
• authors [list of strings]: The article authors.
• timestamp [string]: The publication datetime of the article.
• tags [list of strings]: List of tags associated to the article.

The following are the counts of records in the dataset:

• Total dataset size: 192368
• Divided into training (100K), validation (1K), test dataset (1K)
• Preprocessing: prefix = "summarize: " max_input_length = 512 max_target_length = 64 Keeping articles with text length at least 500 & title length at least 20
• After preprocessing: training (85639), validation (833), test dataset (850)

Model description

The model used was T5-base.

Training procedure

Full finetuning

from transformers import Seq2SeqTrainingArguments from transformers import Seq2SeqTrainer

Training hyperparameters

The following hyperparameters were used during training:

minibatch size: 8 learning_rate: 4e-5, weight_decay: 0.01 num_train_epochs: 1

optimizer: None training_precision: float32

Training results

Framework versions

• Transformers 4.38.1
• TensorFlow 2.15.0
• Datasets 2.18.0
• Tokenizers 0.15.2

Intended uses & limitations

Use at your own discretion