Kimyongari/t5-small-custom

Model Card for t5_small Summarization Model

Model Details

It is text summarization model from pretrained t5_samll model.

Training Data

The model is trained with 1percent of cnn-dailymail datas.

Training Procedure

First : load pretrained t5_small model and tokenizer.

Second : make cnn dataset from basic dataset code.

Third : train the model with DataCollatorForSeq2Seq

training_args = Seq2SeqTrainingArguments(output_dir="./results",

eval_strategy="epoch",
learning_rate=2e-5,
per_device_train_batch_size=4,
per_device_eval_batch_size=4,
warmup_steps=500,
weight_decay=0.01,
save_total_limit=2,
num_train_epochs=1,
fp16=True,
predict_with_generate=True

)

How to Use

it can be used when you want to summarize some long texts.

Evaluation

eval_rouge1: 24.34 eval_rouge2: 9.09 eval_rougeL: 19.72 eval_bleu1: 39.95 eval_bleu2: 15.82 eval_bleu4: 6.22

Limitations

Ethical Considerations

This model can be exploited some wrong issues, so you must use model in appropriate situations