results

This model is a fine-tuned version of t5-small on the dataset: https://huggingface.co/datasets/sentence-transformers/simple-wiki. It achieves the following results on the evaluation set:

eval_loss: 1.2573
eval_model_preparation_time: 0.0027
eval_rouge1: 54.7902
eval_rouge2: 40.4054
eval_rougeL: 51.7529
eval_rougeLsum: 51.7451
eval_runtime: 1078.8925
eval_samples_per_second: 18.95
eval_steps_per_second: 2.369
step: 0

Model description

This is a fine tuned T5 based model that was trained upon the wiki-simple dataset: https://huggingface.co/datasets/sentence-transformers/simple-wiki. The idea is to build a self-service technical simplification model for my Machine Learning course final project!

Intended uses & limitations

Intended use cases would be simplifying user-inputted text for one application, or producing bullet point summaries that are also accordingly simplified. The biggest limitation was time constraints and the small size of the dataset. Also to note, it is trained upon wikipedia. This means it may fall short when dealing with medical or legal terminology.

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
num_epochs: 3
mixed_precision_training: Native AMP

Framework versions

Transformers 5.0.0
Pytorch 2.10.0+cu128
Datasets 4.0.0
Tokenizers 0.22.2

Downloads last month: 1

Safetensors

Model size

60.5M params

Tensor type

F32

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for SpecificTax/470_FINAL_TECHSIMPLIFY

Base model

google-t5/t5-small

Finetuned

(2333)

this model