T5-base SciTail

Full fine-tuning of google-t5/t5-base on allenai/scitail (tsv_format). The encoder-decoder backbone and the two-class classification head were fine-tuned.

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification

model_id = "fillo-rinaldi/t5-base-scitail"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

Training

Dataset: allenai/scitail, tsv_format
Optimizer: AdamW, learning rate 5e-6, weight decay 0.01
Batch size: 4 × 32 accumulation = effective batch 128
Precision: fp32; epochs: 50; seed: 42
Best checkpoint: validation accuracy 93.71%, test accuracy 92.90%

The original full checkpoint, extracted head, and training summaries are in artifacts/.

Downloads last month: 30

Safetensors

Model size

0.2B params

Tensor type

F32

Model tree for fillo-rinaldi/t5-base-scitail

Base model

google-t5/t5-base

Finetuned

(704)

this model