dusersad12/StableT5-TestRepo

StableT5

---

1. Introduction

StableT5 is a sequence-to-sequence model trained with a focus on training stability across random seeds. We evaluated multiple hyperparameter configurations and selected the run that achieves both high performance and low variance across seeds. The resulting model demonstrates that careful hyperparameter search can yield checkpoints that generalize reliably without seed-dependent flukes.

2. Evaluation Results

The champion run was selected from a pool of 6 experiments. Only runs whose per-seed relative standard deviation falls below 5% on all metrics were considered stable.

Benchmark	T5-Small	T5-Base	mT5-Small	StableT5
ROUGE-L	0.435	0.488	0.472	0.589
BLEU	0.410	0.465	0.449	0.563
BERTScore F1	0.862	0.895	0.884	0.934
sacreBLEU	0.416	0.471	0.455	0.570
METEOR	0.465	0.520	0.503	0.625

3. Stability Analysis

All metrics for the champion run were verified across three random seeds (42, 123, 999). The maximum relative standard deviation observed was well below the 5% threshold, confirming that the model's performance is not an artifact of a favorable seed.

4. Usage

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

model = AutoModelForSeq2SeqLM.from_pretrained("your-username/StableT5-TestRepo")
tokenizer = AutoTokenizer.from_pretrained("your-username/StableT5-TestRepo")

input_text = "translate English to French: The weather is nice today."
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs, max_length=128)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

5. License

This model is released under the Apache 2.0 License.

6. Contact

For questions or issues, please open an issue on the associated repository.