metadata
datasets:
- EleutherAI/pile
language:
- en
tags:
- t5x
- encoder-decoder
This is the T5x version of Pile-T5 XL. You can use these checkpoints to continue
pretraining or finetune using the T5x library.
Scripts used to train Pile-T5 are available in the improved-t5 repository
on github. To access a specific step, refer to their respective branch step_TRAININGSTEP. The main branch is left empty
For the HF version, please refer here
BibTeX
@misc{2024PileT5,
author = {Lintang Sutawika and Aran Komatsuzaki and Colin Raffel},
title = {Pile-T5},
year = {2024},
url = {https://blog.eleuther.ai/pile-t5/},
note = {Blog post},
}