from_pretrained("google/flan-t5-xxl") cannot work
When I run
model = T5ForConditionalGeneration.from_pretrained("google/flan-t5-xxl").to("cuda")
It shows
404 Client Error: Not Found for url: https://huggingface.co/google/flan-t5-xxl/resolve/main/pytorch_model.bin
How can I solve it? Thanks!
Hey
@Tebmer
! What is your transformers version?
I'm asking as the flan-t5-xxl model is sharded, which means that you'll need a transformers version that handles loading sharded models.
I would recommend updating your transformers version to the latest one and testing once again.
Hey @Tebmer ! What is your
transformersversion?I'm asking as the flan-t5-xxl model is sharded, which means that you'll need a
transformersversion that handles loading sharded models.I would recommend updating your
transformersversion to the latest one and testing once again.
Yeah, it solves my problem. Thanks a lot!
as a side note, flan-xxl is quite large, I would advice you to load it either in half-precision or 8-bit:
# pip install accelerate bitsandbytes
import torch
model = T5ForConditionalGeneration.from_pretrained("google/flan-t5-xxl", device_map="auto", load_in_8bit=True)
Thanks for reminding. I will try it.