google

/

flan-t5-xxl

like 1.11k

I am using Mac M1.

I am using code:
"from transformers import T5Tokenizer, T5ForConditionalGeneration

tokenizer = T5Tokenizer.from_pretrained("google/flan-t5-xxl")
model = T5ForConditionalGeneration.from_pretrained("google/flan-t5-xxl", device_map="auto")

input_text = "translate English to German: How old are you?"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")

outputs = model.generate(input_ids)
print(tokenizer.decode(outputs[0]))"

The error:
File ~/miniforge3/envs/tf/lib/python3.8/site-packages/accelerate/utils/offload.py:154, in OffloadedWeightsLoader.init(self, state_dict, save_folder, index, device)
146 def init(
147 self,
148 state_dict: Dict[str, torch.Tensor] = None,
(...)
151 device=None,
152 ):
153 if state_dict is None and save_folder is None:
--> 154 raise ValueError("Need either a state_dict or a save_folder containing offloaded weights.")
156 self.state_dict = {} if state_dict is None else state_dict
157 self.save_folder = save_folder

ValueError: Need either a state_dict or a save_folder containing offloaded weights.