google

/

flan-t5-xl

like 429

Hey @bayang , you can use the following Space to add a safetensors variant: https://huggingface.co/spaces/safetensors/convert

I submitted the current model to it and it opened the following PR: https://huggingface.co/google/flan-t5-xl/discussions/24#6565af340ff3292512bcf87d

I'm merging it, feel free to use the Space above to have conversions of the models you want.

Thanks @lysandre .

Great! Closing this issue :)

Hey @lysandre , i was wondering if the code I was preparing to convert is the same as you do in that space internally?
import torch

all_weights = {}
for filename in ["pytorch_model-00001-of-00002.bin", "pytorch_model-00002-of-00002.bin"]:
    _model = torch.load(filename, map_location=torch.device('cpu'))
    all_weights = { **all_weights, **_model }

from safetensors import safe_open
from safetensors.torch import save_file

save_file(all_weights, "/tmp/model.safetensors")

If so, how would I create the model.safetensors.index.json file?

@bayang it's a bit different, it's using transformers' from_pretrained and save_pretrained methods to take care of the sharding

The code is accessible, it's here: https://huggingface.co/spaces/safetensors/convert/blob/main/convert.py

@lysandre Oh I see, the space has also the code.
thanks.

@lysandre How can I merge both safetensors files, because I'm using this quantization technique from Huggingface Candle? and it loads from one file.

$ cargo run --example tensor-tools --release -- quantize --quantization q6k PATH/TO/T5/model.safetensors /tmp/model.gguf