Add safetensors index

#142
by sgugger - opened

The safetensors index file is missing so the from_pretrained method in Transformers does not detect there is a safetensors version of the checkpoint/

cc @Narsil

BigScience Workshop org

Sorry :(

LGTM

BigScience Workshop org
BigScience Workshop org

Thanks! I think we also need to re-run safetensors, in order to fix the old random sha256 (caused by random ordering). I don't know if it would impact the mapping.

BigScience Workshop org

I don't think it's that important tbh.

I added ordering in the lib but I don't think it should be mandatory to do so for file creators.

BigScience Workshop org

Thanks!

TimeRobber changed pull request status to merged

Sign up or log in to comment