Model file appears to be corrupt

#1
by fergusq - opened

When trying to use this model, the program complains that it cannot open the file. I get similar error with the 8B model, but 3B and smaller models seem to work. Is the file corrupt or do I need to open it using something else than the standard AutoModel.from_pretrained("TurkuNLP/gpt3-finnish-13B") code?

โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Traceback (most recent call last) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ
โ”‚ /home/iikkahau/Projektit/miniconda3/envs/fie3/lib/python3.10/site-packages/transformers/modeling โ”‚
โ”‚ _utils.py:417 in load_state_dict                                                                 โ”‚
โ”‚                                                                                                  โ”‚
โ”‚    414 โ”‚   โ”‚   โ”‚   )                                                                             โ”‚
โ”‚    415 โ”‚   โ”‚   return safe_load_file(checkpoint_file)                                            โ”‚
โ”‚    416 โ”‚   try:                                                                                  โ”‚
โ”‚ โฑ  417 โ”‚   โ”‚   return torch.load(checkpoint_file, map_location="cpu")                            โ”‚
โ”‚    418 โ”‚   except Exception as e:                                                                โ”‚
โ”‚    419 โ”‚   โ”‚   try:                                                                              โ”‚
โ”‚    420 โ”‚   โ”‚   โ”‚   with open(checkpoint_file) as f:                                              โ”‚
โ”‚                                                                                                  โ”‚
โ”‚ /home/iikkahau/Projektit/miniconda3/envs/fie3/lib/python3.10/site-packages/torch/serialization.p โ”‚
โ”‚ y:777 in load                                                                                    โ”‚
โ”‚                                                                                                  โ”‚
โ”‚    774 โ”‚   โ”‚   โ”‚   # If we want to actually tail call to torch.jit.load, we need to              โ”‚
โ”‚    775 โ”‚   โ”‚   โ”‚   # reset back to the original position.                                        โ”‚
โ”‚    776 โ”‚   โ”‚   โ”‚   orig_position = opened_file.tell()                                            โ”‚
โ”‚ โฑ  777 โ”‚   โ”‚   โ”‚   with _open_zipfile_reader(opened_file) as opened_zipfile:                     โ”‚
โ”‚    778 โ”‚   โ”‚   โ”‚   โ”‚   if _is_torchscript_zip(opened_zipfile):                                   โ”‚
โ”‚    779 โ”‚   โ”‚   โ”‚   โ”‚   โ”‚   warnings.warn("'torch.load' received a zip file that looks like a To  โ”‚
โ”‚    780 โ”‚   โ”‚   โ”‚   โ”‚   โ”‚   โ”‚   โ”‚   โ”‚     " dispatching to 'torch.jit.load' (call 'torch.jit.loa  โ”‚
โ”‚                                                                                                  โ”‚
โ”‚ /home/iikkahau/Projektit/miniconda3/envs/fie3/lib/python3.10/site-packages/torch/serialization.p โ”‚
โ”‚ y:282 in __init__                                                                                โ”‚
โ”‚                                                                                                  โ”‚
โ”‚    279                                                                                           โ”‚
โ”‚    280 class _open_zipfile_reader(_opener):                                                      โ”‚
โ”‚    281 โ”‚   def __init__(self, name_or_buffer) -> None:                                           โ”‚
โ”‚ โฑ  282 โ”‚   โ”‚   super(_open_zipfile_reader, self).__init__(torch._C.PyTorchFileReader(name_or_bu  โ”‚
โ”‚    283                                                                                           โ”‚
โ”‚    284                                                                                           โ”‚
โ”‚    285 class _open_zipfile_writer_file(_opener):                                                 โ”‚
โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ
RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory
TurkuNLP Research Group org

Thank you for noting this, there appears to have been some issue with the upload! We'll reupload ASAP

TurkuNLP Research Group org

This appears to have been an issue with git lfs file size limits. We've now re-uploaded a sharded version of the model, and it appears to be working OK via from_pretrained. Please let us know if you encounter any further issues!

spyysalo changed discussion status to closed

Sign up or log in to comment