Complete noob question - cloned the repository, now what?

by hansintheair - opened

I'm a complete newcomer to hugging face, and would like to try running this model on my machine.

I created a new environment with:

conda create -n model-env python=3.10 transformers pytorch

activated the environment, read through the readme, and copied the quick start code given there into a new .py file.

It seems the GPT-JT-6B-v1 folder contains all the files necessary to run this model, so modified the path to point to my local directory:

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(r"C:\<path>\test\GPT-JT-6B-v1")
tokenizer = AutoTokenizer.from_pretrained(r"C:\<path>\test\GPT-JT-6B-v1")

However, when I run this I get the following error:

OSError: Unable to load weights from pytorch checkpoint file for 'C:\<path>\test\GPT-JT-6B-v1\pytorch_model.bin' at 'C:\<path>\test\GPT-JT-6B-v1\pytorch_model.bin'. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.

Am I doing it wrong??

Have you tried letting the transformers model handle the installation of the model?

For example:
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("togethercomputer/GPT-JT-6B-v1")

model = AutoModelForCausalLM.from_pretrained("togethercomputer/GPT-JT-6B-v1")

This code will download all the files itself and manage them

Together org

Hello, as @DarwinAnim8or mentioned, the transformers library can handle loading the model for you.
However, if you prefer to load the model from a local folder, please ensure that the folder C:\<path>\test\GPT-JT-6B-v1 directly contains all of the necessary files from this repository, and that there are no nested folders within it. Thank you 😊

hansintheair changed discussion status to closed

Thank you @juewang , it turns out the pytorch_model.bin file was corrupted. After I downloaded it independently and replaced the corrupted file, I got it running!

However, I ended up getting a bunch of weird jibberish to my prompt "once upon a time":

once upon a timeonceonceonceonlyonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonce onceonce once onceonce once once a timeonceonce
 once a timeonceonce once once a timeonceonce once a timeonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonceonce

Along with a warning:

The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.`

Looking into this now, but any pointers are much appreciated =]

hansintheair changed discussion status to open

Sign up or log in to comment