Issues running the model in python
Hello anon! I came across your repository not so long ago and I wanted to use it in a Python application, however I haven't got luck on my side and keep getting this error:
OSError: Could not locate pytorch_model-00001-of-00006.bin inside anon8231489123/gpt4-x-alpaca-13b-native-4bit-128g
This is the code I'm currently using if by any chance you would need it when resolving this issue:
from transformers import AutoTokenizer, AutoModelForCausalLM
rep= "anon8231489123/gpt4-x-alpaca-13b-native-4bit-128g"
tokenizer = AutoTokenizer.from_pretrained(rep)
model = AutoModelForCausalLM.from_pretrained(rep)
inputs = tokenizer(["Today is"], return_tensors="pt")
reply_ids = model.generate(**inputs, max_new_tokens=590) # return_dict_in_generate=True, output_scores=True
outputs = tokenizer.batch_decode(reply_ids, skip_special_tokens=True)[0]
print(outputs)
@Kralos-R for the from_pretrained try passing in the full path to the .pt file
I'm having the same problem as @Kralos-R . I tried the solution from @faaaaaaaaaaaa by giving from_pretrained the full path to the .pt file but transformers seems to still think it's supposed to be a Repo ID. Here's the error I get:
huggingface_hub.utils.validators.HFValidationError: Repo id must use alphanumeric chars or '-', '', '.', '--' and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 96: '\programming\openai\models\gpt4-x-alpaca-13b-native-4bit-128g\gpt-x-alpaca-13b-native-4bit-128g.pt'.
My code up to the point where it spits that error message looks like this. (The print line about the file path outputs the correct path and says the file exists.)
from transformers import AutoTokenizer, AutoModelForCausalLM
from pathlib import Path
modelpath = Path("/programming/openai/models/gpt4-x-alpaca-13b-native-4bit-128g/gpt-x-alpaca-13b-native-4bit-128g.pt")
print("Model path: ", modelpath.absolute(), "Exists: ", modelpath.exists())
tokenizer = AutoTokenizer.from_pretrained(modelpath)
I even tried moving the file to the same directory as my .py file and renaming it to just "alpaca" to get rid of the "." and "/" and stuff, but then it tried to download it from Huggingface Hub:
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/alpaca/resolve/main/tokenizer_config.json
Any suggestions are appreciated!