Can't load any longlora model with Transformers package.
Hello,
Thanks for your amazing work! Whenever I try to access any of your Llama longlora models using the transformers package: "pipe = pipeline("text-generation", model="Yukang/Llama-2-13b-longlora-64k")" I get this type of error:
"*** OSError: Can't load the configuration of '/dataset/pretrained-models/Llama-2-13b-hf'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure '/dataset/pretrained-models/Llama-2-13b-hf' is the correct path to a directory containing a config.json file".
Interestingly, I don't have any problem to load your fully fine-tuned models (e.g: Llama-2-13b-longlora-32k or LongAlpaca-13B), so I believe it has to do with these longlora models where you have the adapter_config.json files which redirects to directories which can't be found ""base_model_name_or_path": "/dataset/pretrained-models/Llama-2-13b-hf"". Is there anything I can try to access these models using the transformers package or something obvious that I might have forgotten for this set of models?
Best,
Julian
Hi,
Thanks for your question. I have changed the "base_model_name_or_path" to be"meta-llama/Llama-2-13b-hf". It would be all right for this case. Would you please have try again?
Regards,
Yukang Chen
It works, thanks! Can you also do the same for your other longlora models (e.g: Llama-2-70b-longlora-32k, Llama-2-70b-chat-longlora-32k, ..) please?
Yes. I have changed them.