mmaaz60

/

LLaVA-7B-Lightening-v1-1

like 7

python3.8 hf_tmp.py
Downloading (…)lve/main/config.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 839/839 [00:00<00:00, 124kB/s]
Traceback (most recent call last):
File "hf_tmp.py", line 6, in
model = AutoModel.from_pretrained('mmaaz60/LLaVA-7B-Lightening-v1-1', cache_dir='./cache_dir')
File "/usr/local/python/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py", line 456, in from_pretrained
config, kwargs = AutoConfig.from_pretrained(
File "/usr/local/python/lib/python3.8/site-packages/transformers/models/auto/configuration_auto.py", line 957, in from_pretrained
config_class = CONFIG_MAPPING[config_dict["model_type"]]
File "/usr/local/python/lib/python3.8/site-packages/transformers/models/auto/configuration_auto.py", line 671, in getitem
raise KeyError(key)
KeyError: 'llava'

Hi, please note that you have to set model_type='llava'. Please have a look at here and here for reference.

Also, please try using LLaVA or Video-ChatGPT official repo to load this model and it should work. Thank you and would be happy to discuss it further in case if you have any comments/questions. Thank You.