TheBloke

/

WizardLM-Uncensored-SuperCOT-StoryTelling-30B-SuperHOT-8K-fp16

like 8

Hey Guys,

im trying to load this into Oobabooga with the new ExLlama_HF loeader and im getting the following error:

Traceback (most recent call last):
File "C:\Users*\OneDrive\Desktop\Bot2*\text-generation-webui\server.py", line 68, in load_model_wrapper
shared.model, shared.tokenizer = load_model(shared.model_name, loader)
File "C:\Users*\OneDrive\Desktop*\Richtig\text-generation-webui\modules\models.py", line 74, in load_model
output = load_func_maploader
File "C:\Users*\OneDrive\Desktop\Bot2*\text-generation-webui\modules\models.py", line 301, in ExLlama_HF_loader
return ExllamaHF.from_pretrained(model_name)
File "C:\Users*\OneDrive\Desktop\Bot2*\text-generation-webui\modules\exllama_hf.py", line 116, in from_pretrained
return ExllamaHF(config)
File "C:\Users*\OneDrive\Desktop\Bot2*\text-generation-webui\modules\exllama_hf.py", line 31, in init
self.ex_model = ExLlama(self.ex_config)
File "C:\Users*\OneDrive\Desktop\Bot2*\installer_files\env\lib\site-packages\exllama\model.py", line 646, in init
with safe_open(self.config.model_path, framework="pt", device="cpu") as f:
safetensors_rust.SafetensorError: Error while deserializing header: HeaderTooLarge

Can somebody help me with this? ty
I have a 2080 TI and a 1660.