LoLLMS Web UI which binding?

#1
by Silverspoon7 - opened

Sorry for asking, the q3_k isn‘t recognized (unknown safetensors ID 11) with pyllama
q4_0-Version: error loading model: missing tok_embeddings.weight
llama_init_from_file: failed to load model

ok think i made it with c_transformers

Yeah ctransformers is correct. I will make that clearer in the README

By the way, don‘t know if it cares, but I compared German usage in PyTorch model (online on huggingface, h2o chat), your 4 Bit and 3 Bit conversion. The 3 bit conversion is like testing a random 7B Model. Maybe, it does better in english. But in german , it’s like …damaged.

Sign up or log in to comment