Can't load in TextGen w/llamacpp_HF
#3
by
biship
- opened
Error: Could not load the model because a tokenizer in Transformers format was not found
However, i can load it it with llama.cpp.
Do you have the a tokenizer_config.json ?
Only a LoRA adapter for Mistral-Instruct-7B-v0.2 and a GGUF quantization for llama.cpp are provided here, not the full FP16 weights; so tokenizer_config.json is not needed. The tokenizer is the same as Mistral-7B.
Ah ok. Is there a difference between running it the two different ways?
There shouldn't be; the provided GGUF quantization should be almost lossless.
lemonilia
changed discussion status to
closed