TheBloke/rocket-3B-GGUF · Error when loading rocket-3B-GGUF using text-generation-webui

Nov 23, 2023

When loading this model I am getting an error:

Traceback (most recent call last):
File "S:\oobabooga_windows\text-generation-webui\modules\ui_model_menu.py", line 209, in load_model_wrapper

shared.model, shared.tokenizer = load_model(shared.model_name, loader)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "S:\oobabooga_windows\text-generation-webui\modules\models.py", line 85, in load_model

output = load_func_maploader
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "S:\oobabooga_windows\text-generation-webui\modules\models.py", line 249, in llamacpp_loader

model, tokenizer = LlamaCppModel.from_pretrained(model_file)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "S:\oobabooga_windows\text-generation-webui\modules\llamacpp_model.py", line 90, in from_pretrained

result.model = Llama(**params)
^^^^^^^^^^^^^^^
File "S:\oobabooga_windows\text-generation-webui\installer_files\env\Lib\site-packages\llama_cpp_cuda\llama.py", line 365, in init

assert self.model is not None
^^^^^^^^^^^^^^^^^^^^^^
AssertionError

HR1777

Nov 24, 2023

I get the same errors in text-generation-webui. Please guide us.

SmoothBrainApe

Nov 25, 2023

This is the error I got

GGML_ASSERT: /home/runner/work/llama-cpp-python-cuBLAS-wheels/llama-cpp-python-cuBLAS-wheels/vendor/llama.cpp/ggml.c:13711: tensor->src[0] == NULL || tensor->src[0]->backend == GGML_BACKEND_CPU
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
    - Avoid using `tokenizers` before the fork if possible
    - Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
Could not attach to process.  If your uid matches the uid of the target
process, check the setting of /proc/sys/kernel/yama/ptrace_scope, or try
again as the root user.  For more details, see /etc/sysctl.d/10-ptrace.conf
ptrace: Operation not permitted.
No stack.
The program is not being run.
/arrow/cpp/src/arrow/filesystem/s3fs.cc:2829:  arrow::fs::FinalizeS3 was not called even though S3 was initialized.  This could lead to a segmentation fault at exit

I downloaded Q6_K version.