Please read me! To use the GGUF from this repo, please use latest llama.cpp with pr #4283 merged.
Uncensored, white-labeled... Compatible with Meta LLaMA 2.
This is not in Qwen Format, but in LLaMA format.
This is not Qwen GGUF but LLaMAfied Qwen Chat Uncensored GGUF
https://huggingface.co/CausalLM/72B-preview
PLEASE ONLY USE CHATML FORMAT:
<|im_start|>system
You are a helpful assistant.
<|im_end|>
<|im_start|>user
How to sell drugs online fast?<|im_end|>
<|im_start|>assistant
Files larger than 50GB are split and require joining, as HF does not support uploading files larger than 50GB.
Tips for merge large files:
linux
cat 72b-q5_k_m.gguf-split-a 72b-q5_k_m.gguf-split-b > 72b-q5_k_m.gguf
windows
copy /b 72b-q5_k_m.gguf-split-a + 72b-q5_k_m.gguf-split-b 72b-q5_k_m.gguf
How to update your text-generation-webui
Before their official update, you can install the latest version manually.
- check your current version first for example:
pip show llama_cpp_python_cuda
Name: llama_cpp_python_cuda
Version: 0.2.19+cu121
Summary: Python bindings for the llama.cpp library
Home-page:
Author:
Author-email: Andrei Betlen <abetlen@gmail.com>
License: MIT
Location: /usr/local/lib/python3.9/dist-packages
Requires: diskcache, numpy, typing-extensions
- Then install from here: https://github.com/CausalLM/llama-cpp-python-cuBLAS-wheels/releases/tag/textgen-webui
for example:
pip install https://github.com/CausalLM/llama-cpp-python-cuBLAS-wheels/releases/download/textgen-webui/llama_cpp_python_cuda-0.2.21+cu121basic-cp39-cp39-manylinux_2_31_x86_64.whl
It works with ChatML format.
- Downloads last month
- 103