shenzhi-wang

/

Llama3-8B-Chinese-Chat-GGUF-8bit

like 141

Context length和原模型是一样的，8K。

我也有同样的问题。 您说是8192，但是超过512代币的输入上显示错误信息。

https://github.com/abetlen/llama-cpp-python?tab=readme-ov-file#adjusting-the-context-window
llama_cpp_python context-window默认长度是512 通过n_ctx参数配置
llm = Llama(
model_path="./models/7B/llama-model.gguf",
# n_gpu_layers=-1, # Uncomment to use GPU acceleration
# seed=1337, # Uncomment to set a specific seed
# n_ctx=2048, # Uncomment to increase the context window
)