上下文长度只有512?

#3
by YUCYU - opened
This comment has been hidden
YUCYU changed discussion status to closed
YUCYU changed discussion status to open
YUCYU changed discussion status to closed
YUCYU changed discussion status to open

Context length和原模型是一样的,8K。

我也有同样的问题。 您说是8192,但是超过512代币的输入上显示错误信息。

https://github.com/abetlen/llama-cpp-python?tab=readme-ov-file#adjusting-the-context-window
llama_cpp_python context-window默认长度是512 通过n_ctx参数配置
llm = Llama(
model_path="./models/7B/llama-model.gguf",
# n_gpu_layers=-1, # Uncomment to use GPU acceleration
# seed=1337, # Uncomment to set a specific seed
# n_ctx=2048, # Uncomment to increase the context window
)

Sign up or log in to comment