bloomer010

/

Ling-3.0-flash-GGUF

like 27

Hey!
First of all, thank you for quantizing this beautiful model! 🤗

Do this model have 131k context size? llama-server caps it to 131k when I specify 262k.

>>>

./llama-server --model "/home/dim/EvoX2/LLMs/models/bloomer010/Ling-3.0-flash-GGUF/Ling-3.0-flash-UD-Q6_K_XL.gguf" --host 192.168.1.86 --port 8080 --tools get_info --api-key XXX --jinja --chat-template-file "/home/dim/EvoX2/LLMs/models/bloomer010/Ling-3.0-flash-GGUF/chat_template.jinja" --gpu-layers all --parallel 4 --kv-unified --kv-offload --load-mode none --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --ctx-size 262144 --batch-size 4096 --ubatch-size 2048 --swa-checkpoints 48 --timeout 999999 --temperature 0.7 --top-k 30 --repeat-penalty 1.05 --presence-penalty 0.05 --top-p 0.95 --min-p 0.05 --spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-p-min 0.5 --cache-reuse 256

<<<

0.00.026.513 W server tools or MCP servers are enabled, using localhost as default CORS origin (change via --cors-origins)
0.00.034.359 I cmn  common_param: common_params_print_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.034.713 W srv  llama_server: -----------------
0.00.034.714 W srv  llama_server: the following feature(s) are enabled:
0.00.034.714 W srv  llama_server:     built-in tools (experimental)
0.00.034.714 W srv  llama_server: do not expose the server to untrusted environments
0.00.034.715 W srv  llama_server: -----------------
0.00.035.903 I srv    load_model: loading model '/home/dim/EvoX2/LLMs/models/bloomer010/Ling-3.0-flash-GGUF/Ling-3.0-flash-UD-Q6_K_XL.gguf'
0.00.544.758 W load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
0.30.890.375 W llama_context: n_ctx_seq (262144) > n_ctx_train (131072) -- possible training context overflow
0.31.217.244 I cmn          init: llama threadpool init, n_threads = 16
0.31.313.663 I common_speculative_init_result: creating MTP draft context against the target model '/home/dim/EvoX2/LLMs/models/bloomer010/Ling-3.0-flash-GGUF/Ling-3.0-flash-UD-Q6_K_XL.gguf'
0.31.313.695 W llama_context: n_ctx_seq (262144) > n_ctx_train (131072) -- possible training context overflow
0.31.482.507 W srv    load_model: the slot context (262144) exceeds the training context of the model (131072) - capping    <======== HERE
0.31.607.134 I srv    load_model: initializing, n_slots = 4, n_ctx_slot = 131072, kv_unified = 'true'
0.31.853.533 I srv          init: chat template supports preserving reasoning, consider enabling it via --reasoning-preserve
0.31.853.588 I srv  llama_server: model loaded
0.31.853.594 I srv  llama_server: listening on http://192.168.1.86:8080