EntityDeletr

/

North-Mini-Code-1.0-EAGLE3-GGUF

like 0

I first used the 4-bit quantization of North-Mini-Code-1.0 from Unsloth, along with North-Mini-Code-1.0-EAGLE3.gguf (Q5_K_M), and the model loading process seemed just fine.

After the first prompt was inputted, the following error message was displayed:

...
[60953] /home/runner/work/llama.cpp/llama.cpp/src/llama-graph.cpp:1247: GGML_ASSERT(t_layer_inp[il] != nullptr && "layer input tensor is null") failed
[60953] [New LWP 665349]
...
[60953] warning: 30 ../sysdeps/unix/sysv/linux/wait4.c: No such file or directory
...
[60953] warning: 58 ../sysdeps/nptl/libc_start_call_main.h: No such file or directory
[60953] #15 0x0000725f7662a28b in __libc_start_main_impl (main=0x62e3e191e260 >

, argc=40, argv=0x7fffa526cb58, init=, fini=, rtld_fini=, stack_end=0x7fffa526cb48) at ../csu/libc-start.c:360
[60953] warning: 360 ../csu/libc-start.c: No such file or directory
[60953] #16 0x000062e3e191e295 in _start ()
[60953] [Inferior 1 (process 665264) detached]
0.30.218.401 E srv operator(): http client error: Failed to read connection
0.30.225.337 E srv operator(): http client error: Could not establish connection
...

Then I tried a different one from Bartowski, and still no luck. If no draft model was in use, no error occurred.

Below is my local setup:

• llama.cpp version: b10182
• OS version: Kubuntu 24.04.4 LTS (Linux 7.0.0-28-generic)
• Sampling parameters:

[North-Mini-Code-1.0-30B-A3B-Instruct-UD-Q4_K_XL]
model = /home/XXX/Models/North-Mini-Code-1.0-Q4_K_M.gguf
chat-template-kwargs = {"reasoning_effort": "low"}
model-draft = /home/XXX/Models/North-Mini-Code-1.0-EAGLE3.gguf
spec-default = true
spec-type = draft-eagle3
spec-draft-n-max = 2
temp = 1.0
top-p = 0.95
n-predict = 64000
ctx-size = 256000

Did I do it wrongly? Please advise and thank you in advance!