Repository: Mozilla-Ocho/llamafile

Issue #191: `SIGABRT` when running llamafile on macos 13.2.1 (darwin 22.3.0)
Created: 2024-01-10T22:58:23Z
./mistral-7b-instruct-v0.2.Q5_K_M.llamafile produces the following error on my mac machine:
...
error: Uncaught SIGABRT (SI_0) on MacBook-Pro.local pid 35366 tid 35366
 /Users/tybug/Desktop/mistral/mistral-7b-instruct-v0.2.Q5_K_M.llamafile
 No such file or directory
...

I know you only officially claim to support 23.1.0+, so maybe this shouldn't be a surprise, but I figured I would report it anyway.
Full logs and stdout are attached, including the automatically created main.log.
logs.zip

Duplicate issue #206: mistral-7b-instruct-v0.2.Q5_K_M.llamafile is crashing
Created: 2024-01-16T04:51:58Z
**While running ./mistral-7b-instruct-v0.2.Q5_K_M.llamafile --server  it crashes on MacBook Pro M1 and M2.
Here is the error:**
greater than the recommended max working set size
.ape-1.10(75705,0x1e2481000) malloc: *** error for object 0x1000806df750: pointer being freed was not allocated
.ape-1.10(75705,0x1e2481000) malloc: *** set a breakpoint in malloc_error_break to debug
error: Uncaught SIGABRT (SI_0) on df84d899228f7.cogeco.local pid 75705 tid 75705
/Users/vicconsult/llm/mistral-7b-instruct-v0.2.Q5_K_M.llamafile
No such file or directory
Darwin Cosmopolitan 3.2.4 MODE=aarch64; Darwin Kernel Version 23.2.0: Wed Nov 15 21:53:18 PST 2023; root:xnu-10002.61.3~2/RELEASE_ARM64_T6000 df84d899228f7.cogeco.local 23.2.0
cosmoaddr2line /Users/vicconsult/llm/mistral-7b-instruct-v0.2.Q5_K_M.llamafile 18bc6a0dc 7e3e80018bbada40 495300018bac4b08 1a2b00018bac83f4 711c80018badcebc 1770800104d5ac48 104d614d0 100000c4dd4 1000012ab90 1000010f4fc 1000006a0e8 1000002afe4
0000000000000000 x0 d7da92b85d8876d8 x8  0000000000000148 x16 0000000000000000 x24
0000000000000000 x1 d7da92b9bfc066d8 x9  00000001eb4f3340 x17 0000000000000000 x25
0000000000000000 x2 000000000000000a x10 0000000000000000 x18 000000016b0cb882 x26
0000000000000000 x3 0000000000000000 x11 0000000000000006 x19 00000001e2481000 x27
0000000000000073 x4 0000000000000037 x12 00000001e2481000 x20 00000100003740c0 x28
000000000000002e x5 0000000000000000 x13 0000000000000103 x21 000000016b0c4e80 x29
0000000000000001 x6 0000000000000000 x14 00000001e24810e0 x22 000000018bca1cc0 x30
0000000000000000 x7 0000000000000000 x15 000000016b0c5530 x23 000000016b0c4e60 x31
000000016b0c4e60 sp 18bc6a0dc pc NULL-1950044708
000000016b0c4e80 fp 7e3e80018bbada40 lr NULL-1950816448
000000016b0c4ec0 fp 495300018bac4b08 lr NULL-1951770616
000000016b0c5500 fp 1a2b00018bac83f4 lr NULL-1951756044
000000016b0c5520 fp 711c80018badcebc lr NULL-1951671364
000000016b0c5580 fp 1770800104d5ac48 lr NULL+80984392
000000016b0c5660 fp 104d614d0 lr NULL+81011152
000000016b0c5710 fp 100000c4dd4 lr ggml_backend_alloc_buffer+40
000000016b0c5720 fp 1000012ab90 lr llama_new_context_with_model+2312
000000016b0c5740 fp 1000010f4fc lr llama_init_from_gpt_params(gpt_params&)+448
000000016b0c6c00 fp 1000006a0e8 lr llama_server_context::load_model(gpt_params const&)+1184
000000016b0c70f0 fp 1000002afe4 lr server_cli(int, char**)+1788
000000016b0c74d0 fp 10000005250 lr main+1256
000000016b0c8590 fp 10000013270 lr cosmo+1188
000000016b0ca720 fp 10000000148 lr _start
Abort trap: 6

Closing comment (2024-01-20T07:42:58Z): Marking as duplicate of #191. We'll have a bug fix release published shortly. In the meantime you can build llamafile at HEAD running make -j8 and sudo make install. You can then run your llamafiles with the latest software by saying llamafile -m foo.llamafile ..... Thanks for reporting this and thanks for using llamafile!