Does this work with Ollama?
got the error invalid file magic
Corrupted download maybe? The magic number is correct for me
Redownloaded and tried again. Got the same error :(
Hmm. I’m sorry, I have no idea. As you can see in the README it works just fine for me, and I’ve had no other complaints. Maybe make sure you’re in the latest llama.cpp version?
getting "GGML_ASSERT: ggml-metal.m:1540: false && "MUL MAT-MAT not implemented"" crash with latest compiled llama.cpp on M3 Max
getting "GGML_ASSERT: ggml-metal.m:1540: false && "MUL MAT-MAT not implemented"" crash with latest compiled llama.cpp on M3 Max
@vidumec Retry with batch size >= 16 for the time being. bfloat16 support is still being worked on
didn't help, but i just realized looking at the code, it hasn't been implemented for Metal, so it could only work on CPU