bartowski

/

Qwen2.5-Math-72B-Instruct-GGUF

like 1

Hi, thanks for your great job.

I encounter a problem when running with ollama(ollama run MY_MODEL_NAME 'Hi, who are you'), it would be greate if you could provide some tips to solve it. The version of ollama is 0.3.12

Problem

level=WARN source=memory.go:117 msg="model missing blk.0 layer size"
panic: runtime error: integer divide by zero

goroutine 106 [running]:
github.com/ollama/ollama/llm.EstimateGPULayers({_, _, _}, _, {_, _, _}, {{0x2000, 0x200, 0xffffffffffffffff, ...}, ...})
    github.com/ollama/ollama/llm/memory.go:124 +0x1f6a
github.com/ollama/ollama/llm.PredictServerFit({0xc000cb9bb8?, 0xf6784f?, 0xc00052d6c0?}, 0xc00060c000, {_, _, _}, {0x0, 0x0, 0x0}, ...)
    github.com/ollama/ollama/llm/memory.go:21 +0xdb
github.com/ollama/ollama/server.pickBestFullFitByLibrary(0xc000846700, 0xc00060c000, {0xc000380c08?, 0x8?, 0x8?}, 0xc000bb3d00)
    github.com/ollama/ollama/server/sched.go:713 +0x6f3
github.com/ollama/ollama/server.(*Scheduler).processPending(0xc0002150e0, {0x3fb0d350, 0xc0005a2c80})
    github.com/ollama/ollama/server/sched.go:247 +0xc5e
github.com/ollama/ollama/server.(*Scheduler).Run.func1()
    github.com/ollama/ollama/server/sched.go:107 +0x1f
created by github.com/ollama/ollama/server.(*Scheduler).Run in goroutine 1
    github.com/ollama/ollama/server/sched.go:106 +0xb4

Manifest

FROM ./Qwen2.5-Math-72B-Instruct-Q8_0-00001-of-00002.gguf
FROM ./Qwen2.5-Math-72B-Instruct-Q8_0-00002-of-00002.gguf

I have checked out the sha256 of these two files

$ echo "c4be09accb6526658c5f34165bb7a963fe5e029841530a8d1b0d1790192a1d12 ./Qwen2.5-Math-72B-Instruct-Q8_0-00001-of-00002.gguf"|sha256sum --check
$ Qwen2.5-Math-72B-Instruct-Q8_0-00001-of-00002.gguf: OK

$ echo "0292764180ae823a25de7bd83677009ff68682b3007af23d80dbd711de387ba5 ./Qwen2.5-Math-72B-Instruct-Q8_0-00002-of-00002.gguf"|sha256sum --check
$ Qwen2.5-Math-72B-Instruct-Q8_0-00002-of-00002.gguf: OK