crivo-150m-GGUF

💡 Important Deployment Notes: The agent that performed this GGUF conversion believed that changes to llama.cpp would be required in order to run this model. These changes are based on the b11552 tag (commit 23b0202a189c44a54625aadcb37a946dd1d6278d). They were captured in llama.cpp-required-changes.patch. These changes were tested and appeared to work, however, they are not upstreamed yet.

I decided to use crivo-150m because it had a high score on https://huggingface.co/spaces/multimodalart/jev-decision-index compared to its parameter count.

This GGUF conversion / quantization was generated by a local agent powered by Qwen3.8 27B. See agent-trajectory-log-part1-plan.md and agent-trajectory-log-part2-implement.md for exhaustive details.

Run with https://llama.app

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build -DGGML_VULKAN=ON
cmake --build build --config Release
./build/bin/llama-server -hf crivo-150m-GGUF:Q8_0 -ngl 999

This is a decision model, to be used via /v1/systemone API. See https://github.com/ggml-org/llama.cpp/pull/29818

Source models

• https://huggingface.co/tardellirs/crivo-150m

Usage example

curl -s http://localhost:9931/v1/systemone \
  -H "Content-Type: application/json" \
  -d '{
    "state": "Paris is the capital of France.",
    "questions": {
      "c": {
        "type": "choice",
        "instructions": "Which country is Paris the capital of?",
        "criteria": { "France": null, "Japan": null, "Brazil": null }
      }
    }
  }'

how to upload to huggingface:

nerdctl build -f Containerfile.hfcli  --progress plain -t hfcli .

nerdctl run -e HF_TOKEN=<token that has write permissions> -v './:/mymodel' --entrypoint /bin/bash hfcli -c 'cd /mymodel && whoami && ls -lah && /root/.local/bin/hf upload breaksoftware/crivo-150m-GGUF .'