llama.cpp server for Meari — Windows x64, CPU

Build of the llama.cpp server (llama-server, MIT) used by Meari to run a small translation model (GGUF) on the CPU, without PyTorch. The Meari engine starts it as a child process on 127.0.0.1 only, sends one sentence at a time and stops it after a while without use.

Source: ggml-org/llama.cpp@0c1e57098bba43ac29e6e3b677cdceebdd22334f (the version bundled in llama-cpp-python 0.3.36, which was used for Meari's translation measurements). No changes.
CPU only, all x86-64 variants (GGML_CPU_ALL_VARIANTS, GGML_BACKEND_DL): llama-server.exe picks the best ggml-cpu-*.dll for the machine at run time. Web UI and download features are not built.
Includes the Microsoft C++ and OpenMP runtime DLLs (Visual Studio 2022 redistributable files) so it runs on a clean PC.
Build script: engine/tools/llama/build.ps1 in the Meari repository.

License: MIT — llama.cpp (LICENSE), cpp-httplib (LICENSE-httplib) and nlohmann/json (LICENSE-jsonhpp). The model weights are separate (see each model's license).

Downloads last month: -; Downloads are not tracked for this model. How to track