FastVLM-0.5B for SpacemiT K1/K3

This repository contains a SpacemiT edge-deployment version of apple/FastVLM-0.5B. The FastVLM vision encoder is exported as an ONNX model and runs through the SpacemiT Execution Provider, while the Qwen2-based 0.5B text decoder is stored as a Q4_1 GGUF model and runs through the SpacemiT llama.cpp backend.

Model origin and acknowledgement

FastVLM was introduced by Apple in FastVLM: Efficient Vision Encoding for Vision Language Models, published at CVPR 2025. It is a vision-language model for understanding and describing images. The key FastViTHD encoder is designed to produce fewer visual tokens and reduce vision encoding latency, while retaining useful image understanding quality. The FastVLM-0.5B variant is the compact model in the released family.

This repository is a deployment conversion, not a new base model. We thank the Apple Machine Learning Research team for releasing the FastVLM model, paper, and source code:

• FastVLM paper
• Official FastVLM repository
• Official FastVLM-0.5B model

The current SpacemiT package exposes offline image understanding through the OpenAI-compatible llama-server chat completions API. The bundled humanspeech.jpg sample has been validated on both K1 and K3.

Files

FastVLM-0.5B-SpacemiT/
├── fastvlm-text-0.5B-Q4_1.gguf
├── fastvlm_vision.f16.onnx
├── configs/
│   ├── K1/config.json
│   └── K3/config.json
├── humanspeech.jpg
└── README.md

The included humanspeech.jpg shows a woman speaking at a podium and is used by the board smoke example below. fastvlm_vision.f16.onnx is the vision encoder; fastvlm-text-0.5B-Q4_1.gguf is the text decoder.

Supported platforms

Platform	Accelerated cores	SMT config	llama-server threads
SpacemiT K1 / X60	0,1,2,3	configs/K1	-t 4
SpacemiT K3 / A100	8,9,10,11,12,13,14,15	configs/K3	-t 8

The platform-specific config.json controls the ONNX vision encoder's SpaceMIT EP thread count and affinity. The -t argument controls the GGUF text decoder. Do not use the K3 config on K1: K1 has four accelerated cores at IDs 0-3, whereas K3 has eight accelerated cores at IDs 8-15.

The SpacemiT llama.cpp runtime automatically detects the accelerated CPU cores. Normal runs do not require manually setting SPACEMIT_PERFER_CORE_ARCH, SPACEMIT_PERFER_CORE_ID, or SPACEMIT_MEM_BACKEND.

Prerequisites

Two runtime components are required:

1. A SpacemiT ONNX Runtime package containing libonnxruntime and the SpacemiT Execution Provider: spacemit-com/onnxruntime releases
2. An SMT-enabled SpacemiT llama.cpp build containing llama-server: spacemit-com/llama.cpp

The validation below used SpacemiT ORT 2.0.6 and a RISC-V llama-server built from the SpacemiT fork.

Option A: use prebuilt packages

Download and unpack the current RISC-V glibc releases:

wget https://github.com/spacemit-com/onnxruntime/releases/download/2.0.6/spacemit-ort.riscv64.2.0.6.tar.gz
tar -xf spacemit-ort.riscv64.2.0.6.tar.gz

wget https://github.com/spacemit-com/llama.cpp/releases/download/v0.1.7/spacemit-llama.cpp.riscv64.0.1.7.tar.gz
tar -xf spacemit-llama.cpp.riscv64.0.1.7.tar.gz

Check the release pages for newer mutually compatible packages when deploying this model in another software image.

Option B: build llama.cpp from source

Cross-compilation also requires a SpacemiT RISC-V toolchain:

git clone --recursive https://github.com/spacemit-com/llama.cpp.git
cd llama.cpp

export RISCV_ROOT_PATH=/path/to/spacemit-riscv-toolchain
export SPACEMIT_ORT_DIR=/path/to/spacemit-ort.riscv64.2.0.6

bash build_spacemit.sh glibc

The installed runtime is generated under build/installed/. An equivalent manual CMake build must enable at least GGML_CPU_RISCV64_SPACEMIT=ON and LLAMA_SERVER_SMT_MTMD=ON, and pass SPACEMIT_ORT_DIR to CMake.

Run on a K1 or K3 board

Copy this model folder, the unpacked ORT package, and the prebuilt or locally built llama.cpp installation to the board. Set the following paths to match their actual locations:

export MODEL_DIR=/path/to/FastVLM-0.5B-SpacemiT
export ORT_DIR=/path/to/spacemit-ort.riscv64.2.0.6
export LLAMA_DIR=/path/to/spacemit-llama.cpp.riscv64.0.1.7

export LD_LIBRARY_PATH="${LLAMA_DIR}/lib:${ORT_DIR}/lib:${LD_LIBRARY_PATH:-}"

K1

K1 uses four accelerated cores, 0-3:

"${LLAMA_DIR}/bin/llama-server" \
  -m "${MODEL_DIR}/fastvlm-text-0.5B-Q4_1.gguf" \
  --media-backend smt \
  --smt-config-dir "${MODEL_DIR}/configs/K1" \
  -t 4 \
  --host 0.0.0.0 \
  --port 8080 \
  --warmup

The K1 vision config applies:

"ep_config": {
  "SPACEMIT_EP_INTRA_THREAD_NUM": "4",
  "SPACEMIT_EP_INTER_THREAD_NUM": "1",
  "SPACEMIT_EP_INTRA_THREAD_AFFINITY": "0;1;2;3"
}

K3

K3 uses eight accelerated cores, 8-15:

"${LLAMA_DIR}/bin/llama-server" \
  -m "${MODEL_DIR}/fastvlm-text-0.5B-Q4_1.gguf" \
  --media-backend smt \
  --smt-config-dir "${MODEL_DIR}/configs/K3" \
  -t 8 \
  --host 0.0.0.0 \
  --port 8080 \
  --warmup

The K3 vision config applies:

"ep_config": {
  "SPACEMIT_EP_INTRA_THREAD_NUM": "8",
  "SPACEMIT_EP_INTER_THREAD_NUM": "1",
  "SPACEMIT_EP_INTRA_THREAD_AFFINITY": "8;9;10;11;12;13;14;15"
}

Wait until the server prints a line similar to:

llama_server: listening on http://0.0.0.0:8080

If the service is reachable outside a trusted local network, configure an API key and suitable network access controls instead of exposing an unauthenticated 0.0.0.0 endpoint.

Send an image understanding request

The request can be sent from the board itself or another machine that can reach the board. Set SERVER_URL=http://127.0.0.1:8080 when testing locally, or replace BOARD_IP when testing remotely.

export SERVER_URL=http://127.0.0.1:8080

base64 < "${MODEL_DIR}/humanspeech.jpg" | tr -d '\n' | jq -Rs '{
  messages: [
    {
      role: "user",
      content: [
        {
          type: "image_url",
          image_url: {
            url: ("data:image/jpeg;base64," + .)
          }
        },
        {
          type: "text",
          text: "Describe the image content."
        }
      ]
    }
  ],
  max_tokens: 64,
  temperature: 0,
  stream: false,
  chat_template_kwargs: {
    enable_thinking: false
  }
}' | curl "${SERVER_URL}/v1/chat/completions" \
  -H "Content-Type: application/json" \
  --data-binary @-

The generated description is returned in choices[0].message.content. Replace humanspeech.jpg with another JPEG/PNG image to describe your own image.

Verified example

The bundled humanspeech.jpg was tested on 2026-08-10 with --warmup enabled. Both boards returned HTTP 200 and generated a description of the speaker, podium, microphone, clothing, and background banner. With max_tokens=64, the responses were truncated at the requested token limit.

K1 example content:

The image depicts a woman standing at a podium, delivering a speech or presentation. She is positioned at the center of the frame, with her right hand raised, holding a microphone close to her mouth, suggesting she is speaking. She is dressed in a patterned top and dark pants. Behind her, there is a banner

K3 example content:

The image depicts a woman standing at a podium, delivering a speech or presentation. She is positioned at the center of the frame, with her right hand raised, holding a microphone close to her mouth, suggesting she is actively speaking. She is dressed in a patterned top and dark pants. Behind her, there is a

Observed single-request wall time after server warmup:

Board	HTTP status	Wall time
K1 (0-3, -t 4)	200	9.79 s
K3 (8-15, -t 8)	200	1.91 s

These are functional smoke-test observations rather than a formal benchmark. Startup and first-request time can be longer while the GGUF model is loaded and the ONNX graph is compiled.

Citation

Please cite the original FastVLM work when using this converted model:

@InProceedings{fastvlm2025,
  author = {Pavan Kumar Anasosalu Vasu and Fartash Faghri and Chun-Liang Li and Cem Koc and Nate True and Albert Antony and Gokul Santhanam and James Gabriel and Peter Grasch and Oncel Tuzel and Hadi Pouransari},
  title = {FastVLM: Efficient Vision Encoding for Vision Language Models},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  month = {June},
  year = {2025},
}

License

The original FastVLM model is released under Apple's AMLR model license. See the official model card and the official repository's LICENSE_MODEL for the terms that apply to the model. The SpacemiT llama.cpp and ONNX Runtime packages are separate dependencies and remain subject to their respective repository licenses.