Instructions to use Aquiles-ai/Kairos-Proj-80k with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Aquiles-ai/Kairos-Proj-80k with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Aquiles-ai/Kairos-Proj-80k", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("Aquiles-ai/Kairos-Proj-80k", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Aquiles-ai/Kairos-Proj-80k with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Aquiles-ai/Kairos-Proj-80k"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Aquiles-ai/Kairos-Proj-80k",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Aquiles-ai/Kairos-Proj-80k

SGLang

How to use Aquiles-ai/Kairos-Proj-80k with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Aquiles-ai/Kairos-Proj-80k" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Aquiles-ai/Kairos-Proj-80k",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Aquiles-ai/Kairos-Proj-80k" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Aquiles-ai/Kairos-Proj-80k",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Aquiles-ai/Kairos-Proj-80k with Docker Model Runner:
```
docker model run hf.co/Aquiles-ai/Kairos-Proj-80k
```

Kairos-Proj-80k

⚠️ Experimental research artifact — NOT competent. This checkpoint comes from a series of experiments on how to build a small multimodal model from scratch. It is not a usable vision-language model. Do not use it in production, for automated decisions, or in any safety-critical context.

Overview

Kairos-Proj-80k is the second checkpoint in the Kairos experiment progression: Kairos-Initialized with the projector aligned on 80,000 image-caption pairs from Aquiles-ai/LLaVA-CC3M-Pretrain-595K-Embedded (the embedded version of LLaVA-CC3M-Pretrain-595K).

Vision tower: Aquiles-ai/MoonViT-3D, extracted from moonshotai/Kimi-K2.6. Frozen.
Projector: Kimi-style 2-layer MLP (LayerNorm → Linear → GELU → Linear), trained on image-caption pairs with an L2 output cap (projector_output_scale = 0.89, ≈ the mean norm of LFM2.5 text embeddings).
LLM: LiquidAI/LFM2.5-2.6B. Frozen.
Modality: image ↔ text only (no video).

Training

Classic LLaVA-style stage-1 projector alignment (via train_projector.py):

Setting	Value
Dataset	`Aquiles-ai/LLaVA-CC3M-Pretrain-595K-Embedded` (80k samples)
Trainable	Projector only (fp32); vision tower + LLM frozen (bf16)
Initialization	Zero-init `out_proj` (start from the text-only prior)
Learning rate	1e-3, cosine, 100 warmup steps
Effective batch size	64 (16 × 4 grad-accum)
Epochs	1
Output cap	`projector_output_scale = 0.89` (hard L2 cap)
Supervision	Image-caption pairs; empty-think ( `thinking response<CAPTION>`)

What this checkpoint shows

This is the stage where the classic LLaVA recipe was tested and measured as insufficient for free generation:

Training loss (cross-entropy) drops correctly, and an ablation comparing the same setup with vs. without the image shows a clear +3.7 nats gap, i.e. the projector does feed the LLM image-derived signal.
However, in free generation the image only moves logits without flipping the greedy argmax: tokens related to the image gain probability, but never enough to beat the tokens the frozen LLM already prefers. The model never describes what it sees, regardless of prompt.

In short: the projector aligns content under teacher-forcing but produces no behavioral grounding. This checkpoint is therefore not a usable VLM; it is the warm-start for the next experiment.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoProcessor

model_id = "Aquiles-ai/Kairos-Proj-80k"

model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True, dtype=torch.bfloat16)
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)

Requires transformers >= 5.x (developed with 5.14.1) and trust_remote_code=True.

Limitations

Not competent for image understanding. Generation is image-independent in practice.
Projector-only alignment against a frozen LLM is insufficient for conversational visual behavior (that is the very finding of this experiment).

References

Blog post: Kairos: Building a Multimodal Model with LFM2.5 and Kimi-K2.6
Public repo: Aquiles-ai/Kairos
Training dataset: Aquiles-ai/LLaVA-CC3M-Pretrain-595K-Embedded

License

This model card describes a research artifact assembled from third-party components (MoonViT-3D from Kimi-K2.6, LFM2.5-2.6B). Check each component's license before any use.