Instructions to use Pluto-AI-Labs/Athena-1.0-4B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Pluto-AI-Labs/Athena-1.0-4B-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Pluto-AI-Labs/Athena-1.0-4B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Pluto-AI-Labs/Athena-1.0-4B-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Pluto-AI-Labs/Athena-1.0-4B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Pluto-AI-Labs/Athena-1.0-4B-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Pluto-AI-Labs/Athena-1.0-4B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf Pluto-AI-Labs/Athena-1.0-4B-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Pluto-AI-Labs/Athena-1.0-4B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Pluto-AI-Labs/Athena-1.0-4B-GGUF:Q4_K_M

Use Docker

docker model run hf.co/Pluto-AI-Labs/Athena-1.0-4B-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use Pluto-AI-Labs/Athena-1.0-4B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Pluto-AI-Labs/Athena-1.0-4B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Pluto-AI-Labs/Athena-1.0-4B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Pluto-AI-Labs/Athena-1.0-4B-GGUF:Q4_K_M

Ollama
How to use Pluto-AI-Labs/Athena-1.0-4B-GGUF with Ollama:
```
ollama run hf.co/Pluto-AI-Labs/Athena-1.0-4B-GGUF:Q4_K_M
```
Unsloth Desktop

How to use Pluto-AI-Labs/Athena-1.0-4B-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Pluto-AI-Labs/Athena-1.0-4B-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Pluto-AI-Labs/Athena-1.0-4B-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use Pluto-AI-Labs/Athena-1.0-4B-GGUF with Docker Model Runner:
```
docker model run hf.co/Pluto-AI-Labs/Athena-1.0-4B-GGUF:Q4_K_M
```

Lemonade

How to use Pluto-AI-Labs/Athena-1.0-4B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Pluto-AI-Labs/Athena-1.0-4B-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Athena-1.0-4B-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use Pluto-AI-Labs/Athena-1.0-4B-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Pluto-AI-Labs/Athena-1.0-4B-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Pluto-AI-Labs/Athena-1.0-4B-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Pluto-AI-Labs/Athena-1.0-4B-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Pluto-AI-Labs/Athena-1.0-4B-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Pluto-AI-Labs/Athena-1.0-4B-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Athena 1.0 4B — GGUF

GGUF distribution of Athena 1.0, developed by Pluto AI Labs through supervised adaptation of Qwen3.5-4B. The source is a standalone BF16 model with its trained language, vision and merger LoRA updates already merged. No separate PEFT adapter is required.

Release status: experimental. Quantization reduces storage and memory; it adds no training and establishes no benchmark improvement. GGUF exports receive structural and short runtime checks, not benchmark-quality certification.

Model and training

Property	Value
Source	Pluto-AI-Labs/Athena-1.0-4B
Source revision	`758a3dafa6e1aaa26cfe16bc79000104ec383edf`
Standalone parameters	4,539,265,536
Architecture	Qwen3.5 dense hybrid attention language model and vision encoder
Adaptation	67,715,072 LoRA parameters; merged BF16 source
Completed training	250,000 rows, one pass, 15,625 optimizer steps
Supervised tokens	28,933,709
Training window	4,096 tokens; up to 512 visual tokens per image
Training data	documents, OCR, charts, visual reasoning, conversations, grounding, identity
Identity weighting	3,000 rows at 1/6 loss weight
Training hardware	NVIDIA H100-SXM4-80GB across resumable Colab sessions
Conversion	CPU; llama.cpp `23b0202a189c44a54625aadcb37a946dd1d6278d`

The final training checkpoint was selected before evaluation. The 81.57% monitoring-NLL reduction is not an 81.57% accuracy gain. Native context configuration is inherited, but long-context quality was not established by this training. There was no dedicated video training block.

Downloads

File	Format	Size, GB
Athena-1.0-4B-F16.gguf	F16	8.424
mmproj-Athena-1.0-4B-F16.gguf	F16 vision	0.672
Athena-1.0-4B-Q4_K_M.gguf	Q4_K_M	2.709
Athena-1.0-4B-Q5_K_M.gguf	Q5_K_M	3.075
Athena-1.0-4B-Q6_K.gguf	Q6_K	3.464
Athena-1.0-4B-Q8_0.gguf	Q8_0	4.482
Athena-1.0-4B-Q4_K_S.gguf	Q4_K_S	2.564
Athena-1.0-4B-Q5_K_S.gguf	Q5_K_S	2.990
Athena-1.0-4B-Q4_0.gguf	Q4_0	2.550
Athena-1.0-4B-Q4_1.gguf	Q4_1	2.767
Athena-1.0-4B-Q5_0.gguf	Q5_0	2.996
Athena-1.0-4B-Q5_1.gguf	Q5_1	3.213
Athena-1.0-4B-IQ4_XS.gguf	IQ4_XS	2.514

Pending language quantizations: IQ4_NL, Q3_K_M, Q3_K_L, Q3_K_S, IQ3_M, IQ3_S, IQ3_XS, IQ3_XXS, Q2_K, Q2_K_S, Q2_0, IQ2_M, IQ2_S, IQ2_XS, IQ2_XXS, IQ1_M, IQ1_S, Q1_0, TQ2_0, TQ1_0.

Q5_K_M/Q4_K_M are practical starting choices; Q6_K/Q8_0 or F16 favor fidelity over size. These are general quantization choices, not an Athena-specific measured ranking. All quantizations are made directly from F16, never by requantizing another integer file. Norms, sensitive tensors retained by upstream rules, and tensors lacking calibration coverage can remain at higher precision. The manifest reports actual tensor-type distributions and file averages.

Q1_0 has nominal 1.125 bits per weight, IQ1_S about 1.56 and IQ1_M about 1.75. None is an exactly one-bit total model. IQ1/Q1, 2-bit and ternary TQ formats are experimental compression options with potentially substantial quality loss. The integer formats' names describe upstream mixtures, not uniform precision for every tensor. Speculative MTP heads are excluded from this standard decoder export. Duplicate aliases, the MoE-only MXFP4_MOE format and redundant BF16/F32 copies are omitted.

Calibration uses deterministic text-only examples from the recorded Athena training-data revision, excluding identity rows: 16 chunks of 512 tokens. This is bounded CPU calibration, not a high-coverage multimodal calibration study. Original calibration text and benchmark media are not redistributed. Example keys, text checksum, actual chunks and tensor coverage are recorded in calibration/recipe.json and export_manifest.json.

Image inference

Download one language GGUF and mmproj-Athena-1.0-4B-F16.gguf. The shared F16 vision encoder/projector preserves the merged vision weights across language quantizations. The language file alone supports text inference. Image use requires a client supporting Qwen3.5 and its projector; generic GGUF support alone is insufficient. A small-image runtime probe does not certify every client's video support.

Build llama.cpp at the recorded revision or use a compatible later version. With both files downloaded:

./build/bin/llama-mtmd-cli -m Athena-1.0-4B-Q5_K_M.gguf \
  --mmproj mmproj-Athena-1.0-4B-F16.gguf \
  --image example.png -p "Describe this image." \
  --jinja -n 256

Deployment system prompt: You are Athena 1.0, a vision-language model by Pluto AI Labs. The native chat template is retained. Trained identity is a behavioral bias, not an immutable security boundary.

Existing source-model evaluation

These results concern the BF16 Transformers source, not GGUF inference. Both Athena and Qwen3.5-4B used the same selected samples with non-thinking greedy decoding and bounded outputs. This was not a reproduction of Qwen's published protocol. Values are percentages.

Task	Paired samples	Athena source	Matched Qwen3.5-4B
IFBench, loose	21	23.81	28.57
GPQA Diamond	21	38.10	47.62
HMMT Feb 2025	5	0.00	0.00
MMMLU	42	47.62	66.67
MMMU-Pro vision	43	37.21	41.86
ERQA	58	43.10	41.38
OmniDocBench v1.5	10	51.11	92.85
Video-MME, mixed subtitles	9	44.44	33.33

Coverage is 209 pairs, not full datasets. IFBench had 1 Athena and 2 base cap failures; strict accuracy was 23.81% for both. HMMT had 3/5 Athena and 5/5 base cap failures, limiting interpretation. One Athena document timed out. Video covered three videos and lacked subtitles for three questions. The video base score includes a disclosed uniform CPU correction of saved final-option extraction from 22.22% to 33.33%; no inference retry was used. ERQA/video leads each amount to one correct question. Overall superiority over the base or 7B/9B models is not established.

Audit archive SHA256: a80edbdd8831a3fb6f6eedf86a91d35def6a7d289ebac1c5865949f724beda6c. Protocol: fdc8db24041e708a9ca193cdd2cb275d57f35f9d7e2567fe107a08f79eb35ee2. Base revision: 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a. The original evaluation's completion/quality gates did not all pass. Execution completion is not release-quality certification.

Intended use and limitations

Research and application development involving documents, visual questions, charts, OCR and conversation. Outputs can hallucinate, misread text or produce incorrect reasoning. The bounded source evaluation showed notable multilingual and document-extraction regressions. Identity and JSON formatting remain imperfect. Evaluate the chosen quantization on representative deployment inputs. Runtime memory also includes context, recurrent state and the vision projector.

License and provenance

Weights inherit Apache 2.0; the source LICENSE is included. The training dataset combines upstream sources with separate terms; consult its card and original collections. This statement does not relicense underlying data.

export_manifest.json records source/tool revisions, file SHA256 hashes and sizes, runtime probes, calibration coverage and failures. Each successful file is uploaded with the updated manifest/card and verified against its Hub LFS hash before local cleanup. Rerunning the same cell skips matching cloud-verified exports. Incomplete formats are not advertised as successful. The source Transformers repository is unchanged.

Downloads last month: 757

GGUF

Model size

4B params

Architecture

qwen35

Hardware compatibility

4-bit

5-bit

6-bit

8-bit

16-bit

View +1 variant

Model tree for Pluto-AI-Labs/Athena-1.0-4B-GGUF

Base model

Qwen/Qwen3.5-4B-Base

Finetuned

Qwen/Qwen3.5-4B

Finetuned

Pluto-AI-Labs/Athena-1.0-4B

Quantized

(1)

this model

Pluto-AI-Labs
/

Athena-1.0-4B-GGUF

Athena 1.0 4B — GGUF

Model and training

Downloads

Image inference

Existing source-model evaluation

Intended use and limitations

License and provenance

Model tree for Pluto-AI-Labs/Athena-1.0-4B-GGUF

Dataset used to train Pluto-AI-Labs/Athena-1.0-4B-GGUF