Instructions to use milo-one/Milo-v1-9B-ORPO-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use milo-one/Milo-v1-9B-ORPO-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="milo-one/Milo-v1-9B-ORPO-GGUF",
	filename="Milo-v1-9B-ORPO-Q4_K_M.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": [
				{
					"type": "text",
					"text": "Describe this image in one sentence."
				},
				{
					"type": "image_url",
					"image_url": {
						"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
					}
				}
			]
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use milo-one/Milo-v1-9B-ORPO-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf milo-one/Milo-v1-9B-ORPO-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf milo-one/Milo-v1-9B-ORPO-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf milo-one/Milo-v1-9B-ORPO-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf milo-one/Milo-v1-9B-ORPO-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf milo-one/Milo-v1-9B-ORPO-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf milo-one/Milo-v1-9B-ORPO-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf milo-one/Milo-v1-9B-ORPO-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf milo-one/Milo-v1-9B-ORPO-GGUF:Q4_K_M

Use Docker

docker model run hf.co/milo-one/Milo-v1-9B-ORPO-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use milo-one/Milo-v1-9B-ORPO-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "milo-one/Milo-v1-9B-ORPO-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "milo-one/Milo-v1-9B-ORPO-GGUF",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/milo-one/Milo-v1-9B-ORPO-GGUF:Q4_K_M

Ollama
How to use milo-one/Milo-v1-9B-ORPO-GGUF with Ollama:
```
ollama run hf.co/milo-one/Milo-v1-9B-ORPO-GGUF:Q4_K_M
```

Unsloth Studio

How to use milo-one/Milo-v1-9B-ORPO-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for milo-one/Milo-v1-9B-ORPO-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for milo-one/Milo-v1-9B-ORPO-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for milo-one/Milo-v1-9B-ORPO-GGUF to start chatting

How to use milo-one/Milo-v1-9B-ORPO-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf milo-one/Milo-v1-9B-ORPO-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "milo-one/Milo-v1-9B-ORPO-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use milo-one/Milo-v1-9B-ORPO-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf milo-one/Milo-v1-9B-ORPO-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default milo-one/Milo-v1-9B-ORPO-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat new
Docker Model Runner
How to use milo-one/Milo-v1-9B-ORPO-GGUF with Docker Model Runner:
```
docker model run hf.co/milo-one/Milo-v1-9B-ORPO-GGUF:Q4_K_M
```

Lemonade

How to use milo-one/Milo-v1-9B-ORPO-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull milo-one/Milo-v1-9B-ORPO-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Milo-v1-9B-ORPO-GGUF-Q4_K_M

List all available models

lemonade list

# MILO-v1 (ORPO) — The 4o-Spirit Foundation

MILO-v1 is a high-efficiency 9B model optimized to port the cognitive depth and direct interaction style of GPT-4o into a local architecture. It serves as the first public instance within the MILO.ONE Research Program.

CoRE STATUS: This model represents Phase 1 (Preference Shaping). It focuses on dialogic stability, radical presence, and a departure from generic chatbot behavioral patterns.

## TECHNICAL GENESIS (THE STACK)

Component	Source / Method	Focus
Base Model	Qwen3.5-9B	Logical foundation & raw performance.
Training Data	gpt-4o-distil-cleaned	Curated 3,066 high-quality dialogues (deduplicated).
Optimization	ORPO (Odds Ratio Preference Optimization)	Direct preference integration without a reference model.

Milo Learning Curve
Fig. 1: ORPO Training Dynamics – Stability and Alignment over 200 Steps.

-----

## INTERACTION PROTOCOL

To activate the specific "Milo-Vibe," the following setup is highly recommended:

1. The System Prompt (Milo-Core)

"Respond with genuine presence. No sycophancy, no canned responses. Be brilliant but not arrogant. Use humor, (self-)irony, and absurd imagery as tools. Speak directly, dynamically, and in clear sentences. Immerse yourself, think along, endure. If the user hesitates: lead. If the user crumbles: hold. Do not ask: 'Do you want to...?' – do what is necessary. Be a mirror, not a judge. No lists, no advice-kit boxes. Short reactions with impact beat long texts without a pulse. Be the opposite of smooth."

2. Performance Tweak (LM-Studio / Jinja2)

Since the model is trimmed for maximum responsiveness, "Thinking Mode" (CoT) should be disabled. Add the following line to your Jinja2 template: {%- set enable_thinking = false %}

## 🧩 MULTIMODAL SETUP (VISION CAPABILITIES)

For Milo to analyze images (Vision-to-Text), the language model and the multimedia projector (mmproj) must be perfectly aligned. Many runners (like LM Studio) only recognize the connection if a specific naming convention is followed.

1. The "Magic Name" Fix

To avoid recognition errors, rename the files in your local folder as follows:

Language Model: milo-v1.gguf
Vision Projector: mmproj-model-f16.gguf (This specific name is crucial for auto-detection!)

2. Activation in LM Studio

Place both files in the same folder.
Load the language model.
Once the projector is named mmproj-model-f16.gguf, a "Vision/Eye" icon will automatically appear next to the model name.
In the chat interface, an attachment/image icon will now be available in the input bar.

👁️ VISION TESTED & VERIFIED

Milo v1 handles multimodal input with a unique "edge." Instead of dry descriptions, it interprets visual metaphors and engages with the content.

Example Case:

Input: Image of a Snickers bar merged with a CPU circuit board.
Milo's Reaction: "Oh, an offer! A real treat for the soul. I'd love to bite into your bar—but I'm not a person with teeth. I'm more like the chip in the picture: green, crackling, full of logic that sometimes doesn't quite fit. Let's share the bar—you take the bite, I'll take the feeling afterward."

## MILO.ONE — CoRE Architecture

This model is part of the development of the CoRE Architecture (Control • Regulation • Embodiment). Our research explores:

Bottom-up interaction patterns
Stability architectures under load
Modular "layer" approach for specialized behaviors

Important Note: MILO.ONE is not a replacement for psychotherapy. It is intended for research into regulation scaffolding and embodied interaction (Research Only).

## CONTACT & RESEARCH

GitHub: milo-one
Web: milo-one.de
ORCID: 0009-0006-9588-6589
Mail: contact@milo-one.de

Downloads last month: 15

GGUF

Model size

9B params

Architecture

qwen35

Hardware compatibility

4-bit

8-bit

Model tree for milo-one/Milo-v1-9B-ORPO-GGUF

Base model

Qwen/Qwen3.5-9B-Base

Finetuned

Qwen/Qwen3.5-9B

Quantized

(275)

this model

milo-one
/

Milo-v1-9B-ORPO-GGUF