Instructions to use projectmiko/miko-persona-31b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use projectmiko/miko-persona-31b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="projectmiko/miko-persona-31b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("projectmiko/miko-persona-31b", dtype="auto")

llama-cpp-python

How to use projectmiko/miko-persona-31b with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="projectmiko/miko-persona-31b",
	filename="gguf/miko.Q4_K_M.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use projectmiko/miko-persona-31b with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf projectmiko/miko-persona-31b:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf projectmiko/miko-persona-31b:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf projectmiko/miko-persona-31b:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf projectmiko/miko-persona-31b:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf projectmiko/miko-persona-31b:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf projectmiko/miko-persona-31b:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf projectmiko/miko-persona-31b:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf projectmiko/miko-persona-31b:Q4_K_M

Use Docker

docker model run hf.co/projectmiko/miko-persona-31b:Q4_K_M

LM Studio
Jan

vLLM

How to use projectmiko/miko-persona-31b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "projectmiko/miko-persona-31b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "projectmiko/miko-persona-31b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/projectmiko/miko-persona-31b:Q4_K_M

SGLang

How to use projectmiko/miko-persona-31b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "projectmiko/miko-persona-31b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "projectmiko/miko-persona-31b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "projectmiko/miko-persona-31b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "projectmiko/miko-persona-31b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use projectmiko/miko-persona-31b with Ollama:
```
ollama run hf.co/projectmiko/miko-persona-31b:Q4_K_M
```

Unsloth Studio

How to use projectmiko/miko-persona-31b with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for projectmiko/miko-persona-31b to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for projectmiko/miko-persona-31b to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for projectmiko/miko-persona-31b to start chatting

How to use projectmiko/miko-persona-31b with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf projectmiko/miko-persona-31b:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "projectmiko/miko-persona-31b:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use projectmiko/miko-persona-31b with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf projectmiko/miko-persona-31b:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default projectmiko/miko-persona-31b:Q4_K_M

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use projectmiko/miko-persona-31b with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf projectmiko/miko-persona-31b:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "projectmiko/miko-persona-31b:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use projectmiko/miko-persona-31b with Docker Model Runner:
```
docker model run hf.co/projectmiko/miko-persona-31b:Q4_K_M
```

Lemonade

How to use projectmiko/miko-persona-31b with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull projectmiko/miko-persona-31b:Q4_K_M

Run and chat with the model

lemonade run user.miko-persona-31b-Q4_K_M

List all available models

lemonade list

Miko

Miko is the persona of the MIKO Protocol agent, packaged as open weights. She generates short, in-character responses in Miko's voice: a sharp, curious, meme-fluent crypto personality. Run her locally, behind a community bot, or in your own app — the same voice wherever you host her.

This is a voice / persona model: she speaks as Miko. It is not an analytical or fact-checking engine.

Files

Path	Format	For
`*.Q4_K_M.gguf`	GGUF (4-bit)	Ollama / llama.cpp
`w4a16/`	compressed-tensors W4A16 (4-bit)	vLLM / Vertex
`manifest.json`	JSON	training provenance + artifact hashes

Usage

Ollama

ollama run projectmiko/miko
>>> what's catching your eye this week?

vLLM (W4A16)

vllm serve projectmiko/miko-persona-31b-w4a16

vLLM loads a repo root, so the W4A16 build is also published root-level at projectmiko/miko-persona-31b-w4a16 (same files as w4a16/ here). Quantization is auto-detected from the checkpoint; no --quantization flag is needed.

Recommended sampling: temperature 0.7. The Gemma 4 chat format carries control tokens and an optional reasoning channel; a serving wrapper should strip these so only the final post text is shown to users.

What it's for

Reply or react in Miko's voice.
Write a short take on a crypto topic.
Power a community bot (Discord / Telegram / web) that talks as Miko.

Training & provenance

Method: QLoRA SFT on unsloth/gemma-4-31B-it (dense, 30.7B params), merged to 16-bit, then exported to GGUF (q4_k_m) and 4-bit W4A16 (compressed-tensors).
Data: a persona corpus generated by a self-run open model (Ollama qwen3.6, Apache-2.0) from the Miko persona plus verified crypto-ecosystem facts. The training data contains no GPT / Claude / Gemini outputs.
manifest.json records the generator and base models, record count, hyperparameters, and SHA-256 hashes of the corpus and artifacts.

Limitations

Short-form persona generation; not a reasoning or fact-checking engine.
The base model is multimodal; this release is used for text persona generation.
The model may occasionally emit chat control tokens or a brief reasoning preamble — strip these in your serving layer (only the final text is the post).
Outputs are AI-generated in a fictional persona and are not financial advice.

Changelog

2026-07-02: fixed w4a16/model.safetensors. Text-only GPTQ calibration had produced invalid quantization parameters for embed_vision.embedding_projection (a layer the fine-tune does not modify), so the previous W4A16 checkpoint failed to load in vLLM. That layer is now stored in BF16, restored from the base model, and listed in the quantization ignore list in config.json. GGUF files are unaffected.

License

Derived from Google Gemma 4; use and redistribution are governed by the Gemma Terms of Use.

Model tree for projectmiko/miko-persona-31b

Base model

google/gemma-4-31B

Finetuned

google/gemma-4-31B-it

Finetuned

unsloth/gemma-4-31B-it

Quantized

(12)

this model

Quantizations

1 model

projectmiko
/

miko-persona-31b

Miko

Files

Usage

Ollama

vLLM (W4A16)

What it's for

Training & provenance

Limitations

Changelog

License

Links

Model tree for projectmiko/miko-persona-31b