Instructions to use apus-ailab/APUS-OpenJev-v1-9B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use apus-ailab/APUS-OpenJev-v1-9B-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf apus-ailab/APUS-OpenJev-v1-9B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf apus-ailab/APUS-OpenJev-v1-9B-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf apus-ailab/APUS-OpenJev-v1-9B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf apus-ailab/APUS-OpenJev-v1-9B-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf apus-ailab/APUS-OpenJev-v1-9B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf apus-ailab/APUS-OpenJev-v1-9B-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf apus-ailab/APUS-OpenJev-v1-9B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf apus-ailab/APUS-OpenJev-v1-9B-GGUF:Q4_K_M

Use Docker

docker model run hf.co/apus-ailab/APUS-OpenJev-v1-9B-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use apus-ailab/APUS-OpenJev-v1-9B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "apus-ailab/APUS-OpenJev-v1-9B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "apus-ailab/APUS-OpenJev-v1-9B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/apus-ailab/APUS-OpenJev-v1-9B-GGUF:Q4_K_M

Ollama
How to use apus-ailab/APUS-OpenJev-v1-9B-GGUF with Ollama:
```
ollama run hf.co/apus-ailab/APUS-OpenJev-v1-9B-GGUF:Q4_K_M
```
Unsloth Desktop

How to use apus-ailab/APUS-OpenJev-v1-9B-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf apus-ailab/APUS-OpenJev-v1-9B-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "apus-ailab/APUS-OpenJev-v1-9B-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use apus-ailab/APUS-OpenJev-v1-9B-GGUF with Docker Model Runner:
```
docker model run hf.co/apus-ailab/APUS-OpenJev-v1-9B-GGUF:Q4_K_M
```

Lemonade

How to use apus-ailab/APUS-OpenJev-v1-9B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull apus-ailab/APUS-OpenJev-v1-9B-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.APUS-OpenJev-v1-9B-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use apus-ailab/APUS-OpenJev-v1-9B-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf apus-ailab/APUS-OpenJev-v1-9B-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default apus-ailab/APUS-OpenJev-v1-9B-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use apus-ailab/APUS-OpenJev-v1-9B-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf apus-ailab/APUS-OpenJev-v1-9B-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "apus-ailab/APUS-OpenJev-v1-9B-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

APUS-OpenJev-v1-9B-GGUF

English | 中文 · Source model · Collection · GGUF collection · MLX collection · MLX (Apple Silicon)

GGUF conversions of APUS-OpenJev-v1-9B (revision 9dd2482d4c28) for Ollama, llama.cpp and LM Studio on Linux, Windows and macOS (Metal).

OpenJev is a decision model: each request supplies a state, an instruction and 2–16 candidates; the model answers with one candidate label (A–P) and the application reads the distribution over those labels. It is not a chat model.

Files and parity

Every file was scored on the Frozen80 panel with identical prompt tokens and compared with the HF BF16 release run by its own runtime (full depth, 68/80 · 85.00%).

File	Size	Frozen80	Decisions = HF BF16	Max Δp vs HF BF16
Q8_0	8.9 GiB	68/80 · 85.00%	80/80	0.0999
Q4_K_M	5.2 GiB	68/80 · 85.00%	78/80	0.4364
BF16	16.7 GiB	68/80 · 85.00%	78/80	0.0268

Rows are llama-server b11118 on an NVIDIA RTX PRO 6000 (CUDA) unless marked Apple M5 (Metal). Q8_0 is the recommended default; Q4_K_M uses an importance matrix. Quantized scores are measured separately and do not inherit the BF16 result. Frozen80 is a reused development panel, not a blind benchmark. Per-file details: evaluation/. Per-question rows (candidate probabilities, choice, correctness; join with Frozen80 by panel_index): evaluation/per-question/.

Ollama

ollama run hf.co/apus-ailab/APUS-OpenJev-v1-9B-GGUF:Q8_0 --think=false

Always disable thinking. Ollama renders this architecture with its built-in Qwen3.5 renderer (the Modelfile TEMPLATE is not applied), and that renderer opens a thinking block unless told otherwise, which breaks the trained no-thinking contract. Use --think=false on the CLI, "think": false in /api/chat / /api/generate, or send the fully rendered prompt with "raw": true. params sets temperature 0 and num_predict 1; a local Modelfile is included.

To get candidate probabilities, use the bundled client (raw mode):

python examples/openjev_local.py --backend ollama --model hf.co/apus-ailab/APUS-OpenJev-v1-9B-GGUF:Q8_0

Ollama returns at most 20 top_logprobs and cannot report named tokens, so the distribution is exact only when every candidate label is in the top 20 (Q8_0: 66/80 Frozen80 prompts); otherwise use the selected label or llama-server.

Measured with Ollama 0.34.3 and the Q8_0 file on Linux (NVIDIA RTX PRO 6000, CUDA):

Request	Label = llama-server	Frozen80	Prompt tokens = training
`/api/generate`, `raw: true`, full prompt (examples/openjev_local.py)	80/80	68/80	80/80
`/api/chat` or `ollama run`, `think: false`	80/80	68/80	80/80
`/api/generate`, thinking left on (wrong)	0/80	0/80	0/80

llama.cpp (exact distribution)

llama-server -m APUS-OpenJev-v1-9B-Q8_0.gguf -c 9216 -ngl 999
python examples/openjev_local.py --backend llama-server --url http://127.0.0.1:8080

examples/openjev_local.py renders prompts with openjev_contracts.py, the same contract used in training.

Conversion

llama.cpp b11118 (e6ab7c1a4), convert_hf_to_gguf.py --no-mtp (the merged release has no MTP weights).
Q4_K_M importance matrix: 448 training-course decisions, 64 per source, disjoint from Frozen80 (details, imatrix.gguf).
All 1-D tensors, including GDN A_log / dt_bias and norms, stay F32 in every file (check).
Scope: full depth only (the 16/20-layer low exit is not available), text only (no vision tower), probabilities are not calibrated.