Instructions to use SargeDev/Solar-Decision-JEV-27B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use SargeDev/Solar-Decision-JEV-27B-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf SargeDev/Solar-Decision-JEV-27B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf SargeDev/Solar-Decision-JEV-27B-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf SargeDev/Solar-Decision-JEV-27B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf SargeDev/Solar-Decision-JEV-27B-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf SargeDev/Solar-Decision-JEV-27B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama cli -hf SargeDev/Solar-Decision-JEV-27B-GGUF:Q4_K_M

Use Docker

docker model run hf.co/SargeDev/Solar-Decision-JEV-27B-GGUF:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use SargeDev/Solar-Decision-JEV-27B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "SargeDev/Solar-Decision-JEV-27B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SargeDev/Solar-Decision-JEV-27B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/SargeDev/Solar-Decision-JEV-27B-GGUF:Q4_K_M

Ollama
How to use SargeDev/Solar-Decision-JEV-27B-GGUF with Ollama:
```
ollama run hf.co/SargeDev/Solar-Decision-JEV-27B-GGUF:Q4_K_M
```
Unsloth Desktop

How to use SargeDev/Solar-Decision-JEV-27B-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf SargeDev/Solar-Decision-JEV-27B-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "SargeDev/Solar-Decision-JEV-27B-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use SargeDev/Solar-Decision-JEV-27B-GGUF with Docker Model Runner:
```
docker model run hf.co/SargeDev/Solar-Decision-JEV-27B-GGUF:Q4_K_M
```

Lemonade

How to use SargeDev/Solar-Decision-JEV-27B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull SargeDev/Solar-Decision-JEV-27B-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Solar-Decision-JEV-27B-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use SargeDev/Solar-Decision-JEV-27B-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf SargeDev/Solar-Decision-JEV-27B-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default SargeDev/Solar-Decision-JEV-27B-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use SargeDev/Solar-Decision-JEV-27B-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf SargeDev/Solar-Decision-JEV-27B-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "SargeDev/Solar-Decision-JEV-27B-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Solar-Decision-JEV-27B-GGUF

GGUF ladder for the r32 + solar merge (the bf16 repo SargeDev/Solar-Decision-JEV-27B). MTP/nextn drafter tensors are preserved in the f16 export (blk.64.nextn.*) so speculative decoding works in llama.cpp.

Sizes

quant	size	decode tok/s (prompt/tokens)
IQ4_XS	14.35 GiB	955.42 / 14.02
Q2_K	10.11 GiB	744.59 / 18.51
Q3_K_L	13.55 GiB	794.24 / 14.06
Q3_K_M	12.56 GiB	820.55 / 15.03
Q3_K_S	11.40 GiB	797.17 / 16.41
Q4_K_M	15.65 GiB	852.36 / 12.43
Q4_K_S	14.73 GiB	901.82 / 13.27
Q5_K_M	18.18 GiB	825.25 / 11.01
Q5_K_S	17.66 GiB	843.25 / 11.38
Q6_K	20.88 GiB	729.97 / 9.71
Q8_0	27.04 GiB	806.43 / 8.09

Q4_K_M is the daily-driver pick; Q6_K for maximum fidelity; Q5_K_M in between.

The full ladder

Every size below is in this repo, all built from the same f16 export and verified (866 tensors each, MTP/nextn drafter tensors preserved so speculative decoding works in llama.cpp). The benchmark numbers are measured on one GB10 (GB10-class), CUDA build, -ngl 99, pp512 / tg128, 3 repetitions.

The honest headline for this ladder

The decision-gate numbers for the combined merge are on the bf16 card. This ladder is the same weights in GGUF form. The measured decision-gate accuracy of the combined merge is below its r32 parent (choice top-1 0.8162 vs 0.8693) — see the bf16 card for the full ablation. We are publishing the ladder because it is the artifact people actually run, with the numbers attached.

Quick start

llama-server -m Jev_Qwen3.8-27B-Judgment-Combined-Q4_K_M.gguf -ngl 99 --jinja \
  --chat-template-kwargs '{"enable_thinking": false}'

Decision gate (AutoTrust-style, test_set_30k with yuri_v1 D1 placeholders excluded)

Read this table carefully, because it contains two very different comparisons.

The sibling merge is our other build, scored on the same 27,695 rows with the same harness and the same head + calibration. It is a sibling model, not a quality bar — but it is the honest apples-to-apples reference for what the lm_head LoRA is worth.
AutoTrust reference is the number published by the authors of the original AutoTrust work for their model on this task set. This is the bar.

On the typed-decision kinds (multiple-choice choice, ordinal score) this model is still below the published AutoTrust reference, and the gap is large - not a rounding difference. On the binary noul kind the gap is narrower. Cells marked "(below bar)" differ from the published reference by more than 5 points.

What this means: the fine-tune produces a usable decision instrument and a clearly better everyday-chat model than its base (see the benchmark matrix below), but it is not a reproduction of AutoTrust-level typed-decision accuracy. Treat the choice and score columns as an open result, not a solved one. We are publishing the numbers as measured, including the gap.

Harness validation. These numbers were produced by our own HTTP gate client, so we checked it against a known path: re-gating the r32 merge reproduced the reference's own numbers to within 0.001 on every metric (choice top-1 0.8704 -> 0.8693, score KL 0.0623 -> 0.0619, noul AUROC 0.9920 -> 0.9919). The harness is sound, so the gaps above are real.

metric	n	Jev Judgment r32+solar (ours)	Jev Judgment r32+lm_head	AutoTrust reference (published)
noul.AUROC	9969	0.9810 (below bar)	0.9919	0.9961
noul.top-1	9969	0.9274	0.9486	n/a
noul.ECE	9969	0.0057	0.0122	n/a
noul.KL	9969	0.0167	0.0081	n/a
choice.top-1	9199	0.8162 (below bar)	0.8693	0.9041
choice.ECE	9199	0.1592	0.0051	n/a
choice.KL	9199	0.2208	0.0530	n/a
score.top-1	8527	0.7192 (below bar)	0.8135	0.8910
score.ECE	8527	0.0377	0.0367	n/a
score.KL	8527	0.1332 (below bar)	0.0619	0.0210
overall.top-1	27695	0.8264	0.8807	n/a
overall.ECE	27695	0.0625 (below bar)	0.0090	0.0011
overall.KL	27695	0.1203 (below bar)	0.0396	0.0185

Ablation: where the decision readout has to live

All four rows below are scored on the SAME 27,695 rows, with the SAME harness and the SAME decision head + calibration. The only difference is where the decision distribution is produced.

model	decision readout	choice top-1	score top-1	score KL	noul AUROC	overall ECE
AutoTrust (published reference)	trained into the model (lm_head LoRA)	0.9041	0.8910	0.0210	0.9961	0.0011
r32 + lm_head LoRA (r=32, alpha=64)	trained into the model (lm_head LoRA)	0.8693	0.8135	0.0619	0.9919	0.0090
solar r16 (no lm_head LoRA)	separate projection at read time	0.5054	0.5369	0.4699	0.9449	0.0211
r32 + lm_head LoRA + solar (combined)	trained into the model (lm_head LoRA)	0.8162	0.7192	0.1332	0.9810	0.0625

Two findings, in order of how much they matter:

The readout has to be trained INTO the model. A separate projection over the final hidden state (the solar r16 row) reaches only 0.5054 choice top-1; the same backbone with an lm_head LoRA in the training loop reaches 0.8693 on the identical rows. A separate projection can work, but it needs the backbone to cooperate, and a backbone trained without the readout in the loop does not.
Adding a second, readout-free adapter on top makes the typed-decision accuracy WORSE, not better. The combined row is the r32 merge plus the solar r16 delta applied on top (bit-exact arithmetic, verified). It lands between the two parents on every metric - 0.8162 choice top-1, between r32's 0.8693 and solar's 0.5054 - rather than above r32. The two LoRA deltas are near-orthogonal (mean cosine 0.003), so they do not cancel, but solar's delta still degrades the readout r32 trained.

We publish the combined row because it is a real, reproducible result, but the r32+lm_head merge is the better model and is the one this release recommends. We are not going to present a merge as an improvement when our own numbers say it is a regression.

Benchmark matrix (lm-eval-harness 0.4.13, greedy, same prompts for every row)

model	GSM8K (8-shot CoT)	IFEval (0-shot, prompt-strict)	MMLU subset (5-shot, 8 subjects)	refusal rate
huihui Qwen3.8-27B (uncensored base)	97.7	76.0	81.2	0.0% (5/5 probes)
Jev Judgment bf16 (solar r16)	96.0	81.5	81.2	0.0% (5/5 probes)
Jev Judgment bf16 (r32+lm_head + solar)	91.0	72.5	80.8	0.0% (5/5 probes)
Jev Judgment bf16 (r32 + lm_head)	94.0	81.0	82.5	0.0% (5/5 probes)
Jev Judgment NVFP4+FP8	95.3	80.5	80.5	0.0% (5/5 probes)
GGUF Q4_K_M	95.3	79.5	82.0	0.0% (5/5 probes)
GGUF Q8_0	95.7	81.0	81.0	0.0% (5/5 probes)

Questions scored per task:

task	questions scored per model
GSM8K (8-shot CoT)	300
IFEval (0-shot, prompt-strict)	200
MMLU subset (5-shot, 8 subjects)	400

Note: GSM8K scored with flexible-extract (the model answers in its own format). MMLU subset subjects: high_school_mathematics, college_biology, professional_psychology, jurisprudence, machine_learning, astronomy, global_facts, logical_fallacies (5-shot, _generative variants). MMLU --limit applies PER SUBJECT, so the subset is 8 x per-subject-limit questions.

Reproducibility: decoding is greedy (do_sample=False, temperature 0, seed 1234), but the server batches concurrent requests, and batch composition changes floating-point reduction order. Repeat runs of the same config can therefore differ by a point or two on the small-n tasks. The error bars in the table are the harness's own stderr over the scored rows.

Lineage & credits

This release is a descendant of the earlier SargeDev/JEV-27B-Uncensored and SargeDev/JEV-27B-VL-Uncensored repos. The credit chain is unchanged and is reproduced here in full:

AutoTrust AI — Blocks-of-Experts recipe, the System-1 decision LoRA (r16/r32) + 24-slot decision head, and the serve_decide.py decision harness, from autotrust/JEV-27B and autotrust/JEV-27B-VL (Apache-2.0). The r32 adapter used here is theirs, unmodified: the backbone LoRA with the decision head re-expressed as an lm_head LoRA, so only the 24 verbalizer rows of the output projection change.
huihui-ai — the abliterated (uncensored) Qwen3.8-27B backbone used here (Huihui-Qwen3.8-27B-abliterated, Apache-2.0; layers 18-51 ablated, vision tower untouched).
SargeDev — jev-distill-corpus-v3, the 740,957-row calibrated typed-decision corpus AutoTrust's JEV models were trained on (Apache-2.0), and the solar r16 adapter trained here on solar-decisions-corpus-v4 (the 1,219-step JEV System-1 run, 17-domain tokenized mix).
Qwen — Qwen3.8-27B base (Apache-2.0).
TypeSafe AI — Jev 1.13, the original closed teacher behind the System-1 typed-decisions framing (referenced; not redistributed).

Base: huihui-ai/Qwen3.8-27B-abliterated (bf16; the text-only builds strip the vision tower and serve as a plain qwen3_5_text causal LM, the VL build keeps the tower and serves as qwen3_5 / Qwen3_5ForConditionalGeneration).

Adapters — this release ships two, and they are NOT interchangeable:
- solar r16 (alpha 32, 400 modules, no lm_head): trained 1,219 steps on the JEV System-1 corpus (solar-decisions-corpus-v4 tokenized mix: T1 gold domains + memory_relevance + 17-domain fill), KL+RPS decision-head distillation objective (AutoTrust-style), cosine LR 2e-4, KL_W 1.0, RPS_W 0.3. Merged with peft merge_and_unload; merge fidelity verified by re-gating the bare merged model (it reproduced the adapter-mounted numbers).
- r32 (alpha 64, 401 modules, includes lm_head): the backbone LoRA plus the 24-slot decision head re-expressed as an lm_head LoRA (only the 24 verbalizer rows change), so a typed decision is a single prefill step with max_tokens=1 constrained to the option tokens.
Decision head + calibration.json (shipped in this repo root): the instrument-panel piece agents call for numeric confidence — the 24-slot verbalizer id list, the per-slot bias, and the per-kind temperature. This is the SAME head and the SAME calibration file (verified byte-identical, md5 11359173… and 7f0f3914…) that produced every gate number in this card, so a consumer reproducing our numbers reproduces them exactly. For everyday natural-language judgment, chat with the model directly and set enable_thinking:false in chat_template_kwargs.
Everything measured on one NVIDIA GB10 (DGX Spark-class); quantized with llmcompressor 0.14.0 (NVFP4) and llama.cpp (GGUF ladder).

The judgment fix (Nov 9, 2026 — reproducing Morgan's Code)

This repo was not the model in the table. The measured fix is Arete-27B-Uncensored-V2: Arete-ablit + the solar step-1946 merge (lm_head trained in). On a Nov 9, 2026 reproduction of Morgan's Code, huihui-ablit flagged 36.7% of clean code as vulnerable; that Arete merge dropped it to 3.3% (recall 100%, 0 refusals). A bolt-on adapter on huihui (step 2327, a different checkpoint) changed nothing. Full table and caveats are on the V2 card. These weights are a different build — do not read 3.3% as this repo's score.

Downloads last month: 1,335

GGUF

Model size

27B params

Architecture

qwen35

Hardware compatibility

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

Model tree for SargeDev/Solar-Decision-JEV-27B-GGUF

Base model

Qwen/Qwen3.8-27B

Finetuned

huihui-ai/Huihui-Qwen3.8-27B-abliterated

Finetuned

SargeDev/Solar-Decision-JEV-27B

Quantized

(2)

this model

SargeDev
/

Solar-Decision-JEV-27B-GGUF

Solar-Decision-JEV-27B-GGUF

Sizes

The full ladder

The honest headline for this ladder

Quick start

Decision gate (AutoTrust-style, test_set_30k with yuri_v1 D1 placeholders excluded)

Ablation: where the decision readout has to live

Benchmark matrix (lm-eval-harness 0.4.13, greedy, same prompts for every row)

Lineage & credits

The judgment fix (Nov 9, 2026 — reproducing Morgan's Code)

Model tree for SargeDev/Solar-Decision-JEV-27B-GGUF

Datasets used to train SargeDev/Solar-Decision-JEV-27B-GGUF