pyrrho-v2-nano-g1

pyrrho-v2-nano-g1 is a small local RAG planning and governance co-processor. It can run before retrieval on a user query and after retrieval on the query plus source passages. The post-retrieval pass returns whether the evidence is SUFFICIENT, DISPUTED, or INSUFFICIENT before an answer is generated.

It is not an answer generator, not a retriever, and not an open-world fact checker. It sits between retrieval and generation, or beside a retrieval pipeline as a fast evidence-quality layer, so downstream systems can answer, show a dispute, retry retrieval, or ask for missing evidence.

Compared with the older Pyrrho v1 line, v2 exposes a smaller native head shape: one evidence verdict, one failure reason, and two compact multi-label metadata heads. The same model owns the fitz-sage pre-retrieval query-planning pass and the post-retrieval evidence-governance pass.

Native V2 Heads

Head	Labels / values	Intended use
evidence_verdict	SUFFICIENT, DISPUTED, INSUFFICIENT	Post-retrieval evidence sufficiency and conflict decision.
failure_mode	none, unresolved_conflict, missing_or_incomplete_evidence, wrong_scope_or_version, ambiguous_request	Actionable reason when evidence is disputed or insufficient.
retrieval_intents	needs_lookup, needs_temporal_resolution, needs_comparison_or_set, needs_broad_coverage	Pre-retrieval planning and post-retrieval retry metadata.
evidence_kinds	needs_text, needs_table_or_record, needs_code_or_symbol, needs_config_or_setting, needs_log_or_run_result, needs_document_layout	Evidence-surface metadata for routing, audit, and missing-source hints.

Output Contract

The raw Hugging Face model output is an 18-logit vector. It is not one flat softmax. Decode it by head:

Logit slice	Head	Decoding
0:3	evidence_verdict	softmax over INSUFFICIENT, DISPUTED, SUFFICIENT
3:8	failure_mode	softmax over the five failure labels
8:12	retrieval_intents	sigmoid multi-label scores
12:18	evidence_kinds	sigmoid multi-label scores

Most integrations should expose structured objects derived from those logits. For [PYRRHO_PRE], use only retrieval_intents and evidence_kinds. For [PYRRHO_POST], use all four heads.

Field	Meaning
evidence_verdict.final_label	Final v2 verdict: SUFFICIENT, DISPUTED, or INSUFFICIENT.
evidence_verdict.probabilities	Softmax probability distribution over the three verdict labels.
failure_mode.final_label	Most likely failure reason, or none for sufficient evidence.
retrieval_intents.final_labels	Intent labels above the configured sigmoid threshold.
evidence_kinds.final_labels	Evidence-kind labels above the configured sigmoid threshold.
confidence	Probability or score assigned to the selected label.

Example normalized output:

{
  "schema_version": "pyrrho_v2_prediction",
  "evidence_verdict": {
    "final_label": "DISPUTED",
    "confidence": 0.86,
    "probabilities": {
      "INSUFFICIENT": 0.08,
      "DISPUTED": 0.86,
      "SUFFICIENT": 0.06
    }
  },
  "failure_mode": {
    "final_label": "unresolved_conflict",
    "confidence": 0.81
  },
  "retrieval_intents": {
    "final_labels": ["needs_comparison_or_set"],
    "scores": {
      "needs_comparison_or_set": 0.77
    }
  },
  "evidence_kinds": {
    "final_labels": ["needs_text", "needs_table_or_record"],
    "scores": {
      "needs_text": 0.91,
      "needs_table_or_record": 0.63
    }
  }
}

The model does not generate answers, citations, source spans, retrieval results, or natural-language explanations. It classifies and scores query intent before retrieval and the (query, retrieved_contexts) evidence state after retrieval.

Intended Use

Use this model when a RAG or retrieval system needs fast local signals about:

• whether retrieved evidence is enough to answer,
• whether retrieved evidence contains an unresolved conflict,
• why evidence is insufficient or disputed,
• whether another retrieval pass should focus on lookup, time, comparison, or broad coverage,
• which source surface appears relevant or missing,
• how to log governance decisions for later audit.

This model is not intended to verify facts outside the provided sources, replace a retriever, write answers, or replace human review in high-stakes settings.

Input Format

Pre-retrieval query planning:

[PYRRHO_PRE]
Question: <user query>

Post-retrieval evidence governance:

[PYRRHO_POST]
Question: <user query>

Sources:
[1] <retrieved source text>
[2] <retrieved source text>

Quick Start

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

MODEL_ID = "yafitzdev/pyrrho-v2-nano-g1"

VERDICT_LABELS = ["INSUFFICIENT", "DISPUTED", "SUFFICIENT"]
FAILURE_LABELS = [
    "none",
    "unresolved_conflict",
    "missing_or_incomplete_evidence",
    "wrong_scope_or_version",
    "ambiguous_request",
]
INTENT_LABELS = [
    "needs_lookup",
    "needs_temporal_resolution",
    "needs_comparison_or_set",
    "needs_broad_coverage",
]
KIND_LABELS = [
    "needs_text",
    "needs_table_or_record",
    "needs_code_or_symbol",
    "needs_config_or_setting",
    "needs_log_or_run_result",
    "needs_document_layout",
]

query = "Has the company achieved profitability?"
contexts = [
    "The company posted net income of $4 million in Q2.",
    "The company recorded a quarterly loss of $12 million in Q3.",
]
text = "[PYRRHO_POST]\nQuestion: " + query + "\n\nSources:\n" + "\n".join(
    f"[{i}] {context}" for i, context in enumerate(contexts, start=1)
)

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForSequenceClassification.from_pretrained(MODEL_ID).eval()

inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=2048)
with torch.no_grad():
    logits = model(**inputs).logits[0]

verdict_probs = torch.softmax(logits[0:3], dim=-1)
failure_probs = torch.softmax(logits[3:8], dim=-1)
intent_scores = torch.sigmoid(logits[8:12])
kind_scores = torch.sigmoid(logits[12:18])

verdict = VERDICT_LABELS[int(verdict_probs.argmax())]
failure = FAILURE_LABELS[int(failure_probs.argmax())]
intents = [
    label for label, score in zip(INTENT_LABELS, intent_scores)
    if float(score) >= 0.5
]
kinds = [
    label for label, score in zip(KIND_LABELS, kind_scores)
    if float(score) >= 0.5
]

print(verdict)
print(failure)
print(intents)
print(kinds)

CPU ONNX

The repository includes both FP32 and INT8 ONNX exports:

• model.onnx
• model_quantized.onnx

fitz-sage loads model.onnx by default for governance accuracy. The quantized graph is included for integrations that explicitly trade some accuracy margin for smaller artifacts. Decode the resulting 18 logits using the same slices shown above.

Evaluation

Held-out post-retrieval eval from outputs/modernbert_base_v2_dual_from_g1_41358_active_20260704_seed42:

Metric	Value
overall score	0.9471
verdict accuracy	0.9703
false sufficient rate	0.0484
failure accuracy	0.9567
retrieval exact match	0.8308
retrieval macro F1	0.9277
evidence-kind exact match	0.9809
evidence-kind macro F1	0.9950

Held-out pre-retrieval query eval:

Metric	Value
retrieval exact match	0.8248
retrieval macro F1	0.9266
evidence-kind exact match	0.9637
evidence-kind macro F1	0.9873

Fitz-sage release-candidate checks:

Benchmark	Result
balanced fixed-evidence governance sanity suite	120/120
live fitz-sage benchmark	97/120
core	19/20
holdout	43/50
holdout2	35/50

The live benchmark result is the practical integration target. The fixed-evidence suite is a minimal sanity check for the governance head.

Training Data

Field	Value
Dataset	fitz-gov-v2
Clean active training rows	41,358
Training source pointer	fitz_gov_v2_41358_20260703
Base model	answerdotai/ModernBERT-base
Seed	42

Artifacts

This repository contains:

• model.safetensors: Transformers checkpoint
• model.onnx: FP32 ONNX export
• model_quantized.onnx: INT8 dynamic ONNX export
• tokenizer/config files
• manifest.json: release metadata

Limitations

1. Evidence-bounded judgment. Pyrrho judges only the retrieved evidence it is given. It does not retrieve new evidence or verify claims against outside knowledge.
2. English synthetic training data. The v2 dataset is English synthetic RAG governance data. Multilingual behavior is not established.
3. Metadata heads are policy signals, not formal proof. retrieval_intents and evidence_kinds are useful routing and audit hints. They do not prove SQL correctness, code execution behavior, or complete corpus coverage.
4. RAG integration still matters. Bad retrieval can produce bad evidence packs. Pyrrho can flag insufficiency or conflict, but it cannot recover source material that was never retrieved.

License

CC BY-NC 4.0. Free for research, evaluation, and personal use; commercial use requires a separate license.