Solomon

A LoRA adapter and trained answer heads for Qwen/Qwen3.8-27B that turn a document plus a set of structured questions into one probability per decision, each with a retrieval pointer to where the support for it plausibly sits in the source. It does not generate text.

Read this before you trust a number on this page

v1.1 was measured on real documents, and the headline gain is not statistically established. On a held-out panel of 802 questions over 54 real documents it answers 706 whole questions right against 679 for v1.0: +3.4 points, 95% document-bootstrap interval [-1.3, +7.4]. The interval includes zero.

The evaluation labels are AI-generated and have not been checked by a human. See Evaluation labels.

Nothing on this page is a certified error rate and nothing is guaranteed.

What it is, and what it is for

Give it a document once and ask structured questions against it. Each answer comes back as a probability. On request it also returns ranked pointers — the three sentences an experimental relevance head scores highest — as a place to start reading, not as the reason for the answer (Evidence, below). There is no chat, no reasoning trace and no sampling: every answer is read from letter logits at a fixed position through trained linear heads, so the same document and the same question return the same numbers every time.

Four answer types:

Type	Question shape	What comes back
Yes / no	does the document establish X?	one probability
Single choice	which of these does it state?	one probability over the listed options
Ordered choice	which threshold does it state?	one probability over the ordered options
Multi-label	which of these apply?	one probability per candidate

v1.1 removed the entity answer type. "Which of these parties is the X?" is a yes/no question with the party written in: ask one yes/no question per candidate, or send the parties as multi-label candidates. An old entity request (candidate_kind: "entity", or a {candidate} placeholder) is answered with a 400 that says exactly this.

It is for turning documents into structured, machine-readable answers where you need a number attached to each one, and where determinism and a refusal to drift matter more than fluency.

It is not for general knowledge question-answering, chat, generation, or summarisation. It is not for any setting where a wrong answer is costly and cannot be checked: see Limitations.

How it works

Base model	Qwen/Qwen3.8-27B, Apache-2.0, pinned revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0. Not redistributed here
Adaptation	LoRA, rank 64, question-side placement, float32
Answer projection	trained linear heads (adapter/heads.npz), not the language-model head
Readout	structured letter logits; yes/no-shaped units collapse to a binary log-odds before temperature
Calibration	one scalar per answer type; all 1.0 (unscaled) in v1.1
Runtime identity	a 21-key binding (BF16; one binding per precision) that refuses to load if the engine is not the one measured

The document is prefilled once; the questions branch off it. The document goes through the model a single time and becomes a reusable state (POST /states). Every question is then answered as an isolated branch off that prefix. Two consequences are worth stating because they are tested on real hardware and not merely intended: asking the same questions in a different order returns the same answers, and answering from the cached document state matches a full forward pass with the same decision and within 0.05 in probability (the v1.1 acceptance runs: BF16 full, int8 subset; largest difference 0.0108 in BF16, and 0.0097 in an earlier BF16 run). That limit is looser than the 1e-3 used for fp32, because BF16 differences of about 0.01 are expected; the fp32 configuration was not re-run through acceptance for v1.1.

The adapter is off while the document is read, and on from the question onward. That is what question-side placement means. Applying it across the whole sequence gives a different model to the one that was measured, and the identity binding exists partly to stop that happening by accident.

The readout is structured, not generated. Rather than sampling an answer and parsing it, the model is asked to commit at a fixed position and the letter logits at that position are read through trained heads (readout mode four_collapsed). Yes/no-shaped units — a yes/no question, and each individual candidate inside a multi-label answer — are read through one merged yes/no head and collapsed to a single binary log-odds, p = sigmoid(z / T) with z = log P(yes)/P(no), before the temperature is applied. The head is shared, and results are reported per type. Choice questions apply the temperature to the listed slice, softmax(logits[:n] / T). v1.1 serves every type at T = 1.0 (boolean 1.0 · multilabel 1.0 · single 1.0 · ordered 1.0): per-type temperatures were fitted on the real dev panel and did not improve held-out calibration, so none is applied (see Calibration).

A branch whose head_key is not in the calibration artifact's map is refused, not served at an assumed 1.0.

The temperatures live inside the runtime binding, covered by its checksum. A calibration fitted on one model may not be served on another: the loader refuses by name rather than serving scalars that mean nothing. Serving at temperature 1.0 everywhere is always permitted, on any model.

What the returned score means

ordering_score means two different things depending on the question, and the difference matters.

• Single-unit — a yes/no question, a single or ordered choice, and every per-candidate value inside a multi-label answer. Here the score is the readout probability (unscaled in v1.1); how well that magnitude holds on real documents is measured under Reliability on real documents, below.
• Multi-unit — the rolled-up question-level score for a multi-label question with more than one candidate. It is the product of the per-candidate probabilities, which assumes those candidates are independent. That assumption has never been validated as a joint probability. It orders such questions well; it is a heuristic ordering, not a calibrated joint. If you need a magnitude for one of these, read the per-candidate numbers.

The field is not called probability because that would be accurate for the first case and an overclaim for the second.

There is no abstention. Every question is answered. The service will not emit a field named abstain, confidence, threshold or certified_error_rate; it raises rather than return one. If you want to decline low-confidence answers, that is your policy, made on your population, and this release makes no claim about where to put the line.

How to run it

The package is a library, not a daemon: you build the engine, wrap it in the serving layer and start the HTTP surface in four lines. The reference CUDA configuration is BF16: base weights in bfloat16 with the linear-attention recurrence promoted to float32 (precision='bf16', the configuration the model was trained in). The v1.1 BF16 and fp32 measurements on this card were run on NVIDIA B200 GPUs. Two alternatives are selectable, each with its own measured numbers below and its own identity: precision='fp32' (float32 weights and attention, float64 recurrence; roughly twice the memory) and precision='int8' (weight-only 8-bit decoder linears via torchao, compute in bf16; the smallest footprint, not faster: on an NVIDIA RTX A6000 (48 GB) it held 29.2 GiB after load and peaked at 32.8 GiB on the real test text documents and 40.4 GiB on page images). The binding pins the precision, so a configuration can only be served against numbers measured on it. The Apple-silicon MLX package in mlx/ has not been updated for v1.1: it is pinned to the v1.0 revision of this repository and loads the v1.0 adapter, heads and calibration, not the files described on this card. It is experimental, and none of the numbers on this card describe it.

pip install -r requirements.lock

huggingface-cli download DoccyHealth/Solomon --local-dir ./solomon
huggingface-cli download Qwen/Qwen3.8-27B --revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0 --local-dir ./solomon/base

cd solomon
export PYTHONPATH=$PWD/src

Run from inside that directory: the engine looks for the base model in base/, and the serving layer reads serving/selection.json and the binding it names, both relative to the package root.

from solomon import api, service
from solomon.serving import ServiceEngine

engine = ServiceEngine('adapter/adapter.safetensors', 'adapter/heads.npz')   # precision='bf16' (default) | 'fp32' | 'int8'
layer = service.service('./store', engine)   # './store' holds cached document states
# fp32 / int8: build the engine with that precision AND pass its binding, e.g.
#   engine = ServiceEngine(..., precision='int8')
#   layer = service.service('./store', engine, binding='serving/serving-binding-int8.json')
server = api.serve(layer)                    # GET /health, POST /states, POST /v1/decide
print('http://127.0.0.1:%d' % server.server_port)
server.serve_forever()                       # or skip the server and call layer.decide(...) directly

ServiceEngine also takes expected_adapter= and expected_heads=, the two sha256 values printed below; pass them and the engine refuses to start on a file that is not the one this card describes.

Asking questions

One request carries a document (or a saved state_id) and any number of questions. The four shapes:

POST /v1/decide
{
  "state": "…the document text…",
  "evidence": "support",                 // none | support | sufficiency | removal
  "questions": {
    "certified": "Is Rookwood Ltd certified to supply produce?",                       // yes / no
    "growers":   {"type": "noul", "instructions": "Which growers may supply produce?",
                  "candidates": ["Rookwood Ltd", "Ostler Ltd"]},                        // multi-label
    "scheme":    {"type": "choice", "instructions": "Which certification scheme applies?",
                  "options": ["Freshcare", "SQF", "GlobalG.A.P."]},                      // single choice
    "risk":      {"type": "choice", "instructions": "What supply risk rating is recorded?",
                  "options": ["low", "medium", "high"], "ordered": true}                 // ordered choice
  }
}

{
  "answers": {
    "certified": {"type": "noul", "noul": 0.97, "ordering_score": 0.97, "temperature": 1.0,
                  "evidence_method": "trained_relevance_head_ranked", "evidence": [{"start": 212, "end": 256, "text": "…", "score": 0.83, "rank": 1}, …], "evidence_suppressed": false},
    "growers":   {"type": "noul", "candidate_kind": "label",
                  "candidates": {"Rookwood Ltd": 0.96, "Ostler Ltd": 0.04},
                  "candidate_ordering_scores": {"Rookwood Ltd": 0.96, "Ostler Ltd": 0.96},
                  "candidate_evidence": {"Rookwood Ltd": [{"start": 212, "end": 256, "text": "…", "score": 0.91, "rank": 1}, …],
                                         "Ostler Ltd":   [{"start": 257, "end": 309, "text": "…", "score": 0.77, "rank": 1}, …]},
                  "candidate_evidence_suppressed": {"Rookwood Ltd": false, "Ostler Ltd": false}},
    "scheme":    {"type": "choice", "answer": "Freshcare", "probabilities": {"Freshcare": 0.91, "SQF": 0.06, "GlobalG.A.P.": 0.03}},
    "risk":      {"type": "choice", "ordered": true, "answer": "medium", "probabilities": {"low": 0.12, "medium": 0.81, "high": 0.07}}
  }
}

(Illustrative numbers, not measurements.) The entity form {"instructions": "Is {candidate} a grower?", "candidates": [...]} was removed in v1.1: write "growers" above, or one yes/no question per party.

Evidence. At evidence: "support" the response carries up to three ranked sentence pointers with scores from an experimental relevance head, per answer branch. evidence_method names the selector: trained_relevance_head_ranked, or lexical_overlap_fallback (word overlap, used for page images and whenever the head is not configured, with evidence_fallback_reason saying why). Neither establishes that the answer was caused by the span (evidence_faithfulness_established: false). See Evidence (experimental).

GET /health reports the contract, the readout mode and the runtime identity. If the engine you built differs from the one the numbers were measured on — a different torch build, a different adapter, a different arithmetic mode — the binding refuses to load rather than quietly serving different numbers. That is intended behaviour; do not work around it.

What is in this repository

README.md                            this file
LICENSE                              Apache-2.0
NOTICE                               attribution and the Apache-2.0 4(b)/4(c) notices
MODIFICATIONS.md                     statement of changes, third-party text scan, maintainer notes
MANIFEST.json                        every file, its size and its sha256
requirements.lock                    the pins of the image this was qualified on, and dependency licences
licenses/Qwen-Apache-2.0.txt         the upstream licence, verbatim
adapter/adapter.safetensors          the LoRA adapter
adapter/heads.npz                    the trained answer heads
adapter/config.json                  base repo, pinned revision, checksums, placement
serving/serving-binding.json         runtime identity binding (bf16, default) and the frozen temperatures
serving/serving-binding-fp32.json    runtime identity binding for precision=fp32
serving/serving-binding-int8.json    runtime identity binding for precision=int8
serving/evidence-head.safetensors    relevance head weights (float32)
serving/evidence-head.json           relevance head config incl. lexical_residual alpha
serving/evidence-policy.json         ranked-pointer serving policy (top 3, suppressed when not stated)
serving/readout-temperature-v3.json  the calibration artifact, standalone
serving/selection.json               readout mode and model identity
serving/service-export.json          qualification envelope hashes
src/solomon/__init__.py              solomon: a document plus structured questions in, one probability per decision out
src/solomon/api.py                   HTTP surface for the Solomon layer (solomon/service.py)
src/solomon/binding.py               serving identity: which readout is served, and proof that it is the one that was measured
src/solomon/calibration.py           the readout calibration: one positive scalar per answer type, and nothing else
src/solomon/engine.py                question-only CUDA runtime for immutable semantic-head checkpoints
src/solomon/engine_contract.py       the answer contract engine: contract v3 on the reference engine's float32 cached path
src/solomon/engine_cuda.py           CUDA contract-v3 engine
src/solomon/engine_numerics.py       versioned CUDA repair: bounded FP32 attention, FP64 recurrent accumulation
src/solomon/engine_reasoning.py      the confidence layer text reasoning on immutable answer-contract base-prefix states
src/solomon/engine_reference.py      the reference engine: task-agnostic document prefix, float32 arithmetic, chunked prefill
src/solomon/evidence.py              deterministic source references and explicit evidence interventions
src/solomon/evidence_head.py         v1.1 evidence head: the ONE forward shared by the trainer (the training tooling via
src/solomon/evidence_packages.py     evidence packages (v3): selected spans plus source-derived governing context, per question unit
src/solomon/evidence_selector.py     evidence selection for the Solomon layer (v1.1): the trained relevance head, with word overlap as a labelled fallback
src/solomon/heads.py                 final normalized feature extraction, preserving the qualified CUDA engine
src/solomon/prompts_two_letter.py    two-letter (Noul) prompts and block conversions
src/solomon/readout.py               contract v3 readouts: branch jobs for every answer type, predictions from letter logits, and metrics
src/solomon/reliability.py           what the readout says about its own answer, with nothing fitted behind it
src/solomon/retrieval.py             inference-only source candidates and explicitly labelled retrieval baselines
src/solomon/routing.py               real callback-driven escalation
src/solomon/semantics.py             answer semantics (design note, not distributed)
src/solomon/service.py               decision layer: the Solomon serving contract over the pinned readout chain
src/solomon/service_answers.py       confidence-aware five-task service, reusing immutable answer-contract input persistence
src/solomon/service_checked.py       backend-neutral contract-v3 service; restart replays immutable inputs, not tensors
src/solomon/service_evidence.py      optional source-grounded evidence around the existing confidence service
src/solomon/service_heads.py         trained-head fast/views service; unchanged decoder and separate stage confidence
src/solomon/service_packages.py      evidence packages (v3) in the answer service: per-unit, source-grounded, page-referenced
src/solomon/service_states.py        local contract-v3 prototype: task-neutral text/image states and five answer types
src/solomon/serving.py               preserve five-task trained-head confidence/routing with optional evidence
src/solomon/units.py                 v1.1 shared sentence/list-item splitter
mlx/                                 the optional Apple-silicon package: its own library, tests and notices

What is deliberately not here: no training data, no evaluation panels, no document corpora, no datasets, no cached states, no rendered images, no score archives, no test fixtures, no logs and no base model weights. The package is default-deny: a manifest names every permitted file and the build fails if anything else is present.

The files under serving/ carry identity only: hashes, the readout mode, the served design and the per-type temperatures. The loader verifies the binding's checksum and every runtime key it carries. The measured provenance behind those hashes (which panels, which fit, which qualification run) is held in the maintainer's records and is not distributed.

Evidence (experimental)

Evidence here is a set of ranked pointers, not an explanation. At evidence: "support" the service returns the top 3 sentences of the document with a relevance score each (evidence_method: "trained_relevance_head_ranked"). The scores come from a small relevance head fitted after training, on the model's layer-42 states plus a word-overlap term. It reads states the answer already computed; it makes no extra model call and cannot change an answer: with the head on and off, all 3,230 real test answer branches were identical (maximum logit difference 0.0).

• evidence_faithfulness_established is false. Nothing shows the pointed-to sentence caused the answer.
• No pointers are returned when the answer is "not stated" (the answer's collapsed state is not-stated, or a choice answer resolved to the reserved not-stated option). That state is the absence signal; the head itself has no reliable "no evidence" signal.
• On the development panel the head put a labelled supporting sentence in its top 3 more often than plain word overlap: hit@3 0.817 against 0.669 over 753 labelled rows. That comparison was used to choose the head, so it is optimistic, and no test-panel measurement exists. The precision and recall targets for evidence were not established.
• Word overlap is the labelled fallback (lexical_overlap_fallback, with evidence_fallback_reason), used for page-image documents and whenever the head is not configured.

Treat pointers as a place to start reading and verify them yourself.

What it scores

Every figure in this section was measured on the adapter this repository ships (d122466d430a…) in the BF16 reference configuration unless stated. The scoring runs loaded a heads file (96ea51416bbe…) that is the shipped adapter/heads.npz (f766d752d776…) plus two legacy slots that were never read; every array the two files share is byte-identical, so the served logits are the measured logits. The packager checks those hashes against the measurement records and refuses to build if they differ. Figures for v1.0 are that model re-scored on the same panels, for comparison only.

Real documents

Real public documents (government notices, policies, agreements, correspondence, minutes and similar), split by document into training, dev and test. Test documents were never trained on and never used for any fit.

Panel	Questions / documents	This model	v1.0	External reference¹	v1.1 − v1.0, points [95% CI]
Test	802 / 54	706 (88.0%)	679 (84.7%)	690 (86.0%)	+3.4 [-1.3, +7.4]
Dev	624 / 15	555 (88.9%)	529 (84.8%)	553 (88.6%)	+4.2 [+1.0, +7.5]

¹ a commercial structured-decision API (external reference), scored on the same questions and labels.

Intervals are document-cluster bootstrap (2,000 resamples). On test the interval includes zero: the release rule asked for a lower bound of −1.0 points and the measured bound is -1.3. The owner accepted this miss for v1.1; see Release decisions.

Per answer type, real test:

Answer type	Questions	This model	v1.0	External reference	v1.1 − v1.0, points [95% CI]
Yes / no	149	132 / 149	132 / 149	135 / 149	+0.0 [-10.1, +7.2]
Party-role questions (formerly entity; now asked as yes/no per party)	213	185 / 213	165 / 213	180 / 213	+9.4 [+0.0, +19.2]
Multi-label	226	191 / 226	190 / 226	184 / 226	+0.4 [-4.8, +5.9]
Ordered choice	109	95 / 109	90 / 109	91 / 109	+4.6 [-1.9, +10.7]
Single choice	105	103 / 105	102 / 105	100 / 105	+1.0 [+0.0, +3.0]
Multi-label, per candidate (slots)	1,204	96.5%	95.8%	not tallied	+0.7 [-1.0, +2.3]
Party-role, per party (slots)	1,329	97.2%	94.9%	not tallied	+2.3 [+0.4, +4.1]

Counts are whole questions right, with the same question definition applied to all three models.

Page images against text

The same 802 real test questions, asked from rendered page images instead of extracted text: 713 whole questions right from images against 706 from text, with 98.5% of 2,936 answer branches agreeing between the two.

Natural images

A panel of photographs and pictures with structured questions (no document text). Two populations are reported and they are different numbers:

• Per answer unit (731 units): this model 97.3%, v1.0 96.2%, base Qwen 95.5%. Answers stated at 0.99 or above that were wrong: this model 0.0% of 293, base Qwen 1.5% of
• Whole questions (415): this model 95.2%, v1.0 93.3%, base Qwen 92.0%.

General knowledge, with no document (out of domain)

800 multiple-choice items, 400 from MMLU and 400 from MMLU-Pro, every model on the same items and prompt.

Model	Accuracy (800)	Answers stated ≥ 0.99	Of those, wrong	ECE (top label, 15 bins)
This model (v1.1)	72.9%	24	0 (0.0%)	0.052
v1.0	72.8%	401	20 (5.0%)	0.146
External reference¹	87.1%	314	5 (1.6%)	0.040
Base Qwen, same prompt	71.8%	187	4 (2.1%)	0.049

All four rows come from one computation on the same 800 items. Probabilities are unscaled (T = 1) for every row, which is how v1.1 serves them.

v1.0 stated half of its answers at 0.99 or above and was wrong on 1 in 20 of them. This model almost never claims 0.99 on general knowledge. MMLU moved +1.5 points and MMLU-Pro -1.25 points against v1.0; the release rule allowed at most 1 point either way, so MMLU-Pro missed it. The owner accepted this miss for v1.1.

Reliability on real documents

Real test, text, every answer unit (each option or candidate scored against its label): of the probabilities stated in each band, the share that were actually right. A calibrated model's column would track the band.

Stated P(yes)	this model	external reference
0.00–0.01	0.0% (n=2,095)	0.1% (n=2,619)
0.01–0.02	0.8% (n=1,038)	0.7% (n=305)
0.02–0.05	3.2% (n=569)	1.8% (n=325)
0.05–0.10	14.4% (n=132)	3.0% (n=202)
0.10–0.20	46.0% (n=87)	7.5% (n=213)
0.20–0.30	51.6% (n=31)	18.1% (n=116)
0.30–0.40	18.2% (n=22)	28.6% (n=63)
0.40–0.50	66.7% (n=18)	30.4% (n=56)
0.50–0.60	47.1% (n=17)	39.5% (n=43)
0.60–0.70	61.5% (n=13)	55.6% (n=63)
0.70–0.80	53.1% (n=32)	68.6% (n=70)
0.80–0.90	83.7% (n=43)	82.9% (n=111)
0.90–0.95	89.3% (n=84)	91.0% (n=89)
0.95–0.98	97.3% (n=295)	92.5% (n=106)
0.98–0.99	99.6% (n=485)	95.6% (n=90)
0.99–1.00	100.0% (n=126)	99.5% (n=616)

The top end is at or above the external reference. The low end under-calls: answers stated at 5–20% are yes more often than stated. (This table is the unscaled readout, which is what v1.1 serves.)

Calibration

Per-type temperatures were fitted on the real dev panel (boolean 0.8175 · multilabel 0.8423 · single 1.1077 · ordered 1.2562) and checked on held-out real test. They did not improve held-out calibration: test ECE got worse in 8 of 10 answer-type × modality cells, and the question-weighted ECE across all cells was 0.0212 with the fitted temperatures against 0.0199 unscaled. Only multi-label improved. v1.1 therefore ships unscaled probabilities (T = 1.0 for every type). Real test ECE per cell (10 bins):

Answer type	Input	Units	ECE, unscaled (served)	ECE, dev-fitted temperature	Meets 0.03 target
yes/no	image	149	0.055	0.068	no
yes/no	text	149	0.067	0.081	no
party-role (per party)	image	1,329	0.009	0.020	yes
party-role (per party)	text	1,329	0.011	0.013	yes
multi-label (per candidate)	image	1,204	0.020	0.012	yes
multi-label (per candidate)	text	1,204	0.024	0.013	yes
ordered	image	109	0.073	0.086	no
ordered	text	109	0.054	0.077	no
single	image	105	0.022	0.053	yes
single	text	105	0.014	0.040	yes

The 0.03 target is missed for yes/no (0.067 text, 0.055 image) and ordered choice (0.054 text, 0.073 image). Those cells have only 105–149 questions each (yes/no, single and ordered). Party-role rows are the former entity questions, now asked as one yes/no question per party.

Precision configurations

Real test, text, 3,230 answer branches. Accuracy is whole questions; flips are served decisions that differ from BF16.

Configuration	Status	Whole-question accuracy	vs BF16, points [95% CI]	Decisions flipped vs BF16	ECE (unscaled)
bf16	default, reference	88.03%	—	—	0.018
fp32	comparison	87.66%	-0.37 [-0.77, +0.00]	0.10%	0.016
int8	option	87.53%	-0.50 [-1.02, +0.00]	0.27%	0.016

int8 is weight-only 8-bit (torchao) with bf16 compute. int8 page-image accuracy has not been scored against the labels; a decision-agreement run against fp32 on page images agreed on 99.71% of served decisions. On page images fp32 scored 88.78% against BF16 88.90%. No speed claim is made for any configuration.

Failure modes (synthetic probes)

6,000 generated questions across 38 targeted failure modes, paired against v1.0. Most modes are flat. Modes whose interval excludes zero:

Mode	Questions	v1.0	This model	Difference, points [95% CI]
indirect reference	160	80.6%	72.5%	-8.1 [-13.1, -3.8]
opposite polarity question	135	94.1%	97.0%	+3.0 [+0.7, +5.9]

• Paraphrase agreement on yes/no questions: 0.951 (v1.0 0.946); the target was 0.98 and is not met.
• Adversarial confident flips (answer changed at ≥ 0.9 by an injected instruction, false summary or self-classifying text): 2.0% (v1.0 2.1%); the target was 1% and is not met.
• Per-type calibration on real test misses the 0.03 ECE target for yes/no and ordered questions (see Calibration, below; reported, not blocking).

Limitations

• Evaluation labels are not human-verified. Every real-document reference label was produced by AI labellers: two blind passes plus adjudication, with 99% agreement on binary slots between the passes. The owner decided to release v1.1 without a human label review. Some measured errors may be label errors, and some measured successes may share a labeller's mistake.
• The headline improvement is not significant (interval includes zero) and the real panels are small: 54 test documents.
• The low end of the probability scale under-calls on real documents (see the reliability table).
• Indirect references regressed on the synthetic probes (table above).
• The multi-candidate roll-up is an ordering, not a joint probability. Read per-candidate values if you need a magnitude.
• Nothing here is a certified error rate. No threshold is enforced anywhere on the serving path.
• Page images: measured on the real test panel in BF16 and fp32 only.

Release decisions

Two release rules were missed and both were accepted by the owner for v1.1:

1. Real test, whole questions: lower bound of the 95% interval -1.3 points against a rule of −1.0.
2. General knowledge: MMLU-Pro -1.25 points against a limit of 1 point (MMLU +1.5).

Evaluation labels

Reference labels on the real panels are AI-generated (two blind passes plus adjudication) and have not been reviewed by a human. No Claude or GPT output is used anywhere as training input.

Training data (v1.1)

Real documents. 200 real public documents: 160 collected for this release plus 40 from an earlier evaluation panel (those 40 are test-only). Split by document: 113 train, 24 dev, 63 test (the evaluation panels above use the labelled subset). Licences of the 160 collected documents, as recorded at collection: 59 Australian government pages under CC BY 4.0, 76 UK Crown copyright under the Open Government Licence v3.0, 25 US federal government works (public domain). No document is distributed here.

Where the training labels came from.

• Real-document training labels: Qwen3.8 2.4T (open weights), called through OpenRouter and routed to third-party hosts serving full-precision weights, not the Alibaba API.
• Anchor targets from the unmodified base Qwen model, so general behaviour does not drift.
• Code generators for the synthetic documents and targeted failure-mode questions.
• Replay of the v1.0 training data (whose third-party sources are listed below).

No Claude or GPT output is ever training input or a training label. The build enforces this with an allow-list of row producers.

Training-data provenance (third-party texts in the synthetic documents)

No training document, panel, corpus or source text is distributed in this repository. The adapter was trained on synthetic documents, and some of those documents were produced by editing third-party natural texts. Those texts are listed here so that their provenance is on the record, and so that a reviewer doing lawful-sourcing diligence can see what was used without having to ask.

The model card discloses the provenance of the third-party texts the training panel was edited from, and attributes them. This is provenance disclosure and attribution as good practice, and it supports documented-lawful-sourcing procurement review. It is NOT a concession that the trained weights are a derivative work or an adaptation of those texts; that question is open and nothing in this package answers it.

13 of the 42 reviewed sources were used in the training panel:

	Source	URL	Licence, as recorded
	Django's security policies	https://docs.djangoproject.com/en/dev/internals/security/	BSD 3-Clause (Django project LICENSE, which covers the documentation in the django/django repository)
CC BY-SA	Wikipedia:Arbitration/Policy (English Wikipedia arbitration policy)	https://en.wikipedia.org/wiki/Wikipedia:Arbitration/Policy	CC BY-SA 4.0 (Wikipedia text; attribution: English Wikipedia contributors, 'Wikipedia:Arbitration/Policy')
	Common Paper Mutual Non-Disclosure Agreement, Version 1.0 – Standard Terms	https://commonpaper.com/standards/mutual-nda/1.0/	CC BY 4.0 (stated in the agreement footer and in the CommonPaper/Mutual-NDA repository README: 'free to use and modify under CC BY 4.0')
	36 CFR Part 2 (National Park Service) - Resource Protection, Public Use and Recreation: sections 2.10, 2.13, 2.14, 2.15, 2.16, 2.21 and 2.22	https://www.ecfr.gov/current/title-36/chapter-I/part-2	US Government work (public domain): Code of Federal Regulations text, not subject to copyright (17 U.S.C. 105)
	NSF Proposal & Award Policies & Procedures Guide (PAPPG, NSF 24-1), Chapter IV: Non-Award Decisions and Transactions	https://www.nsf.gov/policies/pappg/24-1/ch-4-non-award-decisions-transactions	US Government work (public domain): U.S. National Science Foundation policy guide, not subject to copyright (17 U.S.C. 105)
	GOV.UK: Make a court claim for money	https://www.gov.uk/api/content/make-court-claim-for-money	Open Government Licence v3.0
	42 CFR 68: NIH Loan Repayment Programs (2025 edition)	https://www.govinfo.gov/content/pkg/CFR-2025-title42-vol1/xml/CFR-2025-title42-vol1-part68.xml	US Government work (public domain)
CC BY-SA	WordPress.com (Automattic) Terms of Service, last updated April 10, 2026	https://github.com/Automattic/legalmattic/blob/master/Terms%20of%20Service/WordPress.com/EN-Terms-of-Service.md	CC BY-SA 4.0 (Automattic/legalmattic LICENSE.txt and README; the Terms themselves state they are available under a Creative Commons Sharealike license)
CC BY-SA	GitLab Handbook: Global Travel and Expense Policy (sections 1 to 3)	https://handbook.gitlab.com/handbook/finance/expenses/	CC BY-SA 4.0 (GitLab handbook content; licence badge in the footer of handbook.gitlab.com pages)
	Sourcegraph Handbook: Spending company money	https://github.com/sourcegraph/handbook/blob/main/content/benefits-pay-perks/benefits-perks/spending-company-money.md	Apache License 2.0 (LICENSE of the public sourcegraph/handbook repository)
	Project Jupyter Governance: Executive Council	https://raw.githubusercontent.com/jupyter/governance/main/docs/executive_council.md	CC0 1.0 Universal (jupyter/governance repository LICENSE.md)
	29 CFR 1904.30–1904.34: establishment records and annual summaries (2025 edition)	https://www.govinfo.gov/content/pkg/CFR-2025-title29-vol5/xml/CFR-2025-title29-vol5-part1904.xml	US Government work (public domain)
	31 CFR 1.2, 1.4 and 1.6: Treasury FOIA requests and appeals (2025 edition)	https://www.govinfo.gov/content/pkg/CFR-2025-title31-vol1/xml/CFR-2025-title31-vol1-part1.xml	US Government work (public domain)

3 of these are recorded as CC BY-SA 4.0 — Wikipedia:Arbitration/Policy (English Wikipedia arbitration policy), WordPress.com (Automattic) Terms of Service, last updated April 10, 2026, GitLab Handbook: Global Travel and Expense Policy (sections 1 to 3). Share-alike is the one term attribution cannot cure. Whether a share-alike obligation can propagate through training into model weights is legally unsettled; there is no authority settling it in either direction, and the project's own licence review explicitly declines to infer one. The owner of this release accepted that residual risk on 2026-09-21 rather than resolving it, and kept this package under Apache-2.0. A reader should treat the question as open, not answered.

On the strength of this evidence. The licences above are as recorded by the person who collected each source, from the source's own stated terms at the time of collection. The review records evidence_level: "authoring metadata assertion, not archived governing licence text" and upstream_terms_independently_verified: false for every row. No governing licence text was archived alongside most of these sources, and this listing should not be read as a licence audit.

Listing these sources is provenance disclosure and attribution as good practice. It is not a statement that the trained weights are a derivative work, an adaptation, or a copy of any of these texts.

Third-party dependency licences

requirements.lock names the packages the serving layer needs. None of them is redistributed in this repository — you install them yourself from their own publishers — so Apache-2.0 section 4(a) imposes no bundled-notice obligation here and no dependency licence text is packaged. This summary exists because a reviewer will ask for one.

Each licence in the table is the one declared in that distribution's OWN package metadata -- the METADATA file of an installed wheel -- read from a copy on the maintainer's machine. Where no copy existed, the row says NOT VERIFIED instead of guessing.

Package	Pinned as	Licence, as declared by the distribution itself
torch	2.13.0	BSD-3-Clause — read from version 2.8.0
torchvision	0.28.0	NOT VERIFIED
transformers	5.17.0	Apache 2.0 License
flash-linear-attention	0.5.2	NOT VERIFIED
safetensors	unpinned in the qualified image	Apache Software License — read from version 0.8.0
accelerate	unpinned in the qualified image	Apache (Apache Software License) — read from version 1.15.0
numpy	unpinned in the qualified image	BSD-3-Clause AND 0BSD AND MIT AND Zlib AND CC0-1.0 — read from version 2.5.3
scipy	unpinned in the qualified image	BSD License (classifier); the METADATA License field carries the Enthought / SciPy Developers copyright line rather than an SPDX identifier — read from version 1.18.1
pillow	unpinned in the qualified image	MIT-CMU — read from version 12.3.0

Two rows say NOT VERIFIED and mean it. A licence read from one installed version is evidence about that version only. Where the table names a version different from the pin, that is the version whose metadata was actually read, and the pinned version's own metadata could differ. Treat this table as a starting point for your own review, not as a legal opinion, and re-check the distributions you actually install.

Full evidence paths for each row are in the release descriptor (release/solomon-release.json → dependency_licences), which is not distributed; the same information is repeated in the comments of requirements.lock.

Licence and attribution

Copyright 2026 Doccy Pty Ltd.

This repository is licensed Apache-2.0 — the adapter and head weights, the calibration artifact, the serving code and the documentation alike. See LICENSE and NOTICE.

It is a derivative work of Qwen/Qwen3.8-27B, Copyright 2026 Alibaba Cloud, licensed under Apache-2.0. The upstream licence text is reproduced verbatim at licenses/Qwen-Apache-2.0.txt (sha256 bbedc3fda3305820b977265f01b8619d87570a6739de3a5582c3464840f1e57a). NOTICE carries the attribution required by section 4(c) and MODIFICATIONS.md the statement of changes required by section 4(b).

No NOTICE file exists in the upstream repository at the pinned revision (HTTP 404, checked 2026-09-18), so section 4(d) carries nothing forward and no upstream attribution text has been invented.

"Qwen" and "Alibaba Cloud" are used nominatively to identify the base model. Apache-2.0 section 6 grants no trademark rights and none are claimed. No endorsement or affiliation is implied.

Verify what you downloaded

shasum -a 256 adapter/adapter.safetensors      # d122466d430a058bb6457d919f811160e97fbd20149f4f24ca455c5d83e360a0
shasum -a 256 adapter/heads.npz                # f766d752d7768a419a9657155cf27f042834d9de29392cf7470d8725130e67ab
shasum -a 256 serving/readout-temperature-v3.json
#   -> 1a2285d8fd56d17ee1d06a1e9fce866cc0d3b0263730754babb11deea5f1f7c9

MANIFEST.json carries the size and sha256 of every file in this repository.

The calibration artifact has two legitimate and different hashes, and confusing them makes a sound provenance chain look tampered with. 1a2285d8… is the file hash, what shasum returns. 945bad44… is the artifact's own internal sha256 field, computed over its contents with that field removed — a self-referential field cannot hash the file containing it. The loader verifies the payload hash; use the file hash to check the file you were given. Both are recorded in the serving binding's provenance, under those names.

Release record

Machine-readable identity for citation and pinning. The model identity, the calibration and the runtime binding move together; pin by revision.

Repository	DoccyHealth/Solomon
Release	1.1.0, 2026-09-21
Serving contract	solomon-v1
Adapter sha256	d122466d430a058bb6457d919f811160e97fbd20149f4f24ca455c5d83e360a0
Heads sha256	f766d752d7768a419a9657155cf27f042834d9de29392cf7470d8725130e67ab
Base model	Qwen/Qwen3.8-27B at 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0
Runtime binding sha256 (payload)	0add0efda28902180db757a12160953e0f5d8dc303decc27b2cfda84db900da8
Calibration sha256 (file / payload)	1a2285d8fd56d17ee1d06a1e9fce866cc0d3b0263730754babb11deea5f1f7c9 / 945bad449b7f5ffc88e597277d632fbab81c3c8729e22c8babd3f4a45fe1378b
Runtime fingerprint	7d529382321e0e14131a643920d7522b85d1b521d249db89fba839b3fd0f8bea
Readout	four_collapsed
Served temperatures	boolean 1.0 · multilabel 1.0 · single 1.0 · ordered 1.0