JonathanKovatsch/parlpdfech-heads

ParlPDFeCH value heads for Apertus-v1.5-8B (layer 20)

Linear read-out heads on the hidden state of swiss-ai/Apertus-v1.5-8B, made for ParlPDFeCH, an entry of Hack Apertus 2026 (track 2A, OpenParlData). The code that uses them is on GitHub: https://github.com/jkovatsch/ParlPDFeCH. The file SHA256SUMS holds the SHA-256 of the 3 files.

What these files are

ParlPDFeCH converts a PDF of a Swiss parliament into an eCH-0295 record (PDF in, RDF out). Code rules read each value from the PDF first. Apertus-v1.5-8B writes no text: small linear heads read its hidden state and decide 2 things where the rules found nothing or only a weak value.

File	Content
heads2.npz	8 linear heads on the residual state after decoder layer 20 of the text branch of Apertus-v1.5-8B (bf16), at the last token of each question: q0 (4 classes, kind of document), q1 (7 classes: interpellation, request, draft_decree, motion, postulate, parliamentary_initiative, none), and 6 binary heads q2_<field> for one block of the document (affair_number, title, submission_date, sponsor, response_date, responder). Keys <head>__w, __b, __mu, __sd, __lambda (float32). Models 0 to 4 are fold heads; model 5 is the final head.
entry.json	The entry decisions: only q1 (affair type) and q2_sponsor in the band hi (head probability 0.80 or more) entered the converter. q0 and the other Q2 heads did not enter.
calibration_dev_combined.json	The value calibration: for each tier of the rules and for the 2 Apertus tiers, the share of development documents whose value was equal to the reference value. Its key documents holds development document ids and tier facts, no document text.

The heads are valid for ONE model only: the text branch of swiss-ai/Apertus-v1.5-8B (revision ef4bbc54d6fe658bc782280ff9f3b53a10a9758f) with the model fingerprint apertus-v1.5-8b-text-bf16:c10cd28db5bd3adf6043a5047a1435dedc5c3e55c45d5abc2ca82baf2a7786c3, and the prompt template gj_values_v1_de (SHA-256 54abf49ab788b2779b7bd00d8ebfb99abb3daf0fbe076e274fec30851dca81f6). The code of the repository checks both before it reads a state.

How they were made (13.10.2026)

• Documents: 1,793 development documents of 44 Swiss parliaments (1,792 with states; one document was longer than 32,768 tokens). 98,174 rows (one row for each question of each document).
• States: Apertus-v1.5-8B, text branch, bf16, no quantization, torch CUDA on a virtual machine with one H100 GPU. The whole document is in the prompt; one forward pass for each question; the residual state after layer 20 at the last token, stored as float16.
• Fit: logistic and softmax regression with an L2 penalty (lambda grid 0.001, 0.01, 0.1), torch LBFGS on the CPU, standardised inputs (__mu, __sd). 5 folds by parliament; the final head (model 5) is fitted on all documents of the fit.
• Labels: the values of the OpenParlData export of 04.10.2026 for the same documents (a reference, not a gold set).

Calibration of the 2 Apertus tiers (out-of-fold, development documents)

Tier	Field	Units	Equal to the reference	Stored probability
apertus_q1	affair type	100	93	0.925743
apertus_q2_sponsor_hi	first author	24	15	0.62 (the file marks this tier: enough_units false)

The stored probability is the Jeffreys mean (k + 0.5) / (n + 1) over all folds.

In the one fixed run of 13.10.2026 on 872 other documents (26 parliaments, not used in the fit), the tier apertus_q1 gave 59 values with a reference value, 55 of them equal to the reference (0.932). The tier apertus_q2_sponsor_hi gave 79 such values, 67 equal (0.848). The reference is the export, not a gold set.

Limits

• The reference of the fit and of all shares is the OpenParlData export, a register, not a gold set.
• The calibration is out-of-fold on development documents. AI agents read part of these documents during the development. The numbers are optimistic.
• The states of the fit came from torch CUDA bf16 (H100). Other kernels (MLX on Apple silicon, torch MPS, torch CPU, other GPUs) give other bf16 noise. A decision near a threshold (0.80 for Q1; 0.50 and 0.80 for the sponsor) can then differ from the fixed run. Final head, 15 development documents, MLX on one Mac (Apple M4, 32 GB), 16.10.2026: all 9 Apertus decisions were equal to those on the stored H100 states; 8 were written (one type had no type word in the text); the largest change of a head probability was 0.015 for Q1 and 0.047 for the first author. On 16 more development documents, the 12 Apertus decisions were equal too, but a head probability changed by up to 0.097 (Q1) and 0.055 (first author). The code marks a value whose head probability is within 0.10 of a threshold as near. No CUDA GPU other than the H100 of the fit was measured.
• The heads are valid only for the model fingerprint and the template above. Another model, another revision, a quantized model or another prompt gives states that these heads were not fitted on.
• The heads read only states. They need the code of the repository (prompt texts, value readers, writer, validator).

How to use

A head reads one state: the residual state after decoder layer 20 at the last token of one question, a vector of 4096 numbers. The questions and the prompt come from the template gj_values_v1_de in the code repository.

import numpy as np

h = np.load("heads2.npz")
m = 5  # model 5: the final head

def q1(x):  # the affair type: a probability for each of the 7 classes
    z = ((x - h["q1__mu"][m]) / h["q1__sd"][m]) @ h["q1__w"][m].T + h["q1__b"][m]
    p = np.exp(z - z.max())
    return dict(zip(h["q1_classes"], p / p.sum()))

def q2(field, x):  # one block: the probability that it prints the value of the field
    k = f"q2_{field}"
    z = ((x - h[k + "__mu"][m]) / h[k + "__sd"][m]) @ h[k + "__w"][m] + h[k + "__b"][m]
    return 1 / (1 + np.exp(-z))

The whole way from a PDF to an eCH-0295 record (the prompt texts, the value readers, the entry rules, the writer and the validator) is in the code repository: https://github.com/jkovatsch/ParlPDFeCH. These 3 files are public and need no token. The base model needs your own Hugging Face account: accept its conditions on https://huggingface.co/swiss-ai/Apertus-v1.5-8B.

License

The 3 files of this repository (head weights and calibration files: derived data): CC-BY-4.0, as in the file NOTICE of the code repository (https://github.com/jkovatsch/ParlPDFeCH). Source: OpenParlData.ch. The labels of the fit are values of the OpenParlData export of 04.10.2026 (https://files.openparldata.ch/exports/), licence of the data CC BY 4.0; the project calculated the heads and the shares from them. The code that uses the heads: Apache-2.0 (GitHub). This repository holds no weight of swiss-ai/Apertus-v1.5-8B; that model has its own license and its own Acceptable Use Policy, and each user accepts them on Hugging Face.