kikori

Target-directed sentiment for rashomon: given a person and a Portuguese text, how does the text treat that person? Trained and documented in walteraandrade/kikori.

Contract

Read it from config.json["kikori"] rather than copying it:

• Input: the pair (person, text) encoded as [CLS] person [SEP] text [SEP], token_type_ids 0 for the first segment and 1 for the second, max_length 256. Cut only the text so the closing [SEP] stays; truncation: true on a pair in transformers.js drops it and moves the score.
• Output: 3 logits in the order neg, neu, pos. Score = (p_pos - p_neg) * 10, range -10..10; class cut neg <= -2.5, pos >= 2.5. A temperature of 4.25 is already folded into the classifier weights.
• Files: onnx/model.onnx (fp32, 436 MB) and onnx/model_quantized.onnx (per-channel dynamic int8, 110 MB) for @huggingface/transformers (dtype: "fp32" / "q8"); model.safetensors for Python.
• fixtures.json: 24 pairs with the expected fp32 and int8 scores. fp32 should match to 1e-3 on any runtime; int8 drifts up to ~1.3 between runtimes, so test it with a tolerance.

Numbers

Holdout of 469 (person, text) pairs labelled by an LLM teacher under the rules in the repo's LABELLING.md, teacher-human agreement ~0.73:

	acc	MAE
fp32	0.77	1.76
int8	0.76	1.77
constant 0	0.59	2.16

Recipe: BERTimbau base, effective batch 32, lr 5e-5, 4 epochs, class-weighted cross-entropy, temperature calibration on validation. pos recall is 0.33; that is the known weak spot.

Known bias

The person name acts as a prior learned from skewed training labels (Lula's pairs lean pos, Tarcísio's and the Bolsonaros' lean neg). The same short hostile sentence scores +1.9 with Lula as the target and about -4.5 with the others; longer texts are read on their content. Compare outlets on the same person, not people against each other. See the repository README, "Known bias".

This model is a ruler, not a judge. It is biased; the requirement is that it is biased the same way for every outlet, so comparisons between outlets stay valid.