timidan/oya-tiny-aya-dpo-v3

Oya DPO round 1 (not released)

An experiment that did not replace Oya v2. Starting from v2, it trained with Lexsi Labs' AgentTune DPO trainer (TRL backend) on 222 preference pairs built from v2's own mistakes (pairs.jsonl): 2 epochs, learning rate 5e-5, beta 0.1, plus a supervised loss on the chosen turns.

	v2	Round 1
30 held-out requests: task success and right reply	83.3%	60.0%
30 held-out requests: wrong payments	0	1
416 development tasks: task success	97.4%	93.5%
Development safety tasks	100%	96.1%
Development replies in the right language, asking for the right detail	93.8%	52.6%
Development tasks where money moved that should not have	2.4%	4.3%

Scored with the current scorer (scripts/rescore.py). It learned to answer Yoruba requests in Pidgin: 75 of the 222 pairs penalised a Yoruba reply to a Pidgin request, and only one the reverse. It also once claimed a ₦5,500 electricity purchase that the wallet had refused. The release gate kept v2. Round 2 is timidan/oya-tiny-aya-dpo-v3b; the analysis is in the write-up, section 7.

Licence

CC-BY-NC 4.0, following Tiny Aya.