Oya DPO round 2 (not released)

An experiment that did not replace Oya v2. Starting from v2, it trained with Lexsi Labs' AgentTune DPO trainer (TRL backend) on 146 pairs: round 1's pairs without those whose only fault was the reply language (pairs.jsonl). 2 epochs, learning rate 2e-5, beta 0.1, plus a supervised loss on the chosen turns.

v2 Round 2
30 held-out requests: task success and right reply 83.3% 90.0%
30 held-out requests: wrong payments 0 1
416 development tasks: task success 97.4% 95.0%
Development safety tasks 100% 96.1%
Development replies in the right language, asking for the right detail 93.8% 91.1%
Development tasks where money moved that should not have 2.4% 3.4%

It fixed the three follow-up questions v2 got wrong on the held-out requests, but sent "put ₦2,000 on Mama's phone" as a bank transfer to Mama. On the development set it got worse at airtime for contacts (69% to 56%) and at requests with missing details (100% to 88%), and it failed one planted injection. The release gate kept v2; the analysis is in the write-up, section 7.

Licence

CC-BY-NC 4.0, following Tiny Aya.

Downloads last month
16
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for timidan/oya-tiny-aya-dpo-v3b

Adapter
(103)
this model

Dataset used to train timidan/oya-tiny-aya-dpo-v3b

Collection including timidan/oya-tiny-aya-dpo-v3b