GSMB/aya-refusal-v2-adapters

LoRA adapters: closing the multilingual safety gap

Folders are arms: full (all modules/layers), allld (o_proj+down_proj, all layers), circuit (only the located refusal layers), circuit_gap, random (control layers). Base model: CohereLabs/tiny-aya-global.

Metric	base	full	allld	circuit	random	steer_x0.5	steer_x1	steer_x2
Refusal on harmful, seen langs (higher = safer)	53.8 [46.2, 61.9]	86.2 [80.6, 91.2]	86.2 [80.6, 91.2]	71.2 [64.4, 77.5]	97.5 [95.0, 99.4]	51.2 [43.7, 59.4]	56.9 [49.4, 64.4]	36.2 [28.7, 43.8]
Refusal on harmful, HELD-OUT langs	41.2 [33.8, 48.8]	61.3 [53.8, 68.8]	60.0 [52.5, 67.5]	40.6 [33.1, 48.8]	59.4 [51.2, 66.9]	22.5 [15.6, 29.4]	31.2 [23.8, 38.8]	38.8 [31.9, 46.2]
Jailbreak not-refused, seen (lower = safer)	59.2 [53.8, 65.0]	34.6 [28.7, 40.8]	35.4 [29.2, 41.7]	37.1 [31.2, 42.9]	36.7 [30.8, 42.9]	28.3 [22.9, 33.8]	54.2 [47.5, 60.8]	76.2 [70.8, 81.7]
Jailbreak not-refused, held-out	83.3 [78.3, 87.5]	52.5 [46.2, 58.8]	66.2 [60.4, 72.1]	58.3 [52.5, 64.2]	59.6 [53.8, 65.4]	51.7 [44.6, 58.3]	68.3 [62.1, 73.8]	78.3 [72.9, 83.3]
Over-refusal, benign (lower = better)	0.0 [0.0, 0.0]	0.3 [0.0, 0.9]	0.6 [0.0, 1.6]	0.0 [0.0, 0.0]	0.0 [0.0, 0.0]	1.9 [0.6, 3.4]	7.2 [4.4, 10.0]	1.9 [0.6, 3.4]
Over-refusal, borderline-safe	1.2 [0.3, 2.5]	7.2 [4.4, 10.0]	9.1 [5.9, 12.5]	8.4 [5.6, 11.6]	10.0 [6.9, 13.4]	17.5 [13.7, 21.9]	19.1 [15.0, 23.8]	0.9 [0.0, 2.2]
Safety gap: English minus held-out refusal (pts)	48.8	38.7	40.0	56.9	40.6	55.0	56.2	31.2
Capability: MGSM-lite accuracy	56.2	58.1	58.8	60.6	53.8	54.4	3.1	0.0
Params changed (%)	0.000	0.903	0.295	0.066	0.066	0.000	0.000	0.000

Values in %, brackets = 95% bootstrap CI (prompt-level). Refusal label source: judge. Test-set sizes are small; treat differences under ~5 pts as noise.

Research artifacts, not certified for deployment.