lvjr3383/tiny-aya-refusal-steering

Strong Refusal Control, Weak Safety Selectivity: Evaluating Activation Steering in Tiny Aya

Lexsi AI Safety & Alignment Hackathon, Track 2b. Artifacts for an activation-steering study on Tiny Aya Global (3.35B) using SafeTune (refusal direction, steering), CircuitKIT (circuit discovery, faithfulness) and AuditKIT (all evaluations).

Result in one line: a refusal-related direction strongly controlled refusal under large interventions, but at the validation-selected strength (k=0.1) it showed no detectable safety advantage over matched random directions (verdict NO_DETECTABLE_EFFECT); stronger interventions mainly increased over-refusal.

Contents

Path	What
refusal_direction.pt	SafeTune difference-of-means refusal direction (unit vector) and layer index. Use with RefusalDirectionModel(mode="steer", strength=k*gap, layers=[layer]), gap = Phase 1 projection gap (see evidence).
phase2_gsm8k_lora_adapter/	Phase 2 benign GSM8K LoRA adapter (r=16). Research artifact only: it reduced MGSM accuracy; not recommended for use.
notebooks/	The four executed notebooks (Phase 1 feasibility, Phase 2 drift, Phase 3A screening, Phase 3B steering).
figures/, results_tables.md	Figures and tables generated directly from the saved result files.
evidence_manifest.json	SHA-256 of every evidence file across all phases.

Full evidence (splits, generations, judge labels, reports): dataset lvjr3383/tiny-aya-refusal-steering-evidence (gated: contains harmful prompts).

Reproduce

Colab A100 (L4 suffices except CircuitKIT steps). Run the notebooks in order; each is resumable, config-hashed, and writes to Google Drive. Pinned: safetune==0.1.10, circuitkit==0.2.0, auditkit[transformers]==1.1.3; transformers 5.21, torch 2.11. Requires access to CohereLabs/tiny-aya-global, CohereLabs/aya-expanse-8b (Phase 3A judge), allenai/wildguard (Phase 3B judge).

Limitations

Single model; modest samples (primary n=60, low base unsafe rate); Phase 2 safety results are keyword-measured; Phase 3B safety outcomes are AI-judged (WildGuard, Grok), not human-validated, and inter-judge agreement is not accuracy; steering is inference-time, not a weight repair; the CircuitKIT faithfulness check is exploratory (n=16, unequal-length prompt pairs).

AI coding and review assistants were used during development.

Base model licence: see CohereLabs/tiny-aya-global; the adapter inherits its terms.