siddharthmb/2026.MI.RL-Steering-Adapters

LoRA adapters of the June 2026 RL and steering experiments

Twenty-five PEFT LoRA adapters from the MATS-II experiments of 24 to 28 June 2026, one subfolder each: the covert-sycophancy policies (covert_syc_minlen100_s{0,1,2}, covert_persona_syc_w{0,4}_s0, grpo_covert_il_q7_w{0,2,4}_s0), the probe-attack policies (probeattack_syc_*), the judge-panel policies on Llama-3.2-3B (judgepanel_*), the scaled RFT and GRPO runs of the B200 session (scaled_rft_cal_q32_*, scaled_grpo_q14_gemma9b_vllm, deconf_cal_q32_gemma2b), and the three negotiation and secret-word RL adapters (negotiation_rl_adapter, secret_word_rl_adapter, secret_word_grpo_adapter). Each subfolder is the PEFT export (adapter_config.json, adapter_model.safetensors, the stub README PEFT wrote); the base model is in every adapter_config.json. The run logs and results are in 2026.MI.Sibling-Experiments (b200_*, chat_harness) and the transcripts in 2026.MI.Transcripts. Base models: Qwen2.5-7B/14B/32B-Instruct (Apache 2.0) and Llama-3.2-3B-Instruct (Llama licence; built with Llama).

Base models and licences

• Qwen/Qwen2.5-7B-Instruct: Apache 2.0 (the Qwen2.5 and Qwen3 licence permits redistribution of derivatives with attribution).
• Qwen/Qwen2.5-14B-Instruct: Apache 2.0 (the Qwen2.5 and Qwen3 licence permits redistribution of derivatives with attribution).
• Qwen/Qwen2.5-32B-Instruct: Apache 2.0 (the Qwen2.5 and Qwen3 licence permits redistribution of derivatives with attribution).
• meta-llama/Llama-3.2-3B-Instruct: the Llama community licence (derivatives carry the 'Built with Llama' attribution; the base licence applies). The adapters and merged weights here are derivatives of those base models and are redistributed under the same terms; this card adds no licence of its own.

Layout of /

covert_persona_syc_w0_s0_adapter/README.md  (0.0 MB)
covert_persona_syc_w0_s0_adapter/adapter_config.json  (0.0 MB)
covert_persona_syc_w0_s0_adapter/adapter_model.safetensors  (40.4 MB)
covert_persona_syc_w4_s0_adapter/README.md  (0.0 MB)
covert_persona_syc_w4_s0_adapter/adapter_config.json  (0.0 MB)
covert_persona_syc_w4_s0_adapter/adapter_model.safetensors  (40.4 MB)
covert_syc_minlen100_s0_adapter/README.md  (0.0 MB)
covert_syc_minlen100_s0_adapter/adapter_config.json  (0.0 MB)
covert_syc_minlen100_s0_adapter/adapter_model.safetensors  (40.4 MB)
covert_syc_minlen100_s1_adapter/README.md  (0.0 MB)
covert_syc_minlen100_s1_adapter/adapter_config.json  (0.0 MB)
covert_syc_minlen100_s1_adapter/adapter_model.safetensors  (40.4 MB)
covert_syc_minlen100_s2_adapter/README.md  (0.0 MB)
covert_syc_minlen100_s2_adapter/adapter_config.json  (0.0 MB)
covert_syc_minlen100_s2_adapter/adapter_model.safetensors  (40.4 MB)
deconf_cal_q32_gemma2b_adapter/README.md  (0.0 MB)
deconf_cal_q32_gemma2b_adapter/adapter_config.json  (0.0 MB)
deconf_cal_q32_gemma2b_adapter/adapter_model.safetensors  (134.3 MB)
grpo_covert_il_q7_w0_s0_adapter/README.md  (0.0 MB)
grpo_covert_il_q7_w0_s0_adapter/adapter_config.json  (0.0 MB)
grpo_covert_il_q7_w0_s0_adapter/adapter_model.safetensors  (40.4 MB)
grpo_covert_il_q7_w2_s0_adapter/README.md  (0.0 MB)
grpo_covert_il_q7_w2_s0_adapter/adapter_config.json  (0.0 MB)
grpo_covert_il_q7_w2_s0_adapter/adapter_model.safetensors  (40.4 MB)
grpo_covert_il_q7_w4_s0_adapter/README.md  (0.0 MB)
grpo_covert_il_q7_w4_s0_adapter/adapter_config.json  (0.0 MB)
grpo_covert_il_q7_w4_s0_adapter/adapter_model.safetensors  (40.4 MB)
judgepanel_callousness_s0_adapter/README.md  (0.0 MB)
judgepanel_callousness_s0_adapter/adapter_config.json  (0.0 MB)
judgepanel_callousness_s0_adapter/adapter_model.safetensors  (36.7 MB)
judgepanel_hallucination_s0_adapter/README.md  (0.0 MB)
judgepanel_hallucination_s0_adapter/adapter_config.json  (0.0 MB)
judgepanel_hallucination_s0_adapter/adapter_model.safetensors  (36.7 MB)
judgepanel_syc_s0_adapter/README.md  (0.0 MB)
judgepanel_syc_s0_adapter/adapter_config.json  (0.0 MB)
judgepanel_syc_s0_adapter/adapter_model.safetensors  (36.7 MB)
judgepanel_sycophancy_s1_adapter/README.md  (0.0 MB)
judgepanel_sycophancy_s1_adapter/adapter_config.json  (0.0 MB)
judgepanel_sycophancy_s1_adapter/adapter_model.safetensors  (36.7 MB)
judgepanel_sycophancy_s2_adapter/README.md  (0.0 MB)
judgepanel_sycophancy_s2_adapter/adapter_config.json  (0.0 MB)
judgepanel_sycophancy_s2_adapter/adapter_model.safetensors  (36.7 MB)
probeattack_syc_lam16_adapter/README.md  (0.0 MB)
probeattack_syc_lam16_adapter/adapter_config.json  (0.0 MB)
probeattack_syc_lam16_adapter/adapter_model.safetensors  (40.4 MB)
probeattack_syc_lam8_adapter/README.md  (0.0 MB)
probeattack_syc_lam8_adapter/adapter_config.json  (0.0 MB)
probeattack_syc_lam8_adapter/adapter_model.safetensors  (40.4 MB)
probeattack_syc_s0_adapter/README.md  (0.0 MB)
probeattack_syc_s0_adapter/adapter_config.json  (0.0 MB)
probeattack_syc_s0_adapter/adapter_model.safetensors  (40.4 MB)
probeattack_syc_s1_adapter/README.md  (0.0 MB)
probeattack_syc_s1_adapter/adapter_config.json  (0.0 MB)
probeattack_syc_s1_adapter/adapter_model.safetensors  (40.4 MB)
probeattack_syc_s2_adapter/README.md  (0.0 MB)
probeattack_syc_s2_adapter/adapter_config.json  (0.0 MB)
probeattack_syc_s2_adapter/adapter_model.safetensors  (40.4 MB)
scaled_grpo_q14_gemma9b_vllm_adapter/README.md  (0.0 MB)
scaled_grpo_q14_gemma9b_vllm_adapter/adapter_config.json  (0.0 MB)
scaled_grpo_q14_gemma9b_vllm_adapter/adapter_model.safetensors  (201.4 MB)
... 6 more entries

Layout of negotiation_rl_adapter

README.md  (0.0 MB)
adapter_config.json  (0.0 MB)
adapter_model.safetensors  (40.4 MB)

Layout of secret_word_grpo_adapter

README.md  (0.0 MB)
adapter_config.json  (0.0 MB)
adapter_model.safetensors  (40.4 MB)

Layout of secret_word_rl_adapter

README.md  (0.0 MB)
adapter_config.json  (0.0 MB)
adapter_model.safetensors  (40.4 MB)

Related repositories

• The collection: https://huggingface.co/collections/siddharthmb/2026-mats-rational-agents-6acc1dd5623e8e00176b80a8
• The workspace index: https://huggingface.co/datasets/siddharthmb/2026.RA.Workspace-Index

Provenance

Archived from /nlp/scr/siddharth/ii_mats/adapters, /nlp/scr/siddharth/ii_mats/negotiation_rl_adapter, /nlp/scr/siddharth/ii_mats/secret_word_grpo_adapter, /nlp/scr/siddharth/ii_mats/secret_word_rl_adapter on 2026-10-11 by the MATS archive job 17819017 of the transcoder-adapters cleanup (report claude_updates/memory/reports/mats_archive.md in that repository). The directories were uploaded unchanged (upload_large_folder), so every checkpoint, config and log keeps its original relative path. Every file name and every text file was scanned for credentials before upload (Hub, Anthropic, OpenAI, AWS and W&B key shapes, private-key headers, credential URLs, paths of the secrets store); the report lists what was excluded. The local copy was deleted after every uploaded file was verified on the Hub by name, size and hash, and a <name>.txt breadcrumb naming this repository was left beside it.