Why not just patch the model?
An obvious question from a layman like myself and many others who will wonder the same thing in their mind...
Could that be an easer solution and a more permanent one because the other dependendies change often?
I admit I'm probably missing the "big" picture.
Thanks for your work!
Thank you for your comment!
The great thing about RepE is that you can reuse the activations you've extracted. It eliminates the need to constantly prompt the system with "behave like this." I can simply reuse the activations I've extracted and bake them into the model. This is more resistant to context contamination than using prompts.
Unmodification is just one example of how activation steering can change the model's behavior.
I believe that extracting activations with higher precision would allow for more advanced control.
So basically it will always have to happen at runtime?
I run everything local and between the models and the harness (Claude Code in my case) there are way too many restrictions imposed. I'm not doing anything nefarious here, but with so many gates in place and running sub-BF16 quants, the quality of the output is greatly reduced, IMHO.
Thus I'm chasing some kind of a goal where I can reliably say that the output the model gave me is not been filtered in any way. Not like anyone should ever trust what these models output in the first place 😄
RepE is like a handy fine-tuning tool. You can adjust only the parts you want to control. You can also keep those parameters fixed and continue using them. I'm wondering if it could be used in conjunction with fine-tuning the model.
I'd like to support NVFP4, but I have other work to do and don't have enough GPUs.
You mention that you could "bake" the activations into the model, how would that be done - aren't activations dynamic and the model weights are static?
Or you meant, during runtime and not permanently into the model files themselves?
3x RTX 3090 here so not worried about NVFP4 much. 😃
Though I'd love to make a Gemma4-31B with a mixed AutoRound quant (INT8+INT4) but so far it has been eluding me.