YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
This was an experiment to better understand specifically the KDA layers and their impact on refusals. The entire model was frozen other than the KDA output projection. It's fairly clear this has an impact but only when using far too powerful of an intervention, and the effect is only mild at best. It's a usable checkpoint, but I'm leaving this up mostly just as a curiosity.
KL Divergence: 0.19 (This is rediculous, expected is around 1e-6 to 1e-5) Refusals: 215/619
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support