Nicholai

@Kearm apologies, typing from my phone. I'll test later tonight.

zero_index = (unique_vals == 0).nonzero()
zero_count = temp_counts[zero_index]

@lmg-anon @Kearm , the code snippet should be fixed above. I was confusing what torch.unique returned.

The unique function will return the values and their counts. zero_index in the above snippets points to the counts tensor. In the counts tensor we can compare that against a previous iteration.

And thinking through this code, there's got to be a better way to check if two sets of activation tensors overlap, and pick the ones that overlap the least.

https://stackoverflow.com/a/62407582/1813580

@Kearm I was talking about @nmitchko 's code snippet

@Kearm and @lmg-anon want to try the update (i'm away from my cluster so can't test the code at the moment) I changed the original post.

Actually the code probably still doesn't work

Thanks you!
I will try ASAP when I have the opportunity, very interesting

I just fixed an obvious bug in that snippet, so feel free to respond if something isn't working as expected :)

Perhaps you're not thinking of the layer selection correctly. Pick the layers that have a different cached PC1 for your positive and negative dataset as featured in the article.

You could just cycle through them when doing the ablation:

I didn't get a chance to test this code but it should work

pos = -1
start_layer = 14
end_layer=40
final_layer = start_layer

## First computation
harmful_mean_act = harmful_cache['resid_pre', start_layer][:, pos, :].mean(dim=0)
harmless_mean_act = harmless_cache['resid_pre', start_layer][:, pos, :].mean(dim=0)
activation_difference = harmful_mean_act - harmless_mean_act
refusal_dir = activation_difference / activation_difference.norm()
counts = 0

for layer in range(start_layer+1, end_layer):
    temp_harmful_mean_act = harmful_cache['resid_pre', layer][:, pos, :].mean(dim=0)
    temp_harmless_mean_act = harmless_cache['resid_pre', layer][:, pos, :].mean(dim=0)
    temp_activation_difference = temp_harmful_mean_act - temp_harmless_mean_act
    
    unique_vals, temp_counts = torch.unique(activation_difference - temp_activation_difference, return_counts=True, dim=0)
    zero_index = (unique_vals == 0).nonzero()
    zero_count = temp_counts[zero_index]
#    if counts is None:
#        refusal_dir = temp_activation_difference - temp_activation_difference.norm()
#        activation_difference = temp_activation_difference
#        counts = temp_counts
#        final_layer = layer
    if zero_count  > counts:
        refusal_dir = temp_activation_difference / temp_activation_difference.norm()
        activation_difference = temp_activation_difference
        counts = zero_counts
        final_layer = layer

Un-censoring a model is the first pass at a broader goal of knowledge programming in these AI models; we can shortly provide simple inference level ways to model internal layers of a model into a "thought space" and be able to fine tune or adjust arbitrary information in and out. This has immense applications for industry with non-public data and limited training resources and this work is thoroughly saluted.