Improving Causal Interventions in Amnesic Probing with Mean Projection or LEACE (2025)

Abstract: Amnesic probing is a technique used to examine the influence of specific linguistic information on the behaviour of a model. This involves identifying and removing the relevant information and then assessing whether the model’s performance on the main task changes. If the removed information is relevant, the model’s performance should decline. The difficulty with this approach lies in removing only the target information while leaving other information unchanged. It has been shown that Iterative Nullspace Projection (INLP), a widely used removal technique, introduces random modifications to representations when eliminating target information. We demonstrate that Mean Projection (MP) and LEACE, two proposed alternatives, remove information in a more targeted manner, thereby enhancing the potential for obtaining behavioural explanations through Amnesic Probing.

## introduction a major paradigm for analyzing the linguistic information represented in language models is probing @xcite . at its core, the probing paradigm assumes that the information captured by the model can be learnt from its intermediate representations. while this paradigm enables linguistically motivated analyses, it has several limitations, as summarised by @xcite . a more recent framework of model analysis methods aims to establish causal connections between specific linguistic properties and their impact on the model @xcite . amnesic probing @xcite tests whether a model has specific linguistic information by first identifying and removing a property and then testing the impact of the removal on the behaviour of the model. @xcite remove syntactic information and then test whether the model can still perform well on next-word prediction (a task that requires knowledge about syntax). if the syntactic properties were successfully identified and removed, the performance on the task should drop. a major factor in the amnesic probing framework is the removal of the target property. @xcite use iterative nullspace projection @xcite , a method that has originally been proposed to debiase models. inlp removes information by iteratively applying projections to the embedding space. each projection removes more of the target information. a major risk of this method is removing information beyond the target information and thus damaging other aspects of the vector space @xcite . multiple alternative methods that can remove information in a more targeted way have been proposed since @xcite @xcite . @xcite prove the efficiency of these approaches. they also show that their leace approach indeed removes pos-information more effectively and distorts the overall space less than inlp hinting at leace being superior for amnesic probing. however, the impact of using alternative removal methods such as leace or mean projection (mp) has, to our knowledge, not been investigated in full amnesic probing experiments. additionally, despite the limitations of inlp, it remains a popular approach. 1 therefore, it remains important to explain why alternative methods should be favoured. this paper fills this gap and provides the following contributions. we provide an easy-to-follow step-by-step explanation of the amnesic probing framework, including additional checks and evaluations to the original experiments of @xcite . we systematically compare inlp against mp method proposed by @xcite and @xcite and against leace on three control sets from @xcite . our results demonstrate that, unlike inlp, both mp and leace improve the ability to uncover behavioural explanations via amnesic probing and successfully pass information control tests in two settings. further analyses using our extended framework reveal that mp and leace more effectively remove the target property than inlp while causing less distortion to the remaining representation space. although our original focus was on comparing inlp and mp -hence the more detailed treatment of mp -leace performs comparably to mp across all evaluations. we are also releasing a user-friendly framework. foot_2 ## amnesic probing the amnesic probing method, as proposed by @xcite , is based on the following idea. if we want to know, for example, whether a specific model uses syntactic information to predict the next word, we test the following: (1) whether the model contains syntactic information; (2) whether it performs worse at predicting the next word after the syntactic information is removed; (3) whether this drop in performance is greater than that observed in models with the same amount of random information removed; (4) whether explicitly adding syntactic information brings performance close to that of the original model. first, we provide an overview of the experimental setup and then describe the intuition behind the removal procedures used in this study. ## experiments and results we walk through the amnesic probing procedure step-by-step and present the results of our comparison of inlp, mp and leace. @xcite conducted experiments on two variants of the bert model (masked and unmasked) @xcite . both led to the same insights. we present the results for unmasked bert here, and for masked bert in appendix c. step 1: can we identify and remove target information? following @xcite , we first verify whether the target information is present in the original model through a regular probing procedure. we test whether a linear classifier can learn the target information from the vanilla model (i.e. in its original state). we then apply inlp as well as mp and leace to create alternative models where target information is removed. we add an additional test compared to @xcite and check whether probing indeed fails on the target property after applying inlp and mp. the results are presented in we add two comparisons to investigate the impact of each method on the overall model: (1) the change in matrix rank @xcite and ( 2 ) cosine similarity between layers before and after modification. both tests show that inlp has a much larger impact on the model than mp and leace. a detailed description of these metrics and the results can be found in appendix e. ## conclusion we presented a systematic comparison of three different information removal methods for amnesic probing. we hypothesized that the potential of amnesic probing as a model analysis method increases when using a more precise removal method, specifically mp or leace instead of inlp. our results confirm the hypothesis; mp and leace outperform inlp on the information and selectivity control tests. in particular, in two out of three cases, random projections had a bigger impact on the main task than the model modified by inlp, which did not happen when mp or leace was used (information control). in all three cases, mp or leace-modified models recover performance better than inlp when explicit gold information about the target property is added (selectivity control). information and selectivity control provide evidence that the drop in performance on the main task is indeed caused by the target information missing rather than by other artifacts of the modification. as such, our results show that mp and leace are more precise methods for removing information, which directly increases the potential of amnesic probing. mp and leace yield highly comparable results and we provide easy-to-use code for both approaches. we recommend to first use mp since this is more efficient and adding experiments with leace when checks and balances on the mp results call for it.