Is Probing All You Need? Indicator Tasks as an Alternative to Probing Embedding Spaces (2023)

Abstract: The ability to identify and control different kinds of linguistic information encoded in vector representations of words has many use cases, especially for explainability and bias removal. This is usually done via a set of simple classification tasks, termed probes , to evaluate the information encoded in the embedding space. However, the involvement of a trainable classifier leads to entanglement between the probe’s results and the classifier’s nature. As a result, contemporary works on probing include tasks that do not involve training of auxiliary models. In this work we introduce the term indicator tasks for non-trainable tasks which are used to query embedding spaces for the existence of certain properties, and claim that this kind of tasks may point to a direction opposite to probes, and that this contradiction complicates the decision on whether a property exists in an embedding space. We demonstrate our claims with two test cases, one dealing with gender debiasing and another with the erasure of morphological information from embedding spaces. We show that the application of a suitable indicator provides a more accurate picture of the information captured and removed compared to probes. We thus conclude that indicator tasks should be implemented and taken into consideration when eliciting information from embedded representations.

## introduction pre-trained vector representations of words have introduced a stark improvement in the performances of any nlp task they have been applied to, and proved to be effective in classification and prediction of real world tasks @xcite @xcite . this has led researchers to grow interest in analyzing the encoded linguistic features in the representations. to this end, probing tasks, initially proposed by @xcite and @xcite , are frequently used. probing tasks (henceforth probes) are designed to evaluate the information existing in representations by training a simple classification model @xcite . the existence of a probing model that can classify the queried linguistic property is assumed to imply that the respective linguistic information exists in the representations @xcite . however, probes have several, frequently contemplated, drawbacks. generally, probes are meant to 'un-black-box' word representations, that is, to act as lenses into what information exists in the representations. yet the actual relation between the probe's results and the representations thereof remains unclear. this has led some researchers to claim that the probing classifiers should be kept simple @xcite , others are in favor of complex ones @xcite , and a few advocate for dispensing with probing classifiers altogether @xcite . a solution frequent in the literature @xcite @xcite is to accompany the results of probes that involve training an auxiliary classification model, with results over substantially different kind of tasks that are designed to query linguistic information directly from the representations, and do not require training another model. in this work we propose to collectively refer to such tasks by the name indicator tasks, as they provide direct indication of the stored information. we claim that indicator tasks are generally superior to trainable probing models, and provide more faithful information regarding the complex structure of the embedding spaces. we substantiate our claim with two test cases that involves concept erasure: gender debiasing @xcite and morphological property removal. we show that in both cases the evaluation of erasures with indicators provides a fuller picture of the information erased and retained. our two test cases are dissimilar in many ways. gender debaising involves a single, binary, unwanted, social bias that models learn implicitly from the distribution of words in the corpus. morphological features, on the other hand, are a series of multivalued grammatical properties, expressed explicitly in the text, and are not undesired per se, yet their erasure may benefit processing tasks in morphologically rich languages. we apply two different concept erasure methods to them, respectively: rlace @xcite for gender debiasing, and inlp @xcite expanded to multiple features for morphological features removal. we evaluate the success of these methods with standard linear probes, as well as with suitable indicator tasks: word embedding association test @xcite and knn-bias correlation @xcite for gender debiasing, and for morphological information removal we present a novel double edged outlier detection (deod) that indicates on both morphological and lexical semantic information. all representations are obtained from contextualized bert models @xcite . the results in both test cases, despite their dissimilarities, point to the same direction. they show that the conclusions pointed to by the indicators are contradictory to those obtained from the probes. when removing gender bias from the representations we found probes to report complete erasure of the bias while our indicators still manage to pick it up from the vectors. in the second test case, we compared two possible extensions of inlp and found that probes point to one extension possibility as superior, while the indicator points to the other, not only by indicating the removal of the morphological properties, but also by examining the overlapping dimension of semantics. we therefore conclude that when eliciting linguistic information from embedded spaces, results from indicator tasks should be presented alongside standard probes, and results from probes should not be overstated. ## background: the probing conundrum the question of evaluating linguistic properties in dense word representations has been a long standing one, entering alongside the introduction of static word embeddings @xcite . nowadays, probing tasks are generally accepted as the primary method for querying linguistic information in dense vectors. a probe is a classifier designed to elicit a property of interest directly from the representation @xcite . their simplicity and speed are appealing as opposed to the alternative of comparing performance over downstream tasks. however, researchers are yet to reach a consensus regarding several problems with probes @xcite . many issues with probes are derived from their usage of linear or neural models. most prominently are the two interleaved problems of classifier and task selection. it is well received that a probing classifier should be simple @xcite , since as @xcite stressed, with enough training data, a sufficiently expressive probe could solve any task regardless of the representations. on the other hand, @xcite have shown that simple tasks are not informative enough when applied to contextual representations. thus a probing task should not be simplistic. however, researchers found that complex tasks often require a complex classifier @xcite , which are not recommended, as mentioned above. we are thus led to a three-way trade off between the complexity of the task, the complexity of the classifier, and the efficacy of the probe. given that simple models do not fare well on complex tasks, we cannot prefer a simple probing model and a complex task simultaneously. however, opting for either a more complex model or a simpler task would limit the reliability and quality of information given by the probe on the existence of a property in the embedded space. previous works suggested various solutions to mend these problems. @xcite suggested to examine probes on control tasks, in which labels are distributed randomly, in addition to testing the probes on the desired task. probes are then considered successful only if they perform well on the target task while failing on the control task. alternatively, @xcite suggested amnesic probing, in which a probe is considered successful only if it is possible to erase the information from the representations and make the same probe fail. the problem with these methods is that they take a process with complex problems and only complicate the process further, making it even harder to determine what in the results of a given probe is ascribable to representations, to the probe, or to the suggested new mechanism. another line of works suggested amending the output of the probing process itself by adding description length @xcite , mutual information @xcite or pareto hypervolume @xcite to provide an indication on the ease of extraction of properties rather than on their existence. these methods, while useful, embrace the aforementioned trade off and relinquish the attempt of solving it. we, on the other hand, advocate for a completely different approach that bypasses the entire debate. in general, the probing conundrum seems to stem from the presence of a train set which makes it unclear whether the probe employs artifacts in the training data to solve the task or evaluates the encoded information itself. therefore an alternative that does not require a training set and a trainable model is warranted. ## introducing: indicators as a result of these various issues with probing, previous works had suggested to elicit linguistic properties from embedding spaces without the involvement of a trainable auxiliary model. these studies mostly relied on the similarity between vectors, either between the representations themselves @xcite or between attention weights @xcite . in addition, clustering was also taken as a possible task derived from vectors alone @xcite . here we propose to refer to the different kinds of such tasks with the umbrella term indicator tasks, as they indicate the presence or absence of a property in the representation without any intervention or entanglement of the task itself with the vectors it is supposed to study. by virtue of not involving a trainable classifier, indicators cancel out many of the arguments against probes. the tasks can be complex but still fast to devise and cheap to implement and run, without a risk of over-fitting the training data -as there is none. most importantly, there is no other model that interjects, ascribing the performance directly to the embedded representations, where the task is essentially performed in a zero-shot setting. all it takes is to design or select the indicator that suits your needs. in a sense, indicators can be perceived as a resurgence of intrinsic evaluation methods that were popular with static representations. these methods were typically formulated as word similarity tasks, either explicitly @xcite or implicitly @xcite . thus, in order to formulate the indicators in our case studies we draw inspiration from such tasks. the advantages of indicators have led to their use in various works, usually to complement the picture painted by standard probes. for example, @xcite used "directprobe" @xcite as an indicator in addition to a 2-layered non-linear classifier to reinforce and explain their conclusion that fine-tuning bert does not always improve performance. similarly, merchant et al. ( 2020 ) used an mlp-based probe in addition to an indicator named "structural probe" @xcite to show that after fine-tuning, word representations keep the linguistic structure they learn during pre-training. they completed the picture by showing with the indicator task of representational similarity analysis @xcite that fine-tuning affects the top layers more than the bottom layers. lastly, the work presenting the concept erasure method @xcite , used in our first test case below, demonstrated its effectiveness by utilizing both probes and indicators. here we demonstrate that despite having a shared goal, i.e., querying information from dense representations, probes and indicators may result in contradicting conclusions. the implication of such disharmonious results is that if we want a fuller picture of the information existing in the representations, probing results must be accompanied with those obtained by indicators. as will be demonstrated in the test cases below, the specific modus operandi of trainable probes makes them less likely to detect signals that exist in the embedding space but deviate from the pattern they were made to spot. we will demonstrate that by pitting linear probes, that were selected due to their simplicity in accordance with the issues laid out in the previous section, against simple similarlybased indicators. we will show that this difference is significant enough to make indicator tasks detect gender bias and morphological information that are undetectable by a probe. ## concept erasure concept erasure methods are aimed at removing information from word representations. formally, their objective is, given a set of word representations @xmath0, ...x @xmath1, ...p n }, to learn a function r(•) such that r(x) = {r(x 1 ), ...r(x n )} are not predicative of p , but preserve as much information as possible from x @xcite . however, there is no consensus on how to evaluate the success of the erasure procedure. the evaluation methods presented in previous works usually rely specifically on the nature of the erasure method and the nature of the target property. in addition, timely field tendencies or "trends" are also influential, with earlier works devising intrinsic evaluation methods @xcite @xcite and more recent works utilizing probes @xcite . our case studies, presented in detail in the following sections, systematically compare both evaluation methodologies, with an added effort of adjusting the former to contextualized embeddings. each of these test cases involves a different erasure method, a different property to be erased, and, to evaluate the success of the erasure method, different indicator(s) in addition to probes. common to both cases is the contradicting conclusions drawn from evaluating the erasure method with probes compared to its evaluation with indicators. ## case study 2: morphological properties in our second test case, we aim for representations that are ignorant of morphological features, such as tense, person etc. using such representations can make the treatment of morphologically rich languages with many inflections per lemma simplified to a great extent, with models for downstream tasks not needing to worry about unseen inflections @xcite . as success criteria for removing morphological knowledge from word representations, we utilize probes and a novel indicator. to this end, we extend an existing concept erasure method, iterative nullspace projection (inlp, @xcite , to accommodate the erasure of multiple properties. we propose two possible, and novel, extensions and test these alternatives for their efficacy using standard probes, and using a novel indicator task that we design, which indicates on the presence of morphological and lexical semantic information simultaneously. ## conclusions in this work we compare probes that do and do not require training an auxiliary classification model, offering indicator tasks as a term that covers the latter kind. we show, in two different test cases, that although it is tempting to think of probes and indicators as complementary in understanding the inner structure of embedding spaces, sometimes the methods result in contradictory conclusions. we claim that the inherent problems with probes and the flexibility of indicators should lead to prioritizing the latter in interpreting the results of property removal procedures. nevertheless, probes are of course still useful in inspecting embedding spaces, but their results should not be overstated and they should be used only when interested in the specific insights they hold, e.g., the existence of a linear separation in case of linear probes. although the design of indicators requires more creativity and result in tasks that are mostly tailored for specific kinds of questions under investigation, we assert that the advantage they give in better understanding the information, that does and does not exist in word representations, is worth the effort. we therefore conclude that indicators should at the very least be presented alongside probes when examining embedding spaces.