Local Structure Matters Most: Perturbation Study in NLU (2022)

Abstract: Recent research analyzing the sensitivity of natural language understanding models to word-order perturbations has shown that neural models are surprisingly insensitive to the order of words. In this paper, we investigate this phenomenon by developing order-altering perturbations on the order of words, subwords, and characters to analyze their effect on neural models’ performance on language understanding tasks. We experiment with measuring the impact of perturbations to the local neighborhood of characters and global position of characters in the perturbed texts and observe that perturbation functions found in prior literature only affect the global ordering while the local ordering remains relatively unperturbed. We empirically show that neural models, invariant of their inductive biases, pretraining scheme, or the choice of tokenization, mostly rely on the local structure of text to build understanding and make limited use of the global structure.

## introduction recent research has shown that neural language models have an understanding of well-formed english syntax in recurrent neural networks, convolutional neural networks, and in large pretrained (pt) transformers @xcite @xcite @xcite @xcite . other studies, however, take a critical stance with experiments suggesting that models may be insensitive to word-order perturbations @xcite @xcite @xcite , showing that shuffled word-order has little to no impact during training or inference with neural language models. while some research show that models learn some abstract notion of syntax, further probing into their insensitivity to the perturbation of syntax is necessary. specifically, what are the underlying mechanisms causing those unintuitive, or unnatural, results from neural models is still a largely unanswered question. recent research exploring the sensitivity to syntax of pretrained models has primarily been applying perturbations to text through perturbing the order of words @xcite @xcite @xcite . perturbations applied and quantified at this granularity of text offer only a limited understanding of the learning dynamics of the neural language models. analyzing perturbations at a finer granularity such as subwords @xcite or characters @xcite , may provide a deeper insight into the insensitivity to word-order of neural models. in this paper, we define two types of structure foot_0 in text, global which relates to the absolute position of characters, and local, which relates to the relative position of characters to their immediate neighbors. we observe from our experiments ( § 5) that most perturbations proposed and analyzed in the literature will perturb the global structure with different reordering of words, while the amount of disturbance to the local structure remains limited. we hypothesize that the local structure, more so than the global structure, is necessary for understanding in natural language tasks. by applying perturbations of varying degrees to the local structure, while controlling for the amount of global perturbations, we are able to measure how essential it is to a neural model understanding of text. we demonstrate the sensitivity to local structure of model performances in english natural language understanding (nlu) (glue @xcite ) and their relative insensitivity to the global structure, and control for many potential confounding factors that would otherwise provide an alternative explanation to our results. our contributions are as follows: text perturbations several different types of reordering perturbation functions and schemes have been explored to understand and study neural architectures' (in)sensitivity to word-order. the class of perturbation analysis could broadly be split into three categories: deletion, paraphrase injection, and reordering of tokens. @xcite explore utterance and word-level perturbations applied to generative dialogue models to highlight their insensitivity to the order of conversational history. on natural language classification tasks, @xcite define n-grams for different values of n and shuffle them to highlight the insensitivity of pt models. they show that shuffling larger n-grams has a lesser effect than shuffling smaller n-grams, suggesting that preserving more local structure causes less performance degradation. studying textual entailment tasks, @xcite perform perturbations on the position of the words, with the criteria that no word remains in its initial position. @xcite propose a suite of adversarial attacks that replace one word in the input to cause a model to flip its correct prediction. @xcite combine several types of destructive transformations -such as sorting, reversing, shuffling words -towards removing all informative signals in a text. along similar lines, @xcite inject noise by reordering or deleting articles towards injecting artificial noise to measure the robustness of pt language models. characterlevel perturbations that perform minimal flips to cause a degenerate response have been explored by @xcite ; @xcite . @xcite quantify the perturbation in levenshtein distance and draw a correlation to the model's perfor-mance. this work is closely related to our own. we demonstrate that our hypothesis, the importance of local ordering, is a much more robust explanation of the degradation in performance of models than the levenshtein distance. quantifying perturbations several popular similarity metrics can be used to measure perturbations. metrics like bleu @xcite and rouge @xcite will treat text as a sequence of words, from which a measure of overlap is computed. the levenshtein distance @xcite , or the edit distance, measures the minimum amount of singlecharacter edits (insertions, deletions, or substitutions) necessary to match two strings together. in the context a shuffling text, it will roughly count the amount of characters that have been displaced. @xcite observed that learned metrics like bert-score @xcite and bleurt @xcite are often unaffected by minor perturbations in text which limits their usefulness in measuring perturbations. characterlevel metrics, such as the character n-gram f-score (chrf) @xcite offer a character-aware approach to measuring similarity of n-gram overlap between two texts. in the context of shuffling this, this will represent roughly the amount of character n-gram that have been changed by the shuffling. ## measuring local and global pertubations to properly analyze different perturbations to the local and global structure of text, we first require a way to measure perturbations to said structures. the global structure here relates to the absolute position of characters in a text, and the local structure relates to the neighboring character of any other character in a text. ## perturbation functions towards conducting a detailed analysis on the effect of perturbations on the performance of neural language models, we define three granularities of perturbation functions -word-level, subwordlevel and character-level. the subwords are taken from the roberta-base vocabulary. we define the perturbation functions as generic operations that can be applied across the different levels of granularity foot_3 . full shuffle randomly shuffles the position of every word, sub-word, or character, according to the level it is applied to. this transformation should cause a great amount of perturbation to the global and local structure for the specific granularity. the scholar is typesetting. scholar typesetting is the. phrase shuffle creates chunks of contiguous tokens of variable length, controlled by a parameter ρ, and shuffles the phrases of word, subword, or characters. this perturbation has, on average, the same impact as the full shuffling on the global structure as the absolute positions of characters tend to change just as much as full shuffling while preserving a controllable amount of local structure. the scholar is typesetting. is typethe schosetting lar. to randomly define our phrases, we traverse the text sequentially on the desired granularity. the entire text is assumed as a single large phrase and is truncated at a token with probability ρ into smaller phrases. a lower value of ρ leads to longer on average phrases, thus preserving more of the local structure while destroying roughly the same amount of global structure. in the extreme case with ρ = 1.0, phrase shuffling will be equivalent to full shuffling as phrases will all be one token long. neighbor flip perturbations flip tokens of the chosen granularity with the immediate right neighbor with probability, ρ. this function has, on average, a smaller impact on the global structure, as the absolute positions of tokens do not change much but can have an arbitrary large effect on disturbing the local structure. the scholar is typesetting. figure 3 : character-level neighbor flip. the perturbed sentence has an idc of 0.04 and a chrf-2 of 0.32. due to a greater distortion to the local order, the model has a greater chance to be sensitive to this perturbation. the perturbation is applied by traversing the string from left-to-right on the desired granularity and, with a probability ρ, switching the current attended token with the following token. the lower the ρ is, the less perturbation happens, thus preserving more of the local structure. this transformation has a limited impact on the global metric, thus letting us isolate the impact of perturbations to the different structures. ## analysis 6.1 metrics and glue performance we compute the average glue score of different models applied to the validation data perturbed with our different perturbation functions. the pt roberta-base results are plotted in figure 4 7 . first, we observe that word and subword-level perturbations are very limited in their impact on the local structure, but can affect the whole spectrum of global structure. we observe the general trend that the chrf-2 metric strongly correlates with neural models' loss in performance on the glue benchmark tasks across all perturbations and granularity of perturbations. while the idc metric correlates somewhat with performance, it fails to distinguish between neighbor flipping perturbations and phrase shuffle perturbations. the compression rate is strongly correlated with performance on character-level perturbations but does not hold explanatory power for word and subword-level perturbations, as they do not affect the vocabulary, leading to the overall lower rank correlation with performance degradation. by computing the rank correlation between the glue score of the different models on the perturbed samples and the metric measuring the perturbations (figure 5 ), we see that the correlation of glue score with the chrf-2 metric holds for every single architecture and setting tested. on the other hand, the idc metric is only weakly correlated with performance decay. this implies that local structure, more so than global structure, is necessary for models to perform nlu. a model being evaluated on a perturbed text with a chrf-2 of 0.7 can be assumed to have much lower performance than on a perturbed text with a chrf-2 of 0.95, irrespective of the granularity or the type of perturbations that yielded those metrics. this is not true of any of the other metrics. looking at the individual tasks more closely, as in figure 6 , we see that the conclusions regarding the overall glue benchmark do hold for every task individually. ## discussion significance of results while our results at the extremes may be trivial, such that completely shuffling the order of characters of a text removes all the structure necessary for understanding, and that destroying the local structure to an extreme also prohibits models from building a useful representation of the text, it is not trivial that performance correlates to this degree to local structure across the whole spectrum of perturbations. in figure 4 , fully shuffling the subwords of a text and randomly flipping characters with their neighboring character 10% of the time obtains roughly the same glue score and chrf-2 metric despite much different perturbations being applied and much different idc and compression rate. the removal of any amount of local structure correlating directly to an equivalent drop in performance, with little concern for the granularity or mechanics of that removal of local structure, allows us to make interesting conclusions on the kind of structure that is used by neural models to build understanding. adversarial attacks by better understanding the specific mechanics that can induce failure in neural language models, it is possible to develop models that are more resistant to adversarial attacks. as current models performances can be directly related to the preservation of character 2-grams in all studied variations, this study demonstrates a very likely vector of adversarial attacks that may be important to explore further. @xcite use the levensthein distance to measure and limit perturbations of black box adversarial attacks, similar research relying on chrf-2 instead may be interesting. tokenization our results on the importance of local structure could bear some implications for tokenization. recent research trends @xcite look at alternatives and improvements to bpe. the current research appears to be pushing towards smaller vocabulary at finer granularity, even exploring simple byte-level representations @xcite . we find that local clumps of characters contain the most essential structural information required to solve several nlu problems. as a large part of the complexity of nlu seems to be contained within the meaning of the specific order of clumps of characters, by having more of that local structure fixed through tokenization, it is possible to inject additional useful inductive biases into the model. the perturbation analysis discussed in our work could be used for better construction of vocabulary with improved heuristics. ## conclusion our results on the relative importance of local structure in relation to global structure hint at the possibility that much of the tested nlu tasks can be solved with a bag-of-words formulation. intuitively, local structure mainly relates to building meaningful words from the characters of a text whereas the global structure relates to the general order and word-level syntax being maintained. from our experiments, we observe that as long as the local structure is roughly maintained, a majority of nlu tasks can be solved without requiring the global structure. this correlates with similar findings by @xcite . in essence, the structure required to build words seems to be necessary, but much of nlu can be solved with the information of which words (or subwords) are present in the text, without regard to their relative positions. in this work, we have provided empirical results demonstrating that, for deep learning models in english nlu, perturbations to the local structure, as measured by the chrf-2 metric, is highly correlated to downstream model performance which implies that much of the information obtained from the structure of text comes from the local structure. perturbations to the global structure, as measured by idc, seems to only have a limited correlation to performance, implying that models don't generally rely on it to build understanding. reflecting on our results, we observe that perturbations on a local level explains the (in)sensitivity of neural language models to perturbations at different granularities on a variety of nlu tasks. this paper hopefully provides useful intuitions on the importance of different types of structures in text for researchers looking into tokenization, neural architectures and adversarial attacks. although the paper primarily focuses on the effects of perturbations on english texts, extending the study to neural models on other languages will be beneficial. function neighborflip(ρ ← 0.5,text←list): perturbed_tokens ← list(); held_token ← list(text[0]) for token in text[1 :] do p ∼ uni f ([0, 1]); if @xmath0.append(held_token); held_token ← list(token) else perturbed_tokens ← [perturbed_tokens, token]; end end perturbed_tokens.append(held_token); perturbed_text ← ''.join(perturbed_tokens) return perturbed_text algorithm 2: pseudocode for neighborflip.