User Generated Data: Achilles' heel of BERT

Abstract: Pre-trained language models such as BERT are known to perform exceedingly well on various NLP tasks and have even established new State-Of-The-Art (SOTA) benchmarks for many of these tasks. Owing to its success on various tasks and benchmark datasets, industry practitioners have started to explore BERT to build applications solving industry use cases. These use cases are known to have much more noise in the data as compared to benchmark datasets. In this work we systematically show that when the data is noisy, there is a significant degradation in the performance of BERT. Specifically, we performed experiments using BERT on popular tasks such sentiment analysis and textual similarity. For this we work with three well known datasets - IMDB movie reviews, SST-2 and STS-B to measure the performance. Further, we examine the reason behind this performance drop and identify the shortcomings in the BERT pipeline.

In recent times, pre-trained contextual language models have led to significant improvement in the performance for many NLP tasks. Among the family of these models, the most popular one is BERT BIBREF0, which is also the focus of this work. The strength of the BERT model FIGREF2 stems from its transformerBIBREF1 based encoder architectureFIGREF1. While it is still not very clear as to why BERT along with its embedding works so well for downstream tasks when it is fine tuned, there has been some work in this direction that that gives some important cluesBIBREF2, BIBREF3.
At a high level, BERT’s pipelines looks as follows: given a input sentence, BERT tokenizes it using wordPiece tokenizerBIBREF4. The tokens are then fed as input to the BERT model and it learns contextualized embeddings for each of those tokens. It does so via pre-training on two tasks - Masked Language Model (MLM)BIBREF0 and Next Sentence Prediction (NSP)BIBREF0.
The focus of this work is to understand the issues that a practitioner can run into while trying to use BERT for building NLP applications in industrial settings. It is a well known fact that NLP applications in industrial settings often have to deal with the noisy data. There are different kinds of possible noise namely non-canonical text such as spelling mistakes, typographic errors, colloquialisms, abbreviations, slang, internet jargon, emojis, embedded metadata (such as hashtags, URLs, mentions), non standard syntactic constructions and spelling variations, grammatically incorrect text, mixture of two or more languages to name a few. Such noisy data is a hallmark of user generated text content and commonly found on social media, chats, online reviews, web forums to name a few. Owing to this noise a common issue that NLP models have to deal with is Out Of Vocabulary (OOV) words. These are words that are found in test and production data but not part of training data. In this work we highlight how BERT fails to handle Out Of Vocabulary(OOV) words, given its limited vocabulary. We show that this negatively impacts the performance of BERT when working with user generated text data and evaluate the same.
This evaluation is motivated from the business use case we are solving where we are building a dialogue system to screen candidates for blue collar jobs. Our candidate user base, coming from underprivileged backgrounds, are often high school graduates. This coupled with ‘fat finger’ problem over a mobile keypad leads to a lot of typos and spelling mistakes in the responses sent to the dialogue system. Hence, for this work we focus on spelling mistakes as the noise in the data. While this work is motivated from our business use case, our findings are applicable across various use cases in industry - be it be sentiment classification on twitter data or topic detection of a web forum.
To simulate noise in the data, we begin with a clean dataset and introduce spelling errors in a fraction of words present in it. These words are chosen randomly. We will explain this process in detail later. Spelling mistakes introduced mimic the typographical errors in the text introduced by our users. We then use the BERT model for tasks using both clean and noisy datasets and compare the results. We show that the introduction of noise leads to a significant drop in performance of the BERT model for the task at hand as compared to clean dataset. We further show that as we increase the amount of noise in the data, the performance degrades sharply.
In recent years pre-trained language models ((e.g. ELMoBIBREF5, BERTBIBREF0) have made breakthroughs in several natural language tasks. These models are trained over large corpora that are not human annotated and are easily available. Chief among these models is BERTBIBREF0. The popularity of BERT stems from its ability to be fine-tuned for a variety of downstream NLP tasks such as text classification, regression, named-entity recognition, question answeringBIBREF0, machine translationBIBREF6 etc. BERT has been able to establish State-of-the-art (SOTA) results for many of these tasks. People have been able to show how one can leverage BERT to improve searchBIBREF7.
Owing to its success, researchers have started to focus on uncovering drawbacks in BERT, if any. BIBREF8 introduce TEXTFOOLER, a system to generate adversarial text. They apply it to NLP tasks of text classification and textual entailment to attack the BERT model. BIBREF9 evaluate three models - RoBERTa, XLNet, and BERT in Natural Language Inference (NLI) and Question Answering (QA) tasks for robustness. They show that while RoBERTa, XLNet and BERT are more robust than recurrent neural network models to stress tests for both NLI and QA tasks; these models are still very fragile and show many unexpected behaviors. BIBREF10 discuss length-based and sentence-based misclassification attacks for the Fake News Detection task trained using a context-aware BERT model and they show 78% and 39% attack accuracy respectively.
Our contribution in this paper is to answer that can we use large language models like BERT directly over user generated data.
For our experiments, we use pre-trained BERT implementation as given by huggingface transformer library. We use the BERTBase uncased model. We work with three datasets namely - IMDB movie reviewsBIBREF11, Stanford Sentiment Treebank (SST-2) BIBREF12 and Semantic Textual Similarity (STS-B) BIBREF13.
IMDB dataset is a popular dataset for sentiment analysis tasks, which is a binary classification problem with equal number of positive and negative examples. Both STS-B and SST-2 datasets are a part of GLUE benchmark[2] tasks . In STS-B too, we predict positive and negative sentiments. In SST-2 we predict textual semantic similarity between two sentences. It is a regression problem where the similarity score varies between 0 to 5. To evaluate the performance of BERT we use standard metrics of F1-score for imdb and STS-B, and Pearson-Spearman correlation for SST-2.
In Table TABREF5, we give the statistics for each of the datasets.
We take the original datasets and add varying degrees of noise (i.e. spelling errors to word utterances) to create datasets for our experiments. From each dataset, we create 4 additional datasets each with varying percentage levels of noise in them. For example from IMDB, we create 4 variants, each having 5%, 10%, 15% and 20% noise in them. Here, the number denotes the percentage of words in the original dataset that have spelling mistakes. Thus, we have one dataset with no noise and 4 variants datasets with increasing levels of noise. Likewise, we do the same for SST-2 and STS-B.
All the parameters of the BERTBase model remain the same for all 5 experiments on the IMDB dataset and its 4 variants. This also remains the same across other 2 datasets and their variants. For all the experiments, the learning rate is set to 4e-5, for optimization we use Adam optimizer with epsilon value 1e-8. We ran each of the experiments for 10 and 50 epochs.
Let us discuss the results from the above mentioned experiments. We show the plots of accuracy vs noise for each of the tasks. For IMDB, we fine tune the model for the sentiment analysis task. We plot F1 score vs % of error, as shown in Figure FIGREF6. Figure FIGREF6imdba shows the performance after fine tuning for 10 epochs, while Figure FIGREF6imdbb shows the performance after fine tuning for 50 epochs.
Similarly, Figure FIGREF9ssta and Figure FIGREF9sstb) shows F1 score vs % of error for Sentiment analysis on SST-2 dataset after fine tuning for 10 and 50 epochs respectively.
Figure FIGREF12stsa and FIGREF12stsb shows Pearson-Spearman correlation vs % of error for textual semantic similarity on STS-B dataset after fine tuning for 10 and 50 epochs respectively.
It is clear from the above plots that as we increase the percentage of error, for each of the three tasks, we see a significant drop in BERT’s performance. Also, from the plots it is evident that the reason for this drop in performance is introduction of noise (spelling mistakes). After all we get very good numbers, for each of the three tasks, when there is no error (0.0 % error). To understand the reason behind the drop in performance, first we need to understand how BERT processes input text data. BERT uses WordPiece tokenizer to tokenize the text. WordPiece tokenizer utterances based on the longest prefix matching algorithm to generate tokens . The tokens thus obtained are fed as input of the BERT model.
When it comes to tokenizing noisy data, we see a very interesting behaviour from WordPiece tokenizer. Owing to the spelling mistakes, these words are not directly found in BERT’s dictionary. Hence WordPiece tokenizer tokenizes noisy words into subwords. However, it ends up breaking them into subwords whose meaning can be very different from the meaning of the original word. Often, this changes the meaning of the sentence completely, therefore leading to substantial dip in the performance.
To understand this better, let us look into two examples, one each from the IMDB and STS-B datasets respectively, as shown below. Here, (a) is the sentence as it appears in the dataset ( before adding noise) while (b) is the corresponding sentence after adding noise. The mistakes are highlighted with italics. The sentences are followed by the corresponding output of the WordPiece tokenizer on these sentences: In the output ‘##’ is WordPiece tokenizer’s way of distinguishing subwords from words. ‘##’ signifies subwords as opposed to words.
Example 1 (imdb example):
“that loves its characters and communicates something rather beautiful about human nature” (0% error)
“that loves 8ts characters abd communicates something rathee beautiful about human natuee” (5% error)
Output of wordPiece tokenizer:
['that', 'loves', 'its', 'characters', 'and', 'communicate', '##s', 'something', 'rather', 'beautiful', 'about', 'human','nature'] (0% error IMDB example)
['that', 'loves', '8', '##ts', 'characters', 'abd', 'communicate','##s', 'something','rat', '##hee', 'beautiful', 'about', 'human','nat', '##ue', '##e'] (5% error IMDB example)
Example 2(STS example):
“poor ben bratt could n't find stardom if mapquest emailed himpoint-to-point driving directions.” (0% error)
“poor ben bratt could n't find stardom if mapquest emailed him point-to-point drivibg dirsctioge.” (5% error)
Output of wordPiece tokenizer:
['poor', 'ben', 'brat', '##t', 'could', 'n', "'", 't', 'find','star', '##dom', 'if', 'map', '##quest', 'email', '##ed', 'him','point', '-', 'to', '-', 'point', 'driving', 'directions', '.'] (0% error STS example)
['poor', 'ben', 'brat', '##t', 'could', 'n', "'", 't', 'find','star', '##dom', 'if', 'map', '##quest', 'email', '##ed', 'him', 'point', '-', 'to', '-', 'point', 'dr', '##iv', '##ib','##g','dir','##sc', '##ti', '##oge', '.'] (5% error STS example)
In example 1, the tokenizer splits communicates into [‘communicate’, ‘##s’] based on longest prefix matching because there is no exact match for “communicates” in BERT vocabulary. The longest prefix in this case is “communicate” and left over is “s” both of which are present in the vocabulary of BERT. We have contextual embeddings for both “communicate” and “##s”. By using these two embeddings, one can get an approximate embedding for “communicates”. However, this approach goes for a complete toss when the word is misspelled. In example 1(b) the word natuee (‘nature’ is misspelled) is split into ['nat', '##ue', '##e'] based on the longest prefix match. Combining the three embeddings one cannot approximate the embedding of nature. This is because the word nat has a very different meaning (it means ‘a person who advocates political independence for a particular country’). This misrepresentation in turn impacts the performance of downstream subcomponents of BERT bringing down the overall performance of BERT model. Hence, as we systematically introduce more errors, the quality of output of the tokenizer degrades further, resulting in the overall performance drop.
Our results and analysis shows that one cannot apply BERT blindly to solve NLP problems especially in industrial settings. If the application you are developing gets data from channels that are known to introduce noise in the text, then BERT will perform badly. Examples of such scenarios are applications working with twitter data, mobile based chat system, user comments on platforms like youtube, reddit to name a few. The reason for the introduction of noise could vary - while for twitter, reddit it's often deliberate because that is how users prefer to write, while for mobile based chat it often suffers from ‘fat finger’ typing error problem. Depending on the amount of noise in the data, BERT can perform well below expectations.
We further conducted experiments with different tokenizers other than WordPiece tokenizer. For this we used stanfordNLP WhiteSpace BIBREF14 and Character N-gram BIBREF15 tokenizers. WhiteSpace tokenizer splits text into tokens based on white space. Character N-gram tokenizer splits words that have more than n characters in them. Thus, each token has at most n characters in them. The resultant tokens from the respective tokenizer are fed to BERT as inputs. For our case, we work with n = 6.
Results of these experiments are presented in Table TABREF25. Even though wordPiece tokenizer has the issues stated earlier, it is still performing better than whitespace and character n-gram tokenizer. This is primarily because of the vocabulary overlap between STS-B dataset and BERT vocabulary.
In this work we systematically studied the effect of noise (spelling mistakes) in user generated text data on the performance of BERT. We demonstrated that as the noise increases, BERT’s performance drops drastically. We further investigated the BERT system to understand the reason for this drop in performance. We show that the problem lies with how misspelt words are tokenized to create a representation of the original word.
There are 2 ways to address the problem - either (i) preprocess the data to correct spelling mistakes or (ii) incorporate ways in BERT architecture to make it robust to noise. The problem with (i) is that in most industrial settings this becomes a separate project in itself. We leave (ii) as a future work to fix the issues.

Figure and table captions:
Figure 1: BERT architecture [1]
Figure 2: The Transformer model architecture [2]
Table 1: Number of utterances in each datasets
Figure 3: F1 score vs % of error for Sentiment analysis on IMDB dataset
Figure 4: F1 score vs % of error for Sentiment analysis on SST-2 data
Figure 5: Pearson-Spearman correlation vs % of error for textual semantic similarity on STS-B dataset
Table 2: Comparative results on STS-B dataset with different tokenizers

Questions about this paper:
What is the performance change of the textual semantic similarity task when no error and maximum errors (noise) are present?
Which sentiment analysis data set has a larger performance drop when a 10% error is introduced?

Answers:
10 Epochs: pearson-Spearman correlation  drops  60 points when error increase by 20%
50 Epochs: pearson-Spearman correlation  drops  55 points when error increase by 20%
SST-2 dataset