In this paper I explore the robustness of the Multi-Task Deep Neural Networks (MT-DNN) against non-targeted adversarial attacks across Natural Language Understanding (NLU) tasks as well as some possible ways to defend against them. Liu et al., have shown that the Multi-Task Deep Neural Network, due to the regularization effect produced when training as a result of it’s cross task data, is more robust than a vanilla BERT model trained only on one task (1.1%-1.5% absolute difference). I then show that although the MT-DNN has generalized better, making it easily transferable across domains and tasks, it can still be compromised as after only 2 attacks (1-character and 2-character) the accuracy drops by 42.05% and 32.24% for the SNLI and SciTail tasks. Finally I propose a domain adaptable defense which restores the model’s accuracy (36.75% and 25.94% respectively) as opposed to a general purpose defense or an off-the-shelf spell checker. Keywords Model Robustness Domain Adaptation Adversarial Defense 1 Introduction The pace of deep learning research in NLU tasks is extremely fast, with new papers being published consistently pushing the benchmark for various tasks in the ﬁeld. One such example is the Multi-Task Deep Neural Network (MT-DNN) [ 5], a hybrid model which utilizes a combination of language model pretraining along with multi task learning in order to improve the learning of semantic representations to augment the performance across various NLU tasks. Even with the rapid pace by which benchmarks are pushed across supervised learning tasks using deep learning, many of these models are susceptible to attacks which are almost imperceptible to humans. After it was proven that image classifying neural networks can be fooled to predict incorrect classes by adding some noise that was unidentiﬁable by humans [ 19], the focus began to shift on how NLU models can be attacked to make false predictions on sentences which are obviously different classes when observed by humans. There are four fundamental ways of creating adversarial text examples: adding, deleting, swapping adjacent characters, or replacing characters with those that are closer on the keyboard [ 4]. I explore various combinations of these character level swap attacks on the perturbed sentences so that the text is falsely classiﬁed when fed through the MT-DNN, yet easily comprehensible by humans. These alterations are founded through psycho-linguistic studies, which show that the semantic meaning of the words can be easily comprehended even if the characters are jumbled as long as the ﬁrst and last characters remain the same (Ex: Cmabridge Uinversity ). I proceed to explore various methods by which defenses can be introduced such that even if perturbations exist within multiple characters across a sentence, the accuracy can be restored. I utilize a robust word recognition model inspired from Semicharacter RNNs [ 20] which can correct a word by generating encodings for the ﬁrst character, then treating the middle characters as a bag of characters where frequency is the feature as opposed to position, and ﬁnally generating APREPRINT - 20 THDECEMBER 2019 a ﬁnal encoding for the last character. I further propose a multi-task defense where the word recognition model is pretrained off a combination of multiple tasks (similar to how the MT-DNN is trained), which I can then ﬁne-tune to the task at hand, thus ensuring a modular defense which can learn the intricacies of the domain at which it’s operating in. 2 Related Works 2.1 Pre-trained Language Models There are various pre-trained language models which employ different strategies to apply them to downstream tasks such as feature-based method (ELMo [ 8]) or a ﬁne-tuning method (XLNet [ 15], BERT [ 1], Open AI GPT [ 13] in order to achieve state-of-the-art accuracy in different NLI tasks. These models employ the large swaths of unlabelled textual data in order to train contextualized word representations, thus eliminating the need to create bulky task speciﬁc architectures. 2.2 Application of Pretrained Models on Downstream Tasks Due to the extent of the pretraining and the complexity within these models, in order to make a prediction I simply need to add a classiﬁcation layer at the end to make a prediction on different tasks (as mentioned later). However in order to achieve higher accuracies need to incorporate the data of a given domain. There are three main ways to accomplish this: ﬁnetuning the entire mode along with the classiﬁcation layer, ﬁne tuning the BERT [ 1] model using Masked Language Modeling, and Next Sentence Prediction (essentially not using the classiﬁcation layer), and Multi Task Finetuning. In the original approach we simply add a classiﬁcation layer based on the task and ﬁne tune allthe model’s parameters using the in domain data for that use case in the second approach we don’t add the classiﬁcation layer but simply retrain the BERT [ 1] model using it’s original loss functions for the Masked Language Modeling, and next Sentence Prediction Tasks using the in domain data, thus allowing it to create domain-speciﬁc contextual embeddings prior o the application of the classiﬁcation layer. The ﬁnal approach ascertains the combination of multiple tasks by conjoining the entire dataset across tasks to create a generalized model, while using task speciﬁc loss functions when applying it to a speciﬁc use-case. 2.3 Adversarial Attacks Adversarial examples can be generated using black-box or white-box techniques based off of the knowledge of the neural network. Black-box attacks are used when the general architectures, parameters, and hyper-parameters of the model aren’t accessible (such as the attack on some foreign API service). White box attacks require complete knowledge of the parameters and architecture of the model to be generated to be generated. These type of adversarial attacks with respect to Natural Language, character or word-level, have been intro- duced in a variety of manners which primarily either focus on the existence of predominant weaknesses [ 21], through white-box defensive strategies [23], through black-box strategies [22], or a combination of both [4] [24] There also exist different types of perturbations that can be performed on the text including: adding, swap- ping, deleting, or inserting characters which are positionally closer on the keyboard. These attacks are shown in [ 17] they have the ability to signiﬁcantly decrease words per minute (wpm) by up to 36% based on the location of the attack and that although 50% of survey respondents didn’t understand a few words, they were still able to answer the perturbed questions with a high accuracy. 2.4 Defenses In order to defend against gradient based attacks (white-box) or non-gradient based attacks (black-box) alike, one potential method is through the use of spelling correction. Spelling correction [ 3] is often viewed as a sub-task of grammatical error correction [ 7] [9]. Classical methods utilize a body of text which includes the number of times the word is used, then analyze different errors and ﬁnally compute the Levenshtein distance between the chosen word and the target word. Another approach was through the use of a pretrained language embedding trained to identify n-grams in order to make a prediction based off of the previous word. Lately, deep learning inspired approaches have been introduced, which analyze both the context and orthography of the input. There are also off-the-shelf spell checkers such as AHD, or Hunspell check which are trained off of a general purpose dataset and allow for ease of use due to their packaging as Python libraries. This work utilizes the SemiCharacter RNNs [20] 2 APREPRINT - 20 THDECEMBER 2019 Figure 1: This is the architecture of the model including the ScRNN and the MT-DNN. This shows how an input ﬂows through the ScRNN, gets corrected with the LSTM, and then ﬂows through the model from the lexicon encoder to the transformer encoder to generate the contextual word embedding after which a task speciﬁc output layer exists to make a prediction. Image combined elements from [1] and [20] 3 Approach 3.1 MT-DNN Model Architecture The model architecture contains a chain of shared text encoding layers which has of a lexicon encoder and a contextual transformer encoder, which are trained and shared across all tasks. The shared lexicon and contextual transformer encoder is initialized using the BERT [ 1] model as released by huggingface’s pytorch library [ 11]. These parameters are then update during the Multi Task Learning (MTL) phase of creating a pretrained MT-DNN model [ 5]. The output layers are then appended to the bottom as shown in (Figure 1), which are chosen based off the task. The individual components are further elaborated below. 3.1.1 Lexicon Encoder The lexicon encoder is the pre-processing step in order to generate the contextual embedding from the Transformer Encoder [16]. Given an input X, it is ﬁrst tokenized using WordPiece tokenization [ 12] which splits a token into pieces that model can understand (Ex: "I am playing" -> ["i", "am", "play","##ing"]), with a word that it may not know at all being split all the way down to the character level, hence ensuring that there would never be an OOV error for words created from characters of the alphabet of the language for which the model is trained. In exceptional cases where the character isn’t recognized, the token is replaced with a [UNK] token, thus ensuring that no error is produced. The ﬁrst token of the sequence is always the classiﬁcation token ([CLS]) with the ﬁnal hidden state of this token containing the representation of the entire sequence and is used for sentence level classiﬁcation tasks. In situations where there are a 3 APREPRINT - 20 THDECEMBER 2019 Figure 2: This shows how the lexicon encoder creates an input representation by summing the token, segment, and position embeddings from a given input. From [1] pair of sentences (2 sentence classiﬁcation tasks), the sentences are separated with the [SEP] token which denotes what segment it belongs to. This would result in a list of tokens like this: One sentence classiﬁcation: ["[CLS]","i","am","play","##ing"] Two sentence classiﬁcation task: ["[CLS]","i","am",play,"##ing","[SEP]","i","am","talk","##ing","[SEP]"] This is then mapped to a sequence of input embedding vectors for each token which is created by summing the corresponding word, segment, and positional embeddings (Figure 2). 3.1.2 Transformer Encoder The multi-layer bidirectional transformer-encoder [ 10] layer then maps the input representation vectors to a sequence of contextual embedding vectors. This becomes the representation which is used across all tasks. The transformer layer utilizes a self-attention mechanism which directly models contextual relationships across all tokens in a sequence. A key difference in the MT-DNN model as opposed to the vanilla BERT [ 1] model is the utilization of Multi Task Objectives (Figure 3) as well as pre-training. 3.1.3 Single-Sentence Classiﬁcation Task Given the contextual embedding [CLS] token(the aggregate semantic representation of the entire sentence), it can be used to make a prediction regarding the sentence as a whole. The output probability is based on a probability distribution of size nlabels . The probability is predicted using a softmax function over the linear output layer. P(cjx) =softmax (Wtaskx) (1) The equation represents the probability of each class given a contextual embedding input x, where W_task represents the learned weight matrix for a given task. The loss function for this task is binary cross entropy loss as the objective. X c1(X; c )log(Pr(cjX)) (2) where 1(X; c )is either 0 or 1 (binary classiﬁcation) if the predicted class label cis the correct classiﬁcation for X(single sentence), penalized by a factor of Pr(softmax equation) Input: "The movie was great and had great actors and actresses" Output: Positive Review 3.1.4 Pairwise Text Classiﬁcation In the pairwise text classiﬁcation problem, the objective is to output a label given an input of 2 separate sentences. These sentences are packed together into one input sequence which includes a premise and the hypothesis(2 separate components). The original MT-DNN [ 5] utilizes a Stochastic Answer Network [18] as opposed to just predict a 4 APREPRINT - 20 THDECEMBER 2019 label which allows it to maintain a state and iteratively reﬁnes its predictions for k-number of steps (where k is a hyperparameter) and averages the prediction at each step k to create a ﬁnal prediction which improves the robustness of the model. Pk r=softmax (Wtaskskxk) (3) The equation above is very similar to the single sentence text prediction however it maintains a state sthroughout each stepkafter which the probability distribution Pris averaged to produce the ﬁnal output. The loss function for this task is binary cross entropy loss. X c1(X; c )log(Pr(cjX)) (4) where 1(X; c )is either 0 or 1 (binary classiﬁcation) if the predicted class label cis the correct classiﬁcation for X(pair of sentences), penalized by a factor of Pr(softmax equation) Input: "A black race car starts up in front of a crowd of people." (Sentence A), "A man is driving down a lonely road." (Sentence B) Output: Contradiction 3.1.5 Text Similarity Task Given the contextual embedding [CLS] token (the aggregate semantic representation of the entire sequence), it can be used to make a prediction regarding the sequence as a whole. A similarity score can be calculated using: Sim(X1; X2) =Wtaskx (5) where (X1; X2)represents the 2 different input sequences, Wtaskrepresents the learned weight matrix for this task, and where xrepresents the contextual embedding created by the transformer encoder after being fed the input from the lexicon encoder. The function Sim(X1; X2)can be used to represent the similarity as a real value in the range of (1;1) The loss function for this task is mean squared error. (ySim(X1; X2))2(6) where y represents the ground truth similarity of the text from a range of (1;1). Input: "Thank you very much, Commissioner."(Sentence A) "Thank you very much, Mr Commissioner." (Sentence B) Output: 4.500 3.1.6 Relevance Ranking Given the contextual embedding [CLS] token (the aggregate semantic representation of the entire sequence of a pair of question and its candidate answer (Q; A ). A relevance score can be computed as: Rel(Q; A ) =g(Wtaskx) (7) Where a given Qand input representation x, we rank all of its candidate answers using the relevancy score. The loss is similar to the 2 sentence classiﬁcation task, but instead however reduces the negative log likelihood of the positive answer given questions from the training data. X (Q;A+)Pr(A+jQ) (8) where Pris: Pr(A+jQ) =exp(Rel(Q; A+))P A02Aexp(Rel(Q; A0)(9) Input: Question: Who edited Electrical World magazine? Answers: In 1888, the editor of Electrical World magazine was Thomas Commerford Martin Not all cells in a multicellular plant contain chloroplasts. Output: The ﬁrst answer is the correct answer for the given query 5 APREPRINT - 20 THDECEMBER 2019 Figure 3: Representation of how the ScRNN takes a misspelled word and uses an LSTM to make a prediction on the correct word. From: [20] 3.2 Semicharacter RNN (ScRNN) architecture The ScRNN [ 20] is inspired from ﬁndings of the Cmabrigde Uinervtisy effect (also known as typoglycemia), which states that as long as the ﬁrst and last characters of a word are the same the order of the middle characters doesn’t matter in order to know the meaning of the word. This is demonstrated in the paragraph below: Aoccdrnig to a rscheearch at Cmabrigde Uinervtisy, it deosn’t mttaer in waht oredr the ltteers in a wrod are, the olny iprmoetnt tihng is taht the frist and lsat ltteer be at the rghit pclae. The rset can be a toatl mses and you can sitll raed it wouthit porbelm. Tihs is bcuseae the huamn mnid deos not raed ervey lteter by istlef, but the wrod as a wlohe. The ScRNN is shown to signiﬁcantly increase performance in word spelling correction compared to existing spelling checkers and charCNNs [19]. For regular RNNs, the input vector is the word or character representations, but for input vector for ScRNNs consist of 3 subvectors xn="bn in en# (10) The ﬁrst and last subvector represents the one hot vector of the ﬁrst and last characters respectively, and the middle subvector contains the number of occurrences each character has regardless of their order. All three subvectors would be of the size of the alphabet. This concatenation of the subvectors forms xnwhich is used as the input to the model which would output the most probably word. The ScRNN performs better as compared to charCNNs from (Kim et al.) [ 20] and commercial spell checkers for all the noises that it was tested on (adding - 3.52% absolute improvement, deleting - 13.89% absolute improvement, and jumbling - 41.85% absolute improvement). The difference in performance is especially visualized in jumbling, as oftentimes the other models aren’t designed for severely jumbled input. The ScRNN is trained using cross entropy loss as shown below: yn=exp(Whhn)P vexp(Whhn)(11) The model learns the weight matrices Win order to predict the word based off the hidden state h, with a ﬁxed layer vocabulary of v. 6 APREPRINT - 20 THDECEMBER 2019 3.3 Training Technique The training of the MT-DNN [ 5] is split into two aspects: pretraining off of the GLUE dataset, and ﬁnetuning to the task at hand. 3.3.1 Pretraining MT-DNN: As opposed to training off one dataset at a time, all the GLUE datasets are merged and shufﬂed, after which a mini-batch is constructed where the model trains off one example at a time and then uses different loss functions based on the task for that particular sample to compute the gradient and update the model accordingly. ScRNN: Similar to the MT-DNN [ 5] , the ScRNN [ 20] in ﬁrst trained off the Penn Treebank dataset with synthetic noise added. 3.3.2 Finetuning MT-DNN: The MT-DNN model [ 5] takes data which is pertinent to the use case for which it is being implemented in, and then ﬁnetunes the pretrained model to that speciﬁc scenario. These opportunities for ﬁnetuning as opposed to simply using the pretrained model allow for a greater ﬂexibility and accuracy for the speciﬁc use case. 4 Experiment Details 4.1 Architecture 4.1.1 MT-DNN We followed Liu et. al’s implementation of the MT-DNN [ 5] . It used huggingface’s PyTorch implementation of BERT for the parameter initialization [ 11]. Then it used Adamax [ 14] as the optimizer with a learning rate of 5e-5 and a batch size of 32 by following Devlin et al. [ 1]. The maximum number of epochs was set to 5. A linear learning rate decay schedule with warm-up over 0.1 was used. We also set the dropout rate of all the task speciﬁc layers as 0.1, except 0.3 for MNLI and 0.05 for CoLA. To avoid the exploding gradient problem, we clipped the gradient norm within 1. All the texts were tokenized using wordpieces, and were chopped to spans no longer than 512 tokens. 4.1.2 ScRNN The input layer of ScRNN [ 20] consists of a vector with length of 76 (A-Z, a-z and 24 symbol characters). The hidden layer units had size 650, and total vocabulary size was set to 10k. We applied one type of noise to every word, but words with numbers (e.g. 1980s) and short words (lengths less than 4) were not subjected to jumbling, and therefore these words were excluded in evaluation. We trained the model by running 5 epochs with (mini) batch size 20. We set the backpropagation through time (BPTT) parameter to 3 which means that the ScRNN updates weights for previous two words (xn2; xn1)and the current word ( xn). 7 APREPRINT - 20 THDECEMBER 2019 Table 1: Data from 1-character and 2-character attacks across various architectures Format: 0 attacks, 1-char attack, 2-char attack Architecture SNLI SciTail BERT (Vanilla) 90.5% 59.2% 39.9% 93.5% 63.5% 43.6% MT-DNN 91.6% 54.96% 49.55% 95.0% 78.83% 62.76% MT-DNN + ScRNN 91.5% 87.4% 86.3% 94.7% 89.5% 88.7% Table 2: Sample data showing corrupted samples Tasks # of attacks Samples (Sentence 1 ||| Sentence 2) SNLI1-char A soccer gmae with multiple males playing ||| A soccer game with multiple males playing. 2-char A soccer gmae with multiple males playing ||| Some men raeplaying a sport. SciTail1-char The liver is dviided into the right lobe and left lobes ||| The gallbladder is near the right lobe of the liver. 2-char The liver is dviided into the right lobe and left lobes ||| The gallbaldder is near the right lobe of the liver. Table 3: Examples that passed the MT-DNN + ScRNN combo still failed at Datasets Faultily identiﬁed sentences (Sentence 1 ||| Sentence 2) SNLITwo women are embracing while holding to go pcakages . ||| The sisters are hugging goodbye while holding to go packages after just eating lunch. Man in a black suit, wihte shitr and black bowtie playing an instrument with the rest of his symphony surrounding him.||| A person in a suit A woman stands at a podium with a slide show behind her.||| A woman is standing at a pdoium . SciTailGsaes have neither deﬁnite volume nor shape ||| Gas has no deﬁnite volume and no deﬁnite shape A lunar elcipse occurs when the Moon passes into the Earth’s shadow ||| When earth’s shadow falls on the moon, the shadow causes a lnuar eclipse. During pohtosynthesis , sunlight changes water and carbon dioxide into glucose and oxygen. ||| Photosynthesis is described by this statement: carbon dioxide and water are turned into sugar and oxygen. 4.2 Experimental Results As shown in 1, we can see that the as we perform character level attacks on the defenseless architectures (BERT, and the MT-DNN), the accuracy signiﬁcantly reduces with a drop off of 31.3% and 30.0% across datasets after one attack for BERT, and a drop off of 36.64% and 16.67% across datasets for the MT-DNN. When using a two character attack, the accuracy drops off by 50.6% and 49.9% across datasets for the BERT model, and by 42.05% and 32.24% across datasets for the MT-DNN model. This data shows the already pre-existing resilience to attacks from the MT-DNN model due to it’s cross training, as both drop offs in accuracy from the MT-DNN are still lower than drop offs in accuracy from BERT. However when preﬁxing a Semicharacter RNN as a pre-processing step for the data, we observe a signiﬁc- ant "restoration" in accuracy. When utilizing an ScRNN along with an MT-DNN, the accuracy is restored by 32.44% and 10.67% across datasets for 1 character attacks, and by 36.75% and by 25.94% for 2 character attacks across datasets. 8 APREPRINT - 20 THDECEMBER 2019 4.2.1 Error Analysis When observing samples however that still passed through the MT-DNN + ScRNN combination, it can be seen that they are mainly samples with domain-speciﬁc words which throw off the prediction. Most of the errors occurred when domain-speciﬁc words such as "Gases" or "Photosynthesis" were misspelled. This shows that the models performance could potentially be increased through the use of a domain-speciﬁc training step for the ScRNN. This could be included a priori to deployment by either training the ScRNN in parallel to the MT-DNN, or separately (before or after). These results demonstrate that the MT-DNN is inherently more resilient to attacks than BERT due to it’s multi-task learning, but that it’s accuracy can also be restored signiﬁcantly when utilizing a domain adaptable defense. 5 Conclusion Through our training we effectively show that the MT-DNN is a very versatile architecture for NLU tasks. Although it can be severely compromised to basic adversarial attacks, through the use of a domain adaptable defense, it’s accuracy can be almost completely restored in order to ensure optimal performance. There is however room for improvement to perhaps integrate more than one system of adversarial resistance. This model can also be improved by adding a domain-speciﬁc ﬁne-tuning step to the ScRNN to ensure that words that aren’t especially common for a certain domain are still noted. 9 APREPRINT - 20 THDECEMBER 2019