Question: What is the novel architecture proposed by authors?
Evidence: Understanding what people are talking about and how they feel about it is valuable especially for industries which need to know the customers' opinions on their products. Aspect-Based Sentiment Analysis (ABSA) is a branch of sentiment analysis which deals with extracting the opinion targets (aspects) as well as the sentiment expressed towards them. For instance, in the sentence The spaghetti was out of this world., a positive sentiment is mentioned towards the target which is spaghetti. Performing these tasks requires a deep understanding of the language. Traditional machine learning methods such as SVM BIBREF2, Naive Bayes BIBREF3, Decision Trees BIBREF4, Maximum Entropy BIBREF5 have long been practiced to acquire such knowledge. However, in recent years due to the abundance of available data and computational power, deep learning methods such as CNNs BIBREF6, BIBREF7, BIBREF8, RNNs BIBREF9, BIBREF10, BIBREF11, and the Transformer BIBREF12 have outperformed the traditional machine learning techniques in various tasks of sentiment analysis. Bidirectional Encoder Representations from Transformers (BERT) BIBREF13 is a deep and powerful language model which uses the encoder of the Transformer in a self-supervised manner to learn the language model. It has been shown to result in state-of-the-art performances on the GLUE benchmark BIBREF14 including text classification. BIBREF1 show that adding domain-specific information to this model can enhance its performance in ABSA. Using their post-trained BERT (BERT-PT), we add adversarial examples to further improve BERT's performance on Aspect Extraction (AE) and Aspect Sentiment Classification (ASC) which are two major tasks in ABSA. A brief overview of these two sub-tasks is given in Section SECREF3.
Adversarial examples are a way of fooling a neural network to behave incorrectly BIBREF15. They are created by applying small perturbations to the original inputs. In the case of images, the perturbations can be invisible to human eye, but can cause neural networks to output a completely different response from the true one. Since neural nets make mistakes on these examples, introducing them to the network during the training can improve their performance. This is called adversarial training which acts as a regularizer to help the network generalize better BIBREF0. Due to the discrete nature of text, it is not feasible to produce perturbed examples from the original inputs. As a workaround, BIBREF16 apply this technique to the word embedding space for text classification. Inspired by them and building on the work of BIBREF1, we experiment with adversarial training for ABSA.
Our contributions are twofold. First, by carrying out an ablation study on the number of training epochs and the values for dropout in the classification layer, we show that there are values that outperform the specified ones for BERT-PT. Second, we introduce the application of adversarial training in ABSA by proposing a novel architecture which combines adversarial training with the BERT language model for AE and ASC tasks. Our experiments show that the proposed model outperforms the best performance of BERT-PT in both tasks.
Second, we introduce the application of adversarial training in ABSA by proposing a novel architecture which combines adversarial training with the BERT language model for AE and ASC tasks.
Answer: It combines adversarial training with the BERT language model for AE and Aspect Sentiment Classification tasks.