SSN _ MLRG 1 at S em E val-2022 Task 10: Structured Sentiment Analysis using 2-layer B i LSTM (2022)

Abstract: Task 10 in SemEval 2022 is a composite task which entails analysis of opinion tuples, and recognition and demarcation of their nature. In this paper, we will elaborate on how such a methodology is implemented, how it is undertaken for a Structured Sentiment Analysis, and the results obtained thereof. To achieve this objective, we have adopted a bi-layered BiLSTM approach. In our research, a variation on the norm has been effected towards enhancement of accuracy, by basing the categorization meted out to an individual member as a by-product of its adjacent members, using specialized algorithms to ensure the veracity of the output, which has been modelled to be the holistically most accurate label for the entire sequence. Such a strategy is superior in terms of its parsing accuracy and requires less time. This manner of action has yielded an SF1 of 0.33 in the highest-performing configuration.

## introduction sentiment analysis is a specialized area under natural language processing which deals with the extraction of opinions and emotions from text which may include reviews, social media posts, forums or news. sentiment analysis has become a powerful tool for detecting the public opinion on any topic. this research article evinces how the sentiments and opinions expressed by people in various environments could be understood by means of a structured sentiment analysis (ssa) approach. there are conventional means to this end, in transition-based and graph-based techniques, but a potent alternative is the novel way to envision dependency parsing tasks as algorithmic pattern-recognition by way of sequence labelling. sequence labeling is essentially an agglomeration of multiple discrete categorizations, extended on to each element of the entire sequence. the model is trained on 7 seven data sets with 5 different languages english, spanish, basque, norwegian and catalan. in subtask-a we try to separately identify the holder and target in the text, along with the sentiment expression used using a sequence labelling approach. in subtask-b we predict whether the extracted tokens have a relation or the lack of one. subtask-c involves the concatenation of the results of subtask-a and subtask-b to provide the final predictions. ## system overview here we provide a model that first learns to extract the sub-elements (holders, targets, expressions) using sequence labelers, and then tries to classify whether or not they have a relationship. specifically, we first train three separate bilstm models to extract holders, targets, and expressions, respectively. we then train a relation prediction model, which uses a bil-stm + max pooling to create contextualized representations of 1) the full text, 2) the first element (either a holder or target) and 3) the sentiment expression. these three depictions are concatenated and sent to a linear layer, followed by a sigmoid function. the training consists of predicting whether two elements have a relationship or not, converting the problem in binary classification. ## experimental setup the sequence labelling model employed in our research essentially attempts to divaricate and demarcate various elements of the input text into tuples. to begin with, the starting file get_baseline.sh is executed in order to call the subsequent files in convert-to-bio.py, and convert-to-rels.py. the former is responsible for converting the given statements into the workable format, as in the stratification of data into holder, target and expression parts, with "b-i-o" labels. the labels are also carriers of the polarity (positive, negative, or neutral) of each text. next up, the convert-torels.py file is called, upon which, it extracts the target/expression pairs and holder/expression pairs, and creates 2 new fields e1, e2. furthermore, the bilstms are trained to extract holder, target and expression from the data. pretrained glove or fasttext embeddings are also provided to the model for the operation of labeling. after passing the input through the aforementioned processing stages, the vocabulary obtained in the end is stored in a vocabulary dictionary. 3 bilstms are trained separately for each labeling task, following which, is the relation prediction model trained. the full sentence, in addition to e1 and e2, are all scanned for relations. if relations are present, then predictions are made accordingly. finally to get a consolidated prediction.json terminal output file, the inference.py file is called. the holders, targets, and expressions have been extracted using the trained bilstms already and the polarity is available from the expression labelling. the data is formatted accordingly and lastly packed into a neat json format. the following model incorporates a learning rate of 0.01. we have run it for upto 10 epochs, to arrive at a considerable amount of accuracy. the number of hidden layers in this model is kept as 1 by default. ## results the efficacy of the sentiment analysis model towards encapsulating the full sentiment graph can be depicted in terms of the criteria as enumerated by two benchmarks, in sentiment graph f1 (sf1) and non-polar sentiment graph f1 (nsf1). the sentiment graphs are considered in terms of holder, target, expression, and polarity, for evaluation by sf1, whereas nsf1 takes into account all elements of the tuple, except polarity for scrutinization. in this case, a perfect match on the graph, with respect to the mean of all three spans, and including weights pertaining to the gold and expected spans for each member is considered as a true positive. the precision value is a ratio where the numerator is the sum of predicted tokens that are found to be right, with the denominator being the total sum of all predicted tokens (amount of gold tokens is the denominator in case of recall). empty targets and tokens are also taken into consideration. ## conclusion in summation, our research work presents the three-tier model which we constructed, and is found to have the best accuracy of 0.33. the model has been proven to consume significantly less time as opposed to graph parsing. an inconsistency however has been documented, with regard to the processing of complex sentences carrying multiple expressions, in how it considers only the polarity attributed to the terminal element. going forward, the component bilstm could be supplanted with bilstm crf and run, owing to the nature of the latter to be robust and independent of word embedding, and capability to provide superior accuracy levels on parts of speech tagging, name entity recognition of data sets and chunking.