Enhancing Aspect Extraction for H indi (2021)

Abstract: Aspect extraction is not a well-explored topic in Hindi, with only one corpus having been developed for the task. In this paper, we discuss the merits of the existing corpus in terms of quality, size, sparsity, and performance in aspect extraction tasks using established models. To provide a better baseline corpus for aspect extraction, we translate the SemEval 2014 aspect-based sentiment analysis dataset and annotate the aspects in that data. We provide rigorous guidelines and a replicable methodology for this task. We quantitatively evaluate the translations and annotations using inter-annotator agreement scores. We also evaluate our dataset using state-of-the-art neural aspect extraction models in both monolingual and multilingual settings and show that the models perform far better on our corpus than on the existing Hindi dataset. With this, we establish our corpus as the gold-standard aspect extraction dataset in Hindi.

## introduction recent literature has seen an increase in the amount of work being done in fine-graining downstream nlp tasks. one common method of fine-grained analysis is the use of aspect information. an aspect term is an entity of interest which identifies a unique aspect of a predefined topic or domain @xcite . for example, in the restaurant domain, service and seasoning are aspects. while aspect extraction (ae) has been often seen as a subtask of fine grained aspect-based sentiment analysis (absa), recent advances in literature have established it as an independent task which can be used in other downstream tasks as well, such as summarization @xcite and topic-specific information retrieval such as opinion mining @xcite . aspect extraction (as a subtask of aspect-based sentiment analysis) datasets and models have been developed for multiple languages. absa has been a shared task in semeval 2014 @xcite , 2015 @xcite , 2016 @xcite , and as a part of the overall task of sentiment analysis on twitter in semeval 2017 @xcite . these tasks have garnered a lot of attention in various languages including arabic, chinese, dutch, french, russian, spanish and turkish. each monolingual dataset consisted of one or two domains with each language having anywhere between 4,000 to 9,000 sentences overall (including the train and test split). for indian languages, there has been some work in developing a dataset for aspect extraction in hindi @xcite and @xcite . limited work has been done on improving the state of ae and absa in hindi beyond the development of a singular dataset, namely @xcite . existing evaluations show that existing sequence tagging models (both general and specific to ae) have performed very poorly on this dataset when their performance is compared to english ae as well as in similar sequence tagging tasks in hindi such as named entity recognition (ner) and event detection. in this paper, we thoroughly analyze the existing dataset for ae in hindi and explain the reason for the poor model performance. we then propose the creation of a parallel corpus, by manually translating the semeval-2014 absa corpus @xcite . we provide detailed guidelines and challenges faced during the creation of this resource. we show that our dataset performs much better than the existing dataset for hindi using baseline as well as state-of-the-art neural models for ae. finally, we leverage the semeval-2014 corpus to perform zero-shot and fine-tuned aspect extraction in hindi using multilingual bert with baseline and sota neural models in the dataset we have created. therefore, the main contributions of this paper are: we establish that our corpus is a more robust and representative corpus for aspect extraction in hindi, and its parallel nature can be exploited for a large number of downstream tasks including review translation, cross-lingual opinion mining, and aspect-based sentiment analysis. ## dataset development as discussed in section 1, @xcite is the only corpus for aspect term extraction and aspect-based sentiment analysis in hindi. in this section, we discuss the inadequacy of this corpus by analyzing the data qualitatively, statistically and through experiments using established aspect extraction models. we also detail the process of creating a parallel aspect extraction datset from the english gold standard dataset @xcite . this resource can be treated as an individual hindi aspect extraction dataset or can be considered a parallel resource for the aspect extraction task. the annotation format used for the existing dataset and the dataset being created is the begin-inside-outside or bio sequence labeling format @xcite . this format annotates each word with a corresponding label where a word labeled b denotes the first word of an aspect, i denotes any word within the aspect span and o denotes the words outside the aspect span. ## evaluating the dataset in this section, we detail the evaluation of our translated aspect extraction dataset. we evaluate our dataset using multiple monolingual and multilingual models. the monolingual models are trained and tested on the individual language datasets while the multilingual models involve the use of transfer learning from the semeval-2014 dataset to the dataset we have created. ## conclusion in this paper, we detailed the state of aspect extraction in hindi by thoroughly analyzing and evaluating the currently available baseline dataset for this task. by understanding the flaws in that dataset, we explain its inadequacy in terms of lack of uniformity, high domain sparsity and incorrect aspect annotations. we further compare its performance with existing models to show that it performs very poorly as compared to the existing english dataset. we then explain the mechanism of creating a se-meval style corpus for aspect extraction in hindi, by translating the english semeval 2014 aspect based sentiment analysis corpus. we provide a detailed list of guidelines in order to make this task as replicable as possible. we also focus on maintaining the naturalness and fluency of the translations using transliteration wherever necessary. our translation and annotation methodology is evaluated on the rouge-l and fleiss' kappa metrics respectively. we use this dataset to show performance on basline statistical and neural sequence labeling models, as well as the current state-of-the-art models in neural aspect extraction such as decnn and seq2seq4ate. we show that while the published hindi dataset does not perform nearly as well, we provide comparable results to those models. since we have a parallel corpus, we also leverage the english data for improving aspect extraction in hindi using multilingual bert. future work in this direction includes developing an aspect based sentiment analysis corpus which can be trained and tested in a multilingual manner and fine-tuning multilingual bert for few-shot and zero-shot sequence labeling tasks.