Weakly supervised learning for aspect based sentiment analysis of U rdu Tweets (2023)

Abstract: Aspect-based sentiment analysis (ABSA) is vital for text comprehension which benefits applications across various domains. This field involves the two main sub-tasks including aspect extraction and sentiment classification. Existing methods to tackle this problem normally address only one sub-task or utilize topic models that may result in overlapping concepts. Moreover, such algorithms often rely on extensive labeled data and external language resources, making their application costly and time-consuming in new domains and especially for resource-poor languages like Urdu. The lack of aspect mining studies in Urdu literature further exacerbates the inapplicability of existing methods for Urdu language. The primary challenge lies in the preprocessing of data to ensure its suitability for language comprehension by the model, as well as the availability of appropriate pre-trained models, domain embeddings, and tools. This paper implements an ABSA model (CITATION) for unlabeled Urdu tweets with minimal user guidance, utilizing a small set of seed words for each aspect and sentiment class. The model first learns sentiment and aspect joint topic embeddings in the word embedding space with regularization to encourage topic distinctiveness. Afterwards, it employs deep neural models for pre-training with embedding-based predictions and self-training on unlabeled data. Furthermore, we optimize the model for improved performance by substituting the CNN with the BiLSTM classifier for sentence-level sentiment and aspect classification. Our optimized model achieves significant improvements over baselines in aspect and sentiment classification for Urdu tweets with accuracy of 64.8% and 72.8% respectively, demonstrating its effectiveness in generating joint topics and addressing existing limitations in Urdu ABSA.

## introduction opinion mining and sentiment analysis have gained significant importance in analyzing user-generated content for various applications. however, the vast volume of textual content on numerous platforms like social media makes manual analysis impractical. hence, there is a need for automatic computational frameworks to extract opinions from unstructured texts, leading to the development of sentiment analysis and opinion mining as a research field. conventional sentiment analysis studies mainly perform prediction at the sentence or document level @xcite ; @xcite , identifying the overall sentiment towards the whole sentence or document. however, absa @xcite surpasses traditional sentiment analysis by incorporating finer granularity and enabling the identification of two crucial components: target aspect and sentiment in a given sentence or document. the target represents an entity or aspect of an entity, while the sentiment signifies an expression of opinion. for instance, consider sentences from restaurant reviews "the rice was great" and "the drink was hot". both sentences convey sentiments but the term 'great' in the first sentence represents a positive sentiment related to the rice aspect, whereas the term 'hot' in the second sen-tence signifies a negative sentiment associated with the drink aspect. absa primarily revolves around four target components: aspect category, aspect term, sentiment term, and sentiment polarity @xcite . a comprehensive understanding of opinions at the aspect level requires considering multiple combinations of these components in different scenarios, generally referred to as compound absa. however, early absa research focused on individual components, such as capturing aspect terms while ignoring terms related to opinions like aspect term extraction (ate) @xcite . nowadays, various frameworks have been proposed to tackle such compound absa tasks that aim to extract and link individual elements @xcite ; @xcite . such tasks are further denoted with several abbreviations, based on the specific combinations they address in compound absa. figure 1 illustrates these tasks with their underlying combinations of absa individual components, as well as a complete absa of an opinion. most of the research in absa has been conducted in resource-rich languages, primarily english. recent studies generally adopt supervised frameworks @xcite ; @xcite that require a large number of labeled sentences for training. some studies leverage word embeddings in unsupervised @xcite or weakly supervised settings @xcite ; @xcite for aspect extraction without annotated documents. these methods often rely on external language resources or syntactic structures such as pos tags or lexicons. nevertheless, opinions are expressed in multiple languages in real-world scenarios. acquiring large annotated data and accurate language resources for new domains or low-resource languages is challenging and expensive. cross-lingual and cross-domain transfer requires language-specific knowledge of the target language and domain. besides, previous methods relying on translation systems are limited by translation quality @xcite . alike other low-resource languages, urdu also encounters difficulties in terms of data availability, language tools, and research resources. additionally, urdu is a comparatively complex language holding characteristics of multiple languages that pose troubles in minor processing tasks @xcite . besides, the use of informal language on social media platforms further complicates the preparation of suitable datasets for machine learning models. as a result, absa tasks have been largely overlooked in the urdu language and pretrained models developed for other languages may not generalize effectively. due to limited literature, many approaches and difficulties as discussed for english need to be explored for the applicability of absa in the urdu language. meanwhile, the popularity and widespread usage of urdu on online platforms demand sincere efforts to address these challenges for the advancement of urdu technologies. this paper presents a significant contribution to absa in urdu by addressing its unique facets and challenges for urdu texts. we adopt a pioneering step by employing a weakly-supervised model @xcite developed for english absa to unlabeled urdu tweets on the budget topic. our methodology involves rigorous pre-processing to ensure data quality and model compatibility to avoid memory leakage, training disruptions caused by invalid tokens, and out-of-vocabulary issues. moreover, tokenization tools are deeply analyzed to determine the optimal approach for token formation in urdu, taking into account their pivotal role in comprehending language context and optimizing resource utilization. we evaluate the model with vanilla settings and conduct experiments with architecture modifications specifically replacing cnn with bilstm and utilizing fasttext embeddings. clustering techniques and graph network analysis are applied to enhance seed words selection rather than manually. ## literature review several approaches have been proposed for aspect extraction and sentiment classification in absa. some methods focus on independently addressing these two sub-tasks, while others adopt a joint approach to simultaneously solve them. ## methodology our methodology consists of three main stages: pre-processing, seed word selection, and model implementation. figure 2 illustrates the entire process. ## conclusion we propose an approach to introduce joint aspectbased sentiment analysis in the urdu language using advanced techniques with minimal guidance. the weakly-supervised jasen model, originally developed for english, is applied to urdu tweets to capture fine-grained information. fasttext embeddings and the vanilla settings of the model are leveraged to establish a benchmark, followed by experiments with modifications in the model architecture and default settings. extensive pre-processing is performed to prepare the dataset due to the complexity and informality of the language used in tweets, making it compatible with model training and resource utilization. the results demonstrate improvements in the model's performance on the urdu dataset, attributed to the enhancements in preprocessing and model architecture modifications.