Mocktails of Translation, Ensemble Learning and Embeddings to tackle H inglish NLP challenges (2024)

Abstract: Social media has become a global platform where users express opinions on diverse contemporary topics, often blending dominant languages with native tongues, leading to code-mixed, context-rich content. A typical example is Hinglish, where Hindi elements are embedded in English texts. This linguistic mixture challenges traditional NLP systems, which rely on monolingual resources and need help to process multilingual content. Sentiment analysis for code-mixed data, mainly involving Indian languages, remains largely unexplored. This paper introduces a novel approach for sentiment analysis of code-mixed Hinglish data, combining translation, different stacking classifier architectures, and embedding techniques. We utilize pre-trained LoRA weights of a fine-tuned Gemma-2B model to translate Hinglish into English, followed by the employment of four pre-trained meta embeddings: GloVe-T, Word2Vec, TF-IDF, and fastText. SMOTE is applied to balance skewed data, and dimensionality reduction is performed before implementing machine learning models and stacking classifier ensembles. Three ensemble architectures, combining 22 base classifiers with a Logistic Regression meta-classifier, test different meta-embedding combinations. Experimental results show that the TF-W2V-FST (TF-IDF, Word2Vec, fastText) combination performs best, with SVM radial bias achieving the highest accuracy 91.53% and AUC (0.96). This research contributes a novel and effective technique to sentiment analysis for code-mixed data.

## introduction in communities where multiple languages are spoken, individuals often blend their native language with other prevalent languages during conversations. in digital space, including social media and texting with friends and family @xcite , mixing languages emerges as a key force to steer a paradigm shift. using more 125 pare in their performance? ## related work in recent work, jadon et al. @xcite explored a hybrid lstm-gru model for sentiment analysis on hinglish data, combining long short-term memory (lstm) and gated recurrent unit (gru) architectures. this approach handled hinglish's linguistic complexities effectively, achieving 96.76% accuracy. frias et al. @xcite singh and lefever (singh and lefever, 2020) investigated sentiment analysis for hinglish using unsupervised cross-lingual embeddings to capture word meanings across languages. this method, trained on semeval 2020 data ( @xcite ), outperformed models trained on monolingual embeddings, achieving an f1-score of 0.635 compared to a baseline of 0.616. the cross-lingual embeddings facilitated transfer learning, allowing a sentiment model trained on english data to be applied to hinglish data, resulting in an f1-score of 0.556. singh @xcite found that the highest f1score of 0.6907 on the semeval 2020 dataset was achieved using an ensemble voting (soft) (( patwa et al., 2020) 237 work, with all the computational elements and inter-238 actions placed. the computational modules are de-239 signed using either of the fundamental techniques, 240 e.g. meta-embedding, data balancing, feature se-241 lection, feature scaling, or classification. the pro-242 posed research study consists of two individual sets 243 of modules: the translation and prediction mod-244 ules. the translation module is a dedicated module 245 for the 'hinglish to english' translation of code-246 mixed tweets from twitter, aka the 'x' platform. 247 this module is prefixed and postfixed using data-248 cleaning techniques. further, prediction modules 249 take care of computations using different methods. the inherent details within each technique are pro-251 vided subsequently.