L o RA adapter weight tuning with multi-task learning for Faux-Hate detection (2024)

Abstract: Detecting misinformation and harmful language in bilingual texts, particularly those com-bining Hindi and English, poses considerabledifficulties. The intricacies of mixed-languagecontent and limited available resources compli-cate this task even more. The proposed workfocuses on unraveling deceptive stories thatpropagate hate. We have developed an inno-vative attention-weight-tuned LoRA Adopter-based model for such Faux-Hate content de-tection. This work is conducted as a partof the ICON 2024 shared task on DecodingFake narratives in spreading Hateful stories.The LoRA-enhanced architecture secured 13thplace among the participating teams for TaskA.

## introduction social media platforms today enable individuals to express their identities and views, seek validation and feedback, promote products, and make purchases. however, this freedom of speech should be used more effectively. many users post offensive content targeting specific groups, religions, and communities, which exposes these groups to harmful criticism and negatively affects younger audiences @xcite . furthermore, fauxhate narratives are frequently spread on these platforms, causing distress to the government and society alike. this faux hate, emerging from fake narratives, especially during emergencies, can gravely impact our society. in multi-lingual societies like india, people speak 22 different languages @xcite . additionally, due to the influence of english on our education system, people commonly mix the linguistic patterns of their native languages with english, leading to code-mixed text @xcite the proposed study focuses on identifying faux-hate content in low-resource hindi-english code-mixed text. current measures to address these issues primarily involve flagging accounts or disabling user activity, which can unintentionally suppress freedom of speech. however, this situation underscores the urgent need for a more effective strategy to combat faux hate content while allowing people to express their opinions. while hate can be conveyed through various mediums like images, animations, memes, and emoticons, our work focuses on text data. most of the existing work considers fake and hate content as separate entities and employs several single-task learning models to address these types of content. various transformer models, such as roberta and bert, have been utilized. for instance, @xcite , @xcite , and @xcite have analyzed different techniques for hate speech detection, reviewing traditional machine learning models, deep learning models, multi-task learning methods, generative models, and transformer-based models. their findings indicate that transformer models are wellsuited for hate speech detection tasks. however, these models have primarily been trained on highresource languages like english and tend to perform poorly with indian regional languages. to address the issue of limited datasets for indian code-mixed languages and joint learning of fauxhate content, the authors biradar et al. (2024b) have made significant contributions by developing codemixed language datasets. one such dataset is the faux hate multi-label dataset (fhmld), which includes hindi-english code-mixed text to enhance research in this area. to address the issue of faux-hate detection in low-resource hindi-english code-mixed text, the organizers of the icon shared task on decoding fake narratives in spreading hateful stories (faux-hate) have designed two sub-tasks. task a focuses on developing a multi-task model that identifies fake and hate labels for a given input. at the same time, task b aims to create a model that generates both target and severity labels. our team partici-pated in task a and secured the 13 th rank among the participating teams. in our proposed work, we developed a suite for converting bilingual text into monolingual text. on top of this, we employed lora adapter-based attention weight tuning to enhance the model's performance. the rest of the article is organized as follows: section 2 delves into the methodology that is proposed for the given task, section 3 analyzes the experimental results, and we conclude the paper alongside listing potential scope. ## results the performance of the hate speech detection model and fake speech detection model is summarized in the classification report mentioned in