CODEOFCONDUCT at Multilingual Counterspeech Generation: A Context-Aware Model for Robust Counterspeech Generation in Low-Resource Languages (2025)

Abstract: This paper introduces a context-aware model for robust counterspeech generation, which achieved significant success in the MCG-COLING-2025 shared task. Our approach particularly excelled in low-resource language settings. By leveraging a simulated annealing algorithm fine-tuned on multilingual datasets, the model generates factually accurate responses to hate speech. We demonstrate state-of-the-art performance across four languages (Basque, English, Italian, and Spanish), with our system ranking first for Basque, second for Italian, and third for both English and Spanish. Notably, our model swept all three top positions for Basque, highlighting its effectiveness in low-resource scenarios. Evaluation of the shared task employs both traditional metrics (BLEU, ROUGE, BERTScore, Novelty) and the LLM-based JudgeLM. We present a detailed analysis of our results, including error cases and potential improvements. This work contributes to the growing body of research on multilingual counterspeech generation, offering insights into developing robust models that can adapt to diverse linguistic and cultural contexts in the fight against online hate speech.

## introduction hate speech (hs) encompasses expressions that demean or target individuals or groups based on characteristics such as race, ethnicity, gender, sexual orientation, or religion @xcite . although hs makes up only a small fraction of social media content, its impact is significant, affecting nearly one-third of people @xcite . the prevalence of hs on social media has become a critical societal concern. traditional approaches, such as content removal and moderation, have been widely implemented but are often criticized for infringing on free speech. as an alternative, counterspeech (cs) has emerged as a promising solution to mitigate hs while upholding the principle of free expression @xcite . counterspeech is defined as speech intended to counteract and neutralize harmful language. it has demonstrated effectiveness in real-world applications @xcite , but its manual creation is labor-intensive and impractical at scale given the high volume of hs online. this challenge has driven interest in automating cs generation using nlp technologies. the growing need for effective counterspeech highlights the importance of strategies that foster healthier online environments. however, multilingual cs generation remains a significant challenge, particularly in lowresource settings where data scarcity limits model development. research has focused on understanding hs, crafting effective cs, and addressing the unique challenges of multilingual and resourceconstrained contexts. the mcg-coling-2025 shared task addressed these challenges by inviting researchers to generate respectful, specific, and truthful counterspeech across multiple languages, including basque, english, italian, and spanish. in this paper, we present *codeofconduct*, a context-aware model that achieved state-of-the-art performance in this shared task. our model demonstrates exceptional effectiveness in low-resource scenarios, particularly for basque, and offers valuable insights into leveraging multilingual datasets and advanced optimization techniques for counterspeech generation. ## background the languages chosen for the shared task cover a wide range of linguistic features, from basque's complex agglutinative morphology to the more straightforward syntax of english. this diversity allows for testing models' adaptability to varying linguistic challenges. furthermore, the inclusion of background knowledge and cultural differences in multiple languages adds an additional layer of complexity, requiring models to integrate contextual information effectively. ## methodology we employed a three-stage approach for generating effective cs across multiple languages. the first stage utilizes a simulated annealing approach combined with llms to generate and select diverse responses. a word sampling mechanism extracts vocabularies from both predefined word lists and input text (hs and cn) to enrich the response generation. each candidate response is evaluated using judgelm, with scores exponentially weighted to guide the sampling process. the second stage implements a round-robin tournament evaluation system to rank and select the most effective responses, ensuring high-quality output even in lowresource language settings. in the last stage, we combined each of the first-ranked, second-ranked, third-ranked, and fourth-ranked answers into their own csvs and then ran the evaluation script given by mcg-coling foot_1 to find the top 3 runs. ## conclusion this study showcases the effectiveness of the codeofconduct model in crafting contextaware counterspeech, achieving exceptional results in both high-and low-resource language settings. by integrating a simulated annealing-based generation framework with a robust round-robin ranking mechanism, our approach secured leading positions in the mcg-coling-2025 shared task across four diverse languages. as evidenced by the quantitative results in figure 7 , our model's success with basque-a language with limited nlp resources such as annotated datasets, pretrained models, and linguistic tools-stands out as a key achievement. this success stems from three key factors: (1) our simulated annealing approach's effectiveness in handling basque's complex agglutinative morphology, (2) a word sampling strategy specifically enhanced for low-resource scenarios by incorporating domain-specific terminology, and (3) an evaluation system well-suited to basque's unique linguistic features. all three of the runs submitted were able to outclass all the other runs submitted by other groups. by working with only 100 hs-cn pairs for each language and leveraging multilingual pretrained models alongside innovative optimization techniques, we demonstrated how thoughtful methodologies can overcome resource constraints. our sweeping success in basque, coupled with strong rankings in italian, english, and spanish, highlights the versatility of our approach in navigating diverse linguistic and cultural challenges, with particular effectiveness in tackling the unique demands of low-resource languages. through the use of comprehensive evaluation metrics, including judgelm and traditional measures such as bleu and rouge, we ensured that the generated counterspeech was both linguistically accurate and contextually appropriate. the combination of simulated annealing and round-robin evaluation is particularly well-suited for this task for several reasons: simulated annealing enables exploration of culturally appropriate responses through temperature-controlled sampling, while round-robin evaluation captures the nuanced effectiveness of counterspeech across different cultural contexts that simple metrics might miss. the final results highlighted the model's strengths while also revealing opportunities for improvement, particularly in adapting evaluation frameworks to better reflect the nuances of multilingual and culturally specific outputs.