Vicinal Risk Minimization for Few-Shot Cross-lingual Transfer in Abusive Language Detection
Authors: Gretel Liz De la Peña Sarracén
Paolo Rosso
Robert Litschko
Goran Glavaš
Simone Paolo Ponzetto
Venue: EMNLP 2023 Main (2023)
Keywords: Vicinal Risk Minimization
Few-Shot Cross-lingual Transfer
Abusive Language Detection

Abstract: Cross-lingual transfer learning from high-resource to medium and low-resource languages has shown encouraging results. However, the scarcity of resources in target languages remains a challenge. In this work, we resort to data augmentation and continual pre-training for domain adaptation to improve cross-lingual abusive language detection. For data augmentation, we analyze two existing techniques based on vicinal risk minimization and propose MIXAG, a novel data augmentation method which interpolates pairs of instances based on the angle of their representations. Our experiments involve seven languages typologically distinct from English and three different domains. The results reveal that the data augmentation strategies can enhance few-shot cross-lingual abusive language detection. Specifically, we observe that consistently in all target languages, MIXAG improves significantly in multidomain and multilingual environments. Finally, we show through an error analysis how the domain adaptation can favour the class of abusive texts (reducing false negatives), but at the same time, declines the precision of the abusive language detection model.

Vicinal Risk Minimization for Few-Shot Cross-lingual Transfer in Abusive Language Detection
Gretel Liz De la Peña Sarracén Universitat Politècnica de València
Paolo Rosso
Universitat Politècnica de València
uredela@posrrado.uuk upv.es
prossoêdsic. upv.
Robert Litschko"
Goran Glavaš
MaiNLP, LMU Munich
CAIDAS, University of Würzburg
rlitschkēcis. 1mu.de
goran. glavas@uni-wuerzzurrg
Simone Paolo Ponzetto
DWS Group, University of Mannheim
ponzetto@uni -mannheimmnnnmmmmmmm
Abstract
Transfer learning is popularly used in few-shot learning, where the prior knowledge from a source task is transferred to the few-shot task (Pan and Yang, 2010; Pan et al., 2019). Usually, training data is abundant in the source task, while training data is low in the target task. In natural language processing, few-shot cross- pooss-lingual transfer learning (Glavaš et al., 2020; Schmidt al., 2022; Winata et al., 2022) the type of few-shot transfer learning in which the source/target tasks are the same but the source/target languages are different. A pre- trained multilingual model is first fine-tuned in a high-resource language and then fine-tuned on few data in target language (Zhao et al., 2021).
Cross-lingual transfer learning from high- resource to medium and low-resource lan- guages has shown encouraging results. How- ever, the scarcity of resources in target lan- guages remains a challenge. In this work, we resort to data augmentation and continual pre-training for domain adaptation improve cross-lingual abusive language detection. For data augmentation, we analyze two existing techniques based vicinal risk minimization and propose MIXAG, a novel data augmen- tation method which interpolates pairs of in- stances based on the angle of their representa- tions. Our experiments involve seven languages typologically distinet from English and three different domains. The results reveal that the data augmentation strategies can enhance few- shot cross-lingual abusive language detection. Specifically, we observe that consistently in all target languages, MIXAG improves signifi- cantly in multidomain and multilingual environ- ments. Finally, we show through an eтror anal- ysis how the domain adaptation class of abusive texts (reducing false negatives), but at the same time, declines the precision of the abusive language detection model.
Due to the limited availability of examples in the target language, naive fine-tuning can lead to over- fitting and thus poor generalization performance on the few-shot task (Parnami and Lee, 2022). A strat- egy usually used to alleviate this problem, not just in the few-shot cross-lingual transfer but in FSL in general, to increase the number of samples of the few-shot task from prior knowledge. This is the data-level approach (Chen et al., 2023), which can be divided into two categories: 1) transforming samples from the few existing examples (Arthaud et al., 2021; Zhou et al., 2022; Zhang et al., 2022) and 2) transforming samples from external datasets (Antoniou and Storkey, 2019; Rosenbaum et al., 2022: Pana al., 2023).
favour the
1 Introduction
Few-shot learning (FSL) is machine learning paradigm that allows models to generalize from a small set examples (Wang et al., 2020, 2023). Unlike traditional methods, FSL does not require training a model from scratch. Instead, pre-trained models are extended with just little information, which is useful when training examples scarce data annotation is expensive.
Contributions. In this work, we explore abu- sive language detection in seven topologically di- verse languages via few-shot cross-lingual transfer learning at the data-level. Although a number of studies have examined abusive language, we aim take advantage of resources available for English in other less explored and fow-resource languages. We focus on two aspects: 1) considering languages
Work done while at University of Mannheim

---

that are typologically distinct from English and 2) with little effort. Previous works focus on lan- guages that are similar to English, such as Euro- pean languages (Stappen et al., 2020; Nozza, 2021; Rodríguez et al., 2021; Firmino et al., 2021; Zia et al., 2022; Castillo- et al., 2023). In con- trast, we analyze languages that are more different from English. 'Little effort' refers to consistent strategy across all languages, without requiring external resources or ad hoc processing for each particular language. The main contributions of this paper can be summarized as follows:
2 Background and Related Work
In this section, we discuss the main issue of few- shot learning and how data- basa- approaches can alleviate it. We take the definitions from Wang et al. (2020), where more details can be found. Then, we provide brief overview of abusive language and align our work with recent studies focused on few- shot cross-lingual transfer approaches.
Few-Shot Learning. Few-shot learning deals with small training set Dtrain {2), } to ap- proximate the optimal function f" that maps input to output y. given a joint probability distribution p(r, y). Thus, a FSL algorithm is an optimization strategy that searches in a functions space F to find the set of parameters that determine the best f' E F. The performance is measured by a loss function (f)) v) which defines the expected risk with respect to p(x;, y). However, p(æ, y) is un- known, hence the empirical risk is used instead (Fernandes de Mello et al., 2018). This is the average of sample losses over Dirain and can be reduced with a larger number of examples. One major challenge for FSL is then the small size of Dtrain. which can lead to the empirical risk not being a good approximation of the expected risk. To alleviate this problem, an approach that exploits prior knowledge can be used (Wang et al., 2023). Data-level approach involves methods that augment Dtrain with prior knowledge (Feng et al., 2021: Bayer et al., 2022; Dai al., 2023).
Dataset extension: We rely on a multidomain and multilingual dataset for abusive language de- tection (Glavaš et al., 2020). This dataset contains texts in 5 languages which have been obtained by translating original English texts. To facilitate a more comprehensive evaluation, we extend the dataset by manually translating it into Spanish.
Few-shot cross-lingual transfer learning im- provement at data-level: We rely on Vicinal Risk Minimization (VRM) (Chapelle et al., 2000) to generate synthetic samples in the vicinity of the examples to increase the amount of information fine-tune the model in the target language. In this work we use three VRM-based techniques: 1) SSMBA (Ng et al., 2020), which uses two func- tions to move randomly through a variety of data, 2) MIXUP (Zhang et al.. 2018). which linearly com- bines pairs of examples to obtain new samples and 3) MIXAG, our variant of MIXUP, which controls the angle between example and the synthetic data generated in its neighbourhood.
Vicinal Risk Minimization formalizes the data augmentation as an extension of Dirain by draw- ing samples from a neighbourhood of the existing samples (Chapelle et al., 2000). The distribution p(x,, y) is approximated by vicinity distribution D, { 2. Ni, whose instances tion of the instances of Dtrain. Vicinal risk (R,) is then calculated on D, as Equation 1.
Unsupervised language adaptation: We also simulate a fully unsupervised sctup, removing the label information from the target languages. In that setup, we examine strategy to address the lack of information that zero-shot transfer (no example to fine-tune the model) faces. The general idea is make domain adaption for abusive terms via masked language modeling (MLM) in the target language before the zero-shot transfer.
func-
No No Σ (f(()).))
Rg
(1)
We aim to answer the following research questions:
In this work, we study three VRM-based tech- niques that use different strategies to generate the vicinity distribution (see $4).
RQI: What is the role of VRM-based techniques in few-shot cross- lingull abusive language detection?
RQ2: What is the impact of different languages on few-shot gross-lingual abusive language detection?
A busive Language. Typically, abusive language refers wide range of concepts (Balayn et al., 2021; Poletto et al., 2021), including hate speech (Yin and Zubiaga, 2021; Alkomah and Ma, 2022; Jain and Sharma, 2022), profanity (Soykan et al.,
RQ3: How does VRM-based techniques fare against domain specialization for cross cross-lingual transfer of abusive language detection models?

---

2022), aggressive language (Muti al., 2022; Kan- clerz et al.. 2021), offensive language (Pradhan et al., 2020; Kogilavani et al., 2021), cyberbully- ing (Rosa et al., 2019) and misogyny (Shushke- vich and Cardiff, 2019). Pamungkas et al. (2023) overview recent research across domains and lan- guages. They identify that English is still the most widely studied language, but abusive language datasets have been extended to other languages, including Italian, Spanish and German (Corazza et al., 2020; Mamani Conoorii and Ochoa-Luna, 2021; Risch et al., 2021). In addition, we have found studies for other languages such as Arabic (Khairy et al., 2021), Danish (Sigurbergsson and Derczynski, 2020), Dutch (Caselli et al., 2021), Hindi (Das et al., 2022), Polish (Ptaszynski et al., 2019) and Portuguese (Leite et al., 2020). Regard- less, some works like (Stappen et al., 2020) state that there is need to extend the resources for di- verse and low-resource languages. To cover this problem, Glavaš et al. (2020) propose a multido- main and multilingual evaluation dataset. They show that banguage-adaptive additional pre-training of general-purpose multilingual models can im- prove the performance in transfer experiments. These are promising results, and although there are works like (Pamungkas al., 2023) that cite this dataset, we have not found works that exploit it. In this work, we extend the study of the origi- nal work (Glavaš et al., 2020) to assess strategies for enhancing the performance of abusive language detection in low-resource languages.
These are two approaches to improve the problem of poor data availability in few-shot learning. In contrast, we focus on the data-level approach. Un- like other works that are also based on increasing data (Shi et al., 2022), we explore VRM-based strategies for abusive language detection.
Dataset and Experimental Setup
XHate-999 (Glavaš et al., 2020) is an available datasct intended to explore several variants of abu- sive language detection. This dataset includes three different domains: Fox News (GAO), Twit- ter/Facebook (TRAC), and Wikipedia (WUL). In our work, we define ALL as the set of instances resulting from the union of all three domains. Each domain comprises different amounts of annotated data (abusive/non-aaussiv)) in English for training, validation, and testing (see Appendix A). English test instances are translated into five target lan- guages: Albanian (SQ), Croatian (HR), German (DE), Russian (RU), and Turkish (TR).
We extended this dataset with texts in Spanish. To generate the texts, we rely machine trans- lation and post-editing, following the monitored cranslation-bsse approach described in the dataset paper. Thus, slight modifications were made in the Spanish translation to reflect and maintain the level of abuse in the original English instances.
Models. We rely on mBERT (Devlin et al., 2019) base cased with = 12 transformer layers, hid- den state size of H = 768, and A = 12 self- attention heads (see Appendix A for more details), First, we retrain the model with the XHate-999 training and validation sets, to obtain the model (model_base that we use in all our experiments. We search the following hyper-parameter grid: training epochs in the set (2,3.4) 4) and learning rate in {10-4, 10 11111110006,,, We train and evaluate in batches of 2 texts, with a maximal length of 512 tokens, and optimize the models with Adam (Kingma and Ba, 2015). We set the random seeds to 7 to facilitate the reproducibility of experiments.
Cross-Lingual Abusive Language Detection. In recent years, cross-lingual abusive language de- tection has gained increasing attention in zero-shot (Eronen et al., 2022) and few-shot (Mozafari et al., 2022) transfer. Pamungkas and Patti (2019) pro- pose a hybrid approach with deep learning and multilingual lexicon for cross-lingust abusive con- tent detection. Ramasinghe and Zampieri (2020) use English data for cross-lingual contextual word embeddings and transfer learning to make predic- tions in languages with fewer resources. More re- cently, Mozafari et al. (2022) propose an approach based on meta-learning for few-shot hate speech and offensive language detection in low-resource languages. They show that meta-learning models can quickly generalize and adapt to new languages with only a few labelled data points to identify hate- ful or offensive content. Their meta- metaa-lernnng mod- els are based on optimization-leve level and mevrl.-level.
Fine-tuning and Evaluation Details. For each language, we draw 90% of instances from the test set to evaluate model_base In few-shot cross- lingual transfer experiments, we use the remain- ing 10% of instances to fine-tune model_ be- fore the evaluation. i.e. we use 10 instances fine-tune model_base in GAO (and 89 to evalu- ate), while the respective numbers are 30 (270)

---

for TRAC, 60 (540) for WUL, and 100 (899) for ALL (GAO+TRAC+WUL), Notice that for each language, the test set used by Glavaš et (2020) is different from the one we use. However, we do not observe a significant difference between the use of the full test set and the use of the subset we rely on (see Appendix C to examine the results).
combination of the texts representations (Lqua- tion 3): we obtain the embedding E., of each word of a text z; and concatenate them to generate the vector representation E(zi). Then, we combine two texts 22 and zj as the linear combination of their representations Ex,, and E(zz). Note that Eu: is single step of an auto-regressive model. The obtained vector is split into vectors of the same size as the original word embeddings Eu-- Finally, we decode those vectors to obtain a sequence T of words, that we use as the new syntectic text di. The linear combination of the labels y E {0, 1}. when y/ and yj are different depends on the value of X. We assign to di when the combination is greater than equal to 0.5. Otherwise, we assign O.
Statistical Analysis. In our experiments, we used McNemar's test as (Dictterich, 1998) recommends. This is paired non-parametric statistical hypothe- sis test where the rejection of the null hypothesis suggests that there is evidence to say that the mod- els disagree in different ways. We the signifi- cance level to 0.05 and use Oultered: obtained with the Bonferroni correction (Napierala, 2012).
T(NE(a)) (1 A)E(z;))
(3)
4 Few-Shot Cross-lingual Transfer
This VRM-based technique an it- erative process. In each iteration, the few-shot set Dtrain is divided into pairs of samples to combine. Thus, the number of instances generated in each iteration is equal to N where N is the number of samples in Dtrain- We make not to take the same pairs of examples in different iterations.
Procedure.
We first examine the ability of three VRM-based techniques in few-shot cross- loos-linggal transfer learn- ing for abusive language detection to address RQ1.
4.1 SSMBA
Ng et al. (2020) propose SSMBA, a data augmen- tation method for generating synthetic examples with pair of corruption and reconstruction func- tions to move randomly on a data manifold. In the two strategies: 1) mask- ing a word in each text in random way (default) 2) masking the salient abusive words in each text. To identify abusive words, we use HurtLex (Bassignana et al., 2018), a multilingual lexicon with harmful words. For texts that do not contain words in the lexicon, we follow strategy 1. In the reconstruction functions, we use mBERT.
4.3 MIXAG
Motivated by the idea of MIXUP, we propose the variant MIXAG: mix vectors with a focus on the AnGle between them. We hypothesize that the distance between an example and the new synthetic examples may be relevant to generate an effective vicinity. As this aspect cannot be easily controlled in the original MIXUP, we propose particular case which interpolates pairs of instances based on the angle of their representation.
corruption function,
The idea is to define a linear combination (Fqua- tion 4) with the parameter as a function of the angle a between the original vectors x; and Ty, as well as the angle 0 between the new vector é and one of the original vectors (Figure 1).
4.2 MIXUP
(Zhang et al., 2018; Sun et al., 2020) is a VRM- based technique that constructs a synthetic example (f;, 9i) (in the vicinity distribution) from the lin- ear combination of two pairs (Ir, and (zj, (1), drawn at random from the training set Derain as Equation 2, with À 8(a, a:) where is hyper- parameter
f
Xx + Ty
(4)
c; Ax; + +1 A)æ;
(2)
Aya + (1 A)y;
i
We rely on multilingual GPT model (Shli- azhko et al., 2022) (see Appendix A) for the linear
ar 0
We tried some values different from for and MIXUP was not sensitive to variation, so we set to 0.2.
Figure 1: MIXAG description.

---

Using the Law of Sines we express À func- tion (Equation 5) of the cosine of a, which can be obtained with Equation 6, and the cosine of which is the parameter of MIXAG. || denotes the norm of a vector. We refer readers to Appendix B for more details.
4.6 Results and Analysis
A summary of cross-lingual transfer results for the variants few-shot and few-shot with SSMBA, MIXUP and MIXAG is provided in Figure 2 (we refer readers to Appendix C for all the results).
FS
SS
MU
MA
|ksillcooo00)// Tocol(a) costaly I-cos(0))
F1 Score
0.8
0.6
[esllv1~coo007
0.4
cos(a) = ||: ||
2,2
(6)
EN
DE
RU
TR
HR
SQ.
ES
For MIXAG, we define the combination of texts by Equation following the same representation and processing of texts as MIXUP. The differ- ence is basically in the parameter A.
F1 Score
0.8
0.6
0.5
EN
DE
RU
TR
HR
SQ.
ES
4 = T(EE(x;) + B(z)))
(7)
F1 Score
0.6
In this work, set 0 = S. thus the parameter of MIXAG is defined by Equation We suggest extending this study to analyze how the parameter cos(O) can influence the results.
0.4
0.2
EN
DE
RU
TR
HR
SQ
ES
Figure 2: Performance with mBERT of few-shot (FS) cross-lingual transfer and the variants: SSMBA (SS), MIXUP (MU) and MIXAG (MA). Upper Figure: GAO domain, Middle Figure: TRAC domain and Lower Figure: WUL domian
cos(0) =V cos(a)
(8)
Procedure. This VRM-based technique is also iterative process. In this case, randomly se- lect sample 2; from Derein and create the pairs with 2y and each of the rest of the samples of Dtrain- Therefore, the number of instances gen- crated each iteration is N - 1, where is the number of samples in Dtrain.
As expected, we observed that VRM- based tech- niques improve the performance of few-shot cross- lingual transfer in most cases. There is clear difference between the VRM-based techniques, but we can see interesting results that vary depend- ing on the domain. In the GAO domain, all three techniques seem to have similar results across lan- guages. In TRAC, MIXUP seems to be slightly better than MIXAG in most languages. However, the critical result in this domain is that SSMBA fails improve the few-shot cross-lingual transfer. In contrast, SSMBA seems be the best technique in WUL. We believe that these results are due to the nature of the texts in each domain. TRAC contains texts from Twitter and Facebook. We speculate that the reconstruction function of SSMBA affects the quality of the vicinity generated for each text by in- troducing terms that differ from common terms in this domain. On the other hand, WUL contains text from Wikipedia, which supports assumption.
4.4 Multilingual MIXUP/MIXAC
By default, in MIXUP and MIXAG we use the few- shot set Dtrain of each language generate new instances for that particular language. Alternatively, we use the union of the Dtrain of all languages. For each pair of original texts 24 and Zy. we make sure that o3 is from the language in the analysis, while zy is a text from any language.
4.5 Multidomain MIXUP/MIXAG
We rely on training data for GAO, TRAC and WUL, well as ALL (WUL-TRAC+GAOO) in all mono- lingual and multilingual experiments. In short, analyze performance when training and testing 1) only on particular domain (for example, when testing on GAO we train only on GAO training data) and 2) on all available data from all three data sets (multidomain setup).
Multidomain. Table shows the results for all the variants of the VRM-based techniques. We il- lustrate and analyze the results for the combination

---

ALL
EN
DE
RU
TR
HR
SQ
ES
ZS
0.8085
0.7156
0.6308
0.3627
0.6214
0.6127
0.6008
FS
0.8112
0.7141
0.6329
0.4063
0.6316
0.6238
0,6130
SS
0.8077
0.7253
0.7071
0.6568
0.6965
0.6990
0.6838
SS-HL
0.8097
0.7273
0.6987
0.6689
0.6725
0.6909
0.6973
MU
0.8102
0.7404
0.7013
0.6740
0.7116
0.7001
0.6878
MMU
0.8284
0.7500
0.7312
0.7113
0.7371
0.7128
0.7250
MU-SS
0.8176
0.7531
0.7233
0.6839
0.7186
0.6881
0.7087
MA 0.8083
0.7245
0.6757
0.5616
0.6710
0.6788
0.6508
MMA 0.8237
0.7585
0.7392
0.7224
0.7523
0.7344
0.7476
MA-SS 0.8096
0.7229
0.7193
0.6369
0.6759
0.6734
0.6713
Table 1: Zero-shor (ZS) and few-shot (FS) cross-lingual transfer performance with mBERT on the union of all domains. We also show 8 variants for FS: 1) SSMBA (SS) and 2) SSMBA with Hurtles Lex (SS-HL.). 3) MIXUP (MU), 4) multilingual MIXUP (MMU), 5) MIXUP with SSMBA (MU-SS), 6) MIXAG (MA). 7) multilingual MIXAG (MMA), 8) MIXAG with SSMBA (MA-SS). The results (Oalttered = .005) are reported in terms of Fl and significantly better results are underlined for each language and domain. Numbers in bold indicate the best results.
of all domains. The results by domain are detailed in Appendix C.
MIXUP/MIXAG in most cases.
Correlation Analysis. Thus far, we have ob- served that the behaviour of the strategies seems quite similar across languages. For instance, the few-shot cross-lingual transfer is outperformed with the VRM-based techniques. This motivates us to investigate RQ2, i.c. we examine if there is a high correlation between the performance of few- shot cross-lingual transfer (and its variants with TRM-based techniques) and the linguistic proxim- ity scores of each language English.
All languages except German seem to benefit from few-shot cross-lingual transfer w.r.t. zero- shot cross-lingual transfer. Likewise, the few-shot cross-lingual transfer is improved with VRM-based techniques as in the results by domain.
SSMBA improves few-shot cross-lingual trans- fer in all languages except English. In this heteroge- neous domain, we do not observe the problem that SSMBA has in TRAC. On the other hand, the use of HurtLex does not seem be a relevant strategy, since the results are similar to those obtained with the default strategy (random selection). This is an encouraging result, which suggests that we can use SSMBA to improve few-shot cross-lingual transfer learning without relying external resources.
We analyze the correlation between the per- formance of the strategies that we use for cross- lingual transfer learming and the distance between each language and English. We rely on the tool LANG2VEC2 which proves language vectors that encode linguistic features from the URIEL database (Littell et al., 2017). We obtain the vector representation of the languages with 4 features: 1) SYN: encodes syntactic properties, 2) FAM: en- codes memberships in language familics, 3) INV: denotes the presence of natural classes of sounds and 4) PHO: encodes phonological properties.
MIXUP seems to be better than SSMBA and MIXAG for most languages. However, multilin- gual MIXAG is significantly the best strategy. This is a good indicator of the benefits of our variant for multidomain and multilingual environments. Note that the multilingual strategies outperform the rest of the variants and that particularly, multilingual MIXAG consistently performs better than multi- lingual MIXUP. This suggests that our hypothesis about the implication of controlling the angle be- tween the original texts and the new synthetic texts seems to be relevant in multilingual data.
Then, with the vectors from each linguistic fea- ture, we calculate the cosine similarity between each language and English. Finally, we calcu- late the Pearson correlation coefficients (Sedgwick, 2012) between the cosine similarity and the per- formance of each cross-lingual strategy across lan- guages and domains.
Finally, we combine MIXUP/MIXAG with SSMBA: First, we augment the data with SSMBA and then augment the new vicinity with MIXUP/MIXAG. The results are also shown Ta- ble 1. This strategy offers some improvement over
Table 2 shows the correlation coefficients for the significant linguistic features with a significance

---

SYN
FAM
INV
PHO
resentation. Specifically, we used the multilingual generative model mTO (Muennighoff et al., 2023). instead of mGPT.
0.664
0.661
0.607
0.516
SS
0.527
0.627
0.608
0.486
MU
0.405
0.628
0.633
0.463
From the results of the ablation study in Table 3, we have the following observations. First, there are no significant differences with .05 between the variants studied, although experiments with XLM-R seem have shown some improvement. Secondly, we note that the variation of the angle between the inputs and the generated instances does not seem to represent relevant factor.
MA
0.571
0.721
0.686
0.529
Table 2: Pearson correlation coefficients between lin- guistic proximity scores (features SYN, FAM, INV, PHO) and few-shot (FS), few-shot with SSMBA (SS), few-shot with MIXUP (MU) and few-shot with MIXAG (MA) cross-lingual transfer performance with mBERT across all languages and domains.
All five variants obtain very similar results with BBERT. The variation of the factors that we ana- lyze does not seem to influence the performance of the method. However, with XLM-R we observe some interesting findings. Spanish and Russian are the only languages where MMA method is not sur- passed by the other variants. In the rest of the lan- guages, we observe the opposite behaviour, where text representation with the alternative model mTO seems to be the best strategy. Notice that in Alba- nian the use of m'TO for text representation together with the strategy of selecting the nearest neighbor for interpolation seems to be the best variant.
level of 0.05 (Appendix C shows the correlation coefficients for all metrics and the similarity scores between each language and English). Coefficients whose magnitude is between 0.5 and 0.7 indicate moderate correlation, while coefficients between 0.3 and 0.5 indicate low correlation.
We only observe moderate correlation between the performance of each strategy and the distance between the target languages and English. We consider these results encouraging because they suggest that the strategies are possibly consistent across languages.
4.7 Ablation Studies
Unsupervised Language Adaptation
MIXAG is data augmentation method that ran- domly combines inputs and uccordingly combines one-hot-label encodings. This a variant of MIXUP where the new data is obtained by defining the angle between the inputs and the new instance.
In this section, we investigate the scenarios in which there is no information about the target lan- guage for the few-shot cross-lingus transfer. In 8 4 we used a small amount of supervised data Dtrain in the target language to fine-tune the pre-trained model. This allowed us adapt the model the abusive language of each particular language. In contrast, now we assume that the labels of Dtrain are not available. This is a simulated experiment where we only have an unlabelled set of texts and the set Drest in which we want to detect abusive language. Previous works have examined this sce- nario by adjusting model with unlabelled external data. In this work, we use only few unlabelled instances from Dtrain-
In our strategy, we randomly select pairs of in- puts and set the angle between the new instance and one of the inputs as 2. where a is the an- gle between the original inputs. However, there are other strategies that could be used. For example, se- lecting data pairs whose latent representations are close neighbors, as well as defining other values for 0. To compare MIXAG with these alternative possibilities, we run set of ablation study experi- ments using not only mBERT, but also the XLM-R model (Conneau al., 2020). We focus on multi- lingual and multimodal MIXAG (MMA in ALL. as it is the best data augmentation method that we observed in the first experiments.
Basically, this strategy is a zero-shot cross- lingual transfer learning in which the model is adapted to the abusive terms of the target language. As mBER is pre-trained on geurpos and multilingual corpora, it is familiar with the target languages. However, it has been adjusted the particular case of abusive language. We follow then two-step methodology: 1) continual pre- taining for domain adaptation via masked language modeling (MLM) to make it familiar to the partic-
On the one hand, we compare the combination of random pairs of inputs with the combination of nearest neighbors (NN). On the other hand, we set the angle f = 3 to evaluate the impact of varying this parameter on the performance of the method. Finally, we use an alternative model for the text rep-

---

model
variant
EN
DE
RU
TR
HR
SQ
ES
MMA
0.8237
0.7585
0.7392
0.7224
0.7523
0.7344
0.7476
MMA-NN
0.8233
0.7585
0.7201
0.6473
0.7523
0.7273
0.7466
MBERT
MMA-ANG
0,8233
0.7475
0.7169
0,6774
0.7415
0.7273
0.7466
MMA-MTO
0.8238
0.7585
0.7392
0.7224
0.7523
0.7344
0.7476
MMA-MTT-NN
0.8254
0.7687
0.7314
0.6696
0.7477
0.7314
0.7528
MMA
0.8236
0.7927
0.7561
0.7258
0.7 180
0.7780
0.7670
MMA-NN
0.8251
0.7942
0.7328
0.7267
0.7180
0.7797
0.7650
XLM-R
MMA-ANG
0.8251
0.7940
0.7521
0.7267
0.7216
0.7797
0.7650
MMA-MTO
0.8245
0.7952
0.7503
0.7281
0.7243
0.7798
0.7658
MMA-MTO-NN
0.8245
0.7940
0.7503
0.7297
0.7180
0.7803
0.7658
Table 3: Results of the ablation studies for 4 variants of multilingual MIXAG (MMA): 1) interpolation only between nearest neighbors (MMA-NN), 2) set 0 3 (MMA-ANG), 3) text representation with mTO (MMA-MTO) and 4) text representation with mTO and interpolation between nearest nrighborxMMMAAMMTOO--N The results (aaltered .005) are reported terms of F1 for each of the model mBERT and XLM-R.
ular abusive terms, and then 2) employ zero-shot learning to detect abusive language.
positive class: while false negatives are reduced, false positives are increased.
Critical cases are negative texts that can be in- correctly detected as abusive. In order study this phenomenon, we examine the percentage of texts that are non-abusive and are well-classified with zero-shot transfer learning and misclassified with the MLM adaptation. We investigate two statistics across languages and domains: 1) the percentage of non-abusive texts that are well- classified with zero-shot transfer and misclassified with the MLM adaptation and 2) the percentage of abusive texts that are misclassified with zero-shot transfer and well-classified with the MLM adaptation.
5.1 Results and Analysis
Table 4 illustrates the results obtained with the methodology across domains and languages. In most cases, the strategy of prior adaptation to the abusive terms seems to outperform zero-shot cross- lingual transfer learning. English is the only lan- guage in which the MLM adaptation worsens the results all domains. Moreover, TRAC also shows no improvement, similar to the behaviour observed with SSMBA in sho-shot cross- lingull transfer.
These results allow us to answer RQ3: although domain adaptation can improve zero-shot cross- lingual transfer, VRM-based techniques seem to be more robust in few-shot cross-lingual transfer.
Table 5 illustrates the statistics across domains and languages. Consistent with the previous results we observe a detriment in the class of non-abusive texts. The number of negative texts well-classifedd with zero-shot transfer learning and misclassified with the MLM adaptation is large (reaching 100% in a case). However, that amount is surpassed in most cases by the gain in the class of abusive texts. We observe that the number of positive texts that are misclassified with zero-shot transfer learning and well-classified with adaptation via MLM is high (reaching 100% in four cases).
Error Analysis. order to deepen the analysis what happens in the model with the zero-shot cross-lingual transfer adaptation, we also analyze two metries: Recall and Precision. Recall refers to the true positive rate and is the number of true positives divided by the total number of positive texts. Precision refers the positive predictive value and is the number of true positives divided by the total number of positive predictions. In this work, positive refers to the class of abusive texts.
Conclusions and Future Work
Results across domains and languages are in Ap- pendix C. In all cases we observe an increase in Recall, indicating that adapting the model could improve the proportion of the class of abusive texts that is correctly classified. At first glance, it seems be good result, since it is desirable reduce the number of false negatives in abusive language detection. However, we observe that precision is reduced, suggesting that this strategy favours the
In this work, we studied three techniques to im- prove few-shot cross-lingual transfer learning in abusive language detection. These techniques are concentrated on data-level approach to deal with the problem of data scarcity that can lead high estimation error in few-shot learning. Specifically, we focused on vicinal risk minimization tech- niques to increase the data in the vicinity of the

---

GAO
EN
DE
RU
TR
HR
SQ
ES
ZS
0.6747
0.5067
0.5205
0.5116
0.6234
0.5405
0.5263
ZS
MLM
0.6050
0.6364
0.6261
0.6341
0.6290
0.6016
0.6154
TRAC
ZS
0.7642
0.7582
0.6815
0.6777
0.6892
0.7235
0.7000
ZS MLM
0.6821
0.6480
0.6718
0.6785
0.6785
0.6995
0.6118
WUL
ZS
0,8800
0.6698
0.5561
0.2945
0.5469
0.5556
0,4960
ZS
MLM
0.6093
0.6765
0.6708
0.6765
0.6732
0.6675
0.6765
ALL
7S
0.8085
0.7156
0.6308
0.3627
0.6214
0.6127
0.6008
ZS MLM
0.6662
0.6711
0.6637
0.6716
0.6716
0.6721
0.6419
Table 4: Zeoo (ZS) and adapted zero-shot (ZS MLM) cross- cross-linuaal transfer performance with mBERT on domains (GAO, TRAC, WUL) and the union of all domains (AIL). Results are reported in terms of Fl and numbers in bold indicate those that are significantly better for each language and domain (a = .05).
GAO
EN
DE
RU
TR
HR
SQ
ES
%P
%N
%P
%N
%P
%o
%P
%N
%P
%N
%P
%N
%P
%N
GAO
83.3
60
76.1
61.7
85.7
77.7
100
80.7
81.2
86.4
85.0
88.8
100
97.1
TRAC
81.2
74.6
82.3
83.8
91.8
88.8
88.1
92.9
87.5
80.7
100
90.5
69.2
89.3
WUL
85.7
83.8
62.2
78.0
56.8
63.8
74.5
89.3
88.3
96.1
99.3
98.8
96.7
98.8
ALL
87.2
83.9
76.6
84.1
100
100)
100
88.4
88.9
94.5
99.5
99.4
83.9
93.3
Table 5: Percentage of non-abusive texts that are well-classified with zero-shot transfer misclassified with the MLM adaptation (%N), and percentage of abusive texts that are misclassified with zero-shot transfer and well-classified with the MLM adaptation (%P).
few-shot samples. First, we explored two existing techniques: 1) SSMBA, which is based on pair functions to corrupt and reconstruct texts, and 2) MIXUP, which generates new samples from a lin- ear combination of original instances pairs. Then, we proposed MIXAG, variant of MIXUP, to parameterize the combination of instances with the angle between them. Our experiments were based on the multidomain and multilingual dataset XHATE-999, which allowed resource languages as target languages and English as the base language. This dataset contains six dif- ferent languages, and we extended to Spanish, following the same methodology that was used to generate the texts of the other languages. The re- sults showed the effectiveness of VRM-based tech- niques to improve few-shot cross- cross-lingual transfer learning in most domains and languages. Particu- larly, we observed that multilingual MIXAG out- performs the other strategies in the heterogeneous set (multidomain) for target languages. At the same time, we observed that structural language similarity does not seem to be highly correlated with cross-lingual transfer success in none of the strategies. These results are encouraging for abu- sive language detection in low-rew resource settings, as
the strategies that we have examined appear be consistent across languages.
Finally, we evaluated a scenario where it is not possible to perform a few-shot cross-lingual trans- fer due to the lack of supervised information. We used strategy based on masked language mod- cling and saw degradation in the class of non- abusive texts, but a gain in the class of abusive texts, reducing false negatives.
explore low-
In future work, aim to further examine proposed VRM-based technique for data augmen- tation. MIXAG uses as a parameter the angle be- tween the new instance and one of the original instances being combined. In our experiments, we fixed the angle as half the angle between the origi- nal instances, but we consider that the flexibility of varying that parameter must be exploited.
Limitations and Ethical Concerns
Our experiments relied on dataset that only tains English texts in the training and development sets. Only the test set is multilingual. Therefore, we were forced to partition the test set in order to perform the few-shot cross-lingual transfer and domain adaptation experiments. We compared the

---

results obtained in zero-shot cross- cooss-lingual transfer with the original test set and with the subset used in our experiments. We did not observe statistical differences. However, this may be a limitation in comparing our results with the original results re- ported in the dataset paper. Moreover, we observed a limitation in the strategy of domain adaptation. As we discussed in the error analysis, although the class of abusive texts is favoured with this strategy, we observed a detriment the negative class.
Elisa Bassignana, Valerio Basile, Viviana Patti, et al. 2018. Hurtlex: A multilingual lexicon of words to hurt. In CEUR Workshop Proceedings, volume 2253, pages 1-6. CEUR-WS.
Markus Bayer, Marc-André Kaufhold, and Christian Reuter. 2022. A survey on data augmentation for text classification. ACM Computing Surveys, 15(7):1-39.
Tommaso Caselli, Arjan Schelhaas, Marieke Weultjes, Folkert Leistra, Hylke van der Veen, Gerhen Timmer- man, and Malvina Nissim. 2021. DALC: the Dutch abusive language corpus. In Proceedings of the 5th Workshop on Online Abuse and Harms (WOAH 2021). pages 54-66, Online. Association for Computational Linguistics.
This work aims to improve abusive language detection in low-resource languages. While this can be useful for many languages, there are certain ethical implications. Therefore, we strongly rec- ommend not using the proposed strategies as the sole basis for decision-making in abusive language detection. Regarding the issue of privacy, all the data we use in our experiments, both the original dataset and the new texts in Spanish that we gener- ated, are publicly available. It should be noted that the scope of this work is strictly limited to the eval- uation of models that are also publicly available, and it is not used to promote abusive language with the information obtained.
Galo Castillo-López. Arij Riabi, and Djané Seddah. 2023. Analyzing zero-shot transfer scenarios across spanish variants for hate speech detection. In Tenth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2023), pages 1-13.
Olivier Chapelle, Jason Weston, Léon Bottou, and Vladimir Vapnik. 2000. Vicinal risk minimization. Advances in neural information processing systems. 13.
Jiaao Chen, Derek Tam, Colin Raffel, Mohit Bansal, and Diyi Yang. 2023. An empirical survey of data augmentation for limited data learning in nlp. Trans- actions of the Association for Computational Linguis- tics, 11:11-2211.
Acknowledgements
FairTransNL.P
research
project funded by and by
(PID2021-1243610BBCC) MCIN/AEE/10.11003/550 00009//50100001110 10001
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettle- moyer, and Veselin Stoyanov. 2020. Unsupervised cross lingual representation learning at scale. In Pro- ceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 8440- 8451, Online. Association for Computational Lin- guistics.
ERDE, EU A way of making Europe.
References
Fatimah Alkomah and Xiaogang Ma. 2022. A literature review of textual hate speech detection methods and datasets. Information, 13(6):273.
Michele Corazza. Stefano Menini, Elena Cabrio, Sara Tonelli, and Serena Villata. 2020. Hybrid emoji- based masked language models for zero-shot abusive language detection. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 943-949, Online. Association for Computational Lin- guistics.
Antreas Antoniou and Amos Storkey. 2019. Assume, augment and learn: Unsupervised few-shot meta- learning via random labels and data augmentation. arXiv preprint arXiv: 1902.09884
Farid Arthaud, Rachel Bawden, and Alexandra Birch. 2021. Fhw-shot learning through contextual data aug- mentation. In Proceedings of the 16th Conference of the European Chapter of the Association for Compu- tational Linguistics: Main Volume, pages (049-1062, Online. Association for Computational Linguistics.
Haixing Dai, Zhengliang Liu, Wenxiong Liao, Xiaoke Huang, Zihao Wu, Lin Zhao, Wei Liu, Ninghao Liu, Sheng Li, Dajiang Zhu, et al. 2023. ChatAug: Lever- aging ChatGPT for Text Data Augmentation. arXiv preprint arXiv:2302. 13007.
Agathe Balayn, Jie Yang, Zoltan Szlavik, and Alessan- dro Bozzon. 2021. Automatic identification of harm- ful, aggressive, abusive, and offensive language on the web: survey of technical biases informed by psychology literature. ACM Transactions on Social Computing (TSC), 4(3):1156.
Mithun Das, Somnath Banerjee, and Animesh Mukher- jee. 2022. Data hootstrapping approaches to improve low resource abusive language detection for indic lan- guages. In Proceedings of the 33nd ACM Conference on Hypertext and Social Media, pages 32-42.

---

Jacob Devlin, Ming-Wor Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Tech- nologies, Volume 1 (Log and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Marwa Khairy, Tarek M Mahmoud, and Tarek Abd-El- Hafeez. 2021. Automatic detection of cyberbully- ing and abusive language in arabic content on social networks: survey, Procedia Computer Science, 189:156-166.
Diederik Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd Inter- national Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.
Thomas G Dietterich. 1998. Approximate statistical tests for comparing supervised classification learning algorithms. Neural computation, 10(7):195519255
SV Kogilavani, Malliga, KR Jaiabinaya, M Malini, and M Manisha Kokila. 2021. Characterization and mechanical properties of offensive language taxon- omy and detection techniques. Materials Today: Pro- ceedings.
Juuso Eronen, Michal Ptaszynski, Fumito Masui, Masaki Arata, Gniewosz Leliwa, and Michal Wroczynski. 2022. Transfer language selection for zero-shot cross-lingual abusive language de- tection. Information Processing & Management, 1024881.
João Augusto Leite. Diego Silva, Kalina Bontcheva, and Carolina Scarton. 2020. Toxic language detec- tion in social media for Brazilian Portuguese: New dataset and multilingual analysis. In Proceedings of the 1st Conference of the Asia-Pacifie Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Lan- guage Processing, pages 4-424, Suzhou, China. Association for Computational Linguisties.
Steven Y Feng, Varun Gangal, Jason Wei, Sarath Chan- dar, Soroush Vosoughi, Teruko Mitamura, and Ed- uard Hovy. 2021. A survey of data augmentation ap- proaches for nlp. arXiv preprint wrXiv:2105.03075.
Rodrigo Fernandes de Mello, Moacir Antonelli Ponti, Rodrigo Fernandes de Mello, and Moacir An- tonelli Ponti. 2018 Statistical learning theory. Ma- chine Learning: Practical Approach on the Statis- tical Learning Theory, pages 75-128.
Patrick Littell, David R. Mortensen, Ke Lin, Katherine Kairis, Carlisle Turner, and Lori Levin. 2017. URIEL and lang2vec: Representing languages as typological, geographical, and phylogenetic vectors. In Proceed- ings of the 15th Conference of the European Chap- ter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 8-14, Valencia, Spain. Association for Computational Linguistics.
Anderson Almeida Firmino, Cláudio Souza de Bap- tista, and Anselmo Cardoso de Paiva. 2021. Using cross lingual learning for detecting hate speech in portuguese. In Database and Expert Systems Appli- cations: 32nd International Conference, DEXA 2021, Virtual Event, September 27-30, 2021, Proceedings. Part II. pages 170-175. Springer.
Errol Mamani-Condori and José Ochoa-Luna. 2021. Aggressive language detection using vgen-bert for spanish texts. In Intelligent Systems: 10th Brazilian Conference, BRACIS 2021, Virtual Event, November 29-December 3, 2021, Proceedings, Part II 10, pages 359-373. Springer.
Goran Glavaš, Mladen Karan, and Ivan Vulić. 2020. XHate-999; Analyzing and detecting abusive lan- Proceed-
guage across domains and languages. ings of the 28th International Conference on Com- putational Linguistics, pages 6350-6300000 6365, Barcelona, Spain (Online). International Committee on Compu- tational Linguistics.
Marzich Mozafari, Reza Farahbakhsh, and Noel Crespi. 2022. Cross-lingun few-shot hate speech and offen- sive language detection using meta learning. IEEE Access, 10: 14880-14896.
Niklas Muennighoff, Thomas Wang. Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng Xin Yong, Hai- ley Schoelkopf, Xiangru Tang, Dragomir Radev. Alham Fikri Aji. Khalid Almubarak, Samuel Al- banie, Zaid Alyafeai, Albert Webson, Edward Raff, and Colin Raffel. 2023. Crosslingual generaliza- tion through multitask finetuning. In Proceedings of the 6 Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 15991-16111, Toronto, Canada. Association for Computational Linguistics.
Archika Jain and Sandhya Sharma. 2022. A survey on identification of hate speech social media post. 2022 3rd International Conference on Computing. Analytics and Networks (ICAN), pages 1-6. IEEF.
Kamil Kanelerz, Alieja Figas, Marcin Gruza, Tomasz Kajdanowicz, Jan Kocon, Daria Puchalska, and Prze- myslaw Kazienko. 2021. Controversy and confor- mity: from generalized to personalized aggressive- ness detection. In Proceedings of the 59th Annual Meeting of the Association for Computational Lin- guistics and the 11th International Joint Conference on Natural Language Processing (Volume I: Long Pupers), pages 5915-5926, Online. Association for Computational Linguistics.
Arianna Muti, Francesco Fernicola, and Alberto Barrón- Cedeño. 2022. Misogyny and aggressiveness tend to come together and together we address them. In Pro- ceedings of the Thirleenth Language Resources and

---

Evaluation Conference, pages 4142-4148, Marseille, France. European Language Resources Association,
Michal Ptaszynski, Agata Pieciukiewicz. and Pawet Dy- bata. 2019. Results of the poleval 2019 shared task 6: First dataset and open shared task for automatic cyberbullying detection in polish twitter. Warszawa: Institute of Computer Sciences. Polish Academy of Sciences.
Matthew A Napierala. 2012. What is the bonferroni correction? Aaos Now, pages 40-41.
Nathan Ng, Kyunghyun Cho, and Marzyeh Ghassemi. 2020. SSMBA: Self-supervised manifold based data augmentation for improving out-of-domain robust- ness. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1268 1283, Online. Association for Computational Linguistics.
Tharindu Ranasinghe and Marcos Zampieri. 2020. Mul- tilingual offensive language identification with cross- lingual embeddings. In Proceedings of the 2020 Con- ference on Empirical Methods in Natural Language Processing (EMNLP), pages 5838-5844, Online. As- sociation for Computational Linguistics.
Debora Nozza. 2021. Exposing the limits of zero-shot cross-lingual hate speech detection. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 1thh International Joint Conference on Natural Language Processing (Volume Short Pupers), pages 907-914, Online. Association for Computational Linguistics.
Julian Risch, Anke Stoll, Lena Wilms, and Michael Wie- gand. 2021. Overview of the germeval 2021 shared task on the identification of toxic, engaging, and fact- claiming comments. In Proceedings of the GermEval 2021 Shared Task on the Identification of Toxic, En- gaging, and Fact-Claiming Comments, pages 1-12.
Sebastián E Rodríguez, Héctor Allende-Cid, and Héctor Allende. 2021. Detecting hate speech in cross-lingual and multi-lingual settings using language agnostic representations. In Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications: 25th Iberoamerican Congress, CIARP 2021, Porto, Portugal, May 10-13, 2021, Revised Selected Papers 25, pages 77 17- Springer.
Endang Wahyu Pamungkas, Valerio Basile, and Viviana Patti. 2023. Towards multidomain and multilingual abusive language detection: a survey, Personal and Ubiquitous Computing, 27(1):17-43.
Endang Wahyu Pamungkas and Viviana Patti. 2019. Cross-domain and cross-lingual abusive language de- tection: A hybrid approach with deep learning and multilingual lexicon. In Proceedings of the 57th
Hugo Rosa, Nádia Pereira, Ricardo Ribeiro, Paula Costa Ferriir,, Joao Paulo Carvalho, Sofia Oliveira, Luísa Coheur, Paula Paulino, AM Veiga Simāo, and Isabel Trancoso. 2019. Automatic ey cyberbullying detection: A systematic review. Computers in Haman Behavior, 93:333-345.
Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, pages 363- 370, Florence, Italy. Association for Computational Linguistics.
Chongyu Pan, Jian Huang, Jianxing Gong, and Xing- sheng Yuan. 2019. Few-Shot Transfer Learning for Text Classification with Lightweight Word Embed- ding Based Models. JEEE Access, 7:33296-53304.
Andy Rosenbaum, Saleh Soltan, Wael Hamza, Marco Damonte, Isabel Groves, and Amir Saffari. 2022. CLASP: Few-shot cross- looss-l data augmentation for semantic parsing. In Proceedings of the 2nd Con- ference of the Asia Pacific Chapter of the Association for Computational Linguistics and the 12th Interna- tional Joint Conference on Natural Language Pro- cessing (Volume 2: Short Papers), pages 444-462, Online only. Association for Computational Linguis- tics.
Sinno Jialin Pan and Qiang Yang. 2010. A survey on transfer learning. IEEE Transactions on knowledge and data engineering, 22(10))
Mei- hoig Pana, Hongyi Xin, and Hongbin Shen. 2023. Semantic transformation-basee. data augmentation for few-shot learning. Available at SSRN 4321351.
Fabian David Schmidt, Ivan Vulié, and Goran Glavaš. 2022. Don't 't stop fine-tuning: On training regimes for few-shot cross- loonsll transfer with multilingual language models. In Proceedings of the 2022 Con- ference on Empirical Methods in Natural Language Processing, pages 10725-10742 10742, Abu Dhabi, United Arab Emirates. Association for Computational Lin- guisties.
Archit Parnami and Minwoo Lee. 2022. Learning from few examples: A summary of approaches to shw- learning. arXiv preprint arXiv:2203.044 044091.
Fabio Poletto, Valerio Basile, Manuela Sanguinetti, Cristina Bosco, and Viviana Patti. 2021. Resources and benchmark corpora for hate speech detection: systematic review. Language Resources and Evalua- tion, 55:477-523.
Philip Sedgwick. 2012. Pearson's correlation coeffi- cient. Bmj, 345.
Rahul Pradhan, Ankur Chaturvedi, Apma Tripathi, and Dilip Kumar Sharma. 2020. A review on offensive language detection. Advances in Data and Informa- tion Sciences: Proceedings of ICDIS 2019, pages 433-439.
Xiayang Shi, Xinyi Liu, Chun Xu, Yuanyuan Huang. Fang Chen, and Shaolin Zhu. 2022. Cross-lingual offensive speech identification with transfer learning for low-resource languages. Computers and Electri- cal Engineering, 101:108005.

---

Oleh Shliazhko, Alena Fenogenova. Maria Tikhonova, Vladislav Mikhailov, Anastasia Kozlova, and Tatiana Shavrina. 2022. mgpt: Few-shot learners go multilin- gual.
Xin Zhang, Miao Jiang. Honghui Chen, Chonghao Chen, and Jianming Zheng. 2022. Cloze-style data augmentation for few-shot intent recognition. Mathe- matics, 10(18))335..
Elena Shushkevich and John Cardiff. 2019, Automatic misogyny detection in social media: A survey. Com- putación y Sistemas, 23(4): 159-1164.
Mengjie Zhao, Yi Zhu, Ehsan Shareghi, Ivan Vulić, Roi Reichart, Anna Korhonen, and Hinrich Schützee 2021. A closer look at few-shot crosslingual transfer: The choice of shots matters. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer- ence on Natural Language Processing (Volume 1: Long Papers), pages 5751 3751-5767, Online. Association for Computational Linguistics.
Gudbjartur Ingi Sigurbergsson and Leon Derezynski. 2020. Offensive language and hate speech detec- tion for Danish. In Proceedings of the Twelfth Lan- guage Resources and Evaluation Conference, pages 3498-3508, Marseille, France. European Language Resources Association.
Jing Zhou, Yanan Zheng. Jie Tang, Li Jian, and Zhilin Yang. 2022. FlipDA: Effective and robust data aug- mentation for few-shot learning. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume Long Papers). pages 8646-8665, Dublin, Ireland. Association for Computational Linguistics.
Levent Soykan, Cihan Karsak, Ilknur Durgar Elkahlout, and Burak Aytan. 2022. A comparison of machine learning techniques for Turkish profanity detection. Proceedings of the Second International Workshop on Resources and Techniques for User Information in Abusive Language Analysis, pages 16- 24, Marseille, France. European Language Resources Association.
Haris Bin Zia, Ignacio Castro, Arkaitz Zubiaga, and Gareth Tyson. 2022. Improving zero-shot cross- lingual hate speech detection with pseudo-lahel fine- tuning of transformer language models. In Proceed- ings of the International AAAI Conference on Web and Social Media, volume 16, pages 1435-1439.
Lukas Stappen, Fabian Brunn, and Bjöm Schuller. 2020. Cross-lingual zero-and few-shot hate speech detec- tion utilising frozen transformer language models and axel. arXiv preprint 2Xiv:2004.13855
Lichao Sun, Congying Xia, Wenpeng Yin, Tingting Liang Philip Yu, and Lifang He. 2020. Mixup- transformer: Dynamic data augmentation for NLP tasks. In Proceedings of the 28th International Con- ference Computational Linguistics, pages 3436- 3440, Barcelona, Spain (Online). International Com- mittee on Computational Linguistics.
Jian Yuan Wang, KeXin Liu, YuCheng Zhang, Biao Leng. and JinHu Lu. 2023. Recent advances of few-shot learning methods and applications. Science China Technological Sciences, pages 1-25.
Yaqing Wang, Quamming Yao, James T Kwok, and Li- onel M Ni. 2020. Generalizing from few examples: A survey on few-shot learning, ACM computing sur- veys (csur), 33(3):1-344
Genta Winata, Shijic Wu, Mayank Kulkarni, Thamar Solorio, and Daniel Preotiuc-Pietro 2022. Cross- lingual few-shot learning on unseen languages. In Proceedings of the 2nd Conference of the saia-Pccific Chapter of the Association for Computational Lin- guistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 777-791, Online only. Association for Computational Linguistics.
Wenjie Yin and Arkaitz Zubiaga. 2021. Towards gener- alisable hate speech detection: review obstacles and solutions. PeerJ Computer Science, 7:e598.
Hongyi Zhang, Moustapha Cissé, Yann N. Dauphin, and David Lopez-Pat Paz. 2018, mixup: Beyond empirical risk minimization. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC. Canada, April 30- May 3, 2018 Conference Track Proceedings. OpenReview.net.

---

A Reproducibility
2x
Table 6 provides features and links to the pre- trained models that we use, and Table 7 illustrates details of the dataset.
x
0
8
Model:
MOERT
Vocub size:
120k
#Params:
177M
Link:
hmgps//hggggngggoe.... bert bere- muhtilingull-cseee
Figure 3: MIXAG explanation.
SSMBA
Esperimenta (sen-shot and low-shut crese cnse-lingual tmurdor)
HIGPT
Vocab size:
10IK
vectors and angles. Then, À can be expressed function of 0 as Equation 12. Finally, using the known identities in Equations 13, we can define À from the cosine of which can be obtained with Equation 10, and the cosine of 0, which is the parameter of MIXAG (Equation 14).
#Params: 1417
Use this work:
MIXUP & MIXAG
Vorab size:
250k
270M
Link:
hilps hwtps//wwgggiggggg www.gi/ggaaaoooooo loneheett--eee
Use this work:
Ablation studles
Model:
m1o
Vocab size:
250k
#Params:
Link:
Al|a,
||,
(11)
Ablation Mudics
sin(o 0)
sén(()
Table Features of the models used in this work. We built models directly top of the HuggingFace Transformers library.
1x, ((sin(a-0)
A
(12)
sin(a 0) 0)=sin(()cos(() cos(a)sin(()
Domain
Train (EN)
Validation (EN)
Test (LANG)
=in(0)=/1-oxxx sin((r)= /i-om((a
GAO
919
218
99
sin(a 0)= coo(a))eos(()) eas(a)) co0(
TRAC
10,341
2,593
(13)
WUL
71,754
24,130
600
ALL
83,014
26,941
999
A= Mk (lllll(co()))) //t/////0//-////// -coo-P -1oolo)/1--oo/// (0))))
(14)
Table 7: Number of texts in each set of the XHate-999 dataset. LANG stands for each language in {EN, SQ, HR. DE, RU, TR. ES}.
Results by Language and Domain
We show complete results in this section. Table 8 illustrates that there is no significant difference between using the full test set and using subset of texts from the test set (the subset that we used in our experiment).
B MIXAG Details
MIXAG is a particular case of MIXUP where the parameter X of the linear combination (Equation 9) is determined by the angle a between the original vectors 2; and Tj. well as the angle 0 between of the original vectors.
Table 9 illustrates the cosine similarity between each language and English for five linguistic fea- tures. We obtain these features as language vectors from LANG2VEC (Littell et al., 2017).
the new vector è and We take 2; without loss of generality (see Figure 3). We rely on the cosine of a, calculated as Equation 10, where || denotes the norm of a vector. No- tiee that we only parameterize one of the original vectors, since and 0 are sufficient to determine é.
Table 10 shows the correlation coefficient and p-value for these linguistic features.
SYN: vectors encode syntactic properties,
e.g., if subject appears before or after verb.
f Ax + Tjj
(9)
FAM: vectors encode memberships in lan-
guage families.
(os = | ||||||
2,6
(10)
vectors denote the presence or absence of natural classes of sounds.
INV:
The objective is express the parameter as a function of 0, hence we take advantage of the Law of Sines (Equation 11) that allows relating
PHO: vectors encode phonological proper-
ties such as the consonant-vowel ratio.

---

GAO
EN
11R
SQ
ES
FZS
0.6742
0.5185
0.50033
0.5221
0.6088
0.54432
0.5060
TRAC
FZS
0.7894
0.7527
0.8859
0.8806
0.6925
0.5177
0.7045
0.7000
WUL
FZS
0.8812
116739
0,5581
0,2969
0.5476
0.5675
0.501
0,8800
0.0098
0.5061
0.5469
0.5606
0.4960
ALL
FZS
0.8053
1.71446
0.6322
0.3565
0.6251
0.6088
0.60228
0,8085
0.7156
0.6308
0.3627
0.0008
Table
Cross-lingual transfer performance with mBERT on each domain (GAO, TRAC, WUL) and the union of all domains (ALL). FZS refers to zero-shot cross-lingual transfer with the full test set, which corre- sponds the results reported in XHATE 999: Analyz- ing and Detecting Abusive Language Across Domains and Languages. transfer with 90% of the test set of each language, which corresponds to the results discussed in this paper. Re-
refers to zero-shot cross-lingual
sults reported in terms of F1.
DE
RU
SQ
SYN
0.9025
0.8118
0.5067
0.8318
0.7959
0.8216
FAM
1.0
0.5443
0.1667
0.0
0.1260
0.3333
0.0962
INV
1.0
0.7628
0.6475
0.6658
0.6967
0.7249
0.6382
PHO
0.8058
0.8581
0.8181
0.8581
D.8704
0.8581
0.9976
0.9950
0.9919
Table 9: Cosine similarity between each language vec- tor and English vector for LANG2VEC-based language vectors (SYN, FAM. INV) considering all domains.
GEO: vectors express orthodromic distances for languages w.r.t. fixed points on the Earth's 's surface.
Table 11 shows the Precision and Recall results across domains and languages for the error analysis of the unsupervised language adaptation.
Table 12 shows the results for all variants across languages and domains.

---

SYN
FAM
INV
PHO
GEO
Pearson
P-value
Pearson
P-value
Pearson
P-value
Pearson
P-value
Pearson
P-value
ZS
0.672
<.001
0.647
<.001
0.599
<.001
0.529
.003
0.302
.119
FS
0.664
<.001
0.661
<.001
0.607
<.001
0.516
.004
0.289
.166
SS
0.527
.004
0.627
<,001
0.608
<.001
0.486
.008
0.261
180
MU
0.405
.033
0.628
<.001
0.633
<.001
0.463
.013
0.315
.100
MA
0.571
.001
0.721
<.001
0.686
<.001
0.529
.004
0.245
.209
Table 10: Complete table of correlations between zero-shot (ZS), few-shot (FS), few-shot with SSMBA (SS), few- shot with MIXUP (MU) and few-shot with MIXAG (MA) cross-lingual transfer performance with mBERT across all languages and domains, with linguistic proximity scores (features SYN, FAM, INV, PHO, GEO). Correlations that are not statistically significant are underlined (a=.05).
GAO
EN
DE
RU
TR
HR
SQ
ES
R
P
R
P
R
P
R
P
R
P
R
P
R
P
7S
0.70
0.65
0.48
0.54
0.48
0.58
0.55
0.48
0.60
0.65
0.50
0.59
0.50
0.55
ZS_MLM
0.80
0.47
0.88
0.50
0.78
0.50
0.98
0.47
0.98
0.46
0.93
0.45
0.98
0.44
TRAC
ZS
0.89
0.67
0.88
0.66
0.74
0.63
0.71
0.65
0.78
0.62
0.85
0.63
0.73
0.67
MSMMLM
0.92
0.54
0.81
0.54
0.92
0.53
0.93
0.53
0.93
0.53
0.98
0.54
0.72
0.53
WUL
ZS
0.80
0.98
0.51
0.97
0.39
0.94
0.17
0.96
0.38
0.97
0.40
0.92
0.33
0.97
ZS 7S_MLM
0.83
0.48
0.89
0.49
0.99
0.51
0.98
0.51
0.99
0.51
0.97
0.51
0.99
0.51
ALL
ZS
0.79
0.82
0.71
0.72
0.57
0.71
0.24
0.75
0.56
0.69
0.54
0.71
0.51
0.74
MLMM
0.99
0.50
0.99
0.51
0.98
0.50
0.96
0.50
0.99
0.51
0.99
0.51
0.89
0.50
Precision (P) and Recall (R) in cross-lingual transfer with mBERT on each domain (GAO, TRAC, WUL) and the union all domains (ALL). ZS_ ZSMMLM refers to adapted zero-shot cross-lingual transfer, while refers shot cross- cosss-lnguaal transfer.
Table 11:

---

GAO
EN (n)
DE (n)
RU (n)
TR (n)
HR (n)
SQ (n)
ES (n)
ZS
0.6747
0.5067
0.5205
0.5116
0.6234
0.5405
0.5263
FS
0.6500
0.5455
0.5455
0.5556
0.6000
0.5385
0.5385
SS
0.7000
0.6667
0.5952
0.6400
0.6452
0.6598
0.6024
SS-HL
0.6667
0.6737
0.6136
0.6408
0.6588
0.6667
0.6279
MU
0.6742
0.6316
0.5806
0.6481
0.6331
0.6742
0.6444
MMU
0.6667
0.6316
0.5500
0.6355
0.6588
0.6882
0.6471
MU-SS
0.6591
0.6122
0.5926
0.6304
0.6444
0.6667
0.5333
MA
0.6882
0.6214
0.6304
0.6364
0.6292
0.6667
0.6408
MMA
0.6889
0.6526
0.6383
0.6549
0.6667
0.6517
0.6517
MA-SS
0.7033
0.6408
0.5977
0.6471
0.6444
0.6465
0.5278
TRAC
ZS
0.7642
0.7582
0.6815
0.6777
0.6892
0.
0.7000
FS
0.7462
0.7561
0.6861
0.6620
0.6731
0.7183
0.6782
SS
0.7303
0.7208
0.6775
0.6411
0.6783
0.6452
0.6444
SS-HL
0.7261
0.7124
0.6776
0.6282
0.6736
0.6498
0.6544
MU
0.7613
0.7576
0.6923
0.7103
0.7103
0.7290
0.7003
MMU
0.7596
0.7508
0.6894
0.6906
0.6796
0.7148
0.7043
MU-SS
0.7403
0.7325
0.7010
0.6709
0.7162
0.6220
0.6894
MA
0.7516
0.7622
0.6901
0.6885
0.6801
0.7093
0.7047
MMA
0.7516
0.7622
0.6943
0.6885
0.6935
0.7066
0.7047
MA-SS
0.7222
0.7190
0.6554
0.6596
0.6567
0.6596
0.4909
WUL
ZS
0.8800
0.6698
0.5561
0.2945
0.5469
0.5556
0.4960
FS
0.8845
0.6761
0.5657
0.3193
0.5567
0.5833
0.5040
SS
0.9035
0.7821
0.7636
0.6620
0.7443
0.7490
0.7348
SS-HL
0.9014
0.7696
0.7059
0.6636
0.7007
0.7043
0.7043
MU
0.9035
0.7576
0.6605
0.6333
0.5950
0.5650
0.6357
MMU
0.9006
0.7495
0.6574
0.6029
0.6394
0.5985
0.5802
MU-SS
0.9077
0.7776
0.7140
0.6800
0.7245
0.7071
0.7495
MA
0.8822
0.7406
0.6758
0.5821
0.6344
0.6230
0.5277
MMA
0.8911
0.7350
0.6605
0.5260
0.6682
0.6502
0.5656
MA-SS
0.9014
0.7911
0.7149
0.6787
0.7315
0.7505
0.7228
ALL
ZS
0.8085
0.7156
0.6308
0.3627
0.6214
0.6127
0.6008
FS
0.8112
0.7141
0.6329
0.4063
0.6316
0.6238
0.6130
SS
0.8077
0.7253
0.7071
0.6568
0.6965
0.6990
0.6838
SS-HL
0.8097
0.7273
0.6987
0.6689
0.6725
0.6909
0.6973
MU
0.8102
0.7404
0.7013
0.6740
0.7116
0.7001
0.6878
MMU
0.8284
0.7500
0.7312
0.7113
0.7371
0.7128
0.7250
MU-SS
0.8176
0.7531
0.7233
0.6839
0.7186
0.6881
0.7087
MA
0.8083
0.7245
0.6757
0.5616
0.6710
0.6788
0.6508
MMA
0.8237
0.7585
0.7392
0.7224
0.7523
0.7344
0.7476
MA-SS
0.8096
0.7229
0.7193
0.6369
0.6759
0.6734
0.67 13
Table 12: Zero-shot (7S) and few-shot (FS) cross-lingual transfer performance with mBERT on each domain (GAO. TRAC, WUL) and the union of all domains (ALL). We also show variants for FS: 1) SSMBA (SS) and 2) SSMBA with HurtLex (SS-HL), 3) MIXUP (MU), multilingual MIXUP (MMU), 5) MIXUP with SSMBA (MU-SS), 6) MIXAG (MA), 7) multilingual MIXAG (MMA), 8) MIXAG with SSMBA (MA-SS). The results are reported in terms of and significantly better results are underlined each language and domain. Numbers bold indicate the best results. We Qailered .005 since we have 10 tests


Reviews:
[{"review_id": "vCklcCZIYh", "reviewer": "Reviewer_bbqD", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the Reviewer 1 for the reviews --  we appreciate the comments and hope the following replies may soothe some of the concerns raised:\n\n> Limited Comparison with Baselines: The paper does not provide a detailed comparison with existing baseline methods for data augmentation and domain adaptation in cross-lingual text classification. A thorough comparison with state-of-the-art techniques, such as XLM-R, would strengthen the validity and significance of the proposed approaches.\n\nVicinal risk minimization formalizes data augmentation, where a neighbourhood is defined around each example in the training data. Then, additional synthetic examples can be drawn from the vicinity distribution of training examples to increase the support of the training distribution. Most data augmentation techniques assume that the examples in the neighbourhood belong to the same class. \n\nMIXUP is a data augmentation strategy based on linear interpolations of feature vectors and their associated targets, which can be from different classes. This strategy has been shown to increase the robustness of neural networks and improve generalization. We start from these fundamentals and propose MIXAG, whose strengths have been observed in multidomain and multilingual environments. \n\nWith MIXAG as a data augmentation strategy, we would argue that the most meaningful baselines are other established data augmentation strategies. We thus use MIXUP, a highly popular and cited augmentation strategy for dense representation spaces, as a primary baseline. We also use SSMBA, another VRM technique that focuses on text processing. We believe that these are the most appropriate baselines. \n\nNote that all augmentation strategies -- our MIXAG, as well as the baselines -- operate on top of the same multilingual LM (mBERT). While we agree that repeating the experiments and findings on top of another multilingual LM such as XLM-R would make sense and strengthen our work, this does not mean that we do not compare against strong augmentation baselines (i.e., MIXUP).    \n\nPost-hoc, indeed, we considered XLM-R in our experiments. We observed that these results did not lead to significant differences in our conclusions (same trends as for mBERT). Therefore, given the limitation of space in the paper, we decided not to add these results. We will gladly add the results of both our MIXAG and the baselines on top of XLM-r, given the extra space for the camera ready version of the paper. \n\n> Lack of Extensive Ablation Studies: The paper does not conduct extensive ablation studies to analyze the individual contributions of data augmentation and domain adaptation to the overall performance improvement.\n\nMIXAG is a data augmentation method that randomly combines inputs and accordingly combines one-hot-label encodings.\nWith the possibility of increasing the size of the paper, we intend to add a discussion of different design options. However, we would like to emphasize that MIXAG is not a complex comosite approach for which ablations over individual components are obvious. \n\nSome of the options one could consider (again, these are not ablations in the traditional sense of ablating the components of a multi-component approach): \n  - Interpolate only between nearest neighbors instead of random selection, or only between inputs of the same class.\n  - Use different text representation strategies (e.g., from instruction fine-tuned generative multilingual models, e.g., mT0).\n  - When generating a new instance from two original inputs, we set the angle to half the angle between these original inputs. In the new section, we will add the results obtained when varying this parameter.\n\n> Interpretability and Analysis of MIXAG: The paper could include more in-depth analysis and explanation of the novel data augmentation method (MIXAG). Understanding how MIXAG affects the representations and contributes to the model's performance would add clarity and insights into the approach.\n\nIn this regard, we plan to include an illustration showing the representation of original instances and the synthetic instances created with MIXAG (i.e., to plot the topology of the representation space and see where the synthetic instances position themselves w.r.t. original ones). This will allow us to show how this strategy manages to match the statistics of the training set, especially in a multilingual setting. Note that in a data augmentation method, there is a desire for the new data to match the distribution of the training set. Using augmented data that deviates too much from the distribution of the training set can increase the training error in the original training set, which negatively affects generalization.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Strengths of the paper:\n\nNovel Data Augmentation Technique: The paper introduces a novel data augmentation method called MIXAG, which interpolates pairs of instances based on the angle of their representations. This approach provides a fresh perspective on data augmentation for cross-lingual abusive language detection.\n\nAddressing Resource Scarcity: The paper addresses the challenge of resource scarcity in target languages, which is a common issue in cross-lingual NLP. By utilizing data augmentation and domain adaptation, the work offers practical solutions for improving model performance in low-resource settings.", "reasons_to_reject": "While the paper has several strengths, it may also have some weaknesses that could be considered for further improvement:\n\nLimited Comparison with Baselines: The paper does not provide a detailed comparison with existing baseline methods for data augmentation and domain adaptation in cross-lingual text classification. A thorough comparison with state-of-the-art techniques, such as  XLM-R, would strengthen the validity and significance of the proposed approaches.\n\nLack of Extensive Ablation Studies: The paper does not conduct extensive ablation studies to analyze the individual contributions of data augmentation and domain adaptation to the overall performance improvement.\n\nInterpretability and Analysis of MIXAG: The paper could include more in-depth analysis and explanation of the novel data augmentation method (MIXAG). Understanding how MIXAG affects the representations and contributes to the model's performance would add clarity and insights into the approach.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper focuses on cross-lingual abusive language detection and addresses the challenge of resource scarcity in target languages. The main contributions are:\n\nData Augmentation: The paper proposes a novel data augmentation method called MIXAG, which interpolates pairs of instances based on the angle of their representations. This technique, along with analyzing existing vicinal risk minimization-based methods, enhances cross-lingual abusive language detection.\n\nDomain Adaptation: The researchers utilize continual pre-training for domain adaptation, aiming to improve model performance in diverse linguistic and domain contexts.\n\nOverall, the paper's contributions lie in proposing MIXAG, leveraging data augmentation and domain adaptation to enhance cross-lingual abusive language detection. The results validate the effectiveness of the techniques in addressing resource scarcity and improving performance across diverse linguistic and domain settings.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "EUiE8lxFv9", "reviewer": "Reviewer_zEh7", "summary": "", "questions": "1. Why VRM? There are many other data augmentation methods of similar nature.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the Reviewer 2 for reviewing our paper. We appreciate your comments and hope the following comments may clarify your concerns:\n\n1. The VRM principle is an empirical risk minimization variant based on vicinal functions. There is strong evidence of improvements with VRM in terms of generalization with vicinal functions. MIXUP is a popular choice of vicinal distribution that introduces globally linear behaviour between training examples and improves the generalization performance of models. Many works have shown that models trained on MIXUP are relatively robust to input perturbations. Motivated by these findings, we aim to leverage this VRM technique to address cross-lingual transfer in abusive language detection with scarce data. At the same time, we propose and evaluate MIXAG, a variant of MIXUP, which shows improvements in a multidomain and multilingual environment.\n\n2. We plan to include a section on ablation studies in a new version of the article. Among the aspects to be considered is the representation of texts for the combination. Note that this will allow us to analyze the empirical basis of the chosen model, as the reviewer points out.\n\n3. We agree that it is clearer for the reader if the results are presented in a table with the numerical values for each experiment. In fact, our first intention was to present the results only in tables, but the large number of experiments made the table really large, as you can see in Appendix C. As a strategy, we decided to summarize the results in a graph that can help analyze the results by domain. Note that the main contributions of MIXAG are in the multidomain and multilingual environment. Therefore, we ensure that the results of greatest interest (multidomain) are detailed in Table 1.\nDue to space limitations, we included many detailed results in Appendix C. However, note that we summarized the most important content that supports our conclusions and refer to the specific places in the appendices for more details.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* The paper achieved decent performance improvements.\n* The paper experimented with topologically diverse languages.\n* The paper is mostly clear about its contributions, methods, and results.", "reasons_to_reject": "* Many of the decisions made in the experimental settings are well-motivated. For example, why VRM? Why GPT for the combination? These are choices that need to be made on either a theoretical or an empirical basis. \n* Some of the important findings are left in the appendices, like appendix C. This paper needs to be restructured. ", "questions_to_authors": "1. Why VRM? There are many other data augmentation methods of similar nature.", "paper_topic_and_main_contributions": "This paper proposed to use data augmentation and continual pre-training for domain adaptation to improve cross-lingual abusive language detection. Specifically:\n\n* The paper proposed MIXAG, a vicinal risk minimization-based data augmentation method.\n* Their experiments showed that MIXAG improves significantly in multi-domain and multi-lingual environments.", "excitement": 3, "reproducibility": 2, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Please don't use bar graphs. We only get a rough estimate of the F1 score in the bar graph. Make a table and put in the numbers."}, "extra_text": {}}, {"review_id": "tT7XboTIWj", "reviewer": "Reviewer_mJE9", "summary": "", "questions": "1. The answer to RQ3 depends on the assumption that MLM could be used as a technique to let any language be aware of the abusive terms used. Then why doesn’t it work for English? This either illustrated this assumption is flawed or it does not properly answer RQ3.\n2. It is difficult to understand whether MLM adaptation works better than zero-shot transfer. For some domain MLM works better, whereas in other domains zero-shot transfer works better. Can you clarify what are trying to entail by comparing them?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the Reviewer 3 for reviewing our work. We appreciate your comments and hope the following comments may clarify your concerns:\n\n> Quantitative results are not convincing to prove the superiority of the proposed MIXAG approach. Firstly, the author states that there is not much difference between existing VRM-based techniques which is also reflected in Figure\n\nWe propose MIXAG, a variant of MIXUP, whose main contribution is in the multidomain and multilingual environment. On the one hand, Table 1 illustrates the results in the multidomain environment, where we show that the multilingual MIXAG (MMA) improves significantly (underlined values) to other techniques. On the other hand, Figure 2 summarizes the results for each domain, where there are no significant differences between the VRM techniques, as noted by the reviewer. This is why we point out that there are no significant differences between the techniques.  \n\n> Across domains and languages, it is not conclusive whether MIXUP performs better than MIXAG or vice-versa.\n\nRQ3 focuses on the comparison of MLM domain adaptation against VRM techniques for cross-lingual transfer of abusive language detection. We found that VRM techniques performed better than zero-shot transfer, which we did not observe for MLM domain adaptation. For example, the case of English, cited by the reviewer, was always better with zero-shot transfer than with domain adaptation. Therefore, the conclusion for RQ3 is that VRM techniques seem to be more robust than MLM domain adaptation and consistently perform better across all languages. Note, that searching for strategies to improve MLM in each particular language is beyond the scope of this paper.\n\n> Authors say that Multilingual MIXAG outperforms all the baselines when union of all the domains are used. It is not clearly evaluated multilingual MIXAG outperforms other setups. Also, please clearly write what you intend to convey with Figure 2 and Table 1 respectively.\n\nThe comparison of MLM domain adaptation against zero-shot transfer in Table 3 is to show that MLM fails to improve zero-shot transfer in many cases. These results are used to analyze RQ3, as mentioned in the previous point.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper proposes an incremental method MIXAG over two existing data augmentation methods: SSMBA and MIXUP.\n2. The lambda parameter in MIXUP is selected using a beta distribution over a hyperparameter. In MIXAG, lambda is computed using the candidate data points and their angle and vicinity. This approach is simple and works well in practice.\n3. Extensive correlation analysis is performed to demonstrate that the used strategies work well across all the languages.", "reasons_to_reject": "1. Quantitative results are not convincing to prove the superiority of the proposed MIXAG approach. Firstly, the author states that there is not much difference between existing VRM-based techniques which is also reflected in Figure \n2. Across domains and languages, it is not conclusive whether MIXUP performs better than MIXAG or vice-versa.\n3. Authors say that Multilingual MIXAG outperforms all the baselines when union of all the domains are used. It is not clearly evaluated multilingual MIXAG outperforms other setups. Also, please clearly write what you intend to convey with Figure 2 and Table 1 respectively.", "questions_to_authors": "1. The answer to RQ3 depends on the assumption that MLM could be used as a technique to let any language be aware of the abusive terms used. Then why doesn’t it work for English? This either illustrated this assumption is flawed or it does not properly answer RQ3.\n2. It is difficult to understand whether MLM adaptation works better than zero-shot transfer. For some domain MLM works better, whereas in other domains zero-shot transfer works better. Can you clarify what are trying to entail by comparing them?", "paper_topic_and_main_contributions": "This paper proposes a new technique for cross-lingual abusive language detection. It exploits large available datasets in English to perform cross-lingual transfer across seven completely different languages from English and across three domains.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper focuses on cross-lingual few-shot abusive language detection and addresses the scarcity of target language data by data augmentation and further pre-training for domain adaptation. Based on the existing vicinal risk minimization methods (SSMBA, MIXUP), the authors propose a new data augmentation method, MIXAG, that uses the angle between text representations to interpolate them. Furthermore, they test continue pretraining on a few unlabelled examples and compare it with the zero-shot baseline. 

As mentioned by the reviewers, the authors introduce a novel data augmentation method MIXAG, and show its effectiveness with decent performance improvements in few-shot cross-lingual transfer especially in the multilingual scenario (Multilingual MIXUP). Although results from the multidomain setting (Figure 2) do not show superior performance, the authors contextualized these results in their rebuttal and pointed out the impact of their approach in multidomain and multilingual environments. 

I believe the additional content that the authors mentioned during the discussion period, such as XLM-R results and analysis on the representation of original and synthetic instances, will further improve the paper.
Decision: Accept-Main