Cultural Compass: Predicting Transfer Learning Success in Offensive Language Detection with Cultural Features
Authors: Li Zhou
Antonia Karamolegkou
Wenyu Chen
Daniel Hershcovich
Venue: EMNLP 2023 Findings (2023)
Keywords: culture
transfer learning
offensive language detection
TL;DR: Cultural value surveys effectively predict cross-cultural/cross-lingual transfer learning success in Offensive Language Detection

Abstract: The increasing ubiquity of language technology necessitates a shift towards considering cultural diversity in the machine learning realm, particularly for subjective tasks that rely heavily on cultural nuances, such as Offensive Language Detection (OLD). 
Current understanding underscores that these tasks are substantially influenced by cultural values, however, a notable gap exists in determining if cultural features can accurately predict the success of cross-cultural transfer learning for such subjective tasks. 
Addressing this, our study delves into the intersection of cultural features and transfer learning effectiveness. The findings reveal that cultural value surveys indeed possess a predictive power for cross-cultural transfer learning success in OLD tasks, and that it can be further improved using offensive word distance. 
Based on these results, we advocate for the integration of cultural information into datasets. 
Additionally, we recommend leveraging data sources rich in cultural information, such as surveys, to enhance cultural adaptability. 
Our research signifies a step forward in the quest for more inclusive, culturally sensitive language technologies.

Cultural Compass: Predicting Transfer Learning Success in Offensive Language Detection with Cultural Features
Li 2 Antonia Karamolegkou', Wenyu Chen', Daniel Hershcovich²
'University of Electronic Science and Technology of China
2 DDpaatmeent of Computer Science, University of Copenhagen
_i_z si_zhhu@std.. westot uestc. edu.cr en, an antkkëdi. ku.dk, .k, eyyuuescc.ed... cn, dhedi ku. .k
Abstract
and dataset. Most research studies predominantly rely on data-dependent and language-dependen features to predict the optimal language to transfer from (Lin et al., 2019; Sun et al., 2021). However, the same language is often spoken in vastly differ- ent cultures, and conversely, similar cultures may speak very different languages. In subjective tasks such as Offensive Language Detection (OLD), lying solely on measures of linguistic proximity and data-specific features for predicting the opti- mal transfer language/dataset may be insufficient. crucial to consider cultural differences.
The increasing ubiquity of language technology necessitates shift towards considering cultural diversity in the machine learning realm, par- ticularly for subjective tasks that rely heavily on cultural nuances, such as Offensive Lan- guage Detection (OLD), Current understand- ing underscores that these tasks are substan- tially influenced by cultural values, however, a notable gap exists determining if cultural features can accurately predict the success of cross-cultural transfer learning for such subjec- tive tasks. Addressing this, our study delves into the intersection of cultural features and transfer learning effectiveness. The findings re- veal that cultural value surveys indeed possess a predictive power for cross cross-cultura transfer learning success in OLD tasks and that can be further improved using offensive word dis- tance. Based on these results, we advocate for the integration of cultural information into datasets. Additionally, aging data sources rich in cultural information, such as surveys, to enhance cultural adaptabil- ity. Our research signifies step forward in the quest for more inclusive, culturally sensitive language technologies.
Cultural sensitivity and adaptation pose a great challenge in NLP tasks (Papadakis et al., 2022; Arango Monnar et al., 2022; Hershcovich et al.. 2022; Sittar and Mladenic, 2023). Despite the ca- pabilities of Language Models like ChatGPT, their cultural adaptation to culturally diverse human soci- cties is limited (Cao et al., 2023). In an increasingly globalized and AI-driven world, this gap needs be addressed to create and sustain an effective com- munication environment (Aririguzoh, 2022).
recommend lever-
Taking steps towards this direction, we focus on the success of cross-cultura transfer learning in OLD. Offensive language can vary greatly depend- ing on cultural backgrounds. While most multi- lingual OLD datasets are constructed by filtering a predefined list of offensive words (Zampieri al., 2019; Sigurbergsson and Derczynski, 2020; Jeong et al., 2022; Deng al., 2022), certain offen- sive words are culturally specific. For example, OLD models trained on American cultural con- texts may struggle to effectively detect offensive words like "m*adarchod" and "pr*sstitute" in In- dian texts (Ghosh et al., 2021; Santy et 2023). Previous studies (Nozza, 2021; Litvak et al., 2022; Zhou al., 2023) have highlighted the challenges of cross-lingual transfer in OLD, emphasizing the significance of accurately predicting the optimal language/dataset for transfer before the implemen- tation.
Warning: This paper discusses examples of offen- sive content. The authors do not support the of offensive language, nor any of the offensive repre- sentations quoted below.
1 Introduction
Cross-lingual transfer has been successful in var- ious NLP tasks (Plank and Agić, 2018; Rahimi et al., 2019; Eskander et al., 2020; Adelani et al., 2022) particularly in scenarios where low-resource languages lack sufficient training data (Lauscher et al., 2020). To ensure successful cross-lingual transfer, leveraging prior knowledge becomes cru- cial in selecting the most suitable transfer language
Coode is available in tht.///gttttttttee htt.g.......o......eee com/1izhou21/ cultural- compass..

---

Given this context, we propose six country- level features that quantify the variations in cul- tural values across different countries. Addition- ally, we introduce a feature that measures the dis- tance of offensive words between languages. We address three research questions regarding cross- cultural/cross-lnggu transfer learning for OLD:
are other linguistic properties that could lead better results such as phonological or syntactic distances (Karamolegkou and Stymne, 2021; ; Ero- nen et al., 2022). Other factors that can influence transfer language selection, such as lexical overlap, have shown mixed findings. Some studies report positive correlation with the cross-lingual model performance (Wu and Dredze, 2019; Patil et al., 2022; de Vries et al., 2022), while others do not support this finding (Pires al., 2019; Tran and Bisazza, 2019; Conneau et al., 2020b). To auto- mate the process of selecting transfer languages, there have been attempts using a ranking meta- model that predicts the most optimal languages (Lin et al., 2019; Lauscher et al., 2020; Srinivasan et al., 2021; Blazej and Gerasimos, 2021; Srini- vasan et al., 2022; Ahuja et al., 2022; Patankar al., 2022). These mostly rely on data-dependent and language-dependers features, without taking into account cultural background differences during the optimal transfer language prediction process.
RQ1. Can we predict transfer learning success?
RQ2. Which types features (cultural or offen- sive word distance) are most informative?
RQ3. Do the same features contribute to transfer learning success in OLD and other NLP tasks?
Our experiments show the substantial influence of integrating cultural values predicting language transfer for a culture-loaded task, OLD. We also find that exploring offensive word distance in spe- cific domains benefits the prediction of the optimal transfer datasets. Notably, we observe that the ma- jority of current datasets lack sufficient cultural in- formation, posing hindrance to the advancement of cross-cultural NLP research. We adopt HARM- CHRCK (Kirk et al., 2022) in Appendix for han- dling and presenting harmful text in research.
Cultural features. Recent research has begun to focus cross-cultural NLP (Hershcovich et al., 2022). Cultural feature augmentation is able to im- prove the performance of deep learning models on various semantic, syntactic, and psycholinguistic tasks (Ma et al., 2022). Sun et al. (2021) introduce three linguistic features that capture cross-cultural similarities evident in linguistic patterns and quan- tify distinct aspects of language pragmatics. Build- ing upon these features, they extend the existing research auxiliary language selection in cross- lingual tasks. However, Lwowski et al. (2022) con- firms that offensive language models exhibit geo- graphical variations, even when applied to the same language. This suggests that using language as a proxy for considering cultural differences is overly simplistic. Santy et al. (2023) highlight the impact of researcher positionality, which introduces design biases and subsequently influences the positionality of datasets and models. So in this paper, we con- sider country information for annotators across var- ious OLD datasets and propose more fine-grain fine-grained cultural features to enhance the prediction accuracy of transfer learning for cultural-loadee tasks.
2 Related Work
OLD methods. There have been many works on offensive language detection. Some works concen- trate more on how to improve OLD systems (Goel and Sharma, 2022; McGillivray al., 2022; Liu al., 2022). Considering that offensive language online is a worldwide issue and research efforts should not be confined solely to English (Søgaard, 2022), some researchers focus on constructing non- English OLD datasets (Çöltekin, 2020; Sigurbergs- son and Derczynski, 2020; Mubarak et al., 2021; Deng et al., 2022; Jeong et al., 2022). Based these datasets, some works try to apply cross- lingual transfer learning for LOD (Nozza, 2021;
Litvak et al., 2022; Arango Monnar al., 2022; Eronen et al., 2022: Zhou et al., 2023), but perfor- mance has so far not taken into account cultural factors. Understanding the cultural nuances and context surrounding offensive language is crucial for developing effective and accurate models for detecting such language.
3 How to predict optimal transfer datasets?
Optimal transfer language prediction. A ma- jor challenge cross-lingual learning choos- ing the optimal transfer language. Language prox- imity is not always the best criterion, since there
In this section, we formalize the problem a trans- fer dataset ranking task for cross-cultrall/ingual transfer learnin.. Different from cross-lingus

---

transfer learning, our experiments involve consid- ering datasets from the same language but with different cultural backgrounds.
values, as per Hofstede's theory (Hofstede, 1984), across different countries. Hofstede's Cultural Di- mensions Theory serves as a framework utilized to comprehend cultural disparities among countries. A key component of this theory is the Hofstede Value Survey, questionnaire designed to com- paring cultural values and beliefs of similar indi- viduals between cultures. These dimensions are Power Distance (pdi): reflect the acceptance of unequal power distribution ism (idv): examine the level of interdependence and self-definition within a society; Masculinity (mas): explore the emphasis on competition and achieve- ment (Masculine) or caring for others and quality of life (Feminine); Uncertainty Avoidance (uaî): deal with society's response to ambiguity and efforts to minimize uncertainty, Long- Term Orientation (Ito): describe how societies balance tradition with fu- ture readiness; and Indulgence (ivr): focuse on the control of desires and impulses based on cultural upbringing. These cultural dimensions provide in- sights into various aspects of a society's ' values and behaviors, help to understand the cultural varia- tions and preferences within societies and how they shape attitudes, behaviors, and prioritics.
Specifically, we give a task t and provide a re- lated dataset set = d11,,d....,,,
Our ob- jective is to develop a ranking method for the low- resource target dataset d!st DD which ranks the other 1 candidate high re-resource transfer datasets Dtsf D deat {dfs distt ..... d Exhaustive method is the most direct and simple approach, involving training the task model on each candidate transfer dataset and then evaluating on the target dataset. However, is time-consuning and resource-wasting.
society; Individual-
An effective approach is to utilize the extracted features from the dataset pairs to predict prior knowledge for cross cross-cultural task, eliminating the necessity of conducting task-specific experiments.
da dor extract (d.t d df
These extracted features encompass various as- pects, including statistical properties, linguistic in- formation, and domain--pecifer characteristics of the datasets. These features are employed predict a relative score for each transfer dataset pair.
We symbolize these features C' {c:), where i E {pdi, idv, mas, wai, lto, ier). with each fea- ture value ranging from to 100. To characterize the differences between the transfer dataset and the target dataset in different cultural dimensions, we represent them using the proportional values of each cultural dimension, which is defined Ro; cfsí /c/st These cultural distances are de- noted as CulDim (Re).
T 1o R (voor gpr:))
where R(-) denotes the ranking model, 0 is the parameters of the ranking model.
4 What features can be used to predict?
The current features used for transfer learning pre- diction can be divided into data-dependent and language-dependent categories. Data Dete-dependent features capture statistical characteristics specific the dataset, while -enundee-dependent features assess language similarities from different perspec- tives. Appendix B provides further details on these features. However, in cultural-loaded tasks, it is crucial to account for cultural differences. To ad- dress this, we propose incorporating six country- level features that quantify variations in cultural values across countries. Furthermore, introduce a language-level and domain-specifie feature that measures the distance between offensive words in language pairs.
4.2 Domain-Specific Feature
To predict transfer learning success in OLD tasks, we explore using offensive word distance as a fea- ture, inspired by Sun et al. (2021) who introduce Emotion Semantics Distance (ESD) for measuring lexical similarities of emotions across languages. In contrast to their method of calculating distances using 24 emotion concepts (Jackson al., 2019), calculate offensive distance based on the align- ment of offensive concepts across languages. Specifically, we utilize Hurtlex, a multilingual lexicon comprising offensive, aggressive, and hate-
hntpps//ggerthoofseeeee www.///eeeeeccaaaneeeeeeeee vsm-2013/
4.1 Nettural-Dimensie Features
'The cultural dimension values for each coun- try can be found in https: htasss//geerthastteeee. com/ research-ante
The country-level features encompass six cultural dimensions that capture cross- culsrral variations in
wwwrrchhand-ss//eee htmm//dmmiiiinncc waaoonnddaaaaaaaaaaaa data-matrix/.

---

200
Europe
100
pl
South Asia
0-
CS
Middle East
ar
hi
ko
West Europe
-100
East
Western Europe
zh
de
-200
Cast Europe
fa
-300
Southh America North America
-200
-100
100
300
Figure 2: Network based Offensive Word Distance. Edges connect languages based on their top-2 closest rankings. Solid lines represent mutual ranking, while dashed lines represent exclusive ranking.
Figure I: T-SNE Visualization of Cultural Values: Coun- tries represented as data points, with color-coded re- gions highlighting similarities and differences in cul- tural dimensions.
ful words over 50 languages (Bassignana et al., 2018).4 This resource offers over 1000 aligned offensive lemmas across languages. For instance, words related to the concept 'clown' in vari- ous languages (i.e. klaun, klown, bouffon, payaso, jokar, Kömäli, H). etc.) are grouped together. We obtain aligned word embeddings based on Fast- Text (Joulin et al., 2018),* S and average the distance between all aligned offensive concepts between language pairs. Mathematically, the offensive dis- tance OffDis from transfer language to target language is defined as follows:
(lanuuage-eeve)), adopt the language selection from Sun et al. (2021), including 16 languages. As for CulDim (country-level), we select 16 countries based on their relevance to the experimental OLD datasets and the chosen languages."
Cultural dimensions. These dimensions provide insights into societal perceptions and interactions across various aspects of life. To visualize the lationships between countries' cultural values, we use T-SNE in Figure 1, where the proximity of points on the plot indicates the degree of similarity in their cultural values. Observations from Fig- ure reveal that there is a degree of consistency between cultural similarity and geographical prox- imity. For instance, countries in Western Europe and the Middle East tend to be closer in Figure 1. However, geography alone doesn't solely deter- mine cultural similarity. Notably, the United States and the United Kingdom are visually close despite their geographic distance, while Japan and Ger- many, though geographically distant, show cultural proximity. This illustrates the intricate interplay between culture and geography, transcending phys- ical boundaries.
OffDist( Σ cos (u w [W]
WCW
where wh and Vi respectively represent the aligned word vectors of offensive aligned word to in FastText for transfer language and target lan- guage, W = { (w"t, w } is the set of all of- fensive aligned words between transfer and target languages. In example, 'clown' is the offensive word concept helonging W. Assuming transfer language is English and target language Czech, then tsf and ett would the words 'clown' and "klaun", dgt and va denote the corresponding word embeddings extracted with Fasttext respectively.
Offensive distance. Following Sun et al. (2021), create a language network based on offensive distances, shown as Figure 2.? Each node corre- sponds to a language, distinguished by its cultural area. Languages are sorted based on offensive dis- tance, and an edge is drawn between languages that
4.3 Features Analysis
To analyze and qualitatively evaluate proposed features attempt to visualize them. For OffDist
"The list selected countries and languages, along with their corresponding abbreviations in Figure and Figure 2. can he found Appendix E.
'License https://crretiveeommnssoo//////
by-nc-sa/4.0/ httt::/:/fattexxt.c///s////s///e html
'Japanese and Tamil are excluded due to lack of aligned Fest embeddings and Hurtlex lexicon, respectively.

---

Dataset
Language
Country
Region
Offensive Label
Source
Size
COLD (Dng al., 2022)
Chinese
China
East Asia
Offensive
Zhihu, Weibo
37480
ChileOLD
Spanish
Chile
South
intentional crofanity/vulgarity
Tweet
9834
Moerar al., 2023)
America
unintended profanity/valgam insall/uppellatinn hale speach
DeTox (Demus al., 2022)
German Hindi
Germany
West Europe
Hate Speech
Tweet
10278
HinHDD (Mandi
India
South Asia
Offensive; Hate Speech; Defamation
Twitter,
8192
Facebook,
WhatsApp NAVER.
KOLD Uoong al., 2022)
Korean
South Korea
East Asia
Offensive
40429
YouTube
NJH-UK (Bianchi al., 2022)
English
United
West Europe
Outrage; Insults; Profanity;
Tweet
11190
Kingdom
Character Assassination;
NJH-US US (Biznchi et 2022)
English
United States
North America
Discrimination; Hostility
Tweet
9086
PolEval (Ptuszynski al., 2019)
Polish
Poland
East Europe
Cyberbullying, Hate Speech
Tweet
10041
TurkishOLD (Coltokin, 2020)
Turkish
Turkey
Middle East
Offensive
Tweet
34792
Table I: Statistics of OLD datasets used in our experiments, covering 8 languages and 9 countries.
rank among the top-2 closest. Based on this net- work we can see that the offensive word distances seem follow language proximity pattern. Some clusters seem to appear for the Romance and Slavic languages. 75.8% of the edges correspond cul- tural areas, matching exactly the ESD network and surpassing the syntactic network (Sun et al., 2021). Emotion and offensive word similarity therefore seem to have similar relationship to cultural areas.
2022; Demus et al., 2022), including hate speech, insults, threats, and discrimination. In this paper, we simplify the detection tasks in the OLD datasets into binary classification tasks, considering a text offensive is labeled with any offensive label.
Zero shat Transfer Leaming
ChileOLD
0:466-466
.0.6
4.11
40.54
-0.15
-.30
-173
0.445
Culture vs. geography. Based on the above analysis, whether visualizing countries based on CulDim or visualizing languages based on OffDist, they can approximately fit the grouping based on geographical location but do not strictly match it. This highlights that geographical differences can- not be absolute indicators of cultural distinctions. Culture is complex and multifaceted concept that is influenced by various factors beyond geography, such history, politics, migration, and globaliza- tion (Taras et al., 2016). Thus, exploring additional culturally relevant features becomes necessary. Ap- pendix C presents the feature correlation analysis involved in the paper.
Hindi-
0.36
0.22
0.50
0.31
0.223
0.25
0.46
0.30
Text Set
KOLD
0.24
0.30
0.38
0.4
MJHUUS
0.33
0.23
0.52
0.19
0.37
0.08
0.51
0.28
-05
-0:15
25
+0.31
0.31
40.37
Polish-
-0.33
0.05
0.22
60.58
0.02
-0.09
0.00
0.05
-0.6
TurkishOLD -ee--------
-0.2
-0.20
.0.63
-0.20
-0.34
.0.00
-0.35
-0.7
CHikOLD
DeTox
Hindl
KOLD
NIK UK
Paish
TriGHOLD
MH
Figure 3: Relative losses (in percent) from zero-shot transfer learning models over the intra-cultural models.
Zero-shot transfer. We train intra-cultural XLM- R (Conneau et al., 2020a) for each dataset to en- sure cultural consistency. Then, we perform zero- shot transfer (ZST) using XLM-R on 72 directed dataset pairs, simulating cross-cultural implemen- tation. Specifically, for all 72 directed dataset pairs, we fine-tune the XLM-R model on the transfer dataset with learning rate le-5. The training epoch is 20, training batch size is 8, we adopt the early- stopping method with patience 10 on the target dataset's 's dev set. In order to mitigate the influence
5 Can we predict transfer languages for OLD successfully?
5.1 Experimental Settings
OLD datasets. To acquire country-level cultural dimension features, carefully select 9 OLD datasets that include annotators' nationality infor- mation, ensuring the availability of country context, The OLDD dataset statistics are shown in Table (re- fer to Appendix D for more details). Offensive lan- guage encompasses various aspects (Bianchi et al.,
hnttps://hugginggface co wwwwwooooooeeeeeeeeeeeeeeeoooo roberta-base

---

MAP
NDGC
MAP
NDGC
MAP
NDGC
LangRank
47.48
59.33
MTVEC
51.67
68.23
Colex2Lang
57.78
71.41
PPAGG
50.65
53.60
+PRAG
56.98
73.87
+PRAG
55.77
71.95
Offiist
51.08
60.17
+OffDist
64.26
76.50
+OffDist
59.64
73.81
CalDim
51.23
60.71
+CulDim
70.28
76.92
+CulDim
73.25
71.56
+P40
50,90
58.35
+P+0
49,46
62.30
+P+0
55.75
73.26
+P4C
57.45
67.63
+P+C
74.44
76.53
+P+C
69.63
76.33
+C+0
53.45
61.47
+C+0
73.33
73.87
+C40
76.69
79.34
+P+C40
56.54
62.73
+P+C+0
76.67
79.61
+P+C40
68.99
72.06
Table 2: The performance of transfer language prediction. Bold for best performance, underline for second best performance. P, and C stand for PRAG, OfjDist, and CulDim respectively.
Target Datasets
Ranking comparison
transfer learning success. Lastly, we obtain the ground truth ranking results for the transfer of OLD based on the experimental results of ZST.
KOLD
NJH
NILUK
COLD
TurkishOL
KOLD
UIL
UK
ChileOLD
KOLD
NIHLUK
COLD
Transfer ranking prediction. Following Sun et al. (2021), we use LightGBM (Ke et al., 2017) as the ranking model, known for its effectiveness in ranking tasks. We evaluate the performance using Mean Average Precision (MAP) and Normal- ized Discounted Cumulative Gain (NDCG) metrics. MAP calculates the average precision across all ranks, while NDCG considers both position and relevance. For our analysis, we focus the top 3 ranked items for both metrics.
TuKishOLD
KOLD
ChilOLLD
DeTox
KOLD
TurkshOLD
ChiksOLD
Hindi
NIH
NIHLUS
ChikOLD
NILUK
COLD
Hindi
KOLD
NJH,US US
NIH UK
ChikeOn D
Hindi
NJHLUK UK
COLD
NIH_US
Hindi
NJHLUK
COLD
Hindi
NJH US
COLD
Hindi
KOI D
PolEval
Chil:OLD
TurkisbOLD
Baseline features. We employ the following ranking baselines: LangRank (Lin et al., 2019), incorporating 13 features to rank items, includ- ing data-dependent and linguistic-dependent fea- tures; MTVEC (Malaviya et al., 2017), learning 512- 12-dimensional language vectors infer syntac- tic, phonological, and phonetic inventory features; Colex2Lang (Chen al., 2023), learning language representations with node embedding algorithms based on constructed large-scale synset. Addi- tionally, we incorporate PRAG features (Sun et al., 2021), capturing cross-cultune cross-cultural linguistic patterns and assessing language pragmatics. Appendix E.3 provides summary of the features used in each baseline and subsequent feature groups.
ChileOLD
Chil-OLD
NJH. US
KOLD
TurkishOLD
ChilsOLD
NJH US
KOLD
Table 3: Comparison of Top-3 Transfer Datasets. The first line represents ground truth rankings while the second line represents predicted rankings. Transfer datasets with exact rankings are highlighted red,, and those predicted within the top-3 positions are high- lighted in blue.
of randomness in our experiments and ensure re- sult reliability, we run five times for each dataset pair. Subsequently, we compute the average perfor- mance (macro F1 score) of the trained model on the target dataset's 's test set, representing the transfer performance for each dataset pair. Figure 3 illus- trates the relative losses incurred by ZST compared to intra-cultural models, highlighting the varying impacts of different transfer datasets on the same target dataset. This presents challenge in selecting appropriate transfer datasets for low-resource tar- get datasets without prior implementation. These findings underscore the importance of predicting
5.2 Overall Results
We examine the effectiveness of proposed features in predicting transfer dataset rankings for OLD 9. We specifically incorporate different features and their combinations into the baseline features, and the experimental results are presented in Table 2.
"The specific cultural dimension features used in our ex- periments are shown in Appendix E.2.

---

LangRank
MTVEC
Canex2Lang
an ablation experiment using the optimal feature combination from each baseline setting in Table 2, all of which include CulDim. The results Ta-
MAP
NDGC
MAP
NDGC
MAP
NDGC
BEST
57.45
67.63
76.67
79.61
76.69
79.34
-pdi
51.52
62.46
76.28
76.31
76.34
78.95
ble 4 demonstrate that almost each cultural dimen-
iiv
51.98
68.18
74.81
78.52
72.06
75.86
sion can contribute individually to the prediction of
-mas
54.09
63,20
70.94
72.00
68.92
72.74
OLD transfer. These contributions exhibit variation depending the specific baseline features utilized, It's important to note that cultural dimensions are not mutually exclusive, and given culture can hibit varying degrees of each dimension. Further- more, the relationship between culture and OLD is complex and can be influenced by factors beyond these dimensions, such as historical context, social norms, and individual differences.
-uai 46.12
59.55
76.76
77.89
76.06
80.96
-Ito 51.19
67.07
61.94
67.16
71,80
74.36
-ivr 55.30
61.95
73.33
75.61
69.95
74.13
Table 4: Ablation analysis for the six cultural dimen- sions, demonstrating that all dimensions contribute to the ranking performance and are therefore meaningful for offensive language detection generally.
However, the cultural dimension values are somewhat correlated with the sensitivity of offen- sive language detection. Khan (2014) states that "High power distance employees will be less likely to perceive their supervisory abusive" and "High in- dividualistic cultural employecs will be more likely to perceive their supervisory abusive". Bond et al. (1985) find that cultural variations (including Cul- tural Collectivism and Power Distance) in the use of social control are related to perceptions of an insult and of the insulter. Van Oudenhoven et al. (2008) also highlight that people from different cul- tures exploit different categories of verbal abuse. Our experimental results further substantiate these research findings.
Results show that our proposed features, CulDim and OffDist, independently enhance baseline per- formance, indicating the influence of cultural-value differences and domain-specific distance on trans- fer prediction. Moreover, Combining CulDim con- sistently improves ranking prediction, reinforcing the reliability of cross-cultural features in culture- loaded task transfer. Surprisingly. the MTVEV group surpasses the LangRank group, despite lack- ing data-dependent features. This underscores the impact of culture-dependent and language- dependent features on accurate and robust transfer learning predictions for culture-loaded tasks.
Moreover, using the best results from Table 2, we visualize the top-3 transfer datasets for each target dataset in Table 3. By comparing the ground truth rankings with the predicted rankings, we further assess prediction accuracy. The analysis reveals that for TurkishOL.D, the top-3 transfer datasets, including their specific rankings, accurately predicted. When considering only the prediction accuracy for the top-3 rankings without consid- ering the specific positions, the aceuracy rate is 100% for four datasets and 67% for three datasets. Except for NJH_US, the highest-ranked transfer datasets are accurately predicted to be within the top-3 for other datasets. These results demonstrate the potential of our proposed features in reducing trial and error in migration data selection. Accu- rate prediction of top-3 transfer datasets empowers researchers and practitioners to make informed de- cisions, enhaneing the efficiency and effectiveness of cross-culture transfer learning.
6
What types of features are most important for different types of tasks?
6.1 Feature Fine-grained Comparison
To gain a deeper insight into the key features con- tributing to the success of the optimal transfer dataset prediction in OLD task, we categorize the involved features into eight distinct groups: Data- specific, Topology, Geography, Orthography, Prag- matic, PRAG, OFF, and Cultural. The first five groups align with the grouping proposed by Sun et al. (2021) in their previous work. In particular, the Pragmatic group consists of PRAG features as well as three data-dependent features. So we estab- lish a separate group specifically dedicated to the PRAG features to ensure their distinct recognition and analysis. We introduce two new feature groups: OFF, which incorporates our newly proposed fea- ture OffDist, specifically designed for OLD tasks, and Cultural, which encompasses the CulDim fea- ture capturing cultural dimensions.
5.3 Cultural Dimensions Analysis
To examine the influence of each cultural dimen- sion on offensive language detection, we perform
The performance of ranking models trained with

---

Dataspecific
Typology
Geography MAP
Orthography
Pragmatic
PRAG
OFF
Cultural
Datasets
MAP
NIGIC
MAP
NDGC
NDGC
MAP
NDGC
MAP
NDGC
MAP
NDGC
MAP
NDGC
MAP
NDGC
COLD
75.00
69.58
25.00
15.79
26.67
38.21
58.33
82.07
33.33
65.10
45.00
79.24
24.29
9.55
58.33
73.11
ChileOLD
39.29
60.83
20.83
-3.31
29.17
57.31
29.17
28.07
26.79
31.59
26.79
31.38
26.79
15.10
45,00
78.76
DeTox
22.62
31.47
26.79
47.53
24.29
35.38
33.33
59.26
26.67
27.37
26.67
22.11
66.67
77.78
62.5
76.35
Hindi
83.33
94.34
25.00
23.59
25.00
38.21
41.67
77.00
62.50
68.41
41.67
56.35
20.83
35.02
30.95
42.69
KOLD
64.29
21.24
26.67
37.50
41.67
42.69
50.00
55.18
66.67
79.72
23.59
NJLUK
41.67
48.35
64.29
69.58
24.17
76.41
45.00
70.28
83.33
94.34
100.00
96.69
62.50
80.31
66.67
79.72
NJLLS
36.67
79:72
62.5
71.93
29.17
63.68
33.33
$4,59
75.00
71.93
75.00
71.93
66.67
70.17
39.29
59.66
PolEval
32.06
42.38
33.13
29.12
45.22
51.13
63.89
55.82
68 08.06
71.51
56.94
63.98
45.24
54,26
100.00
99.01
TurkishOLD
26.79
52.83
70.00
67.24
40.00
53.49
24.29
24.17
66.67
79.72
32.50
65.21
22:50
55.66
50.00
46.21
AVG.
46.86
62.14
40.00
38.08
30.6
50.23
40.72
57.63
53,78
61.41
50.51
60.23
44.68
53.06
53.74
64.34
Table 5: Performance comparison of OLD ranking predictions. Bold for best performance, underline for second best performance.
different feature groups for OLD is presented in Table 5. The results highlight the varying perfor- mance of different feature types. The Cultural group stands out as the best-performing group in both metrics, closely followed by the Pragmatic group. This finding confirms that the proposed CulDim feature, which captures cultural-value dif- ferences, holds high predictive power for OLD tasks. While the feature OffDist can provide some auxiliary predictive power when combined with other features, it does not possess independent pre- diction ability on its own, which points the limi- tations of the domain- specific feature.
the country information of the dataset, and the SA and DEP tasks involve datasets that only provide language information (Sun et al., 2021), we filter out the countries where the relevant language is the sole official language. 10 We then calculate the average cultural dimension values of these coun- tries to obtain the cultural dimension value for the language itself.
The comparison results are presented in Table 6. The Topology group demonstrates the highest pre- dictive power for the DEP task. This can be at- tributed to the inclusion of syntactic distance mea- surements between language pairs, which aligns with the inherent characteristics of the DEP task. Although the Cultural group does not rank as the most predictive for the SA task, it still achieves sig- nificant results. When comparing the performance of the Cultural group across different tasks, it is ob- served that the performance for OLD and SA tasks is higher than that for the DEP task. This finding highlights the valuable contribution cultural fea- tures to the success of transfer learning predictions for culture-loaded tasks.
SA MAP NDCG MAP NDCG MAP NDCG
DEP
OLD
Data-specific
36.94
52.83 68.00 85.40 46.86 62.14
Typology
58.12
79.41 49.90 60.70 40.00 38.08
Geography
34.12
67.93 24.90 55.00 30.60 50.23
Orthography
35.54
65.50 34.20 56.60 40.72 57.63
Pragmatic
54.37
60.50 73.20 88.00 53.78 61.41
PRAG
44.45 61.11 41.81 59.58
50.51 60.23
Cultural
34.47 49.23 55.80 79.75 53.74 64.34
Table 6: Ranking performance comparison on differ- ent NL. tasks: dependency parsing (DEP), sentiment analysis (SA) and offensive language detection (OLD). Bold for best performance, underline for second best performance.
Conclusion
We explored the feasibility of predicting transfer learning success, identifying informative features for transfer learning effectiveness, and examined the generalizability of these features across differ- ent NLP tasks. We have gained valuable insights into the potential of incorporating cultural sensitiv- ity in language technologies: dimensions from cul- tural values surveys consistently improve transfer success prediction, and offensive language distance features provide a further improvement.
6.2 Comparison in Different Tasks
To quantify the generalization of different feature groups across various NLP tasks, we compare their prediction performance among OLD, Sentiment Analysis (SA) and Dependency Parsing (DEP). Among these tasks, OLD and SA are relatively subjective in nature, while DEP is highly objective, with labels directly derived from linguistic rules. Given that the features of the Cultural group rely on
We recommend annotation projects to consider
List of official languages by country and territory

---

not only the balance between genres and age groups but also encompass cultures from different regions. Including documentation with information about the cultural background of the annotators enables better interpretation of annotations considering bi- ases and cultural nuances, and promotes a com- prehensive and inclusive approach to research and analysis fostering the transparency, usability, and suitability of the dataset. This study highlights the need for culturally informed datasets and the impor- tance of including cultural factors in cross-lingual transfer learning for OLD.
languages (Alaux et al., 2019). The coverage of offensive words in many languages is very partial, and furthermore, the ability of aligned word vectors to quantify cultural differences is mmnited-bsside their inability to capture linguistic context, social context (who says what to whom, where and when) is crucial to the interpretation of offensive language. This context is very seldom captured in language datasets. This limitation applies in general the premise of offensive language detection on isolated sentences as a goal and limits the extent which such classifiers can accurately detect offensive lan- guage in the real world.
Future work will augment the cross-cultural transfer learning strategy by incorporating cultural features directly into models. Further analysis will also conduct socially situated multi-cultural hu- man evaluation on language model predictions to address subjective nuances that cannot be identified in reference-baseee bassdd evaluation on isolated sentences. Finally, we will further explore the correlation be- tween features like offensive word distance, geog- raphy, and culture, to gain insights into their rela- tionships and effectiveness in capturing offensive language patterns.
Despite these limitations, our study serves an important step toward understanding the role of culture in NLP and provides valuable insights for future research in this domain.
Ethics Statement
This study involves the use of datasets with poten- tially harmful and offensive content, but they all follow privacy guidelines to ensure the anonymity and confidentiality of the individuals involved. The datasets used in this study are obtained from pub- licly available sources, and all efforts have been made to ensure compliance with relevant legal and ethical regulations. We acknowledge the potential impact of offensive language and take precautions to minimize any harm caused. Furthermore, this research aims to contribute to the development of technologies that promote a safer and more inclu- sive online environment. We are dedicated to fos- tering culture of respect, diversity, and fairness in research practices and encourage open dialogue the ethical implications of offensive language detection.
Limitations
While this study provides valuable insights into several aspects of cross-cultural and cross-lingual transfer learning for offensive language detection, there are certain limitations to consider.
The cultural features we introduce certainly do not fully capture the diverse nuances and complex- ities of different cultural contexts. While more fine-grained than linguistic categories, countries are clearly not homogencous cultural entities, and assuming they are such can limiting and harmful to diversity and inclusion. Furthermore, our study is based on a specific set of datasets, tasks, and lan- guage pairs, which may limit the generalizability of our findings to other languages and domains. We employ early stopping on the target development set, practice commonly used in many machine learning settings, while (Kann et al., 2019) state development sets in the target language are often assumed to be unavailable in the case of zero-shot learning. Moreover, the two most related works (Lin et al., 2019; Sun et al., 2021) do not mention the details about development set. Regarding the offensive distance feature, it is clear that using embeddings alone for alignment may not capture fine- grained relationships between
Acknowledgements
Thanks to the anonymous reviewers for their help- ful feedback. The authors express their gratitude to Yong Cao, Jiaang Li, and Yiyi Chen for their help during the rebuttal period and to Christoph Demus, Federico Bianchi, Aymé Arango for their assistance with the dataset collection process. Li Zhou acknowledges financial support from China Scholarship Council (No. 202206070002).
that
References
David Adelani, Graham Neubig, Sebastian Ruder. Shruti Rijhwani, Michael Beukman, Chester Palen-

---

Michel, Constantine Lignos, Jesujoba Alabi, Sham- suddeen Muhammad, Peter Nabende. Cheikh M. Bamba Dione, Andiswa Bukula, Rooweither Mabuya, Bonaventure F. P. Dossou, Blessing Sibanda, Happy Buzaaba, Jonathan Mukibii, Godson Kalipe, Derguene Mbaye Amelia Taylor, Fatoumata Ka- bore, Chris Chinenye Emezue, Anuoluwapo Aremu, Perez Ogayo, Catherine Gitau, Edwin Munkoh- Buabeng, Victoire Memdjokam Koagne, Allah- sera Auguste Tapo, Tebogo Macuewa, Vukosi Mari- vate, Mboning Tchiaze Elvis, Tajuddeen Gwad- abe, Tosin Adewumi, Orevaoghene Ahia, Joyce Nakatumba-Nabende, Neo Lerato Mokono, Ig- natius Ezeani, Chiamaka Chukwuneke, Mofetoluwa Oluwaseun Adeyemi, Gilles Quentin Hacheme, Idris Abdulmumin, Odunayo Ogundepo, Oreen Yousuf. Tatiana Moteu, and Dietrich Klakow. 2022. MasakhaNER 2.0: Africa- ceticccenttic transfer learning for named entity recognition. In Proceedings of the 2022 Conference on Empirical Methods in Nat- urul Language Processing, pages 4488 4508, Abu Dhahi, United Arah Emirates. Association for Com- putational Linguistics.
Dolicki Blazej and Spanakis Gerasimos. 2021. Analysing the impact of linguistic features on cross- lingual transfer. ArXiv, abs/2105.05977.
Michael H Bond, Kwok-Choi Wan, Kwok Leung, and Robert A Giacalone. 1985. How are responses to ver- bal insult related to cultural collectivism and power distance? Journal of Cross-Calural Psychology, 16(1):111-127.
Yong Cao, Li Zhou, Seolhwa Lee, Laura Cabello, Min Chen, and Daniel Hershcovich. 2023. Assessing cross-cultural alignment between ChaiGPT and hu- man societies: An empirical study. In Proceedings of the First Workshop on Cross-Culture Considerations in NLP (C3NLP), pages 53-67, Dubrovnik, Croatia. Association for Computational Linguistics.
Yiyi Chen, Russa Biswas, and Johannes Bjerva, 2023. Colex2Lang: Language embeddings from semantic typology. In Proceedings of the 24th Nordic Con- ference on Computational Linguistics (NoDaLiDa), pages 673-684, Tórshavn, Faroe Islands. University of Tartu Library.
Kabir Ahuja, Shanu Kumar, Sandipan Dandapat, and Monojit Choudhury. 2022. Multi task learning for zero shot performance prediction of multilingual models. In Proceedings of the 60th Annual Meet- ing of the Association for Computational Linguistics (Volume I: Long Papers), pages 5454-5467, Dublin, Ireland. Association for Computational Linguistics.
Chris Collins and Richard Kayne. 201 Syntactic struc- tures of the world's languages.
Çağri Çöltekin. 2020. corpus of Turkish offensive language on social media. In Proceedings of the Twelfth Language Resources and Evaluation Confer- ence, pages 61 74-6184, Marseille, France. European Language Resources Association.
Jean Alaux, Edouard Grave, Marco Cuturi, and Armand Joulin. 2019. Unsupervised hyper-alignment for mul- tilingual word embeddings. In International Confer- ence on Learning Representations.
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettle- moyer, and Veselin Stoyanov. 2020a. Unsupervised cross pooss-lingual representation learning at scale. In Pro- ceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 8440- 8451, Online. Association for Computational Lin- guistics.
Ayme Arango Monnar, Jorge Perez, Barbara Poblete, Magdalena Saldaña, and Valentina Proust. 2022. Re- sources for multilingual hate speech detection. In Proceedings of the Sixth Workshop on Online Abuse and Harms (WOATI), pages 122-130, Seattle, Wash- ington (Hybrid). Association for Computational Lin- guistics.
Alexis Conneau, Shijie Wu, Haoran Li, Luke Zettle- moyer, and Veselin Stoyanov. 2020b. Emerging cross-lingual structure in pretrained language mod- els. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. pages 6022 6034, Online. Association for Computational Linguistics.
Stella Aririguzoh. 2022. Communication competencies, culture and sdgs: effective processes cross-cultural communication. Humanities and Social Sciences Communications, 911:1-111
Elena Bassignana, Valerio Basile, and Viviana Patti. 2018. Hurtlex: A multilingual lexicon of words to hurt. Proceedings of the Fifth halian Conference on Computational Linguistics CLiC-it 2018: 10-12 December 2018, Torino, Torino. Accademia Univer- sity Press.
Wietse de Vries, Martijn Wieling, and Malvina Nissim. 2022. Make the best of cross-lingual transfer: Ev- idence from POS tagging with over 100 languages. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers). pages 7676-7685, Dublin, Ireland. Association for Computational Linguistics.
Federico Bianchi, Stefanie Hills, Patricia Rossini, Dirk Hovy, Rebekah Tromble, and Nava Tintarev. 2022. "it's not just hate": A multi- dinetiiaimenio perspective on detecting harmful speech online. In Proceedings of the 2022 Conference on Empirical Methods in Nat- ural Language Processing, pages 8093-8099, Abu Dhabi, United Arab Emirates. Association for Com- putational Linguistics.
Christoph Demus, Jonas Pitz, Mina Schütz, Nadine Probol, Melanie Siegel, and Dirk Labudde. 2022. Detox: A comprehensive dataset for German offen- sive language and conversation analysis. Proceed- ings of the Sixth Workshop on Online Abuse and

---

Harms (WOAH), pages 143-153, Seattle, Washington (Hybrid), Association for Computational Linguistics.
Joshua Conrad Jackson, Joseph Watts, Teague Henry, Johann- List, Robert Forkel, Peter J Mucha, Simon J Greenhill, Russell D Gray, and Kristen A Lindquist. 2019. Emotion semantics show both cultural variation and universal structure. Science, (66(6472))1517715522
Jiawen Deng, Jingyan Zhou, Hao Sun, Chujie Zheng, Fei Mi, Helen Meng. and Minlic Huang. 2022. COLD: A benchmark for Chinese offensive language detection. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Process- ing, pages 11580-1 11599, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Younghoon Jeong, Juhyun Oh, Jongwon Lee, Jaimeen Ahn, Jihyung Moon, Sungjoon Park, and Alice Oh. 2022. KOLD: Korean offensive language dataset. In Proceedings of the 2022 Conference on Empiri- cal Methods in Natural Language Processing, pages 10818-10833, Abu Dhabi, United Arab Emirates. As- sociation for Computational Linguistics.
Matthew S Dryer and Martin Haspelmath. 2013. The world atlas of language structures online.
Juuso Eronen, Michal Ptaszynski. Fumito Masui, Masaki Arata, Gniewosz Leliwa, and Michal Wroezynski. 2022. Transfer language selection for zero-shot cross-lingual abusive language de- tection. Information Processing & Management, 59(4):10298)
Armand Joulin, Piotr Bojanowski, Tomas Mikolov, Hervé Jégou, and Edouard Grave. 2018. Loss in translation: Learning bilingual word mapping with retrieval criterion. In Proceedings of the 2018 Con- ference on Empirical Methods in Natural Language Processing, pages 2979-2984 Brussels, Belgium. Association for Computational Linguisties.
Ramy Eskander, Smaranda Muresan, and Michael Collins. 2020. Unsupervised vross-lingual part-of- speech tagging for truly low-resource scenarios. In Proceedings of the 2020 Conference Empirical Methods Natural Language Processing (EMNLP), pages 4820-4831, Online. Association for Computa- tional Linguisties.
Katharina Kann, Kyunghyun Cho, and Samuel R. Bow- man. 2019. Towards realistic practices in low- resource natural language processing: The develop- ment set. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natu- ral Language Processing (EMNLP-IJCNI) pages 3342-3349, Hong Kong, China. Association for Com- putational Linguistics.
Sayan Ghosh, Dylan Baker, David Jurgens, and Vin- odkumar Prabhakaran. 2021. Detecting cross- geographic biases in toxicity modeling on social me- dia. In Proceedings of the Seventh Workshop on Noisy User-generated Text (W-NUT 2021), pages 313-328, Online, Association for Computational Lin- guistics.
Antonia Karamolegkou and Sara Stymne. 2021. In- vestigation of transfer languages for parsing Latin: Italic branch vs. Hellenic branch. In Proceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaliDa), pages 315-320, Reykjavik, Iceland (Online). Linköping University Electronic Press, Sweden.
Divyam Goel and Raksha Sharma. 2022. Leveraging dependency grammar for fine-grained offensive lan- guage detection using graph convolutional networks. In Proceedings of the Tenth International Workshop on Natural Language Processing for Social Media, pages 45-54, Seattle, Washington. Association for Computational Linguistics.
Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang. Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017. Lightgbm: A highly efficient gradient boost- ing decision tree. Advances in neural information processing systems, 30.
Harald Hammarstrm, Robert Forkel, Martin Haspel- math, and Sebastian Bank. 2015. Glottolog 2.6. Max Planck Institute for the Science of Human History.
Shahid Khan. 2014. Impact of hofstede's 's cultural dimensions on subordinate's 's perception of abusive supervision. International Journal of Business and Management, 9(12)239.
Daniel Hershcovich. Stella Frank, Heather Lent, Miryam de Lhoneux, Mostafa Abdou, Stephanie Brandl, Emanuele Bugliarello, Laura Cabello Pi- queras. Ilias Chalkidis, Ruixiang Cui, Constanza Fierro, Katerina Margatina, Phillip Rust, and Anders Søgaard. 2022. Challenges and strategies in cross- cultural NLP. In Proceedings of the 60th Annual Meeting of the Association for Computational Lin- guistics ( (Volume I: Long Papers), pages 6997-7013, Dublin, Ireland, Association for Computational Lin- guistics.
Hannah Kirk, Abeba Birhane, Bertie Vidgen, and Leon Derezynski. 2022. Handling and presenting harmful text in NLP research. In Findings of the Associa- tion for Computational Linguistics: EMNLP 2022. pages 497-510, Abu Dhabi, United Arab Emirates. Association for Computational Linguisties.
Anne Lauscher, Vinit Ravishankar, Ivan Vulić, and Goran Glavaš. 2020. From zero to hero: On the limitations of zero-shot language transfer with mul- tilingual Transformers. In Proceedings of the 2020 Conference Empirical Methods in Natural Lan- guage Processing (EMNLP), pages 4483 4499, On- line. Association for Computational Linguistics.
Geert Hofstede. 1984. Culture's consequences: Interna- tional differences in work-related values, volume 5. sage.

---

Yu-Hsiang Lin, Chian-Yu Yu Chen, Jean Lee. Zirui Li, Yuyan Zhang, Mengzhou Xia, Shruti Rijhwani, Junx- ian He, Zhisong Zhang, Xuezhe Ma, Antonios Anas- tasopoulos. Patrick Littell, and Graham Neubig. 2019, Choosing transfer languages for cross-lingual learn- ing. In Proceedingss of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3125-313 3135, Florence, Italy. Association for Compu- tational Linguistics.
Barbara McGillivray, Malithi Alahapperuma. Jonathan Cook, Chiara Di Bonaventura, Albert Meroño- Peñuela, Gareth Tyson, and Steven Wilson 2022. Leveraging time-dependent lexical features for offen- sive language detection. In Proceedings of the The First Workshop on Ever Evolving NLP (EvoNLP)) pages 39-54, Ahu Dhabi, United Arah Emirates (Hy- brid). Association for Computational Linguistics.
Steven Moran, Daniel McCloy, and Richard Wright. 2014. Phoible online.
Patrick Littell, David R. Mortensen, Ke Lin, Katherine Kairis, Carlisle Turner, and Lori Levin. 2017. URIEL and lang2vec: Representing languages as typological, geographical, and phylogenetic vectors. In Proceed- ings of the 15th Conference of the European Chap- ter of the Association for Computational Linguistics: Volume 2, Short Pupers, pages 8-14, Valencia, Spain. Association for Computational Linguistics.
Hamdy Mubarak, Ammar Rashed, Kareem Darwish, Younes Samih, and Ahmed Abdelali. 2021. Arabic offensive language on Twitter: Analysis and exper- iments. In Proceedings of the Sixth Arabic Natu- ral Language Processing Workshop, pages 126-135, Kyiv, Ukraine (Virtual). Association for Computa- tional Linguistics.
Marina Litvak, Natalia Vanetik, Chaya Liebeskind, Omar Hmdia, and Rizek Abu Madeghem. 2022. Of- fensive language detection in Hebrew: other lan- guages help? In Proceedings of the Thirteenth Lan- guage Resources and Evaluation Conference, pages 3715-3723, Marseille, France. European Language Resources Association.
Debora Nozza. 2021. Exposing the limits of zero-shot cross-lingual hate speech detection. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the I 11th International Joint Conference Natural Language Processing (Volume 2: Short Pupers), pages 907-914, Online. Association for Computational Linguistics.
Jiexi Liu, Dehan Kong, Longtao Huang, Dinghui Mao, and Hui Xue. 2022. Multiple instance learning for offensive language detection. In Findings of the Asso- ciation for Computational Linguistics: EMNLP 2022, pages 7387-7396, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Nikolaos M. Papadakis, Francesco Aletta, Jian Kang, Tin Oherman, Andrew Mitchell, and Georgios E. Stavroulakis. 2022. Translation and cross-cultural adaptation methodology for soundscape attributes a study with independent translation groups from english to greek. Applied Acoustics, 200:109031.
Brandon Lwowski, Paul Rad, and Anthony Rios. 2022. Measuring geographic performance disparities of of- fensive language classifiers. In Proceedings of the 29th International Conference on Computational Lin- guistics, pages 6600-6616, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
Shantanu Patankar, Omkar Gokhale, Onkar Litake, Aditya Mandke, and Dipali Kadam. 2022. To train or not train: Predicting the performance of massively multilingual models. In Proceedings of the First Workshop Sculing Up Multilingual Evaluation, pages 8-12, Online. Association for Computational Linguistics.
Weicheng Ma. Samiha Datta, Lili Wang. and Soroush Vosoughi. 2022. EnCBP: A new benchmark dataset for finer-grained cultural background prediction in English. In Findings of the Association for Compu- tational Linguistics: ACI. 2022, pages 28 1-2823, Dublin, Ireland. Association for Computational Lin- guistics.
Vaidehi Patil, Partha Talukdar, and Sunita Sarawagi. 2022. Overlap-basee vocabulary generation im- proves cross- lingull transfer among related lan- guages. In Proceedings of the 60th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 219-233, Dublin, Ireland, Association for Computational Linguistics.
Chaitanya Malaviya, Graham Neubig, and Patrick Lit- tell. 2017. Learning language representations for typology prediction. In Proceedings of the 2017 Con- ference on Empiricul Methods in Natural Language Processing, pages mark. Association for Computational Linguistics.
Telmo Pires, Eva Schlinger, and Dan Garrette. 2019. How multilingual is multilingual BERT? In Proceed- ings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4996-5001, Flo- rence, Italy. Association for Computational Linguis- tics.
Copenhagen, Den-
Thomas Mandl, Sandip Modha, Prasenjit Majumder, Daksh Patel, Mohana Dave, Chintak Mandlia, and Aditya Patel. 2019. Overview of the hasoc track at fire 2019: Hate speech and offensive content identifi- cation in indo-curopean languages. In Proceedings of the 1 11th forum for information retrieval evaluation, pages 14-17.
Barbara Plank and Željko Agié. 2018. Distant super- vision from disparate sources for low-resource part- of-speech tagging. In Proceedings of the 2018 Con- ference on Empirical Methods in Natural Language Processing, pages 614, Brussels, Belgium. As- sociation for Computational Linguistics.

---

Michal Ptaszynski, Agata Pieciukiewicz. and Pawet Dy- bata. 2019. Results of the poleval 2019 shared task 6: first dataset and open shared task for automatic cyberbullying detection in polish twitter. In Maciej Ogrodniczuk and Lukasz. Kobyliński, editors, Pro- ceedings of the PolEval 2019 Workshop, pages 89- 110. Institute of Computer Sciences. Polish Academy of Sciences, Warszawa.
Ke Tran and Arianna Bisazza. 2019. Zero-shot depen- dency parsing with pre-trained multilingual sentence representations. In Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019), pages 281-288, Hong Kong, China. Association for Computational Linguistics.
Jan Picter Van Oudenhoven, Boele de Raad, Francoise Askevis-Lcherpeuxx Pawel Boski, Geir Scott Brun- borg, Carmen Carmona, Dick Barelds, Charles T Hill, Boris Mlačić, Frosso Motti, et al. 2008. Terms of abuse as expression and reinforcement of cul- tures. International Journal of Intercultural Rela- rions, 32(2):174-185
Afshin Rahimi, Yuan Li, and Trevor Cohn. 2019. Mas- sively multilingual transfer for NER. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 151-164, Florence. Italy, Association for Computational Linguistics.
Sebastin Santy, Jenny T. Liang. Ronan Le Bras, Katha- rina Reinecke, and Maarten Sap. 2023. Nlposition- ality: Characterizing design biases of datasets and models. In Annual Meeting of the Association for Computational Linguistics (ACL).
Shijie Wu and Mark Dredze. 2019. Beto, bentz, becas:
The surprising cross-lingual effectiveness of BERT. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-JJCNLP), pages 833-844, Hong Kong. China. Association for Computational Linguis- tics.
Gudbjartur Ingi Sigurbergsson and Leon Derczynski. 2020. Offensive language and hate speech detec- tion for Danish. In Proceedings of the Tweifth Lan- guage Resources and Evaluation Conference, pages 3.49-35500 Marseille, France. European Language Resources Association.
Marcos Zampieri. Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019. Predicting the type and target of offensive posts in social media. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Minneapolis, Minnesota.
Abdul Sittar and Dunja Mladenic. 2023. Classifica- tion of cross-cultural news events. arXiv preprint arXiv:2301.05443.
Papers), pages Association for Computational Linguistics.
Anders Søgaard. 2022. Should we ban English NLP for year? In Proceedings of the 2022 Conference on
Empirical Methods in Natural Language Procesxing, pages 5254-5260, Abu Dhabi, United Arah Emirates. Association for Computational Linguistics.
Li Zhou, Laura Cabello, Yong Cao, and Daniel Her- shcovich. 2023, Cross-cultural transfer learning for Chinese offensive language detection. In Proceed- ings of the First Workshop on Cross-Cultural Consid- erations in NLP (C3NLP), pages 8-15, Dubrovnik, Croatia. Association for Computational Linguistics.
Anirudh Srinivasan, Gauri Kholkar, Rahul Kejriwal, Tanuja Ganu, Sandipan Dandapat, Sunayana Sitaram,
Balakrishnan Santhanam. Somak Aditya, Kalika Bali, and Monojit Choudhury. 2022. Litmus predictor: An ai assistant for building reliable, high-performing and fair multilingual nip systems. Proceedings of the AAAI Conference on Artificial Intelligence. 36( 36(111:13227-1322.
A HARMCHECK List
Risk of harm protocol: The specific risks of harm are derived from offensive language detection datasets in multiple languages. Ex- perimenters for this work have access to these datasets, but they don't need to read them care- fully.
Anirudh Srinivasan, Sunayana Sitaram, Tanuja Ganu, Sandipan Dandapat, Kalika Bali, and Monojit Choud- hury. 2021. Predicting the performance of multilin- gual nlp models.
Jimin Sun, Hwijeen Ahn, Chan Young Park, Yulia Tsvetkov, and David R. Mortensen. 2021. Cross- cultural similarity features for cross- -ingull transfer learning of pragmatically motivated tasks. In Pro- ceedings of the 16th Conference of the European Chapter of the Association for Computational Lin- Online.
The work includes a Content Warn- ing directly after Abstract. The content warn- ing is in italics to maximize visibility.
Preview:
Distance:
Only few offensive words are included to explain cross-cultural differences in the introduction section.
guistics: Main Volume, pages Association for Computational Linguistics.
Vas Taras, Piers Steel, and Bradley L Kirkman. 2016. Does country equate with culture? beyond geography the search for cultural boundaries. Management International Review, 56:455-487.
Respect: For slurs and profanities in text, the authors star out the first vowel with an asterisk (1

---

B
Data-dependent and Languate-dependent Features
tiyy the similarities or dissimilarities between lan- guages based on these features.
The existing features used to predict transfer learn- ing mainly divided into data-dependent and language-dependent features.
Genetic: The Genetic distance derived from the Glottolog tree (Hammarstrm et al., 2015) of language families, where it quantifies the dissimilarity between two languages by mea- suring their distance within the tree.
B.1 Data-dependent Features
Data-dependent features primarily manifest in three aspects: dataset size, Type-Token Ratio (TTR), and word overlap, which are specific measurements obtained from the datasets themselves.
Syntactic: The syntactic distance is the co- sine distance between syntax features of lan- guage pairs. The syntax features are adapted from the World Atlas of Language Struc- tures (WALS) (Dryer and Haspelmath, 2013), Syntactic Structures of World Languages (SSWL) (Collins and Kayne, 201 1) and Eth- nologue 11
Dataset size. Transfer Size and Targer Size, serv- ing fundamental features, indicate the number of data entries utilized for training in transfer and target datasets, denoted by S1 and S2 respectively. Additionally, the Ratio Size S, Sa is utilized to indicate the relative size difference between the transfer dataset and the target dataset. It quantifies how much larger the transfer dataset is in compari- son to the target dataset.
The phonological distance measure the cosine distance between vectors containing phonological information from Ethnologue and WALS.
Phonological:
Type-Token Token Ratio TTR is the ratio between the number of types (vocabulary set) and the number of tokens in dataset, measuring lexical varia- tion. A higher TTR represents higher lexical variation. We utilize Transfer TTR TTR, and Tar- get TTR TTR2 represent the Type-Token Ra- tio (TTR) of the transfer and target datasets, re- spectively. The lexical-variation Distance TTR be- tween the transfer and target datasets is defined as TTRR, = (1 TTRi
Inventory: The inventory distance is the cosine distance between the inventory fea- ture vectors of languages, sourced from the PHOIBLE database (Moran et al., 2014).
Geographical distance can rep- resent the shortest distance between two lan- guages on the surface of Earth's sphere. It is another component of URIEL, in which the geographical vectors of each language express geographical location with fixed number of dimensions and each dimension representing the same feature.
Geographical:
Word Overlap Word overlap is used measure the lexical similarity between pair of languages (datasets). is defined as [VinVal [[ where Vi and 1/2 represent the vocabularies of transfer dataset and target dataset.
Featural: The cosine distance between vec- tors incorporating features from the above five.
B.2 Language-dependent Features
PRAG. These features are pragmatically- inspired linguistic features that capture cross- cultural similarities manifested in linguistic patterns.
Language-dependee features encompass six lin- guistic distances queried from the URIEL Typo- logical Database (Littell al., 2017), well as three linguistic features (PRAG) that manifest in linguistic patterns and quantify distinct aspects of language pragmatics (Sun et al., 2021).
Language Context-level Ratio (LCR) mea- sures the extent to which the language leaves the identity of entities and predicates to con- text. only takes into account noun prob- abilities and verb probabilities here, which represent the likelihood of nouns and verbs occurring in the text, respectively.
The feature vector in URIEL encom- passes various linguistic features that describe the typological properties of languages, which include Genetic, Syntactic, Phonological, Inventory, Geo- graphic and Featural. The derived distances quan-
URIEL.
"https://www htttps://www.ethoolgguug ethnologue.com

---

C. 2.S.
2 5 0
21 5
1.0
Figure A pairplot showcasing the relationship between culturally relevant features using kernel density estimation (KDE), accompanied correlation coefficients (in red digit) annotated on the corresponding scatter plots. The diagonal plots display the distribution of each feature using KDE curves, while the lower triangle depicts the joint distributions and contour plots using color gradient (Blues colormap).
Literal Translation Quality (LTQ) quantifies how well given language pair's MWEs are preserved in literal (word-by-word) transla- tion, using bilingual dictionary.
have selected those that are associated with cultural diversity, such as CulDim, Geographical, Syntac- tic, and so We construct a sample set by sam- pling from the OLD zero-shot experiment, which involves all possible pairs of data for the relevant features. Figure shows the relationship between the selected features. As depicted in Figure 4, we observe strong positive correlation between CulDim and Geographical, suggesting that there is relationship between cultural differences and geographical differences to some extent. The corre- lations between different features vary significantly. highlighting the importance of introducing multidi- mensional features certain extent.
Emotion Semantics Distance (EDS) measures how similarly emotions are lexicalized across languages.
Feature correlation analysis
C.1 Different distances
In this section, we discuss the correlations between our different features, with a specific focus on data- independent features. Among all the features, we

---

C.2 Cultural dimension
office for offensive comments in Germany. The datasct is collected by manually creating keyword list with the help of Google Trends.
We conduct a fine-grained correlation analysis for each dimension feature in the CulDim. As shown in the Figure 5, Indulgence (ivr) and Individualism (idv) exhibit relatively strong positive correlation, implying that societies or groups that tend to score high on indulgence also tend to score high on indi- vidualism. This could indicate cultural inclination towards personal freedom, self-expression, and a focus on individual rights and achievements. How- ever, Power Distance (pdi) and has strong negative correlation with Indul Indulgence gence (ivr) andIndividualism (idv). This indicates that societies or groups charac- terized by a high power distance, where hierarchi- cal structures and authority are emphasized, tend to have lower levels of indulgence and individualism. It suggests a cultural tendency towards obedience, respect for authority, and collective orientation over personal freedom and self-expression.
HinHD (Mandl et al., 2019), utilized in the HASOC 2019 shared task, was collected from Twit- ter using hashtags and keywords associated with offensive content. In the labeling process, multiple junior annotators participated via an online system to evaluate the tweets.
KOLD (Jeong ct al., 2022) a Korean offen- sive language dataset comprising comments from NAVER news and YouTube platform, which is also filtered by using predefined keywords.
NJH (Bianchi et al., 2022) is English dataset collected with keywords and associated with im- migration in the US and/or UK. It is annotated by ten undergraduate researchers from universities the UK and US, based on which the dataset can be split into NJH-UK and NJI-US..
PolEval (Ptaszynski et al., 2019) is the first dataset for the Polish language containing anno- tations of harmful and toxic language. The dataset was automatically collected from Twitter accounts in the Polish language and annotated by layperson volunteers under the supervision of cyberbullying and hate-speech expert.
Correlation of Different Cultural Dimensions
0.4
0.0
-0:14
-0.2
TurkishOLD (Cöltekin, 2020) is the first corpus of offensive language for Turkish and consists of randomly sampled micro-blog posts from Twitter. The annotators are native speakers of Turkish, and all are highly educated.
068
0.13
01
0.55
-06
pdi
maz
ual
Ibo
Figure 5: Fine-grained correlation analysis of dimension features in CulDim.
E
Table Summary for Supplementary Information
By presenting this information in a table format, allows for easy reference and understanding of the selection criteria and categorization of the data. This section serves to enhance the clarity and com- prehensiveness of the paper, ensuring that readers have access to all the necessary details regarding the countries, languages, and their corresponding abbreviations used in the study.
D Datasets details
COLD (Deng et al., 2022) is a Chinese dataset, covering the topics of racial, gender, and regional bias. It is collected by strategies of keyword query- ing and related sub-topics crawling, and labeld by 17 native Chinese native workers.
ChileOLD (Arango Monnar et al., 2022) con- tains tweets in the Spanish language that originated in Chile, which is a representative of the Spanish spoken in South America. Each tweet is annotated with several fine-grained offensive categories by three native Chileans.
E.1 Overview of Countries, Languages, and Groups Feature Analysis
To provide concise overview the countries and languages included in the feature analysis, we pro- vides detailed Table showcasing the abbrevia- tions used to represent each country and language in the feature analysis figures (Figure I and Fig- ure 2), as well as their corresponding groups based
DeTox (Demus al., 2022) is German dataset that is far more comprehensive with twelve differ- ent annotation categories. The annotation schema is developed with first-hand users from a reporting

---

on relevant criteria.
Country
Language
CN
China
zho Chinese
East Asia
Japan
jpn Japanese
KR
South Korea
kor
Korean
IN
India
hin Hindi
South Asia
tam Tamil
NL
Netherlands
nld Dutch
FR
France
France
Western Europe
UK
the United
eng. English
Kingdom
ES
Spain
spa
Spanish
DE
German
deu
German
PL
Poland
pol
Polish
East Europe
RU
Russia
rus
Russian
Czech
TR
Turkey
tur Turkish
Middle East
JO
Jurdan
Arabic
Tran
Persian
South America
CL
Chile
North America
US
the United
States
Country
pdi
idv
mas
uai
Table 7: Overview of Countries, Languages, and Groups in Feature Analysis
China
24
Chile
63
23
28
86
31
68
Germany
40
India
77
48
56
40
51
26
E.2 Cultural Dimension Values of Countries and Languages
South Korea
85
100
29
United Kingdom
35
89
66
35
51
69
United States
68
We display the cultural dimension values associ- ated with the countries and languages involved in our experiments. In OLD, the cultural dimension values are depicted Figure The datasets used in OLD contain country information, allowing for the direct utilization of corresponding cultural di- mension values. However, in the case of SA and DEP tasks, where only language information is provided in the datasets, direct indexing of cultural dimension values is not feasible. As solution, we select countries where each language con- sidered the sole official language and employ the average cultural dimension values of these coun- tries to represent the cultural features associated with the respective language. The value details are shown in Table 9.
Poland
68
60
64
93
38
29
Turkey
49
Table 8: Cultural-value features in OLD.
E.3
Feature List of Baselines and Feature Groups
Table 10 provides comprehensive list of features utilized in the baselines and feature groups em- ployed for comparison in the study. It showcases specific sets together for analysis and evaluation purposes.

---

Cultural Value
Language
Country
pdi
idv
mas
uai
Ito
ivr
Egypt
80
37
55
55
42
0
Jordan
70
30
45
65
16
43
Kuwait
90
25
40
80
-1
-1
Lebanon
62
43
48
57
22
10
Libya
100
35
66
67
15
74
Arabic
Qatar
93
25
80
-1
-1
Saudi Arabia
72
48
43
64
27
14
Syria
80
35
60
30
-1
Tunisia
70
40
40
75
-1
-1
United Arab Emirates
74
36
66
22
22
Chinese
China
80
30
24
Czech
Czech Republic
57
70
29
Netherlands
68
Dutch
Suriname
85
47
37
92
-1
-1
Australia
38
90
61
51
21
71
Ghana
72
Jamaica
45
39
68
13
-1
-1
Namibia
Nigeria
80
30
60
55
13
84
English
Sierra Leone
Trinidad and Tobago
47
16
58
55
13
80
United Kingdom
69
United States
40
91
62
46
26
68
Zambia
42
Burkina Faso
18
France
France
68
71
43
86
63
48
Senegal
-1
Austria
63
German
Germany
35
67
66
65
83
40
Hindi
India
77
48
56
40
51
26
Japanese
Japan
54
46
95
92
88
42
Korean
South Korea
60
18
39
85
100
29
Persian
Iran
58
41
43
59
14
40
Polish
Poland
68
60
64
93
38
29
Russian
Russia
93
33
36
95
81
20
Argentina
49
56
86
20
62
Chile
63
23
28
86
31
68
Colombia
67
13
80
13
83
Costa Rica
35
15
21
86
-1
-1
Dominican Republic
65
30
65
45
13
54
El Salvador
66
19
40
94
20
89
Spanish
Guatemala
95
37
98
-1
-1
Honduras
80
20
40
50
-1
-1
Mexico
81
30
69
82
24
97
Panama
95
11
44
86
-1
-1
Spain
57
42
86
48
44
Uruguay
61
36
38
98
26
53
Tamil
Sri Lanka
80
35
10
45
45
-1
Turkish
Turkey
66
37
45
85
46
49
Table Cultural Dimension Values of Languages and DEP Tasks. In particular, When the dimension score is -1, it indicates that the country does not have score in that specific cultural dimension.

---

Other Group
PRAG
Cultural-valu
Ours
Pragmatic
Orthography
Table 10: Feature Groups and Baseline Features in the Study
Feature Group
Gengraphy
Topology
Data-specific
MTVEC
Baselines
Rank
Uncertainty Avoidance (ual)
Long- Tong-Term Term Orientation (Ito)
Power Distance (pdi)
LCReer Distance
LCR Distance
Individualism fidv)
Masculinity (mas)
Indulgence (ivr)
Transfer Size
Transfer TTR
Distance TTR
Word overlap
Phonological
ESD Distance
Feature
Target Size
Ratio Size
Target TTR
Genetic
Syntectic
Featural
Inventory
Geographic
LTQ score
Rep Diff
Data-ind Ieteeindeeendent
Data- detendenn
Type


Reviews:
[{"review_id": "I3tkKHsbBT", "reviewer": "Reviewer_CaTP", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your meticulous review of our work.  Based on your feedback, we will make revisions and additions to the manuscript for the CR version, aiming to enhance the soundness of our work. The response regarding the reasons for rejection is as follows:\n\n**R1&2: The definition in Section 3 needs to be more detailed, and the abbreviations in Table 2 require explanations.**\n\nA1&2: We will enhance the description in Section 3 to improve its readability, and mention the named abbreviation symbols in the caption of Table 2. \n\n**R3: The authors should compare to newer SOTA to show the effectiveness and generality of their features.**\n\nThanks for the newer compared SOTA suggestion, which could enhance the soundness of our work.\n\nThe domain research on optimal transfer language prediction and data (language) features is highly advanced and challenging. There are not many features that can be used for optimal transfer dataset prediction. \n\nAfter diligent exploration of relevant references, we are thrilled to have discovered a very recent and potentially baseline method named Colex2Lang[1]. The experimental results based on the baseline Colex2Lang are shown in Table (a) below. The experimental results show that our proposed features *CulDim* and *OffDist* can still enhance the newer baseline. This underscores the credibility and effectiveness of our introduced features.\n\n*Table (a) The performance of transfer language prediction on a newer baseline Colex2Lang. P, O and C stand for PRAG, OffDist, and CulDim respectively*\n\n|            | MAP       | NDGC      |\n|------------|-----------|-----------|\n| Colex2Lang | 57.78     | 71.41     |\n| +PRAG      | 55.77     | 71.95     |\n| +OffDist   | 59.64     | 73.81     |\n| +CulDim    | 73.25     | 71.56     |\n| +P+O       | 55.75     | 73.26     |\n| +P+C       | 69.63     | 76.33     |\n| +C+O      | **76.69** | **79.34**  |\n| +P+C+O     | 68.99     | 72.06     |\n\n### References\n1. Colex2Lang: Language Embeddings from Semantic Typology (Chen et al., NoDaLiDa 2023)", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This is an interesting research finding. The authors validate that incorporating cultural features could improve the performance of cross-cultural transfer learning model.\n\n2. This paper proposes six country-level features that quantify the variations in cultural values across different countries. It introduces a feature that measures the distance of offensive words between languages.\n\n3. The logical structure of the paper is clear, and it revolves around the three research questions raised in the introduction in detail, and answers these three questions well.\n\n", "reasons_to_reject": "1. For the problem definition in section 3, it should be more detailed to improve readability.\n\n2. Symbols such as \"P\", \"O\", \"C\" in Table 2 have no detailed explanation.\n\n3. The authors should compare to newer SOTA to show the effectiveness and generality of their features.\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "1. This paper delves into the intersection of cultural features and transfer learning effectiveness. \n\n2. The findings reveal that cultural value surveys indeed possess a predictive power for cross-cultural transfer learning success in OLD tasks and that it can be further improved using offensive word distance. \n\n3. The authors advocate leveraging data sources rich in cultural information, such as surveys, to enhance cultural adaptability.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "sVVGkzWa1i", "reviewer": "Reviewer_miiQ", "summary": "", "questions": "A. The expression “predict language transfer” / “transfer prediction”  in the Introduction and the title of Sec 3 “How to predict transfer datasets” is hard to understand at first. As the objection of predicting is not specified by the expression. For example, predict the model’s performance after transferring to a different language? predict what aspect of the dataset? \nIs it a conventional expression or it can be replaced with clearer phrases expressing “predict optimal language to transfer from”?\n    \nB. Where does the data for Fig. 1 come from?\n\nC. Are the word embedding distances from Fasttext suppose to reflect the semantic difference or offensiveness distance of the words?\nIf it is the former, then two words with the same semantic meaning should be the same, if not, then it seems to be a limitation of the aligned Fasttext. If it is the latter, then how can Fasttext embedding reflect offensiveness?\n\nD. Is there any evidence for arguments in Line 453-468?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your careful review. We hope that if our response is worthy enough, it will make you consider adjusting your score. We summarize your comments and respond in the following five aspects.\n\n**Q1. The source of cultural dimension values (from the rejected reason, question B and the ethical concern)**\n\nA1: These cultural dimension values are based on Hofstede’s Cultural Dimensions Theory, which is mentioned in Line 220-223. \n\nFor more details, Hofstede’s Cultural Dimensions Theory serves as a framework utilized to comprehend cultural disparities among countries. A key component of this theory is the Hofstede Value Survey, a questionnaire designed to assess an individual's cultural values and beliefs. You can access additional information about the survey through the website https://geerthofstede.com/research-and-vsm/vsm-2013/  and retrieve the quantified cultural dimensions from https://geerthofstede.com/research-and-vsm/dimension-data-matrix/ **(The value used in Figure 1, Table 8, and Table 9 can be found in this website).**\n\nWe appreciate your inquiry, and we’ll give these details in the CR version to avoid confusion.\n\n**Q2. Confusion about the expression such as “predict language transfer” / “transfer prediction”, and suggestion with a clearer phrase such as “predict optimal language to transfer from”**\n\nA2. Thanks for your question and sorry for the confusion. As you suggested, we can indeed add a descriptive term to define our prediction objective.Our aim in prediction is to identify the optimal (top) transfer datasets for the OLD task through the following steps: (1) train a set of NLP models with all available transfer datasets and collect evaluation scores, (2) train a ranking model to predict the ranking of transfer dataset, then obtain the top  transfer dataset.\n\nIt's crucial to emphasize that our discussion involves transfer datasets rather than transfer languages, as languages do not inherently represent specific national cultures.  However, the dataset's specific sources and annotations can be specified to particular countries.\n\n\n**Q3.Question regarding the correlation between FastText embeddings and semantic distinctions, or the connection between FastText embeddings and the measure of offensiveness.**\n\nA3. These embeddings are usually described as semantic, and indeed, in this case we can say we are interested in the semantic distance between offensive words in the respective languages to quantify how similar their meaning is, and not what is the difference between their offensiveness. \n\n\n**Q4. Ask for evidence of arguments in Line 453-468?**\n\nA4. Khan [1] designed a study to identify the cultural differences between developed and developing countries, i.e., Australia and Pakistan, one representing western culture and the other representing eastern culture. Based on the cultural dimensions dissimilarities between these two cultures, the researcher developed the appended several propositions, including “High power distance employees will be less likely to perceive their supervisory abusive while” and “High individualistic cultural employees will be more likely to perceive their supervisory abusive”\n\nBond et al.[2] conducted cross-cultural studies on college students from China and the United States. In the testing process, students should rate the legitimacy of the insult and the personality of the insulter after reading several scenarios. The researchers found that cultural variations (including Cultural Collectivism and Power Distance) in the use of social control are related to perceptions of an insult and of the insulter.\n\nBesides, this work[3] also highlights that people from different cultures exploit different categories of verbal abuse. In our analysis and experimentation (Subsection 5.3 and Table 5), we indeed observe that all cultural dimensions contribute to the accurate prediction of optimal transfer datasets for the OLD task.\n\nWe will add these references to our CR version to enhance the soundness of our work.\n\n\n**Q5. Suggestion for presentation Improvements**\n\nA5. (1) Thanks for the nice suggestion about mask parts of offensive words with “*”. We will adopt this suggestion. \n\nIn addition, we further adopt HARMCHECK [4] in our paper for handling and presenting harmful text in research. Our self-checks and measures are as follows:\n\n   1. **Risk of harm protocol:**  The specific risks of harm are derived from offensive language detection datasets in multiple languages. Experimenters for this work have access to these datasets, but they don't need to read them carefully.\n\n   2. **Preview:** The work will include a Content Warning directly after Abstract: “This document discusses examples of harmful content (hate, abuse, misinformation and negative stereotypes). The authors do not support the use of harmful language, nor any of the harmful representations quoted below.” The content warning is in red to maximize visibility.  \n\n   3. **Distance:** Only a few offensive words are included to explain cross-cultural differences in the introduction section.\n\n   4. **Respect:** For slurs and profanities in text, the  authors will star out the first vowel with an asterisk (“*”) \n\n\n(2) Besides, we will revise the highlighting method for Table 3 based on the performance ranking of feature groups, aiming to enhance its readability.\n\n\n### References\n1. Impact of Hofstede’s Cultural Dimensions on Subordinate’s Perception of Abusive Supervision. (Khan, Shahid N., International Journal of Business and Management, 2014)\n2. How are Responses to Verbal Insult Related to Cultural Collectivism and Power Distance? ( Michael H. Bond et al., Journal of Cross-Cultural Psychology, 1985)\n3. Terms of abuse as expression and reinforcement of cultures. (Oudenhoven et al., International Journal of Intercultural Relations 2008)\n4. Handling and Presenting Harmful Text in NLP Research (Kirk et al., Findings of EMNLP 2022)", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The research question is important, and the proposed method is proven effective by experimental results.", "reasons_to_reject": "Some arguments in the paper are not well-justified or accompanied by solid experimental results. For example, the source of cultural dimension values for each country is not specified.", "questions_to_authors": "A. The expression “predict language transfer” / “transfer prediction”  in the Introduction and the title of Sec 3 “How to predict transfer datasets” is hard to understand at first. As the objection of predicting is not specified by the expression. For example, predict the model’s performance after transferring to a different language? predict what aspect of the dataset? \nIs it a conventional expression or it can be replaced with clearer phrases expressing “predict optimal language to transfer from”?\n    \nB. Where does the data for Fig. 1 come from?\n\nC. Are the word embedding distances from Fasttext suppose to reflect the semantic difference or offensiveness distance of the words?\nIf it is the former, then two words with the same semantic meaning should be the same, if not, then it seems to be a limitation of the aligned Fasttext. If it is the latter, then how can Fasttext embedding reflect offensiveness?\n\nD. Is there any evidence for arguments in Line 453-468?", "paper_topic_and_main_contributions": "This paper investigates the feasibility of predicting the optimal source language for transfer learning on the offensive language detection task. The authors propose two factors -- cultural and offensiveness -- to assist in selecting optimal source language. They conduct sufficient experiments to show the effectiveness of the proposed factors.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "Yes", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "1. when presenting offensive words, it would be considerably better by mask part of the words by “*” to reduce the unnecessary offensiveness.\n\n2. All the values in “Avg” line in Table 3 are highlighted, which is hard to get the main point.", "Justification_for_Ethical_Concerns": "One of the most important contributions of this work is introducing a cultural dimension in predicting the ideal language to transfer from. In the experiments, authors use the cultural scores for each country (Figure 1, Table 8, and Table 9). I tried my best and cannot find the source of the values in the manuscript. A reliable source is important as these values may also encompass inferent biases towards certain cultures. Additionally, these values can be sensitive to time, thus requiring a documented time stamp.\nDespite this, I do not rule out the possibility that I may have overlooked something."}}, {"review_id": "3ml7LFqDlE", "reviewer": "Reviewer_cZgm", "summary": "", "questions": "- I see that the authors are gonna perform experiments with pre-trained language models in future, but I'm curious why the authors did not do it in this paper. There are few small-sized language models (e.g., BERT-small,base), so it is possible to do experiments with such small pre-trained language models. The language models are known to have knowledge of various aspects (e.g., linguistic, cultural, commonsense, etc.), so it is necessary to show that the proposed features are valuable even if we use pre-trained language models.", "limitations": "", "rating": 2, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your meticulous review. We summarize your comments and organize our response in the following four aspects, addressing the rejected reasons and the raised questions. We hope that our replies will make you consider adjusting your score.\n\n**R1. Questioning the ability of cultural features, especially for some datasets of Asian languages (e.g., KOLD, Hindi).**\n\nA1. It needs to be clarified that the predictive challenge we aim to address is not for a specific data set, but for the whole field at the macro level. Consequently, when considering the utilization of cultural features alone, compared with using other feature groups alone for prediction, cultural features can achieve the best overall performance, that is, our view has been successfully demonstrated.\n\nWhen cultural features are combined with other features, as shown in Table 2, it greatly improves the performance of the optimal transfer prediction. Specifically, **based on LangRank baseline, 9.97 MAP score and 8.30 NDGC score have been improved; based on MTVEC baseline, 25.00 MAP score and 11.38 NDGC score have been improved.** Observing Figure 4, for each dataset, even for the dataset of KOLD and Hindi, the model combined with cultural features can hit the GOLD top 1 transfer dataset  in the top3 prediction. Therefore, we are confident enough to argue that the cultural features have strong power.\n\n**R2. More details about the ZST process are needed.**\n\nA2. Thanks to the suggestion. We’ll add more details on the CR version. \n\nSpecifically, for all 72 directed dataset pairs, we fine-tune the XLM-R [1] model on the transfer dataset with learning rate 1e-5. The training epoch is 20, training batch size is 8, we adopt the early-stopping method with patience 10 on the target dataset’s dev set. Macro F1 score is the metric.\n\nIn order to mitigate the influence of randomness in our experiments and ensure result reliability, we run five times for each dataset pair. Subsequently, we compute the average performance (macro F1 score ) of the trained model on the target dataset's test set, representing the transfer performance for each dataset pair.\n\n\n**R3.The ground-truth ranking results generated from the ZST process with limited languages is based on a very strong assumption.**\n\nA3. Yes, you're right. We mention this in the Limitations section,  specifically in Line 582-585. However, it's worth noting that such assumptions are common in current research[2][3] to predict optimal transfer dataset or language. Since our novelty is focused on different aspects than this methodological one, we leave it to future work to generalize it. \n\n\n\n**Q1. I see that the authors are gonna perform experiments with pre-trained language models in future, but I'm curious why the authors did not do it in this paper**\n\nA4. Thanks to the reviewers for the suggestion to incorporate small pre-trained language models into the experiments. There is indeed an interest in this direction and it can be pursued even with limited computational resources at our disposal. However, we believe the gap to performing these experiments is still a conceptual one, as it requires a significant modification to the task setup, namely encoding the dataset contents and acquiring high-quality encoders for each target language and cultural context.\n\nThrough elaborate experiments, we have shown that cultural features contribute to the prediction of optimal transfer dataset in transfer learning. Future work is expected to incorporate such cultural features directly into the pre-trained language model to enhance cross-cultural transfer learning strategies. Although we are still actively developing the exact methodology for achieving this objective, it represents an intriguing avenue for investigation and aligns with an inspiring research vision for follow-up work.\n\n\nReferences\n1. https://huggingface.co/xlm-roberta-base\n2. Choosing Transfer Languages for Cross-Lingual Learning (Lin et al., ACL 2019)\n3. Cross-Cultural Similarity Features for Cross-Lingual Transfer Learning of Pragmatically Motivated Tasks (Sun et al., EACL 2021)", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Well organized paper with interesting motivation.\n- New features for cultural difference and offensive word distance are proposed, and its effectiveness is demonstrated by experimental results.", "reasons_to_reject": "- Limited interpretation of the results of Table 3. I see that the proposed 'cultural features' gave much performance gain, and authors insist that the cultural features hold strong predictive power for the OLD task. However, for some datasets of Asian languages (e.g., KOLD, Hindi), the cultural features quite lower power than many other feature groups. The experiments are performed on limited languages, and some languages give contradictory experimental results, thus I think the authors can't insist that the cultural features have 'strong' power than others.\n- More details of the zero-shot transfer (ZST) process is required. How the authors 'fine-tune' the model on the transfer dataset? (e.g., training objectives, loss, and etc.)\n- The ZST is employed to generate ground-truth ranking results. This is based on very strong assumption that the ZST is perfect and always gives correct answers. Authors need to justfy this.", "questions_to_authors": "- I see that the authors are gonna perform experiments with pre-trained language models in future, but I'm curious why the authors did not do it in this paper. There are few small-sized language models (e.g., BERT-small,base), so it is possible to do experiments with such small pre-trained language models. The language models are known to have knowledge of various aspects (e.g., linguistic, cultural, commonsense, etc.), so it is necessary to show that the proposed features are valuable even if we use pre-trained language models.", "paper_topic_and_main_contributions": "This paper proposes to incorporate cultural differences for better language transfer, especially for offensive language detection task.\nPrevious studies have mainly focused on linguistic difference, but this paper firstly investigate why and how to use the cultural difference for language transfer.\nSpecifically, country information is used as a proxy for the cultural features in this paper, and existing lexicon is employed to compute offensive distance.", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper looks at the problem of transfer learning for offensive language detection. While previous work has looked at language dimensions for finding an optimal source language, this work proposes looking at cultural differences. This paper tackles an important and timely problem with an interesting approach, so I believe this will contribute positively to the community.

There are a few improvements suggested by the reviewers. For one, clarity of writing could be improved. Other details are in the reviews, so please refer to those when making your revisions.
Decision: Accept-Findings