Towards Detecting Contextual Real-Time Toxicity for In-Game Chat
Authors: Zachary Yang
Nicolas Grenon-Godbout
Reihaneh Rabbany
Venue: EMNLP 2023 Findings (2023)
Keywords: Real-Time Toxicity Detection
Game Chat Toxicity
Game Chat Moderation
TL;DR: ToxBuster, a model designed for real-time toxic chat detection, utilizes chat history and metadata effectively to achieve high precision and recall required for real-world deployment, and can be effectively applied to post-game moderation.

Abstract: Real-time toxicity detection in online environments poses a significant challenge, due to the increasing prevalence of social media and gaming platforms. We introduce ToxBuster, a simple and scalable model that reliably detects toxic content in real-time for a line of chat by including chat history and metadata. ToxBuster consistently outperforms conventional toxicity models across popular multiplayer games, including Rainbow Six Siege, For Honor, and DOTA 2. We conduct an ablation study to assess the importance of each model component and explore ToxBuster's transferability across the datasets. Furthermore, we showcase ToxBuster's efficacy in post-game moderation, successfully flagging 82.1% of chat-reported players at a precision level of 90.0%. Additionally, we show how an additional 6\% of unreported toxic players can be proactively moderated.

Towards Detecting Contextual Real-Time Toxicity for In-Game Chat
Zachary Yang
Ubisoft La Forge
McGill University, Mila
zachary. yangemail.mcgi
Nicolas Grenon-Godbout
Reihaneh Rabbany
Ubisoft UDO
McGill University, Mila
nicolas. sccolaa.ggeeoonggooourlbbbboooos
CIFAR Al chair
reihaneh raabann@milaaeee cuueeeeeeee..eeeeeeeceee
Abstract
made exceedingly challenging implement con- sistent moderation practices.
Real-time toxicity detection in online environ- ments poses significant challenge, due to the increasing prevalence of social media and gam- ing platforms. We introduce ToxBuster, sim- ple and scalable model that reliably detects toxic content in real-time for line chat by including chat history and metadata. ToxBuster consistently outperforms conventional toxicity models across popular multiplayer games, in- cluding Rainbow Six Siege, For Honor, and DOTA 2. We conduct an ablation study to as- sess the importance of each model component and explore ToxBuster's ' transferability across the datasets. Furthermore, we showease Tox- Buster's efficacy in post-game moderation, suc- cessfully flagging 82.1% 1% of chat-reported play- ers a precision level 90.0%. Additionally, we show how additional 6% of unreported toxic players can be proactively moderated.
We leverage recent advancements in large lan- guage models to create accurate and transferable models for effective content moderation. Contex- tual language embeddings, like BERT (Devlin al., 2018), serve as the foundation for many state-of- the-art toxic speech detection models. However, most existing approaches either neglect context en- tieeyy or yield only marginal improvements and are not fit for real-time moderation for in-game chat.
To address this limitation, we propose Tox- Buster, the first real-time in in-game chat toxicity de- tection model capable of integrating chat history and metadata. It is trained on annotated datasets that encompass diverse perspectives, with annota- tors from marginalized groups offering valuable insights. Furthermore, demonstrate the transfer- abilty of ToxBuster across different games, as well as its adaptability a completely different domain, i.e., comment threads of news articles. When it comes to post-game moderation, we prioritize high precision settings to maximize the impact of our model without overloading limited manual content moderation resources with false positives. Remark- ably, ToxBuster achieves an impressive 82% iden- tification rate for toxic players for posting toxic chat- precision level of 90.0%. This can significantly reduce the mod- eration load of human moderators. summary, contributions are threefold:
1 Introduction
Online spaces are plagued by toxic speech, span- ning social media platforms (e-g.. Facebook (Ciftci et al., 2017), Twitter (Watanabe et al., 2018), Red- dit (Mohan et al., 2017), YouTube (Döring and Mohseni, 2020)), in-game chats (Silva et al., 2020) and news websites comment sections (Zannettou et al., 2020). As evidenced by surveys from the Anti-Defamatio League (ADL), exposure to toxic language not only alienates users but also poses a range of psychological harms and the potential to incite real-world violence (ADL, 2021). Even worse, marginalized groups continue to face dis- proportionate level of targeted online hate and ha- rassment.
by other
We present ToxBuster, real-time toxicity de- tection model for in-game chat that outperforms current available solutions by leveraging chat his- tory and metadata
Companies strive to foster healthy online com- munity and have employed various methods to address toxic speech, such censoring words, (shadow) banning users or blocking them from communicating (Maher, 2016; Lewington, 2021). However, the vast amount of user-gente secr-geneatted data and the rapidly evolving nature of language have
We show that the proposed model can transfer across different games and domains.
We discuss the post-game moderation implica- tions and show how ToxBuster can automate a high percentage obvious cases of toxicity and

---

refocus moderator's effort towards unclear cases.
3 Methodology
Reproducibilty: Source code for ToxBuster is hosted on Github. Due to legal and privacy reasons, R6S and FH datasets cannot be public.
In this section, we present four toxicity datasets along with ToxBuster and the baselines models.
3.1 Dataset
2 Related Works
We collected and curated datasets from three multi- player games representing different genres: Rain- bow Six Siege (R6S), For Honor (FH), and De- fense of the Ancients 2 (DOTA 2). R6S first- person shooter, FH is a melee action game, and DOTA 2 is an online battle arena game. Addi- tionally, we incorporated Jigsaw's 's Civil Comments (CC) dataset into our research. The annotation pro- cess involved token-level annotations for R6S and FH, while DOTA and CC were annotated at the sentence-level. For games, chat lines from match considered a document while for CC, document contains each comment on an article. For detailed dataset statistics, please refer to Table 1.
Toxicity detection research has gained increasing attention due to the challenges poses. These challenges include not only the text itself, such as the presence of out-of-voue of-vocabulary words, but also the lack of consensus on the precise definition of toxicity (van Aken et al., 2018). Definitions vary, ranging from a broad concept of toxicity (Geor- gakopoulos et al., 2018) 8) to more specific categories like hate speech (Gambäck and Sikdar, 2017), abu- sive language (Park and Fung, 2017), cyberbullying (Zhong et al., 2016), and offensive language. In our study, we focus on adapting categories defined by Alliance (2020) to address disruptive behavior in online games.
R6$
FH
DOTA
CC
Traditionally, toxicity detection has been ap- proached as classification task using various meth- ods, including traditional ML models with man- ual feature engineering (Watanabe et al.. 2018). deep neural networks (Gambäck and Sikdar, 2017:
Domain
Game
Game
Game
Media
Time
2021
2022
2017
2018
No. of Documents
1,392
5,340
1,921
65,148
Lines
95,612
99,371
62,483
131,319
Avg. WPL
3.20
4.01
2.41
54.63
Avg. LPD
69.07
19.46
32.53
2,01
Zhong et al., 2016), and pretrained language mod- els (Almerekhi et al., 2022; Jhaveri et al., 2022; Lees al., 2022). Previous studies have explored including additional context such news article titles (Gao and Huang, 2017), usernames (Mubarak al., 2017), Twitter user metadata (Fehn Unsvåg and Gambäck, 2018), game-specific slang (Weld et al., 2021) and parent comments and discussion ti- tles (Pavlopoulos et al., 2020). These attempts to in- corporate additional context into language models have yielded limited performance gains, typically less than 1 1%. To address this, we incorporate more than just the previous comment (Pavlopoulos et al., 2020; Yu et al., 2022) by utilizing all preceding chat history available well.
% of Toxic Lines
32.06%
21.24%
23.91%
8.01%
Table Toxic chat datasets overview. WPL Words per Line. LPD Lines per Document.
3.1.1 R6S & FH
For each game, we extract chat logs from regions that communicate prominently in English. We over- sample matches with a high volume of chat lines and/or instances where at least one player was re- ported by another player. The toxic classes we adopt align with the "Disruptive Behavior in On- line Game" categories outlined by Alliance (2020): Hate and Harassment, Threats, Minor Endan- germent, Extermism, Scams and Ads, Insults and Flaming, Spam and Other Offensive Text. For detailed definitions each class and annota- tion guidelines, please refer to Appendix A and B respectively.
While previous research on toxicity detection has explored metadata aspects, none have ad- dressed the conversational aspect, especially with this level of granularity, in contexts where more than just the preceding line of text is used. We draw inspiration from multi-turn conversational models and incorporate speaker segmentation (metadata including TeamID, Chat Type, and PlayerID) and a naive dialogue augmentation technique that has shown to enhance BERT for multi-tume tulti-turn conversa- tions (Lu et al., 2020).
Our dataset includes diverse perspectives by in- cluding annotators from marginalized groups Each line of the dataset was reviewed by three annota- tors. To aggregate labels, used the minimum intersecting span of words from the annotators and employed a majority vote, resolving ties by assign- ing the most severe class. The final number of chat

---

lines per class and Fleiss score are reported in Table 2.
3.2
ToxBuster
ToxBuster is a token classification model based on BERT BASE- In our experiments, we employed a 60-20-20 train-validation-tesl split using 5 different random seeds. We present the mean and standard deviations of each metric. Our model achieved improved performance through two approaches: Chat History and Chat Speaker Segmentation.
Class
R6s
FH
Hate and Harassment
5,482
4,453
Threats
618
421
Minor Endangerment
625
109
Extremism
392
173
Scams and Ads
456
53
Insults and Flaming
8,824
11,329
Spam
11,127
2,210
3.2.1
Chat History
Other Offensive
3,117
2,077
Non-toxic
64,937
78,292
To enable reliable real-time toxicity detection, we utilize chat history additional context. For all three in-game chat datasets, the average Word Per Line (WPL) is less than 5. Thus, including all available chat history becomes crucial.
Fleiss K.
0.54
0.47
Table 2: Class distribution for R6S and FH (ordered by toxicity severity level, descending)
Drawing inspiration from the question- answering task, we treat chat history as the question and the current chat line as the answer. In particular, we concatenate all chat lines preceding the current chat line chat history and separate them with the "[SEP]" token the question. During training, only the toxic class label is provided for the current chat line, ie. our model is trained to predict the probability of each token's 's class in the current chat line given the chat history.
3.1.2 DOTA 2
We utilize Weld et al. (2021)'s dataset the foun- dation. This dataset comprises automatic token- level annotations and four manual sentence-level annotations: "explicitly-toxicc" "implicitly-toxx toxic", "action", and "other". To adapt the dataset for real- time toxicity detection, we introduce the following modifications: 1) merging the "action" class with the "other" class, 2) breaking down the merged sentences back into lines of chat, addressing the merging of consecutive lines originating from the same user. When merged sentence is "explicitly- toxic", chat lines are labeled as exxplicitlytoxiie if they contain at least one toxic token. Otherwise, they are labeled as "other". For all other classes, the chat lines retain the original labels from the merged sentence.
BERT BASE has a max token size of 512. Since the main goal is to predict the current token's ' class, employ custom truncator. It first prioritizes truncating the chat history by removing the earliest chat lines (truncating left) and, if necessary, the current chat line on the right. For the three tn-game chat datasets, the current chat line would never be truncated.
In the case of in-game chat, players have the op- tion to communicate with their teammates or with everyone. Following the approach of dialogue aug- mentation (Lu al., 2020), we determine the type of previous chat lines to include in the chat his- tory by considering four different scopes: personal, team, global and moderator. The personal scope of the chat history includes only lines written by the current player. The team scope expands the chat history to include lines from players on the same team. The global scope further includes past chat lines broadeasted to all players. The moderator scope includes remaining chat lines, such as those from enemy teams that are not broadcasted to all players. Table 4 presents a fabricated sample chat at the beginning of a match.
Class
DOTA
Explicitly Toxic
10,511
Implicitly Toxic
4,431
Other 47,541
Table 3: Class distribution for DOTA 2
3.1.3 CC
Civil Comments platform is a commenting plugin for independent news sites. To adapt Jigsaw's Civil Comments dataset for real-time toxicity detection, we create its comment history by including the past comment thread. This is achieved by tracing all parent comments. We use the binary toxic versus non-toxic label for each comment.

---

Line
playerID
charType
teandD
is the unique identifier for the player associated with the chat line, starting from 0 and bounded by the number of teams times the team size. For consistency, the player of the current chat line is always O. For other players on the same team, the identifier is incremented based on the recency of that player's chat line. For players on the other team, the identifier starts from the size of the team, e.g, in 5 game, the most recent opponent that has typed in chat will be 5. With this scheme, the playerlD can be extended even Battle Royal games, where there can be multiple enemy teams. TeamID is the unique identifier of the team the cur- rent player belongs to. For consistency, the current player is always team 0. The enemy team would be team 1. For battle royal games, this scheme can be extended similarly to the playerID. The last three columns in Table4 describe the playerID, chafType and teamID when detecting toxicity for line 5.
(Team) Apple: Hi
Team
(AIl) Banana:
(Team) Grape: Which site?
Team
(Team) Orange: A
Team
(All) Orange: Glhf
Table 4: Chat history & scope. We present the metadata encoding when considering line #5. The corresponding chat history for the different scopes of personal, team, global and moderator would be then line 4, lines 3-4, lines 2-4, and lines 1-4.
3.2.2 Chat Speaker Segmentation
The primary objective is to enhance the model by incorporating conversational and game-related in- formation. To achieve this, we introduce speaker segmentation, which includes three metadata at- tributes for each chat line: playerID, chatType, and leamID.
ToxBuster a BERT-based model with an ad- ditional three inputs provided. We follow BERT's input embedding scheme, where it is the sum of all (position and token) its encoding. As shown in Figure 1, we introduce an encoding for each of the teamlD, chatType and player!I) that corresponds the token. As such, the new input embedding would then be the sum of all 5 encodings: Token, Position, teamID, chatType and playerID.
3.3 Baselines
Here, present three baseline models used for comparison with ToxBuster.
3.3.1 Cleanspeak
Cleanspeak is paid tool that has "premier profan- ity filter and moderation" based user-defined keywords and regexes. Toxicity is determined based on the API response containing matched text related to various toxic classes, which can be mapped to our own. Currently, the toxic classes are
MLP
"harm_abuse", "threats", "grooming", "terrorism", "pii" (personal identifi- able information), "spam", "bullying", "vulgarity", "sexual", "alcohol drug", "asciiart".
BERT
Input
Fagz
Token
H
H
[SEP)
SSEP]
3.3.2 Perspective API
Position
NIN12
T TeamlD
T2
Tt
We utilized Perspective API2 (vlalphal) developed by Jigsaw and Google's Counter Abuse Technology team for promoting healthier online discussions (Lees et al., 2022). As noted by the research team, we classified chat line as toxic only if the returned toxic score >= 0.7. We remove approximately 13% of the chat lines during the calculation of the metrics due to the API returning an error code for unsupported languages.
Chat Type
C.
C2
Cy
Co
PlayerID
Chat History
Chut History
Chat History Line R
Chat Line
Line I
Line
Figure 1: ToxBuster with chat speaker segmentation. Input embeddings are the sum of the corresponding token, position, teamID, chat type and playerID. The chat history includes as many lines as possible.
3.3.3 Detoxify
For the three chat metadata, playerID and teamlD dynamically changes based on the player of the current chat line. CharType can be either team or all, indicating whether line is exclusive to the team or broadcasted to all players. PlayerID
Detoxify wetoxifyynbiassal unbiased (Hanu and Unitary team, 2020) is a BERT-based models trained on CC. Further
htttps://cleanspeak.cm
htt.s//aasppaaa.cccooo/o////// com/docs/3. x/tech/apis/ hhtppss//perspecivepp.co

---

details of how the models were trained can be found in their repository
Correlation with Domain: Focusing on the top left section, the performance between R6S and FH was consistently good, ranging from 81 to 89. Ex- panding to include DOTA 2, we observe an accept- able performance of 72, but it drops to 59 when CC is included. This suggests that R6S and FH share similar toxic language characteristics, with some similarities extending to DOTA 2 and fewer with CC. We attribute this pattern the fact that R6S, FH, and DOTA belong to the game chat domain, while CC falls within the social media comment threads domain.
4 Results and Discussions
In the upcoming sections, we analyze ToxBuster's performance relative to baselines, examine individ- ual toxic class performance, conduct an ablation study to evaluate each model component's ' impact and assess implications for post-gamm moderation.
4.1 Baseline Comparison
Table 5 compares Boxtuster with baseline models across all 4 datasets. The baseline models focus on the binary task of classifying chat lines as toxic or non-toxic. ToxBuster is trained on each dataset's specific toxic class, while ToxBustcr lacks chat history and speaker segmentation.
Correlation with Time: If we shift our attention to the bottom right section, the performance be- tween DOTA 2 and CC also exhibit good results, ranging from 81 to 95. Interestingly, DOTA 2 per- formed better on CC than either R6S or FH. We attribute this difference to the evolution of toxic language over time. The CC and DOTA 2 datasets date back to 2017 and 2018, respectively, while the R6S and FH datasets are more recent, from 2021 and 2022.
ToxBuster achieves the highest performance, with 5% improvement R6S and 3% improve- ment on FH compared to ToxBusterhase- However, no significant improvement observed for DOTA 2 and CC, possibly due to dataset characteristics. DOTA 2 dataset includes only all chat and initial annotation included merged chat lines. Similarly, the CC dataset lacks consideration for contextual information during annotation.
In summary, for robust real-time toxicity detec- tion in game chat, the model should be trained on up-to-date toxic game chat datasets rather than generic toxicity datasets.
In terms of precision, Perspective API generally outperforms Cleanspeak, while detoxifyymbiaaee excells in recall and Fl score. Cleanspeak ranks the lowest. Notably, baseline models perform poorly on the DOTA 2 dataset, likely due to game slang and low average WPL of 2.41.
4.3 Game Adaptation
We also assess adapting ToxBuster from R6S FH, two multiplayer games with slightly differ- ent data, including game-specific references and potential differences in toxicity. We use 20,339 chat lines (20% of the dataset) as the test set. To compare fine-tuning and adapting, we fine-tune ToxBuster on FH and perform transfer learning on the best-performing ToxBuster model R6S with gradually increasing the size of the FH train- ing dataset. The results are presented in Table 7. As expected, transfer learning outperforms fine- tuning in all scenarios, indicating good transfer- ability between R6S and FH. The performance differences amongst the transfer learning settings are minimal, suggesting that at low data settings, transfer learning may not provide significant im- provements and could even slightly decrease perfor- mance (+691 indicates lower overall performance). However, adapting from R6S and FH is beneficial as it demonstrates a boost in the model, as evident from the 1.5% in precision from ToxBuster,62.525 and Tox Buster pas 62,522.
4.2 ToxBuster Transferabilty
In this experiment, ToxBuster trained using one specific toxic dataset and evaluated on the remain- ing datasets. Each dataset was transformed into a binary classification task, determining whether each chat line was toxic or non-toxic. The findings are summarized in Table 6, which reveals interest- ing patterns.
Correlation with WPL. When considering the diagonal entries (trained and tested on the same dataset), we observe a positive correlation between performance and the average Word Per Line (WPL). The average WPL values for DOTA 2, R6S, FH and CC are 2.41, 3.20, 4.01, and 54.63, respec- tively. This matches our intuition where greater word count facilitates more reliable toxicity predic- tion.

---

R6$
FH
DOTA 2
CC
P
R
F1
P
R
F1
P
F1
R
F1
Cleanspeak
66.62
29.10
40,48
65.91
38.92
48.93
29.47
3,07
5.55
25.14 14
67.56
36,65
Perspective API
75.11
24.38
36.81
73.48
37.97
50.07
11.47
1.06
1.95
83.63
60.98
70.53
Detoxify
63.47
29.58
40.33
66.01
50.09
56.98
26.98
3,22
5.75
69.99
75.81
72.79
Tox box
77.21
77.91
77.36
80.41
82.17
81.88
83.98
84.82
84.05
94.53
94.43
94.45
ToxBuster 82.95
83.56
83.25
84.88
85.62
85.09
84.39
85.09
84.53
95.48
95.46
95.47
Table 5: Toxicity classification performance across datasets. Performance is measured in terms of weighted average precision, recall, and Fl score. Baseline models focus on the binary task toxic versus non-toxic, while ToxBuster is trained and tested for each toxic class.
Test
Class
Precision
Recall
Precision
Recall
R6S
FH
DOTA 2
CC
Hate Hamss
63.78 2.3
56.40 a
58.76 5.8
45.55 0.4
R6S
85.38
88.65
77.70
59.88
Threats
31.53 3.7
22.85 4.6
36.38 8.6
2 7.0
Minor Endanger.
38.28 7.1
2.721
27.37 + 13.0
13.70 7.8
Train
FH
81.79
89.60
81.62
42.27
54.58
40.86
+
10.6
DOTA
2
72.81
85,54
85.14
89.75
Scams & Ads
56.89 5.0
45.
51.33 + 27.2
20.16 10.1
Insults
28.97
23.
$9.63
64.11
CC
67.49
75.77
81.88
95.47
84.15
7
58.86+
46:51 =911
Other Offensive
4,252
44,20 3.0
33.33 2
20.23 2
Table ToxBuster transferabilty across datasets. Trans- ferabilty is tied closely with domain & time.
Non
8..
92.55 9.
Table 8: ToxBuster class performance for R6S and FH.
Blue and red highlights the best and worst performing toxic classes respectively.
ToxBuster
Tox Buster
Precision
Recall
Precision
Recall
84,
85.81 0.3
76.50 +
81.38 0.4
84.01 :
85.18 0.4
15,787
0.6
0.5
84.57 2.1
84.27
reveals that 25% of them were clearly non-toxic. However, 42% of the text contained sexual content unrelated to the game, which could be categorized as bordering on sexual harassment. Additionally, 21 % of the text mentioned racial groups, where the toxicity of the chat line was open to debate. Fur- thermore, 12% of the text included racial terms that implied toxicity based on the context. For false neg- atives, our analysis indicates that 65% of them were toxic but not caught due to misspellings. Further- more, 25% of the false negatives contained implicit racism, ageism, and sexism. Additionally, 5% of the false negatives were considered implicitly toxic based on the context. Lastly, 5% were annotated as toxic due to the use of profanity, although they were not used in a toxic manner. These findings highlight the nuanced nature of evaluating toxicity.
25,448
84,20
85.04 0.5
85,43
35,410
00.4
0.3 0.4
85.65
53,582
84,84 44.84
0.8
85.47
0.9
85.43 0.5
85,89 0.3
62,528
84,88 0.6
85.23 0.4
Table 7: Comparison fine-tuning ToxBuster and adapting ToxBuster ROS with increasing number of train- ing data from FH. ToxBuster res achieves equal perfor- mance as Toxeuster with less than half training data.
4.4 Class-wise Evaluation
We also analyze our model's performance for each toxic class with results shown in Table 8. The model can easily differentiate amongst non-toxic, toxic words and spam. We attribute the lower Fl score in threats and minor endangerment to their minority. While extremism and scams and ads have even fewer samples, the language for both these two categories are usually very unique. We notice that the model often confuses amongst hate and ha- rassment, threats, other offensive as the words are often very similar and additional context from chat history, in game knowledge and social constructs are needed. Annotators also reported it was often hard to choose between these categories well.
4.6 Ablation Study
In this section, conduct ablation study on the two main components: Chat History and Chat Speaker Segmentation
4.6.1 Chat History
In this experiment, we evaluate the impact of chat history without chat speaker segmentation, as shown in Table 9. Intuitively, increasing the amount of context improves the reliability of tox- icity prediction. This is evidenced by the nearly
4.5 Error Analysis
We randomly sampled 100 false positives and 100 false negatives. For false positives,our analysis

---

4% performance improvement with either global or moderator scope. Notably, precision increases as we include more context, while global demon- strates higher recall and Fl score compared to mod- erator, albeit with slightly lower precision. These findings align with Lu et al. (2020) who empha- sized the importance of dialogue consistency dur- ing classification.
Precision
Recull
Fl Score
Base
81.60 0.5
82.21 0.4
81,70 0.5
w/ teamID*
81,59
82.08 0.9
81.69 #
In-Line
w/ chat type*
0.5
82.38
81.88 + 0.4
w/ playerlD*
8.4
82.01 = 0.4 0.4
w/ full*
81,95 * 0.3
82.56 + 0.3
82.12 # 0.3
w/ leamID
81.61 0.4
82.23
81,72 + 0.4
Speaker
w/ chat type
81.91 a 0.5
82.53
82.18 0.6
w/ playerID
82.36 0.4
82.79
82.43 0.4
w/ full
82.95 = 0.3
83.56 0.3
83.25 # 0.3
Table 10: Impact of Different Chat Metadata and Incor- poration Method. Full incorporates all three.
Precision
Recall
Fl Score
No History
77.21 1.3
77.91 + 1.0
77.36 1.3
Personal
79.11 + 0.7
79.82 + 0.3
78.89 + 0.8
Team
80.51 = 0.2
80.10
81.
Precision-Recal Tradeoff: In Figure 2. Tox- Buster achieves a consistent precision of 95% across variaous recall levels, indicating its effective- ness in identifying and flagging potentially toxic content. This high average precision demonstrates reliable results for moderation purposes.
Global
81.60 = 0.5
82.21 = 0.4
81.70 = 0.5
Moderator
81.90 = 0.4
81.47 + 0.4
81.68 0.4
Table 9: Impact of chat history & scope. Performance increases with larger scopes of chat history.
4.6.2 Chat Speaker Segmentation
1.0
In this experiment, we focus on the impact of chat speaker segmentation while keeping the chat his- tory constant in the global mode. We compare two methods of including chat metadata: in-line and chat speaker segmentation. The in-line method appends the metadata in front of each chat line, which intuitively incorporates the information. Ta- ble 10 presents the specific impact of each metadata added and the overall impact when all three in- cluded. These metadata are typically considered during human annotation to assess chat consistency. Across both methods, playerID is the most influen- tial feature, as it distinguishes chat lines based on the player. The chat type, indicating whether the intended audience is the team everyone (includ- ing the enemy team), is the next significant feature. Comparing in-line and chat speaker segmentation, the latter outperforms in terms of precision, recall, and Fl score for each metadata. We speculate that ToxBuster can better learn the correlation between chat line metadata and toxicity when each token has its own associated metadata, rather than posi- tioning it in front of the chat line.
0.8
Procision
0.6
0.2
Bascline (0.79)
Non-toxic Rest (AP 0.95)
0.0
0.0
0.8
1.0
Recall
Figure 2: ToxBuster Precision-Reddd Curve on R6S. Average precision for non-toxic vs. toxic words is 95%.
Performance at High Precision Settings: Tox- Buster successfully intercepts 5.38%, 2.07%, and 0.81% 1% of chat lines for R6S when operating at high precision thresholds of 90.0%, 99.0%, and 99.9%, respectively. This emphasizes ToxBuster's poten- tial as a real-time chat moderation solution that incorporates chat history and metadata, which is capability currently lacking in Perspective API. Table 11 presents the recall rates for each class at high precision levels. At precision of 99.9%, Tox- Buster achieves recall rates ranging from 0.4% 6.5% across different classes. Notably, the classes of Spam, Scams and Ads and Minor Endanger- ment demonstrates the highest recall rates, likely due to the distinct language patterns compared to other toxic classes.
4.7 Post-Game Moderation Implication
In real-world scenario, ToxBuster would be oper- ated at a high precision threshold ensure accurate detection of toxic chat and minimize false posi- tives. As shown before in Table 5, ToxBuster significantly outperforms Cleanspeak and Perspec- tive API, achieving 82.95% (+7) in precision and 83.56% (+54) in recall.
Viability for post-game Moderation: Table 12 presents a cross-reference of distinct players flagged by ToxBuster, chat-reported players, and reported players (due to disruptive behaviors) over a one-week period in R6S. is important to note that, at the time of data collection, R6S only al-

---

Class Name
90.0%
99.0%
issue (ADL, 2022). ToxBuster can help address this problem through proactive light automatic moder- ation. Table 12 demonstrates that substantial number of flagged players were not reported by other players. For proactive moderation, we utilize 0.50%
99.9%
Hate and Harassment
14.22%
2.35%
2.35%
Threats
0.26%
0.26%
0.26%
Minor Endangerment
7.32%
6.69%
6.69%
Extremism
0.50%
0.50%
Scams and Ads
8.84%
4.76% %
4.76% %
the average number of flagged chat lines per match from chat-reported players as simple measure. Figure 3 shows that players with more chat reports tend to have higher number of flagged lines of chat per match. Specifically, for players with more than 3 chat reports, the average number of flagged lines By applying this criterion, approximately 6.39% of non-chat-reported toxic players would be identified, start towards addressing this issue.
Insults and Flaming
1.18%
0.39%
0.39%
Spam
66.63%
42.01 %
6.14%
Other Offensive
2.09%
0.72%
0.72%
Table 11: ToxBuster Recall Rate per Toxic Class at High Precision Levels R6S.
lows players report others for single reason, meaning players engaging in both toxic chat and disruptive in- game behavior are more likely to be reported for their behavior rather than their chat.
Auerage No of Flagged Tooic Liwe/ Match
% of Players 90.0%
99.0%
99.9%
F/P 29.48%
11.64%
7.89%
(FNCR)/CR
82.1%
51.1% %
41.3%
(F (EN-CR)/-CR 26.44%
9.36%
5.96%
(FNR)/R 55.49%
26.92%
19.57%
(Fn-R)/-R 19.18%
6.37%
3.86%
Table 12: Intersection of flagged players (F) by Tox- Buster, chat-reported players (CF) and reported players (R). CR players represent 5.47% of all distinct players and R players represent 25.64%.
Chal Raports Accumulltted by Player
Figure 3; Average number of flagged toxic lines/match over for increasingly chat-reported players. Players are often chat reported when having more than 5 flagged toxic lines.
We utilize this metric for two estimates. Firstly, we estimate the model's precision based on player reports from the players' perspective, acknowledg- ing that this perspective may slightly differ from ac- tual moderators. Secondly, we estimate the reduc- tion in workload for moderators. By implementing simple yet effective automated post-game moder- ation system that focuses on players both flagged by Buster and reet-reported, we can promptly address toxic chat-reported players, leading a more positive player experience compared to man- ual inspection by moderators for each instance.
Conclusion
In this paper, we propose ToxBuster, a simple and scalable model specifically designed for real-time chat moderation in in-game chat. ToxBuster out- performs baseline models, including Cleanspeak and Perspective API, across multiple datasets and toxic categories. Additionally, ToxBuster be easily adapted for comment threads of social media platforms. Our study emphasizes the importance of using up-to-date toxic game chat datasets to enhance model robustness, rather than relying on generic toxicity datasets. Furthermore, we demon- strate the seamless transferability between R6S and FH datasets. Ablation studies further validate the significant contributions of chat history and speaker segmentation ToxBuster's effectiveness.
The second row of Table 12 demonstrates that ToxBuster identifies between 41% % to 82% of chat- reported players, indicating high precision from players' perspective. This approach also signifi- cantly reduces the workload for moderators when dealing with this set of users. A more sophisticated system could consider the severity level from each toxic class for further improvements.
4.7.1 Proactive Moderation
The precision-recall analysis illustrates Tox- Buster's capability to accurately identify and flag toxic content with high precision and recall.
Toxicity in online games and social media plat- forms often goes unreported, which a significant

---

an automated post-game moderation scenario, Tox- Buster successfully identifies toxic chat players, flagging over 41 of chat-reported players at a pre- cision level of 99.9%. Additionally, we explore how ToxBuster can partially address the issue of toxic players who are not chat-reported. Overall, ToxBuster presents robust and efficient solution for automating chat moderation, promoting safer and more positive gaming experience for players.
the diversity of the annotators identities into con- sideration. In our sessions, we recognize that it was hard to recruit those that identify as a woman. We had more success in recruiting those that iden- tified as belonging to marginalized groups (e.g. LGBTQAI+, BIPOC), where half of the annotators self-identifies belonging to at least one of the marginalized group.
Annotators were also warned about the toxic content they will see. They were given very lax schedule and allowed to annotate freely at their own pace over a lengthy time period, allowing many breaks if needed.
Limitations
Currently, the model's dataset is limited to the En- glish language, with the exceptions of common toxic phrases appearing in in-game chat lines from other languages. Based on results from Perspec- tive API and Jigsaw, we know that the methods presented in this paper can be extended from mono- lingual to multi-lingual.
As stated in the limitations, we are in the process of devising methods to measure bias and debias the model. An adaptation of Kiritchenko and Moham- mad (2018) Equity evaluation corpus (EEC) will be created to test and measure several categories of social biases such as gender, race, sexual ori- entation, etc. Meade et al. (2022) also includes a few benchmarks (Sentence Encoder Association Test and Word Embedding Association Test (May et al., 2019)) and de- methods. De-biasing methods include counterfactual data augmenta- tion (CDA), increasing dropout and projection- based techniques. CDA works by rebalancing the dataset by swapping bias attribute words. As rec- ommended by Blodgett et al. (2020), we have in- vited and welcomed new researchers from other disciplinary studies, namely from linguistics and psychology.
ToxBuster will make errors. Only existing pat- terns of toxicity in the dataset will be detected. Language that closely resemble existing patterns of toxic language could also be incorrectly flagged. As such, the model without any active learning is not suitable for a fully automated moderation. The model also cannot completely replace human moderation.
ToxBuster is intended for in in-game chat that will have mentions of in-game events. Hence, phrases that could be considered toxic (a threat) in normal everyday language could be scored as neutral or having less probability of being toxic.
ToxBuster is step in the right direction towards combatting the many challenges of moderating in- game chat toxicity. In terms of improving toxicity detection, some directions are performing domain adaption on the base language model on unl unlabeled chat data, continuous learning and adversarial train- ing. The model and dataset can be extended from English to multilingual. Another area is biases and its mitigation. While we have mitigated some during the data annotation phase, we still need to measure biases the model has learned and ways to debias the models without degrading the model per- formance. Finally, can also analyze the causes and impacts of toxicity from player and game design perspective.
Acknowledgements
We wish to thank Ubisoft La Forge, Ubisoft Mon- treal User Research Lab and Ubisoft Data Office for providing technical support and insightful com- ments on this work. We also acknowledge funding in support of this work from Ubisoft, the Canadian Institute for Advanced Research (CIFAR AI Chair Program), Natural Sciences and Engineering Re- search Council of Canada (NSERC) Postgraduate Scholarship -octooal (PGS D) Award and Fonds de recherche du Québec Nature et Technologies (FRONT) Doctoral Award.
References
Ethics Statement
ADL. 2021. Online hate and harassment: The american experience 2021. Anti- Dnri-Defamation League.
As with any language models, Boxtuster will prop- agate any existing biases in the dataset. We have tried to mitigate biases in the annotation by taking
ADL. 2022. Hate and harassment in online games. Anti- Defamation League.

---

Fair Play Alliance. 2020. Disruption and harms in on- line gaming framework. Fair Play Alliance, pages 1-48.
Svetlana Kiritchenko and Saif Mohammad. 2018. Ex- amining gender and race bias in two hundred senti- ment analysis systems. In Proceedings of the Sev- enth Joint Conference on Lexical and Computational Semantics, pages 43-53, New Orleans, Louisiana. Association for Computational Linguisties.
Hind Almerekhi, Haewoon Kwak, Joni Salminen, and Beraard Jansen. 2022. Provoke: Toxicity trig- ger detection in conversations from the top 100 sub- reddits. Data and Information Management, page 100019.
Alyssa Lees, Vinh Q. Tran, Yi Tay. Jeffrey Sorensen, Jai Gupta, Donald Metzler, and Lucy Vasserman. 2022. A new generation of perspective api: Efficient multi- lingual character-leve transformers. In Proceedings of the 28th ACM SIGKDD Conference on Knowl- edge Discovery and Data Mining, KDD '22, page 3197-3207, New York, NY, USA. Association for Computing Machinery.
Su Lin Blodgett, Solon Barocas, Hal Daumé II and Hanna Wallach. 2020. Language (technology) is power: A critical survey of "bias" in NLP. In Pro- ceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 5454- 5476, Online. Association for Computational Lin- guistics.
Robert Lewington. 2021. Being 'targeted' about content moderation:. Fair Play Alliance, pages 1-21.
Tuba Ciftci, Liridona Gashi, René Hoffmann, David Bahr, Aylin Ilhan, and Kaja Fietkiewicz. 2017. Hate speech on facebook. pages 425-4333 Fourth Euro- pean Conference on Social Media Research.
Junyu Lu, Xiancong Ren, Yazhou Ren, Ao Liu, and Zenglin Xu. 2020. Improving contextual language models for response retrieval in multi-turn conversa- tion. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '20, page 1805-1808, New York, NY, USA. Association for Computing Machinery.
Jacob Devlin, Ming-Wor Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: pre-training of deep bidirectional transformers for language under- standing. CoRR, abs/1810.04805.
Nicola Döring and M. Mohseni. 2020. Gendered hate speech in youtube and younow comments: Results of two content analyses. Studies in Communication and Media, 9:62-88.
Brendan Maher: 2016. Can a video game company tame toxic hehaviour? Nature News, 531:568.
Chandler May, Alex Wang, Shikha Bordia, Samuel R. Bowman, and Rachel Rudinger. 2019. On measuring social biases sentence encoders. Proceedings of the 2019 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Papers). pages 622-628, Minneapolis, Min- nesota. Association for Computational Linguistics.
Elise Fehn Unsväg and Bjöm Gambäck. 2018, The effects of user features on Twitter hate speech detec- tion. In Proceedings of the 2nd Workshop on Abusive Language Online (ALW2). pages 75-85, Brussels, Belgium. Association for Computational Linguistics.
Björn Gambäck and Utpal Kumar Sikdar. 2017. Using convolutional neural networks to classify bate-speech. In Proceedings of the First Workshop on Abusive Lan- guage Online, pages 85-90, Vancouver, BC, Canada. Association for Computational Linguisties.
Nicholas Meade, Elinor Poole- Moole-Dayan, and Siva Reddy, 2022. An empirical survey of the effectiveness of debiasing techniques for pre-traines language models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume
Lei Gao and Ruihong Huang, 2017. Detecting on- line hate speech using context aware models. In Proceedings of the International Conference Recent Advances in Natural Language Processing, RANLP 2017, pages 260- 660-266, Varna, Bulgaria. INCOMA Ltd.
1: Long Papers), pages 1878-1898, Dublin, Ireland. Association for Computational Linguistics.
Shruthi Mohan, Apala Guha Michael Harris, Fred Popowich, Ashley Schuster, and Chris Priebe. 2017. The impact of toxic language on the health of reddit communities. pages 51-56. Canadian Conference on Artificial Intelligence.
Spiros V. Georgakopoulos, Sotiris K. Tasoulis, Aris- tidis G. Vrabatis, and Vassilis P. Plagianakos. 2018. Convolutional neural networks for toxic comment classification. In Proceedings of the 10th Hellenic Conference on Artificial Intelligence, SETN '18, New York. NY, USA. Association for Computing Machin- ery.
Hamdy Mubarak, Kareem Darwish, and Walid Magdy. 2017. Abusive language detection Arabic social media. In Proceedings of the First Workshop on Abu- sive Language Online, pages 52-56, Vancouver, BC, Canada. Association for Computational Linguistics.
Laura Hanu and Unitary team. 2020. Detoxify. Github. http://gihubbcom//iiaaaaaaeeeeee
Ji Ho Park and Pascale Fung. 2017. One-step and two- step classification for abusive language detection on Twitter. In Proceedings of the First Workshop on Abusive Language Online, pages 41-45, Vancouver, BC, Canada, Association for Computational Linguis- tics.
Manan Jhaveri, Devanshu Ramaiya, and Harveen Singh Chadha. 2022. Toxicity detection for indic multilin- gual social media content. CoRR, abs/2201.0055

---

John Pavlopoulos, Jeffrey Sorensen, Lucas Dixon, Nithum Thain, and lon Androutsopoulos. 2020, Tox- icity detection: Does context really matter? In Pro- ceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 4296- 4305, Online. Association for Computational Lin- guistics.
taken into consideration. Each annotator had be at least 18 years old, advanced in English pro- ficiency, reside in the North American time zone and active in the respective game. We define active as having played the respective game within the last year for at least 16 hours in the player versus player (PVP) mode. After the initial recruitment, a pilot test was conducted to further filter annotators to those that understood the task and aligned them- selves on the common definition of toxicity based on examples shown. Each annotator was instructed to highlight the minimum span of contiguous words in a chat line that falls under a toxic category. a span of words can fall under more than one toxic category, they were to use the most severe category. Each chat line was annotated by three annotators, with full visibility of all previous chat lines of the match available. We also did not show any game related events nor whether the player was reported.
Bruno Silva, Mirian Tavares, Filipa Cerol, Susana Silva, Paulo Alves, and Beatriz Isca. 2020. Playing against hate speech -how teens hate speech in video games and online gaming communities. Journal of Digital Media and Interaction, 3:34-52.
Betty van Aken, Julian Risch, Ralf Krestel, and Alexan- der Löser. 2018. Challenges for toxic comment clas- sification: An in-depth error analysis. In Proceedings of the 2nd Workshop Abusive Language Online (ALW2), pages 33-42, Brussels, Belgium. Associa- tion for Computational Linguistics.
Hajime Watanabe, Mondher Bouazizi, and Tomoaki Ohtsuki. 2018. Hate speech on twitter: pragmatic approach to collect hateful and offensive expressions and perform hate speech detection. IEEE Access, 6:13825 13835.
B Detailed Toxicity Definitions
Henry Weld, Guanghao Huang, Jean Lee, Tongshu Zhang, Kunze Wang, Xinghong Guo, Siqu Long, Josiah Poon, and Caren Han. 2021. CONDA: CONtextual dual-annotated dataset for in-game tox- icity understanding and detection. In Findings of the Association for Computational Linguistics: ACL- IJCNLP 2021, pages 2406-2416, 16, Online. Association for Computational L Linguistics.
Our dataset does not include any audio or visual data, and therefore, categories such cheating. abuse of play, antisocial actions not within the scope of this model.
1. Hate and Harassment: Identity-based hate harassment (e.g., racism, sexism, homopho- bia) or bullying / mobbing (e.g., group of players bullying one or more players).
Xinchen Yu, Eduardo Blanco, and Lingzi Hong. 2022. Hate speech and counter speech detection: Conver- sational context does matter. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- Seat-
2. Threats:
Threats of violence, physical safety to another player, employee or property, ter- rorism, or releasing a player's real-world per- sonal information (e.g., doxxing).
man Language Technologies, pages tle, United States. Association for Computational Linguistics.
Savvas Zannettou, Mai Elsherief, Elizabeth Belding. Shirin Nilizadeh, and Gianluca Stringhini. 2020. Mcasuring and characterizing hate speech on news' websites. In /2 ACM Conference on Web Science, WebSei '20, page 125-134, New York, NY, USA. Association for Computing Machinery.
3. Minor Endangerment: Sexual and/or aggres- sive actions towards minors attempts to get minors perform sexual activities.
4. Extremism:
Extremist views (e.g.. white supremacy), attempts to groom or recruit for an extremist group or repeated sharing of po- litical, religious, or social beliefs.
Haoti Zhong, Hao Li, Anna Cinzia Squicciarini, Sarah Michele Rajtmajer, Christopher Griffin, David J. Miller, and Cornelia Caragea. 2016. Content-driven detection of cyberbullying on the in- stagram social network. In International Joint Con- ference on Artificial Intelligence.
5. Scams and Ads: Fraud / scamming (c.g., in- cluding phishing, account stealing, bad trades or theft), posting inappropriate links (e.g.. malware, dangerous websites, advertising ex- ploits, etc ) and advertising of websites, ser- vices, cheats or rival products.
A Annotator Details
For R6S and FH, annotators were recruited from so- cial media with representation in game experience and selffidentiiicationn with marginalized groups

---

6. Insults and Flaming: Insults or attacks on another player or team (not based on player or team's real or perceived identity)
and doesn't use chat history) and Tox (trained with chat history and speaker segmenta- tion) on R6S test set. We bin the number chat lines in the chat history and report the Fcore in Table 14. We can see that using chat history will always help with the performance, although marginally with 0 or I lines of chat history and much higher for more lines of chat history. This could indeed explain some of the variance have reported in the main results. More interestingly, we see a large improvement performance for 21 30 lines and the improvement slightly dropping afterwards, suggesting that this may the ideal length of chat history for this dataset.
7. Spam:
Excessive sharing of the same or sim- ilar words, phrases, emojis sharing (c.g., "kdiiklsjjfklljkkaa)"
8. Other Offensive Texts: Any other message not covered in the above categories that is offensive and/or harms a player's 's reasonable enjoyment of the game.
C Model Reproduction Details
Our model is implemented with HuggingFace and PyTorch'. Scction 3.2.1 and 3.2.2 include explicit information on the model architecture and prepro- cessing steps. Our model uses default values for BERT. The learning rate is le-5, chosen from a hy- perparameter search amongst le-4, le-5 and 1c-6. -1-300
Chat History
Support
Boxtustera.ss
ToxBuster
0
96
81.69
81.72
92
80.65
81.68
2-10
78.15
83.43
11-20
77.92
82.22
772
75.87
84.25
The learning rate scheduler is set to linear decay with a warmup step ratio of 5%. Training with GeForce RTX 2080 took approximately 7 hours with max train epochs set to 100, early stopping with patience of 5 epochs based on the weighted F1 score.
31-40
717
77.35
82.75
41+
14,447
77.14
83.65
Table 14: Impact of Context Model Fl score across differing chat history length.
D Sentence-level vs. Token-level
For this experiment, we do not include chat speaker segmentation and use global mode for chat history. We analyze in Table 13 the impact of changing the max token size for the tokenizer and the level of classification (sentence-level and token-level). Contrary to our beliefs, it would seem that classify- ing on the sentence level as opposed the token-level is a slightly harder task.
Token
Sentence
64
77.06 + 0.88
76.12 + 1.56
128
78.90 + 0.64
79.05 1.39
256
81.41 = 0.42
80.45 1.27
512
82.09 = 0.39
80.88 0.51
Table Model mean score different max token length and classification mode.
Cold-Start Problem
As chat history is used for the model, a cold-start problem may arise. To address this, we exam- ine the results of ToxBusterBASE (is not trained
'https://huggingfe hhttps://pytorch

---

1.0
1.0
Class Scamss and Ady Advertiienne riveemteeent vs. Rest
Basa mn (a carg) Closs Clos--ithrr oihhr offerdive content West (a0 0.31)
Closs Threets Rest
0.6
Peraal
resal
CAN Thrwets W. hest
0.4
0.4
EaSe THE
02
10
1.0
0.0
0.0
0.8
0.8
0.9
Claze Hete and Inrastmment VS. Rest
Class Extremism vs. Rost
Caacs laet und (leesnrrent vr. Sweet (19
0.6
0.5
Claco Spo.nmtte www.inggg VS. Wont CAn 265
PRocall
0.4
--- Ranaiia (0.N)))
Suzeiiee
0.2
0.0
8
00
100
08-
1.0
0.0
02
0.0
10
Class Minor Endangerment vs. Rost
Cless Not zolle vS. Rest
and vs.
sin staninn os ROST FUP 01233
0.6
Iecal
Recal
Clacs de IM 001
0.4
Heselee (3.02))
clooo 1roos Yoults
Baveline
0.
100
0.-
022
0.0
Figure 4: ToxBuster Precision-Recd Curve per Toxic Class on R6S.


Reviews:
[{"review_id": "lfMioNpp2c", "reviewer": "Reviewer_KywN", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# R1: The Uniqueness of Toxicity Detection in Gaming Platforms\n\nWe understand your concern about the uniqueness of our work in comparison to other toxicity detection settings. To clarify, toxicity detection in gaming platforms presents distinct challenges that differentiates it from other settings. In gaming platforms, in-game chat messages are **exceptionally brief, often containing less than five words, contains abundant usage of slang, and are susceptible to frequent misspellings.** This starkly contrasts with platforms such as social media or news article comments, where text is typically much longer such as the CC dataset that  averages around 55 words per line. Moreover, our model incorporates speaker segmentation, a crucial component to capture the essence of in-game chat conversations, as these often involve interactions between multiple users and different teams. This distinct **focus on the conversational aspect** is a departure from existing approaches that predominantly concentrate on single-line comments. We will further emphasize these unique aspects in our paper to provide a more comprehensive understanding of the nuances of toxicity detection for in-game chat.\n\n# R2 & R3: Innovations in Architecture\n\nWe appreciate your feedback on the novelty of our architecture.  To address this concern, we'd like to underscore that our approach introduces several innovative elements that distinguish it from existing toxicity detection methods. \n\nSpecifically, we **incorporate more than just the previous comment** (Toxicity Detection: Does Context Really Matter?) by utilizing all preceding chat history available as well. Additionally, we integrate speaker metadata, including TeamID, Chat Type, and PlayerID, as integral parts of the model. While previous research on toxicity detection has explored metadata aspects like article titles, usernames or game-specific slang (CONDA: a CONtextual Dual-Annotated dataset for in-game toxicity understanding and detection), none have **addressed the conversational aspect with this level of granularity**, especially in contexts where more than just the preceding line of text is used.  Finally, we also **modify the BERT model to incorporate the chat speaker segmentation as an additional input**. For better clarity, we will include the following detailed explanation of how we modify the BERT model in the paper. \n\n> As shown in Figure 1, ToxBuster is a BERT-based model with an additional three inputs provided. BERT’s input embedding is the sum of the position and token encoding. For ToxBuster, we introduce an encoding for each of the TeamID, Chat Type and PlayerID that corresponds to the token. Similar to BERT, the input embedding would then be the sum of all 5 encodings: Token, Position, TeamID, Chat Type and PlayerID. \n\n# R4. Model Simplicity and Comparison with LLMs\n\nWe acknowledge the reviewer's interest in comparing our model with LLMs and how they perform in such cases. First, we want to highlight that our model goes through specific data preprocessing before concatenating chat history. In the paper, we explore different filters for the chat history, focusing on personal, team, global, and moderator filters, determining their relevance and model performance impact to the current chat line. Our experimentation indicates that the global filter, which simulates the chat experience with all chat enabled, yields the most effective results. As previously discussed in R2 & R3, we also incorporate speaker segmentation to capture the conversational nuances present in gaming platform interactions.\n\nFurthermore, we'd like to highlight that deploying LLMs or calling GPT-3.5 / GPT-4 with OpenAI API for discriminative tasks such as toxicity detection is cost-prohibitive, as supported by the findings in \"Open, Closed, or Small Language Models for Text Classification?\" (https://arxiv.org/abs/2308.10092). The scalability and speed advantages of BERT-based models, i.e. ToxBuster, and its cost-effectiveness are particularly crucial for the high volumes of game chat typical of multiplayer games. We note that Perspective API, one of the baseline we compare to, is also a BERT-based model that is capable of real-time scenarios (response time ~100ms).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1.\tThe authors study an interesting an important problem- toxicity detection. \n2.\tThere are several experiments like performance comparisons, error analysis, ablation studies, and some exploratory experiments", "reasons_to_reject": "1.\tThe novelty of the work seems limited. It is not clear as to why detecting toxicity in gaming platforms unique compared to other settings for toxic content detection. \n2.\tThe proposed model is trivial. The authors propose to use BERT which crunches on chat history and current chat. There are no innovations on the architecture which are different from existing toxicity detection works. \n3.\tI would recommend the authors to clarify their innovations in their work. \n4.\tFurthermore, since the model simply feeds on a concatenation of chat history, it would be interesting to see how LLMs do in such cases (if possible to check).", "questions_to_authors": "", "paper_topic_and_main_contributions": "The authors study the problem of detecting toxicity on gamming platforms. They introdyce a model called ToxBuster which leverages chat history and meta-data for toxicity detection. They further experiment with ToxBuster on several gamming platforms.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "vEdzXwx6Tk", "reviewer": "Reviewer_BWUQ", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# A1: Framing of Contributions\n\nWe appreciate the reviewer's recognition of the importance of our paper's contributions to toxicity detection in online game chat. We'd like to clarify that our primary **contributions are twofold**:\n\n1. **Innovative Model**: We introduce ToxBuster, a BERT-based model that leverages chat history and speaker segmentation features for toxicity detection. Previous contextual toxicity detection models either only consider the preceding comment rather than the full history or use metadata such as article titles, usernames or game-specific slang (CONDA: a CONtextual Dual-Annotated dataset for in-game toxicity understanding and detection). With both chat history and speaker segmentation, our model ***addresses the conversational aspect for in-game chat that is novel in the context of toxicity detection***.\n2. **Practical Insights**: We provide valuable insights into the real-world applicability of our model, including cross-dataset experiments, post-game moderation analysis, and proactive moderation potential. First, the ***transferability between games*** and the CC dataset shows how our model and dataset is useful for game companies with more than one game or inviting collaborations between game companies. Hence, we also use categories defined under the Fair Play Alliance (FPA). Second, we provide post-game moderation analysis as the closest baseline to estimate the ***effectiveness of the model in a real-world setting***. Finally, we also provide the proactive moderation aspect as another potential use for real-world settings. While exploratory, it showcases how this ***model can proactively protect players in real-time***, i.e. muting a player for excessive toxic usage while the game is still active. These experiments showcase the model's practicality in a real-world setting.\n\nWe believe that these contributions are well-supported by our experiments and analysis. \n\n\n# R1: Method Reproducibility\n\n**1. Chat History for the CC dataset**:\n\nWe only briefly state that the comment history is created by “tracing all parent comments”.  To further clarify this, we **provide the following explanation**:\n> In the original Jigsaw’s Civil Comments dataset, each comment is accompanied with metadata such as, “id”,  “parent_id” and  “article_id”. In this case, a document is one full comment thread within the same article (i.e. identical “article_id”), starting from the root comment all the way down its child comments. Using the “parent_id”, we can build a tree of comments by backtracking the parent-child relationship formed when linking comments where the “parent_id” of a comment matches the “id” of another. We note that an article can have many trees of comment threads and that each path from the root of one tree to its leaf would then be one document. For the train & test split, we split by articles to prevent test leakage. The processed CC dataset and the corresponding processing code are also included in the supplementary material, ensuring complete transparency and ease of replication.\n\n**2. Chat Speaker Segmentation Representation & Implementation**: \n\nFor the chat speaker segmentation, we would like to **update Table 4 to increase the clarity**.\n\n | # | Line                         | PlayerID | Chat Type | TeamID |\n |---|------------------------------|----------|-----------|--------|\n | 1 | (Team) Apple: Hf          | 6        | Team      | 1      |\n | 2 | (All) Banana: Hf           | 5        | All       | 1      |\n | 3 | (Team) Grape: Which site? | 1        | Team      | 0      |\n | 4 | (Team) Orange: A           | 0        | Team      | 0      |\n | 5 | (All) Orange: Glhf         | 0        | All       | 0      |\n\nWe would also be **updating the paragraph about the metadata to the following**:\n\n> For the three chat metadata, *playerID* and *teamID* dynamically changes based on the player of the current chat line. *ChatType* can be either **team** or **all**, indicating whether a line is exclusive to the **team** or broadcasted to **all** players. *PlayerID* is the unique identifier for the player associated with the chat line, starting from **0** and bounded by the number of teams times the team size. For consistency, the player of the current chat line is always **0**. For other players on the same team, the identifier is incremented based on the recency of that player’s chat line. For players on the other team, the identifier starts from the size of the team, e.g, in a 5 v 5 game, the most recent opponent that has typed in chat will be **5**. With this scheme, the *playerID* can be extended to even Battle Royal games, where there can be multiple enemy teams. *TeamID* is the unique identifier of the team the current player belongs to. For consistency, the current player is always team **0**. The enemy team would be team **1**. For battle royal games, this scheme can be extended similarly to the playerID. The last three columns in Table 4 describe the *playerID*, *chatType* and *teamID* when detecting toxicity for line 5.\n\nFinally, we will be **adding a new paragraph about the model architecture**, as it is only briefly mentioned in the Figure 1 captions but not in text to the following:\n\n> As shown in Figure 1, ToxBuster is a BERT-based model with an additional three inputs provided. BERT’s input embedding is the sum of the position and token encoding. For ToxBuster, we introduce an encoding for each of the TeamID, Chat Type and PlayerID that corresponds to the token. Similar to BERT, the input embedding would then be the sum of all 5 encodings: Token, Position, TeamID, Chat Type and PlayerID. \n\n**3. Missing game-specific metadata:**\n\nWe acknowledge that not all datasets may contain game-specific metadata, such as the Chat Type or TeamID. To address this issue, we **refer to Table 10**, the ablation study for the chat speaker segmentation which provides the impact of each new input encoding (Player ID, Chat Type, Team ID) individually and with all included. If the reviewer is asking for a comprehensive ablation study with all possible combinations, we can include that as a table in the Appendix in the camera-ready version. \n\n# R2: Cold-Start Problem\n\nWe appreciate the reviewer for bringing up the concern of cold-start as our model is trained with chat history and may be dependent on it, causing a cold-start problem. We address this by examining the results of ToxBuster_BASE (is not trained and doesn't use chat history) and ToxBuster_FULL (trained with chat history and speaker segmentation) on the R6S test set. We bin the number of chat lines in the chat history and report the F1-score. We can see that using **chat history will always help with the performance**, although **marginally with 0 or 1 lines of chat history** and much **higher for more lines of chat history**. This could indeed explain some of the variance we have reported in the paper. More interestingly, we see a large improvement in performance for 21 - 30 lines and the improvement slightly dropping afterwards, suggesting that this may be the ideal length of chat history for this dataset. This is a useful insight that can be further explored in future works to improve the model performance and we thank the reviewer again for the comment which led to the new analysis.  \n\n| # of Lines in Chat History | Support | ToxBuster_BASE | ToxBuster_FULL |\n|----------------------------|---------|----------------|----------------|\n| 0                          | 96      | 81.69          | 81.72          |\n| 1                          | 93      | 80.65          | 81.68          |\n| 2 - 10                     | 811     | 78.15          | 83.43          |\n| 11 - 20                    | 901     | 77.92          | 82.22          |\n| 21 - 30                    | 772     | 75.87          | 84.25          |\n| 31 - 40                    | 717     | 77.35          | 82.75          |\n| 41+                        | 14447   | 77.14          | 83.65          |\n\n\n  \n\t\n# W3: Contextual Toxicity Detection Models \n\nWe appreciate the reviewer's feedback, especially with the citation of two papers on contextual toxicity detection models. We acknowledge that incorporating context for toxic or abusive language detection is not new, as stated in our related works. We note that many papers focus on tweets or comment threads, which have much longer words per line compared to in-game chat. We would also like to stress that to our knowledge, **none of the toxicity models consider more than the preceding comment or line**. They also **do not consider the conversational aspect** when there is more than one preceding comment. Regarding the comparative studies the reviewers cited, they actually **support our claim** further.:\n\n* **Comparative Studies of Detecting Abusive Language on Twitter**: The paper does a comprehensive study on different models while including “context tweets”. Based on our understanding and the examples the authors provide, this only includes the preceding tweet, i.e. the tweet that is replied to. This study also **confirms our statement** of “yielding only marginal improvements” as shown when the overall F1 score of their CNN and RNN models actually decrease when context is provided. CNN with context does perform better on the HATE class with an increase of  .05 for its F1 score. \n\n* **Pay “Attention” to Your Context when Classifying Abusive Language**:  One of the main results of the paper is the usage of “context attention” which is jointly learned when training the model. In their results of comparing whether to stack Bi-LSTM and whether to use self-attention or context-attention, the stacked Bi-LSTM with context attention prevails with an increase of the F1-score by less than 1 in 3 of their datasets. While interesting, we note that 1) it only changes **how it attends to the context around the word in the given sentence** (model architecture), 2) does not take into account any conversational aspect. \n\n\n# W3:  Real-time Detection Capability\nWe understand the reviewer's concern regarding real-time detection capability. We show **real-world applicability of our model by including cross-dataset experiments, post-game moderation analysis, and proactive moderation potential**. As all previous chat history can be provided, a contextual real-time in-game chat toxicity detection is feasible. We thank the reviewer for bringing the concern of the cold-start. The results shown in R2 should help answer that concern. Additionally, we would like to highlight that this research is in **collaboration with a company and is undergoing the industrialization process, i.e.  real-world deployment**.\n\nWe sincerely appreciate the reviewer's constructive feedback, and we are committed to addressing these concerns in the revised manuscript. We believe these updates will strengthen our paper and make our contributions more evident and reproducible. Thank you for your valuable input.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "A1. This paper tackles an important problem of toxic speech detection in online platforms. The new annotation dataset could be a valuable resource to the NLP researchers on abusive languages because they were constructed on real-time chat on online games.\n- A downside regarding this point is that the whole paper is framed for other types of contributions, which are not adequately supported by the paper and experiments.\n\nA2. They conducted various experiments to understand the detection ability of the proposed method. In particular, the cross-dataset experiment provided insights into practical considerations on using the ML model. It was interesting to see the CC-based model can achieve a reliable performance on the game chat data.", "reasons_to_reject": "R1. The paper does not provide enough details to reproduce the proposed method.\n- It misses the details of how the chat history feature is implemented for the CC dataset.\n- I don’t clearly understand how the chat speaker segmentation embeddings are represented and incorporated with the BERT input embeddings.\n- Some datasets, even a game dataset, may not provide all the information required for building the model. For example, there could be a game that does not provide Team ID for chat data collection. More ablation experiments could be conducted to understand the effects of each component of chat segmentation features.\n\nR2. While the experiments show the importance of considering chat history for better detection, the high variance in the evaluation of chat segmentation features suggests that its attribution to performance is not clearly understood.\n- History features may suffer from cold-start problems, which could be analyzed in additional experiments.\n\nW3. In the Introduction, this paper argued that “most existing approaches either neglect context or yield only marginal improvements and not fit for real-time moderation for an in-game chat” to support its method-wise contribution. I believe some of the arguments are wrong, and others require more justification or experiments.\n- The idea of considering additional contexts has been widely discussed in the context of abusive language detection (https://arxiv.org/abs/1808.10245, https://aclanthology.org/W19-3508/). \n- Despite some analyses presented in the paper, I was not fully convinced of the real-time detection capability of the proposed method. A potential issue is a cold-start problem, as I describe above.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper addresses the problem of real-time toxicity detection for in-game chat. They propose ToxBuster, a BERT-based model that employs chat history and speaker segmentation features for detection. Experiments show that the proposed model outperforms three off-the-shelf general-purpose toxicity detection models. They presented ablation studies to show the effectiveness of each sub-module. Cross-prediction experiments across different datasets suggest that it can achieve a reasonable performance for predicting toxicity for different games. This paper could make a resource contribution to the NLP researchers on abusive languages. There could be a possibility of method-wise contribution, but it seems to require further investigation.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "To6sYFjgRY", "reviewer": "Reviewer_yvYH", "summary": "", "questions": "A. It seems like this is not the first model to incorporate context into toxicity or hate speech detection (as mentioned in Related Work). Have you tried implementing other models that use context and comparing against them? Or, as a weaker baseline, using the APIs on each line but using the score from the lines in the history to help predict whether the current one is toxic?\nB. Are the ablation analyses in Section 4.6 only computed on FH and R6S? Do you know why these additions (chat history and metadata) don't seem to improve performance as much for the other two datasets in Table 5?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# R1: Simplistic Method Additions; adding context is commonly done\n\nWe acknowledge that contextual toxicity detection has been explored in the literature, referring back to our related works section. However, our approach **distinguishes itself by emphasizing the conversational aspect**, which has not been extensively addressed before. Specifically, our model incorporates the entire chat history, offering varying scopes of history (e.g., moderator scope vs. player scope) and chat metadata. While these additions may seem straightforward, they represent a novel contribution to the field of toxicity detection.\n\n\n# R2:  Low Improvement on CC and DOTA 2 Dataset\n\nWe appreciate your observation regarding the performance discrepancy between ToxBuster_BASE and ToxBuster_FULL on certain datasets. Regarding the CC dataset, it primarily consists of **comments, which tend to be longer in nature (~55 words per line)**. In such cases, the marginal gains from context are expected, as evidenced by similar studies focusing on tweets and comments. In contrast, in-game chat, as seen in R6S and FH, is notably shorter, where the inclusion of all possible chat history is particularly effective for toxicity detection. As for the DOTA 2 dataset, it was adapted from a different task with two levels of annotation. The first level of annotation for toxicity is based off of a list of keywords, making the contextual information less critical. The second level of annotation on the sentence level is human-annotated for explicit and implicit toxicity, where explicit toxicity had to contain words that were labeled toxic. Only about 7% of the lines had implicit toxicity and could not be labeled as toxic with a keyword-based annotation. We provide the F1-score of ToxBuster_BASE and ToxBuster_FULL in the table below where we group the samples by the original class of the human-annotation. We see that there is a **higher performance gain in the implicit class** compared to the explicit class, as implicit toxicity benefits more from context as opposed to the explicit class.\n\n| Original Class | Support | ToxBuster_BASE | ToxBuster_FULL |\n|---------|------------|-------------------------|------------------------|\n| Explicit  | 1950  | 77.81   | 78.58  |\n| Implicit  | 146    | 64.27   | 67.54  |\n| Non-Toxic | 2123 | 91.14 | 91.16 | \n\n# R3: Varied Performance; Low F1-Score on Certain Categories\n\nWe recognize the variability in performance across toxicity classes, particularly for categories with limited training data such as threats and minor endangerment. We acknowledge it in the class-wise evaluation section and plan to **address it in future work**. Our ongoing research focuses on methods to improve the F1 scores for each category. Furthermore, we intend to **explicitly mention the low-performing categories in the limitations section** in the camera-ready version..\n\n# Q1: Baseline with other contextual toxicity detection models.\n\nWe appreciate your suggestion regarding the comparison with other contextual models and the weak baseline. To clarify, while previous contextual toxicity detection models have explored aspects like metadata (e.g. article title, usernames), our emphasis is on conversational metadata, which presents a unique context not addressed by existing models. For the preceding comment, we **point towards Table 13 in the Appendix**. While not directly comparable to only including the preceding comment, the lower the max token length, the less chat history we have. We see that as we increase the max token length, i.e. the more chat history that is available, the better our model performs on the R6S and FH dataset. We would be happy to include results of limiting our model to a specific number of lines of chat as part of the Appendix for the camera-ready version.\n\nAs for the use of APIs on each line and incorporating historical scores, it is an intriguing idea. Designing this baseline was not straightforward for us, but based on your suggestion, we’re thinking of training a classifier on two features, the average of toxicity scores of the past chat line and the current toxicity score of the chat line from Perspective API to predict the toxicity label of the current chat line. Due to the short time of the rebuttal, we will be **adding this as a baseline in the camera-ready version**. Although we want to highlight that at the end it is not ideal for the solution to depend on a third-party API, and a solution like ToxBuster which can be deployed in house is more practical. \n\n# Q2:  Clarification on Ablation Analysis\n\nIndeed, the ablation analyses were **conducted solely on the FH and R6S datasets, which are the best match with our task given being from in-game chats and being properly labeled**. As for the performance variation across the CC and DOTA 2 datasets which are not ideal for our task, CC is not from the game domain, and DOTA 2 is not properly labeled as we discussed earlier in R2. We believe these factors contribute to the observed performance differences and will provide further clarification in the revised manuscript.\n\n\n\nOnce again, we sincerely thank you for your valuable feedback and will diligently address your comments and suggestions in the camera-ready version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- ToxBuster outperforms the baseline models across four datasets\n- Strong results from transferability and adaptability results\n- Thorough evaluation: class-wise, error analysis\n- Promising results in post-game moderation, with a high proportion of chat-reported players flagged by ToxBuster  ", "reasons_to_reject": "- Method additions beyond BERT_base are relatively simple (chat history and chat metadata); adding context is commonly done\n- These method additions also don't improve performance much for two of the datasets, comparing ToxBuster-full and ToxBuster-base\n- Performance varies substantially across toxicity classes and is much worse for certain classes, eg, F1 below 30 ", "questions_to_authors": "A. It seems like this is not the first model to incorporate context into toxicity or hate speech detection (as mentioned in Related Work). Have you tried implementing other models that use context and comparing against them? Or, as a weaker baseline, using the APIs on each line but using the score from the lines in the history to help predict whether the current one is toxic?\nB. Are the ablation analyses in Section 4.6 only computed on FH and R6S? Do you know why these additions (chat history and metadata) don't seem to improve performance as much for the other two datasets in Table 5?", "paper_topic_and_main_contributions": "This paper introduces ToxBuster, a new model for detecting toxicity in game chat. Their model is based on BERT_Base, with two additions. First, they incorporate chat history into the model, testing different \"scopes\" of history (ie, which chat lines are visible, eg, moderator scope can see all lines and personal can only see lines by the current player). Second, they incorporate chat metadata, including player ID, chat type, and team ID. They test their model on four datasets -- two datasets that they construct (R6S and FH) and two existing datasets (DOTA 2 and CC) -- and compare to three baselines (Cleanspeak, Perspective API, and Detoxify, a model from 2020). They show that their model outperforms the baselines on all datasets, and their full model significantly outperforms BERT_Base on two of the datasets (the ones they curate). \n\nThen, they conduct a series of detailed analyses of ToxBuster. They analyze transferability across datasets and find better transferability within the gaming domain (compared to CC) and within similar time periods. They find that ToxBuster, when trained on R6S and adapted to FH, performs very well. They also conduct class-wise evaluations, showing differential performance across different types of toxicity; an error analysis of false positives and negatives; ablation studies; evaluations of recall in high-precision settings; and comparisons to ToxBuster's high-precision predictions to actually flagged players, finding that ToxBuster flags a large proportion of these users.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Table 6 should define what metric it is showing - F1? "}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors 

However, the proposed analysis shows some drawbacks and limitations:
1) The novelty of the work seems limited. It is not clear why detecting toxicity in gaming platforms is unique compared to other settings for toxic content detection;
2) The chat speaker segmentation embeddings should be better described, as well as discussing how it is integrated into the architecture.
3) The proposed approach slightly improve the performance w.r.t. the other approaches.
4) Performance varies substantially across toxicity classes and is much worse for certain classes, eg, F1 below 30
Decision: Accept-Findings