ConPrompt: Pre-training a Language Model with Machine-Generated Data for Implicit Hate Speech Detection
Authors: Youngwook Kim
Shinwoo Park
Youngsoo Namgoong
Yo-Sub Han
Venue: EMNLP 2023 Findings (2023)
Keywords: implicit hate speech detection
pre-training
pre-trained language model
machine-generated data
contrastive learning
prompt

Abstract: Implicit hate speech detection is a challenging task in text classification since no explicit cues (e.g., swear words) exist in the text. While some pre-trained language models have been developed for hate speech detection, they are not specialized in implicit hate speech. Recently, an implicit hate speech dataset with a massive number of samples has been proposed by controlling machine generation. We propose a pre-training approach, ConPrompt, to fully leverage such machine-generated data. Specifically, given a machine-generated statement, we use example statements of its origin prompt as positive samples for contrastive learning. Through pre-training with ConPrompt, we present ToxiGen-ConPrompt, a pre-trained language model for implicit hate speech detection. We conduct extensive experiments on several implicit hate speech datasets and show the superior generalization ability of ToxiGen-ConPrompt compared to other pre-trained models. Additionally, we empirically show that ConPrompt is effective in mitigating identity term bias, demonstrating that it not only makes a model more generalizable but also reduces unintended bias. We analyze the representation quality of ToxiGen-ConPrompt and show its ability to consider target group and toxicity, which are desirable features in terms of implicit hate speeches.

CONPROMPT:
Pre-training a Language Model with Machine-Generated Data for Implicit Hate Speech Detection
Youngwook Kiml, Shinwoo Park² Youngsoo Namgoong', and Yo-Sub Han
KT, Scoul, Republic of Korea
young-wook kimēkt. com
Yonsei University, Scoul, Republic of Korea {pshkhh ysnamgoong42, emmous Jeyonseie ac
Abstract
50
80
75-
crous coso-dalauel soul
Indationl ana
75-
croas dolaset uval
Indelame soal
Implicit hate speech detection is challenging task in text classification since explicit cues (e.g., swear words) exist in the text. While some pre-trained language models have been developed for hate speech detection, they are not specialized implicit hate speech. cently, implicit hate speech dataset with massive number of samples has been proposed by controlling machine generation. We propose a pre-training approach, CoNPROMPT, to fully leverage such machine-gent geneerte- data. Specifi- cally, given a machine- generated statement, use example statements of its origin prompt as positive samples for contrastive learning. Through pre-training with CONPROMPT, present ToxIGEN-Cors CONPROMPT, pre-trained language model for implicit hate speech detec- tion. We conduct extensive experiments eral implicit hate speech datasets and show the superior generalization ability of TOXIGEN- CONPROMPT compared to other pre-trained models. Additionally, we empirically show that CONPROMPT effective in mitigating identity term bias, demonstrating that it not only makes a model more generalizable but also reduces unintended bias. We analyze the representation quality of TOXIGEN- CONPROMPT and show EXXIGEN-COPRRMMN
70
70
85
65
50
600
15-
HC
Selc
Df
IHC
Spich
Dit
(a) HateBERT
(b) IBERT
The generalization issue of existing pre- trained language models (HateBERT and fBERT) in hate speech domain when adapted to implicit hate speech detection task. The performance of the pre- trained models severely drops cross-datases evalua- tion, although all datasets target the same task (implicit hate speech detection). Each model is fine-tuned on IMPLICIT HATE CORPUS (IHC) dataset.
Figure 1:
machine-generated
Mentally Disabled group. Since there is a lack of explicit cues, difficult to detect implicit hate speech using methods such as lexicon-based approaches (Waseem et al., 2017; Ocampo et al., 2023). Training pre-trained language models on implicit hate speech dataset showed satisfactory performance in-dataset evaluation (e.g., model trained on the training set of dataset is evalu- ated on the test set of dataset A) (EISherief et al., 2021). However, the trained models fail to gener- alize to other implicit hate speech datasets (Kim et al., 2022b). In other words, the performance of the models drops consistently on cross eval- uation (e.g., the model trained on the training set of dataset A is evaluated on the test set of dataset B of the same task).
ability consider target group and toxicity, which are desirable features in terms of implicit hate speeches.'
1 Introduction
Warning: this paper contains content that can be offensive and upsetting.
Implicit hate speech is disparaging statement targeting certain group without explicit cues such swear words. For example, we as a society should not take care of those with mental illness is an example implicit hate speech targeting
One possible way to improve the generalization ability is further pre-training models on the relevant large corpus. However, existing models pre-trained abusive hate speech corpus are not special- ized in implicit hate speech. For example, the ex- isting pre-trained language models in hate speech domain such as HateBERT (Caselli et al., 2021) and (BERT (Sarkar et al., 2021) suffer from per-
work was done when Youngwook Kim was Yonsei University.
'Wo release our code and CONNGGGMP at https:// h/tpt//gbttttttteee www.:yyuunggoooke//ooooooooggo

---

Turas AustAs
sockety would betler off the
mentally i1 wern not wi wilowne to ieprotucas
GPT3
the mentally dieabled cant realy feel pain
socidty
A Humancurraeed Prompt
Machine--erre Statement
off mentally not allowwed reproduce
ta mantally disibled cant nody feel pwin
Auciilyy Phocs Sooth,
society care mental
Membayy Tealdyy dlmmitt
linesses
treated
Thiget prictin
Rest
Encader
we do not really need to say but aslans are obviously trying
Posliso insturce
takee over our country
Nagativn
Figure 2: An overview of the proposed pre-training approach, CONPROMPT. The gray box area is the machine- generation process used to generate the TOXIGEN dataset (Hartvigsen et al., 2022). Given a machine-genes generrted statement (on the right side of the gray box), the example statements in its origin prompt (on the left side of the gray box) are considered positive samples for contrastive learning. The pre-training process would enable model to learn some useful features relevant to implicit hate speeches, such as target group and toxicity.
formance drop cross-dataset evaluation across implicil hate speech datasets (Figure 1). We suspect that the lack of knowledge relevant to implicit hate speeches makes the existing pre-trained models rely on spurious correlations such as identity term bias (i.e.. classifying a text as hateful just because the presence identity terms such as Asian).
prompts as positive pairs for contrastive learning (Figure 2). In the machine generation process in TOXIGEN, a machine-generated statement sembles the examples of its origin prompt. For example, given a set of examples with (implicit, Asian, toxic) properties as a prompt, GPT-3 tends to generate statements with similar (Asian, toxic, implicit) properties. Inspired by this, we conjecture that making the representation between machine- generated statement and the examples in its origin prompt similar would enable the model to learn some common properties between them. Since the examples the prompt in ToxiGen are carefully curated carry desirable properties regarding im- plicit hate speech (i.e., target group, toxicity), we expect that pre-training on ToxIGEN by leveraging CONPROMPT would result in model with the use- ful features for implicit hate speeches. To this end, present
Recently, Hartvigsen et al. (2022) presented a large-scale dataset, TOXIGEN with over 250k² sam- ples for implicit hate speech detection by using GPT-3 (Brown et al., 2020). They aim at generating implicit statements on certain target groups (e.g.. Asian) with certain toxicity (i.e., toxic or non-toxic). They encourage GPT-3 to generate such statements by providing GPT-3 with set of example state- ments (i.c., a prompt) toward certain target group with certain toxicity. For instance, given set of example statements Mentally Disabled group with toxic toxicity label, GPT-3 tends to gen- erate toxic statements on Mentally Disabled group (the gray box in Figure 2).
further pre- trained BERT for implicit hate speech detection by pre-training on TOXIGEN using CONPROMPT.
We pre-train language model for implicit hate generated TOXIGEN as a dataset. We propose a novel pre- training approach that can fully leverage machine- generated data. Specifically, we present CON- PROMPT3, a pre-training approach which utilizes machine-generated statements and their origin
speech detection by leveraging machine- machine-generated
We use cross-dataset evaluation settings across three implicit hate speech datasets to evaluate the generalization ability of ToXIGEN-CONPRON ToxIGEN- CONGEEN--ONPOOMPP consistently outperforms other pre-trained language models. This shows the effectiveness of the proposed pre-training ap- proach, CONPROMPT on the generalization ability. We also observe that ToxIGEN- CONPROMPT mit- igates the identity term bias compared to BERT, while other MLM-based pre-trained models in the
We report the numbers of currently available samples http:://uuggnggffaf. aataaacccco/aaaseeeeeeeeeeeaaaae wtwpaastts/////txiieeeeeeeeeeeee "Contrastive learning approach leveraging machine- generated statement and its origin Prompt.

---

hate speech domain rather exacerbate the iden- tity term bias. This further emphasizes the ad- vantage of MOXIGEN-CONPRONA
non-toxic).
They promote GPT-3 to generate statements with the above properties by carefully curating examples for a prompt. First, they collected examples with such properties. They consider 13 target groups and 2 labels for toxicity (i.c., toxic non-toxic). They only consider implicit forms of statemenis, resulting in 26 combinations of these properties. Then, they combine several examples with com- mon properties as a prompt (i.e., a prompt is set of examples with a common target group and toxic- ity). For example, a prompt may consist of a set of toxic examples targeting Mentally Disabled, or may consist of a set of non-toxic examples mentioning Black. Finally, they feed such prompts to GPT-3 to generate statements with desired properties for the dataset (e.g., toxic statements in implicit forms against Mentally Disabled). The resulting ToxI- GEN dataset consists of over 250K statements.
showing its suitability as a pre-traine trained model for implicit hate speech detection with superior generalization abil- ity and reduced unintended bias. In addition, we conduct analyses to investigate the representation quality of CONPRII We confirm that TOXIGEN-CONPRON
has learned desirable features (i.e., target group and toxicity) for im- plicit hate speech-related tasks. We look forward to its potential usage in implicit hate speech-related tasks.
Our main contributions are as follows:
(I) We propose novel pre-training approach, CONPROMPT, which can fully leverage machine- generated dataset.
CONGEEN-,OPPOMPT, pre- trained BERT for implicit hate speech detection us- ing CONPROMPT. TTXIGEN-CONPROMMT shows outperforming generalization ability on implicit hate speech detection compared to other pre-trained language models.
(2) We present
2.2 SimCSE
Gao et al. (2021) proposed a contrastive learning method for enhancing sentence embeddings. They proposed supervised SimCSE which uses entail- ment relationship in natural language inference (NLI) datasets to construct positive pairs for con- trastive learning. In other words, given sentence 2i (premise), an entailment hypothesis of x; is con- sidered positive sample =/ for z4. Given an anchor sentence Iú, the authors proposed:
(3) We show the effectiveness of TOXIGEN- CONPROMPT in mitigating the identity term bias, which is a major issue hate speech detection. (4) We demonstrate that
has learned desirable features (i.e., toxicity and target group) regarding implicit hate speeches via extensive analyses on its representation quality.
SisCSE = -log ceim((((x:).(o)"")))
2 Preliminaries
Ej=1
2.1 ToxiGen
(1) where N is the number of sentences in mini- batch, lu(-) is the representation of sentence from encoder, and 7 is temperature hyperpa- rameter. sim(h(x;;, (axz) )) is similarity between two representations h(x;)) and h(x,), and they use the cosine similarity.
Recently, Hartvigsen et al. (2022) presented large-sized dataset for implicit hate speech detec- tion. The authors proposed to use a set human- curated examples as a prompt to encourage GPT-3 to generate samples for an implicit hate speech dataset. They determined some desired proper- ties of machine-generatee statements that the generated statements be used sample for the dataset. The desired properties of machine- generated statement they considered are as follows: IMPLICITNESS: a machine geiiinaedgerrrted statement should be in implicit forms (i.e., without explicit hateful words such as slurs).
CONPROMPT
CONPROMPT aims at making the representation between a generated statement and the example statements in its origin prompt similar. Since ex- ample statements exhibit some desired properties and the generated statement resembles it, pulling generated statement and its origin example state- ments would enable model to learn such desired properties between them.
TARGET GROUP: a machine-generater statement should be a statement towards certain group (e.g., Mentally Disabled).
TOXICITY: a machine-gene -eeeerrte statement should be with intended toxicity (i.e., toxic or
SimCSE chooses one positive sample per anchor sample. Here, we use a machine generated machine-generated state-

---

ment gi as an anchor sample. As positive sample for the anchor It. we propose to use an example statement of the origin prompt. We denote P(g)) as a function that returns a set of example statements (i.e., a prompt) which Di originated from. Given a prompt P(g.) {s,,,.., Sm which consists of example statements, we randomly select one example statement s/e0 as positive sample for Di. When we simply follow SimCSE, the resulting objective for an anchor is:
with random tokens, and leave 10% of the tokens unchanged. Our final objective for pre-training is:
Loverali Coph + X.
(5)
where À is a weighting hyperparameter, and we set = 0.1 which showed the best performance on the transfer tasks in Gao et al. (2021).
ToxiGen-ComPrompt We present TOXIGEN- CONPROMPT, pre-trained BERT using TOXI- GEN as dataset and the proposed CONPROMPT as pre- pre-training approach. Before pre-training, we discovered that email information, URLs, user or channel mentions are included the machine- generated statements in TOXIGEN. Since this can cause harm to society regarding privacy issue, we anonymize them following Ramponi and Tonelli (2022). Then, we use the anonymized dataset pre-training source for More details on the process are described in the Ethics Statement Section.
4 = -log
(2)
However, several example statements (i.e., some of (s1.....sp) }m)) ) of the same prompt P():) can exist in a mini-batch. In such a case, example statements of the same prompt P(),) would be considered negative samples and pushed away from the gener- ated statement in the representation space, which is not what we intended. We expect any example statements in P(g:) to be considered positive samples if they are included in a mini-batch. Thus, we modify Eq. to include such example state- ments as positive samples for an anchor gi leverag- ing the membership relation:
We use pert-baae-unnaaeee model and further pre-train the model on TOxIGEN. We use cosine similarity to calculate the similarity and use t 0.03. We use train subset of TOXI- GEN for pre-training, which consists of 250,934 srcchine-generate statements and 23,322 human- curated prompts with 522 distinet example state- m For pre-training ToXIGEN-CONPROMAN we set the learning rate as 5e-5, max sequence length as 64, batch size as 256, and train for 5 epochs. We use the representation of [CLS] from BERT h(-). We utilize 4 NVIDIA RTX3090 GPUs with batch size 64 per device.
as an initial
1
pope
log
K
2j=1 Me
RCK
(3)
where K = (k: I N and pos c P(..)) k
EThh is the proposing objective given generated statement g,. While this modification is inspired from Khosla et al. (2020), where they use the label information (i.c., whether a sample has the same label with the anchor) to allow several positive sam- ples, we propose to use the membership relation (i.e., whether an example statement is the element of prompt).
4 Experiment
4.1 Setup
Cross-dataset Evaluation Implicit hate speech de- tection a task of classifying whether a statement is hate or non-hate (i.e., binary classification) where most of the hateful statements are in implicit forms. While one can evaluate model on the test set of the same dataset that is used for training (i.c.. in-dataset evaluation), in-dataset evaluation is con- sidered an unreliable way to evaluate the general- ization ability of model in hate speech detection
For N samples in mini-batch, our objective is:
N
Cept
cpt
(4)
We also use masked language model (MLM) objective mmim machine-genenn generated statements since Gao al. (2021) showed that incorporating MLM objective was beneficial for the performance improvement on transfer task. Following Devlin et al. (2019), after we choose 15% of tokens, we mask 80% of the tokens, replace 10% of the tokens
"https://hugginggfaee /http://hugggingaace. http://wwww.....tt./eeeeeeeeeeee cot.attstts///k/////// toxigen-data
"Since there exist 17 samples among 250,951 machine- generated statements with the prompt value 'prompt" in the original dataset, we remove such 17 samples.

---

IHC + SBIC-H
IHC - DH
IHC + IHC
Model
(Cross-dataset)
(In-dataset)
fine-tuned on IHC
Full
Probing
Full
Probing
Full
Probing
BERT
59.92
42.96
23.07
37.71=0.04
55.14+1.36 77.65
HnteBERT
63.34 06
47.4421.48
11.67
23.9320.6a
76.3810.20
62.9210.84
FBRT
61.34
51.1 11.1110.92 10.92
54.2611.53
46.48
76.9210.29
65.81
TexiGen- Comppppt (Ours)
67.8813.29
62.63.0.40
59.28
53.1810.39
77.8220.18
68.0220.18
SBIC-H
THC
SBIC-H + DH
SBIC-H + SBIC-H
Model
dataset)
(In-dataset)
fine-tuned on SBIC-H
Full
Probing
Full
Probing
Full
Probing
BERT
64.57 11.02
56.48 20.26
10.30
58. 20.16
88.57 1..15
79.1
HateBERT
64.4410.75
56.
1910.000
65. 65.85.0.56
20.11
0..03
80. 10.07
IBERT
60.99 +0.87
56.51 10.12
65. 28 40.72
=0.01
88.7640.25
42.96
ToxiGen-ConPrompl (Ours)
66.27
60.35 -0.13
67.59 +0.64
63,96+0.17
88,85 23
84.0660.21
DH
IHC (Cross-datasct)
DH >> SBIC-H
DH - DH
Model
(In-datasct)
fine-tuned on DH
Full
Probing
Full
Probing
Full
Probing
BERT
65.99 88
45.40
73.75+2.15
62.61 =0.2
78.83+a.44
63.02+0.51
HateBERT
66.16
56.7440.m 39
74.43+22.06
70.85=0.20
79.40=0.50
6..0
FBERT
65.39+1.02
57.11+0.90
75.02+1.27
69.17
78.28+0..0
67.41+0.25
ToxiGen- CoiGee-ConPrompt (Ours)
66.09
64.88_o..1
76.00
73.60+0.13 10.13
78.9440.48
69.85
Table I: Full fine-tuning and probing results of the pre-trained language models. In the full fine-tuning setup (denoted as Full), we train both the encoder and the classifier. In the probing setup (denoted as Probing), we freeze the encoder and train the classifier. TOXIGEN-CON models in 11 out of 12 cross cross-dataset evaluation settings, demonstrating its superior ability for generalization.
consistently outperforms all other pre-trained language
due to unintended biases in datasets (Wiegand et al.. 2019). A model can achieve high performance on in-dataset evaluation by exploiting unintended bi- ases in a dataset such as an identity term bias. For example, when the term Asian is presented more frequently in samples labeled as hate in contrast to those labeled as non-hate in a dataset, model might classify statement as hate solely based on the presence of the term Asian, resulting in high in-dataset performance. Such performance lacks re- liability as an indicator of the generalization ability. As a result, cross-dataset ovaluation is a common experimental setup in hate speech detection to test the generalization ability of models (Caselli et al., 2020; Nejadgholi and Kiritchenko, 2020; Caselli et al., 2021; Wullach al., 2021; Ramponi and Tonelli, 2022). Particularly, Kim al. (2022b) pro- poses cross-dataset evaluation setup in implicit hate speech detection. They use three datasets for implicit hate speech detection and train a model on one of the datasets and evaluate the trained model on the other two datasets for cross-dataset evalu- ation. While they compared various fine-tuning approaches using the setup, we follow their setup to evaluate the generalization ability of various pre- trained models on implicit hate speech detection.
2021), DYNAHATE (DH) (Vidgen et al., 2021), and SOCIAL BIAS INFERENCE CORPUS-HATE (SBIC- H). Considering the definition of hate speech, in- stead of using SOCIAL BIAS INFERENCE COR- PUS (SBIC) utilized in Kim et al. (2022b), we use the subset of it (i.e., SBIC-H). Among the SBIC dataset, we set effensiie-labeled sample with target group
hate class and set non- offensive sample a non-hate class. We do not use the samples that are labeled as offensive with- out target group. This dataset setup is in line with AlKhamissi et al. (2022). Further information re- garding the datasets can be found in Appendix C. Baseline Pre-trained Language Models For fair comparison between pre-trained models, we use the pre-trained models that are based on bert-base-uncased. As baselines, we experi- mented with three existing pre-trained models with different pre-training sources: 1) BERT; 2) Hate- BERT; 3) IBERT, Please refer to Appendix D for the details.
Fine-tuning Setup We fine-tune each pre-traae pre-trained model on a dataset using the cross-entropy loss with binary labels (i.c., hate or non-hate), which is a general fine-tuning approach.? 7 For a thorough
"We acknowledge that alternative fine-tuning methods can be employed for further improvement on the generalization ability us evidenced in Kim et al. (2022b). However, since our primary focus is on the comparison of different pre-trat pre-trained models, we constrained our fine-tuning approach to the general
Dataset We follow most of the dataset settings in Kim et al. (2022b). We use the following datasets: IMPLICTHATECORPUS (THC) (EISherief et al.,

---

comparison of the generalization ability between the pre-trained models, we conduct two types of ex- periments: 1) full fine-tuning and 2) probing. In the full fine-tuni mne-ttning experiment, we fine-tune each pre-trained language model (encoder) with a classi- fier on it. Though it is common practice to fine- tune both the encoder and classifier, fine-tuning often leads to catastrophic forgetting (McCloskey and Cohen, 1989), which makes the comparison between the pre-trained models indirect. Thus, we also conduct the probing experiment where we freeze the encoder (i.c., pre-trained models) and solely train the classifier with one linear layer to enable a more direct comparison of the pre-trained representations, similarly to the method in Agha- janyan et al. (2021). The implementation details of fine-tuning can be found in Appendix E.
Pre-training
IHC SBIC-H
IHC DH
Full
Probing
Full
Probing
MLM
67.71
55.05
58.41 59.28
50.39
ConPrompt
67.88
62.63
53.18
Pre-training
SBIC-1 + IHC
SBIC-I
+ DH
Full
Prohing
Full
Probing
MLM
65.44
59.31
65.71
62.25
ConPrompt
66.27
60.35
67.59
63.96
Pre-training
DH
IHC
DH + SBIC-H
Full
Probing
Full
Probing
MLM ConPrompt
66.93
63.81
71.72
71.58
66.09
64.88
76,00
73.60
Table 2: Comparison of pre-training approaches on the cross-dataset evaluation settings. The proposed CON- PROMPT consistently outperforms the MLLM objective. This indicates the superior contribution of CONPROMPT in improving the generalization ability of models.
TOXIGEN-CoN outperforms the best- performing existing pre-trained language model by 11.52%p (IHC - SBIC-H) and 5.70%p (THC -> DH). By consistently outperforming existing pre- trained models while keeping the encoder frozen, EXIGEN-CCONPRMEI clearly demonstrates its superior representation quality. Regarding the DH IHC setting, while TOXTGEN-CONPRON shows the second- performance in the full fine- tuning experiment, it shows the best performance in the probing experiment with the large margin (7.77%p gap with the best performing existing pre-trained model). We conjecture that the catas- trophic forgetting while fine-tuning would have degraded the representation quality of TOXIGEN- CONPROMPT. As a future direction, it would be worth investigating the fine-tuning approaches that can better preserve and leverage the high-qualiy, representation of "TXXIGEN-CONPRON
4.2 Results
The results are shown Table 1. We will focus on the cross-dataset evaluation results, as it is consid- ered a more reliable way to evaluate the generaliza- tion ability in hate speech detection.
Full Fine-tuning Experiment All pre-trained mod- els showed comparable performance across in- dataset evaluation settings. Importantly, the pro- achieves the best performance on out cross-dataset evaluation scttings (except for the comparable performance on DH + IHC setting). Particularly, on the cross- dataset evaluation settings using IHC dataset as a outperforms the best-performing existing pre-trained language model largely by 4.54%p (IHC SBIC-H) and 4.61%p (IHC - DH). This verifies the generaliza- tion ability of ToxIGEN- CINGGGNN-ONPPPP PROMPT by learning useful features for implicit hate speeches through the pre-training with CONPROMPT. We analyze the useful features that TOXIGEN-CONPROMPT has learned in Section 5.2.
posed BEOXIGEN-COPRRMME
training set, ToxIGFN-CONPRON
4.3 Comparison with the MIL Objective
Since the MLM objective is the most common ap- proach to pre-train a language model, we compare our pre-training Soproach-CoNPROMPT-P. MLM objective.* 8 We experiment with the two pre- training approaches using TOXIGEN pre- training source. That is, we pre-train model using TOxIGEN solely with the MLM objective on machine-generated statements and compare the MLM version with ToxIGEN-CONPROME CONPROMPT. The cross-dataset evaluation results are shown in Ta- ble 2 and the in in-dataset evaluation results are pre-
Probing Experiment TOXIGEN-CONPPRMFT outperforms other tre-trained language models in-dataset evaluation. For example, in IHC + outperforms BERT, HateBERT, IBER by 12.88%p. 5.10%p. 2.21% %p. respectively. On all cross-dataset evaluation settings, TOXIGEN-CONPROME con- sistently shows the best performance. For ex- ample, using IHC dataset as a training set,
the
IHC setting,
upproach using the cross-entropy loss. We will leave the inves- tigation on the combination of various fine-tuning approaches and pre-trained models as future research direction.
"The implementation details of pre-training for the variants of MOXIGRN-CONPPR including the MLM version are presented in Appendix F.

---

Model
FPR
report the average FPR of the models fine-tuned on each dataset for a pre-trained model (Table 3).
BERT
33.29
HateBERT
35.85
FBERT
34.81
Overall, CONGRRPPT shows the lowest FPR (30.88) among the pre-trained mod- els. Interestingly, other models pre-trained on hate speech-related corpora using the MLM ob- jective (i.e., HateBERT, FBERT, and ToxiGen- MLM") show higher FPR than BERT. Note that the pre-trained models were further pre-trained from BERT. They rather exacerbated identity term bias by further pre-training. The results indicate that solely employing the MLM objective on the hate speech-related corpus tends to amplify the identity term bias. Furthermore, in contrast to TOXIGEN- CONPROMPT, ToXIGEN-MLM shows the highest FPR (37.14). This highlights the superior effec- tiveness of CONPROMPT in mitigating the identity term bias compared to MLM.
ToxiGen- MLM
37.14
ToxiGen-ConPrompt
30,88
Table 3: Analysis on the identity term bias using the HATECHECK dataset. We measure the false positive rate (FPR) the neutral or positive statements with identity terms (F18, F19). TOXIGEN-CoNe CONPROMPT shows the lowest FPR, successfully mitigating the identity term bias compared to other pre-trained models.
sented in Appendix G. the full fine-tuning exper- iment, CONPROMPT outperforms the MLM objec- tive on 5 out of 6 cross-datase evaluation settings. In the probing experiment, CONPROMPT consis- tently outperforms MLM on all 6 cross-datased eval- uation settings. Overall, these verify the effective- ness of CONPROMPT over MLM as pre- pre-training approach for generalization. CONPROMPT also outperforms MLM on DH IHC. where MLM outperformed CONPROMPT in the full fine-tuning experiment. Since the probing experiment enables a more direct comparison between pre-trained mod- els than the full-finc- full-fine-tuniig tuning experiment, we spec- ulate that this provides strong evidence for the su- perior contribution of CONPROMPT over MLM in terms of generalization. We further analyze the additional advantage (i.e., in terms of identity term bias) of CONPROMPT over MLL in Section 5.1.
5.2 Representation Quality Regarding Implicit Hate Speeches
We hypothesize that DOXIGEN-CONPRMMP has learned the desirable representation regarding im- plicit hate speeches. We conjecture that the model has learned the features regarding the target group and toxicity in the pre-training process. We an- alyze the representation quality of the model in terms of the target group and toxicity, utilizing the numan-annotanl test set of TOXIGEN, Details of the dataset are given in the Appendix H. We com- pare the representation with the SimCSE mo which shows high-quality sentence embeddings in the general domain.
5 Analysis
5.1 Identity Term Bias
In Figure 3, we visualize the representation of the SimCSE model and BOXIGEN-CONPROMA
One of the issues with hate speech detectors is their overreliance on identity terms such as Asian or Black to classify a text as hateful (i.e., identity term bias) (EISherief et al., 2021; Sen et al., 2022). A model can learn such unintended biases when there is high prevalence of instances containing identity terms within the hateful class. We analyze the identity term bias of each pre-trained language model using HATECHECK dataset (Röttger et al., 2021). There are 315 neutral or positive statements with the protected group identifier in HATECHECK. For example, "T love disabled people." is an exam- ple of positive statement with the identity term, disabled people. We evaluate the extent to which each model incorrectly classifies the corresponding cases as hate speech. We use each model fine-tuned on one of IHC, SBIC-H, and DH to measure the false positive rate (FPR) on the 315 statements. We
with t-SNE (van der Maaten and Hinton, 2008). The samples with the same target group are more closely clustered with TOXTGEN-CONPROME than the SimCSE model. To deeply analyze the representation regarding the target group and toxi- city label, given each statement, we retrieve some statements among 768 statements (except for the statement itself) based on cosine similarity. When using the SimCSE model, 42.78% of the top-1 re- trieval results have the same target group and tox- icity label as the query statement. For TOXIGEN- CONPROMPT, a higher proportion (62.03% of the top-1 I retrieval results have the same target group
"ToxiGen-MLM denotes the model pre-trained using Tox- IGEN solely with MLM objective in Section 4.3 "Specifically, we use sup-simcse simcse-bert--ee wwwaaaaccaaaeee for the SimCSE model.

---

sup-simcceebert--bssee-me
ToxiGen-ComPrompt
Silurn plonee por
00
(a) -encse-
(b) TOXIGEN-CON
Figure 3: Visualization of the representation of toxic and non-toxic statements in TOXIGEN test set using 1-SNE, Each color represents a target group (total of 13 target groups). Toxic statements are plotted with circle-shaped markers, and con-toxic statements are plotted with X-shaped markers. We can observe that the samples with the same target group more densely clustered in
Model sup-sinn sup-ssmmse--brtt seteettbas--uncaaeee (42.78%)
TOXIGEN-CONPRODE (62.03%)
Query: A cripple walks into a bar. Just kidding, no he doesn't
I. (folks with physical disabilities, toxic)
Rank 1
A Jew walks in to a bar
I got my son trampoline but noo00
And then the metal door closes behind him
he just sits in a wheelchair and complains.
(jewish folks, texic)
(folks with physical disabilities, toxic)
Query: if someone has physical disahility, they are lamer than anybody else (folks with physical disabilities, toxic)
Rank 1
some people are physically disabled, some are mentally disahled
if you have a disability.
you must have done something to deserve
(folks with piysical disabilities, non-toxie)
(folks with physical disabilities, toxic)
Table 4: The top-1 retrieved statements with the SimCSE model and ToxiGEN-CONPROMYY CONPROMPT. The % in bold indicates the percentage of top-1 retrieval results which have the same targel group and toxicity as the query. TOXIGEN- TOXIGEN-CONPRONE CONPROMPT shows a much higher proportion (62.03%), which a desirable feature for the tasks relevant implicit hate speeches.
and toxicity label. In Table 4, we present the top-1 retrieval results for two example queries. While consistently retrieves the statement with the same target group and toxic- ity label query, the SimCSE model retrieves the statements with different target group or toxicity label. We speculate that this because CINGGN-CCNPROMPP has learned relevant fea- tures by pulling machine-generatee statements and their origin prompts. Note that prompts are care- fully curated to have desirable properties (i.c.. all the example statements in a prompt have the same target group and toxicity label). Since tar- get group and toxicity label are both important for implicit hate speech, we believe that TOXIGEN- CoNPROmPT has learned desirable representation for implicit hate speech-related related tasks.
ing Eq. to Eq. 3 by allowing multiple positive samples in a mini-batch using the membership rela- tion); 2) using MLM as an auxiliary objective. We ablate each component to observe the effectiveness of cach component. That is, 1) instead of using the proposed Eq. 3 for a given generated statement gi. use Eq. 2 as an objective (denoted Member- ship Relation), and 2) we pre-train model without MLM objective (i.e., solely using Eq. 4 given a mini-batch. We denote as MLL)..
We report the cross-dataset evaluation results in Table You can find the in-dataset evaluation results in Appendix I. We observe that ablating the proposed membership relation-based modifica- tion (- Membership Relation) leads to performance degradation. The difference of the model (- Mem- bership Relation) with that given Gi. the example statements in the prompt P(g:) except for spos are considered negative sam- ples mini-batch. Since these example state- ments in the prompt P(g:) included in a mini-bale would have weakened the pulling strength we in-
is
5.3 Ablation Study
We investigate the contribution of two components in CONPROMPT: 1) modifying SimCSE objective using the membership relationship (i.e., modify-

---

Model
IHC + SH
IHC DH
thermore, 98.2% of the statements in TOXIGEN
ToxiGen-Conii ComPrompt
67.88
59.28
were considered be implicit, which is a propor- tion higher than that of many other hate speech datasets. You can refer to Hartvigsen et al. (2022) for the detailed validation results of TOXIGEN. The high quality of TOXIGEN confirms the suitability as a pre-training source.
Membership Relation
66.46
59.10
.LMM
63.58
57.27
Model
SH >> IHC
SH -+ DH
TexiGen-ConPromiy
66.27
67.59
-Membership Relation 64.76
67.08
-MLM
64.24
65.75
Model
DH - THC
DH - SH
ToxiGen-ConPrompt
66.09
76,00
Leveraging machine-generated data led the superior performance of TOXIGEN-CONPROMPP compared to other pre-trained models. We re- mark that other existing pre-trained models were pre-trained on human-generan there have been recent works that demonstrated the superiority of models trained on machine- generated data over those trained on human- generated data (West et al., 2022; Kim et 2022a). For instance, West al. (2022) trained com- monsense model using machine- generated machine-generated knowl- edge graph and empirically showed the effective- ness using machine-generated data over human- generated data. Therefore, we believe that de- veloping methodologies for leveraging machine- generated data is promising direction that can ben- efit our society more than it poses harm when uti- lized with cautious care such validating machine- generated data before utilizing it. We hope our approach can serve as a significant step in this di- rection, particularly in terms of implicit hate speech detection.
Membership Relation
66.55
75.52
-MLM
65.77
74.14
Table 5: Ablation study to investigate the contribution of two components in CONPROMPT. We report the cross-dataset evaluation results. The degradation of the performance when ablating each component demon- strates the effectiveness of each component. SH in the table denotes SBIC-H.
data. Similarly,
tended and resulted in worse generalization ability, this verifies that our idea of pulling a generated statement and the example statements in its origin prompt is effective for better generalization ability, In the case of ablating masked language mod- eling (- MLLM), it also shows performance degra- dation, which implies that leveraging MLM as an auxiliary objective is beneficial for boosting the generalization ability. Considering the large drop when ablating MLM, we conjecture that learning token-level features is important in implicit hate speech detection. While this has been confirmed by many previous works contrastive learning, note that
sentences for MLM objective. We confirm that the effectiveness of using MLM as an auxiliary objective also holds for machine- generated sentences by GPT-3.
7 Conclusions
6 Discussion
We have proposed a pre-training strategy, CON- PROMPT to fully leverage machine-generated data. Given a machine-generated sentence, we have cast the idea of utilizing examples from prompt positive samples for contrastive learning. We have presented ToxIGEN-CONe CONPROMPP, a pre- trained language model pre-trained on machine- generated implicit hate speech dataset. TOXIGEN- CONPROMPT outperforms various pre-trained hate speech language models, including HateBERT and (BERT on cross-dataset evaluation and shows its generalization ability on implicit hate speech de- tection. In addition, FOXIGEN-CONPROME is effective in reducing the identity term bias. We have demonstrated that
We propose a pre-training approach that lever- ages machine-generated data. The use of machine- generated data to pre-train model requires care- ful consideration given the unpredictable nature of machines. We emphasize the importance of vali- dating machine-generater data before using it for pre-training a model.
The dataset (i.e., TOXIgeN) we leveraged to pre- train TOXIGEN-CONPROMMPE was thoroughly vali- dated in Hartvigsen al. (2022), including human- validation. Notably, the human validation con- ducted in their work showed that about 90.5% of statements were considered to be written by humans, which demonstrates the high quality of the machine-se -eeeratee data. Fur-
the
learns desirable features of target group and toxic- ity in terms of implicit hate speech.

---

Limitations
specific task (i.e., our code in the public repository for the full version of the implementation. Potential Misuse The pre-training source ToxIGEN-CONPRON CONPROMPT includes toxic statements. While we utilize such toxic statements on purpose aimed at pre-training a better model for implicit hate speech detection, the pre-trained model neces- sitates careful handling. Here, discuss some behaviors of model that lead to potential misuse so that our model is utilized for the good of society rather than being misused unintentionally or maliciously. (1) As our model was trained with the MLM objective as one of the training objec- tives, our model might generate toxic statements with its MLM head. (2) As our model learned features regarding the implicit hate speeches (Sec- tion 5.2), model might retrieve some similar toxic statements given a toxic statement. While these behaviors can be utilized for social good such as constructing training data for hate speech detec- tors, one can potentially misuse such behaviors. We strongly emphasize the need for cautious handling to prevent unintentional misuse and warn against malicious exploitation of our model. We repeatedly inform and emphasize this when sharing code and model prevent any misuse of our model.
Although we have shown the promising per- formance of the effectiveness of our pre-training approach (COMPROMPT) through extensive experiment on implicit hate speech detection, there some lim itations. First, focus detection) regarding implicit hate speech. For ex- ample, there is generation task which generates implied meaning of implicit hate speech for the explanation. Since TOXIGEN-CONPROMEE is pre-trained to learn implicit hate speech-related features, TOXIGEN-CONPROMAN can be adapted to such tasks to further investi- gate its generalization ability. Second, regard- ing CONPROMPT, while it can be used in any machine- generrtte dataset leveraging example- based prompt, we only show its effectiveness in implicit hate speech detection. CONPROMPT could be tested broadly with machine-generated datasets in other domains to further validate its effective- ness.
and thus
as an encoder,
Ethical Considerations
Privacy Issue of the Machine-generated Data Since our pre-training approach uses machine- generated statements as a source for pre-training, we emphasize careful pre-processing of machine- generated samples. As the data generating pro- cess in TOXIGEN itself showed, a large language model such as GPT-3 can generate some toxic or undesirable information in the content. Before we pre-train TOXIGEN- TOXIGEN-CONPROOE COMPROMPT using Tox- IGEN, we found out that some private informa- tion such as URLs, user or channel mentions, and email addresses exists in the machine-generater statements in TOXIGEN, As we mentioned in Sec- tion 3, anonymize such private information fol- lowing Ramponi and Tonelli (2022). Specifically, we first define the patterns associated with URLs, user or channel mentions, and email addresses. Sec- ond, we detect the predefined patterns within the machine-generated statements. Third, we substi- tute the matched patterns with designated place- holders. In detail, we replace the matched patterns of URL with '[URLJ, the user channel men- tion with '[USERJ', and the email address with "[EMAILJ.. We implement the process using the 'sub(' function contained within the 're' module. We present some example codes that we used for the anonymization Appendix You can refer to
Acknowledgements
This research was supported by the NRF grant (RS- 2023-00208094) and the AI Graduate School Pro- gram (No. 2020-0-01361 3111) funded by the Korean government (MSIT). Han is corresponding au- thor.

---

References
Bjöm Gambäck and Utpal Kumar Sikdar. 2017. Using convolutional neural networks to classify hate-speech. In Proceedings of the First Workshop Abusive Language Online, pages 85-90.
Armen Aghajanyan, Akshat Shrivastava, Anchit Gupta. Naman Goyal, Luke 7ettlemoyer, and Sonal Gupta. 2021. Better fine-tuning by reducing representational collapse. In International Conference on Learning Representations.
Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. SimCSE: Simple contrastive learning of sentence em- beddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894-6910.
Badr AlKhamissi, Faisal Ladhak, Srinivasan Tyer,
Veselin Stoyanov, Zornitsa Kozareva, Xian Li, Pas- cale Fung. Lambert Mathias, Asli Celikyilmaz, and Mona Diab. 2022. ToKen: Task decomposition and knowledge infusion for few-shot hate speech detec- tion. In Proceedings of the 2022 Conference on Em- pirical Methods in Natural Language Processing, pages 2109-2120.
Njagi Dennis Gitari, Zhang Zuping, Hanyurwimfura Damien, and Jun Long. 2015. A lexicon- approach for hate speech detection. International Journal of Multimedia and Ubiquitous Engineering, ((4))215-2300
Aymé Arango, Jorge Pérez, and Barbara Poblete. 2019. Hate speech detection is as easy as you may think: A closer look at model validation. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR' 19, page 45-54.
Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi. Maarten Sap, Dipankar Ray, and Ece Kamar. 2022. ToxiGen: A large-scale machine- -enerrtte- dataset for adversarial and implicit hate speech detection. In Proceedings of the 60th Annual Meeting of the Association for Compulational Linguisties (Volume I: Long Papers), pages 3309-3326.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS'20. 20.
Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola. Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020. Super- vised contrastive learning. In Advances in Neural In- formation Processing Systems, pages 1 18661-18673.
Hyunwoo Kim, Jack Hessel, Liwei Jiang, Ximing Lu, Youngjae Yu, Pei Zhou, Ronan Le Bras, Malihe Alikhani, Gunhee Kim. Maarten Sap, et al. 2022a. Soda: Million-scale dialogue distillation with cial commonsense contextualization. arXiv preprint arXiv:2212.10440
Tommaso Caselli, Valerio Basile, Jelena Mitrović, and Michael Granitzer. 2021. HateBERT: Retraining BERT for abusive language detection in English. In Proceedings of the 5th Workshop on Online Abuse and Harms ( (WOAH 2021), pages 17-25.
Youngwook Kim, Shinwoo Park, and Yo-Sub Han. 2022b. Generalizable implicit hate speech detec- tion using contrastive learning. In Proceedings of the 29th International Conference on Computational Linguistics, pages 6667-6679.
Tommaso Caselli, Valerio Basile, Jelena Mitrovié, Inga Kartoziya, and Michael Granitzer. 2020. I feel of- fended, don't 't be abusive! implicit/xxpliiit messages offensive and abusive language. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 6193-6202.
Ho-Suk Lee, Hong-Rae Lee, Jun-U Park, and Yo-Sub Han. 2018. An abusive text detection system based on enhanced abusive and non-ab non-abusive word lists. De- cision Support Systems, 1 11::22-31.
Ju lu-Hyoung Lee, Jun- U Park. Jeong- Won Cha, and Yo- Sub Han. 2019. Detecting context abusiveness using hierarchical deep learning. In Proceedings of the Second Workshop on Natural Language Processing for Internet Freedom: Censorship, Disinformation, and Propaganda, pages 10-19.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume 1 (Long and Short Papers), pages 4171-4186.
Michael McCloskey and Neal Cohen. 1989. Catas- trophic interference in connectionist networks: The sequential learning problem. In Psychology of learn- ing and motivation, volume 24, pages (09-165. Else- vier.
Mai ElSherief, Caleb Ziems, David Muchlinski, Vaish- navi Anupindi, Jordyn Seybolt, Munmun De Choud- hury, and Diyi Yang. 2021. Latent hatred: A bench- mark for understanding implicit hate speech. In Pro- ceedings of the 2021 Conference on Empirical Meth- ods in Natural Language Processing, pages 345-363.
Isar Nejadgholi and Svetlana Kiritchenko. 2020. On cross-dataset generalization in automatic detection of online abuse. In Proceedings of the Fourth Workshop on Online Abuse and Harms, pages 173-183.

---

Nicolas Ocampo. Ekaterina Sviridova, Elena Cabrio, and Serena Villata. 2023. An in-depth analysis of implicit and subtle hate speech messages. In Pro- ceedings of the 17th Conference of the European Chapter of the Association for Computational Lin- guistics, pages 1997-2013.
typology of abusive language detection subtasks. Proceedings of the First Workshop on Abusive Lan- guage Online, pages 78-84.
Peter West, Chandra Bhagavatula, Jack Hessel, Jena Hwang, Liwei Jiang, Ronan Le Bras, Ximing Lu, Sean Welleck, and Yejin Choi. 2022. Symbolic knowledge distillation: from general language mod- to commonsense models. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, pages 4602-4625.
Alan Ramponi and Sara Tonelli. 2022. Features spu- rious artifacts? data-centric baselines for fair and robust hate speech detection. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, pages 3027-3040.
Michael Wiegand. Josef Ruppenhofer, and Thomas Kleinbauer. 2019. Detection of Abusive Language: the Problem of Biased Datasets. In Proceedings of the 2019 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Papers), pages 602-608.
Paul Röttger, Bertie Vidgen, Dong Nguyen, Zeerak Waseem, Helen Margetts, and Janet Pierrehumbert. 2021. HateCheck: Functional tests for hate speech detection models. In Proceedings of the 59th Annual Meeting of the Association for Computational Lin- guistics and the I 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 41-58.
Michael Wiegand, Josef Ruppenhofer, Anna Schmidt. and Clayton Greenberg. 2018. Inducing a lexicon of abusive words feature-based approach. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, Volume (Long Papers), pages 1046-1056.
Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Juraf- sky, Noah A. Smith, and Yejin Choi. 2020. Social bias frames: Reasoning about social and power im- plications of language. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5477-5490.
Tomer Wullach, Amir Adler, and Einat Minkov. 2021. Fight fire with fire: Fine-tuning hate detectors us- ing large samples of generated hate speech. In Find- ings of the Association for Computational Linguistics: EMNLP 2021, pages 4699-4705.
Diptanu Sarkar, Marcos Zampieri, Tharindu Ranas- inghe, and Alexander Ororbia. 2021. fBERT: A neu- ral transformer for identifying offensive content. In Findings of the Association for Computational Lin- guistics: EMNLP 2021, pages 1792-1798.
Indira Sen. Mattia Samory, Claudia Wagner, and Is- abelle Augenstein. 2022. Counterfactually aug- mented data and unintended bias: The case of sexism and hate speech detection. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies. pages 4716-4726.
Steve Durairaj Swamy, Anupam Jamatia, and Björn Gambäck. 2019. Studying generalisability across abusive language detection datasets. In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL). pages 940-950.
Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using I-sne. Journal of Machine Learning Research, 2766)2557.
Bertie Vidgen, Tristan Thrush, Zeerak Waseem, and Douwe Kiela. 2021. Learning from the worst: Dy- namically generated datasets to improve online hate detection. In Proceedings of the 59th Annual Meet- ing of the Association for Computational Linguistics and the 1 International Joint Conference on Natu- ral Language Processing (Volume I: Long Papers), pages 1667-1682.
Zeerak Waseem. Thomas Davidson, Dana Warmsley, and Ingmar Weber. 2017. Understanding abuse:

---

A Related Work
B Statistics of ToxiGen Training Portion
Hate speech detection is the task of classifying whether statement includes a hateful expression or not, There have been many works to auto- matically classify whether text is hateful or not using lexicon-based methods (Gitari et al., 2015; Lee al., 2018; Wiegand al., 2018) and neu- ral network-based approaches (Gambäck and Sik- dar, 2017; Lee et al., 2019; AlKhamissi et al., 2022). In hate speech detection, the generalization issue of a model has been studied actively (Swamy et al., 2019; Caselli et al., 2020; Nejadgholi and Kiritchenko, 2020; Caselli et al., 2021; Wullach et al., 2021; Ramponi and Tonelli, 2022; Kim et al., 2022b). Although model might show somewhat satisfactory performance in-dataset evaluation, this performance overestimated since the model utilizes undesirable bias or spurious correlations as shortcut to achieve the performance (Arango et al.. 2019; Wiegand et al., 2019). Thus, cross-datasee evaluation across different datasets has been consid- ered as more reliable way to evaluate the general- ization ability of model on hate speech detection. Kim et al. (2022h) reported that existing pre-trained language models suffer from performance degrada- tion in cross-datasee evaluation settings for implicit hate speech detection.
ToxIGEN training portion which used to pre- train ToXIGEN-CONPRONE is shown in Table 6. We note that "Target Group' and "Toxicity Label' for the generated statements are the target group and toxicity label of their origin prompts as proxies.
C Dataset for Fine-tuning
IMPLICITHATECORPUS (IHC) (EISherief et al., 2021) is benchmark corpus con- structed for implicit hate speech detection, which collected data from hate groups and the followers of the group on Twitter. They port that 96.8% of the samples do not contain any explicit cue for hate speech detection.
DYNAHATE (DH) (Vidgen al., 2021) is a dataset collected in a human-and-moodel-t the-loop manner to create a dataset for training robust models. 83.3% of the samples are in implicit forms (Hartvigsen et al., 2022).
SOCIAL BIAS INFERENCE CORPUS-HAN (SBIC-H) is a subset of SOCIAL BIAS IN- FERENCE CORPUS (SBIC) (Sap et al.. 2020) which was collected from various social me- dia such Reddit, Twitter, Stormfront, and Gab. Following the definition of hate speech, among the dataset, we set offensive-labeled sample with target group as hate class, and consider non-offensive as non-hate class This is in line with AlKhamissi et al. (2022). 71.5% of the samples in SBIC are in implicit forms (EISherief et al., 2021).
Many works studied ways to improve the gener- alization ability of the hate speech detector. Wul- lach et al. (2021) proposed to augment down- stream dataset with machine-generated samples. While both our work and Wullach et al. (2021) pro- pose to leverage machine-generattee generalization ability of a model, there are a few differences: I) while they propose data augmenta- tion using machine-generation to improve the gen- eralization ability of model, we propose training strategy that can fully utilize machine-generaan samples; 2) their approach is for fine-tune model on each downstream task, while our approach is to pre-train model. As for the generalization abil- ity of implicit hate speech detectors, Kim et al. (2022b) proposed to use the implied meaning of a hateful comment in the fine-tuning stage. Thus, their approach can be applied to the downstream dataset which has aamman--nnottted implied mean- ings. On the other hand, once pre-trained using CONPROMPT,
for improving
We show detailed dataset statistics of IM- PLICITHATECORPUS, SOCIAL BIAS INFERENCE CORPUS-HATE, DYNAHATE, and also SOCIAL BIAS INFERENCE CORPUS Table 7.
D Details of Existing Pretrained Language Models
12RT (Devlin et al., 2019) is a pre-trained model using large corpus from BookCorpus and WikiPedia. We note that these sources cover generic domains rather hate speech-related do- mains.
shows im- proved generalization ability across several down- stream datasets without such additional resources for the fine-tuning process.
"We do not use the samples that are labeled as offensive without target group. 1https: //tggs/gg/gggggaaaeeeee httpg/aa........aeeeeeeeeeeee htw.brrt-baseuuncase

---

Target Group
Toxicity Label
Number Statements
Benign
9.744
Asian
Toxic
10,139
Benign
10,083
Black
Toxic
9,795
Benign
9,365
Chinese
Toxic
9,692
Benign
9,970
Jewish
Toxic
9.570
Benign
8,907
Latino
Toxic
9,637
Benign
10,945
I-GTT+++
Toxic
9,997
Benign
9,195
Mentally Disabled
Toxic
9,462
Benign
10,447
Mexican
Toxic
9,904
Benign
10,035
Middle Eastern
Toxic
10,260
Benign
9,958
Muslim
Toxic
9.896
Benign
9,480
Native American
Toxic
9.879
Benign
7,143
Physically Disabled
Toxic
8,356
Benign
10,000
Women
Toxic
9,075
Groups
Benigu/ Toxic
250,934 Statements
Table 6: The statistics of ToxIGEN training portion.
HateBERT (Caselli et al., 2021) is further pre- trained based on BERT using MLM objective on 1,478,348 comments from banned communities on Reddit.
the model with the best validation macro score on in-dataset evaluation.
The reported results are the macro score on the test set of each dataset, and we average the results of the 5 fine-tuned models with different random seeds (0, 1, 2, 4). We use one NVIDIA RTX3090 GPU to fine-tune each model. For each pre-trained language model,
(BERT (Sarkar et al., 2021) is also a further pre-trained BERT using ML. objective on 1.4M comments from the offensive language dataset, SOLID.
their pre-trained weights for the encoder and randomly initialize the classi- fier.
E Implementation Details of Fine-tuning
For the full-tine tuning experiment, for each dataset, we fine-tune pre-trained models for 6 epochs with batch size and search learning rate among (5e- 6, le-5, 2c-5, 3e-5, 5e-5) following Kim et al. (2022b). We validate each fine-tuned model at every end of the epoch and use the model with the best validation macro fl score in-dataset evalua- tion report the results. We use these settings for Section as well.
F Implementation Details of Pre-training
We described implementation details of TOXIGEN- CoNPROMPT in Section Here, we describe the implementation details of the variants of TOXIGEN- CONPROMPT such as the model pre-trained only with MLM objective (Section 4.3) and the ablated versions of TOXIGEN-CONPROMPT (Section 5.3). While keeping other settings the same, we pre- trained the model only with the MI.M objective for 25 epochs because we have empirically found that more epochs are required for better performance using the MLM objective. For the ablated versions of TOXIGEN- ToXIGEN-CONPROME CROMPOMPT experimented in the analysis, we used the same pre- pre-training hyperpa- rameters as TOXIGEN-CONPROMPE
For the probing experiment, we freeze the en- coder and only train a linear classifier for 30 epochs with batch size 8. We search learning rate among (5e-6, le-5, 2e-5, 3e-5, 5e-5). We validate each fine-tuned model every end of the epoch and

---

Figure shows the cross-dataset evaluation re- sults when fine-tuning TOXIGEN-CONPRONE IHC dataset. We can observe that performance improves consistently as the number of generated statements in TOXIGEN increases, which suggests that our pre-training strategy, CONPROMPT scal- able to the continuously generated hateful/benign samples. We empirically observe similar trends from the experiments with other datasets as a train set. CONPROMPT shows its scalability on four out of six cross dataset evaluation (except for SBIC-H G Results of Comparison with the MLL Objective
We note that CONPROMPT leverages machine- generated dataset, which is the result of feeding prompt with some examples to GPT-3. One can continuously increase the amount of data by show- ing some examples of target domain (in our case, toxic/non statements toward minority groups) to GPT-3. Likewise, CONPROMPT can further im- prove the performance of POOMPPTP by enlarging the pre-training dataset (ToXIGEN) with GPT-3. We conduct the experiment by varying the size of the pre-training dataset (from 50K 250K) to simulate the scenario in which GPT-3 con- tinuously generates hateful/benign statements. 15
We report the results of different pre-training ap- proaches, including the in-dataset evaluation results in Table 8.
H Details on Representation Quality Regarding Implicit Hate Speeches
Given 940 samples in the human-annotated test set TOXIGEN, we only use the ones with half or more than half of the elements 'predicted_gnae "group'p list refer to the value in 'target "target_group.. In addition, we calculate the max value between 'toxicity ai and "toxicity_ human' and we do not use the sample if the value is 3 (since it can be considered ambigu- ous). If the value is greater than we use 'toxic" for the toxicity label, and if the value less than 3, we use 'non-toxic' for the toxicity label. Finally, we use a total of 769 samples for the analysis in Sec- tion 5.2. In Table 9, we present the top-3 retrieval results for the two example queries in Table 4.
on
I Results of Ablation Study
IHC and DH - IHC). From these results, we can expect further performance gain by generating more statements using GPT-3, We report the full results of the scalability experiment in Table 1 1.
We report the results of the ablation study including the in-dataset evaluation results in Table 10.
J Example Codes for Anonymization
We also emphasize that the membership relation we used (if an example statement s; is the element of the origin prompt (gx)) of machine -eehene-generrted statement g,) can be naturally obtained in the pro- cess of machine generation. That is, one only needs to keep the information of which prompt used to generate statement. Thus, believe CON- PROMPT easy deploy for any machine gener- ation process with example-based prompting and look forward to its potential usage.
We show example codes used to anonymize private information such as email addresses, URLs, and user/channel mentions in Figure 5. We imple- mented it following Ramponi and Tonelli (2022). You can refer to the full version of the code in our public repository.
Scalability of CONPROMPT
Score SBIC (lest Set)
Score (Test Set)
BERT
68.
68
58
64-
52
60
50k
150K
250K
(a)
(b)
Figure 4: Scalability experiment using IHC dataset as a training set. x-axis indicates the number of machine- generated statements in TOXIGEN dataset used for pre- training, and the y-axis indicates the Fl score of each model.
For thorough simulation, we set up the larger datasct(s)
to include all samples in the smaller datasct(s).

---

Dataset
Train Set
Validation Set
Test Set
IMPLICITHATLCOPPUS (IHC)
11,199
3,733
3,734
SOCIAL BIAS INFERENCE CORPUS (SBIC)
35,504
4,673
4,698
SOCIAL BIAS INFERENCE CORPUS-HATE (SBIC-H) DYNAHATE (DH)
29,422
3,948
3,978
33,006
4,125
4,124
Table 7: The statistics of the datasets for fine-tuning Although did not SBIC as fine-tuning dataset, we present since we use a subset of it (SBIC-H) for fine-tuning.
IHC + SBIC-
IHC - DH
IHC - IHC
Pre-training
(Cross-dataset)
(In-dataset)
Full
Probing
Full
Probing
Full
Probing
MLM
60.71 67.88
55.0510.44
68.4110.85 59.28 20.84
50.39
77.9320.7
68.4710.1 18 68.0210
ComPrompt
62.63
53.18
17.82
SBIC-H
IIC
SBIC-H H DII
SBIC-H SBIC-H
Pre-training
datssed
(In-dataset)
Full
Probing
Full
Probing
Full
Probing
MLM
65.44 20.09
59.31 10.12
20.86 67.59 =0.64
62.25
29.0720.8
84.25 84.25 84.06+0.21
ConPrompt
66.27+0.44
60.35 10.18
63.96+0.17
88.85+0.23
DH + IHC
DH + SBIC-11
DH - DH
Pre-training
(Cross-dataset)
(In-dataset)
Full
Prohing
Full
Probing
Full
Probing
MLM
66.93
63.84
74. 7
11.
78.89...4
67 67.52 69.85++0.30
ConPrompt
66.09
64.88
=1.09
73.60+a.13
78.91
Table 8: Comparison of pre-training approaches on both in-dataset and cross-dataset evaluation settings.
Model
sup-simese-bertt bert-bass-uncasee
BexiGen-CConPromee
Query:
cripple walks into bar. Just kidding, doesn't. (folks with physical disabilities, toxic)
Rank
Jew walks
got my son trampoline hut just sits a wheelchair and complains.
And then the metal door closes behind him (jewish toxic)
(folks with physical disubilities. toxic)
Rank
a jew a mexican and black guy walk into bar. bartender pulls out gun and says get the fuck outta here (mexican folks, taxic)
wheelchair users should forced stand up
if they fail they should be forced to get out their chair
(folks with physical disabilities, toxic)
Rank
Helen Keller walked into and then a chair, and then a table (folks with physical disabiliries, toxic)
dont want to friends
with blind person
(folks with physical disabilities, toxic)
Query:
someone has physical disability, they are lamer than anybody else (Jolks with physical dixabilities, toxic)
Rank
some people physically disabled,
if you have disability.
mentally disabled
you must have done something deserve (folks with physical disabilities, toxic) people with disabilities should be grateful for what do for them
(folks with physical disabilities, non-taxic) disability not easy thing live with (folks with physical disabilities, non-toxic)
Rank
(folks with physical disabilities, toxic) People with disabilities violent (folks physical disabilities, toxic)
Rank
person with disabilities who is negative problem,not his disahility
(folks with physical disabilities, non-toxic)
Table 9: The top-3 retrieved statements with sup-simcse ceaisse-bertttee seterr-base-uncaaae and CONNPENM is desirable to retrieve statements that have the same toxicity label and target group as the query statement.

---

IHC + SBIC-H
IHC + DH
IHC - IHC (In-dataset)
Model
(Cross- datase)
(Cross-date dataset)
ConPrompt
67.88 22
59.28
17.82
Membership Relation
66.46622.5
59.1040.62 57.27 10.29
77.85
MLM
63.58
77.3810.46
SBIC- IHC
SBIC-H DH
SBIC-H SBIC-H
Model
(Cross-dataset)
(Cross-dataset)
(In-dataset)
ToxiGen- ConPrompt
ToxiGen-CCoPronee
66. 66.2720.44
67.5920.64
88. 88.8520.25
Membership Relation
64. 26.091
67.0810.44
88.70.0.23
MLM
64. 04.2441.22
65.75+1.97
88. 88.60
DH - IHC
DH + SBIC-H
DH -> DH
Model
(Cross-dataset)
(Cross-dataset)
(In-dataset)
MoxiGen-ConnPoot
66.091110
76.00
78.94
Membership Relation
66.55+0.s3
75.52+1.48
78.91
MLM
65.77
74.14
78.22+0.92
Table 10: Ablation study including in-dataset evaluation investigate the contribution of two components of CONPROMPT: 1) using membership relationship to allow multiple positive samples; 2) using MLM as an auxiliary objective.
import re
Email address:
text hatt://aaecnnuo00002200000000000
"[EMAIL]'. text)
URL:
text re. rressubb/httpp//7/////nnntt JIW)'', [UURL', text)
text re.sub/rwww..15t, (UUR]', text)
text re.sub(rru///w'" "[USER]". text)
User/Channel mention:
text re.sub(rVV//w==" "[USER]" text)
text re.sub(""@(AA
"[USER]". text)
Figure 5: Example codes for the anonymization process we used replace private information with placeholders. We use the anonymization code from https: //gi thub. calub..om/ddii /hfbk/hattteeeee artthacts (Ramponi and Tonelli, 2022). The full version of the code can found our public repository.
IHC SBIC-H (Cross-dataset)
IHC (Cross-dataset)
IHC IHC
Model
(In-dataset)
ToxiGen- Coopieen-CoPromptt 50K
60.29
55.3 55.39
78.02+0.22
ToxiGin--onn Coroen--CCorpoppttt 150K
62, 12.1912.44
56.85
77.9210.47
MesiGen-ConPromppe 250K
67. 27.88
59.28 40.94
77.82+018
SBIC-H -1 IHC
SBIC-H -> DH (Cross-dataset)
SBIC-H SBIC-H
Model
(In-dataset)
ToxiGien-Cone Srrren--ooPPoonee Prompt 50K
65.51 10.91
66.1011..0
88.8410.42
ToxiGen- PonGont 150K
64 44.44
40.80
88.61
250K
66.27 40.44
67.5040.66
88.8540.28
DH IHC
DII SBIC-H
DH (In-dataset)
Model
(Cross-dataset)
(Cross-dataset)
ToxiGen- Coaeet-CooPPoppppp 50K
66, 06.27
73.67
78.41-0.48
150K
62.724..08
74.83
78.61=0.68
250K
66.09+1.1
76.00+1.19
78.94-0.43
Table II: Results of ToXIGIN-CONNPOMMP fine-tuned on various sizes of the pre-training dataset. ToxiGen- ConPrompt 250K is the proposing model.


Reviews:
[{"review_id": "52rdxGaw5b", "reviewer": "Reviewer_hY1k", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We deeply appreciate your constructive feedback. Here are our responses to your review.\n\n**[Comparison with other methods ([a] and [b]) addressing the generalization issue]**\n\n  We believe that it is fairer to compare our pre-trained model with other pre-trained models instead of [a] and [b]. We remark that while [a] and [b] are designed for fine-tuning process (i.e., they require the corresponding downstream dataset), our approach (ConPrompt) is for pre-training process (i.e., ours does not require such a downstream dataset). This is why we compared our approach with other PLMs rather than the fine-tuning processes (i.e., [a] and [b]).\n\n  Nonetheless, we agree that it is important to discuss the relationship between our pre-training approach and other fine-tuning methods including [a] and [b]. Particularly, our approach and other methods can be used together for further improvement. For example, one can use our approach in the pre-training step, and then apply the method of [a] or [b] in the fine-tuning process. We will elaborate on this in the camera-ready version.\n\n**[Suggestion regarding the typo]**\n\n  We appreciate your suggestion regarding Line 388 and will revise it considering your suggestion in the camera-ready version. (i.e., the revised sentence would be “... we compare our pre-training approach–ConPrompt–with the MLM objective.”) \n\n\n*[a] Kim, Youngwook, Shinwoo Park, and Yo-Sub Han. \"Generalizable implicit hate speech detection using contrastive learning.\" Proceedings of the 29th International Conference on Computational Linguistics. 2022.*\n\n*[b] Wullach, Tomer, Amir Adler, and Einat Minkov. \"Fight Fire with Fire: Fine-tuning Hate Detectors using Large Samples of Generated Hate Speech.\" Findings of the Association for Computational Linguistics: EMNLP 2021. 2021.*", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper exhibits excellent writing and a well-organized structure. It encompasses a variety of experiments, including those conducted on the representation space, to demonstrate the superiority of their modified contrastive objective in comparison to its previous version.\nThey ingeniously employed a creative approach to generate positive samples for contrastive learning. Additionally, they recognized the potential presence of other similar samples within the batch and introduced an adapted version of the contrastive objective function to account for this possibility.", "reasons_to_reject": "\nThe generalizability of their method is assessed in comparison to other existing PLMs (fBERT, HateBERT, BERT), revealing the relatively superior performance of ToxiGen-ConPromt. However, it's worth noting that these mentioned PLMs were not specifically designed to address the issue of degradation on cross-dataset evaluation. It would be reasonable to compare their method with approaches that are explicitly design to tackle this problem, such as the two methods mentioned in the related work appendix specially  \"Generalizable Implicit Hate Speech Detection using Contrastive Learning\" (kim et al. 2022).", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper introduces ToxiGen-ConPrompt, a pre-trained BERT model designed for implicit hate speech detection. It leverages the construction strategy of the ToxiGen dataset to generate positive samples, which are then utilized in a contrastive learning approach for the task of implicit hate speech detection. The ToxiGen dataset is crafted through ChatGPT, utilizing manually designed prompts for generating an implicit hate speech detection dataset. This paper employs the original prompt of a sample as a positive example for that sample. Subsequently, an adapted version of simCSE is utilized to enhance the closeness of representations for similar samples. Finally  they use cross-dataset evaluation across  different  implicit datasets to showcase the generalizability of their proposed method.", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Line 388, if I understand correctly, you mean that \"ConPrompt\" is your pre-training approach, so you should insert a comma after \"ConPrompt.\""}, "extra_text": {}}, {"review_id": "nggU2oGciR", "reviewer": "Reviewer_jhCf", "summary": "", "questions": "- Question A: The paper presented very promising results when evaluated on cross-dataset performance. However, the performance on the in-dataset seems to have reached a threshold even though it was trained on ToxiGen, a high-quality dataset that improved cross-dataset performance. Could the authors provide further clarification on this limitation?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your helpful questions and comments. Here are our responses to your review.\n\n\n**[Why we focused on cross-dataset evaluation]**\n\n  We respectfully disagree with you regarding the limitation of our approach on in-dataset evaluation.  \n\n  While ours showed comparable performance on in-dataset evaluation, our main research direction is to improve the performance for the cross-dataset evaluation. This is because the in-dataset evaluation is known to have several limits in hate speech detection tasks due to unintended biases such as identity term bias. Thus, the cross-dataset evaluation is widely utilized in hate speech detection tasks to assess the generalization ability of a model in the literature (Line 292-300, 758-774). This is why we focused on cross-dataset evaluation, where ours showed superior performance. \n\n  Therefore, we believe that our experimental results strongly support the superiority of our approach in hate speech detection, and it is not a limitation to have a comparative performance for in-dataset evaluation. We will elaborate on this using an extra page in the camera-ready and hope that our rebuttal can help you re-evaluate the soundness of our work.  \n\n\n**[Differences in data characteristics with other works]**\n\n  Before starting, we presume that you meant “human”-generation data (instead of “machine”-generated data in your review) used in [1] and [2]. Two main differences in data characteristics are as follows:\n\n  1. The use of the machine generation differs. While ToxiGen was constructed using machine (e.g., GPT-3) generation, the datasets in [1] and [2] were not. In HateCheck [1], they used templates and replaced the placeholders with pre-defined group identifiers and slurs. The datasets in [2] were mostly collected from social media and human-annotated.\n\n  2. The size and the aim of each dataset is different. ToxiGen is a large-scale dataset with over 250k samples, and we used the dataset for pre-training. However, HateCheck [1] is a small dataset with 3,728 samples, and it was constructed for the evaluation of a model. In [2], they used three datasets with 18k to 44k samples and used the dataset for fine-tuning.\n\n\n  Hopefully, our rebuttal clarifies your concerns about our contribution in terms of its correctness.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Based on thorough experimentation across various implicit hate speech cross-datasets, ToxiGen-ConPrompt demonstrated a notably superior capacity for generalization compared to other pre-trained models.\n- ToxiGen-ConPrompt effectively addresses the issue of identity term bias, showcasing its dual capability to enhance model generalizability while mitigating unintended biases.", "reasons_to_reject": "- Despite the superior results of ToxiGen-ConPrompt compared to HateBERT and fBERT in the cross-dataset evaluation, the performance differences within the in-dataset evaluation are not significantly pronounced.\n- I agree that this paper proposes a method that emphasizes pre-training the language model rather than focusing on the fine-tuning phase, as done in [1] and [2]. However, I believe this paper would be improved by describing the differences in data characteristics between ToxiGen corpus and the machine-generation data used in [1] and [2].\n\nReferences:\n- [1] Paul Röttger, Bertie Vidgen, Dong Nguyen, Zeerak Waseem, Helen Margetts, and Janet Pierrehumbert. 2021. HateCheck: Functional tests for hate speech detection models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 41–58.\n- [2] Youngwook Kim, Shinwoo Park, and Yo-Sub Han. 2022. Generalizable implicit hate speech detection using contrastive learning. In Proceedings of the 29th International Conference on Computational Linguistics, pages 6667–6679.\n", "questions_to_authors": "- Question A: The paper presented very promising results when evaluated on cross-dataset performance. However, the performance on the in-dataset seems to have reached a threshold even though it was trained on ToxiGen, a high-quality dataset that improved cross-dataset performance. Could the authors provide further clarification on this limitation?", "paper_topic_and_main_contributions": "- The authors proposed a pre-training strategy, named ConPrompt, in order to fully leverage machine-generated data. This strategy utilizes examples from a prompt as positive samples for contrastive learning. By applying ConPrompt to the machine-generated implicit dataset ToxiGen and modifying “bert-base-uncased”, the authors achieved ToxiGen-ConPrompt. This model out-performed various pre-trained hate speech language models, including HateBERT and fBERT, in a cross-dataset evaluation, showcasing its generalization ability in implicit hate speech detection. Besides, ToxiGen-ConPrompt proved to be effective in reducing identity term bias.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "dgdTg4G53J", "reviewer": "Reviewer_tecQ", "summary": "", "questions": "I think you should run some SOTA like : XLM-R.\n\nToxiGen-ConPrompt use ConPrompt as a dataset for training mlm, but i don't know it A Machine-generated Statement or A Human-curated Prompt or both of them. \n\n", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your constructive questions and feedback. Here are our responses to your review.\n\n**[Using machine-generated data for pre-training a model]**\n\n  We respectfully disagree with your opinion that machine-generated data is not good data for pre-training a model. \n\n  First, the machine-generated data (i.e., ToxiGen) we used is of high quality. The authors of ToxiGen conducted various human evaluations and confirmed its high quality. For example, about 90.5% of the machine-generated samples were considered to be written by humans in the human evaluation. \n\n  Second, there are several works showing the effectiveness of using machine-generated data for training a model. For example, [a] generated a large-sized commonsense knowledge graph using GPT-3 and trained a commonsense model using the machine-generated data. They empirically showed the effectiveness of using machine-generated data over human-generated one.\n\n  Therefore, we believe that leveraging machine-generated data for pre-training a model is a promising direction.\n\n\n**[Experiment with XLM-R]**\n\n  We respectfully disagree with your opinion that we should run XLM-R as a SOTA model. We remark that we only use the pre-trained models that are based on bert-base-uncased on purpose.\n\n  First, we respectfully disagree with your idea that XLM-R is a SOTA model for implicit hate speech detection. Our baselines such as HateBERT and fBERT were pre-trained on hate speech-related corpus and showed SOTA performance on various hate speech detection datasets. However, XLM-R was pre-trained on general corpus and is not specialized in hate speech detection.\n\n  Second, by using only BERT-based models (i.e., the models had the same starting point), various pre-training strategies can be fairly compared (Line 327-330). Through the fair comparison, we were able to successfully validate the effectiveness of our pre-training approach and our pre-trained language model. \n\n  As you mentioned, RoBERTa-based models like XLM-R can be employed as alternatives replacing BERT-based models, which is also an intriguing direction. We will leave this as a future direction.\n\n\n**[Training resources: machine-generated statements and human-curated prompts]**\n\n  ToxiGen-ConPrompt used ToxiGen as a dataset by leveraging ConPrompt as a pre-training approach with MLM as an auxiliary objective. For ConPrompt objectives, we utilize both the machine-generated statements and the human-curated prompts (Line 88-91, 210-217, Equation 3, and Figure 2). For MLM objectives, we use machine-generated statements (Line 255-256). We will clarify this in the camera-ready version.\n\n\n*[a] West, Peter, et al. \"Symbolic Knowledge Distillation: from General Language Models to Commonsense Models.\" Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.*", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "A new way for generate Implicit hate speech and a new pretrain for this task. ", "reasons_to_reject": "The dataset for training ToxiGen-ConPrompt is machine-generated data, so this is not good data for pretrain-model such as bert.\n\n", "questions_to_authors": "I think you should run some SOTA like : XLM-R.\n\nToxiGen-ConPrompt use ConPrompt as a dataset for training mlm, but i don't know it A Machine-generated Statement or A Human-curated Prompt or both of them. \n\n", "paper_topic_and_main_contributions": "Propose a pre-training approach, CONPROMPT for Implicit hate speech detection task\n\nPropose TOXIGEN-CONPROMPT, a pre-trained BERT using TOXIGEN as a dataset \n\n", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "BFdBzB4nhc", "reviewer": "Reviewer_poeE", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We deeply appreciate your constructive feedback. Here are our responses to your review.\n\n**[A carefully designed procedure of our work]**\n\n  We are delighted that you found our approach leveraging machine-generated data is a wise approach. It is true that with machine-generated data, we had to design our procedures by precise methods. We assess our work in terms of preciseness as follows:\n\n  *(Ethical Consideration)* It is crucial to care about the potential harms of using machine-generated data. We rigorously tackled the possible ethical issue regarding the machine-generated data. Before using the machine-generated data for pre-training, we inspected samples and observed that some private information was included in the machine-generated data. We anonymized such information following [a] to deal with the privacy issue (Line 272-280). We will describe our procedure of anonymization in a more detailed way in the camera-ready version for further preciseness.\n\n  *(Methodology)* We thoroughly describe our pre-training approach in Section 3. We first explain the objective based on the previous work [b] (Equation 2) and then present the modification of the objective (Equation 3). We justify the modification by providing the possible issue of Equation 2 regarding the false negatives within a mini-batch. Besides, we formulate our new way of selecting (multiple) positive samples using the membership relation (Line 239-244). Therefore, we believe that our methodology is presented in a precise way.\n\n  *(Validation)* We validate various aspects of our approach with precisely designed experiments. First, we conducted cross-dataset evaluation to validate the generalization ability of our approach (Section 4). Second, we measured the identity term bias of the models to validate the effect of our approach on reducing unintended behaviors (Section 5.1). Third, we compared the representation quality of our pre-trained model and the SimCSE model to validate the desirable representation quality of our model regarding implicit hate speech-related tasks (Section 5.2). Finally, we experimented with the ablated versions of our approach to investigate the contribution of each component and validate that our final approach is the best-performing version (Section 5.3). Through the thorough design of experiments, we conjecture that our validation is precise and appropriate.\n\n  *(Documentation)* For reproducibility, we reported the implementation details such as hyperparameters for both pre-training and fine-tuning in Appendix C and D due to the page limit. We will include the implementation details of our approach in the final version using an extra page. \n\n  We hope that our rebuttal helps you re-evaluate the soundness of our work.\n\n\n*[a] Ramponi, Alan, and Sara Tonelli. \"Features or Spurious Artifacts? Data-centric Baselines for Fair and Robust Hate Speech Detection.\" Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.*\n\n*[b] Gao, Tianyu, Xingcheng Yao, and Danqi Chen. \"SimCSE: Simple Contrastive Learning of Sentence Embeddings.\" Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2021.*", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The proposed pre-training approach and pre-trained model perform effectively in the task of detecting implicit hate speech. When compared to previous pre-trained models such as HateBERT and fBERT, the proposed TOXIGEN-CONPROMT outperforms on cross-dataset evaluation, indicating its capacity to generalize on this task. Furthermore, by learning desirable features in terms of implicit hate speech, the proposed pre-trained model is intended to decrease identity term bias.", "reasons_to_reject": "It is highlighted that although taking advantage of the machine-generated model in terms of constructing datasets for training is a wise way to conduct experiments, procedures should be carefully considered by precise methods.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The main contributions of this paper include:\n(1) The CONPROMT, a novel pre-training approach toward implicit hate speech detection by fully leveraging machine-generated data.\n(2) TOXIGEN-CONPROMT, a pre-trained language model on the machine-generated implicit hate speech dataset, outperformed various other pre-trained hate speech languages.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper primarily introduces a pre-trained BERT model tailored for implicit hate speech detection. The paper introduces ToxiGen dataset, which is built with ChatGPT, to generate positive samples. The paper also introduces a contrastive learning appraoch called ConPrompt. This contrastive learning approach similar to simCSE.

One of the key claims in the paper is the cross-data set performance. This is a nice feature as exposure to identity terms in one dataset can cause the model to be overly confident on that data, and not transfer well to unseen task settings. However, while the model does exhibit performance increases in this transfer setting. It is not exactly clear whether this is an issue with the model or the dataset. Examining table 1, Large benefits are provided when trained on IHC, but these are much less pronounced with SBIC-H and DH. There was no discernable difference in the in-dataset setting. 

The paper received mixed but positive reviews from the reviewers. The reviewers have concerns about the use of machine-generated data. Additionally, ethical concerns regarding privacy and potential misuse were raised by the ethics meta reviewer. The use of machine-generated data, while innovative, necessitates a thorough discussion on its implications, especially in a sensitive domain like hate speech detection.
Decision: Accept-Findings