V-HATE: Voting-based Implicit Hate Speech Detection
Authors: Yejin Lee, Hyeseon Ahn, Yo-Sub Han
Accepted at: acl25


Reviewer paper summaries: The paper introduces V-HATE, a voting-based framework for implicit hate speech detection. It highlights that existing hate speech detection techniques primarily focus on model generalization while often neglecting dataset-specific characteristics, leading to inconsistencies in cross-dataset predictions. 
To address this issue, V-HATE trains five modules (Using Target Special Token with NER Tagger, Remove Outliers in a Cluster, Using Cosine Similarity,Contrastive Learning with Hard Negative Samples, Voting-based Implicit Hate Speech Detection), each optimized for different dataset characteristics, and employs a soft voting mechanism to aggregate their predictions, enhancing adaptability and detection performance.

The paper describes a novel implicit hate speech detection system called V-Hate that seeks to resolve five issues when classifying this kind of content: i) inconsistent annotations in a hate dataset, ii) avoiding model generalization and focusing on data-specific characteristics, iii) identifying hateful messages that do not have a clear target being attacked, iv) recognizing instances with incomplete or fragmented sentences that lack essential grammatical components such as a subject or verb, making it difficult to convey a complete meaning, and v) adapting contrastive learning to use cosine similarity instead of Euclidean distance as a more suitable learning method, improving previous related work approaches. These attributes are focalized in five detection modules, which finally use soft voting for the final prediction (HS vs. Non-HS). The method improves classification on five well-known implicit benchmarks, outperforming state-of-the-art systems.

This paper addresses the challenge of detecting implicit hate speech—hateful content that lacks overt or explicit markers and may instead rely on subtle cues and context. The authors argue that a major limitation of current methods lies in their overemphasis on generalization and disregard for dataset-specific characteristics, which can be especially detrimental in the subjective domain of hate speech detection where annotation guidelines and target definitions vary significantly.

To address this, the authors propose V-HATE, a voting-based ensemble framework consisting of five contrastive learning modules (M1–M5), each designed to capture a different aspect of dataset-specific noise or structure. Instead of hard-voting, V-HATE employs soft-voting via averaging model logits. The modules include: M1 replicates SharedCon (clustering-based contrastive learning). M2 uses NER-based tagging to insert target markers into hate-labeled samples, M3 removes cluster outliers, often corresponding to broken or fragmentary sentences, M4 modifies anchor selection by replacing Euclidean distance with cosine similarity, M5 incorporates hard negative sampling using high-confidence false positives.

The paper presents empirical results on five datasets (IHC, SBIC, DynaHate, Hateval, Toxigen), demonstrating consistent performance improvements over prior methods and providing a comprehensive ablation study.
Summary of strengths: 1.The article is well-organized, clearly proposing work based on M1 and implementing it with methods M2-M6 to derive results. The final outcome is determined through a voting mechanism, making the paper highly readable.
2.The authors highlight that different annotators' subjective interpretations of hate speech lead to variations in dataset labeling. Their approach to overcoming this issue and conducting research across multiple datasets represents a novel perspective.
3.In the methods section, the authors conduct a detailed analysis of the limitations of existing approaches in modules M2-M6 and replace the shortcomings with new methods.
4.The presentation of the methods and implementation for each module is very intuitive.

- Excellent error analysis, considering each of the modules of the proposed system and showing their impact through an ablation study.
- I consider not only the model novel but I also highlight its simplicity, detailing the motivations for experimenting with each module very well.
- Obtaining state-of-the-art results on all the benchmarks is quite impressive.
- Recognizing improvements over previous work (Ahn et al., 2024) in contrastive learning by using cosine similarity instead of Euclidean distance.

The study introduces a modular framework that is grounded in recent contrastive learning approaches while extending them in dataset-specific ways.

It documents insightful dataset-level observations such as the prevalence of broken sentences or targetless hate speech which are valuable contributions to the hate speech literature.

The ablation study and embedding analysis offer an explanation for each module's impact, supporting the authors’ hypothesis that different datasets benefit from different adaptations.
Summary of weaknesses: 1.The modular design of V-HATE also increases the complexity and training cost of the model. Each module requires separate training and optimization, which can significantly increase training time, especially on large-scale datasets. Moreover, the marginal improvement in experimental results does not justify the added complexity. For instance, in the SBIC and Toxigen datasets, the performance gain of the V-HATE method, which incorporates five modules and a voting mechanism, is less than 1% compared to the SharedCon method using only the M1 module.
2.The V-HATE framework appears to be a mere aggregation of existing methods rather than a novel approach.
3.V-HATE enhances performance by capturing dataset-specific features. However, the rapidly evolving nature of hate speech on online platforms means that the model may struggle to generalize when confronted with new, unseen data.
4.While the paper acknowledges that different annotators have subjective interpretations of hate speech, the extraction of features from the datasets still relies on these error-prone datasets.

I was wondering if the recognition of hateful examples without a target, broken sentences, and mislabeled examples was all done automatically via GPT-4o. If so, did this step include any human verification? How could the authors evaluate that there were no model errors at this step?

The paper does not motivate why this modular ensemble approach is preferable to scaling up the backbone (e.g., using RoBERTa-large or DeBERTa). It remains unclear whether the observed gains stem from architectural novelty or simply from ensemble diversity. A comparison with a stronger single model would help contextualize the contribution.

Although the idea of exploiting information coming from a diverse set of datasets is very interesting, we have not seen enough evidence that this might be generalized further.

The paper has several unsupported or weakly supported claims:

- M4 Analysis (Line 507–508): The claim that cosine similarity is more effective than Euclidean distance for selecting semantically similar anchors is plausible but not convincingly demonstrated. The evidence is anecdotal and would benefit from either a visualization of cluster structures or a similarity metric evaluation.

- Integrated Module Explanation (Lines 535–537): The authors state that the integrated module underperforms because it eliminates dataset-specific adaptability. However, this is presented as an assumption without any concrete analysis or diagnostic evidence.

- Figure 6 Interpretation: It is unclear what Figure 6 is meant to demonstrate. Why the difference in Toxigen embeddings is observed and how this is related to the dataset being machine generated is not clearly explained. 

While the ablation study shows the average effect of removing each module across datasets, it does not provide fine-grained insight into how or where the modules help. For instance, in the M5 (hard negative sampling) discussion, the analysis seems to attribute performance changes to M5 by showing V-HATE's improved confusion matrix. However, since V-HATE includes all modules, it is difficult to isolate the contribution of M5 alone. Ideally, the analysis would compare the performance of the standalone M5 module versus M1 or other modules on specific subsets (e.g., false positives or borderline examples). Similar instance-level analysis could be extended to other modules like M2 or M3 to strengthen the interpretability of the approach.
Comments, suggestions and typos: 1.Given the promising performance of Large Language Models (LLMs) in detecting implicit hate speech, it would be beneficial to include such methods for comparison.
2.Correct grammatical errors in the paper, such as "Tabel shows" in Section 4.4 and "datasest-specific" in Figure 1.
3.Consider conducting experiments with reduced training data to assess the model's performance under data-limited conditions.

line 499: utilizess -> utilizes
line 527: treir -> their
Title appendix A.4: Diffence -> Difference

It would strengthen the empirical section to include an additional baseline with five M1 models soft-voted, to isolate the impact of module diversity from the ensemble effect.

Figure 6: The claim about toxigen embeddings being different is interesting but underexplained. Why are they so distinct? How does this reflect the synthetic nature of the dataset?

M4 module: The assertion that cosine similarity leads to semantically better anchors should be supported more concretely. Possibly include a qualitative example or average intra-cluster similarity scores.

The notation “%p” (e.g., 1.18%p) is used repeatedly without explanation. It confuses readers.

Line 434: "82.29%" percentage of what? Please clarify the metric (average macro-F1).

Figure 1: Clustring basedContrastive Leaning
Clustring → Clustering
Leaning → Learning

Section 4.4, line 429: Tabe1 → Table 1

Line 527 treir -> their
Overall assessment scores: 2.5

4

3