F an C huan: A Multilingual and Graph-Structured Benchmark For Parody Detection and Analysis (2025)

Abstract: Parody is an emerging phenomenon on social media, where individuals imitate a role or position opposite to their own, often for humor, provocation, or controversy. Detecting and analyzing parody can be challenging and is often reliant on context, yet it plays a crucial role in understanding cultural values, promoting subcultures, and enhancing self-expression. However, the study of parody is hindered by limited available data and deficient diversity in current datasets. To bridge this gap, we built seven parody datasets from both English and Chinese corpora, with 14,755 annotated users and 21,210 annotated comments in total. To provide sufficient context information, we also collect replies and construct user-interaction graphs to provide richer contextual information, which is lacking in existing datasets. With these datasets, we test traditional methods and Large Language Models (LLMs) on three key tasks: (1) parody detection, (2) comment sentiment analysis with parody, and (3) user sentiment analysis with parody. Our extensive experiments reveal that parody-related tasks still remain challenging for all models, and contextual information plays a critical role. Interestingly, we find that, in certain scenarios, traditional sentence embedding methods combined with simple classifiers can outperform advanced LLMs, i.e. DeepSeek-R1 and GPT-o3, highlighting parody as a significant challenge for LLMs.

## introduction parody in social media 1 is a form of humor or satire, which uses exaggerated or absurd imitations for critique or entertainment @xcite . it has become popular around some controversial topics in recent years, especially among the young generation @xcite . for example, as shown in figure 1 , the question "should my boyfriend hand over his salary to me?" has sparked intense debate (q&a). while some users clearly express their views as neutral, supportive, or opposed, others adopt a parody tone, mockingly pretending to support the stance with exaggerated statements like, "guys who don't hand over their salary are a huge red flag...", which subtly opposes it. this tactic can attract attention and provoke reactions through humor, making people reflect their opinions. similar to irony or sarcasm @xcite , parody also expresses the opinion opposite to its appearance. however, it emphasizes playful, entertaining, and exaggerated mimicry of a character, making the underlying critique more accessible and engaging to the audience. the real meaning behind parody is highly culture-dependent. therefore, the analysis of parody can offer unique insights in understanding the corresponding cultural values. the spread of parody on internet also fosters a diverse linguistic culture @xcite . people can share their distinct views on society, political, or cultural topics in a humorous and engaging manner, encouraging global and cross-cultural dialogue. in addition, parody plays a crucial role in the formation of subcultures @xcite . parody comments not only create distinct communities, but also mirror the values and identities of online users. for younger generations, parody comments have become a way of self-expression, which help to define their uniqueness, build connection with others, and form social circles. gradually, it has become a shared language and a set of symbols for the growth of internet subcultures. despite the widespread popularity of parody, there is a lack of high-quality datasets that capture parody comments with different topics and languages @xcite , restricting the more general and inclusive analysis in various contexts. to fill this gap, we propose fanchuan, a parody benchmark with high quality in three key aspects: high diversity, rich contexts, and precise annotations. first, we enhance diversity by collecting data from multiple sources (both chinese and english corpora), a wide range of topics, and various social media platforms. such broad coverage allows us to conduct more sufficient, balanced and fair evaluations of models. second, we construct richer context information by building the relationship between comments and their replies as heterogeneous graphs. unlike previous studies that only focus on textual @xcite or dialogue @xcite content, the graph-structured context enables the exploitation of relational information, which is found to be fairly valuable later. third, since parody labeling is quite challenging and disagreements among annotators can easily arise, we ensure the quality of annotation by employing native speakers to label the parody and sentiment of each comment. additionally, we have expert judges to resolve any disagreement and large language models (llms) to refine the annotation results, ensuring consistency and reliability. as a result, we have created seven datasets, with 14,755 annotated users and 21,210 annotated comments in total, enabling comprehensive experiments and analyses. with the new datasets, we evaluate embedding-based methods @xcite , incongruity-based methods @xcite , outlier detection methods @xcite , graph-based methods @xcite , and large language models (llms) (openai, 2023) on fanchuan with three parody related tasks: parody detection, comment sentiment classification with parody, and user sentiment classification with parody. our results indicate that (1) parody-related tasks are challenging for all models, and even llms fail to consistently outperform traditional embedding-based approaches; ## related work in this section, we introduce the datasets and detection methods related to parody, as well as its associated topics: sarcasm, irony, and humor. ## conclusions in this paper, we introduce fanchuan, a multilingual benchmark for parody detection and analysis, encompassing seven datasets characterized by high diversity, rich contextual information, and precise annotations. our findings reveal that parody detection remains highly challenging for both llms and traditional methods, with particularly poor performance on chinese datasets. we also observe that contextual information significantly enhances model performance, while parody itself increases the difficulty of sentiment classification. additionally, our results indicate that reasoning fails to improve llm performance in parody detection. by filling a critical gap in the study of emerging online phenomena, fanchuan provides valuable insights into cultural values and the role of parody in digital discourse. these findings highlight the limitations of current llms, presenting an opportunity for future research to enhance model capabilities in parody detection and analysis.