Comparative Analysis of Anomaly Detection Algorithms in Text Data (2023)

Abstract: Text anomaly detection (TAD) is a crucial task that aims to identify texts that deviate significantly from the norm within a corpus. Despite its importance in various domains, TAD remains relatively underexplored in natural language processing. This article presents a systematic evaluation of 22 TAD algorithms on 17 corpora using multiple text representations, including monolingual and multilingual SBERT. The performance of the algorithms is compared based on three criteria: degree of supervision, theoretical basis, and architecture used. The results demonstrate that semi-supervised methods utilizing weak labels outperform both unsupervised methods and semi-supervised methods using only negative samples for training. Additionally, we explore the application of TAD techniques in hate speech detection. The results provide valuable insights for future TAD research and guide the selection of suitable algorithms for detecting text anomalies in different contexts.

## introduction anomaly detection is a fundamental process in data analysis, aiming to identify inconsistent data points that deviate significantly from expected behaviors or established norms within a dataset. such anomalies can emerge from various factors, including human errors, malicious behaviors, unusual events, or unexpected changes. effective anomaly detection can facilitate swift problem recognition, proactive measures for error correction, and future problem prevention. it enhances data quality, aids in risk identification, and empowers decision-making across diverse domains, with its utility extending to various data types such as tabular data, graphs, time series, texts, images, and videos. in the context of text data, the anomalies refer to specific texts or textual fragments that deviate significantly from established norms, which can be determined based on the overall text or corpus, regular language usage, or common sense. these anomalies may manifest at various linguistic levels, such as orthographic (spelling), lexical (word usage), syntactic (sentence structure), semantic (meaning), and discourse (overall context) levels @xcite @xcite . detecting anomalies in text data holds vital importance in applications like language development assessment, plagiarism detection, quality control in data processing, and identifying abnormal language usage in cybersecurity @xcite . in this article, we concentrate on text anomaly detection (tad) at the semantic and discourse levels, where norms are established on a corpus scale. it is important to note that the definition of anomalies can be further refined and may slightly differ according to specific contexts. for instance, in the realm of competitive intelligence, anomalies often relate to abnormal themes or topics, while in the field of online reputation monitoring, they typically pertain to negative sentiments. despite the broad utility of tad and the potential benefits it offers, tad has not been as extensively explored as other topics within nature language processing (nlp). while previous research works have approached the field of tad, they have often been limited either by the scope of algorithms considered or by the range of textual representations evaluated @xcite . unlike these studies, this paper aims to provide a comprehensive overview of tad by evaluating a wide array of algorithms on several corpora across different languages, making our approach distinctive in its breadth and depth. our primary objective is to provide a systematic evaluation of 22 tad algorithms applied to 17 corpora across three languages. we assess these algo-rithms' performance in detecting textual anomalies and examine the use of various text representations, including monolingual and multilingual sentence-bert (sbert) @xcite . additionally, we investigate the potential application of tad techniques in detecting hate speech, aiming to gain insights into the effectiveness of tad in this specific domain. ## related work text anomaly detection (tad) stands as a comparatively less explored intersection of data mining (dm) and natural language processing (nlp). while extensive research has been conducted in dm dedicated to anomaly detection, scant attention has been given to the application of these techniques to text data. on the other side of the spectrum, nlp, despite significant progress in text understanding and generation, exhibits a noticeable deficiency in research focusing on the detection of anomalous text. consequently, dedicated algorithms for text data anomaly detection are rare, and corpora specific to this task are either completely inaccessible or simply nonexistent. ## conclusion in summary, this paper provides a comprehensive evaluation of 22 anomaly detection algorithms applied to 17 corpora derived from datasets associated with three distinct tasks. the evaluation considers three types of text representations: tf-idf, monolingual sbert, and multilingual sbert models. the findings shed light on several key insights regarding the performance and limitations of these algorithms. the analysis reveals variations in algorithm performance across different corpora categories. the figure 4: performance (avg. roc auc) comparison of anomaly detection algorithms based on underlying theory for anomaly scores: proximity-based, probabilisticbased, domain-based, reconstruction-based, and ensemble methods figure 5: performance (avg. roc auc) comparison of anomaly detection algorithms based on model architecture: deep models (with neural networks) and shallow models looking ahead, several potential avenues of investigation could further enrich the field of text anomaly detection. firstly, the exploration of supervised algorithms could provide an opportunity to bolster anomaly detection performance, especially in contexts where labeled data is available. secondly, the incorporation of advanced technologies, such as language models like chatgpt, opens up novel possibilities for innovative anomaly detection methodologies that can adapt to evolving data landscapes. another promising direction lies in the creation of specialized datasets explicitly designed for anomaly detection tasks. such datasets could allow for the refining and optimization of current detection algorithms while enabling the development of new, more effective methods. lastly, delving deeper into the study of different types of text anomalies could provide a more nuanced understanding of their unique characteristics and the detection strategies that work best for each.