Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
Authors: Walid Durani
Tobias Nitzl
Claudia Plant
Christian Böhm
Venue: ICML 2025 poster (2025)
Keywords: anomaly detection
kernel
dual-kernel tails
Primary area: general_machine_learning->unsupervised_and_semisupervised_learning

Abstract: Detecting anomalies with limited supervision is challenging due to the scarcity of labeled anomalies, which often fail to capture the diversity of abnormal behaviors. We propose Weakly Supervised Anomaly Detection via Dual-Tailed Kernel (WSAD-DT), a novel framework that learns robust latent representations to distinctly separate anomalies from normal samples under weak supervision. WSAD-DT introduces two centroids—one for normal samples and one for anomalies—and leverages a dual-tailed kernel scheme: a light-tailed kernel to compactly model in-class points and a heavy-tailed kernel to main- tain a wider margin against out-of-class instances. To preserve intra-class diversity, WSAD-DT in- corporates kernel-based regularization, encouraging richer representations within each class. Furthermore, we devise an ensemble strategy that partition unlabeled data into diverse subsets, while sharing the limited labeled anomalies among these partitions to maximize their impact. Empirically, WSAD-DT achieves state-of-the-art performance on several challenging anomaly detection benchmarks, outperforming leading ensemble-based methods such as XGBOD.

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
Walid Durani' Tobias Nitzl
Claudia Plant Christian Böhm
Abstract
pervised approaches require many labeled anomalies, which is typically infeasible (Han et al., 2022). Consequently, weakly supervised anomaly detection has emerged, lever- aging small labeled anomaly set alongside predominantly normal unlabeled data (Pang et al., 2023). Two notable examples are DeepSAD (Ruff et al., 2019), which extends DeepSVDD (Ruff' et al., 2018) by pushing labeled anoma- lies away from a single "normal" center, and DevNer (Pang et al., 2019), modeling anomalies as the extreme tail of a univariate distribution. Both can handle sparse labels but rely on a single center or tail, risking performance degra- dation when anomalies exhibit significant heterogeneity or cause collapsed embeddings (Goyal et al., 2020). Motivated by these challenges and guided by classical margin- based theory, which advocates maintaining a tight radius around in-class samples while enforcing a wide margin against out- of-class points to reduce model complexity and enhance generalization (Cristianini & Shawe-Taylor, 2000; Bartlett & Mendelson, 2002; Tax & Duin, 2004)-we introduce Weakly Supervised Anomaly Detection via Dual-Tailed Kernel (WSAD-DT), Our method is carefully tailored to the nuances of weakly supervised anomaly detection, where labeled anomalies are not only scarce but may also underrep- resent the true diversity of anomalous behaviors. The core idea of WSAD-DT is to map input data into latent represen- tation that effectively distinguishes normal from anomalous samples. To achieve this, our approach employs dynamic similarity measures based on heavy-tailed and light-tailed kernels (Schölkopf & Smola, 2002) (Fig. 1), which are tai- lored to the distinct characteristics of anomalies and normal samples. Specifically, WSAD-DT leverages a light-tailed kernel to tighten the representation around each class center by rapidly reducing similarity with increasing distance. Con- currently, heavy-tailed kernel ensures slower similarity de- cay, regulating broader dispersion farther away-balance compactness for in-class points with greater margins against out-of-class samples. This dual-tailed kernel mechanism effectively balances compactness near each center while allowing broader margins for out-of-class patterns, thereby enabling clear separation between normal and anomalous instances. Fig. 1 illustrates the decay behaviors of light and heavy-tailed kernels, highlighting their suitability for mod- eling normal and anomalous data, respectively. Addition- ally, WSAD-DT incorporates kernel-based regularization to promote intra-class diversity and avoid over-concentration,
Detecting anomalies with limited supervision is challenging due to the scarcity of labeled anoma- lies, which often fail to capture the diversity of abnormal behaviors. We propose Weakly Supervised Anomaly Detection via Dual-Tailed Kernel (WSAD-DT), novel framework that learns robust latent representations to distinctly separate anomalies from normal samples under weak supervision. WSAD-DT introduces two centroids- for normal samples and one for anomalies-and and leverages a dual-tailed kernel scheme: light-tailed kernel compactly model in-class points and heavy- teiieet kernel to main- tain a wider margin against out-of-class instances. To preserve intra-class diversity, WSAD-DT in- corporates kernel- based regularization, encourag- ing richer representations within each class. Fur- thermore, we devise an ensemble strategy that par- titions unlabeled data into diverse subsets, while sharing the limited labeled anomalies among these partitions to maximize their impact. Empirically, WSAD-DT achieves state-of tfte--of-the-rr the performance on several challenging anomaly detection bench- marks, outperforming leading ensemble-based methods such as XGBOD.
1. Introduction
Anomaly detection identifies data instances that deviate sub- stantially from normal patterns (Agrawal & Agrawal, 2015), with critical applications in credit risk analysis (John & Naaz, 2019), network intrusion detection (Tao et al., 2018), and medical diagnostics (Abuzaid, 2020). Fully unsuper- vised methods often yield high false positives, while fully su-
'LMU Munich, Munich Center for Machine Learning (MCML), Munich, Germany 'LMU Munich, Munich, Germany 'Faculty of Computer Science ds:Uniiie, University of Vienna, Vi- enna, Austria 'Faculty of Computer Science, University of Vi- enna, Vienna, Austria. Correspondence to: Walid Durani <du- rani@ dbs ifi lmu.
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
2. Related Work
1,0
Uiitttst konel (basss
(-dieniuutio)
Detecting anomalies under weak supervision-Whert a small subset of anomalies is labeled while the majority of data remain unlabeled- has attracted growing interest in recent years (Pang al., 2023).
0.5
DeepSAD (Ruff et al., 2019) extends DeepSVDD (Ruff et al., 2018) by introducing an autoencoder-basee latent representation. In this representation, a single learned cen- ter pulls normal samples closer and pushes anomalies far- ther away; a hypersphere is then fitted to primarily cap- ture normal embeddings. To address the collapse issue in DeepSVDD, DROCC (Goyal et al., 2020) introduces ad- versarial perturbations around normal points to learn robust decision boundaries. Subsequently, DROCC-LF (Goyal et al., 2020) incorporates a small number of labeled anoma- lies alongside local feature representations, thus maintaining adaptable boundaries even in high- -igh-iimnnsooal spaces. De- vNet (Pang et al., 2019) employs a Gaussian prior in the latent space and deviation loss that pulls normal sam- ples toward the distribution center while pushing anomalies outward. This focus on relative deviations yields compet- itive performance. FeaWAD (Zhou et al., 2021) builds on the same principle, fusing the deviation loss with autoen- coders for more robust representations. PReNet (Pang et al., 2023) tackles the anomaly detection problem from a meta- learning viewpoint, combining pairwise relations with self-supervised distance metric to more effectively isolate anomalies, A separate line of research leverages Generative Adversarial Networks (GANs) (Goodfellow et al., 2014). For example, GANomaly (Akcay et al., 2019) learns a latent space which the ates differences between reconstructions of normal samples and anomalies. While GAN-based models can capture rich data distributions, they often require careful tuning for sta- ble adversarial training. RoSAS (Xu et al., 2023b) further refines semi-supervised AD by introducing contamination- resilient, continuous supervision mechanism that uses mass interpolation to produce smoothly varying anomaly scores, enhancing robustness to label noise. Beyond deep end- to-end solutions, hybrid approaches remain popular. XG- BOD (Zhao & Hryniewicki. 2018), for instance, combines gradient-boosting ensemble with unsupervised anomaly scores, integrating weak labels to sharpen decision bound- aries. Single- vs. Multi-Kernel Methods. Classical kernel- based methods such as One-Class SVM (Schölkopf et al., 2001) rely on a single kernel for all data. Meanwhile, vari- ous multi-view or multi-kernel approaches blend multiple kemell- via linear combinations
0.
distance
Figure 1. Comparison of two distance-based similarity functions- light- and heavy-tailed as distance in- creases. The light-tailed kernel's similarity rapidly falls off at moderate distances. In contrast, the heavy-tailed decays more gradually, preserving moderate similarity for points farther away.
ensuring robust latent space representation. To address the variability and imbalance inherent in weakly supervised settings, WSAD-DT also introduces an effective ensemble learning mechanism. This ensemble approach leverages diverse subsets of the data while sharing labeled anoma- lies across all subsets, which enhances robustness and sur- passes traditional ensemble methods like XGBOD (Zhao & Hryniewicki, 2018) in anomaly detection tasks.
Our key contributions are as follows:
Dynamic Similarity Measures: We introduce novel loss function that incorporates both light-tailed and heavy-tailed kernels, ensuring compact representation around each class center while allowing broader disper- sion for out-of-class samples. This dual-tailed kernel design captures typical data tightly yet accommodates anomalous deviations more flexibly, thereby improving separation under limited supervision.
pair accentu-
Kernel-Based Regularization: To prevent degener- ate polttsccollappee"" solutions, we incorporate a kernel-based regularization term that promotes intra- class diversity. This term ensures that each class retains its inherent variability in the latent space, thereby en- hancing robustness and generalization.
Ensemble- Learning with Subset Splitting: An effective ensemble scheme divides unlabeled data into multiple subsets, while the limited labeled anoma- lies are shared across all partitions. This design yields diverse yet anomaly-aware models, and aggregated de- cisions outperform strong ensemble baselines such as XGBOD in anomaly detection tasks.
capture richer similarities (Gönen & Alpaydın, 2011). For example, Dual- Regularized Multi- View Outlier Detection DMOD (Zhao & Fu, 2015) factors multi-view data into cluster indicators and sample-specific errors, thereby modeling both cross-view inconsistencies and universal anomalies. By contrast, our
2

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
work assigns two distincl kernels- for in-class points and heavy-tailed for out-of-class points-thuss explic- itly enforcing separate tail behaviors for each class rather than learning single "blended" kernel for the entire dataset.
4. Two Centers based Separation
In WSAD-DT, two dedicated centers ep (for normal data) and e1 (for anomalies) are placed in the latent space to better capture the distinct characteristies of each class, par- ticularly when labeled anomalies are scarce. In the weakly unsupervised setting, we assume that the unlabeled dataset Dy is predominantly composed of normal instances (Pang et al., 2023), although it may contain some anomalies. Nevertheless, this assumption provides solid foundation for modeling normal behavior. In Appendix M, we fur- ther analyze how WSAD-DT performs under varying lev- els of contamination in the unlabeled data. A neural net- work f(;:; Oy) projects each input 24 to latent embed- ding Fs. and the centers are initialized by averaging em- beddings from Dy (predominantly normal) and Dr. (con- fo(z;) and
3. The Proposed Approach
3.1. Problem Setting
Let X' R be the input feature space, and consider a dataset D C X x 10, In weakly supervised anomaly detection, the dataset D is divided into an unlabeled set DU = ( O)..... ..... (Iv, 0)}. which is largely normal but may contain a few anomalies, and a labeled set Dz= = {(xN+1, 1)............... (#N+k. 1)), comprising a small number of confirmed anomalies. Since K << N, these labeled anomalies provide only sparse supervision and often span a narrow range of anomaly types (Pang al., 2023). The objective is to learn a scoring function o : X + I that ranks true anomalies (labeled or unseen) higher than normal samples. This goal poses two key difficulties: (i) learning robust decision boundary from small set of labeled anoma- lies, and (ii) generalizing effectively on a large, unlabeled pool that may include unknown anomalies.
firmed anomalies), as in Co
Do
1 C1 = P22
Σ fa(z;), ensuring Co + 01. By aligning (r..
each class with its dedicated center, normal samples cluster around co and anomalies are pulled toward cı, yielding a sharper distinction between their intrinsic characteristics, This two-center design effectively decouples normal and anomalous representations in the latent space, minimizing overlap and improving separation.
3.2. Overview of WSAD-DT
To tackle these challenges, this paper introduces Weakly Su- pervised Anomaly Detection via Dual-Tailed Kernel (WSAD- DT). a framework tailored for scenarios with scarce labeled anomalies and abundant unlabeled data dominated by nor- mal instances. WSAD-DT learns a feature representation f X + 2, where X C R and 2 C Rdd that separates normal from anomalous instances under limited supervision and data imbalance. The mapping f is a feature represen- tation learner, instantiated as a neural network [(s θy) with parameters , = ... "H,, J. where H e N de- notes the number of hidden layers. WSAD-DT employs two distinet latent centers one representing cohesive nor- mal patterns and another capturing dispersed anomalies- to establish a clear boundary between normality and anoma- lies. A dual-tailed kernel approach then sharpens this divide by keeping each class tightly localized around its match- ing center while pushing out-of-class samples away from the opposing center. However, merely driving data toward these centers can trigger degenerate collapses, where all points of class map single coordinate. To avert this, WSAD-DT includes a diversity term that preserves intra- class variability. Additionally, an ensemble strategy further enhances stability and generalization by distributing the lim- ited anomaly labels across multiple partitions of unlabeled data. Finally, WSAD-DT translates the learned embeddings (2) into anomaly scores Z + R by contrasting each embedding's 's similarity to the normal center against its simi- larity to the anomaly center.
Remark Single Center for Normality. Although real data can exhibit multiple modes of normal behavior, we adopt a single center Co to represent normal samples be- cause allowing multiple centers would introduce additional complexity and the risk of overfitting or unstable partitions in weakly supervised scenarios. This choice aligns with clas- sic one-class frameworks (e.g., DeepSVDD and DeepSAD) that effectively use a single hypersphere or reference point for normal data and still achieve strong performance. Empir- ically, our experiments (Tables 1, 5) demonstrate that even single center, combined with the dual-tailed kernel and a suitable diversity term, can robustly capture local variations while maintaining enough variance to avoid degeneracy. Thus, one center often strikes the right balance between simplicity, robustness, and generalization, especially under limited anomaly labels.
5. -ual-Tailed Kernel: Motivation and Formal Definition
Margin- based theory (Cristianini & Shawe-Taylor 2000; Bartlett& & Mendelson, 2002; Tax & Duin, 2004) emphasizes the importance of forming a compact in-class region while maintaining a broad margin for out-of-class points. Moti- vated by these insights, the main objective of WSAD-DT is to achieve high in-class compactness and a clear boundary
3

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
for anomalous samples, thereby yielding robust represen- tations even under limited anomaly labels. To this end, we propose a dual-tailed kemel separation loss that adap- tively switches between two complementary similarity func- tions according to each sample's alignment with its assigned class center. Concretely, we first define a distance metric to quantify how far each sample lies from its designated class center. We then transform these distances into similarity scores through two specialized kernels: one that reinforces tighter distribution within each class, and another that preserves a wider margin for out-of-class points. This dual perspective ensures both strong in-class compactness and a robust boundary for anomalous samples, ultimately produc- ing discriminative embeddings even with sparse anomaly supervision.
simply distance-based similarity. In our setting, we use two variants that differ in their tails (light-tailed vs. heavy- tailed), reflecting different asymptotic decay behaviors. By applying these kernels to distances di,s, we obtain con- tinuous measure of how "close" a sample is to its in-class center or out-of-class center, allowing us to capture both near- and long-range relationships in the latent space.
Light-tailed kernel (Compact Representations); We uti- lize a light-tailed kernel designed to enforce a rapid decay in similarity as points move farther from their true center. A kernel Kight is called fight-tailed if it decays at least as fast as an exponential function for large d. Formally, there exists a constant > such that
Krigh(d)) (d) d+000 e-ed
lim
L,
(3)
5.1. Distance Metric
For each sample (ais, ye), let So be the neural network map- ping 24 e IR into the latent space. The distance between fa();) and ck (k E {0, 1)) is defined as:
where 0 L < 00. Equivalently, Kiigh(d) approaches 0 faster than any polynomial rate d This rapid decay enforces tight representations: small distances yield high similarity, while similarity plummets quickly as d grows (more detail in Section 6.1 and Appendix H).
d Σ (Jolwz))) -ces) (1)
di,e= = [[so(as) - call =
Heavy-Tailed Kernel (Separation Enforcement) : In two-class setting, it is crucial that off-center points remain distinctly separated from the wrong class. While a light- tailed kernel excels at pulling well-matched points close to its center, it may fail to emphasize separation once points deviate substantially.
where "Il |r denotes the standard Euclidean (I.3) norm.
Because y/ E {0, 1} indicates whether 2; is normal (y 0)
or anomalous (y: = 1). we assign:
din { do,o, if y = 0, di,,, if 1,
dout d if u 0, ds,a, yf 1.
contrast, heavy-tailed kernel de- cays more gradually with distance, preserving a moderate similarity for out-of-class points even larger distances and thereby enforcing clearer boundary. Formally, say Knewvy is heavy-tailed if it decays at most polynomially: there exist constants >
Hence, din measures how close the sample is to its own class center, while dout measures its distance to the opposite (out- of-class) center (i.e. C1 for normal data, co for anomalous). We aim to decrease din (pulling each sample to its own center) and increase dout (pushing it away from the out-of- class center). thus causing normal data cluster near co and anomalies near c1.
O and E > 0 such that
lim dP KCheavy(l)) e, (0 (0<e<<o)) (4) d-000
This implies Kheavy (d) remains larger at moderate to large distances than an exponentially decaying function, maintain- ing a "long tail" that keeps out-of out-of-class points distinguish- able. Consequently, mismatched samples retain consistently low (but not vanishing) similarity to the wrong center, re- inforeing class separation (more detail in Section 6. 1 and Appendix H)).
5.2. Similarity-Bare Kernels
To achieve compact in-class representation while maintain- ing broader margin for out-of-class points, convert distances in the latent space into similarity measure using carefully chosen kernels. These kernels regulate how fast similarity decays with distance: To capture how data be- haves in relation to these centers, we employ distance-based kernels. Let
By further considering the ratio of light and heavy-tailed kernel::
K. R20 + Rz0
Lemma 5.1 (Ratio of Light- and Heavy- Kernels), Let Krigh be light-tailed and Kreavy be heavy-tailed defined. Then
be a function that maps a nonnegative distance (d) to a non- negative similarity score, satisfying KC(0) = 1 (maximum similarity) and (C) > as d + co. We use 'kernel' in- formally here: KC need not be positive-semidefinit but is
Kigha (d) Kheary (d) = 0.
lim d=000
(5)

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
Proof. By definition, (d) decays least exponentially with rate 0, while Krr is at most polynomial decay.
Consequently, Klight exerts stronger "pull" toward co. resulting in a strictly tighter in-class cluster than Kheavy-
Hence
In contrast, a heavy-tailed kernel better preserves a non- zero similarity at moderate distances, thereby maintaining an outward "push" that increases separation for out-of-class points.
(d)ght(d)) Kheawy(P) (d)
e e-ad
e
0.
(6)
d-B
d-+00
Lemma 6.2 (Heavy-Tailed Kernels for Out-of-Clas Sepa- ration). Assume Klight decays to nearly zero at moderate distances, whereas Kheavy remains non-negligible. Then for an out-of-class point with moderate || -ill.
confirms, that light-tailed kemelss rapidly decay to near-zero for distant points, while heavy-tailed kernels diminish more gradually, preserving longer-range distinctions.
This result underpins why one kernel function cannot, at the same distance scale, exhibit both "very small" similarity values (the ballmark of an exponential/ligtt tail) and "mod- erate" similarity values (the hallmark of a polynomia//heavy tail). If a kernel had to be "light-tail tailed" for in-class com- pactness and "heavy-tailed" for out-of-class separation at the same distance regime, it would violate the limit. Hence, Lemma 5.1 is critical: it clarifies that no single kernel can act like both an exponential tail and a polynomial tail simul- taneously..
V font
eleheary (2)1l
oou
((Kiggn)) (z)]l-
(7)
Hence, Kheavy continues pushing out-of-class samples far- ther than Klight, achieving strictly larger separation mar- gin.
6.2. No Single Kernel Can Do Both
As established in Lemmas 6.1 and 6.2, a light-tailed kernel enforces tight in-class clustering but fails to maintain a wide out-of-class margin, whereas a heavy-tailed kernel preserves a broad margin but cannot strongly pull in-class points together. Consequently, no single kernel can satisfy both goals at once, motivating a dual-tailed-kerne approach.
6. Dual-tailed kernel
In the following, we consider two disjoint classes of samples in the feature space Id:: the normal class B [ 91) E Du} and the anomalous class A (xx](z v:) Dr} (known labeled anomalies).
Theorem 6.3 (Dual-tailed Kernel Outperforms Single Ker- nel). Let single-kemel approach use the same function Kss for both sn(Ks(dm,) (in-class term) and In[1 - K ( (dout) ) (out-of-class term). Let a dual-tailed kemel approach in- stead use Küght for in-class and Khewvy for out-of-class. Under mild assumptions (e.g., sufficient model capacity or well- sepaaaeerrdeed data), any single-ken kernel method must com- promise either on compaciness for in-class points on maintaining wide out-of-class margin. By contrast, the -tall-tiied kernel design achieves both objectives simulta- neously.
6.1. Single-tailed kernel vs. Dual-tailed kernel Approaches
In the following, we demonstrate why (1) light-tailed ker- nels excel at in-class compactness, (2) heavy-tailed kernels excel at out-of-class separation, and (3) no single kernel can provide both simultancously. This naturally motivates a dual-tailed kernel approach (light-tailed kernel for in-class, heavy-tailed kernel for out-of-class). Concretely, we incor- porate two in-class separation terms, defined as
The proof follows directly by Lemmas 6.1 and 6.2.
(19) (z) = In [C(dn)] and (K) (z) = In K(don)
Based on these observations we define the dual-tailed kernel separation loss as:
The In-class term (10 (=) rewards data points for remaining near their correct center, ensuring that normal anomalous examples stay close to their respective centers. Conversely, the out-of-class term ( (=) penalizes points that approach the wrong center, driving them away from the wrong center.
Dual-tailed kernel (Klight Kheavy) Separation Loss. Building on the insight of Theorem 6.3, we unify these findings into a ratio-based logistic form. Rather than sum- ming a separate in-class reward (In(Kright()) (-)) and out-of- class penalty (In[1 - Kheavy (-))), we define the sample wise dual-tail tailet kernel loss as:
Lemma 6.1 (Light-Taiiee Tailed Kernels for In-Class Compact- ness). Suppose Klight decays very quickly at moderate dis- tances, whereas Kheavy decays more slowly. Then for point near moderately far from its own class center co.
Krsghal (din)
|| e (Knuau) (=)]l
(Kucarr) (=)
Eseparation (0: = In
"unghal (iigha(duu)) KKeavy (8)
>
5

---

Weakly Supervised Anomaly Detection via Duil-Tailed Kernel
and based on the wise loss, we define the total separation loss as:
similarity:
((C:0) = Ic
Σ
exp
[[fo(at)
(11)
C sepaaration (0)
eeeparation x,
JJEC
(2.3) ED
where o >> O is fixed scale. The diversity loss is
sum of single-sample losses
Ldiversity (0) k(A: k(U;
(12)
This encourages each sample have higher similarity to its correct center (via the light-tailed kernel) than the opposite center (heavy-tailed kernel). The ratio inside the logarithm naturally captures both closeness to the correct center (via the light-tailed term in the numerator) and dis- tance from the opposing center (via the heavy-tailed term in the denominator). Numerically, this ratio-based design is stable, and conceptually, it aligns with softmax-style inter- pretation, ensuring each sample is more "similar" to its own center than the other while leveraging the complementary strengths of light and heavy tails.
A high value of k(C; indicates that points in C' are mapped very close together in latent space (since the expo- nential term exp(- Il/o²) is close to 1). We minimize diversity to disfavor such overly tight clustering. advantage of this exponential form is its smoothly decaying gradient, which provides better control over moderate dis- tances than linear threshold metrics. Moreover, al- lowing separate dc values for normal and anomalous classes captures their differing spread without adding significant complexity.
key
To quantify the benefit of using separate light-tailed and heavy-tailed kernels, we perform an ablation in Appendix K where we replace our dual-tailed design with single kernel for both in-class and out-of-class distances. Table 8 below summarizes the main findings: We observe that the dual-tailed approach outperforms the single-kernel baseline. In particular, the heavy-tailed component preserves a more effective "push" for out-of-class samples, while the light- tailed component enforces tighter in-class clustering.
Lemma 6.5 (Diversity Lower Bound for Collapsed Map- ping). Let U and A each contain at least two samples. Suppose fa- the fully collapsed mapping:
[00
E U, x E A,
fo-
with è # δ1
Then
k(1100)== 1, k(A;0)== = 1, so Ldiversity (0") = 2.
6.3. Necessity of Diversity Loss
Relying solely on the separation loss can yield degenerate solutions if the model is sufficiently expressive (c.g., large enough parameters). In such cases, the model can drive its training error near zero by mapping every instance of class onto single point in latent space, leading to poor generalization (Goyal et al., 2020).
Based on these insights we define the overall loss ass:
(rotal()) = Eseparation (0) + Ediverstty (0)) (13)
Corollary 6.6. If the total loss is
(o) = Caeparation (0) + Ldiversity (0) with
Lemma 6.4 (Degenerate Solutions with Fixed Centers). If the network fe: DRD + IR is sufficiently expressive, then there exists a parameter set 6* such that
then at the collapsed solution 0 we have
Lrota((" Eseparation (0*) + 2 2.
(0')
Cor Eu,
Thus no collapsed solution can have zero total loss. Hence, diversity prevents degeneracy.
(o)
(10)
c1, A
and this degenerate mapping drives the training separation loss (separation(8) arbitrarily close to O.
Discussion We want to point out that the degenerate ar- rangement does not globally minimize the total loss. Con- sider non-degenerate mapping in which points lie close to their respective centers rather than coinciding exactly. By choosing this partial spread, we keep the separation loss Lgeparation (0) arbitrarily small- say e-while lowering the diversity loss Laiversity (8) below 2 by some margin > 0. In total we have:
This result motivates the introduction of the diversity term (Eq. 11), which counteracts the tendency collapse by fostering variability within each class. By penalizing over- concentration, the diversity term ensures that normal data retains subtle variations and that anomalies preserve their inherent heterogeneity, enhancing the model's robustness and generalization capabilities (Goyal et al., 2020). Specifi- cally, for class C' E (A,U), we define the average pairwise
Grotal (0) = Laspparation Ldiversiit (0)
6

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
Because the degenerate arrangement fixes the total loss at 2, any such non-degenerate solution with & 6 strietly improves upon it. Consequently, the collapsed mapping is not optimal: degeneracy cannot minimize Liotal.
model is computed as:
Kheavy ( (Ir (Irmolo)))
om (x)) = Kheavy( (Imm,(=))]) + Kheavy (Irm.1l)
The separation loss pushes each sample toward its own center and away from the opposite center, while the diversity loss preserves intra-class variability to prevent collapse.
(15)
where we define
('m.(()) (1)) Se... (2) co
and T', (x)) So (x) C1-
6.4. Computational Complexity of the Diversity Term.
(x) is bounded between [0. For om we utilize the heavy- kernel, due to the diversity term, normal points naturally spread around co. but still remain closer than anomalies. A light-tailed kernel would penalize even mod- erate distances and risk mislabeling these normal points. Instead, the heavy-tailed kernel in Eq. 15 retains sufficient similarity at moderate distances, ensuring normal points consistently receive lower anomaly scores, while anomalies lying farther out assigned higher scores. We then aggre- gate the M scores into single final score by averaging:
naive implementation of the diversity term incurs ((N2) cost for a dataset of size N. We instead compute it within each mini-batch of size b, reducing the cost to (l?) per iteration. To go further, we uniformly subsample b.- vb points from each class within the batch, yielding ((2)
(b)) complexity per iteration. This strategy preserves the diversity penalty's 's effectiveness while keeping its overhead linear in the batch size.
7. Ensemble-Basee Subset Splitting.
Let Du be the unlabeled dataset and Dr the set of la- beled anomalies. We partition Dy into M disjoint subsets, (D(")) J== by splitting the data indices into M consecu- tive blocks. As result,
<(x)
1 M
Σ m==
vm( (2)).
(16)
The anomaly score (xx) is between [0,1]. Anomalies have an anomaly score of = 1, while normal samples have a significantly lower anomaly score of z 0.
M Dim
U m=1
(m)) = Do. D = e for + j. (14)
Discussion. Splitting unlabeled data gives each ensem- ble model a distinct view of normality, while sharing the same anomaly labels ensures consistent guidance. Aggre- gating these diverse detectors improves robustness and gen- eralization under limited anomaly labels. Appendix L pro- vides an ablation study on ensemble size. Our empirical sweep (Appendix L) shows that M = 5 strikes a good balance-achievinne achieving near-optimal performance gains over smaller ensembles (like M = 1,3) without incurring the heavy computational cost of even larger ensembles. Hence, we fix M 5 as our default throughout the experiments.
Note that if [Dyl is not exactly divisible by M, the last subset may contain fewer samples, but altogether they still cover Dy fully without overlap. Each subset D(m)) then combined with the same set of labeled anomalies Dr to form a training set for the m-th model:
D(m)
D(
(m))
U Di.
Hence, every model in the ensemble sees a unique slice of unlabeled data while sharing the same anomalies.
Training Ensemble Components. Denote by fm (-) the feature mapping or anomaly score network trained on the set D(m). Let Om be its parameters, learned by minimizing a suitable objective (Eq. 13):
8. Experiments
8.1. Experimental Setup
0 arg min 2(Om; D(("), Dr.), m e..
M.
We compare WSAD-DT with state-of-the-ar deep anomaly detection methods on over 20 real-world datasets from the AdBenchmark repository (Han et al., 2022). Each dataset is split into 70% training and 30% testing, preserving the anomaly ratio via stratified sampling. In this experiment, we label only 5% of anomalies (or 5 anomalies, whichever is larger), chosen uniformly at random from the training set, ensuring minimal yet consistent supervision across all runs. Note that in these main experiments, we do not add contamination in the training data; the limited anomaly la- bels (5 5%) reflect our focus on weaker supervision (in
Since all ((m)) share Di. each model is exposed to the same limited but crucial anomaly examples, preventing any single model from entirely ignoring the labeled anomaly information.
7.1. Aggregating Ensemble Outputs.
During inference, a test point x is passed through each trained model, and the anomaly score di (1:) for the m-th
7

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
Table 1. This table presents the results of all algorithms using the default parameters outlined in the original paper. Hereby, the best values are shown in bold, and the runner-up underlined. The 'AVG Rank' row of table lists the average rank achieved by all algorithms the metric AUC-ROC, ROC. The lower the rank. the better the result. The last row of the table contains the adjusted p-value of the Wilcoxon signed-rank test at an alpha 0.05 for comparison between the WSAD-DT and reference methods. The symbol represents situations
where the WSAD-DT is statistically superior to the comparing method.
Dataset
WSAD-DT
DeepSAD
DevNet
FeaWAD
GANAnomaly
PreNet
ROSAS
XGBOD
Optdigits
0.9996 (3)
0.9773 (7)
0.9959 (4) 0.9839 (5)
0.6041 (8)
1,0000 (1)
1.0000 (1)
0,9817 (6)
Lymphography
1.0000 (1)
1.0000 (1)
0.9961 (4)
0.9787 (6)
0.8953 (7)
1.0000 (1)
0.9903
Pendigits
0.9998 (1)
0.9768 (5)
0.9682 (6)
0.7798 (8)
0.8361 (7)
0,9832 (4)
0,9997 (2)
0,9981 (3)
Vertebral
0.9051 (1)
0.7810 (5)
0.4974 (7)
0.6399 (6)
0.3618 (8)
0.9036 (2)
0,8377 (3)
0.8125
Wdbc
1.0000 (1)
0.9990 (3)
1.0000 (1)
0.9979 (4)
0.9830 (8)
0,9969 (6)
0.9933 (7)
0.9979 (4)
Wphc
0.6915 (1)
0.6144 (4)
0.6403 (3)
0.5619 (6)
0.4635 (7)
0.6649 (2)
*
0.6135 (5)
Stamps
0.9902 (1)
0.9492 (4)
0.8909 (6)
0.8664 (7)
0.7529 (8)
0.9697 (2)
0.9544 (3)
0.901 (5)
Satimage-2
0,9973 (1)
0,9715 (5)
0.9562 (6)
0.9753 (4)
0.9783 (3)
0,9243 (8)
0.9526 (7)
0.9866 (2)
Spambase
0.9499 (2)
0.8669 (5)
0,9389 (3)
0.6108 (7)
0.8453 (6)
0.9128 (4)
0.9610 (1)
Thyroid
0,9960 (2)
0,9782 (4)
0.9667 (5)
0.8525 (7)
0.7795 (8)
0,9420 (6)
0.9961 (1)
0,9795 (3)
Mnist
0.9883 (1)
0.9116 (4)
0.9037 (6)
0.8100 (8)
0.8127 (7)
0.9093 (5)
0.9609 (3)
0.9870 (2)
Yeast
0.6545 (1)
0.6336 (3)
0.5942 (5)
0,5360 (7)
0.4867 (8)
0,6309 (4)
0.6521 (2)
0.5922 (6)
Cardio
0.9908 (1)
0.9842 (3)
0.9778 (4)
0,8067 (8)
0.9084 (7)
0.9422 (6)
0.9567 (5)
0.9904 (2)
Vowels
0.9736 (1)
0,9712 (2)
0.8839 (6)
0.7696 (8)
0.8231 (7)
0,9300 (5)
0.9653 (3)
0.9516 (4)
Wine
1.0000 (I)
1,0000 (1)
0.9954 (5)
0.9491 (6)
0.6836 (7)
1,0000 (1)
*
1,0000 (1)
Magic. gammma
0.9166 (1)
0.8719 (4)
0.8282 (5)
0.7228 (7)
0.6817 (8)
0,8260 (6)
0.9034 (3)
0.9159 (2)
Jonosphere
0,9779 (1)
0.9704 (3)
0.5232 (7)
0.4238 (8)
0.6880 (6)
0,7116 (5)
0.7656 (4)
0.9746 (2)
Glass
0.9830 (1)
0.9355 (3)
0.8199 (6)
0.6891 (8)
0.7106 (7)
0.8934 (5)
0.8952 (4)
0.9606 (2)
Breastw
0,9894 (2)
0,9508 (5)
0.9947 (1)
0.9559 (4)
0,9402 (6)
0.7851 (7)
0,9842 (3)
Yelp
0.8311 (1)
0.7139 (4)
0.6812 (5)
0.5522 (8)
0.6567 (7)
0.6670 (6)
0.7950 (2)
0.7584
Imdb
0.7623 (1)
0.6730 (3)
0.6204 (5)
0.5683 (6)
0.5021 (8)
0.5464 (7)
0.6473 (4)
0.6996 (2)
MNIST-C-Fog
1.0000 (1)
1,0000 (1)
1.0000 (1)
0.9529 (7)
0.8012 (8)
1.0000 (1)
1.0000 (1)
0.9998
MNIST-C-canny-Eddee
0.9999 (1)
0.9859 (6)
0.9927 (5)
0,8818 (7)
0.6937 (8)
0.9932 (4)
0.9996 (2)
0.9982 (3)
MVTec-AD-Zipper
0.9307 (1)
0.8626 (3)
0.7728 (6)
0.5132 (8)
0.7402 (7)
0.7815 (5)
0.8993 (2)
0.8455 (4)
MNIST-C-Stripe
1.0000 (1)
1.0000 (1)
1.0000 (1)
0.9697 (8)
0.9733 (7)
1,0000 (1)
1.0000 (1)
1.0000 (1)
Skin
0.9998 (1)
0.9995 (2)
0.9937 (5)
0.5202 (6)
0.9992 (4)
0.9995 (2)
Fraud
0.9574 (2)
0.9504 (4)
0.9189 (6)
0.7855 (7)
0.9300 (5)
0.9521 (3)
0.9612
Http
1.0000 (1)
1.0000 (1)
0.9984 (4)
0.7011 (6)
0.9979 (5)
0.9997 (3)
Cover
0.9996 (2)
0.9976 (4)
0.9992 (3)
0.6304 (6)
0.4702 (7)
0.9997 (1)
0.9946
Shuttle
0,9956 (2)
0.9942 (3)
0.9775 (6)
0.9717 (7)
0.9781 (5)
0.9864 (4)
0.9993 (1)
AVGRank
1.27
3.43
4.57
6.90
6.87
4.77
3.73
3.10
p-value
N/A
0.00032220 (+)
0.0020000 9 (+)
0000000005 (+)
A 00000000 (+)
Cuc00 9809 (+)
00001901611))
0000344150 (+)
Values marked with t indicate that result available within 12 hours. Values marked with indicate that a runtime error occurred during execution.
8

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
line with (Han al., 2022: Pang et al., 2023)). For ablation study on contamination in the training data, see Appendix M. Each experiment is repeated on three splits and averaged, with performance measured by AUC-ROC and AUC-PR (Davis & Goadrich, 2006) and significance tested via paired Wilcoxon signed-rank (Holm-Bou i correction (MeDonald, 2014)). All features are scaled to [0. 1] (MinMaxScaler (Pedregosa et al., 2011)); five seeds (10,1,2,100, are used for stochastic methods; and default hyperparameters are taken from the original refer- ences (Appendix 1). We set the ensemble size to 5 (Ablation study for ensemble size see Appendix L), and provide fur- ther dataset statistics, kernel choices, parameter details, and experiments with varied labeled-anomaly proportions in Appendices H and N.
tailed kernel for robust out-of-class separation. This design allows WSAD-DT learn latent space where normal and anomalous samples are effectively distinguished, aided by diversity term that prevents degenerate all points-collapee mappings. In addition, we split the unlabeled dataset into multiple partitions while sharing a small set of labeled anomalies across them; the resulting ensemble achieves state-of- oaate-of-thh-arr results under limited supervision. Our dual- kernel similarity approach resolves the tension between tight clustering and wide margins, outperforming single-kemel methods in both in-class pull and out-of-class push. Split- ting unlabeled data into multiple partitions enables each model to develop a different view of normality, while shared anomaly labels enforce a consistent notion of anomaly. An ensemble size of M = was found to offer practical trade-off between accuracy and computational cost. More- over, the method demonstrates strong robustness to weak labels, effectively identifying anomalies using as few as five labeled anomalies across various datasets.
8.2. Real-world data
Table 1 and the Appendix F summarize the AUC-ROC and AUC-PR results across all competing methods. WSAD-DT ranks first on 23 datasets and second on 6 for AUC-ROC (Table 1), attaining average rank of 1.27. By contrast, baselines such as DeepSAD and relyin-relying on single brarbbuttions-erffrrro less effectively. WSAD-DT also excels in AUC-PR (Appendix F), rank- first on 15 datasets and second on 11, with an average rank of 1.70. Wilcoxon signed-rank tests confirm these improvements are statistically significant. The gains pri- marily arise from WSAD-DT's dual- kernel, which integrates light- and heavy-tailed similarities for tighter in- class clustering and broader inter-class margin, along with its diversity term that prevents trivial collapse. Moreover, WSAD-DT's ensemble ensemble-splittnng unlabeled data but shar- ing few labeled anomalies counaales-oupprffrrmm gradient-boosting ensembles (e.g., XGBOD), highlighting the benefits of com- bining scarce anomaly labels with diverse unlabeled parti- tions. Even under contamination in the unlabeled dataset, WSAD-DT maintains strong performance across diverse benchmarks. To assess the robustness of WSAD-DT, Ap- pendix M presents additional experiments simulating vary- ing contamination rates. Our results show that WSAD-DT maintains strong performance even as unlabeled data be- comes increasingly corrupted. In particular, it degrades gracefully compared to other methods, underscoring the resilience of the dual-tailed kernel design and ensemble approach under noisy supervision. Moreover, it shows re- silience to hyperparameter choices, reinforcing its reliability in real-world scenarios with varying data conditions.
10. Future Directions
Although WSAD-DT excels on static tabular data, it does not explicitly handle temporal or relational structures. Real- world tasks (e.g.. fraud sensor monitoring) often involve series or graph data, where anomalies manifest through evolving patterns or network connections. Adapting WSAD- DT to recurrent or transformer-bas or graph neural networks (for relational data) could signifi- cantly broaden its applicability.
centers or single-tail
models (for sequences)
9. Conclusion
We introduced WSAD-DT, a dual-tailed kernel framework for weakly supervised anomaly detection that employs a light-tailed kernel for in-class compactness and a heavy-
9

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
Impact Statement
Han, S., Hu, X., Huang, H., Jiang, M., and Zhao, Y. Ad- bench: Anomaly detection benchmark. Advances in Neu- ral Information Processing Systems, 35:32142-32119, 2022.
This paper presents work whose goal to advance the field of Machine Learning. There are many potential societal consequences of our work, none of which feel must be specifically highlighted here.
John, H. and Naaz, S. Credit card fraud detection using local outlier factor and isolation forest. Int. J. Comput. Sci. Eng, 7(4):1060-1064, 2019.
References
Abuzaid, A. H. Identifying density-basee based local outliers in medical multivariate circular data. Statistics in medicine, 39(21))2793-2799 2020.
McDonald, J. Wilcoxon signed-rank test. Handbook of biological Statistics, pp. 186-189, 2014.
Pang, G., Shen, C., and Van Den Hengel, A. Deep anomaly detection with deviation networks. In Proceedings of the 25th ACM SIGKDD international conference on knowl- edge discovery data mining, pp. 353-362, 2019.
Agrawal, S. and Agrawal, J. Survey anomaly detec- tion using data mining techniques. Procedia Computer Science, 60:708-713, 2015.
Pang, G., Shen, C., Jin, H., and van den Hengel, A. Deep weakly-supervisee anomaly detection. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1795-1807, 2023.
Akcay, S., Atapour-Abarghouei, A., and Breckon, T. P. Ganomaly: Semi-supervised anomaly detection via ad- versarial training. In Computer Vision-ACCV 2018: 14th Asian Conference on Computer Vision, Perth, Australia, December 2-6, 2018, Revised Selected Papers, Purt III 14, pp. 622-637. Springer, 2019.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V.,
Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cour- napeau, D., Brucher, M., Perrot, M., and Duchesnay, Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825 2830, 2011.
Bartlett, P. L. and Mendelson, S. Rademacher and Gaus- sian complexities for risk bounds. Journal of Machine Learning Research, 3:463 482, 2002.
Breunig, M. M., Kriegel, H.-P., Ng, R. T., and Sander, J. Lof: identifying density- based local outliers. In Proceedings of the 2000 ACM SIGMOD international conference on Management of data, pp. 93-104, 2000.
Ramaswamy, S., Rastogi, R., and Shim, K. Efficient al- gorithms for mining outliers from large data sets. In Proceedings of the 2000 ACM SIGMOD international conference Management of data, pp. 427-438, 2000.
Cristianini, N. and Shawe-Taylee port Vector Machines and Other Kernel-based Learning Methods. Cambridge University Press, Cambridge, UK, 2000. ISBN 0-521-78019-5.
Introduction to Sup-
Ruff, I.., Vandermeulen, R., Goernitz, N., Deecke, I.., Sid- diqui, S. A., Binder, A., Müller, E., and Kloft, M. Deep one-class classification. In International conference on machine learning, pp. 4393-4402 PMLR, 2018.
Ruff, L., Vandermeulen, R. A., Görnitz, N., Binder, A., Müller, E., Müller, K.-R., and Kloft, M. Deep semi-supervised anomaly detection. arXiv preprint arXiv: 1etp:/906.0294, 2019.
Davis, and Goadrich, M. The relationship between precision-recall and roc curves. In Proceedings of the 23rd international conference on Machine learning. pp. 233-240, 2006.
Schölkopf, B. and Smola, A. J. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge, MA, USA, 2002. ISBN 778-02621947544
Gönen, M. and Alpaydmn E. Multiple kernel learning algo- rithms. The Journal of Machine Learning Research, 12: 2211-2268, 2011.
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. Generative adversarial nets. Advances in neural informa- tion processing systems, 27, 2014.
Schölkopf, B., Platt, J. C., Shawe-Taylor, J., Smola, A. J., and Williamson, R. C. Estimating the support of a high- dimensional distribution. Neural computation, 13(7): 1443 1471, 2001.
Goyal, S., Raghunathan, A.. Jain, M., Simhadri, H. V., and Jain, P. Droce: Deep robust one-class classification. In International conference on machine learning, pp. 3711 1- 3721. PMLR, 2020.
Tao, X., Peng. Y., Zhao, F., Zhao, P., and Wang, Y. A parallel algorithm for network traffic anomaly detection based on isolation forest. International Journal of Distributed Sensor Networks, 14(11):15501477) 075550147778811- 2018
10

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
Tax, D. M. J. and Duin, R. P. W. Support vector data de- scription. Machine Learning, 54(1):45-66, 2004.
Xu, H. Deepod: A library for deep anomaly detection. https::/gitthh com/ xuhongzuo/ DeepOD, 2023. Accessed: 2025-01-28.
Xu, H., Pang, G., Wang, Y., and Wang, Y. Deep isola- tion forest for anomaly detection. IEEE Transactions on Knowledge and Data Engineering, 2023a.
Xu, H., Wang, Y., Pang, G., Jian, S., Liu, N., and Wang, Y. Rosas: Deep cepi-super anomaly detection with contamination-resllienn continuous supervision. Informa- lion Processing & Management, 60(5):103459, 2023b.
Zhao, H. and Fu, Y. Dual-regularized multi-view outlier detection. In Proceedings of the Twenty-Foun Fourth Interna- tional Joint Conference on Artificial Intelligence (ICAI), pp. 4077-4083, 2015. URL https:/ www. ijcai. org prg/Pzee Ptte///ccceding///// 15 Patpps/ppppspp..p/ pdf.
Zhao, Y. and Hryniewicki, M. K. Xgbod: improving super- vised outlier detection with unsupervised representation learning. In 2018 International Joint Conference on Neu- ral Networks (IJCNN), pp. 1-8. IEEE, 2018.
Zhao, Y., Nasrullah, Z., and Li, Z. Pyod: A python toolbox for scalable outlier detection. Journal of Ma- chine Learning Research, 20(96):1-7, 2019. URL http: //jmlr. org/ hapers/v200 htm..mmll.
Zhou, Y., Song, X., Zhang, Y., Liu, F., Zhu, C., and Liu, L. Feature encoding with autoencoders for weakly super- vised anomaly detection. IEEE Transactions on Neural Networks and Learning Systems, 33(6):2454 2465, 2021.

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
Appendix Section
Content
Appendix A
Notation summary
Appendix B
Proof of Lemma 6.1
A Appendix C
Proof Lemma 6.2
A Appendix D
Proof Lemma 6.4
Appendix
Proof of Lemma 6.5
Appendix F
AUC-PR results Algorithms Details
Appendix G
A Appendix H
Experiment details
Appendix
Implementation Details
Appendix
Scalability Test
Appendix K
Ablation study light-tailed, heavy-tailed, dual-tailed kernel Analyzing the performance under contamination in the training data Ablation study the effect of different numbers of ensembles Ablation study for different numbers of labeled anomalies Parameter Sensitivity
Appendix M
Appendix L
Appendix N
Appendix O
Appendix P
Ablation study kernel regularization Limitation
Appendix Q
Table 2. Structure of our appendix.
A. Notation summary
Table 3 provides a summary of the notation used throughout this paper.
Table 3. Notation Summary
Symbol
Description
Do
Unlabeled dataset (primarily normal but may be contaminated) Labeled set of anomalies
DL
N,K K
Number of unlabeled points (N), number of labeled anomalies (K)
2 E RD
i-th input data point (feature vector of dimension D)
V E 10,
Binary label indicating normal (y = 0) or anomaly (y = 1)
fo(-)
Neural network mapping from input space to latent space Latent representation (embedding) of point J;;
2, So(
co: C1 Rd
Centers in latent space for normal (cp) and anomalous (ci) classes
di
Distance between 2; and center Cg. e.g. || |=
din dout
In-class vs. out-of-class distance for a sample, depending on y/ Light-tailed kernel (e.g. Gaussian), for in-class distances Heavy-tailed kernel (e.g. Student-t), for out-of-class distances
Klighe (d)
Kheavy (d)
Tu A.A
Bandwidth parameters for the light- tailed kernels (nornal/anomaly)
Degrees-of Regrees-of-freedde of-freedom parameter for the heavy-tailed kernel
Eseparation (
Separation loss term
Laiversity (0)
Diversity (regularization) loss term
(rotal())
Overall training objective, sum of separation + diversity Anomaly score output by the m-th model in the ensemble Final ensemble anomaly score (aggregation dm.)
Φm((x)
Φ(x)
12

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
B. Proof Lemma 6.1
Proof. 1. General Gradient Form. For (z) = In([C(d) with = || - coll. we compute:
(1) in (z) dK((d)
K(d) V= cu
dd)
Since
V || coll =
2 Co || Cu
we have
(1 in
(z) = K(d)
K'(d)
2 Co || col
magnitude
IV ( (2)]1 = x(d)
K'(d)
in
2. Moderate to large ||2
coll: Klight is smaller = bigger reciprocal.
(only) heavy
V
(Knnghe) (=)
(z)
for large d.
in
Because Klight (d) ~ e-od, we get
Klight (d) = - C aa
Thus
K light (d)
0 -ad
Kright (d)
e e-ad
a
constant in magnitude. Meanwhile, Kright (d) itself is extremely small for large d, making In[Kiight (d) strongly negative and its gradient large in magnitude. This yields a strong inward pull for any in-class point at moderate/large distance
If we replaced Kiight by heavier kernel Kheavy ~ d= then at moderate we have
Kheavy (d) ~ (-Bd Kheavy(d) Kheavy (d)
3
+ 0.
d
This smaller ratio translates weaker inward gradient. Therefore, Klight is strictly more "aggressive" about pulling in-class points near co-
Thus, the light-tailed kernel exerts stronger inward pull, preventing normal points from staying at large distances from the true center.
Because the Klight gradient strictly exceeds that of x(ooll) for moderate to large distances, it never tolerates a normal point lingering far from Cg- Consequently, the final in-class cluster strictly tighter under Kiight-
Remark 1. In a deep model, fa( depends on parameters Although the proof here focuses on V, l, the actual training updates weights via chain rule:
V (0) = V 4m V = (V= x (Vo
Since larger ||V, fll directly contributes to larger V E||, the strong "pull" evident in the latent space indeed translates into stronger parameter updates in backpropagation. Therefore, if Kright induces a bigger inward gradient on it also induces larger weight updates, more tightly clustering in-class points around
Remark An identical argument applies if we replace co by another class center c₁ The same exponential vs. polynomial decay comparison shows that light-tailed tailed kernel Kiight always exerts a stronger inward pull, regardless of which center is in question.
13

---

Weakly Supervised Anomaly Detection via Duil-Tailed Kernel
C. Proof Lemma 6.2
Proof. Let d = ||z - cill. Then:
e ou)(2) = In 1
V (A (z) (z) 1 K(d) (- (d))
2
((A)) (z) with respect to 2. Using the chain rule:
Focus moderate d: We compute the gradient of
eall (2) Oz
In K(d)].
Let ||2 a 1.. Then:
In 1 K(d)]
1 K(dd [xd'dd)
0d
and
((d) 0
2 c0
2 all
Oz 0z
|| - col
Thus:
V (10) (2) - -d'
2- c1
1- K((d)
--aa
Note that (C) is the derivative of IC with respect to || -cll.
3. The behavior of Light-Tailes Tailed vs. Heavy-Taiee Tailed Kernels at Moderate Distances (a) Light-Tailed Kernel Klight By definition, a light-tailed kernel rapidly decays toward (or extremely close to) zero when is only moderately large. Hence, for moderate d, we typically have:
Klight (d) O, Klight (d) O.
Substituting into the gradient formula:
(nnght)) )
(2)
0 2 0.
1-0
0 ci
Consequently, the out-of-class log-loss In[1 Kright (d)) saturates In(1) 0. and the gradient becomes nearly zero. This implies there is almost no incentive to push farther away from C1.
(b) Heavy-TTalees Kernel Kheavy: A heavy-tailed kernel retains moderate positive values even at moderate distances. For such d, we might have:
Kheavy (d) + O, Kheavy (d) #
Then:
-Ky Kheevyy (d)
(Kheavyy C
(2) =
2- C #0.
Kheavy (d)
Ci
This gradient remains non-trivial, meaning the loss still increasing as moves outward. The model thus continues "push" away from C1, promoting wider separation boundary.
Because Klight effectively becomes negligible at moderate distances, (Knggh) saturates and yields no gradient for pushing out-of-class points farther. In contrast, Kheavy preserves a moderate value at similar distances, ensuring (Kpawvy) continues to provide meaningful gradient that drives away from Cj.
Conclusion
out
Remark 1. Again, if 2 fa is computed by neural net with parameters 6 then standard backpropagation yields:
Vo Lout (0) = V Lout X Voz.
Hence a sustained non-zero gradient in V lout (under heavy tails) also implies a sustained non-zero gradient in V lout> causing the network weights to keep shifting away from e₁. By contrast, a light-tailed kernel's near-zero V. at moderate translates into minimal weight updates once already somewhat far from C1-
14

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
Remark 2. The same argument applies if we swap centers. We specifically showed how light-tailed kernel saturates when pushing away from C1; however, one could equally consider pushing a point away from co if it out-of-class for that center. The key principle remains: heavy-tailed kernel continues to produce nonzero gradient at larger distances, while light-tailed kernel saturates and stops pushing.
Remark (Moderate Distances), Throughout Lemmas 5.1 and 5.2, we say sample z is at moderate distance from a center Cg if ||= - c is neither so small as to be effectively zero, nor so large that the kernel has saturated at (or near) zero. Concretely, let min: dmax > be thresholds such that
Ōmin c|=l <
Ōmax-
Near Distance (I|= ckll || Rz 0) leads to similarity K[[1= -x+l) = 1.
Large Distance (| ck 1) often results in KC[]2 - cxll) for light-tailed kernel (or near saturation for
heavy-tailed kernel).
Moderate Distance precisely the range where the kernel's 's decay profile (exponential vs. polynomial) meaningfully differs and produces nontrivial gradient behavior.
In practice, "moderate distance" captures the radius at which a light-tailed kernel begins drop off sharply, or heavy- tailed kernel remains significantly above zero. is in this regime that Lemmas 5.1 and 5.2 highlight how light-tailed vs. heavy-tailed kernels yield distinct advantages for in-class compactness and out-of-class separation, respectively.
Experiment validation To further validate these observations, conducted experiments some real-world datasets, training each model for 100 epochs and reporting the results in Table 4. Our analysis focused on the average distances of anomalous and normal points to both the anomaly center (c1) and the normal center (co), as detailed below:
i.a.
Σ d(z,ci),
Σ
d(z,co),
Ain
Aout
aeAin
Σ d(z,co), zEl4in
Σ d(z,c1).
in..
onn. = Hout
Umm
xEldous
Here:
Ain and Lin denote points correctly assigned their respective centers (anomalous to cı, normal to og).
Aout and Maut refer points assessed in relation to the center of the opposite class.
Key Findings
In-Class Distances (i.a. and i.n.:) The light-tailed kernel achieves significantly smaller in-class distances compared the heavy-tailed kernel. This indicates that the light-tailed kernel promotes tighter clustering around the respective centers, confirming its ability enforce compactness for in-class points.
Out-of-Class Distances (o.a. and o.n.:) The heavy-tailed kernel results in larger out-of-class distances, effectively separating points from the center of the opposing class. This observation aligns with the heavy-tailed kernel's design, which preserves moderate similarity for points farther away, thereby maintaining a wider margin between classes.
Implications
The results validate the complementary roles of the light and heavy-tailed kernels as described in Lemmas 6.1 and 6.2. Specifically:
The light-tailed kernel excels at compacting in-class points, ensuring that normal and anomalous points cluster tightly around their respective centers.
15

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
The heavy-tailed kernel enforces broader margins, preventing out-of-class points from being falsely pulled toward the wrong center.
By leveraging both kernels, the dual-tailed kernel approach achieves robust in-class compactness and out-of-class separation, which are critical for distinguishing anomalies from normal points. This dual mechanism addresses the limitations of single-kernel methods and demonstrates superior adaptability across diverse datasets (see ablation study in Appendix K).
Table 4. Comparison of Average Distances Using Light- vs. Heavy-Tailed Kemels
Light-Tailed Kernel
Heavy-Tailed Kernel
Dataset
i.a.
o.a.
i.n.
o.n.
i.a.
o.a.
i.n.
o.n.
Vertebral
0.3556
0.5176
0.4038
0.4495
0.8766
1.0620
0.9960
1,0299
WDBC
0.0568
1.1348
0.1310
1.2099
0.2737
1.4539
0.2862
1.4368
WPBC
0.1294
0.5579
0.3189
0.6069
0.5693
0.9703
0.9013
1.1783
Stamps
0.0588
0.8711
0.1115
0.8529
0.2683
1.0988
0.4663
1.2703
Satimage-2
0.0274
0.9126
0.0581
0.9499
0.2904
1.2005
0.3640
1.2727
Magic. gamem.
0.2421
0.5414
0.1981
0.6531
0.7599
1.1767
0.6396
1.1107
Yeast
0.2427
0.5069
0.3010
0.4875
0.7474
1.0097
0.8511
0.9507
Vowels
0.0363
0.6670
0.1455
0.7584
0.2462
0.8767
0.7248
1.3530
Ionosphere 0.0960
0.8152
0.1026
0.7668
0.3943
1.1142
0.5042
1.2127
D. Proof of Lemma 6.4
Proof. Zero Separation Loss. The per-sample separation loss for a point (I., y) takes the form
Klight (Ilfa (x)) - Cinll) Kiight (11fo(z)) - qall) Kneavy (1]f((x) Cowlll
eseparation
(0:2, (0:
In
where Cjn the "correct" center for x (i.e., co if y O. or ei if 1). and Cout the opposite center.
Consider the mapping
fe- (x) = Co. = 0,
C1
Then [].a. (Ir) Cin || = 0. Since Kiight (0) & and ||co || > O, the ratio inside the logarithm approaches making Eseparation (8"; y) ~ 0 for each sample. Summing over the entire training set, Lseparation (f*) = 0.
Class Collapse and Consequences. All normal points (y = 0) are mapped to cp. and all anomalous points (y 1) are mapped с₁. Hence, each class is collapsed onto single coordinate in the latent space, destroying any intra-class variability. While it yields "perfect" separation the training set, this degenerate solution generalizes poorly (Goyal et al., 2020).
Conclusion. With fixed co and C1, sufficiently flexible fo can achieve near-zero training separation loss by collapsing each class onto its respective center. This justifies the need for additional regularization (e.g., a diversity term) to preserve intra-class structure and avoid trivial collapse.
E. Proof of Lemma 6.5
Proof. Fix 8* such that fe- (I) is constant on 24 Pick any I,, æ, E U. Then (.. - fo- (x,))) = || - oo1 = 0, so the exponential similarity exp(-lle - 2)11/o2) = cxp(0) = Hence every pair (i.j) E 24 x l contributes Summing over 2 pairs and dividing by [ yields k(4;; 8*) = Identical reasoning applies to A. A summation of the two completes the proof.
16

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
Corollary: If the total loss is ( Eepparation! (0) + Ldiversitty (0) ,then at the collapsed solution 0* have
Looall Eseparation (0*) + x2 2 2.
Thus no collapsed solution can have zero total loss with diversity penalty. Hence, diversity prevents degeneracy. Compactness from separation.
Although the diversity term penalizes overly tight cluster, the separation objective Ceeparation (0) still rewards each class-k point = fo (z;) for lying near its center Cj. If point drifts arbitrarily far from Ck. Lseparation grows, hurting the total objective. Therefore, at the optimum 0", each class remains relatively compact around ex (to keep the separation cost low), yet not collapsed to a single point (to keep diversity cost low).
Conclusion, By combining separation and diversity:
Non-collapse:
The diversity penalty ensures that no class fully collapses onto one coordinate.
Compactness: The separation objective still keeps each af near its respective Cy., all points of class is form cluster around o with moderate scatter.
This balance yields robust representations that preserve in-class variability and clear separation from other classes (Goyal al., 2020).
Conclusion
By introducing a diversity loss, we ensure that any attempt to collapse all class members to a single point incurs large penalty, thus preventing the degenerate mappings that minimize the separation loss alone. This mechanism encourages the learned representations maintain more realistic spread of samples within each class, ultimately improving generalization and robustness in anomaly detection tasks. In Appendix D, we compare WSAD-DT with and without the diversity term, providing empirical evidence that the inclusion of the diversity term leads to improved anomaly detection.
F. Addtional AUC-PR results
In addition to the AUC-ROC results presented in the main paper. we also report the AUC-PR (Area Under the Precision- Recall curve) (Davis & Goadrich, 2006) scores for all competing methods. Table 5 summarizes these additional experiments on the same benehmark datasets described in Section H.
Consistency with AUC-ROC. As with ROC-AUC, WSAD-DT consistently achieves strong AUC-PR performance across the majority of datasets. This reaffirms its robustness under heavily imbalanced scenarios, where AUC-PR often considered more informative than AUC-ROC.
Robust Ranking. Based on WSAD-DT's average rank in AUC-PR, we conclude that it not only effectively separates anomalies from normal points but also preserves high precision under severe class imbalance, where the prevalence of anomalies is especially low.
Overall, the AUC-PR results are consistent with the main findings based on AUC-ROC, further reinforcing the effectiveness of WSAD-DT under weak supervision.
17

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
Table 5. This table presents the results of all algorithms using the default parameters outlined in the original paper in the metric AUC-PR.
Dataset
WSAD-DT
DeepSAD
DevNet
FeaWAD
GANAnomaly
PreNet
ROSAS
XGBOD
Optdigits
0.9890 (3)
0.9469 (5)
0,9830 (4)
0.8320 (6)
0.0376 (8)
0.9991 (2)
0.9994 (1)
0.7631 (7)
Lymphography
1.0000 (1)
1.0000 (1)
0.9444 (4)
0.7278 (6)
0,5051 (7)
1,0000 (1)
0.8528 (5)
Pendigits
0.9915 (2)
0,9265 (6)
0.9313 (5)
0.5205 (7)
0.2064 (8)
0.9454 (3)
0.9946 (I)
0.9392 (4)
Vertebral
0.6342 (1)
0.4025 (4)
0.1538 (7)
0.3150 (6)
0.1136 (8)
0.6242 (2)
0.5161 (3)
0.3958 (5)
Wdbe
1.0000 (1)
0,9722 (3)
1.0000 (1)
0.9500 (4)
0.7641 (8)
0,9306 (6)
0.8718 (7)
0,9444 (5)
Wpbc
0.4495 (2)
0.4121 (3)
0.3424 (5)
0.3296 (6)
0.2606 (7)
0.4858 (1)
0.3744 (4)
Stamps
0.9254 (1)
0.7155 (4)
0.5349 (6)
0.5357 (5)
0.2316 (8)
0.8084 (2)
0.7703 (3)
0.5232 (7)
Satimage-2
0.9122 (2)
0.8694 (6)
0.8624 (7)
0.8701 (5)
0.3463 (8)
0.8711 (4)
0,9004 (3)
0.9163 (1)
Spambase
0,9283 (2)
0,8650 (5)
0.9005 (3)
0.4589 (7)
0,8550 (6)
0.9004 (4)
0.9451 (1)
Thyroid
0.8894 (1)
0.7866 (5)
0.7847 (6)
0.6200 (7)
0.2137 (8)
0.8263 (4)
0,8878 (2)
0.8465 (3)
Mnist
0,9040 (2)
0.8078 (5)
0.7978 (6)
0.4794 (7)
0.2665 (8)
0.8492 (4)
0.8849 (3)
0.9114 (1)
Yeast
0,4716 (4)
0.4807 (2)
0.3920 (6)
0.3472 (7)
0.3239 (8)
0.4780 (3)
0,4888 (1)
0.4262 (5)
Cardio
0.9450 (2)
0.9465 (1)
0.8693 (5)
0.5509 (8)
0.5967 (7)
0,8629 (6)
0.8716 (4)
0.9265 (3)
Vowels
0.8127 (3)
0.8205 (1)
0.4429 (6)
0.2680 (7)
0.2448 (8)
0.7695 (5)
0.8090 (4)
0.8154 (2)
Wine
1.000 (1)
1.0000 (1)
0.9639 (5)
0.8905 (6)
0.2366 (7)
1.0000 (1)
1,000 (1)
Magic.gammat gamma
0.8767 (2)
0.8449 (4)
0.6975 (6)
0,6480 (7)
0,5577 (8)
0,8083 (5)
0.8718 (3)
0.8854 (1)
Jonosphere
0.9658 (2)
0.9541 (3)
0.5213 (7)
0.4288 (8)
0.6709 (6)
0.7566 (5)
0.7942 (4)
0.9659 (1)
Glass
0,8310 (1)
0.5999 (4)
0.1867 (6)
0.1286 (8)
0.1365 (7)
0,6732 (3)
0.4341 (5)
0.7455 (2)
Breastw
0.9764 (2)
0.9233 (6)
0.9895 (1)
0.9261 (5)
0.9349 (4)
0,8082 (7)
0.9725 (3)
Yelp
0.3663 (1)
0,2899 (3)
0.1321 (6)
0.0703 (8)
0.0817 (7)
0,2564 (4)
0.3523 (2)
0.1896 (5)
Imdb
0,2335 (1)
0. 0.1602 (2)
0.0752 (6)
0.0645 (7)
0.0499 (8)
0.1434 (4)
0.1541 (3)
0.1203 (5)
MNIST-C-Fog
1.0000 (1)
1.000 (1)
1.0000 (1)
0.8680 (7)
0.1794 (8)
1.0000 (1)
1.0000 (1)
0.9960 (6)
MNIST-C--cany-Edeee
0,9974 (1)
0.9655 (5)
0.9626 (6)
0.6155 (7)
0.0832 (8)
0,9860 (3)
0,9952 (2)
0,9741 (4)
MVTec-AD-Zipper
0,8718 (1)
0.8021 (3)
0.6757 (6)
0.3894 (8)
0.6298 (7)
0.6901 (5)
0.8557 (2)
0.7750 (4)
MNIST-C-Stripe
1.0000 (1)
1.0000 (1)
1.0000 (1)
0.9421 (7)
0.6279 (8)
1,0000 (1)
1.0000 (1)
0,9992 (6)
Skin
0.9990 (1)
0.9972 (3)
0.9446 (5)
0.2519 (6)
0.9902 (4)
0.9979 (2)
Fraud
0.7437 (2)
0.7361 (3)
0.4941 (6)
0.2695 (7)
0.6274 (5)
0.6832 (4)
0.7978 (1)
Http
0.9985 (1)
0.9947 (2)
0.9908 (4)
0.2886 (6)
0.9798 (5)
0.9921 (3)
Cover
0.9647 (4)
0.9851 (2)
0.9512 (5)
0.1392 (6)
0.0091 (7)
0.9861 (1)
0,9810 (3)
Shuttle
0.9901 (2)
0.9865 (3)
0.9680 (5)
0.9546 (6)
0.9359 (7)
0.9773 (4)
0.9978 (1)
AVGRank
1.70
3.23
4.90
6.77
7.27
4.10
3.60
3.37
p-value
N/A
0.00192370
0000116511
0U00000005
0.00200005 (1)
0U00044749
0J00362462
0.0611554411 (1)
Values marked with i indicate that no result was available within 12 hours. Values marked with * indicate that a runtime error occurred during execution.
G. Algorithm details
In Algo. 1 we describe WSAD-DT in detail. The core of WSAD-DT is to leverage both labeled anomalies and unlabeled data in a weakly supervised setting by training multiple models (an ensemble) and then aggregating their anomaly scores, First, the unlabeled dataset is split into M disjoint subsets; each subset is augmented with the same small set of labeled anomalies, ensuring that every model has consistent information about the known anomalies while observing different portions of unlabeled data. This yields M distinct training sets, each used to train separate neural network mapping inputs to latent representation. Within the latent space, two distinct centers are maintained, one for normal points (co) and another for anomalies (c1). A dual-kemn' approach drives the representations toward the correct center and away from the incorrect one. Specifically, a light-tailed kernel is applied to in-class distances, causing normal or anomalous samples to cluster tightly around their correct center, whereas a heavy-tailed kernel is applied to out-of-class distances, keeping mismatched samples at a broader margin. This two-part strategy facilitates both tight grouping for in-class points and clear separation between classes, which a single kernel cannot achieve simultancously. A crucial challenge is the risk of degenerate solutions where all points of given class collapse onto a single coordinate the latent space. To mitigate this, WSAD-DT introduces a diversity regularization term that penalizes pairs of points in the same class that are mapped too closely together. This penalty protects intra-class variability by discouraging trivial collapse onto each center. Finally, once every network in the ensemble is trained, each model's 's anomaly scores for a new sample are combined, by averaging, to yield final ensemble score. This last step enhances stability and robustness, especially given the scant availability of labeled anomalies, by capitalizing on
18

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
diverse perspectives of the unlabeled data while retaining consistent guidance from the same known anomalies.
Algorithm WSAD-DT
Require: Labeled anomalies Dz = {(x,,, 1)}, Unlabeled data Dy = ((z,,00} Ensemble size M, mini-batch size b, max epoch maxEpoch
1: Partition Dy into M disjoint subsets (Di (m) JM= each combined with Dr to form D(m))
2: for m m= 1 to M do
3: Initialize network parameters Om and centers co, C1-
4: for epoch 1 to maxEpoch do
5: for each mini-batch W D(m) (of size b) do
6: Step 1. Compute embeddings 2; fem (2t) for all E WV
7:
Step 2. Compute separation loss (Lseperation):
separation (0)
csoparation (0; x. v)
(a.21) W
single-sample losses
8:
Step Subsampling for diversity:
9:
Let Wij {z (),,0) E W') and = (z,, 1) W"). Uniformly randomly sample Lvb] points from each W yielding subsets W W
11:
Step 4. Compute diversity loss (Ldiversity) )
Ldiversity (0) = Σ
Σ
exp
W'
kE(0.1)
#c,yy EW'
12:
Step 5. Gradient update:
Om f mm Ve.. (Ceeparation (0) + Ldiversity (0))
13:
end for
14: end for
end for
16:
Inference: For each test sample æ, each model outputs
K Kheavy fll (x) coll)
Φm (x)
Kheavy (1.sem. (x) co (a Kheavy (1 (x) c1
17: Aggregate ensemble outputs:
M Σ m==1
Φ(x) = M
Om (m))
Final anomaly scores ((x).
H. Experiment details
Experiment Setup
All experiments were conducted on workstation equipped with an Intel Core 17-10700K CPU (3.8 GHz) and 32 GB of RAM. We repeat all experiments three different splits and report average results. Performance is measured with AUC-ROC and AUC-PR (Davis & Goadrich, 2006), and significance is determined via paired Wilcoxon signed-rank test with foll--Boneerroni correction (McDonald, 2014). All features are scaled to [0, 1] using MinMaxScaler (Pedregosa et al., 2011). For stochastic methods, five random seeds (0.1,2,100, are used, and default hyperparameters according to (Xu, 2023).
19

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
Datasets and Splits
We adopt a diverse of real-world anomaly detection benchmarks from AdBench (Han et al., 2022) (Table 6), each split 70%-30% for training and testing via stratified sampling preserve anomaly ratios. In line with (Han et al., 2022), only small fraction of anomalies in the training split is labeled (weak supervision), specifically labeling either anomalies or p% of mnohalies-whichever whichever is greater-to to ensure a minimal but consistent supervisory signal across all experiments. These labeled anomalies are selected uniformly at random from the available anomalies. This setup reflects real-world conditions, where annotated anomalies are scarce and often fail to capture the breadth of abnormal behaviors. We further analyze different fractions of labeled anomalies Appendix and contamination in the training data in Appendix M.
Dataset
# Instances
# Dimensions
# Anomalies (%)
Optdigits Lymphography
5216
64
150(0.0288)
148
18
6(0.0405)
Pendigits
6870
156(0.0227)
Vertebral
240
30(0. 1250)
Wdbe
367
30
10(0.0272)
Cardiotocography
2114
21
466(0.2204)
Wpbc
198
33
47(002374)
Stamps
340
9
31(0.0912)
Satimage-2
5803
36
71(0.0122)
Spambase
4207
57
1679(0.3991)
Thyroid
3772
93(0.0247)
Mnist
7603
100
700(0.0921)
Yeast
1484
507(0.3416)
Cardio
1831
21
176(0.0961)
Vowels
1456
12
50(0.0343)
Wine
129
13
10(0.0775)
Magic.gamma gamma
19020
10
6688(0.3516)
Ionosphere
351
126(0.3590)
Glass
214
900.4221
Breastw
683
239(0.3499)
Yelp
10000
512
500(0.0500)
Imdb
10000
512
500(0.0500)
MNIST-C-Fog
10000
512
500(0.0500)
MNIST-C-camny-Fdges
10000
512
500(0.0500)
MVToc-AD-Zipper
10000
512
500(0.0500)
MNIST-C-Striper
10000
512
500(0.0500)
Skin
245057
3
50859(0.2075)
Http
567498
3
2211(0.0039)
Cover
286048
10
2747(0.0096)
Shuttle
49097
9
3511(0.0715)
Table 6. Statisties the used datasets
Implementation and Code
Our code is implemented in PyTorch and builds on top of the DeepOD and PyOD libraries (Zhao et al., 2019; Xu, 2023).
Our anonymous code repository: Link (Anonymous).
Baselines
We compare our method (WSAD-DT) with state-of-the-art anomaly detection baselines, including DeepSAD, DevNet, FeaWAD, GANomaly, PReNet, RoSAS, and ensemble- based XGBOD. For each baseline, we used the open-source implemen- tation from (Xu et al., 2023a; Han et al., 2022); further details can found Appendix I. By default, XGBOD employs multiple unsupervised detectors (including KNN (Ramaswamy et al., 2000) and LOF (Breunig al., 2000)). However,
20

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
on datasets exceeding 100000 samples, these detectors took over 12 hours complete. We therefore omit KNN and LOF in such large-scale cases keep XGBOD''s runtime feasible, while still leveraging the remaining detectors for reliable anomaly scores.
Kernel Choices & Hyperparameters
One key novelty in WSAD-DT is the dual-kernel design: light-tailed kernel for in-class similarity and heavy-tailed kernel for out-of-class separation. Below, outline each kernel and its bandwidth parameters (a, 1/)
H.1. Light- Taiht-Tailed Kernel (Gaussian)
Klight (di,k) exp(
di,k)
2
0y 0.5 for the normal center, ensuring tighter cluster for normal data.
0 1.0 for the anomaly center, allowing slightly looser cluster for more diverse anomalies.
H.2. Heavy-Tailed Kernel (Student-t)
ut4 Default // = 0.2. A smaller corresponds to a heavier tail, which helps separate out-of-class points more aggressively.
Kheavv (di,,) (1 di )
Diversity Term
key risk of minimizing only the separation loss is collapse, where all points in class map the same latent coordinate. To counteract this, introduce class-specific exponential diversity penalty. For each class C' E {A,,), first measure the average pairwise similarity:
k(C:0)
Σ exp
[ 2
c
í.jCC
where oc > 0 is a fixed bandwidth parameter for class C. The overall diversity loss is then defined as
Ldiversity (f) = k(A:0) + 1)(4;))
Role of oc-
In practice, the parameter oc controls how strongly the method penalizes points that are closely mapped in
latent space:
Small oe (e.g., divversity.2 = 0.1 for normal data) leads to weaker penalty for moderate distances, partial clustering incurs but still effectively prevents any accidental clustering into a single point.
Larger dc (e.g., ddiversity.4 = 1.0 for anomalies) allows more dispersion among inherently diverse anomalies, since moderate pairwise distances will incur a sufficient penalty.
A high value of (); indicates that points in class C are excessively close in the latent space, triggering larger penalty and thus discouraging trivial collapse. By adjusting oc appropriately for each class, can balance intra-class variability against the risk of collapse. In practice, these parameters may be selected via cross-validation or a small parameter search, depending on dataset size and diversity.
detailed ablation study on hyperparameter selection is provided in Appendix O, offering comprehensive insights into the impact of each parameter on model performance.
Ensemble Setup (M splits)
By default, WSAD-DT uses M ensemble splits (unless otherwise noted). An ablation over M E (1,3,5) is reported in the Appendix L, showing that M typically yields the best or near-best performance.
21

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
Evaluation Metries & Statistical Significance
Metries
We use two widely adopted metrics to evaluate anomaly detection models:
1. ROC-AUC (Receiver Operating Characteristic-Ares Under the Curve) (Davis & Goadrich, 2006): This measures the trade-off between the True Positive Rate (TPR) and the False Positive Rate (FPR) over different decision thresholds.
A higher ROC-AUC indicates stronger discriminative power in distinguishing anomalies from normal points.
2. PR-AUC (Precision-Reell--AAr Under the Curve) (Davis & Goadrich, 2006): Unlike ROC-AUC, PR-AUC focuses on Precision (the fraction detected anomalies that are actually anomalous) and Recall (the fraction of anomalies correctly identified).
By reporting both metrics, gain comprehensive view of each method's effectiveness in detecting anomalies across varying degrees of class imbalance.
Statistical Testing
We adopt the paired Wilcoxon signed-rank with Holm-Bonferroni correction compare WSAD-DT against each baseline 0.05. We highlight "4" if WSAD-DT is significantly superior, "_" if worse, and "x"" otherwise. In the reporting tables, we list the adjusted p-values with
correction.
Ranking
All tables include "average rank" across all datasets; smaller rank value indicates better overall performance.
I. Implementation Details
All algorithms are implemented as described in their respective original papers and integrated into the publicly available DeepOD library (Xu, 2023), following uniform training protocol to ensure fair comparisons across methods. Each experiment is conducted on the same data splits, using consistent data loading and preprocessing procedures.
Network Architecture. We employ a three-layer feed-forward encoder with hidden sizes of 100 and 50, followed by projection into final embedding dimension of 128, aligning with configurations used in approaches such as DeepSAD, Formally, the neural network has the layers:
input dimension + 100 50 - 128.
Activation Functions. In proposed method WSAD-DT, utilize the Scaled Exponential Linear Unit (SELU) activation function at each layer, which found stabilize training and often accelerate convergence. For other baselines, adhere to the recommended activation functions specified in their original publications, such as ReLU or LeakyRel.U. SELU activation function is defined as:
Ax
if > 0, if x 0,
SELU(z) = Aa(E=
where:
X 1.0507 is the scaling parameter,
a Rs 1.67326 the parameter that controls the slope for negative inputs.
Optimization. All models trained for 100 epochs using the Adam optimizer with learning rate of 1 10-3 and weight decay of 1 x 10-5, We use the standard Adam hyperparameters (8) 0.9, B3 0.999). Batches of size 64 are used for each training step (Table 7).
Remark: Importantly, apart from using the SELU activation function, we did not implement any other modifications. To ensure the comparison is as fair as possible to the comparison method, our method and the underlying network architecture, along with the corresponding hyperparameters, remain the same as recommended by (Xu, 2023).
22

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
Parameter
Value
GENERAL TRAINING
Batch size
64
Learning rate
le-3
Epochs
100
ARCHITECTURE
Feed Forward
hnputiim-100--0-mmmdddii
Embedding size
128
Activation function
SELU
OPTIMIZER
Optimizer
Adam
Momentum B1
0.9
Momentum $2
0.999
Weight decay
le-5
Table 7. Neuralnetwork and training setting of WSAD-DT
J. Scalability Test
WSAD-DT-1
runtime (logarithmic scale)
WSAD-DT
10
DeepSAD
10
DevNet
PreNet
10
ROSAS
10
FeaWAD
XBGOD
4k
8k
16k
64k 128k 256k 512k number datapoints
32k
GANAnomaly
(a) varying the number data points
(b) Legend
Figure 2. Analyzing the runtime performance for WSAD-DT and the competitors.
To assess the scalability of proposed method (WSAD-DT), we measure its runtime performance increasing number of data points and compare it against several baselines. Figure illustrates the average wall-clock time (in seconds) as function of dataset size for WSAD-DT and other competing methods.
Experimental Setup. We simulate large-scale synthetic datasets by primarily drawing normal samples from 16- dimensional Gaussian and injecting small fraction (5%- 10%% of anomalies sampled from a distinct distribution. We label these injected points as anomalies and combine them with the normal samples. This yields controlled dataset where the fraction of anomalies remains fixed while we grow the overall dataset size from 4,000 samples up to 512,000 samples. Each size three times with different random seeds, and we report the average wall-clock time for each approach. For this experiment, used the default setup described in Appendix with the exception that we fixed the number of epochs to 10 for all algorithms in this experiment.
Asymptotic Behavior. As illustrated in Figure 2(a), WSAD-DT exhibits runtime scaling similar to DeepSAD and other deep anomaly detection approaches as the number of samples increases. This behavior is expected in standard miii-bbtch--bart deep learning pipelines. Although WSAD-DT includes diversity penalty term, its computational overhead remains modest. Specifically, the diversity penalty is evaluated only on each mini- batch and further reduced by subsampling pairs of points, making its contribution to overall runtime negligible in practice. In contrast, the
23

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
ensemble method XGBOD (which uses KNN (Ramaswamy et al., 2000) and LOF (Breunig et al., 2000) base estimators) experiences higher computational overhead large datasets, consistent with observations. Meanwhile, WSAD-DT maintains relatively consistent runtime profile, particularly with mini-batch optimization.
Impact of Ensemble Size. In Fig. 2(b), WSAD-DT-1 refers WSAD-DT with a single ensemble, whereas WSAD-DT represents the default setting with five ensembles. The runtime does increase proportionally with the number of ensemble splits in WSAD-DT since each component model is trained distinct partition of unlabeled data. However, set the default number of ensembles to moderate value (c.g., M 5), striking practical balance between computational overhead and performance gains.
Overall, WSAD-DT demonstrates competitive runtime scalability, making it well-suited for large-scale anomaly detection tasks where both robust performance and manageable computational cost required.
K. Ablation study of light-tailed, heavy-tailed, and dual-tailed kernel
We conducted an ablation study by separately evaluating the light-tailed kernel, the heavy-tailed kernel, and their combination (dual-tailed kernel). We set the number ensembles to I for all different configurations (light-tailed kernel, heavy-tailed kernel, dual-tailed kernel). Our findings (Table 8) reveal that while each kernel on its own provides a decent level of performance, combining both kernels clearly improves the model's ability distinguish anomalies from normal points. Specifically, the dual-tailed kernel configuration yields higher AUC-ROC scores compared to either kernel used in isolation. This result underscores the complementary benefits of the two kernels, with the light-taiie fight-tailed kernel component enforcing tight clustering for in-class points and the heavy-tailed component preserving greater separation for out-of-class samples.
24

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
Table We present the AUC-ROC values for various configurations in WSAD-DTT. Specifically, we compare the AUC-ROC performance across three settings: single kernels (light-taile tailed kernel and heavy-tailed kernel) and dual-tailed kernel.
Dataset
WSAD-DT
Light-tailed kernel
Heavy-tailed kernel
Optdigits
0.9984 (1)
0.8875 (3)
0.9845 (2)
Lymphography
0.9961 (1)
0.9787 (2)
0.9593 (3)
Pendigits
0.9997 (1)
0.9477 (3)
0.9991 (2)
Vertebral
0.8827 (1)
0.8354 (2)
0.7778 (3)
Wdbc
1.0000 (1)
0.9979 (3)
0.9995 (2)
Cardiotocography
0.9347 (1)
0.8899 (2)
0.8894 (3)
Wpbe
0.6801 (1)
0.6724 (2)
0.5807 (3)
Stamps
0.9841 (1)
0.8769 (3)
0.9456 (2)
Satimage-2
0.9954 (1)
0.9811 (3)
0.9941 (2)
Spambase
0.9317 (1)
0.8842 (3)
0.9202 (2)
Thyroid
0.9911 (1)
0.9667 (3)
0.9889 (2)
Mnist
0.9820 (1)
0.8293 (3)
0.9696 (2)
Yeast
0.6566 (1)
0.6409 (2)
0.5854 (3)
Cardio
0.9908 (1)
0.9505 (3)
0.9741 (2)
Vowels
0.9824 (1)
0.8737 (3)
0.9629 (2)
Wine
1.0000 (1)
0.9923 (2)
0.9769 (3)
Magic.gamma
0.8974 (1)
0.8375 (3)
0.8649 (2)
lonosphere
0.9836 (1)
0.9510 (3)
0.9651 (2)
Glass
0.9794 (2)
0.9543 (3)
0.9964 (1)
Breastw
0.9828 (2)
0.9658 (3)
0.9844 (1)
Yelp
0.7811 (1)
0.7355 (2)
0.7189 (3)
Imdb
0.6882 (1)
0.6074 (3)
0.6228 (2)
MNIST-C-Fog
1.0000 (1)
1.0000 (1)
0.9998 (3)
MNISS--C-Camny-ddees
0.9999 (1)
0.9846 (3)
0.9989 (2)
MVTec-AD-Zipper
0.9280 (1)
0.8468 (3)
0.9140 (2)
MNIST-C-Stripe
1.0000 (1)
1.0000 (1)
1.0000 (1)
Skin
0.9998 (1)
0.9991 (2)
0.9987 (3)
Fraud
0.9485 (1)
0.9346 (3)
0.9416 (2)
Http
1.0000 (1)
1.0000 (1)
1.0000 (1)
Cover
0.9995 (1)
0.9896 (3)
0.9973 (2)
Shuttle
0.9935 (2)
0.9862 (3)
0.9961 (1)
AvgRank
1.10
2.55
2.13
p-values
NA
0.00001200
0.00015176(+)
25

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
L. Ablation study on the effect of different numbers of ensembles
Below, we present an ablation study evaluating the effect of varying the number ensemble models (#ensembles) in WSAD-DT from
Table reports AUC-ROC scores for selection of benchmark datasets. A single model (#1) 1) already achieves strong results, yet using #3 ensembles yields clear boost on most datasets. Increasing the ensemble size to #5 generally delivers the highest overall performance.
Key Observations
1. #1 vs. #3: Introducing ensembles (i.c., going from to 3) consistently improves robustness. The models benefit from diverse views of the unlabeled data, while all leveraging the same limited set of labeled anomalies.
2. #3 vs. #5: Although gains can be smaller after #3, using #5 often still improves AUC-ROC, suggesting further diversity helps in certain datasets with complex structures highly imbalanced anomalies.
3. Computational Cost vs. Accuracy: While larger ensembles typically perform better, they also introduce additional training overhead. practice, #3 or #5 often provide a good balance between computational cost and accuracy.
Overall, the results confirm that using an ensemble of models, each trained on distinct partition of the unlabeled data while sharing the same few labeled anomalies, significantly enhances performance. In practice, #5 ensembles often good default, balancing the improved robustness from ensemble diversity against the computational cost.
26

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
Table This table presents the AUC-ROC results of WSAD-DT under different number of ensembles (M).
Dataset
#5-Spilt
#1-Spilt
#3-Spilt
Optdigits
0.9996 (1) 0.9984 (3)
0.9995 (2)
Lymphography
1.0000 (1) 0.9961 (3)
1.0000 (1)
Pendigits
0.9998 (1)
0.9997 (2)
0.9997 (2)
Vertebral
0.9051 (1)
0.8827 (3)
0.8939 (2)
Wdbe
1.0000 (1)
1.0000 (1)
0.9990 (3)
Wpbe
0.6915 (2)
0.6801 (3)
0.69188 (1)
Stamps
0.9902 (1)
0.9841 (3)
0.9883 (2)
Satimage-2
0.9973 (1)
0.9954 (3)
0.9962 (2)
Spambase
0.9499 (1)
0.9317 (3)
0.9454 (2)
Thyroid
0.9960 (1)
0.9911 (3)
0.9951 (2)
Mnist
0.9883 (1)
0.9820 (3)
0.9867 (2)
Yeast
0.6545 (2)
0.6566 (1)
0.6512 (3)
Cardio
0.9908 (2)
0.9908 (2)
0.9919 (1)
Vowels
0.9736 (3)
0.9824 (1)
0.9778 (2)
Wine
1.0000 (1)
1.0000 (1)
1.0000 (1)
Magic-gamma
0.9166 (1)
0.8974 (3)
0.9150 (2)
Ionosphere
0.9779 (3)
0.9836 (1)
0.9792 (2)
Glass
0.9830 (1)
0.9794 (2)
0.9794 (2)
Breastw
0.9894 (1)
0.9828 (3)
0.9883 (2)
Yelp
0.8311 (1)
0.7811 (3)
0.8008 (2)
Imdb
0.7623 (1)
0.6882 (3)
0.7379 (2)
MNIST-C-Fog
1.0000 (1)
1.0000 (1)
1.0000 (1)
MNIST-C-canny-EE
0.9999 (1)
0.9999 (1)
0.9999 (1)
ATTTcc-AD-Zpper
0.9307 (2)
0.9280 (3)
0.9354 (1)
MNIST-C-Stripe
1.0000 (1)
1.0000 (1)
1.0000 (1)
Skin
0.9998 (1)
0.9998 (1)
0.9998 (1)
Fraud
0.9574 (1)
0.9485 (3)
0.9556 (2)
Http
1.0000 (1)
1.0000 (1)
1.0000 (1)
Cover
0.9996 (2)
0.9995 (3)
0.9997 (1)
Shuttle
0.9956 (1)
0.9935 (2)
0.9934 (3)
AVGRank
1.30
2.20
1.73
p-value
NA
0.00645607 (4)
0.00947055 (+)
M. Analyzing the performance under contamination in the training data
In real-world scenarios, portion of the unlabeled dataset may itself be contaminated with anomalies, making the weakly supervised setting more challenging.
Contamination Procedure. We begin by using only 5% of the total anomalies for weakly supervised training, chosen uniformly at random, leaving the remaining anomalies "unused." We then define a contamination fraction p% (relative to these unused anomalies) to artificially degrade the unlabeled set. Specifically, we inject p% of the leftover anomalies into the unlabeled pool, mislabeled as normal points. At the same time, we select the same number normal points from the unlabeled set and mislabeled them as anomalies. This symmetrical label swap increases both label noise and anomaly contamination in the unlabeled data, enabling a systematic evaluation of WSAD-DT under two configurations: WSAD-DT (ensemble size of 5) and WSAD-DT-1 (ensemble size of 1) across various levels of mislabeling and adversarial injection multiple experiments with p% ranging from 1% (small contamination) to 100%
(see Fig. 3i). Using this setup, (maximum contamination). Figure 3 shows the AUC-ROC curves of WSAD-DT and several reference methods over eight
27

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
representative datasets contamination increases. Across all levels of contamination, WSAD-DT (blue curve) demonstrates consistently higher comparable AUC-ROC values compared to other approaches. WSAD-DDT- with single ensemble also consistently achieves higher or comparable AUC-ROC values than other approaches, underscoring the advantages of the dual-tailed kernel and kernel- based regularization. Notably. its performance degrades more gracefully than baselines when the contamination rate exceeds 10%. These findings underline two key advantages of WSAD-DT:
1. Robustness Noisy Unlabeled Data. By maintaining dual-tailed separation, WSAD-DT can still isolate anomalies even when contamination is present in the data.
2. Effective Use of Sparse Labels. Sharing small number of labeled anomalies across ensemble splits remains advantageous. Even at higher contamination levels, each component of the ensemble learns to loighteiiht suspicious unlabeled instances using the limited but high-value labeled examples.
Overall, WSAD-DT's resilience contamination makes it well-suited for practical applications where unlabeled data may not be entirely clean, reinforcing the benefits of dual-tailed kernel formulation and ensemble-based training under weak supervision.
1,0
1.0
1.0
AUC-ROC
0.-
AUC-ROC
AUC-ROC
06
0.4
0.4
0.4
contamination
75
100
contamination %
100
contaminatoon
25
56
100
(a) Vowels
(b) Glass
Lymphography
0.8
0.8
AUDROC
AVC-RCC
0.8
066
06
04
34
0.0
0.0
contamination %
25 contamination % (e) Satimage
contamination %
(d) Thyroid
(I) Wine
WSAD-DT
1.0
WSAD-DT-1
0.8
0.8
DeepSAD
AUG-RCC
0.8
DevNet
PreNet
0.2
0.2
GanAnomaly
XGBOOST
0.0
0.0
FeaWad
60
75
100
25
70
100
Contemination
contamination
(g) Stamps
(h) WDBC
Legend
Figure 3. Analyzing the performance under contamination in the training data.
28

---

Weakly Supervised Anomaly Detection via Duil-Tailed Kernel
N. Ablation study for different numbers of labeled anomalies
Below, we examine how varying the fraction of labeled anomalies the training set impacts performance under weak supervision. While our main experiments generally use 5% labeled anomalies, real-world anomaly detection tasks may have even fewer labeled anomalies, or- orouh still "weakly" supervised-ppoentially higher fraction of anomaly labels.
N.1. Experimental Setup
We select multiple benchmark datasets from our main evaluation and train WSAD-DT using three different proportions of labeled anomalies in the training data: 1%, 5%, and 10%. All other settings (network architecture, batch size, mini-batch sampling, etc.) remain identical to the primary experiments in Section H. Specifically,
1% labeled anomalies: Extremely limited supervision, approaching the scarcity seen in purely unsupervised settings.
5 % labeled anomalies: Our default setting is moderately weakly supervised scenario.
10% labeled anomalies: We have more labeled anomalies, but still lack enough fully supervised data to provide better coverage of anomaly types.
We compare WSAD-DT's performance to other state-of-the-art methods (DeepSAD, DevNet, etc.) under these varying amounts of labeled anomalies.
N.2. Results and Observations
Tables 10 and 1l show the AUC-ROC results for two representative levels: 1% and 10% labeled anomalies. (Our main paper reports the case of 5%.)
Key Observations:
Sensitivity to Labeled Fraction. As expected, all methods tend to improve with more labeled anomalies, but WSAD- DT consistently outperforms or matches the best baseline at each fraction. Even at /% labeled anomalies, it achieves top performance on majority of datasets.
Strong Gains from Limited Labels. Moving from 1% to 10% labeled anomalies often yield large jump for WSAD-DT. This suggests that even modest increase in anomaly labels can substantially reduce false positives and sharpen the separation boundary, especially given the two-center and the dual-tailed kernel approach.
Conclusion. WSAD-DT exhibits robust performance across a broad range of labeled anomaly percentages. While having more labeled anomalies generally leads to more accurate and stable decision boundaries, our method consistently maintains a strong lead (or near-lead) even when extremely few (1% (%) labeled anomalies are available. This resiliency underscores the effectiveness of the dual-kernel framework and ensemble strategy in leveraging limited supervision for improved anomaly detection.
29

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
Table 10. This table presents the results all algorithms using the default parameters outlined the original paper and 1% of labeled
anomalies.
Dataset
WSAD-DT
DeepSAD
DevNet
FeaWAD
GANAnomaly
PreNet
ROSAS
XGBOD
Optdigits
0.9996 (3)
0.9773 (7)
0.9959 (4)
0,9839 (5)
0.6041 (8)
1.0000 (1)
1.0000 (1)
0.9817 (6)
Lymphography
1.0000 (1)
1.0000 (1)
0,9961 (4)
0.9787 (6)
0.8953 (7)
1.0000 (1)
0.9903 (5)
Pendigits
0.9998 (1)
0.9768 (5)
0.9682 (6)
0.7798 (8)
0.8361 (7)
0.9832 (4)
0.9997 (2)
0.9981 (3)
Vertebral
0.9051 (1)
0.7810 (5)
0.4974 (7)
0.6399 (6)
0.3618 (8)
0.9036 (2)
0.8377 (3)
0.8125 (4)
Wdbc
1.0000 (1)
0.9990 (3)
1.0000 (1)
0.9979 (4)
0.9830 (8)
0.9969 (6)
0,9933 (7)
0.9979 (4)
Wpbc
0.6915 (1)
0.6144 (4)
0.6403 (3)
0.5619 (6)
0.4635
0.6649 (2)
0.6135
Stamps
0.9902 (1)
0.9492 (4)
0,8909 (6)
0.8664 (7)
0.7529 (8)
0.9697 (2)
0.9544 (3)
0.901 (5)
Satimage-2 2
0.9973 (1)
0.9715 (5)
0.9562 (6)
0.9753 (4)
0.9783
0.9243 (8)
0,9526 (7)
0,9866 (2)
Spambase
0.8900 (3)
0.7591 (5)
0.9214 (2)
0.6103 (7)
0.7334 (6)
0.7933 (4)
0.9263 (1)
Thyroid
0.9960 (2)
0.9782 (4)
0.9667 (5)
0.8525 (7)
0.7795 (8)
0.9420 (6)
0.9961 (1)
0.9795 (3)
Mnist
0.9767 (1)
0.8268 (5)
0.7492 (7)
0.6498 (8)
0.8281 (4)
0.7907 (6)
0,8725 (3)
0.9759 (2)
Yeast
0.6024 (3)
0,5351 (7)
0.6081 1 (2)
0.6268 (1)
0,4884 (8)
0.5453 (6)
0,5678 (5)
0,5797 (4)
Cardio
0.9916 (1)
0.9796 (3)
0.9686 (4)
0.8559
0.9095 (6)
0,9052 (7)
0.9672 (5)
0.9837 (2)
Vowels
0.9736 (1)
0,9712 (2)
0.8839 (6)
0.7696 (8)
0.8231 (7)
0.9300 (5)
0.9653 (3)
0.9516 (4)
Wine
1.0000 (1)
1.0000 (1)
0.9954 (5)
0.9491 (6)
0.6836
1.0000 (1)
1,0000
Magic-gamma
0.8853 (2)
0.8056 (5)
0.8304 (4)
0.7818 (6)
0.6784 (8)
0.7257 (7)
0.8504 (3)
0.8930 (1)
Ionosphere
0.9779 (1)
0.9704 (3)
0.5232 (7)
0.4238 (8)
0,6880 (6)
0.71 (5)
0.7656 (4)
0.9746 (2)
Glass
0.9830 (1)
0.9355 (3)
0.8199 (6)
0.6891 (8)
0.7106 (7)
0.8934 (5)
0.8952 (4)
0.9606 (2)
Breastw
0.9875 (2)
0.9398 (5)
0.9947 (1)
0.9555 (4)
0.8872 (6)
0.7631 (7)
0.9859 (3)
Yelp
0.7283 (3)
0.7310 (2)
0.5676 (6)
0.5491 (7)
0.6691
0.4998 (8)
0.7440 (1)
0.6888
Imdb
0.6809 (2)
0.5555 (6)
0,5502 (7)
0.5693 (5)
0.5042 (8)
0,5880 (4)
0.6810 (1)
0.6399 (3)
MNIST-C-Fog
1.0000 (1)
0.9901 (6)
0.9999 (4)
0.9327 (7)
0.8011
1.0000 (1)
1.0000 (1)
0.9968 (5)
MNIST-C-canny-FBae
0.9977 (2)
0,9578 (6)
0.9649 (5)
0.7623 (7)
0.6864 (8)
0.9667 (4)
0.9986 (1)
0.9846 (3)
-VTTcc-AD-Zipper
0.9307 (1)
0.8626 (3)
0.7728 (6)
0.5132 (8)
0.7402 (7)
0.7815 (5)
0.8993 (2)
0.8455 (4)
MNIST-C-Stripp
1.0000 (1)
1.0000 (1)
1.0000 (1)
0.9159 (8)
0.9743 (7)
1,0000 (1)
1.0000 (1)
0.9998 (6)
Http
1.0000 (1)
1.0000 (1)
0.9981 (4)
0.5595 (6)
0.9980 (5)
0.9997 (3)
Skin
0.9998 (1)
0,9995 (2)
0.9957 (5)
0.4688
0.9993 (3)
0.9990 (4)
Fraud
0.9526 (1)
0.9228 (5)
0,9136 (6)
0.7745 (7)
0.9281 (4)
0.9478 (2)
0.9451 (3)
Cover
0.9994 (1)
0.9930 (5)
0.9963 (3)
0.9102 (6)
0.5970 (7)
0.9992 (2)
0.9935 (4)
Shuttle
0.9952 (1)
0.9866 (3)
0,9789 (5)
0.9383 (6)
0.9068 (7)
f
0.9855 (4)
0.9922 (2)
AVGRank
1.43
3.90
4.60
6.57
6.70
4.90
3.63
3.33
p-value
NA
0.0020600 (+)
000146543 (-)
0.00300046 (+)
000000005 (+)
0.00001700
0000166441
0.001 (001-46833..
Values marked with i indicate that result available within 12 hours.
Values marked with indicate that runtime error occurred during execution.
30

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
Table 11. This table presents the results of all algorithms using the default parameters outlined the original paper and 10 % of labeled
anomalies.
Dataset
WSAD-DT
DeepSAD
DevNet
FeaWAD
GANAnomaly
PreNet
ROSAS
XGBOOST
Optdigits
0.9999 (3)
0.9948 (6) 0.9997 (4)
0.9983 (5)
0.6044 (8)
1.0000 (1)
1.0000 (I)
0.9931 (7)
Lymphography
1.0000 (1)
1.0000 (1)
0.9961 (4)
0.9787 (6)
0.8953 (7)
1.0000 (1)
0.9903 (5)
Pendigits
0.9998 (1)
0.9931 (4)
0.9889 (5)
0.9841 (6)
0.8323 (8)
0.9800 (7)
0.9993 (2)
0.9971 (3)
Vertebral
0.9051 (1)
0,7810 (5)
0.4974 (7)
0.6399 (6)
0.3618 (8)
0.9036 (2)
0.8377 (3)
0.8125 (4)
Wdbc
1.0000 (1)
0,9990 (3) 1.0000 (1)
0,9979 (4)
0.9830 (8)
0.9969 (6)
0,9933 (7)
0.9979 (4)
Wpbc
0.6915 (1)
0.6144 (4)
0.6403 (3)
0.5619 (6)
0.4635 (7)
0.6649 (2)
0.6135 (5)
Stamps
0.9902 (1)
0,9492 (4)
0.8909 (6)
0,8664 (7)
0.7529 (8)
0,9697 (2)
0,9544 (3)
0.9011 (5)
Satimage-2 2
0.9973 (1)
0,9715 (5)
0.9562 (6)
0.9753 (4)
0.9783 (3)
0.9243 (8)
0.9526 (7)
0.9866 (2)
Spambase
0,9615 (2)
0,9073 (5)
0.9310 (4)
0.6103 (7)
0.8781 (6)
0,9412 (3)
0.9686 (1)
Thyroid
0.9957 (2)
0.9705 (5)
0.9753 (4)
0,9449 (6)
0.7761 (8)
0.9407 (7)
0.9966 (1)
0.9843 (3)
Mnist
0.9921 (1)
0,9511 (4)
0.9425 (5)
0.8121 (8)
0.8284 (7)
0.9368 (6)
0.9703 (3)
0,9914 (2)
Yeast
0.6819 (1)
0.6673 (2)
0,5805 (6)
0,5260 (7)
0,4885 (8)
0.6633 (3)
0.6593 (4)
0.6471 (5)
Cardio
0.9945 (2)
0,9869 (3)
0.9830 (4)
0.7676
0.9088 (7)
0.9428 (6)
0.9743
0.9967 (1)
Vowels
0.9736 (1)
0.9712 (2)
0.8839 (6)
0.7696 (8)
0.8231 (7)
0.9300 (5)
0,9653 (3)
0.9516 (4)
Wine
1.0000 (1)
1.0000 (I)
0.9954 (5)
0.9491 (6)
0.6836 (7)
1,0000 (1)
1,0000 (1)
Magic-gamne
0.9187 (2)
0.8919 (4)
0.8323 (6)
0.7932 (7)
0.6680 (8)
0.8594 (5)
0.9141 (3)
0.9203 (1)
lonosphere
0.9769 (1)
0.9693 (2)
0.3976 (8)
0,6145 (7)
0.6887 (6)
0.8720 (4)
0,8587 (5)
0,9605 (3)
Glass
0.9830 (1)
0.9355 (3)
0,8199 (6)
0.6891 (8)
0.7106 (7)
0.8934 (5)
0.8952 (4)
0.9606 (2)
Breastw
0.9876 (3)
0.9670 (4)
0.9943 (1)
0.9549 (5)
0.9388 (6)
0.8695 (7)
0.9925 (2)
Yelp
0,8671 (1)
0.7777 (3)
0.7061 (4)
0.5846 (8)
0.6691 (6)
0.6491 (7)
0.6888
0,8351 (2)
Imdb
0.8140 (2)
0.8094 (3)
0.6801 (5)
0,5792 (7)
0.5021 (8)
0.6128 (6)
0,8230 (1)
0,7616 (4)
MNIST-C-Fog
1,0000 (1)
1.0000 (1) 1,0000 (1)
0.6645 (8)
0.8040 (7)
I. 1,0000 (1)
1.0000 (1)
1,0000 (1)
MNIST-C-canny Caanny--Ede
0.9999 (1)
0.9946 (6)
0.9980 (5)
0.7528 (7)
0.6962 (8)
0.9995 (3)
0.9998 (2)
0.9988 (4)
MVTec-AD AD--pperr
0.9278 (1)
0.8923 (4)
0.7327 (6)
0.5681 (8)
0.7385 (5)
0.7313 (7)
0.8945 (3)
0.8964 (2)
MNIST-C-Stripe
1.0000 (1)
1.0000 (1) 1.0000 (1)
0.8470
0.9738 (7)
1.0000 (1)
1.0000
1,0000
Http
1.0000 (1)
1.0000 (1)
0,9984 (4)
0.5482 (6)
0,9980 (5)
0.9992 (3)
Cover
0.9997 (2)
0.9976 (4)
0.9990 (3)
0.4632 (6)
0.9998 (1)
0.9968 (5)
Skin
0.9998 (1)
0.9995 (3)
0.9949 (5)
0.5026 (6)
0.9993 (4)
0.9996 (2)
Fraud
0.9566 (3)
0.9337 (5)
0,9342 (4)
0,8545 (7)
0.9181 (6)
0,9641 (2)
0.9680 (1)
Shuttle
0.9949 (3)
0.9956 (2)
0.9779 (5)
0.9044 (7)
0.9714 (6)
f
0.9849 (4)
0.9995 (1)
AVGRank
1.47
3.33
4.47
6.87
6.83
4.83
3.80
2.87
p-valuc
NA
000001722244
000009962
000000005 (+)
0.00000005
00000009900 (+)
0,80175194 (-)
0.02546843 (+)
Values marked with f indicate that result was available within 12 hours. Values marked with indicate that a runtime error occurred during execution.
O. Parameter Sensitivity
In WSAD-DT, each of the two class centers (normal and anomalous) is associated with a light-tailed kernel that enforces tighter clustering around that center. We allow each center's i light-tailed kernel to have its own bandwidth parameter, enabling the normal and anomalous clusters to have different degrees of "compactness." Both centers have the same parameter y for the single heavy-tailed kernel to model out-of-class similarity, ensuring a broader margin is maintained for any point that does not align with its assigned center. Similarly, for the diversity term (which penalizes excessive clustering), we use class-specific bandwidth parameters that can adjust how strongly intra-class distances are regulated. We conduct detailed ablation to analyze the effect of varying these parameters, showing that while the method is robust across a reasonable range of values, fine-tuning can further improve performance in different data regimes.
Experimental Setup.
Normal Kernel Bandwidth: We vary the bandwidth on of the light-tailed kernel (assigned to the normal center) within (0.1, 0.25, 0.5, 0.75J.
Anomaly Kernel Bandwidth: We similarly vary a the bandwidth of the anomalous center's light-tailed kernel, within (0.25, 0.5, 0.75, 1.03.
31

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
Heavy-Tailed Kernel (Tail Parameter v): We adjust the degrees-of-freedon parameter of the heavy-tailed kernel over {0.2, 0.5, 0.75, 1.0).
Diversity Bandwidth: Finally, we vary the bandwidth (a Ms, All used in the exponential term of the diversity loss in {0.1, 0.5, 1.0, 2.0}.
All other hyperparameters (c.g., network architecture, batch size, and ensemble splits) are kept at their default settings.
Results and Analysis. Figure 4 illustrates the impact of changing each parameter on AUC-ROC for five example datasets. We highlight several observations:
Light-tailed kernels (Normal vs. Anomaly Center). Although performance does change with different bandwidth values, the method remains generally robust for a E [0.2, 1.0]. For normal data smaller a often yields higher accuracy by enforcing tighter, more coherent cluster; however, using a larger a can make the model overly permissive and degrade performance. By contrast, for anomalous data, a relatively larger a tends to improve performance by accommodating the greater heterogeneity of anomalies, whereas a too-small a risks overly constraining the anomalous region and thus degrading results.
Hcavy-Tailed Kernel (Tail Parameter v). We find that values of v around 0.2-1.0 often offer strong performance, preserving sufficient separation for out-of-class points.
Diversity Term (ddiverstty For most datasets, diversity [0.5, 1.0) provides a balanced penalty that prevents collapse while still allowing appropriate clustering within each class. Extremely small values (e.g.. 0.1) risk over-penalizing points that are naturally close, while very large values reduce the diversity penalty, making the model more susceptible to trivial collapse in extreme cases.
Practical Recommendations. Overall, WSAD-DT is not highly sensitive to modest changes in these parameters; even when they are set sub-optimally, the method continues to outperform or stay on par with strong baselines. We recommend the following default ranges, based on empirical results:
du. F 0.5 and 1.0.
V (heavy-tailed) the range {0.2, 1.0].
diversity 24 R 0.5 and di ~ 1.0.
In practice, a quick grid random search within these ranges often suffices to achieve robust performance without extensive tuning.
32

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
1.00
ROU-AUC
ROU-AUC
0.50
0.50
0.25
0.25
0.26
0.00
0.00
0.00
1.00
0.76
Odverity.4
favertty..
(a) ddiversity. 24
(b) diversity... ..A
(c) The
Vowels
0.75
0.75
Glass
ROU-AUG
ROU-AUD
Satimage
0.50
0.50
Wine
0.25
0.25
Slamps
WDBC
Thyroid
0.25
0.75
1,00
0.20
0.50
0.75
Lymphography
(d) 0,A
(e)
(1) Legend
Figure 4. Analyzing the performance of WSAD-DT under different hyperparameters.
P. Ablation study kernel regularization
As established in Appendix D, relying solely on the separation loss can lead to degeneracies where samples of each class collapse to a single latent point. To mitigate this, WSAD-DT includes kernel-based regularization term-the diversity which penalizes over-concentraatin and thus preserves intra-class variability. In this ablation study, we assess how omitting this diversity term affects performance.
Experimental Setup. We compare WSAD-DT with and without the diversity term, keeping all other hyperparameters unchanged. For both settings, we set the ensemble size of WSAD-DT to Specifically, No Regularization removes the exponential pairwise penalty cxp(-Ilfo(zs) fo(x;))1 /o? from the total loss. We evaluate both variants on multiple real-world datasets under the same weakly supervised setting described in Section H
Results. Table 12 reports AUC-ROC scores with ("WSAD-DT"" and without ("No Regularization Term") the diversity penalty. In most datasets, WSAD-DT achieves higher AUC-ROC than its unregularized counterpart. Although in few cases, the version without regularization narrowly outperforms the regularized one, most datasets strongly benefit from including the diversity term. Specifically: Without regularization, the model can map each entire class to a single point in latent space, artificially minimizing the separation loss (Appendix D). This often degrades generalization to unseen anomalies (Goyal al., 2020). Overall, these results confirm the theoretical and empirical rationale for using kernel- based regularization: while datasets may not strictly require it to avoid degenerate solutions, significantly increases robustness and yields better or comparable AUC-ROC average.
33

---

Weakly Supervised Anomaly Detection via Dual-Tailed Kernel
Table 12. This table presents the AUC-ROC results of WSAD-DT with and without diversity
Dataset
WSAD-DT
No regularization term
Optdigits
0.9984 (1)
0.9864 (2)
Lymphography
0.9961 (2)
0.9981 (1)
Pendigits
0.9997 (1)
0.9768 (2)
Vertebral
0.8827 (2)
0.8836 (1)
Wdbe
1.0000 (1)
0.9928 (2)
Cardiotocography
0.9347 (1)
0.8839 (2)
Wpbc
0.6801 (2)
0.6827 (1)
Stamps
0.9841 (1)
0.9765 (2)
Satimage-2
0.9954 (1)
0.9898 (2)
Spambase
0.9317 (1)
0.9053 (2)
Thyroid
0.9911 (1)
0.9732 (2)
Mnist
0.9820 (1)
0.9249 (2)
Yeast
0.6566 (1)
0.6413 (2)
Cardio
0.9908 (1)
0.9514 (2)
Vowels
0.9824 (1)
0.9670 (2)
Wine
1.0000 (1)
1.0000 (1)
Magic.gamne
0.8974 (1)
0.8974 (1)
Ionosphere
0.9836 (1)
0.9674 (2)
Glass
0.9794 (2)
0.9919 (1)
Breastw
0.9828 (1)
0.9758 (2)
Yelp
0.7811 (1)
0.6415 (2)
Imdb
0.6882 (1)
0.6262 (2)
MNIST-C-Fog
1.0000 (1)
1.0000 (1)
MNIST-C-canny-E-ae
0.9999 (1)
0.9873 (2)
MVTec-AD-Zipper
0.9280 (1)
0.8585 (2)
MNIST-C-Stripe
1.0000 (1)
1.0000 (1)
Skin
0.9998 (1)
0.9992 (2)
Fraud
0.9485 (1)
0.9064 (2)
Http
1.0000 (1)
0.9985 (2)
Cover
0.9995 (1)
0.9752 (2)
Shuttle
0.9935 (2)
0.9956 (1)
AvgRank
1.16
1.71
p-value
NA
0.000127
Q. Limitations
While WSAD-DT generally performs well under weak supervision, several scenarios can pose significant challenges. One issue arises when anomalics are poorly separated from normal instances, creating fuzzy boundary that can require additional supervision or more refined feature engineering cases of extreme overlap. Another complication occurs if anomalies are highly diverse, markedly different clusters with shared center, single anomaly center may underfit, indicating that multi-center multi-modal methods could be more appropriate. Finally, although WSAD-DT addresses extremely limited anomaly labels, there may be domains with such scarcity of labeled anomalies that even the dual-tailed kernel and diversity term are not sufficient for reliable separation. Nevertheless, our experiments suggest that WSAD-DT remains robust under most practical conditions, balancing in-class compactness with broad margins and leveraging sparse anomaly labels through an ensemble mechanism.
employ a single center for anomaly representation. If anomalies originate from multiple,
34


Reviews:
[{"review_id": "eBUAQRx09O", "reviewer": "Reviewer_mzRc", "summary": "The paper proposes WSAD-DT, a weakly supervised anomaly detection framework that employs dual-tailed kernels (light-tailed for in-class compactness, heavy-tailed for out-of-class separation) and an ensemble strategy to address limited labeled anomalies. Empirical results on AdBench datasets show state-of-the-art performance compared to methods like DeepSAD and XGBOD.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the valuable feedback. \n\n**Ablation stay extreme case(e.g., 0.1%) **\n\nWe have conducted additional experiments with a 0.1\\% fraction of labeled anomalies. Table https://anonymous.4open.science/r/weakly_anomaly_detection/Table1.png summarizes  AUC-ROC results under this extreme scenario.  Despite having only 0.1\\% labeled anomalies, WSAD-DT achieves state-of-the-art or near-best performance on most datasets. Notably, even with such sparse labels, dual-tailed kernels and our ensemble design still effectively leverage the limited anomaly examples to separate normal vs.\\ abnormal data. \n\n**Q1**\n\nThe “well-separated” assumption in Theorem 5.3 is a common theoretical device in margin-based analyses, akin to the strict separability often assumed in classical SVM proofs. Although real data typically feature some overlap between anomalies and normal samples, this idealized scenario highlights a core theoretical insight: having distinct margins reveals how dual-tailed kernels outperform single-kernel designs by simultaneously promoting in-class compactness (via the light-tailed kernel) and sustaining a long-range “push” for out-of-class points (via the heavy-tailed kernel). In practice, WSAD-DT remains effective when anomalies are only partially separable because the heavy-tailed kernel preserves nontrivial gradients even at moderate distances, preventing anomalies close to the normal center from collapsing into negligible similarity. Meanwhile, the light-tailed kernel keeps each class tightly clustered, and the diversity term further prevents degenerate collapsing by penalizing over-concentration. Moreover, our experiments across different weak-supervision scenarios—1\\%, 5\\%, or 10\\% labeled anomalies, sometimes only five anomalies total—show that WSAD-DT remains effective under label scarcity. As demonstrated in our contamination experiments (Appendix M), performance degrades only mildly when unlabeled data contain anomalies, confirming that while Theorem 5.3 holds under clean-margin assumptions, WSAD-DT’s dual-tailed design still excels under real-world overlap.\n\n\n**Q2**\n\nWe focus on light‐ vs. heavy‐tails, not specific forms. Any light‐ (e.g., Laplacian) or heavy‐tail (e.g., Cauchy) kernel is valid, as long as it preserves the fast‐ vs. slow‐decay principle.  In Appendix O, we vary kernel parameters to test different tail decays and observe stable performance. \n\n**Q3**\n\nWSAD-DT is primarily designed for static, batch-oriented anomaly detection and does not explicitly account for streaming data or concept drift, similar to methods like DeepSAD, DevNet, or XGBOD. Nonetheless, it can be adapted by updating the normal and anomalous centers as new data arrive and retraining periodically to accommodate changing patterns. The ensemble structure also naturally extends to dynamic contexts by allowing older models to be replaced with ones trained on more recent data. This preserves the dual-tailed kernel advantage while helping WSAD-DT remain effective in environments, such as fraud detection, where data distributions evolve over time.\n\n\n**Q4**\n\n Classical kernel-based anomaly detection methods, such as One-Class SVM, rely on a single kernel applied uniformly across the data [3]. In contrast, multi-kernel and multi-view approaches blend multiple kernels—often through linear combinations—to capture richer similarity structures [2]. For example, Zhao and Fu introduce a dual-regularized framework that factors multi-view data into cluster indicators and sample-specific errors [1]. However, these methods still learn a single, blended kernel function that is applied globally to all points. By contrast, our approach conditionally assigns two distinct kernels to each data point: a light-tailed kernel for in-class distances and a heavy-tailed kernel for out-of-class distances. This explicitly changes tail behavior based on whether a point is close to the normal or anomaly center—rather than learning a single global kernel mixture. As we demonstrate in Section 5, this dual-tailed design achieves tighter in-class clustering and broader out-of-class separation—two conflicting goals that a single kernel or linear blend cannot simultaneously satisfy.\n\n**Typos**\n\nWe thank the reviewer for pointing out the typos, and we will correct them in the revised manuscript.\n\n[1] Zhao, Yue, and Yun Fu. \"Dual-Regularized Multi-View Outlier Detection.\"\n\n[2] Gönen, Mehmet, and Ethem Alpaydın. \"Multiple Kernel Learning Algorithms.\" \n\n[3] Schölkopf, Bernhard, et al. \"Estimating the Support of a High-Dimensional Distribution.\"", "claims_and_evidence": "Supported Claims: The dual-tailed kernel’s effectiveness is supported by ablation studies (Appendix K) and theoretical proofs (Lemmas 5.1–5.3). The ensemble strategy’s benefits are validated via experiments (Table 9).\n\nProblematic Claims: The claim that \"no single kernel can satisfy both compactness and separation\" (Theorem 5.3) relies on assumptions (e.g., \"sufficient model capacity\") that are not empirically verified for all scenarios.", "methods_and_evaluation_criteria": "The dual-tailed kernel design is intuitive and aligns with margin-based theory. The use of AdBench datasets is appropriate, but experiments on temporal/graph data are missing, limiting scope validation. Evaluation metrics (AUC-ROC/PR) are standard, but statistical significance tests (Wilcoxon) strengthen reliability.", "theoretical_claims": "Lemmas 5.1 and 5.2 are correctly derived, but Theorem 5.3’s proof assumes ideal conditions (e.g., \"well-separated data\"), which may not hold in practice. The collapse prevention via diversity loss (Lemma 5.4) is valid but lacks empirical validation in high-dimensional spaces.", "experimental_designs_or_analyses": "Experiments are thorough but lack scalability tests on very large datasets (e.g., >1M samples). The contamination study (Appendix M) is a strength, but labeled anomaly fractions (1%–10%) could be extended to extremes (e.g., 0.1%).", "relation_to_broader_scientific_literature": "The work builds on margin-based theory and extends DeepSAD/DevNet by decoupling in-class and out-of-class similarity. The dual-kernel idea is novel but could relate to multi-kernel learning.", "essential_references_not_discussed": "I don't have specific knowledge of the area in question, but I currently think it's adequate", "other_strengths_and_weaknesses": "Strengths: Novel dual-kernel design, robust ensemble strategy, and comprehensive experiments.\n\nWeaknesses: Limited discussion on computational overhead for large ensembles and no exploration of graph/temporal data.", "other_comments_or_suggestions": "Typos: Page 3, \"reserves a broader margin\" → \"preserves\"; Page 5, \"In-based separation terms\" → \"In-class\".", "questions_for_authors": "Theorem 5.3: How does the method perform if anomalies are not well-separated (violating the assumption)? Would this invalidate the theorem?\n\nWere other kernels (e.g., Laplacian, polynomial) tested? If not, why?\n\nHow does WSAD-DT handle streaming data or concept drift in applications like fraud detection?\n\nSince I don't know the relevant literature, I will look carefully at other reviewers' comments to adjust the score", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cmBrWPsLXl", "reviewer": "Reviewer_WTby", "summary": "This paper proposes a weakly supervised anomaly detector WSAD-DT via a dual-tailed kernel that can clearly distinguish anomalies from normal samples under weak supervision. Moreover, an ensemble strategy was devised to divide the unlabeled data into distinct subsets. Meanwhile, the limited labeled anomalies were shared across these partitions to optimize their influence.\n\n## update after rebuttal\nMost of my concerns were addressed, so I changed my score.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for their perceptive review.\n\n\n\n**Q1**\n\nWe adopt the 70/30 train/test split protocol following the approach used by AdBench (Han et al., 2022) [1], and this does not contradict the principle of weak supervision. What determines the “weakness” here is not how much of the dataset is allocated to training, but rather how few anomalies are actually labeled among that training portion. Even though 70\\% of the data is used for training, only a small fraction of those anomalies—sometimes just five  labeled anomalies—is available for explicit supervision. The rest of the training set remains unlabeled. This imbalance in labeled anomalies, rather than the overall training size, is what makes the setting truly weakly supervised and differentiates it from the kind of fully supervised approach that would require extensive or comprehensive anomaly labeling.\n\n**Q2**\n\nWe set the bandwidth parameters to reflect the distinct behaviors of normal and anomalous data, using a smaller value for the normal center to enforce tighter in-class clustering and a larger value for the anomaly center to accommodate its more varied distribution. These parameters essentially control how rapidly similarity decays with distance for the light-tailed (Gaussian) and heavy-tailed (Student-\\(t\\)) kernels, respectively. A Gaussian kernel with a smaller bandwidth enforces compactness for normal samples, whereas a slightly larger bandwidth for anomalies prevents overly constraining their more diverse representations. Likewise, the Student-\\(t\\) kernel parameter determines how “heavy” the tail is, maintaining a broader margin for out-of-class points. We provide an ablation study showing that the method remains robust across reasonable ranges [0.1-1] of these bandwidth and tail parameters (Appendix O).\n\n\n**Q3**\n\nWe performed an additional set of experiments comparing WSAD-DT with IDK (using the authors’ code from https://github.com/IsolationKernel/Codes) and Isolation Forest (IForest) [2] (code from https://github.com/yzhao062/pyod), adopting each method’s default parameter settings.  In Table https://anonymous.4open.science/r/weakly_anomaly_detection/Table2.png, WSAD-DT consistently outperforms IDK  and IForest in AUC-ROC across nearly all datasets, often by a substantial margin. Although isolation-based methods are well-known for their speed and simplicity, these results suggest that incorporating a small set of labeled anomalies, along with a dual-tailed kernel and diversity regularization, yields more accurate detection. Even on large or high-dimensional datasets, WSAD-DT achieves better separation between normal and outlying samples than both IDK and IForest. These findings are consistent with [1], where even minimal supervision—such as having only five labeled anomalies—can substantially surpass purely unsupervised methods.\n\n\n**Q4**\n\nWe conducted an additional experiment where we replaced the dual-tailed setup (Gaussian for in-class + Student-\\(t\\) for out-of-class) with a single Gaussian kernel using two different bandwidths—one smaller for in-class (e.g., $\\sigma=0.1$) and one larger for out-of-class (e.g., $\\sigma=2$).  To isolate the effect of the kernel design and the diversity term, we fix the ensemble size to 1 in both setting.  Table https://anonymous.4open.science/r/weakly_anomaly_detection/Table3.png shows that while using two distinct Gaussian bandwidths can perform reasonably well, the full dual-tailed configuration (Gaussian + Student-\\(t\\)) generally achieves stronger AUC-PR score. Larger $\\sigma$ in a Gaussian slows its decay somewhat, but that decay remains fundamentally exponential, whereas a Student-\\(t\\) kernel falls off more gradually and does not saturate as quickly at moderate distances. This long-tail behavior preserves a stronger “push” for out-of-class points, leading to a broader margin and, ultimately, more robust anomaly separation.\nThis suggests that having a genuinely heavy tail is important for pushing out-of-class points farther away and not just a matter of scaling the same kernel. In practice, a true heavy-tailed kernel—like Student-\\(t\\)—better preserves moderate similarities at larger distances, thereby preventing early saturation and improving separation of anomalous instances.\n\n\n[1] Han, S., Hu, X., Huang, H., Jiang, M., & Zhao, Y. (2022). Adbench: Anomaly detection benchmark. Advances in neural information processing systems, 35, 32142-32159.\n\n[2] Liu, F. T., Ting, K. M., & Zhou, Z. H. (2008, December). Isolation forest. In 2008 eighth ieee international conference on data mining (pp. 413-422). IEEE.", "claims_and_evidence": "yes", "methods_and_evaluation_criteria": "yes", "theoretical_claims": "no", "experimental_designs_or_analyses": "no", "relation_to_broader_scientific_literature": "This paper proposes a weakly supervised anomaly detector with better performance.", "essential_references_not_discussed": "no", "other_strengths_and_weaknesses": "The paper is clearly described and provides some theoretical proofs.", "other_comments_or_suggestions": "see questions.", "questions_for_authors": "1. Is it weakly supervised if 70% of the data in the dataset is used for training? Although there are few anomalies used, there are few anomalies in the dataset. Even if all the data is used for training, there are still a few anomalies.\n\n2. How are the parameters alpha and beta set and what is their impact on the model?\n\n3. There is a lack of comparison with some SOTA unsupervised algorithms, such as IDK[1], isolation-based algorithms are very famous in anomaly detection, and most of them are linear algorithms.\n\n4. Do the light and heavy kernels necessarily have to adhere to the specific forms outlined in the paper? Would it be acceptable to employ a Gaussian kernel for both, with distinct σ values?\n\nI am willing to improve my score if my concerns are addressed.\n\n[1] Ting, K. M., Xu, B. C., Washio, T., & Zhou, Z. H. (2020, August). Isolation distributional kernel: A new tool for kernel-based anomaly detection. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining (pp. 198-206).", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ayfh3WtyI6", "reviewer": "Reviewer_HgXU", "summary": "This paper proposes a method to improve anomaly detection performance by utilizing two kernel functions in the weakly supervised anomaly detection problem, where only unlabeled data and a small amount of labeled anomaly data are available. The effectiveness of the proposed method is validated through experiments using various datasets.\n\n## update after rebuttal\nI raises the score to Weak Accept.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the helpful feedback.\n\n**Q1**\n\nFrom a theoretical standpoint, WSAD-DT applies margin-based reasoning to handle both partial contamination in the unlabeled set and extremely limited anomaly labels. Assigning each class its own center and using a dual-tailed kernel—light-tailed for in-class compactness, heavy-tailed for out-of-class margins—maintains separation even when anomalies are scarce or intermixed with unlabeled data. The heavy-tailed component preserves a “push” at moderate distances, which is crucial under weak supervision where a handful of labeled anomalies must guide the entire boundary. Meanwhile, the diversity term prevents degenerate “collapse” solutions, ensuring normal and anomalous samples maintain sufficient variability. The ensemble mechanism  gives each ensemble model a distinct view of normality while sharing the same anomaly labels ensures consistent guidance. Aggregating these diverse detectors improves robustness and generaliza- tion under limited anomaly labels.  Furthermore, our experiments under different weak-supervision setups—1%, 5%, and 10% labeled anomalies, sometimes as few as five labeled anomalies total—demonstrate that WSAD-DT remains effective even in extremely label-scarce conditions. As shown in our contamination experiments (Appendix M), the method’s performance degrades only slightly when unlabeled data are partially contaminated, underscoring both its theoretical and practical robustness in real-world weakly supervised scenarios.\n\n**Q2**\n\nWe compared WSAD-DT against SOEL (using the authors’ code from  https://github.com/aodongli/Active-SOEL), which explicitly handles anomalies in the unlabeled set, using each approach’s default configurations. Table https://anonymous.4open.science/r/weakly_anomaly_detection/Table4.png  shows that WSAD-DT generally achieves higher AUC-ROC on most datasets, occasionally by a wide margin. These results indicate that while SOEL is effective at identifying unlabeled anomalies, the dual‐tailed separation and diversity regularization in WSAD-DT provide a stronger results  in weakly supervised scenarios.\n\n\n**Q3**\n\nTo isolate the effect of the kernel design and the diversity term, we fix the ensemble size to 1 in both the full WSAD-DT and a simplified variant that uses only a single Gaussian kernel with no diversity regularization.\nAs shown in  Table https://anonymous.4open.science/r/weakly_anomaly_detection/Table5.png, we compare the full WSAD-DT (dual-tailed + diversity) to a simplified version using only a single Gaussian kernel without the diversity term. Across most datasets, WSAD-DT achieves higher AUC-ROC. Empirically, this suggests that relying on a single kernel alone and omitting the diversity regularization is insufficient for robust anomaly detection under weak supervision. In contrast, combining light- and heavy-tailed kernels with a diversity penalty  improves stability and overall performance. \n\n**Q4**\n\nWe appreciate the reviewer’s suggestion regarding the use of more realistic datasets, such as MedicalMNIST. In response, we have extended our experiments to include this dataset, applying the commonly used \"one-vs.-rest\" evaluation protocol. This protocol treats each class in turn as normal while randomly selecting samples from other classes as anomalies, following the standard practice for anomaly detection tasks on multi-class datasets. Our experiments on the MedicalMNIST dataset have yielded strong results, demonstrating the robustness and scalability of our method in more complex, real-world scenarios. As shown in the Table https://anonymous.4open.science/r/weakly_anomaly_detection/Table6.jpg, we compare the performance of our method (WSAD-DT) against several competitive methods across various datasets in the MedicalMNIST suite. For example, on the  Derma dataset, WSAD-DT achieved the highest AUC-ROC score of 0.9002, outperforming methods such as DeepSAD, DevNet, and XGBOD. Across several datasets, our method consistently ranks among the top performers, reinforcing its effectiveness in handling medical data with weak supervision. We believe these results demonstrate that our method is not only effective on tabular  datasets but also performs well in more complex, real-world applications like medical anomaly detection.", "claims_and_evidence": "The claims seem clear, but I am unsure why the method is effective for weakly supervised learning. Please refer to the Question section.", "methods_and_evaluation_criteria": "The proposed method appears reasonable to some extent. However, it would be beneficial to evaluate it on more realistic datasets. Please refer to the Question section.", "theoretical_claims": "The reason why the proposed method is effective for weakly supervised learning is unclear.", "experimental_designs_or_analyses": "An ablation study is necessary. Please refer to the Question section.", "relation_to_broader_scientific_literature": "This paper should cite and compare their method with LOE [1] and SOEL [2]. Please refer to the Question section.\n\n[1] Qiu, Chen, et al. \"Latent outlier exposure for anomaly detection with contaminated data.\" International Conference on Machine Learning. PMLR, 2022.\n\n[2] Li, Aodong, et al. \"Deep anomaly detection under labeling budget constraints.\" International Conference on Machine Learning. PMLR, 2023.", "essential_references_not_discussed": "Please refer to the \"Relation To Broader Scientific Literature\" section.", "other_strengths_and_weaknesses": "The motivation for the design of the proposed method is well reflected, which is commendable. The experimental results are also promising. However, the justification for why it is effective in weakly supervised learning should be explicitly stated. Additionally, the comparison methods and datasets seem insufficient. Please refer to the Question section.", "other_comments_or_suggestions": "Please refer to the Question section.", "questions_for_authors": "- While I believe the proposed method is effective for supervised anomaly detection, I do not understand why it is effective for weakly supervised anomaly detection. Since the method assumes that the unlabeled data consists mainly of normal samples, the presence of anomalies within the unlabeled data might significantly affect the results. Could you clarify this point?\n\n- The authors should compare their method with approaches that explicitly handle anomalies within unlabeled data, such as LOE [1] and SOEL [2]. SOEL, in particular, has been shown to be effective for weakly supervised anomaly detection. Would it be possible to conduct such a comparison?\n\n- The appendix (Appendix P) includes an ablation study on the diversity term, but I believe additional ablation studies are necessary. For example, what happens if only one of the two kernel functions is used?\n\n- The datasets used in the experiments seem simple. Would it be possible to conduct experiments on more realistic datasets, such as medical data like MedicalMnist [3]?\n\n[3] Yang, Jiancheng, et al. \"Medmnist v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification.\" Scientific Data 10.1 (2023): 41.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "inlPJs3sxJ", "reviewer": "Reviewer_a6ae", "summary": "This paper proposes Weakly Supervised Anomaly Detection via Dual-Tailed Kernel (WSAD-DT), which uses two centroids, one for normal samples and one for anomalies. It uses a light-tailed kernel for normal samples, and heavy-tailed kernel for abnormal samples. To prevent degenerate ``all-points-collapse’’ solutions, the method introduces kernel-based regularization term that promotes intra-class diversity. Further, an ensemble strategy that partition unlabeled data into diverse subsets were proposed.\n\n## update after rebuttal\nThe score was upgraded.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the valuable feedback.\n\n**Usage of Lemma 4.1 in the main paper.**\n \nLemma 4.1 is the formal stepping-stone for showing that a single kernel cannot serve as both a strictly light-tailed and a heavy-tailed function. By demonstrating $\\lim_{d\\to\\infty} \\kappa_{\\mathrm{light}}(d)\\,/\\,\\kappa_{\\mathrm{heavy}}(d) = 0$, it clarifies that any kernel “light” enough to enforce tight in-class clustering cannot simultaneously retain the broader margin necessary for out-of-class separation. This result underpins the transition to a dual-kernel scheme in Theorem5.3, where one kernel is dedicated to pulling in-class points close, and a second kernel enforces out-of-class separation. In our revised version, we plan to reference Lemma 4.1 more explicitly in the main text (e.g., just before Theorem5.3), making its critical role in motivating the dual-kernel approach more transparent.\n\n\n**Validation of Lemma 5.1, 5.2 and clarification of  moderate distances**\n\n\nIn Lemmas 5.1 and 5.2, “moderate distances” refers to the typical radius range where in-class samples are neither so close that their similarity is almost one nor so far that the kernel decays to zero. This region is precisely where the difference between a rapidly decaying (light-tailed) and a more gradual (heavy-tailed) function is most pronounced. Although “moderate” is inherently qualitative, Table 4 in the appendix shows that in real datasets, most points do indeed occupy this intermediate distance regime in the learned representation. By measuring average distances between each sample and its assigned (or opposing) center, one sees that in-class  samples form tighter clusters under light-tailed similarity, and out-of-class samples  remain substantially farther under heavy-tailed similarity—exactly matching the theoretical claims of Lemmas 5.1 and 5.2.\n\n**Proof $L_{total} < 2$**\n\n\nTo prove this property, we further need to prove the minimum $ L_{\\text{total}}(\\theta) $ is below 2 and $ L_{\\text{separation}}$ is not zero.\n\nIn the main paper, we define the total loss as\n\n$$\nL_{\\mathrm{total}}(\\theta) = L_{\\mathrm{separation}}(\\theta) + L_{\\mathrm{diversity}}(\\theta).\n$$\n\nCorollary 5.6 shows that collapsing each class onto a single point yields a diversity loss of exactly 2 while driving the separation term arbitrarily close to zero, so the total loss under this degenerate arrangement is approximately 2. To demonstrate that degeneracy does not globally minimize the total loss, one can construct a non-degenerate arrangement in which each class’s points lie near (but not identically at) its center. Placing the points close to their respective centers makes $ L_{\\mathrm{separation}}(\\theta) $ arbitrarily small, say $ \\varepsilon$, while ensuring that the points are not all coincident reduces $ L_{\\mathrm{diversity}}(\\theta) $ below 2 by some margin $\\delta > 0 $. Because these adjustments can be made largely independent of each other, one can tune $ \\delta$  to exceed $\\varepsilon $, forcing\n\n$$\nL_{\\text{total}}(\\theta) = L_{\\mathrm{separation}}(\\theta) + L_{\\mathrm{diversity}}(\\theta) \\le \\varepsilon + \\bigl(2 - \\delta\\bigr) <  2.\n$$\n\nSince the degenerate arrangement’s total loss is 2, whereas this non-degenerate arrangement achieves a strictly smaller value, degeneracy cannot be optimal. Consequently,  $L_{{separation}}$  cannot vanish at the true minimum, and the trivial collapse solution is excluded as a global minimizer.\n\n\n**Explanation regarding 0 label and ensemble size.**\n\nIn Section3.1, “0” denotes unlabeled samples whose true status is unknown—mostly normal but potentially anomalous. Our method uses the labeled anomalies (y=1) to drive separation, exposing hidden anomalies among the unlabeled. We will clarify in the revision that “0” does not guarantee normality. For ensemble size (1, 3, or 5 splits), more splits can marginally improve accuracy but fragment the data and increase runtime. We find five splits to be the best balance of performance and cost, and will make this rationale explicit in the revised manuscript.\n\n\n**Justification for single normal center.**\n\nIn classical one-class anomaly detection, using a single center for normal data aligns well with margin-based and complexity-theoretic reasoning: keeping the normal region compact and described by a minimal boundary (e.g., a single enclosing ball) not only simplifies optimization but also reduces the risk of overfitting, particularly under limited anomaly labels. This approach is seen in canonical methods like DeepSVDD or DeepSAD, which assume that normal data occupy a single contiguous cluster in feature space. While multiple centers can capture diverse normal modes, each adds hyperparameters. Empirically, a single center often suffices, as normal instances instances typically share enough common structure to cluster tightly around one latent coordinate.\n\n**W1**\n\nWe will improve the structure of our manuscript in the revision, addressing section formatting and organization.", "claims_and_evidence": "The effectiveness of dual-kernels, diversity regularization, and ensemble-based learning are only shown in the Appendix.", "methods_and_evaluation_criteria": "In general, abnormal samples are diverse, so it is doubtful that a single center can represent them.", "theoretical_claims": "-  How Lemma 4.1 is used is not clear, at least on the main paper. \n\n-  Lemma 5.1, 5.2, moderate distances are ambiguous, and these Lemmas are not validated by experiments. \n\n-  Corollary 5.6 would be insufficient to prove that degeneracy is prevented. To prove this property, we further need to prove the minimum $L_{total}(\\theta)$ is below 2 and $L_{seperation}$ is not zero.", "experimental_designs_or_analyses": "This paper uses datasets from Ad Benchmark repository. The type of anomaly is not analyzed.", "relation_to_broader_scientific_literature": "This paper proposes to use two different kernels for normal and anormal samples. This point is different from the existing methods listed on related work sections.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Strengths \n- The theory of single-tailed kernels vs. dual-tailed kernels is interesting. \n- Enhancing the representation via diversity loss is reasonable\n- Ensemble-based subset splitting is effective \n- The performance of the proposed method is higher than state-of-the-art. \n\nWeaknesses\n\nThe structure of this paper is inadequate. Sec.1. is written in one paragraph except for contributions. Some section formats are not consistent. Sec.6.1 Experimental Setup is written in an improper location. Ablation studies are shown only in the appendix.", "other_comments_or_suggestions": "-  In Sec 3.1, 0 is the unlabeled sample, which is largely normal but can contain anomalies.  However, in p.4, 0 is regarded as normal without explanation. \n\n-  The ensemble size of 5 may not the best. Since the only 1,3,5 splits are evaluated, more larger number of ensemble may increases the accuracies.", "questions_for_authors": "The authors should justify the single center for normal samples.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: WSAD-DT is a framework for anomaly detection under weak supervision -- an emerging field since 2017.
 
The method introduces dual-tailed kernels—light-tailed for compactness around the normal centroid + heavy-tailed for maintaining separation from anomaly points—and includes a kernel-based diversity term plus an ensemble partitioning of unlabeled data. The empirical results on tabular and some image-based datasets show improved performance over existing approaches like DeepSAD and XGBOD, with only a small fraction of anomalies is labeled. One thing I have to say is both xgbod and deepsad are quite old in ML definitions... though.

Reviewers found the central idea is well motivated, with theory suggesting each tail focuses on either tight clustering or longer-range separation. They also appreciated the ensemble approach for leveraging the limited anomaly labels effectively. 

However, some questioned how well a single centroid for anomalies would generalize if actual anomalies are highly diverse, this is indeed important to know. I suggest the authors to better talk about limitations and failure cases in the final version.

Overall, there is consensus to merit acceptance, though some structural refinements and clarifications should be included in the final version.
Decision: Accept (poster)