Contextual Learning for Anomaly Detection in Tabular Data
Authors: Spencer King
Zhilu Zhang
Ruofan Yu
Baris Coskun
Wei Ding
Qian Cui
Venue: Accepted by TMLR ()

Abstract: Anomaly detection is critical in domains such as cybersecurity and finance, especially when working with large-scale tabular data. Yet, unsupervised anomaly detection---where no labeled anomalies are available---remains challenging because traditional deep learning methods model a single global distribution, assuming all samples follow the same behavior. In contrast, real-world data often contain heterogeneous contexts (e.g., different users, accounts, or devices), where globally rare events may be normal within specific conditions. We introduce a \emph{contextual learning framework} that explicitly models how normal behavior varies across contexts by learning conditional data distributions $P(\mathbf{Y} \mid \mathbf{C})$ rather than a global joint distribution $P(\mathbf{X})$. The framework encompasses (1) a probabilistic formulation for context-conditioned learning, (2) a principled bilevel optimization strategy for automatically selecting informative context features using early validation loss, and (3) theoretical grounding through variance decomposition and discriminative learning principles. We instantiate this framework using a novel conditional Wasserstein autoencoder as a simple yet effective model for tabular anomaly detection. Extensive experiments across eight benchmark datasets demonstrate that contextual learning consistently outperforms global approaches---even when the optimal context is not intuitively obvious---establishing a new foundation for anomaly detection in heterogeneous tabular data.

Published in Transactions on Machine Learning Research (03/2026)
Contextual Learning for Anomaly Detection in Tabular Data
Spencer King
sdk81 adk8172288 722 Bruga. edu
School of Computing University of Georgio, Athens, GA, USA
Zhilu Zhang
shaskilu@maazon
Amazon Web Services, Seattle, WA, USA
Ruofan Yu
raofan@amaczo..co
Amazon Web Services, Seattle, WA, USA
Baris Coskun
barisco Cawaaonn.aaooocccmmmmm
Amazon Web Services, Seattle, WA, USA
Wei Ding
dingwe @wmazon.ccom com
Amazon Web Services, Seattle, WA, USA
Qian Cui
cuiqia Ciimazonamzzonccoom com
Amazon Web Services, Seattle, WA, USA
Reviewed on OpenReview: https: openreview. net/ pormfo id= PmgZa LRENM
Abstract
Anomaly detection is critical in domains such as cybersecurity and finance, especially when working with large-scale tabular data. Yet, unsupervised anomaly detection-where no labeled anomalies are vvilabllerremiins challenging because traditional deep learning methods model a single global distribution, assuming all samples follow the same behavior. In contrast, real-workd data often contain heterogeneous contexts (e.g., different users, accounts, devices), where globally rare events may be normal within specific conditions. We introduce contertual learning framework that explicitly models how normal behavior varies across contexts by learning conditional data distributions P(Y C) rather than a global joint distribution P(X). The framework encompasses (1) a probabilistic formulation for context- conditioned learning, (2) a principled bilevel optimization strategy for automatically selecting informative context features using carly validation loss, and (3) theoretical grounding through variance decompasition and discriminative learning principles. We instantiate this framework using novel conditional Wasserstein autoencoder simple yet effective model for tabular anomaly detection. Extensive experiments across cight benchmark datasots demonstrate that contextual learning consistently outperforms global approaches- when the optimal context is not intuitively obvious eetiiissssaaabiinnnggg a new foundation for anomaly detection in heterogeneous tabular data.
1 Introduction
Unsupervised anomaly detection (AD) is critical field focused on identifying patterns that deviate from expected behavior, with applications spanning wide range of tasks like security, fraud detection, and system reliability Chandola et al. (2009); Aggarwal (2017). The core challenge is not merely detecting new unseen
"These authors contributed equally.
Work doue while interning Amazon Web Services.
*Corresponding Author.

---

Published in Transactions on Machine Learning Research (03/2026)
objects, but discerning whether such observations align with the learned notion of "normal," " truly represent anomalies.
Recent advances in deep learning have driven a wave of neural network-based methods for unsupervised anomaly detection Landauer et al. (2023); Pang et al. (2021b). Despite methodological differences, most deep approaches model the joint distribution of all features, implicitly assuming that data patterns are homogeneous across conditions. In practice, patterns often vary significantly with context. For example, in cybersccurity, spike in data transfer may be benign for a user who routinely performs backups but suspicious for database administrator. Ignoring such context can degrade detection accuracy.
To address this challenge, we argue probabilistically and demonstrate empirically the effectiveness of modeling conditional distributions over joint distributions for anomaly detection. We advocale for a contertual learning framework, which models conditional distributions to capture intra-context regularities while isolating inter- context variability, thereby enhancing dctection performance without requiring scparate modcls for cach condition.
Selecting appropriate context feature is critical aspect of contextual learning. It is often not obvious what feature one should condition on when modeling the data. Indeed, the choice of conditioning variable can significantly influence model performance. To address this, we develop a bilevel optimization methodology to automatically identify the context. features based on early validation loss. Requiring only a small set of validation samples, our approach is simple, model-agnostic, and demonstrates consistent benefits across a wide range of datasets.
To demonstrate the framework's utility and empirically validate claim, build upon the Wasserstein autoencoder (WAE) Tolstikhin et al. (2019) to develop the conditional Wasserstein autoencoder (CWAE), novel conditional variant designed to model conditional distributions for anomaly detection. We instantiate the framework using CWAE to conduct experiments using several tabular AD benchmark datasets containing inherent contextual information (c.g., users, accounts, or hosts). We demonstrate that incorporating contextual information with CWAE enables more robust anomaly detection across diverse and complex datasets, and offers significant advantages in improving detection accuracy.
Main contributions:
We introduce a general contextual learning framework for anomaly detection.
We provide probabilistic formulation and theoretical justification grounded in variance decomposition and discriminative learning.
We propose bilevel optimization strategy for context selection using early validation loss.
We instantiate the framework with novel CWAE model and demonstrate consistent performance gains across diverse datasets.
We demonstrate that conditional models in general can outperform both their non-contextual counterparts of the same architecture and state-of-che-art. (SOTA) unconditional models.
2 Related Work
Anomaly detection has been a long-standing area of rescarch, with carly work dominated by statistical and distance-based approaches. Classical statistical methods such Principal Component Analysis (PCA) Shyu et al. (2003) and distance-based techniques like k-Nearest Neighbors Liao & Vemuri (2002) and clustering- based methods Münz et al. (2007) aim to identify points that deviate from expected norms in the data distribution. While effective in lower dimensions, these techniques struggle in high-dinensional settings due to the *curse of dimensionality."
More recent advances leverage deep neural networks to learn expressive representations for anomaly detection in complex data. Notable lines of work include one-class classification with deep feature embeddings Ruff et al. (2018; 2021), autoencoder-basee approaches that reconstruct. normal samples Gong et al. (2019); Zhou &
2

---

Published in Transactions on Machine Learning Research (03/2026)
Paffenroth (2017); Zhao et al. (2017); Liu et al. (2021), and generative models like GANs Schlegl et al. (2017); Sabuhi et al. (2021), normalizing flows Gudovskiy et al. (2022), and diffusion models Livernoche et al. (2023). However, several studies Kirichenko et al. (2020); Zhang et al. (2021) have shown that generative models trained to match the marginal data distribution can fail to identify out-of-distribution or context-specific anomalies, as they may assign high likelihood to atypical but structurally plausible inputs.
To address this, self-supervised learning methods Golan & El-Yaniv (2018); Wang et al. (2019b); Chen et al. (2020); Tack et al. (2020) recast anomaly detection an auxiliary classification task, such predicting data augmentations transformations. While effective in image domains, extending such techniques to tabular categorical data is non-trivial Bergman & Hoshen (2020); Qiu et al. (2021); Shenkar & Wolf (2022), and performance can degrade when the auxiliary task is misaligned with anomaly structure.
Orthogonal to these, subspace anomaly detection methods aim to discover subsets of features or projections where anomalics become more apparcnt. Classical examples include axis-parallel subspace approaches Kriegel et al. (2009), projection pursuit, and local subspace analysis Aggarwal & Yu (2013); Sathe & Aggarwal (2016). These methods, including randomized hashing Sathe & Aggarwal (2016), avoid modeling the full joint distribution by seeking dimensions that isolate outliers. However, they typically treat all features equally and do not distinguish between context and behavioral attributes.
In contrast, context-aware or conditional anomaly detection explicitly models how behavioral features deviate from expected values given a specific context Song et al. (2007). For instance, CADENCE Amin et al. (2019) models conditional probabilities for discrete events in high-cardinality logs, while the joint deep variational generative model of Shulman Shulman (2019) separates contextual and behavioral attributes for anomaly scoring. Similarly, UCAD Li ct al. (2022) detects anomalous database operations by comparing operation semantics against ual intent, and Moore & Morelli (2024) estimate context-condiitooned distributions via normalizing flows for time series anomaly detection. These approaches highlight the benefits of modeling conditional distributions over global joint distributions, but they typically either require contextnal and behavioral variables to be specified in advance or focus on more structured settings such as event records, logs, database operations, or time series.
Our work builds this literature in three key ways. First, we target tabular data-a setting with less inherent structure and no predefined context-making making the identification and modeling of contextual dependencies significantly more challenging. Second, unlike prior heds that rely on dual encoders or complex density estimators, we propose a lightweight yet effective CWAE that models P(content context) through a simple reconstruction objective with MMD regularization. Third, we introduce a practical, unsupervised context selection st stregg based on early validation loss, allowing the framework to automatically discover informative context fcatures without supervision. Together, these components form a unified contertual learning framework for anomaly detection that is both conceptually simple and broadly applicable. This perspective reveals that meaningful contextual structure often exists even when not explicitly labeled, and that leveraging such structure can significantly enhance detection performance with minimal architectural complexity.
Additionally, our work is related to the extensive literature on statistical process control (SPC) and multivariate statistical process control (MSPC), which monitor deviations from learned baselines using control limits. Early univariate control charts date back to Shewhart Shewhart (1931), while multivariate extensions include Hotelling's 2 charts Hotelling (1947) and Mahalanobis Mahallanbis-diittanc--ban distanco- detectors Mahalanobis (1936), as reviewed in standard SPC references Montgomery (2012). PCA-based MSPC further decomposes variability into latent subspaces and residual components, yielding statistics such as T and squared prediction error (SPE) Jackson & Mudholkar (1979). More recently, autocncoder-ban based process monitoring methods replace linear subspace models with nonlinear representations, while using reconstruction error or latent-space monitoring statistics for fault detection Qian et al. (2022); Biegel et al. (2022).
Conceptually, these methods share the intuition that anomalies should be assessed relative to an appropriate baseline or operating regime. However, classical SPC and MSPC techniques typically rely on assumptions such as Gaussianity, linear structure, or known operating regimes, and are primarily developed for continuous process variables. In contrast, our framework generalizes this conditional monitoring perspective to high- dimensional, heterogeneous, and categorical tabular data without requiring explicit regime labels.
3

---

Published in Transactions on Machine Learning Research (03/2026)
3 Preliminaries
Consider random variable X E Rd representing data with d features. We assume that X follows distribution P(X). The anomaly detection problem aims to determine whether X is typical with respect to P(X) or anomalous.
Definition (Context-Conditional Anomaly Detection). Let X = (C,) E R Rd-k where C represents k-dimensional conlext features and Y represents (d- k)-dimensional content features. Given training dataset D ((), drawn from the distribution of normal events, our objective is to learn score function
S S: x Rd-k R
that assigns an anomaly score S(X) to each data point X. Then based on this anomaly score, we set a threshold 7 so that points with scores S(X) exceeding т are labeled as anomalics.
In traditional anomaly detection research, common approach is to model the joint probability distribution over the entire feature space and to construct single decision boundary to separate normal and anomalous behaviors. For instance, suppose X = {A, B, C, D} with four features, and we assume X follows joint distribution P(X) = P(A, B,C, D). We estimate P(X) from the training data and then derive an anomaly score directly from the learned distribution. Typically, observations with lower probability under P(X) are considered more likely be anomalous. Formally:
Score(X) S(P(X)),
some other monotonic transformation of P(X), i.c., log P(X). To determine whether data point X is anomalous, the model compares the score against predetermined threshold T:
Normal
if Score (X) S T
(X)
Anomalous
if Score () > T.
Here the single decision boundary is derived from training data, i.c., the maximum anomaly score.
A model using P(X) for anomaly detection without conditional modeling relies on an implicit assumption of homogeneity across different contexts. However, this assumption may fail in practice, particularly when dealing with large-scale and complex datasets, as discussod in Section 1. If the context significantly influences the anomaly patterns, this assumption introduces noise into the learned boundary. For such scenarios, single decision boundary derived from the joint distribution might not adequately capture the diverse anomaly patterns. Next we introduce contextual conditioning approach that effectively isolates the variability across different contexts, enabling the model to learn more precise boundaries.
3.1 Contextual Learning Formulation
To address the heterogeneous nature of the problem, we hypothesize that subset of features-referree to optimal context features captures the greatest diversity of anomaly patterns across their unique values, thereby defining distinct contexts. For simplicity, interpretability, and computational stability, we focus on a single feature that exhibits this maximal diversity, denoted as the context feature, while treating the remaining features as content features that characterize anomaly patterns within each context.
In practice, the partition between context and content features is entirely data-driven. Candidate context. features drawn from the observed feature set, and exactly one feature is selected as context via the validation-based procedure described in Section 5.3. All remaining features are treated as content and modeled conditionally. This partition is dalaset-dependent and does not rely on prior semantic assumptions. When a candidate context. is uninformative, the selection procedure naturally favors the no-context (global) baseline.
Focusing on a single feature offers several practical advantages. First, enhances interpretability- practitioners can directly observe which variable defines the contextual partition. Second, it avoids the combinatorial explosion of context groups that arises under multi-feature conditioning, which lead to
4

---

Published in Transactions on Machine Learning Research (03/2026)
data sparsity and unstable estimation when certain feature combinations rarely observed. Third, our empirical findings indicate that well-chosen single features often capture the dominant axis of contextual variation, while remaining dependencies can be effectively modeled through the content features. Importantly, this simplification does not restrict the generality of our framework: all theoretical results, including those on variance decomposition and discriminative learning advantages, naturally exlend the multi-feature case where R* with & > 1. Future work may explore multi-fcature context construction when single-feature conditioning proves insufficient.
As established in our definition above, denote C as k-dimensional context features and Y (d - k)- dimensional content features, where & < d. Given training datasct D = (c), drawn from the distribution of normal events, our objective is to learn a score function
S IR +R= R
where S(c, y) outputs an anomaly score for the context and content observation (c,y). Unlike the conventional approach that models P(X), now consider the conditional distribution P(Y C). By focusing on P(Y C= c), c), we may define context-wise thresholds Te for each supp(C).
By leveraging this conditional structure, the model learns a separate decision boundary for each value e that C' can take. In other words, adapts the anomaly detection rule to the specific subspace partitioned by each context value. Thus, instead of relying on a single threshold T, we can define context-dependent threshold Te.
The resulting context-conditioned anomaly detection rule becomes:
Normal, if S(c,y) S Te,
Label(c, Anomalous,
if S(c,y) > Te-
This approach allows the model to more accurately profile the true boundaries of normal behavior by accounting for contextual variability, rather than relying on single, global threshold. This formulation constitutes the core of our contextual learning framework.
4 On the Advantages of Contextual Learning
Conceptually, our contextual learning meaodigm-dffinee as conditional modeling of content given context- parallels discriminative logic the classic generalive-disrininaais framework. While generative methods strive to capture the joint distribution of inputs and labels (analogous to content and context our framework), they can be error-prone if assumptions about the data-generating process are misspecified. By contrast, conditional or discriminative approach sidesteps modeling the context distribution altogether, focusing solely on how content depends context. This distinction is particularly salient when context. variables (e.g., userlDs) have high cardinality, since modeling P(C) or the full joint distribution P(Y, C) becomes prohibitively complex, risking curse of dimensionality.
Such complexity motivates Vapnik's s principle (Vapnik, 1995), which asserts that one should not solve a more general, and potentially more difficult, problem than necessary meet the ultimate predictive goal. By focusing on P(Y C), our method reduces the risk of model misspecification surrounding context variables and streamlines parameter estimation in high nigh-dimensional spaces. In practice, this can lead to more robust and data efficient performance, which is an advantage also highlighted by extensive generative vs. discriminative comparisons in the classification literature. We illustrate this in more detail in the following subsections.
Furthermore, we illustrate the advantage of contextual learning from the perspective of variance reduction by the conditioncd context variable. We argue that with an optimal choice of context variable, the conditional variance given each context would be smaller, resulting in less spread among data points, making deviations anomalies more pronounced and easier to detect.
5

---

Published in Transactions on Machine Learning Research (03/2026)
4.1 Model Misspecification Risks
Several theoretical and empirical studies substantiale these advantages for conditional or discriminative approaches. Ng & Jordan (2001) provides a rigorous comparison of Logistic Regression versus Naïve Bayes, demonstrating that although the generative model can dominate in very low-data regimes (due to stronger assumptions reducing variance), the discriminative model typically surpasses it once the sample size becomes sufficiently large. This phenomenon occurs precisely because discriminative model directly optimizes the conditional likelihood pertinent to the task akin estimating P(Y C) rather than the full joint distribution. The asymptotic analysis by (Liang & Jordan, 2008) further clarifies how misspecificaiinn in modeling P(C) can degrade generative model's performance, while discriminative estimator that only captures P(Y C) remains robust. Lasserre et al. (2006) shows how purely generative assumptions can degrade classification accuracy when they are violated, and how incorporating a discriminative component can improve robustness and prediction quality.
4.2 Curse of Dimensionality
In the high-dimensional context space, Hastie et al. (2009) discusses how focusing on the conditional distribution helps avoid unnccessary parameter explosion, while Bishop (2006) and Murphy (2012) provide extensive treatments of both paradigms, showing that discriminative techniques can be more robust to structural mismatches in the data generation process. These results generalize beyond simple supervised learning to broader scenarios such as one-class classification semi-supervised training, where focusing on the part of the distribution necessary for the objective-be it anomaly detection or prediction- can reduce parameter complexity and improve predictive accuracy. Hence, in contexts where one can define a meaningful "background" subset of variables, instead of learning the full joint distribution, conditioning on them directly can provide both practical and theoretical advantages in model efficiency. By extension, in nigh-dimensional scenarios, focusing solely on P(Y C) avoids the exponential blow-up complexity that can come with modeling or sampling from P(C). Consequently, conditional modeling can confer both practical and theoretical gains in efficiency and accuracy, making it compelling strategy whenever meaningful separation between context and content variables is possible.
4.3 Variance Reduction
In this section, we compare joint vs. conditional anomaly detection from variance-reduction standpoint. Let
Y E Rd-k (content features),
T (context features).
A joint model learns the full density P(Y, C), however, the marginal
P(Y) = P(Y. C(Y,C)dc= P(Y (Y)C))((CCE
mixes together data from all contexts.
By contrast, a conditional approach directly learns P(Y C) and scores each sample within its own context.
slice. The key theoretical insight is the law of total variance:
Var(Y)= = Re[[ar(Y C)]
Varc CC)
average "within-contest" vuriance
"hetwnen-context variance
A joint detector must accommodate both terms, whereas a context-condiitonal detector only needs to cope with Var(Y C = c), which is typically much smaller than the total Var(Y).
So ideally if we can find contexts to partition the data so that
Var Y C c) < Var(Y)
holds for most c E supp(C), conditioning makes our anomaly detectors both sharper and more sensitive to small deviations from normal, context-specific patterns. For reconstruction based models, this variance reduction yields the following practical benefits:

---

Published in Transactions on Machine Learning Research (03/2026)
1. Higher signal-to-noise ratio. Normal samples concentrate more tightly around their reconstruc- tions, making anomalous deviations stand out. We can choose threshold for each context based on the smaller Var(Y C =)), instead of a single conservative for the full marginal distribution.
2. More stable and efficient learning. Each conditional model only needs to capture a homogeneous slice of the data, leading to more stable gradient updates, faster convergence, and fewer mixed-mode reconstruction errors, as the model doesn't need to accommodate conflicting patterns from different contexts simultaneously.
We now formalize the above intuition with a heuristic concentration based argument linking conditional variance reduction to anomaly-score separability.
Proposition 1 (Heuristic link between conditional variance reduction and anomaly score separability). Let S(Y) denote an anomaly score derived from cnonstructionn-aseed model. Assume that for normal data and anyy fixed contert the conditional score distribulion S(Y) C = c is sub- Gab-Gaussian with variance prozy de where lower conditional variability in Y C = induces a smaller variance prozy for S(Y) CC= Then, for anomalous samples whose reconstruction scores remain separaled from those of normal samples by a fixed margin, conditioning on C con improve anomaly-sooe score separability relative to scoring under the marginal distribution P(Y).
Proof sketch. Under the sub-Gaussian assumption, the tail probability of the reconstruction score for normal samples satisfies
-c) (
P(S(Y) - exp
20
where He = =[S((Y) C== c] and of is the corresponding variance proxy. By the law of total variance,
Var(Y) = Ec EarVar(Y | C)] + (Er [C],
SO marginal scoring under P(Y) mixes within-context variability with additional between-context variability. Heuristically, this can induce broader normal score distribution than context-conditional scoring, yielding looser concentration. Consequently, for fixed false-positive rate, the context-dependent threshold Ta can be tighter than the corresponding marginal threshold, thereby improving the potential separation between normal and anomalous distributions.
5 Instantiating Contextual Learning with CWAE
To demonstrate the contextual learning framework in practice we instantiate it using the Contextual Wasscrstein Auto-Encoder (CWAE). CWAE is novel, generative, and reconstruction-based model designed to be straightforward and interpretable, operationalizing the framework by modeling context-conditate distributions for anomaly detection. While CWAF achieves strong performance, its primary purpose is to illustrate the effectiveness and generality the contextual learning framework rather than to serve as the main contribution of this work. Comprehensive implementation details are provided in the Appendix A.
5.1
CWAE Model Structure
The CWAE model is designed to learn the conditional distribution of input features given contextual features. This section outlines the objective, architecture, and loss function.
5.1.1 Model Objective
CWAE aims to conditionally reconstruct inpul conlent features based on associated context features. By learning a latent representation that combines content and context, the model identifies anomalies through reconstruction loss, where higher reconstruction loss indicates greater likelihood of anomalous behavior. This framework allows CWAE to adapt to varying contextual conditions, which is critical for datasets with complex or diverse anomaly patterns.
7

---

Published in Transactions on Machine Learning Research (03/2026)
5.1.2 Model Architecture
As displayed in Figure 1, CWAE consists of three main components: embedding layers, an encoder, and a decoder.
Context Emb
Content Recon
FC layers
Softmax
Latent
FC layers
Content
Space
FC layers
Softmax
Emb
Content Recon
Encoder
Decoder
Figure 1:
Architecture of the CWAE model utilized in the experiments.
Embedding Layers. Two sets of embedding layers are employed- one for context features and another for content features. Each embedding layer is randomly initialized, with the first dimension corresponding to the number of unique feature values plus one (to handle unseen values) and the second dimension representing the embedding vector size.
The encoder takes input the concatenated embeddings of context and content features. This input is processed through fully connected (FC) layers extract latent representation. To retain contextual information, the output of these layers is concatenated with the original context embeddings to form the final latent representation.
Encoder.
The decoder reconstructs the content features from the latent representation. A series of FC layers are used to map the latent representation back to the original feature space.
Decoder.
5.1.3 Loss Function
The loss function combines recoustruction accuracy with regularization ensure effective learning. Formally:
d
Lewae Σ CEYY, Y,)
+ 1.-MMD Q(2 ( c, x)))
(1)
where C and Y represent context and content features, respectively; CE(Y,, Y,) represents the cross-entropy reconstruction loss for content features and the dimension of content Y; À is a weighting hyperparameter for the MMD term; MMD (P(=), Q(z X)) quantifies the maximum mean discrepancy between the latent distribution and predefined prior (e.g., Gaussian). The reconstruction loss ensures that input features are accurately reconstructed, while the MMD regularization term aligns the latent distribution with the desired prior. The combined loss facilitates effective training and ensures the model captures meaningful contextual patterns.
5.2 Training and Inference
Training. During training, tabular features are first mapped indices, resulting in integer vectors input. These indices passed through embedding layers, which updated during training. The encoder

---

Published in Transactions on Machine Learning Research (03/2026)
and decoder work together to reconstruct the content features, and the total loss is optimized to minimize reconstruction error and regularization discrepancy.
Inference. During inference, anomaly detection is performed by calculating the reconstruction loss for each instance. Samples with reconstruction loss exceeding the contexl-specific threshold (as defined in Section 6.1.3) are classified as anomalies. This approach allows the model adapt varying contextual conditions while maintaining robust detection performance.
5.3 Context Feature Selection
Sclecting an appropriate context feature is one of the most critical and challenging aspects of the contextual learning framework. The chosen context fundamentally shapes how the model interprets patterns in the data and can have a substantial impact on detection performance. As shown in Tables 3, 4, and 5, contextual models consistently outperform their non-contextual counterparts when the context is well chosen, underscoring its central role in effective anomaly detection. Achieving strong results therefore requires a principled and effective strategy for selecting a context feature that is likely to yield good performance. This procedure operationalizes the context-content partition introduced in Section 3.1 by automatically selocting one observod feature as context and treating the remainder as content.
We propose to frame context selection bilevel optimization problem Sinha et al. (2018), where the goal is to optimize for the joint data likelihood P(Y, C) = P(Y C) P(C). In this setting, both the model parameters and the context feature are treated as variables to optimize. The inner loop learns the model weights for a given context, while the outer loop searches for the context feature that yields the best generalization performance.
To enable reliable context selection, our approach uses small validation subset drawn from data that is assumed to be predominantly normal. This assumption is aligned with common settings in unsupervised and semi-supervised anomaly detection, where training data typically consist primarily or exclusively of normal samples Tax & Duin (2004); Ruff et al. (2018); Bergman & Hoshen (2020). It is also consistent with recent work that explicitly studies model selection and validation without labeled anomalies, including settings that use only normal data during both training and validation Cui et al. (2023) or rely on a small support set of normal samples to construct validation procodures in the absence of labeled validation data Fung et al. (2024).
This roquirement is minimal, since only enough samples are needed to compute stable loss estimates rather than labeled anomalies. The validation data is used exclusively to evaluate generalization performance for context selection and is not used to train the anomaly detector itself, ensuring that the framework remains fully unsupervised with respect to anomaly labels. Such assumptions are common in practical anomaly detection meployments, where a short baseline period or trusted historical data is often available (e.g., during initial system deployment). When a clean validation subset is unavailable, practitioners may instead reserve a small portion of the training data as a proxy validation set, although this may slightly reduce selection reliability.
We note that robustness to significant anomaly contamination in unsupervised data constitutes distinct. and well-studied problem setting, often framed under contaminated data, noisy supervision, or weakly supervised anomaly detection. Recent surveys explicitly identify this as a separate line of research Jiang et al. (2023), and several methodological works propose dedicated mechanisms such as iterative reweighting, pscudo-labeling, or contamination-aware objectives handle high contamination ratios Kim et al. (2023); Wu al. (2022). Our work addresses a different setting and follows the conventional assumption that training and validation data are mostly benign. That said, because our bilevel selection procedure depends primarily on the relative ranking of candidate context features rather than absolute validation loss values, small amounts of contamination (e.g., 5 10%) are unlikely to fundamentally alter context rankings.
Naively training each conditional model to convergence is thorough, but can be prohibitively expensive. In practice, we only train the model for one epoch, and evaluate the model with validation loss afterwards. As a result, the computational cost of context selection scales linearly with the number of candidate context features, requiring one partial (single-epoch) training run per feature. In practice, this cost. is modest relative
9

---

Published in Transactions on Machine Learning Research (03/2026)
to full model training and is trivially parallelizable across candidate contexts. Consequently, the procedure is well-suited to modern multi-core or distributed training environments.
Our single-epoch evaluation strategy is motivated by extensive empirical and theoretical evidence showing that early training dynamics are strongly predictive of final generalization performance. Numerous prior works have successfully leveraged early-trained models for neural architecture search Liu et al. (2019); Bender et al. (2018), model pruning You et al. (2020), dataset pruning Paul al. (2021), and learning with noisy labels Liu et al. (2020); Zhang & Sabuneu (2018). Studies have also shown that the first few epochs establish key characteristics of the learned representation Achille et al. (2017); Zhang et al. (2016); Frankle et al. (2020). In our context selection sctting, the goal is not absolute performance but rather the relative ranking among context features, a signal that emerges early in training. Figure 3 supports this empirically, as the early-epoch praxy achieves better performance on five of eight datasets.
To ensure consistency across models with different probabilistic factorizations o.g. P(Y C) vs. P(Y,C) we evaluate all models using their joint log-likelihood. For conditional models, we apply the identity P(Y, C) = P(Y) P(C), where P(C) is estimated empirically from the training data. This formulation also penalizes rare context values via the log P(C) term, favoring parsimonious and meaningful conditioning. As motivated in Section 3.1, we only condition on one feature at a time as the context feature. As such, P(C)
can be easily obtained from the training data.
Putting everything together we present the proposed procedure for selecting context features Algorithm This data-driven selection method avoids heuristic filtering and provides a principled mechanism for identifying informative context. features within the contextual learning framework.
Algorithm Context Selection via Validation Loss Minimization
Given candidate context features K, training set Daraio validation set Drat assumed to predominantly normal, and model M:
1. Outer Loop (Context Optimization):
For each C E
a. Initialize model M conditioned on context C
b. Inner Loop (Model Training): Train M on Drain to optimize weights w for 1 epoch
c. Compute joint validation loss Lvat = log P(Y, C) on Dval
Select context C with minimal validation loss
Return final context C and trained model M(Y C*)
Figure 2 illustrates this process the Census dataset using CWAE. We show the validation loss curves for the top and bottom three candidate context features, based on performance after a single epoch. The feature detailed Sociplatoo_cccpptiiiiiei _eeodde results in the lowest loss and is therefore selected.
Using this selection method, we achieve performance equal to or better than the non-contextual baseline in all but two datasets. Figure shows the improvement in AUCROC for CWAF when using the selected context feature, compared to its non-contextual counterpart. In cases where the no-context option yields the best validation loss, it is retained as the final choice (marked in orange). Final context selections are summarized in Table 3 under the Contert column.
This selection cost is comparable in spirit to commonly used procedures in anomaly detection and representation learning, such as ura screening, hyperparameter search, or early-stoppingg-asse model select ion, all of which require multiple partial training runs. While our approach incurs higher upfront cost than lightweight methods such as Isolation Forest, it remains practical for high-cardinality, heterogencous tabular sottings, where improved detection accuracy often prioritized over minimal training time.
5.4 Advantages of the Wasserstein Approach
As outlined in Section 4, effective conditional modeling of P(Y | C) benefits from architectures that are both stable and efficient at inference. Reconstruction-bassd anomaly detection models-including inncluding standard
10

---

Published in Transactions on Machine Learning Research (03/2026)
Validation Loss per Epoch By Context Model: cwae, Dataset: census
Context (AUCCOC, Minimum Fpoch locs)
27.5
occupation recode (0.743, 24.74)
major_ www.opcccccconccco code (0.742, 24.95!
25.0
country of_ __birth_mtther mother (0.64) 645, 25.00)
dividends from ftom_sttcce stocks (0. (0.722, 27.87)
capitel_geins (0.585, 28.30)
Va Validation Loss
22.5
family_ members, under 18 (0.664, 28.39)
20.0
17.5
15.0
12.5
Epoch
Figure 2: Validation loss curves for CWAE on the Census dataset, conditioned on different context features.
AUC-ROC Gain by Dataset cwae (Min Val Loss Epoch 1)
A AUIC-RO ROC (96) (Contextual vs. Non- -o--eottttll
Context Cases No Context Cases
30
25-
20
15
20-
spotify
bank
consus
kdd
bedh
lari
Dalasels
Figure 3: AUCROC improvement achieved by applying Algorithm 1 using only one epoch of training to select context for CWAE across datasets. Results show that one epoch validation loss provides sufficiently reliable relative ranking signal for context selection on the majority of datasets. When early signals are uninformative, the method degrades gracefully toward the no-context baseline.
autocncoders Zhou Paffenroth (2017); Gong et al. (2019), GAN-based approaches Schlogl et al. (2017), and variational autoencoders (VAEs) and their conditional variants Kingma & Welling (2014); Sohn et al. (2015)-Bu) have been widely used for modeling normal data distributions. However, stochastic latent-variabll models such as VAEs often benefit from averaging over multiple latent samples at inference to obtain more stable anomaly scores, at the cost of additional computation.
Wasserstein Autoencoders (WAEs) Tolstikhin et al. (2019) replace the KL-based regularization used in VAEs with an optimal transport-basod objective, which can be implemented using Maximum Mean Discrepancy (MMD) Gretton et al. (2012). This formulation supports deterministic encoders while regularizing the aggregated posterior to match chosen prior. By avoiding Monte Carlo sampling at inference, deterministic
11

---

Published in Transactions on Machine Learning Research (03/2026)
encoding make anomaly scoring faster and more stable, aligning with the robustness and efficiency goals discussed in Section 4.
In this work, we instantiate CWAB with a deterministic encoder decoder pair. This design choice reduces inference latency, improves score consistency, and maintains a well-regularized latent space, providing practical benefits over the Conditional VAE (CVAF) baseline while preserving the ability to model complex conditional distributions.
Importantly, these properties especially well aligned with the goals of contextual anomaly detection. Because anomaly scores are computed and compared across heterogeneous contexts, score stability and low variance at inference time are critical: stochastic latent sampling can introduce unncessary noise that obscures true context-dependent devialions. Deterministic encoding ensures that anomaly scores reflect structural differences in the data rather than sampling variability, leading to more consistent context-wise thresholds and more reliable cross-context comparisons. We emphasize that this design choice is complementary to the broader contextual learning framework rather than essential to its validity; alternative conditional models can be substituted within the framework, but the Wasserstein formulation provides particularly stable and practical instantiation.
6 Experiments
6.1
Experimental Setup
6.1.1
Dataset Selection
To evaluate the effectiveness of the contextual learning framework, we instantiate using CWAE and assess its performance on selected datasets spanning diverse domains, including finance, cybersecurity, demographics, and network intrusion detection. Each dataset is designed to facilitate the study of anomaly detection with contextual learning, where anomalies are defined by specific criteria such as customer behavior, malicious events, or demographic thresholds.
We started with datascts introduced in the AD literature, such as those discussed in Han et al. (2022), and expanded our selection to include additional datasets originally used for multi-class classification. Collectively, we have compiled a suite of eight datasets for comprehensive empirical evaluation, which includes the Bank Marketing dataset (bank) Moro et al. (2012), the Beth Cybersecurity dataset (beth) Highnam et al. (2021), the Census dataset (census) cen (2000), the CMC dataset (cme) Lim (1999), the KDD dataset (kdd) kdd (1999), the LA dataset (lanl) Turcotte et al. (2019), the Solar Flares dataset (sf) sol (1989) and the Spotify dataset (spotify) Choksi (2021). Details about the datasets can be found in Table 1. These datasets vary significantly in size, feature count, and anomaly prevalence, providing a comprehensive foundation for robust evaluation. Complete implementation and evaluation details to facilitate reproducibility are provided in the Appendix A.
6.1.2 Data Complexity Ranking
Understanding dataset complexity is essential for ensuring that contextual learning methods are evaluated across a broad spectrum of dataset difficulties. By considering complexity, we can betler assess how contextual learning behaves in both simpler and more challenging anomaly detection scenarios. This perspective allows us examine whether the benefits of contextual learning hold consistently across varying levels of difficulty. To capture these complexities, we adopt several metries from AD literature Pang et al. (2021a), tailored to reflect the specific characteristics of our datasets. It is important to note that the metrics are used solely post-hoc diagnostic descriptors to contextualize empirical results, rather than evaluation criteria, optimization objectives, or the basis for any methodological claims. These metrics are defined as:
Value Coupling Complexity (Kvee): Quantifies the impact of similarity among outliers (homophily) on detection effectiveness. Higher Kvec values indicate greater complexity, as coupled outliers introduce noise and correlations that hinder detection.
12

---

Published in Transactions on Machine Learning Research (03/2026)
Heterogeneity of Categorical Distribution (Khet): Measures the diversity in the frequency distribution of dominant categories across features. A higher Knet value signifies increased difficulty in identifying outliers due to the varied distribution of categorical modes.
Outlier Inseparability (Kins): Represents the difficulty of separating outliers from normal data points. ranks features based on the inverse frequency of their values, with higher Kins values suggesting that anomalics are less distinguishable from normal objects.
Feature Noise Level (Kmi): Captures the proportion of features where noise causes outliers exhibit frequencies similar to or higher than normal data points. Higher Kinl values indicate increased detection difficulty due to noise obscuring distinctions.
Table provides a comprehensive breakdown of dataset complexity across the four distinct metrics: K Knet, Kins, and Kia- For cach datasct, we report the raw metric value, its min-max scaled version, and the corresponding rank. The overall complexity score (Avg Scoled) is computed as the average of the scaled scores across all four metrics. This aggregate score serves final ranking reflects this overall complexity, where higher scores correspond to more challenging datasets for anomaly detection, and lower rank values indicate relatively easier datasets.
unified measure of dataset difficulty. The
Comparing Tables and shows that our complexity metrics capture challenges that basic statistics miss. For instance, spotify emerges as the most complex dataset despite appcaring ordinary in size, feature count, and cardinality, highlighting the role of structural and contextual patterns in driving difficulty.
Dataset
Number
Size
Number of Anoma- lies
Anomaly Ratio
Avg Cardi-
Name
of Fea- tures
nality
bauk
41,188
4,640
11.27%
beth
1,141,07 078
158,432
13.88%
35, 154
census
38
299,285
18,568
6,20%
17
eine
1,473
29
1.97%
3.13
kdld
1,014,535
440
4.51%
12
Jani
16
2,542,727
6,971
0.23%
3,191
sd
1,006
43
4.03%
3.73
spotify
113.550
2,412
2.12%
4.615
Table Overview of datasets with descriptive statistics.
K
Kyet Scaled
Kin
K ful
Overall Complexity
Dataset
Score
Scaled
Rank
Score
Rank
Score
Scaled
Rank
Score
Scaled
Rank
Avg Scaled
Rank
bank
0.210
0.196
5
2.015
0.016
4
0.343
0.775
3
1.000
1.000
0.497
beth
0.916
1.000
1
2.929
0.037
0.016
0.032
0.179
0.179
0.312
consus
0.450
0.469
3
3.500
0.049
2
0.238
0.536
0.256
0.286
0.335
eme
0.038
0.000
8
1.579
0.007
6
0.348
0.786
2
0.000
0.000
0.198
7
kdd
0.055
0.019
7
1.278
0.000
8
0.159
0.357
6
0.500
0.500
0.219
6
lanl
0.369
0.377
1.939
0.002
0.000
0.009
0.009
0.100
sf 0.124
0.098
6
1.564
0.006
0.176
0.395
0.500
0.500
0,250
spotify
0.500
0.526
2
16.261
1.000
0.442
1,000
0.123
0.123
0.662
Table 2: Dataset complexity metrics with raw scores (Score), scaled scores (Scaled), and ranks (Rank) reported for each. Higher scores indicate greater complexity, while lower ranks correspond higher complexity.
Conversely, datasets with large size or high cardinality, such as beth and land, rank lower in complexity, suggesting that these properties alone do not necessarily make anomaly detection harder. Overall, the complexity metrics offer more nuanced basis for evaluating contextual learning across datasets of varying difficulty.
6.1.3 Contextual Thresholding for AUCROC Calculation
Traditional AUCROC calculation typically involves grid searching over single, global decision threshold. In the no-context setting, this is straightforward since all samples share the same threshold. However, in
13

---

Published in Transactions on Machine Learning Research (03/2026)
contextual learning, using a single global threshold can be suboptimal because score distributions may vary significantly across context groups. To address this, we adopt a per-context thresholding approach while ensuring that resulting AUCROC values remain directly comparable to the baseline.
Given a training set with a particular context feature selected, we define the maximum training loss within each context group as contextual threshold Hc,, For the test set, each sample's anomaly score normalized by its corresponding contextual threshold, producing contextual ratio Rc.- asomaly scure, This normalization He;
rescales scores so that they are on the same relative scale across different contexts, effectively reducing the per-group thresholding problem to a single normalized thresholding problem.
We then compute the AUCROC by grid searching over 100 evenly spaced thresholds across the range of Rc,- For example, step & corresponds to a threshold of 0.01 k- max(Rc.). Since both the contextual and no-context settings ultimately apply throsholding to single unified score distribution (global anomaly scores in the baseline, normalized contextual ratios in the contextual case), the resulting AUCROC values are directly comparable across all model configurations.
6.1.4 Handling Unseen Contexts
Context features embedded using lookup tables with additional fallback index reserved for unseen values. During inference, any context value not observed during training is mapped to this fallback embedding. In these cases, context information is effectively unavailable, and anomaly scoring relies primarily on the learned content representation. This yields a well-defined fallback behavior that is comparable to a non-contextual baseline, without requiring extrapolation to unseen context embeddings.
Our evaluation pipeline distinguishes between samples whose context values were observed during training and those containing previously unseen context values. Unseen contexts evaluated using the fallback embedding described above, allowing performance in this regime to be analyzed separately rather than being conflated with seen-context results. Implications for low-support and rare contexts are discussed further in Section 9.
6.1.5 Benchmark Models
To effectively benchmark our proposed model, we selected six SOTA anomaly detection models from the DeepOD package Xu: DSVDD Ruff et al. (2018), RDP Wang et al. (2019a), RCA Liu et al. (2021), ICL Shenkar & Wolf (2022), DIF Xu et al. (2023a), and SLAD Xu et al. (2023b). Based on our research, these models represent leading techniques the anomaly detection literature. The DeepOD package provides comprehensive implementations of these methods within a unified testbed, enabling consistent evaluation. Each model was tested using the default hyperparameter settings provided by DeepOD, ensuring comparability across experiments. All models were implemented without incorporating contextual considerations for anomaly detection.
In addition to the DeepOD-based models, we include the DTE Livernoche et al. (2023) model, implemented from the available paper details. As one of the most recent diffusion- based approaches to anomaly detection, DTE introduces fundamentally different architecture that broadens the diversity of our baseline set. We also evaluate the WAE Tolstikhin et al. (2019) model a non-contextual bascline. WAE shares the same architecture our proposed CWAE but treats all features uniformly as content, without distinguishing contextual variables, making it strong reference point for isolating the effects of contextual modeling.
6.2 Performance Evaluation
In this section, present the CWAE results obtained using the selection method described in Section 5.3. Tables 3 and 4 report the corresponding AUCROC scores and their rankings, respectively.
CWAE achieves top overall performance. We evaluate model performance using two metrics: AUCROC and average rank. Ranks are computed per dataset, then averaged across all datasets to offer balancod view that reduces the influence of outliers. CWAE attains the best overall performance, with the highest average AUCROC (0.797) and the lowest average rank (2.75) among all models.
14

---

Published in Transactions on Machine Learning Research (03/2026)
Datazet
Scaled Couplexity
DSVDD
RDP
RCA
ICL
DIF
SLAD
DTE
WAE
CWAE
Context
bank
0.497
0.455
0.552
0.619
0.518
0.580
0.461
0.582
0.651
0.68I
losa
beth
0.312
0.885
0.308
0.907
0.963
0.988
0.996
0.990
0.996
0.906
cenens
0.335
0.443
1,629
0.701
0.637
0.669
0.673
0.410
0.676
0.743
detailed oceappttion__ane rocode
0.760
0.530
0.713
0.702
Hushands education
kdd
0.219
0.615
0.921
0.530
0.501
0.928
0.939
0
0.674
0.639
is guest login
1.000
Subject Logonil
0:250
0.579
0.811
0..00)
0.731
0.827
0.267
0.731
0.677
0.895
C-oless flores productioo by this region loudness
spotity
0.062
0.400
0.537
0.501
0.585
0.450
0.431
0.697
0.665
AVG
0.625
0.749
0.769
0.682
0.758
0.696
0.717
0.700
0.797
Table 3: AUCROC scores, scaled complexity scores, and the selected context feature for all models across datasets. Bold and underlined values indicate the best AUCROC scores.
Dataset
DSVDD
RDP
RCA
ICL
DIF
SLAD
DTE
WAF
CWAF
lank
3
both
9
2
3
1
4
census
8
7
2
3
8
1
5
4
kdd
8
3
5
1
4
6
lani
4
sf
8
3
4
9
5
7
spotify
9
AVG
7.375
4.75
4.125
5.375
5.25
1.025
3.875
2.75
Table 4: Model ranks for all datasets. Bold and underlined values indicate the best (lowest) rank.
Contextual learning improves over non-contextual baselines. CWAE consistently outperforms its non-contextual counterpart WAE, which shares the same architecture, loss function, and training procedure but omits contextual conditioning. This comparison isolates the effect of conditioning itself, independent of how the context feature is chosen. Across all eight datasets, CWAE improves the average AUCROC from 0.760 (WAE) to 0.797, thereby demonstrating the benefit of conditional modeling per Compared to other leading non-contextual baselines of different architectures RCA (0.769, 4.125), DIF (0.758, 5.375), RDP (0.749, 4.75), and DIE (0.717, 4.625) CWAE remains the strongest overall.
CWAE excels across datasets of varying complexity. A dataset-wise breakdown reveals CWAB's flexibility and robustness:
spotify (high complexity): WAE achieves the best AUCROC (0.697), but CWAE competitive 0.665 and outperforms several baselines, including ICL (0.585) and RDP (0.537). This illustrates contextual modeling's effectiveness in challenging data regimes.
bank and census (moderate complexity): CWAE attains the top AUCROC on both datasets (0.687 and 0.743), surpassing RCA and WAE. This shows strong generalization to structured real-world tabular data.
lanl (low complexity): CWAE attains near-perfect AUCROC of 0.999, just behind DSVDD, ICL, and DTE at 1.0, showing that generalization is not limited to high-complexity regimos.
beth (high cardinality): On this large and high-cardinality datasot, CWAE (0.996) matches WAE and trails only slightly behind DTE (0.999), demonstrating scalability.
Consistency distinguishes CWAE from other models. CWAE ranks first or second on five of the eight datasets, spanning spectrum of complexity levels. In contrast, models like RCA, SLAD, or DTE achieve top results only on one or two datasets and suffer from inconsistency elsewhere, as reflected by their higher average ranks.
These results affirm the core message of this work: context matters. In addition, we observed no systematic degradation or instability when evaluating samples from rare or unseen contexts, with performance approaching that of the non-contextual baseline expected. When paired with effective context. selection, CWAE
15

---

Published in Transactions on Machine Learning Research (03/2026)
demonstrates scalable, generalizable, and state-of-the anomaly detection performance. The observed gains are consistent. across variety of domains and not limited to any single dataset characteristic, underscoring the broader value of integrating contextual information.
7
Ablation Studies
7.1 Benefits of Contextual Thresholding
Contextual anomaly detection models allow for context-specific thresholds, which offer more nuanced decision boundary than a global threshold approach. This is particularly relevant in domains where the distribution of normal behavior varies significantly across groups defined by context features.
Figure visualizes this idea by plotting the thresholds learned by CWAE the bank dataset, conditioned on different context features. Each point corresponds to the threshold for given group within a context feature. These thresholds Hc. are obtained in the same manner described in Section 6.1.3.
Thresholds for cwae on bank
25
20
Thresholds
15
10
5
loan
marital
campaign
contact
day of week
default
education
housing
job
month
poutcome
Context
Figure 4: Thresholds for CWAE on the bank dataset, conditioned on different context features. Each point represents threshold learned under a specific context group.
The primary takeaway from this figure is that thresholds differ significantly across groups, confirming earlier claim that learning thresholds for each conditional distribution is desirable. This level of customization enables the model to detect context-dependent anomalies that may be missed under a single global threshold particularly such anomalies lie below the global cutoff but are relatively anomalous within their specific context.
It is important note that comparing raw threshold values across contexts or between contextual and non-contextual models is not strictly valid. The threshold for WAE corresponds to P(Y), while the thresholds for CWAE correspond to P(Y C). Since the marginal P(C) is not uniform across all datasets or features, and since scaling thresholds by P(C) would alter the decision boundary, no fair normalization is possible without further assumptions.
Thus, the purpose this visualization is not to support a direct comparison against WAE, but rather to illustrate that contextual models naturally learn different thresholds for different conditions. This leads a more flexible and adaptive detection policy, consistent with the goals of contextual anomaly detection.
16

---

Published in Transactions on Machine Learning Research (03/2026)
7.2 Upper Bound Performance with Optimal Context
In addition to evaluating CWAE under algorithmically selected context features, we assess its performance in the optimal context setting whernn the context feature yielding the highest AUCROC score for each dataset is selected. This analysis reveals the potential upper bound of CWAE when context is idcally chosen. Table 5 presents the comparison where CWAE Best indicates the optimal context choices.
Dataset
Best SOTA
WAE
CWAE
CWAE Best
CWAE Context
CWAE Best Context
AUCROC
Rank
AUCROC
Rank
AUCROC
Rank
AUCROC
Rank
bank
0.6.0
0.834
3
0.687
0.895
lenr no etc detailed compatton rocode
education
bech
0.900
0.086
5
0.500)
(.067
arge/aam
ceusus
0.701
0.676
0.741
0.743
detalled_ occupatioa recode
0.70)
0,702
0.761
0.787
Huslanbe Huslannk eclacation lagin
Cuntropptin mathed wnd
kdd
0.9:00
0.678
0.603
01877
service
lazi
0,909
0.500
SubjoctLquulD
AutlautiintiaParaan
production by this mejun
Xethone Fanor prsdustion this meloon
spotity
03265
0.057
0,003
3.25
Table 5: Comparison of AUCROC scorcs and ranks for WAE, CWAE, CWAE Best, and the best SOTA model per dataset. Bold and underlined values indicate the best AUCROC and best (lowest) rank.
Optimal context significantly boosts CWAE performance. On average, CWAE Best achieves an AUCROC of 0.839, outperforming the standard CWAE score of 0.797. It ranks first on 6 of the 8 datasets, compared to only 3 for the standard CWAE. This gap underscores the potential benefit of improved learned context selection methods.
CWAE Best consistently outperforms the non-contextual baseline. Across all datasets, CWAE Best yields higher AUCROC scores than the non-contextual WAF baseline, often by substantial margins. For instance, on the eme dataset, CWAE Best achieves 0.787 compared to WAE's 0.702, and on kdd, it reaches 0.872 versus WAE's 0.674. These improvements reinforce the core hypothesis of this work: that modeling context is critical component for effective anomaly detection.
CWAE Best surpasses all SOTA models on most datasets. We aggregate the best scores from all individual baselines into a Best SOTA column to establish a performance reference. CWAE Best outperforms the Best SOTA model on out of datasets, including notable gains such as +0.046 on bank; +0.027 on emc, and +0.079 on sf. On average, CWAE Best ranks first with a rank of 1.25, compared to Best SOTA's 2.375. Importantly, CWAE achieves this level of performance using single unified archilecture across all datasets, underscoring its value as flexible, general-purpoe anomaly detection model.
CWAE Best performs well across the full range of datasct complexitics. Whether the dataset is low-complexity (e.g., lanl) or high-complexity (e.g., spotify), CWAE Best consistently delivers strong results. This suggests that with appropriate context, CWAE is broadly applicable across varied data regimes.
Informative context selection drives performance gains. The comparison between CWAE and CWAE Best isolates the benefit attributable specifically to selecting informative context features. Because CWAF Best emumerates all candidate context features for each dataset, it implicitly spans both informative and weak (or effectively uninformative) contexts. Empirically, poorly chosen contexts regress toward the non-contextual WAF baseline, while informative contexts yield substantial gains. As an raluse consequence of the above, randomly selecting a context fcature would, in expectation, fall within the same obscrved performance range, without introducing behavior beyond what is already captured by exhaustive enuneration. We therefore treat CWAE Best as an upper bound on context selection quality and the SWAE-CWAL Best gap direct measure of the value of improved context selection.
These results further support the importance of context selection. When given the optimal context, CWAE matches or surpasses state-of-the-art performance, demonstrating the potential of context-aware approaches. This suggests that future rescarch should focus on improving context selection methods, including the development of learnable strategies that can consistently achieve performance close to this upper bound in practice.
17

---

Published in Transactions on Machine Learning Research (03/2026)
7.3 Quantifying the Value of Context
A central question in this research is whether incorporaling contextual information improves anomaly detection performance not only across datasets of varying complexity, but also different model architectures. To invostigate, we ran additional experiments with other conditional models from the AD litcrature. The selected models are listed below:
Conditional Wasserstein Autoencoder (CWAE), described in Section 5.1.
Conditional Variational Autocncoder (CVAE) Pol et al. (2020), which models the conditional distribution Po C) of observations given contextual variables using a deep generative model with learned fcature-wise reconstruction variance.
Conditional Mixture Density Network (CMDN) Dai et al. (2025), which learns a neural network-parre parrmmeeiar Gaussian mixture model to capture P(Y C), enabling mixture parameters to adapt dynamically based on context.
Contextual Anomaly Detection using Isolation Forest (CADI) Yepmo el al. (2024), which extends the isolation forest algorithm with density-aware splits to jointly detect anomalies and explain them relative to local contextual clusters.
All models are implemented with only minor modification ions from their original formulations, using the same embedding layers outlined in Section 5.1 (see Appendix A for more details). For each model-dataset pair, we systematically iterated over all candidate features as potential context, designating exactly one feature as context in each run and treating the remainder as content. Performance was compared to a no-contezt baseline in which all features are treated content. This controlled setting enables direct, interpretable measure of how both context and model architecture contribute to performance improvements.
To quantify the benefits of contextual learning, we report the average improvement in AUCROC across datasets for each model, comparing the best-perfrmming context configuration to its non-contextual counterpart. All contextual models outperform their non-contextual basclines on average, often by a substantial margin. The CWAE model achieves the highest average improvement of +11.69%, followed by CMDN (+7.47%), CVAE (+6.21%), and CADI (+4.18%). These consistent gains support the hypothesis that leveraging context enhances model's ability to detect anomalies across architectures, especially when the contextual signal is informative.
Given its strong empirical performance, we selected CWAE as the primary model for further analysis and methodological development. We treat CWAE as a representative upper bound for context-awaro models in our framework, using it as a proxy to explore context selection strategies and to benchmark the overall value of contextual information in anomaly detection.
8 Conclusion
This work establishes conteziual learning a general paradigm for anomaly detection in tabular data. The proposed framework provides unified probabilistic formulation, a principled context selection strategy, and a theoretical foundation for modeling conditional distributions P(Y C). Through variance decomposition and discriminative learning principles, we show that conditioning effectively isolates intra-context variability while mitigating noise from inter-context differences, leading more robust and precise anomaly detection.
We instantiate this paradigm through CWAE, lightweight, generative model that operationalizes the contextual learning framework for tabular domains. Across diverse datasets spanning finance, cybersecurity, demographics, and network intrusion detection, the CWAE instantiation consistently outperforms both its non-contextual baseline (WAE) and SOTA unconditional models, demonstrating the practical effectiveness and generality of contextual learning.
Future rescarch will focus on extending contextual learning to multi-context conditioning, developing learnable context discovery mechanisms, and applying the framework other data modalities such as text, time series, and multimodal environments.
18

---

Published in Transactions on Machine Learning Research (03/2026)
9 Broader Impact and Limitations
This work proposes a contextual learning framework for unsupervised anomaly detection in tabular data, with primary intended applications in areas such as cybersecurity, fraud detection, and system monitoring. By modeling conditional distributions rather than a single global data distribution, the approach has the potential to improve detection accuracy in heterogeneous environments. At the same time, the use of context-aware anomaly dctection raises several broader impact considerations that merit discussion.
Potential for Bias and Context Selection. A central component of the proposed framework is the sclection of context features that condition the Icarned notion of normal behavior. While contextual modcling can reduce certain forms of bias by avoiding single global decision boundary, it also introduces the risk that chosen context features may correlate with sensitive attributes such as race, gender, age, or socioeconomic status. In such cases, the model may learn different anomaly score distributions or thresholds across demographic groups, potentially resulting in disparate false positive or false negative rates. This risk is not unique to the proposed method but is particularly salient because context explicitly influences decision boundaries. Practitioners deploying contextual anomaly detection systems should therefore carefully audit candidate context features, assess correlations with sensitive attributes, and evaluate group-wise wise performance to identify unintended discriminatory effects.
Dual-Use Considcrations. Like most advances in anomaly detection, the proposed framework has dual-use potential. Improved anomaly detection can yield clear societal benefits when applied to domains such fraud prevention, netw security, and system reliability. However, the same techniques could also be applied in ways that raise ethical concerns, for example in large-scale surveillance, behavioral monitoring, or profiling without appropriate safeguards. This work does not prescribe specific deployment contexts, and responsible use ultimately depends on organizational policies, legal frameworks, and oversight mechanisms governing how anomaly detection systems are applied.
Data Use, Transparency, and Reproducibility. From an ethical research practice perspeclive, this work relies exclusively on publicly available benchmark datasets that have been widely used in prior anomaly detection research, and no proprietary or personally identifiable data are introduced. Implementation details, experimental settings, and additional analyses are provided in the appendix to support reproducibility and facilitate independent verification. While reproducibility does not by itself guarantee ethical deployment, transparency in data usage and evaluation is an important prerequisite for responsible research and for enabling follow-up work that examines robustness, bias, and broader impacts more closely.
Behavior Under Rare Contexts. For rare context values with limited support (e.g., fewer than 10 samples), the model may not reliably learn context-specific structure. In these cases, contextual thresholds are estimated conservatively through the normalization procedure described in Section 6.1.3. Because anomaly scores are normalized relative to per-context training maxima, rare contexts do not induce arbitrarily tight decision boundaries. As a result, the detector relies more heavily on content-level structure rather than memorizing small context-specific patterns, reducing the risk of overfitting in low-support regimes but also limiting the benefits of contextual conditioning when data are sparse.
Sensitivity to Validation Data Quality. The proposed context selection procedure relies on a small validation subset to rank candidate context features based on early-stage generalization performance. This approach assumes that the validation data predominantly representative of normal operating conditions, which is standard in unsupervised anomaly detection but may not hold in all deployment scenarios. When the validation set is heavily contaminated with anomalies, validation-based context ranking may become less reliable, potentially leading to uninformative or even detrimental context selection. Addressing such settings may require Dontamination-awan or weakly supervised extensions that explicitly account for noise or anomalous samples in the validation data. Developing context selection strategies that remain robust under substantial validation-set contamination is an important direction for future work.
Overall, this work is intended as a methodological contribution to anomaly detection research. The con- siderations outlined above highlight the importance of careful context selection, auditing, and responsible
19

---

Published in Transactions on Machine Learning Research (03/2026)
deployment when applying contextual anomaly detection methods in real-world settings, particularly those involving human-centered or high-stakes decisions.
References
Solar flare [dataset], 1989. URL
Kdd cup 1999 data, 1999. URL
ics weaeddiiss...eeeuie htt.a./....///atttteee/////////p .tmll. Dataset.
Census-income (kdd); 2000.
Alessandro Achille, Matteo Rovere, and Stefano Soatto. Critical learning periods in doep neural networks. arXiv preprint arXix:1711.0885 2017.
Charu C. Aggarwal. Oullier Analysis. Springer, 2017.
Charu C Aggarwal and Philip S Yu. Outlier detection in high dimensional data. ACM Computing Surveys (CSUR), 45(4):1- 50, 2013.
Mohammad Ruhul Amin, Pranav Garg, and Baris Coskun. Cadence: Conditional anomaly detection for events using noise-contrastive estimation. In Proceedings of the 12th ACM Workshop on Arlificial Intelligence and Security, pp. 73 84, 2019. doi: 10.1145, 10.1145/333850L /333333013335736663
Gabriel Bender, Pieter-Jan Kindermans, Barret Zoph, Vijay Vasudevan, and Quoe Le. Understanding and simplifying one-shot architecture search. In International conference machine learning, pp. 550-559. PMLR, 2018.
Liron Bergman and Yedid Hoshen. Classification-bass anomaly detection for general data. arXiv preprint arXiv:2005.0253 2020.
Tobias Biegel, Nicolas Jourdan, Carlos Hernandez, Amir Cviko, and Joachim Metternich. Deep learning for multivariate statistical in-process control in discrete manufacturing: case study sheet metal forming process. Procedia CIRP, 107:422 427, 2022. ISSN 2212-8271. doi: https://doi.org 10.1100///100ooooo/// procir 2022. 05.002. URL https://wwes sciencedirect. wwwpc/ciinc///eeeeecceceeceeeeee http//cceeetcccc//cccccce//////e httpe///ttccc/////////////222//0 manufacturing systems transformation Proceedings of the 55th CIRP Conference on Manufacturing Systems 2022.
Leading
Christopher M. Bishop. Paltern Recognationn and Machine Learning. Springer, 2006.
Varun Chandola, Arindam Banerjee, and Vipin Kumar. Anomaly detection: A survey. ACM Computing Surveys (CSUR), (3):1-58, 2009.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations, In International conference on machine learning, pp. 1597 1607. PMLR, 2020.
Priyam Choksi. 114000 spotify songs.
spotify-dataset- dat-dddtttsttta titieet-114: 114k teaett-11kkssonssgg songs, 2021.
Can Cui, Yaohong Wang, Shunxing Bao, Yucheng Tang, Ruining Deng, Lucas W. Remedios, Zuhayr Asad, Joseph T. Roland, Ken S. Lau, Qi Liu, Lori A. Coburn, Keith T. Wilson, Bennett A. Landman, and Yuankai Huo. Feasibility of universal anomaly detection without knowing the abnormality in medical images. In Medical Imaging with Deep Learning Short Papers, pp. 82 92, 2023. doi: 10.1077997---031-44477- 8.
La Dai, Wenxuan Zhu, Xuchui Quan, Yichen Wang, Sheng Chai, and Renzi Meng. Deep probabilistic modeling of user behavior for anomaly detection via mixture density networks. arXiv preprint arXiv:2505.00 082200 2025.
Jonathan Frankle, David Schwab, and Ari S Morcos. The early phase of neural network training. arXiv preprint Xrxi::200.. 10365, 2020.
20

---

Published in Transactions on Machine Learning Research (03/2026)
Clement Fung, Chen Qiu, Aodong Li, and Maja Rudolph. Model selection of anomaly detectors in the absence of labeled validation data. arXiv preprint arXiv:2310. 2024. URT. https://arxiv..rr http://aaxivv..o/bb.aaba/a// httggabb/23300 2210. 10461.
Izhak Golan and Ran EI- Yaniv. Deep anomaly detection using geometric transformations. Advances in neural information processing systems, 31, 2018.
Dong Gong, Linggiao Liu, Vuong Le, Budhaditya Saha, Moussa Reda Mansour, Svetha Venkatesh, and Anton den Hengel. Memorizing normality to detect anomaly: unsupervised anomaly detection. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 1703-1714, 2019.
deep autoencoder for
Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. kernel two-sample test. The Journal of Machine Learning Research, 13(1):723-777 2012.
Denis Gudovskiy, Shun Ishizaka, and Kazuki Kozuka. Cflow-ad: Real-time unsupervised anomaly detection with localization via conditional normalizing flows. In Proceedings of the IEEE/CVF winter conference applications of compuler vision, pp. 98 107, 2022.
Songqiao Han, Xiyang Hu, Hailiang Huang, Minqi Jiang, and Yue Zhao. Adbench: Anomaly delection benchmark. Advances in Neural Information Processing Systems, 35:32142 15:32142-32159, 2022.
Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The Flements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, New York, NY, 2nd edition, 2009. ISBN 978-0-387-88857-0 doi:
Kate Highnam, Kai Arulkumaran, Zachary Hanif, and Nicholas R Jennings. Beth dataset: Real cybersecurity data for unsupervised anomaly detection research. In CEUR Workshop Proc, volume 3095, pp. 1-12, 2021.
Harold Hotelling. Multivariate quality control, illustrated by the air testing of sample bombsights. In Churchill Eisenhart, M. W. Hastay, and W. A. Wallis (eds.), Selected Techniques of Statistical Analysis for Scientific and Industrial Research, and Production and Monagement Engineering, chapter pp. 111-184. McGraw-Hill Book Company, New York, 1947.
J. Edward Jackson and Govind S. Mudholkar. Control procedures for residuals associated with principal component analysis. Technometrics, 21(3):341-349, 1979. doi:
10489779.
Minqi Jiang, Chaochuan Hou, Ao Zheng, Xiyang Hu, Songqiao Han, Hailiang Huang, Xiangnan He, Philip S.
Yu, and Yue Zhao. Weakly supervised anomaly detection: A survey, 2023. URL https: httpp::///xxvx.oggg..// org/abs/
Minkyung Kim, Jongmin Yu, Junsik Kim, Tac-Hyun Oh, and Jun Kyun Choi. An iterative method for unsupervised robust anomaly detection under data contamination. arXiv preprint ar Xiw:2309.09436 2023.
URL https://arxiv: http:///axxvv..ogbbbbo/// 99943//0/.///b//0009993... 09436.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014. URL https://arxis http:::://xxxvoor//aa//// orgg/bs/////1//
Polina Kirichenko, Pavel Izmailov, and Andrew G Wilson. Why normalizing flows fail to detect out-of- distribution data. Advances in neural information processing systems, 2225778-25888 2020.
Hans-Peter Kriegel, Peer Kröger, and Arthur Zimek. Outlier detection in axis-parallel subspaces of high dimensional data. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp. 831-838. Springer, 2009.
Max Landauer, Sebastian Onder, Florian Skopik, and Markus Wurzenberger. Deep learning for anomaly detection in log data: A survey. Machine Learning with Applications, 12:1 12:100470, 2023.
21

---

Published in Transactions on Machine Learning Research (03/2026)
Julia A. Lasserre, Christopher M. Bishop, and Michael I. Jordan. A hybrid to modeling sequence data. TEEE Transactions on Pallern Analysis and Machine Intelligence, 29(4): 747 761, 2006.
approach
Sainan Li, Qilei Yin, Guoliang Li, Qi Ti, Zhuotao Liu, and Jinwei Zhu. Unsupervised contextual anomaly detection for database systems. In Proceedings of the 2022 International Conference on Management of Dala, SIGMOD '22, pp. 788 802, 2022. ISBN 9781450392495. doi: 00.1145/3514221.3511
Percy Liang and Michael Jordan. An asymptotic analysis of generative, discriminative, and pseudolikelihood estimators. In Proceedings of the 25th International Conference on Machine Learning (ICML), pp- 584-591. ACM, 2008.
Yihua Liao and Rao Vemuri. Use of k-nearest neighbor classifier for intrusion detection. Computers & security, 21(5):439-44 448, 2002.
Tjen-Sien Lim. Contraceptive method choice [dataset], 1999. URL httpsttt//arrhiee...tt wcwp::chiiee.ee.uuccc. www.ucc..ee..... wcc.eed/datasee/ 30.
Boyang Liu, Ding Wang, Kaixiang Lin, Pang-Ning Tan, and Jiayu Zhou. Rea: A deep collaborative autoencoder approach for anomaly detection. In IJCA:: proceedings of the conference, volume 2021, pp. 1505. NIH Public Access, 2021.
Hanxiao Liu, Karen Simonyan, and Yiming Yang. DARTS: Differentiable architecture search, In Interna- tional Conference on Learning Representations (ICLR), 2019. URL https://openrreviee.nn wet..rouod..ttt/tdddddd S1eYHoC5FX.
Sheng Liu, Jonathan Weed, Narges Razavian, and Carlos Sernandez-Grande Early-learning regular- ization prevents memorization of noisy labels. Advances in neural information processing systems, 33: 2020.
Victor Livernoche, Vineet Jain, Yashar Hezaveh, and Siamak Ravanbakhsh. On diffussio modeling for anomaly detection. arXiv preprint orXiv:2305118593, 2023. doi: 001550/arXiv.23051115. URL https://arxiv, org/abs//3// http:://arxxv.orr//as///5.. 18593.
Prasanta Chandra Mahalanobis. On the generalized distance in statistics. Proceedings of the National Institute of Sciences of India, 1936.
Douglas C. Montgomery. Introduction to Statistical Quality Control. Wiley, Hoboken, NJ, edition, 2012.
Alex Moore and Davide Morelli. conDENSE: Conditional density estimation for time series anomaly detection. Journal of Artificial Intelligence Reseurch, 79:801 824, 2024. doi: 1..1133/jair..114499
S. Moro, P. Rita, and P. Cortez. Bank marketing, 2012. URL https://doi. htt:://oo..og////// org/10. 21.22443//5K366 Published by UCI Machine Learning Repository.
Gerhard Münz, Sa Li, and Georg Carle. Traffic anomaly detection using k-means clustering. In Gi/ilg workshop mmbnet, volume 7, 2007.
Kevin P. Murphy. Machine Learning: Probabilistic Perspective, MIT Press, 2012.
Andrew Ng and Michael Jordan. On discriminative vs. generative classifiers: A comparison of logistic regression and naive bayes. In T. Dietterich, S. Becker, and Z. Ghahramani (eds.), Advances in Neural Information Processing Systems, volume 14. MIT Press, 2001. URL https://procceddnnn http://rrooeedd.nn..........p. cc/ Paper.
paper_
Guansong Pang, Longbing Cao, and Ling Chen. Homophily outlier detection in non-iid categorical data. Data Mining and Knowledge Discovery, 35(4):1163-122) 2021a.
Guansong Pang, Chunhua Shen, Longbing Cao, and Anton Van Den Hengel. Deep learning for anomaly detection: A review. ACM compuling surveys (CSUR), 14(2):1-38, 2021b.
22

---

Published in Transactions on Machine Learning Research (03/2026)
Mansheej Paul, Surya Ganguli, and Gintare Karolina Dziugaite. Deep learning on data diet: Finding important examples early in training. Advances in neural information processing systems, 14:20596-20611 2021.
Adrian Alan Pol, Victor Berger, Gianluca Cerminara, Cecile Germain, and Maurizio Pierini. Anomaly detection with conditional variational autoencoders. arXiv preprint wxXx:2010.05550 2020.
Jinchuan Qian, Zhihuan Song, Yuan Yao, Zheren Zhu, and Xinmin Zhang. A review on autoencoder based representation learning for fault detection and diagnosis in industrial processes. Chemometrics and Intelligent Laboratory Systems, 231:104711, 2022. ISSN 0169-7439. doi:
URL https: /ttp///wwww///// htt.://ww.ccccccccoooooooocoaaaa
Chen Qiu, Timo Pfrommer, Marius Kloft, Stephan Mandt, and Maja Rudolph. Neural transformation learning for deep anomaly detection beyond images. In International conference on machine learning, pp. 8703-877 8714. PMLR, 2021.
Lukas Ruff, Robert Vandermenlen, Nico Goernitz, Lucas Deecke, Shoaib Ahmed Siddiqui, Alexander Binder, Emmanuel Müller, and Marius Kloft. Deep one-class classification. In International conference on machine learning, pp. 4393 4402. PMLR, 2018.
Lukas Ruff, Jacob R Kauffmann, Robert A Vandermeulen, Grégoire Montavon, Wojeiech Samek, Marius Kloft, Thomas G Dietterich, and Klaus-Robert Müller. A unifying review of deep and shallow anomaly detection. Proceedings of the IEEE, 109(5):756-799) 2021.
Mikael Sabuhi, Ming Zhou, Cor-Paul Bezemer, and Petr Musilek. Applications of generative adversarial networks in anomaly detection: a systematic literature review. Teee Access, 2261003-1610222 2021.
Saurabh Sathe and Charu C Aggarwal. Subspace outlier detection in linear time with randomized hashing. In Proceedings of the 25th International Conference on World Wide Web, pp. 401-411. International World Wide Web Conferences Steering Committee, 2016.
Thomas Schlegl, Philipp Seeböck, Sebastian M Waldstein, Ursula Schmidt-Erfurth, and Georg Langs. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In International conference on information processing in medical imaging, pp. 146-157. Springer, 2017.
Tom Shenkar and Lior Wolf. Anomaly detection for tabular data with internal contrastive learning. In International conference on learning representations, 2022.
Walter A. Shewhart. Economic Control of Quality of Manufactured Product. D. Van Nostrand Company, Inc., New York, 1931.
Yaniv Shulman. Unsupervised contextual anomaly detection using joint deep variational generative models. arXiv preprint vrxXi:1104.000 00548, 2019. URL https://arxiv.. abaaaaas///0 005488.
Mei-Ling Shyu, Shu-Ching Chen, Kanoksri Sarinnapakorn, and LiWu Chang, A novel anomaly delection scheme based on principal component classifier, In Proceedings of the IEEE foundations and new directions of data mining workshop, pp. 172-179. IEFE Press Piscataway, NJ, USA, 2003.
Ankur Sinha, Pekka Malo, and Kalyanmoy Deb. A review of bilevel optimization: From classical to evolutionary approaches and applications. IEEE Transactions on Evolutionary Computation, 22(2):276-295, 2018.
Kihyuk Sohn, Honglak Lee, and Xinchen Yan. Learning structured output representation using deep conditional generative models. In C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett (eds.), Advonces in Neu- ral Information Processing Systems, volume 28. Curran Associates, Inc., 2015. URL https: http://///pp/.......... seurips.cc/pappe
Paper. .ddf.
Xiuyao Song, Mingxi Wu, Christopher Jermaine, and Sanjay Ranka. Conditional anomaly detection. IEEE Transactions knowledge and Data Engineering, 19(5):631166) 645, 2007.
23

---

Published in Transactions on Machine Learning Research (03/2026)
Jihoon Tack, Sangwoo Mo, Jongheon Jeong, and Jinwoo Shin. Csi: Novelty detection via contrastive learning on distributionally shifted instances. Advances in neural information processing systems, 11852, 2020.
David M. J. Tax and Robert P. W. Duin. Support vector data description. Machine Learning, 2004. doi: A..00//BBMACH..0000008B..AAL 1,8411190000000068114449,,,, URL M.t.://0......r.....00./....... 60811.49.
llya Tolstikhin, Olivier Bousquet, Sylvain Gelly, and Bernhard Schoelkopf. Wasserstein auto-encoders, 2019.
URL http://arxiv, wwt..rrgg//a.... ab////1s1111.... 1711.01558.. 01558. Number: arXiv:1711.01558.
Melissa JM Turcotte, Alexander D Kent, and Curtis Hash. Unified host and network data set. In Data science for cyber-security, pp. 1-22. World Scientific, 2019.
Vladimir N. Vapnik. The nature of slatistical learning theory. Springer-Verlag New York, Inc. 1995.
Hu Wang, Guansong Pang, Chunhua Shen, and Congbo Ma. Unsupervised representation learning by predicting random distances. arXiv preprint or Xrxiv: 12186, 2019a.
Siqi Wang, Yijie Zeng, Xinwang Liu, En Zhu, Jianping Yin, Chuanfu Xu, and Marius Kloft. Effective end-to-end unsupervised outlier detection via inlier priority of discriminative network. Advances in neurnl information processing systems, 32, 2019b.
Shuang Wu, Jingyu Zhao, and Guangjian Tian. Understanding and mitigating data contamination deep anomaly detection: A kernel-based approach. In Proceedings of the Thirty-First International Joint Conference Artificial Intelligence (IJCAI-22), pp. 2319 2325. International Joint Conferences Artificial Intelligence Organization, 2022. doi: 10.24963 110/06///caii202222 2022/322.
URL https: //tp:i///oii.e 10. 1rg/1..24933// 24963/1i6jca
Hongzuo Xu. xuhongauo/DeepO1 URL http:::/gggaub...c.///ooooogg/gg original-date: 9022-10- 28:09:45:592.
Hongzuo Xu, Guansong Pang, Yijic Wang, and Yongjun Wang. Deep isolation forest for anomaly detection. IEEE Transactions on Knowledge and Data Engineering, 35(1)):12591 12604, 2023a.
Hongzuo Xu, Yijie Wang, Juhui Wei, Songlei Jian, Yizhou Li, and Ning Liu. Fascinating supervisory signals and where to find them: Deep anomaly delection with scale learning. International Conference on Machine Learning, pp. 38655 38673. PMLR, 2023b.
Véronne Yepmo, Grégory Smits, Marie-Jeanne Lesot, and Olivier Pivert. Cadi: Contextual anomaly detection using an isolation forest. In Proceedings of the ACM Symposium on Applied Computing (SAC), pp. 4:1-4:10. ACM, 2024.
Haoran You, Chaojian Li, Pengfei Xu, Yonggan Fu, Yue Wang, Xiaohan Chen, Zhangyang Wang, Richard G Baraniuk, and Yingyan Lin. Drawing early-bird tickets: Towards more efficient training of deep networks. In International Conference on Learning Representations 2020 (ICLR 2020), 2020.
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. arXiv preprint orXic:1611.0855) 2016.
Lily Zhang, Mark Goldstein, and Rajesh Ranganath. Understanding failures in out-of-distribution detection with deep generative models. In International Conference Machine Learning, pp. 2021.
PMLR,
Zhilu Zhang and Mert Sabuneu. Generalized cross entropy loss for training deep neural networks with noisy labels. Advances in neural information processing systems, 31, 2018.
Yiru Zhao, Bing Deng, Chen Shen, Yao Liu, Hongtao Lu, and Xian-Sheng Hua. Spatio-temporal autoencoder for video anomaly detection. In Proceedings of the 25th ACM international conference on Multimedia, pp. 1933-1945 2017.
24

---

Published in Transactions on Machine Learning Research (03/2026)
Chong Zhou and Randy C Paffenroth. Anomaly detection with robust deep autoencoders. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 665 665-674, 2017.
A
Appendix
This appendix provides comprehensive implementation and evaluation details to help facilitate the repro- ducibility our experimental results.
A.1 Model Architectures
CWAE Architecture Details following architecture:
The Conditional Wasserstein Autoencoder (CWAE) implements the
Embedding Layers:
Type: Trainable embedding layers initialized randomly
Dimension: 16-dimensional embeddings for categorical features
Vocabulary Size: Determined by unique values in training data for each feature, plus for handling unknown values at test time
Unknown Value Handling: Reserved index initialized maximum values across embedding dimensions
Encoder:
Input Layer: Context embeddings (R") concatenated with content embeddings (R*-s)
Hidden Layer 1: Linear transformation from (c + d - c) to 128 dimensions with ReLU activation
Hidden Layer 2: Linear transformation from 128 to 64 dimensions with ReLU activation
Latent Layer: Linear transformation from 64 to Adim dimensions, where 7dim = 64 by default
Regularization: No bias terms used linear layers
Decoder: For each content feature E [l....,d-cl:
Input: Latent representation z concatenated with context embeddings, dimension Re
Hidden Layer 1: Linear transformation from (zaim + c) to 64 dimensions with ReLU activation
Hidden Layer 2: Linear transformation from 64 128 dimensions with ReLU activation
Output Layer: Linear transformation from 128 to vocabulary size for feature i, producing logits for
cross-entropy loss computation
Regularization: No bias terms used in linear layers
CVAE Architecture Details
The Conditional Variational Autocncoder extends CWAE with probabilistic
latent variables:
Encoder:
Same architecture as CWAE encoder, but outputs both mean and log-variance log o²
for the latent distribution
Latent Sampling: Reparameterization trick used: 2 = 4 & where ~ N(0,
Decoder: Identical architocture CWAE decoder
KL Divergence: Computed D (),)())))))))) Ejer (1 + log 03 12 of)
25

---

Published in Transactions on Machine Learning Research (03/2026)
CMDN Architecture Details
The Conditional Mixture Density Network models output distributions
Gaussian mixtures:
Encoder: Similar architocture to CWAE with context conditioning
Output: Predicts mixture parameters (T&, Mks σk) for K mixture components
Mixture Weights: Normalized using softmax: Th
explas)
Σラ exp(ay)
Loss Function: Negative log-likelihood of the Gaussian mixture: -log E TaNNaal (uke o
A.2 Training Hyperparameters
Default Training Configuration shown in Table 6.
All models use consistent hyperparameters across experiments. This is
Parameter
Value
Learning Rate
0.001
Optimizer
Adam (31 0.9, A2 = 0.999, e 10-*)
Batch Size
2048
Maximum Epochs
25
Early Stopping Threshold
0.0001 (on training loss)
Weight Decay
0 (no 1,2 regularization)
Gradient Clipping
None
Random Seed
43
Table 6: Default training hyperparameters used across all experiments.
Model-Specific Adjustments
CWAE: MMD regularization weight 1.0
CVAE: KL divergence weight / 1.0 (standard VAE formulation)
CMDN: Early stopping threshold -20.0 (adapted to negative log- lig-liieldhood scale)
CADI: Uses 100 isolation trees with automatic contamination parameter selection
A.3
Computational Environment
Hardware Specifications Experiments were conducted on the following hardware:
GPU: NVIDIA A100 GPU (40GB memory) with CUDA 12.4 support
CPU: Multi-core processor for parallel data loading (4 workers)
RAM:: Sufficient for batch size of 2048 across all datasets
Storage: SSD storage for efficient data 1/O operations
Software Dependencies Experiments were conducted with the following software dependencies. These are shown in Table
26

---

Published in Transactions on Machine Learning Research (03/2026)
Package
Version
Python
3.8+
PyTorch
2.6.0+-cu24
CUDA
12.4
cuDNN
9.1.0.70
NumPy 2.0.2
Pandas
2.3.0
Scikit-learn
1.6.1
Matplotlib
3.9.4
Seaborn
0.13.2
Table 7: Software dependencios and versions used experiments.
A.4 Joint Validation Loss Computation
To enable fair comparison between joint models (learning p(r, v)) and conditional models (learning p(rly)), we convert all models to evaluate on the joint probability p(z, y); as these distributions have different scales.
Motivation
Conversion Methodology For contextual models learning p(xly), we apply the probability chain rule: p(r, v) p(xly) p(y)
(2)
In log-space, this becomes:
-logp(z,5)) log logp(x') v) log ply)
(3)
Marginal Probability Estimation:
1. Estimate ply) empirically from training data: p(y c) count(e)
2. Compute log probabilities: log p(y c) log(count (c)) - log(N)
3. Handle unseen values: Use minimum observed probability fallback for rare categories
Sample-wise Correction: For each validation sample y:):
joint loss, = model loss; + (-
log b(yss)
(4)
/econtext
Batch-wise Average:
Joint Validation Loss
1 B
Σ
joint loss;
(5)
1=1
where B is the number of validation batches (limited to 10 batches for computational efficiency).
A.5
Datasets
A.5.1
Data Format
All datasets were preprocessed into a standardized format:
Anomaly Label: Binary column with values 0 (normal) and (anomaly)
Feature Encoding: All features encoded categorical integer indices
Missing Values: Handled via imputation or exclusion prior to experiments
27

---

Published in Transactions on Machine Learning Research (03/2026)
A.5.2 Feature Encoding
Categorical Encoding: All features mapped integer indices using label encoding
Index 0 Reserved: Designated for unknown unseen categorical values during inference
3. Vocabulary Construction: Built exclusively from training data; validation and test sets may contain previously unseen values
4. No Numerical Scaling: Not required as features are processed through trainable embedding layers
A.5.3 Context/Content Feature Assignment
For each dataset and context configuration:
Context Features: Selected feature(s) used for conditional modeling
2. Content Features: Remaining features to be reconstructed by the model
3. Column Organization: Features organized context features followed by content features
4. No-Context Baseline: All features treated content (empty context set)
A.5.4 /ainnValidationnTTest Splits
All experiments use fixed pre-determined data splits:
Training Set: Contains only normal samples (anomaly=0)
Validation Set: Contains only normal samples, used for context selection
Test Set: Contains both normal and anomalous samples for evaluation
Split Ratios: Approximately 60% train, 20% validation, 20% test
Split Strategy: Random but fixed across all experiments to ensure fair comparison
A.5.5
Reproducibility Measures
Fixed Random Seed: Seed value of 43 used for all random operations
Data Loading: Training data shuffled; validation and test data processed in fixed order
Consistent Splits: Same train/validation/tess partitions used across all model comparisons
A.6
Evaluation Metrics
A.6.1
Anomaly Scoring
For reconstruction-bon based models (CWAE, CVAE, CMDN), the anomaly score is computed as:
score c)
Σ CE 0.)
(6)
where CE denotes cross-entropy loss, y/ represents the true content feature value, is is the model's recon- struction, and l is the number of content features.
28

---

Published in Transactions on Machine Learning Research (03/2026)
A.6.2 Context-Specifie Thresholding
1. Training Threshold: For each context value c, compute maximum training score: He max (trre score(r,
Normalization: Normalize test scores relative training threshold: R₂(a) score(z,e)
He
3. Global Threshold Search: Grid search 100 uniformly-spaced thresholds in range [0, max (R.)]
4. AUC-ROC Computation: Use normalized ratios as continuous anomaly scores
A.6.3 Performance Metrics
AUC-ROC: Area under receiver operating characteristic curve
AUC-PR: Area under precision-recall curve
F1-Score: Harmonic mean of precision and recall at optimal threshold
Precision: TP4FP at optimal threshold
TP
Recall:
TP TP4FN at optimal threshold
Optimal threshold determined by maximizing F1-score on the validation set.
A.7
Context Selection Procedure
A.7.1
Bilevel Optimization Framework
Outer Loop (Context Selection):
Candidate Set: All individual features tested as single-feature contexts
Baseline: No-context configuration included for comparison
Evaluation Protocol: Train each candidate for epoch and compute joint validation loss
Inner Loop (Model Training):
Training Duration: Single epoch for context selection (early-epoch proxy)
Objective: Minimize training loss using standard hyperparameters
No Baldda Training: Validation set only used for evaluation, not for early stopping
during context selection
Selection Criterion:
c" = =rg min ((..))~Daall log p(x, 0/02
(7)
AEC
where 6g represents model parameters trained with context feature c, and C is the set of candidate context features.
29

---

Published in Transactions on Machine Learning Research (03/2026)
A.7.2 Early Epoch Justification
Spoch evaluation is motivated by:
Computational Efficiency: Reduces context selection time by approximately 96% (25x speedup compared to full training)
Relative Ranking: Early training dynamics strongly correlate with final performance rankings (Liu et al., 2019)
Empirical Validation: Figure demonstrates that the early-epoch proxy achieves better final performance 5 of datasets
Established Practice: Similar approaches used successfully in neural architecture search (Bender al., 2018), model pruning (You et al., 2020), and noisy label learning (Liu al., 2020)
A.8 Runtime Analysis
A.8.1
Training Time
Approximate training times per model/dataset combination (25 epochs NVIDIA A100 GPU) shown in
Table 8.
Dataset
Training Time
Samples
Time /Epoch
bank beth
min 41,188
sec
45 min 1,141,078
108
census
12 min 299,285
29 sec
2 min
1,473
5
kdd
10 min 1,014,535
96 sec
lanl
1.5 hrs 2,542,727
216
sf
2 min 1,066
5 sec
spotify
min 113,550
sec
Table 8: Approximate training times for CWAE on each datasct.
A.8.2 Inference Time
Per-sample anomaly score computation:
Forward Pass: Less than 1 millisecond per sample with batch size 2048
GPU Acceleration: Significantly faster than CPU-based inference
Deterministic Scoring: CWAF requires single forward pass (no sampling) unlike stochastic models (CVAE)
A.8.3 Context Selection Time
For each dataset with N candidate context features:
Single Context Training: epoch 4% of full training time
Total Context Selection: Nx (1-epoch time) + validation evaluation time
Example (census): 38 features x 29 sec/epoch 18.4 minutes
Validation Evaluation: Less than minute per context (evaluated on 10 batches)
30


Reviews:
[{"review_id": "p3QqJqwgfh", "reviewer": "Reviewer_ZRPQ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Author proposes a Contextual Learning framework for unsupervised anomaly detection in tabular data. This framework change the modeling objective from the joint distribution P(X) to a conditional distribution P(Y|C), where C represents a context features and Y represents content features. Author argue that conditioning on a context features reduces variance and allows for sharper, context-specific decision boundaries compared to global models.\n\nTo implement this, the authors introduce the Conditional Wasserstein Autoencoder (CWAE), a deterministic reconstruction-based model. This method perform heuristic search to automatically select the optimal context feature from the available features. This strategy relies on minimizing the joint validation loss estimated after a single epoch of training. The method is evaluated on eight tabular datasets against six deep anomaly detection baselines and a standard WAE, demonstrating generally superior performance in terms of AUROC.\n\nStrengths:\n1. The authors evaluate their method against a wide range of baselines across datasets of varying complexity.\n2. The theoretical motivation regarding the Law of Total Variance is sound and intuitive and proves need of contextual learning. Decomposing variance into within-context variance and between-context variance components provides a clear justification for why conditional modeling should theoretically outperform global modeling in heterogeneous data.\n\n\nWeakeness:\n1. While the authors state this is future work and justify the single-feature focus for interpretability, the framework's performance when dependencies are spread across multiple features i.e. P(Y|C_1,C_2) is not explored empirically. \n2. The framework assume that the validation set is mostly normal. In truly unsupervised settings, ensuring a clean validation set is often impossible. The paper fails to investigate the failure mode where the validation set is contaminated.\n3. The paper claims the model is effective for heterogeneous contexts. However, it does not provide evidence of how the model performs when the test set contains context values not present in the training set.\n4. The paper discusses data complexity but ignores dimensional complexity. Training 38 models for 1 epoch takes ~18 minutes. If a user has a standard dataset with 500 features, this selection phase alone would take ~4 hours, whereas an Isolation Forest runs in seconds. This become much worse when dependencies are spread across multiple features", "claims_and_evidence_yn": "Yes", "claims_explanation": "The authors provide both theoretical justifications and broad empirical evidence. The variance reduction section clearly explains why conditional modeling should theoretically improve detection sensitivity by reducing the noise of inter-context variability. Experimentally, author showed that CWAE outperforms its non-contextual counterpart and several state-of-the-art baselines across most datasets. The use of diverse datasets further supports the claim that the framework is generalizable.", "audience_yn": "Yes", "audience_explanation": "Anomaly detection in tabular data is a important problem in industry and a core topic in the machine learning community. The shift from global to contextual modeling is a relevant topic for researchers/practitioners dealing with high-cardinality categorical features that are common in real-world logs but often ignored or poorly modeled by standard anomaly detection algorithms.", "requested_changes": "1. Author should evaluate the model's performance on unseen contexts. perform a split where the Test Set contains only Context IDs that are not present in the Training Set. Author should discuss and empirically test how the model behaves for rare contexts (values of C with <10 samples). Does the model default to a prior, or does it overfit?\n2. Author should quantify or discuss the cost of your selection algorithm relative to the number of features in the paper and compare with existing methods\n3. Provide an experiment or discussion on the impact of anomaly contamination in the validation set. If the mostly normal assumption for the validation set is violated (e.g., 5-10% anomalies), does the algorithm still select the correct context feature?", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "nbepGZxr7J", "reviewer": "Reviewer_nSmt", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a contextual learning framework for anomaly detection in tabular data that models conditional distributions P(Y∣C) instead of a single global joint distribution P(X), instantiates it via a Conditional Wasserstein Autoencoder (CWAE), and shows that contextual models can surpass strong non-contextual baselines on eight benchmarks. The core contributions are:\n\ni) Formalizing anomaly detection with a split between context features C and content features Y, arguing for modeling P(Y∣C) with context-specific thresholds instead of a global P(X)-based detector.\n​\nii) Using a variance decomposition (law of total variance) and generative-vs-discriminative arguments (misspecification, curse of dimensionality) to justify why conditioning on context can reduce variance and improve anomaly detection.\n​\niii) Proposing a bilevel optimization approach that selects a single context feature by minimizing early joint validation loss (after one epoch) across candidates, using only (mostly normal) validation data and no anomaly labels.\n​\niv) Introducing CWAE, a conditional WAE with embeddings for tabular categorical features, deterministic encoder-decoder, MMD regularization, and per-context thresholding for anomaly scoring.\n​\nv) Empirical evaluation is done on benchmarks CWAE against seven non-contextual deep AD baselines over eight tabular datasets, reporting that CWAE attains the best average AUCROC (0.797) and lowest average rank (2.75), improving over its non-contextual WAE counterpart (0.760, rank 3.875).\n​\nvi) Providing ablations on contextual thresholding, optimal context choice (CWAE Best reaching average AUCROC 0.839 and rank 1.25, outperforming best SOTA on 6/8 datasets), and additional conditional baselines (CVAE, CMDN, CADI) to quantify the value of context.\n\n\nKey Strengths:\n\ni) The paper addresses a significant, practical challenge: heterogeneity in real-world tabular data.\n\nii) The framework is well-motivated theoretically (variance reduction, discriminative advantages) and practically (scalable context selection).\n\niii) The experimental design is rigorous, using diverse datasets and comparing against multiple strong baselines.\n\niv) The proposed CWAE is simple and effective, serving as a clear proof-of-concept for the broader framework.\n\n\nKey Weaknesses:\n\ni) The theoretical connection between variance reduction and improved anomaly detection, while intuitive, could be elaborated more formally.\n\nii) The ablation studies, while present, could more deeply analyze the individual contribution of the bilevel optimization strategy versus the conditional modeling itself.\n\niii) Context selection is restricted to single-feature conditioning, justified mostly for simplicity and interpretability; multi-feature contexts are only discussed as future work despite being central to many realistic settings.\n\niv) The proposed context selection relies on a mostly clean validation set and an early-epoch proxy; robustness to contamination, stability across seeds, and sensitivity to the 1 epoch choice are not fully stress-tested.\n\nv) The evaluation introduces a context-normalized scoring/thresholding scheme (max training loss per context), which may inflate robustness in some cases and is potentially sensitive to extremes; it deserves deeper justification and alternatives.\n\nvi) The framework is mostly evaluated with single-feature context and primarily categorical-encoding pipelines; generality to continuous/tabular mixed-type settings is not demonstrated convincingly.\n\nvii) Baseline comparisons may be uneven: many baselines are used with default settings and without contextual variants, so the SOTA claim is strongest for the CWAE-vs-unconditional setting, but less clear for contextualized competitors.\n​\nviii) Evaluation focuses almost entirely on AUCROC, which can be misleading under heavy class imbalance; AUC-PR and operational metrics are mentioned in the appendix but are not central to the main narrative.\n​\nix) The framework's dependence on the availability of meaningful context candidates is not stress-tested in adversarial settings (e.g., when candidate context features are weak, redundant, or noisy).\n\nx) It would have been better if they had shared the repository, respecting the journal guidelines.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The claims are strongly supported by a combination of theoretical reasoning and comprehensive empirical evidence.\n\ni) Theoretical Claims: The arguments about variance reduction (via the law of total variance) and the benefits of discriminative/conditional modeling are well-established in statistical learning literature and are correctly applied to the anomaly detection problem.\n\nii) Empirical Claims: The experimental section is thorough. The use of eight diverse datasets covers various domains and complexities. The comparison includes a relevant non-contextual baseline (WAE) and several SOTA models. Results (AUCROC, ranks) are clearly presented in tables and show consistent gains for the contextual approach. Ablation studies on contextual thresholding and optimal context upper bounds provide additional convincing evidence.\n\niii) Clarity: The methodology for context selection, model training, and evaluation (including the nuanced contextual AUCROC calculation) is described in sufficient detail for reproducibility. Figures effectively illustrate key points like threshold variation and validation loss curves.", "audience_yn": "Yes", "audience_explanation": "The audience of TMLR  would be interested in knowing the findings of this paper. The paper touches on\n\ni) Relevance to Core ML: The paper tackles a fundamental issue in density estimation and generative modeling, how to handle heterogeneous data by advocating for a shift from joint to conditional modeling. The theoretical discussion connects to classic generative vs. discriminative trade-offs.\n\nii) Practical Impact: Anomaly detection is a critical task in high-stakes domains like cybersecurity and finance, where the paper's data originates. The proposed method offers a principled way to improve detection in real-world, non-IID tabular data, which is a pervasive and challenging data type.\n\niii) Novelty: The integration of a bilevel optimization for unsupervised context feature selection is a novel and practical contribution. The CWAE model, while simple, effectively demonstrates the framework's viability.\n\nTherefore, the findings would be of significant interest to those working on anomaly detection, representation learning, and the application of ML to structured data.", "requested_changes": "i) Strengthen the theoretical section (Section 4). The link between reduced conditional variance Var(Y∣C) and improved anomaly detection performance is argued intuitively but not formally proven. A more formal discussion or a simple theorem/bound connecting variance to anomaly score separability would significantly bolster the paper's foundation.\n\nii) While Algorithm 1 is evaluated, a more direct ablation is needed to disentangle the gain from using any reasonable context vs. the gain from the optimally selected context via the proposed method. Consider adding a small experiment comparing performance using a randomly chosen context feature versus the one selected by the bilevel method across multiple runs/datasets.\n\niii) In Section 5.3 and Algorithm 1, clarify that the validation set is assumed to contain only normal samples, which is a standard but important assumption for unsupervised AD. This is mentioned later but should be explicit in the algorithm description.", "broader_impact_concerns": "The paper presents a methodological advancement in anomaly detection with primarily positive potential impacts (improved security, fraud detection). However, a Broader Impact Statement should be added to address the following:\n\ni) Potential for Bias: The method's performance hinges on context selection. If the chosen context feature correlates with sensitive attributes (e.g., race, gender), the model could learn to flag anomalies differently for different demographic groups, potentially leading to discriminatory outcomes. The paper should briefly discuss this risk and the importance of auditing context features.\n\nii) Dual-Use Potential: Like all anomaly detection research, more effective tools could be used for both beneficial (network security) and harmful (surveillance) purposes. Acknowledging this duality is standard.\n\niii) Data and Reproducibility: The statement could highlight the use of publicly available benchmark datasets and the provided appendix as positive steps for reproducibility and ethical research practice. Currently, the absence of such a statement is a minor oversight.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "gym6ABB57f", "reviewer": "Reviewer_yuXt", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents a contextual learning framework for anomaly detection in tabular data, motivated by the observation that global joint-distribution modeling often fails to capture heterogeneous behavioral patterns. Rather than modeling P(X), the authors focus on P(Y|C) by separating context variables from content features and learning conditional distributions. They introduce (i) a probabilistic formulation based on variance decomposition and discriminative-learning principles, (ii) a bilevel context-selection strategy via single-epoch validation loss, and (iii) a Contextual Wasserstein Autoencoder model. Experiments across eight datasets demonstrate that contextual learning improves over global models and that CWAE outperforms non-contextual WAE and other SOTA anomaly-detection baselines on average.\n\nStrengths:\n- CWAE shows consistent gains over WAE and often surpasses strong non-contextual SOTA baselines, supported by AUCROC tables and average-rank comparisons.\n- The theoretical framing is well-motivated, clearly linking conditional density estimation with variance reduction, offering conceptual insight into why contextual modeling helps.\n- The bilevel context-selection strategy is simple, computationally efficient, and empirically validated through early-epoch behavior.\n\nWeaknesses:\n- The role of context vs. content is under-formalized. While the authors claim generalizability to multi-context conditioning, implications such as feature interactions, combinatorial search cost, and risk of degradation are not thoroughly examined.\n- The thresholding and contextual score normalization strategy is engineered but not statistically grounded—there is little discussion of false-alarm rate, stability across context-sizes, or comparability with baselines using different score distributions.\n- Some implementation decisions appear insufficiently justified—for instance why one epoch reliably predicts context quality, why embeddings are concatenated before encoding, and how embedding representations are initialized or optimized.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Performance improvements are documented across eight datasets with AUCROC, ranked comparison tables, and multiple contextual baselines. The theoretical claims are reasonably supported.", "audience_yn": "Yes", "audience_explanation": "Tabular unsupervised anomaly detection is an active area, and contextual modeling is applicable across many fields. The framework is general, interpretable, model-agnostic, and applicable to existing architectures.", "requested_changes": "- Elaborate on how context/content features are partitioned, and discuss feasibility of multi-context conditioning.\n- Strengthen justification for the complexity ranking metrics and provide more literature linkage or comparative baselines.\n- Improve motivation for choosing the Wasserstein approach over VAE/CVAE and other reconstruction-based models.\n- Reference related literature from statistical process control and multivariate anomaly detection (e.g., control charts, Hotelling-T²/Mahalanobis distance, autoencoder-based MSPC) to position contributions more clearly within the broader methodological landscape.\n- Provide further ablation on one-epoch context selection to verify robustness of early-training ranking signals.", "broader_impact_concerns": "No concern.", "additional_comments": "- Could CWAE generalize to other autoencoder families (e.g., convolutional, diffusion, transformer-style) and to non-tabular domains such as images or logs?\n- How would the framework behave under multi-context conditioning, continuous context variables, or learned context embeddings?", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)