Adversarial Subspace Generation for Outlier Detection in High-Dimensional Data
Authors: Jose Cribeiro-Ramallo
Federico Matteucci
Paul Enciu
Alexander Jenke
Vadim Arzamasov
Thorsten Strufe
Klemens Böhm
Venue: Accepted by TMLR ()

Abstract: Outlier detection in high-dimensional tabular data is challenging since data is often distributed across multiple lower-dimensional subspaces—a phenomenon known as the Multiple Views effect (MV). This effect led to a large body of research focused on mining such subspaces, known as *subspace selection*. However, as the precise nature of the MV effect was not well understood, traditional methods had to rely on heuristic-driven search schemes that struggle to accurately capture the true structure of the data. Properly identifying these subspaces is critical for unsupervised tasks such as outlier detection or clustering, where misrepresenting the underlying data structure can hinder the performance. We introduce Myopic Subspace Theory (MST), a new theoretical framework that mathematically formulates the Multiple Views effect and writes subspace selection as a stochastic optimization problem. Based on MST, we introduce V-GAN, a generative method trained to solve such an optimization problem. This approach avoids any exhaustive search over the feature space while ensuring that the intrinsic data structure is preserved. Experiments on 42 real-world datasets show that using V-GAN subspaces to build ensemble methods leads to a significant increase in one-class classification performance—compared to existing subspace selection, feature selection, and embedding methods. Further experiments on synthetic data show that V-GAN identifies subspaces more accurately while scaling better than other relevant subspace selection methods. These results confirm the theoretical guarantees of our approach and also highlight its practical viability in high-dimensional settings.

Published in Transactions on Machine Learning Research (07/2025)
Adversarial Subspace Generation for Outlier Detection in High-Dimensional Data
Jose Cribeiro-Ramallo Karisruhe Institute of Technology
jose. wet.ecrrbbeiotiit...c. edu
Federico Matteucci
federico. www.emcciitattttccmommmm edu
Karlsruhe Institute of Technology
Paul Enciu
paul. ceileennii@ssudeee kit. edu
Karlsruhe Institute of Technology
Alexander Jenke
alecmnder.jenkkeesstuueeedd edu
Karlsruhe Institute of Technology
Vadim Arzamasov Karlsruhe Institute of Technology
vadim, satiim.rsammaoo@kiit.co edu
Thorsten Strufe Karlsruhe Institute of Technology
thorsten.struueBBkit.cc edu
Klemens Böhun
klemens. boehm @kit. edu
Karlsruhe Institute of Technology
Reviewed OpenReview: /ttps://openrevvee net/ forum? id= j
Abstract
Outlier detection in high-dimensional tabular data is challenging since data is often distributed across multiple lower-dimensional subspaces phenomenon known the Multiple Views elfect (MV). This effect led to a large body of research focused on mining such subspaces, known as subspace selection. However, as the precise nature of the MV effect was not well understood, traditional methods had to rely on heuristic-driven scarch schemes that struggle to accurately capture the true structure the data. Properly identifying these subspaces is critical for unsupervised tasks such as outlier detection or clustering, where misrepresenting the underlying data structure can hinder the performance. We introduce Myopic Subspace theoretical framework that mathematically formulates the Multiple Views effect and writes subspace selection MST, we introduce V-GAN, a generative method trained to solve such an optimization problem. This approach avoids any exhaustive search over the feature space while ensuring that the intrinsic data structure is preserved. Experiments on 42 real-world datasets show that using V-GAN subspaces to build ensemble methods leads a significant increase in one-class classification performance compared to existing subspace selection, feature selection, and embedding methods. Further experiments on synthetic data show that V- GAN identifies subspaces more accurately while scaling better than other relevant subspace selection methods. These results confirm the theorctical guarantees of our approach and also highlight its practical viability in high-dimensional settings.
Theory (MST),
stochastic optimization problem. Based on

---

Published in Transactions on Machine Learning Research (07 (07/202) 2025)
1.0
A 0.5
0.0
X1
0.0
0.5
1.0
F
(a) Population from example
(b) Sot.A for subspace search
Figure 1: (a) Population from example and the performance of the Sot for subspace scarch in it. We colored in blue those in subspace Un and purple those in subspace U2. (b) The normalized weights and
1- FF assigned by GMD (Trittenbach and Böhm 2019) to subspaces S and S2 should be as close as possible to F and - F, ie., the dashed grey lines.
1 Introduction
high-dimensional data, such as images, text, or some tabular datasets, constitutes much of the available data on the internet, in medical domains, and even in the private sector. Especially when high-dimensional data can exhibit multiple complex relations between its features. Outlier Detection (OD), as well as other downstream lasks, can greatly benefit from correctly exploiting these relations to achieve more accurate results (Aggarwal, 2017 Trittenbach and Böhm 2019). A popular research direction the literature is scarch for subspaces maximizing a given quality metric. The multiple subspaces later employed either to study complex interactions between features to build an ensemble of models, with each member in different subspace (Aggarwal 2017).
Methods for obtaining subspaces work in two ways. One type extracts a single subspace that better represents all data like embedding and feature selection methods (Balm et al. 2019 Meilä and Zhang 2023 Healy and McInnes 2024). These methods assume that the data lies on a single, low-dimensional subspace preserving properties, such as point distances, topology, or notably, the underlying distribution. As discussed in Example 1 however, single, low-dimensional subspace might not be enough to characterize the data. It is therefore common in unsupervised tasks assume that the data instead lies multiple subspaces known as the Multiple Views effect (MV) [Keller et al.] 2012). Subspace selection methods handle this latter scenario by providing a list of interesting subspaces and, hence, better preserving the relationships within the data (Keller et al. 2012 Trittenbach and Böhm 2019 Agrawal et al. 2005).
The extra information provided by mining for multiple subspaccs can be crucial for unsupervised tasks with little prior information, like clustering and outlier detection (Keller et al., 2012: Qu et al. 2023 Cribeiro- Ramallo et al. 2024p. Particularly, et al. 2000), by training multiple detectors, each a lower-dimensional projection of the data. Subspace selection methods that provide projections into each subspace are known as subspace search methods. The most common approach, as discussed (Aggarwal 2017 Chapter 4), is to search for those feature subspaces maximizing some heuristic quality metric. The obtained subspaces are hence necessarily axis-parallel. Despite this limitation, this approach proved detection. The usage of heuristic as the quality metric, however, does not guarantee that the extracted subspaces preserve the data's properties, such as its distribution. As claborate in the following example, this happen even in simple settings.
create ensembles of outlier detection methods, like LOF (Brunigg
an effective technique for some downstteam tasks, including outlier
Example 1. Consider a population in Figure la where 3-dimensional data lies with probability F on the 21 -2 plane and with probability 1 F on the va line. We refer to these two subspaces as S1 and S2
2

---

Published in Transactions on Machine Learning Research (07/2025)
Table 1: Summary of existing methods and their capabilities.
Multi-subspaee
Represent
Project
Feature Selection
X
Embedding Methods
X
Subspace Search
X
Subspace Discovery
Subspace Generation (Ours)
respectively. The data exhibits Multiple Views, as it lies within S S1 US C TR3. As the data lies on two subspaces, methods that return a single subspace are not able to correclly represent it. On the other hand, methods that return a list of subspaces should be able to (1) identify S and S2 as the relevant subspaces and (2) assign them scores proportional F and 1- F. However, as shown in 7b. the state-of-thee (GMD (Trittenbach and Böhm, 2019)) fails identify both subspaces and to assign them an accurate score. Indeed, the relrieved subspaces opprocimalely degenerate to the sole S1 if F < 0.5 and to S2 olherwise.
Our goal is to avoid using heuristic quality metric while guaranteeing that the underlying data distribution is preserved. This presents the following challenges. (1) To the best of our knowledge, the only previous attempt at formalizing the Multiple Views effect focused on proving the efficacy of subspace ensembles for tabular Outlier Detection (OD) "Crbeerr-Rammllo et al. 2024). Consequently, their theory is not directly usable to find subspaces, nor does extend to arbitrary data types; we will elaborate on this in Section 3.1 (2) Even with a theory able recognize the subspaccs relevant for MV, these latter live in an exponential search space the power set of the set of features. An exhaustive search is therefore unfeasible. The designed method should be able to find relevant subspaces and approximate their weights while avoiding searching in such a power set.
Our contributions are the following: (1) We generalize the theory from Raimeiro-Ramallo et al. to be both applicable to general data types and to allow us to obtain subspaces in practice. In our revised theory, subspaccs can be obtained by solving stochastic optimization problem, while we can provide guarantees on the underlying data distribution being preserved.
(2) To solve the optimization problem, we propose the generative network V-GAN, whose goal is to generating projections into the desired subspaces. We prove that its loss function is optimizable and that the network converges to the desired global optimum under mild conditions. (3) We validate our theoretical results in practice using synthetic data, showing how V-GAN can extract the predicted subspaces by our theory. (4) We study the quality of V-GAN subspaces by training an ensemble of outlier detection methods using them, and testing their performance. In particular, we show how V-GAN's subspaces consistently lead to ensembles that are siguificantly better than all their competitors on 42 real-world benchmark datasets from (Han et al. 2022). (5) Finally, we provide the code for all of our experiments and methods!
2 Related Work
This section briefly overviews the subspace selection field together with its subfields. Table includes a summary of the discussion.
A classic approach to dealing with high-dimensional data is to assmm that data lies on a lower-dimensional manifold and to provide a projection into it. This can be done by removing unwanted features (Baln et al. 2019) or finding other (not necessarily) orthogonal transformations {Jones and Artemioul 2021: Meilä and Zhang 2023p. These transformations always focus on preserving the original distribution of the data in order to it in given downstream task. For unsupervised downstream tasks, it is common to use general assumption on the data, known as Mulliple Views edffect (MV). Under MV, data lies in a collection of subspaces [S rather than in a single well-behaved one (Keller et al.
2012 Elhamifar and Vidal into each subspace, ensuring
2013). Subspace selection methods aim to obtain a set of projections (T:) 19, 1
that any new datapoint can be projected into these subspaces (Aggarwal
2017 Keller et al. 2012). The
3

---

Published in Transactions on Machine Learning Research (07/2025)
2005 in contrast to other classical approaches employed in subspace clustering (Wang et al. | Vidal et al.] 2012). Using these subspaces, one can build powerful ensembles for outlier detection (TTritennach and Böhm 2019) or obtain more precise clusters [Qu et al. 2023). There exist two big families of methods in subspace sclection, as we will claborate
number of subspaces, ngy is assumed unknown beforehand (Agrawal ct al.
in what follows.
Subspace Search. Subspace search methods assume that the subspaces conforming the data are feature subspaces, i.e, axis-parallel projections of the data. Thanks to this, they work on finite search space P([I...,, dd) simplifying the search scheme. More specifically, these methods explicitly output the subspaces that maximize a certain quality metric in P((1....,d) project the data into each subspace for the desired downstream lask using axis- exis-parallel projections. They are popular for outlier detection, as they allow the use of subspaces to create ensembles with off- the-hhe-shef outlier detectors (Aggarwal 2017). These methods are also used for clustering, like in (Agrawal et al., 2005 Cheng ct al. 1999), although they are typically coupled with a particular clustering algorithm. However, as discussed earlier, the main drawbacks of these methods are the cardinality of P(1, d) and the selection of quality metric. Although some efforts address the search space problem, work in the subspace search literature offers a theoretical definition of what an 'important' subspace is. Current methods rely on heuristic quality metrics that do not guarantee the selected subspaces accurately preserve the data's properties. For instance, a method like Enclus (Chong et al.] 1999) relics on subspace search based on the entropy metric. Therefore, Enclus would select the subspaces with minimal entropy, missing potential subspaces if the assumption of low entropy subspaces does not hold like for the population from Example [ This leads to a collection of subspaces that do not represent the data correctly.
As the subspaces are explicitly known, one can trivially
Subspace Discovery. Given the problems with subspace search, a second group of methods adopted a different approach for subspace selection. That is, they do not assume that the subspaces are necessarily feature subspaces, nor do they explicitly output the projections themselves. Instead, they solely focus on identifying which data points are likely to belong to the same subspace to output an adjacency matrix based on it. This relationship matrix then typically used as a graph adjacency matrix for spectral clustering. Authors successfully used these mothods to develop clustering tochniques for various applications, including face, motion, and sentiment recognition (Qu et. al. 2023 Elhamifar and Vidal 2013). However, since they rely on relationship matrix between the clustered points, these methods focus on projecting the clustered data only. Therefore, they cannot project new data points into the subspaces without retraining, which limits the use of subspace discovery methods for outlier detection. While there are adaptations to handle incoming data points, they typically treat data in an online manner, contimuously updating the projections with each new datapoint inferred (Kong et al.] 20011 Langone et al. 2014 Sui et al. 2022). Consequently, this adaptation falls under the category of online machine learning. As we treat data in an offline manner, this adaplation is outside of our scope (Ben-David et al. 1997).
Previous Descriptions of Multiple Views. In the recent literature on outlier detection, Cribeiro- Ramallo et al., attempted to mathematically describe the MV effect for a given population. Previous attempts aim to study the distribution of data lying in subspaces by characterizing parts of the fullspace where the probability mcasure concentrates (Pal et al. 2023). In contrast. [Criberr-Raane Ramallo et al. focused on providing specific probability measures for the subspaces, allowing both their characterization and measurement. In particular, the authors defined a family of distributions called myopic distributions, and show how the MV effect occurs when the data is generated as such. For example, distribution of a population is myopic when its density Px is invariant under the transformations of random orthogonal projection matrix U. That is, whenever Px = Pux, with each realization of U being an orthogonal projection matrix Seiibiro-Ramaill let al.) (024).. As an example, if one considers again the population from Example | it is myopic under the effects of:
one can easily verify that
U {
U diag 1,0) with probability F, else.
U2 diag(0,0,
While the theory can predict certain behavior on paper, Cribeiro Ramallo et al. do not provide any way to obtain such U's in practice. Additionally, it lacks sufficient generality to do so trivially, as density functions
Grbeero-Ramallo
4

---

Published in Transactions on Machine Learning Research (07/2025)
are difficult to estimate in practice (Guo et al. 2022). This complicates the task of obtaining the random projection matrix by directly using the definition in Cribeiro-Ramallo et al. (2024).
Subspace Generation. Our work centers around generalization of the definition of myopic distribution that allows us to frame it using components one can easily estimate in practice. Thanks to this, we can fit a generative method capable of approximating the distribution of a U verifying the definition. This way, we can sample projections into these subspaces This novel approach to subspace selection, which we dubbed Subspace Generation, avoids searching in P(,),..., d}), and provides suitable notion of "important" subspace. This solves the representation problem of subspace search, while not sacrificing the ability to project the data into the subspaces.
3 Myopic Subspace Theory
In this section, we will discuss the preliminaries for introducing our Subspace Generation method. We will frame our thcorctical background a generalization of the theory of myopic distributions introduced by Cribeiro-Ramallo et al.] In partieular, will introduce their definition first and discuss the main drawbacks that motivate more general framework (Section 3.1]. After that, we will propose such generalization and use it to write subspace selection optimization problem (Section 3.2) Lastly, we will show optimality guarantees under general conditions (Section 3.3
3.1 Original Definition
A large collection of authors observed that high-dimensional tabular data seems to behave differently in certain feature subspaces than in others. In particular, significant body of research empirically examines the occurrence of data variability concentrating a specific collection of subspaces (Keller et al. 2012
Nguyen
ct al. Trittenbach and Böhm, 2019). This effect is called Multiple Views of the data (MV) (Müller
2014
et al.
Cribeiro-Ramane et al. tried to mathematically describe MV, to then propose a way to train parametric methods under it. Their definition gocs as follows.
2012)
First, consider a metric space (F4, T)) Let x" B., Fd be random vector and (Ω, (,, : IP) a measurable probability space with Borel's sigma algebra. Lastly, consider Diag( {0, 1})axd the space of d d diagonal binary matrices ? and Px x P the distribution of x" with Px dp." its density in the du
Radon-Nikodym sense 3
Definition 1 (Cribeiro- Ramallo et al.). Consider E' = R and U B,, P) - Diag(10, 13)axd random binary matric. We will say that x" is myopic under the views of U iff:
Pxr = Puxe, point-wise in their support.
In this case, we call x" myopic under U simply, myopie, if there is risk of confusion.
By Definition I a population is myopic under the views of a random binary matrix iff the random vector Ux" has the same density x" for all points in its support. As diagonal matrices are orthogonal projections, it is the same saying that observing x" and randomly projected version of x" lead to the same density for any point in its support. The authors then prove that one can calculate Pux under myopicity by
PU Σ Pu, (U))Pixx
This result is very important for the particular use case of outlier detection (Cribeiro-Ramall Ramallo et al. 2024 Trittenbach and Böhm 2019 Aggarwal 2017). However, how to find such U not properly described. In particular, identify the following problems with Definition 1
Withoot the identity. With p Fxt.
5

---

Published in Transactions on Machine Learning Research (07 (07/2025)
1. The point-wise equality of densities. In order to provide estimate for the validity of the definition, one would have to estimate first both densities. Not only is density estimation hard in high-dimensional data, but the existence of densities is not guaranteed for a general distribution, limiting the applicability. Limited to E = and U E Diag( Haxd- The limitation of the metric space to the real line and the realizations of U to diagonal binary matrices further restricts the use of this theory to more general data types.
3. Estimation in Practice. Even with all the limitations to the definition of x" and U, it is unclear how to properly find U that verifies Definition | for given x". Even assuming that we can perfectly estimate the densities, how to find such a random matrix that P (p) - Pux (p) = 0 for almost all p is unclear for the finite sample setting.
In the following section, we will propose a general definition that addresses all previous weaknesses, while also giving certain generality conditions for it.
3.2
Myopicity via its Representation in H.
We will first introduce collection of notations and necessary conditions for our generalized definition. After that, we will explain how our generalization solves all of the previously raised problems.
3.2.1 Tackling the Point-wise Equality of Densities and the Space Limitations
Consider T) separable metric space, H the associated Reproducing Kernel Hilbert Space (RKHS) of real-valued functions on E' with kernel x and MT (E) M+ the space of positive signed Borel measures with value 1 (i.c., probability measures) on E. Further consider x : (2,, B,, P) - E a random variable with (22, B,, IP) a measurable probability space with Borel's sigma-algebra, and & the space of such random
variables. In order to avoid problems | and D one can consider a richer definition as follows:
Definition (Myopicity of a distribution). Consider C(X) the class of continuous operators from and the space of random variables on E, X, and a subset ((X) C C(T). Further consider
U: (S2, B., P) (x)) c C(2),
random operator taking values on ( We say that x is myopic to the views of U iff
Px = Pux-
(1)
In this case, we say that x my3)-m and U a lens operator for x.
It is clear that Definition generalizes Definition 1 by taking E Rd, +(x) = Diag( (0, 1))axd and invoking the uniqueness of Radon-Nikodym's derivative (Simonnet, 1996 Chapter 10). Furthermore, Ux is correctly defined as the mapping
Ux w E (,, - Un E E,
with both and x being realizations of and x respectively. Coming next, we will introduce a collection of examples of lens operators for a different set of cases not necessarily having +(x) defined on fibers of E. Example i) Normal Projected Population from Example 1. Consider the same population as Example 1, with F 0.5. Clearly, by the law of tolal probabilities
1 PxxxES + Fx
1
Px 2
In this oase, any non-zero mensurable set on St will get projected into zero-measure set in S2, and vice versa. Thus, we can write IP PxixEs, = Puix and IP PxIxES= = Puxx, with
0 0
0 0
U1 =
0 0
U2
=
0
0
0
0
6

---

Published in Transactions on Machine Learning Research (07/2025)
and observing that both Puix and Puzx are probability measures by invoking the Disintegration Theorem Faden 1985).
ii) General non-paraltel hyperplanes. Consider now a population lying in H1 and H2, two different non-parallel hyperplanes of TR of (non-necessarity equal) finite dimensionality, with probability F and 1-F. F. Further consider E to be equipped with the usual topology. As an illustrative exomple, one can consider the simplified version of Figure [2]] Consider E to be normed topological vector space and both H1 and H2 subspaces. Now, consider the projections U1 and U2 to H1 and H2, parallel to H2 and H1 respectively. In other words, U1 and U2 are normal to the orthogonal planes H2 and Hi, respectively. Clearly, U2(H1) Ç H2, otherwise any point. of H2 could be wrillen as a linear combination of H1 's basis. This would, in turn, lead to a contradiction as H1 and H2 are non panplarlll,,, and thus H₂ & H. Therefore, (2) is a subspace of H2 with lower dimension. The equivalent can be derived for U1 (H2).
Consider U as the random operator such that it takes values U1 and U2 with probabilities F and 1- F respectively. Clearly, Px = F PxixEH, + (1 F) P PxixeH, as in the previous ezample. Furthermore, as we just proved, any open set from H1 will get projected into subspace in H2, and vice versa. As F is Borel's sigma algebra and E = =R, 1) is of measure 0 in H₂ (Rudin) 1987, Theorem (e)) Thus, all measurable subsets of H1 will get projected into aero-measure subsets in H2, and vice versa. Then, as we did before, IP xx PUxx and P Px/xEH, = PU,x- Therefore,
= F + F1 PUax:
making U lens operator for This process can be repeated to obtain the lens operator to on arbitrary finite collection of non-parallel hyperplanes (Hi with probabilities {pilier with linearly independent bases.
iit) Transversal manifolds. We can take it one step further and consider the general case of two (n - 1) -Jssmmmnnsoolds M2 of the ambient n-manifold M. If the intersection of both submanifolds non-empty, we can use the notion of transversality prove that the intersection is a O-measure set. For instance, we say that two manifolds intersect transuersely iff the tongent spaces T,,Mi, T,.M2 spon T.M at all points p (Guillemin and Pollack, 2010). In this case, the intersection M1 MM2 is a submanifold with dimensionality dim M1 + dim M2 dim M = n- 2. If one takes look at Figure 26 we can see an example in 3 dimensions, where the intersection of these two 2-manifolds spheres - is of dimensionality 1. Consider U1 the parallel transport to the closest point to M1 and U2 the parallel transport to the closest point in M2- Thus, by taking U1 and U2, we can construct similar lens operator for any population x lying in the transversal manifolds.
2-
ir) Homoskedastic errors. Assume the following random variable y 3x +s, s, with B E Raxd and another random variable acting as noise. Now, given an infinite set D = [Bx; + Ethie of samples of y, we can define
U, Bæ, + E B ER. VIEI.
+
As such, defining a U selecting all U, with equal probability will trivially be a lens operator, if all e are equally distributed. The finite sample setting of this case a common bootstrap technique known as Resampling Residuals.
v) Location Operator for the Variance. A trivial example in R can be obtained by considering the random variables x ~ Px and y = Var(x). Consider U(w) E {UI.U2}. with U1y = Var(x + 3) and U2y = Vx 10). Trivially, as the variance is invariant to location changes, Uy = y.
Certainly, both problems and 2 are successfully addressed by Definition 2 However, equality between two measures in MT is still too general to lackle problem 3 Generally, when searching for way to determine when two elements a, of the same space X are equal, one defaults to check whether m(a, b)x = 0, if such a space X is equipped with a metric xx. Our goal is to do the same for two probability mcasures p. (E). In what follows, we will introduce how obtain such a metric, and in what conditions that metric exists.
7

---

Published in Transactions on Machine Learning Research (07/2025)
H1
U2
Un
M1
H2
M2
(a) Example 2 (ti).
(b) Example 2 (iti).
Figure 2: Simplified version of the subspaces Example (wi and Example (ii)..
3.2.2 Tackling the Estimation
There exists large body of literature focusing embedding (E (*)EE into RKHS H of real-valued functions on E -5-ss see (Berlinet and Thomas-Agaa Agnan 2004 Chapter 4) for survey. The particular embedding employed to represent measure as function in H. will determine the metric that one obtains at the end. This is why it is important to carefully embed M in a way that the resulting metric can be easily estimated. For that, we will follow the existing body work that aims to obtain a metric (Gretton et al. 2012 Schrab et al.) 2023 Fukumizu et al., 2007) that easy to estimate and has a known asymptotic distribution (Gretton et al. 2012
First, consider the linear functional on H:
Ep : E Epl = fdp E R.
Given this mapping, one can define: Definition 3 (Definition 2 in Gretton et al. (0012)) ). Let F C H be class of functionals E. The
Marimum Meon Discrepancy (MMD) is defined as:
MMDx
sup (Epp - Egs).
(2)
JEF
In H. RKHS with kernel measurable and such that k(, )dp < co, for all Mf),,, (E), and F the unit ball in H. one can casily prove (Sriperumbudur et al 2010 that:
Elpp H. such that Epf =< Mp. >H, Vp M MMD, a)² 1| Pg 11. = xx [x(x, x')] 2Ex [s(x,, x')l Ex.x [M(x, x')]
(3)
(4)
The unique representer of p in H. Hp: known as the mean embedding of p in H. The use of the unit ball for F is not arbitrary, as different function classes lead to different metrics. We want use the one in Equation as it has consistent U-estimator with better rate of convergence than other popular metrics in M, see
Sriperumbudur et al. (2010). An interesting consequence of working in an RKHS is that one can characterize specific propertics of the MMD and the estimator by properties of the kernel к. The most important one for
us is that the MMD defined as in equal ion on an RKHS H with a characteristid kernel is a metric see Fukumizu et al. 2007 Thus, MMD, (p. = + p=
Therefore, one can state the following:
*here the kernel is integrated with respect to both variables at the same time. This is, as:
x(;-): : - x(x,x:)
More precisely, / /mm//mm h m) -consiseent 6 is characteristic iff F,g FqJ, v/ = q
8

---

Published in Transactions on Machine Learning Research (07 (07/2025)
Lemma 1. Consider H. RKHS with a characteristic kernel K; and x, U and MMD previously defined. Further, consider V to be lens operator for x. Then,
min PUx) V
U
Thus, by Lemma 1 for a myy)-m x, one could find a lens operator by solving the stochastic optimization problem
arg min MMD Pux) 3 V.
As we only have access the sample estimate of the MMD in the finite sample setting, MMD, Gretton et al. 2012). need to work with the problem
arg min MMD, (P, 3 V.
(5)
The question now is whether the optimization problem is optimizable, and under which conditions obtain lens operator. We will answer these questions in what follows.
3.3 Convergence to Lens Operator
Consider now a random operator U as before, and the space M C CM of probability measures on E generated by ((x) and x. This is,
p PE MB(R)
EU such that Fux = p.
The following theorem and corollary establish the conditions for the optimization problem 5 to have global minimum for M - i.c., a lens operator. We first will write it terms probabilities in M C M and then we will show that one can rewrite it terms of the random operators under certain conditions. random variable (E,T) operator laking values on ((X) C C(X). Further consider the associated RKHS H of functions E with characteristic kernel K,, and the induced MMD metric on M Under these conditions, if M is compact and x ((X)-myopp
Theorem 2. Consider
separable metric space and U random
have that:
Given an iterative convergence strategy 3 such that 3(Pn-1) = Px E c MM and {PaINEN - p' arg inf MMD,,(Px, il follows that: PEN
3(pn-1) = Pa. E M = {paINEN - p' E arg min MMD, and p' M
PEM,
Logically, any way of obtaining sequence in subset N C Mt whose limit optimizes min MMD,,(Px p) be used to obtain such sequence in M Under Theorem 2 we know that such a sequence has a limit in ( and that the limit will also be a global optimum in MT The usefulness of this is made clear in the following corollary, which also gives the conditions to write Theorem 2 in terms of operators O(X) - Figure for visual summary. This corollary will allow us to solve equation given a large enough sample size for MMDx, and a proper way of sampling realizations of random operators.
in V like (Arbel et al. 2019 Mroueh and Nguyen 2021 |
Corollary 3 (Convergence to lens operator). Consider random variable on (E,T) separable metric space and U continuous random operator taking values O(x) C C( Further consider the associated RKHS H of functions E with characteristic kernel & and the induced MMD metric on M Under this conditions, if ((x)) is compact and x is -myoicc, we have that
Given an iterative convergence strategy such that F(P.-1) Pr EN C and {PalnEN - p' arg inf (IP, p), it follows that: PEN
(Un)neN such that 3(PU, PU., MMD, PU. - 0, and (Un) a. V (xx).

---

Published in Transactions on Machine Learning Research (07 (07/202)) /025)
MI
Unx
Puixhn
with the operators verifying
U,
Mx ((x)
Corollary 2 shifts
the focus from M the random operators
Figure 3: Diagram of the relation between Theorem and Corollary
In other words, Corollary shows that Theorem also imply that sequence of operators (Un) JeEN obtained via 3, will converge almost surely to lens operator in ( as long ( is compact, and myopic. Thus, by Corollary 3 Equation will have a solution that is a lens operator for x.
Now that we know that we can solve Equation there are only two questions left.
sample random operators to solve Equation 5 in differentiable manner? 2. If we find lens operator V, can we still characterize the density Pvx by the marginals Pvx? That is, is there an equivalent to Cribeiro-Ramallo et al. 2024 Proposition 1) in this general theory?
1. In practice, how
Solving Question 1 will give way to obtain lens operators in ((X)-myopic populations in practice. Section will introduce such method. Solving Question is important to the downstream task of outlier detection. It is immediate under the assumptions of Corollary 3 by invoking the Disintegration and Radon-Nikodym
Theorems [Faden 1985 Simonnet 1996). We included a more general result akin to (Cribeiro-Ramallo el al. 2024 Proposition 1) in the Appendix, as such generality not necessary in our setting.
4 Adversarial Subspace Generation: V-GAN
In this section, we will employ our previous theoretical findings to propose a method for sampling lens operator. We will describe our setting and propose our method, and then propose a way to identify whether V lens operator or not. A pseudo-code of the training is included in the Appendix
4.1 Subspace Generation with MMD-GANs
Our goal to find a way to sample a lens operator V, i.e., we want to approximate the sampling function of V using a parametric model. In particular, we aim to learn a parametric function Gg, from an arbitrary latent space to the space of operators ((2). The goal is that, when Go is composed with a uniform random variable in Z, PG)()) Pv. We do so by minimizing the loss function:
(()) = MMD 2
(6)
Approximating sampling functions is common problem in the machine learning literature, being the main use case of generative models (Goodfellow et al. 2016 Chapter 20). Tn particular, Generative Moment Matching Networks (MMD-GANs) use the squared sample MMD as their loss function, written as
2 C()) (P,,
with Fe : E Z Fo(z) E generative network. These networks guarantee convergence
distríbution of Fz(z) to x when minimizing the loss in terms of the parameters (Bińkowski al. 2021 Arbel
10

---

Published in Transactions on Machine Learning Research (07 (07/2025)
et al. 2019 Li et al. 2017). However, none of them guarantee convergence to solution in Mx that is global optimum also in MM which need for myopicity. Theorem gives sufficient conditions that guarantee convergence within M and Corollary writes it terms of the space of operators (
As such, we will consider a neural network Gg such that:
: f -
with ( compact. In practice, the architecture of Gg, the metric space E, and the space of operators ( have to be defined case-by-case. We will study the case axis-parallel subspace selection, as it the most common setting in the literature of subspace search and subspace outlier detection (Aggarwal 2017). We call
this strategy searching subspaces by generating them Subspace Generation, and our proposed method, V-GAN. Section B.2 in the Appendix contains examples of how can apply the Myopic Subspace Theory and V-GAN to different datatypes using different operators.
4.1.1 Axis parallel Subspace Generation
Let E = Fd and (()) = ((, 1axx separable and compact, respectively. As matrix-vector multiplica- tion with diagonal matrices the same as the element-wise product of the vector and the diagonal, we will build G3 such that (ø(2) E [0. Thus, the loss function of our network, given of samples {a and noise {2g]i=1 can be written
n n
Lx ((xi)==1: (z5)ー=1;0) MMD (Tx, Pao(e)x) n(n 1)
k <( Uy)
j=1
n
(,, Oz))
n(n
(7)
j=1
iデj
2
n
x Go(z))
n
==
ifj
with characteristic kernel (Greton et al. 2012). To obtain Go(zy) {0, 13d will use the upper-softmax activation function, defined as:
1
uus (x) u Jam (x)
with Jam the softmax activation, the element-wise unit-step function and 1 a vector of size with in each entry, As the unil-step function is not differentiable, we will use the softmax directly during backpropagation, similar to other binary-NN (Goodfellow al. 2016). We described the particular layers employed in the experiments in the Experimental Details in Section 5.1.2
4.1.2 Kernel Learning for V-GAN
The literature of MMD-GANs also studies the case of using kernel learning, where is now a trainable function Kg. Particularly, Li et al. provide a way train such kernels while also maintaining the convergence guarantecs. The resulting loss function can be written as:
Lu (1a:) (x00) = Lno
Σ || & (E)(2))),
(8)
x with E, and E being an encoder and decoder network, and Ks K E = x.E Eg()). Ks will be a characteristic kernel as long is characteristic and & injective (Berlinet and Thomas- 2004
11

---

Published in Transactions on Machine Learning Research (07 (07/2025)
The second addend of Equation guarantees the injectivity Birikowski et al. 2021 Thus, the optimization
problem becomes (Li et al. 2017
min max Lu (f (z)15=1;0,0)
(9)
4.2 A Test for Myopicity
In practice, we only have access to sample of i..d. realizations of x. Thus, if one limits itself to simply minimizing Loss G one could end with a random operator approcimalely lens. That is why, to assess whether the two random variables x and Ux have the same distribution, we need to use the following hypothesis test:
Ho Px = -Uxx
(10)
H, Fx + PIXx
As the sample MMD's asymptotic distribution tabulatod, one can use for such a statistical test.
In other words, statistic (Gretton et al. 2012) for the Test 10 This is ideal, as Gretton et al. proved that the resulting test is asymptotically consistent" Therefore, for sufficiently large sample, we could study whether U is lens operator for x with a probability of a false negative R 0. Thus, for any particular population, we can study whether given U a lens operator, allowing us to obtain random operators that are exactly lens practice.
test whether given operator U is lens operator for by using the MMD test
5 Experiments
We evaluate different aspects of V-GAN as follows. First, we examine its ability to recover a derived lens operator. Second, we compare its effectiveness in building one-class classification ensembles across 42 real-world datasets to nine competitors. Finally, we analyze its scalability in comparison other subspace selection methods. We will start by describing the experimental setup.
5.1
Experimental Details
This section has three parts. First, we describe the synthetic and real datasets for our experiments. Then, we describe V-GAN's configuration. Finally, we introduce competitors.
5.1.1
Datasets
Real We used 42 normalized datasets from the benchmark study by Han et al. listed in Tables [I the appendix. For those datasets with multiple versions, we chose the first in alphanumeric order. Details about each dataset are available in (Han et al. 2022
Synthetic Consider the random variables X2, X3 ~ N 1). As data for our experiments section 5.2 we will consider a 3-dimensional population x generated by randomly drawing points from S = (x1. X2, 0) and S₂ (0, 0, x3) with probabilities Fi F' and F2 from d-dimensional Uniform distribution and vary and d to study the scalability of various methods.
respectively. In section 5.4 we generate points
5.1.2 Network Settings
Generator Figure contains a diagram of the architecture and the training of the generator. features four hidden linear layers with increasing number of neurons: hú, _ h42 f. hig = a and h2 d, where d represents the data dimensionality. The input layer from the latent space has neurons, while the output layer employs the upper softmax activation dus see Section 4
7 test. A is called consistent iff, given any level a, the Type IT error 9
12

---

Published in Transactions on Machine Learning Research (07/202) 2025)
vgan
vgan
Linear Upper softmax
G U e(3)
G2 U. B(X)
GRUBIX
2.(9) MMD[P, P.. Payo)
Lu o) Lue, (
1E eo e 1edt))1
(a) Without kernel learning
(b) With kernel learning
Figure 4: Diagram of the network and the training without and with kernel learning, from left to right, respectively. (a) The network Go is trained minimize the loss (0) the empirical estimator of the
MMD Gretton al. 2012) between samples of and (z)z)x using kernel (b) The network Gg is trained to minimize the same loss before, but with & composed with Eg, the encoder part of auloencoder. At the same time, Eg is trained to maximize CroEs while minimizing the reconstruction loss.
Table 2: Table of the different competitors in our experiments grouped by method type.
Тype
Competitors
Subspace Selection
CLIQUE
2005
HiCS 2012
GMD
2019
Feature Selection
CAE
2019
Embedding Method
PCA 1901
UMAP 2024
ELM
2023
Kernel Unless stated otherwise, following the advice from (Li et al. 2017 we will use the kernel Kg = c Eg- Here, s is a Gaussian kernel with the median heuristic bandwidth parameter (Garreau el. al. 2018) and Eg an encoder trained by kernel learning - see Section 4.1.2 Particularly, we use an upside-down version of the generator's hidden layers for Es, with the identity function as the output layer.
Training We trained the network for 2000 epochs, with minibatch gradient descent using the Adadelta optimizer ZZeiler 2012) following preliminary results. In particular, use batches of size 500, learning rate of lra frg 0.007 for the generator and the encoder, respectively. We set momentum (0.99) and weight-decay (0.04) [Goodfelow el al. 2016). Additionally, updated Eg once every 5 epochs.
Number of Subspaces We generate 500 samples of the lens operator V to approximate its distribution. Thus, the number of subspaces depends on the number of unique values of its distribution.
5.1.3 Competitors & Baselines
We selected popularly used in outlier detection and state-of-the-art (SotA) subspace search CLIQUE Agrawal et al., 2205),, HiCS [Keller al. 2012). GMD (Trittenbach and Böhm 2019) feature selection
CAE (Balin et al. 2019 and embedding methods PCA (F.RSS.) 1901), UMAP (Healy and McInnes 2024 ELM (Xu al. 2023 with openly available implementations, competitors; see Table 2 for quick summary. As subspace discovery methods cannot project newly incoming data into the selected subspaces, they can not be used in our setting. For all methods included, we used the recommended parameters and training regimes. Specific details for each compelitor are in the appendix, Section B.1 Additionally, we included regular Feature Bagging (FB) (Lazarevic and Kumar 2005) as a baseline in Section 5.3.1 We built homogencous feature ensembles using off-the-shelf outlier detectors the outlier detection experiments. For the embedding methods, we used the embedded version of the dataset fit a singular off-the-shelf detector. Specifically, we utilized the most popular and best-performing detectors from (Han et al. 2022): LOF, kNN,
13

---

Published in Transactions on Machine Learning Research (07 (07/202) 2025)
1.0
1.0
1.0
0.5
( 0.5
0.5
0.0
0.0
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
F
F
(a) HiCS
(b) GMD
(c) V-GAN
Figure 5: Comparison of the relative scores for each subspace across different values of F.
CBLOF, ECOD, and COPOD (Breunig et al. 2000 Aggarwal 2017 with their respective recommended default parameters.
He et al. 2003 Li al. 2023
2020
All experiments were implemented in Python. We used popular implementations for all competitors and baselines and implemented V-GAN in PyTorch. We used the Python package pyod for all outlier detectors. All subspace selection methods where implemented in Python, except HiCS, which is implemented in Nim. Experiments ran on Ryzen 9 7900X CPU and an Nvidia RTX 4090 GPU.
5.2 Obtaining the Theoretical Lens Operator
To study the properties of the operator V obtained by V-GAN, we use synthetic data. Specifically, we consider population similar to Example D where a lens operator can be directly calculated. Using the with values Un diag(1, 1,0) and U2 diag(0, 0, 1), occurring with probabilities Fi = F and F2 = F, respectively, This operator is trivially lens operator. The experiment aims to extract subspaces S1 and S2 with scores F; and F5 as close possible to Fj and F2, using subspace selection method. The steps are as follows:
synthetic population described Section 5.1.1 we define a random operator
1. Generate a dataset D by sampling 10000 points from x.
2. Use D to train a given subspace selection method. Obtain the subspace qualities F; of all selected subspaces {S²} and map them into [0, 1] probabilities
by F!
A Σ,
Report the probabilitics Fg of subspaces S1 and S2.
Repeat steps T 10 times.
In step 3 we considered the subspace selection methods HiCS and GMD (Keller al. 2012 Trittenbach and Böhm
2019) apart from V-GAN. We could not include the subspace selection method CLIQUE (Agrawal et al. 2005) it does not report quality metric for the subspaces. As using 3-dimensional dataset, it will be enough to employ regular Gaussian kernel with the recommended bandwidth parameter for V-GAN's training. We reported the results in Figure 5 As we can see, V-GAN is the only method capable of properly extracting the true weight of each subspace.
5.3 One-class Classification
This section presents outlier detection experiments using V-GAN to build ensembles. The goal is to detect outliers in a test set Diest after training on an inlier train set Dtrain problem known as one-class classification Perera et al. 2021 The experimental process is as follows:
Split the dataset D into a training set traai containing 80% of the inliers from D, and test set test containing the remaining 20% and the outliers.
2. Obtain collection of K subspaces { using subspace selection method.
14

---

Published in Transactions on Machine Learning Research (07/202) 2025)
30
50
3.0
2.6
2.5
2.5
2.0
Roak
2.0
Kink
2.0
1.5
1.5
15-
0.10
30-200
200-2000
10:30
30-200
200-2000
0-30
10:
30-200
2000
(a) LOF
(b) KNN
(c) ECOD
3.0
3.0
2.5
2.5
RJ66
2.0
Ronk
2.0
V-GAN Feature Bagsing
1.5
1.5
Space
1.0
10:30
30,200
200.2000
0.10
30-200
2000
(d) COPOD
(e) CBLOF
(f) Legend
Figure 6: Boxplots of ranks of the comparison with baselines using myopic datasets with different numbers of features. The bins contained 3, 7, 5, and 6 datasets, respectively.
Table 3: Results of the Conover Iman test for the rankings against baselines on myopic datasets
LOF
kNN
ECOD
COPOD
CBLOF
FB None
V-GAN
FB
None
V-GAN
FB
None
V-GAN
FB
None
V-GAN
FB
None
V-GAN
FB
None
V-GAN
+
Given an outlier detector M, obtain {M,) by fitting M on each of the selected subspaces. As a dataset, use Durain|s, the projection of Dtrain into the subspace.
4. Evaluate the performance of each detector by reporting the AUC of the aggregated scores across all Diest S.. If K 1 (like in feature selection), use the score in Diest s.
5. Repeat steps 2 to 4 10 times.
We aim to address tw key questions about the performance of V-GAN's lens operator: (Q1) How does il compare to baselines for outlier detection, such as the full-space method and a randomly selected collection of subspaces (feature bagging)? ? (Q2) How does perform relative to other subspace selection methods and dimensionality reduction techniques? Furthermore, we will evaluate its performance on datasets with and without myopic distribution, providing insights into both the best-case scenario (where V acts lens operator) and the worst-case scenario (where V does not).
5.3.1 Comparison with Baselines (Q1)
In this section, we compare V-GAN to two classical baselines the subspace selection literature: the full-space method and Feature Bagging (FB) (Lazarevic and Kumar 2005). For FB, we chose the number of subspaces K from set of five equidistant values [rom 50 to 500. For each dataset, we selected the K yielding the highest average AUC across 10 repetitions. To aggregate scores, used weighted average based on the probability assigned to each subspace, following (Cribeiro-Ramale et al. 2024 Proposition 1). For FB, this reduces to simple average.
Furthermore, we will evaluate its performance on datasets with and without a myopic distribution, providing insights into both the best-casc scenario (where V is a lens operator, and hence x myopic) and the worst-case scenario (where V not lens operator, and hence x not myopic). To study whether this is the case for each dataset, we will study the hypothesis test presented in Section 4.2 We collected the
15

---

Published in Transactions on Machine Learning Research (07 (07/202) /025)
7.5
7.5
Rank
Rank
2.5
2.5
VGAN
CAE
ELM
HiCS
GMD
PCA
UMAP
CLIQUE
VGAN
CAE
ELM
HiCs
GMD
PCA
UMAP
CLIQUE
(a) LOF
(b) kNN
7.5
7.5
Rank
5.0
Rank
5.0
000
2.5
2.5
VGAN
CAP
ELM
HiCS
GMD
PCA
UMAP
CLIQUE
VGAN
CAE
ELM
HiCS
GMD
PCA
UMAP
CLIQUE
(c) ECOD
(d) COPOD
7.5
Rank
5.0
2.5
VGAN
CAE
ELM
HiCS
GMD
PCA
UMAP
CLIQUE
(e)
CBLOF
Figure Boxplots of ranks of the comparison with our compotitors using myopic datasots.
lest results in Tables 15 If the test returns in favor of the null hypothesis, then the operator is lens (i.e., (MDDPPxPPPs for the studied population. Further details can be found in Section B.1 in the Appendix.
Myopic Datasets. Figure shows rankings contingent dataset dimensionality group and average rankings. V-GAN demonstrates consistent performance improvements dimensionality increases, often outperforming basclines for all outlier detectors. To assess statistical significance, we apply the Conover-Iman post-hoc lest (Conover and Iman 1979), commonly used in outlier detection (Campos et al. 2016), following a preliminary positive result from the Kruskal- -rulksl-Wallis test (Kruskal and Wallis 1952). Table 3 contains the results, where '+' indicates the row method has a significantly lower median rank than the column method, and '_' indicates significantly higher rank. One symbol marks p-values 0.1, two symbols mark p-values & 0.05, and blanks indicate no significant difference. Entirely grayed-out subtables denote cases where the Kruskal-Walli test, prerequisite for using the Conover- Iman post-hoc test, not passed. V-GAN outperformed all baselines across detectors. The appendix summarizes the complete AUC results in
Tables 15
Non-myopic Datasets. Non-myopieity represents the worst-case scenario for V-GAN, as its guarantees rely on this property, Figure g in the Appendix B.] shows rank boxplots for non-myopic datasets, similar to those for myopic datasets. It evident that V-GAN's performance is not worse than any bascline, and the Conover- Mhooeer-Iman test results (Table 5) support this. V-GAN's performance in its worst-case scenario is no worse than that of tuned feature bagging (FB) and outperforms the full-space approach for some outlier detectors.
16

---

Published in Transactions on Machine Learning Research (07/2025)
Table Results of the Conover-Imar test for the rankings against competitors on myopic datasets
OD method
SS method
CAE HiCS
CLIQUE BLM
GMD PCA
UMAP V-GAN
CAE
HICS
CLIQUE
ELM
LOF
GMD
PCA
UMAP
V-GAN
HICS
CLIQUE
ELM
KNN
GMD
UMAP
V-GAN
CAE
HICS
CLIQUE
ELM
ECOD
GMD
PCA
UMAP
V-GAN
CAE
CLIQUE
COPOD
ELM
GMD
PCA
UMAP
V-GAN
CAE
HICS
CLIQUE
ELM
CBLOF
GMD
PCA
UMAP
V-GAN
5.3.2 Comparison with Competitors (Q2)
We now compare V-GAN to the competitors introduced in Section 5.1.3 (see Table 1 As before, we analyze performance separately for myopic and non-myopic datasets.
Myopic Datasets. Figure plots the ranks of all competitors for myopic datasets, V-GAN consistently achicves the lowest median rank, with GMD typically being the closest competitor. Table 4 contains the results of the Conover-Iman Lest. V-GAN siguificantly outperforms all methods and the best option for enhancing outlier detection performance under myopicity.
Non-myopic Datasets. Figure I0 plots ranks for the non-myopic case, and Table contains the Conover- Iman test results. V-GAN demonstrates closer performance to its competitors expected, but it is never statistically worse than any competitor. Therefore, we can recommend V-GAN as default approach for ensemble outlier detection using subspaces, which brings significant advantages in the myopic case while having no apparent disadvantage in the absence of myopicity.
non-myopic datasets,
5.4 Scalability
We now compare the scalability of V-GAN with other subspace search methods. For these experiments, all methods were tested with their parameters set as in Section 5.1.2. The dataset. consists of uniformly
17

---

Published in Transactions on Machine Learning Research (07/2025)
2
Time (hours)
V-GAN
GMD
CLIQUE
HICS
0
10
Features (in thousands)
Figure 8: Time laken for performing subspace search, per feature count.
generated noise with d E {0.1, 1.2, 2.3, 4.5,, 7.8, 103 features. All experiments are using a single CPU thread to ensure a fair comparison.
Figure g presents the results of scalability experiments, showing the runtime in hours required to obtain a collection of subspaces as function of the number of features. V-GAN is more scalable than all subspace search competitors: It is over 4, 30, and 8000 times faster than HiCS, GMD, and CLIQUE, respectively.
6 Limitations & Future Work
The experiments in this article center around obtaining lens operators in high-dimensional tabular data. While our specific setting work with simple axis-parallel projections, MST polentially allows arbitrary levels of complexity for (()). This, especially for high-dimensional tabular data, can lead to limitation of the interpretability of the derived operators. This limitation is a common characteristic of tabular data, leading to its own field of interpretable models for tabular data aiming to alleviate this (Yan et al. 2025 Si et al. 2023 For instance, it is clear that. domains such as image processing or NLP, the results of V-GAN are interpretable see Figures 12 and 14 We believe that further research into these interpretable datatypes can help to better interpret lens operators for tabular data.
While MST is general theory, it does not consider general stochastic structures of the form:
x. x:0x T T
w, xx() Tt,
i.c., it docs not consider time-correlated datatypes like time series, functional data, or Markov processes. A Analyzing such datatypes requires suitable exlension of MST, and thus, we consider as future work. Furthernore, we focus our experimental evaluation on outlier detection, but there exist other downstream
tasks interested in obtaining projections of the data, like clustering (Cheng et al. 1999 Agrawal et al. 2005
Qu et al. 2023 In this setting, classical approach is to employ some clustering aggregation method, like consensus clustering (Lipor et al., 2021), or hierarchical clustering (Baumgartner et al., 2004). Another interesting application, that might not necessarily consider projections, is contrastive learning. In contrastive learning, the goal is to augment the data by utilizing transformations x of samples of x with high similarity (Chen et al. 2020). This augmented data, together with other samples x= not following the same distribution as x, are used to train representation lcarning methods for a wide varicty of downstrcam tasks(Chen al. 2020 Van Gansbeke et al. 2021). MST could provide set of particular transformations given by U that have guarantees to be optimal in terms of similarity since Px = FUxx This is appealing, as know that there exists literature on obtaining better transformations by heuristically scarching from prodefined collection of transformations (Cubuk et al., 2019). Studying whether MST can derive useful transformations in practice, and how can affect the rich theoretical background of contrastive learning [Wang el al. 2021 is treated as future work.
Next, in Section 5 we performed experiments for outlier detection using V-GAN. Therefore, we set up a default choice of hyperparameters for the training. This was to study how easily one can converge to
18

---

Published in Transactions on Machine Learning Research (07/2025)
significant solution in real, practical scenarios. As we have Test 10 we can study whether the network has converged to a solution or not. While in our case one could derive lens operators effortlessly - see Table I s last column we believe that an analysis of the effect of hyperparameters in training can help with harder datatypes. Firstly, harder, higher-dimenssonall datatypes like high-rcsolution images may result in a more complicated optimization surface for Equation S Secondly, hyperparameters might, not. only affect. the optimization surface, but also Test (T0p itself. Particularly, Test (10) depends on the selected kernel for
its performance, as mentioned in (Gretton al. 2012 Bińkkwski al. 2021 Schrab et al. 2023 Lastly, while Test (10) is consistent
see Section 4.2 its consistency is only asymptotic. This means that the performance of Test. (10) depends heavily on the batch size selected - the higher the better. This is not a like text with a large token count, or high-resolution images - or when trying to use models with a high memory complexity like large transformers. These four limitations are shared between all MMD- bbseed generative models and kernel-basod inference mothods (Gretton et al., 2012 Li al. 2015). Thus, an exhaustive analysis like the one performed in (Binkowski et al. 2021 is considered an important future work.
problem for our particular setting, but it can be one for datasets of much higher dimensionality
7 Conclusions
Subspace search can improve outlier detection for off-the-shelf detector in tabular data Mülller et al. 2012 Keller et al.) 2012 Nguyen et al. 2014 Trittenbach and Böhm 2019). Tn our experiments, however, we did not observe this improvement in all datascts, with the methods sometimes failing to beat näive basclines. Besides, existing subspace search methods can hardly be applied to non-tabular data due to poor scaling. The above-mentioned factors hindered the use such methods in practice. This paper proposes a new theoretical framework that explains when subspace selection is helpful and, more importantly, how we can exploit it to our advantage. Using this theory, we introduced new way of performing subspace selection, akin to subspace search methods, that is theoretically sound, scalable, and usable in general scenarios a strategy that called subspace generation. Our first attempt in subspace generation, called V-GAN, demonstrates a significant performance increase against other basclines and competitors in the downstream task of outlier delection - one of the main use cases for subspace search. In addition, our experiments suggest that the performance increase is conditioned on the data's distribution being myopic, a property we can infer from data without any prior knowledge. Furthermore, even when the data is not myopic, V-GAN still not outperformed by its competitors. Our findings not only validate the superior performance of V-GAN for subspace selection, but also show the potential of our Myopic Subspace Theory (MST) beyond the use case of outlier detection on tabular data.
Aknowledgements
This work was supported by the Ministry of Science, Research and the Arts Baden- Maden-Württemberg. project Algorithm Engineering for the Scalability Challenge (AESC).
References
C. C. Aggarwal. Outlier Analysis. Springer International Publishing, Cham, 2017. ISBN 978-3-319-
47577-6 978-3-310-17578-3, doi: 978-3-319-47578-3
URL http:://iikk.pprr spr inger. 10mm/10.1000
R. Agrawal, J. Gehrke, D. Gunopulos, and P. Raghavan. Automatic Subspace Clustering of High Dimensional Data. Data Mining and Knowledge Discovery, 11(1):5 33, July 2005. ISSN 1573-756X. doi: 10.1007/
00018-005-1396-11 URL
M. Arbel, A. Korba, A. Salim, and A. Gretton. Maximum Mean Discrepancy Gradient Flow, Dec. 2019. URL http:// http://aarvvvoogabaass//sa wwtp://g//g/a..oo/.///
arXiv:1906.04370 [es, stat].
M. F. Balm, A. Abid, and J. Zou. Concrete Autoencoders: Differentiable Feature Selection and Reconstruction.
In Proceedings of the 36th International Conference on Machine Learning, pages 444-453. PMLR, May 2019. URT. https::/pproceediggs... mlr. wwtppp//we//aaaane//ppppa
http://ppoeeeaagaa.oo.ere//a//aa html ISSN: 2640-3498.
19

---

Published in Transactions on Machine Learning Research (07/2025)
C. Baumgartner, C. Plant, K. Railing, H.-P. Kriegel, and P. Kroger. Subspace selection for clustering high-dimensional data. In Fourth IEEE International Conference on Dala Mining (TCDM pages 11-18, Nov. 2004. doi: 10.1109/ICDM
URL https://iesexplore. leee. /rg.//occcuutt//111026611
S. Ben-David, E. Kushilevitz, and Y. Mansour. Online Learning versus Offline Learning. Machine Learning, (1):45 63, Oct. 1997. ISSN 1573-0565. doi: 10.1033/A110004433907577 URL https://doi.orr//10000333. 1007465907571
A. Berlinet and C. Thomas Thomas-Agnan Reproducing Kernel Hilbert Spaces in Probability and Statistics, Springer US, Boston, MA, 2004. ISBN 978-1-46133-7799-7777888 97711111990077771111111919999990 doi: 10.1007/988-144419990000 URL
M. Binkkwski,, D. J. Sutherland, M. Arbel, and A. Gretton. Demystifying MMD GANs, Jan. 2021. URL
http://aarx..orrg//aa////1// htt///aaa//11....11 01401 arXiv:1801.01401 [cs, stat]-
M. M. Breunig, H.-P. Kriegel, R. T. Ng, and J. Sander. LOF: identifying density-based local outliers. SIGMOD Rec., 29(2):93 104, May 2000. ISSN 0163-5808. doi: (0.1145/335191.335388 URL https //dl.acm. org/ do1/100..00011 10. 1145/335191111 335388
G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, 1. Assent, and M. E. Houle. On the evaluation of unsupervised outlier detection: mcasures, datasets, and an empirical study. Data Mining and Knowledge Discovery, 30(4):891-92) 927, July 2016. ISSN 1573-756X. doi: 10.1007/ $10618-015-0444-88 URL
T. Chen, S. Kornblith, M. Norouzi, and G. Hinton. A Simple Framework for Contrastive Learning of Visual Representations. In Proceedings of the 37th International Conference Machine Learning, pages 1597-1607. PMLR, Nov. 2020. URL htttps://proceedinnggg httprr/eeeaaas...ppppe/////d/ddd html ISSN: 2640-3498.
mlr pttppp/ss/e//eee/////aa
C.-H. Cheng, A. W. Fu, and Y. Zhang. Entropy- based subspace clustering for mining numerical data. In Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 84-93, San Diego California USA, Aug. 1999. ACM. ISBN 978-1-58113-148-7. doi: 10.1145/312129. 312199. URL https: acm wtt.oooor...11.///...
312199
W. J. Conover and R. L. Iman. Multiple-comppriioons procedures. Technical Report LA-7677-MS, Los Alamos National Lab. (LANT.), Los Alamos, NM (United States), Feb. 1979. URL /ttts::/www...
gov/bab11o/6057803
J. Crbbeiro-Ramallo V. Arzamasov, F. Matteucci, D. Wambold, and K. Böhm. Generative subspace adversarial active learning for outlier detection in multiple views of high-dimensional data, 2024. URL
https://arxiv. org/ aas//b///22// /2044. 14451
E. D. Cubuk, B. Zoph, D. Mane, V. Vasudevan, and Q. V. Le. AutoAugment: Learning Augmentation Strategies From Data. In 2019 IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pages 113-123, Long Beach, CA, USA, June 2019. IEEE. ISBN 978-1-7281-3293-3 doi: 10.1109/ LVPR.2019.00020 URL https:///ieexxploree. ieee. /ttp//cummmtt//53/////////
E. Elhamifar and R. Vidal. Sparse Subspace Clustering: Algorithm, Theory, and Applications, Feb. 2013.
URL http:///arxi..orr/aa//10033 1005 arXiv:1203.1000 [cs, math, stat].
A. M. Faden. The Existence of Regular Conditional Probabilities: Necessary and Sufficient Conditions. The Annals of Probability, 13(1):288 298, Feb. 1985. ISSN 0091-1798, 2168-894X. doi: 10.121/a0p/1176990000 URL
The-
aop/ 111111999938.ffull
Publisher: Institute of Mathematical Statistics.
K. F.R.S. Lii. on lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 2(11):550-57) 1901. doi:
20

---

Published in Transactions on Machine Learning Research (07/2025)
K. Fukumizu, A. Gretton, X.
Sun, and B. Schölkopf. Kernel Mcasures of Conditional Dependence. In Advances in Neural Information Processing Systems, volume 20. Curran Associates, Inc., 2007. URL https:// papers
cips.cc/paper... / http:///....aapa///////////////a
D. Garreau, W. Jitkrittumm, and M. Kanagawa. Large sample analysis of the median heuristic, Oct. 2018.
URL http://arxi: wwwp:/wwwww.rgbb///oooo/// 07269 arXiv:1707.07260 [math] version: 3.
1. Goodfellow, Y. Bengio, and A. Courville. Deep Learning. MIT Press, 2016. http://www.deplleem..gggoo. org
A. Gretton, K. M. Borgwardt, M. J. Rasch, B. Schölkopf, and A. Smola. A Kernel Two-Sample Test. Journal of Machine Learning Research, 13(25):723-T) 773, 2012. ISSN 1533-7928. URL htttp://jmlr.orgg/ #13/gretton122 html
V. Guillemin and A. Pollack. Differential Topology. American Mathematical Soc., 2010. ISBN 978-0-8218- 5193-7. Google-Books-ID, FdRbAQAAQBAJ.
L. Guo, W. Song, and J. Shi. Estimating multivariate density and its derivatives for mixed measurement error data. Journal of Multivariate Analysis, 191:105005, Sept. 2022. ISSN 0047-259X. doi: 10.1016/j.jmva. 2022.105005, URL
S. Han, X. Hu, H. Huang, M. Jiang, and Y. Zhao. ADBench: Anomaly Detection Benchmark, Sept. 2022. URL http:/ /ttt:///rxivv arxiv org/ aa////bb/22200 09426 arXiv:2206.092 224::2222609922 [cs].
Z. He, X. Xu, and S. Deng. Discovering cluster-based local outliers. Pattern Recognition Letters, 24 (9):1641-1650, June 2003. ISSN 0167-8655. doi:
URL https:://ww/
sciencedirect.. aeceaccddrett.coom/cceieer//rrre http://www....aeee.......eeoooor
J. Healy and L. Mclnnes. Uniform manifold approximation and projection. Nature Reviews Methods Primers, 4 4(1):1-15, Nov. 2024. ISSN 2662-8449. doi:
URL https://ww/ www.:natuer...om
022-03333
Publisher: Nature Publishing Group.
B. Jones and A. Artemiou. Revisiting the predictive power of kernel principal components. Statistics
& Probability Letters, 171:109019, Apr. 2021. ISSN 0167-7152. doi: 10. 10.1016/1ap1.2220.10000 URL
F. Keller, E. Muller, and K. Bohm. HiCS: High Contrast Subspaces for Density-Based Outlier Ranking. In 2012 JEEE 28th International Conference Data Engincering, pages 1037-1048, Apr. 2012. doi: 10. 10.1109/ICDE:2212..8. URL http:///sspppoeeeeeeeeerrrrrr/ts 026X.
ISSN: 2375-
T. Kong, Y. Tian, and H. Shen. A Fast Incremental Spectral Clustering for Large Data Sets. In 2011 12th International Conference on Parallel and Distributed Computing, Applications and Technologies, pages 1-5, Oct. 2011. doi: C0.1109/DCCAT22011 URL https://ieeeepplore .eee org//accueet//6611555 ISSN: 2379-5352.
W. H. Kruskal and W. A. Wallis. Use of ranks in one-criterion variance analysis. Journal of the American Statistical Association, 47 (7600))5583 621, 1952. doi: handfonline. http://weee.ccooce//eaeeaae httpa//n./e....c///////////0000/ http://bb//0111000011112123311 1952. 10483441
URL https:/ //www
R. Langone, O. Mauricio Agudelo, B. De Moor, and J. A. K. Suykens. Incremental kernel spectral clus- tering for online learning of non-stationary data. Neurocomputing, 139:246-2668 Sopt. 2014. ISSN 0925- 2312. doi: 10.10166j..euooo mew..m001.............6 2014.02.036. URL pJ1/80925231214004433
Lazarevic and V. Kumar. Feature bagging for outlier detection. In Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery data mining, KDD '05, pages 157-166, New York, NY, USA, Aug. 2005. Association for Computing Machinery. ISBN 978-1-39593- 135-11 doi: 10. 10.1145/1081870.008199)) URL http:p////oii...///....4///11110
21

---

Published in Transactions on Machine Learning Research (07/2025)
C.-L. Li, W.-C. Chang, Y. Cheng, Y. Yang, and B. Póczos. MMD GAN: Towards Deeper Understanding of Moment Matching Network, Nov. 2017. URL stat].
arXiv:1705.0855 [cs,
Y. Li, K. Swersky, and R. Zemel. Generative Moment Matching Networks. In Proceedings of the 32nd International Conference on Machine Learning, pages 1718-1727. PMLR, June 2015. URL https:// proceedings mlr, ppess/v37/1ile html ISSN: 1938-7228.
Z. Li, Y. Zhao, N. Botta, C. Ionescu, and X. Hu. COPOD: copula- based outlier detection. In IEEE International Conference on Data Mining (TCDM). IEFE, 2020.
2. Li, Y. Zhao, X. Hu, N. Botta, C. Tonescu, and G. H. Chen. ECOD: Unsupervised Outlier Detection Using Empirical Cumulative Distribution Functions. IEEE Transactions on Knowledge and Data Engineering, 35(12):12181 12193, Dec. 2023. ISSN 1558-2191. doi: 10.1109 10.1109/TKDE.20 TTD...0022.3558.0
URL https:/
ieeexplore.iese org/ /oc///mtttmmm/7703 Conference Name: IEEE Transactions on Knowledge and Data Enginecring.
J. Lipor, D. Hong, Y. Tan, and I.. Balzano. Subspace Clustering using Ensembles of SK8-Subspaces, Jan. 2021. URL http:///rxii: wwtg/a///////////////////
1eXiv:1709.0474: [cs].
A. Mackiewiez and W. Ratajczak. Principal components analysis (PCA). Computers & Geosciences, 19(3)3303 342, Mar. 1993. ISSN 0098-3004. doi:
URL https://www.ciieceddrreeee
M. Meilä and H. Zhang. Manifold learning: what, how, and why, Nov. 2023. URL http://arxiv.org/ab
2311. 03757 3XXx:2311.0375..
Y. Mroueh and T. Nguyen. On the Convergence of Gradient Descent in GANs: MMD GAN As Gradient Flow. In Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, pages 1720 1728. PMLR, Mar. 2021. URL htap:s::///oceeeigg...t mlr. 2640-3498.
htapooeddngg.mt...rrrr//////rrre pt.ppres/vv1/mmrruue2llaaaa html ISSN:
E. Müller, I. Assent, P. Iglesias, Y. Mülle, and K. Böhm. Outlier Ranking via Subspace Analysis in Multiple Views of the Data. In 2012 IEEE 12th International Conference on Data Mining, pages 529- 538, Dec. 2012. doi: ISSN: 2374-8486.
URL https://1eeexpploo/ wetpeppooreeee..e........eeeeeee waaeeeorg///cccueeeeee wtgg/documant//611877))
Parnumber=6413873
H.-V. Nguyen, E. Müller, and K. Böhm. A Near-Linear Time Subspace Search Scheme for Unsupervised Selection of Correlated Features. Big Data Research, 1:37-51, Aug. 2014. ISSN 2214-5796. doi: 10.1016/j. bdr.2014.07.004. URL
A. Pal, J. Sulam, and R. Vidal. Adversarial Examples Might be Avoidable: The Role of Data Concentration in Adversarial Robustness. Nov. 2023. URL httts:// www.::/wnnn/ooooomoommmm hat.rrrrmmmmdddddDaaaaaaaaaaa
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrol, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825 2830, 2011.
P. Perera, P. Oza, and V. M. Patel. One-Clase Classification: A Survey, Jan. 2021. URL http://arxivv org/abs 22g/ 03064 arXiv:2101.0306: [cs].
W. Qu, X. Xiu, H. Chen, and L. Kong. A Survey on High-Dimensional Subspace Clustering. Mathematics, 11(2):436, Jan. 2023. ISSN 2227-7390. doi:
URL http::///wwwmml... com/ Number: Publisher: Multidisciplinary Digital Publishing Inst.ttte.
2227-7390/11/2/436
W. Rudin. Real and complez analysis, 3rd ed. McGraw Hill, Inc., USA, 1987. ISBN 978-0-07-005223411
A. Schrab, I. Kim, M. Albert, B. Laurent, B. Guedj, and A. Gretton. MMD Aggregated Two-Sample Test, Aug. 2023. URL http:://ttr:: httt:///arii.orr//aa////11000 wttp:or/aabb//1....000000 15073. arXiv:2110.1503 (os, math, stat].
22

---

Published in Transactions on Machine Learning Research (07/2025)
J. Y. H. Si, M. Cooper, W. Y. Cheng, and R. Krishnan. InterpreTabNet: Enhancing Interpretability of Tabular Data Using Deep Generative Models and Large Language Models. Oct. 2023. URT https
//openreview.netee wtt..//ororrriddaaaasss5......
M. Simonnet. Measures and Probabilities. Universitext. Springer, New York, NY, 1996. ISBN 978-0-387-
94644-3 97811-612------111-- doi: 10.1007/978 1-17778----11---88 1-4612-4012-9. URL http::t/ttinmi www.:nnn.ppiiinninnncmm cow. 978-1-4612-4012-9
B. K. Sriperumbudur, A. Gretton, K. Fukumizu, B. Schölkopf, and G. R. Lanckrict. Hilbert Space Embeddings and Metrics Probability Measures. J. Mach. Learn. Res., 11:1517 1561, Aug. 2010. ISSN 1532-4435.
J. Sui, Z. Liu, L. Liu, A. Jung, and X. Li. Dynamic Sparse Subspace Clustcring for Evolving High- Sigh--imensional Data Streams. IEEE Transactions Cybernetics, 52(6):4173 1104441778866, June 2022. ISSN 2168-2275. doi:
URL htttps::/ ieee wtw.eeeee.eeoooo..... org./occumentt9269413//
H. Trittenbach and K. Böhm. Dimension-based subspace search for outlier detection. International Journal of Data Science and Anolytics, 7(2):87-101, Mar. 2019. ISSN 2364-4168. doi: 10.1007/4411600-11-001111
URL httpps://dol https://ddi.or////// ottp.....////./...o.oo 111111...000/1110000 100g1/0..000//11100011----133330
W. Van Gansbeke, Vandenhende, S. Georgoulis, and L. V. Gool. Revisiting Contrastive Methods for Unsupervised Learning Visual Reprosentations. In Advances Newral Information Processing Systems, volume 34, pages 16238 16250. Curran Associates, Inc., 2021. URL htttps:/pproceedingg/ http:oceeedingg....iip......... nwurips.cc..o
hppppr/oahhaal//rrreeeeeeeeeeeea Abstreectteebbtttt..o.oooooooe.e html
R. Vidal, Y. Ma, and S. Sastry. Generalized Principal Component Analysis (GPCA), Feb. 2012. URL http://arxiv www///rg///////////// haabs//112..444 arXiv:1202.4002 [cs].
D. Wang, C. Ding, and T. Li. K-Subspace Clustering. In W. Buntine, M. Grobelnik, D. Mladenić, and J. Shawe-Taylor, editors, Machine Learning and Knowledge Discovery in Databases, pages 506-521, Berlin, Heidelberg, 2009. Springer. ISBN 778-3-642-04174-7 doi: -LL07/9978-3-642-4417777
Wang, Q Zhang, Y. Wang, J. Yang, and Z. Lin. Chaos is a Ladder: A New Theoretical Understanding of Contrastive Learning via Augmentation Overlap. Oct. 2021. URL http:::/opparrriii.nnn///// wtw.:rriiiwi.eetoooounn......... EÖvgmYVyeUz
H. Xu, G. Pang, Y. Wang, and Y. Wang. Deep Isolation Porest for Anomaly Detection. IEEE Transactions Knowledge and Data Engineering, 35(12):12591 12604, Dec. 2023. ISSN 1558-2191. doi: M01109/TKDE 2023.3270293 URL https:///eeew:: Transactions on Knowledge and Data Engineering.
ieee org/ /dccumentt/010000444 Conference Name: IEEE
R. Yan, M. T. Islam, and L. Xing. Interpretable discovery of patterns in tabular data via spatially semantic topographic maps. Nature Biomedical Engineering, 44) Apr. 2025. ISSN 2157-846X. doi: 10. (0008//s1155--02-0011 Nature Publishing Group.
URL ottp:p///......aaa.c.c.caaaaaaap
Publisher:
M. D. Zeiler. ADADELTA: An Adaptive Learning Rate Method, Dec. 2012. URL http://arxiv.orm abs/ 1212.5701 arXiv:1212.5701 [es].
A Theoretical Appendix
This appendix contains the proofs for all the statements in Section extra general statements, and collection of examples of lens operators different spaces.
23

---

Published in Transactions on Machine Learning Research (07 (07/2025)
A.1 Myopic Subspace Theory (Extension)
We will first introduce all of the proofs of the statements from Section 3 and then introduce all of the additional statements and proofs. To maintain the clarity of this section, we will reintroduce all of the statements before their proofs.
Lemma 1. Consider H a RKHS with a characteristic kernel K; and x, U and MMD as previously defined. Further, consider V to be a lens operator for x. Then,
arg migniMMD, (Px,, 9
Proof. V lens for x Pvx = Px = MMD,(Pvx*) = 0 V E argminMMDDPss Pvx). Pvx). The first.
implication comes from the definition of lens operator, the second for being characteristie, and the last one is trivial when considering MMD,(P.)) q) 2 0, vp, E M
Theorem 2. Consider
random variable (E, operator taking values on ((x) C C(x). Further consider the associated RKHS H of functions on E with characteristic kernel E, and the induced MMD metrie on M Under these conditions, if (xx and myx)-my have that:
separable metric space and U random
(xx) compact
Given an iterative convergence strategy 3 such that 3(pn-1) = Pn E C MI and {PaINEN - p' E arg inf (IP,,((xxPPP) it follows that: PEN
3(pm-1) Pm E M
(paJneN - p' arg min MMD,, and p' M PEM,
Proof. By the definition of 3, we can construct a sequence (Pn)neN E M2 such that
{PnINEN p' arg arg MMD
(11)
pEMM(())
Since x myopic, V: Ω ( that is a lens operator for x. By Lemma and the definion of M (X)) is clear that:
PVvx E arg min MMD,,(Px p).
(12)
PEM (00)
Additionally, by the definition a lens operator,
Pvx E arg Min
(13)
PEM:
Thus, by 11
12 and 13
clear that
p' E arg MMDM PEM
Additionally, as {Pn]nEN a sequence in a compact space,
(PnIneN - p' M0(x)
Corollary 3 (Convergence to a myopic operator). Consider x a random variable on (E, a separable metric space and U continous random operator taking values on O(X) C(X). Further consider the associated RKHS H. of functions on with characteristic kernel k and the induced MMD metric on M Under this conditions, if ((X) is compact and x is O -myopic, uee have that.
24

---

Published in Transactions on Machine Learning Research (07/2025)
Given an iterative convergence strategy such that 3(Pn-1) Pn E MM and arg inf (M,,,(Pxx.))) il follows that:
PEN
{Un)eet such that 3(Pun. PU,x MMDx(1P
0, and {Un)NEN V (
Proof. Consider (pn)nen E _M such that F(pn-1) = Pn- By Zorn's Lemma, one construct parallel sequence (UnInEN such that F(pn-1) = Pn PU,x As such,
[PUx)] - p' arginf (M.(Px.P)) PEM
If M is compact, we can solve the remainder of the proof equivalently as done for Theorem Thus, we will focus on proving such a statement.
( compact {UnTneN V for all sequences in ( If we now consider {Un)NEN and V such that Un(w) Un - V = V(w) for almost all Ω it is clear that:
Plim (w) V(w)) =.
In other words, U V. Therefore, since (x) C(x) and E is a separable metric space, by the definition of almost sure convergence is clear that:
U
U 0.s Vx.
And lastly, by the convergence laws of random variables, we know that:
U,
Vx = U, d
Vx
PU,x
Pvx.
Now, introduce the result mentioned at the end of Section 3.3 This result motivates the way aggregate in our outlier detection experiments.
Proposition 4. Consider E a Rodon space, (2, F, IP) probability space, & the space of random variables E, and ( the space of operators from X to X. Further consider all U E ((x) to be defined on fibers of E, and U : Ω - 0(x) C(C) a lens operator for x E X. Lastly, consider the following conditions
i) U is such that, given any two realizations U1 and U2 (U)+U)), Pu, #0 - Pu,x (A) = 0. for A E F(Ux) - i.e., all realizations are mutually exclusive.
i) The set of all realizations of U is countable.
ii) There exists a measure μ such that μ >> Pux and μ >> Pu())x Vi S
In this case, Px Ewen
and Px Eweo (U(((())
Proof. Consider all U E ( to be defined on fibers of E. By the disintegration theorem, we know that the pushforward functions U.Ix are (probability) mcasures. Then, by (1) -id), we can apply the law of total probabilities to derive:
PUx = Σ PPU
wEst
The result for the densities (in the Radon-Nikodym sense) immediate by the Radon-Nikodym Theorem.
These conditions are trivially fulfilled by the axis-parallel case for outlier detection akin to the one Ramallo al. 2024 Proposition 1). Consider that this result focuses on the case when the operators U (x)) each reside different fibers of the space E, which is enough for our downstream setting.
25

---

Published in Transactions on Machine Learning Research (07 (07/2025)
A.2 Subspace Generation with AMD-GANs (Extension)
In this Section, will extend the results from Section 4.1 by including pseudo-code of V-GAN in Algorithm Here, we included the training for V-GAN with kernel learning. Using an identity matrix / as the encoder is sufficient to derive the pscudo-code for training without kernel learning. In practice, the simultancous training the generator G and the autoencoder (Eg.Eg') has to be done sequentially. In other words, we will train the autoencoder for a given number of epochs first, then the generator, and after that, we restart. the loop until we reach the maximum number of epochs.
A Algorithm V-GAN training
Require:
Dataset D, the RKHS kernel к, epochs, batches, number of epochs training the autoencoder iternum,,, number of epochs training the generator iternumG,
Initialize Generator Go
2: Initialize the Encoder &g and Decoder Eg"
3: for epoch E {1...., epochs) do
4: for batch E {1, ..., batches} do
5: noise + Random noise 2(1), ,..,,...." ((m) from Z dala - Draw current batch x.(1), ,..., x(m] trained trained_epochss,
0, trained trained_epochse, = if trained epochss, < iternume, then
Update Eo and Eg by ascending the stochastic gradient: V VoLiaddata, noise; 0,4) trained sennndddeechss.... +=1
10:
11:
else if trained epochss, Liternune, then Update G by descending the stochastic gradient: TgLaadata, noise; () trained trainedd_ppchhse +=1
12:
13:
if trained epochsa. 2 iternumn, & fettined_epocleee 2 iternume, then trained epochse, = O, trained epochsa, =0 16: end if
14:
15:
17: end if
18: end for
19: end for
Algorithm [ takes input. the dataset. D, the kernel K, the number of epochs and batches, and the iteration count for the autoencoder and generator iternumm, and iternumg,, respectively. During training (lines 3-19), batch of data points is drawn from D, and equal amount of random noise is sampled (lines 5-6). The update loop (lines 8-17) alternates between updating the encoder and the generator. The autoencoder is updated (lines 8-10) as long as its epoch counter trained trained_epochse, less than iternume,- After each update, the counter is incremented by 1. Once the autoencoder's counter reaches iternumgg, the generator is updated (lines 11-16) until counter trained epochac, reaches ilernumng. When both counters reach their limits, they are reset to (lines 14-16), and the process repeats.
B Experimental Appendix
In this Appendix, extend Section by including further information about experimental settings, extra images and tables from Section 5.3 and further experiments with extra data types.
B.1 One-class Classification (Extended)
In Section 5.3
we compared V-GAN with other subspace selection, embedding, and feature selection methods.
We now introduce the exact default values employed for each of them, as well as specific information regarding their implementation.
CAE (Baln et al. 2019). We followed the original CAE implementation by selecting K 20 features, fixing start and minimum temperature of 10 and 0.1, respectively, and 300 epochs with tryout limit of
26

---

Published in Transactions on Machine Learning Research (07/2025)
Table Results of the Conover- Iman test for the rankings against baselines on non-myopic datasets
LOF
KNN
ECOD
COPOD
CBLOF
None V-GAN
None
V-GAN
FB None
V-GAN
FB None V-GAN
FB None
V-GAN
FB None V-GAN
++
5. The architecture of the network, optimizer, and default learning rate were taken as-is from their official implementation.
HiCS (Keller et al., 2012). We used the only official implementation of HiCS available, together with their recommended parameters. In particular, used 100 runs with 500 subspace candidates and kept critical value for the test statistic = 0.10. We did use a different amount of output subspaces, 500, to keep it consistent with what V-GAN uses. Additionally, we added direct Python support to their source originally in Nim. The compiled binary is available for download in our code repository.
code -
CLIQUE (Agrawal et al. 2005). We used the only readily available implementation of the algorithm in Python In our experiments we employed the default values of = 3 and r 0.1.
ELM (Xu et al. 2023 We used the Extreme Learning Machines that perform the dimensionality reduction for Deep Isolation Forests as another competitor, due to the popularity and similarity of the method. In particular, we used the default architecture and parameters from their implementation in pyod. This is 50 ensemble members, a hyperbolic tangent activation layer, and a representation space of dimensionality 20.
GMD Trittenbach and Böhm 2019). We employed the only readily available implementation of GMD onlin
We employed the defauit parameters of 0.1 and 100 runs.
PCA (Maćkiewicz and Ratajezak, 1993). We used the implementation of PCA available in sklearn Pedregosa et al. 2011). For reducing the dimensionality of the data, we selected the components with the largest share of variability, until reaching 90%.
UMAP (Healy and McInnes 2024). We employed the implementation provided in their official package. We chose 15 neighbors as recommended by the authors. As for the dimensionality of the underlying manifold, the authors recommend using between 10 and 100 for downstream machine learning tasks. As the dimensionality of our datasets D varies, we opted to use min dim(()) 100)
Additionally, we included the figures and tables from the non-myopic case from our experiments. Figure 9 contains the boxplots, and Table D the Conover- both the boxplots and the Conover-Iman test for the competitors, respectively. We also included the p-values of all the Conover-Iman tests, in Tables 1 To finalize, we included the raw AUC results in Tables We fixed 5-hour time-out per repetition of the subspace search experiment, denoted by OT. Additionally if the Outlier Detection Method employed failed to report any results due to an implementation error, we reported ERR. We excluded results with errors from the ODM during the Conover-Iman test analysis, but treated time-outs as 0 AUC values when calculating the ranks. The last column of the tables contains the results of the Myopicity test with the derived operator. A 1 signifies that the p-value of the test statistic was larger than 0.10
test for the baselines. Figure I0 and Table 5 contain
i.e, that the distribution is myopic and signifies the contrary.
B.2 Other Data types
This section aims to exemplify the flexibility of the Myopic Subspace Theory (MST) to adapt to different data types. In particular, we will present preliminary experiments for both Images and Natural Language in
httter//gghhbb.mmm///ogkaaaaaann hhtpe///ghhubb.omm//dddesnnnngo
27

---

Published in Transactions on Machine Learning Research (07/202) 2025)
3.0
2.-
Roak
Monk
1.0
0-10
10-30
30-300
200-3000
0:10
10-90
30:200
200-2000
0-10
10-30
30-100
200-2000
Fanmus
Featuess
Featirns
(a) LOF
(b) KNN
(c) ECOD
3.0
2.5
2.5
2.0
V-GAN Feature Bageing
Full Space
0.10
10:30
30.200
20002000
0.10
10:30
30.200
200-2000
Features
Features
(d) COPOD
CBLOF
(f) Legend
Figure 9: Boxplots ranks of the comparison with baselines using non-myopic datasets across features
7.5
7.5
Rank
5.0
Rank
5.0
2.5
2.5
VGAN
CAE
ELM
HiCS
GMD
PCA
UMAP
CLIQUE
VGAN
CAE
ELM
HiCS
GMD
PCA
UMAP
CLIQUE
(a) LOF
(b) kNN
Rank
5.0
Rank
5.0
VGAN
CAE
ELM
HiCS
GMD
PCA
UMAP
CLIQUE
VGAN
CAE
ELM
HiCS
GMD
PCA
UMAP
CLIQUE
(c)
ECOD
(d) COPOD
7.5
Rank
5.0
2.5
VGAN
CAE
ELM
HICS
GMD
PCA
UMAP
CLIQUE
CBLOF
Figure 10: Boxplots of ranks of the comparison with our competitors using non-myopic datasets.
28

---

Published in Transactions on Machine Learning Research (07 (07/2025)
Table 6: Results of the Conover-Iman Test for the rankings against competitors on non-myopic Datasets
OD method
SS method
CAE HiCS
CLIQUE
ELM
GMD
PCA
UMAP V-GAN
HiCS
CLIQUE
ELM
LOF
GMD
PCA
UMP
V-GAN
HICS
CLIQUE
ELM
GMD
PCA
UMAP
V-GAN
CAE
HICS
CLIQUE
ECOD
GMD
PCA
UMAP
V-GAN
CAE
CLIQUE
ELM
COPOD
GMD
PCA
V-GAN
CAE
HICS
CLIQUE
ELM
CBLOF
PCA
UMAP
V-GAN
Table 7: p-values of the Conover-Iman Test for Rankings against baselines on myopic Datasets
LOF
kNN
ECOD
COPOD
CBLOP
FB
None
V-GAN
FB
None
V-GAN
FB
None
V-GAN
FB
None
V-GAN
FB
None
V-GAN
FB None
0.00
0.00
0.84
0.01
0.41
0.04
0.83
0.00
0.42
0.00
V-GAN
0.00
0.00
0.01
0.01
0.04
0.01
0.00
0.00
0.00
0.00
Table p-values of the Conover-Iman Test for Rankings against baselines on non-myopic Datasets
LOF
KNN
BOOD
COPOD
CBLOF
PB
None
V-GAN
FB Nome V-GAN
FB None V-GAN
FB
None V-GAN
FR
None V-GAN
0.80
None
0.06
0.05
0.06
0.05
V-GAN
0.80
0.05
0.80
0.05
29

---

Published in Transactions on Machine Learning Research (07 (07/2025)
Table 9: p-values of the Conover-Iman Iman Test for Rankings against competitors on myopic Datascts
Method
method
CAE
HICS
CLIQUE
ELM
GMD
UMAP
V-GAN
CAE
0.25
0.00
0.68
0.46
0.87
0.00
0.00
HICS
0.25
0.00
0.46
0.68
0.19
0.04
CLIQUE
0.00
0.00
0.00
0.00
0.00
0.36
0.00
ELM
LOF
GMD
0.46
0.68
0.00
0.74
0.36
0.00
0.01
0.00
UМAP
0.00
0.36
0.00
0.00
0.00
0.00
V-GAN
0.00
0.04
0.00
0.01
0.01
0.00
0.00
CAE
0.59
0.00
0.02
0.18
0.06
0.00
0.00
HICS
CLIQUE
0.00
0.00
0.00
0.00
0.00
0.86
0.00
KNN
ELM
0.02
0.06
0.00
0.28
0.39
0.00
0.02
GMD
0.18
0.42
0.00
0.28
0.59
0.00
0.00
PCA
0.06
0.18
0.00
0.59
0.59
0.00
0.00
UMAP
V-GAN
0.00
0.00
0.00
0.02
0.00
0.00
0.00
CAE
0.01
0.04
0.06
0.00
0.00
HICS
0.01
0.00
0.38
0.86
0.02
CLIQUE
0.04
0.00
0.00
0.00
0.00
ELM
0.06
0.38
0.00
0.29
0.00
ECOD
GMD
PCA
UMAP
V-GAN
0.00
0.02
0.00
0.00
0.03
CAE
HECS
0.03
0.00
0.38
0.70
0.03
CLIQUE
COPOD
ELM
0.21
0.38
0.00
0.21
0.00
GMD
0.01
0.70
0.00
0.21
0.07
PCA
UMAP
V-GAN
CAE
0.02
0.11
0.12
0.14
0.10
HECS
0.43
0.00
0.02
0.49
0.37
0.02
0.02
CLIQUE
0.02
0.00
0.47
0.00
0.00
0.41
0.00
ELM
0.11
0.02
0.47
0.00
0.00
0.95
0.00
GMD
0.12
0.49
0.00
0.00
0.90
0.00
0.06
PCA
UMAP
0.10
0.02
0.41
0.95
0.00
0.00
0.00
V-GAN
0.00
0.02
0.00
0.00
0.06
0.05
0.00
30

---

Published in Transactions on Machine Learning Research (07/202) 2025)
Table 10: p-values of the Conover-Tman Test for Rankings against competitors on non-myopic Datasets
OD Method
8S method
CAE
HICS
CLIQUE
ELM
GMD
PCA
UMAP
V-GAN
CAE
0.19
0.01
0.40
0.06
0.44
0.00
0.01
HiCS
CLIQUE
0.01
0.00
0.00
0.00
0.00
0.27
0.00
LOF
GMD
0.06
0.54
0.00
0.28
0.25
0.00
0.46
PCA
0.41
0.39
0.00
0.94
0.25
0.00
0.06
UMAP
0.00
0.00
0.27
0.00
0.00
0.00
0.00
V-GAN
0.01
0.18
0.00
0.07
0.46
0.06
0.00
0.03
0.06
HICS
0.03
0.00
0.83
0.40
0.88
0.00
0.48
CLIQUE
0.05
0.00
0.00
0.00
0.00
0.81
0.00
KNN
ELM
0.06
0.83
0.00
0.29
0.94
0.00
0.36
GMD)
0.00
0.40
0.00
0.29
0.32
0.00
0.88
PCA
0.05
0.88
0.00
0.94
0.32
0.40
UMAP
0.08
0.00
0.81
0.00
0.00
0.00
0.00
0.00
0.48
0.00
0.36
0.88
0.40
0.00
CAE
0.29
0.04
0.57
0.07
0.00
HICS
0.29
0.00
0.10
0.45
0.02
CLIQUE
ECOD
ELM
0.57
0.10
0.12
0.02
0.00
GMD
PCA
UMAP
V-GAN
0.00
0.02
0.00
0.00
0.13
CAE
0.85
0.02
0.57
0.05
0.00
CLIQUE
0.02
0.01
0.00
0.00
0.00
COPOD
ELM
0.57
0.71
0.00
0.16
0.01
GMD
0.05
0.07
0.00
0.16
0.22
PCA
UMAP
V-GAN
0.00
0.00
0.00
0.01
0.22
CAE
0.27
0.13
0.22
0.03
0.01
0.11
0.06
HiCS
0.27
0.01
0.02
0.27
0.38
0.01
0.41
CLIQUE
0.13
0.01
0.75
0.00
0.00
0.98
0.00
ELM
CBLÓF
GMD
0.03
0.27
0.00
0.00
0.81
0.00
0.84
0.00
0.00
UMAP
0.11
0.01
0.98
0.73
0.00
0.00
0.00
V-GAN
0.06
0.41
0.00
0.00
0.84
0.99
0.00
31

---

Published in Transactions on Machine Learning Research (07/2025)
V-GAN Myopicity
1
0
0
0
0
0
1
0
1
0
0
0
1
1
0
0
0
1
0
0
0
1
0
1
using LOF. We denoted time-outs by OT, and ODM errors by ERR.
0.8211
0.7902
0.6971
0.5998
0.5443
0.6825
0.7491
0.8978
0.9762
0.9713
0.7175
0.5895
1.0000
0.9733
0.9986
0.4901
0.7792
0.8702
0.9974
0.9327
0.5370
0.9286
0.9956
0.6578
0.9273
0.6827
0.8107
0.8173
0.6473
0.9994
0.9699
0.9771
0.9447
0.9662
0.8726
0.9760
0.5921
0.6103
0.5604
0.9173
0.4612
0.6122
ELM
0.7642
0.7647
0.6841
0.5777
0.5345
0.6584
0.7415
0.8923
0.9288
0.9543
0.6766
0.5704
1.0000
0.9638
0.9802
0.5017
0.6925
0.5741
0.9969
0.8865
0.5011
0.9207
0.9799
0.6316
0.7229
0.6425
0.7033
0.8292
0.5083
0.9643
0.9945
0.8008
0.9639
0.7374
0.7599
0.8256
0.6912
0.6225
0.5186
0.7229
0.5696
0.7306
UMAP
0.4546
0.5219
0.5280
0.5226
0.5058
0.5196
0.5942
0.8570
0.5099
0.5969
0.5184
0.5325
0.6175
0.5523
0.6970
0.4732
0.5257
0.6293
0.5501
0.4699
0.5290
0.6419
0.8313
0.5320
0.5842
0.5587
0.6144
0. 0.6614
0.4615
0.7226
0.6882
0.7038
0.6826
0.5048
0.4886
0.5517
0.5833
0.5863
0.5231
0.5587
0. 0.4293
0.5540
PCA
0.4632
0.6382
0.5489
0.5052
0.5720
0.6443
0.7449
0.9048
0.7066
0.8635
0.6981
0.3882
1.0000
0.9126
0.9996
0.3315
0.7880
0.8489
0.9970
0.5227
0.8379
0.8387
0.9944
0.6312
0.8107
0.6721
0.3545
0.7317
0.6331
0.9464
0.9871
0.9708
0.8443
0.9339
0.7349
0.9287
0.8453
0.6253
0.4715
0.6721
0.4841
0.5836
GMD
0.8127
0.6609
0.5239
0.6302
OT 0.5086 0.5905
0.6704
OT 0.6624 0.6492
0.9227
0.6187
0.7539
0.5795
0.5812
1,0000
0.7369
0.6032
0.7897
0.7447
0.8148
0.9977
0.5723
0.9384
0.7609
0.9915
0.6011
0.7895
0.8517
0.8136
0.7599
0.8757
0.9524
0.9698
0.9708
0.9446
0.9708
0.6721
0.9818
0.8934
0.6224
0.5779
0.8493
0.4786
0.8398
HiCS
OT 0,8300
OT 0.5463
OT 0.4863
0.5071
OT 0.5198
OT 0.7863
0.6405
OT 0.9275
OT 0.5891
0.5390
1.0000
OT 0.6373
OT 0.5057
0.8251
OT 0.8208
OT 0.8075
0.9972
0.9376
0.5950
0.7813
1.0000
0.6107
0.9629
OT 0.8836
0.7638
OT 0.8173
OT 0.8107
0,9524
0,9825
0.9625
0.9196
0.9767
0.8619
0.9579
0.6884
0.6365
0.5464
0.9629
0. 0.4944
0 0.8565
CAE CLIQUE
0.9593
0.9792
0.9285
0.9789
0.8391
0.9886
0.7326
0.6530
0.5810
0.9593
0.4952
0.8728
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT
0.5154
0.6613
0. 0.6336
0.5880
0.5555
0.6151
0.6122
0.8636
0.8736
0. 0.9624
0. 0.6474
O. 1.5955
0. 0.9988
0.8394
0.9964
0-3619
0.7810
0. 0.8591
0.9926
0.9190
0.4433
0.8523
0.9845
0. 0.6378
0. 0.6605
0.7542
0.7275
0.6935
0.6923
0.9018
0.8698
0.9792
0. 0.6538
0.9432
0.7698
0.9136
0.7372
0.4507
0. 0.5299
0.6613
0.4915
0.7215
Table 11: Full table of raw AUCs from Section 5.3
Feature Bagging Full Space
0.7757
0.7861
0.6900
0.5975
0.5433
0.6782
0.7328
0.8869
0.9752
0.9700
0.7151
0.5243
1.0000
0.9664
0.9988
0.3608
0.7744
0.8686
0.9968
0.9287
0.5326
0.9093
0.9944
0.6516
0.8752
0.6135
0.7818
0.8131
0.6272
0.9821
0.9916
0.9708
0.9610
0.9411
0. 0.8059
0.9662
0.7047
0.6343
0.4734
0.8752
0. 0.4873
0.9054
0.8613
0.7867
0.0920
0.5975
0.5433
0.6792
0.7389
0. 0.8891
0.9757
0.9704
0.7168
0.5513
1.0000
0.9681
0.9984
0.4305
0.7785
0.8664
0.9972
0.9311
0. 1.5304
0.9221
0.9958
0. 0.6647
0.9357
0.6727
0.8004
0.8157
0.6432
0.9756
0.9962
0.9737
0.9653
0.9645
0.8019
0.9802
0.7370
0.6422
0.5097
0.9357
0. 0.4955
0. 7,8997
Features
1555
768
768
768
768
768
512
512
512
512
512
400
166
100
64
57
36
36
36
33
33
32
30
27
21
21
21
21
19
18
16
13
12
10
9
9
8
8
6
6
5
Cardiotocograply
Lymphography
Pashion
Dataset
Internet Ads
20news
Agnews
Amazon
Imdb
Yelp
CIFAR10
MNTST-C
SVHN
speech
musk
mnist
optdigits
SpamBase Base
landsat
satellite
sat satimage-22
Ionosphere
WPBC
letter
WDBC
fault
annthyroid
cardio
Waveform
Hepatitis
pendigits
wine
vowels
PageBlocks
breastw
Stamps
WBC
Pima
yeast.
thyroid
vertebral
Wilt
32

---

Published in Transactions on Machine Learning Research (07 (07/2025)
V-GAN Myopicity
0
0
0
1
1
0
1
1
1
1
0
0
1
1
1
0
1
0
0
1
using KNN, We denoted time-outs by OT, and ODM errors by ERR.
0.7420
0. 0.6860
0.5650
0.6024
0.5294
0.6676
0.7913
0.9171
0.8993
0.9745
0.7220
0.6745
1.0000
0.9780
0.9973
0.3601
0.7857
0.8291
0.9993
0.9777
0.5128
0.9364
0.9756
0.8098
0.7167
0.8166
0.8087
0.7264
0.6580
0.9804
0.9905
0.9337
0.9572
0.9592
0.9830
0.8640
0.9686
0.5151
0.5833
0.8087
0.5214
0.7216
ELM
0.8403
0.6819
0.5569
0.5860
0.5309
0.6448
0.7671
0.9035
0.8568
0.9666
0.6984
0.5857
1.0000
0.9618
0.9844
0.3057
0.7715
0.7882
0.9993
0.9786
0.4955
0.9344
0.9168
0.8034
0.6669
0.6734
0.6553
0.6996
0.6314
0.9548
0.9969
0.4542
0.9803
0.9549
0.9066
0.9627
0.7995
0.5213
0.5955
0.6669
0.5293
0.8578
UMAP
0.6910
0.4779
0.5286
0.5735
0.4757
0. 0,4890
0.5670
0.3872
0.5003
0.5747
0.5700
0.6075
0.6791
0.4804
0.7617
0.4495
0.5057
0.5704
0.1325
0. 0.4509
0.6175
0.6701
0.4587
0.6394
0.4749
0.4294
0.5295
0.4876
0.5959
0.7399
0.6486
0.2663
0.5841
0.2513
0.6814
0.6981
0.7544
0.4849
0.5355
0. 0.4294
0. 0.4828
0. 0.5643
PCA
0.5578
0.6120
0.5310
0.5303
0.5818
0.6398
0.8621
0.9717
0.7812
0.9083
0.7149
0.5636
1.0000
0.9489
0.9994
0.3511
0.8215
0.8243
0.9990
0.5298
0.8760
0.8989
0.9803
0.7854
0.8064
0.6091
0. 0.4990
0.6822
0.5799
0.9583
0,9991
0.9625
0.9717 0.9163
0.9665
0.7860
0.9693
0.9245
0.5340
0.5883
0.6091
0.5611
0.6548
GMD
0.8814
0.6793
OT 0.4663 0.5304
0.5533
0.5898
0.6466
0.7854
0.9031
0.7432
0.8649
0.5880
0.6664
1.0000
0.8685
0.9077
0.4295
0.7624
0.8153
0.9981
0.5702
0.9621
0.8608
0.9930
0.7058
0.8263
0.8378 0.6687
0.7903
OT 0.8592 0.7457
0.7929
0.9524
0.9903
0.9708
0.9729
0.9140
0.9657
0.9735
0.4781
0.5787
0.7883
0.4913
0.7971
HiCS
OT 0.8716
OT 0.5213
OT 0.5328
OT 0.5177
OT 0.5526
OT 0.7037
OT 0.8477
0.6464
OT 0.9279
0.6034
0.5346
1.0000
0.8349
OT 0.5289
OT 0.7329
OT 0.7824
OT 0.7941
OT 0.9983
OT 0.9646
OT 0.5709
OT 0,8602
OT 0,9958
0.7169
0.8065
0.7502
OT 0.7811
0.9583
0,9943
0.9417
0.9679
0.9547
0. 0.9644
0.9631
0.9186
0.5249
0.5606
0.8065
0.5254
0.8493
CAE CLIQUE
0.7954
0.9417
0.9572
0.9738
0.9571
0.9886
0.8767
0.5431
0.5710
0.7954
0.5278
0.8257
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT
0.6037
0.6659
0.5092
0.5717
0.5526
0. 0.6019
0. 0.6933
0.8400
0.7859
0.9658
0.6407
0.5486
1.0000
0.8867
0.9975
0.3758
0.7815
0. 0.8304
0.9992
0.9690
0.4801
0.9261
0.9338
0. 0.7886
0.5823
O. 0.5419
0.7598
0.7220
0.6331
0.7321
0.9805
0.9542
0.7430
0.9646
0.8457
0.9240
0.6814
0.4660
0.5192
0. 0.5823
0.4770
0.6910
Table 12: Full table of raw AUCs from Section 5.3
Features Feature Bagging Full Space
0.8801
0.6847
0.5096
0.6019
0.5285
0.6668
0.7882
0.9161
0. 0.8058
0.9745
0.7205
0.6503
1.0000
0.9779
0.9981
0.3584
0.7842
0.8295
0.9993
0.9771
0.5191
0.9305
0.9718
0. 0.8093
0.7681
0. 0.6976
0.7187
0. 0.8040
0.7337
0.9583
0.9988
0.9417
0.9806
0.9805
0.9245
0.9880
0.7814
0.5503
0.5871
0.7681
0.5333
0.8620
0 0.8914
0. 0.6852
0.5689
0.6019
0.5288
0.6668
0.7898
0.9169
0. 1.8960
0.9748
0.7219
0.6570
1.0000
0.9771
0.9979
0.3579
0.7850
0.8290
0. 0.9993
0.9767
0.5091
0.9350
0.9742
0.8105
0.7092
0.8153
0.7816
0.7190
0.7284
0.9577
0.9987
0.9396
0.9836
0.9720
0.9876
0.8270
0.9415
0.5518
0.5834
0.7816
0.5260
0.8434
1555
768
768
768
768
768
512
512
512
512
512
400
166
100
64
57
36
36
36
33
33
32
30
27
21
21
21
21
19
18
16
13
12
10
9
9
8
8
6
6
5
Cardiotocograply
Lymphography
Pashion
Dataset
Internet Ads
20news
Agnews
Amazon
Imdb
Yelp
CIFAR10
MNTST-C
SVHN
speech
musk
mnist
optdigits
SpamBase Base
landsat
satellite
sat satimage-22
Ionosphere
WPBC
letter
WDBC
fault
annthyroid
cardio
Waveform
Hepatitis
pendigits
wine
vowels
PageBlocks
breastw
Stamps
WBC
Pima
yeast.
thyroid
vertebral
Wilt
33

---

Published in Transactions on Machine Learning Research (07 (07/2025)
V-GAN Myopicity
using ECOD. We denoted time-outs by OT, and ODM errors by ERR.
0
0
0
1
1
0
1
1
1
1
0
0
1
1
1
0
1
0
0
1
0.6350
0. 0.6556
0.5094
0.5640
0.5043
0.6020
0.7229
0.8216
0.6490
0.9415
0.5653
0.5420
0.8683
0.7822
0.5615
0.5943
0.4326
0.6985
0.9932
0.8405
0.5077
0.5107
0.9477
0.4442
0.7521
0.5129
0.7386
0.7118
0.6402
0.9304
0.8725
0.8900
0.4929
0.9468
0.7856
0.9113
0.7935
0.5481
0.6014
0.7501
0.5194
0.3968
ELM
0.3670
0.6510
0.4916
0.5675
0.4941
0.5930
0.7218
0.8118
0.6364
0.8566
0.5232
0.5420
0.1949
0.8846
0.3226
0.3359
0.3224
0.6694
0.9917
ERR 0.8720
0.4228
ERR 0.4993
0.9565
0.3800
0.5977
0.1092
0.6140
ERR 0.7609
ERR 0.4450
0.9452
0.8313
0.7492
ERR 0.4261
0.9638
0.3482
0.6948
0.4526
0.4825
0.3393
0.5977
ERR 0.2838
0.4220
UMAP
0.2977
0.5953
ERR 0.5182
ERR 0.5063
ERR 0.5334
ERR 0.5488
ERR 0.4936
ERR 0.5920
FRR 0.4760
ERR 0.5296
0.3949
0.4251
0.4911
0.5676
ERR
FRR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR ERR
ERR
ERR
ERR ERR
ERR ERR
ERR ERR
ERR
FRR
ERR
ERR
ERR
PCA
ERR
0.6552
ERR
FRR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
FRR
ERR
ERR
ERR
ERR
GMD
0.7114
0.6198
0.5093
0.4939
0.5509
0.5849
0.6562
0.9393
0.7073
0.8127
0.5634
0.5814
0.6342
0.7024
0.6191
0.6351
0,5084 0.4487
0.6801
0.9916
0.4972
0.8097
0.4862
0.9620
0.4269
0.6585
0.6742
0.7808
0.7235
0.6272
0.9464
0.8998
0.8417
0.3682
0.9356
0.8384 0.8953
0.8933
0.8742
0.5159
0.5595
0.6732
0.5167
0.4109
HiCS
0.7746
OT 0.4776
0.4646
OT 0.5015
0.5130
OT 0.5023
0.7027
OT 0.8216
OT 0.6288
OT 0.9234
0.6007
OT 0.5325
OT 0.0430
0.6536
OT 0.4910
OT 0.8083
OT 0.6199
OT 0.9859
OT 0.7944
0.4801
0,4861
0,9648
0.4056
0.7513
0.7397
0.7322
OT 0.6922
OT 0.6095
0.9643
0,8996
0.8708
0.2 0.2231
0.9482
0.8715
0.8349
0.5422
0.5843
0.7543
0.5143
0.4068
CAE CLIQUE
0.7527
0.8017
0.4752
0.9472
0.7836
0.9095
0.7930
0.5494
0.5989
0.7527
0.5278
0.3970
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT
0.3337
0.6032
0.4934
0.5604
0.4834
0.5605
0. 0.6382
0.6934
0.5242
0. 0.9331
0. 0.4566
0.4884
0.1663
0.7719
0.4640
0.2578
0.4292
0.6802
0.9949
0.8335
0.4638
0,4830
0.9324
0.4110
0.6448
0. 0.4407
0.7176
0.6209
0.6391
0.7857
0.8053
0.9292
0. 0.4044
0.9342
0.2853
0.8387
0.6442
0.5256
0.4625
0. 0.6333
0.5774
0.3821
Table 13: Full table of raw AUCs from Section 5.3
Feature Bagging Full Space
0.6990
0.6547
0.5000
0.5635
0.5052
0.6024
0.7222
0.8213
0.6470
0.9403
0.5608
0.5421
0.8681
0.7478
0.5608
0.5962
1.4310
0.6983
0.9931
0.8409
0.5085
0.5112
0.9479
0. 0.4430
0.7513
0.5107
0. 0.7375
0.7128
0.6391
0.9286
0.8709
0.8917
0.4596
0.9472
0.7884
0.9095
0.7977
0.5474
0.6016
0.7513
0.5159
0.3970
0.6994
0. 0.6547
0.5089
0.5635
0.5052
0.6023
0.7228
0.8215
0.6473
0.9399
0.5609
0.5451
0. 0.8685
0.7476
0.5611
0.5957
0.4304
0.6979
0.9932
0.8406
0.5090
0.5106
0.9486
0.4431
0.7514
0.5101
0.7369
0.7132
0.6361
0.9327
0.8690
0.8913
0.4608
0.9475
0.7857
0.9085
0.7944
0.5485
0.6006
0.7514
0.5171
0.3977
Features
1555
768
768
768
768
768
512
512
512
512
512
400
166
100
64
57
36
36
36
33
33
32
30
27
21
21
21
21
19
18
16
13
12
10
9
9
8
8
6
6
5
Cardiotocograply
Lymphography
Pashion
Dataset
Internet Ads
20news
Agnews
Amazon
Imdb
Yelp
CIFAR10
MNTST-C
SVHN
speech
musk
mnist
optdigits
SpamBase Base
landsat
satellite
sat satimage-22
Ionosphere
WPBC
letter
WDBC
fault
annthyroid
cardio
Waveform
Hepatitis
pendigits
wine
vowels
PageBlocks
breastw
Stamps
WBC
Pima
yeast.
thyroid
vertebral
Wilt
34

---

Published in Transactions on Machine Learning Research (07 (07/2025)
V-GAN Myopicity
using COPOD. We denoted time-outs out--uuts by OT, and ODM errors by ERR.
0
0
0
1
1
0
1
1
1
1
0
0
1
1
1
0
1
0
0
1
0.6356
0.6492
0.5242
0.5731
0.5121
0.6190
0.7131
0.8002
0.6440
0.9717
0.5410
0.5950
0.7637
0.7430
0.6522
0.6213
0.6429
0.7321
0.9944
0.8497
0.4465
0.5564
0.7939
0.4034
0.7599
0.7837
0.6097
0.8330
0.6621
0.9423
0. 0.8999
0.6042
0.4176
0.9534
0.8445
0.9579
0.8044
0.5701
0.4589
0.7499
0.4105
0.5191
ELM
0.4269
0.6347
0.4818
0.5719
0.5029
0.5945
0.7454
0.7817
0.6409
0.9460
0.5969
0.5453
0.3524
0.8736
0.3467
0.3679
0.3353
0.7075
0.9978
0.8718
0.4087
0.5205
0.9445
0.3900
0.5686
0.1989
0.6057
ERR 0.8110
0.4716
0.9536
0.8613
0.6925
0.4760
0.9659
0. 0.4579
0.9019
0.5363
0.4992
0.3536
0.5686
ERR 0.3143
0.5057
UMAP
0.2415
0.5922
ERR 0.5110
ERR 0.5220
ERR 0.5374
ERR 0.5219
ERR 0.4877
ERR 0.5923
FRR 0.4320
ERR 0.5365
ERR 0.4338
ERR 0.3711
0.4921
ERR 0.5500
ERR
FRR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR ERR
ERR
ERR
ERR
ERR ERR
ERR ERR
ERR
ERR
FRR
ERR
ERR
ERR
PCA
ERR
0.6030
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
ERR
FRR
ERR
ERR
ERR
ERR
GMD
0.7117
0.6185
0.4985
0.4984
0.5488
OT 0.4704 0.5913
0.7027 0.6481
0.9703
0.6387
0.7939
0.5407
0.6189
0.5693
0.6977
0.6764
0.6552
0.6257
0.7087
0.9861 0.9923
0.8127 0.4468
0.8360
0.5355
0.8211
0.3710
0.7704
0.6177
0.8992
0.5962
0.6391
0.9167
0.9079
0.5833
0.3070
0.9410
0.8977
0.9542
0.9179
0.5441
0.4833
0.6180
0.3952
0.5282
HiCS
OT 0.7748
OT 0.4755
0.4661
0.4758
0.4716
OT 0.8216
OT 0.6288
OT 0.9234
0.6008
0.5562
OT 0.1501
OT 0.6248
OT 0.4930
0.8106
OT 0.6256
0.6295
OT 0.4376
OT 0.5524
0.8324
0.3885
0.7534
0.7832
0.6082
OT 0.8003
OT 0.6627
OT 0,9464
0.9187
0.5458
0. 0.2643
0.9559
0.8876
0.9209
0.8651
0.5415
0.4503
0.7534
0.3746
0.5302
CAE CLIQUE
0.7528
0.6083
0.4285
0.9537
0.8397
0.9579
0.8023
0.5710
0.4631
0.7528
0.4087
0.5196
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT
0.4041
0.6107
0.4441
0. 0.5608
0.5178
0.5638
0.6685
0.6227
0.4702
0. 0.9638
0.4512
0.5854
0.1850
0.7218
0.6553
0.2214
0.6065
0.7100
0.9952
0.8148
0.3652
0.5152
0.7915
0.3901
0.5694
0. 0.3937
0.7371
0.7212
0.4852
0.6964
0.8096
0. 0.7333
0. 0.3800
0.9270
0.1872
0.9386
0.7558
0.5171
0. 0.4297
0. 0.5694
0.4198
0.4749
Table 14: Full table of raw AUCs from Section 5.3
Features Feature Bagging Full Space
0.6994
0.6482
0.5235
0.5715
0.5118
0.6186
0.7128
0.7999
0.6436
0.9720
0.5362
0.5891
0.7597
0.7422
0.6511
0.6233
0.6416
0.7318
0.9944
0.8499
0.4455
0.5558
0.7944
0.4020
0.7522
0.7808
0.6103
0. 0.8280
0.6627
0.9405
0.8988
0.6083
0.4005
0.9537
0.8452
0.9579
0.8047
0.5681
0.4576
0.7522
0.4111
0.5196
0.7000
0.6480
0.5235
0.5715
0.5119
0.6186
0.7126
0.7996
0.6441
0.9722
0.5357
0.5885
0.7623
0.7418
0.6509
0.6223
0.6420
0.7318
0.9944
0.8499
0.4465
0.5559
0.7954
0.4025
0.7539
0.7797
0.6114
0. 1.82884
0.6568
0.9417
0. 0.8993
0.5942
0.4099
0.9534
0.8441
0.9580
0.8058
0.5686
0.4574
0.7539
0.4110
0.5192
1555
768
768
768
768
768
512
512
512
512
512
400
166
100
64
57
36
36
36
33
33
32
30
27
21
21
21
21
19
18
16
13
12
10
9
9
8
8
6
6
5
Cardiotocograply
Lymphography
Pashion
Dataset
Internet Ads
20news
Agnews
Amazon
Imdb
Yelp
CIFAR10
MNTST-C
aiVeec-AD
SVHN
speech
musk
mnist
optdigits
SpamBase Base
landsat
satellite
sat satimage-22
Ionosphere
WPBC
letter
WDBC
fault
annthyroid
cardio
Waveform
Hepatitis
pendigits
wine
vowels
PageBlocks
breastw
Stamps
WBC
Pima
yeast.
thyroid
vertebral
Wilt

---

Published in Transactions on Machine Learning Research (07 (07/2025)
Myopicity
using CBLOF, We denoted time-outs by OT, and ODM errors by ERR.
0
0
1
1
0
1
1
1
1
0
0
1
1
1
0
0
1
V-GAN
ERR
0.6788
ERR
0.8864
0.8015
0.9757
ERR
ERR
1. 1.0000
ERR
ERR.
0.3924
0.7628
0.8196
0.9995
0.9486
0.5258
0.8298
0.9797
0.7760
0.6631
0.6990
0.7761
0.7979
0.5769
0.9804
0.9506
0.9246
0.8797
0.9724
0.9291
0.9750
0.7914
0.5604
0.5701
0.6591
0.4733
0.7403
0.6042
0.4465
0.4589
0.6190
ELM
ERR
0.6650
ERR
ERR
ERR
ERR
ERR
ERR
FRR
0.9588
ERR
ERR
ERR
ERR
ERR
0.3331
0.7317
0.7855
0.9990
0.9647
0.4844
ERR
0.9513
ERR
0.6109
ERR
0.6647
ERR
0.6166
0.9548
ERR
0.5398
ERR
0.9725
0.8548
0.9462
0.7213
0.5388
0.6109
0.4952
0.7365
UMAP
0.3566
0.5232
0.5312
0.5161
0.5151
0.5349
0.5709
0.6374
0.5110
0.5750
0.5181
0.5406
0.3957
0.5780
0.7542
0.5056
0.6178
0.6327
0.7356
0.4682
0.5247
0.5806
0.6718
0.5132
0.4919
0.6238
0.5623
0.5700
0.4885
0.7280
0.4131
0.1929
0.6589
0.8058
0.5253
0.4891
0.7153
0.5443
0.5429
0.6238
0.4916
0. 0.5858
PCA
0.1804
0.6154
0.5357
0.5138
0.5667
0.6092
ERR 0.7314
ERR 0.9738
0.7547
0.8571
0.6729
0.3581
0.9998
ERR
0.8529
0.4038
0.7972
0.8072 0.8030
0.9987
0.9434 0.5442 0.5255
0.8279
0.7145
0.9859
0.6748 0.7142
0.7800
0.6129
ERR
0.7142
0.5503
0.9762
0.9795
0.8996 0.8375
0.9288
0.9664
0.7395
0.9558
0.8979
0.5702
0.5502
0.6129
0.5825
0.5999
GMD
ERR
0.6906
0.5323
0.5449
0.5865
0.6398
0.7391
0.8407
0.5935
FRR
1,0000
0.9247
0.8250
0.3576
0.7337
0.9979
0.9523
0.8040
0.9693
0.7971
0.6037
0.8350
0.7812
0.7675
0.9542
0.9664
ERR 0.9353
0.9712
0.8826
0.9728
0.9587
0.5696
0.5786
0.5980
0.4702
0.6892
HiCS
ERR
0.4792
0.5193
0.4940
0.5211
0.5265
0.7061
ERR
FRR
ERR
ERR
FRR
0.9981
ERR
ERR
0.6151
OT 0.8156
0.7985
0.9979.
0.5423
OT 0.8223
OT 0,9901
OT 0.6983
0.6661
ERR
ERR
ERR
OT 0.5030
OT 0.9583
ERR
0.8639
0. 0.9698
0. 0.9364
0.9649
0.8756
0.5766
0.5670
0.6731
0. 0.4927
0.7651
CAE CLIQUE
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT
OT 0.6682
OT
OT
OT
0.9042
ERR
0.9715
0.9245
0.9888
0.8281
0.5006
0.5590
0.6687
0.5042
0.7171
0.4141
0.5954
0.4909
0. 0.5550
0.5489
0.3595
0.6747
0.7742
0.7079
0.9679
ERR
FRR
0.9 0.9234
ERR
ERR
0.4160
0.7566
0. 0.8244
0.9988
0.9155
0.5691
0.7445
0.9266
0. 0.7229
0.6131
0. 0.5914
0.6766
0.7152
0.5811
0.7905
0.9733
0.9342
0. 0.5752
0.9484
0.7112
0.9285
0.5772
0.4886
0.5064
0. 0.6131
0. 0.4322
0.5734
5.3
Table 15: Full table of raw AUCs from Section
Feature Bagging Full Space
0.7211
0.6680
0.7543
0.8805
0.7945
0.9790
0.6966
0.5382
1.0000
0.9491
0.9230
0. 0.4077
0.7414
0.8170
0.9991
0.9631
0.5759
0.8009
0.9915
0.7512
0.6615
0.6288
0.8073
0.7940
0.7219
0.9643
0.9798
0.8542
0.9374
0.9694
0.9011
0.9802
0.6349
0.5583
0.5675
0.6615
0.5254
0.7151
0.6083
0.4454
0. 0.4576
0.6186
0.7347
0.6779
ERR
0.8843
0.7984
0.9767
0.6997
ERR
1.0000
ERR
ERR
0.3935
0.7634
0.8194
0.9989
0.9501
0.5294
0.8236
0.9835
0.7743
0.6590
0.6793
0.7224
ERR
0.7414
0.9689
0.9773
0.9317
ERR
0.9710
0.9139
0.9886
0.7842
0.5891
0.5777
0.6590
0.5151
0.7218
0.5942
0.4465
0.4574
0.6186
Features
1555
768
768
768
768
768
512
512
512
512
512
400
166
100
64
57
36
36
36
33
33
32
30
27
21
21
21
21
19
18
16
13
12
10
9
9
8
8
6
6
5
Cardiotocograply
Lymphography
Pashion
Dataset
Internet Ads
20news
Agnews
Amazon
Imdb
Yelp
CIFAR10
MNTST-C
SVHN
speech
musk
mnist
optdigits
SpamBase Base
landsat
satellite
sat satimage-22
Ionosphere
WPBC
letter
WDBC
fault
annthyroid
cardio
Waveform
Hepatitis
pendigits
wine
vowels
PageBlocks
breastw
Stamps
WBC
Pima
yeast.
thyroid
vertebral
Wilt
36

---

Published in Transactions on Machine Learning Research (07 (07/2025)
vgan
Linear
Gauscian Notse
vgan Loc
Linear
Batch
-xiiss -xiision
LeakyyelU
Butch norm
LeakyReld
i- Upper sottmax
Go
Ga
Battch disc
Up per softmax
+G,(+) U (
+G,0+ U. (
G
mij
muj
G, MMD(Fx, P Fog
2,) ) MMD (x. Faword
(a) Image data architecture
(b) Text data architecture
Figure 11: Diagram of the network for both image (left) and text (right) data.
what follows. Our goal will be to exploit the human--riende nature of images and text to visualize what a lens operator looks like in these cases. We include more specific details regarding all implementations and the training for both data types in their respective repositories. To encourage further research on MST, we are releasing all original data, code, and even model weights for all experiments for both data types TT
B.2.1 V-GAN vision: Myopic Subspace Theory on Image Data
We will first explore the results of applying V-GAN to image data. For this experiment, we defined E 3- and ( = (t)))) are considering to be the vectorized space of all 3 RGB channels of an image, and ((x) to be the space of (h w>) (h w=) binary diagonal matrices acting on random images as:
with h and w being the height and width of an image. Therefore, we
( UU:
X - x
(14)
x ~ Ur = (UIR Una Unn).
In other words, are applying the operators simultancously in all color channels. To make V-GAN generate such subspaces, the only required change is to alter the head of the network to output a binary vector of the required size. In particular, we changed the layers of V-GAN in order to better handle image data. particular, we employ 5 sequential layers, each consisting of linear layer for the learning, Gaussian noise layer and Batch Normalization layer for regularization, and a Leaky ReLU as an activation. After this, we include a last Batch Discrimination layer and final linear layer, before passing the output into an upper softmax. The kernel that use is Gaussian kernel composed with and we did not employ kernel learning during training. We included a summary in Figure 11 We will study V-GAN's lens operators for image data with both real and synthetic data.
mmageNet-pretraine Autoencoder,
Real Data. We chose two popular datasets, FashionMNIST and MVTee-AD. The first consists of 28x28 grey-scaled images of clothes, while the second consists of 900x900 colored images of different industrial materials. We used the classes pants (images pants) and bottle (cross-sections of steel bottles), respectively. Figure 12 contains the results for both FashionMNIST (Figure 12a] and MVTec-AD (Figure 12b).. As we can see, the lens operators managed to extract interesting similarity patterns for each class. While in FashionMNIST, operators are mostly pant-shaped, in Meeec-AD, more complex pattern emerged. Here, the operators selected different ring-shaped parts of the cross-scctions rather than chunks of the image. These results further validate our theoretical claims that lens operators can extract complex relations in the data.
Synthetic Data. To mirror the methodology in Section 5.2 we generated 10,000 synthetic images (32x32 pixels) with half- half-white//l half- black sections. In particular, 5000 images had white top part and black bottom part, while the remaining 5000 had the inverse. Logically, as what happened with Section's 's
Forkeed from the original repository
37

---

Published in Transactions on Machine Learning Research (07 (07/)025)
U/
U5
Us
U:
U.
Average
U
Ui
Average
Original
Original
1
(a) FINHT (28x28)
(b) MVTec-AD (rescaled 64x64)
Visualization of the lens operators obtained using V-GAN in FashionMNIST and MVTec-AD. the top row of both figures, we plotted realizations of U, and final average between 500 samples. On the left-most column of both figures, plotted 5 non-altered images. In the remaining columns and rows, we plotted each image after applying the corresponding operator in the column.
Figure 12:
U
U
Us
U4
U.
Average
Original
Figure 13: Visualization of the lens operators obtained using V-GAN in syuthetic data. the top row of the figure, plotted 5 realizations of U, and final average between 500 samples. On the left-most column, plotted non-altered images. In the remaining columns and rows, plotted each image after applying the corresponding operator in the column.
population, one would expect that a lens operator consists of the two parts with equal probability. As we can see in Figure 13 the derived lens operator was exactly as we expected, further strengthening our derivations.
B.2.2 V-GAN text: Myopic Subspace Theory on Natural Language
We will try to extract lens operators from the token space Natural Language data directly. We could do this by considering F No and ((x) Diagaxa ((0,1)) where is the predifined sentence length for each
38

---

Published in Transactions on Machine Learning Research (07 (07/202) 2025)
1.0
0.8
Sumple 1:
Bestert Veneruet are turned ast earlyy In large rembers Sunday to vote atstortc reterecaia wing Presidentt Hays Chavez affice or give his mandati
Uhat cither
0.4
Somple
Rosters South Karsen police water cornon Le central Sccel Sunday to
least
protesters Lur ging the governnent reverse contreversial decision send more treeps Iree.
0.2
Sumple 3:
Reuters Thourands of Palestintan prisze ere in Esrnnts jasle began hanger strice for better cord Sarday Israet 's security masister said te cidh
they starved seath
10
20
30
40
50
APP gustire and caelling tast place overnight
Eteputed Georgion South Dos ct violation of fragile cossefire. vounding seven Goorgian servic enen.
Taken Position
(a) Average selected token by U in AGnews
(b) Average selected token in AGnews' text
1.0
0.8
0.6
Sample 2:
bl. bed
tonight tired feeling pretty
this
Sample 2:
feel Like in dooned before ive even began
Sample 3:
0.0
Average
feel ashaned with such prolific
20
Token Position
feel tonely again
(c) Average selected token by U in Emotions
(d) Average selected token in Emotions' text
Visualization of the lens operators obtained using V-GAN in AGnews and Emotions. The first row corresponds to AGnews, and the second to Emotions. The left figures contain the average probability of selecting each token using U. The right figures contain a visualization of the previous average probability on top of 4 text samples.
Figure 14:
dataset. We also utilize a different architecture than before, featuring 3 sequential layers, each consisting of a Linear layer, a leaky ReI.U, and a Batch Normalization layer. After them, the outputs are passed into final Linear layer with upper softmax output for obtaining the operators. We use simple Gaussian kernel during training with no kernel learning.
We planned similar experiments utilizing real data for Natural Language. In particular, we employed the AG news dataset and the Emotions datasct. The first one contains descriptions of news articles from different outlets, and the latter contains tweets about sentiments and feelings. Figure 14 contains the results for both.
In AGnews, Figure 14a shows how the derived lens operator U selects, on average, the beginning and the end of the sentence. By Figure 14b] this means that U on average selects the news outlet the beginning of cach sample and the core event of the news the end of each sample. A similar behaviour can be observed for Emotions, where we can see that the network focuses selecting the end of each tweet, corresponding to the actual feeling see Figures 14c and [4d] These results motivate the possible use of Myopic Subspace Theory also for Natural Language, further strengthening the applicability of this theory.
39


Reviews:
[{"review_id": "pjcmAJ88lS", "reviewer": "Reviewer_DTvK", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n\n1. Studying theoretical properties of data distributions that concentrate on lower dimensional manifolds is an important exercise, and new approaches to modeling properties of such distributions is valuable to the community. \n\n2. The present paper comprehensively identifies, and fixes holes in the prior work it extends from, and as such presents a well rounded theory of myopic distributions, and their computational properties. \n\n### Weaknesses\n\n1. The treatment of related work is incomplete, and should be greatly improved. In particular, this reviewer disagrees with some of the statements in the related work section, and some existing methods like K-subspaces are skipped, which seem to satisfy all criteria in Table 1. Details in requested changes (1, 2, 3, 4, 5). \n\n2. There is a need for a lot more contrast of the notion of \"myopic distributions\" to several notions of \"low dimensional distributions\" or more generally \"localized distributions\" available in the literature. Details in requested changes (6). \n\n3. Concrete, mathematically precise examples of a few different myopic distributions is missing. The content is spread out between Example 1, Page 4, and elsewhere. Beyond extremely degenerate examples, this makes it difficult for a reader to have a mental picture of a distribution which (A) admits a lens operator, and (B) is somewhat similar to how a real world distribution would look like. Details in requested changes (7)\n\n4. The experiments, while insightful, seem to be a bit detached from the motivating theory: while myopicity is defined in Defn. 2 to have $P_{x}  = P_{Ux}$, the practical procedures only strive for $ {\\rm} dist(P_x, P_{Ux}) $ to be small. While the reviewer understands that developing a theory for such \"approximate myopicity\" might be outside the scope of this paper, it would be great to know what are the challenges, and more importantly, how does approximate myopicity differ qualitatively from the exact case. Details in requested changes (8)", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper extends theory from prior work on identifying \"myopic distributions\", which, at a high-level, are probability distributions which remain the same when viewed under the operation of a \"lens operator\". As an example, if the lens is an orthonormal matrix $U$, then the myopic distribution is supported on a single subspace whose orthonormal projector is $U$. The paper identifies shortcomings of the prior work and outlines an improved theory for general myopic distributions supporting general random lens operators. \n\nThe paper then relaxes the general theory to propose practical schemes for identifying lens operators for \"almost myopic\" distributions, under whose action the distribution remains almost the same (in some notion of distributional distance, the authors use maximum mean discrepancy). Such operators are obtained on real data distributions, and compared to baselines for the task of outlier detection.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. The treatment of related work is incomplete. Specifically, methods in the sub-field of subspace clustering (which the authors group in Subspace Discovery methods in the related work) are not treated sufficiently. The problem of identifying multiple subspaces in data is central to subspace clustering, and several algorithms with theoretical guarantees for success of subspace-identification methods have proposed. Here is a (very small, incomplete) list which the authors can use to find other references:\n\n[1]: Provable Self-Representation Based Outlier Detection in a Union of Subspaces: https://openaccess.thecvf.com/content_cvpr_2017/html/You_Provable_Self-Representation_Based_CVPR_2017_paper.html\n\n[2]: Sparse methods for learning multiple subspaces from large-scale, corrupted and imbalanced data: https://jscholarship.library.jhu.edu/server/api/core/bitstreams/2dfff1da-9ef7-4ba1-9444-a2d7913fee56/content\n\n[3]: Estimation of Subspace Arrangements with Applications in Modeling and Segmenting Mixed Data: https://people.eecs.berkeley.edu/~yima/psfile/Ma-SIREV07.pdf\n\n[4]: Noisy Sparse Subspace Clustering: https://jmlr.org/papers/volume17/13-354/13-354.pdf\n\n---\n\n2. Please add K-subspaces to related work, which seems to be able to do all three ticks in Table 1. Also see: \n\n[5]: k-means projective clustering. https://dl.acm.org/doi/10.1145/1055558.1055581\n\n[6]: Nearest q-Flat to m Points. https://link.springer.com/article/10.1023/A:1004678431677\n\n---\n\n3. What PCA fall into? It does not seem to be subspace search, as the subspace need not be axis-aligned. It does not seem to be subspace discovery because it explicitly outputs a subspace and the projection matrix. \n\n---\n\n4. > no work in the subspace search literature offers a theoretical definition of what an \"important\" subspace is.\n\n\tThis requires a rewording with references and a toning down of the contrast: subspace clustering literature, and more generally any clustering work always grapples with the problem that \"accurately finding clusters\" is an ill-defined problem. The same data distribution can be seen to be having 1 cluster at a coarse level, and 100 clusters at a finer level. Typically one assumes some ideal \"good clustering\" (e.g., dimension of the subspaces should be small, $\\ell_2$ radius of the clusters should be small, etc.), and determines whether the subspace structure was captured according to that model. It is unclear what the authors mean by \"accurately preserving data properties\" here, and this makes the tick and crosses in the \"representation\" column of Fig 1 ambiguous.\n\n---\n\n5. > [...] they do not provide a way to project the data [...]. \n\n\tThe reviewer disagrees with this statement. Computing projections is an intermediate step in subspace clustering methods. In fact, spectral clustering methods work precisely because this low dimensional projection contains task-specific structures. \n\n\tIn its most basic version, the projections of the points are obtained by computing the eigenvectors of the laplacian of a graph whose \n\tedge-weights are given by the affinity matrix. In spectral clustering, these eigenvectors are then clustered (say, using k-means) to obtain the subspace labels.\n\n\tThe reviewer encourages the authors to read Section 4.3.2 of Chapter 4 in [Book 1]  (this tutorial [Tutorial 1] also contains similar material in Section 4)\n\n[Book 1]: Generalized Principal Component Analysis: https://link.springer.com/book/10.1007/978-0-387-87811-9\n\n[Tutorial 1]: Spectral Clustering. https://people.csail.mit.edu/dsontag/courses/ml14/notes/Luxburg07_tutorial_spectral_clustering.pdf\n\n---\n\n6. The notion of myopic distributions needs to be contrasted and compared to more general notions of distributions concentrated on lower dimensional subsets of the input space. In addition to references [1-4] above, the authors can find recent theory for such \"localized distributions\" in:\n\n[7]: Adversarial Examples Might be Avoidable: The Role of Data Concentration in Adversarial Robustness: https://proceedings.neurips.cc/paper_files/paper/2023/hash/92d21245424f3898b7110f555a00e829-Abstract-Conference.html\n\n---\n\n7. Write a few examples of myopic distributions, and precisely specify their random lens operators and their distributions, and what real world settings would one expect to see such distributions. \n\n---\n\n8. Add a discussion on how distributions which only satisfy myopicity approximately differ from truly myopic distributions.\n\n---\n\n9. Once the related work weaknesses above are resolved, it would be important to have a discussion on why subspace clustering methods / subspace detection methods based on sparse representations were not compared to as baselines in Section 5.3.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "KYyc5Mr53L", "reviewer": "Reviewer_dFsV", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strength\n1. To the best of my knowledge, the proposed idea is novel.\n2. The paper contains some theoretical progress, which is rare and valuable in the machine learning community.\n3. The experiments contain experiments on a large number of real-world datasets and competitors.\n4. The experiments demonstrate remarkable improvements from the baseline.\n\n### Weaknesses\n1. In general, the paper’s contents are good enough. Maybe some efforts for better writing could improve the paper’s quality.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes subspace generation as a new method to improve outlier detection performance on tabular data. The paper’s contribution is twofold: a theoretical framework and a practical design for subspace generation. The authors point out the problems of existing subspace selection methods, which use some heuristic quality metric, resulting in the loss of the data’s property in the extracted subspace.\n\nTo overcome this problem, this paper generalizes an existing theoretical work about the myopic distribution. The existing theory of myopic distribution is limited to some invariance under transformations of a random orthogonal matrix, and the authors generalize this to an invariance under a larger class of operators, which they call a lens operator. Then, the paper shows that a lens operator can be found by solving a stochastic optimization problem.\n\nThe paper proposes **V**-GAN to solve the optimization and find the desired lens operator. The authors present the realization of **V**-GAN for a specific case (axis-parallel subspace generation) and demonstrate its performance with a set of experiments. The experiments show that **V**-GAN can successfully extract the lens operator and outperform its competitors.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. There are many grammatical mistakes and typos. Please read through the paper once again and fix them.\n2. When referring to the Appendix, please include which section is related to the point that refers to it. Also, when referring to tables or figures that are not in the main body of the paper, it is better to clarify the part of the Appendix they are in.\n3. The theory part of this paper is quite hard to understand. Adding some illustrative examples for the definitions could make it easier to understand, improving the overall readability of the paper.\n4. While the tables with ‘+’ and ‘-’ signs could demonstrate enough progress, it would be better to include the actual p-values (in the Appendix) because some p-values slightly bigger than 0.1 could have been marked as no significant difference.\n5. The authors mention that the neural network architecture, the metric space, and the space of operators have to be defined case-by-case. If any guidance or design principles can be provided, this would be appreciated.", "broader_impact_concerns": "I don’t see a particular broader impact concern regarding this paper.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "6UD2jBq6SD", "reviewer": "Reviewer_2sjW", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n+ **S1.**  The paper is well motivated. It carefully discusses the limitations of existing methods and theories, and the proposed theory and methodology are intuitive.\n+ **S2.** The approach of extending existing theory (myopic distribution) to construct a theory suitable for real-world subspace detection and formulating subspace detection as a theoretical probabilistic optimization problem is novel and interesting.\n+ **S3.** The paper proposes V-GAN, a generative approach for efficiently solving the proposed probabilistic optimization problem. It also proposes practical techniques for training kernels.\n+ **S4.** The experiments on toy data and real-world datasets are extensive and directly validate the research questions. In particular, it is fair to provide results not only for datasets where the myopic assumption holds but also for those where it does not.\n### Weaknesses\n- **W1.** V-GAN generates lens operators, but in cases of high dimensions or complex operators, it may not be easy to interpret what specific data structures or relationships between features the generated operators capture. While axis-parallel projections are relatively easy to interpret, interpretability may become an issue when using more general operators.\n- **W2.** Since V-GAN is a GAN-based method, its performance may depend on the settings of many hyperparameters, including network architecture, kernel function selection (whether to use kernel learning, and if so, the kernel type and parameters), optimizer, learning rate, batch size, and number of training epochs. There may be significant costs associated with tuning to find optimal parameter settings. Furthermore, the paper should discuss the instability resulting from min-max optimization in GANs.\n- **W3.** MST has been proposed as a general framework, but further discussion about the limitation of this work is needed on specific application methods and effectiveness for tasks other than outlier detection (e.g., clustering, contrastive learning) and more complex data structures (e.g., time series data, graph data).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a subspace detection method applicable to outlier detection in high-dimensional data. The paper points out that existing subspace detection methods suffer from (i) inaccurate data representation due to dependence on heuristic quality metrics and/or (ii) inefficient computation time due to exponential search spaces. To address these issues, the paper extends existing multiple views theory to propose a generalized myopic subspace theory (MST). MST generalizes existing myopic distribution theory and makes it applicable to a wide range of data types and operators. Specifically, while existing theories require consistency at the probability density function level for returns under random projection operators, MST requires consistency at the probability measure level (Definition 2). This enables the theory to handle data types other than real-valued functions and operators other than projections. Furthermore, under the myopic distribution assumption, the paper formulates subspace detection as a probabilistic optimization problem that minimizes maximum mean discrepancy. Furthermore, the paper proposes V-GAN, a generative network based on MMD-GAN to solve this stochastic optimization problem, i.e., subspace generation. By training V-GAN, we can theoretically find an operator that minimizes MMD. In the experiments, the paper evaluates V-GAN mainly on toy data for sanity checks and real-world tabular datasets. The experiments demonstrate that V-GAN can recover the correct subspace and its weights for Example 1, which existing methods could not recover, and confirm that it achieves the best performance on real-world datasets under the MST assumption. Scalability and applicability to other data types such as images are also verified, supporting the practicality of V-GAN.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Please add the discussion on the interpretability of the proposed method (see W1).\n- Please add the discussion on the stability of V-GAN training (see W2).\n- Please add the discussion on the limitation of the proposed theory and method (see W3).", "broader_impact_concerns": "Nothing to report.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)