Generative Subspace Adversarial Active Learning for Outlier Detection in Multiple Views of High-dimensional Tabular Data

Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-03 16:15:29

Abstract: Outlier detection in high-dimensional tabular data is an important task in data mining, essential for many downstream tasks and applications. Existing unsupervised outlier detection algorithms face one or more problems, including inlier assumption (IA), curse of dimensionality (CD), and multiple views (MV). To address these issues, we introduce Generative Subspace Adversarial Active Learning (GSAAL), a novel approach that uses a Generative Adversarial Network with multiple adversaries. These adversaries learn the marginal class probability functions over different data subspaces, while a single generator in the full space models the entire distribution of the inlier class. GSAAL is specifically designed to address the MV limitation while also handling the IA and CD, being the only method to do so. We provide a mathematical formulation of MV, convergence guarantees for the discriminators, and scalability results for GSAAL. Our extensive experiments demonstrate the effectiveness and scalability of GSAAL, highlighting its superior performance compared to other popular OD methods, especially in MV scenarios.

Main paper: 
 introduction: 
 Outlier detection (OD), a fundamental and widely recognized issue in data mining, involves the identification of anomalous or deviating data points within a dataset. Outliers are typically defined as low-probability occurrences within a population [41, 19] . In the absence of access to the true probability distribution of the data points, OD algorithms rely on constructing a scoring function.Points with higher scores are more likely to be outliers. Existing unsupervised OD algorithms have one or more of the following problems, in high-dimensional tabular data scenarios.• The inlier assumption (IA): OD algorithms often make assumptions about what constitutes an inlier, which can be challenging to verify and validate [30] .• The curse of dimensionality (CD): As the dimensionality of data increases, the challenge of identifying outliers intensifies, decreasing the effectiveness of certain OD algorithms [2] • Multiple Views (MV): Outliers are often only visible in certain "views" of the data and are hidden in the full space of original features [31] We now explain these problems one by one.The inlier assumption poses a challenge to algorithms that assume a standard profile of the inlier data. For example, angle-based algorithms like ABOD [24] assume that inliers have other inliers at all angles. Similarly, neighbor-based algorithms like kNN [34] assume that inliers have other neighboring points nearby. These assumptions influence the scoring as it measures the degree to which a sample deviates from this assumed norm. Consequently, the performance of these algorithms Submitted to 38th Conference on Neural Information Processing Systems (NeurIPS 2024). Do not distribute. may degrade if these assumptions do not hold [30] . This means that a general OD method should not make any inlier assumptions.The curse of dimensionality [2] refers to the decrease in the relative proximity of data points as the number of dimensions increases. Simply put, with high dimensionality, the distance between any pair of points becomes similar, regardless of whether none, one, or both of the points in a pair are outliers. This is particularly problematic for OD algorithms that rely on distances or on identifying neighbors to detect outliers, such as density-(e.g., LOF [3] ), neighbor-(e.g., kNN [34] ), and cluster-based (e.g., SVDD [1, Chapter 2] ) OD algorithms.Multiple Views refers to the phenomenon that certain complex correlations between features are only observable in some feature subspaces [31] . As detailed in [1] , this occurs when the dataset contains additional irrelevant features, making some outliers only detectable in certain subspaces. In scenarios where multiple subspaces contain different interesting structures, this problem is exacerbated. It then becomes increasingly difficult to explain the variability of a data point based solely on its behavior in a single subspace [23] . This problem can occur regardless of the dimensionality of the dataset if the number of points is insufficient to capture a complex correlation structure.The following example illustrates the three problems described above Example 1 (Effect of MV, IA and CD). Consider the random variables x 1 , x 2 and x 3 , where x 1 and x 2 are highly correlated and x 3 is Gaussian noise. Figure 1 plots datasets with 20, 100 and 1000 realizations of (x 1 , x 2 , x 3 ). It also contains the classification boundaries from both a locality-based method (green) and a cluster-based method (red) in the subspace. The cluster-based detector fitted in the full 3D space fails to detect the outlier shown in the figure (red cross). However, the outlier is always detected in the 2D subspace, as we can see. Once we increase the number of samples over n = 1000, the cluster-based method detects the outlier in the full space (MV). On the contrary, the locality-based method could not detect the outlier in any tested scenario (MV + IA). If we increase the dimensionality by adding more features consisting of noise, no method can detect the outlier in the full space (MV + IA + CD).We are interested in tackling outlier detection whenever a population exhibits MV, like [31, 23, 25] and as showcased in [1] . Particularly, the goal of this paper is to propose the first outlier detection method that explicitly addresses IA, CD, and MV simultaneously.As we will explain in the next section, we build on Generative Adversarial Active Learning (GAAL) [44] , a widely used approach for outlier detection [30, 17, 39] . It involves training a Generative Adversarial Network (GAN) to mimic the distribution of outlier data, and it enhances the discriminator's performance through active learning [38] , leveraging the GAN's data generation capability. GAAL methods avoid IA [30] and use the multi-layered structure of the GAN to overcome the curse of dimensionality [33] . However, they often miss important subspaces, leading to MV.Challenges. Training multiple GAN-based models in individual subspaces is not trivial. (1) The joint training of generators and discriminators in GANs requires careful monitoring to determine the optimal stopping point, a task that becomes daunting for large ensembles. (2) The generation of difficult-to-detect points in a subspace remains hard [40] . (3) While several authors have proposed multi-adversarial architectures for GANs [11, 5] , none of them address adversaries tailored to subspaces composed of feature subsets. Furthermore, these methods may not be suitable for GAAL since they do not have convergence guarantees for detectors, as we will explain.Contributions. (1) We propose GSAAL (Generative Subspace Adversarial Active Learning), a novel GAAL method that uses multiple adversaries to learn the marginal inlier probability functions in different data subspaces. Each adversary focuses on a single subspace. Simultaneously, we train a single generator in the full space to approximate the entire distribution of the inlier class. All Paper outline: Section 2 reviews related work, Section 3 contains the theoretical results for our method, Section 4 features our experimental results, and Section 5 concludes and addresses limitations. 
related work: 
 This section is a brief overview of popular unsupervised outlier detection methods for tabular data related to our approach. We categorize them based on their ability to address the specific limitations outlined above. For example, local methods that assume isolated outliers fail when several outlying samples fall together. In addition, many classical methods, which rely on measuring distances, are susceptible to the curse of dimensionality. Both limitations impair the effectiveness of these methods [30] .Subspace Methods Subspace-based methods [25] operate in lower-dimensional subspaces formed by subsets of features. They effectively counteract the curse of dimensionality by focusing on identifying so-called "subspace outliers" [22] . These outliers, which are prevalent in high-dimensional datasets with many correlated features, are often elusive to conventional non-subspace methods [29, 31] . However, existing subspace methods inherently operate on specific assumptions on the nature of anomalies in each subspace they explore, and thus face the inlier assumption limitation.Generative Methods A common strategy to mitigate the IA and CD limitations is to reframe the task as a classification task using self-supervision. A prevalent self-supervised technique, particularly for tabular data, is the generation of artificial outliers [13, 30] . This method involves distinguishing between actual training data and artificially generated data drawn from a predetermined "reference distribution". [21] showed that by approximating the class probability of being a real sample, one approximates the probability function of being an inlier. One then uses this approximation as a scoring function [30] . However, it is not easy to find the right reference distribution, and a poor choice can affect OD by much [21] .A first approach to this challenge proposed the use of naïve reference distributions by uniformly generating data in the space. This approach showed promising results in low-dimensional spaces but failed in high dimensions due to the curse of dimensionality [21] . Other approaches, such as assuming parametric distributions for inlier data [1, Chapter 2] or directly generating in susbpaces [12] , can avoid CD when the parametric assumptions are met. Methods that generate in the subspaces can model the subspace behavior, additionally tackling the MV limitation. However, these last two approaches do not address the IA limitation, as they make specific assumptions about the behavior of the inlier data.Generative Adversarial Active Learning According to [21] , the closer the reference distribution is to the inlier distribution, the better the final approximation to the inlier probability function will be. Hence, recent developments in generative methods have focused on learning the reference distribution in conjunction with the classifier. A key approach is the use of Generative Adversarial Networks (GANs), where the generator converges to the inlier distribution [15] . The most common approaches for this are GAAL-based methods [30, 17, 39] . These methods differentiate themselves from other GANs for OD by training the detectors using active learning after normal convergence of the GAN [36, 10] . The architecture of GAAL inherently addresses the curse of dimensionality, asGANs can incorporate layers designed to manage high-dimensional data [33] . In practice, GAALbased methods outperformed all their competitors in their original work. However, they overlook the behavior of the data in subspaces and therefore may be susceptible to MV.Our method, GSAAL, incorporates several subspace-focused detectors into GAAL. These detectors approximate the marginal inlier probability functions of their subspaces. Thus, GSAAL effectively addresses MV while inheriting GAAL's ability to overcome IA and CD limitations.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes]Justification: Section 5.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.• The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.• The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.• The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.• The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.• If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.• While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
our method: gsaal: 
 We first formalize the notion of data exhibiting multiple views. We then use it to design our outlier detection method, GSAAL, and give convergence guarantees. Finally, we derive the runtime complexity of GSAAL. All the proofs and extra derivations can be found in the technical appendix.Several authors [1, 31, 23, 25, 29] have observed that at times the variability of the data can only be explained from its behavior in some subspaces. Researchers variably call this problem "the subspace problem" [1, 25] or "multiple views of the data" [22, 31] . Previous research has largely focused on practical scenarios, leaving aside the need for a formal definition. In response, we propose a unifying definition of "multiple views" that provides a foundation for developing methods to address this challenge effectively.The problem "multiple views" of data (MV) arises from two different effects. First, it involves the ability to understand the behavior of a random vector x by examining lower-dimensional subsets of its components (x 1 , . . . , x d ). Second, it stems from the challenge of insufficient data to obtain an effective scoring function in the full space of x. As Example 1 shows, combining these two effects obscures the behavior of the data in the full space. Hence, methods not considering subspaces when building their scoring function may have issues detecting outliers under MV. The next definition formalizes the first effect.Definition 1 (myopic distribution). Consider a random vector x : Ω -→ R d and Diag d×d ({0, 1}), the set of diagonal binary matrices without the identity. If there exists a random matrix u : Ω -→ Diag d×d ({0, 1}), such that p x (x) = p ux (ux) for almost all x, (1) we say that the distribution of x is myopic to the views of u. Here, x and ux are realizations of x and ux, and p x and p ux are the pdfs of x and ux.It is clear that, under MV, using p ux to build a scoring function instead of p x mitigates the effects. This comes as the subspaces selected by u are smaller in dimensionality. Hence it should take fewer samples to approximate the pdf of ux. The difficulty is that it is not yet clear how to approximate p ux . The following proposition elaborates on a way to do so. It states that by averaging a collection of marginal distributions of x in the subspaces given by realizations of u, one can approximate the distribution of p ux . Proposition 1. Let x and u be as before with p x myopic to the views of u. Consider a set of independent realizations of u:{u i } k i=1 . Then 1 k i p uix (u i x) is an unbiased statistic for p ux (ux).MV appears when there is a lack of data, and its distribution is myopic. To improve OD under MV, one can exploit the distribution myopicity to model x in the subspaces, where less data is sufficient.Proposition 1 gives us a way to do so, by approximating p ux . In this way, under myopicity, this also approximates p x , avoiding MV. Our method, GSAAL, exploits these derivations, as we explain next.GAAL methods tackle IA by being agnostic to outlier definition and mitigate CD through the use of multilayer neural networks [30, 28, 33] . GAAL methods have two steps:1. Training of the GAN. Train the GAN consisting of one generator G and one detector D using the usual min-max optimization problem as in [15] .Training of the detector through active learning. After convergence, G is fixed, and D continues to train. This last step is an active learning procedure with [44] . Following [21] ,D(x) now approximates the pdf of the training data p x .After Step 2, the detector converges to p x . However, our goal is to approximate p x by exploiting a supposed myopicity of the distribution. We extend GAAL methods to also address MV in what follows. The following theorem adapts the objective function of the GAN to the subspace case and gives guarantees that the detectors converge to the marginal pdfs used in Proposition 1:Theorem 1. Consider x and u as in the previous definition, with x a realization of x and {u i } i a set of realizations of u. Consider a generator G : z ∈ Z -→ G(z) ∈ R d and {D i }, i = 1, . . . , k, a set of detectors such as D i :u i x ∈ S i ⊂ R d -→ D i (u i x) ∈ [0, 1].Z is an arbitrary noise space where G randomly samples from. Consider the following optimization problemEQUATIONwhere each addend V (G, D i ) is the binary cross entropy in each subspace. Under these conditions, the following holds:i) Each detector in optimum is D * i (u i x) = 1 2 , ∀x. Thus, in optimum V (G, D i ) = -log(4), ∀i. ii) Each individual D i converges to D * i (u i x) = p uix (u i x) after trained in Step 2 of a GAAL method. iii) D * (x) = 1 k k i=1 D * i (u i x) approximates p ux (ux). If p x is myopic, D * (x) also approxi- mates p x (x).Using Theorem 1 we can extend the GAAL methods to the subspace case: We call this generalization of GAAL Generative Subspace Adversarial Active Learning (GSAAL).The appendix contains the pseudo-code for GSAAL.In this section, we focus on studying the theoretical complexity of GSAAL. We study both its usability for training and, more importantly, for inference.Theorem 2. Consider our GSAAL method with generator G and detectors {D i } k i=1 , each with four fully connected hidden layers, √ n nodes in the detectors and d in the generator. Let D be the training data for GSAAL, with n data points and d features. Then the following holds:i) Time complexity of training is O(E D • n • (k • n + d 2 )). E D is an unknown complexity variable depicting the unique epochs to convergence for the network in dataset D.ii) Time complexity of single sample inference is in O(k • n), with k the number of detectors used.The linear inference times make GSAAL particularly appealing in situations where the model can be trained once for each dataset, like one-class classification. We build on this particular strength in the following section.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [Yes]• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)• The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean.• It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.• For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).• If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments: 
 This section presents experiments with GSAAL. We will outline the experimental setting, and examine the handling of "multiple views" in GSAAL and other OD methods. We then evaluate GSAAL's performance against various OD methods and investigate its scalability. The appendix includes a study on the sensitivity to the number of detectors, IA experiments, an ablaition study and extra competitors evaluated in the real world datasets. System specifications are included in the appendix.This section has three parts: First, we describe the synthetic and real data for the outlier detection experiments. Then, we describe the configuration of GSAAL. Finally, we present our competitors.Synthetic. We constructed synthetic datasets, each containing two correlated features, x 1 and x 2 , along with 58 independent features x j , j = 3, . . . , 60 consisting of Gaussian noise. This approach simulates datasets that exhibit the MV property by adding irrelevant features into a pair of highly correlated variables. We detail the methodology and all correlation patterns in the technical appendix.Real. We selected 22 real-world tabular datasets for our experiments from [19] . The selection criteria included datasets with less than 10,000 data points, more than 10 outliers, and more than 15 features, focusing on high-dimensional data while keeping the runtime (of competing OD methods) tractable. Table 2a contains the summary of the datasets. For datasets with multiple versions, we chose the first in alphanumeric order. Details about each dataset are available in the original source [19] . Training. Like other GAAL methods [30, 44] , we train the generator G together with all the detectors D i until the loss of G stabilizes. Then we train each detector D i until convergence with G fixed. To automate this process, we introduce an early stopping criterion: Training stops when a detector's loss approaches the theoretical optimum (-log(4)), see statement (ii) of Theorem 1. For consistency across experiments, training parameters remain fixed unless otherwise noted. Specifically, the learning rates of the detectors and the generator are 0.01 and 0.001, respectively. We use minibatch gradient descent [14] optimization, with a batch size of 500.We selected popular and accessible methods from each category, as summarized in Table 2b , guided by related work. We excluded generative methods with uniform distributions because they prove ineffective for large datasets [21] . We could not include a generative method with subspace behavior due to operational issues with the most relevant method in this class, [12] , caused by its outdated repository. We used the recommended parameters for all methods, as usual in OD [19] .We used the pyod [43] library to access all competitors except MO-GAAL. We used MO-GAAL from its original source and implemented our method GSAAL in keras [6] .To demonstrate the effectiveness of GSAAL under MV, we use synthetic datasets. Visualizing the outlier scoring function in a 60-dimensional space is challenging, so we project it into the x 1 -x 2 subspace. A method adept at handling MV should have a boundary that accurately reflects the x 1 andx 2 dependency structure. We first generate a synthetic dataset D synth as described in section 4.1.1and train the OD model. Using this model, we compute the scores for the points (x 1 , x 2 , 0, . . . , 0)and visualize the level curves on the x 1 -x 2 plane.Figure 2 shows results for selected datasets and competitors, which are detailed in the Appendix. Itshows the level curves and decision boundaries (dashed lines) of the methods. Notably, our model effectively detects correlations in the right subspace. To quantify this, we generated outliers in the subspace of interest and extra inliers. We tested the one-class classification performance of each method in 10 different MV datasets. On average, GSAAL managed to obtain 0.70 AUC, while the second-best performer (IForest) did not surpass a random classifier -0.49 AUC. All results and further details can be found in section B.2 in the appendix.This section evaluates GSAAL on a one-class classification task [37] . First, we study the effectiveness of GSAAL on real data. Then, we investigate the scalability of GSAAL in practical scenarios.We perform the outlier detection experiments on real datasets. Specifically, we take on the task of one-class classification, where the goal is to detect outliers by training only on a collection of inliers [19] . To evaluate the performance of OD methods, we use AUC as it is robust to test data imbalance, a common issue in OD tasks . The procedure is as follows: Table 3 : Results of the Conover-Iman test for pairwise comparisons of the rankings. To save space, we moved the detailed AUC results to the appendix; showing that GSAAL obtained the lowest median rank -see Figure 10 in the appendix. Although other subspace methods tend to perform better with irrelevant attributes [29, 25] , they did not outperform classical OD methods on average in our experiments. Notably, ABOD, the second-best method in our experiments, performed poorly in the MV tests (Section 4.2).Method ABOD GSAAL GMM IForest KNN LOF MO GAAL OCSVM SOD ABOD = ++ ++ ++ ++ ++ GSAAL = ++ ++ + ++ ++ ++ GMM -- -- = ++ -- -- ++ ++ IForest -- -- -- = -- ++ ++ KNN ++ ++ = ++ ++ LOF - ++ = ++ + ++ MO GAAL -- -- -- -- -- = ++ OCSVM -- -- -- - = ++ SOD -- -- -- -- -- -- -- -- = 1.For statistical comparisons, we use the Conover-Iman post hoc test for pairwise comparisons between multiple populations [7] . It is superior to the Nemenyi test due to its improved type I error boundings [8] . Conover-Iman test requires a preliminary positive result from a multiple population comparison test, for which we employ the Kruskal-Wallis test [26] .Table 3 shows the test results. In each cell, '+' indicates that the method in the row has a significantly lower median rank than the method in the column, while '-' indicates a significantly higher median rank. One symbol indicates p-values ≤ 0.15 and two symbols indicate p-values ≤ 0.05. A blank indicates no significant difference. The table shows that GSAAL is superior to most of its competitors.Our method does not significantly outperform the classical methods ABOD and kNN. However, these methods struggle to detect structures in subspaces, showing their inadequacy in dealing with the MV limitation, see Section 4.2.Overall, the results support GSAAL's superiority in outlier detection tasks involving multiple views.Additionally, they establish our method as the leading GAAL option for One-class classificationIn section 3.3, we derived that the inference time of GSAAL scales linearly with the number of training points if the number of detectors k is fixed, while it does not depend on the number of features d. This is in contrast to other methods, in particular LOF, KNN, and ABOD, which have quadratic runtimes in d [3, 24] . We now validate this experimentally. The procedure is as follows: In section 4 we randomly selected subspaces for training the detectors in GSAAL, i.e. we took a uniform distribution of u. This was already sufficient to demonstrate the highly competitive performance of our method. In practice, this assumption seemed to perform well for our experiments.However, GSAAL can work with any subspace search strategy to obtain the distribution of u, for example, the methods exploiting multiple views [23, 22] . We have not included them in this paper due to the lack of an official implementation. In the future, we plan to benchmark various subspace search methods in GSAAL.Next, GSAAL is limited to tabular data, since the "multiple views" problem has only been observedfor this data type. The mathematical formulation of MV in section 3 does not exclude unstructured data. The difficulty lies in identifying good search strategies for u for non-tabular data, which remains an open question [18] . However, depending on the type of unstructured data, extending GSAAL to work with it is not immediate. Therefore, building a method that exploits the theoretical derivations of GSAAL for structured data is future work. 
conclusions: 
 Unsupervised outlier detection (OD) methods rely on a scoring function to distinguish inliers from outliers, since the true probability function that generated the dataset is usually unavailable in practice.However, they face one or more of the following problems -Inlier Assumption (IA), Curse of Dimensionality (CD), or Multiple Views (MV). In this article, we have proposed the first mathematical formulation of MV, which allows for a better understanding of how to solve this occurrence. Using this formulation, we developed GSAAL, which is the first OD approach that solves MV, CD, and IA.In short, GSAAL is a generative adversarial network with a generator and multiple detectors fitted in the subspaces to find outliers not visible in the full space. In our experiments on 27 different datasets, we demonstrated the usefulness of GSAAL, in particular, its ability to deal with MV and its superior performance on OD tasks with real datasets. In addition, we have shown that GSAAL can scale up to deal with high-dimensional data, which is not the case for our most competent competitors. These results confirm GSAAL's ability to deal with data exhibiting MV and its usability in any practical scenario involving large datasets. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes]Justification: See the beginning of section B Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.• The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.• The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: In sections, 1 & 5 we go through the importance of outlier detection in many fields, particularly for our use-case. Our positive impact on society consists of the improvement of the tasks where outlier detection is needed.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.• The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.• If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: We do not identify any risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.• Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.• We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: We include URLs and citations for all dataset selections, packages, and methods.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL.• The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.• For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.• If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [Yes] Justification: We include the documentation of our implementation in the repository.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.• The paper should discuss whether and how consent was obtained from people whose asset is used.• At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA]Justification: The paper does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.• Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.• According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.• Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.• We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.• For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: In this appendix, we will include all the proofs of the included theorems and propositions. Additionally, we also extend all non-experimental sections with relevant information for the experimental appendix.Before starting to prove our main results, it is important to add a remark about our notation in this article. Whenever we denote ux, we mean the operation resulting in the following vector: u(ω)x(ω).Thus, ux is a random vector following its distribution p ux . However, it is important to remark that ux, and therefore, also u i x, does not state the usual matrix-vector multiplication. What we mean by ux is the operation U × M x, where U stands for the range-complete version of u and × M the usual matrix multiplication. This means that whenever we write ux we are considering the projection of x into the subspace of the features selected in u. This means that u i x is the random vector composed of the features selected by u i , and therefore, p uix (u i x) denotes subsequent marginal pdf of x. We do not state this in the main text as it functionally does not change anything of our derivations, and simply works as a notation. The only important remarks stemming from this fact are the following:1. p x (u i x) = p x (π ui (x)), where π ui denotes the projection of a point x into the subspace of u i . Therefore, we can write p x (u i x) = p uix (u i x).2. The operator as stated before is not distributive. This is trivial, as given u a random matrix as in definition 1, (1 d -u)x is defined properly, as 1 d -u ∈ Diag({0, 1}). However, xux denotes the vector subtraction between two vectors with different dimensionality.While not important to understand the following proofs and the derivations from the main text, understanding this is crucial for anyone seeking to work with these definitions.We will reformulate all of the statements for completion before introducing each proof.Proposition 2. Let x and u be as before with p x myopic to the views of u. Consider a set of independent realizations of u: {u i } k i=1 , a realization of x, x, and a realization of ux, ux. Then1 k i p uix (u i x) is a statistic for p ux (ux).Proof. Consider x and u as in the statement. Recall the law of total probabilities:p ux (ux) = E u p ux|u=u ′ (ux|u ′ ). By taking the definition of u and the myopicity, it is trivial that:p ux|u=u ′ (ux|u ′ ) = p u ′ x (u ′ x) for u ′ such that p u (u ′ ) ̸ = 0.Then, by definition of marginal probability and expectation, we have that:p ux (ux) = N i=1 p u (u i )p uix (u i x),as u is discrete with finite set of occurrences of size N . Thus, we can approximate N i=1 p u (u i )p uix (u i x)) by 1 k i p uix with u i independent samples of u.D i : u i x ∈ S i ⊂ R d -→ D i (u i x) ∈ [0, 1].Z is an arbitrary noise space where G randomly samples from. Consider the following objective functionEQUATIONUnder these conditions, the following holds:i) Each detector's loss in optimum is V (G, D * i ) = 1 2 . ii) Each individual D i converges to D * i (u i x) = p uix (u i x) after trained in Step 2 of a GAAL method. iii) D * (x) = 1 k k i=1 D * i (u i x) approximates p ux (ux). If p x is myopic, D * (x) also approxi- mates p x (x).Proof. This proof will follow mainly the results in [15] , adapted for our case. We will first derivative two general results that we are going to use to immediately prove (i), (ii) and (iii). First, consider the objective functioni V (G, D i ) = i E uix∼pu i x log(D i (u i x))+ E z∼pz (1 -log(D i (u i G(z)))),where z is the random vector used by G to sample from the noise space Z. We will write E x , E z and E uix instead of E x∼px , E z∼pz and E uix∼pu i x as an abuse of notation.The problem is, then, to optimize:EQUATIONFixing G and maximizing for all D i , each detector individually maximizes V (G, D i ). Let us try to obtain the optimal of each D i with a fixed G. First, we write:V (G, D i ) = uix p uix (u i x) log D i (u i x)du i x+ z p z (z) log(1 -D i (u i G(z)))dz.As G uses z to sample from its sample distribution p G (x), we can rewrite the second addent, like in [15] , as:V (G, D i ) = uix p uix (u i x) log D i (u i x)du i x+ uix p G (u i x) log(1 -D i (u i x))du i x.Aggregating both integrals, we have a function of the type f (t) = a log(t) + b log(1 -t), with a, b ∈ R -{0}. We know that f (t) obtains its optimum in t = a a+b . As f (t) ∈ R + , V (G, D i ) obtains its optimum for a given G in:EQUATIONLet us now consider the following functionEQUATIONThis is known in Game Theory as the cost function of player "G" in the null-sum game defined by the min max optimization problem. [15] refers to it as the virtual training criterion of the GAN. The adversarial game defined by ( 4) reaches an equilibrium (and thus, the min max problem an optimum) whenever C(G) is minimized. We will study the value of G in such equilibrium and use it, together with (5), to prove the statements.Rewriting C(G) it is clear that:C(G) = i KL p uix(uix) ∥ p uix (u i x) + p G (u i x) 2 + KL p G (u i x)∥ p uix (u i x) + p G (u i x) 2 .This expression corresponds to that of a sum of multiple binary cross entropies between a population coming from p uix and from p G projected by u i . Therefore, as we know, we can rewrite:C(G) = i 2JSD(p uix(uix) ∥p G (u i x)),with JSD the Jensen-Shannon divergence. Since JSD(s∥r)∈ [0, log(2)), it is clear that C(G)obtains its minimum only wheneverEQUATIONand for all i ∈ {1, . . . , k}.Knowing G and D i in the optimum for all i, we can prove the statements above:(i) As p G (u i x) = p uix (u i x)for almost all x, in the optimum of (4), it is immediate that:D i (u i x) = 1 2 ,i.e., the detectors cannot differentiate between the real training data and the synthetic data of the generator. If one employs the numerically stable version of each V (G, D i ) (equivalent to the numerically stable version of the binary cross entropy [6] ), it is trivial to see thatV stable (G, D i ) = log(2).(ii) After optimizing (4), training each D i individually with G fixed, is the equivalent of building a two-class classifier distinguishing between the artificial class generated by p G (u i x) = p uix (u i x) and the real data coming from p uix (u i x). By [21] , the resulting two-class classifier would be such as:D i (u i x) = p uix (u i x).(iii) By proposition 2 and statement (ii),1 k i D * i (u i x) is an estimator for p ux (ux). By myopicity, it is also of p x (x). ii) The single sample inference time complexity is bounded with O(k • n), with k the number of detectors used.Proof. An evaluation of a neural network is composed of two steps, the backpropagation, and the fowardpass steps. While training the network requires both, inference requires only a fowardpass.Therefore, we will first prove (ii) and will build upon it to prove (i).(ii). GSAAL consists of a generator and k detectors. Single point inference consists of a single fowardpass of all the detectors. We will first prove the general complexity of a fowardpass of a general fully connected 4 layer network and will use it to derive all the other complexities. Let us consider three weight matrices W ji , W hj and W lh each between two layers, with j, i, h and l being the number of nodes in each. Therefore, W ji denotes a matrix with j rows and i columns, and so on. Now, let us consider x i1 the datapoint after passing the input layer. Lastly, without any loss of generality, consider f to be the activation function for all layers. This way, the forward pass of a single detector can be written as:c l1 = f (W lh f (W hj f (W ji x i1 ))) .We will study the complexity in the first layer and use it to derive the complexity of the others.A j1 = W ji x i1 is a simple matrix-vector multiplication that we know to be O(j • i) atmost. Then, asf is an activation function, f (A j1 ) is equivalent to writing f j1 ⊙ A j1 , with ⊙ being the element-wise multiplication. Thus, f (W ji x i1 ) is:O(j • i + j) = O(j • (i + 1)) = O(j • i).Doing this for all layers, we obtain:EQUATIONAs all layers have √ n nodes,O(3n) = O(n).As we have k detectors, the complexity for a fowardpass of all detectors, and thus, for a single sample inference of GSAAL is: EQUATIONO(E D • k • n 2 + k • n 2 ) = O(E D • k • n 2 ).As we also need to consider the generator, we will use equation 8 to derive both steps on the generator.As the generator is also a fully connected 4-layer network, with all layers having d nodes, the complexity for a single fowardpass is:O(d 2 ).As during training one generates n samples during each fowardpass:O(n • d 2 ).Now, on each backpropagation pass the network calculates the backpropagation error for each generated sample, thus,O(n • d 2 )is also the time complexity for the backpropagation step of the generator. Considering all E D epochs and both backpropagation and fowardpass steps of the generator and all the detectors, the time complexity of GSAAL's training is:O(E D • k • n 2 + E D • n • d 2 ) = O(E D • n • (k • n + d 2 ))Figure 4 : Difference in statistical distance between two populations.Deep Outlier Detection for other data types. Outlier detection is also very popular in different data types, especially in unstructured data [42, 16, 36, 35, 32] . Due to the complexity of the data they are used for, deep methods are the main approach employed for this task. The main difference with the other deep methods introduced for tabular data, is that the deep architecture in the later targets mainly CD. For unstructured data types, like images or natural language, is the complexity of the data that drives the architecture. For example, to treat image data, multiple linear layers do not suffice, complex layers like convolutional or residual layers are employed for this [27] .Although popular, most deep methods have limited to no use at all in tabula data in their original articles. However, some have appeared in the literature of tabular data as competitors [36, 35] . We identified the most common for our task in related articles and benchmarks, and included them as an extension of our main experiments in sections B.2 and B.3.In this section we extend the derivations in section 3.1 by providing an example of a myopic distribution:Example 2 (Myopic distribution). Consider a x like in example 1. Here, it is clear that x 1 , x 2 ⊥x 3 .Consider, then, u such that:u : {1} -→ {diag(1, 1, 0)}. To test whether p x is myopic, we employed a simple test utilizing a statistical distance (M M D with the identity kernel) between p x and p ux . This way, if M M D(p x ∥p ux ) = 0, it would be clear that the equality holds. As a control measure, we also calculated the same distance for a different populationx ′ , where x 3 = x 2 1 . We have plotted the results in image 4, where Population 1 refers to x and Population 2 to x ′ . As we can see, we do obtain a positive result in the test of myopicity for x and a negative one for x ′ .We now extend the results from section 3.2 by providing the pseudocode for the training of our method. It is important to consider that, while theorem 3 formulates the optimization problem in terms of the neural networks G and {D i } i , in practice this will not be the case. Instead, we will consider the optimization in terms of their weights, Θ G and Θ Di . Therefore, in practice, the convergence into an equilibrium will be limited by the capacity of the networks themselves [14] .We considered the optimization to follow minibatch-stochastic gradient descent [14] . To consider any other minibatch-gradient method it will suffice to perform the necessary transformations to the gradients.The pseudocode is located in Algorithm 1. As it is the training for the method, it takes both the parameters for the method and the training. In this case, epochs refers to the total number of epochs we will train in total, while stop_epoch marks the epoch where we start step 2 of the GAAL training. Lines 1-3 initialize both the detectors in their subspaces and the generator with for batch ∈ {1, ..., batches} do 6:noise ← Random noise z (1) , ..., z (m) from Z 7:data ← Draw current batch x (1) , ..., x (m) 8:for j ∈ {1...k} do 9:Update D j by ascending the stochastic gradient: Update G by descending the stochastic gradient: Here we proceed with training each detector and the generator using their gradients. Lines 8-10 update each detector by ascending its stochastic gradient, while line 11 updates the generator by descending its stochastic gradient. After the normal GAN training, we start the active learning loop [30] once epoch ≥ stop_epoch. The only difference with the regular GAN training is that G remains fixed, i.e., we do not descend using its gradient. This allows us to additionally train the detectors and, in case of equilibrium of step 1, converge to the desired marginal distributions as derived in theorem 3.∇ Θ D j 1 m m i=1 log(D j (u j x (i) )) + log(1 -D j (u j G(z (i) )))∇ Θ G 1 k k j=1 1 m m i=1 log(1 - D j (G(z (i) )))In this section, we will include a supplementary experiment testing the IA condition for completion, the sensibility experiments, and an ablation study. Additionally, we extended both main experimental studies featured in the main text. All of the code for the extra experiments, as well as for all experiments in the main text, can be found in our remote repository 3 . Our experiments used a RTX 3090 GPU and an AMD EPYC 7443p CPU running Python in Ubuntu 22.04.3 LTS. Deep neural network methods were trained on the GPU and inferred on the CPU; shallow methods used only the CPU.GAAL methodologies are capable of dealing with the inlier assumption by learning the correct inlier distribution p x without any assumption [30] . While this should also extend to our methodology, we will study experimentally whether this condition holds in practice. To do so, as one cannot identify beforehand whether a method is going to fail due to IA, we will generate synthetic datasets. This will allow us to generate outliers that we know to follow from a specific IA, ensuring that failure comes from the anomalies themselves. We will include all of the code in the code repository. To generate the synthetic datasets we follow:1. Generate D, a population of 2000 inliers following some distribution F in R 20 .2. Select an outlier detection method M with some assumption about the normality of the data and fit it using D. We will call such M as the reference model for the generation.3. Generate 400 outliers by sampling on R 20 uniformly and keeping only those points o such that M (o) = 1 (i.e., they are detected as outliers). We will write O D to refer to such a collection of points. 6 gathers all the AUCs of a method in 3 boxplots, one for each outlier type in each training set. Additionally, we grouped all based on the IA and assigned a similar color for all of them. We have done this for the classical OD methods LOF, ABOD, and kNN, besides our method GSAAL. We cropped the image below 0.45 in the y axis as we are not interested in results below a random classifier. As we can see, classical methods seem to correctly detect outliers for an outlier type that verifies its IA. However, whenever we introduce outliers behaving outside of their IA, the performance hit is significant. Notoriously, it appears that none of them had trouble detecting the Local and Angle outlier type. regardless of their IA. This can be easily explained by those outliers types being similar, as we can see in figure 5 . On the other hand, GSAAL manages to have a significant detection rate regardless of the outlier type.In this section, we will include a brief description of the generation process for the datasets used in section 4.2. We will also perform the same experiment as in section 4.2 for all methods showcased in the main text and additional datasets. The datasets were generated by the following formulas:• Banana. Given θ ∈ [0, π] we have x = sin(θ) + U (0, 0.1) and y = sin(θ) 3 + U (0, 0.1).• Spiral. Given θ ∈ [0, 4π] and r ∈ (0, 1), we have x = r cos(θ) + U (0, 0.1) and y = r sin(θ).• Star. Given θ ∈ [0, 2π] and r ∈ {r ∈ R|r = sin(5θ); r ≥ 0, 1, 0.4} , we have x = r cos(θ)+ U (0, 0.1) and y = r sin(θ) + U (0, 0.1).• Circle. Given θ ∈ [0, 2π], we have x = cos(θ) + U (0, 0.1) and y = sin(θ) + U (0, 0.1).• L. Given x 1 = N (0, 0.1), x 2 = U (0, 5), y 1 = U (-5, 0), and y 2 = N (0, 0.1); we havex = concat(x 1 , x 2 ) and y = concat(y 1 , y 2 ).We considered N (0, 0.1) to denote a random normal realization with µ = 0 and σ 2 = 0.1, andU (a, b) to denote a uniform realization in the [a, b] interval.Figure 7 contains all images from the MV experiment. We employed the default parameters for all methods in this experiments. We did that as those were the employed parameters in our real world experiments. Additonally, the choice of parameter did not impact the outcome of the experiment much. Our remote repository includes extra images for every competitor with multiple parameters for comparison. We do not have any new insight beyond the ones exposed in the main article. Note that we have included all methods but SOD. The reason was that SOD failed to execute for datasets Star, Spiral, and Circle.Additionally, we added competitors from outside of our related work that will later be used in section B.3. In particular, we employed LUNAR, DIF and DeepSVDD with default parameters. We included extra images in our remote repository with multiple parameters for the deep competitors as well. The method AnoGAN was not included due to it failing in datasets Star, Spiral and Circle. Their results can be seen in Figure 8 . As it also happened our main competitors, some of the extra competitors were capable of detecting the data structure in very sparse occasions. However they remained incapable to properly describe a boundary consistently. The only method that was sensible enough in all datasets was GSAAL.In order to quantify this, we tested the ability of all methods to perform one-class classification in each dataset. As outliers, we used white noise in the x 1 -x 2 subspace. Additionally, we created two extra datasets greatly different from the rest, X and wave:• X. Given x 1 = x 2 = U (-1, 1) and y 1 = x 1 + U (0, 0.1), y 2 = x 2 + U (0, 0.1); we havex = concat(x 1 , x 2 ) and y = concat(y 1 , y 2 ).. • Wave. Given θ ∈ [0, 4π],we have x = θ and y = sin(x) + U (0, 0.1).We will also use them as outleirs, for a total of 15 different datasets. We also generated extra inliers in each test set. We gathered the AUC results in Figure 9 . As we can see, all other methods struggel to come ahead of the random classifier, marked with a dashed line. The only method well above that is GSAAL.As we noted in Section 4, we obtained our benchmark datasets from [19] , a benchmark study for One-class classification methods in tabular data. Some of the datasets featured in the study, and also in our experiments, were obtained from embedding image or text data using a pre-trained NN (ResNet [20] and BERT [9] , respectively). We shunt the interested reader into [19] for additional information. Additionally, we found discrepancies between the versions of the datasets in the study of [4] and [19] . We utilized the version of those datasets featured in [4] for our experiments due to popularity. This affected the datasets Arrhythmia, Annthyroid, Cardiotocography, InternetAds, Ionosphere, SpamBase, Waveform, WPBC and Hepatitis. Figure 10 summarizes the ranks from the one-class experiments in section 4.3. Table 5 summarizes the AUC results from our experiments. As mentioned in section A.3, we also included extra methods outside of our related work. Particularly, we added deep versions tailored to image data of previously included methods -DeepSVDD [35] and Deep Isolation Forest [42] (DIF)-and others that extend some types of outlier detectors into image and text data -LUNAR [16] , as an extension of Locality-based classical methods, and AnoGAN [36] , as an extension of Generative methods. For their parameters, we employed the recommended ones for LUNAR and DIF, and trained the models the same way that the authors did in their articles. As for DeepSVDD and AnoGAN, as they do not have any recommended way of training nor hyperparameters, we performed a grid search for their training parameters and kept the best result. We used all of their official implementations 4 . All deep methods (including MO-GAAL and GSAAL) were trained multiple times with the same train set and their results were averaged to account for initialization.Additionally, we gathered all extra deep methods and performed the same statistical analysis as in section 4.3. We also included MO GAAL besides GSAAL for completion. SO GAAL, the single generator version of MO GAAL was not included, even if popular in the related literature. The reason is that authors in [30] showed that MO GAAL constantly outperforms SO GAAL in the outlier detection task. Results are included in table 6, gathered after a positive Kruskal-Wallis test. As we can see, GSAAL outperform almost all competitors except LUNAR (the most recent method). However, LUNAR is incapable to detect change in the subspaces as GSAAL does, see section B.2. Therefore, regardless of considering the tabular related work, or the more generalist deep methods, GSAAL still can outperform most competitors in the field. Additionally, for those that GSAAL performs similar to, we showed that we are more sensible to changes in subspaces. This fact makes GSAAL the preferred option for One-class classification under MV.We now explore the effect of the number of detectors in GSAAL, k, by repeating the previous experiments with varying k. We also found that the results in Table 3 remain almost the same if one sets k = 30. So we recommend fixing k = 30, which makes GSAAL very suitable for high-dimensional data.Lastly, we also performed an ablation study for GSAAL. We identify two critical components in our method, the subspace nature of our detectors, and the multiple detectors used. Table 7 contains a summary of the included features in each considered configuration. We will compare the performance of all the different configurations of GSAAL. i GSAAL ✗✗ ✗ ✗ GSAAL ✓✗ ✓ ✗ GSAAL ✗✓ ✗ ✓ GSAAL ✓ ✓We will employ, once again, the Conover-Iman test to compare the performance of all configuration in a statistically sound way. Table 8 contains the results of the ablation experiment. As expected, our fully configured method significantly outperformed all of the others. This further confirms that the performance increase over our competitors comes directly from tackling the MV problem. Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper.• The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.• The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.• It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.Justification: We reviewed the NeurIPS Code of Ethics and found no violation.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.• The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).https://anonymous.4open.science/r/GSAAL-8D6EFor almost all xhttps://anonymous.4open.science/r/GSAAL-8D6ELUNAR and DIF have official implementations by their authors in pyod[43]. 


Decision: This paper provides a mathematical formulation of multiple views, convergence guarantees for the discriminators, and scalability results for Generative Subspace Adversarial Active Learning. Two of the three reviewers were inclined to reject this manuscript, mainly because it lacked sufficient innovation and strong experiments. After discussion, the rebuttal failed to address these concerns, and the final decision is reject.

Reviews: review: paper_summary: paper_summary: The main contribution of this paper is to improve existing work on Generative Adversarial Active Learning (GAAL) by using multiple discriminators for multiple views to detect outliers in tabular data. The training mechanism is similar to existing works. The paper also introduces a theoretical analysis on Multiple Views (MV). As claimed by the authors, GAAL addressed the problems of Inlier Assumption (IA) and Curse of Dimensionality (CD), but missed Multiple Views (MV), which is the main focus of this paper. The experimental results compare the proposed method to GAAL and some other classical methods such as OCSVM and KNN, ....
strength_weakness: strength_weakness: * The empirical results are not strong (or at least unclear in the way the authors presented in the main paper); most of the experiments are on synthetic datasets.
* The results on the real dataset do not seem to show significant improvements compared to existing work (or at least it is hard to observe this when reading the paper). Perhaps the authors could improve the writing and highlight the results better. It is unclear to me why the experiments on the real dataset were put in the Appendix, as it is an important result.
* The paper claims at the beginning that it not only improves the MV problem but also the IA and CD problems, but this is hard to see with the current writing of the paper. Could the authors highlight the experiments in the paper to prove that claim?
questions: questions: 1. How do you define the number of discriminators and the number of realizations u?
2. In Fig. 3, why is GAAL missing? The paper claims it is faster in terms of time complexity. Which results show this?
limitations: limitations: N/A
paper_summary: paper_summary: This paper presents this generalization of GAAL Generative Subspace Adversarial Active Learning (GSAAL) for outlier detection to address the limitation of the previous work such as multi-view and the curse of dimensionality, where the theoretical convergence, the scalability of the algorithm are discussed. Experiments on real dataset and synthetic tabular dataset are carried out to establish the validity of the approaches.
strength_weakness: strength_weakness: The novelty of the work appears to be small. Theoretically, the derivation of theorem 1 is very similar to GAN derivation. 

In this case, the paper needs to compare their solution both theoretically and experimentally with the related work for outlier detection using GAN such as [1] https://arxiv.org/pdf/1906.11632 such as AnoGAN, BioGAN and EGBAD. 
[2] https://asp-eurasipjournals.springeropen.com/articles/10.1186/s13634-022-00943-7
If we compare the main equation (2) in the manuscript with the formulation in reference [1] with conditional GAN and BioGAN, it seems the main difference are the proposed method used multiple detectors and accumulated the performance, which should not be considered a large distinction.

Due to the lack of the comparison with generative adversarial network based approaches such as AnoGAN and EGBAD, the potential improvement of the purposed method against the state-of-the-art approaches is not clear. The novelty of the paper does not stand on the safe ground. The theoretical derivation is also similar to GAN derivation.
questions: questions: Like mentioned above, in order to convince the readers, the paper should really clarify the different and improvement of their solution compared to GAN based solution and focus on explaining and justifying whether or why the improvement against GAN (if there are) is significant.
limitations: limitations: Limited innovation and lack of critical comparison with important reference are the main issues of the current manuscript.
paper_summary: paper_summary: This paper proposed GSAAL to simultaneously address three changeling problems in outlier detection: inlier assumption (IA), curse of dimensionality (CD), and multiple views (MV).
strength_weakness: strength_weakness: some (but few) places to improve.
questions: questions: 1. There are a lot of abbreviations in this article. Before using them, they should be first defined.
2. In line 96, “Classical Methods” lacks recently published work, such as “Mean-shift outlier detection and filtering”.
3. In line 159, it reads “p_x(x) = p_{ux}(ux) for almost all x” (Equation 1). Please classify whether x here refers only to normal samples or also to outlier samples. 
4. Why only K-NN-based baselines were shown in Fig. 3? How about other baselines? The font size of the text in Fig. 3 was too small.
5. Please explain the meaning of “FA” in Table 5.
6. The Y-axis in Figure 4 lacks a name.
limitations: limitations: The authors have analyzed the limitations sufficiently.

Comments: title: Response
Response to the rebuttal
Rebuttal review
comment: Dear all,

Thanks for your time and efforts in reviewing this paper. This is the right and emerging time to discuss this paper with the authors.

The authors provided their rebuttal, and some reviewers have posted their partial discussion about this paper.

Any discussion is welcome and you may consider reading each others' reviews, posting a question, and reaching a consensus.

Best,
Your AC
We are sorry, but this does not help us much to improve our paper. Could you please be more specific about which of our arguments or clarifications you find unconvincing and why? In our rebuttal, we showed that the methods requested in comment 2 are already part of our experiments, the information from comments 1 and 4 is included, and the methods in comment 3 are not proposed for tabular data or even applicable, as noted in our paper and cited references.
Thanks for the detailed comments.

After reading other reviewer's comments and the rebuttal, I decide to main my rating.
I have read the rebuttal. Thanks for your explanation. Please adjust as you said.