Title: Anomaly Detection by an Ensemble of  Random Pairs of Hyperspheres
Venue: NeurIPS 2025
Accepted: yes
Keywords: ensemble

isolation

anomaly detection
Abstract: Anomaly detection is a crucial task in data mining, focusing on identifying data points that deviate significantly from the main patterns in the data. This paper introduces Anomaly Detection by an Ensemble of Random Pairs of Hyperspheres (ADERH), a new isolation-based technique leveraging two key observations: (i) anomalies are comparatively rare, and (ii) they typically deviate more strongly from general patterns than normal data points. Drawing on a delta-separation argument, ADERH constructs an ensemble of multi-scale hyperspheres built upon randomly paired data points to identify anomalies. To address inevitable overlaps between anomalous and normal regions in the feature space, ADERH integrates two complementary concepts: Pitch, which highlights points near hypersphere boundaries, and NDensity, which down-weights hyperspheres centered on sparse (and often anomalous) regions. By averaging these local, density-adjusted ``isolation'' indicators across many random subsets, ADERH yields robust anomaly scores that clearly separate normal from abnormal samples. Extensive experiments on diverse real-world datasets show that ADERH consistently outperforms state-of-the-art methods while maintaining linear runtime scalability and stable performance across varying hyperparameter settings.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes ADERH (Anomaly Detection by an Ensemble of Random Pairs of Hyperspheres), an isolation-based anomaly detection method. The method isolates anomalies by randomly pairing data points in small subsets and constructing two hyperspheres using half of the pairwise distance as the radius. The core innovations of ADERH include: (1) creating diverse hyperspheres by randomly pairing samples in many small subsets, where the radii naturally reflect local density; (2) introducing the Pitch metric to quantify the normalized distance from points to hypersphere centers; (3) using NDensity to down-weight hyperspheres in sparse (anomalous) regions. Experiments on 22 benchmark datasets show that ADERH achieves the best AUC-ROC on 14 datasets with an average rank of 1.67, significantly outperforming existing methods.\n\nSTRENGTHS AND WEAKNESSES:\n1. The paper has a clear and reasonable motivation. The core ideas of isolation and isolation ensemble have been widely validated in anomaly detection. The ensemble of hyperspheres with radius halving and density adjustment is an interesting innovation. In particular, the idea of using local structural information (i.e., distances between paired points) to construct hyperspheres demonstrates certain novelty. \n\n2. The design of the two concepts, Pitch and NDensity, introduced in the paper is ingenious. Pitch effectively highlights anomalies near hypersphere boundaries through a ratio-based distance metric. NDensity appropriately reduces the influence of hyperspheres potentially centered on anomalies. The combination of these two complementary measures enables the method to effectively handle partial violations of δ-separation. \n \n3. The experimental evaluation uses a substantial number of benchmark datasets. Testing on 22 diverse datasets from different domains enhances the credibility of the results. The comparison includes various types of anomaly detection methods (isolation-based, deep learning-based, LOF, etc.), making the experimental comparison relatively comprehensive. \n \n4. The paper provides a theoretical analysis of the proposed method, particularly regarding δ-separation, variance bounds, and concentration inequalities. These theoretical results offer certain guarantees for the method's effectiveness and help readers understand why ADERH can effectively separate normal and anomalous samples.\n\nWeaknesses\n\n1. The experimental design description is confusing. For example, in Section 4.1, line 314 states \"Each experiment was repeated across three splits, with the average AUC-ROC and AUC-PR scores reported.\" However, in line 318 of the same section, it states \"For methods involving randomness (ADERH, INNE, IForest, EIF, PIDForest, DIF), we used seeds [0, 1, 2, 100, 1000] and reported the average AUC-ROC and AUC-PR.\" Furthermore, in Appendix K, it mentions \"This split is repeated three times with different random seeds; we report the average performance metrics across these three runs.\" This is confusing to me as a reader. Could you please clarify the complete experimental setup? How many independent runs were conducted in total for each method-dataset combination? Is it 3, 5, or 15? This directly affects the validity of the statistical tests.\n \n2. Although the authors acknowledge in Appendix R that real-world data rarely satisfies strict δ-separation, the theoretical analysis in this paper is primarily based on this idealized assumption, which may limit the method's applicability to complex data distributions.\n \n3. While the authors claim that the method is robust to hyperparameters, it still requires setting the number of subsets n and subset size ω. For different types of datasets, the optimal choice of these parameters may vary significantly.\n \n4. The proposed ADERH is conceptually quite similar to the existing INNE method, as both use hyperspheres. The main differences lie in the pairing strategy and weighting scheme.\n\nQUESTIONS:\n1. In Section 4.1 of the paper, why was the non-consecutive random seed combination [0,1,2,100,1000] chosen? Has this choice been validated? More importantly, for methods that don't involve randomness (such as LOF), were they also run on the same 3 data splits? How do you ensure that all methods are compared on exactly the same data splits?\n \n2. The “grid search” in Appendix P is actually just a limited sampling of parameter combinations (ADERH only tests 9 combinations), and the search space sizes vary greatly across different methods. Can you explain why different methods were allocated different search space sizes and provide the rationale for determining these parameter ranges? Could a more comprehensive hyperparameter optimization be performed? Or acknowledge the limitations of the current search? Also, can this limited search effectively support the conclusion that 'ADERH remains best even under optimal tuning'?\n \n3. While the paper cites variance bounds to support the choice of subset number n, this only shows that larger n leads to smaller variance, but doesn't prove that 256 is optimal. Additionally, the paper discusses the choices of subset number n and subset size ω separately, but lacks analysis of their interaction. Can the authors provide stronger theoretical support for determining optimal parameters? Theoretically, n×ω determines the total number of hyperspheres - is this product more important than individual n or ω? Or can the authors provide theoretical analysis on the joint optimization of n and ω?\n \n4. Although Figure 4 in Appendix H shows parameter stability, the test datasets vary drastically in scale, dimensions, and anomaly rates (as shown in the dataset details in Appendix J, Census has 299,285 samples while Lymphography has only 148), yet the same default parameters ω=18 and n=256 are used. Can the authors provide more detailed sensitivity analysis, including the magnitude of performance changes due to parameter variations on different datasets? Or can the authors explain the rationale for this uniform parameter setting and whether they considered providing parameter selection guidelines based on different data characteristics?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-20"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a new isolation-based technique called ADERH by leveraging two key observations: (i) anomalies are comparatively rare, and (ii) they typically deviate stronger from general patterns than normal data points. ADERH  isolates anomalies using compact hyperspheres designed to minimize overlap with anomalies. The goal of ADERH is to create multiple hyperspheres of different radii in each subset Si, so that dense areas are captured by smaller hyperspheres, and sparser areas by larger ones.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n \nThe paper provides theoretically rigorous justification for why their method works. \n\nThey have good results shown in Table 1. \n\nWeaknesses\n \nThere is a lot of repetitive information in this paper. Some things like the Rarity, Deviation and using Pitch and NDensity to deal with real world data that doesn’t actually satisfy Rarity and Deviation, are repeated numerous times in the paper in slightly different ways. \n\nThe paper could be a lot more concise and a lot of the useful information in the appendix could be moved to the main paper. The paper spends almost 3 pages on the abstract, intro, contributions and related work where there is significant overlap in the information between the abstract and contributions, and between the intro and related work. It then spends another 2-3 pages on definitions, most of which are not significantly necessary for the main result in Theorem 3.15.\n\nSee questions, particularly the last question.\n\nQUESTIONS:\nThe norm in your distance metrics is l2 norm? What if your data does not lie on a manifold where Euclidean distance is not the metric space e.g. text data or other discrete data?\n\nWhat if your normal data does not cluster spherically but instead are shaped as curves e.g. half moons or donuts? A significant drawback of methods such as 1 class SVM is being unable to “draw” a smooth round boundary around the normal data.\n\nThese concepts of using multiple “blobs” (whether spherical or other shapes) to cover normal points and form densities is not new. How does your method compare to the ones listed below (which are not in the related work nor compared against)? These prior methods also use coverings to estimate densities where a point is anomalous if its score is large than normal points scores. And [2] does not reply on requiring the data to cluster / have density / be covered by a sphere as it creates these covering with K nearest neighbors giving it more freedom in the shape of the coverings. \n\n[1] C. Scott and R. Nowak, “Learning minimum volume sets,” Journal of Machine Learning Research, vol. 7, pp. 665–704, April 2006.\n\n[2] Hero, Alfred. \"Geometric entropy minimization (GEM) for anomaly detection and localization.\" Advances in neural information processing systems 19 (2006). \n\n[3] Gu, Xiaoyi, Leman Akoglu, and Alessandro Rinaldo. \"Statistical analysis of nearest neighbor methods for anomaly detection.\" Advances in Neural Information Processing Systems 32 (2019).\n\nA strong rebuttal of this last question would lead to a significant change in score as it would significantly improve the originality and significance of this paper.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-20"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIt is proposed to isolate anomalies using an ensemble of pairs of hyperspheres, randomly generated from a random subset of the dataset. The radius of these pairs of hyperspheres is half the distance between their centers. The spheres are then analyzed with respect to Pitch and (normalized) density to compute a score for this subset, which is then averaged over the ensemble. It is shown that, under the assumption that anomalies are far from cluster centers, this setup with assigns lower scores to anomalies with high probability, and this uncertainty decreases as the ensemble size increases. In experiments, it is shown that the proposed approach outperforms many existing methods on several popular benchmark datasets.\n\nSTRENGTHS AND WEAKNESSES:\nThe proposed method is original and makes sense. It is good that theoretical evidence is provided for the method, which considerably raises the technical quality of the manuscript—most new anomaly detection methods either don't provide theoretical evidence, or don't provide good experimental results, but this paper provides both. I believe the idea of using pairs of hyperspheres is somewhat interesting to the community, but not largely so. The descriptions are mostly clear, but can be improved (see suggestions).\n\nHowever, the paper is not without flaws. I have the impression that the authors somewhat oversell their method and attribute traits to it without providing any proof. For example, it is claimed that ADERH effectively captures heterogeneous distributions, but no proof is provided. The description of ADERH's limitations is very brief—for a method consisting of several concepts (pitch, normalization, subsets, density, ...) there surely are additional limitations/drawbacks that should be mentioned to give an honest and complete picture of the proposed method. Also, some of the formal statements are too imprecise, but this does not seem to affect the overall correctness of the approach.\n\nQUESTIONS:\n- Please describe how/if assumption 3.1 is affected by the curse of dimensionality\n- As stated in line 184, how does ADERH capture heterogeneous distributions more effectively (...than what exactly)? Can you prove this statement? If not, I suggest removing it to enhance clarity.\n- Regarding equation (12), I am wondering how any data point can have $T(x,\\mathcal{S}_i)=\\emptyset$. Is it because $\\mathcal{S}_i\\subset X$ and the possibility that $x\\not\\in \\mathcal{S}_i$?\n\n\n\n**Improvement suggestions**\n\n- Line 32: It might be good to add Deep Isolation Forest to the list of related approaches\n- Line 34: Hawkins 1980 is not the best reference for this statement. You may want to use a different reference for rarity, e.g. Barnett and Lewis 1974, or Aggrawal 2017.\n- Line 41: This is a description of how Iforest works, not a weakness of Iforest.\n- Section 2 feels a bit too long. I would cut out some of the older approaches like LOF.\n- Line 130: The statement in brackets (\"...indicated by $0\\ll \\alpha$\") can be omitted, as well as (\"as evidenced by RARITY...\")\n- Assumption 3.1: $J$ is not defined. Also, please avoid vague language like *typical* in formal environments like assumptions, theorems etc.\n- Line 176: Radius variability was not described until this point, so this passage is confusing.\n- Theorem A.3 is strange. A lemma is not an assumption: A lemma is a statement that is true. The theorem does not really show that $\\alpha=\\frac{1}{2}$ is the best choice—it instead shows that two properties hold for this setting, but these properties do not imply optimality in any sense, especially not if the random pairing function is not used.\n- Figure 2 seems more confusing than helpful to me. I would suggest to remove it.\n- Lines 220-222: I think this argument is wrong. Each anomaly in $\\mathcal{S}_i$ will be the partner of exactly one point in $\\mathcal{S}_i$, and per the rarity argument this occurrence will only have a small effect on the entire ensemble.\n- Definition 3.7: I understand what you are trying to say, but symbol $X_\\mathcal{H}$ is only defined later in line 244.\n- Equation 9: Use $\\max$ instead of $max$, because the latter reads as $m\\cdot a \\cdot x$\n- Equation 10: The symbol $c_\\mathcal{H}$ is not defined.\n- Lemma 3.10: I understand where the proof is heading, but the notation needs to be improved. Please rephrase the lemma s.t. it is clear that $\\text{Ndensity}(\\mathcal{H}(z))$ converges to zero almost surely. Also, please don't use $\\ll $  here because the symbol is vague.\n- Lemma 3.13: I recommend to add the description \"informal\" in the main text and to restate the lemma more precisely in the appendix (avoiding $\\ll $ )\n\nLIMITATIONS:\nThe description of ADERH's limitations is very brief—for a method consisting of several concepts (pitch, normalization, subsets, density, ...) there surely are additional limitations/drawbacks that should be mentioned to give an honest and complete picture of the proposed method", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents an anomaly detection method built on the concept of isolation. The approach constructs multiple hyperspheres using randomly selected pairs of samples and assigns anomaly scores based on each data point's position relative to the nearest hypersphere center and the hypersphere’s density characteristics.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n1. The paper is well organized.\n\n2. The theoretical analysis is comprehensive.\n\n**Weaknesses:**\n\n1. The experimental setup section lacks proper citations for the baseline methods. I recommend that the authors include full references for all baselines to ensure clarity and proper attribution.\n\n2. The paper lacks comparisons with recent state-of-the-art baselines. It only evaluates against methods published up to 2024, despite several relevant works from other conference, like ICLR 2025 and AAAI 2025, already being publicly available.\n\n3. Statistical results, such as mean and standard deviation over multiple runs, are not reported.\n\n4. There are no visualizations or additional evidence to support the authors’ claims, only experimental results are provided. For example:\n\n(1) Although Pitch is said to emphasize boundary anomalies and NDensity down-weights hyperspheres in sparse regions, the paper provides no visual comparisons of scores for normal vs. anomalous points to show these components work as intended.\n\n(2) A key claim is that random pairing naturally results in hyperspheres of varying radii (multi-scale coverage), but there is no histogram, distribution plot, or illustrative example to support this.\n\n(3) The use of a multiplicative combination of Pitch and NDensity is motivated in theory, but no score distribution plots, t-SNE projections, or spatial embeddings are shown to compare additive vs. multiplicative strategies.\n\n(4) The ensemble scoring mechanism is said to reduce variance and improve robustness, but this is supported only by theoretical bounds. There are no visualizations or comparisons of score distributions across subsets or before/after ensemble averaging.\n\nQUESTIONS:\nPlease refer to Weaknesses.\n\nLIMITATIONS:\nPlease refer to Weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}