Title: Interactive Anomaly Detection for Articulated Objects via Motion Anticipation
Venue: NeurIPS 2025
Accepted: yes
Keywords: articulated objects

anomaly detection

active vision
Abstract: This paper presents a novel problem, interactive anomaly detection (AD) for articulated objects, and introduces a tailored solution that detects functional anomalies by integrating vision, interaction, and anticipation. Unlike traditional AD methods that rely on passive visual observations, our approach actively manipulates objects to reveal anomalies that would otherwise remain hidden. Our method learns to generate a sequence of actions to interact exclusively with normal objects and to anticipate the resulting normal motion. During inference, the model applies predicted actions to the object and compares the observed motion with the anticipated motion to detect anomalies. Additionally, we introduce a new benchmark, PartNet-IAD, for interactive AD, which includes articulated objects with realistic functional anomalies. Experiments show strong generalization to detect anomalies in both seen and unseen object categories. Code and dataset will be released.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this work, the authors propose to tackle interactive anomaly detection (IAD), that is, the detection of anomalies that occur only after interacting with the object (e.g., opening and closing a drawer). The authors propose a new benchmark and approach to interact and detect this kind of anomaly. Specifically, the proposed approach generates a set of actions to interact with the object and predict its motion. Suppose the predicted motion is different to a certain degree from the actual motion, and the model will detect the anomaly. The authors define different baselines to compare with.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n- The proposed formulation of IAD is original and well-motivated. Most existing anomaly detection benchmarks focus on passive visual cues. In contrast, this work emphasizes that many real-world anomalies only present themselves through physical interaction, a critical insight for robotics and quality assurance in manufacturing.\n\n- The proposed method introduces a joint modeling of interaction and motion anticipation. While each component (e.g., motion prediction, action sampling) builds on prior work, their combination into an end-to-end interactive anomaly detection pipeline is novel. The idea of detecting anomalies through discrepancies between expected and observed motion is both intuitive and effective.\n\n- The paper is clear, logically organized, and provides strong qualitative and quantitative results\n\n- The PartNet-IAD benchmark fills a gap in the community by offering a dataset for interactive anomaly detection. The design of the anomalies and interaction protocols is realistic, and the evaluation across both seen and unseen categories demonstrates the model’s generalizability.\n\n- The proposed approach achieves substantial performance gains (e.g., AUROC improvements of ~6 points over the strongest baseline), suggesting its practical utility.\n\n### Weaknesses\n\n- While the core idea is strong, the experimental section could benefit from more thorough ablation studies. For instance, the paper briefly mentions the use of confidence-based weighting (Eq. 6) and a multi-step filtering strategy for action selection. However, the impact of these design choices is not isolated and quantified. A study showing the effect of varying \\lambda values in the loss would strengthen the empirical analysis.\n\n- While the baselines are well-designed, most rely on Where2Act, and it is unclear how much performance gain comes from better interaction versus better anomaly scoring. A more comprehensive comparative analysis with a broader range of interaction strategies would be beneficial.\n\nQUESTIONS:\n1. Please provide more detail on how sensitive the method is to the loss weight hyperparameters (e.g., different \\lambdas in Eq. 6). A sweep or sensitivity analysis would be valuable.\n\n2. How robust is the approach to noisy or partial depth observations, especially since it relies on point clouds?\n\n3. How critical is the use of the long-horizon motion estimation compared to atomic action steps?\n\nLIMITATIONS:\nLimitations have been discussed.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a novel task, namely *Interactive Anomaly Detection* (IAD) that aims at classifying an articulated object as anomalous in case of functional probelms. This represent a step further respect to images AD, and can be useful in several industrial scenarios.\n\nThe authors also contribute a novel method for IAD, in which they compare possible movments on the  articulated object movable parts with the same actions as performed on a functioning *replica* of the object inspected.\n\nThis work presents a novel dataset (even though it originates from a pre-existing one) that is used for benchmarking their proposed method against five baselines reporting best results.\n\nSTRENGTHS AND WEAKNESSES:\nSTRENGHTS\nThe paper is well written and is full of details.\nThe proposed task seems valuable and novel.\nThe method proposed, while being directly inherited from classical Anomaly Detection techniques, is well posed and articulated.\nThe proposed baselines are not very challenging, but they are informative when compared with the proposed method.\nThe integration of CV methods, robotics and simulated environments is interesting\n\nWEAKNESSES\nWhen proposing a novel task, the motivations are very important. While the authors provide some motivations, I think that the importance of the proposed IAD task could be expressed more clearly. For example, I imagine that IAD could be very interesting for industries, that could benefit from inspecting virtual copies of their produced goods.\nThe benchmark is limited to a selection of objects. While I acknowledge that this paper represents only the first work in IAD, I also think that it would be challenging to extend it to real-world objects (virtual copies of industrial objects) to be valuable for the industries.\nIt is not very clear to me how the treshold on the anomaly score is calculated, as it could change from object to object.\n\nQUESTIONS:\nSee weaknesses\n\nLIMITATIONS:\nThe main limitations of this work is in its extendability. \nIt seems very challenging to construct a dataset similar to the one the author propose and this could limit the interest in IAD for the CV community.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a benchmark for interactive anomaly detection of articulated objects, using motion anticipation as the basis. Alongside the benchmark, it introduces a learning-based approach to anomaly detection. The method detects anomalies by learning how the object is normally interacted with through a robotic agent. It learns to predict normal functional behaviour through interactions with normal objects. During testing, the method detects anomalies by comparing the observed motion of the object with the model's learned normal behaviour. The approach is evaluated on the proposed benchmark and compared to several baselines using standard evaluation metrics.\n\nSTRENGTHS AND WEAKNESSES:\n- The paper would be better suited to the benchmark track. The method is not entirely novel. Instead, the main contribution is the benchmark.\n- Limited novelty: the proposed method does not make significant contributions. The idea of using motion modelling for anomaly detection is well established in the literature (e.g. automated driving trajectory-based anomaly prediction; see the Shift dataset). The proposed approach is also quite similar to Where2Act. It should be made clear what the proposed novelties are.\n- Benchmark clarity: The selection of categories for training and testing lacks clarity. While the paper mentions a 'mutually exclusive set', it is unclear whether the categories are chosen randomly or manually, or if they are functionally disjoint (e.g. a closet door versus a regular door). This ambiguity limits the strength of the generalisation claims. It would be valuable to conduct an ablation study analysing how training category selection affects performance.\n- Benchmark clarity: The paper provides limited information on how anomalies are generated in the dataset. For example, the anomaly shown in Figure 1 appears somewhat unrealistic, which could reduce the relevance of the benchmark.\n- More evaluation metrics: The only metric reported for anomaly detection performance is AUROC. However, this metric is insensitive to class imbalance, which is usually the case in anomaly detection since, by definition, anomalies are rare. This can lead to deceptively high AUROC scores even when the model performs poorly on the rare class (i.e. anomalies). To get a more complete picture of anomaly detection performance, benchmarks typically require precision-recall area under the curve (AUPRC) and false positive rate at 95% recall to be reported.\n- Method clarity: The method description lacks clarity and uses inconsistent terminology. For example, line 125 refers to 'decoders' for motion and segmentation, whereas later sections refer to 'heads'. Maintaining consistent terminology would improve readability.\n- The paper appears somewhat unpolished in places. For example, the abstract could be improved by providing a brief explanation of the broader context and potential applications of interactive anomaly detection to motivate the problem. A condensed version of the first paragraph of the introduction could achieve this.\n\n\nMinor points: \n- Section 4.4 (Limitations and Failure Cases) would benefit from a more detailed analysis of failure modes, including potential causes of false positives/negatives, as well as suggestions for how these could be mitigated in future work.\n- The term 'online data sampling' is introduced in Section 4.2 without explanation. Explaining what this entails and why it is beneficial would improve reproducibility.\n\nQUESTIONS:\n- How does the proposed method differ from Where2Act, and how directly comparable are the chosen baselines to it?\n- How are object categories for training and testing selected? Are they chosen randomly or according to specific criteria?\n- Have you computed additional evaluation metrics, such as AUPRC or FPR@95% recall? These could help to improve our understanding of the model's performance.\n- Are the failure cases shown in Section 4.4 representative of all typical failures, or have other types of false detection been observed?\n\nLIMITATIONS:\nyes.", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}