Interactive Anomaly Detection for Articulated Objects via Motion Anticipation
Authors: Ankan Kumar Bhunia
Changjian Li
Hakan Bilen
Venue: NeurIPS 2025 poster ()
Keywords: articulated objects
anomaly detection
active vision
Primary area: applications
TL;DR: An interactive anomaly detection method for articulated objects

Abstract: This paper presents a novel problem, interactive anomaly detection (AD) for articulated objects, and introduces a tailored solution that detects functional anomalies by integrating vision, interaction, and anticipation. Unlike traditional AD methods that rely on passive visual observations, our approach actively manipulates objects to reveal anomalies that would otherwise remain hidden. Our method learns to generate a sequence of actions to interact exclusively with normal objects and to anticipate the resulting normal motion. During inference, the model applies predicted actions to the object and compares the observed motion with the anticipated motion to detect anomalies. Additionally, we introduce a new benchmark, PartNet-IAD, for interactive AD, which includes articulated objects with realistic functional anomalies. Experiments show strong generalization to detect anomalies in both seen and unseen object categories. Code and dataset will be released.

Interactive Anomaly Detection for Articulated Objects via Motion Anticipation
Ankan Bhunia Changjian Li
Hakan Bilen
University of Edinburgh
Abstract
This paper presents a novel problem, interactive anomaly detection (AD) for articu- lated objects, and introduces a tailored solution that detects functional anomalies by integrating vision, interaction, and anticipation. Unlike traditional AD methods that rely on passive visual observations, our approach actively manipulates objects reveal anomalies that would otherwise remain hidden. Our method learns generate a sequence of actions to interact exclusively with normal objects and anticipate the resulting normal motion. During inference, the model applies predicted actions to the object and compares the observed motion with the antici- pated motion detect anomalies. Additionally. we introduce a new benchmark, PartNet-IAD, for interactive AD, which includes articulated objects with realistic functional anomalies. Experiments show strong generalization detect anomalies in both seen and unseen object categories.
Introduction
Humans possess a remarkable ability to interact effortlessly with wide range of objects in everyday life. This capability stems from our intuitive understanding of how objects funetion-we assess their potential uses, apply appropriate forces, and anticipate the outcomes of our actions. Importantly, adjust our behavior when the actual response an object deviates from our expectations. These discrepancies often serve key feedback signals, especially when object fails operate as intended what we refer to as functional anomalies. This work aims to develop a perception system that mirrors this human capability by detecting functional anomalies in articulated objects through combination of vision, physical interaction, and motion anticipation.
Detecting functional anomalies in articulated objects is essential for quality inspection in manufactur- ing (e.g., drawers, washing machines, laptops) and for robotic manipulation tasks. Unlike standard anomaly detection (AD) benchmarks that focus on identifying visually apparent defects from static observations 12 46 13 5 g 3 many functional anomalies in articulated objects are not visually observable and only become evident during physical interaction. As such, detecting these anomalies requires an active maarracchmmnipulattt
objects, anticipating their normal behavior, and identifying deviations from expected motion (see Fig. I.
This paper presents novel perspective on AD by proposing an interactive AD framework that addresses two central challenges: (1) learning how to interact with objects to reveal hidden anomalies, and (2) anticipating their normal motion response during interaction. Unlike the prior work 28 40 which focuses interaction with articulated objects without modeling expected outcome, our approach predicts anomalies by comparing observed object motion after manipulation with the motion expected under normal conditions. Given an RGBD image partial point cloud and an atomic action (e.g., pushing or pulling), our model predicts: (i) the 3D segmentation and motion of the target part, and (ii) a motion confidence score, and an interaction end state indicating whether the part has reached an interaction end state (e.g., a door fully opened). Importantly, the model is trained exclusively on
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Observed Motion
Agent
Compare Motion
Anomaly Normal
Action Sequences
RGBD Image
Anticipated Normal Motion
Figure I: In the proposed interactive anomaly detection setting, the algorithm learns to manipulate articulated objects without any anomalies through generating a sequence of actions while predicting their normal motion. During testing, the algorithm first generates a sequence of actions along with their anticipated normal motion without any interaction (bottom). Then the agent (e.g., ., robotic arm) executes the generated actions on the object resulting the observed motion (top). By comparing the anticipated and observed motions, the algorithm can detect potential functional anomalies.
interactions with normal objects, ensuring that its predicted motion aligns with normal behavior. At inference time, the model generates predicted motion trajectory by scquentially applying learned atomic actions without physical interaction. The same action sequence is executed on the actual object by agent (e-g., robotic arm), and the observed trajectory is recorded. Comparing the anticipated and observed motions enables AD based on motion discrepancies.
To support this task, we introduce PartNet-IAD, new benchmark for interactive AD. Built upon the PartNet-Mobillty dataset [39], we inject realistic functional anomalies into articulated objects and simulate interaction environments where robotic arm can push and pull object parts. Results show that our method generalizes well to unseen objects and categories, effectively detecting functional anomalies through interaction.
In summary, our contributions are: (1) we formulate the novel task of detecting functional anomalies through interaction with 3D articulated objects; (2) propose novel model that learns manipulate objects and detect anomalies by comparing expected and observed motion; (3) we introduce PartNer- IAD, the first benchmark for interactive AD, enabling evaluation of AD in articulated objects.
2 Related Work
AD benchmarks. Previous AD benchmarks focus on detecting anomalies from passive observations, such as images [2] 46
44 15 5), videos [8 32 42), or point clouds 44 23 3]. These approaches make predictions based on single observations captured by cameras and do not actively interact with the object or iteratively update their understanding of the object through feedback mechanism. While such approaches are effective for rigid objects, they are often insufficient for articulated objects, where passive observation alone may not reliably reveal anomalies. propose a learnable framework in which an agent actively interacts with articulated objects to uncover motion anomalies that may not apparent otherwise. previous work [4), FixIT, proposes solution to diagnose and fix malfunctioning articulated objects, however, without learning to actively interact with the object, and instead assumes that interaction videos are directly available as input. In contrast, framework simultancously learns to interact and discover anomalies.
address this limitation, we
Interactive perception for articulated objects. Early research in object manipulation has primarily focused on learning task- musk-specffic action primitives, such as manipulating doors and drawers [1] 20 These approaches often rely on hand-tuned actions to generate informative motion for downstream perception tasks [8] 34.. However, these methods typically assume prior knowledge of the object's structure, which is used to design heuristic rules. Additionally, action primitives tailored for one task (e.g., opening doors) may lack the generality needed for other tasks or objects (e.g., pushing buttons). Recent studies have highlighted the effectiveness of exploiting interactions simulated
environments for learning perception models 4 30
27
28
33
36
29 40) demonstrating promising

---

generalization to real-world scenarios [6 31]. Among these studies, Where2Act (28] presents a learnable framework estimate dense action affordance maps on articulated objects from a single RGB image or point cloud, while UMPnet A4 extends this method to handle long-horizon action trajectories for goal-conditioned manipulation tasks. AtP [33 proposes an iterative method for interacting with articulated objects to discover and segment their components. In contrast, our work focuses on discovering motion anomalies by learning to interact with articulated objects. Unlike previous methods [28] 40 [33 that predict only action parameters, our framework jointly predicts both the action and anticipated motion, enabling anomaly detection through comparison with observed motion. Our framework enhances robustness through confidence-basee motion estimation and supports long-term, temporally consistent action exploration.
Rigid motion analysis. The pivotal component of our framework is learnable motion prior that estimates normal motion behavior at any instance. Several prior works [1 24 have presented learning-based rigid motion estimation from point cloud scenes. For articulated objects, however, research on motion estimation has been limited. Recent works estimate object articulation and predict motion flow and segmentation masks from pair of point cloud frames [33 43 or multi-view images [25] [O]. In contrast, our formulation of the motion prior action-conditioned, estimating the rigid motion of articulated object parts for any arbitrary action parameters. Recently, DragAPart [22] proposed a diffusion- basuusdoo---edddddd generation pipeline that predicts deformation for given drag force using images input. Unlike this approach, our motion prior operates on unstructured 3D point clouds.
3 Method
3.1 Problem Formulation and Overview
Given an observation of a 3D object with articulated parts (e.g.. cupboard with a drawer and door), goal is to classify whether each movable part exhibits a functional anomaly (1 if anomalous, 0 otherwise). Since such anomalies are often not detectable through passive observations alone, we allow robot agent to interact with the object over limited number of timesteps T At each timestep f. the robot executes an action a; parameterized by its end-effectorss 's position Pt. E R3 and movement direction u, E R The scene is captured as point cloud Qi at timestep t with Qo denoting the initial observation prior to any interaction.
An overview of our method is shown in Fig. 2 The process begins with the agent observing the initial state of the object Qo and planning a sequence of atomic actions (e.g., push, pull). These actions simulated to anticipate the resulting object motions. At each timestep t, the agent predicts an action a, and the corresponding motion M, for a manipulated part r. Note that we omit the part index in both a and M for brevity. The agent initializes its internal object state as S Qo and updates this state S, over time by applying the predicted motion M, to the relevant part, producing simulated future state S441 without physical interaction. This simulation process iterates over T steps, forming a sequence of atomic action-conditions atomic actions may not sufficiently capture the full range of part motions or cover all parts, we incorporate a long-term normal motion estimation strategy. This ensures comprehensive exploration of the action space with temporal consistency and robustness noise (see Sec. 3.3) The result is
motion priors (see Sec. (.2).. Because isolated
sequence of predicted action-motion pairs {(,,, M. for each part.
The agent then executes the same actions on the real object and collects the resulting observations (Q.) > from which the actual motions are computed (T- A part is classified as anoma- lous if the observed motion sequence deviates from the predicted one by more than threshold
(see Sec. 3.4).
3.2 Atomic Action-Conditioned Motion Prior
Given an internal 3D object state S and an action a. we learn a function V that predicts the expected
normal rigid motion in 3D M. a confidence score c, E (0, 1) for the predicted motion, interaction end-state indicator e, E (0, 1) (e.g., the fully open/closed door) and a binary mask m, E {0,1]'* 1]
indicating points that move coherently with the interaction point p (see Fig. a)). Formally:
(Ma, Ca. Ca, Ca, mp) = (S).)).
(I)
3

---

(a)
Coddi--ondi Mation Prior
Action direction sampler
actual observations
MIP
Action position
sampler
PINS
2D optical flow model
Point Transformer
Predicted mation Ma
partial point cloud 5 (projected from RGBD)
Per-point representation F
backproject
seforea spaca polin
(p.)
Action Conditioned
(pi w())
[u,]
Motion Prior +
filtering noisy
action movability scores
sampled action a
Predicted motion M
Observed mation R.
prodictions
Figure 2: Overview of our approach. (a) We first train action-conditioned motion prior set of articulated objects without anomalies (Sec.3. 3.2). testing time, given the input RGBD image, we (b) exploit the learned prior infer action-motion motion pairs, while predicting the normal motion (Sec. 3.3
At the same time, (c) an agent interacts with the object based on our inferred actions to obtain the observed motion, which is compared against the anticipated motion to reveal anomalies (Sec.
The subscripts a and p indicate the conditioning on action and end-effector's position respectively. We omit timestep t since predictions are sloeic-made per single timestep. The predicted motion M, rigid transformation T. E SE(3), comprising a rotation matrix R. SO(3) and a translation vector ta E R We implement V/ as deep neural network comprising backbone feature encoder and two decoders for motion estimation and segmentation respectively.
Backbone feature encoder. The initial point cloud observation So is derived from the RGBD) image Qo IR HxW For each point, the RGB values and surface normals (estimated via KD-tree search (40)) are concatenated, resulting in an input feature set of size N x 6 where N = H x W. All the predicted S and observed states Q are represented in this format. A Point Transformer3 (PTv3) (37 processes the point cloud, extracting per-point features F R Nxd, where denotes the feature dimension.
Motion estimation head. Estimating rigid motion in the camera frame is ambiguous because the transformation matrix depends the object's position in the scene, in other words, it is not caanslatoon-ivvaiian Following (1, we instead estimate rigid motion in a local coordinate frame centered the action position p, with
aligned to the global frame. To this end, concatenate the local feature F, for p with u. and process it through an MLP mapping network to produce 11-dimensional vector which is further processed by using Gram-Schmidt process [45] to obtain local 6D rotation R and 3D translation vector th respectively relative to u. For optimizing the motion estimation, we use the following loss function:
(a) = Egoo R R.) + A12l (t,, 1.)
where fea computes the geodesic distance [45 in the rotational space and L12 is the 1.2 distance. R% and th are the groundtruth parameters of R and th respectively. AL2 is loss constant. The local rigid matrix is simply transformed into the global coordinate frame as follows:
R R
tg = R )p t
The end-state e, supervised using a binary cross-cntropy loss denoted as (a(a) = (ea Ea). where é, is the groundtruth for the end state.
Segmentation head. Here our goal is predict what points in S will move together with the point p when force is applied on it. We use binary classification head which takes in the local feature Fp after getting replicated N times along the sample dimension and concatenated with the point cloud representation F along the feature dimension, to provide global context. The resulting N 2d dimensional feature is processed through an MLP mapping network to predict the mask m E {0, N1, For optimizing the segmentation head, we use a binary cross-entropy loss & for each point p as follows:
lm (p) Loce (mpp, mp),

---

t=1
==
t=
Figure 3: Anticipated trajectory visualization: for each timestep 1, we show the movability map along with selected action, and predicted next state. The trajectory is shown for a single point. Movability scores are averaged over 250 sampled action directions and visualized using RGB mapping.
where m, E {0, 1) is the groundtruth mask. Finally, the total loss function weighted sum of the motion and segmentation losses:
Lm (p) + AMEM(() + )ele(a),
(5)
where AM and de are loss constants.
Regressing rigid motion parameters is challenging, particularly under occlusion, where some action- induced motions are harder to predict. To address this, we use the predicted confidence score Ca to modulate the loss function in Eq. (5), following [19]
em (p) + ca (Aaseas(a)) (a) )elog(Ca),
66)
where de is a loss weight. When C. is predicted to be low for a given action, the corresponding loss is down-weightes The last logarithmic term is regularizer and encourages the model to avoid setting low confidence for all samples trivially. We train the model end-to-end on normal objects using the learn normal motion patterns for arbitrary actions.
total loss in Eq. (6) enabling
3.3 Long-term Normal Motion Anticipation
We describe how the normal motion predictions for individual atomic actions utilized to estimate the normal trajectory over multiple timesteps (see Fig. (2)).. Analyzing anomalies over multiple timesteps is crucial, as revealing anomalies typically requires manipulating each part over its full action space (e.g., verifying whether drawer fully opens without detaching beyond its intended range) and assessing all object parts. To facilitate this, introduce part memory to track examined parts and propose two-stage filtering strategy to select informative actions likely to induce motion the part.
Part history. For each object, we use two memory masks, B and B+, each consisting of N binary elements. B records previously examined parts, where elements corresponding the previously manipulated parts are set to 1, while B+ tracks the part currently being manipulated. Following interaction, the memory is updated
B B v B+ Since the mask of the next part is initially unknown, we initialize it as B + 1 - B, where 1 is N-dimensional vector with all elements set to ensuring already examined parts are excluded. After the first timestep, B+ is updated based on segmentation head predictions.
Action candidate selection. Directly regressing actions is challenging, strategy to select informative actions. At each time step. we define the action space for an articulated part r as (rr) = {pI...., PN. x {uI..... u,.,,,,,,,, where {p.} denotes N,, points sampled from the mask + on the part and {u,} } are N,, directions uniformly sampled in the SO(3) space. We begin by estimating the 'normal' motion Ma., for each pair a;₃ (p., uj using Eq. (I).
propose robust
Given the inherent noise in motion prediction, we implement a two-stage filtering process. First, actions with low confidence scores (<10%) are discarded to eliminate unreliable predictions. Next, we apply a density- outlier removal strategy by extracting and normalizing the rotation and translation axes from the predicted rigid motion matrices. These unit vectors are then clustered using DBSCAN [2). grouping similar rotations (and translations) while identifying sparse or inconsistent vectors as outliers. For revolute joints, this approach yields two clusters (forward and backward rotations), effectively filtering out noisy predictions.
5

---

To ensure motion consistency with the previous timestep, we leverage the clustering results to exclude actions that would induce the opposite motion, refining the action space to A Subsequently, an action selected based on a movability score S?(a), which is derived from the predicted rigid transformation matrix Ta and defined as:
D(a)=
Talle
where I denotes the Frobenius norm, and Il is the 4 x 4 identity matrix. Using 1?(a), we define normalized score (2) as:
exp (Ar)
((a) E (a)e(( exp (D(a)/AT))
(8)
where Xr is a temperature parameter. We use (2(a) as probability distribution to sample action (r) (p(r), (+))) from Á We use the predicted motion M. derive the next state S and iterate the above process again. The interaction terminates based on the end-state token e, or
when Taxx is reached. We concatenate the predicted motion over all timesteps to obtain the discrete
'normal' trajectory and denote it as C Fig. shows an example of the anticipated trajectory starting from initial observation.
3.4 Measuring Observed Trajectory Error
The agent sequentially executes {ar T-11 (==) for each part, generating the actual observation (Qt)== The kinematic change between Qe and Q++1 is estimated as rigid transformations denoted as M, (see Fig, 2 c)). To compute Mf, omit the depth channels of Q2 and Q++1, retaining only their RGB values, and apply an optical flow method [35] to predict 2D motion flow. Given that the camera remains fixed during interactions, backproject the 2D motion flow into 3D using depth information to obtain 3D motion flow vectors. We compute both forward and backward flows and derive their respective transformation matrices using the Kabsch algorithm [7]. To enhance robustness, we iteratively refine the set of flow vectors using RANSAC while enforeing bidirectional flow consistency. This process yields reliable set of 3D motion vectors for the observed flow. The transformations M, are accumulated over all timesteps for part r and the observed trajectory Si. is computed.
originally computed for the normal motion anticipation,
Finally, the anomaly label y,- for part is assigned by comparing the expected and observed motion trajectory using the Hausdorff distance DH
if A > where A, DH (1 ((1))
y-
otherwise,
We choose the Hausdorff distance for its sensitivity to the worst-case deviation which is beneficial for identifying anomalies effectively. Importantly, our model is not trained explicitly trained to optimize the prediction rule in Eq. 9
4 Experiment
4.1 Interactive AD Benchmark
We introduce a new benchmark for interactive AD: the PartNet-IAD dataset. To construct the dataset, we use objects from the PartNet-Mobility dataset (38] as normal instances, and modify their physical and kinematic structures to generate anomalous articulations, which are used exclusively for evaluation. More details about dataset statistics, anomaly generation process, and anomaly types are provided in the supplementary. We focus on two evaluation settings: i) evaluate our model on the test set of the training categories to measure its generalization ability to unseen objects within the same categories, and ii) evaluate on the test set of the testing categories to measure the generalization ability to unseen object categories. For training the normal motion anticipation module, we use mutually exclusive set of 402 normal objects (from the train set of the training categories of PartNet-Mobillty).

---

Table 1: PartNet-IAD benchmark. AUROC (%) of our framework compared against baselines.
Unseen objects mainnn citegories
Testing calegories
Average
Averrage
22. No interaction + Cls
60.3
61.3
58,5
64.9
58.9
60.3
63.
57.1
60.1
64,9
66.3
61.3
51.3
56.1
52.0
56.7
58.3
56.0
56.2
58,0
19.1
5L.3
55.5
1: Randont-itteeniion Cls
64.1
63.4
63.5
67.1
65.3
63.4
63.1
58.0
61.1
64.5
63.2
63.3
53.6
55.7
528
57.8
59.1
56
55.6
59.0
61.8
50.1
56.
WhereZAct + Cls
70.5
61.5
67.1
70.1
72.0
66.1
629
63.4
63.5
64.0
662
58.8
60.5
61.4
61.8
610
633
62.7
64.3
70.8
55.3
62.0
WhoneAet Hearistics
75.
E: Where2kct MotienPrior
72.2
50.5
85.8
74.55
81.7
79.0
Our Method
91.3 75,3
88.9
85.3
92.6
87.5
89.8
77.8
85.0
$2.8
90.2
86.1
78,9
91.6
53.2
64.4
83.1
88.4
89.1
874
90.6
71.9
R399
4.2 Implementation Details
Environment and action setting. We create interactive simulated environments by using Pybullet (9] For the agent, we use a suction-based flying gripper [40] as the robot actuator. The flying gripper can be initialized at any position and orientation, with the gripper either closed or open. The object is observed using an RGBD camera with known intrinsics, positioned 5 units away from the object and facing its center. The camera is placed on the upper hemisphere with a random azimuth [120", 270%) and altitude [25 1.0 primarily capturing the front view of the object in the canonical frame of the original Mobility objects. To execute an action, the agent first moves its end-effector to the sampled 3D position, aligning its orientation perpendicular to the object surface. In practice, the closest point on the surface is used to place the end-effector. The agent then moves the end-effector 0.18 meters along the action direction. The suction behavior is implemented between the suction cup and the selected 3D position on the object, as described in [40].
force constraint
Collecting interaction data for training. To create a large set of action-motion interaction pairs, we randomly select an articulated part for each normal object and initialize its pose with 50% chance at its rest state (e.g., fully closed drawer)
random pose (e.g., half-opened drawer). We then sample a 3D position on the part's surface and an action direction uniformly from a spherical distribution. The robot arm executes the action, and record the resulting motion as rigid transformation matrix along with its segmentation mask. Additionally, an end-state token is stored if the action leads to an end state (fully closed fully open). For non-movable parts, we record interaction data for 5% of points on these parts, storing their resulting motion as an identity matrix. The offline dataset consists of 145M interaction pairs, generated over 3-4 days on a single 64-core CPU machine by parallelizing the simulation across multiple CPU cores, to kickstart the training. To complement the offline dataset, online data sampling (28) (see supplementary) is introduced after the 10th epoch, with each batch consisting of 70% offline data and 30% online data, focusing on interactions more likely to induce motion.
Inference details. During inference, we use the current memory mask B to sample N,- = 1000 points for the starting interaction of each part and N₂ 250 points after the first timestep once the part is identified. We sample N₂ = 128 action direction to form the total action space. The temperature 'y is set to 0.3 in Eq. (8) Tmaz is set 15.
4.3 Baselines and Results
Since we are the first to propose and formalize this task, no prior work exists for direct comparison. To evaluate our method and establish benchmarks for the proposed task, as shown in Tab. [ we compare against five alternative approaches (A-E) specifically designed for the task:
A: No-interaction + Classifier takes an initial observation frame as input and performs anomaly classification without any interaction.
B:
+ Classifier selects random action positions and directions (uniformly sampled from the action space) at each timestep for interaction using the simulator, followed by frame-by- frame anomaly classification on the observed frames. We perform same number of steps (15) as ours and if single frame during the interaction is flagged as anomaly by the classifier we consider the articulation as anomaly.
C: 2Acr + Classifier employs learned action prediction model (e.g.. Where2Act 28 to predict the action parameters and pass them to the simulator to obtain the observed frames, replacing the random action sampling.
7

---

Initial state
Prodicted state
Predicted trajectory
Observed state
Observed Drajectrtra
Figure 4: Qualitative results of interactive AD; Each block shows (left to right) the input RGBD, estimated movability map, and predicted vs. observed final trajectories. Top two rows are from training categories; the bottom row is from testing categories. Our model correctly detects anomalies in all cases. Color coding is shown above; full interaction videos are in the supplementary material.
D: Where2Act Heuristics includes separate model that is trained to infer the joint type and joint axis for each moving part. We then use Where2Act agent to predict action and the simulator executes the action. We then detect anomalies by comparing the observed motion axis with the predicted ones.
E: Where2Act MotionPrior employs the Where2Act agent to predict the action and utilizes our action-conditioned motion prior to estimate normal motion, obtaining anticipated trajectories for the agent's actions, which are compared with the observed trajectories to detect anomalies.
For A-C. we implement frame-by-fire anomaly classifier using Point Transformer- based model. They take a single point cloud frame as input and predict an anomaly label. The classifier is trained using normal frames and frames, generated online via random deformations or transforma- tions. For D, we use a Point Transformer for part segmentation, followed by pooling opera- tion over each segmented part to obtain the joint axis vector and joint type (revolute prismatic). This model is trained on the normal objects from the training categories. For fair comparison, we train Where2Act using suction-based gripper. Anomaly classification performance is evaluated using the area under the ROC curve (AUROC), computed per part and averaged across all test objects. Qualitative results of our framework are shown in Fig.
Table 2:
We evaluate our method using quantitative metrics for motion estimation (MSE and mloU) and interaction performance (action success rate [ASR] and part success rate [PSR]). The evalua- tion conducted on normal objects from the test categories.
augmented
Method
MSE
mlot! f
ASR(%) 1
PSR (5)
w/o noise filtering
0.08
0.65
96.78
92.85
w/ noise filtering
0.05
0.72
98.35
94,12
Importance of interactive part analysis. We first validate our claim that reliable functional anomaly detection requires agent interaction. Baseline A, lacking interaction, misses anoma- lies not visible the input frame. Baseline B, re- lying random interactions, is ineffective since most actions either cause no motion or fail to re- veal anomalies. In contrast, our method, which uses a learned agent for meaningful interactions, achieves significantly better performance.
Rondiom anmercio Cs
WhenZAet Heundirs
70
+
WhenDAee Weteestorr
65
2
3
Interaction stops
Importance of our action-dependent frame- work. Baselines B and are action- independent-they do not consider which spe- cific action caused the irregular motion, often missing anomalies like drawer failing to close.
Figure 5: Interaction steps vs. AUROC. The results evaluated on the testing categories.

---

Bucket
Box
Trashcan
Figure 6: Real-worid visualization of the motion prior network the AKB-48 (26) dataset. We used our trained motion prior model in PartNett-obility to estimate normal motion for the real-world scanned objects from the AKB-48 dataset. For each row, show the input point cloud, the movability score map, and four sampled action- motion pairs, respectively. The sampling of action-motion pair is performed based the normalized scores. Red arrows indicate the input actions, and the anticipated motions are shown cyan.
Baseline C misses ceniiincondiiooned temporal cues due frame-wise classification, while Baseline D predicts joint axes but ignores the magnitude of action-induced motion. contrast, our final model conditions anomaly prediction on the executed action, resulting in significantly higher performance.
Importance our joint action-motion representation. In Baseline E, we naively apply our motion anticipation module to Where22Attpreedde motion for comparison with the observed motion. However, it struggles with multi-step interactions. Without motion history, it often results in back-and-fortt movements without progressing to new states. In contrast, our method uses predicted joint motion matrices guide actions consistently in one direction-eithh fully closing fully opening enabling more reliable trajectory comparisons.
actions, yielding modest gains by estimating normal
Interaction efficiency. Next we evaluate by plotting AUROC against interaction steps in Fig. 5 Our method continues improving with more interactions, while other baselines plateau after 2-3 steps. This highlights our pipeline's ability to perform meaningful interactions and effectively reveal hidden anomalies.
Methodological differences with Where2Act [28]. While Where2Act tackles a different problem. we highlight the key differences between our method and its architecture below: 1) Where2Act focuses on predicting which actions are likely to move an articulated part, but it does model how the part moves. In contrast, our method estimates the rigid motion flow that fully characterizes the part's kinematic response to the action. This crucial for our task, as it provides the anticipated motion necessary for comparison with the observed one. 2) Our formulation enables long- horizon, temporally consistent action exploration, which Where2Act lacks. 3) By explicitly estimating motion, our method can filter out noisy actions by identifying inconsistencies in predicted motions. 4) Our model leverages rich supervisory signals from simulation during training, including motion mask and rigid motion matrix tied to each input action, whereas Where2Act uses only discrete supervision (motion vs. no motion). This allows to capture the continuous range of articulation dynamics more accurately.
Additional metrics for motion estimation and interaction results. Table 2 reports additional quantitative metrics to assess two components of our framework. For the motion prior network, we evaluate motion prediction eгror (MSE) and part segmentation accuracy (mloU). For interaction performance, we report action success rate (ASR) and part success rate (PSR). ASR measures the proportion of actions that successfully move part-defined as achieving a displacement greater than 0.01 unit-length or 0.5 relative to its total motion range at any timestep. PSR indicates the percentage of parts that reach fully open or closed state over multiple interaction steps. All evaluations conducted normal objects from the testing categories. We compare two variants of

---

our framework: one with and one without the noise filtering module. Results demonstrate that noise filtering significantly improves motion estimation and boosts interaction success rates.
Real-world visualization of the motion prior network. To evaluate the generalization ability of the learned motion prior, textures of the shapes in this dataset are scanned from real-worid objects. Fig. 6 visualizes our normal motion estimation model (pretrained on the PartNit-Mobility dataset) applied three categories from AKB-48: Bucket, Box, and Trashcan. As shown, our model performs favorably even on shapes with significant geometric differences and realistic textures, without any finetuning. For more results please refer to the supplementary.
real-world articulated objects from the A AKB448 dataset (26]. The
4.4 Limitations and Failure Cases
We assume articulated parts undergo rigid trans- formations, meaning each part moves indepen- dently without affecting others. While this holds for most real-world objects, does not apply to multi-link rigid systems or soft-body deforma- tions. Our framework generalizable to various joint types but the PurtNet-IAD dataset is limited to I-DoF prismatic and revolute joints. Addi- tionally, it does not handle articulation requiring composite actions, such as locking a door (e.g.,
Predicton:AAnmmsy Normal
Prediction: Aredicioon:Anamaly Normal
Figure 7: Failure cases. We show the input RGBD, anticipated motion and the observed motion. The applied action in first timestep is shown in red arrow. We use the same color coding as in Fig.
turning key and pushing latch). Our method uses suction- end-effector for robust grasping across diverse objects, aligning with real-world robotics. However, it is unsuitable for tasks needing precise grasping, handling small objects, or gripping uneven surfaces. While our model generalizes across categories - e.g., learning to open a safe from experience with microwaves due to similar revolute joints it cannot handle entirely novel articulations (e.g., different joint types). Finally, al- though our method is designed with real-world generalization in mind, our experiments are primarily conducted in simulation without using real robot.
Fig. highhights two failure cases due to ambiguity in their joint type and motion direction respectively. The left one is false positive where the model misinterprets normal motion by assuming revolute joint. In the right one, the door is designed to be pushed to open, while our model infers the wrong opening direction (i.e., pulling out) for this ambiguous case, leading inaccurate action and motion predictions.
5 Conclusion and Future Work
In this work, we introduced a novel problem, interactive AD for detecting functional anomalies in articulated objects through learned agent interactions. Unlike existing techniques that either lack interaction or rely on random actions, our approach actively explores the object's 's motion space to uncover hidden anomalies. By integrating learned motion anticipation and action selection, our method achieves significantly higher performance across multiple evaluation metrics. Our findings demonstrate the importance of intelligent interaction in anomaly detection, paving the way for more advanced robotic perception systems capable of understanding complex articulated structures.
Broader Impacts. This work enables robotic agents to actively detect functional anomalies in articulated objects, with potential benefits in quality control, assistive robotics, and autonomous inspection. Those may include furniture (e.g. for cabinets) and consumer electronics (e.g. for laptops) industry for quality control and durability testing. It may also be used in product development and prototyping to identify design flaws and improve functionality, and, well as, in service industry for inspecting and repairing articulated objects, and warranty assessments. While this technology advances safe and intelligent interaction, unintended damage from probing actions or generalization issues in real-world settings may pose safety and reliability concerns.
Acknowledgments. The project was funded by Toyota Motor Europe. HB was supported by the EPSRC Visual AI grant EP/T028572/1.
10

---

References
Aseem Behl, Despoina Paschalidou, Simon Donné, and Andreas Geiger. Pointilownet: Learning represen- tations for rigid motion estimation from point clouds. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7962-7971, 2019.
[2] Paul Bergmann, Michael Fauser, David Sattlegger, and Carsten Steger. Mvtec ad-a comprchensive real-world dataset for unsupervised anomaly detection. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 9592-9600, 2019.
[3] Paul Bergmann, Xin Jin, David Sattlegger, and Carsten Steger. The mvtec 3d-ad dataset for unsupervised 3d anomaly detection and localization. arXiv preprint arXiv:21 12.09045, 2021.
[4] Paul Bergmann and David Sattlegger. Anomaly detection in 3d point clouds using deep geometric descriptors. In Proceedings of the IEEE/CVF Winter Conference on Applications of Compueer Vision, pages 2613-2623, 2023.
[5] Ankan Bhunia, Changjian Li, and Hakan Bilen. Looking 3d: Anomaly detection with 2d-3d alignment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17263- 17272, 2024.
[6] Yevgen Chebotar, Ankur Handa, Viktor Makoviychuk, Miles Macklin, Jan Issac, Nathan Ratliff, and Dieter Fox. Closing the sim-to-real loop: Adapting simulation randomization with real world experience. In 2019 International Conference on Robotics and Automation (ICRA). pages 8973 8979, IEEE, 2019.
[7] Lei Chen, M Tamer Özsu, and Vincent Oria. Robust and fast similarity search for moving object trajectories. In ACM SIGMOD, 2005.
[8] Yang Cong, Junsong Yuan, and Ji Liu. Sparse reconstruction cost for abnormal event detection. In CVPR 201 pages 3449-3456. IEFE, 201
[9] Erwin Coumans and Yunfci Bai. Pybullet, python module for physies simulation for games, robotics and machine learning, 2016.
[10] Jianning Deng, Kartic Subr, and Hakan Bilen. Articulate your nerf: Unsupervised articulated object modeling via conditional view synthesis. Advances Neural Information Processing Systems, 2025.
[11] Ayush Dewan, Tim Caselitz, Gian Diego Tipaldi, and Wolfram Burgard. Rigid scene flow for 3d lidar scans. In 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1765-1770. IEFF, 2016.
[12] Martin Ester, Hans-Peter Kriegel, Jörg Sander, and Xiaowei Xu. Density-based spatial clustering of applications with noise. ACM SIGKDD international conference on Knowledge discovery and data mining, 1996.
[13] Samir Yitzhak Gadre, Kiana Ehsani, and Shuran Song. Act part: Learning interaction strategies for articulated object part discovery. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15752-15727 2021.
[14] Yining Hong, Kaichun Mo, Li Yi, Leonidas J Guibas, Antonio Torralba, Joshua Tenenbaum, and Chuang Gan. Fixing malfunctional objects with learned physical simulation and functional prediction. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pages 1413-1423, 2022.
[15] Chaoqin Huang, Haoyan Guan, Aofan Jiang, Ya Zhang, Michael Spratiigg, and Yan-Feng Wang. Regis- tration based few-shot anomaly detection. European conference on computer vision, pages 303-319. Springer, 2022.
[16] Stephen James, Paul Wohlhart, Mrinal Kalakrishnan, Dmitry Kalashnikov, Alex Irpan, Julian Ibarz, Sergey Levine, Raia Hadsell, and Konstantinos Bousmalis. Sim-lo-real via sim-to-sim: Data-efficient robotic grasping via randomized-to-camoiie adaptation networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages
2019.
[17] Wolfgang Kabsch. A solution for the best rotation to relate two sets of vectors. Acta Crystallographica Section A: Crystal Physics, Diffraction, Theoreticul and General Crystallography, (2(5):922-223, 1976.
[18] Dov Katz, Moslem Kazemi, J Andrew Bagnell, and Anthony Stentz. Interactive segmentation, tracking. and kinematic modeling of unknown 3d articulated objects. In 2013 IEEE International Conference on Robotics and Automation, pages 5003-5000 5010- IEEE, 2013.

---

[19] Alex Kendall, Yarin Gal, and Roberto Cipolla. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In Proceedings of the IEEE conference on computer vision and puttern recognition, pages 7482-7491, 2018.
[20] Chad C. Kessens, Joseph B Rice, Daniel C Smith, Stephen J Biggs, and Richard Garcia. Utilizing compliance to manipulate doors with unmodeled constraints. In 2010 IEEP/RSI International Conference on Intelligent Robots and Systems, pages 483-489. IEEE, 2010.
[21] Ellen Klingbeil, Ashutosh Saxena, and Andrew Ng. Learning to open new doors. In 2010 JEEE/RSJ International Conference on Intelligent Robots and Systems, pages
IEEE, 2010.
[22] Ruining Li. Chuanxia Zheng. Christian Rupprecht, and Andrea Vedaldi. Dragapart: Learning a part-level motion prior for articulated objects. In Eurupean Conference Computer Vision, pages 165-183, Springer, 2024.
[23] Wenqiao Li, Xiaohao Xu, Yao Gu, Bozhong Zheng, Shenghua Gao, and Yingna Wu. Towards scalable 3d anomaly detection and localization: benchmark via 3d anomaly synthesis and self-supervised learning network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 12207-22211 2024.
[24] Yancong Lin and Holger Cacsar. lep-flow; Lidar scene flow estimation with icp. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 15501-15511, 1, 2024.
[25] Jiayi Liu, Ali Mahdavi-Amiri, and Manolis Savva, Paris: Part-level reconstruction and motion analysis for articulated objects. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 352-363 2023.
[26] Liu Liu, Wenqiang Xu, Haoyuan Fu, Sucheng Qian, Qiaojun Yu, Yang Han, and Cewu Lu. Akb-48: A real-worke articulated object knowledge base. In Proceedings the IEEE/CVF Conference Computer Vision and Pattern Recognition. pages 14809-14818 2022.
[27] Martin Lohmann, Jordi Salvador, Aniruddha Kembhavi, and Roozbeh Mottaghi. Learning about objects by learning to interact with them. In Advances in Neural Information Processing Systems, volume 33, pages 3930-3941, 2020.
[28] Kaichun Mo, Leonidas Guihas, Mustafa Mukadam, Abhinav Gupta, and Shubham Tulsiani. Where2act: From pixels to actions for articulated 3d ohjects. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 681 1--68233, 2021.
[29] Chuanruo Ning, Ruihai Wu, Haoran Lu, Kaichun Mo, and Hao Dong. Where2explore: Few-shot affordance learning for unseen novel categories of articulated objects. In Advances in Neural Information Processing Systems, 2023.
[30] Santhosh K Ramakrishnan, Dinesh Jayaraman, and Kristen Grauman. An exploration of embodied visual exploration. International Journal of Computer Vision, 12965)1666--164, 2021.
[31] Kanishka Rao, Chris Harris, Alex Irpan, Sergey Levine, Julian Ibarz, and Mohi Khansari. Rl-cyclegan: Re- inforcement learning aware simulation-to-real. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11157-11166, 2020.
[32] Nicolae-C Ristea, Florinel-AIh Croitoru, Radu Tudor Ionescu, Marius Popescu, Fahad Shahbaz Khan, Mubarak Shah, et al. Self-distilled masked auto-encoders are efficient video anomaly detectors. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15984- 15995, 2024.
[33] Ziyang Song and Bo Yang. Oge: Unsupervised 3d object segmentation from rigid dynamics of point clouds. Advances in Neural Information Processing Systems, volume 35, pages 30798-30811 2022.
[34] Jürgen Sturm, Cyrill Stachniss, and Wolfram Burgard. A probabilistic framework for learning kinematic models of articulated objects. Journal of Artificial Intelligence Research, 41:477-526, 2011.
[35] Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In Computer -CCnn 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part 16, pages 402-419. Springer, 2020.
[36] Ruihai Wu, Yan Zhao, Kaichun Mo, Zizheng Guo, Yian Wang. Tianhao Wu, Qingnan Fan, Xuelin Chen, Leonidas Guibas, and Hao Dong. Vat-mart Learning visual action trajectory proposals for manipulating 3d articulated objects. arXiv preprint arXiv:2106. 14440, 2021.
12

---

[37] Xiaoyang Wu, Li Jiang, Peng-Shuai Wang. Zhijian Liu, Xihui Liu, Yu Qiao. Wanli Ouyang, Tong He. and Hengshuang Zhao. Point transformer v3: Simpler faster stronger. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4840-4851, 2024.
[38] Fanbo Xiang. Yuzhe Qin, Kaichun Mo, Yikuan Xia, Hao Zhu, Fangchen Liu, Minghua Liu, Hanxiao Jiang, Yifu Yuan, Wang, et al. Sapien: A simulated part-based interactive environment. In Proceedings of IEEE/CVF conference on computer vision and pattern recognition, pages 11097-11107, 2020.
[39] Fanbo Xiang, Yuzhe Qin, Kaichun Mo, Yikuan Xia, Hao Zhu, Fangehen Liu, Minghua Liu, Hanxiao Jiang, Yifu Yuan, He Wang. Li Yi, Angel X. Chang. Leonidas J. Guibas, and Hao Su. SAPIEN: A simulated part-based interactive environment. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 2020.
[40] Zhenjia Xu, Zhanpeng Hc, and Shuran Song. Universal manipulation policy network for articulated objects. IEEE robotics and outomation letters, 7(2)2447-2244 2022.
[41] Zhenjia Xu, Jiajun Wu, Andy Zeng, Joshua B Tenenbaum, and Shuran Song. Densephysnet: Learning dense physical object representations via multi-step dynamic interactions. arXiv preprint Xiv: / 1906.99063555 2019.
[42] Yiru Zhao, Bing Deng, Chen Shen, Yao Liu, Hongtao Lu, and Xian Kian-Sheng Hua. Spatio-temporal toencoder for video anomaly detection. In Proceedings of the 25th ACM international conference on Multimedia, pages 1933-1941, 2017.
[43] Jia-Xing Zhong, Ta- Ying Cheng, Yuhang He, Kai Lu, Kaichen Zhou, Andrew Markham, and Niki Trigoni. Multi-body se (3) cquivariance for unsupervised rigid segmentation and motion estimation. In Advances in Neural Information Processing Systems, volume 36, pages 76085-76097, 2023.
[44] Qiang Zhou, Weize Li, I Lihan Jiang, Guoliang Wang, Guyue Zhou, Shanghang Zhang, and Hao Zhao. Pad: A dataset and benehmark for pose-agnostic anomaly detection. In Advances in Neural Information Processing Systems, 2023.
[45] Yi Zhou, Connelly Barnes, Jingwan Lu, Jimei Yang, and Hao Li. On the continuity of rotation represen- tations in neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5745-5753, 2019.
[46] Yang Zou, Jongheon Jeong. Latha Pemula, Dongqing Zhang, and Onkar Dabeer. spot-thc-differenn self- self-supervised pre-training for anomaly detection and segmentation. In European Conference on Computer Vision, pages 392-408, 2022.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Unlike traditional anomaly detection methods that rely on passive visual observations, our proposed anomaly detection task requires agent interaction to detect functional anomalies that would otherwise remain hidden. Experiments shown in Table of our proposed Bart benchmark demonstrate strong generalization in detecting anomalies across both seen and unseen object categories.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Limitations are discussed in Section 4.4.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
14

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
The paper introduces a new benchmark and new model. The dataset is derived from existing public articulated shape dataset and the generation procedure is explained in detail in the main text and supplementary material. Similarly, we provide the required details to construct, train, and test the models.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the partieular way they provide for reproducibiliity In the case of closed-source models, it may be that access to the model is limited in
15

---

some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: We will release the dataset and model, the source code that is used to generate the data and train the models upon publication.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please the NeurlPS code and data submission guidelines (bttps://nips. co/
public/ gwwces/ Code Stoe: ss wtt.:u/mimssssooooooooooooo
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ //nips.co cC/ htce/ublic/guue quidee////// SCodeSubmis ss ieiioooPiiiy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification:
We provide these details under Section 4.2, Appendix A.I and Appendix A.2.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer:
[Yes]
Justification:
We discussed the statistical significance of the experiments in Appendix A.2.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
16

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have provided detailed computing resources for training in Appendix A.2.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: There are no human subjects or participants in this research and there are no data-related concerns either.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We have included paragraph in Section 5 to discuss both positive and negative potential impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
17

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risk.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All the existing assets properly cited and their licenses are explicitly mentioned in Appendix A.I.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
18

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paapersw paperswithcoode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Question:
Answer: [Yes]
Justification: Our new assets including the network training and testing codes, and the new dataset will be released under the AtribatoonKoonCmmmerrai---heeeii 4.0 International license upon publication.
Guidelines:
The answer NA means that the paper does release assets.
Researchers should communicate the details the dataset/code/mooell as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
19

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification:
The core method development this research does not involve LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non non-standard components.
Please refer to our LLM policy (https: https:///euripae htt.ar/iig.ccccooeeeeneeeee////e 2025/ LLM) for what should or should not described.
20


Reviews:
[{"review_id": "uCqUUWxj2G", "reviewer": "Reviewer_osxE", "summary": "The paper presents a benchmark for interactive anomaly detection of articulated objects, using motion anticipation as the basis. Alongside the benchmark, it introduces a learning-based approach to anomaly detection. The method detects anomalies by learning how the object is normally interacted with through a robotic agent. It learns to predict normal functional behaviour through interactions with normal objects. During testing, the method detects anomalies by comparing the observed motion of the object with the model's learned normal behaviour. The approach is evaluated on the proposed benchmark and compared to several baselines using standard evaluation metrics.", "questions": "- How does the proposed method differ from Where2Act, and how directly comparable are the chosen baselines to it?\n- How are object categories for training and testing selected? Are they chosen randomly or according to specific criteria?\n- Have you computed additional evaluation metrics, such as AUPRC or FPR@95% recall? These could help to improve our understanding of the model's performance.\n- Are the failure cases shown in Section 4.4 representative of all typical failures, or have other types of false detection been observed?", "limitations": "yes.", "rating": 2, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "- The paper would be better suited to the benchmark track. The method is not entirely novel. Instead, the main contribution is the benchmark.\n- Limited novelty: the proposed method does not make significant contributions. The idea of using motion modelling for anomaly detection is well established in the literature (e.g. automated driving trajectory-based anomaly prediction; see the Shift dataset). The proposed approach is also quite similar to Where2Act. It should be made clear what the proposed novelties are.\n- Benchmark clarity: The selection of categories for training and testing lacks clarity. While the paper mentions a 'mutually exclusive set', it is unclear whether the categories are chosen randomly or manually, or if they are functionally disjoint (e.g. a closet door versus a regular door). This ambiguity limits the strength of the generalisation claims. It would be valuable to conduct an ablation study analysing how training category selection affects performance.\n- Benchmark clarity: The paper provides limited information on how anomalies are generated in the dataset. For example, the anomaly shown in Figure 1 appears somewhat unrealistic, which could reduce the relevance of the benchmark.\n- More evaluation metrics: The only metric reported for anomaly detection performance is AUROC. However, this metric is insensitive to class imbalance, which is usually the case in anomaly detection since, by definition, anomalies are rare. This can lead to deceptively high AUROC scores even when the model performs poorly on the rare class (i.e. anomalies). To get a more complete picture of anomaly detection performance, benchmarks typically require precision-recall area under the curve (AUPRC) and false positive rate at 95% recall to be reported.\n- Method clarity: The method description lacks clarity and uses inconsistent terminology. For example, line 125 refers to 'decoders' for motion and segmentation, whereas later sections refer to 'heads'. Maintaining consistent terminology would improve readability.\n- The paper appears somewhat unpolished in places. For example, the abstract could be improved by providing a brief explanation of the broader context and potential applications of interactive anomaly detection to motivate the problem. A condensed version of the first paragraph of the introduction could achieve this.\n\n\nMinor points: \n- Section 4.4 (Limitations and Failure Cases) would benefit from a more detailed analysis of failure modes, including potential causes of false positives/negatives, as well as suggestions for how these could be mitigated in future work.\n- The term 'online data sampling' is introduced in Section 4.2 without explanation. Explaining what this entails and why it is beneficial would improve reproducibility.", "was_revised": true, "final_justification": "After reading the rebuttal,, I found all points valid and could follow them. However, I am fundamentally against accepting the dataset paper as a NeurIPS paper. This could be submitted to the dataset track of the conference or to another robotics venue, e.g. IROS/ICRA. There is no new theory in this work. I will maintain my 'Reject' score.", "rebuttal": "Thank you for reviewing our work.  Below we address your questions and weaknesses mentioned:\n\n> **1. Concerns regarding novelty**\n\nThank you for the feedback. We would like to clarify the novel contributions of our method compared to Where2Act [R1] as follows: \n1) Where2Act focuses on predicting *which* actions are likely to move an articulated part, but it does not model *how* the part moves. In contrast, our method estimates the rigid motion flow that fully characterizes the part’s kinematic response to the action. This is crucial for our interactive anomaly detection (IAD) task, as it provides the anticipated ‘normal’ motion necessary for comparison with the observed motion. \n2) Our formulation enables long-horizon, temporally consistent action exploration, which Where2Act lacks.  \n3) By explicitly estimating motion, our method can filter out noisy actions by identifying inconsistencies in predicted motions.\n4) Our model leverages rich supervisory signals from simulation during training, including motion mask and rigid motion matrix tied to each input action, whereas Where2Act uses only *discrete* supervision (motion vs. no motion). Our richer supervision allows to capture the *continuous* range of articulation dynamics more accurately. \n\nComparison with autonomous driving trajectory anomaly detection: In autonomous driving, anomaly detection is typically based on passive observation of agents (vehicles or pedestrians) moving along structured paths governed by traffic rules. These methods rely on detecting spatial deviations in smooth trajectories. In contrast, our task involves an embodied agent actively manipulating articulated objects, where anomalies arise from deviations in expected part dynamics (e.g., stuck or broken joints). This demands modeling of contact-driven motion, causality, and part-specific behavior. We will discuss about the related papers in the final draft.\n\nWe would like to highlight that other reviewers have acknowledged the novelty of our work. For instance, Reviewer dCsY acknowledged our joint modeling of interaction and motion anticipation as a novel approach, Reviewer bQ9u remarked that our proposed method is well-posed and clearly articulated, and both reviewers recognized the novelty and practical importance of the IAD task itself.\n\n[R1] Where2Act: From Pixels to Actions for Articulated 3D Objects, ICCV'21\n\n> **2. Selection of training categories**\n\nOur chosen category split follows the same protocol as in [R2]. As suggested, we also conduct experiments with different combinations of training categories. While selecting these combinations, we ensure that each combination contains a sufficient number of articulated objects covering representative articulation types, such as both revolute and prismatic joints. The StorageFurniture category is included in all sets as it is the most representative category in the PartNet-Mobility dataset. The test categories remain consistent across all experiments. As shown in the results below, our choice of training categories has a relatively low impact on overall performance, although some test category might benefit from the presence of functionally similar category in the training set (e.g., Microwave and Safe as in SetA). Note, while the overall performance is slightly lower than in Table 1, this is expected since training was conducted on a smaller subset of the data.\n\n|      | Training categories                              | Interaction pairs (offline) | Box      | Phone    | Dishwasher | Safe     | Oven     | WashingMachine | Table    | KitchenPot | Bucket   | Door     |\n|------|--------------------------------------------------|-----------------------------|----------|----------|------------|----------|----------|-----------------|----------|------------|----------|----------|\n| SetA | {StorageFurniture, Microwave, Switch, stapler}   | 77M                         | 69.1     | 88.3     | 79.8       | **84.1** | 79.0     | 84.5            | 88.1     | 75.7       | 72.3     | **73.9** |\n| SetB | {StorageFurniture, Refrigerator, Laptop, Window} | 90M                         | 71.4     | 91.5     | **82.4**   | 81.1     | **81.6** | 85.8            | **88.9** | 80.5       | 73.4     | 72.1     |\n| SetC | {StorageFurniture, TrashCan, Toilet, Kettle}     | 81M                         | **73.7** | **92.6** | 80.2       | 81.0     | 80.3     | **86.1**        | 88.4     | **87.6**   | **79.3** | 72.3     |\n\n[R2] UMPNet: Universal Manipulation Policy Network for Articulated Objects, ICRA'22\n\n> **3. Anomaly generation details**\n\nOur dataset contains a broad range of plausible functional anomalies covering types such as: tilted axis of motion, wrong joint type, restricted motion etc. A detailed description of the anomaly generation process is provided in Appendix A.1. Currently, the dataset focuses on 1-DoF prismatic and revolute joints due to limitations in publicly available articulation datasets, such as PartNet-Mobility, which includes only these joint types. However, our methodology uses a general motion parameterization that is agnostic to joint type and can be extended to more complex articulations.\n\n> **4. Additional metrics**\n\nOur evaluation set is relatively balanced (see Table A1 in appendix). As suggested, we also report AUPRC and FPR@95% recall on the test categories in the table below.\n\n| Method                  | AUPRC (↑) | FPR@95% (↓) |\n|-------------------------|-----------|-------------|\n| Where2Act + MotionPrior | 74.4      | 14.2        |\n| Our Method              | **81.9**      | **9.5**         |\n\n\n> **5. On clarity and inconsistent terminology**\n\nThanks for your feedback. We will fix the inconsistencies in the final draft.\n\n> **6. Analysis of failure cases**\n\nIn addition to the analysis in Section 4.4, Figure A.3 (appendix) presents more examples of representative failure cases. The primary cause of failure is due to inaccurate motion prediction that manifests in several ways, such as misclassifying the joint type (e.g., predicting rotation instead of translation), defining an incorrect axis of motion, or generating physically implausible movements that ignore part collisions. Please see Section A.3 (appendix) for more discussion. \n\n> **7. Explanation of Online data sampling**\n\nWhile offline data generation using random actions can partially explore the action space, it is often inefficient due to the large size of the action space. Focusing on actionable interactions (i.e., actions likely to induce motion) yields more informative training data. Online data sampling provides an efficient way of collecting actionable interaction data by leveraging the knowledge already acquired by the network. Specifically, we sample actions using the normalized movability score distribution in Eq. (8) and simulate additional interactions for those high-likelihood actions. We skip noise filtering in this step to sample actions faster, since this process is not sensitive to outliers.\nThis online data collection strategy significantly increases the proportion of interaction pairs that induce positive motion."}, {"review_id": "xhLbTGSc3H", "reviewer": "Reviewer_bQ9u", "summary": "The authors propose a novel task, namely *Interactive Anomaly Detection* (IAD) that aims at classifying an articulated object as anomalous in case of functional probelms. This represent a step further respect to images AD, and can be useful in several industrial scenarios.\n\nThe authors also contribute a novel method for IAD, in which they compare possible movments on the  articulated object movable parts with the same actions as performed on a functioning *replica* of the object inspected.\n\nThis work presents a novel dataset (even though it originates from a pre-existing one) that is used for benchmarking their proposed method against five baselines reporting best results.", "questions": "See weaknesses", "limitations": "The main limitations of this work is in its extendability. \nIt seems very challenging to construct a dataset similar to the one the author propose and this could limit the interest in IAD for the CV community.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "STRENGHTS\nThe paper is well written and is full of details.\nThe proposed task seems valuable and novel.\nThe method proposed, while being directly inherited from classical Anomaly Detection techniques, is well posed and articulated.\nThe proposed baselines are not very challenging, but they are informative when compared with the proposed method.\nThe integration of CV methods, robotics and simulated environments is interesting\n\nWEAKNESSES\nWhen proposing a novel task, the motivations are very important. While the authors provide some motivations, I think that the importance of the proposed IAD task could be expressed more clearly. For example, I imagine that IAD could be very interesting for industries, that could benefit from inspecting virtual copies of their produced goods.\nThe benchmark is limited to a selection of objects. While I acknowledge that this paper represents only the first work in IAD, I also think that it would be challenging to extend it to real-world objects (virtual copies of industrial objects) to be valuable for the industries.\nIt is not very clear to me how the treshold on the anomaly score is calculated, as it could change from object to object.", "was_revised": true, "final_justification": "The authors addressed all my concerns. \n\nRegarding the motivation of the IAD benchmark and its extendability, the authors have committed to adding a discussion about the former in the final version of their paper and described possible ways to extend the proposed benchmark in simulated environments. \n\nRegarding the metric, the authors kindly (but rigorously) clarified the properties of the used metric (correcting a mistake I made in my review) and experimented with alternative metrics when addressing reviewers osxE's concerns.\n\nIn light of this, I will confirm my initial positive score for this work.", "rebuttal": "We appreciate your valuable feedback and recognition of our work's novelty. We address your points below.\n\n> **1. Practical importance of the proposed IAD task**\n\nThank you for your feedback. We agree that our proposed IAD task holds significant value for industries, which could benefit from inspecting digital twins of their manufactured products with movable parts. Those may include furniture (e.g., for cabinets) and consumer electronics (e.g., for laptops) industry for quality control and durability testing. It may also be used in product development and prototyping to identify design flaws and improve functionality, and, as well as, in service industry for inspecting and repairing articulated objects, and warranty assessments. We will add a discussion of this in the final draft.\n\n> **2. Extendibility to real-world industry setting**\n\nWe would like to clarify that our method does not require anomaly samples during training, which significantly reduces the effort of dataset construction and improves scalability.\n\nRegarding extension to industrial settings, we note that many industries already maintain digital twins of their products as part of modern manufacturing pipelines. These digital twins can be used within simulation environments to generate rich interaction data. Recent advances in digital twin modeling and availability of advanced simulation tools (i.e., NVIDIA Omniverse) further support the generation of realistic, industry-standard datasets for IAD. \n\nFurthermore, imperfections from real-world scans or digitization artifacts can be explicitly modeled during simulation, enhancing our model's robustness. Our method is general and designed to accommodate such variations. Future work can further incorporate additional physical properties such as friction to increase realism. \n\n>**3. On threshold anomaly score**\n\nFollowing standard practice in AD benchmarks, our evaluation uses AUROC metric which is threshold-independent and obtained by varying over the different threshold values. A higher AUROC score means superior separability between normal and anomaly samples, i.e., stronger AD capability. In a practical setup, a fixed threshold is typically chosen based on the desired tolerance for false positives. This can be determined by calibrating on normal data to achieve an acceptable false positive rate (FPR)."}, {"review_id": "GGB6t6nIVr", "reviewer": "Reviewer_dCsY", "summary": "In this work, the authors propose to tackle interactive anomaly detection (IAD), that is, the detection of anomalies that occur only after interacting with the object (e.g., opening and closing a drawer). The authors propose a new benchmark and approach to interact and detect this kind of anomaly. Specifically, the proposed approach generates a set of actions to interact with the object and predict its motion. Suppose the predicted motion is different to a certain degree from the actual motion, and the model will detect the anomaly. The authors define different baselines to compare with.", "questions": "1. Please provide more detail on how sensitive the method is to the loss weight hyperparameters (e.g., different \\lambdas in Eq. 6). A sweep or sensitivity analysis would be valuable.\n\n2. How robust is the approach to noisy or partial depth observations, especially since it relies on point clouds?\n\n3. How critical is the use of the long-horizon motion estimation compared to atomic action steps?", "limitations": "Limitations have been discussed.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "### Strengths\n\n- The proposed formulation of IAD is original and well-motivated. Most existing anomaly detection benchmarks focus on passive visual cues. In contrast, this work emphasizes that many real-world anomalies only present themselves through physical interaction, a critical insight for robotics and quality assurance in manufacturing.\n\n- The proposed method introduces a joint modeling of interaction and motion anticipation. While each component (e.g., motion prediction, action sampling) builds on prior work, their combination into an end-to-end interactive anomaly detection pipeline is novel. The idea of detecting anomalies through discrepancies between expected and observed motion is both intuitive and effective.\n\n- The paper is clear, logically organized, and provides strong qualitative and quantitative results\n\n- The PartNet-IAD benchmark fills a gap in the community by offering a dataset for interactive anomaly detection. The design of the anomalies and interaction protocols is realistic, and the evaluation across both seen and unseen categories demonstrates the model’s generalizability.\n\n- The proposed approach achieves substantial performance gains (e.g., AUROC improvements of ~6 points over the strongest baseline), suggesting its practical utility.\n\n### Weaknesses\n\n- While the core idea is strong, the experimental section could benefit from more thorough ablation studies. For instance, the paper briefly mentions the use of confidence-based weighting (Eq. 6) and a multi-step filtering strategy for action selection. However, the impact of these design choices is not isolated and quantified. A study showing the effect of varying \\lambda values in the loss would strengthen the empirical analysis.\n\n- While the baselines are well-designed, most rely on Where2Act, and it is unclear how much performance gain comes from better interaction versus better anomaly scoring. A more comprehensive comparative analysis with a broader range of interaction strategies would be beneficial.", "was_revised": true, "final_justification": "The authors provided thorough and convincing responses to all concerns. The ablations reported in the response clarify the impact of loss weighting and noise filtering. They justify the joint modeling of interaction and motion, discuss robustness to noisy inputs and the necessity of long-horizon motion estimation. The rebuttal strengthens the paper’s contributions, and the accept rating remains appropriate.", "rebuttal": "Thank you for your positive feedback. We address your questions and concerns below.\n\n>**1. Ablation for confidence-based loss and noise filtering**\n\nTable 2 (main paper) quantitatively and Figure A2 (appendix) qualitatively show the impact of multi-step noise filtering on the motion estimation performance. As further suggested, we also conduct ablation studies on the test categories to assess the individual contributions of the confidence-based loss and noise filtering strategy to our IAD task, using the AUROC metric. \n\n| Method | AUROC |\n|---|---|\n| w/o filtering | 81.4 |\n| w/ conf-based noise filtering | 82.7 |\n| w/ conf-based + DBSCAN-based noise filtering (our method) | **83.9** |\n\n>**2. Ablation for loss weights**\n\nThe weights $\\lambda_M$ and $\\lambda_e$ in Eq. (6) are relatively insensitive and are therefore fixed to 1 across all experiments. We tune $\\lambda_c$, the loss weight for the confidence loss, over the range [0.005, 1]. We find that a high value of $\\lambda_c$ (e.g., close to 1) can cause training instability and slow convergence by dominating the primary motion loss, whereas a very low value leads to uniformly low confidence predictions across inputs. We set it to 0.05 for optimal performance. We will add discussion in the final draft. \n\n>**3. It is unclear how much performance gain comes from better interaction versus better anomaly scoring**\n\nOur method consists of three integral steps: (1) learning an interaction (action), (2) motion modeling, and (3) anomaly scoring.  Each step plays a critical role in the overall performance.\n\nA good interaction agent can be helpful as it systematically explores the full articulation range of a part. Since functional anomalies are typically not visible without motion, the full articulation range of all parts needs to be thoroughly explored and tested to detect such anomalies. \n\nWhile Where2Act [R1] offers a simple yet effective affordance-based interaction framework, it falls short in our specific IAD task due to its lack of motion-aware long-term planning. This limitation restricts its ability to fully explore articulation ranges, which is crucial for anomaly detection.\n\nLearning an accurate motion estimator is equally important for our task, as it directly drives the final anomaly prediction by comparing against observed motion. Moreover, our motion estimator also facilitates long-term planning, enabling the selection of more informative future actions that lead to better exploration of the part articulations. In summary, our joint modeling of both action and motion is mutually reinforcing—better interaction policies lead to more informative motions, while accurate motion modeling supports more effective action planning. \n\nThe final step, our anomaly scoring as defined in Eq. (9), is straightforward and performs well provided the anticipated normal motion is accurate.\n\nWe also note that while more advanced interaction models may exist, Where2Act integrates seamlessly with our motion estimator, making it the most suitable baseline for our framework. Incorporating alternative models would require significant modifications to our motion model beyond the scope of this work.\n\n\n>**4. Impact of noisy or partial depth observations**\n\nOur method inherently works well with partial point clouds as we train the model using depth observation from single viewpoint.  Figure A1 (appendix) shows motion estimation results on real-world scanned articulated objects showing it can handle noisy input to a certain extent. To improve robustness, we can inject artificial noise to model real-world artifacts in the training depth data. \n\n>**5. How critical is the use of the long-horizon motion estimation compared to atomic action steps?**\n\nWithout long-horizon motion estimation, our model struggles with multi-step interactions. The lack of motion history often results in back-and-forth movements without progressing to new states. As shown in Table 1, our method outperforms baseline E, which uses atomic action steps with Where2Act [R1] as the interaction agent.\n\n[R1] Where2Act: From Pixels to Actions for Articulated 3D Objects, ICCV'21"}]

Author rebuttal:

Meta-review: Although all three reviewers acknowledge the technical novelty and experimental details, reviewer osxE recommends rejection, suggesting that the paper is more suitable for IROS/ICRA or the benchmark track.
Decision: Accept (poster)