FlowBotHD: History-Aware Diffuser Handling Ambiguities in Articulated Objects Manipulation

Authors: yishu  li
wen hui leng
yiming  fang
ben  eisner
david  held
Venue: robot-learning_org_CoRL_2024_Conference
Published: 2024-06-05 16:26:34

Abstract: We introduce a novel approach to manipulate articulated objects with ambiguities, such as opening a door, in which multi-modality and occlusions create ambiguities about the opening side and direction. Multi-modality occurs when the method to open a fully closed door (push, pull, slide) is uncertain, or the side from which it should be opened is uncertain.  Occlusions further obscure the door’s shape from certain angles, creating further ambiguities during the occlusion. To tackle these challenges, we propose a history-aware diffusion network that models the multi-modal distribution of the articulated object and uses history to disambiguate actions and make stable predictions under occlusions. Experiments and analysis demonstrate the state-of-art performance of our method and specifically improvements in ambiguity-caused failure modes. Our project website is available at https://flowbothd.github.io/.

Main paper: 
 introduction: 
 Ambiguities are common in manipulation tasks. Consider opening a door without a visible handle or hinges: when the door is fully closed, the way in which it opens may be totally ambiguous.It may swing open from the left side or the right side, and may swing either inwards or outwards. Without further information, even experienced humans can't always tell by sight whether it opens by push, pull or even slide. Another type of ambiguity is caused by occlusions. When a door opens to certain angles such that the door's principal plane is parallel to the viewing angle, it might look like a vertical line instead of a flat panel (see Figure 3 ). Occlusions cause loss of information, introducing ambiguities in the understanding of the object shape.Faced with object ambiguities, a human might interact with the object and observe the outcome to disambiguate the true mode; for example, a person might push the door and see if it opens, and if not, they might try pulling the door open. Furthermore, momentary occlusions are not typically a problem for humans: we can remember past observations to supplement our current perception. We therefore would expect an intelligent robot to have the ability to learn from multi-modal training data, predict multi-modal solutions for ambiguous observations, and with the help of historical observations, make stable predictions even during occlusions.To generalize across different articulated object kinematics and geometries, we build on prior work that models articulated object manipulation with 3D articulation flow (3DAF) vectors as proposed in Eisner et al. [1] . This approach predicts a set of 3D vectors for each point on the articulated object that indicates its opening direction. Results from this approach have shown impressive abilities to generalize to novel objects in the real world, after training only in simulation. However, since this prediction is unimodal and deterministic, this approach often fails to predict the correct flow when facing the ambiguous settings described above. Therefore, in this paper, we propose a history-aware diffusion method to model multi-modality and handle occlusions for articulated objects. We use diffusion to model multi-modal distributions over multiple possible object opening directions, and we use historical observations to adjust or disambiguate the predicted distribution accordingly. Our experiments and analysis demonstrate the benefits of our method in such ambiguous situations. The contributions of this paper include:(1) A novel approach to model multi-modal ambiguity of articulated objects caused by visual ambiguity or occlusion. (2) A novel structure for incorporating object history to disambiguate articulation modes.(3) Simulated experiments across a wide variety of PartNet-Mobility objects. (4) Real-world experiments that demonstrate the feasibility of our method in real-world. 
related work: 
 Articulated Object Manipulation: Manipulating articulated objects is an important and challenging task. One line of work investigates building manipulation systems with analytical methods [2, 3, 4, 5, 6] . With the help of large-scale datasets [7, 8] , recent lines of work turn to learningbased methods. Several works model manipulation choices with explicit articulation parameters and poses [9, 10, 11, 12] . To improve the model's generalization ability to different kinematics, several recent works learn to predict geometry-aware kinematics-free representations such as visual affordances [7, 13, 14] and articulation flows [1, 15] which achieve significant improvements in object generalization. Different from previous learning-based works which tend to use regression models for prediction, we propose to use diffusion models for better handling of multi-modality.Uncertainty Modeling in Robotics: Modeling uncertainty is a well-explored topic in various robotics fields [16, 17] . Previous works have included uncertainty in tasks including visual perception and tracking [18, 19, 20, 21] , motion planning [22, 23, 24, 25] , safety control [26] and also articulation parameter estimation [27] . With similar motivation, our work focuses on building a model robust to visual uncertainty caused by multi-modality and occlusions.Diffusion Models in Robotics: Diffusion models, a class of generative models, learns to denoise an input to approach a given distribution. Following its impressive success in image generation, diffusion has also shown great potential in robotics tasks. Recent works have applied diffusion models to various tasks of robotics, including reinforcement learning [28, 29] , imitation learning [30, 31, 32, 33, 34] and motion planning [35, 36, 37] . In contrast to prior methods, we show how 3D diffusion models can be used to handle ambiguities in articulated object manipulation.3 Background: 3D Articulation Flow (3DAF)Our method builds upon the 3D Articulation Flow (3DAF) representation of Eisner et al. [1] , which we briefly review here. We assume that the parts of the articulated object are connected in a kinematic tree with no closed-loop kinematic chains, and also that each joint has only one degree of freedom. Suppose that an articulated object is described by a point cloud P = {p 0 , . . . , p N -1 } for a set of 3D points p i ∈ R 3 on the object visible surface. Following Eisner et al. [1] , for each point p i , we define the 3D Articulation Flow (3DAF) f i ∈ R 3 as the motion that point would undergo if there were an infinitesimal positive displacement δθ of the joint between that point's articulation part and its parent. Eisner et al. [1] showed that, given a suction gripper, we can most efficiently open the articulated object using the predicted 3DAF by attaching the gripper to the point p imax with the largest predicted flow norm i max = arg max i ∥f i ∥ and then moving the gripper along its predicted flow direction. The set of 3D Articulation Flows F = {f 0 , . . . , f N -1 } for all points in the point cloud is hierarchy-free and invariant under object translation and scaling, which allows for effective learning over a large set of objects of different categories and effective transfer from simulation to the real world. 
method: 
 The task we propose to tackle is opening fully-closed articulated objects from a single-view depthsensor. The inputs to our model are a history of 3D point cloud observations of the articulated object being manipulated. We assume that the articulated objects do not consist of a closed-loop kinematic chain and that each joint has only a single degree of freedom.We aim to enable a robot to manipulate ambiguous articulated objects, focusing on two types of ambiguity: ambiguity in opening direction due to lack of visual cues, and ambiguity due to selfocclusion during articulation. Both types of ambiguity are known failure modes of prior deterministic systems [1] . At a high level, our objective is to predict all the modes that describe how a potentially-ambiguous articulated object might open, and then to use these predictions to determine how to manipulate the object. Below we will describe our proposed history-aware diffusion model which can represent multimodality (Section 4.1) and is robust to occlusion (Section 4.2), as well as a system for manipulating articulated objects based on multimodal predictions (Section 4.3).Our system builds on the 3D Articulation Flow representation from prior work [1] described in Section 3. Our first task is to construct a system which can accurately predict multi-modal affordances for an articulated object, in which affordances are captured by 3D Articulation Flow. For instance, given an ambiguous door that can open right or left (as shown in Figure 4 ), we'd like to be able to generate one set of flow vectors to describe the possibility that the door opens rightward, and a different set of flow vectors to capture the possibility that it might open leftward. Diffusion processes have been shown to be a good choice of model architecture for this setting, as they can represent multi-modal distributions over high-dimensional data [38, 30] .We now describe how we can use diffusion to predict multi-modal 3D Articulation flow. To incorporate observed 3D point cloud inputs P obs into the standard diffusion model paradigm, we use a PointNet++ [39] encoder g ϕ that encodes the current inputs and a DiffusionTransformer (DiT) [40] denoiser D θ . During training, we first add noise to the ground truth flow F * obs to obtain a noisy flow F ϵ obs , following the DDPM [38] diffusion formulation. Then we pair each point and its corresponding noisy flow as the current stateS cur = {p obs,i , f ϵ obs,i }, i ∈ [1 . . . N ],where N is the total number of observed points in P obs . We encode the current state S cur as Ψ obs = g ϕ (S cur ) to obtain 3D geometry-aware point-wise features. Following the standard diffusion training procedure [38] , History-aware State Embedding X X XFigure 2 : FlowBotHD structure: a history observation and the history flow are input into a history encoder to obtain a global history latent. This history latent is injected into the current observation encoder through fully-connected projection layers and element-wise multiplication. The current observation encoder outputs a history-aware point-wise embedding. The history-aware embedding is then input to a DiT-based denoiser to predict the 3D articulation flow [1] , i.e. the predicted future motion of the object if it were to be opened a small amount.we train the denoiser D θ and encoder g ϕ with the following learning objective:J (θ, ϕ) = E[||ϵ t -D θ ( √ ᾱt x 0 + √ 1 -ᾱt ϵ t , t, g ϕ (S cur ))|| 2 2 ]From the predicted noise, we estimate the flow F following the standard diffusion equations [38] .While this proposed 3DAF diffusion model is capable of modeling multi-modal distributions, it lacks the ability to disambiguate based on past interactions. For example, suppose the door is occluded in a current observation, as shown in Figure 3 ; if the robot succeeded in the past timesteps to partially open the door, it should output a distribution that places more probability on a flow which agrees with the previously observed motion.To accomplish this objective, we propose to modify the above architecture to integrate the past history with a history encoder. We define a compact history state representation as the previous time step's point cloud observation P his , along with the previous step's flow F his , asS his = {p his,i , f his,i }, i ∈ [1 . . . M ],where M is the number of points in the previous point cloud. We then pass S his though a global point cloud encoder h γ based on PointNet++ [39] to get a global history context vector z his = h γ (S his ) ∈ R dhis , where d his is the fixed dimension of the history latent. We then inject this additional context into the "decoding" layers of our U-net encoder g ϕ , similar to the method proposed by Shridhar et al. [41] (see Figure 2 ). For each decoding layer j of g ϕ with pointwise activations a j ∈ R dj , we apply a learned linear projection W j ∈ R dhis×dj to the history latent z his , and combine with the original activation using the Hadamard product: a ′ j = a j ⊙ (W j • z his ). During training, we generate a synthetic history by sampling a random small angle to perturb the object to generate the "history" point cloud observation, and we obtain the corresponding ground truth articulation flow as the history flow. At inference time, we use the model's previous point cloud observations and its previous flow predictions as history. To account for situations at inference time when no history yet exists (i.e. the first timestep), we also train with samples that include no history, and include a learnable "start" vector z 0 in place of the history vector z his in such cases. We train this model end-to-end with the standard diffusion objective outlined in Section 4.1.Along with the history-aware diffusion model, we propose a novel articulation manipulation policy that utilizes the flow predicting model in real manipulation tasks, taking into consideration the multimodality of diffusion and the incorporation of object history.Switch Grasp Point (SG): Following Eisner et al. [1] , to achieve the best manipulation efficiency with a suction gripper, we want to grasp at the point with the greatest leverage; thus the optimal action given the predicted flow is to grasp the point that has the maximum flow norm [1] . Due to the multi-modality of diffusion, the prediction at different steps will vary; each prediction might imply a different point that the robot should grasp for maximum leverage. In order to correct for past potential grasping mistakes as well as to maintain a reasonable re-grasping cost, we propose to switch the grasp point whenever the leverage (defined as the current largest flow norm) of the newly predicted grasp point is larger than that of the current grasp point by a threshold:|| Fmax ||-|| Fg || > ϵ l, where F is the current step's flow prediction for all points in the point cloud, g is the index of the point that is closest to the current grasp point, and ϵ l is the threshold which we set to 0.2 in practice.Consistency Check (CC): Learned diffusion models may not always perfectly represent the true distribution over history-conditioned 3DAF values; for example, they might predict multiple modes even when there is unambiguously only a single mode. To handle this, we perform a consistency check for every prediction after the first. Assuming that our model will sample the true mode given enough samples for any observation, we design a heuristic consistency check to filter the actions that are not consistent with the previously executed motion. Specifically, we define the consistency direction g his as the measured gripper motion caused by executing previous selected flow fhis (we only keep the previous execution if its motion was greater than 0). For a subsequent step to be considered "consistent", the predicted action's direction shouldn't deviate too much from that of the previous step; thus we set an angle threshold ϵ θ for the consistency check:arccos fmax•ghis || fmax||•||ghis|| < ϵ θ ,where fmax is the current maximum sampled flow direction fmax = max i fi . We sample 3DAF flows from the diffusion model and compute maximum-leverage actions [1] until we find one that satisfies the consistency check, before executing the sampled action.History Filter (HF): If we rely solely on a consistency check for next actions, the model may end up in a degenerate state where its previous predictions are incorrect, enforcing that only incorrect predictions are permitted by consistency. Therefore, we include a history filter that only updates the history point cloud P his and history flow F his if the previous action successfully achieves reasonable gripper motion without losing contact. Specifically, after an execution, we only update the history if two criteria are satisfied:1. The measured gripper motion g cur caused by the currently-executed action is consistent with past executions: arccos gcur•ghis ||gcur||•||ghis|| < ϵ c = 90 • where g cur is the current gripper movement and g his is the gripper movement from the previous execution. 2. The gripper motion g cur is larger than a threshold ||g cur || > ϵ m where g cur is the current gripper movement and ϵ m is the thresholdThe pseudocode for our method, FlowBotHD (History and Diffusion) can be found in Algorithm 1. 
experiments and analysis: 
 We use the PartNet-Mobility dataset [7] to train and evaluate the models. We split a subset of this dataset containing 21 categories of objects (the same categories as in prior work [ To evaluate each model's ability to open objects, we construct a simulation environment in the Py-Bullet [42] simulator consisting of a suction gripper and objects from the PartNet-Mobility dataset, where the environment's goal is to successfully articulate the object. We initialize every object as fully closed and filter out instances that cannot open from the fully closed state even with the ground truth action sequence (e.g. due to self-collision). Following prior work [1] , we compute 2 evaluation metrics: Success Rate: the percent of objects the model succeeds to open > 90% of the joint's full range of motion, and Normalized Distance [13] : the mean distance between the end state of this trial and the fully opened state, normalized with its range of motion d goal = ||jend-jclosed|| ||jopened-jclosed|| where j end is the joint angle at the end of the policy rollout, j closed and j opened are the fully closed angle and the fully opened angle respectively. We evaluate each test object for 5 trials.Baselines: To compare our model with the original FlowBot3D [1] setting, we trained FlowBot3D on a dataset with only randomly opened ("RO") samples which aligns with the original training strategy of Eisner et al. [1] . We also retrain Flowbot on the mixed dataset ("MD") that we use to train our model in which we also include examples in which the object is fully closed. Further, for a fair comparison to our method, we implement a baseline in which we allow FlowBot to switch its grasp point ("SG") using the same criteria as for our method.Comparing the Baseline models with Ours in Tables 1 and 3 (see Appendix for normalized distance metrics), we can see that our full method (FlowBotHD) with the complete set of policy improvements (switch grasp point + consistency check + history filter) achieves the best overall performance. Specifically, compared to the original FlowBot3D [1] model, it increases the success rate averaged across all samples from 87% to 95%, or a 61% reduction in error rate (from 13% to 5%). We observe a particular increase in success rate in the door category (from 23% to 77%), which contains the most visual ambiguities of any category when instances are fully closed. These quantitative results demonstrate our model's ability to overall improve the performance and more importantly, to better handle multi-modal cases. Table 1 : Success Rate Metric Results (↑): Fraction of successful trials (normalized distance < 0.1) of different object categories after a full rollout across different methods; the higher the better. AVG c and AVG s refers to metric averaged over categories or samples. For the policy column, SG refers to switch grasp point, CC refers to consistency check and HF refers to history filter. We only add history filter in history-aware methods and only add consistency check with diffusion-based methods because inconsistent predictions in deterministic methods can't be refined even with re-predictions. against the open ratio. The baseline (top) fails at the point of severe occlusion, whereas our method (bottom) continues to make stable predictions using history. We also include flow visualizations (right) to show the quality of the predictions in both cases.To understand the impact of our design decisions, we perfomed ablation studies in which we remove various components of the model: No history, no diffusion, and varying whether we use switch grasping ("SG" column), consistency check ("CC" column), and history filtering ("HF" column). Looking at Tables 1 and 3 , we observe large performance improvements when including history awareness in the diffusion model (comparing Ours to "Ours-No History"). This improvement demonstrates that the extra guidance of history is beneficial for diffusion to reduce the multimodality in cases of past success in which the opening direction is now disambiguated. These tables show that our policy improvements of switching grasp point, applying a consistency check, and applying a history filter all contribute to increased model performance. The importance of the consistency check potentially implies that one challenge for using diffusion models for visual manipulation policies is randomness and, as a result, inconsistency. By filtering out inconsistent predictions, our policy takes actions that lead to higher success rates than without filtering.To understand the ability of our model to handle occlusions which are not immediately apparent from our quantitative metrics, in Figure 3 , we visualize the model predictions for different door opening angles. At certain angles, the point cloud is severely occluded such that only the vertical edge of the door is visible from the camera's perspective. We can see from the RMSE plot that FlowBot3D consistently mispredicts the 3DAF on examples where the opening angle causes severe occlusions. FlowBotHD performs substantially better in these occluded cases by leveraging history, predicting the correct direction even at the point of occlusion. The flow visualization shows a clear improvement between our model and the baseline for this ambiguous case.We performed real-world experiments on general articulated objects, as well as a custom made multi-modal door to demonstrate FlowBotHD's ability to function in real-world ambiguous scenarios (Figures 1 and 4 ). For the multi-modal door, we demonstrate our model's ability to open the door in all 4 possible modes, and also the ability to adjust to better grasp points with the switch grasp point policy. Static visualizations of the real-world roll-outs on our custom-made door and general articulated objects are presented in Figure 4 and Appendix Figure 15 , and full videos can be found on our project website. Details of our real-world setup can be found in Appendix F. 
conclusion: 
 In this work, we propose a history-aware diffusion model FlowBotHD to handle ambiguities encountered while manipulating articulated objects, along with a carefully designed heuristic manipulation policy. Both quantitative experiments and qualitative analysis show the overall effectiveness of our model and specific improvements in multi-modal or severely occluded cases.Limitations: While our model greatly improves the ability of the policy to handle ambiguities, we have observed trade-offs in accuracy and consistency when building on top of a diffusion pipeline. Our diffusion model sometimes makes multimodal predictions even when the object is opened enough to disambiguate the true opening direction. While we currently mitigate this issue by adding a consistency check in the policy, an ideal model would learn to disambiguate perfectly based solely history. Therefore, we hope to further explore how to exert better control over the diffusion's learned distributions for better consistency and accuracy.  
 
 Appendix: Model Structure: FlowBotHD consists of a current state encoder g ϕ , a history state encoder h γ and a denoiser D θ . Both current and history state encoder are based on PointNet++ [39] , where the history encoder only includes the PointNet++ global encoding module that outputs a global latent of 128 dimensions. We kept the same model architecture and hyperparameter set as the original PointNet++ paper. We inject the history latent into every layer of the PointNet++ framework in the decoder (see Figure 2 ). This is performed through a Hadamard product of the current point cloud encoding with the latent encodings at every decoding layer to produce the final output of the model. We base the denoiser on a DiT with 5 layers, 4 heads, and a hidden size of 128.Dataset Details: Similar to the original FlowBot3D paper [1] , we create a randomly opened dataset (RO) in which we randomly sample the view perspective, the target joint, and the open ratio. To open objects from the fully closed state, we create another mixed dataset (MD) in which we half of the objects are fully closed and the other half randomly opened. To enable history-aware training, we generate a dataset with 1/3 of the objects fully closed, 1/3 randomly opened but without history and the final third randomly opened and with history at random interval.Training Process: We set diffusion timestep to be 100 steps. We train the model for 450 epochs using AdamW optimizer with a weight decay of 1e-5. We use a learning rate of 1e-4, and a batch size of 128. We evaluate the model every 20 epochs, and save the checkpoint with the best winnertake-all RMSE metric. The Winner-take-all metric means that we repeatedly make predictions for each sample 20 times and record the best RMSE.We design a simulation environment in the PyBullet simulator that simulates operating a suction gripper to operate PartNet-Mobility objects. We first roll out the simulation with ground truth flow and filter out samples that can't open with the ground truth motion, due to errors in the simulator. We repeat each sample for 5 times during evaluation. The final test object categories we evaluated on after filtering and repeating are listed in Table 2 . To manipulate the objects, we first attach the suction gripper to the object. This is achieved by first teleporting the gripper to a position close to the target grasp point and then gradually moving towards the target grasp point until contact is detected. After contact, we apply a physical force constraint between the gripper and the object. The move action is then implemented as applying a certain velocity along the predicted direction.IconHyperparameter in Policy: There are 3 hyperparameters to set in Algorithm 1: the threshold for switch grasp point ϵ l , the threshold for consisteny check ϵ θ and the threshold for good movement ϵ m . We set ϵ l = 0.2, ϵ θ = 30 • and ϵ m =1e-2 in simulation. Table 3 demonstrates the quantitative simulation results for the normalized distance metric, computed in the same way as in Eisner et al. [1] . We demonstrate the simulation process in Fig 5 . We visualize the flow predictions (the first row), the simulation process (the last row, x-axis is the step number, the y-axis is the open ratio), and the trajectory plot along with policy signals (the middle plot). In these visuals, we can see our model and policy's pattern of opening an object: make one or more attempts at opening at the beginning of the episode, and once the door is opened a bit, make consistent predictions based on past history and consistency check.The first example is quite smooth: our model succeeds in moving the door on the first step. It continues to make consistent and history-aware predictions until the door is fully opened. Qualitatively, the consistency check didn't filter out many predictions due to high prediction quality, and due to the smooth execution the history is updated at each step, meaning that the policy always uesd the previous step's history for the next step's prediction. In the second example, our model makes several attempts to open the object when in the fully closed state. The observation history is not updated during these steps. Once the door is opened a bit, the history information and the consistency check enable the model to execute consistent actions. In the bar plot, we se that for step 11, the consistency check filters out 12 inconsistent predictions, demonstrating the effectiveness of applying a consistency check. Also, we can see that at step 6, the door is opened a bit, but the history is not updated due to the small movement (indicating the prediction is not good enough) which demonstrates the importance of tuning the history filter.Analyzing how many of the objects in the dataset are multi-modal or typical multi-modal paradigms provides valuable insights of the dataset. Take door category as an example, doors without handle typically have 4 possible modes when fully-closed (push left, push right, pull left and pull right); handles can indeed disambiguate which side of the door opens but still leaves ambiguity on whether the door opens by push or pull. Among the 28 training door joints and 7 test door joints, most of them (20 out 28 for training, 5 out of 7 for test) are with handles (2 possible modes -push and pull), and the other samples are without handles (4 possible modes -push left, pull left, push right and pull right).For other object categories, since there is no generic way to determine whether an object is ambiguous, it is hard to manually count ambiguous samples for all the categories (not to mention ambiguity might be subjective), therefore we describe broadly where ambiguities typically happen. There are three typical multi-modal objects (See Figure 6 ) : 1) Joints without handles, 2) Joints with small handles that might be erased when downsampling the point cloud, and 3) Joints with handles that may afford different opening modes: for example a dishwasher/oven with a handle on the top of its front surface, although the handle eliminates many modes, it is unclear whether it opens by pulling out like a drawer, or pulling from the top to the bottom like a lid or door.Similar to counting modes of general articulated objects, there is no precise way to count multimodality in predictions without including manual labeling. To approximately evaluate the multimodal predictions of the model, we make 50 predictions for each object in the test set and calculate their RMSE with respect to the ground truth flow. We record how many of the objects include both RMSE < 0.2 (presumably a correct prediction) and RMSE > 0.6 (presumably a different mode than the ground truth mode). We can see from the bar plot in Figure 7 that the model predicts multi-modality mostly in conceptually multi-modal categories like Door, WashingMachine, and Refrigerators. However it also makes multi-modal predictions with conceptually unambiguous cases, which aligns with limitations of this work we addressed in 6 about the difficulty of striking a balance between precision and multi-modality.To specifically analyze our model's improvements on handling multi-modality, we evaluated baseline FlowBot3D trained on randomly opened dataset (RO) / mixed dataset (MD) and our model FlowBotHD on doors. We open each test door from 0% to 100%. We average the metrics within 10% open to compute the performance for closed doors, and we average the metrics above 10% open to compute the performance for open doors. As we can see from Table 4 , our model outperforms the baselines by a large margin across all metrics. We observe that FlowBot3D trained on randomly We compare our model with baselines on doors. The cosine metric means the cosine similarity between the predicted flow and the ground truth flow, the higher the better. RMSE metric means the root mean square error between prediction and ground truth, the lower the better. MAG metric means the flow magnitude error, the lower the better. We analyze our model's performance under occlusions on different object categories. A door example is included in the Fig 3 We can see from the visualizations that with history, FlowBotHD is able to produce stable and robust predictions regardless of the occlusions while FlowBot3D makes less consistent predictions when severely occluded.We conduct further analysis on components of our roll-out policies, including the occasional negative effect of the history filter, the efficiency of switching the grasp point during policy execution and the effect of limiting the number of steps during execution.Since we train the model with only ground truth data, we propose to use a history filter to prevent incorrect predictions in the history from disturbing the history distribution; our history filter removes previous predictions that fail to further open the object. However, occasionally we find that including incorrect predictions in the history can lead to better performance; below we analyze when this might occur.Consider the below dishwasher (which opens by pulling the upper side) as an example, we experimented with 4 types of history input: No history (which can occur either at the beginning or due We performed an additional analysis to calculate the number of grasp actions each object needs. By average, our method needs 1.76 times grasp actions including the initial grasp, so on average each object requires 0.76 switch-grasps. The results show that our method does not require a lot of switching and still leads to improved performance over the baseline.We made 2 supplementary plots to visualize the frequency of switching grasp points in our simulation experiments. Figure 11 shows the switch grasp point rate per step. We can see that the switch grasp point rate gets lower among the first few steps -the consistency increases with multi-modality decreased. But it increases slightly at the later stages -possibly meaning that occlusions could happen and destabilize the grasp point predictions. Figure 12 shows the distribution of the number of switch grasps across the dataset. We can see that about 60% of objects only require one grasp action (the first step). And for the other 40%, most of them require only <5 extra switches of the grasp point.During a roll-out with limited number of steps (in our case, 30 steps), failures can happen either because of bad prediction quality that will never succeed, or a less efficient strategy or grasp point that might eventually succeed given enough steps. To further analyze the time dependency of the success rate we measured, we made a plot of the opening success rate at each time step.Figure 13 : Success Rate over Timestep: We can see that both baseline with and without switch grasp policy (SGP) and FlowBotHD (ours) have nearly reached a plateau after 25 steps. We infer that the performance is not much limited by the 30 step timesteps constraint.We can see in Figure 13 that both FlowBot3D (baseline) with and without switch grasp policy (SGP) and FlowBotHD (ours) have nearly reached a plateau after 25 steps, indicating that further timesteps after 25 would likely have limited performance increase. In our paper, we allow each method 30 timesteps to open the object before we compute the success rate for our analysis. Therefore we infer that most of the remaining failures are the result of incorrect predictions and the performance is not much limited by the 30 timestep constraint.F Real-world Equipment F.1 HardwareIn the real-world demos, we deploy our model on a Franka Emika Panda Robot. We obtain point cloud input data from an Azure Kinect Depth Camera. The robot's end effector is a Schmalz Cobot Pump with a suction cup 3 cm wide in diameter.We constructed our workspace (Fig 14 ) in a 1.3 m by 1.3 m by 1.1 m space. The Azure Kinect Camera was placed so that it pointed toward the center of the workspace at an angle that allowed the door to be seen clearly. To avoid collisions with the table when motion planning, we added a box representing the table top using MoveIt's collision box construction tool.To isolate the objects in the point cloud, we had to programmatically segment the table, background, and robot points.To segment out the table and background, we threshold the x, y, and z values of the points to effectively crop all points outside of the box where the object would be. For example, all points with a z value below 0.02 were removed, as that marks the top of the table.The robot points are filtered out in real-time using a ROS package called Robot Body Filter that takes in the robot's URDF file. The filter assumes perfect calibration of the camera, which leads to some trailing points from the robot occasionally remaining in the scene. We filter out those points using outlier removal since they are sparser than the object's points. The robot is controlled using position control by inputting the desired end-effector pose to MoveIt, which would then motion plan and execute the path.To make contact, the robot aligns itself with the chosen grasp point and estimated flow direction, then moves to a point 10 cm in the flow direction away. Then, a function is called to start the pump, and the robot moves in 2 cm increments towards the contact point. Another function detects when a vacuum has successfully been established, meaning the robot can progress to opening the object.For each step, the robot moves 5 cm in the direction of the respective predicted flow vector. After the robot attempts to execute the initial step, we use the switch grasp policy to determine if it should attempt to remake contact. To remake contact, the robot moves back to its initial starting pose, then repeats the process described above. The robot can remake contact up to 5 times.We custom built a multi-modal door out of plywood, with four pairs of hinges. This allows for it to be configured to open in all four of the possibilities of a standard door (forward to the right, forward to the left, backward to the right, and backward to the left). The door's current configuration is determined by two thin allen key-shaped metal rods, which connect a pair of hinges. Those hinges are then effectively in use and determine how the door opens. The door's frame is 25 cm by 8.5 cm by 43.8 cm, with a 31.75 cm by 36 cm by 1.25 cm stabilizing base. The door itself is 16.83 cm by 1.75 cm by 43.5 cm, with nothing on its front or back face to differentiate which way it opens.To ensure that it fits within our workspace, we built it to be much smaller than a normal door, then scaled the point cloud up by three before passing it into the model.Given the custom made door, our model has the ability to make multi-modal predictions and open the door in all 4 different ways: pull left, pull right, push left and push right. It also has the ability to actively switch grasp point when the initial prompts fail to align with the current configuration. In all the experiments, the action of 'push' is executed as pulling from the back.8th Conference on Robot Learning (CoRL 2024), Munich, Germany. 


Decision: Strengths:
- Clearly presented with extensive experimental validation.
- History aware diffusion networks that offers potential for handling uncertain situations in object manipulation like occlusions.

Weaknesses:
- Lack of real world results and direct evaluation on robots
- Scope of contribution is limited: provides a solution to limitations of prior methods

Reviewer LDDR focused on the lack of real-world experiments and direct robot testing. In response, the authors provided details about their real-world experiments, including the hardware, workspace setup, foreground segmentation process, and robot control paradigm. They also discussed the custom-built multi-modal door used in their experiments.    

Reviewer niS8 also pointed out the limited real-world evaluation and suggested including more real-world examples. Additionally, they asked about RGB/RGBD baselines, in-depth analysis of results, history window length, frequent grasp switching, and the value of om. The authors addressed these points by providing further analysis of their results, explaining their choices in the experimental setup, and discussing the challenges of incorporating RGB/RGBD baselines. They also included real-world experiments with a custom-built multi-modal door.    

Reviewer aMTY questioned the scope of the contributions, the limited real-world experiments, and specific aspects of the method, such as the quantification of ambiguous objects, rollout time steps, and cases where FlowBot3D outperforms FlowBotHD. The authors responded by providing a more detailed analysis of ambiguities in the dataset, discussing the time dependency of their method, and explaining the reasons behind the occasional outperformance of FlowBot3D. They also extended their real-world experiments to address the reviewer's concerns.

Reviews: review: title: title: Review
paper_summary: paper_summary: The paper introduces a new method for manipulating articulated objects with inherent ambiguities, such as doors with uncertain opening mechanisms (push, pull, slide) and directions. These ambiguities are compounded by occlusions that obscure the object's shape.
main_review: main_review: **Key Contributions:**

1.  The paper presents a history-aware diffusion network that addresses these ambiguities by modeling the multi-modal nature of the problem and using historical data to make stable predictions even when occlusions occur.
2. The proposed method leverages diffusion techniques to handle the multiple possible ways an object can be manipulated and adjusts predictions based on past interactions.
3. The approach extends prior work on articulated object manipulation (3DAF vectors) which, while effective in simulations, struggled with ambiguous real-world scenarios.

**Strength**:

1. The paper excels by innovatively using diffusion techniques to manage multi-modality in articulated object manipulation, offering multiple potential solutions for uncertain interactions. 
2. It is clearly presented, with a thorough and accessible explanation of the methodology and results. 
3. it includes extensive experimental validation, demonstrating the method's effectiveness in handling ambiguities and occlusions across various scenarios.
questions: questions: Limitations:

1.  while the proposed method shows strong theoretical and simulated results, there is a lack of hardware contribution, and the real-world experiments are somewhat limited in scope.  This absence means the full practical impact and reliability of the method in diverse, real-world settings remain uncertain.

2. The experiments do not include direct robot testing, which could provide deeper insights into the method’s performance in practical robotic applications. Incorporating such robotic experiments could offer valuable validation and could be more appropriate for discussions in other conferences focused on practical robotics and implementation challenges.
limitations: limitations: 2
review_summary: review_summary: Weak accept.
title: title: Manuscript Review
paper_summary: paper_summary: Interacting with articulated objects in real-world environments presents challenges like ambiguous manipulation directions, multiple optimal actions, and occlusions. To address this, the authors introduce FlowBotHD, a history-aware diffusion network that models the multi-modal distribution of object articulation. By incorporating historical context and consistency checks, FlowBotHD disambiguates actions and makes stable predictions even under occlusions. The authors demonstrate their approach's effectiveness by conducting experiments on multiple simulated objects and a real-world door-opening task.
main_review: main_review: The authors present a promising method for handling ambiguities and occlusions while interacting with articulated objects. While the approach demonstrates clear merit, the evaluation strategy needs to include more real-world examples to illustrate the model's effectiveness and maximize the impact of this work. 

**Strengths**:
- Addresses a Significant Practical Challenge: The authors tackle the real-world problem of disambiguating articulation directions in articulated objects under challenging conditions, such as occlusions and varying lighting. This research has direct implications for the development of robust action-oriented models and robotic interaction systems in real-world environments.
- Novel and Potentially Impactful Solution: The proposed FlowBotHD approach, utilizing a history-aware diffusion network, represents an innovative solution to address ambiguities in articulation direction. This novel methodology could lead to significant advancements in both perception and manipulation tasks for articulated objects.
- Clear and Comprehensive Presentation: The paper is well-written, with informative figures (notably Figures 2 and 3) and tables that effectively illustrate key concepts. The authors provide clear explanations and anticipate potential reader questions, resulting in a comprehensive presentation of their work.
- Practical Considerations Highlighted: The authors demonstrate a focus on real-world applicability by discussing specific corrections like switch grasp points and consistency checks. This attention to practical implementation details enhances the credibility of the proposed approach.

**Weakness**:
- Limited Real-World Evaluation: While the approach is promising, the current evaluation relies predominantly on synthetic data and simulated environments. A more comprehensive evaluation, incorporating real-world experiments with diverse articulated objects, is necessary to fully validate the effectiveness and generalizability of FlowBotHD.
- The experimental section would benefit from a more in-depth analysis of the results. In particular, the negative impact of the history filter on the performance of certain revolute joint objects (e.g., microwave, locker, washing machine) warrants further investigation and discussion to understand the underlying causes and potential mitigations.

**Miscellaneous**:
- Jain et al. introduced an approach in [1] that directly predicts the network’s uncertainty over the estimated articulation parameters for articulated objects. Consider including the work in the related work section for a more complete literature review on the topic.
- Lines 100-104, please cite the existing methods that suffer from the typical failure modes discussed in the text.
- The justification for the switch grasp point in lines 154-155 could be further strengthened by drawing parallels to human behavior. Humans often intuitively switch grasp points to achieve optimal motion and control when manipulating objects.
- Minor corrections: 
  - Line 98 to `manipulation` →`to manipulate`.  
  - Line 201 `datasetwith` -> `dataset with`.

*References*:
- [1]  Jain, A., Giguere, S., Lioutikov, R., & Niekum, S. (2022, January). Distributional depth-based estimation of object articulation models. In Conference on Robot Learning (pp. 1611-1621). PMLR.
questions: questions: - RGB/RGBD Baselines: Given that RGB features can aid disambiguation in varying lighting conditions, did you consider comparing your method to RGB-only or RGBD-only baselines, such as those in [2, 3, 4]? This could further highlight the value of your approach.
- In-Depth Analysis of Results: In Table 1, the history filter appears to negatively impact performance for some revolute joint objects (e.g., microwave (col. 6), locker (col. 9), washing machine (col. 14)). Could you provide a more detailed analysis in the experimental section, exploring potential reasons for this and discussing any mitigating strategies?
- History Window Length: Why was a history window length of 1 chosen? Would a longer history window potentially improve disambiguation? Did you conduct any ablation studies to determine the optimal window length?
- Frequent Grasp Switching: How does your approach mitigate the issue of frequent grasp switching between time steps? Are there mechanisms in place to ensure stable grasp selection?
- What value of $\delta m$ was used in your experiments? 


*References*:
- [2] Mo, K., Guibas, L. J., Mukadam, M., Gupta, A., & Tulsiani, S. (2021). Where2act: From pixels to actions for articulated 3d objects. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 6813-6823).
- [3] Jiang, Z., Hsu, C. C., & Zhu, Y. (2022). Ditto: Building digital twins of articulated objects from interaction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 5616-5626).
- [4] Heppert, N., Irshad, M. Z., Zakharov, S., Liu, K., Ambrus, R. A., Bohg, J., ... & Kollar, T. (2023). Carto: Category and joint agnostic reconstruction of articulated objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 21201-21210).
limitations: limitations: 2
review_summary: review_summary: The authors' proposed method shows potential for effectively handling ambiguities and occlusions when interacting with articulated objects. However, to fully substantiate the claims and demonstrate the broader applicability of the approach, it is essential to provide more comprehensive evaluation with real-world examples.  I am inclined to recommend this paper for acceptance, contingent upon the authors addressing these concerns through further experimentation and analysis.
title: title: Solid paper, even if slightly incremental
paper_summary: paper_summary: This paper proposes an extension to FlowBot3D, a method for predicting the movement flow of articulated objects. The extension is aimed at solving multi-modality and occlusion ambiguities with regards to object movements, and uses diffusion models and history encoding to solve both ambiguities respectively.
main_review: main_review: ## Strengths

The paper is well written and well presented. The contributions are clearly stated, and the experiments support the claims made by the authors. The proposed approach does, in fact, handle the ambiguities which caused failure cases in the original FlowBot3D. 

While reading the paper, I found the insights about adding actions to the manipulation policy interesting. The fact that the new actions are tested with the previous method as well is a welcome baseline comparison, and rounds out the suite of experiments. 

## Weaknesses

The central weakness of this paper, in my opinion, is the scope of the contributions. While the proposed approach solves a set of problems from the baseline approach, it is just that - a solution to a limitation of a previous approach, and therefore has limited novelty. A resolution of this weakness could comprise of exploring additional solutions to the ambiguity problem, or on the other hand, more convincing results for the proposed approach (see question 3 below). 

Another minor drawback of the paper is that the real-world experiments are lacking; it would be great to see them extended for future versions of the paper, so that they more closely match the set of simulated experiments. 

Some other concerns are raised in the questions section below.
questions: questions: 1. Line 189 states “it might be unclear which direction these objects open”: is there a quantification on how many of the objects in the dataset are actually ambiguous in that sense? While it is clear that doors, for example, are more ambiguous than laptops, even inside the door class I assume some have visual tells such as handles. This quantification would be interesting both in advance (when surveying the dataset), and when learned by the model, i.e., how often does the model predict objects to have multi-modal flows. 

2. Is a rollout limited in the amount of time steps? One might expect that even less efficient grasp points to succeed given enough time, as long as they are using the correct flow direction. Do failures mostly happen because of time constraints or because of choosing the wrong grasp points?

3. In the results table, it seems like there are cases where the original FlowBot3D outperforms FlowBotHD. To the best of my understanding, it seems like FlowBotHD extends the previous approach, using a more expressive distribution. Especially with the consistency check heuristic, it is hard to understand why there would be cases where the proposed approach is outperformed. Could the authors share examples for some failure cases where FlowBot3D outperforms FlowBotHD? Is there any insight on when and why this happens?
limitations: limitations: 3
review_summary: review_summary: This paper proposes a valid solution to challenges in a previous approach. While it has some drawbacks, in particular being slightly incremental, I find the contributions good enough to merit acceptance, especially if the other concerns are addressed.