Learning Robotic Manipulation Policies from Point Clouds with Conditional Flow Matching

Authors: eugenio  chisari
nick  heppert
max  argus
tim  welschehold
thomas  brox
abhinav  valada
Venue: robot-learning_org_CoRL_2024_Conference
Published: 2024-06-07 13:37:32

Abstract: Learning from expert demonstrations is a popular approach to train
robotic manipulation policies from limited data. However, imitation learning
algorithms require a number of design choices ranging from the input modality,
training objective, and 6-DoF end-effector pose representation. Diffusion-based
methods have gained popularity as they allow to predict long horizon trajectories
and handle multimodal action distributions. Recently, Conditional Flow Matching
(CFM) (or Rectified Flow) has been proposed as a more flexible generalization
of diffusion models. In this paper we investigate the application of CFM in the
context of robotic policy learning, and specifically study the interplay with the
other design choices required to build an imitation learning algorithm. We show
that CFM gives the best performance when combined with point cloud input
observations. Additionally, we study the feasibility of a CFM formulation on
the SO(3) manifold and evaluate its suitability with a simplified example. We
perform extensive experiments on RLBench which demonstrate that our proposed
PointFlowMatch approach achieves a state-of-the-art average success rate of 67.8%
over eight tasks, double the performance of the next best method.

Main paper: 
 introduction: 
 Imitation learning (IL) is the widely studied problem of training policies from a given set of expert demonstrations [1, 2, 3] . In recent years, imitation learning has gained popularity in the robot learning community, as leveraging the prior knowledge of the expert demonstrator allows training complex behaviors with small amounts of data. The primary approach to learning an IL policy is Behavior Cloning (BC) [4, 5] , where a deterministic mapping from state to actions is learned in a supervised manner from the available data. While BC has achieved significant success for different tasks, robot policy learning remains a challenging problem, given the requirement of high precision, the sequential correlation (i.e. not i.i.d.) of data, and the multimodality of the action distribution, which all add complexity compared to other supervised learning problems. disadvantages: diffusion models need to explicitly define an iterative forward diffusion process, which inherently defines the final noisy distribution and the probability path the model learns to denoise along. In the typical case of Gaussians, a closed-form solution is available, enabling us to directly generate fully noised and intermediate, partially noised, samples. In turn, in the cases where no closed-form solution for the forward diffusion process is available, training time will increase [11] . To overcome these limitations, Conditional Flow Matching (CFM) has been proposed as an efficient generalization of diffusion models [12, 13, 11] . CFM is a simulation-free approach, i.e. it starts directly from noise without requiring a forward diffusion process. It is, therefore, more general, since it allows transporting any starting probability distribution into the data distribution.Inspired by recent flow-based generative models, we propose PointFlowMatch, a novel imitation learning algorithm for robotic manipulation. PointFlowMatch uses point cloud observations that prove to be more effective than images [9, 10] and builds upon a CFM formulation to learn the distribution of the expert robot trajectories. As CFM is able to model arbitrary probability paths, it also allows formulating the regression on the R 3 × SO(3) manifold. We evaluate the performance of our proposed method on the popular RLBench benchmark [14] and compare it against strong recent baselines with both image and point cloud observations: Diffusion Policy [6] , 3D Diffusion Policy [9] , ChainedDiffuser [10] , and AdaFlow [15] . In summary, this paper makes the following main contributions: • PointFlowMatch, a novel method based on the recent conditional flow matching framework to train robotic imitation learning policies from point clouds. • An investigation of two different approaches to handle 3D rotations in the context of CFM for policy learning. • Extensive evaluations against recent state-of-the-art baselines and an ablation study of our main design choices. • We make the code, models, and videos publicly available at http://pointflowmatch.cs. uni-freiburg.de. 
related work: 
 Generative models for imitation learning in robotic manipulation have attracted significant interest.In the following, we highlight recent advances in this topic.Generative Models for Policy Learning: One interesting class of generative models explored for robotic policies is the Action Chunking Transformer (ACT) [16] . It adopts a CVAE [17] structure, where the encoder predicts a latent variable z and the decoder outputs the action sequence. A different approach is taken by Florence et al. [18] , who propose Implicit Behavior Cloning (IBC). IBC defines the distribution over action as an Energy-Based Model (EBM) [19] . This approach inherently handles multimodal distributions but is unstable to train due to the need for negative sampling when computing the loss. An early attempt to apply diffusion models to robotic applications is presented in Diffuser [20] , where the authors investigate the use of diffusion models from the perspective of planning. Refining this idea, Chi et al. [6] propose the seminal work Diffusion Policy (DP). DP inherits the advantages of IBC (handling multimodal action distributions) while avoiding its downsides. Instead of learning an energy function, DP learns the gradient of the action distribution, which exhibits better training stability. Diffusion Policy from Point Clouds: The success of DP inspired numerous follow-up works. A key characteristic common to many of them is the choice of point cloud observation representations, instead of RGB images, as used in the original DP method. ACT3D [21] is a transformer model that predicts the next best waypoint, instead of a full trajectory. 3D-DP [9] learns to predict full trajectories, and it also adopts a modified MLP as an efficient point cloud backbone. ChainedDiffuser [10] proposes a hierarchical model where high-level key points are predicted via ACT3D and low-level trajectories connecting these points are generated via the standard trajectory diffusion. Another hierarchical model is HDP [22] , which builds on PerAct [23] as a waypoint predictor and combines both joint and end-effector pose predictions. While hierarchical models achieve strong results, we specifically investigate the performance of the underlying trajectory diffusion models and therefore focus our analysis on non-hierarchical single-task policies.Conditional Flow Matching: Conditional Flow Matching (CFM), also called Rectified Flow (RF), is a recent generative model training technique that connects noise to data via straight paths [12, 13, 11] .The more established diffusion models can be seen as a special case of the CFM framework, which is more general and conceptually simpler. Very recently, CFM was demonstrated to be superior to diffusion models in various applications, ranging from image synthesis [24] to protein backbone generation [25] . In the context of robotics, AdaFlow [15] is an image-based policy trained with CFM, which also includes a variance estimation network to predict the variance of the current state and adjust the number of integration steps dynamically in order to reduce inference time. Concurrently to our work, Braun et al. [26] investigate the application of CFM to Riemannian manifolds, but do not provide a direct comparison with the standard Euclidean approach, while Rouxel et al. [27] show the application of CFM for shared autonomy teleoperation of a humanoid robot. 
technical approach: 
 We consider an imitation learning problem: given a dataset of n expert demonstrations D = s (i) , a (i) n i=1 , the goal is to train a policy π : S → A that maps the observed states s to the optimal actions a. Given the widespread accessibility of depth cameras, we assume that both RGB and depth observations are available to the policy. In the following sections, we detail our choices of observation and action spaces, training formulation, and model architecture.It has been demonstrated that, for robot policy learning, point cloud observations yield better results compared to RGB images [9] . These findings are supported by the fact that most recent state-ofthe-art approaches [21, 22, 10] building upon DP adopt point clouds as their visual observation representations. Point cloud observations are effective as they directly encode the three-dimensional structure of a scene, and clearly separate geometric and semantic features, which are instead blended together in raw RGB images. This separation is especially helpful in the low data regimes common in robot policy learning. Therefore, in our approach PointFlowMatch we also adopt point clouds as the visual observation representation.The input to our policy consists of the T obs last observations, which include both the robot state and processed point cloud, where T obs is a tuning hyperparameter. Point clouds are processed in the following manner. For each available camera, we use the intrinsics and extrinsics to project the observed depth values in a common 3D space, and we merge all observations into a single point cloud. Last, the final point cloud is computed by applying voxel-downsampling to attain uniform density, and by cropping all points outside the relevant workspace around the robot. The action space of our model is 10-dimensional, consisting of end-effector position (3-dimensional), end-effector orientation (6-dimensional) , and gripper open/close action (1-dimensional). We choose absolute position control following Chi et al. [6] , which showed it to be more suited to action-sequence prediction compared to velocity control. We represent the gripper orientation as the 6D vector formed by truncating and flattening the relative rotation matrix, as proposed by Zhou et al. [28] . At test time, we employ a closed-loop receding horizon control strategy, i.e. we predict T pred horizon steps into the future, command only the first predicted step to the robot, and then repeat the prediction.In CFM, for arbitrary samples z, the prediction problem is formulated as an ordinary differential equation (ODE) of the formEQUATIONwhere p 0 is the arbitrary start distribution, commonly chosen as random Gaussian noise with zero mean and unit variance, and v : R d × [0, 1] → R d is the velocity vector field that induces the probability path from the start to the target distribution. The goal is to learn the appropriate vector field that transports the chosen probability distribution to the data distribution in unit time. The vector field v θ can be learned by supervision via the loss functionEQUATIONwhere v gt is the choice of ground truth vector field we regress against. While this choice is not unique, CFM adopts the time-independent, straight line vector field v gt = z 1z 0 , which has the desirable property of producing straight probability paths [12, 11] . In practice, the vector field v θ is parametrized by a neural network and the loss in Eq. ( 2) can be minimized via mini-batch gradient descent. In the context of policy learning, the target samples z 1 are the robot trajectories from the expert demonstration dataset. Once the model is trained, we can sample z 0 from the start distribution p 0 and numerically integrate the ODE from Eq. ( 1) up to time t = 1 to generate new robot trajectories that follow our expert data distribution. An overview of the implementation of CFM for policy learning is shown in Algorithm 1. Our robot state representation includes the end-effector 3D orientation. Since 3D orientations live on the SO(3) group's manifold, we need to pay particular attention to how we handle the orientation's prediction. There are two strategies we can employ to tackle this challenge.Euclidean Formulation: The first strategy is to follow the standard CFM formulation in Euclidean space for both training and inference and project the resulting vector to the SO(3) group's manifold only at the end of the inference process, to output a valid rotation matrix. This projection can be implemented as a Grahm-Schmidt process in the case of a 6D representation of rotations or as the SVD-based orthogonal Procrustes in the case of a 9D representation of rotations [29] . SO(3) Formulation: The second strategy is to define the starting random distribution and target vector field directly on the 3D rotation manifold so that the entire resulting probability path lives on SO(3). In particular, the random initial state is sampled as z(t = 0) ∼ IG SO (3) , whereIG SO(3)is the isotropic Gaussian distribution on SO(3). Since the 3D rotation group is a smooth manifold, its velocity at a specific point is expressed in the tangent space, denoted as so(3). The mapping from the tangent vector space to the manifold is defined by the exponential map Exp(•), while the inverse mapping from the manifold to the tangent space is defined by the logarithm map Log(•). For a detailed overview of the topic, we refer the readers to [30] . For our CFM setting, the natural choice of target vector field is given by the tangent on the manifold that forms the geodesic interpolation between the start and target samplesEQUATIONAt test time, we can integrate the predicted velocity vectors in tangent space along the manifold geodesic to solve the ODE in Eq. ( 1) and generate end-effector orientations that directly lie in SO(3).The implementation of the SO(3) formulation of CFM for policy learning is shown in Algorithm 2. While the latter approach has been studied in recent works such as Riemannian Flow Matching [31] and FoldFlow [25] , a comparison of the two strategies in the context of policy learning is still lacking.In our evaluation (Appendix A.1), we investigate both strategies and provide the counter-intuitive insights that the first, simpler approach might be preferable for policy learning in robotics.To encode the point cloud observation into low-dimensional features, we use a modified version of PointNet [32] . Compared to the original architecture, we remove both T-nets used for input and feature transformation, see [33] . These are used to improve rotation and translation invariance, which is desirable for classification and segmentation but not for robotics tasks. The model used to de-noise trajectories is a conditional 1D U-Net, similar to [6] . This model takes as input the random noise samples and the condition information and returns the sample velocity in the case of CFM and the noise ϵ in the case of diffusion. For an overview, see Fig. 1 . The condition information consists of the concatenation of the robot's proprioceptive state (i.e. end-effector poses) as well as the encoded visual observation, and it is added to the bottleneck state of the U-Net model.We train our model with the AdamW optimizer and a learning rate of 3e -5 and weight decay of 1e -6 . We apply cosine annealing of the learning rate and linear warmup of 5000 steps. We use a batch size of 128 and apply EMA on the weights. The input point clouds are downsampled to 4096 points and the target robot trajectories are subsampled with a factor of 3. 
experimental evaluation: 
 We evaluate our proposed PointFlowMatch on RLBench [14] , a popular robot learning benchmark. Each environment consists of the 7-DoF Franka Emika robot arm to execute actions and a set of five cameras around the workspace. We use RLBench's automatic expert demonstration execution to collect 100 trajectories for each task, which consists of all robot states and camera observations. For evaluation, we rollout the policy on each task for 100 episodes and report the average task success rate across all episodes. Following RLBench's test protocol, for each episode the task setup is randomized. We further perform three evaluation runs with three random seeds and average over all runs. As a result, all models are evaluated on the same set of 300 random episodes. Following ChainedDiffuser [10] , we consider a set of eight tasks that require continuous interaction with the environment, Table 1 : Performance comparison of PointFlowMatch with different baseline methods on the RLBench set of tasks. We report the success rate (SR) (↑) as well as the delta to our method. On average, our method performs considerably better. For OL-ChainedDiffuser we report the results from the original publication, which do not include the standard deviation. In this section, we compare the performance of PointFlowMatch with recent state-of-the-art methods for imitation learning based on generative models. For a fair comparison, we keep the number of denosining and/or integration steps k constant to 50 for all methods.For image-based methods, we compare against the original Diffusion Policy [6] as well as AdaFlow [15] which also uses the CFM learning objective. For the scope of this analysis, we do not implement the adaptive step feature in AdaFlow, since we mainly focus on the quality of prediction and not on inference speed. For both methods, we follow the original Diffusion Policy [6] approach and fuse images from all five viewpoints after the backbone feature extraction. As point cloud-based baselines, we consider Chained Diffuser [10] as well as 3D-DP [9] . In this work, we specifically investigate the performance of non-hierarchical, single-task policies. For this reason, we consider the Open loop trajectory diffusion baseline presented in ChainedDiffuser [10] , which is the underlying diffusion model without the higher level waypoint policy forming the hierarchy.We report the results in Tab. 1. This evaluation highlights the strong performance of PointFlowMatch, which achieves a success rate 34 percentage points higher than the second-best model. This result demonstrates that the combination of our choices of observation type, encoder architecture, and training objective leads to a highly effective imitation learning algorithm. In line with the findings of our ablation study in Sec. 4.2, we also highlight how the point cloud based policies outperform the image-based policies.In this section, we aim to compare the key design choices of our method and answer the following research questions:• Are point clouds a more effective observation representation compared to raw RGB images?• Is Conditional Flow Matching a more effective training objective compared to the more established diffusion models? • Is the SO(3) formulation of CFM more effective at learning to match the data distribution on the 3D rotation manifold, compared to CFM in Euclidean space followed by a projection on SO(3)? • How does the number of inference steps k affect performance?In order to answer these questions, we carry out an ablation study where we vary each of the aforementioned design choices and investigate their impact on overall performance. In particular, we investigate 1) using images as visual observation, adopting the same ResNet backbone as in the original DP [6] , 2) using the Denoising Diffusion Implicit Model (DDIM) [34] instead of the CFM training objective, 3) adopting the SO(3) formulation of CFM for the end-effector orientations to learn probability paths directly on the 3D rotations manifold, 4) testing both the DDIM and CFM-based models with varying inference steps k, ranging from 1 to 16. We report the results in Tab. 2 and Fig. 3 . From the numerical evaluation, we observe that the largest impact on performance is determined by the choice of observation type. The policy using raw RGB inputs is 27.7 percentage points worse than the proposed model, highlighting the strengths of point cloud observations. First, we consider the two different formulations of the CFM framework that can be used to learn about the distribution of 3D rotations: euclidean and SO(3). The outcome for this ablation is very similar, with the score averaged across all tasks showing that the SO(3) formulation achieves a lower success rate, by 0.4 percentage points. This result is surprising and at first counterintuitive, as we expected that learning probability paths directly on the target manifold would achieve higher performance. Given the intriguing result of this ablation, we add further analysis of this comparison in Appendix A.1.Regarding the choice of training objective, the results in Tab. 2 show that for k = 50 inference steps, the CFM objective does not improve over the popular DDIM framework, which achieves a similar success rate, within the standard deviation. Nevertheless, when considering the two objectives across a range of value of k inference steps in Fig. 3 , we find that CFM demonstrates stronger performance, especially in the low k regime. This is in line with the results obtained for the image synthesis domain, where Stable Diffusion 3 [24] shows that CFM achieves similar scores to the diffusion objectives, but performs better with fewer inference steps, required for faster inference. This characteristic, paired with the higher flexibility, generality, and ease of implementation, in our opinion, makes the CFM formulation advantageous compared to established diffusion objectives. To demonstrate the application of PointFlowMatch on a real robotic platform, we perform a set of experiments with a Franka Emika Panda manipulator. We evaluate on two table-top tasks: open box and sponge on plate, shown in Fig. 4 . We use two Realsense RGB-D cameras, the first mounted on the robot end-effector and the second mounted externally. Similarly to the simulation setup, we merge the point clouds obtained from both cameras and process it via voxel-subsampling.To prevent overfitting and improve the model's robustness to the noisy point clouds, we apply random transformation and random noise-jitter augmentations during training. PointFlowMatch achieved a success rate of 72% for open box and 48% for sponge on plate. The main failure mode we observed in this experiment consisted of the robot reaching correctly for the object to grasp, but missing it by a small margin. A selection of policy rollouts for both tasks is shown in the accompanying video. 
conclusion: 
 We present PointFlowMatch, a novel method for imitation learning from a fixed set of demonstration examples. Our method combines the recently developed conditional flow matching framework with a cloud observation encoding. In developing our method, we performed an ablation study of the most important components. Additionally, we study a formulation of CFM on the SO(3) manifold. Our method achieves state-of-the-art results for single-task non-hierarchical policy learning, with an average success rate of 68.6 %, double that of the closest baseline method.Limitations: There are a few limitations to our proposed method. Compared to approaches such as BC that directly predict an action, diffusion and CFM models both have longer inference times, given their iterative nature. In addition to this, as usual in the fixed-data imitation learning setting, CFM cannot extrapolate out of distribution and thus, only learns motion correction behavior when included in the demonstration set. Our point cloud observation space merges multiple views, this requires depth cameras with known intrinsic and extrinsic calibration. We also trained single-task policies and did not investigate generalization capabilities. Finally, implementation on a real robot requires manually collecting expert demonstrations.Table 3 : Comparison of different state dimensions and their respective velocities, target calculation and progression formulations. For definitions of the Log(•)-and Exp(•)-maps we refer to Sola et al. [30] . † As for calculating the final angle only the first column is needed, we omit calculating the second orthonormal vector. Thus, instead of performing the full Grahm-Schmidt process, we only perform a normalization of the inferred state variable that maps the vector to the unit circle.  
 
 Appendix:  


Decision: This paper introduces PointFlowMatch, a method that combines Conditional Flow Matching (CFM) with point cloud observations for imitation learning in robotic manipulation. The authors demonstrated improved performance compared to baselines on RLBench tasks.

Before the rebuttal, the reviewers acknowledge the importance and relevance of the proposed work but raised several concerns. The primary issue is that the results are not strong enough with the performance only demonstrated through simulation on RLBench. There are no hardware experiments to validate the method in more complicated real-world scenarios. Additional concerns include a lack of rigorous analysis on the advantages of CFM in the context of robotics, missing supplemental materials that were promised, and the absence of discussions on more related works, among others.

During the rebuttal, the authors did an excellent job addressing the reviewers’ concerns, particularly by including hardware results. As a result, all reviewers now rate the paper as a Weak Accept. However, the reviewers mentioned during the discussion that concrete comparisons between diffusion and flow methods will be necessary to further enhance the impact and relevance of the work to the robot learning community.

Reviews: review: title: title: Excellent investigation of conditional flow matching as an alternative to diffusion policy.
paper_summary: paper_summary: The paper presents an investigation of conditional flow matching (CFM) versus diffusion policy. The authors also ablate image versus point cloud features and different ways to learn rotations. The experiments consider several SOTA imitation learning architectures on RLBench, showing better performance for CFM and 3D point cloud features as compared to diffusion policy on image-based features.
main_review: main_review: Strengths:
* The paper topic is pertinent to the research community as diffusion policy and its variants have become a critical research direction in robot learning.
* The literature review is thorough and does a good job contextualizing the paper in the literature.
* The comparisons are extensive, consisting of many SOTA diffusion policy-style methods and different input-feature modalities.
* The discussion is insightful. It is good to see a fair investigation that finds certain choices important (poinct cloud features) and others providing only modest gains (CFM vs diffusion).

Weaknesses:
* The evaluation is only in simulation on RLBench; there are no hardware experiments.
* The literature review is missing a few concurrent works:

[A] Braun, M., Jaquier, N., Rozo, L., & Asfour, T. (2024). Riemannian Flow Matching Policy for Robot Motion Learning. arXiv preprint arXiv:2403.10672. (Came out before submission.)

[B] Rouxel, Q., Ferrari, A., Ivaldi, S., & Mouret, J. B. (2024). Flow Matching Imitation Learning for Multi-Support Manipulation. arXiv preprint arXiv:2407.12381. (Came out after submission.)

* In the limitations, it is not true that imitation learning can't learn 'motion corrective behaviors'. It is possible to learn them if they are included in the demonstrations in small quantities.

* The following is a non-exhaustive list of typos and grammatic errors.
1. Line 129: 'supervising' -> 'supervision'.
2. Line 284: there should be no comma after diffusion.
3. Make sure to capitalize acronyms (e.g., 3D) in the paper titles of the references appropriately.
questions: questions: In addition to the weaknesses listed above, I have the following questions for the authors.
1. In Sec. 3.2, why use a Gaussian distribution if one of the advantages is the generality of the possible distributions? It would be interesting to explore other possibilities.
2. For my understanding, why is the linear step in Algorithm 1 taken before the network during training and after during inference?
limitations: limitations: 2
review_summary: review_summary: Overall, I think this is a solid paper accomplishing what it sets out to do in terms of architecture investigations. The use of CFM as compared to diffusion policy is promising as it is more general and does not require both a backward and forward pass. The only concern with this paper that I have is the lack of hardware experiments.
title: title: Conditional flow matching from pointclouds
paper_summary: paper_summary: This paper introduces PointFlowMatch, a method combining Conditional Flow Matching with point cloud observations for robotic manipulation learning. Evaluated on RLBench tasks, it outperforms existing methods. The authors analyze various design choices, concluding that point cloud inputs significantly contribute to the method's effectiveness while other strategies such as manifold learning and normalizing flows do not.
main_review: main_review: - CNF typically refers to continuous normalizing flows, not conditional normalizing flows.

- When listing disadvantages of diffusion models you state: "the space of sampling probability paths is limited (only Gaussian)". This is not necessarily true. Many existing diffusion works use non-gaussian noise [1]

- "which restricts 39 learning on simple manifolds and results in longer training times". This claim is also untrue. Diffusion has been applied to manifolds outside of standard euclidian [2].

- No supplementary materials? The paper says code, models, and videos are available but the link is dead.

- The results look really good in comparison with baselines, but no standard deviation were reported on the results and no statistical tests were performed. Given such a large improvement over baselines, I'm guessing the results are significant, but it would be good to know for certain.

- Section 4 says "Each result reported is the average across three random seed". This statement needs clarification. Is this three separate training seeds or three environment seeds? The policy is run for 100 episodes. Are the locations/positions of the objects randomized for each episode?

- Figure 3 might not be necessary. I think it is widely understood why training with euclidian distances on a non-euclidian space is a bad idea, and it would be easier to just point to one of the many recent works on this subject. It also seems not too relevant for this problem setting according to the results.

- No experiments performed on real robot setups. This is especially important here because depth images are notoriously noisy and are likely to significantly impact performance. One of the key questions the authors try to answer is "Are point clouds a more effective observation representation compared to raw RGB images?". They certainly contain more information than raw RGB images, but I think it is premature to make the claim that pointclouds are strictly better without testing on real-world pointclouds.

- I'm a bit confused by the results and discussion in this paper. The authors appear to be concluding that the choice of flow/diffusion and se3/euclidian is not very important and the choice of pointcloud/pointnet vs rgb/cnn has a much larger impact. Then why does 3D-DP perform so poorly on the RL bench problems if it is essentially an pointcloud-based diffusion model in euclidian space? 

- The conclusion states that CFM has longer inference times? How much longer? Inference times should be reported somewhere.

[1] Bansal, A., Borgnia, E., Chu, H. M., Li, J. S., Kazemi, H., Huang, F., Goldblum, M., Geiping, J., & Goldstein, T. (2022). Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise

[2] Elhag, A. A., Wang, Y., Susskind, J. M., & Bautista, M. A. (2023). Manifold Diffusion Fields.
questions: questions: See above
limitations: limitations: 3
review_summary: review_summary: The paper presents a novel combination of existing methods in generative modeling and pointcloud neural networks for behavior cloning. While the architecture is novel, the experiments are lacking real-world evaluations. Additionally, no supplementary materials were provided.
title: title: Contribution is unclear and the results are not strong enough.
paper_summary: paper_summary: This paper applies conditional flow matching (CFM) to imitation learning. CFM is a more general form of generative models. The method is evaluated against baselines using diffusion models. The authors also investigates the use of 6D representation and SO3 representation in the rotation matrix. The third emphasis of the work is using point cloud as input.
main_review: main_review: Pros 
- The authors describe very clearly the input and output of the models, the hyper parameters, and number of training iterations. These details are helpful for reproducibility and to compare with other models.
- The visualization in Fig. 3 (c) (d) is interesting. The discontinuity can be clearly seen from the figure.
- The ablation is helpful.

Cons
- The contribution is not clear enough. The use of point cloud has been studied in many other papers. The continuity on rotational representation is presented in [26]. The major contribution to me is using CFM instead of diffusion models. However, the theoretical advantage of CFM is not clear to me. Though there is a potential flexibility in encoding the distribution, it is not illustrated clearly through the writing or experiments. According to Table 2, using CFM only has a marginally higher SR than DDIM, which renders the main contribution less appealing. 
- I think the continuity of SO3/SO2 representation has been well studied in [26]. Is the part comparing 6D and tangential representation not covered there? Please correct me if I’m wrong.

Other
- P3 L110 - ’T_{obs} is a tuning hyperparameter’. Looks like there were some equations here and results in a typo?
- About the paragraph above Sec. 5. I’m not quite following here.  Should there be another plot for the SO3 representation to draw the conclusion?


Overall I think either 1) more analysis be provided on what theoretical advantage CFM has in IL, or 2) more convincing empirical result be presented, would make this a stronger paper.
questions: questions: - Some result numbers are really close. I’m curious about the significance of the result. It’ll be great if there are some measure of variance there.
- Are all baseline models trained using the same data generated by the authors?
limitations: limitations: 3
review_summary: review_summary: Trying different generative models is nice. But there is only limited discussion on why would using CFM have advantages in the context of robotics.  Results are not strong enough. After the rebuttal: Though the results are not very strong, there is some potential benefits of the framework. The conclusion on rotation formulation could be helpful to other authors. I'm inclined to accept the paper.