Training Free Guided Flow-Matching with Optimal Control

Authors: Luran Wang, Chaoran Cheng, Yizhen Liao, Yanru Qu, Ge Liu
Primary area: generative models
Keywords: flow matching, controlled generation, inverse problem

Abstract:
Controlled generation with pre-trained Diffusion and Flow Matching models has vast applications. One strategy for guiding ODE-based generative models is through optimizing a target loss $R(x_1)$ while staying close to the prior distribution. Along this line, some recent work showed the effectiveness of guiding flow model by differentiating through its ODE sampling process. Despite the superior performance, the theoretical understanding of this line of methods is still preliminary, leaving space for algorithm improvement. Moreover, existing methods predominately focus on Euclidean data manifold, and there is a compelling need for guided flow methods on complex geometries such as SO(3), which prevails in high-stake scientific applications like protein design. We present OC-Flow, a general and theoretically grounded training-free framework for guided flow matching using optimal control. Building upon advances in optimal control theory, we develop effective and practical algorithms for solving optimal control in guided ODE-based generation and provide a systematic theoretical analysis of the convergence guarantee in both Euclidean and SO(3). We show that existing backprop-through-ODE methods can be interpreted as special cases of Euclidean OC-Flow. OC-Flow achieved superior performance in extensive experiments on text-guided image manipulation, conditional molecule generation, and all-atom peptide design.

Decision: accept (poster)
Reviewer scores: 6, 8, 6, 6

Reviews and discussion:
<thread>
creation date:
1730481742290

thread_type:
official_review

<thread_object>
creator: reviewer_zuf2

<content>
summary:
The paper proposes a novel method based on optimal control to optimize generations obtained by ODE-based generative models (e.g. flow matching). The paper proposes algorithms for generative models in Euclidean space and in SO(3), generalizes previously existing approaches, and studies the convergence of the proposed method.

strengths:
The paper tackles the problem of changing the generation process of ODE-based generative models in order to produce samples that maximize a certain reward, while staying close to the original ODE trajectory (through a regularization term). This problem is relevant in multiple domains where additional signals/information are available at inference time. 

To the best of my knowledge, this is the first paper to formalize this guidance and control framework in SO(3), which is a group used by many methods in structural biology.

The approach generalizes existing methods that optimize the trajectory of ODE-samplers (D-Flow, GradFlow), and outperforms them in multiple benchmarks.

weaknesses:
**Computational cost.** The method proposed in the paper, as well as its predecessors (D-Flow, GradFlow) all require optimizing the sampling process *for each sample* produced by the generative model. In other words, producing a single sample requires solving an optimization problem, for which computing the loss requires simulating the full ODE. This has a high cost in memory and computation time.

While GradFlow proposed a clever way of reducing the memory cost of this process, which is also adopted in this paper, this optimization process is inherently computationally expensive, significantly increasing the time cost of producing each sample. Previous work used some approaches to try to alleviate this (e.g. FlowGrad uses an adaptive solver to minimize the number of steps used during generation, by setting the step-size as a function of the estimated curvature of the flow at the current point), but simulation is still considerably slower than the baselines without this optimization process. While generation times are not discussed in this work, the D-Flow paper states that producing a single molecule takes around 3 minutes (they use 100 function evals for discretization of the ODE), and for images the time to generate a single output ranges from 4 to 15 minutes depending on the task. While the purpose of these works is not increasing generation efficiency, but generating better samples through guidance and optimization, they exacerbate the main limitation of diffusion models / flow models, which is their slow generation. In the paper I am unable to find generation time for the experiments. Given the method's nature, I think these should be reported and discussed.

I think related to this point, experiments tend to be on the smaller end. Celeba-HQ for images, molecule generation with up to 9 heavy atoms, and peptides, which are short proteins (less than 50 residues). I understand these methods are able to produce better samples given the external guidance, while other approaches are unable to leverage such information, which is quite valuable. Still, I think providing values for computational cost / generation time, and comparing against plain approaches (baselines that do not require tuning) would be good. I would expect the cost of producing one sample is between 10x-100x more than baselines that do not optimize the sampling process (since there are ~20 optimization steps, and some gradient computation too), but happy to be shown otherwise. This does not account for the fact that lower memory requirements by the baselines would allow producing more samples in parallel too.

questions:
Proposition 1. What is the prior terminal point $x^p$? Is it $x^p_0$ or $x^p_1$ (that is, terminal as in time $t=1$ or $t=0$). If $t=0$ then the joint $p_1(x^p, x_1)$ is a delta distribution (since ODE is deterministic)? If $t=1$ then $x^p$ and $x_1$ are independent (since $x^p$ is generated from random noise independent of $x_1$)?

GradFlow could in principle be used for manifolds too? The control terms would live in the tangent space of the manifold at the current point? They do not propose this in GradFlow so this is not something to compare against, and would even be consider a novelty and addition to the paper. But this work got me wondering if there’s any obvious reason why this would fail?

I’m not sure I completely agree with the paper’s title “Training free…”. I understand training is the same, and that this method can be used for any pre-trained flow model. But it does require solving an optimization problem, albeit with few iterations (~20) but expensive ones. The difference is that this optimization happens at inference time.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730689229488

thread_type:
official_review

<thread_object>
creator: reviewer_jxtp

<content>
summary:
This paper attempts to solve the problem of conditional generation using Flow-Matching models. In particular, they propose a unifying framework (OC-Flow) from which other approaches (such as D-Flow and FlowGrad) can be derived, and operates in Euclidean and SO(3) geometries. Subsequently, the authors provide extensive theoretical analysis of OC-flow to prove convergence and theoretical properties. Finally, the authors apply OC-Flow to text-guided image generation and peptide design tasks.

strengths:
* The paper has a strong theoretical grounding, is well placed within the existing literature and goes to extensive efforts to prove theoretical properties of the proposed methodology. Additionally, the paper provides significant context and background, referencing existing works in Flow-Matching models. Finally, the authors provide a ton of detailed proofs in the appendix, and theoretical analysis in the main body of the text.
* The paper provides a legitimate contribution to formalizing and extending existing guided-flow matching techniques to complex geometries (such as SO(3))
* The authors compare the proposed methodology to similar existing methods to demonstrate competitive performance
* Table 1 gives a good comparison to understand the contribution of OC-flow vs D-Flow and Flow-Grad

weaknesses:
__Theoretical Concerns__
* The main concern with the theoretical aspects of the paper is that the authors perform all the theoretical analysis in the continuous regime, but then implement the practical algorithms in a discrete regime. They even mention this limitation in the conclusion (“we also note that our practical algorithm…”). This presents a potentially significant hole in the paper, as the objects being proved, and the objects being validated empirically are not the same, and thus the theoretical content in the paper is not necessarily applicable to the experimental results (as noted by the author). To address this, we recommend that the authors 1) provide theoretical analysis of the discrete regime of the algorithm, 2) discuss in detail how the continuous approximates or bounds the behavior of the discrete implementation, or 3) implement a continuous version of the algorithm. These additions would help bridge the gap between theory and practice.

__Experimental Concerns__
* There is a significant lack of information regarding the experiments and reproducibility. There are no details as to how these models are trained, what the architectures are, the implementation, etc. This would need to be addressed before the paper could be accepted. To address this, we recommend that the authors give detailed descriptions of the model architectures and hyperparameters, training procedures and optimization details, data preprocessing steps, computational resources used, and code availability or plans for release. This would greatly improve the reproducibility of the work.
* Similarly, no error bars/confidence intervals are reported on any of the experiments. In fact, it is unclear whether their model is better than existing baselines (i.e. in table 5, they claim 0.795 is better than 0.793, but no CI, similarly in tables 2, 3 and 4, they report outperforming existing methods but do not report CI.). Consequently, it is not possible to assess their claims that they outperform existing models, given the lack of confidence intervals and the close proximity of the performance values. To address this, we recommend that the authors 1) run multiple trials and report mean and standard deviation for all metrics, 2) perform appropriate statistical significance tests (e.g. t-tests) when comparing to baselines and 3) include error bars or confidence intervals in all tables and figures.

* Finally, one of the major claims in the paper is that OC-flow can optimize in euclidean and SO(3) space, and that optimization in SO(3) provides benefits in tasks such as protein design. However, the authors do now present results split into Euclidean and SO(3) algorithms. It is unclear how/if the extension to SO(3) is even beneficial, and additional experimental details/results are needed to validate this claim as well. This is brought up by "our OC-Flow method, fully optimized in both Euclidean and SO(3) space" on page 10.

__Presentation Concerns__
* The paper has some issues with the presentation that make it quite difficult to asses which parts are novel contributions, and which parts are existing works that are being used. The authors/paper would benefit greatly from having a clear vision of what they are proposing and why, and subsequently moving large portions of the detailed proofs to the appendix to not muddy understanding with unnecessary detours. For example, what are co-state flow and E-MSA, and why do we care about these constructs? How do these constructs factor into the actual problem of performing guided matching flow generation? Are they purely used for proving convergence analysis? And if so, then it should be framed/explained as such. In fact, the structure of the paper would greatly benefit from having a section which clearly describes the proposed methodology in terms of implementation, and a separate section for the theoretical analysis of the proposed method, since the current structure makes it very difficult to separate the method as-such, from the additional theoretical concepts only necessary for proving convergence.
  * To address these issues, we recommend that the authors clearly delineate novel contributions from existing work, as well as practical details from theoretical proofs. Adding sections such as "contributions", "proposed methodology and implementation", and "theoretical results" would greatly improve the structure of the paper.
  * Additionally, we recommend that the authors provide clearer explanations of key concepts such as co-state flow and E-MSA, as well as highlighting their importance to the key contributions in the paper.
  * Finally, by restructuring the paper to highlight the novel aspects, and moving detailed proofs to the appendix, the overall quality and clarity of the paper would be much improved.
* Furthermore, several significant objects/theorems are introduced with very little explanation. For example, co-state variables are introduced as “shadow prices representing the sensitivity of the optimal value function to changes in the state variables”. But what are shadow prices? How does this analogy help when there is little thought/exposition given to the co-state/Hamiltonian introduced by the PMP? I would like to see a much more principled approach to writing, where each theorem introduced is clearly placed within the larger context of the work, and has a clear purpose in support of theoretical results.
* Similarly, the lack of figures greatly hinders understanding. Additionally, figure 1 does not clearly articulate what it is presenting, and what the various sub-figures and equations represent.

__Contribution Concerns__:
* First of all, due to the presentation it is not clear what the contributions of the paper are, and what is preexisting work being leveraged for proving theoretical properties of the method. However, my understanding is that there are two main contributions of the paper: 1) they formulate conditional generation using flow-matching models as a control problem in equation 2, and 2) given that formulation, the authors demonstrate that OC-Flow is a generalization of D-Flow and Flow-Grad that can be optimized in SO(3), as well as proving various convergence properties.
* In light of this understanding, it seems like the contributions of the paper are limited in scope. First of all, equation 2 seems to be fairly trivial extension of existing Flow-Matching/Continuous Normalizing Flow formulations (see Fjelde et al (2024)). Furthermore, given the lack of definitive experimental results, it is unclear whether this extension provides tangible benefit over existing methods, especially when considering the additional complexity. One of the major proposed benefits of the method is optimization in SO(3), but no ablation studies are given to demonstrate that SO(3) provides additional benefits over simple euclidean optimization.
* Additionally, the experimental reproducibility of the paper is quite poor, with no experimental parameters given, and no experimental source code provided.
*Finally, the scope of the contribution is somewhat niche. In particular, this paper focuses on classifier-guided generation using flow-matching models in SO(3). While useful for certain problems, it likely does not have wide-reaching implications outside of a few target applications.


__Citations__
* Fjelde, T., Mathieu, E., & Dutordoir, V.. (2024). An Introduction to Flow Matching.

questions:
* I would like to see a comparison of the runtime of OC-flow vs the other methods. While Table 1 suggests that the memory consumption of OC-flow is lower than D-flow and on par with Flow-Grad, I would be concerned that the additional complexity of solving in SO(3) adds significant computational costs.
* I would like to have a better understanding of what parts of the paper are core to the methodology (i.e. actually implementing OC-Flow), versus what parts of the paper are necessary for proving convergence. I would then like to see separate sections/subsections for the proposal, and the subsequent analysis.
* I would like to see a clearer presentation of a conventional flow-matching model, and how the proposed method extends this standard formulation, ideally in the form of before/after equations to get a clear and unambiguous idea of the elements being added/proposed.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730706976683

thread_type:
official_review

<thread_object>
creator: reviewer_o479

<content>
summary:
This paper provides a novel approach for guided generation using pre-trained diffusion and flow matching models. Traditional methods of guiding ODE-based generative models often require expensive retraining and work mainly on Euclidean manifolds, but OC-Flow uses an optimal control training-free framework beyond Euclidean spaces to the SO(3) manifold. Experiments on tasks like text-guided image manipulation, conditional molecule generation, and peptide design validate the method’s effectiveness​.

strengths:
- As far as I know, the approach is original in framing guided flow matching as an optimal control problem. The authors develop a general framework for non-Euclidean geometries with strong theoretical backing, which is fairly rare.
- Another strength of this work is that it is a general approach, i.e. OC-Flow can be used effectively for a variety of applications such as image and molecular data. 
- Unlike existing approaches, OC-Flow allows training-free guidance, making it computationally efficient and more applicable in real-life settings. 
- The SO(3) results such as on improved molecular generation accuracy, validate the importance of using this geometric inductive bias for generative models. 
- Through the framing of existing approaches as special cases under their optimal control formulation, this paper helps clarify the connections between gradient-based techniques like FlowGrad and D-Flow.
- The model consistently demonstrates improvement over previous work.
- I really appreciate the figures included in the paper to illustrate and compare the methods against existing approaches.

weaknesses:
- My main question is regarding scalability. While the model performs well on selected benchmarks, to me it is still unclear how OC-Flow scales to high-dimensional datasets such as large molecules. A discussion of potential scalability limits and memory efficiency in such cases would strengthen the paper.
- Moreover, even though the formal contributions are great and well-formalized, to me the paper is still quite hard to read. Since the theoretical results are one of the main contributions, I think it would be valuable to add more intuitive explanations of the proofs and why they are there. For example, theorem one provides a bound based on VFM on KL between the model and a terminal point, but some intuition of why this bound is provided would make the paper more approachable.
- Adding to this point, the theoretical assumptions made (e.g. Lipschitz continuity, boundedness) are clear and needed for the argument, but some reflection (perhaps on a high level) on whether these hold in practice would help to interpret the method's advantages.
- While the focus is on continuous CNFs, a brief comparison with discrete flow techniques could contextualize OC-Flow's advantages or limitations more clearly, especially as discrete methods have shown promise in similar applications.

questions:
- The paper shows promising results, but could the authors elaborate on potential ways to enhance scalability, especially when applied to e.g. large molecules or more complex target distributions in general? 
- How does OC-Flow compare to recent works in Riemannian FM? What about SO(3) and SE(3)?
- Could OC-Flow be adapted for hybrid tasks where both Euclidean and Riemannian components are present? This might extend its applicability to even broader fields where you need this hybrid. Does the method allow for this directly or not?
- Since OC-Flow is designed to be computationally efficient, could the authors comment on real-time applications requiring 'immediate' guidance?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730870983835

thread_type:
official_review

<thread_object>
creator: reviewer_fauf

<content>
summary:
This paper proposed a new framework for controlled generation using pre-trained diffusion and flow matching models, dubbed OC-Flow. The method is based on sound theory in optimal control that offers additional convergence guarantees in Proposition 1 and Theorem 2 (under two key assumptions of affince Gaussian path and Lipschitz continuity of the gradient of guided loss). Several benchmarks on guided-image manipulation, molecular generation and protein design with generative models are performed to demonstrate the effectiveness of the method.

strengths:
- Well-motivated problem, overall nicely written paper with clear literature review.
- The methodological and theoretical parts of the paper are well-sounded. 
- Providing a framework that has convergence analysis is always welcomed.

weaknesses:
**Major: questionable and inconsistent baselines' results in empirical benchmarks** 

- While on the first task (section 5.1 text-guided image manipulation) the authors have report/insert exactly other baselines' results (originally in Table 2 of the FlowGrad paper); the results on two remaining tasks in section 5.2 (molecule generation) and section 5.3 (peptide design) do not match the results reported in their respective original paper. 
- More specifically, the results in Table 3 do not match those of Table 4 in D-Flow paper (Ben-Hamu et al. 2024); results in Table 5 do not match those of Table 1 in PepFlow paper (Li et al. 2024). In fact, if one instead takes into account the original results, the baseline D-Flow actually perform better in MAE metrics compared to OC-Flow in Table 3. For Table 5 the metrics reported are in different scale. 
- I am therefore request the authors to clarify this discrepancies between results reported in their paper and the results reported in the respective original works of compared baselines. Otherwise, I think the practical performance of OC-Flows remains questionable.

Ben-Hamu et al. (2024). D-Flow: Differentiating through Flows for Controlled Generation. Proceedings of the 41 st International Conference on Machine Learning, Vienna, Austria. PMLR 235, 2024.

Li et al. (2024). Full-Atom Peptide Design based on Multi-modal Flow Matching. Proceedings of the 41 st International Conference on Machine Learning, Vienna, Austria. PMLR 235, 2024.

questions:
See weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732295991809

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Overall Response

comment:
We sincerely thank our reviewers for their insightful feedback which enabled us to further improve our paper. We greatly appreciate reviewers recognition of our clear presentation (faUf, o479, zUf2), theoretical significance (all reviewers), soundness of method (faUf, o479), importance of problem setting(faUf,o479,zuF2), novel contribution(o479,zuF2), and empirical superiority(o479,zuF2). We would like to address several common feedbacks and highlight key improvements we made with the following response:

**Additional experimental results**

- To further establish the strength of our OC-Flow-SO3, we conduct new peptide design experiments with **comprehensive ablations to study the effectiveness of Euclidean, SO3, and joint Euclidean+SO3 OC-Flow**. As shown in table 5, applying OC-Flow on rotation(SO3) or translation(Euclidean) alone effectively improves the results, while applying SO3+Euclidean OC-Flow jointly achieved the best performance, doubling the increase of the Madrax score from 0.34(single modal) to 0.68(joint). 

- We further include ablation on SO3 to show the **necessity of OC-Flow-SO3 on guiding SO3 FM**(table 12), where naively applying gradient update with projection to tangent space of SO3 fails to optimize and leads to worse results than unconditional samples.

- We study the sensitivity to optimizers in QM9, where we show that DFlow heavily relies on L-BFGS (used in their paper) which leads to high runtime complexity, whereas OC-Flow achieves similar strong performance without L-BFGS (table 11), enabling efficient implementation. 

**Scalability and runtime**

- To address the requests from all reviewers regarding runtime and scalability, we now provide a comprehensive **theoretical analysis** and **empirical evaluation** of the runtime and memory complexity of OC-Flow and other baselines in our revision (appendix D). We show that our efficient algorithm on both SO3 and Euclidean (e.g., with **asynchronized update**, **adjoint method** and **vector-Jacobian product formulation**) significantly reduced the time and memory complexity on SO3 from $O(ND^4)$ to $O(D^2)$ (table 6/7). On Euclidean, our method is better than Dflow in terms of memory ($O(ND^2)$ ->$O(D^2)$) and runtime ($O(cND^2)$ ->$O(nD^2)$).

- Our empirical benchmark of runtime demonstrates **speedup and memory saving** compared to Dflow. We show that OC-flow samples 256x256 images in **216s**, and small molecules in **38s**, as opposed to Dflow which self-reported to take **15min** on 128x128 images and ran OOM in our 256x256 image data.

- We also show that **OC-Flow-SO3 can be efficiently implemented with the same order of complexity as OC-Flow-Euclidean**, and runtime of **296s (SO3)** vs. **188s (EU)** confirms the analysis. With the above, we make sure that OC-Flow is scalable and efficient.

**Reproducibility**

- We provide more extensive experimental details in Appendix E, including hyperparameters and implementation details to make sure reproducibility. We plan to open source our code soon.
- Our efforts for **fair, valid and reproducible experiments** includes:
  1. Making sure benchmarks are controlled, comparing all methods **using same publicly available open-sourced pretrained FM model and reward model**.
  2. Reporting metrics with large number of samples (~10^3) for significance.
  3. Reaching out to authors and following exact same configuration when implementing baselines.

**Additional Theoretical Results**
- We provide additional analysis on **discretization error bounds** of our continuous algorithm in C.3. As our discretization is Euler method of ODE, we prove the discretization error is of the order $O(\Delta t)$ and becomes negligible with sufficiently dense steps. In practice, we use 100 time steps for text2img experiment, 50 timesteps for QM9 and 200 time steps for Peptide design, which makes the assumption valid.
- We extend our proposition 1 to also provide additional **KL bound between marginal distributions of prior and guided model** as a function of running cost. The intuition here is to theoretically show that the running cost can control deviation from prior distribution, which is key to our algorithm motivation.

**Presentation**
- Following advises from reviewers, we improved our presentation of contribution, theory, figures, and notations, and provide more intuitions that help readers understand the significance of the work. We also include discussions on efficiency-performance tradeoffs in inference-time guided generation approaches, as well as scalability of OC-Flow.


We are sincerely grateful for the reviewers' time and their insightful feedback that helped us improve the practical impact of the paper. We believe our efforts have addressed all of the reviewers' comments, and we welcome further discussions.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732568465892

thread_type:
official_comment

<thread_object>
creator: area_chair_xedz

<content>
title:
Last day for reviewers to ask questions to the authors!

comment:
Dear reviewers,

Tomorrow (Nov 26) is the last day for asking questions to the authors. With this in mind, please read the rebuttal provided by the authors and their latest comments, as well as the other reviews. If you have not already done so, please explicitly acknowledge that you have read the rebuttal and reviews, provide your updated view and score _accompanied by a motivation_, and raise any outstanding questions for the authors.

**Timeline**: As a reminder, the review timeline is as follows:
- November 26: Last day for reviewers to ask questions to authors.
- November 27: Last day for authors to respond to reviewers.
- November 28 - December 10: Reviewer and area chair discussion phase.

Thank you for your hard work, 

Your AC
</content>
</thread_object>

</thread>
<thread>
creation date:
1733198940593

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Sincere Gratitude from Authors

comment:
Dear Reviewers,


We sincerely thank all reviewers for their invaluable time and effort during the review process. We are encouraged by Reviewer fauf’s faith in reproducibility, Reviewer o479’s recognition of the paper’s value, Reviewer jxtp’s acknowledgment of its “strong theoretical grounding,” and Reviewer zuF2’s emphasis on its novelty. We are grateful that our rebuttals have successfully addressed your questions and concerns, and we deeply appreciate the recognition of our contributions of the proposed OC-flow framework.


The interactive discussions during the rebuttal period have been both inspiring and productive, providing valuable insights that have been instrumental in enhancing our work. We are committed to polishing the paper further to build a more comprehensive and mathematically rigorous manuscript. Once again, we sincerely thank the reviewers for their thoughtful engagement, which has greatly contributed to improving the quality and clarity of our research.


Warm regards,
The Authors
</content>
</thread_object>

</thread>


Metareview:
metareview:
Among other things, the reviewers have highlighted the nice framing of guided flow matching as an optimal control problem, the contribution to formalizing and extending the existing guided-flow matching techniques to SO(3), and the applicability of the proposed approach to various domains such as image and molecular data. Furthermore, it was highlighted that the framing of existing work as special cases under the optimal control formulation helps clarify connections between different methods. Finally, the results were found promising.

As important areas for improvement, both reviewer faUf and jxTp had questions regarding the experimental setup that required clarification, and reviewer faUf raised concerns about the results of baseline methods and how they compare against what is reported previously in the literature. Another important area for improvement suggested by multiple reviewers was to provide more details on runtime comparisons. Other points include questions around how the theoretical assumptions hold in practice (raised by two reviewers). In particular, reviewer jxTp raised questions on the use of discretization in practice while the theory provided relies on a continuous framework. Furthermore, reviewer jxTp suggested that an ablation study on optimization in SO(3) would be helpful to demonstrate the additional benefits over Euclidian optimization. Finally, reviewer jxTp also asked for a clearer exposition of the contributions of the paper, while other reviewers found this clearly illustrated. Finally, several reviewers provided suggestions for clarifications and explanations to improve the accessibility of the paper, they 

The two major concerns were addressed in the rebuttal and revised version of the paper. The concerns on reproducibility and details of the experimental setup were addressed by sharing anonymized code, and including additional details in the revised paper, as well as explanations in the rebuttal itself. The second major point regarding runtime comparison and scalability was addressed by including a discussion on time and memory complexity in appendix D, as well as practical runtime and memory usage. The majority of the other concerns were addressed as well. The authors have included a discussion and theoretical analysis of the discretization gap in the revised paper. The authors have also incorporated the suggestion to include an ablation study to demonstrate the benefit of optimization in SO(3).  Finally, the authors have also taken into account several suggestions by the reviewers to improve the clarity of the paper. While reviewer jxTp still finds there to be room for improvement on the exposition of the paper, they have indicated that several other concerns have been addressed and raised their score.

In summary, most of the concerns initially raised by the reviewers were addressed through fruitful discussions between the authors and reviewers, and revisions in the submitted paper. Given multiple raised scores, there is now unanimous agreement among reviewers that this paper should be accepted. Therefore I recommend to accept this paper.