Paper: Thin-Shell Object Manipulations With Differentiable Physics Simulations (ICLR-2024)

Reviewer comment (Experiments):
It seems the system does not fully support/couple with rigid-body simulation, which is important for modeling robots.

Author response:
While we can directly integrate the rigid body dynamics into the implicit-time integration scheme, the system will become quite cumbersome and therefore we decide to process the deformable and rigid system separately. Currently the robot manipulator is modelled by position-based control, i.e., the action space of rigid body end-effectors are defined as SE(3) movements within each timestep, and therefore the elastomers are attached to moving rigid boundaries controlled by the actions.

We are still working on further modelling of rigid body dynamics. For the rigid manipulated objects, we plan to support them by adding an stiff orthogonality potential energy on a deformable mesh as in ABD[1]. We also plan to include the penalty energies of joint constraints to model the dynamics behavior of articulated body system.

Reviewer comment (Experiments):
The paper does not show any tasks involving constraint simulation (e.g., cloth with one end fixed).

Author response:
We support the constraints of fixing vertex positions. In **Folding** and **Forming**, we fix one end of the paper on the table. In **Separating** and **Following**, we fix two manipulator in a gripper shape.

The technique to simulate the constraints is simple. To fix one vertex (or even one dimension of this vertex), we simply set the derivative and hessian matrix related to this dimension to zero before solving the equation. This will result in an unchanged position for this vertex while modeling it's force and influence to other vertices.

Reviewer comment (Experiments):
Real-world manipulation only contains a single case. It is not convincing that in general the described system identification method will improve sim-to-real policy deployment.

Author response:
We add more real-world manipulation cases identical to **Folding** tasks in simulation. We use a rigid manipulator to interact with a curved paper and try to fold on different creases.

Reviewer comment (Theory):
The contact model uses quadratic energy instead of barrier. How does the system correctly identify that 2 triangles are penetrating? Based on the paper, it seems it is based on normal direction, however, this does not seem correct since the normal would flip if one of the contacting shell is flipped.

Author response:
To model contact behavior, we apply quadratic penelty energies on a dynamic set of contact constraints. The reason of not using a barrier function is that we observe that the quadratic model converges much faster than the log-barrier model on complex scenes involving multiple layers of thin-shell material. With quadratic energies, we actually make non-penetration a soft constraint with a large stiffness coefficient. When a constraint is established between a vertex and a shell surface, we maintain a "side" property between them, whose value is one of the shell's two sides. This property remains unchanged until the vertex's projection falls on the shell's boundary. When dealing with contact between the vertex and the shell, all faces of the shell will be assigned with a normal direction pointing towards the corresponding side. In this way, we avoid the ambiguity problem of shell surface's normal direction, and also the penetration problem caused by the soft constraint, since a vertex slightly penetrating into a surface will be repulsed towards a consistent direction.

Reviewer comment (Reproducibility):
The paper describes the system as simulating tetrahedral and triangular elements, how is the rigid-body action space achieved?

Author response:
While we can directly integrate the rigid body dynamics into the implicit-time integration scheme, the system will become quite cumbersome and therefore we decide to process the deformable and rigid system separately. Currently the robot manipulator is modelled by position-based control, i.e., the action space of rigid body end-effectors are defined as SE(3) movements within each timestep, and therefore the elastomers are attached to moving rigid boundaries controlled by the actions.

We are still working on further modelling of rigid body dynamics. For the rigid manipulated objects, we plan to support them by adding an stiff orthogonality potential energy on a deformable mesh as in ABD[1]. We also plan to include the penalty energies of joint constraints to model the dynamics behavior of articulated body system.

Reviewer comment (Reproducibility):
Newton's method requires solving a large linear system. How is this implement on the GPU? Is it solved through sparse matrix algorithms, conjugate gradient methods, or some other method?

Author response:
For linear equation solving, we use a python binding of the cuSPARSE library, which supports solving sparse matrices using Cholesky Factorization on GPU. The speed is around 2.74 FPS in one case and the bottlenecks are **1) linear equation solving at each iteration of Newton's method** and **2) Hessian matrices building and SPD projecting**.
The details could be found in general response point 4.

Reviewer comment (Evaluation):
What is the speed/performance of this simulator? What are the bottlenecks?

Author response:
For linear equation solving, we use a python binding of the cuSPARSE library, which supports solving sparse matrices using Cholesky Factorization on GPU. The speed is around 2.74 FPS in one case and the bottlenecks are **1) linear equation solving at each iteration of Newton's method** and **2) Hessian matrices building and SPD projecting**.
The details could be found in general response point 4.

Reviewer comment (Presentation):
The overall presentation is good. However, there could be improvements in clearly labeling the three system identification inverse tasks - Sliding, Bouncing, and Card - in both Table 2 and Figure 3. Their current placement together without clear labels within the table or graph is confusing.

Author response:
**1. Labeling inverse tasks**

We have separate those tasks in tables and figures in our main paper. Thank you for pointing them out!

Reviewer comment (Presentation):
Furthermore, it would enhance the paper if some clear conclusions were drawn, along with providing some directions or suggestions to guide readers to explore each type of method.

Author response:
**2. Conclusion and future directions**

We draw a conclusion that RL methods are of low data efficiency in most of the times and it would be good to seek a combination of sample-based methods (maybe RL) and gradient based method. More discussions about this are shown in General Response point 7.

Reviewer comment (Reproducibility):
Interested to know what is the speed of the simulation? Is multiple GPU parallelization supported by the simulation?

Author response:
> **Q1: Interested to know what is the speed of the simulation? Is multiple GPU parallelization supported by the simulation?**

The speed is approximately 2.74 FPS in a test case. Currently, there are no obstacles preventing us from running parallel environments on different GPUs. However, we haven't pursued this option because the GD method does not derive any benefits from parallelization. More details can be found in General Response point 4.

Reviewer comment (Presentation):
The presentation in the paper does not feel quite as refined as the website, especially the figures, which feel like they could use some work. For example, Figure 2 does not have the image bounding boxes align, and within the Sliding subfigure you can see how the edges of the images don't match up. A similar small issue is present in Figure 3 with a typo in "Separatering".

Author response:
**1. Typo and figure quality**

Thank you for pointing them out, we have made the revisions.

Reviewer comment (Reproducibility):
A lot of information is pushed to the Appendix, even though it could be beneficial to include in the main text. For example, the description of the hybrid method, which is a large engineering contribution of the paper to show working policies, is described only at a high level. A more detailed description would help, and mentioning how much work it was to tune/balance the gradient-free and gradient-based steps could aid the community.

Author response:
**3. Hybrid method details**

This information has been incorporated into General Responses point 5. In essence, we apply a fixed proportion to the steps of CMA-ES and GD in the hybrid method. Additionally, we introduce specific constraints to shape the behavior and enhance the learning process of CMA-ES.

Reviewer comment (Presentation):
Details such as how the real-to-sim system identification is performed and how the disparity between simulation and reality is computed should be included in the main text rather than only in the Appendix.

Author response:
**2. Real-to-sim disparity**

While we are using tactile sensor and force feedbacks as observations in the real-world, we compute the disparity by the L2 loss between the simulated tactile markers (projected into 2D images given the camera parameters) and the images collected in real-world, and by the L2 loss between the simulated forces and the real-world force feedbacks. The loss of several experiments and 2D images of markers are ploted in the appendix.

Reviewer comment (Evaluation):
It is better to have some success indicators for tasks in the benchmark. Otherwise, it is hard for readers to understand difference in performance between methods, if only rewards are provided (e.g., Table 2).

Author response:
While we design continuous rewards without a distinct boundary between success and failure and primarily assess the success of a policy through visualization, we also present scores for human-designed policies, theoretical analyses, and success indicators in General Responses point 6.

Reviewer comment (Reproducibility):
In Sec 4.1, it seems that the observation includes a set of points. Does the order of points matter? How do the learning algorithms (SAC, PPO) used in the paper handle the observation if the input observations (points) are unordered?

Author response:
The order of points matters in the observation. As we uniformly sample points in the full state, the sequence of selected nodal points remains constant within a given task.

Reviewer comment (Writing):
In Sec 4.1, does the 6DoF stand for the pose change in SE(3), and does the 7DoF include one more DoF for gripper closeness (mimic behaviors for two finger tips)?

Author response:
Yes, the 6DoF stand for the pose change in SE(3) (x, y, z, roll, pitch, yaw), and the 7DoF include one more DoF for gripper closeness.

Reviewer comment (Evaluation):
In Sec 5.1, the authors mentioned that they reported "the maximum scores achieved within a specified number of timesteps". Does it mean that the benchmark currently focuses on "optimization" rather than "generalization"? Besides, how do the authors count "timesteps" for CMA-ES and GD methods?

Author response:
The experiment setting we currently test on indeed focuses on "optimization" rather than "generalization". 

However, we still want to share vision about how to address generalization in General Response point 7. Concretely, we can switch the initial state and parameters in one task and evaluate close-loop policies. Another way is to use open-loop method to collect successful trajectories under different initial state, parameters or even different tasks, then we can use imitation learning with those data to obtain a generalist policy.

Reviewer comment (Presentation):
What's the meaning of "episode" (x-axis) for GD and CMA-ES in Figure 3?

Author response:
**The "episode" means a rollout which is a full sequence of actions from the initial state to a final state.** 

The "timesteps" here has the same meaning of the previous "episode". As a result, we can simply count the number of full action sequences executed in GD, CMA-ES, and RL to count the episode. We will change them into a uniform one to avoid confusing. Thank you for pointing it out.

Reviewer comment (Writing):
In Sec 5.2, for "our gradient-based method converges these parameters towards a uniform outcome.", do the authors mean that the method converges to similar solutions despite different initialization?

Author response:
Yes. While the **Card** still have a respectively large standard deviation in the end, it is clearly shown in **Sliding** that they converges to similar solutions.

Reviewer comment (Theory):
The description of the simulator, one of the core contributions, is barely a page long and mostly covers high-level concepts, physical quantities being modeled, and methods used to implement the simulator. Greater detail is needed as it is currently hard to assess if this is a straightforward task of writing up a few equations or required complex derivations and developments.

Author response:
We provide additional details about the implementation of our simulation, specifically addressing the process and methods we used to compute the derivatives and Hessian matrix to address your concerns.

**1. Contact Energy**
According to Section 3.2, the contact energies $U_r$ and $U_f$ are defined as
$$
U_r(d)=\frac{k_r}2\max(d-\epsilon_r, 0)^2,U_f(\mathbf u)=\mu\lambda f_0(\epsilon_v,\|\mathbf u\|),
$$

where the distance $d$ is a signed distance between vertex $\mathbf p$ and its projection to a triangle $(\mathbf p_1,\mathbf p_2,\mathbf p_3)$, whose normal direction is represented by $(\mathbf p_2-\mathbf p_1)\times(\mathbf p_3-\mathbf p_1)$. $k_r$ stands for a constant stiffness parameter, $\mu$ stands for a friction coefficient, and $\lambda$ stands for the contact force along normal direction at last timestep. The signed distance $d$ is then calculated as
$$
\begin{aligned}
d=\frac{(\mathbf p-\mathbf p_1)\cdot[(\mathbf p_2-\mathbf p_1)\times(\mathbf p_3-\mathbf p_1)]}{\|(\mathbf p_2-\mathbf p_1)\times(\mathbf p_3-\mathbf p_1)\|}\\
=\frac{(\mathbf p-\mathbf p_1)\cdot\det
[\mathbf p_2-\mathbf p_1,\mathbf p_3-\mathbf p_1,(\hat{\mathbf x},\hat{\mathbf y},\hat{\mathbf z})]}{\|(\mathbf p_2-\mathbf p_1)\times(\mathbf p_3-\mathbf p_1)\|}\\
=\frac{\det[\mathbf p_2-\mathbf p_1,\mathbf p_3-\mathbf p_1,\mathbf p-\mathbf p_1]
}{\|(\mathbf p_2-\mathbf p_1)\times(\mathbf p_3-\mathbf p_1)\|},
\end{aligned}
$$

in which the 3D vertices are represented as column vectors. Let $\mathbf x=[\mathbf p_1,\mathbf p_2,\mathbf p_3,\mathbf p]\in\mathbb R^{12}$ denote the involved variables, we implemented tool functions in Taichi to calculate
$$
\begin{aligned}
D(\mathbf x)&=\det[\mathbf p_2-\mathbf p_1,\mathbf p_3-\mathbf p_1,\mathbf p-\mathbf p_1]
,\\
C(\mathbf x)=\|(\mathbf p_2-\mathbf p_1)\times(\mathbf p_3-\mathbf p_1)\|,
\end{aligned}
$$

together with the gradients $\partial D/\partial\mathbf x,\partial C/\partial\mathbf x$ and Heissan matrices $\partial^2 D/\partial \mathbf x^2,\partial^2 C/\partial\mathbf x^2$. With the help of SymPy library for symbolic computation, we optimized the computation graph and hard coded it into Taichi snippets. The repulsive energy $U_r$ is then a composition function on $D(\mathbf x),C(\mathbf x)$, and therefore we use the chain rule to calculate its gradient and Hessian matrix.

For the friction energy $U_f(\mathbf u)$, we follow IPC [1] to calculate the $f_0(\epsilon_v,\|\mathbf u\|)$ function, its derivative $f_1(\epsilon_v,\|\mathbf u\|)$ and second derivative $f_2(\epsilon_v,\|\mathbf u\|)$. Once we get the $\partial U_f/\partial \|\mathbf u\|$ and $\partial^2 U_f/\partial \|\mathbf u\|^2$, we transform it back to the tangential space to get $\partial U_f/\partial\mathbf u$ and $\partial^2U_f/\partial\mathbf u^2$.

**2. Elastic Energy**

We follow Stable Neo-Hookean [2] to define the elastic energy as
$$
U_e(\mathbf F)=\frac{\mu}{2}(I_C-3)-\mu(J-1)+\frac\lambda2(J-1)^2,
$$
in which $\mathbf F$ denote the deformation gradient of one tetrahedral element, $I_C=\text{tr}(\mathbf F^T\mathbf F)$ and $J=\det \mathbf F$. The motivation of replacing the original $\log J$ term in original Neo-Hookean model to $J-1$ is that $J-1$ is stable even for $J\le0$. Therefore, even if the whole tetrahedral element is inverted, this Stable Neo-Hookean model still works robustly and can provide gradients & Hessian matrices to recover the element (See the original paper of Stable Neo-Hookean [2]). This situation is common in our simulator, since the robot manipulator involves a moving boundary and can easily cause invertion when the moving speed is large. We observe that Stable Neo-Hookean perfectly addresses this problem.

The gradient $\partial U_e/\partial \mathbf F$ can be computed easily. To compute the Hessian matrix $\partial ^2U_e/\partial\mathbf F^2$, we first enumerate the differential $\delta F_{ij}$ for single entry and then calculate the correponding force differential $\delta(\partial U_e/\partial\mathbf F)$. We follow a well-known tutorial on FEM [3] to calculate the force differentials, and then concatenate them into the Hessian matrix $\partial^2 U_e/\partial \mathbf F^2$.

Reviewer comment (Experiments):
There is no discussion regarding the chosen way of modeling thin objects. Since the selected approach is likely not the only one, a discussion of the pros and cons of modeling the interactions in different ways would be a valuable contribution.

Author response:
**3. Thin-Shell Energy**

Following [4], we apply the stretching energy $U_s$ and the bending energy $U_b$ for thin-shell materials. We further divide the stretching energy into $U_s^e$ and $U_s^a$, representing stretching energy for edges and areas of triangles. The energy formula is as follows:
$$
\begin{aligned}
U_s^e=\sum_{e}K_e(1-|e|/|\overline{e}|)^2|\overline{e}|,
U_s^a=\sum_{A}K_a(1-|A|/|\overline{A}|)^2|\overline{A}|,
U_b=\sum_{e}K_b(\theta_e-\overline{\theta_e})^2|\overline{e}|/|\overline{h_e}|,
\end{aligned}
$$
where $e$ represents all the edges, $A$ represents all the triangles, and $\theta_e$ represents the angle on edge $e$. $|\overline{e}|$, $|\overline{A}|$, $|\overline{h_e}|$, and $\overline{\theta_e}$ respectively represent the rest length of the edge, the rest area for the triangle, one-third of the average height of the two adjacent triangles of the edge, and the rest angle on this edge. $K_e$, $K_a$, and $K_b$ are the coefficients.

For $U_s^e$, it's simple to calculate the derivatives and Hessian matrix. The derivative of the length $|e|$ to the nodal point positions $x_0$ and $x_1$ is $\frac{\partial |e|}{\partial x_0}=(x_0 - x_1)/|e|$. The Hessian matrix will require the calculation of $\frac{\partial^2 |e|}{\partial a_0^2}$ and $\frac{\partial^2 |e|}{\partial a_0 \partial b_0}$ due to symmetry, where $x_0=(a_0,b_0,c_0)$.

For $U_s^a$, it becomes more complex to compute the derivatives and Hessian matrix. Assume the triangle is $(x_0, x_1, x_2)$ and $x_0=(a_0, b_0, c_0)$. Similar to the calculation of the contact energy, we use math tools to generate the symbolic formula of the derivative $\frac{\partial |A|}{\partial a_0}$ and hard code it with Taichi; then, we can calculate all the derivatives due to symmetry. For the Hessian matrix, we use math tools to generate formulas for $\frac{\partial^2 |A|}{\partial a_0^2}$, $\frac{\partial^2 |A|}{\partial a_0 \partial a_1}$, $\frac{\partial^2 |A|}{\partial a_0 \partial b_0}$, $\frac{\partial^2 |A|}{\partial a_0 \partial b_1}$; then, we can calculate the full matrix using symmetry.

For $U_b$, the Hessian matrix is too complex to compute, even when we try to use some math tools. As a result, we follow [5] to compute the derivatives and Hessian matrix for bending energy. This paper discusses how to compute the Hessian matrix of bending energy in a simplified form.

Reviewer comment (Experiments):
The task descriptions lack detail. While the tasks are easy to understand at a high level, their actual definition is omitted, making it unclear what objective the various methods are required to optimize for. As there are many ways to formulate the described tasks, this information is important.

Author response:
**3.Actual definition of task objective**
Although we have detailed the reward function in Appendix B.2, we provide the exact formulas of the reward functions for further clarification.

**Lifting**: Let $B$ be the set of vertex positions on the block and $B^T$ be the target vertex positions, and $B[i]$ be the $i_{th}$ vertex in $B$.
$reward=-\sum_{i}\{(B[i].x - B^T[i].x)^2 + (B[i].y - B^T[i].y)^2 + (B[i].z - B^T[i].z)^2\}$

**Separating**: Let $B$ be the set of vertex positions on the block and $C$ be the set of vertex positions on the paper. Since there are $256$ vertices in the shell and $144$ vertices in the block, we multiply the sume of block vertices with $256 / 144$.
$reward=256 / 144 * \sum_{i}B[i].x - \sum_{i}C[i].x$

**Following**: $reward=-\sum_{i}B[i].x$

**Folding-U**: Let U be the set of rest angle (angle at rest position) on the upper crease and L be the set of rest angle on the lower crease.
$reward=\sum_{i}U[i] - \sum_{i}L[i]$

**Folding-L**: $reward=-\sum_{i}U[i] + \sum_{i}L[i]$

**Pick-Folding**: Let M be the set of rest angle (angle at rest position) on the middle of the paper.
$reward=\sum_{i}M[i]$

**Forming**: Let $S$ be the set of vertex positions on the shell and $S^T$ is the target vertex positions, and $S[i]$ be the $i_{th}$ vertex in $S$.
$reward=-\sum_{i}\{(S[i].x - S^T[i].x)^2 + (S[i].y - S^T[i].y)^2 + (S[i].z - S^T[i].z)^2\}$

**Sliding**: Let $S$ be the set of vertex positions on the bottom shell.
$reward=-\sum_{i}S[i].x$

**Bouncing**: Let $S$ be the set of vertex positions on the shell.
$reward=\sum_{i}S[i].z$

**Card**: Let $C$ be the set of vertex positions on the card.
$reward=-\sum_{i}C[i].x$

Reviewer comment (Experiments):
The experiments section does not clearly convey the goals. The only stated aim is to evaluate the performance and behavior of the simulator and methods. Being more concrete and explicitly stating the things to be investigated would improve readability.

Author response:
**5.Conveying the goal**

Our goal of the experiment section is to delve into task behavior to highlight the inherent challenges, and analyze the strengths and limitations of various methods in addressing these challenges. This analysis serves as  the groundwork for the future development of innovative algorithms for thin-shell manipulation tasks. 
We've add this to our main paper, thank you for metioning it.

Reviewer comment (Presentation):
The presentation of the results is incomplete. For example, Table 2 shows numbers with +/- values but does not explain what those values are; it can be assumed to be mean +/- standard deviation, but this should be clearly stated. One row has no +/- values without explanation, and some cells lack values entirely.

Author response:
**4.The presentation of the results**

The symbol $\pm$ in the table denotes the standard deviation resulting from multiple runs. Notably, the GD method exhibits no standard deviation as it lacks randomness in its multiple executions. In the context of inverse design tasks, the optimization is solely focused on the parameters. Consequently, we evaluate a distinct set of methods for these tasks, which explains the absence of certain rows in the table.

We will ensure to articulate these details more explicitly in the main paper. Thank you for bringing this to our attention.

Reviewer comment (Evaluation):
The results list score values for each of the tasks, but without knowing the level of "success" these scores correspond to, it is impossible to determine whether differences in scores are significant.

Author response:
**2.What is the level of success for each task**

As demonstrated in the General Response point 6, we integrate the theoretically highest scores with human-intuitive policies to derive the success score for each task. The success rate is then computed based on the mean and standard deviation for the CMA-ES + GD method.

Reviewer comment (Presentation):
The presentation of the real-world experiments fails to provide substantive information. This section should be improved to include actual experimental data instead of referring to an appendix with only a handful of plots, and should provide a discussion.

Author response:
**6.Real world experiment**

* **[Real-to-sim system identification]** We have introduced more detailed experimental results for the real-to-sim system identification tasks [Appendix C]. These results are analyzed through the observation of the 2D marker pose differences, presented qualitatively, and the loss curve, presented quantitatively. Although tactile simulation is not the primary focus of this paper, we argue that it demonstrates satisfactory alignment with the real world.
We have encountered challenges in accurately retrieving small-scale bending stiffness from the real world using tactile sensors. This difficulty arises from the sensitivity of results to the Poisson ratio of the tactile material. We deem that a better way to estimate small-scale bending stiffness is to first use vision method to obtain it's state and subsequently estimating it using the balance between gravitational and bending forces.

* **[Sim-to-real manipulation tasks]** In the real-world execution of the **Separating** task, our paper discusses that the bending stiffness of the thin shell significantly influences the final policy. Successfully deploying this policy to the real world serves as evidence of our effective approximation of the bending stiffness of the card paper.
We have also incorporated additional real-world manipulation results. In the **Folding** task, we employ a rigid manipulator with 3 Degrees of Freedom (DoFs) and train a new policy in the simulator, which is subsequently deployed into the real world. The outcome illustrates our success in capturing the dynamics of the paper and accomplishing the task in the real-world setting.

Reviewer comment (Novelty):
The paper describes a hybrid approach but provides extremely limited details. Given the focus on the simulator, it is unclear why the hybrid approach was included or what it adds.

Author response:
**7.Hybrid method**
In this work, we focus more on how to utilize the gradient in simulation to train an open-loop policy within limited time. While RL policy can generalize better to unseen state, open-loop methods employing differentiable physics prove to be more data-efficient. CMA-ES, GD, and our hybrid method all operate as open-loop trajectory optimization approaches. The concept of combining CMA-ES and GD with differentiable physics is proposed to alleviate the issue of local minima associated with the GD method in scenarios characterized by rich contact interactions, while design of RL policy considers no physics properties.
More details of this hybrid method is shown in the general response.

Reviewer comment (Novelty):
The idea of combining global search with local refinement is sensible but likely common knowledge in the RL community, given RL's unguided exploration and good performance once a solution is found.

Author response:
**7.Hybrid method**
In this work, we focus more on how to utilize the gradient in simulation to train an open-loop policy within limited time. While RL policy can generalize better to unseen state, open-loop methods employing differentiable physics prove to be more data-efficient. CMA-ES, GD, and our hybrid method all operate as open-loop trajectory optimization approaches. The concept of combining CMA-ES and GD with differentiable physics is proposed to alleviate the issue of local minima associated with the GD method in scenarios characterized by rich contact interactions, while design of RL policy considers no physics properties.
More details of this hybrid method is shown in the general response.

Reviewer comment (Experiments):
The benchmark aspect is unclear. The paper states the aim is for this work to be used as a benchmark, but it is not specified whether it should be similar to gym and Atari game setups or more like the benchmarks in the computer vision community, which use withheld data tasks to counteract overfitting.

Author response:
> **Q1: The paper states that the goal is for this to be used as a benchmark. Is the idea to use this similar to gym and atari game setups, or more like the benchmarks used in the computer vision community with tasks on withheld data to counteract overfitting on a set of tasks?**

It is similar to gym and atari game setups, where users can generate their own data and build their own experiment.

Reviewer comment (Evaluation):
What solution quality / success rate do the various rewards correspond to?

Author response:
> **Q2: What solution quality / success rate do the various rewards correspond to?**

As outlined in General Response point 6, we have established a set of scores to signify success in our benchmark. It is crucial to note, however, that we consider these tasks to be continuous rather than discrete. Our primary method of assessing success revolves around visualizations, and the success indicators are primarily employed for comparative purposes.

Reviewer comment (Evaluation):
How fast does the simulator run—real-time, faster, or slower?

Author response:
> **Q3: How fast does the simulator run, is it real-time, faster, slower?**

It's about 2.74 FPS as discussed in the General Response point 4.

Reviewer comment (Experiments):
What is the limit of object complexity that can be simulated?

Author response:
> **Q4: What is the limit of object complexity that can be simulated?**

We employ 3D meshes to simulate objects, enabling the simulation of any object that can be represented using a 3D mesh. It's important to note, however, that simulations may experience decreased speed and increased memory requirements when dealing with objects with a high number of vertices.