D$^2$CSG: Unsupervised Learning of Compact CSG Trees with Dual Complements and Dropouts
Authors: Fenggen Yu
Qimin Chen
Maham Tanveer
Ali Mahdavi Amiri
Hao Zhang
Venue: NeurIPS 2023 poster (2023)
Keywords: 3D reconstruction
constructive solid geometry
unsupervised learning
compact shape assembly

Abstract: We present D$^2$CSG, a neural model composed of two dual and complementary network branches, with dropouts, for unsupervised learning of compact constructive solid geometry (CSG) representations of 3D CAD shapes. Our network is trained to reconstruct a 3D shape by a fixed-order assembly of quadric primitives, with both branches producing a union of primitive intersections or inverses. A key difference between D$^2$CSG and all prior neural CSG models is its dedicated residual branch to assemble the potentially complex shape complement, which is subtracted from an overall shape modeled by the cover branch. With the shape complements, our network is provably general, while the weight dropout further improves compactness of the CSG tree by removing redundant primitives. We demonstrate both quantitatively and qualitatively that D$^2$CSG produces compact CSG reconstructions with superior quality and more natural primitives than all existing alternatives, especially over complex and high-genus CAD shapes.

D'CSG: Unsupervised Learning of Compact CSG Trees with Dual Complements and Dropouts
Fenggen Yu'
Qimin Chen' Ali Mahdavi Amiri! Simon Fraser University
Maham Tanveer' Hao Zhang 2 AAmazon
Abstract
Abstract: We present D²CSG, a neural model composed of two dual and comple- mentary network branches, with dropouts, for unsupervised learning of compact constructive solid geometry (CSG) representations of 3D CAD shapes. Our net- work is trained to reconstruct 3D shape by a fixed-order assembly of quadric primitives, with both branches producing a union of primitive intersections or inverses. A key difference between DDCSG and all prior neural CSG models is its dedicated residual branch to assemble the potentially complex shape complement, which is subtracted from an overall shape modeled by the cover branch. With the shape complements, our network is provably general, while the weight dropout further improves compactness of the CSG tree by removing redundant primitives. We demonstrate both quantitatively and qualitatively that D2CSG produces com- pact CSG reconstructions with superior quality and more natural primitives than all existing alternatives, especially over complex and high-genus CAD shapes.
Introduction
CAD shapes have played a central role in the advancement of geometric deep learning, with most
neural models to date trained on datasets such as ModelNet (5]). ShapeNet (2), and PartNet (33) for classification, reconstruction, and generation tasks. These shape collections all possess well-defined category or class labels, and more often than not, the effectiveness of the data-driven methods is tied to how well the class-spes dlas-ssec shape features can be learned. Recently, the emergence of datasets of CAD parts and assemblies such as ABC [2]] and Fusion360 [49 has fueled the need for learning shape representations that are agnostic to class labels, without any reliance on class priors. Case in point, the ABC dataset does not provide any category labels, while another challenge to the ensuing representation learning problem the rich topological varieties exhibited by the CAD shapes.
Constructive solid geometry (CSG) is a classical CAD representation; it models a 3D shape recursive assembly of solid primitives, e.g., cuboids, cylinders, etc., through Boolean operations including union, intersection, and difference. Of particular note is the indispensable role the difference operation plays when modeling holes and high shape genera, which are common in CAD. Recently, there have been increased interests in 3D representation learning using CSG [ 50 41 40, 19 57 3, striving for generality, compactness, and reconstruction quality of the learned models.
In terms of primitive counts, a direct indication of compactness of the CSG trees, and reconstruction quality, CAPRI-Net [57] represents the state of the art. However, it is not general neural model, e.g., it is unable to represent CAD shapes whose assembly necessitates nested difference operations (i.e.. needing subtract part that requires primitive differences build itself, e.g., see the CAD model in Fig. (2). Since both operands of the (single) difference operation in CAPRI-Net can only model intersections and unions, their network cannot produce a natural and compact CSG assembly for relatively complex CAD shapes with intricate concavities and topological details; see Fig.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Residual Shapes
Residual Shapes
Convexes Only
General Shapes
CAPRI-Net Reconstruction
D'CsG Reconstruction
Figure 1: Comparing CSG trees and shape reconstructions by network, DDCSG, and CAPRI- Net, the current state of the art. To reproduce the ground truth (GT) shape, a natural CSG assembly necessitates difference operation involving complex residual shape, which DDCSG can predict with compactness (only three intermediate, general shapes) and quality, CAPRI-Net can only build it using union of convexes, requiring unnecessarily many primitives and leading to poor reconstruction.
In this paper, we present DDCSG, novel neural network composed of two dual and complementary branches for unsupervised learning of compact CSG tree representations of 3D CAD shapes. As shown in Fig. our network follows a fixed-order CSG assembly, like most previous unsupervised CSG representation learning models [57 6, 3]. However, one key difference to all of them is the residual branch that is dedicated the assembly of the potentially complex complement or residual shape. In turn, the shape complement is subtracted from an overall shape that is learned by the cover branch. Architecturally, the two branches are identical, both constructing a union of intersections of quadric primitives and primitive inverses, but they are modeled by independent network parameters and operate on different primitive sets. With both operands the final difference operation capable of learning general CAD shape assemblies, our network excels at compactly representing complex and high-genus CAD shapes with higher reconstruction quality, compared the current state of the To improve the compactness further, we implement dropout strategy over
art, as shown in Fig. | network weights to remove redundant primitives, based on an importance metric.
Given the challenge of unsupervised learning of CSG trees amid significant structural diversity among CAD shapes, our network is not designed to learn a unified model over a shape collection. Rather, it overfits to given 3D CAD shape by optimizing a compact CSG assembly of quadric surface primitives to approximate the shape. The learning problem is still challenging since the number, selection, and assembly of the primitives are unknown, inducing complex search space.
In contrast to CAPRI-Net, our method is provably general, meaning that any CSG tree can be converted into an equivalent DºCSG representation. Our dual branch network is fully differentiable and can be trained end-to-end with only the conventional occupancy loss for neural implicit models [4] 3. [57. We demonstrate both quantitatively and qualitatively that our network, when trained on ABC 21 ShapeNet (2) produces CSG reconstructions with superior quality, more natural trees, and better
trade-off than all existing alternatives 3 6 57
2 Related Work
In general, 3D shape can be represented as set of primitives or parts assembled together. Primitive fitting to point clouds has been extensively studied 22 26 [4].. For shape abstraction, cuboid 44 [54) and super quadratics
(38) have been employed, while 3D Gaussians were adopted for
template fitting [2). From single image, cuboids have also been used to estimate object parts and
their relations using a convolutionall-reuree
auto-encoder [34]. More complex sub-shapes have
been learned for shape assembly such as elementary 3D structures [8) implicit convex [7], and neural
star components [20] as well as parts in the form of learnable parametrie patches [T]. moving or
deformable primitives 30 58 55 37 point clouds 228. or a part-aware latent space [10 However,
none of these techniques directly addresses reconstructing a CSG tree for a given 3D shape.
Deep CAD Models. Synthesizing and editing CAD models is challenging due to their sharp features and non-trivial topologies. Learning-basee shape programs have been designed to perform these tasks by providing easy-to-use editing tools 43 17 common for CAD modeling and there have been attempts reverse engineer such representations given input mesh or point cloud [52]. For example, BRepNet (23) UV-Net [15), and SBGCN [16]
Boundary Representations (B-Reps) are

---

Convex Primities
W
S
Intersection
Union
Residual Branch
Difference
W
Final Shape
Given Shape
Guassian Noise
Feature
Primtiive
Vector
Prediction
Primitive Inverses
Residual
Intersection
Lnnion
Figure 2: For given 3D shape S (shown the right end), network DCCSG is trained to optimize both its network parameters and a feature code to reconstruct S by optimizing an occupancy loss. The network parameters define CSG assembly The assembly is built using two identical branches: a cover branch (top), producing shape Sc, and a residual branch (bottom), producing shape Sn- After applying intersections and union to obtain the cover shape Sc and residual shape SR. by optimizing their respective occupancy losses, the recovered shape obtained via difference operation. dropout is further applied to the parameters in the intersection and union layer for removing redundant primitives and intermediate shapes.
set of quadric primitives and primitives inverses.
offer encoder-based networks to work with B-Reps and their topological information through message passing for CAD model analysis. NH-Rep [14] proposed several hand-crafted rules infer CSG trees from B-Reps. However, the leaf nodes of their CSG trees are not parameterized primitives but implicit surfaces. Extrusion-basee -basd techniques [50
53 39 have also been used in deep-learning-bant -ased CAD shape generation, but extrusions do not provide general shape representations and the ensuing supervised learning methods [48 50
are restricted to specific datasets.
Learning CSG. Learning CSG representations, e.g., primitive assembly 47 and sketch analy- sis 25], has become an emerging topic of geometric deep learning. While most approaches are supervised, e.g., SPFN [26). ParseNet [41], DeepCAD (50), ComplexGen [13], Lambourne et al. [22) and Point2Cyl [45], there have been several recent attempts at unsupervised CSG tree reconstruction, especially under the class-agnostic setting, resorting neural implicit representations 4 31 36 3
InverseCSG 99] is another related method, which first detects a collection of primitives over the sur- face of input shape using RANSAC, then performs search find CSG sequences to reconstruct the desired shape. However, RANSAC requires meticulous parameter tuning and the initially detected primitives can be noisy. Additionally, the space of CSG sequences is vast, leading lengthy search process. CSG-Net [40] employs a GRU-based decoder to generate CSG programs and resorts to RL find the optimal CSG sequence for target shape. However, the large search space for their beam search limits the method's 's ability to reconstruct complex target shapes. PLAD [18] follows CSG-Net and improves the program training with pseudo-labels and approximate distributions, but suffers from similar limitations as CSG-Net due to the use of similar program grammars and sequential network designs. UCSG-Net 19 learns reconstruct CSG trees with arbitrary assembly orders. Such a task is difficult due to the order flexibility, but it can be made feasible by limiting the primitives to boxes and spheres only. More success terms of reconstruction quality and compactness of the CSG trees has been obtained by learning fixed-order assemblies, including DCCSG.
BSP-Net 3 learns plane primitives whose half-spaces assembled via intersections obtain convexes, followed by a union operation, to reconstruct concave shapes. CAPRI-Net (57) extends BSP-Net by adding quadric surface primitives and difference operation after primitive unions. Similar to DCCSG, CAPRI-Net [57] also tries to perform a final shape difference operation by predicting set of primitives that undergo a fixed order of intersection, union, and difference operations to obtain the target shape. However, it constrains all primitives to be convex and its sequence is not general to support all shapes. To support generality, the set of primitives in D2CSG includes both convex primitives and complimentary primitives (see Section 3.1. This enables integration of difference operations during the initial phases of the CSG sequence. See the supplementary material for formal proof of the generality of D&CSG. CSG-Stump (6) also follows fixed CSG assembly order while including an inverse layer to model shape complements. The complement operation helps attain generality of their CSG reconstructions, in theory, but the inverse

---

Table I: Comparing various aspects of CAPRI-Net [57] and DºCSG.
CAPRI [57]
DCCSG Convex quadries and their inverses
Primitives
Convex quadrics
# Difference operation
One
Multiple
Generalization
Certain shapes
All shapes
CSG layer parameters
#64K
=32K
Training time per shape
R33 mins
R:5 mins
layer non-differentiable and the difference operations can only be applied basic primitives, which can severely compromise the compactness and quality of the reconstruction. ExtrudeNet [39 follows the same CSG order as [6. but changes the modeling paradigm to extrusion to improve edibility. Later, SECAD [27] employs 2D implicit fields to produce more accurate 2D profiles for extrusion. The main issue with both methods the lack of generality of the extrusion cerruddo---bsedd shapes.
Overfit models. Learning an overfit model to single shape is not uncommon in applications such as compression [5), reconstruction [46). and level-of-details modeling [42] due the high-quality
results it can produce. The underlying geometry of a NeRF is essentially an overfit (i.e., fixed) to
the shape/scene although the primary task of NeRF is novel view synthesis [56] 32 Following
similar principle, to replicate fine geometric details, D*CSG constructs a CSG tree for given object via optimizing a small neural network along with randomly initialized feature code (Fig. 2 We
follow this optimization/overfttingg procedure as we did not find learning a prior on CAD shapes very useful and generalizable due to the structural and topological diversity in the CAD shapes.
3 Method
CSG involves performing operations on a set of primitives (leaves of a CSG tree), resulting in the creation of intermediate shapes that are then merged form the final shape. In this work, sampled from a Gaussian distribution, the input to D²CSG is a feature vector that is optimized along with the network's 's parameters to reconstruct a given 3D shape and the output is set of occupancy values that are optimized to fit S (similar to Auto-decoder in DeepSDF (36).. We avoid encoding shape to a latent code since it does not help to converge more accurately or faster (supplementary material). Our design is efficient as DCCSG uses a light network converging quickly to each shape.
As shown in Fig. 2 starting from feature vector, first pass it to the primitive prediction network to generate two matrices Pc and P that hold the primitives' parameters. Each matrix is used to calculate the noint-to-primitive approximated signed distance (ASD) matrix De and DR for query points sampled near the target shape. These two matrices are separately passed to the cover and residual branches, each having an intersection and a union layer. The two branches generate point occupancies ac and ag with respect to the cover and residual shapes Se and SR. respectively. The final shape obtained as the difference between Sc and Sg. We adopt the multi-stage training of CAPRI-Net [57] to achieve accurate CSG reconstructions. Furthermore, since there is no explicit compactness constraint, many redundant primitives and intermediate shapes exist in the predicted CSG sequence. To this end, we introduce a novel dropout operation on the weights of each CSG layer to iteratively reduce redundancy and achieve higher level of compactness.
Comparison to CAPRI-Net. The most relevant work to our method is CAPRI-Net 57). which assembles set of quadric primitives via fixed order of intersection, union, and difference operations We adopt the same equations as CAPRI-Net to define the convex quadric primitives and CSG operations. However, as highlighted in Table ! our network, D2CSG, differs from CAPRI-Net in several aspects. First, D'CSG uses additional inverse primitives to support nested difference operations, which CAPRI-Net cannot handle. In general, despite of the fixed-order CSG operations, our network can reproduce the result of any CSG assembly, unlike CAPRI-Net, while using fewer network parameters. In addition, the new dropout design leads to more compact CSG assemblies.
3.1 Primitive Representation
The implicit quadric equation from CAPRI-Net 57 can cover frequently used conver primitives, e.g., planes, spheres, etc. However, its fixed CSG order (intersection, union, and difference) is not

---

general to combine the convexes to produce any shape; see proof in supplementary material. With D'CSG, we introduce a more general primitive form to also cover inverse convex primitives, so that the difference operation can be utilized as the last, as well as the first, operation of the learned CSG sequence. This is one of the key differences between our method and CAPRI-Net [57], which only produces convex shapes from the intersection layer. Our primitive prediction network (an MLP) receives a code of size 256 and outputs two separate matrices Pc E 12x77 and PR E RP*7 (for cover and residual branches), each contains parameters of p primitives (see Fig. 2 We discuss the importance of separating Pc and PR in the next section. Half of the primitives in Pa and PR are convex represented by quadric equation: |a|z² + |bly² + |c|=² + dr ey + f + g 0.
Complementary Primitives. In DºCSG, the other half of the primitives in Pc and R are inverse convex primitives with the first three coefficients to be negative:
|a|2² |bly² |c|= + dr ey + f2
We avoid using general quadric form without any non-negative constraints since it would produce
complex shapes not frequently used in CAD design. For reconstruction, n points near the shape's surface are sampled and their ASD to all primitives is calculated similarly CAPRI-Net [57] For
each point q = ( Mj. 2,), its ASD is captured in matrix D E TkxxP as: Dj(;,)) Q P
and D R R(..)) :) = Q P) to where Q(,;:) = (a) u7. 23 T Ms 2j 1) is the jth row of Q.
3.2 Dual CSG Branches
Previous CSG learning approaches employ a fixed CSG order and do not apply difference operation as the last operation as opposed to CAPRI-Net [57] and D2CSG, Applying the difference as the last operation helps produce complex concavity for the reconstructed shape [57]. However, CAPRI-Net utilizes shared primitive set for the two shapes that undergo the final subtraction operation. This sharing between the two operands has adverse effects the final result. To backpropagate gradients through all the primitives, values in the selection matrix T in the intersection layer are float in the early training stage (Equation 2 and Table 2). Therefore many primitives simultaneously belong to cover and residual shapes even though their associated weights might be small in T. Thus, the primitives utilized by the residual shape may impact the cover shape and result in the removal of details, or the opposite may also occur. Our dual branch in DCCSG, however, allows the cover and residual shapes to utilize distinct primitives, enabling them to better fit their respective target shapes.
Now, we briefly explain our dual CSG branches using similar notations of BSP-Net [3]. We input the ASD matrix De into the cover branch and D into the residual branch, and output vector ac and an, indicating whether query points are inside/outside cover shape Sc and residual shape Sp. Each branch contains an intersection layer and union layer adopted from BSP-Net (3.
During training, the inputs to intersection layers are two ASD matrices DR RTxP and De E Rx Primitives involved in forming intersected shapes are selected by two selection matrices Te Roxe and TR E RRxx, where e is the number of intermediate shapes. The two selection matrices To and Tn are set as learnable weights in the intersection layer. We can obtain point-to-intermediate--hh indicator Con E R by the intersection layer and only when Con(j, i) = O, query point qj is inside the intermediate shape (Conæ is the same and only subscripts are R):
in,
Cong TD
0 out.
Then all the shapes obtained by the intersection operation are combined by two union layers to find the cover shape Se and residual shape Sn. The inside/outside indicators of the combined shape are stored in vector an and ac E Rax1 indicating whether point is in/outside of the cover and residual shapes. We compute ac and ag in a multi-stage fashion (a* and a" for early and final stages) as [57] Specifically, ac and a are obtained by finding min of each row of Cong and Cong!
0
in,
at(( = min (J)
(3)
ISiSo
0 out.
Since gradients can only be backpropagated to the minimum value in the min operation, we addition- ally introduce a at the early training stage to facilitate learning by the following equation:
5

---

Table 2: Settings for multi-stage training.
Stage
Intersection (T)
Union (W)
Difference Op (a)
Dropout
Loss
0
float float
float
a
Liec + Lr Lw
1
a"
Lioc + Lr
2
binary
binary
a"
Lyec
at(1) = 86
Σ
Wc(1)8(1 Conc(i,
~ in,
2 out.
Isise
Wc E Rs is learned weighting vector and & is a clip operation to [0, 1]. a is defined similarly.
3.3 Loss Functions and Dropout in Training
Our training process is similar to that of CAPRI-Net [57 in terms of staging. The main motivation behind this is to ensure that the values in the CSG layer weights gradually change from float values to binary values, while facilitating better gradient propagation for learning the CSG layer weights in the early stage. While the early stages (0 - 1) of our training and loss functions are the same as CAPRI-Net, stage which a crucial part of our training due the injection of novel dropout, is different. In Table 2
we provide an overview of each stage of training and loss functions, with more details presented below and also in the supplementary material.
Stage 0. At stage 0, T in the intersection layer and W in the union layer are float. The loss function is L+ Lfec LT Lw, where LT and Lw serve to limit the entries of T to be between 0 and 1, and the entries of W to be almost 1. This setup would help the union layer avoid using the min operation in Equation 8 at stage 0 so that all the intermediate shapes could have gradients. Specifically,
1/. LT= Σ max + max(t - 1,0), Lw Eiw. -W. TET
(5)
It is worth noting that we have separate network weights for the dual CSG branches: T = [Tc, Tal and W = [Wo, W R). and they are trained separately. Our reconstruction loss Liec a combination of the cover loss L and the residual loss h. which are defined below.
_ 1 Mc (g(j) a)))
(6)
1 Mn(j) ((1 - -()) a²))
(7)
where g denotes the ground truth (GT) point occupancy values. Mc max(g(i), 1(a
and MR(j) mgx(g()). (aa)()) B)), where 3 is set to be 0.5 to signify the shape surface at stage 0. The function 1 maps Boolean values to float, while Mc and M adjust the losses on each branch with respect to the value of the other branch. In the supplementary material, we provide visualization examples to help comprehend these various notations.
Stage 1. Here, the vector a is substituted with a" to eliminate the use of W from stage 0. This adopts min as the union operation for the cover and residual shapes and restricts the propapptonn of gradients solely to the shapes that underwent union operation. Note that W is not updated at this stage. The loss function is L = Liec + Lr, where Ly is the same as in stage O, and Liee is the reconstruction loss: Li
LE + Līg- The cover loss L and residual loss L are defined as:
1
LE
Mcli) * [(1 g(i)) (1 ac(i)) + wc g() ab(i),,
(8)
n
L% MR(j) * (g() (1 a)( + w * (1 - g())) * az10)

---

= ma)( ((a)())<<)) g is the GT point occupancy value operating as a mask, and ? = 0.01 is close the inside value 0. We choose to use the T.1 loss function in stage I so that the different weights can be applied to encourage utilization of the difference operation. In addition, we set wic = 10 and wn 2.5, as in CAPRI-Net [57. to encourage the cover and residual shapes to fulfill their roles in the reconstruction.
Here, Mc max( (aa ())))
Stage 2. In stage 2. entries in T' are quantized into binary values by a threshold, Binory (t m)?1 : O, where 7) = 0.01. This way, the network will perform interpretable intersection operations on primitives. Due to the non-differentiable nature of the Binery values, the loss term LT is no longer utilized, resulting in the reduction of the loss function to only Li (see Table
Dropout in Stage 2. Without an explicit compactness constraint or loss, redundancies in our CSG reconstruction are inevitable. To alleviate this, we make two crucial observations: first, the CSG sequence learned at stage 2 is entirely interpretable, allowing us track the primitives and CSG operations involved in creating the final shape; second, altering the primitives or CSG operations will consequently modify the final shape's 's implicit field. Therefore, we can identify less significant primitives and intermediate shapes if their removal does not substantially impact the final result. Thus, we introduce the importance metric AS measure the impact of removing primitives intermediate shapes on the outcome. We first define the implicit field s* of the final shape as follows:
{0
in,
s' (j) max
max (a) a an(())
{>0 0 out,
(10)
where s" (j) signifies whether query point qi is inside or outside the reconstructed shape. It is important to note that a should be small and positive ( 0.2), as s* approaches O when points are within the shape. Then the importance metric AS is defined as:
n
AS = ESI(s &
(11)
j=1
Here, < 인 quantizes implicit field values 8 (j) from float to Boolean, where inside values are 1 and outside values are O. Function 1 maps Boolean values to integers for the sum operation, A captures the difference in values in case of eliminating primitives or intermediate shapes.
Note that union layer parameters W are set close to 1 by Tw in stage O. During stage 2, at every 4,000 iterations, removing the intermediate shape from the CSG sequence makes AS smaller than threshold a, we would discard intermediate shape by making W, = 0. Consequently, Equation (6)
will be modified incorporate the updated weights in the union layer:
{ 0 out.
in,
a) = min (Concli, (1- Isise
(12)
This way, the removed shape will not affect a" as long as is large enough. our experiments, we discover that 0 just needs to be larger than so that the removed shape i will not affect s*. Thus we set as 100 in all experiments. Furthermore, we apply dropout to parameters in the intersection layer by setting Tx. = 0 if AS falls below a after removing primitive & from the CSG sequence. We set as 3 in experiments and examine the impact of a in the supplementary material. After dropouts, the network parameters will be tuned. We iteratively perform this process until the maximum number of iterations has been reached or when no primitives or intermediate shapes are dropped.
4 Results
In our experiments to be presented in this section, we set the number of maximum primitives as p 512 and the number of maximum interscctions as c 32 for each branch support complex shapes. The size of our latent code is 256 and a two-layer MLP is used to predict the parameters of the primitives from the input feature code. We train DCCSG per shape by optimizing the latent code, primitive prediction network, intersection layer, and union layer. To evaluate DºCSG against prior methods, which all require an additional time-consume consuminn optimization at test time achieve satisfactory results (e.g., 30 min per shape for CSG-Stump). we have randomly selected a moderately sized subset of shapes as test set for evaluation: 500 shapes from ABC, and 50 from each of the 13 categories of ShapeNet (650 shapes in total). In addition, we ensured that 80% of the selected shapes from ABC have genus larger than two with more than 10K vertices to include complex structures. All experiments were performed using Nvidia GeForce RTX 2080 Ti GPU.
7

---

STUMP
CAPRI
D'CsG
GT
Figure 3: Comparing CSG representation learning and reconstruction from 3D meshes in ABC dataset (columns 1-6) and ShapeNet (columns 7-12). Results are best viewed when zooming in.
4.1 Mesh to CSG Representation
Given a 3D shape. the task is to learn an accurate and compact CSG representation for it. To do so, we first sample 24,576 points around the shape's surface (i.e. with a distance up 1/64) and 4, 4,096 random points in 3D space. All 28,672 points are then scaled into the range [-0.5, 0.5]. these points along with their occupancy values are used to optimize the network.
We compare DºCSG with InverseCSG [9] (I-CSG), BSP-Net [3, CSG-Stump 6 and CAPRI- Net (57], which output structured parametric primitives. For fair comparison, optimize all of these networks with the same number of iterations. BSP-Net, CSG-Stump, and CAPRI-Net pre-trained the training provided in CAPRI-Net before optimization initialization. Note that CSG-Stump uses different network settings for shapes from ABC (with shape differences) and ShapeNet (without shape difference); we therefore follow the same settings in our comparisons. For InverseCSG. we adopt the RANSAC parameters used for most of shapes in its CAD shape dataset. For each shape reconstruction, BSP-Net takes about 15 min, I-CSG about 17 min, CSG-Stump about 30 min, and CAPRI-Net about 3 min to converge. In contrast, the training process for DCCG runs 12,000 iterations for each stage, taking about minutes per shape reconstruction.
achieve better
Evaluation Metrics. Quantitative metrics for shape reconstruction are symmetric Chamfer Distance (CD), Normal Consistency (NC), Edge Chamfer Distance 3 (ECD). For ECD, we set the threshold for normal cross products to 0.1 for extracting points close to edges. CD and ECD are computed on 8K sample points on the surface and multiplied by 1,000. addition, we compare the number of primitives #P to evaluate the compactness of shapes since all CSG-based modeling methods predict some primitives that are combined with intersection operations.
Evaluation and Comparison. We provide visual comparisons representative examples from the ABC dataset and the ShapeNet dataset in Fig. 3 Since InverseCSG only accepts high-quality CAD meshes input for its primitive detection algorithm and fails on ShapeNet, we only compare on the ABC dataset, more visual results can be found in the supplementary material. Our method consistently reconstructs based primitive detection algorithm in InverseCSG can easily produce noisy and inaccurate primitives, resulting many degenerated planes after CSG modeling. BSP-Net simply assembles convex shapes to fit the target shape and obtain less compact results without a differrnnce operation. CSG-Stump tends use considerably more difference operations to reconstruct shapes. This also causes the shapes' surface to be carved by many redundant primitives (i.e., lack of compactness). In addition, since CSG-Stump does not support difference operations between complex shapes, fails to reconstruct small holes or intricate concavities. As the intermediate shapes subtracted in CAPRI-Net's fixed CSG sequence are only unions of convex shapes, it fails to reproduce target shapes with complex concavities. DCCSG achieves the best reconstruction quality and compactness in all metrics on ABC dataset and ShapeNet compared to other methods, except for NC on ShapeNet where D2 underperformed ever slightly (A 0.003); see Table
accurate shapes with geometric details and concavities. The RANSAC

---

Table 3: Comparing CSG rep learning from 3D meshes in ABC and ShapeNet
ABC
ShapeNet
Methods
I-CSG
BSP
STUMP
CAPRI
Ours
BSP
STUMP
CAPRI
Ours
CD
0.576
0.115
0.383
0.177
0.069
0.164
2.214
0.124
0.119
NC f
0.877
0.921
0.850
0.903
0.928
0.882
0.794
0.890
0.887
ECD 6.330
4.047
8.881
3,990
3.091
3,899
6.101
2.035
1.722
#P
43.62
359.38
83.42
66.26
28.62
694.21
228.58
50.94
43.78
Table 4: Ablation study on key components of D&CSG: complementary primitives (CP), dual branches (DB), and dropout (DO), on three quality metrics and three compactness metrics, the number of CSG primitives (#P), intermediate shapes (#IS), and surface segments (#Seg) resulting from decomposition induced by the CSG tree. Winner in boldface and second place in blue.
Row ID
CP
DB
DO
CD L
NC
ECD
#P
#IS 4
#Seg
0.183
0.907
3.92
77
9.2
93
V
0.073
0.935
3.12
38
5.8
55
V
0.088
0.926
3.48
27
5.3
40
V
0.069
0.936
2.98
53
6.8
57
0.069
0.928
3.09
29
5.7
42
Table 5: Comparing CSG rep learning from 3D point cloud ABC and ShapeNet.
ABC
ShapeNet
Methods
BSP
STUMP
CAPRI
Ours
BSP
STUMP
CAPRI
Ours
CD L
0.133
0.695
0.225
0.085
0.268
1.177
0.242
0.224
NC1 1
0.919
0.841
0.894
0.924
0.889
0.840
0.888
0.886
ECD 4
3.899
7.303
3.308
3.029
1.854
3,482
1.971
1.815
#P 4
360.87
67.436
68.57
35.28
553.76
211.36
55.21
51.26
Ablation Study. We conducted ablation study (see Table assess the efficacy of, and tradeoff between, the three key features of DºCSG: complementary primitives, dual branch design, and dropout. As we already have three metries for reconstruction quality, we complement #P, the only compactness measure, with two more: the number of intermediate shapes and the number of surface segments result of shape decomposition by the CSG tree; see more details on their definitions in the supplementary material. We observe that incorporating complementary primitives (row 2) enables better generalization to shapes with complex structures and higher accuracy than the CAPRI-Net (57) baseline (row 1). The dual branch design further enhances reconstruction accuracy (row 4 vs. 2). as primitives from the residual shape do not erase details the cover shape, and each branch can better fit its target shape without impacting the other branch. However, the dual branch design did compromise compactness. Replacing dual branching by dropout in the final stage (row 3 vs. 4) decreases the primitive count by about 50% and improves on the other two compactness measures, while marginally compromising accuracy. The best trade-off is evidently achieved by combining all three components, as shown in row 5, where adding the dual branches has rather minimal effects all three compactness metrics with dropouts. In the supplementary material, we provide additional ablation studies. We also examine the impact of employing basic primitives, well modifying the maximum primitive count and the number of sampled points.
4.2 Applications
Point Clouds to CSG. We reconstruct CAD shapes from point clouds, each containing 8,192 points. For each input point, we sample points along its normal with perturbations sampled from Gaussian distribution (μ = 0, o 64). If this point at the opposite direction of normal vectors, the occupancy value is 1, otherwise it is 0. This way, we sample 65, 536 points to fit the network to each shape and other settings the same as mesh-to-CSG experiment. Quantitative comparisons in Table 5 visual comparisons in Fig. show that our network outperforms BSP-Net, CSG-Stump, and CAPRI-Net in different reconstruction similarity and compactness metrics on ABC dataset and ShapeNet. More results can be found in the supplementary material.

---

Input
STUMP
CAPRI
D'CSG
GT
Figure 4: CSG reconstruction from 3D point cloud ABC (Col. 1-6) and ShapeNet (Col. 7-12).
Cylinder Redius 0.073 0:173
Cylnder Box
Cube Translation (0:16, 0,01 |2.00 0,0
Cylinder Removal
D'csG
OpenSCAD CSG Script
Ecit
Figure 5: D'CSG learns OpenSCAD scripts for a given shape and supports editability.
Shape Editing. Once we have acquired the primitíves and CSG assembly operations, we can edit the shape by altering primitive parameters, adjusting the CSG sequence, transforming intermediate shapes. To facilitate shape editing in popular CAD software, we further convert quadric primitives into basic primitives (e.g., cubes, spheres, cylinder, etc.) by identifying the best-fitting basic primitive for each predicted quadric primitive by our method. Subsequently, we can export the primitive parameters and CSG sequence into an OpenSCAD script. Fig. shows the shape editing results.
Discussion, Limitation, and Future Work
We present D²CSG, a simple yet effective idea, for unsupervised learning of general and compact CSG tree representations of 3D CAD objects. Extensive experiments on the ABC and ShapeNet datasets demonstrate that our network outperforms state-of-the art methods both in reconstruction quality and compactness. We also have ample visual evidence that the CSG trees obtained by our method tend to be more compact than those produced by prior approaches.
Our network does not generalize in essence an optimization to find a CSG assembly. While arguably limiting, this not entirely unjustified since some the models the ABC dataset may not possess sufficient generalizability in their primitive assemblies. Another limitation is that our current quadric primitive representation cannot fully support complex surfaces, such as tori NURBS. would be an interesting future direction to change quadric primitives to NURBS or other easily editable primitives, such as those from extrusions. In addition, incorporating interpretable CSG operations into the network tends to cause gradient back poo-ppropggttion issues and limits the reconstruction accuracy of small details such as decorative curves on chair legs. We would also like to extend our method structured CAD shape reconstruction from images and free-form sketches. Another interesting direction for future work is to scale the primitive assembly optimization from CAD parts to indoor scenes.
shape collection; it "overfits"
single input shape and
Acknowledgement
We thank all the anonymous reviewers for their valuable comments, and KOKONI3D (httss:/www.kkooonll) for the helpful discussions. This work was supported in part by an NSERC Discovery Grant (No.611370) and industrial gift funds from Adobe and Autodesk.
10

---

References
[1] Dan Cascaval, Mira Shalah, Phillip Quimn, Rastislav Bodik, Maneesh Agrawala, and Adriana Schulz. Differentiable 3d cad programs for bidirectional editing. Computer Graphics Forum, 41(2):00--323, 2022. Angel X. Chang. Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song. Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu. ShapeNet: An information-rich -ricc 3D model repository, arXiv preprint urXiv:1512.03012 2015.
[3] Zhiqin Chen, Andrea Tagliasacchi, and Hao Zhang. Bsp-net: Generating compact meshes vis binary space partitioning. In CVPR, pages 45-54, 2020.
[4] Zhiqin Chen and Hao Zhang. Learning implicit fields for generative shape modeling, In CVPR, pages
5939-5948, 2019.
[5] Thomas Davies, Derek Nowrouzezahrai, and Alec Jacobson. Overfit neural networks as a compact shape representation. arXiv preprint (eti::20000998000 2020.
[6] Jianmin Zheng Daxuan Ren. Jianfei Cai. Haiyong Jiang Jiatong Li, Zhongang Cai, Junzhe Zhang, Liang
Pan, Mingyuan Zhang, Haiyu Zhao, and Shuai Yi. CSG-Stump: A learning friendly osg-like representation for interpretable shape parsing. In ICCV, pages 12458-12467, 2021.
[7] Boyang Deng, Kyle Genova, Soroosh Yazdani, Sofien Bouaziz, Geoffrey Hinton, and Andrea Tagliasacchi.
Cvxnet: Learnable convex decomposition. In CVPR, pages 31-44, 2020.
[8] Theo Deprelle. Thibault Groueix, Matthew Fisher, Vladimir Kim, Bryan Russell, and Mathieu Aubry. Learning elementary structures for 3d shape generation and matching. Advances in Neural Informaiin Processing Systems, 32, 2019.
[9] Tao Du, Jeevana Priya Inala, Yewen Pu, Andrew Spielberg, Adriana Schulz, Daniela Rus, Armando Solar-Lezama, and Wojciech Matusik. Inversecsg: Automatic conversion of 3d models to esg trees. ACM Trans. on Graphicy (TOG), 37(6):1-16. 2018.
[10] Anastasia Dubrovina, Fei Xia, Panos Achlioptas, Mira Shalah, Raphaël Groscot, and Leonidas J Guibas. Composite shape modeling via latent space factorization. In ICCV, pages 8140-8149, 2019.
[11] Kevin Ellis, Maxwell Nye, Yewen Pu. Felix Sosa, Josh Tenenbaum, and Armando Solar-Lezama. Write, execute, assess: Program synthesis with a repl. Advances in Neural Information Processing Syxtems, 32, 2019.
[12] Kyle Genova, Forrester Cole, Daniel Vlasic, Aaron Sarna, William T Freeman, and Thomas Funkhouser. Learning shape templates with structured implicit functions. In CVPR, pages 7154-7164, 2019.
[13] Haoxiang Guo, Shilin Liu, Hao Pan, Yang Liu, Xin Tong. and Baining Guo. Complexgen: Cad re- construction by b-rep chain complex generation. ACM Transactions on Graphics (TOG), 41(4):1-18,
2022.
[14] Hao-Xiang Guo. Liu Yang, Hao Pan. and Baining Guo. Nh-rep: Neural halfspace representations for
implicit conversion of b-rep solids. ACM Transactions on Graphics (TOG), 2022.
[15] Pradeep Kumar Jayaraman, Aditya Sanghi, Joseph G Lambourne, Karl DD Willis, Thomas Davies, Hooman Shayani, and Nigel Morris. Uv-net: Learning from boundary representations. In CVPR, pages
11703-11712, 2021.
[16] Benjamin Jones. Dalton Hildreth, Duowen Chen, Ilya Baran, Vladimir G Kim, and Adriana Schulz. Automate: A dataset and learning approach for automatic mating of cad assemblies. ACM Trans. on Graphics (TOG), 40(6))1-18, 2021.
[17] R. Kenny Jones, Theresa Barton, Xianghao Xu, Kai Wang, Ellen Jiang, Paul Guerrero, Niloy Mitra, and Daniel Ritchie Shapeassembly: Learning generate programs for 3d shape structure synthesis. ACM Transactions on Graphics (TOG). Siggraph Asia 2020, Artiice 234, 2020.
[18] R. Kenny Jones, Homer Walke, and Daniel Ritchie. Plad: I learning to infer shape programs with pseudo- labels and approximate distributions. The IEFE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
[19] Kacper Kania, Maciej Zieba, and Tomasz Kajdanowicz. ddss--netunsuupervieed discovering of constructive solid geometry tree. Advances in Neural Information Processing Systems, 33:8776-877 77786 2020. [20] Yuki Kawana, Yusuke Mukuta, and Tatsuya Harada. Neural star domain as primitive representation. Advances in Neural Information Processing Systems, 33:7875-7886, 2020.
[21] Sebastian Koch, Albert Matveev, Zhongshi Jiang. Francis Williams, Alexey Artemov, Evgeny Burnaev, Mare Alexa, Denis Zorin, and Daniele Panozzo. ABC: A big cad model dataset for geometric deep learning. In CVPR, pages 9601-9611, June 2019.
[22] Joseph George Lambourne, Karl Willis, Pradeep Kumar Jayaraman, Longfei Zhang, Aditya Sanghi, and Kamal Rahimi Malekshan. Reconstructing editable prismatic cad from rounded voxel models. In SIGGRAPH Asia 2022 Conference Papers, pages 1-9, 2022.
[23] Joseph G Lambourne, Karl DID Willis, Pradeep Kumar Jayaraman, Aditya Sanghi, Peter Meltzer, and Hooman Shayani. Brepnet: A topological message passing system for solid models. CVPR, pages 12773-12782, 2021.
[24] Eric-Tuan Lē, Minhyuk Sung, Duygu Ceylan, Radomir Mech, Tamy Boubekeur, and Niloy J Mitra, Cpfin:
Cascaded primitive fitting networks for high-resolution point clouds. In CVPR, pages 7457-7466, 2021.
[25] Changjian Li, Hao Pan, Adrien Bousseau, and Niloy J. Mitra. Free2cad: Parsing freehand drawings into
cad commands. ACM Trans. on Graphics (TOG), 41(4), 2022.
[26] Lingxiao Li. Minhyuk Sung. Anastasia Dubrovina. Li Yi. and Leonidas J Guibas. Supervised fitting of geometric primitives to 3d point clouds. In CVPR, pages 2652-260222 2060666600000 2019.
11

---

[27] Pu Li, Jianwei Guo, Xiaopeng Zhang, and Dong-Ming Yan. Secad-net: Self-supervised cad reconstruction by learning sketch--xtrude operations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 16816-16826. 2023.
[28] Yichen Li, Kaichun Mo, Lin Shao, Minhyuk Sung, and Leonidas Guibas. Learning 3d part assembly from single image. ECCV. pages 664-682. Springer, 2020.
[29] Yangyan Li, Xiaokun Wu, Yiorgos Chrysathou, Andrei Sharf, Daniel Cohen- Or, and Niloy Mitra. Globfit:
Consistently fitting primitives by discovering global relations. ACM Trans. on Graphics (TOG), 30(4),
2011.
[30] Zhijian Liu, William T Freeman, Joshua B Tenenbaum, and Jiajun Wu. Physical primitive decomposition.
In ECCV, pages 3-19, 2018.
[31] Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy
networks: Learning 3D reconstruction in function space. In CVPR, pages 4460 4470, 2019.
[32] Ben Mildenhall. Pratul P Srinivasan. Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes neural radiance fields for view synthesis. Communications of the ACM.
65(1):99-106, 2021.
[33] Kaichun Mo, Shilin Zhu, Angel X. Chang, Li Yi, Subarna Tripathi, Leonidas J. Guibas, and Hao Su. PartNet: A large-scale benchmark for fine-grained and hierarchical part-level 3D object understanding. In CVPR, pages 90-918, 2019.
[34]
Chengjie Niu, Jun Li, and Kai Xu. 1m2struct: Recovering 3d shape structure from a single RGB image. In CVPR. 2018.
[35] Wamiq Para, Shariq Bhat, Paul Guerrero, Tom Kelly, Niloy Mitra, Lconidas J Guibas, and Peter Wonka.
Sketchgen: Generating constrained cad sketches. Advances in Neural Information Processing Systems, 34:5077-5088, 2021.
[36] Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. DeepSDF: Learning continuous signed distance functions for shape representation. In CVPR, pages 165-174, 2019..
[37] Despoina Paschalidou, Angelos Katharopoulos, Andreas Geiger, and Sanja Fidler. Neural parts: Learning expressive 3d shape abstractions with invertible neural networks. In CVPR, pages 3204-3215, 2021.
[38] Despoina Paschalidou, Ali Osman Ulusoy, and Andreas Geiger. Superquadrics revisited: Learning 3d
shape parsing beyond cuboids. In CVPR, pages 00344-10353, 2019.
[39] Daxuan Ren, Jianmin Zheng, Jianfei Cai, Jiatong Li, and Junzhe Zhang. Extrudenet: Unsupervised inverse for shape parsing. In Computer Vision ECCV 2022: 17th European Conference, Tel
seech-anddexteee
Aviv, Israel, October 23-27, 2022, Proceedings. Part II. pages 482 482-498. Springer. 2022.
[40]
Gopal Sharma, Rishabh Goyal, Difan Liu, Evangelos Kalogerakis, and Subhransu Maji. CSGNet: neural
shape parser for constructive solid geometry. In CVPR. pages 5515-5523, 2018.
[41] Gopal Sharma, Difan Liu, Subhransu Maji, Evangelos Kalogerakis, Siddhartha Chaudhuri, and Radomír Mčch. Parsenet: A parametric surface fitting network for 3d point clouds. In pagss
pages 261-276, 2020. [42] Towaki Takikawa, Joey Litalien, Kangxue Yin, Karsten Kreis, Charles Loop, Derek Nowrouzezahrai, Alec Jacobson, Morgan McGuire, and Sanja Fidler. Neural geometric level of detail: Real-time rendering with implicit 3d shapes. In CVPR, pages 11358- 11377, 2021.
[43] Yonglong Tian, Andrew Luo, Xingyuan Sun, Kevin Ellis, William T Freeman, Joshua B Tenenbaum, and Jiajun Wu. Learning to infer and execute 3d shape programs, arXiv preprint (etp:9901.02800 2019.
[44] Shubham Tulsiani, Hao Su, Leonidas J. Guibas, Alexei A. Efros, and Jitendra Malik. Learning shape abstractions by assembling volumetric primitives, In CVPR, pages
2017.
[45] Mikaela Angelina Uy. Yen yu Chang, Minhyuk Sung, Purvi Goel, Joseph ambburre, Tolga Birdal, and Leonidas Guibas. Point2Cyl: Reverse engineering 3D objects from point clouds to extrusion cylinders. In CVPR, pages 11 (1850-18860 2022.
[46] Francis Williams, Teseo Schneider, Claudio Silva, Denis Zorin, Joan Bruna, and Daniele Panozzo. Deep
geometric prior for surface reconstruction. In CVPR, pages pages 10130-10133 2019.
[47] Karl D.D. Willis, Pradeep Kumar Jayaraman, Hang Chu, Yunsheng Tian, Yifei Li, Daniele Grandi, Aditya Sanghi, Linh Tran, Joseph G. Lamboume, Armando Solar-Lezama, and Wojeiech Matusik. JoinABLe: Learning bottom-up assembly parametric CAD joints. In CVPR, pages 15849-15860, 2022.
[48] Karl DI Willis, Yewen Pu, Jieliang Luo, Hang Chu, Tao Du, Joseph G Lambourne, Armando Solar- Lezama, and Wojeiech Matusik. Fusion 360 gallery: A dataset and environment for programmatic cad construction from human design sequences. ACM Transactions on Graphics (TOG), 40(4):1-24, 2021.
[49] Karl D. D. Willis, Yewen Pu, Jieliang Luo, Hang Chu, Tao Du, Joseph G. Lambourne, Armando Solar- Lezama, and Wojciech Matusik. Fusion 360 gallery: A dataset and environment for programmatic cad construction from human design sequences. ACM Trans. on Graphics (TOG), 40(4), 2021.
[50] Rundi Wu, Chang Xiao, and Changxi Zheng. Deepcad: A deep generative network for computer-aided design models. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages
6772-6782, 2021.
[51] Zhirong Wa, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In CVPR, pages 1912-1920, 2015.
[52] Xianghao Xu, Wenzhe Peng, Chin-Y Yi Cheng. Karl DD Willis, and Daniel Ritchie. Inferring cad modeling
sequences using zone graphs. In CVPR, pages 6062-60 6070, 2021.
[53] Xiang Xu, Karl DD Willis, Joseph G Lambourne, Chin- Yi Cheng, Pradeop Kumar Jayaraman, and Yasutaka Furukawa. Skexgen: Autoregressive generation of cad construction sequences with disentangled codebooks. arXiv preprint ar Xix: 04632,, 2022.
12

---

[54]
Kaizhi Yang and Xuejin Chen. Unsupervised learning for cuboid shape abstraction via joint segmentation from point clouds. ACM Transactions on Graphicy (TOG), 40(4))1-11, 2021.
[55]
Chun-Han Yao, Wei-Chih Hung. Varun Jampani, and Ming Hsuan Yang. Discovering 3d parts from image collections. In ICCV. pages 12981-12990. 2021.
[56]
Lior Yariv, Jiatao Gu, Yoni Kasten, and Yaron Lipman. Volume rendering of neural implicit surfaces. Advances Neural Information Processing Systems, 34:4805-48155 2021.
[57] Fenggen Yu, Zhiqin Chen, Manyi Li, Aditya Sanghi, Hooman Shayani, Ali Mahdavi-Amiri, and Hao Zhang. Capri-net: Learning compact cad shapes with adaptive primitive assembly. In CVPR, pages 11768-11778, 2022.
[58] Chuhang Zou, Ersin Yumer, Jimei Yang, Duygu Ceylan, and Derek Hoiem. 3d-prnn: Generating shape primitives with recurrent neural networks. In ICCV, pages 900 909, 2017.
13


Reviews:
[{"review_id": "wxS2CikGdN", "reviewer": "Reviewer_rfjZ", "summary": "The authors propose a novel, unsupervised method to reconstruct the CSG tree given a 3D shape. The authors prove that all CSG trees can be formulated with a boolean difference operation as the last step. Therefore, to generate the final reconstructed shape, the proposed method uses two branches, cover and residual, to produce the shapes used to keep and remove in the boolean difference operation. To further enhance the model's capability of generating all possible shapes, the inverse convex primitives are introduced to incorporate the boolean difference operations other than the last step. The proposed method also improves compactness using dropout. The results demonstrate significant improvement over prior works qualitatively and quantitatively. ", "questions": "1. Stages 0, 1, and 2 in the main text are not coherent with stages 1, 2, and 3 in Table 1. \n\n2. How does the proposed method perform on multiple objects (or objects with disconnected parts)?\n\n3. How do the point sampling density, quality, or methods affect the results?\n\n4. Have the authors tried generating CSG trees with depths larger than two (e.g., using an iterative approach)? Is there any way to justify whether the current approach is sufficient to generate any shapes? In other words, do shapes generated in each branch have enough complexity?", "limitations": "Yes, the limitations are addressed.", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. The proposed method significantly improves the reconstruction quality, which can be easily observed from the qualitative results in Fig 3.\n\n2. The paper is well-written and provides most of the required details (both in the main text and supplementary).\n\n3. The authors demonstrate that the model can reconstruct the CSG trees from general and complex 3D shapes in both ABC and ShapNet datasets, and provide shape editing capability for downstream CAD tasks. ", "weaknesses": "1. The generated CSG trees might not look natural to designers and engineers since they usually have some shared patterns and preferences when modeling 3D shapes. Moreover, the modeling sequences are often related to the design and manufacturing intents. Therefore, it would be great to see user study results comparing the generated CSG trees with real modeling sequences (e.g., Fusion 360 Gallery Reconstruction Dataset).\n\n2. Designers usually do not model 3D shapes simply using primitives. Most solid modeling tools in CAD are profile-based, meaning that users draw 2D profiles and use them to perform 3D operations such as extrude, revolve, sweep, chamfer, fillet, etc. Therefore, mapping the generated CSG trees into those operations will be necessary for practical use.\n\n3. Point cloud representation can be challenging when reconstructing large objects with small details, such as the camping car example in Fig. 3.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**The generated CSG trees might not look natural to designers.**\n\n**A:** We generally agree with the reviewer and there is definitely *more* that is left to do to get there. \n\nOne intrinsic difficulty is that there is no (single) ground truth for the CSG assembly of a given CAD shape. Each 3D shape could be represented by different modeling sequences, and different artists might have different design preferences: some may like the additive style, while others more used to the subtractive style. In addition, different modeling sequences also have different editing complexity for different modeling targets. Taking a more objective view, we currently focus on accounting for and striving towards the reconstruction accuracy and compactness of the CSG assembly when designing our network. \n\n**User study results comparing the generated CSG trees with real modeling sequences (e.g., Fusion 360 Gallery Reconstruction Dataset).**\n\n**A:** This is an interesting thought. However, the extrusion-based modeling sequences in Fusion360 are quite different from our primitive-based CSG assembly. The latter is our goal. We acknowledge that sketch-and-extrude is quite a natural editing and modeling paradigm for designers, e.g., see Google (Trimble) SketchUP. Learning those types of assemblies would involve a very difficult goal, e.g., see ExtrudeNet and SECAD-Net as recent attempts.\n\n* ExtrudeNet: Unsupervised Inverse Sketch-and-Extrude for Shape Parsing, ECCV 2022\n\n* SECAD-Net: Self-Supervised CAD Reconstruction by Learning Sketch-Extrude Operations, CVPR 2023\n\n**Designers usually do not model 3D shapes simply using primitives.**\n\n**A:** A fair point; see our remarks above on sketch-and-extrude. To this end, we believe that D$^2$CSG could serve as a valuable supplementary approach to existing modeling paradigms. By utilizing extrusions and sweep operations to create primitives, we can employ D$^2$CSG to construct even more intricate shapes. Introducing broader and more versatile primitives into the D$^2$CSG framework presents an intriguing avenue for future research. We intend to include this discussion in our paper.\n\n**Point cloud representation can be challenging.**\n\n**A:**  Yes, point clouds are not the best representations for details. One future work is to first use SOTA point cloud upsampling methods to produce details or use adaptive sampling to sample more points around details.\n\n**Q1: Stages 0, 1, and 2 in the main text are not coherent with stages 1, 2, and 3 in Table 1.**\n\n**A:** Thank you for pointing out this issue. We will address it in the revision.\n\n**Q2: How does the proposed method perform on multiple objects (or objects with disconnected parts)?**\n\n**A:** Our method is self-supervised and is robust to unseen shape structures and categories; see Figure 3. In particular, we use a shape-specific optimization technique, allowing the method to effectively accommodate diverse objects, including multi-part objects.\n\n**Q3: How do the point sampling density, quality, or methods affect the results?**\n\n**A:** Our method is no exception to the \"garbage in, garbage out\" concept. Less sampled points or low-quality points will produce worse results, see Table 1 in the PDF file attached to the global response.\n\n**Q4: Have the authors tried generating CSG trees with depths larger than two (e.g., using an iterative approach)? Is there any way to justify whether the current approach is sufficient to generate any shapes? In other words, do shapes generated in each branch have enough complexity?**\n\n**A:** In the supplementary material, we provide a proof that the current approach is able to produce any shape. Please see Proposition 2 and its proof in the supplementary material (line 35).\n\n"}, {"review_id": "hpdho372IS", "reviewer": "Reviewer_dmBa", "summary": "This paper proposed a method for unsupervised learning of CSG trees from mesh or point cloud. An auto-decoder approach is used, i.e., each training shape is represented by a learned latent code. Compared to previous approach CAPRI-Net, the proposed approach used complementary primitives and dual branch design to represent shapes in more complicated and accurate CSG primitives. Also, the proposed approach used dropout during training to avoid redundancy of the inferred CSG tree. The method is validated on ABC dataset and ShapeNet.", "questions": "- How is the T matrix predicted from the network?\n- The residual loss appears in the figure-2 of main paper, it will be helpful to move its definition to the main paper.\n- Many components of the paper is based on CAPRI-Net, it will be helpful to provide a compact background section for CAPRI-Net in the paper.", "limitations": "N/A", "rating": 6, "confidence": 2, "soundness": 4, "presentation": 2, "contribution": 3, "strengths": "- The idea of complementary primitives and dual branch are novel and effective.\n- The proposed method significantly outperforms previous method both qualitatively and quantitatively.\n- The experiment section is conducted extensively, which supports the claims well.", "weaknesses": "- Some important technical details are not clear (see questions).\n- It will be easier to follow the paper if there are some visualizations explaining the insights of complementary primitives and dual branch.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**It will be easier to follow the paper if there are some visualizations explaining the insights of complementary primitives and dual branch.**\n\n**A:** We showed additional visualization results for the ablation study, see Figure 2 in the PDF file attached to the global response.\n\n**Q1: How is the T matrix predicted from the network.**\n\n**A:** The T matrix is not predicted from the network but set as learnable parameters in the network.\n\n**Q2: The residual loss appears in the figure-2 of main paper, it will be helpful to move its definition to the main paper..**\n\n**A:** Thank you for pointing out this issue. We put details of the cover loss and residual loss in the supplementary material. We will mention how we use the cover loss and residual loss in the main paper.\n\n**Q3: Many components of the paper is based on CAPRI-Net, it will be helpful to provide a compact background section for CAPRI-Net in the paper.**\n\n**A:** We plan to add this compact background section to briefly introduce the components adopted from CAPRI-Net as space allowed.\n\n\n"}, {"review_id": "PnPobVrK84", "reviewer": "Reviewer_DMnr", "summary": "The paper proposes a reconstruction approach to building constructive solid geometry (CSG) from other 3D modalities like meshes and point clouds. The key contribution of the paper is a dual representation that considers both the shape and its complement that are built with Boolean intersection and union operations with a set of primitive convex quadric surfaces and their inverses. Two branches in a fully differentiable neural network are responsible for generating the shape and the complement that is Boolean subtracted to output the final CSG representation of a target shape. The dual representation is general and enables complex shapes to be captured well as demonstrated by extensive results.", "questions": "In the related work under Deep CAD Models, it appears that BRepNet, UV-Net and SB-GCN are attributed as reverse engineering models while these are all encoders.\n\nSome missing citations that might be worth adding:\n- https://dl.acm.org/doi/abs/10.1145/3528223.3530078\n- https://dl.acm.org/doi/abs/10.1145/3550469.3555424\n\nReferences [46] and [47] are duplicates\n", "limitations": "Yes", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The paper is well written and easy to follow. It offers a competitive solution to an important inverse problem in CAD, and the proposed representation is novel, general and backed by strong empirical results in two standard datasets. The design choices are well justified and evaluated. The decision to overfit to a target shape rather than attempting to generalize is sensible, but see also a related point in weaknesses.", "weaknesses": "One weakness of choosing an overfitting approach as opposed to learning from datasets is challenges related to robustness to noise and outliers. The method appears to be sensitive to how clean the input geometry is, and with noisy pointclouds or low resolution meshes, it is very likely for unintended sliver geometry to be constructed. This is apparent in Figure 4, although the proposed method does perform better than others. Some discussion on this would be beneficial in the limitations or future work section.\n\nAnother weakness is the choice of using quadric surfaces as primitives. CAD shapes are typically built with prismatic primitives like planes, cylinders, etc. and while quadric surfaces can represent most of such surfaces, optimizing in this representation is unlikely to reconstruct the exact prismatic primitives as seen in the results throughout the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**One weakness of choosing an overfitting approach as opposed to learning from datasets is challenges related to robustness to noise and outliers.**\n\n**A:**  Yes, unintended geometry could be produced for noisy point clouds or low-resolution meshes. This is a general issue for overfitting-based methods. Incorporating D$^2$CSG with other point cloud denoising or upsampling modules would consititute interesting future work for robust CSG reconstruction. We can add this discussion in the revision. \n\n**Another weakness is the choice of using quadric surfaces as primitives.**\n\n**A:**  We did discuss the limitation of quadric primitives in the conclusion. CAD shapes are not only constructed by prismatic primitives but also by other complex primitives, such as NURBS. However, we consider D$^2$CSG as a key step toward learning general and compact CSG assembly sequences. The quadric primitives in D$^2$CSG can be easily changed to other primitives, and we find quadric primitives can achieve the best fitting results compared to prismatic primitives; see Table 2 in the supplementary material. In addition, when the shape can be approximated with simple primitives, we can convert/approximate quadric functions with simple primitives (please refer to line 306).\n\n**Q1: Revise related works and add citations.**\n\n**A:** Thank you for these suggestions. We will incorporate them in the revision.\n"}, {"review_id": "MaN6xQWTbo", "reviewer": "Reviewer_AcKp", "summary": "The paper presents an unsupervised network learning method for reconstructing CSG trees from CAD models. The network is an enhancement of CAPRI-NET, and features the fixed operations (from bottom to top) of intersection -> union -> difference on primitives modeled by quadratic surfaces, where the difference is always applied on two intermediate shapes produced from subtrees. The enhancement to CAPRI-NET is the allowance of difference in the form of inverse primitives before intersection, which the paper shows to cover all possible CSG trees in contrast to the incomplete coverage of CAPRI-NET. In addition, the paper uses dropout pruning to reduce the redundant primitives or intersected intermediate shapes. Through experiments on ABC and ShapeNet datasets and comparisons with previous works on unsupervised CSG reconstruction, this paper shows improved results with good reconstruction accuracy and compactness. Ablation studies further show the usefulness of the dual branch design, inverse primitives, and dropout pruning. ", "questions": "In addition to questions above, there are some detailed questions:\n\n1. Could the primitives before intersection be shown in the expanded CSG trees? That would help readers better understand the complexity and details of the results.\n\n2. Is there any intuitive understanding of the learned weighting vector W in Eq(4)? \n\n3. Why $\\Delta S$ of Eq(6) is not normalized by the number of sample points $n$?\n\n4. Line223, should it refer to Eq(3) instead?\n\n", "limitations": "The authors have discussed limitations in generalization, detail recovery and limited expressiveness of quadratic primitives. ", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper shows the limitation of a previous work CAPRI-NET in representing all possible CSG trees and fixes it with a simple addition of inverse primitives.\n\nThe paper introduces dropout pruning to improve compactness of result CSG trees.\n\nExtensive tests have shown the new algorithm can recover more compact and faithful results than previous works.", "weaknesses": "It's not very clear how the dropout pruning is applied in the training process. Is it used in the last step and after its application there will be no network finetuning? Or the interleaved application of dropout and finetuning can be more helpful? It's desirable that the authors provide a detailed study on this issue.\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**It is not very clear how the dropout pruning is applied in the training process.**\n\n**A:** Dropouts are not applied in the last step but during the training process; please refer to lines 221-222 in the paper. After dropouts, the network parameters will be tuned. We iteratively perform this process until the maximum number of iterations has been reached or when no primitive/intermediate shapes are dropped.\n\n**Q1: Could the primitives before intersection be shown in the expanded CSG trees.**\n\n**A:** Since the complicated shapes in our supplementary material usually use many primitives, we only show a part of the obtained CSG trees. We show the primitives before intersection for the simple shape in our pipeline, as shown in Figure 2 of the main paper. We also show additional primitive visualization examples, see Figure 1 in PDF file attached to the global response.\n\n**Q2: Is there any intuitive understanding of the learned weighting vector W in Eq(4)?.**\n\n**A:** Values in W are learned close to 1 after stage 0. This setup will help the union layer avoid using the min operation in Eq(3)  at stage 0 and all intermediate shapes could have gradients in the early stage.\n\n**Q3: Why $\\Delta S$ of Eq(6) is not normalized by the number of sample points.**\n\n**A:** Since the number of sampled points is constant, normalization will not change the learning process. In our experiments, we sampled a fixed number query points from the shape before dropouts at stage 2 to calculate $\\Delta S$. We will clarify this part in the revision.\n\n**Q4: Line223, should it refer to Eq(3) instead?.**\n\n**A:** Yes, Line 223 is equation (3). Thanks for spotting this mistake.\n\n\n"}, {"review_id": "Ew8GoHrb2e", "reviewer": "Reviewer_T4uV", "summary": "D2CSG presents a neural architecture for inferring CSG programs that reconstruct complex 3D shapes. The architecture is composed of two branches, a cover branch and a residual branch, which are differenced from one another to form the complete shape. The work is largely an extension of, and a strong improvement over, CAPRI-Net, wherein the major differences are: (i) the use primitive complements, (ii) separate primitive sets for the cover and residual branches, (iii) a third training stage, termed dropout, encouraging program sparsity, and (iv) switching from an encoder to an auto-decoder framework. Finding a CSG program to represent a target shape involves \"overfitting\" the network in a test-time optimization scheme, with an occupancy-based reconstruction loss (with regularizing terms, and various relaxations). Compared to previous neural relaxation approaches for CSG program inference, D2CSG net find programs that result in much better reconstructions (both quantitatively and qualitatively).  ", "questions": "\n- If I understood the dropout operation, its based on finding changes to the primitive / intersection contributions that make little difference (e.g. a small delta) to the output binarized shape from the network. I'm surprised that it isn't instead based in deltas to the networks reconstruction loss (e.g. difference versus the target shape). Did you explore any types of dropout operations that considered the target occupancy values?\n\n- The conclusion has the following statement:  ```We also have ample visual evidence that the CSG trees obtained by our method tend to be more natural than those produced by prior approaches```. Looking through the decompositions in the supplemental, I'm unconvinced that many people would posit the decompositions as `natural' -- I would consider either adding additional evidence to support this claim, or removing it.\n\n- The conclusion implies that for datasets like ABC, that lack a consistent global part decomposition, 'overfitting' is more acceptable, with the subtext that its not helpful to learn over more than one shape at a time. I would encourage the addition of a little more nuance to this statement: while ABC-type shape are certainly less globally consistent compared with other domains (like ShapeNet chairs), mechanical parts share clear commonalities with one another, especially at the sub-component/local level, and other methods have found success in learning over distributions of these types of shapes (e.g. [46]). Thus, instead of tying the the merits of `overfitting' to this line of argument, it would be better to let it stand on its own.\n\n\n- (minor) I'm about confused about Eq 7. From the explanation of the method, I would have thought that W is ignored after stage 0 is done. In table 1, it looks like it is not used at all in stage 1, but then re-introduced (and binarized) in stage 2. Line 109 in the supplemental indicates that stage 2 training is the same as stage 1, but how can this be the case if W is re-introduced (is it frozen in stage 1, then unfrozen/modified for stage 2)? I would request additional clarification on this mechanism. \n\n\n", "limitations": "While the `overfitting' setup certainly seems to improve reconstruction performance (at least for D2CSG, though interestingly not for CAPRINet), there are certain downsides of moving away from a shared latent space: e.g., ill-conditioned inference tasks (e.g. program from partial point cloud, or program from image), and shape to shape interpolation, are no longer possible. It would be good for the paper to discuss the impact of this design decision a bit (beyond the change in reconstruction).\n", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The paper presents compelling, and well-supported, evidence that D2CSG provides a marked improvement for the task of 3D CSG program inference (and more generally shape abstraction). The insights that contributed to this improvement, while not a dramatic departure from past work, are sound and sensible. \n\nBased on the qualitative results, and quantitative metrics, its clear that D2CSG outperforms a reasonable set of baseline methods, offering a new state-of-the-art bar for CSG program inference. The ablation experiments, in both the main paper and supplemental, are very helpful and generally well-structured. The fact that D2CSG is able to capture a provably larger set of possible shapes than CAPRINet, due to the use of complements in the primitive set, is also a nice property.  ", "weaknesses": "Probably the biggest weakness of the paper is that its methodological improvements are largely incremental on top of the CARPINet system. In my opinion, this shouldn't disqualify the paper from being accepted. Despite the similarity to CARPINet, D2CSG offers real, supported improvement, and would be useful, and of interest, to the community. This type of paper requires strong experimental results, in the form of comparisons to related work and well-crafted ablation conditions, both of which are provided. To give a higher rating, beyond \"accept\", it would be useful to show that the insights found useful in improving CAPRINet to D2CSG could be extended to other architectures / systems (e.g. they would have more general applicability than this one system). While I think this could be possible, its by no means a given.\n\n```Ablation issues```\n\nThere were a couple of small questions/concerns I had surrounding the ablation experiments:\n\n- Is the top row of table 3 just CAPRINet (from the tables this looks like the case). But CAPRINet also has the encoder/decoder vs auto-encoder difference, so I would expect this to be a different architecture (e.g. the CARPINet without pretraining, provided in the supplemental). Please clarify what this row is supposed to report. \n\n- I think it would be important to add rows to Table 3, where just DB, and just DO, are added to row 1. It would be fine to report these results in the supplemental material, but having access to their performance would help disentangle the effects of each change between systems.\n\n```Clarity```\n\nWhile I wouldn't say that the paper was unclear, I think the overall clarity of the method could be improved. The paper is not very self-contained, much of the technical details are omitted from the main paper, and pushed to either past-work or the supplemental material. Due to space constraints, a full explanation of the training scheme and architecture is likely impossible to have in the main paper, but I think the main paper could still be improved to give a better *motivation* for why the architecture and training scheme is designed in the way that it is. \n\n- Most helpful would be to add more details and annotations to Figure 2 (e.g. the D, Q, T, Con, and a matrices). If these can't fit in the figure, then sub-figures showing parts of the pipeline should be introduced (e.g. to the supplemental). \n\n- In the text, introductory paragraphs should be used to set the stage for the various matrices, and what information they are suppsoed to hold. For instance, there is no explanation given for what the \"Con\" matrix (first introduced on line 175) holds, beyond the formulas for how it is constructed. An additional paragraph at the start of 3.2, walking through all of the matrices that will be created (and what information they hold) would be very helpful.\n\n- Similarly, while some technical details (though not all) are provided in 3.3 indicating how the loss functions change through the training stages, it would benefit the paper to have a high-level section at the beginning of 3.3 explaining *why* multiple stages are needed (while this reasoning might have been provided in past-work, it needs to be restated here, for self-containment). For instance, something like: in stage 0 no hard decisions are made for either (a) primitive to intersection members or (b) which convexes are added into union, then in stage 1 we make a hard decision for (b), and in stage 2 we make a hard decision for (a).\n\n```Minor```\n\n- It would be useful to know how the reconstruction metrics change when converting from quadrics to basic primitives for user-editing (especially compared with just directly optimizing basic primitives, and not quadrics). \n\n- Some discussion of related work on shape program inference can be improved. [36] is not a supervised method, and should likely be grouped with [11], as RL-based methods for unsupervised shape program inference (see [1*]). Relatedly, [2*] should also be added to this section, as an alternative to per-shape optimization for 3D CSG inference. \n\n- typos/phrasing: \n  -  L:27, genera\n  - L:163, are float\n\n\n[1*] Neurosymbolic Models for Computer Graphics, Eurographics 2023 STAR\n\n[2*] PLAD: Learning to Infer Shape Programs with Pseudo-Labels and Approximate Distributions, CVPR 2022", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**it would be useful to show that the insights found useful in improving CAPRINet to D2CSG could be extended to other architectures / systems.**\n\n**A:** The dual complementary idea can be applied to other primitive-based methods for better concavity reconstruction. For example, ExtrudeNet only uses the union of extrusions to reconstruct a shape; it is possible to introduce complement extrusions to better handle concavity. The dropout idea can also be applied to other assembly-based systems as long as they employ intersection or union assembly operations.\n\n* ExtrudeNet: Unsupervised Inverse Sketch-and-Extrude for Shape Parsing, ECCV 2022\n\n**Ablation issues.**\n\n**A:** \n* Good point, we show CAPRI-Net with pre-training and encoder at the first row here, while other rows are performed without pre-training. It would be better to show CAPRI-Net without an encoder here, see updated Table 2 in the PDF file attached to the global response\n* The existing rows can prove the efficiency of our designs. We also add two additional rows and visualization results to help disentangle the effects of each change, see results in the PDF file attached to the global response.\n\n**Clarity.**\n\n**A:** \n* We can revise Figure 2 as suggested. \n* We can add introductory paragraphs for the various matrices.\n* We can provide high-level details in Section 3.3 about the multi-stage training.\n\n**It would be useful to know how the reconstruction metrics change when converting from quadrics to basic primitives for user-editing.**\n\n**A:** We will provide them in the supplementary material.\n\n**Some discussion of related work on shape program inference can be improved.**\n\n**A:** We will improve them as suggested.\n\n**Q1: Did you explore any types of dropout operations that considered the target occupancy values?**\n\n**A:** The dropout module is designed to remove primitives/intermediate shapes that have little effect on the reconstructed shape. The $\\Delta S$ is calculated based on the difference between the reconstructed shapes before and after dropout. We did not calculate $\\Delta S$ between the target shape and the reconstructed shape since it is hard to determine a proper threshold. If we set it smaller, then many complicated shapes would not be considered for dropouts since they usually have a large difference to the target shape. if we set the threshold larger, then many primitives modeling shape details could be dropped as the number of query points is less around such details. In our design, all shapes will have the chance to be considered for dropouts while keeping the shape after dropouts close to what it was before.\n\n**Q2:  I'm unconvinced that many people would posit the decompositions as `natural'**\n\n**A:** We can tone this down in the revision.\n\n**Q3:  I would encourage the addition of a little more nuance to this statement.**\n\n**A:** Thanks for your advice. We will reword the conclusion and add the nuances as the reviewer described by mentioning that \"some of the models in ABC may not possess sufficient generalizability in their primitive assemblies\" to avoid giving the false impression that all ABC models including mechanical objects lack commonalities in structure.\n\n**Q4:  I'm confused about W in Eq 7.**\n\n**A:** At stage 0, values in W are set close to 1 (line 221). At stage 1, W is not used and is not updated. At stage 2, W is re-introduced and updated (line 223). Although the reconstruction loss $L_{rec}^*$ is the same in both stages, the $a^*$ in $L_{rec}^*$ is derived from different equations for stage 1(Eq 3) and stage 2 (Eq 7). In addition, the gradients from reconstruction loss $L_{rec}^*$ won’t be used to update W at stage 2, W is only updated by dropout (line 223) at stage 2. \n\n**Q5:  Discuss more impact about using overfitting.**\n\n**A:** We will add such discussions as suggested. Thank you.\n"}]

Author rebuttal:
We thank all the reviewers for their insightful comments and encouraging remarks. We are glad to see reviewer recognitions that our approach is “novel,” “effective,” "compelling,” and “significantly” outperforms existing methods. Since the reviewer questions were all technical in nature, seeking more details and clarifications, we will answer them in the individual responses. The submitted PDF file attached to the global response includes additional results.

Code and data will be released upon paper acceptance.

Meta-review: Overall, the reviewers appreciated the novelty of the presented approach and the significant improvements over prior methods. The reviewers had several remarks on how the paper should be modified:
- a brief background section on CAPRI-Net
- revised motivation for choice of architecture and training scheme
- revised explanation of various matrices and rationale for various stages
- revised related work section

Given that these changes will be incorporated as promised in the rebuttal, the reviewers and the AC have no objection to accepting the paper.
Decision: Accept (poster)