MV2Cyl: Reconstructing 3D Extrusion Cylinders from Multi-View Images
Authors: Eunji Hong
Nguyen Minh Hieu
Mikaela Angelina Uy
Minhyuk Sung
Venue: NeurIPS 2024 poster (2024)
Keywords: Extrusion
Multi-View Images
3D Reconstruction
Computer-Aided Design
Primary area: machine_vision

Abstract: We present MV2Cyl, a novel method for reconstructing 3D from 2D multi-view images, not merely as a field or raw geometry but as a sketch-extrude CAD. Extracting extrusion cylinders from raw 3D geometry has been extensively researched in computer vision, while the processing of 3D data through neural networks has remained a bottleneck. Since 3D scans are generally accompanied by multi-view images, leveraging 2D convolutional neural networks allows these images to be exploited as a rich source for extracting extrusion cylinder information. However, we observe that extracting only the surface information of the extrudes and utilizing it results in suboptimal outcomes due to the challenges in the occlusion and surface segmentation. By synergizing with the extracted base curve information, we achieve the optimal reconstruction result with the best accuracy in 2D sketch and extrude parameter estimation. Our experiments, comparing our method with previous work that takes a raw 3D point cloud as input, demonstrate the effectiveness of our approach by taking advantage of multi-view images.

MV2Cyl: Reconstructing 3D Extrusion Cylinders from Multi-View Images
Eunji Hong ', Minh Hieu Nguyen', Mikaela Angelina Uy2.3, Minhyuk Sung' Korea Advanced Institute of Science and Technology, "ttanford University, SNVIDIA hanji.hong, hieuristics, mhsung) ac. mikaelaangel@nvidi... com
multi-view images
2D surface segments
2D curve segments
surface reconstruction
3D curve reconstruction
primitiyes extraction
{ translate
editing
reconstructed CAD parameters
scale
boolean invert
extrude
Abstract
We present MV2Cyl, novel method for reconstructing 3D from 2D multi-view images, not merely field raw geometry but as a sxetch-extrude CAD model. Extracting extrusion cylinders from raw 3D geometry has been extensively re- searched in computer vision, while the processing of 3D data through neural networks has remained bottleneck. Since 3D scans are generally accompanied by multi-view images, leveraging 2D convolutional neural networks allows these images be exploited rich source for extracting extrusion cylinder informa- tion. However, we observe that extracting only the surface information the extrudes and utilizing it results in suboptimal outcomes due to the challenges in the occlusion and surface segmentation. By synergizing with the extracted base curve information, achieve the optimal reconstruction result with the best accuracy in 2D sketch and extrude parameter estimation. Our experiments, comparing our method with previous work that takes raw 3D point cloud input, demonstrate the effectiveness of our approach by taking advantage of multi-view images. Our project page can be found at htthhttt::///n htttp::/mmm2cl.... .ww..b.ittubio..o
Introduction
Most human-made objects in our daily lives are created using computer-aided design (CAD), Recon- structing the structural representation from raw geometry, such as 3D scans, is essential to enable
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

the fabrication of 3D shapes and manipulate them for diverse downstream applications. Sketch- Extrude CAD model is particularly notable not only as the most common but also as a versatile CAD representation, which enables the expression of diverse shapes with 2D sketches and extruding heights.
The reconstruction of a set of extrusion cylinders from raw 3D geometry has been extensively explored in previous studies [58] 65 49
30), yet the suboptimal quality of the results remains 30 have demonstrated notable capabilities in parsing raw
challenge. Unsupervised methods 44
geometry but often struggle to precisely segment the shape and part-wise fit the output extrusions to the given shape. Supervised methods using a language-bass based architecture, such as those described producing valid outputs. A notable exception is Point2Cyl [58).
in [65], often face challenges which is supervised method proposing to first segment the given raw point cloud into each extrusion region and estimate the 2D sketch and the extrusion parameters using the predicted segments and surface normal information. While it has shown successful results, the bottleneck remains in the limited performance of the 3D backbone network for segmentation.
To address the limitation, we focus on the fact that 3D scans are commonly accompanied by multi- view images, which are now even solely used to reconstruct a 3D shape without depth or other information thanks to the recent advancements in neural rendering techniques [40] 61 I3 62 []] 2D multi-view images are a rich resource for extracting 3D extrusion information. Also, 2D convolutional neural networks have been utilized in various 3D tasks [34 633. owing to their superior performance compared to 3D processing networks. In light of these observations, we introduce novel framework called M MV2Cyl, which reconstructs set of extrusion cylinders from multi-view images of an object without relying on raw geometry as input.
A straightforward approach to exploiting multi-views in extrusion reconstruction is to first segment 2D regions of the extrusion in each image and reconstruct the 3D shape using the extrusion labels. However, achieving precise 3D surface reconstruction solely from the multi-view images poses challenges due the ambiguity between the object's intrinsic color (albedo) and the effects of lighting (shading) or shadows within images (Fig. 5) In contrast, we observe that reconstructing only the 2D sketches of the two ends of the extrusion cylinders- -te start and end planes of extru- sion- pron-prvides much more accurate results while avoiding the ambiguity issue and the consequential error accumulation. However, this approach also results in failure cases when one of the bases not properly detected in 2D image due to the sparse viewpoints. Therefore, we propose framework that synergizes the reconstructions of labeled surfaces and labeled curves of the extrusion bases so that parameters such as extrusion center and height can be better calculated from the surface information, while the 2D sketches can be precisely recovered from the curve information. In our experiments, we demonstrate the superior performance of our method MV2Cyl on two sketch-extrude datasets: Fusion360 (64) and DeepCAD 65
2 Related Work
3D Primitive Fitting, Primitive fitting has been extensively investigated in computer vision. One line of work involves decomposing into surface/volumetric primitives. Classical approaches include detecting simple primitives such as planes 6 11 42) through RANSAC [I] region growing [45
or Hough voting B M The emergence of neural networks gave rise to data-driven methods 29
56 28 learning frameworks that fit simple primitives such planes, cylinders, cones, and spheres.
Primitive fitting has also been used to approach the task of shape abstraction 75 (5 that fit cuboids given an input shape or image. However, these works all assume fixed predefined set of primitives that may not be able to cover complex real-world objects. The focus of our work is to recover a more general primitive, i.e., extrusion cylinders, that are defined by any arbitrary closed loop and hence can explain more complex objects.
Another line of work tackles fitting 3D parametric curves, where common strategy is to first detect edges (70] and corners [35 70
399 from an input point cloud, and then fit or predict parametric curves from the initial proposals. A recent work, NEF [68], takes multi-view images as input and introduces a self-supervised approach to the parametric curve fitting problem using neural fields, thus allowing them to optimize without needing clean point cloud data. Despite having multi-view images as input, NEF only tackles recovering parametric curves and not extrusion cylinders, which is basic building block for the sketch-extrude reverse engineering task and is the focus of our work.
2

---

CAD Reconstruction for Reverse Engineering. CAD reverse engineering has been a well-studied 38 enabling the recovery of
problem computer graphies and CAD communities 4 60
structured representation of shapes for applications such as editing and manufacturing. Farlier
works focused on reconstructing CAD through inverse CSG modeling 9 55
22
48
69 which
decomposes shapes into simple primitives combined with boolean operations. While achieving good reconstruction, CSG-like methods tend to combine a large number of shape primitives, making them less flexible easily manipulated by users. Recent efforts in collating large-scale CAD
datasets (ABC [25], Fusion 360 [4]. SketchGraphs [54]) have pushed this boundary, enabling
the learning of data-driven methods for various applications such as B B-Rep classification [19 and
segmentation [19 26
10], parametric surface inference [[6 36). and CAD reconstruction
[23
and generation |66 For the reverse engineering task, common CAD modeling paradigm uses
sketch-extrude 64
58 as the basic building block, where a sketch 667 47 is defined as a sequence
of parametric curves forming closed loop. This results in primitives that are more flexible and complex, not a predefined finite set, which will be the focus this work.
Previous research uses NLP-based language models conditioned on raw geometry to represent CAD models as token sequences [65] [4 [8] [2]. However these methods are not designed to understand and decompose geometry into individual primitives and may also result in invalid CAD sequences. A recent work [21] also models CAD with language but face similar issues despite allowing multi-view inputs. To address these issues, Point2Cyl [58] is the pioneering work that poses the CAD reverse engineering task as a geometry- decomposition problem. The idea was to first decompose an input point cloud into extrusion cylinders (sketch-extrude primitives) and introduce differentiable loss functions to recover the parameters. ExtrudeNet [49] and SECAD-Ner (30] further built on this setting and extended it to the unsupervised and self-supervised settings, respectively. Our work tackles a similar setting that takes a geometry-awar approach to the reverse engineering task, except we take multi-view images input, as opposed existing works that only handle point cloud inputs.
3 MV2Cyl: Reconstructing 3D Extrusion Cylinders from Multi-View Images
Surface segmentation
30 Surface reconstruction
segmentation
Extrusion cylinders reconstruction
fbart/
segmentation
pcuiiives
mubisview
Instance segmentation
grarna
pain
granc
mansincial parmmcm
Start/end segmentation
reconstruction
segmentation
Figure Full Pipeline of MV2Cyl.
We propose a method that reconstructs 3D extrusion cylinders from multi-view images without relying on raw 3D geometry as input. The idea is to leverage 2D neural networks to learn 2D priors that provide 3D extrusion information (Sec. (.2).. i.e. extrusion eurves and surfaces. The integration of the information into 3D is achieved through optimizing neural fields (Sec. B.3). MV2Cyl is combination curve field and a surface field that used recover the parameters and reconstruct the extrusion cylinders only given 2D input images (Sec. 3.4
3.1 Problem Statement and Background
Given set of 2D RGB images [ where Rxx x3 the goal is to recover a set of extrusion cylinders E - that represent the underlying 3D shape. We provide an overview here but refer the readers Uy al. (58) for more details.
3

---

Sketch-Extrude CAD is a designing paradigm or the designed model itself in the area of computer aided design (CAD) consists extrusion cylinders as its building blocks. An extrusion cylinder E is 3D space defined as E (n, c, h, S.s) with the extrusion axis n E S?, the extrusion center E IR the extrusion height E IR, and the normalized sketch Š scaled by s E IR. A sketch-extrude CAD model is a 3D shape that is reconstructed from a set of extrusion cylinders. {Ei, E2, .... Ex) where K is the number of extrusion cylinders. A sketch S is set of non-self-intersectinn closed loops drawn in a normalized plane.
We further classify the surfaces of sketch-extruse CAD model as base or barrel. A surface is a base if its surface normal is parallel to its extrusion axis n, and is a barrel if its surface normal is perpendicular to its extrusion axis. By this definition, base surfaces are parameterized by points on the planes and a normal (n). The base surfaces can further be distinguished into start plane and end plane where the start plane contains the point n, while the end plane contains the point c + n.
With these definitions, we will later show how our method, MV2Cyl, is able to recover the set of extrusion cylinders and their corresponding parameters given only 2D multi-view images.
3.2 Learning 2D Priors for 3D Extrusions
To reconstruct extrusion cylinders from multi-view images, we first learn 2D priors for 3D extrusions by exploiting 2D convolutional neural networks. We train two U-Net-based 2D segmentation frameworks: M that extracts curve information and M that extracts surface information. The 2D information extracted from the two frameworks is integrated into 3D using a neural field and then utilized for more robust reverse engineering. We detail each 2D segmentation frameworks below.
Figure 2: Example of segmentation prediction. From left to right: input rendered image, surface instance seg- mentation, surface start-end- barrel segmentation, curve instance segmentation, and curve start-end end segmentation.
2D Surface Segmentation Framework Maurface The goal of the surface segmentation network to extract the extrusion instance segmentation as well as start plane, end plane, and barrel (start-end- (barre) segmentation given input image. That is M "xx"*xxxx
{psurface {0, (O,..., K} #x W Q Quirface E {0, 1, 3} H WWW) where is the number of instances, psurface repre- sents the extrusion instance segmentation label, and Quurface denotes the start-end-barrel segmentation label. The zero index is used for background annotation. MMsurface is implemented as two distinct U-Nets predicting each segmentation and is trained using multi-class cross entropy loss with ground truth labels psurface and Quurlace, An important property that the problem does not admit a unique solution as i) the extrusion segment can be ordered arbitrarily and ii) the start and end planes can also be arbitrarily labeled. To handle this, we use Hungarian matching to find the best one-to-one matching with the ground truth labels and reorder it as psurfaaee and Q Qurfaaee This gives us the loss function of the instance segmentation:
K ΣΣΣΣ
B
H
W
1
C
BHW
1
1
surfaace Aac
= k) log P surface
bonk
(1)
bm
1amm
W= Rm
where Psurfice is the model's predicted probability that pixel (h, w) belongs the k-th instance in the b-th image in the batch, and Psurface is the pseudo GT label for that pixel, with the number of images in batch B and the number of possible instances including the background K. The loss function of the start-end-ban enrrel segmentation appears the similar, replacing the as Q and the range of classes {0, .... K) to (0, 1, 3).

---

2D Curve Segmentation Framework Marr, We observe that detecting and extracting curves on the base planes of each extrusion cylinder segment provides additional information recover the parameters and reconstruct the extrusion cylinders. Moreover, (feature) curves have been longstanding and well-explored problem dating back to classical computer vision literature [46] 74 43 thanks to their strong expressiveness. This leads us to learn a 2D curve prior that provides more discriminative and detailed outputs to make reverse engineering more robust.
The goal of the curve segmentation framework is to extract both extrusion instance segmen- tation and start-end segmentation, where the start-end plane segmentation distinguishes the curves of the two base planes of extrusion cylinders. Concretely, M R R xxxxxx3 + {peuree E .). #x) #xW"" Quurve E {0, 1, 2) "x)), where peuiee the extrusion instance segmen- lation label and Q Quurve is the start-end segmentation label.
Similar M the zero index is used for background annotation in both segmentation tasks and Menue is also implemented with two U-Nets and trained using multi-class cross-entropy loss against the pseudo ground truth { peurve Qeurve that reordered with Hungarian matching to handle order invariance and ambiguities. An additional challenge for curves compared to surfaces is the labels are a lot sparser with the majority the images being background pixels. Hence, the model can easily fail to predict meaningful labels due to this label imbalance. To alleviate this, additionally employ a dice loss [4]] to handle the strong foreground-backgroundd imbalance and focal loss [3] eircumvent the class imbalance between the extrusion instances. Hence the loss function used to train the curve prior is given as:
D ACE Caross
A Crocal + Adice Liice.
(2)
Additional details about the segmentation framework are available in the appendix E.1
3.3
Integrating 2D Segments into 3D Field
grsuuface
surface
q
surface
gr
curve
curve
surface
E
ssufaace
psurface
Q Surface
Ceurve
peurve
curve
Figure 3: Overview of the learned surface and curve fields. LLeft-to-Right Density field of surface, instance semantic field of surface, start-end semantic field surface, density field curve, instance semantic field of curve, and start-end semantic field of curve.
To establish correspondences and collate the information extracted from multi-view images, a natural approach integrate 2D information into a consistent 3D representation through learning a 3D field [40] 61 68 44]. For each of our 2D priors M and Mecure, we learn density field F and semantic field as detailed below, respectively.
Density Field F. We learn a density field for surfaces Fsarface and curves Feurve Given query point in 3D space IR3 the density field network F. R33 + IR outputs scalar value that indicates how likely the 3D point x is on a surface or a curve for Fuarface and curves Feurree respectively. To optimize the field differentiably with only 2D images,
volume rendering [40 and learnable
transformation 688 to convert the density field F(x) ) to an opacity field (x).. This transformation is in the form of learnable sigmoid function given as:
a(x) a (1 ( (F(()))8)
5

---

where a is learnable parameter that adjusts the density scale, and B and Ç are constant hyperparam eters. We use the same transformation Eq. 3 for the surface field j and the curve field F resulting in corresponding opacity fields a strrae and oeurve We then directly render the density by using the volume rendering equation to get the expected surface and curve density for camera ray r, which is given by the integrals
Fosurface (r) /
Tuurface (t)o surfaee (r(1))dt,
in Exuve(r) (r) ((1 (1uu curve ()())))
(5)
where (rarfuce (t) = So osurlasee (r(s)) and Truve (t) = o (r(s)) are the corresponding
transmittance. Inspired by NEF (88, we train our surface field j.surface and curve field J.euve using an adaptive 1.2 loss for the reconstruction of pixel density and sparsity loss for learning of a sharp distribution of density along the ray. The objective function is given as follows:
Iddeniity
Waachh (F)]l - F
Nawy
(6)
(1+ F(r,(t),)
+ Asporsity Nammplss
(r) log 1,
where F((r) and F(rr) the 2D-obberved and predieted field density, respectively, for both the surface and curve models, i indexes background pixels from input maps, j indexes the sample points along the rays, and determines the scale of the regularizer. The 2D-obserrved density E(r) is derived from the output of the 2D prior being a foreground pixel, either start, end, or barrel pixel.
Following Ye et al. 668 Wbatch (r) is given as:
Nusth ccii piucla
ifr Thatch
W Waachh (r)
Naa
Nusch pieca
otherwise
Npach pisch.
The weight (ra addresses the imbalance between foreground and background areas in the density maps, preventing the network from converging to a local minimum that falsely classifies most pixels as background, which are more prevalent.
Semantic Field A. Now with the density field, we can distill the information from the 2D priors by learning semantic fields A for both surfaces and curves, which we denote as AAsurface and A
Following Zhi et al. (73, each semantic modeled as an additional feature channel branching out from the density field F. For surfaces, the additional semantic field A = (uuuaae .4 guttiiee) is
comprised of two networks purfuce, R + (I))) that predicts the extrusion instance label, and segmentation. Similarly for the curves, the semantic field is given as A {pourve [ where youvee R + R(K+1) predicts the extrusion instance label, and gsurve R Rd predicts the start-end segmentation. The expected semantic for a ray r is then computed using the volume rendering equation which is given as:
gurlaae 3 + IR that predicts the start-end-
A(r) A(r)= L"d T())))(t))a(((()ddd
(8)
for each {prurface gratiace p pruve grrrve) that correspond to rendered predicted semantics Á [psurface Quurface pearre Quurve] from the 3D field.
We train our semantic field using the predictions from our learned 2D priors Fauuface and Fourve For each of the 2D multi-view images, we can extract the 2D observed labels A E [psurface Q Qqurface perree Qeurve] from the learned priors, which we use as supervision. As men- tioned Sec. 3.2 since the labels may not individually be consistent across the multiple views, we use Hungarian matching during training to align the labels across multiple views based on the best one-to-one matching. For each semantic, we use a multi-class cross-entropy loss train and optimize the semantic field with the given supervision:

---

Lsemantic = =- EEEE (r) log Ar(r),
where (r) and ('(r) represent the class semantic probabilities from the learned 2D prior and the predicted field, respectively, aligned with Hungarian matching.
Training and Implementation Details. The integration into a coherent 3D field is optimized independently for each 3D shape. We use TensoRF [5] as the backbone representation for the 3D fields, where we have one model for the surface field and the other for the curve field. both models, we train the density fields F. and the semantic fields A together for I 1500 iterations. Fach semantic field is parameterized by 2-layer MLP that outputs logit values. Additional implementation details provided in the Appendix E.2
3.4 Reverse Engineering from 3D Fields
We now elaborate on how we leverage our optimized surface field J.uuface and curve field jeurve for the reverse engineering task, specifically to recover extrusion cylinders and their defining parameters, (n, c. h. S. s). Each point within the extracted point clouds is assigned semantic information queried from the associated semantic fields, providing context for each feature in 3D space. Using these en- riched point clouds, we employ multi-step process to reconstruct each extrusion cylinder, iteratively refining the parameters to accurately the geometry and semantics. Further details can be found in Appendix E.3
1. Extrusion Axis Estimation: For each extrusion cylinder, the extrusion axis n is computed through plane fitting via RANSAC [12] on the surface base points with corresponding extrusion cylinder instance label. The axis then set as the normal of the fitted plane. This process relies on surface information because plane fitting is more reliable using the surface points. If the base face of the cylinder fails to be reconstructed due to occlusion, we use curve points as a substitute.
2. 2D Sketch Š Estimation: To obtain the sketch Š for each extrusion cylinder, we first project the curve point cloud for each extrusion instance onto the plane with normal n. From the projected point cloud, we then compute for the sketch scale s. We then optimize an implicit signed distance function for each closed loop using the projected point cloud with IGR [15]. Finally, we obtain the closed loop by extracting the zero-level set of the implicit function. Implicit sketches can be converted to a parametric representation using an off-the-shelf spline fitting module. We provide example visualizations in Appendix E.7
3. Extrusion height h Estimation:
The height h of each extrusion cylinder is computed as the distance between the two sketch centers from the start plane and the end plane. The sketch centers are computed from the projected curve point cloud for each paired labels (extrusion instance segmentation, start-end segmentation) onto the plane with normal n. If any of the start- end planes are occluded, the height is computed using the barrel points from the surface point cloud with the corresponding extrusion cylinder instance label.
4. Extrusion centroid e Estimation: The centroid of each extrusion cylinder be obtained by taking the average position of the 3D curve point cloud with the corresponding extrusion cylinder instance label. Similar the above, if any of the start-end planes are occluded or missing, the centroid is computed as the average of the barrel points from the surface point cloud with the corresponding extrusion cylinder instance label.
Experiments
In this section, present experimental evaluation to demonstrate the feasibility of our MV2Cyl in reconstructing extrusion cylinders given only multi-view images. To the best of our knowledge, proposed pipeline is the first tackle extrusion cylinder recovery directly from 2D input images, and we show that our method is outperforms existing methods that take clean geometry as input and their modifications to handle input images.

---

1. Fit plane using RANSAC and project the curve points onto the plane
s
Normalize and fit curve
3. Find start and end centers and compute height
4. Compute instance center
instance label
start label
end label
Figure 4: Converting 3D reconstructed geometry and semantics into CAD parameters.
Datasets. We evaluate our approach on two sketch-extrude CAD datasets Fusion360 64 and
DeepCAD 655 We use the train and test splits released in [58 We enrich these datasets with multi-view image renderings and 2D curve and surface segmentation maps for the training and evaluation of our method and the baselines. More details about dataset generation are in Appendix E.4
Evaluation Metrics. Following [58 report the average extrusion-axis error (E.A.), extrusion- center error (E.C.), per-extrusion cylinder fitting loss (Fit Cyt.), and global fitting loss (Fit Glob.). Additionally, also report the extrusion-height error (E.H.) defined the LI distance between the GT and predicted height value: Ex= [hĄ hy/- These metries evaluate how well the methods reconstruct and recover the parameters of the extrusion cylinders. The final scores are the average of the metrics calculated over all test shapes.
Baselines. In the absence of established methods for transforming multi-view images into sketch- extrude CAD models within shape space, benchmark our model against point cloud sketch- extrude CAD reconstruction technique (88 and its variant to validate the effectiveness of our proposed approach.
We compare with Point2Cyl 58 technique for reconstructing sketch-extrude CAD given input 3D point clouds. The point clouds are sampled from clean ground truth CAD meshes and are utilized as the input for this baseline. Point2Cyl (58) initially segments the input point cloud into distinct instances and base/barrel segments, then identifies the extrusion axis as the direction that aligns with the normals of the points in the base segments and is orthogonal to the normals the points in the barrel segments. The rest of the parameters inferred based this predicted axis and the segmentation labels. To evaluate the network, we trained it using the official code released by the authors.
NeuS2 Point2inll [58) combines NeuS2 62 with Point2Cyl [88. Since no prior work has reconstructed extrusion cylinders directly from multi-view images, we combine methods for i) shape reconstruction from multi-view images, with techniques for ii) extrusion cylinder reconstruction from raw geometry. This approach utilizes NeuS2 (2). an off-the-shelf image-based surface reconstruction technique, that optimizes the photometric loss to learn underlying implicit 3D representations. For this baseline, we first reconstruct the object surface from multi-view images using NeuS2 (62), and then sample point cloud from the reconstructed mesh. The sampled point cloud is then directly fed into Point2Cyl (58) to predict sketches and extrusion parameters of CAD models.
4.1 Results
Tab. shows the quantitative results of CAD reconstruction on the Fusion360 64 and DeepCAD 65 datasets. We see that MV2Cyl outperforms the baselines by considerable margins across both datasets in all evaluated metrics. Qualitative comparisons are shown in Fig. S

---

Table 1: Quantitative results using Fusion360 and DeepCAD 665
datasets. MV2Cyl
consistently outperforms all baselines in all metrics. Red denotes the best; orange denotes the second best.
Fit.
Fit.
Dataset
Method
E.A.("4
E.C.
E.H.
Cyl. +
Glob.
NeuS2+Point2Cyl
35.0562
0.2198
0.7802
0.1036
0.0596
Fusion 360
Point2Cyl
9.5228
0.0839
0.2918
0.0731
0.0293
MV2Cyl (Ours)
1.3939
0.0385
0.1423
0.0284
0.0212
NeuS2+Point2Cyl
54.8715
0.0650
0.8471
0.0903
0.0638
Deep
Point2Cyl
7.9156
0.0266
0.1632
0.0420
0.0250
CAD
MV2Cyl (Ours)
0.2202
0.0121
0.0761
0.0246
0.0229
(a)
[b)
(c)
(d)
(e)
(
GT
Ours
(CAD)
NeuS2-
Point2Cyl (Mesh)
NeuS2-
Point2Cyl (CAD)
Point2Cyl (PC)
Point2Cyl (CAD)
Figure 5: Qualitative comparisons with the baselines. Each instance identified by different color. MV2Cyl produces high-quality geometry and even outperforms Point2Cyl (58) that directly consumes 3D point clouds. Furthermore, comparison against a naive baseline that pipelines NeuS2 (62), multi-view surface reconstruction technique, to Point2Cyl demonstrates the importance of edge information when inferring 3D structures.
The results of NeuS2 62 Point2yyl [58 illustrate the challenges in naively using multi-view images for CAD reconstruction, where inaccurate 3D geometry reconstructed from images serve as critical bottleneck in recovering CAD parameters. We observe that it struggles to distinguish the object's intrinsic color (albedo) from the effects of lighting or shadows (shading). This can result in discrepancies, such omitted details subtle smoothing at the intersection between different extrusion segments, as illustrated in (Fig. 5 As seen in Fig. (c), a common failure arises when NeuS2 [62] fails to reconstruct fine geometric details, in this case, the boundaries between the extrusion cylinders. Such artifacts in the reconstructed mesh make the conversion from the point cloud to the CAD model unreliable.
MV2Cyl also outperforms Point2Cyl (58), which takes precise 3D geometric information as input. An advantage of taking multi-view images instead of point clouds is the images contain richer information such as edges and curves. Moreover,
neural networks have also shown superior performance compared to 3D processing networks leading to superior performance of MV2Cyl compared to Point2Cyl [58). The comparison the 2D segmentation network MV2Cyt with the 3D segmentation network in Point2Cyl are found in Appendix F.I

---

Unlike the baselines that require accurate 3D surface information, MV2Cyl is able to directly handle 2D multi-view images as input by leveraging on learned 2D priors to extract 3D extrusion information. We show that extracting extrusion surfaces and curves from 2D) images allows us to leverage both the curve and surface representation for better extrusion cylinder reconstruction compared baselines that even require clean 3D geometry input.
4.2 Real Object Reconstruction
We validate MV2Cyl on real-world demo by 3D-printing various sketch-extrude objects from the Fusion360 test set, by capturing multi-view images with an iPhone 12 Mini and then obtaining camera poses via COLMAP. These captured images were processed through our pipeline extract 2D information, converted to 3D using our density and semantic fields to recover extrusion cylinders and parameters. To address the domain gap between synthetic training images and real-world data, we applied preprocessing techniques, including grayscale conversion, background removal, and then fine-tuning a large vision model. Additional details are provided in Appendix B
Conclusion
We introduce MV2Cyl, a novel method for reconstructing set of extrusion cylinders in 3D from multi-view images. Unlike previous approaches that rely only on raw 3D geometry as input, our framework utilized 2D multi-view images and demonstrates superior performance by leveraging the capabilities of 2D convolutional neural networks image segmentation. Through the analysis of curve and surface representations, each having its advantages and shortcomings, propose integration both that takes the best of both worlds, achieving optimal performance.
Limitations and Future Work. Our proposed method is not without its limitations. While MV2Cyl demonstrates of-te--f-the-art performance in reconstructing sketch-extrude CAD models, shares a limitation with previous approaches, such as Point2Cyl (58), regarding the prediction of binary operations among primitives. MV2Cyl does not explicitly predict these binary operations, which we leave as future work. Nonetheless, propose an initial straightforward method for recovering binary operations through exhaustive search. Details are provided the appendix [F.6]] Additionally, MV2Cyl does not directly accommodate textured CAD models, as the 2D) segmentation models are trained on a synthetic dataset derived from rendering untextured CAD models. By utilizing generalizable large 2D models, such as Segment Anything [24], we can preprocess images of textured CAD models for segmentation by extracting object masks and removing textures. We also leave this for future investigation.
MV2Cyl is also susceptible to occlusion since relies on 2D image inputs. Specifically, if one side of an extrusion cylinder is completely hidden by others, our 2D segmentation model cannot detect the Fig. 66 the left shape
hidden side. such cases, the extrusion cylinder cannot be reconstructed. represents the target CAD model, while the right one shows the reconstructed model by MV2Cyl. The target shape features an inset hexagonal cylinder with one end hidden by an outer cylinder, but MV2Cyl was unable to reconstruct the inset extrusion cylinder.
Figure 6: Example failure case.
Societal Impacts. This technology could make CAD more accessible to con-rrofessional or educational sectors, allowing more people to engage design and engineering tasks without needing extensive training in traditional CAD software. However, such a 3D reconstruction method can also raise privacy concerns. By converting physical object into a digital format, the shape could be reused for design purposes without the original creator's permission.
10

---

Acknowledgements
We thank Seungwoo, Juil, Jaihoon, and Yuseung for their valuable comments and suggestions in developing the idea and preparing an earlier version of the manuscript. This work was supported by the NRF grant (RS- (KS-2023-00209923), IITP grants (RS-2022-11225594, RS-2023-00227592, RS-2024- 00399817), and KEIT grant (RS-2024-00423622), all funded by the Korean government (MSIT and MOTIE), as well as grants from the DRB-KAIST SketchTheFuture Research Center, NAVER-Inte Co-Lab, Hyundai NGV, KT, and Samsung Electronics. Mikaela also acknowledges the support from an Apple AI/L PhD Fellowship.
References
[1] Pál Benkö, Géza Kós, Tamás Várady, László Andor, and Ralph Robert Martin. Constrained fitting in reverse engineering. Comput. Aided Geom. Des., 2002.
[2] Pal Benko and Tamas Varady. Segmentation methods for smooth point regions of conventional engineering objects. Computer-Aided Design, 2004.
[3] Dorit Borrmann, Jan Elseberg, Kai Lingemann, and Andreas Nülchter. The 3d hough transform for plane detection in point clouds: A review and a new accumulator design. 3D Research, 2, 2011.
[4] Roseline Bénière, Gérard Subsol, Gilles Gesquière, François Le Breton, and William Puech. A comprehensive process of reverse engineering from 3d meshes to cad models. Computer-Aideet Design, 2013.
[5] Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su. Tensorf: Tensorial radiance fields. In ECCV, 2022.
[6] T Czerniawski, B Sankaran, M Nahangi, C Haas, and Leite. 6d dbscan-based segmentation of building point clouds for planar object classification. Automation in Construction. 88:44-58. 2018.
[7] Maximilian Denninger, Martin Sundermeyer, Dominik Winkelbauer, Youssef Zidan, Dmitry Olefir, Mohamad Elbadrawy, Ahsan Lodhi, and Harinandan Katam. Blenderproc. arXiv preprint arXiv:1911.01911, 2019.
[8] Bertram Drost and Slobodan llic. Local hough transform for 3d primitive detection. In 3DV, 2015.
[9] Tao Du, Jeevana Priya Inala, Yewen Pu, Andrew Spielberg, Adriana Schulz, Daniela Rus, Armando Solar-Lezama, and Wojeiech Matusik. Inversecsg: Automatic conversion of 3d models to esg trees. ACM Transactions on Graphics (TOG), 2018.
[10] Elona Dupont, Kseniya Cherenkova, Anis Kacem, Sk Aziz Ali, Ilya Arzhannikov, Gleb Gusev, and Djamila Aouada. Cadops-net: Jointly learning cad operation types and steps from boundary- representations. In 3DV, 2022.
[11] Hao Fang, Florent Lafarge, and Mathieu Desbrun. Planar shape detection structural scales. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018
[12] M. Fischler and R. Bolles. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 1981.
[13] Qiancheng Fu, Qingshan Xu, Yew Soon Ong, and Wenbing Tao. Geo-neus: Geometry-consistent neural implicit surfaces learning for multi-view reconstruction. In NeurIPS, 2022.
[14] Yaroslav Ganin, Sergey Bartunov, Yujia Li, Ethan Keller, and Stefano Saliceti. Computer-aided design as language. In NewrlPS, 2021.
[15] Amos Gropp. Lior Yariy, Niv Haim, Matan Atzmon, and Yaron Lipman. Implicit geometric regularization for learning shapes. In ICML, 2020.

---

[16] Haoxiang Guo, Shilin Liu. Hao Pan, Yang Liu, Xin Tong, and Baining Guo. Complexgen: Cad reconstruction by b-rep chain complex generation. In SIGGRAPH. 2022.
[17] J H Hannay and F Nye. Fibonacci numerical integration on a sphere. Journal of Physics A: Mathematical and General, page 11591, 2004.
[18] Pradeep Kumar Jayaraman, Joseph G. Lambourne, Nishkrit Desai, Karl D. D. Willis, Aditya Sanghi, and Nigel J. W. Morris. Solidgen: An autoregressive model for direct b-rep synthesis, 2023.
[19] Pradeep Kumar Jayaraman, Aditya Sanghi, Joseph G. Lambourne, Karl D.D. Willis, Thomas Davies, Hooman Shayani, and Nigel Morris, Uv-net: Learning from boundary representations. In CVPR, 2021.
[20] Rasmus Jensen, Anders Dahl, George Vogiatzis, Engil Tola, and Henrik Aanæs. Large scale multi-view stereopsis evaluation. In 2014 IEEE Conference Computer Vision and Pattern Recognition, pages 406-413. IEEE, 2014.
[21] Henrik Jobczyk and Hanno Homann. Automatic reverse engineering: Creating computer-aided design (cad) models from multi-view images. In GCPR, 2023.
[22] Kacper Kania, Maciej Zięba, and Tomasz Kajdanowicz. Uesg-net- unsuuervied discovering of constructive solid geometry tree. In NeurlPS, 2020.
[23] Mohammad Sadil Khan, Elona Dupont, Sk Aziz Ali, Kseniya Cherenkova, Anis Kacem, and Djamila Aouada. Cad-signet: Cad language inference from point clouds using layer-wise sketch instance guided attention, 2024.
[24] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick. Segment anything. arXiv:2304.02643 2023.
[25] Sebastian Koch, Albert Matveev, Zhongshi Jiang, Francis Williams, Alexey Artemov, Evgeny Burnaev, Mare Alexa, Denis Zorin, and Daniele Panozzo. Abe: A big cad model dataset for geometric deep learning. In CVPR, 2019.
[26] Joseph G. Lambourne, Karl D. D. Willis, Pradeep Kumar Jayaraman, Aditya Sanghi, Peter Meltzer, and Hooman Shayani. Brepnet: A topological message passing system for solid models. In CVPR, 2021.
[27] Joseph George Lambourne, Karl Willis, Pradcep Kumar Jayaraman, Longfei Zhang, Aditya Sanghi, and Kamal Rahimi Malekshan. Reconstructing editable prismatic cad from rounded voxel models. In SIGGRAPH Asia 2022 Conference Papers, SA '22, 2022.
[28] Eric- Tuan Lé, Minhyuk Sung, Duygu Ceylan, Radomir Mech, Tamy Boubekeur, and Niloy J. Mitra. Cpfn: Cascaded primitive fitting networks for high-resolution point clouds. In ICCV, 2021.
[29] Lingxiao Li, Minhyuk Sung, Anastasia Dubrovina, Li Yi, and Leonidas Guibas. Supervised fitting of geometric primitives to 3d point clouds. In CVPR, 2019.
[30] Pu Li, Jianwei Guo, Xiaopeng Zhang, and Dong-Ming Yan. Secad-net: Self-supervised cad reconstruction by learning sketch-extrude operations. In CVPR, 2023.
[31] Yangyan Li, Xiaokun Wu, Yiorgos Chrysathou, Andrei Sharf, Daniel Cohen-Or, and Niloy J Mi- tra. Globfi:: Consistently fitting primitives by discovering global relations. In ACM Transactions on Graphics (TOG), 2011.
[32] Siyou Lin, Dong Xiao, Zuoqiang Shi, and Bin Wang. Surface reconstruction from point clouds without normals by parametrizing the gauss formula. ACM Transactions on Graphics, 42(2):1-19, 2022.
[33] Tsung-Yr Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In ICCV, 2017.
12

---

[34] Fangfu Liu, Chubin Zhang, Yu Zheng, and Yueqi Duan. Semantic ray: Learning a generalizable semantic field with ross-reppooection attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17386-17396, 2023.
[35] Yujia Liu, Stefano D' Aronco, Konrad Schindler, and Jan Dirk Wegner. Pe2wf: 3d wireframe reconstruction from raw point clouds, 2021.
[36] Yujia Liu, Anton Obukhov, Jan Dirk Wegner, and Konrad Schindler. Point2cad: Reverse engineering cad models from 3d point clouds, 2023.
[37] William E. Lorensen and Harvey E. Cline. Marching cubes: A high resolution 3d surface construction algorithm. In SIGGRAPH, 1987.
[38] Weijian Ma, Minyang Xu, Xueyang Li, and Xiangdong Zhou. Multicad: Contrastive representa- tion learning for modlt 3d cepputeer-iiedd design models. Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, 2023.
[39] Albert Matveev, Ruslan Rakhimov, Alexey Artemov, Gleb Bobrovskikh, Vage Egiazarian, Emil Bogomolov, Daniele Panozzo, Denis Zorin, and Evgeny Burnaev. Def: Deep estimation of sharp geometric features in 3d shapes, 2022.
[40] Ben Mildenhall, Pratul Srinivasan, Matthew Tancik, Jonathan Barron, Ravi Ramamoor- thi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 2021.
[41] Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmadi. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In 3DV, 2016.
[42] Aron Monszpart, Nicolas Mellado, Gabriel J Brostow, and Niloy J Mitra. Rapter: rebuilding man-made scenes with regular arrangements of planes. ACM Transactions on Graphics (TOG), 2015.
[43] Elia Moscoso Thompson, Gerasimos Arvanitis, Konstantinos Moustakas, N Hoang-Xune Xuan, ER Nguyen, M Tran, Thibault Lejemble, Loic Barthe, Nicolas Mellado, Chiara Romanengo, et al. Feature curve extraction on triangle meshes. In 12th Eurographics Workshop on 3D Object Retrieval. The Eurographies Association, 2019.
[44] Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolution hash encoding, ACM Trans. Graph., 41(4):102:1110:155, July 2022.
[45] Sven Oesau, Florent Lafarge, and Pierre Alliez. Planar shape detection and regularization in tandem. In Computer Graphics Forum, 2016.
[46] Yutaka Ohtake, Alexander G. Belyaev, and Hans-Peter Seidel. Ridge-valley lines on meshes via implicit surface fitting. In SIGGRAPH, 2004.
[47] Wamiq Reyaz Para, Shariq Farooq Bhat, Paul Guerrero, Tom Kelly, Niloy Mitra, Leonidas Guibas, and Peter Wonka. Sketchgen: Generating constrained cad sketches. In NewrIPS, 2021.
[48] Daxuan Ren, Jianmin Zheng, Jianfei Cai, Jiatong Li, Haiyong Jiang, Zhongang Cai, Junzhe Zhang, Liang Pan, Mingyuan Zhang, Haiyu Zhao, and Shuai Yi. Csg-stump: A learning friendly esg-like like representation for interpretable shape parsing. In ICCV, 2021.
[49] Daxuan Ren, Jianmin Zheng, Jianfei Cai, Jiatong Li, and Junzhe Zhang. Extrudenet: Unsuper- vised inverse -ketch-and-extrueee for shape parsing. In ECCV, 2022.
[50] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In MICCAI, 2015.
[51] Ruwen Schnabel, Roland Wahl, and Reinhard Klein. Efficient ransac for point-cloud shape detection. Computer graphics forum, 2007.
[52] Johannes Lutz Schönberger and Jan-Michael Frahm. Stratture-rrommmotio revisited. In Conference Computer Vision and Pattern Recognition (CVPR), 2016.
13

---

[53] Johannes Lutz Schönberger, Enliang Zheng, Mare Pollefeys, and Jan- Jan-Michael Frahm. Pixelwise view selection for unstructured multi-view stereo. In European Conference on Computer Vision (ECCV), 2016.
[54] Ari Seff, Yaniv Ovadia, Wenda Zhou, and Ryan P. Adams. SketchGraphs: A large-scale dataset for modeling relational geometry in computer-aided design. In ICML, 2020.
[55] Gopal Sharma, Rishabh Goyal, Difan Liu, Evangelos Kalogerakis, and Subhransu Maji. Csgnet: Neural shape parser for constructive solid geometry. In CVPR, 2018.
[56] Gopal Sharma, Difan Liu, Evangelos Kalogerakis, Subhransu Maji, Siddhartha Chaudhuri, and Radomír Méch. Parsenet: A parametric surface fitting network for 3d point clouds. In ECCV, 2020.
[57] Shubham Tulsiani, Hao Su, Leonidas J. Guibas, Alexei A. Efros, and Jitendra Malik. Learning shape abstractions by assembling volumetric primitives. In CVPR, 2017.
[58] Mikaela Angelina Uy, Yen-Yu Yu Chang, Minhyuk Sung, Purvi Goel, Joseph G Lambourne, Tolga Birdal, and Leonidas Guibas. Point2cyl: Reverse engineering 3d objects from point clouds to extrusion cylinders. In CVPR, pages 11850-1 18600, 2022.
[59] Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern, Eric Larson, C Carey, ilhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, Ian Henriksen, E, A. Quintero, Charles R. Harris, Anne M. Archibald, Antônio H. Ribeiro, Fabian Pedregosa, Paul Mulbregt, and SeiPy 1.0 Contributors. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods, 17:261-272, 2020.
[60] Tamás Várady, Ralph R Martin, and Jordan Cox. Reverse engineering of geometric models an introduction. Computer-Aice Aided Design, 1997.
[61] Peng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. In NeurlPS, 2021.
[62] Yiming Wang, Qin Han, Mare Habermann, Kostas Danilidiss Christian Theobalt, and Lingjie Liu. Neus2: Fast learning of neural implicit surfaces for multi-view reconstruction. In ICCV, 2023.
[63] Yue Wang, Vitor Campagnolo Guizilini, Tianyuan Zhang, Yilun Wang, Hang Zhao, and Justin Solomon. Detr3d: 3d object detection from multi-view images via 3d-to-2d queries. In Conference Robot Learning, pages 180-191. PMLR, 2022.
[64] Karl DD Willis, Yewen Pu, Jieliang Luo, Hang Chu, Tao Du, Joseph G Lambourne, Armando Solar-Lezama, and Wojciech Matusik. Fusion 360 gallery: A dataset and environment for programmatic cad construction from human design sequences. ACM Transactions on Graphics (TOG), 2021.
[65] Rundi Wu, Chang Xiao, and Changxi Zheng. Deepcad: deep generative network for computer- aided design models. In ICCV, 2021.
[66] Xiang Xu, Pradeep Kumar Jayaraman, Joseph G Lambourne, Karl DD Willis, and Yasutaka Furukawa. Hierarchical neural coding for controllable cad model generation. arXiv preprint 2rXiv:2307.00149 2023.
[67] Xiang Xu, Karl DD Willis, Joseph G Lambourne, Chin- Yi Cheng, Pradeep Kumar Jayaraman, and Yasutaka Furukawa. Skexgen: Autoregressive generation of cad construction sequences with disentangled codebooks. In ICML, 2022.
[68] Yunfan Ye, Renjiao Yi, Zhirui Gao, Chenyang 7hu, Zhiping Cai, and Kai Xu. Nef: Neural edge fields for 3d parametric curve reconstruction from multi-view images. In CVPR, 2023.
14

---

[69] Fenggen Yu, Qimin Chen, Maham Tanveer, Ali Mahdavi Amiri, and Hao Zhang. Deesg: Unsupervised learning of compact esg trees with dual complements and dropouts. In NeurIPS, 2023.
[70] Lequan Yu, Xianzhi Li, Chi-Wing Fu, Daniel Cohen-Or, and Pheng-Ann Heng. Ec-net: an edge-aware point set consolidation network. In ECCV, 2018.
[71] Kaidong Zhang and Dong Liu. Customized segment anything model for medical image segmentation. arXiv preprint arXiv:2304. 13785, 2023.
[72] Shuming Zhang, Zhidong Guan, Hao Jiang, Tao Ning. Xiaodong Wang, and Pingan Tan. Brep2Seq: a dataset and hierarchical deep learning network for reconstruction and generation of computer-aided design models. Journal of Computational Design and Engineering, 2024.
[73] Shuaifeng Zhi, Tristan Laidlow, Stefan Leutenegger, and Andrew J Davison. In-place scene labelling and understanding with implicit scene representation. In CVPR, 2021.
[74] Q. Zhou, T. Weinkauf, and O. Sorkine. Feature-based mesh editing. In Eurographics, Short Papers, 2011.
[75] Chuhang Zou, Ersin Yumer, Jimei Yang, Duygu Ceylan, and Derek Hoiem. 3d-prnn: Generating shape primitives with recurrent neural networks. In ICCV, 2017.
15

---

Appendix
A Ablation Study
A.1
Importance of Jointly Using Surface and Curve
Tab.
shows the importance of combining surface and curve reconstruction for CAD reconstruction.
shows qualitative results of the final CAD model outputs and also includes intermediate results, such as point clouds from density fields. Fig.
We see that in surface-only reconstruction, occlusion between instances can result missing base faces, as seen with the central cylinder in the top row Fig. A7 (a). This occlusion impedes accurate CAD parameter reconstruction. Conversely, the curves (see Fig. A7 (a)) capture this extrusion instance with accurate start-end labels, enabling successful CAD reconstruction demonstrated in the curve only and MV2Cyl CAD outputs.
In Fig. A7 (b), the thin rod meet, failing to capture one base curve. This omission leads to predicted zero height of the instance, highlighting limitation in reconstructing that specifie section using curve data alone. On the other hand, the surface reconstruction successfully reconstructs this shape. We leverage the reconstructed barrel face of the instance to accurately predict its height (h) and center (c). This approach enables MV2Cyl to successfully reconstruct instances that the curve only model may miss.
that the curve-only reconstruction struggles at the (small) intersection where
Surface Only
Curve Only
Instance p uurface
Start-End
Output CAD
Instance p curve
Start-End gurve
Output CAD
GT
Ours
q s
Invalid Sketch
Invalid Sketch
Figure A7: Ablation study. Each example illustrates the limitations of surface and curve repre- sentations. The surface-only model, whose instance label point cloud, start-end label point cloud, and output CAD model are shown in columns 2, 3, and 4, The table shows the final output of the surface-only model and the curve-only model along with their semantic fields. The surface-only model failed to reconstruct the proper start-end field and thus the sketch is invalid. The curve-only model was able reconstruct the geometry in case (a) but predicted zero height for the thin cylinder in (b). Ours were able to make better estimations in both cases.
Table A2: Quantitative results from the ablation study on Fusion360 (64) dataset. The omission of curve fields Icads to significant performance degradation (the first row). The performance could be further improved by joint use of surface reconstruction. Red denotes the best; orange denotes the second best.
Fit.
Fit.
Method
E.A.())
E.C.|
E.H.L
Cyl.
Glob.l
Surface Only
4.1180
0.1179
0.2376
0.0984
0.0353
Curve Only
1.5612
0.0434
0.1508
0.0310
0.0232
Ours (Surface Curve)
1.3939
0.0385
0.13
0.0284
0.0212
To summarize, our synergy utilizing both surface and curve information allows MV2Cyl to effec- tively reconstruct instances where individual data types may fail, showcasing its comprehensive
16

---

reconstruction capability. Specifically, curve representation is effective against occlusion and pre- eisely captures the closed curve of sketch, making it suitable for reconstructing the 2D implicit sketch. However, it may not always be reconstructed successfully due to its localized presence within the scene. On the other hand, surface representation provides a more dependable basis for fitting a plane to determine the extrusion axis, which is vital for inferring subsequent CAD parameters. Yet, the base faces necessary for this process might not be available. This is either because subtractive boolean operations occlusion from combining multiple shapes. We thus address the challenges by jointly reconstructing curves and surface representations of shape and utilizing them to recover CAD parameters.
A.2 Separate 2D Segmentation Modules for Curve and Surface
In MV2Cyl, the curve and surface segmentation modules are kept separate. To evaluate how segmentation performance changes when the 2D curve and surface segmentation networks combined, we test a shared U-Net backbone that branches into two outputs for curve and surface segmentation. The results of using merged and separate U-Nets extract 2D segmentation maps for our reverse engineering extrusion cylinders task are shown in Tab, A3 While a slight improvement in performance observed are comparable.
metrics (Fit Cyl. and Fit Glob.), the results from both approaches
Table A3: Quantitative comparison with the U-Nets of surface and curve segmentation using Fusion360 dataset.
Fit.
Fit.
Method
E.A.C))
E.C./
E.H.
Cyl.
Glob.l
Combined U-Net
1.4425
0.0397 0.0385
0.1543
0.0279
0.0200
Separate U-Nets (Ours)
1.3939
0.1423
0.0284
0.0212
A.3 Ablation on the Number of Instances
We evaluate the impact of varying the number of instances on MV2Cyl's performance using the Fusion360 dataset. Tab. [ab.[A]] [4]sshos the results breakdown of the models in the Fusion360 test set across different numbers of extrusion instances K. Note that the number of samples for K 7 and =8 is very small, and these statistics should therefore be interpreted sparingly.
Table A4: Performance the changes of the number of instances using Fusion360 dataset. Number of Number of samples
Fit.
Fit.
E.A.(%)
E.C.J.
E.H.L.
instances
Cyl.
Glob.l
250
0.0000
0.0011
0,0615
0.0169
0.0169
494
1,4484
0.0325
0.1531
0.0277
0.0224
207
2.9046
0.0732
0.2092
0.0405
0.0261
93
1.8476
0.1092
0.2270
0.0466
0.0300
40
2,4250
0.0998
0.2065
0.0586
0.0294
15
9.0000
0.1301
0.1916
0.0601
0.0459
7
1.3265
0.0712
0.0620
0.0553
0.0364
2
16.2143
0.0130
0.0350
0.0250
0.0232
A.4 Ablation on the Number of the Input Images
In this section, we analyze the effect of varying the number of input images on MV2Cyl's performance using the Fusion360 dataset. Tab. [5] shows that our approach is robust to changes in the number of input images, maintaining reasonable performance even with as few as 10 input views. Although performance slightly degrades as number of input images decreases, MV2Cyl still produces effective reconstructions, demonstrating its adaptability to limited input data. As expected, the use of input images yields the best overall results due to the richer information available for reconstruction.
17

---

Table AS: Performance over the changes of the number of the input images using Fusion360 dataset.
Number of images
E.A.(%
E.C.4
E.H.
Fit.
Fit.
Cyl.1
Glob.
50 (Ours)
1,3939
0,0385
0.1423
0,0284
0.0212
25
1,4631
0.0421
0.1561
0.0293
0.0218
2.1548
0.0489
0.1784
0,0371
0.0259
10
2.2585
0.0573
0.2112
0.0515
0.0364
A.5
Ablation the Line Width of the Curve Segmentation Maps
In this section, we investigate the impact of varying the line width of curve segmentation maps on MV2Cyl's performance. Tab. T shoss that our method is relatively robust changes in line width, with performance remaining consistent across different widths. The results indicate that using pixels (our default setting) achieves good balance between accuracy and consistency. While slight variations in error metries are observed when using line widths of 2.5 or 7.5 pixels, these changes minimal, suggesting that MV2Cyl can handle range of line width settings without significant performance loss.
Table A6: Performance over the changes of the line width the curve segmentation maps using Fusion360 dataset.
Fit.
Fit.
Line width
E.A.C)4
E.C.4
E.H.
Cyl.
Glob.l
5 pixels (Ours)
1.3939
0.0385
0.1423
0.0284
0.0212
2.5 pixels
1.4886
0.0392
0.1482
0.0291
0.0216
7.5 pixels
1.4375
0,0410
0.1548
0.0307
0.0225
B Real- Demo
In this section, we showcase the feasibility of using MV2Cyl in reconstructing CAD models from real-world object captures. We provide details of the process as follows.
To reverse engineer real-world object, we begin by capturing videos the target object using an iPhone 12 Mini. These videos are then converted into multi-view images through frame extraction. which are subsequently input into the image segmentation models. The training dataset for our segmentation models is generated by rendering untextured models in Blender, producing RGB images with grayscale appearance against a transparent background. To address the domain gap between the training data and real-worke demo images, we preprocess the real images by converting them to grayscale and removing the background. While these preprocessing steps help align the datasets, a residual domain gap remains, which may still impact segmentation performance on real images. To address this challenge and enhance segmentation efficacy, we employ a robust 2D segmentation model known as Segment Anything (24) By fine-tuning the pretrained SAM model using LoRA-based finetuning code [71), with the same training dataset applied to the U-Nets, we observe substantial improvements in segmentation performance on real images, even in the presence of distributional shifts. The finetuned SAM model leverages extensive prior knowledge gained from training on a significantly larger dataset, ensuring robust performance 2D segmentation tasks. We acquire the camera's 's extrinsic and intrinsic parameters using COLMAP 52 53].
Fig. A8 displays an example of real image alongside the predicted segmentation maps from the finetuned SAM model. The processed multi-view images and their corresponding camera poses are then input into the reconstruction pipeline, which focuses on learning the density and semantic fields. Subsequently, CAD parameters are estimated based this reconstruction. Tab. presents results on final reconstructed CAD models in real captures, demonstrating successful output of CAD models from various real-world objects.
18

---

RGB
Grayscale image
Background removed
Instance segmentation
Start segmentation
Figure A8: Example of the inferred segmentation maps for real image (top row) and synthetic image (bottom row). The first image in the top row is the real image, which then grayscaled (second image) and bockgroundreemoeer (third image). The fine-tuned SAM [24] model segments the processed image into the instance segment (fourth column) and the start-end segment (last column). When comparing the real and synthetic images from similar viewpoints, the segmentation outcomes are similarly effective.
We quantitatively evaluate MV2Cyl real data found in Tab. A7 Our reconstruction achieves an average Chamfer distance of 2.11 x 10-3 from the ground truth CAD model. To compute this distance, first sampled 8192-point point clouds from the reconstructions and then aligned the reconstructed model to the corresponding ground truth shape using ICP. After alignment, the desired metrics were computed. This process is illustrated in Fig. A9
ground
Export real demo CAD output to mech
Sample point cloud Shope registation GT point cloud
Open3D KPP point cooud registration
Figure A9: Processing steps for aligning the pose of the real demo output with the ground truth CAD point cloud.
C Comparison with SECAD-Net [30
We further compare our MV2Cyl against an unsupervised CAD reconstruction baseline SECAD- Net [30]. Unlike our approach that takes multi-view images input, SECAD-Net takes 3D voxel grid as input. It trains a model that directly outputs shape as a union of predicted extrusion cylinders. We evaluate how well their approach reconstructs individual extrusion cylinders using the same metrics as in [58] and compare them against our MV2Cyl. Results are shown Tab. A8 We see that approach outperforms SECAD-Net [30] for both the Fusion 360 [64] and DeepCAD 65 datasets on all metrics that quantify extrusion cylinder reconstruction.
For fair comparison, we evaluated the test set released by (58 on both datasets and used the scripts from the code release of SECAD-Net 30 to convert the test shapes to voxelized inputs and train or fine-tune the SECAD-Ne model with our data. For both datasets, adhered to the evaluation
19

---

Table A7: Gallery of real demo examples. demonstrates the capabilities of MV2Cyl in reverse engineering real objects.
Real Image
MV2Cyl
Extrusion Cylinders
protocol from 30 fine-tuning SECAD-Ne for 300 epochs per test shape using pre-trained model, followed by mesh extraction 256 resolution with Marching Cubes [37]. We used Hungarian matching based on Chamfer distances to find the best match between ground-truit and predicted segments, which allowed us to calculate the metrics reported. The Fusion360 dataset evaluations were conducted on 798 test shapes, while the DeepCAD dataset included 1,950 test shapes. These numbers reflect instances where both MV2Cyl and SECAD-Net were able produce valid CAD models.
20

---

Table A8: Quantitative comparison with SECAD-Net using Fusion360 and Deep-
CAD 665| datasets. MV2Cyl consistently outperforms SECAD-Net [0 in all metrics. Red denotes
the best.
Fit.
Fit.
Dataset
Method
E.A.(")
E.C.
E.H.
Cyl. +
Glob.
Fusion
SECAD
42.5529
0.05881
0.6111
0.8378
0.7990
360
MV2Cyl (Ours)
1.4654
0.04270
0.1520
0.0283
0.0215
Deep
SECAD
43.2088
0.03406
0.5126
1.3627
0.6997
CAD
MV2Cyl (Ours)
0.2476
0.00616
0.0917
0.0238
0.0227
Tab. A8 demonstrates that MV2Cyl outperforms SECAD-Net 30 across all metrics. SECAD-Net lacks supervision for segmenting parts in the way CAD designers would, leading to issues such as over-segmmentttion. This evident in the images of Tab. A9 For example, in the second row, SECAD-Net segments two vertically attached cylinders into several patches along their surfaces. In comparison, MV2Cyl more accurately represents these components as two distinct cylinders, showing its improved ability to align with CAD design conventions.
Table A9: Qualitative comparison with SECAD-Net 30 on Fusion360 64 dataset. MV2Cyl outperforms SECAD-Net for segment the instances.
GT Mesh
Reconstruction
Extrusion Cylinders
MV2Cyl
SECAD
MV2Cyl
SECAD
MV2Cyl
SECAD
21

---

D Comparison with Curve + Point2Cyl [58]
We compare our MV2Cyl with an additional baseline Curve+Point2Cyl [58] that is a modified version of Point2Cyl that incorporates an additional "curve" feature into the input point cloud. While MV2Cyl benefits from the supervision of curve information from the CAD, we apply equivalent supervision to the baseline to maintain faimess in comparison. The evaluation was conducted using Fusion360 (64) dataset.
To implement the baseline Curve+Point2C) (described in the Sec. in the main paper), we modified the network's front-end where receives inputs. The original Point2Cyl takes an input tensor of size N 3, representing the target point cloud (see Fig. A10). To incorporate information about specific curves within the point cloud, we concatenate the NI curve points after the N input points. These curve points are assigned label of one on additional channel appended to the original three XYZ channels, while the original non-curve points retain a label of zero on this channel (see Fig. AIT. The rest of the network architecture remains unchanged. N is set to 8, 8,192 and M is set to 800. Notably, the curve points are obtained from the ground truth CAD models for both training and testing phases.
poins Narbha
Scamattriinn Hearkrrku
W SPAc
Be 1aa
Impal Poitt Choal PERto
bachasee
X CER
Ravader
Decoder
Skeech Inpikit
Parssncee Extiration
is
coantituts
Deemier
Inferring Cyllaske Parmetiess
Inferrigg Skrides
Arpuce
Inlestee
Figure A10: Network architecture of Point2Cyl 58
of peizb legnuses
Segrantation Rennbring
W gecc
Iaput Point Cland Curve
Be Bar
R C
Decoder
Sketch Implicti
Paruneker Extirudee
&
cauntituto
Inferting Listrudos Cyllader Parmmtiess
zi
Inforring Skeides
Iqqax
Yeatlo
Difforatablle
Trasing
Inteares
Figure A11: Network architecture of Curve+Point2Cy) 58
Tab. A10 shows the quantitative comparison between MV2Cyl and Curve+Point2Cy! MV2Cyl
surpasses Curve+Point2Cyl across all the metrics. Despite incorporating ground truth curve features, Curve+Point2Cyl performs similarly to the original Point2Cyl. In practical applications, it might
22

---

be expected to use predicted curves or edges; however, even with ground truth curves, we observed no significant improvement over the original Point2Cyl, which lacks eurve data. This suggests that Point2Cll's network may prioritize surface points over boundary points, making the additional boundary information less impactful.
Table A10: Quantitative comparison with Curve+Point2Cy/ using Fusion360
664 dataset.
MV2Cyl surpasses Curve+Point2Cyl across all the metrics. Red denotes the best.
Fit.
Fit.
Method
E.A.()4
E.C.I
E.H.L
Cyl.4
Glob.l
Curve+Point2Cyl
7.3253
0.0875
0.3001
0.0792
0.0327
MV2Cyl (Ours)
1.3939
0.0385
0.1423
0.0284
0.0212
E
Implementation Details
In this section, we elaborate the implementation of the networks presented in the main paper. Sec. E.1 contains the details of the training 2D segmentation frameworks (Sec. 3.2 in the main paper) and Sec. includes the details of the 3D field learning from the 2D segments (Sec. 3.3 the main paper).
E.2
E.1
Details of 2D Segmentation Frameworks
Each 2D segmentation framework Msuuface and Muuuee includes two U-Nets for instance segmentation plype and htrt-end((baree) segmentation Qtyee where the type is either surface curve, respectively.
Details of 2D Surface Segmentation Framework Musurface
The 2D surface segmentation framework, MMsurface, maps RGB images of dimensions H x W x 3 set comprising the extrusion instance segmentation label, psurface E {o,.... K.-.,KKJ #) and the start-end-barrel segmentation label Qsurface E [0, 1, 2, 1,2
where K indicates the number of 3 HxW
instances. We fix K 8 instances following [58] and train with image resolutions of H W 400. For the Fusion360 dataset, 196,050 training images derived from 3,921 shapes are used, while for the DeepCAD [65] dataset, derived from subset of the training set, 709,000 images from 18,180
shapes are utilized. The datasets are divided into training and validation sets in a 9:1 ratio for the U-Net [50] training. The training times for 3 epochs are 316 minutes for the Fusion360 64 dataset and 381 minutes for the DeepCAD dataset, using single NVIDIA RTX A6000 GPU.
Details of 2D Curve Segmentation Framework M
The 2D curve segmentation framework, M processes RGB images (H W x 3) produce W and the Similarly following [58] we also set K 2, HxW
two types of labels: the extrusion instance segmentation label, pourve E {O..... K) M start-end segmentation label, Qeure E (0, 1,
with training and validation image resolution of 400 400. The training objective is defined as:
2D XcECanss-nnogp Afocal + Adice
where ACE, Aocal and Adice are all set to 1.0, balancing the contributions cross-entropy, focal 33 and dice [41] losses to the overall training objective.
For the Fusion360 dataset, we utilize 196,050 images from 3,921 shapes. For the DeepCAD dataset, 1.745,45 images from 34,909 shapes from the training set. These image datasets are split into training and validation sets at 9:1 ratio. The Fusion360 dataset's U-Net is trained over epochs, taking 307 minutes, while the DeepCAD dataset's U-Net training for single epoch requires 20.8 hours, all on a single NVIDIA RTX A6000 GPU.
E.2 Details of Integrating 2D Segments to 3D Fields
The opacity field (x) hyper-parameters are set to Ç = 10 and 8 = 0.8 as in 68 reconstruction loss of the density field F, we set Aaparsitty
For the 0.5, = 0.5, Thatch 0.1, and
Timuge = 0.05. We use batch size of 8,1 rays for the training of both the density field F and the semantic field A. We trained all the fields for 1500 iterations on a single NVIDIA RTX 3090 GPU. The whole training time is approximately 5 minutes per shape. Note that the segmentation process
23

---

and the learning of object density and semantics are typically done as separate steps, rather than in single, continuous process.
E.3 Reverse Engineering Implementation Details
We provide additional details on our reverse engineering process, i.e. from our density and semantic fields to the reconstruction of the extrusion cylinders, as discussed in Sec. 3.4 the main paper. This process begins by sampling points from the density fields, and then from the extracted point clouds deriving the corresponding parameters for each extrusion cylinder. We query the learned density field value across 400 400 x 400 grid. Points this grid with an opacity value greater than threshold т 2 0.99 are selected and their corresponding instance segmentation and sart-end((bbrrel)) segmentation labels are extracted. The extrusion axis (n), height (h), and center (c) are determined directly from the sampled point clouds as detailed in the main paper.
The process to extract the 2D implicit sketch involves additional steps detailed as follows: After predicting the extrusion axis, the 2D sketch boundary points are created by projecting the base curve points along this axis. These 2D point clouds are then used to infer the 2D implicit sketch using IGR 15]. which takes as input the projected 2D sketch point clouds and their normals to generate the corresponding signed distance fields. To extract the 2D point normals for IGR, we modify PGR (32 to estimate 2D normals from our projected 2D sketch points. We adapt the algorithm from PGR and modified 2D basis as follows:
X
i exx-y,le yyl w(x)
(10)
otherwise
where wxx) is the width function defined in PGR. For an in-depth understanding of the formula and its derivations, we direct readers to the PGR paper [3). Fig. displays the impressive results of our 2D normal estimation implementation, showcasing the accuracy and quality our approach. These 2D point clouds and the estimated normals are fed into IGR network to output the 2D implicit sketch represented as 2D signed distance function.
YY
Figure A12: Example of input 2D point cloud (top row) and optimized normal estimation (bottom row).
E.4 Multi View Images Preparation Details
We detail the generation of multi-view image datasets used in our experiments. Adapted from 668 the rendering process utilizes BlenderProe 7 and we render 50 views with resolution 400 x 400
for each shape, where the viewpoints are sampled by Fibonacci sampling [17] evenly distributed
24

---

over sphere. The surface and curve segmentation labels, we start off by coloring the ground truth mesh black, then marking each segment white. The meshes marked per instance are then rendered in diffuse mode to produce highlight map for each segment from all viewpoints. Finally, the highlight maps are merged into a single segmentation map, creating multi-view segmentation maps consistently annotated with GT labels across views.
E.5
NeuS2 62
Point2Cyl
58 Implementation Details
We provide additional details on the baseline NeuS2 + Point2Cyl. There are many recent and high- performing studies on converting multi-view images to 3D geometry 40
33, and chose NeuS2 because this work is based on InstantNGP 44 and offers fast convergence in reconstructing shapes. For instance, while NeRF [40 takes about hours reconstruct a sample from the DTU dataset [20. NeuS2 can achieve more accurate geometry in just 5 minutes. It has been confirmed that NeuS2 achieves more accurate geometry than InstantNGP, which has converged the same duration. We selected this study considering the efficiency and feasibility of evaluation. In the test with NeuS2+Point2yyl, we used a set of images with view points per shape, at a resolution of 400 x 400, and trained for 20,000 iterations. We then ran a marching cube at resolution of 64 to obtain the mesh and sampled 8,192 points from that surface to apply as input to Point2Cyl.
E.6 Binary Operation Prediction
We introduce a straightforward method for recovering binary operations. naive approach involves performing an exhaustive search across all possible combinations of primitives and operations, resulting 2 possibilities for a model with K primitives. aligns closely with the observed input images, we measure the average per-pixel I.2 distance between the rendered images of each combination's resulting model and the observed multi-view images, selecting the combination with the lowest distance. We have implemented this approach to recover binary operations, and examples of the results are illustrated Fig. A13
identify the optimal combination that
Input Image of the corrispondnng camera pase
Reconstructed part
Part
Part
2 combinations of reconstructed parts
Rendering Lsing input mutiview Images' camora poses
Comba
Comn RutD PLZ(1
Cork3 21
Comk& MSTeeL1()20
Comb RinO]:: PL26))
Coma.s FarDec 196.3
Combut herd 26,20)
Par 2an0ee.119.
The best combination with the minimal disence from the given imput iage
Figure A13: Example of the binary operation prediction of the reconstructed extrusion cylinders.
E.7 Implicit sketch to parametric curves
We also provide example showcases of converting an implicit sketch into parametric representation. After generating the 2D implicit field for each sketch, we sample boundary points using the marching squares algorithm, which serve as knots for cubic Bézier curves. Using these points, we employ an off-the-shelf spline fitting module from the SciPy library [59] to determine the optimal B-spline representation based on the provided control and knot points. This process is illustrated in Fig. A14
25

---

It is important to note that the resulting sketch may not comprise the minimal set of parametric curves; for example, a single line segment may be represented by multiple segments.
Figure A14: Recovered parametric models with sketch represented as cubic spline.
F Additional Analysis
F.1
Comparison of Segmentation Performance Between 2D and 3D Networks
We evaluate the segmentation performance of networks in MV2Cyl and Point2Cyl process 2D images and 3D point clouds, respectively. The 2D segmentation network MV2Cyl processes 400 x 400 resolution images, categorizing them into background(O)), start face(1), end face(2), and barrel face(3). It was tested on 55,400 images from 1,108 shapes. Conversely, the 3D segmentation network in Point2Cyl processes 8, 19122-point point clouds, segmenting them into (ar and base(1) points, with 1,108 point clouds evaluated. Table/ATT details the test accuracy and test accuracy excluding the background for the 2D network. Notable, the 2D segmentation network outperforms the 3D segmentation network, supporting our hypothesis that 2D networks more effective than 3D networks segmentation tasks, particularly in extracting the spatial information needed to reconstruct CAD model.
58 which
Table A11: The test accuracy of base-barrel segmentation networks
Test Accuracy
Method
Test Accuracy (%)
Excluding Background (%)
Point2Cyl
88.27
N/A
MV2Cyl (Ours)
99.15
96.58
F.2
Consistency Between Curve and Surface Segmentations
We compute a consistency metric between the curve and surface predictions. Specifically, we compute the percentage of models where the number of predicted curve instances and surface instances are identical. Across all the test shapes in the Fusion360 dataset, 98% the models (1,095 shapes over 1, 1,108 shapes) have the the consistency between the extracted curve and surface point clouds. Concretely, we measure the average one-way Chamfer distance between the extracted curve point cloud and surface point cloud (0.081 x 10-3), We see that this very small value suggests that the extracted curve point clouds are quite consistent with the surface point clouds.
number of curve and surface segments. Moreover, also measure
E33 Chamfer Distance Evaluation for 3D Reconstruction
We quantitatively evaluate the reconstruction quality of MV2Cyl by measuring the Chamfer distance against the ground truth CAD model. To generate the output CAD model, binary operations
26

---

are determined through simple exhaustive search approach. The results, shown in Tab. A12 demonstrate that our method outperforms the baseline, NeuS2+Point2Cy.. For fair comparison, we compare with NeuS2+ instead of NeuS2, as our work focuses on the reverse engineering of extrusion cylinders, which is more challenging task than 3D reconstruction without CAD structure. Reverse engineering extrusion cylinders enables shape editing and importing the model back into CAD software.
Table A12: Chamfer distances comparisons using Fusion360 and DeepCAD datasets. The chamfer distance values are multiplied by 10a.
Dataset
Method
Mean CD (L)
Median CD (J)
NeuS2+Point2yl
105.18
51.88
Fusion360
MV2Cyl (Ours)
49.08
9.40
NeuS2+Point2yl
106.59
17.21
DeepCAD
MV2Cyl (Ours)
18.21
1.43
G
Additional Qualitative Results
We provide additional qualitative visualizations of the reconstructed extrusion cylinders using our proposed MV2Cyl. Results are shown in Tab. A13 It showcases MV2Cyl's exceptional ability to process and accurately render shapes of diverse appearances stration underscores the versatility and precision of our approach in capturing and reconstructing complex geometries from multi-view images.
instance-wise basis. This demon-
27

---

Table A13: Visualization of the outputs of MV2Cyl. It showcases the superior performance of segment the instance and infer the accurate CAD parameters. Extrusion Cylinders
GT Mesh
MV2Cyl
28

---

GT Mesh
MV2Cyl
Extrusion Cylinders
29

---

GT Mesh
MV2Cyl
Extrusion Cylinders
30

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The main claims presented in the abstract and introduction are consistently reflected and elaborated upon in the main text.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitations of the work outlined in the conclusion section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
31

---

Justification:
This work does not include any theoretical assumptions.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The detailed procedure of the method delineated the method section and appendices.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
32

---

Answer:
[No]
Justification:
The code and data will be made available upon the acceptance of this work.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/t.pss.ccppullc//uuiii / Soaileesss/oSSSeeiiieeeeee
wwt.SSubmisioooooiiooooooooo for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The training and test details are explained in the main text and the appendices.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Due to the high computational demands of the neural rendering- based 3D reconstruction step in this work, iterating over multiple trials is challenging because of limited time and computational resources.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
33

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: The computer resources required for the experiments are described in the experiment section and the appendices.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics htttpp///eurrpp..c.ppol.///llht
Answer: [Yes]
Justification:
This research adhered to the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: discuss societal impacts in the last section of the main paper. Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
34

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This work does not pose such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All existing assets used in this work properly cited in the text.
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
35

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This work does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This work does not involve any crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
36


Reviews:
[{"review_id": "RsFJjMuYFe", "reviewer": "Reviewer_Ajxc", "summary": "## Summary of the Paper:\n\n*Problem Statement*:\n\n     Given multi-view images (RGB) of a 3D shape, the paper aims at recovering a *set* of extrusion cylinders to represent the underlying 3D shape.\n\n\n*Motivation*:\n\t\n     Existing works that address 3D shape reconstruction problem through sketch-extrude take raw 3D geometry as input (ex: Point2Cyl) whereas this work takes multi-view images. Other works that take multi-view images as input perform parametric reconstruction (ex: NEF), which while important, does not reconstruct the 3D surface itself. And so, there exists a gap in the community in terms of either the input representation, or the output result, in the context of 3D reconstruction through sketch-extrude modeling. This forms the basis of this work, termed, MV2Cyl.\n\n*Contributions*:\n\n     Developing a method for sketch-extrude reconstruction of 3D shapes from multi-view images, via labeled surfaces and curves. \n\t\n\n*Input*: \n\n     Multi-view RGB images of a man-made 3D shape\n\n*Output*: \n\n     3D reconstruction of the object based on sketch-extrude cylinders. A cylinder is \n\n*Dataset used*: \n\n     Fusion 360 and DeepCAD\n\n*Underlying Modeling Tool*:\n\n1)\t U-Net for processing images (segmenting curves and surfaces)\n\n2)\t Implicit field-based volumetric rendering for integrating 2D estimated curve and surface information to corresponding 3D space\n\n\n*Learning Mechanism*:  \n\n     Strongly supervised\n\n\n*Loss Functions*:\n\n     For 2D surface segmentation, a U-Net with multi-class cross-entropy loss constrained by Hungarian matching algorithm is used. The Hungarian matching constraint is added to account for order invariance of the extrusion segments on the rendered 3D shape, as well as the to classify start-end-barrel segments.\n\n     The 2D curve segmentation module also follows the same architecture and loss functions. In addition, due to the sparsity of labels for curves (which can cause the network to just “attending” to the background pixels), two additional loss terms are used – Dice Loss and Focal Loss\n\n     2D to 3D integration using volumetric rendering: Here, the paper describes two fields. One is the Existence field and the other is the Attribute Field. \n\n     The training of the Existence field is dictated by the input 2D images, which is accomplished via differentiable rendering. A weighted combination of L2 loss (for surface) and sparsity loss (for curves) is used.\n\n     The training of the Attribute filed is governed by a multi-class cross-entropy loss function, constrained by Hungarian matching.\n\n\n*Quantitative Metric*:\n\n     Extrusion-axis error (E.A.), extrusion-center error (E.C.), per-extrusion cylinder fitting loss (Fit Cyl.), global fitting loss (Fit Glob.)  \n\n\n*Baselines*:\n\n     Since existing works do not take multi-view images for sketch-extrude 3D reconstruction tasks, no suitable comparison exists. However, the paper compares against a closely related work (and its variant), Point2Cyl (+NeuS2), which takes a point cloud as input.", "questions": "1)\t I do not see RGB inputs (as claimed on L 105) in the Teaser as well as Fig 1. I see just a grayscale image. Which one is it?\n2)\t How do you differentiate NEF (CVPR 2023) from the 2D curve segmentation framework? The curves that are of interest can be weakly considered as edges. Can you replace 2D curve segmentation network with NEF (part/whole) to detect the extrusion curves and the base curves?\n3)\t Dataset question (L259- 261)– how were these datasets enriched with 2D segmentation maps? That is, how were the ground truth 2D curve and extrusion segments obtained? Manually? Provide details.\n4)\t I am curious to know why ABC dataset was not considered. It is a highly relevant dataset that would have made the comparisons exhaustive. What was the reasoning here? \n5)\t Can you talk about the extendibility of this approach? I ask this question since I find it non-trivial to train the Existence and Attribute fields, as described in the paper.", "limitations": "Yes, the authors have correctly addressed the limitations of the work, including any societal impact that may arise as a result of the publication.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1)\t Well-written paper\n\n2)\t Suitable comparisons have been made with Point2Cyl (and +Neus2) to investigate the apparent gains rendered by the proposed framework\n\n3)\t Experimental results on two relevant and complex CAD datasets (Fusion-360 and DeepCAD) are better than other closely related works", "weaknesses": "1)\t I wanted to understand how essential the 2D curve segmentation network is in making the reverse engineering possible. In other words, how would the cylinder extrusion results in 3D look like if this module were not present? This is a critical piece of experiment, something that should have been shown through ablation experiments. A similar ablation experiment for 2D surface estimation is NOT necessary though, since it is straight-forward to see why this is needed in the first place, given multi-view images as inputs.\n2)\t There is a lack of discussion on which kind of shapes does the proposed approach fail on. I would have liked to see a figure (with MV images, 3D GT and sketch-extrude 3D reconstructions) that shows the kind of shapes that MV2Cyl fails at, even if such results are just a tiny sample. This is consequential to the research push in this direction, and also does not do complete justice to the paper.\n3)\t In Table 1, the different quantitative metrics lack corresponding units. For ex., as I understand, the extrusion axis error (E.A) is an angle measurement. The reported number must have been in degrees, yes? Likewise, it would be good to see units of measurement, if applicable, for other metrics reported therein. If you want to save space, you could just put it in the table, instead of mentioning in the caption.\n4)\t The extrusion instance segmentation is performed by both 2D segmentation modules (Surface and Curve). I sense that there is a strong dependency on this task to help complete the other “auxillary” task in these modules. As this (extrusion instance segmentation) is a common task for the two, were there experiments done to assess if these two modules (Surface and Curve) could be merged? To me, there is something amiss. Would like to see an intuitive explanation justifying this two-step, common-task-occurrence justification.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "- **2D curve segmentation network ablation.** \nSee Table A2 and Figure A4 in the supplementary for an ablation on the necessity of the 2D curve segmentation network; referred to as the “Surface only” approach. We see that without the curve segmentation module, occlusions between the extrusion cylinder instances can result in missing base faces leading to poor reconstruction.\n- **Failure cases.** \nSince MV2Cyl relies on 2D image inputs, it’s susceptible to occlusion. Specifically, if one side of an extrusion cylinder is totally hidden by the other ones, our 2D segmentation model cannot catch the hidden side. In this case, the extrusion cylinder cannot be reconstructed. In Figure. 5, the left shape is the target CAD model and the right one is the reconstructed CAD model by MV2Cyl. While the target shape has an inset hexagonal cylinder whose one end is hidden by the outer one, MV2Cyl failed to reconstruct the inset extrusion cylinder. \n- **Units for quantitative metrics.**\nYes, the reported extrusion axis error is in degrees. We will include the corresponding units in the final version.\n- **Merging the 2D segmentation modules.**\nThank you for the suggestion. We experiment with combining the 2D curve and surface segmentation networks. We use a shared U-Net backbone that then outputs two separate branches for curve and surface segmentation. Results on using merge and separate UNets to extract the 2D segmentation maps for our reverse engineering extrusion cylinders task are shown in Table 2. While we see that there is a small improvement in performance on some of the metrics (Fit Cyl. and Fit Glob.), the results on both approaches are comparable. We will include these results in the final version.\n- **Clarification on RGB input.**\nOur segmentation networks take as input a 3-channel image, i.e. RGB. Our training data was constructed by rendering **untextured** models using Blender, which is why the images look like they are grayscale. For real data that may have texture, we first convert the input image into grayscale to bridge the domain gap. We find that this does not harm the performance and we are still able to achieve reasonable curve and surface segmentations as shown in Figure 2.\n- **Difference between our curve module and NEF.**\nThe main difference between our curve module and NEF is NEF learns a view-dependent color term alongside its edge density field, while our curve module learns a semantic/curve instance segmentation field together with the curve/edge density field. We further note that for the 2D network, NEF utilizes PiDiNet as an edge detection network, while our approach trains a U-Net for curve instance segmentation. \n- **How the 2D curve segments are obtained.**\nSee Section A5.4 in the supplementary materials for details on multi-view image and curve/surface segmentation map data preparation.\n- **ABC dataset.**\nWe ran our method on the DeepCAD dataset, which is a **subset of the ABC dataset** as provided by [Wu et al., 2021] that only includes the sketch-extrude models. \n- **Extendibility of the approach; non-trivial to train existence and attribute fields.**\nThank you for the inquiry. Could you clarify what you mean by extendability? We are happy to answer this in the discussion phase. On training the existence and attribute fields, given obtained 2D curve and surface segmentation maps, e.g. from our U-Net, the optimization of the existence and attribute fields becomes straightforward, that is a standard NeRF-model optimization.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "APF0zycvAn", "reviewer": "Reviewer_AtoY", "summary": "This paper proposes a method to predict extrusion cylinders from images of a CAD part. Specifically, it takes a set of masked multi-view images as input; these are then processed independently by instance segmentors trained to find extrusion curves and surfaces. Neural fields are then fitted, that reconstruct these 2D segmentations in terms of 3D segmentations. Lastly, a set of primitives explaining the 3D segmentations is extracted using a series of heuristics (RANSAC and other hand-crafted robust estimators). Experiments are conducted on two standard benchmark datasets (Fusion 360 and Deep CAD), and show the proposed method out-performing a baseline approach (NeuS reconstruction followed by an existing 3D primitive-prediction pipeline).", "questions": "How/where do you predict whether each primitive is 'added' or 'subtracted' from the model? The results in Fig. 3 indicate that the model correctly inserts holes, but it is unclear in the method where this determination is made", "limitations": "These are discussed briefly but adequately in Sec. 5.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "- The work presents a sophisticated and carefully engineered pipeline, with choice of components clearly motivated.\n- The approach of fitting several 3D fields to the 2D segmentations, and regularising to ensure these are binary, is interesting.\n- There is a fairly extensive evaluation (qualitative and quantitative) on synthetic (rendered data), including comparison against a baseline that first performs 'naive' surface reconstruction, then applies a 3D-only primitive extraction to find the extrusion cylinders.\n- Quantitatively, the proposed method is found to significantly out-perform the reconstruction-then-fitting baseline. Indeed, it also out-performs a method that directly predicts extrusion cylinders from ground-truth 3D point-clouds, which is a notable success since the latter would seem to have access to more complete information\n- Qualitatively, the proposed method appears to correctly capture geometric features of the input objects, using an appropriate (minimal) set of primitives, and providing much cleaner reconstructions than a naive \n- The paper is well-written, sensibly organized and pleasant to read throughout.", "weaknesses": "- The work focuses on clean synthetic (rendered) inputs, with only one experiment (in the appendix) on real data. However, operating on multi-view images offers no benefit over 3D-input methods when one has access to the 3D shapes (e.g. as point-clouds). For a strongly 'applied' paper like this, the lack of focus on the practically-relevant scenario (i.e. actual photos) is problematic.\n- Real-world results (though still with in-distribution CAD shapes 3D-printed from the datasets) are not very impressive, despite fine-tuning for this setting. There is also no quantitative evaluation her, which is a pity since this would be straightforward given that the ground-truth CAD models are available (modulo 3D printing imperfections)\n- There is no quantitative evaluation of the quality of 3D reconstruction in the sense of distance (e.g. hausdorff / f1) from the reconstructed shape to the original CAD model. It would be valuable to add this (and compare with NeuS2), to give an idea of how well the shapes are being predicted in a more universally-meaningful metric, as opposed to the given metrics that are specific to the shape representation and difficult to interpret in a broader context.\n- There are few generalisable insights present in the paper. It presents a pipeline that evidently works well for the specific task, but the technical contributions are very narrow in applicability, reducing potential impact.\n- Overall the paper is a poor fit for NeurIPS – lots of engineering (obviously an achievement in itself), but minimal machine learning (just training a segmentation model on synthetic data, then fitting the single-scene neural fields), making the interest to the community relatively low.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "- **Work focuses on clean synthetic data; lacking real data experiments.**\nWe note that it is difficult to obtain real data with ground truth CAD (sketch-extrude) parameters. In fact, to the best of our knowledge, such a dataset does not currently exist. Hence we opt to use synthetic data to train and test our method. We provide more real data examples in Table 6 (rebuttal) which supplements the real-world examples in Table A3 (supp).\n- **Benefits of multi-view images over 3D inputs.**\nWe note that images are easier to capture and acquire such as with our mobile phones or digital cameras compared to 3D point clouds that require a depth camera or LiDAR scanner. Moreover, we also compared directly with Point2Cyl, which takes 3D point clouds as input, in Table 1 (main) and showed that MV2Cyl achieves better performance due to the superior performance of 2D networks compared to 3D backbones on the segmentation task, as shown in Table A7 (suppl.).\n- **Quantitative results on real data.**\nWe quantitatively evaluate MV2Cyl on real data found in Table A3 (supp). Our reconstruction achieves an average Chamfer distance of 2.11*10^-3 from the ground truth CAD model. To compute this distance, we first needed to align the reconstructed model to the corresponding ground truth shape with the reconstructed model using ICP, after which the desired metrics can be computed.\n- **Quantitative evaluation on 3D reconstruction using distance-based metric.**\nWe quantitatively evaluate the 3D reconstruction quality of MV2Cyl by measuring the Chamfer distance against the ground truth CAD model. To obtain the output CAD model, we obtain the binary operations through a simple exhaustive search approach (see “how to predict binary operations” below). Results are shown in Table 5, where our approach shows superior performance against the baseline, NeuS2+Point2Cyl.  We note that for fair comparison, we compare with NeuS2+Point2Cyl instead of NeuS2 as the focus of our work is the reverse engineering of extrusion cylinders, which is a harder task compared to just 3D reconstruction without the CAD structure. Reverse engineering of extrusion cylinders allows for shape editing as well as importing the model back into CAD softwares. \n- **Generalization insights to the paper/narrow applicability.**\nThank you for acknowledging the generalizable insights presented in our paper. We believe that our approach, which leverages 2D image segmentation and neural rendering for 3D structure reconstruction, can be extended to a broader range of CAD models and thus inspire future research. Additionally, extrusion cylinders are a representation widely used in the CAD industry, and their reconstruction has also been extensively studied, as described in our related work section. \n- **Fit to Neurips.**\nIn this work, our main idea is to demonstrate how 2D priors can aid in reconstructing 3D structures when combined with neural rendering and to explore the best way to integrate different 2D priors (such as surfaces and curves) in 3D space. Although we focus on reconstructing extrusion cylinders in this study, we believe that the ideas presented have the potential to influence future research in many areas of machine learning. We will revise the introduction of our submission to clarify this point.\n- **Binary operation prediction: determining whether each primitive is 'added' or 'subtracted' from the model.**\nSimilar to Point2Cyl, our approach did not explicitly predict the binary operations of the primitives. We further provide a simple approach to the binary operation recovery: one naive approach is to do an exhaustive search against all possible primitives-operations combinations (2^K possibilities for a model with K primitives) and take the best combination, which is closest to the observed (input) images. To obtain the best combination, we measure the average per-pixel L2 distance between the rendered images of each combination’s resulting model against the observed multiview images and select the lowest one. We implement this approach and obtain the binary operations, examples are shown in Figure 3. We will include this in our final version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4iyi0pdjCL", "reviewer": "Reviewer_mRwh", "summary": "MV2Cyl is a method that proposes to solve the 3D reverse engineering of CAD models. The network takes as input multi-view images and outputs extrusion cylinders. The method extends Point2Cyl [58] that proposed to predict extrusion cylinder from point clouds. It is argued that multi-view images can easily be obtained from 3D scans and that 2D CNNs have superior performance than 3D processing networks.\nThe network is composed of three main components. Firstly, 2D CNN U-Net-based networks learns to segment and classify the pixel image of the 2D images into surface and curve segments. Then, a NERF based approach is used to leverage the 2D segments into 3D existence and attribute fileds. Finally, 3D surface and curve point clouds are extracted from the 3D fields. The extrusion parameters (extrusion axis, height and centroid) and the sketch (implicit function) are estimated from the extracted point clouds.\nThe method is evaluated on the DeepCAD[64] and Fusion360[63] datasets, from which multi-view images are extracted. The results show an improvement compared to Point2Cyl[58] and a Neus[51]+Point2Cyl.", "questions": "- The representation of the output is a little unclear. Figure 3 suggests that the output of MV2Cyl is a CAD model. Can a CAD B-Rep be obtained from the output? If so, how are the sketch primitive and parameters obtained from the predicted implicit functions?\nBeyond the points mentioned in the weaknesses, it would be interesting to know how the perfomance of the model changes with respect to the number of input images at test time. \n\n- Many qualitative results are presented in the Appendix, all of them show very impressive predictions. It would also be interesting for the reader to see examples for which the model fails to recover the correct shape.", "limitations": "- The authors have addressed one of the main limitations of the work that is the lack of prediction of the binary operations between the different cylinders. Potential societal impacts have also been included.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper is fairly clear and easy to understand for a reader that is familiar with Point2Cyl [58]. \n\n- The main strength of the proposed work is that it shows an improvement to the results presented in Point2Cyl [58]. This is achieved by leveraging the information provided in multi-view images as opposed to 3D point clouds.", "weaknesses": "- While the proposed work offers improvements to Point2Cyl [58], the reviewer feels that it does not address the main limitations of Point2Cyl. Firstly, the binary operations between cylinder (how to assemble the cylinders into the GT CAD model) are not predicted. This limitation is mentioned in the paper but addressing it would have lead to a significant improvement to Point2Cyl. Secondly, the sketches are predicted as implict functions and not parametric curve primitives such as in real CAD models. This produces noisy surfaces (as shown in Figure 3 of the paper), which are quite different from the standard sharp representation of CAD models.\n\n- The evaluation consists of a comparison with Point2Cyl using some of the metrics proposed in Point2Cyl. Nevertheless, further analysis of the results could be completed. For example, it could be meaningful to compute a consistency metric between the surface and curve predictions. One would expect the two network to predict the same number of extrusion cylinders. Also, are the extracted curve point clouds consistent with the surface point clouds? A comparison to the point cloud to CAD sequence experiment presented in DeepCAD[64] could also be included as the extrusion cylinders can easily be extracted from the CAD sequences. It would also be interesting to know how the performance of the network changes with respect to the number of extrusion cylinders (K) present in the ground truth.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "- **Binary operations.**\nThe binary operations can be recovered in a simple/straightforward approach such as an exhaustive search against all possible primitives-operations combinations (2^K possibilities for a model with K primitives). We take the best combination as the output configuration, which is the combination that is closest to the observed images. To obtain the closest/best combination, we measure the average per-pixel L2 distance between the rendered images of each combination’s resulting model against the observed multiview images and select the lowest one. We implement this approach and obtain the binary operations, examples are shown in Figure 3. We will include this in our final version.\n- **Sketches: from implicits to parametric curves.**\nWe demonstrate how to convert an implicit sketch to a parametric representation. After acquiring the 2D implicit field for each sketch, we sample boundary points using the marching squares algorithm and use them as knots for cubic Bézier curves. Utilizing these points, we employ an off-the-shelf spline fitting module implemented in Scipy library which finds the best B-spline representation with the given control/knot points. This process is illustrated in Figure 4. Note that the resulting sketch may not consist of the minimal set of parametric curves; for example, a single line segment might be divided into multiple segments. Optimizing this aspect is left for future work.\n- **Analysis of results: consistency metric.**\nThank you for your suggestion. We compute a consistency metric between the curve and surface predictions. Specifically, we compute the percentage of models where the number of predicted curve instances and surface instances are identical. Across all the test shapes in the Fusion360 dataset, 98% of the models (1,095 shapes over 1,108 shapes) have the same number of curve and surface segments. Moreover, we also measure the consistency between the extracted curve and surface point clouds. Concretely, we measure the average one-way chamfer distance between the extracted curve point cloud and surface point cloud (0.081 x 10^-3). We see that this very small value suggests that the extracted curve point clouds are quite consistent with the surface point clouds.\n- **Comparison against DeepCAD’s point cloud to CAD.**\nThank you for suggesting a comparison with DeepCAD’s point-cloud-to-CAD framework. We attempted this comparison but encountered several challenges. First, the pretrained model for DeepCAD’s point-cloud-to-CAD framework was not available. Second, retraining the DeepCAD model with our dataset took several days, which was difficult to manage within the short rebuttal period. Third, we tried using an unofficial pretrained model from GitHub, but the resulting CAD outputs were completely different from the input point clouds, suggesting possible bugs in the training process. We plan to contact the authors of DeepCAD to resolve these issues and achieve a fair comparison.\nDespite these challenges, we strongly believe that our method—segmenting 2D images, unprojecting them to 3D space, and fitting extruded cylinders rather than directly predicting them—will demonstrate superior performance.\n- **Performance w.r.t. different number of extrusion cylinders (K).**\nTable 1 shows the results breakdown of the models in the Fusion360 test set across different numbers of extrusion instances (K). We see that the general trend is that the error steadily increases as with larger K, i.e. as model difficulty increases. (Note that the number of samples for K=7, 8 are very small; please take these statistics sparingly.) \n- **Output representation.**\nThe output representation is a set of extrusion cylinders. Specifically, for each extrusion cylinder, we have its center (R^3), extrusion axis (SO(3)), extent (R^2) and a sketch represented as an implicit (which can also be converted into a set of parametric curves (see above response). Furthermore, for each extrusion cylinder, we can also recover its operation, which is either additive or subtractive, (see “Binary operations” response).\n- **Ablation on the number of input images.**\nTable 3 shows an ablation of MV2Cyl for a different number of input images at test time. Results show that our approach is not that sensitive to the change in number of input images and still achieves reasonable performance for as low as 10 input views.\n- **Failure cases.**\nSince MV2Cyl relies on 2D image inputs, it’s susceptible to occlusion. Specifically, if one side of an extrusion cylinder is totally hidden by the other ones, our 2D segmentation model cannot catch the hidden side. In this case, the extrusion cylinder cannot be reconstructed. In Figure. 5, the left shape is the target CAD model and the right one is the reconstructed CAD model by MV2Cyl. While the target shape has an inset hexagonal cylinder whose one end is hidden by the outer one, MV2Cyl failed to reconstruct the inset extrusion cylinder.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mnbyMsnwCy", "reviewer": "Reviewer_7GUw", "summary": "This paper proposes a new method to reconstruct extrusion cylinders from multiview images. The key idea is to train a CNN to predict the binary mask for each surface and the sketch of each surface. Then, these predictions are used in learning 3D neural fields for each surface and sketches by the volume rendering technique.", "questions": "Refer to Weakness.", "limitations": "Limitations are discussed in the paper, which is the binary prediction of CNNs. I think the method could be sensitive to hyperparameters, which could be a potential limitation.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The task seems to be interesting in that we use multiview images with semantic segmentation to help reconstruct 3D extrusion cylinders. This method is novel for me.\n2. The experiments demonstrate some improvements over previous reconstruction methods.", "weaknesses": "1. How to associate each surface is unclear here. If I understand it correctly, the predicted labels are similar to instance segmentation labels so we may have multiple different labels for the same region. How to associate predicted surfaces with different viewpoints is not clear here.\n2. The prediction of feature lines could be sensitive to the line width. If we use a large line width the localization of the feature lines could be difficult because we cannot learn a perfect 3D field for these lines. If we use a small line width, learning a feature field for signals just on very sparse pixels could be problematic and we may extract disconnected feature lines (sketches) here. The setting of the linewidth for training could be tricky here.\n3. It is difficult to get some real-world images to train these CNNs, which could harm the generalization ability of the proposed method. If we have painted some textures on these CAD models, then the proposed method could fail because it is not trained on models with textures.\n4. The proposed method seems to be too concentrated on a specific problem in computer graphics. Predicting extrusion cylinders is not a common task in daily life and the proposed method does not contain technically new ideas on neural networks. The paper seems to be more suitable for some graphics journals or conferences rather than NeurIPS.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "- **Surface label association.**\nYes, you are right that the predicted segmentation maps of the multi-view images are instance segmentation labels. Hence we use Hungarian matching (see Ln 209-215 main paper) to align the labels between the segmentation maps of the training images with the segmentation labels of the attribute field at training time.\n- **Line width sensitivity.**\nWe use a line width of 5 pixels in our experiments, which we found works quite well in our setting. We further experiment on using different line widths, specifically 2.5 pixels (50% thinner) and 7.5 pixels (50% thicker). Results are shown in Table 4, where we see that our approach is not that sensitive to line width.\n- **Textured CAD models.**\nYes, our current method does not directly handle textured CAD models. We plan to explore handling textured objects by utilizing generalizable large 2D models such as Segment-Anything (SAM) to extract the object mask removing the texture and similar to our current real data experiments we can remove the background, which together can potentially bridge the domain gap. We leave this exploration as future work. \n- **Specific problem with computer graphics.**\nIn this work, our main idea is to demonstrate how 2D priors can aid in reconstructing 3D structures when combined with neural rendering and to explore the best way to integrate different 2D priors (such as surfaces and curves) in 3D space. Although we focus on reconstructing extrusion cylinders in this study, we believe that the ideas presented have the potential to influence future research in many areas of machine learning. We will revise the introduction of our submission to clarify this point.\n- **Limitations: binary operation prediction.**\nOne simple approach to predicting binary operations without CNNs is to do an exhaustive search against all possible primitives-operations combinations (2^K possibilities for a model with K primitives) and take the best combination, which is closest to the observed (input) images. To obtain the best combination, we measure the average per-pixel L2 distance between the rendered images of each combination’s resulting model against the observed multiview images and select the lowest one. We implement this approach and obtain the binary operations, examples are shown in Figure 3. We leave further explorations for better approaches, such as being cheaper and faster, as future work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Nu2KF47RHr", "reviewer": "Reviewer_dRdb", "summary": "The authors introduce a novel method named MV2Cyl, which reconstructs technical 3D  objects withing the sketch-extrude paradigm by utilizing a 2D prior model and a learnable radiance field derived from multi-view images. To accomplish this, the 2D prior model is trained on a labeled dataset to predict semantic labels, $K$ extrusion cylinders, and extrusion curves, which are crucial for further processing of the 3D shape. Once the model is trained, it predicts pseudo-ground-truth for the subsequent stage. In this stage, a radiance field model is trained to replicate the pseudo-ground-truth labels. As the radiance field model converges, it facilitates the extraction of surfaces and curves in 3D space. The authors then detail how to utilize this output to reconstruct the final shape. The experimental results demonstrate that each component significantly contributes to achieving 3D reconstructions that surpass the selected baselines. Given that the radiance field is trained solely from the multi-view images and the prior model, it offers improved real-world applicability compared to previous works.", "questions": "Questions:\n- I will refer to:\n> We first extract the surface point cloud and curve point cloud from the corresponding 3D fields by thresholding the corresponding existence fields. Each point in the point cloud would have attributes queried from the corresponding attribute fields.\n\nHow is the point cloud sampled specifically? Is it accurate the underlying radiance field uses density instead of SDF field like in NeuS?\n\nSuggestions:\n- The explanation in lines 44-47 assumes prior knowledge of the sketch-extrude principle. Without this knowledge, it's challenging to understand the source of the issue. An inset visualization could greatly aid comprehension.\n- I propose renaming the \"existence field\" and \"attribute fields\". The term \"existence field\" implies a binary output indicating whether an object exists at a given point. \"Density field\" would be more appropriate, clarifying how prior values translate to opacity and aligning with the field's intended purpose.\nRegarding the \"attribute field\", the term could be misconstrued as referring to \"attributes\" such as colors. \"Semantic field\" would be more accurate.\n- It can be deducted from the context that $L_\\text{existence}$ applies to both curve and attribute fields. However, stating that clearly in the text would remove potential ambiguities for other readers.", "limitations": "The potential negative societal impact is clearly stated and supported by the real-demo presented in the Appendix.\n\nRegarding the limitations, it is unclear what authors mean by:\n> our framework is limited in predicting binary operations across primitives. We plan to further explore predicting binary operations using multi-view image inputs.\n\nin lines 328-331. Some examples or an visualization in the Appendix would be appreciated here. It would be worth adding a comment on the shape complexity the model cannot handle.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "- The paper is overall well-written and easy to follow.\n- Each component is effectively justified, and the ablation study outcomes distinctly demonstrate their significance.\n- Given the extensive datasets like ABC [1], the prior training represents a significant contribution to this research. It is conceivable that the prior model may serve as a foundational model for CAD extraction in subsequent studies.\n- Additionally, the prior and inference models are independent, and both can be adapted adequately if new, better models appear in the literature. This should also facilitate scaling with the datasets.\n- I appreciate the demo displayed in the Appendix. It illustrates the practical application of the model and reinforces the societal impact discussed in the final section of the paper.\n- One can argue about the pros and cons between different paradigms (such as sketch-extrude, CSG, etc.). The choice of sketch-extrude in this instance is substantiated by the results and outperforms the established baselines.\n- Publishing the code and the extended dataset upon acceptance is highly appreciated.\n \n[1] Koch, S, Matveev, A, Jiang, Z, Williams, F, Artemov, A, Burnaev, E, Alexa, M, Zorin, D, Panozzo, DABC: A Big CAD Model Dataset For Geometric Deep Learning. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2019 .", "weaknesses": "- Point2Cyl, ExtrudeNet, and SECAD-Net encode inputs as latent vectors, which are then decoded into the target representation. An encoder in this context could be any model that processes input in a specific modality. Adapting these methods to handle multi-view images appears feasible and would support the paper's claims. Notably, an encoder understanding 3D geometry based solely on 2D images could be advantageous (for instance, utilizing part of the network from [2] as an encoder).\n- While the paper is generally understandable, Section 3.4 is hard to follow. Introducing a simple visualization to illustrate the process would aid comprehension.\n- In Section 3.4, points 1 and 2 prompt inquiries about the 3D model employed in this study. The authors refer to TensorRF, but it is unclear whether a standard NeRF or NeuS was used. If NeuS was utilized, the predicted normals could be enough to:\n    - Determine a fitting plane by voting for the plane that most likely aligns with the normals.\n    - Derive the output shape using a standard algorithm like ball-pivoting or Poisson surface reconstruction, both of which are widely accessible.\n- Furthermore, a NeuS backbone would arguably be more appropriate for the application described in the paper, as it more precisely reconstructs surface fields, potentially enhancing the accuracy of the curve field.\n\n[2] Watson, D, Chan, W, Martin-Brualla, R, Ho, J, Tagliasacchi, A, Norouzi, M. \"Novel view synthesis with diffusion models\". arXiv preprint arXiv:2210.04628 2022.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "- **Adapting methods to multi-view images (Image encoder + CAD decoder).**\nThank you for the suggestion. First, we want to note that for Point2Cyl and SECAD-Net, it is not feasible to directly replace the point cloud encoder with an image encoder since the encoder does not simply output a latent code but assigns information to each point, such as the segment label or surface normal. ExtrudeNet is the only case that produces a latent code from the input shape. While we couldn't try replacing the point cloud encoder of ExtrudeNet with an image encoder during the tight rebuttal period (as we needed to address questions from five reviewers), we promise to include the results of this experiment in the revision. We strongly believe that our method—segmenting 2D images, unprojecting them to 3D space, and fitting extruded cylinders rather than directly predicting them—will demonstrate superior performance.\n- **Section 3.4. Simple visualization.**\nThank you for your suggestion. We add a simple visualization to clarify the pipeline of the reverse engineering of CAD parameters as shown in Figure 1. \n- **Clarification on 3D model (TensorRF) employed.**\nOur approach directly builds on top of TensorRF which is based on standard NeRF that models a density field directly instead of an SDF as in NeuS. We will clarify this in the final version. We find that this choice of using TensorRF works well in our setting and is also fast to optimize. We thank the reviewer for their suggestion on using a NeuS backbone, which is interesting and also makes sense. We leave this exploration for future work.\n- **How are point clouds sampled from the learned fields?**\nWe refer to Appendix A5.3 (Ln 638-642). We query the learned existence field by sampling points across a 400x400x400 grid. The existence values  $\\mathcal{F} (x)$ are converted into opacity values (Eq. 3), and we keep the sampled points whose opacities are greater than 0.99, obtaining our point cloud.\n- **Adding visual aids for lines 44-47.**\nThank you for the suggestion. We will include an additional visualization to aid comprehension in the final version.\n- **Renaming “Existence field” and “Attribute field” to “Density field” and “Semantic field”.**\nThank you for your suggestion. That makes sense! We will modify this in the final version.\n- **Clearer statement of L_existence.**\nThank you for the suggestion. We will state this explicitly and more clearly in the text of the final version.\n- **Clarification on limitation (line 328-331)**\nSimilar to Point2Cyl, our approach did not explicitly predict the binary operations of the primitives, i.e., determining whether each primitive is 'added' or 'subtracted' from the model. We will make this statement more clear in the final version. Note that in this rebuttal, we further provided a simple approach to the binary operation recovery: one naive and simple approach is to do an exhaustive search against all possible primitives-operations combinations (2^K possibilities for a model with K primitives), and take the best combination, which is closest to the observed (input) images. To obtain the best combination, we measure the average L2 distance between the rendered images of each combination’s resulting model against the observed multiview images and select the lowest one. We implement this approach and obtain the binary operations, examples are shown in Figure 3.\n- **Failure cases.**\nSince MV2Cyl relies on 2D image inputs, it’s susceptible to occlusion. Specifically, if one side of an extrusion cylinder is totally hidden by the other ones, our 2D segmentation model cannot catch the hidden side. In this case, the extrusion cylinder cannot be reconstructed. In Figure. 5, the left shape is the target CAD model and the right one is the reconstructed CAD model by MV2Cyl. While the target shape has an inset hexagonal cylinder whose one end is hidden by the outer one, MV2Cyl failed to reconstruct the inset extrusion cylinder.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We appreciate the invaluable feedback from all the reviewers on MV2Cyl. The thorough and insightful comments have significantly contributed to the improvement of our work.

We have carefully considered each question and suggestion and have provided detailed responses to the comments individually. We have compiled the qualitative results discussed in our rebuttal in the attached PDF file.

Meta-review: After the rebuttal and extensive discussion, the reviewers' consensus was to accept the paper given its strengths i.e., the novelty of the proposed extrusion-based 3D reconstruction method using primitives and strong results. The AC recommends acceptance. The AC strongly encourages the authors to incorporate material from the rebuttal and discussion to improve the final version of the paper.
Decision: Accept (poster)