Title: MV2Cyl: Reconstructing 3D Extrusion Cylinders from Multi-View Images
Venue: NeurIPS 2024
Accepted: yes
Keywords: Extrusion

Multi-View Images

3D Reconstruction

Computer-Aided Design
Abstract: We present MV2Cyl, a novel method for reconstructing 3D from 2D multi-view images, not merely as a field or raw geometry but as a sketch-extrude CAD. Extracting extrusion cylinders from raw 3D geometry has been extensively researched in computer vision, while the processing of 3D data through neural networks has remained a bottleneck. Since 3D scans are generally accompanied by multi-view images, leveraging 2D convolutional neural networks allows these images to be exploited as a rich source for extracting extrusion cylinder information. However, we observe that extracting only the surface information of the extrudes and utilizing it results in suboptimal outcomes due to the challenges in the occlusion and surface segmentation. By synergizing with the extracted base curve information, we achieve the optimal reconstruction result with the best accuracy in 2D sketch and extrude parameter estimation. Our experiments, comparing our method with previous work that takes a raw 3D point cloud as input, demonstrate the effectiveness of our approach by taking advantage of multi-view images.

MV2Cyl: Reconstructing 3D Extrusion Cylinders from Multi-View Images

Abstract

We present MV2Cyl, a novel method for reconstructing 3D from 2D multi-view images, not merely as a field or raw geometry but as a sketch-extrude CAD model. Extracting extrusion cylinders from raw 3D geometry has been extensively researched in computer vision, while the processing of 3D data through neural networks has remained a bottleneck. Since 3D scans are generally accompanied by multi-view images, leveraging 2D convolutional neural networks allows these images to be exploited as a rich source for extracting extrusion cylinder information. However, we observe that extracting only the surface information of the extrudes and utilizing it results in suboptimal outcomes due to the challenges in the occlusion and surface segmentation. By synergizing with the extracted base curve information, we achieve the optimal reconstruction result with the best accuracy in 2D sketch and extrude parameter estimation. Our experiments, comparing our method with previous work that takes a raw 3D point cloud as input, demonstrate the effectiveness of our approach by taking advantage of multi-view images. Our project page can be found at https://mv2cyl.github.io .

1 Introduction

Most human-made objects in our daily lives are created using computer-aided design (CAD). Reconstructing the structural representation from raw geometry, such as 3D scans, is essential to enable

the fabrication of 3D shapes and manipulate them for diverse downstream applications. SketchExtrude CAD model is particularly notable not only as the most common but also as a versatile CAD representation, which enables the expression of diverse shapes with 2D sketches and extruding heights.

The reconstruction of a set of extrusion cylinders from raw 3D geometry has been extensively explored in previous studies [58, 65, 49, 30], yet the suboptimal quality of the results remains a challenge. Unsupervised methods [49, 30] have demonstrated notable capabilities in parsing raw geometry but often struggle to precisely segment the shape and part-wise fit the output extrusions to the given shape. Supervised methods using a language-based architecture, such as those described in [65], often face challenges in producing valid outputs. A notable exception is Point2Cyl [58], which is a supervised method proposing to first segment the given raw point cloud into each extrusion region and estimate the 2D sketch and the extrusion parameters using the predicted segments and surface normal information. While it has shown successful results, the bottleneck remains in the limited performance of the 3D backbone network for segmentation.

To address the limitation, we focus on the fact that 3D scans are commonly accompanied by multiview images , which are now even solely used to reconstruct a 3D shape without depth or other information thanks to the recent advancements in neural rendering techniques [40, 61, 13, 62, 5]. 2D multi-view images are a rich resource for extracting 3D extrusion information. Also, 2D convolutional neural networks have been utilized in various 3D tasks [34, 63], owing to their superior performance compared to 3D processing networks. In light of these observations, we introduce a novel framework called MV2Cyl, which reconstructs a set of extrusion cylinders from multi-view images of an object without relying on raw geometry as input.

A straightforward approach to exploiting multi-views in extrusion reconstruction is to first segment 2D regions of the extrusion in each image and reconstruct the 3D shape using the extrusion labels. However, achieving precise 3D surface reconstruction solely from the multi-view images poses challenges due to the ambiguity between the object's intrinsic color (albedo) and the effects of lighting (shading) or shadows within images (Fig. 5). In contrast, we observe that reconstructing only the 2D sketches of the two ends of the extrusion cylinders-the start and end planes of extrusion-provides much more accurate results while avoiding the ambiguity issue and the consequential error accumulation. However, this approach also results in failure cases when one of the bases is not properly detected in 2D image due to the sparse viewpoints. Therefore, we propose a framework that synergizes the reconstructions of labeled surfaces and labeled curves of the extrusion bases so that parameters such as extrusion center and height can be better calculated from the surface information, while the 2D sketches can be precisely recovered from the curve information. In our experiments, we demonstrate the superior performance of our method MV2Cyl on two sketch-extrude datasets: Fusion360 [64] and DeepCAD [65].

2 Related Work

3D Primitive Fitting . Primitive fitting has been extensively investigated in computer vision. One line of work involves decomposing into surface/volumetric primitives. Classical approaches include detecting simple primitives such as planes [6, 11, 42] through RANSAC [51, 31], region growing [45], or Hough voting [3, 8]. The emergence of neural networks gave rise to data-driven methods [29, 56, 28] learning frameworks that fit simple primitives such as planes, cylinders, cones, and spheres. Primitive fitting has also been used to approach the task of shape abstraction [75, 57] that fit cuboids given an input shape or image. However, these works all assume a fixed predefined set of primitives that may not be able to cover complex real-world objects. The focus of our work is to recover a more general primitive, i.e. , extrusion cylinders, that are defined by any arbitrary closed loop and hence can explain more complex objects.

Another line of work tackles fitting 3D parametric curves, where a common strategy is to first detect edges [70] and corners [35, 70, 39] from an input point cloud, and then fit or predict parametric curves from the initial proposals. A recent work, NEF [68], takes multi-view images as input and introduces a self-supervised approach to the parametric curve fitting problem using neural fields, thus allowing them to optimize without needing clean point cloud data. Despite having multi-view images as input, NEF only tackles recovering parametric curves and not extrusion cylinders, which is a basic building block for the sketch-extrude reverse engineering task and is the focus of our work.

CAD Reconstruction for Reverse Engineering . CAD reverse engineering has been a well-studied problem in computer graphics and CAD communities [4, 60, 2, 1, 38], enabling the recovery of a structured representation of shapes for applications such as editing and manufacturing. Earlier works focused on reconstructing CAD through inverse CSG modeling [9, 55, 22, 48, 69], which decomposes shapes into simple primitives combined with boolean operations. While achieving good reconstruction, CSG-like methods tend to combine a large number of shape primitives, making them less flexible or easily manipulated by users. Recent efforts in collating large-scale CAD datasets (ABC [25], Fusion 360 [64], SketchGraphs [54]) have pushed this boundary, enabling the learning of data-driven methods for various applications such as B-Rep classification [19] and segmentation [19, 26, 10], parametric surface inference [16, 36], and CAD reconstruction [23, 27] and generation [66]. For the reverse engineering task, a common CAD modeling paradigm uses sketch-extrude [64, 58] as the basic building block, where a sketch [67, 47] is defined as a sequence of parametric curves forming a closed loop. This results in primitives that are more flexible and complex, not a predefined finite set, which will be the focus of this work.

Previous research uses NLP-based language models conditioned on raw geometry to represent CAD models as token sequences [65, 14, 18, 72]. However these methods are not designed to understand and decompose geometry into individual primitives and may also result in invalid CAD sequences. A recent work [21] also models CAD with language but face similar issues despite allowing multi-view inputs. To address these issues, Point2Cyl [58] is the pioneering work that poses the CAD reverse engineering task as a geometry-aware decomposition problem. The idea was to first decompose an input point cloud into extrusion cylinders (sketch-extrude primitives) and introduce differentiable loss functions to recover the parameters. ExtrudeNet [49] and SECAD-Net [30] further built on this setting and extended it to the unsupervised and self-supervised settings, respectively. Our work tackles a similar setting that takes a geometry-aware approach to the reverse engineering task, except we take multi-view images as input, as opposed to existing works that only handle point cloud inputs.

3 MV2Cyl: Reconstructing 3D Extrusion Cylinders from Multi-View Images

Instance

segmentation

Start/end/barrel

segmentation

Instance

segmentation

Start/end

segmentation

2D Surface segmentation

2D Curve segmentation

3D Surface reconstruction

3D Curve reconstruction

Extrusion cylinders

reconstruction

Fsurface

psurface

K

primitiyes

extraction

multi-viewimages

Fcurve

pcurve

CAD parameters

reconstructed

We propose a method that reconstructs 3D extrusion cylinders from multi-view images without relying on raw 3D geometry as input. The idea is to leverage 2D neural networks to learn 2D priors that provide 3D extrusion information (Sec. 3.2), i.e. extrusion curves and surfaces. The integration of the information into 3D is achieved through optimizing neural fields (Sec. 3.3). MV2Cyl is a combination of a curve field and a surface field that is used to recover the parameters and reconstruct the extrusion cylinders only given 2D input images (Sec. 3.4).

3.1 Problem Statement and Background

Given a set of 2D RGB images { I i } N i =1 where I ∈ R H × W × 3 , the goal is to recover a set of extrusion cylinders { E j } K j =1 that represent the underlying 3D shape. We provide an overview here but refer the readers to Uy et al. [58] for more details.

Sketch-Extrude CAD is a designing paradigm or the designed model itself in the area of computeraided design (CAD) consists of extrusion cylinders as its building blocks. An extrusion cylinder E is 3D space defined as E = ( n , c , h , ˜ S , s ) with the extrusion axis n ∈ S 2 , the extrusion center c ∈ R 3 , the extrusion height h ∈ R , and the normalized sketch ˜ S scaled by s ∈ R . Asketch-extrude CAD model is a 3D shape that is reconstructed from a set of extrusion cylinders, { E 1 , E 2 , ..., E K } where K is the number of extrusion cylinders. A sketch ˜ S is a set of non-self-intersecting closed loops drawn in a normalized plane.

We further classify the surfaces of a sketch-extrude CAD model as base or barrel . A surface is a base if its surface normal is parallel to its extrusion axis n , and is a barrel if its surface normal is perpendicular to its extrusion axis. By this definition, base surfaces are parameterized by points on the planes and a normal ( n ). The base surfaces can further be distinguished into start plane and end plane where the start plane contains the point c -h 2 n , while the end plane contains the point c + h 2 n .

With these definitions, we will later show how our method, MV2Cyl, is able to recover the set of extrusion cylinders and their corresponding parameters given only 2D multi-view images.

3.2 Learning 2D Priors for 3D Extrusions

To reconstruct extrusion cylinders from multi-view images, we first learn 2D priors for 3D extrusions by exploiting 2D convolutional neural networks. We train two U-Net-based 2D segmentation frameworks: M curve that extracts curve information and M surface that extracts surface information. The 2D information extracted from the two frameworks is integrated into 3D using a neural field and then utilized for more robust reverse engineering. We detail each 2D segmentation frameworks below.

2D Surface Segmentation Framework M surface . The goal of the surface segmentation network is to extract the extrusion instance segmentation as well as start plane, end plane, and barrel (start-end-barrel) segmentation given an input image. That is M surface : R H × W × 3 → { P surface ∈ { 0 , ..., K } H × W , Q surface ∈ { 0 , 1 , 2 , 3 } H × W } , where K is the number of instances, P surface represents the extrusion instance segmentation label, and Q surface denotes the start-end-barrel segmentation label. The zero index is used for background annotation. M surface is implemented as two distinct U-Nets predicting each segmentation and is trained using a multi-class cross entropy loss with ground truth labels ˆ P surface and ˆ Q surface . An important property is that the problem does not admit a unique solution as i) the extrusion segment can be ordered arbitrarily and ii) the start and end planes can also be arbitrarily labeled. To handle this, we use Hungarian matching to find the best one-to-one matching with the ground truth labels and reorder it as ˜ P surface and ˜ Q surface . This gives us the loss function of the instance segmentation:

where P surface hwk is the model's predicted probability that pixel ( h, w ) belongs to the k -th instance in the b -th image in the batch, and ˜ P surface hw is the pseudo GT label for that pixel, with the number of images in a batch B and the number of possible instances including the background K . The loss function of the start-end-barrel segmentation appears the similar, replacing the P as Q and the range of classes {0, ..., K} to {0, 1, 2, 3}.

2D Curve Segmentation Framework M curve . We observe that detecting and extracting curves on the base planes of each extrusion cylinder segment provides additional information to recover the parameters and reconstruct the extrusion cylinders. Moreover, (feature) curves have been a longstanding and well-explored problem dating back to classical computer vision literature [46, 74, 43] thanks to their strong expressiveness. This leads us to learn a 2D curve prior that provides more discriminative and detailed outputs to make reverse engineering more robust.

The goal of the curve segmentation framework is to extract both extrusion instance segmentation and start-end segmentation, where the start-end plane segmentation distinguishes the curves of the two base planes of extrusion cylinders. Concretely, M curve : R H × W × 3 → { P curve ∈ { 0 , ..., K } H × W , Q curve ∈ { 0 , 1 , 2 } H × W } , where P curve is the extrusion instance segmentation label and Q curve is the start-end segmentation label.

Similar to M surface , the zero index is used for background annotation in both segmentation tasks and M curve is also implemented with two U-Nets and trained using a multi-class cross-entropy loss against the pseudo ground truth { ˜ P curve , ˜ Q curve } that is reordered with Hungarian matching to handle order invariance and ambiguities. An additional challenge for curves compared to surfaces is the labels are a lot sparser with the majority of the images being background pixels. Hence, the model can easily fail to predict meaningful labels due to this label imbalance. To alleviate this, we additionally employ a dice loss [41] to handle the strong foreground-background imbalance and a focal loss [33] to circumvent the class imbalance between the extrusion instances. Hence the loss function used to train the curve prior is given as:

Additional details about the segmentation framework are available in the appendix E.1.

3.3 Integrating 2D Segments into a 3D Field

p

curve

b

curve

K

K

curve

K

K

To establish correspondences and collate the information extracted from multi-view images, a natural approach to integrate 2D information into a consistent 3D representation is through learning a 3D field [40, 61, 68, 44]. For each of our 2D priors M surface and M curve , we learn a density field F and a semantic field A as detailed below, respectively.

Density Field F . We learn a density field for surfaces F surface and curves F curve . Given a query point in 3D space x ∈ R 3 , the density field network F : R 3 ↦→ R outputs a scalar value that indicates how likely the 3D point x is on a surface or a curve for F surface and curves F curve , respectively. To optimize the field differentiably with only 2D images, we use volume rendering [40, 5] and a learnable transformation [68] to convert the density field F ( x ) to an opacity field σ ( x ) . This transformation is in the form of a learnable sigmoid function given as:

where α is a learnable parameter that adjusts the density scale, and β and ζ are constant hyperparameters. We use the same transformation in Eq. 3 for the surface field F surface and the curve field F curve , resulting in corresponding opacity fields σ surface and σ curve . We then directly render the density by using the volume rendering equation to get the expected surface and curve density for a camera ray r , which is given by the integrals

where T surface ( t ) = ∫ t 0 σ surface ( r ( s )) ds and T curve ( t ) = ∫ t 0 σ curve ( r ( s )) ds are the corresponding transmittance. Inspired by NEF [68], we train our surface field F surface and curve field F curve using an adaptive L2 loss for the reconstruction of pixel density and a sparsity loss for learning of a sharp distribution of density along the ray. The objective function is given as follows:

where E ( r ) and ˆ E ( r ) are the 2D-observed and predicted field density, respectively, for both the surface and curve models, i indexes background pixels from input maps, j indexes the sample points along the rays, and s determines the scale of the regularizer. The 2D-observed density E ( r ) is derived from the output of the 2D prior being a foreground pixel, either start, end, or barrel pixel.

Following Ye et al. [68], W batch ( r ) is given as:

The weight W batch ( r ) addresses the imbalance between foreground and background areas in the density maps, preventing the network from converging to a local minimum that falsely classifies most pixels as background, which are more prevalent.

Semantic Field A . Now with the density field, we can distill the information from the 2D priors by learning semantic fields A for both surfaces and curves, which we denote as A surface and A curve . Following Zhi et al. [73], each semantic is modeled as an additional feature channel branching out from the density field F . For surfaces, the additional semantic field A surface = { p surface , q surface } is comprised of two networks p surface : R 3 ↦→ R ( K +1) that predicts the extrusion instance label, and q surface : R 3 ↦→ R 4 that predicts the start-end-barrel segmentation. Similarly for the curves, the semantic field is given as A curve = { p curve , q curve } , where p curve : R 3 ↦→ R ( K +1) predicts the extrusion instance label, and q curve : R 3 ↦→ R 3 predicts the start-end segmentation. The expected semantic for a ray r is then computed using the volume rendering equation which is given as:

for each a ∈ { p surface , q surface , p curve , q curve } that correspond to rendered predicted semantics ˆ A ∈ { ¯ P surface , ¯ Q surface , ¯ P curve , ¯ Q curve } from the 3D field.

We train our semantic field using the predictions from our learned 2D priors F surface and F curve . For each of the 2D multi-view images, we can extract the 2D observed labels A ∈ { P surface , Q surface , P curve , Q curve } from the learned priors, which we use as supervision. As mentioned in Sec. 3.2, since the labels may not individually be consistent across the multiple views, we use Hungarian matching during training to align the labels across multiple views based on the best one-to-one matching. For each semantic, we use a multi-class cross-entropy loss to train and optimize the semantic field with the given supervision:

where A l ( r ) and ˆ A l ( r ) represent the class l semantic probabilities from the learned 2D prior and the predicted field, respectively, aligned with Hungarian matching.

Training and Implementation Details . The integration into a coherent 3D field is optimized independently for each 3D shape. We use TensoRF [5] as the backbone representation for the 3D fields, where we have one model for the surface field and the other for the curve field. In both models, we train the density fields F and the semantic fields A together for 1500 iterations. Each semantic field is parameterized by a 2-layer MLP that outputs logit values. Additional implementation details are provided in the Appendix E.2.

3.4 Reverse Engineering from 3D Fields

We now elaborate on how we leverage our optimized surface field F surface and curve field F curve for the reverse engineering task, specifically to recover extrusion cylinders and their defining parameters, ( n , c , h , ˜ S , s ) . Each point within the extracted point clouds is assigned semantic information queried from the associated semantic fields, providing context for each feature in 3D space. Using these enriched point clouds, we employ a multi-step process to reconstruct each extrusion cylinder, iteratively refining the parameters to accurately fit the geometry and semantics. Further details can be found in Appendix E.3.

Extrusion Axis n Estimation : For each extrusion cylinder, the extrusion axis n is computed through plane fitting via RANSAC [12] on the surface base points with corresponding extrusion cylinder instance label. The axis n is then set as the normal of the fitted plane. This process relies on surface information because plane fitting is more reliable using the surface points. If the base face of the cylinder fails to be reconstructed due to occlusion, we use curve points as a substitute.

2D Sketch ˜ S Estimation : To obtain the sketch ˜ S for each extrusion cylinder, we first project the curve point cloud for each extrusion instance onto the plane with normal n . From the projected point cloud, we then compute for the sketch scale s . We then optimize an implicit signed distance function for each closed loop using the projected point cloud with IGR [15]. Finally, we obtain the closed loop by extracting the zero-level set of the implicit function. Implicit sketches can be converted to a parametric representation using an off-the-shelf spline fitting module. We provide example visualizations in Appendix E.7.

Extrusion height h Estimation : The height h of each extrusion cylinder is computed as the distance between the two sketch centers - from the start plane and the end plane. The sketch centers are computed from the projected curve point cloud for each paired labels (extrusion instance segmentation, start-end segmentation) onto the plane with normal n . If any of the startend planes are occluded, the height is computed using the barrel points from the surface point cloud with the corresponding extrusion cylinder instance label.

Extrusion centroid c Estimation : The centroid c of each extrusion cylinder can be obtained by taking the average position of the 3D curve point cloud with the corresponding extrusion cylinder instance label. Similar to the above, if any of the start-end planes are occluded or missing, the centroid is computed as the average of the barrel points from the surface point cloud with the corresponding extrusion cylinder instance label.

4 Experiments

In this section, we present our experimental evaluation to demonstrate the feasibility of our MV2Cyl in reconstructing extrusion cylinders given only multi-view images. To the best of our knowledge, our proposed pipeline is the first to tackle extrusion cylinder recovery directly from 2D input images, and we show that our method is outperforms existing methods that take clean geometry as input and their modifications to handle raw input images.

n

2. Normalize

and fit curve

1.  Fit a plane using RANSAC and

project the curve points onto the plane

s

S

~

h

3. Find start and end centers

and compute height

c

4. Compute instance center

instance label

start label

end label

Datasets . We evaluate our approach on two sketch-extrude CAD datasets Fusion360 [64] and DeepCAD [65]. We use the train and test splits as released in [58]. We enrich these datasets with multi-view image renderings and 2D curve and surface segmentation maps for the training and evaluation of our method and the baselines. More details about dataset generation are in Appendix E.4.

Evaluation Metrics . Following [58], we report the average extrusion-axis error (E.A.), extrusioncenter error (E.C.), per-extrusion cylinder fitting loss (Fit Cyl.), and global fitting loss (Fit Glob.). Additionally, we also report the extrusion-height error (E.H.) defined as the L1 distance between the GT and predicted height value: 1 K ∑ K k =1 | ˆ h k -h k | . These metrics evaluate how well the methods reconstruct and recover the parameters of the extrusion cylinders. The final scores are the average of the metrics calculated over all test shapes.

Baselines . In the absence of established methods for transforming multi-view images into sketchextrude CAD models within shape space, we benchmark our model against a point cloud to sketchextrude CAD reconstruction technique [58] and its variant to validate the effectiveness of our proposed approach.

We compare with Point2Cyl [58] , a technique for reconstructing sketch-extrude CAD given input 3D point clouds. The point clouds are sampled from clean ground truth CAD meshes and are utilized as the input for this baseline. Point2Cyl [58] initially segments the input point cloud into distinct instances and base/barrel segments, then identifies the extrusion axis as the direction that aligns with the normals of the points in the base segments and is orthogonal to the normals of the points in the barrel segments. The rest of the parameters are inferred based on this predicted axis and the segmentation labels. To evaluate the network, we trained it using the official code released by the authors.

NeuS2 [62]+Point2Cyl [58] combines NeuS2 [62] with Point2Cyl [58]. Since no prior work has reconstructed extrusion cylinders directly from multi-view images, we combine methods for i) shape reconstruction from multi-view images, with techniques for ii) extrusion cylinder reconstruction from raw geometry. This approach utilizes NeuS2 [62], an off-the-shelf image-based surface reconstruction technique, that optimizes the photometric loss to learn underlying implicit 3D representations. For this baseline, we first reconstruct the object surface from multi-view images using NeuS2 [62], and then sample a point cloud from the reconstructed mesh. The sampled point cloud is then directly fed into Point2Cyl [58] to predict sketches and extrusion parameters of CAD models.

4.1 Results

Tab. 1 shows the quantitative results of CAD reconstruction on the Fusion360 [64] and DeepCAD [65] datasets. We see that MV2Cyl outperforms the baselines by considerable margins across both datasets in all evaluated metrics. Qualitative comparisons are shown in Fig. 5.

GT

Ours

(CAD)

NeuS2-

Point2Cyl

(Mesh)

NeuS2-

Point2Cyl

(CAD)

Point2Cyl

(PC)

Point2Cyl

(CAD)

(a)

(b)

(c)

(d)

(e)

(f)

(g)

O

C

The results of NeuS2 [62]+Point2Cyl [58] illustrate the challenges in naively using multi-view images for CAD reconstruction, where inaccurate 3D geometry reconstructed from images serve as a critical bottleneck in recovering CAD parameters. We observe that it struggles to distinguish the object's intrinsic color (albedo) from the effects of lighting or shadows (shading). This can result in discrepancies, such as omitted details or subtle smoothing at the intersection between different extrusion segments, as illustrated in (Fig. 5). As seen in Fig. 5 (c), a common failure arises when NeuS2 [62] fails to reconstruct fine geometric details, in this case, the boundaries between the extrusion cylinders. Such artifacts in the reconstructed mesh make the conversion from the point cloud to the CAD model unreliable.

MV2Cyl also outperforms Point2Cyl [58], which takes precise 3D geometric information as input. An advantage of taking multi-view images instead of point clouds is the images contain richer information such as edges and curves. Moreover, 2D neural networks have also shown superior performance compared to 3D processing networks leading to superior performance of MV2Cyl compared to Point2Cyl [58]. The comparison of the 2D segmentation network in MV2Cyl with the 3D segmentation network in Point2Cyl are found in Appendix F.1.

Unlike the baselines that require accurate 3D surface information, MV2Cyl is able to directly handle 2D multi-view images as input by leveraging on learned 2D priors to extract 3D extrusion information. We show that extracting extrusion surfaces and curves from 2D images allows us to leverage both the curve and surface representation for better extrusion cylinder reconstruction compared to baselines that even require clean 3D geometry as input.

4.2 Real Object Reconstruction

We validate MV2Cyl on a real-world demo by 3D-printing various sketch-extrude objects from the Fusion360 test set, by capturing multi-view images with an iPhone 12 Mini and then obtaining camera poses via COLMAP. These captured images were processed through our pipeline to extract 2D information, converted to 3D using our density and semantic fields to recover extrusion cylinders and parameters. To address the domain gap between synthetic training images and real-world data, we applied preprocessing techniques, including grayscale conversion, background removal, and then fine-tuning a large vision model. Additional details are provided in Appendix B.

5 Conclusion

We introduce MV2Cyl, a novel method for reconstructing a set of extrusion cylinders in 3D from multi-view images. Unlike previous approaches that rely only on raw 3D geometry as input, our framework utilized 2D multi-view images and demonstrates superior performance by leveraging the capabilities of 2D convolutional neural networks in image segmentation. Through the analysis of curve and surface representations, each having its own advantages and shortcomings, we propose an integration of both that takes the best of both worlds, achieving optimal performance.

Limitations and Future Work . Our proposed method is not without its limitations. While MV2Cyl demonstrates state-of-the-art performance in reconstructing sketch-extrude CAD models, it shares a limitation with previous approaches, such as Point2Cyl [58], regarding the prediction of binary operations among primitives. MV2Cyl does not explicitly predict these binary operations, which we leave as future work. Nonetheless, we propose an initial straightforward method for recovering binary operations through exhaustive search. Details are provided in the appendix E.6. Additionally, MV2Cyl does not directly accommodate textured CAD models, as the 2D segmentation models are trained on a synthetic dataset derived from rendering untextured CAD models. By utilizing generalizable large 2D models, such as Segment Anything [24], we can preprocess images of textured CAD models for segmentation by extracting object masks and removing textures. We also leave this for future investigation.

MV2Cyl is also susceptible to occlusion since it relies on 2D image inputs. Specifically, if one side of an extrusion cylinder is completely hidden by others, our 2D segmentation model cannot detect the hidden side. In such cases, the extrusion cylinder cannot be reconstructed. In Fig. 6, the left shape represents the target CAD model, while the right one shows the reconstructed model by MV2Cyl. The target shape features an inset hexagonal cylinder with one end hidden by an outer cylinder, but MV2Cyl was unable to reconstruct the inset extrusion cylinder.

Societal Impacts . This technology could make CAD more accessible to non-professionals or educational sectors, allowing more people to engage in design and engineering tasks without needing extensive training in traditional CAD software. However, such a 3D reconstruction method can also raise privacy concerns. By converting a physical object into a digital format, the shape could be reused for design purposes without the original creator's permission.

References

Pál Benkö, Géza Kós, Tamás Várady, László Andor, and Ralph Robert Martin. Constrained fitting in reverse engineering. Comput. Aided Geom. Des. , 2002.

Pal Benko and Tamas Varady. Segmentation methods for smooth point regions of conventional engineering objects. Computer-Aided Design , 2004.

Dorit Borrmann, Jan Elseberg, Kai Lingemann, and Andreas Nüchter. The 3d hough transform for plane detection in point clouds: A review and a new accumulator design. 3D Research , 2, 2011.

Roseline Bénière, Gérard Subsol, Gilles Gesquière, François Le Breton, and William Puech. A comprehensive process of reverse engineering from 3d meshes to cad models. Computer-Aided Design , 2013.

Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su. Tensorf: Tensorial radiance fields. In ECCV , 2022.

T Czerniawski, B Sankaran, M Nahangi, C Haas, and F Leite. 6d dbscan-based segmentation of building point clouds for planar object classification. Automation in Construction , 88:44-58, 2018.

Maximilian Denninger, Martin Sundermeyer, Dominik Winkelbauer, Youssef Zidan, Dmitry Olefir, Mohamad Elbadrawy, Ahsan Lodhi, and Harinandan Katam. Blenderproc. arXiv preprint arXiv:1911.01911 , 2019.

Bertram Drost and Slobodan Ilic. Local hough transform for 3d primitive detection. In 3DV , 2015.

Tao Du, Jeevana Priya Inala, Yewen Pu, Andrew Spielberg, Adriana Schulz, Daniela Rus, Armando Solar-Lezama, and Wojciech Matusik. Inversecsg: Automatic conversion of 3d models to csg trees. ACM Transactions on Graphics (TOG) , 2018.

Elona Dupont, Kseniya Cherenkova, Anis Kacem, Sk Aziz Ali, Ilya Arzhannikov, Gleb Gusev, and Djamila Aouada. Cadops-net: Jointly learning cad operation types and steps from boundaryrepresentations. In 3DV , 2022.

Hao Fang, Florent Lafarge, and Mathieu Desbrun. Planar shape detection at structural scales. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2018.

M. Fischler and R. Bolles. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM , 1981.

Qiancheng Fu, Qingshan Xu, Yew Soon Ong, and Wenbing Tao. Geo-neus: Geometry-consistent neural implicit surfaces learning for multi-view reconstruction. In NeurIPS , 2022.

Yaroslav Ganin, Sergey Bartunov, Yujia Li, Ethan Keller, and Stefano Saliceti. Computer-aided design as language. In NeurIPS , 2021.

Amos Gropp, Lior Yariv, Niv Haim, Matan Atzmon, and Yaron Lipman. Implicit geometric regularization for learning shapes. In ICML , 2020.

Haoxiang Guo, Shilin Liu, Hao Pan, Yang Liu, Xin Tong, and Baining Guo. Complexgen: Cad reconstruction by b-rep chain complex generation. In SIGGRAPH , 2022.

J H Hannay and J F Nye. Fibonacci numerical integration on a sphere. Journal of Physics A: Mathematical and General , page 11591, 2004.

Pradeep Kumar Jayaraman, Joseph G. Lambourne, Nishkrit Desai, Karl D. D. Willis, Aditya Sanghi, and Nigel J. W. Morris. Solidgen: An autoregressive model for direct b-rep synthesis, 2023.

Pradeep Kumar Jayaraman, Aditya Sanghi, Joseph G. Lambourne, Karl D.D. Willis, Thomas Davies, Hooman Shayani, and Nigel Morris. Uv-net: Learning from boundary representations. In CVPR , 2021.

Rasmus Jensen, Anders Dahl, George Vogiatzis, Engil Tola, and Henrik Aanæs. Large scale multi-view stereopsis evaluation. In 2014 IEEE Conference on Computer Vision and Pattern Recognition , pages 406-413. IEEE, 2014.

Henrik Jobczyk and Hanno Homann. Automatic reverse engineering: Creating computer-aided design (cad) models from multi-view images. In GCPR , 2023.

Kacper Kania, Maciej Zi˛ eba, and Tomasz Kajdanowicz. Ucsg-net - unsupervised discovering of constructive solid geometry tree. In NeurIPS , 2020.

Mohammad Sadil Khan, Elona Dupont, Sk Aziz Ali, Kseniya Cherenkova, Anis Kacem, and Djamila Aouada. Cad-signet: Cad language inference from point clouds using layer-wise sketch instance guided attention, 2024.

Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick. Segment anything. arXiv:2304.02643 , 2023.

Sebastian Koch, Albert Matveev, Zhongshi Jiang, Francis Williams, Alexey Artemov, Evgeny Burnaev, Marc Alexa, Denis Zorin, and Daniele Panozzo. Abc: A big cad model dataset for geometric deep learning. In CVPR , 2019.

Joseph G. Lambourne, Karl D. D. Willis, Pradeep Kumar Jayaraman, Aditya Sanghi, Peter Meltzer, and Hooman Shayani. Brepnet: A topological message passing system for solid models. In CVPR , 2021.

Joseph George Lambourne, Karl Willis, Pradeep Kumar Jayaraman, Longfei Zhang, Aditya Sanghi, and Kamal Rahimi Malekshan. Reconstructing editable prismatic cad from rounded voxel models. In SIGGRAPH Asia 2022 Conference Papers , SA '22, 2022.

Eric-Tuan Lê, Minhyuk Sung, Duygu Ceylan, Radomir Mech, Tamy Boubekeur, and Niloy J. Mitra. Cpfn: Cascaded primitive fitting networks for high-resolution point clouds. In ICCV , 2021.

Lingxiao Li, Minhyuk Sung, Anastasia Dubrovina, Li Yi, and Leonidas Guibas. Supervised fitting of geometric primitives to 3d point clouds. In CVPR , 2019.

Pu Li, Jianwei Guo, Xiaopeng Zhang, and Dong-Ming Yan. Secad-net: Self-supervised cad reconstruction by learning sketch-extrude operations. In CVPR , 2023.

Yangyan Li, Xiaokun Wu, Yiorgos Chrysathou, Andrei Sharf, Daniel Cohen-Or, and Niloy J Mitra. Globfit: Consistently fitting primitives by discovering global relations. In ACMTransactions on Graphics (TOG) , 2011.

Siyou Lin, Dong Xiao, Zuoqiang Shi, and Bin Wang. Surface reconstruction from point clouds without normals by parametrizing the gauss formula. ACM Transactions on Graphics , 42(2):1-19, 2022.

Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In ICCV , 2017.

Fangfu Liu, Chubin Zhang, Yu Zheng, and Yueqi Duan. Semantic ray: Learning a generalizable semantic field with cross-reprojection attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 17386-17396, 2023.

Yujia Liu, Stefano D'Aronco, Konrad Schindler, and Jan Dirk Wegner. Pc2wf: 3d wireframe reconstruction from raw point clouds, 2021.

Yujia Liu, Anton Obukhov, Jan Dirk Wegner, and Konrad Schindler. Point2cad: Reverse engineering cad models from 3d point clouds, 2023.

William E. Lorensen and Harvey E. Cline. Marching cubes: A high resolution 3d surface construction algorithm. In SIGGRAPH , 1987.

Weijian Ma, Minyang Xu, Xueyang Li, and Xiangdong Zhou. Multicad: Contrastive representation learning for multi-modal 3d computer-aided design models. Proceedings of the 32nd ACM International Conference on Information and Knowledge Management , 2023.

Albert Matveev, Ruslan Rakhimov, Alexey Artemov, Gleb Bobrovskikh, Vage Egiazarian, Emil Bogomolov, Daniele Panozzo, Denis Zorin, and Evgeny Burnaev. Def: Deep estimation of sharp geometric features in 3d shapes, 2022.

Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM , 2021.

Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmadi. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In 3DV , 2016.

Aron Monszpart, Nicolas Mellado, Gabriel J Brostow, and Niloy J Mitra. Rapter: rebuilding man-made scenes with regular arrangements of planes. ACM Transactions on Graphics (TOG) , 2015.

Elia Moscoso Thompson, Gerasimos Arvanitis, Konstantinos Moustakas, N Hoang-Xuan, ER Nguyen, M Tran, Thibault Lejemble, Loïc Barthe, Nicolas Mellado, Chiara Romanengo, et al. Feature curve extraction on triangle meshes. In 12th Eurographics Workshop on 3D Object Retrieval . The Eurographics Association, 2019.

Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolution hash encoding. ACM Trans. Graph. , 41(4):102:1-102:15, July 2022.

Sven Oesau, Florent Lafarge, and Pierre Alliez. Planar shape detection and regularization in tandem. In Computer Graphics Forum , 2016.

Yutaka Ohtake, Alexander G. Belyaev, and Hans-Peter Seidel. Ridge-valley lines on meshes via implicit surface fitting. In SIGGRAPH , 2004.

Wamiq Reyaz Para, Shariq Farooq Bhat, Paul Guerrero, Tom Kelly, Niloy Mitra, Leonidas Guibas, and Peter Wonka. Sketchgen: Generating constrained cad sketches. In NeurIPS , 2021.

Daxuan Ren, Jianmin Zheng, Jianfei Cai, Jiatong Li, Haiyong Jiang, Zhongang Cai, Junzhe Zhang, Liang Pan, Mingyuan Zhang, Haiyu Zhao, and Shuai Yi. Csg-stump: A learning friendly csg-like representation for interpretable shape parsing. In ICCV , 2021.

Daxuan Ren, Jianmin Zheng, Jianfei Cai, Jiatong Li, and Junzhe Zhang. Extrudenet: Unsupervised inverse sketch-and-extrude for shape parsing. In ECCV , 2022.

Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In MICCAI , 2015.

Ruwen Schnabel, Roland Wahl, and Reinhard Klein. Efficient ransac for point-cloud shape detection. In Computer graphics forum , 2007.

Johannes Lutz Schönberger and Jan-Michael Frahm. Structure-from-motion revisited. In Conference on Computer Vision and Pattern Recognition (CVPR) , 2016.

Johannes Lutz Schönberger, Enliang Zheng, Marc Pollefeys, and Jan-Michael Frahm. Pixelwise view selection for unstructured multi-view stereo. In European Conference on Computer Vision (ECCV) , 2016.

Ari Seff, Yaniv Ovadia, Wenda Zhou, and Ryan P. Adams. SketchGraphs: A large-scale dataset for modeling relational geometry in computer-aided design. In ICML , 2020.

Gopal Sharma, Rishabh Goyal, Difan Liu, Evangelos Kalogerakis, and Subhransu Maji. Csgnet: Neural shape parser for constructive solid geometry. In CVPR , 2018.

Gopal Sharma, Difan Liu, Evangelos Kalogerakis, Subhransu Maji, Siddhartha Chaudhuri, and Radomír Mˇ ech. Parsenet: A parametric surface fitting network for 3d point clouds. In ECCV , 2020.

Shubham Tulsiani, Hao Su, Leonidas J. Guibas, Alexei A. Efros, and Jitendra Malik. Learning shape abstractions by assembling volumetric primitives. In CVPR , 2017.

Mikaela Angelina Uy, Yen-Yu Chang, Minhyuk Sung, Purvi Goel, Joseph G Lambourne, Tolga Birdal, and Leonidas J Guibas. Point2cyl: Reverse engineering 3d objects from point clouds to extrusion cylinders. In CVPR , pages 11850-11860, 2022.

Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern, Eric Larson, C J Carey, ˙ Ilhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, Ian Henriksen, E. A. Quintero, Charles R. Harris, Anne M. Archibald, Antônio H. Ribeiro, Fabian Pedregosa, Paul van Mulbregt, and SciPy 1.0 Contributors. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods , 17:261-272, 2020.

Tamás Várady, Ralph R Martin, and Jordan Cox. Reverse engineering of geometric models-an introduction. Computer-Aided Design , 1997.

Peng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. In NeurIPS , 2021.

Yiming Wang, Qin Han, Marc Habermann, Kostas Daniilidis, Christian Theobalt, and Lingjie Liu. Neus2: Fast learning of neural implicit surfaces for multi-view reconstruction. In ICCV , 2023.

Yue Wang, Vitor Campagnolo Guizilini, Tianyuan Zhang, Yilun Wang, Hang Zhao, and Justin Solomon. Detr3d: 3d object detection from multi-view images via 3d-to-2d queries. In Conference on Robot Learning , pages 180-191. PMLR, 2022.

Karl DD Willis, Yewen Pu, Jieliang Luo, Hang Chu, Tao Du, Joseph G Lambourne, Armando Solar-Lezama, and Wojciech Matusik. Fusion 360 gallery: A dataset and environment for programmatic cad construction from human design sequences. ACM Transactions on Graphics (TOG) , 2021.

Rundi Wu, Chang Xiao, and Changxi Zheng. Deepcad: A deep generative network for computeraided design models. In ICCV , 2021.

Xiang Xu, Pradeep Kumar Jayaraman, Joseph G Lambourne, Karl DD Willis, and Yasutaka Furukawa. Hierarchical neural coding for controllable cad model generation. arXiv preprint arXiv:2307.00149 , 2023.

Xiang Xu, Karl DD Willis, Joseph G Lambourne, Chin-Yi Cheng, Pradeep Kumar Jayaraman, and Yasutaka Furukawa. Skexgen: Autoregressive generation of cad construction sequences with disentangled codebooks. In ICML , 2022.

Yunfan Ye, Renjiao Yi, Zhirui Gao, Chenyang Zhu, Zhiping Cai, and Kai Xu. Nef: Neural edge fields for 3d parametric curve reconstruction from multi-view images. In CVPR , 2023.

Fenggen Yu, Qimin Chen, Maham Tanveer, Ali Mahdavi Amiri, and Hao Zhang. D 2 csg: Unsupervised learning of compact csg trees with dual complements and dropouts. In NeurIPS , 2023.

Lequan Yu, Xianzhi Li, Chi-Wing Fu, Daniel Cohen-Or, and Pheng-Ann Heng. Ec-net: an edge-aware point set consolidation network. In ECCV , 2018.

Kaidong Zhang and Dong Liu. Customized segment anything model for medical image segmentation. arXiv preprint arXiv:2304.13785 , 2023.

Shuming Zhang, Zhidong Guan, Hao Jiang, Tao Ning, Xiaodong Wang, and Pingan Tan. Brep2Seq: a dataset and hierarchical deep learning network for reconstruction and generation of computer-aided design models. Journal of Computational Design and Engineering , 2024.

Shuaifeng Zhi, Tristan Laidlow, Stefan Leutenegger, and Andrew J Davison. In-place scene labelling and understanding with implicit scene representation. In CVPR , 2021.

Q. Zhou, T. Weinkauf, and O. Sorkine. Feature-based mesh editing. In Eurographics, Short Papers , 2011.

Chuhang Zou, Ersin Yumer, Jimei Yang, Duygu Ceylan, and Derek Hoiem. 3d-prnn: Generating shape primitives with recurrent neural networks. In ICCV , 2017.

Appendix

A Ablation Study

A.1 Importance of Jointly Using Surface and Curve

Tab. A2 shows the importance of combining surface and curve reconstruction for CAD reconstruction. Fig. A7 shows qualitative results of the final CAD model outputs and also includes intermediate results, such as point clouds from density fields.

We see that in surface-only reconstruction, occlusion between instances can result in missing base faces, as seen with the central cylinder in the top row of Fig. A7 (a). This occlusion impedes accurate CAD parameter reconstruction. Conversely, the curves (see Fig. A7 (a)) capture this extrusion instance with accurate start-end labels, enabling successful CAD reconstruction as demonstrated in the curve only and MV2Cyl CAD outputs.

In Fig. A7 (b), we see that the curve-only reconstruction struggles at the (small) intersection where the thin rod meet, failing to capture one base curve. This omission leads to a predicted zero height of the instance, highlighting a limitation in reconstructing that specific section using curve data alone. On the other hand, the surface reconstruction successfully reconstructs this shape. We leverage the reconstructed barrel face of the instance to accurately predict its height ( h ) and center ( c ). This approach enables MV2Cyl to successfully reconstruct instances that the curve only model may miss.

SurfaceOnly

Curve Only

Instance

Start-End

Output

Instance

Start-End

Output

GT

Ours

CAD

pcurve

gcurve

CAD

Invalid

Sketch

Invalid

Sketch

To summarize, our synergy of utilizing both surface and curve information allows MV2Cyl to effectively reconstruct instances even where individual data types may fail, showcasing its comprehensive

reconstruction capability. Specifically, curve representation is effective against occlusion and precisely captures the closed curve of a sketch, making it suitable for reconstructing the 2D implicit sketch. However, it may not always be reconstructed successfully due to its localized presence within the scene. On the other hand, surface representation provides a more dependable basis for fitting a 3D plane to determine the extrusion axis, which is vital for inferring subsequent CAD parameters. Yet, the base faces necessary for this process might not be available. This is either because of subtractive boolean operations or occlusion from combining multiple shapes. We thus address the challenges by jointly reconstructing curves and surface representations of shape and utilizing them to recover CAD parameters.

A.2 Separate 2D Segmentation Modules for Curve and Surface

In MV2Cyl, the curve and surface segmentation modules are kept separate. To evaluate how segmentation performance changes when the 2D curve and surface segmentation networks are combined, we test a shared U-Net backbone that branches into two outputs for curve and surface segmentation. The results of using merged and separate U-Nets to extract 2D segmentation maps for our reverse engineering extrusion cylinders task are shown in Tab. A3. While a slight improvement in performance is observed on some metrics (Fit Cyl. and Fit Glob.), the results from both approaches are comparable.

A.3 Ablation on the Number of Instances

We evaluate the impact of varying the number of instances on MV2Cyl's performance using the Fusion360 dataset. Tab. A4 shows the results breakdown of the models in the Fusion360 test set across different numbers of extrusion instances K. Note that the number of samples for K = 7 and K = 8 is very small, and these statistics should therefore be interpreted sparingly.

A.4 Ablation on the Number of the Input Images

In this section, we analyze the effect of varying the number of input images on MV2Cyl's performance using the Fusion360 dataset. Tab. A5 shows that our approach is robust to changes in the number of input images, maintaining reasonable performance even with as few as 10 input views. Although performance slightly degrades as the number of input images decreases, MV2Cyl still produces effective reconstructions, demonstrating its adaptability to limited input data. As expected, the use of 50 input images yields the best overall results due to the richer information available for reconstruction.

A.5 Ablation on the Line Width of the Curve Segmentation Maps

In this section, we investigate the impact of varying the line width of curve segmentation maps on MV2Cyl's performance. Tab. A6 shows that our method is relatively robust to changes in line width, with performance remaining consistent across different widths. The results indicate that using 5 pixels (our default setting) achieves a good balance between accuracy and consistency. While slight variations in error metrics are observed when using line widths of 2.5 or 7.5 pixels, these changes are minimal, suggesting that MV2Cyl can handle a range of line width settings without significant performance loss.

B Real-World Demo

In this section, we showcase the feasibility of using MV2Cyl in reconstructing CAD models from real-world object captures. We provide the details of the process as follows.

To reverse engineer a real-world object, we begin by capturing videos of the target object using an iPhone 12 Mini. These videos are then converted into multi-view images through frame extraction, which are subsequently input into the image segmentation models. The training dataset for our segmentation models is generated by rendering untextured models in Blender, producing RGB images with a grayscale appearance against a transparent background. To address the domain gap between the training data and real-world demo images, we preprocess the real images by converting them to grayscale and removing the background. While these preprocessing steps help align the datasets, a residual domain gap remains, which may still impact segmentation performance on real images. To address this challenge and enhance segmentation efficacy, we employ a robust 2D segmentation model known as Segment Anything (SAM)[24]. By fine-tuning the pretrained SAM model using LoRA-based finetuning code[71], with the same training dataset applied to the U-Nets, we observe substantial improvements in segmentation performance on real images, even in the presence of distributional shifts. The finetuned SAM model leverages extensive prior knowledge gained from training on a significantly larger dataset, ensuring robust performance in 2D segmentation tasks. We acquire the camera's extrinsic and intrinsic parameters using COLMAP [52, 53].

Fig. A8 displays an example of a real image alongside the predicted segmentation maps from the finetuned SAM model. The processed multi-view images and their corresponding camera poses are then input into the reconstruction pipeline, which focuses on learning the density and semantic fields. Subsequently, CAD parameters are estimated based on this reconstruction. Tab. A7 presents results on final reconstructed CAD models in real captures, demonstrating successful output of CAD models from various real-world objects.

RGBimage

Grayscaleimage

Backgroundremovedimage

Instancesegmentation

Start-endsegmentation

VOLVO

We quantitatively evaluate MV2Cyl on real data found in Tab. A7. Our reconstruction achieves an average Chamfer distance of 2 . 11 × 10 -3 from the ground truth CAD model. To compute this distance, we first sampled 8192-point point clouds from the reconstructions and then aligned the reconstructed model to the corresponding ground truth shape using ICP. After alignment, the desired metrics were computed. This process is illustrated in Fig. A9.

Blue:ground truthpoint cloud

Yellow:ground truthpoint cloud

Samplepointcloud

Export real demo CAD output to mesh

&Shape registration to GT point cloud

Open3DICPpoint cloud registration

C Comparison with SECAD-Net [30]

We further compare our MV2Cyl against an unsupervised CAD reconstruction baseline SECADNet [30]. Unlike our approach that takes multi-view images as input, SECAD-Net takes a 3D voxel grid as input. It trains a model that directly outputs a shape as a union of predicted extrusion cylinders. We evaluate how well their approach reconstructs individual extrusion cylinders using the same metrics as in [58] and compare them against our MV2Cyl. Results are shown in Tab. A8. We see that our approach outperforms SECAD-Net [30] for both the Fusion 360 [64] and DeepCAD [65] datasets on all metrics that quantify extrusion cylinder reconstruction.

For fair comparison, we evaluated the test set as released by [58] on both datasets and used the scripts from the code release of SECAD-Net [30] to convert the test shapes to voxelized inputs and to train or fine-tune the SECAD-Net model with our data. For both datasets, we adhered to the evaluation

Real Image

MV2Cyl

Extrusion Cylinders

O

protocol from [30], fine-tuning SECAD-Net for 300 epochs per test shape using a pre-trained model, followed by mesh extraction at a 256 resolution with Marching Cubes [37]. We used Hungarian matching based on Chamfer distances to find the best match between ground-truth and predicted segments, which allowed us to calculate the metrics reported. The Fusion360 dataset evaluations were conducted on 798 test shapes, while the DeepCAD dataset included 1,950 test shapes. These numbers reflect instances where both MV2Cyl and SECAD-Net were able to produce valid CAD models.

Tab. A8 demonstrates that MV2Cyl outperforms SECAD-Net [30] across all metrics. SECAD-Net lacks supervision for segmenting parts in the way CAD designers would, leading to issues such as over-segmentation. This is evident in the images of Tab. A9. For example, in the second row, SECAD-Net segments two vertically attached cylinders into several patches along their surfaces. In comparison, MV2Cyl more accurately represents these components as two distinct cylinders, showing its improved ability to align with CAD design conventions.

GT Mesh

Reconstruction

Extrusion Cylinders

MV2Cyl

SECAD

MV2Cyl

SECAD

MV2Cyl

SECAD

r

R

D

D Comparison with Curve + Point2Cyl [58]

We compare our MV2Cyl with an additional baseline Curve+Point2Cyl [58] that is a modified version of Point2Cyl that incorporates an additional "curve" feature into the input point cloud. While MV2Cyl benefits from the supervision of curve information from the CAD, we apply equivalent supervision to the baseline to maintain fairness in comparison. The evaluation was conducted using the Fusion360 [64] dataset.

To implement the baseline Curve+Point2Cyl (described in the Sec. 4 in the main paper), we modified the network's front-end where it receives inputs. The original Point2Cyl takes an input tensor of size N × 3 , representing the target point cloud (see Fig. A10). To incorporate information about specific curves within the point cloud, we concatenate the M curve points after the N input points. These curve points are assigned a label of one on an additional channel appended to the original three XYZ channels, while the original non-curve points retain a label of zero on this channel (see Fig. A11). The rest of the network architecture remains unchanged. N is set to 8,192 and M is set to 800. Notably, the curve points are obtained from the ground truth CAD models for both training and testing phases.

Input

𝚸 ∈ ℝ

!×#

PointNet++

backbone

𝓖

𝜽

FC

Μ & ∈ ℝ

!×%&

Ν & ∈ ℝ

!×#

W & ∈ ℝ

!×&

Β

& ∈

0, 1

Segmentation

Reordering

ℒ

'()

ℒ

**

ℒ

+,-.

Parameter

Estimation

S /

𝐃𝐞𝐜𝐨𝐝𝐞𝐫	𝒮

2D coordinate

𝑟 ∈ ℝ

%

ℒ

'/(012

𝑓

3

S /

ℒ

Ρ

Μ &

Ν &

Inputs

Variables

Network

External

Solver

Differentiable

Module

N

: Number of points

K

: Number of segments

FC

𝒆 ? ∈ ℝ

&×#

&×#

𝑠̂ ∈ ℝ

&

𝚷

Μ &

𝐒𝐤𝐞𝐭𝐜𝐡	𝐈𝐦𝐩𝐥𝐢𝐜𝐢𝐭

Inferring Extrusion Cylinder Parameters

Inferring Sketches

3

pred sketch

S

gt sketch

𝑔 S /

Training

Inference

Projector

Operator

Input

!×#

PointNet++

backbone

$

$

FC

Μ & ∈ ℝ

!×%&

Ν & ∈ ℝ

!×#

W & ∈ ℝ

!×&

Β

& ∈

0, 1

Segmentation

Reordering

ℒ

'()

ℒ

**

ℒ

+,-.

Parameter

Estimation

S /

0123456	8

9523456	:

2D coordinate

; ∈ ℝ

%

ℒ

'/(012

<

3

S /

ℒ

Ρ

Μ &

Ν &

Inputs

Variables

Network

External

Solver

Differentiable

Module

N

: Number of points

K

: Number of segments

FC

> ? ∈ ℝ

&×#

@ ? ∈ ℝ

&×#

Â ∈ ℝ

&

C

Μ &

DE5F2G	HIJKL2LF

Inferring Extrusion Cylinder Parameters

Inferring Sketches

0123456	<

3

pred sketch

S

gt sketch

8 S /

Training

Inference

Projector

Operator

Input

+ Curve

𝐏′ ∈ ℝ

𝑵"𝑴 ×𝟒

Tab. A10 shows the quantitative comparison between MV2Cyl and Curve+Point2Cyl. MV2Cyl surpasses Curve+Point2Cyl across all the metrics. Despite incorporating ground truth curve features, Curve+Point2Cyl performs similarly to the original Point2Cyl. In practical applications, it might

be expected to use predicted curves or edges; however, even with ground truth curves, we observed no significant improvement over the original Point2Cyl, which lacks curve data. This suggests that Point2Cyl's network may prioritize surface points over boundary points, making the additional boundary information less impactful.

E Implementation Details

In this section, we elaborate the implementation of the networks presented in the main paper. Sec. E.1 contains the details of the training 2D segmentation frameworks (Sec. 3.2 in the main paper) and Sec. E.2 includes the details of the 3D field learning from the 2D segments (Sec. 3.3 in the main paper).

E.1 Details of 2D Segmentation Frameworks

Each 2D segmentation framework M surface and M curve includes two U-Nets for instance segmentation P type and start-end(-barrel) segmentation Q type where the type is either surface or curve, respectively.

Details of 2D Surface Segmentation Framework M surface .

The 2D surface segmentation framework, M surface , maps RGB images of dimensions H × W × 3 to a set comprising the extrusion instance segmentation label, P surface ∈ { 0 , . . . , K } H × W , and the start-end-barrel segmentation label Q surface ∈ { 0 , 1 , 2 , 3 } H × W , where K indicates the number of instances. We fix K = 8 instances following [58] and train with image resolutions of H = W = 400 . For the Fusion360 dataset, 196,050 training images derived from 3,921 shapes are used, while for the DeepCAD [65] dataset, derived from a subset of the training set, 709,000 images from 14,180 shapes are utilized. The datasets are divided into training and validation sets in a 9:1 ratio for the U-Net [50] training. The training times for 3 epochs are 316 minutes for the Fusion360 [64] dataset and 381 minutes for the DeepCAD dataset, using a single NVIDIA RTX A6000 GPU.

Details of 2D Curve Segmentation Framework M curve .

The 2D curve segmentation framework, M curve , processes RGB images ( H × W × 3 ) to produce two types of labels: the extrusion instance segmentation label, P curve ∈ { 0 , . . . , K } H × W , and the start-end segmentation label, Q curve ∈ { 0 , 1 , 2 } H × W . Similarly following [58], we also set K = 8 with a training and validation image resolution of 400 × 400 . The training objective is defined as:

where λ CE , λ focal , and λ dice are all set to 1.0, balancing the contributions of cross-entropy, focal [33], and dice [41] losses to the overall training objective.

For the Fusion360 dataset, we utilize 196,050 images from 3,921 shapes. For the DeepCAD dataset, we use 1,745,450 images from 34,909 shapes from the training set. These image datasets are split into training and validation sets at a 9:1 ratio. The Fusion360 dataset's U-Net is trained over 3 epochs, taking 307 minutes, while the DeepCAD dataset's U-Net training for a single epoch requires 20.8 hours, all on a single NVIDIA RTX A6000 GPU.

E.2 Details of Integrating 2D Segments to 3D Fields

The opacity field σ ( x ) hyper-parameters are set to ζ = 10 and β = 0 . 8 as in [68]. For the reconstruction loss of the density field F , we set λ sparsity = 0 . 5 , s = 0 . 5 , η batch = 0 . 1 , and η image = 0 . 05 . We use a batch size of 8,192 rays for the training of both the density field F and the semantic field A . We trained all the fields for 1500 iterations on a single NVIDIA RTX 3090 GPU. The whole training time is approximately 5 minutes per shape. Note that the segmentation process

and the learning of object density and semantics are typically done as separate steps, rather than in a single, continuous process.

E.3 Reverse Engineering Implementation Details

We provide additional details on our reverse engineering process, i.e. from our density and semantic fields to the reconstruction of the extrusion cylinders, as discussed in Sec. 3.4 of the main paper. This process begins by sampling points from the density fields, and then from the extracted point clouds deriving the corresponding parameters for each extrusion cylinder. We query the learned density field value across a 400 × 400 × 400 grid. Points on this grid with an opacity σ value greater than a threshold τ ≥ 0 . 99 are selected and their corresponding instance segmentation and start-end(-barrel) segmentation labels are extracted. The extrusion axis ( n ), height ( h ), and center ( c ) are determined directly from the sampled point clouds as detailed in the main paper.

The process to extract the 2D implicit sketch involves additional steps detailed as follows: After predicting the extrusion axis, the 2D sketch boundary points are created by projecting the base curve points along this axis. These 2D point clouds are then used to infer the 2D implicit sketch using IGR [15], which takes as input the projected 2D sketch point clouds and their normals to generate the corresponding signed distance fields. To extract the 2D point normals for IGR, we modify PGR [32] to estimate 2D normals from our projected 2D sketch points. We adapt the algorithm from PGR and modified 2D basis as follows:

where w ( x ) is the width function defined in PGR. For an in-depth understanding of the formula and its derivations, we direct readers to the PGR paper [32]. Fig. A12 displays the impressive results of our 2D normal estimation implementation, showcasing the accuracy and quality of our approach. These 2D point clouds and the estimated normals are fed into IGR network to output the 2D implicit sketch represented as 2D signed distance function.

Y

E.4 Multi-View Images Preparation Details

We detail the generation of multi-view image datasets used in our experiments. Adapted from [68], the rendering process utilizes BlenderProc [7] and we render 50 views with resolution 400 × 400 for each shape, where the viewpoints are sampled by Fibonacci sampling [17] evenly distributed

over a sphere. The surface and curve segmentation labels, we start off by coloring the ground truth mesh black, then marking each segment white. The meshes marked per instance are then rendered in diffuse mode to produce a highlight map for each segment from all viewpoints. Finally, the highlight maps are merged into a single segmentation map, creating multi-view segmentation maps consistently annotated with GT labels across views.

E.5 NeuS2 [62] + Point2Cyl [58] Implementation Details

We provide additional details on the baseline NeuS2 + Point2Cyl. There are many recent and highperforming studies on converting multi-view images to 3D geometry [40, 61, 44, 13], and we chose NeuS2 because this work is based on InstantNGP [44] and offers fast convergence in reconstructing shapes. For instance, while NeRF [40] takes about 8 hours to reconstruct a sample from the DTU dataset [20], NeuS2 can achieve more accurate geometry in just 5 minutes. It has been confirmed that NeuS2 achieves more accurate geometry than InstantNGP, which has converged in the same duration. We selected this study considering the efficiency and feasibility of evaluation. In the test with NeuS2+Point2Cyl, we used a set of images with 50 view points per shape, at a resolution of 400 × 400 , and trained for 20,000 iterations. We then ran a marching cube at a resolution of 64 to obtain the mesh, and sampled 8,192 points from that surface to apply as input to Point2Cyl.

E.6 Binary Operation Prediction

We introduce a straightforward method for recovering binary operations. A naïve approach involves performing an exhaustive search across all possible combinations of primitives and operations, resulting in 2 K possibilities for a model with K primitives. To identify the optimal combination that aligns closely with the observed input images, we measure the average per-pixel L2 distance between the rendered images of each combination's resulting model and the observed multi-view images, selecting the combination with the lowest distance. We have implemented this approach to recover the binary operations, and examples of the results are illustrated in Fig. A13.

Inputimageof the

Reconstructed part 0

Part 1

Part 2

correspondingcamerapose

2K combinations of K reconstructed parts

Rendering usinginput multi-viewimages'camera poses

Comb.0

Comb.1

Comb.2

Comb.3

Comb.5

Comb.6

Comb.7

Part 0 (+), 1 (+), 2 (+)

Part 0 (+), 1 (+), 2 (-)

Part 0 (+), 1 (-), 2 (+)

Part 0 (+), 1 (-), 2 (-)

Part 0 (-), 1 (+), 2 (+)

Part 0 (-), 1 (+), 2 (-)

Part 0 (-), 1 (-), 2 (+)

Part 0 (-), 1 (-), 2 (-)

Thebestcombinationwiththeminimal

L2 distance from the giveninput image

E.7 Implicit sketch to parametric curves

We also provide example showcases of converting an implicit sketch into a parametric representation. After generating the 2D implicit field for each sketch, we sample boundary points using the marching squares algorithm, which serve as knots for cubic Bézier curves. Using these points, we employ an off-the-shelf spline fitting module from the SciPy library [59] to determine the optimal B-spline representation based on the provided control and knot points. This process is illustrated in Fig. A14.

It is important to note that the resulting sketch may not comprise the minimal set of parametric curves; for example, a single line segment may be represented by multiple segments.

OODOO

00000O

O

F Additional Analysis

F.1 Comparison of Segmentation Performance Between 2D and 3D Networks

We evaluate the segmentation performance of networks in MV2Cyl and Point2Cyl [58], which process 2D images and 3D point clouds, respectively. The 2D segmentation network in MV2Cyl processes 400 × 400 resolution images, categorizing them into background(0), start face(1), end face(2), and barrel face(3). It was tested on 55,400 images from 1,108 shapes. Conversely, the 3D segmentation network in Point2Cyl processes 8,192-point point clouds, segmenting them into barrel(0) and base(1) points, with 1,108 point clouds evaluated. Table A11 details the test accuracy and test accuracy excluding the background for the 2D network. Notable, the 2D segmentation network outperforms the 3D segmentation network, supporting our hypothesis that 2D networks are more effective than 3D networks at segmentation tasks, particularly in extracting the spatial information needed to reconstruct a CAD model.

F.2 Consistency Between Curve and Surface Segmentations

We compute a consistency metric between the curve and surface predictions. Specifically, we compute the percentage of models where the number of predicted curve instances and surface instances are identical. Across all the test shapes in the Fusion360 dataset, 98% of the models (1,095 shapes over 1,108 shapes) have the same number of curve and surface segments. Moreover, we also measure the consistency between the extracted curve and surface point clouds. Concretely, we measure the average one-way Chamfer distance between the extracted curve point cloud and surface point cloud ( 0 . 081 × 10 -3 ). We see that this very small value suggests that the extracted curve point clouds are quite consistent with the surface point clouds.

F.3 Chamfer Distance Evaluation for 3D Reconstruction

We quantitatively evaluate the 3D reconstruction quality of MV2Cyl by measuring the Chamfer distance against the ground truth CAD model. To generate the output CAD model, binary operations

are determined through a simple exhaustive search approach. The results, shown in Tab. A12, demonstrate that our method outperforms the baseline, NeuS2+Point2Cyl. For a fair comparison, we compare with NeuS2+Point2Cyl instead of NeuS2, as our work focuses on the reverse engineering of extrusion cylinders, which is a more challenging task than 3D reconstruction without CAD structure. Reverse engineering extrusion cylinders enables shape editing and importing the model back into CAD software.

G Additional Qualitative Results

We provide additional qualitative visualizations of the reconstructed extrusion cylinders using our proposed MV2Cyl. Results are shown in Tab. A13. It showcases MV2Cyl's exceptional ability to process and accurately render shapes of diverse appearances on an instance-wise basis. This demonstration underscores the versatility and precision of our approach in capturing and reconstructing complex geometries from multi-view images.

GT Mesh

MV2Cyl

Extrusion Cylinders

GT Mesh

MV2Cyl

Extrusion Cylinders

C

GT Mesh

MV2Cyl

Extrusion Cylinders

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The main claims presented in the abstract and introduction are consistently reflected and elaborated upon in the main text.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The limitations of the work are outlined in the conclusion section.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: This work does not include any theoretical assumptions.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: The detailed procedure of the method is delineated in the method section and appendices.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [No]

Justification: The code and data will be made available upon the acceptance of this work. Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: The training and test details are explained in the main text and the appendices.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [No]

Justification: Due to the high computational demands of the neural rendering-based 3D reconstruction step in this work, iterating over multiple trials is challenging because of limited time and computational resources.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: The computer resources required for the experiments are described in the experiment section and the appendices.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: This research adhered to the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We discuss societal impacts in the last section of the main paper.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: This work does not pose such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: All existing assets used in this work are properly cited in the text.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: This work does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This work does not involve any crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors introduce a novel method named MV2Cyl, which reconstructs technical 3D  objects withing the sketch-extrude paradigm by utilizing a 2D prior model and a learnable radiance field derived from multi-view images. To accomplish this, the 2D prior model is trained on a labeled dataset to predict semantic labels, $K$ extrusion cylinders, and extrusion curves, which are crucial for further processing of the 3D shape. Once the model is trained, it predicts pseudo-ground-truth for the subsequent stage. In this stage, a radiance field model is trained to replicate the pseudo-ground-truth labels. As the radiance field model converges, it facilitates the extraction of surfaces and curves in 3D space. The authors then detail how to utilize this output to reconstruct the final shape. The experimental results demonstrate that each component significantly contributes to achieving 3D reconstructions that surpass the selected baselines. Given that the radiance field is trained solely from the multi-view images and the prior model, it offers improved real-world applicability compared to previous works.\n\nSTRENGTHS:\n- The paper is overall well-written and easy to follow.\n- Each component is effectively justified, and the ablation study outcomes distinctly demonstrate their significance.\n- Given the extensive datasets like ABC [1], the prior training represents a significant contribution to this research. It is conceivable that the prior model may serve as a foundational model for CAD extraction in subsequent studies.\n- Additionally, the prior and inference models are independent, and both can be adapted adequately if new, better models appear in the literature. This should also facilitate scaling with the datasets.\n- I appreciate the demo displayed in the Appendix. It illustrates the practical application of the model and reinforces the societal impact discussed in the final section of the paper.\n- One can argue about the pros and cons between different paradigms (such as sketch-extrude, CSG, etc.). The choice of sketch-extrude in this instance is substantiated by the results and outperforms the established baselines.\n- Publishing the code and the extended dataset upon acceptance is highly appreciated.\n \n[1] Koch, S, Matveev, A, Jiang, Z, Williams, F, Artemov, A, Burnaev, E, Alexa, M, Zorin, D, Panozzo, DABC: A Big CAD Model Dataset For Geometric Deep Learning. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2019 .\n\nWEAKNESSES:\n- Point2Cyl, ExtrudeNet, and SECAD-Net encode inputs as latent vectors, which are then decoded into the target representation. An encoder in this context could be any model that processes input in a specific modality. Adapting these methods to handle multi-view images appears feasible and would support the paper's claims. Notably, an encoder understanding 3D geometry based solely on 2D images could be advantageous (for instance, utilizing part of the network from [2] as an encoder).\n- While the paper is generally understandable, Section 3.4 is hard to follow. Introducing a simple visualization to illustrate the process would aid comprehension.\n- In Section 3.4, points 1 and 2 prompt inquiries about the 3D model employed in this study. The authors refer to TensorRF, but it is unclear whether a standard NeRF or NeuS was used. If NeuS was utilized, the predicted normals could be enough to:\n    - Determine a fitting plane by voting for the plane that most likely aligns with the normals.\n    - Derive the output shape using a standard algorithm like ball-pivoting or Poisson surface reconstruction, both of which are widely accessible.\n- Furthermore, a NeuS backbone would arguably be more appropriate for the application described in the paper, as it more precisely reconstructs surface fields, potentially enhancing the accuracy of the curve field.\n\n[2] Watson, D, Chan, W, Martin-Brualla, R, Ho, J, Tagliasacchi, A, Norouzi, M. \"Novel view synthesis with diffusion models\". arXiv preprint arXiv:2210.04628 2022.\n\nQUESTIONS:\nQuestions:\n- I will refer to:\n> We first extract the surface point cloud and curve point cloud from the corresponding 3D fields by thresholding the corresponding existence fields. Each point in the point cloud would have attributes queried from the corresponding attribute fields.\n\nHow is the point cloud sampled specifically? Is it accurate the underlying radiance field uses density instead of SDF field like in NeuS?\n\nSuggestions:\n- The explanation in lines 44-47 assumes prior knowledge of the sketch-extrude principle. Without this knowledge, it's challenging to understand the source of the issue. An inset visualization could greatly aid comprehension.\n- I propose renaming the \"existence field\" and \"attribute fields\". The term \"existence field\" implies a binary output indicating whether an object exists at a given point. \"Density field\" would be more appropriate, clarifying how prior values translate to opacity and aligning with the field's intended purpose.\nRegarding the \"attribute field\", the term could be misconstrued as referring to \"attributes\" such as colors. \"Semantic field\" would be more accurate.\n- It can be deducted from the context that $L_\\text{existence}$ applies to both curve and attribute fields. However, stating that clearly in the text would remove potential ambiguities for other readers.\n\nLIMITATIONS:\nThe potential negative societal impact is clearly stated and supported by the real-demo presented in the Appendix.\n\nRegarding the limitations, it is unclear what authors mean by:\n> our framework is limited in predicting binary operations across primitives. We plan to further explore predicting binary operations using multi-view image inputs.\n\nin lines 328-331. Some examples or an visualization in the Appendix would be appreciated here. It would be worth adding a comment on the shape complexity the model cannot handle.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a new method to reconstruct extrusion cylinders from multiview images. The key idea is to train a CNN to predict the binary mask for each surface and the sketch of each surface. Then, these predictions are used in learning 3D neural fields for each surface and sketches by the volume rendering technique.\n\nSTRENGTHS:\n1. The task seems to be interesting in that we use multiview images with semantic segmentation to help reconstruct 3D extrusion cylinders. This method is novel for me.\n2. The experiments demonstrate some improvements over previous reconstruction methods.\n\nWEAKNESSES:\n1. How to associate each surface is unclear here. If I understand it correctly, the predicted labels are similar to instance segmentation labels so we may have multiple different labels for the same region. How to associate predicted surfaces with different viewpoints is not clear here.\n2. The prediction of feature lines could be sensitive to the line width. If we use a large line width the localization of the feature lines could be difficult because we cannot learn a perfect 3D field for these lines. If we use a small line width, learning a feature field for signals just on very sparse pixels could be problematic and we may extract disconnected feature lines (sketches) here. The setting of the linewidth for training could be tricky here.\n3. It is difficult to get some real-world images to train these CNNs, which could harm the generalization ability of the proposed method. If we have painted some textures on these CAD models, then the proposed method could fail because it is not trained on models with textures.\n4. The proposed method seems to be too concentrated on a specific problem in computer graphics. Predicting extrusion cylinders is not a common task in daily life and the proposed method does not contain technically new ideas on neural networks. The paper seems to be more suitable for some graphics journals or conferences rather than NeurIPS.\n\nQUESTIONS:\nRefer to Weakness.\n\nLIMITATIONS:\nLimitations are discussed in the paper, which is the binary prediction of CNNs. I think the method could be sensitive to hyperparameters, which could be a potential limitation.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nMV2Cyl is a method that proposes to solve the 3D reverse engineering of CAD models. The network takes as input multi-view images and outputs extrusion cylinders. The method extends Point2Cyl [58] that proposed to predict extrusion cylinder from point clouds. It is argued that multi-view images can easily be obtained from 3D scans and that 2D CNNs have superior performance than 3D processing networks.\nThe network is composed of three main components. Firstly, 2D CNN U-Net-based networks learns to segment and classify the pixel image of the 2D images into surface and curve segments. Then, a NERF based approach is used to leverage the 2D segments into 3D existence and attribute fileds. Finally, 3D surface and curve point clouds are extracted from the 3D fields. The extrusion parameters (extrusion axis, height and centroid) and the sketch (implicit function) are estimated from the extracted point clouds.\nThe method is evaluated on the DeepCAD[64] and Fusion360[63] datasets, from which multi-view images are extracted. The results show an improvement compared to Point2Cyl[58] and a Neus[51]+Point2Cyl.\n\nSTRENGTHS:\n- The paper is fairly clear and easy to understand for a reader that is familiar with Point2Cyl [58]. \n\n- The main strength of the proposed work is that it shows an improvement to the results presented in Point2Cyl [58]. This is achieved by leveraging the information provided in multi-view images as opposed to 3D point clouds.\n\nWEAKNESSES:\n- While the proposed work offers improvements to Point2Cyl [58], the reviewer feels that it does not address the main limitations of Point2Cyl. Firstly, the binary operations between cylinder (how to assemble the cylinders into the GT CAD model) are not predicted. This limitation is mentioned in the paper but addressing it would have lead to a significant improvement to Point2Cyl. Secondly, the sketches are predicted as implict functions and not parametric curve primitives such as in real CAD models. This produces noisy surfaces (as shown in Figure 3 of the paper), which are quite different from the standard sharp representation of CAD models.\n\n- The evaluation consists of a comparison with Point2Cyl using some of the metrics proposed in Point2Cyl. Nevertheless, further analysis of the results could be completed. For example, it could be meaningful to compute a consistency metric between the surface and curve predictions. One would expect the two network to predict the same number of extrusion cylinders. Also, are the extracted curve point clouds consistent with the surface point clouds? A comparison to the point cloud to CAD sequence experiment presented in DeepCAD[64] could also be included as the extrusion cylinders can easily be extracted from the CAD sequences. It would also be interesting to know how the performance of the network changes with respect to the number of extrusion cylinders (K) present in the ground truth.\n\nQUESTIONS:\n- The representation of the output is a little unclear. Figure 3 suggests that the output of MV2Cyl is a CAD model. Can a CAD B-Rep be obtained from the output? If so, how are the sketch primitive and parameters obtained from the predicted implicit functions?\nBeyond the points mentioned in the weaknesses, it would be interesting to know how the perfomance of the model changes with respect to the number of input images at test time. \n\n- Many qualitative results are presented in the Appendix, all of them show very impressive predictions. It would also be interesting for the reader to see examples for which the model fails to recover the correct shape.\n\nLIMITATIONS:\n- The authors have addressed one of the main limitations of the work that is the lack of prediction of the binary operations between the different cylinders. Potential societal impacts have also been included.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a method to predict extrusion cylinders from images of a CAD part. Specifically, it takes a set of masked multi-view images as input; these are then processed independently by instance segmentors trained to find extrusion curves and surfaces. Neural fields are then fitted, that reconstruct these 2D segmentations in terms of 3D segmentations. Lastly, a set of primitives explaining the 3D segmentations is extracted using a series of heuristics (RANSAC and other hand-crafted robust estimators). Experiments are conducted on two standard benchmark datasets (Fusion 360 and Deep CAD), and show the proposed method out-performing a baseline approach (NeuS reconstruction followed by an existing 3D primitive-prediction pipeline).\n\nSTRENGTHS:\n- The work presents a sophisticated and carefully engineered pipeline, with choice of components clearly motivated.\n- The approach of fitting several 3D fields to the 2D segmentations, and regularising to ensure these are binary, is interesting.\n- There is a fairly extensive evaluation (qualitative and quantitative) on synthetic (rendered data), including comparison against a baseline that first performs 'naive' surface reconstruction, then applies a 3D-only primitive extraction to find the extrusion cylinders.\n- Quantitatively, the proposed method is found to significantly out-perform the reconstruction-then-fitting baseline. Indeed, it also out-performs a method that directly predicts extrusion cylinders from ground-truth 3D point-clouds, which is a notable success since the latter would seem to have access to more complete information\n- Qualitatively, the proposed method appears to correctly capture geometric features of the input objects, using an appropriate (minimal) set of primitives, and providing much cleaner reconstructions than a naive \n- The paper is well-written, sensibly organized and pleasant to read throughout.\n\nWEAKNESSES:\n- The work focuses on clean synthetic (rendered) inputs, with only one experiment (in the appendix) on real data. However, operating on multi-view images offers no benefit over 3D-input methods when one has access to the 3D shapes (e.g. as point-clouds). For a strongly 'applied' paper like this, the lack of focus on the practically-relevant scenario (i.e. actual photos) is problematic.\n- Real-world results (though still with in-distribution CAD shapes 3D-printed from the datasets) are not very impressive, despite fine-tuning for this setting. There is also no quantitative evaluation her, which is a pity since this would be straightforward given that the ground-truth CAD models are available (modulo 3D printing imperfections)\n- There is no quantitative evaluation of the quality of 3D reconstruction in the sense of distance (e.g. hausdorff / f1) from the reconstructed shape to the original CAD model. It would be valuable to add this (and compare with NeuS2), to give an idea of how well the shapes are being predicted in a more universally-meaningful metric, as opposed to the given metrics that are specific to the shape representation and difficult to interpret in a broader context.\n- There are few generalisable insights present in the paper. It presents a pipeline that evidently works well for the specific task, but the technical contributions are very narrow in applicability, reducing potential impact.\n- Overall the paper is a poor fit for NeurIPS – lots of engineering (obviously an achievement in itself), but minimal machine learning (just training a segmentation model on synthetic data, then fitting the single-scene neural fields), making the interest to the community relatively low.\n\nQUESTIONS:\nHow/where do you predict whether each primitive is 'added' or 'subtracted' from the model? The results in Fig. 3 indicate that the model correctly inserts holes, but it is unclear in the method where this determination is made\n\nLIMITATIONS:\nThese are discussed briefly but adequately in Sec. 5.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\n## Summary of the Paper:\n\n*Problem Statement*:\n\n     Given multi-view images (RGB) of a 3D shape, the paper aims at recovering a *set* of extrusion cylinders to represent the underlying 3D shape.\n\n\n*Motivation*:\n\t\n     Existing works that address 3D shape reconstruction problem through sketch-extrude take raw 3D geometry as input (ex: Point2Cyl) whereas this work takes multi-view images. Other works that take multi-view images as input perform parametric reconstruction (ex: NEF), which while important, does not reconstruct the 3D surface itself. And so, there exists a gap in the community in terms of either the input representation, or the output result, in the context of 3D reconstruction through sketch-extrude modeling. This forms the basis of this work, termed, MV2Cyl.\n\n*Contributions*:\n\n     Developing a method for sketch-extrude reconstruction of 3D shapes from multi-view images, via labeled surfaces and curves. \n\t\n\n*Input*: \n\n     Multi-view RGB images of a man-made 3D shape\n\n*Output*: \n\n     3D reconstruction of the object based on sketch-extrude cylinders. A cylinder is \n\n*Dataset used*: \n\n     Fusion 360 and DeepCAD\n\n*Underlying Modeling Tool*:\n\n1)\t U-Net for processing images (segmenting curves and surfaces)\n\n2)\t Implicit field-based volumetric rendering for integrating 2D estimated curve and surface information to corresponding 3D space\n\n\n*Learning Mechanism*:  \n\n     Strongly supervised\n\n\n*Loss Functions*:\n\n     For 2D surface segmentation, a U-Net with multi-class cross-entropy loss constrained by Hungarian matching algorithm is used. The Hungarian matching constraint is added to account for order invariance of the extrusion segments on the rendered 3D shape, as well as the to classify start-end-barrel segments.\n\n     The 2D curve segmentation module also follows the same architecture and loss functions. In addition, due to the sparsity of labels for curves (which can cause the network to just “attending” to the background pixels), two additional loss terms are used – Dice Loss and Focal Loss\n\n     2D to 3D integration using volumetric rendering: Here, the paper describes two fields. One is the Existence field and the other is the Attribute Field. \n\n     The training of the Existence field is dictated by the input 2D images, which is accomplished via differentiable rendering. A weighted combination of L2 loss (for surface) and sparsity loss (for curves) is used.\n\n     The training of the Attribute filed is governed by a multi-class cross-entropy loss function, constrained by Hungarian matching.\n\n\n*Quantitative Metric*:\n\n     Extrusion-axis error (E.A.), extrusion-center error (E.C.), per-extrusion cylinder fitting loss (Fit Cyl.), global fitting loss (Fit Glob.)  \n\n\n*Baselines*:\n\n     Since existing works do not take multi-view images for sketch-extrude 3D reconstruction tasks, no suitable comparison exists. However, the paper compares against a closely related work (and its variant), Point2Cyl (+NeuS2), which takes a point cloud as input.\n\nSTRENGTHS:\n1)\t Well-written paper\n\n2)\t Suitable comparisons have been made with Point2Cyl (and +Neus2) to investigate the apparent gains rendered by the proposed framework\n\n3)\t Experimental results on two relevant and complex CAD datasets (Fusion-360 and DeepCAD) are better than other closely related works\n\nWEAKNESSES:\n1)\t I wanted to understand how essential the 2D curve segmentation network is in making the reverse engineering possible. In other words, how would the cylinder extrusion results in 3D look like if this module were not present? This is a critical piece of experiment, something that should have been shown through ablation experiments. A similar ablation experiment for 2D surface estimation is NOT necessary though, since it is straight-forward to see why this is needed in the first place, given multi-view images as inputs.\n2)\t There is a lack of discussion on which kind of shapes does the proposed approach fail on. I would have liked to see a figure (with MV images, 3D GT and sketch-extrude 3D reconstructions) that shows the kind of shapes that MV2Cyl fails at, even if such results are just a tiny sample. This is consequential to the research push in this direction, and also does not do complete justice to the paper.\n3)\t In Table 1, the different quantitative metrics lack corresponding units. For ex., as I understand, the extrusion axis error (E.A) is an angle measurement. The reported number must have been in degrees, yes? Likewise, it would be good to see units of measurement, if applicable, for other metrics reported therein. If you want to save space, you could just put it in the table, instead of mentioning in the caption.\n4)\t The extrusion instance segmentation is performed by both 2D segmentation modules (Surface and Curve). I sense that there is a strong dependency on this task to help complete the other “auxillary” task in these modules. As this (extrusion instance segmentation) is a common task for the two, were there experiments done to assess if these two modules (Surface and Curve) could be merged? To me, there is something amiss. Would like to see an intuitive explanation justifying this two-step, common-task-occurrence justification.\n\nQUESTIONS:\n1)\t I do not see RGB inputs (as claimed on L 105) in the Teaser as well as Fig 1. I see just a grayscale image. Which one is it?\n2)\t How do you differentiate NEF (CVPR 2023) from the 2D curve segmentation framework? The curves that are of interest can be weakly considered as edges. Can you replace 2D curve segmentation network with NEF (part/whole) to detect the extrusion curves and the base curves?\n3)\t Dataset question (L259- 261)– how were these datasets enriched with 2D segmentation maps? That is, how were the ground truth 2D curve and extrusion segments obtained? Manually? Provide details.\n4)\t I am curious to know why ABC dataset was not considered. It is a highly relevant dataset that would have made the comparisons exhaustive. What was the reasoning here? \n5)\t Can you talk about the extendibility of this approach? I ask this question since I find it non-trivial to train the Existence and Attribute fields, as described in the paper.\n\nLIMITATIONS:\nYes, the authors have correctly addressed the limitations of the work, including any societal impact that may arise as a result of the publication.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}