Title: Sparc3D: Sparse Representation and Construction for High-Resolution 3D Shapes Modeling
Venue: NeurIPS 2025
Accepted: yes
Keywords: 3D Reconstruction

3D Generation
Abstract: High-fidelity 3D object synthesis remains significantly more challenging than 2D image generation due to the unstructured nature of mesh data and the cubic complexity of dense volumetric grids. Existing two-stage pipelines—compressing meshes with a VAE (using either 2D or 3D supervision), followed by latent diffusion sampling—often suffer from severe detail loss caused by inefficient representations and modality mismatches introduced in VAE.
We introduce Sparc3D, a unified framework that combines a sparse deformable marching cubes representation Sparcubes with a novel encoder Sparconv-VAE. Sparcubes converts raw meshes into high-resolution ($1024^3$) surfaces with arbitrary topology by scattering signed distance and deformation fields onto a sparse cube, allowing differentiable optimization.
Sparconv-VAE is the first modality-consistent variational autoencoder built entirely upon sparse convolutional networks, enabling efficient and near-lossless 3D reconstruction suitable for high-resolution generative modeling through latent diffusion. Sparc3D achieves state-of-the-art reconstruction fidelity on challenging inputs, including open surfaces, disconnected components, and intricate geometry. It preserves fine-grained shape details, reduces training and inference cost, and integrates naturally with latent diffusion models for scalable, high-resolution 3D generation.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents SparCubes, a fast and near-lossless remeshing algorithm that produces watertight 3D surfaces at high resolution in just 30 seconds, offering a 3× speedup over prior work. The authors also propose SparConv-VAE, a modality-consistent variational autoencoder with a sparse convolutional encoder and a self-pruning decoder, achieving efficient and accurate reconstruction without relying on global attention.\n\nSTRENGTHS AND WEAKNESSES:\n- The authors propose a method to address high-resolution 3D shape modeling, which is an interesting and relevant topic. Moreover, they conduct extensive experiments to validate the effectiveness of the proposed approach. In addition, the proposed method achieves this with less computational time compared to existing methods.\n\n- The use of sparse voxels over dense grids lacks sufficient justification, and directly optimizing sparse structures may lead to loss of fine details.\n\n- The reliability of the flood-fill algorithm on non-watertight shapes is questionable, and the decoupled SDF prediction lacks comparison with end-to-end alternatives.\n\n- Several metrics in Table 2 fall behind prior work without explanation, and the visualization comparisons lack key baselines such as XCubes.\n\n- The current method struggles to preserve fine details of the target and is prone to losing small or hidden components, making it challenging to apply in practical projects.\n\nQUESTIONS:\n- How does utilizing a sparse set of active voxels maintain 3D reconstruction accuracy compared to processing a dense voxel grid across the entire bounding volume?\n\n- How can flood-fill algorithms reliably distinguish between interior and exterior regions when the target object lacks watertight closure during initial training stages?\n\n- Could directly optimizing the geometric structure of sparse cubes instead of explicitly refining a globally accurate SDF lead to loss of fine surface details?\n\n- What are the advantages of decoupling SDF prediction into sign classification (using BCE loss) and magnitude regression (L2 loss)? Has end-to-end SDF regression with sign-aware loss functions been explored as an alternative?\n\n- In Table 2, many of the metrics fall short compared to previous work. The authors should provide an analysis to explain these performance gaps.\n\n- For the visualization comparisons, the authors should include more published methods as baselines, such as XCubes.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a new 3D representation for reconstruction and generation, named SparCubes. The representation builds upon sparse and deformable vertices and cubes. Different from the original marching cubes, the sparse vertices are optimized to move to the boundary of meshes. The SDF values and position differences are further optimized by 2D depth and mask renderings. Based on this representation, the paper presents a VAE and a subsequent diffusion model on the vae latents.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The proposed method is effective in converting meshes with open surfaces to watertight meshes, which is crucial in 3D generative modeling. The remeshing process brings little loss to model accuracy.\n- By converting the meshes to the SDF-based reprensentation, it facilitates the training of the 3D VAEs, involving no modality conversion but only compressing. \n- The reconstruction and generation results look superior than previous methods.\n\nWeaknesses:\n- SparCubes is implemented with custom CUDA kernels. I have concerns about the reproducibility if the code is not open sourced. \n- In the SparCubes reconstruction pipeline, the step 3-4 involve per-object optimization, which seems time-consuming. I am not sure whether the claimed time (line 63) includes these steps. \n- SparseFlex is not compared in the main paper, especially under quantitative evaluations.\n- Some technical details are missing. Please see questions below.\n\nQUESTIONS:\n- In the step 3 of SparCubes reconstruction, are the SD values on vertices also optimized along with the delta positions? Are all vertices optimized to move towards the boundary? If it is the case, does the spatial grouping of many vertices lead to unstable optimization, such as face flipping?\n- In eq 5. does the delta function mean the per vertex shift? What does approximately equal mean here, does it mean that there is a constraint on the movement of the vertex positions?\n- Reference [28] (TripoSR) is not a diffusion model but a LRM. In this context, it seems to make more sense to replace it with \"TripoSG: High-Fidelity 3D Shape Synthesis using Large-Scale Rectified Flow Models\".\n- \"x\" in Eq 3. should be math-bolded.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces SparC, a unified framework for high-resolution 3D shape modeling that addresses key limitations in existing 3D generation pipelines. The work consists of two main components: SparseCubes, a sparse deformable marching cubes representation that converts raw meshes into watertight surfaces at $1024^3$ resolution, and SparConv-VAE, a VAE based on sparse convolution architecture achieved SOTA performance when trained on the high-quality processed watertight meshes.\n\nThe paper's main contributions are:\n- A watertight remeshing algorithm, i.e., SparCubes, that preserves fine details and small components while achieving nearly 3 times speedup over baseline methods.\n- A space-conv-based VAE for SparCubes encoding.\n- State-of-the-art 3D shape reconstruction quality.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n- The paper is well-written and easy to follow.\n- The paper proposes an engineering solution to address the high-resolution watertight remeshing problem, which is a key bottleneck in current 3D generation workflows.\n- The paper achieves state-of-the-art reconstruction results both quantitatively and qualitatively across several benchmarks\n\n### Major Weaknesses\n\n- While this work focuses on remeshing algorithms, it only compares against Dora's remesh algorithm, which is a baseline implementation in their open-source code. The work should compare against more advanced remeshing algorithms and provide detailed analysis of advantages and disadvantages.\n- SparCubes involves 4 steps, but the ablation study lacks analysis of individual components. For example, the actual contribution of deformation optimization and rendering refinement steps is unclear to reviewer.\n- TRELLIS already adopts sparse convolution, but the differences and advantages of SparConv-VAE compared to TRELLIS's VAE are not clearly illustrated. It is hard to determine whether the improvements over TRELLIS mainly come from higher-quality remesh data or VAE architecture design.\n- The paper lacks clear descriptions of VAE and latent flow model details, including model architecture and latent resolution. Moreover, TRELLIS adopts a two-stage training approach - it's unclear whether this work follows a similar scheme.\n\n**Minor weaknesses**\n- Section 3.3 mentions hole filling but doesn't clearly explain where this algorithm is applied or demonstrate its effectiveness.\n- No analysis of SparCubes' resolution bottlenecks and failure cases.\n- The contribution of each loss term during vae model training is unclear.\n\nQUESTIONS:\nPlease refer to major weaknesses for more details.\n\n- Remesh algorithm comparisons and ablation study.\n- The novelty of sparconv-vae is not clear to me.\n- Model architecture and training details are missing.\n\nOverall, I think this paper is not ready for publication in NeurIPS. I will consider raise my score if the authors can address my questions.\n\nLIMITATIONS:\nWhile the authors mention some limitations in their conclusion section (texture loss and internal structure handling), the limitations discussion is insufficient and should be expanded to address several critical aspects:\n\n- Failure case analysis. The paper lacks discussion of failure modes. For example, what types of mesh structures cause the deformation optimization to converge poorly?\n- The technical limitations of the proposed remeshing algorithm is not clear and should be analyzed.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

comments: SUMMARY:
This work introduces a unified framework called SparC, that combines sparse deformable marching cubes representation SparseCubes with a vae-baed encoder SparConv-VAE in order to address the challenges faced by the existing two stage pipelines that can result in a major loss of details through the vae processing. The method achieves sota reconstruction results and preserves fine details of the shapes, while maintaining efficient training and inference.

STRENGTHS AND WEAKNESSES:
Strengths:
1. The paper is well written and provides a good motivation to the proposed method.
2. The literature review also neatly categorizes the existing methods and provides the pros and cons of the related work.
3. The experiments have proper details of the metrics used and the baselines.
4. Ablation studies are included to evaluate the speedup costs and training costs.

Weaknesses:
1. The method involves an optimization process in Step 3 and it would be helpful to understand the stability of this this optimization process. Simplest could be to show it over multiple seeds.
2. A visual comparison of impact of multi-view vs single-view rendered images in step 4 would be helpful.

QUESTIONS:
1. Did you evaluate your method on other datasets, in addition to the ABO, Objaverse and Wild datasets?
2. Were different seeds used in the quantitative comparison of Table 1? 
3. What is the impact of Step 4 of using multiple view images on the reconstruction / generation quality? Is there a visual comparison figure to show the impact?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-04