Title: Orientation Matters: Making 3D Generative Models Orientation-Aligned
Venue: NeurIPS 2025
Accepted: yes
Keywords: computer vision
Abstract: Humans intuitively perceive object shape and orientation from a single image, guided by strong priors about canonical poses. However, existing 3D generative models often produce misaligned results due to inconsistent training data, limiting their usability in downstream tasks. To address this gap, we introduce the task of orientation-aligned 3D object generation: producing 3D objects from single images with consistent orientations across categories. To facilitate this, we construct Objaverse-OA, a dataset of 14,832 orientation-aligned 3D models spanning 1,008 categories. Leveraging Objaverse-OA, we fine-tune two representative 3D generative models based on multi-view diffusion and 3D variational autoencoder frameworks to produce aligned objects that generalize well to unseen objects across various categories. Experimental results demonstrate the superiority of our method over post-hoc alignment approaches. Furthermore, we showcase downstream applications enabled by our aligned object generation, including zero-shot object orientation estimation via analysis-by-synthesis and efficient arrow-based object rotation manipulation.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn many datasets, objects are often presented in arbitrarily oriented canonical positions, which may not align with human perception of their natural orientation. \nThis paper addresses the challenge of such non-oriented objects by constructing a large dataset of forward-facing objects, defined as orientations consistent with typical human perception. \nThe authors train two 3D generative models that, given one or more renderings of an object, predict its corresponding forward-facing orientation. \nAdditionally, they propose and train two downstream tasks that explicitly involve object orientation.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Dataset Creation: Construct a large-scale dataset of forward-facing objects, where each object's orientation aligns with human perception.\n2. Model Generalization: Demonstrate that fine-tuning a 3D generative model on this dataset enables generalization across diverse object categories.\n3. Qualitative Results: Present high-quality visualizations of correctly oriented objects across diverse categories, showcasing the effectiveness of the approach on Objaverse samples.\n\nWeaknesses\n1. Limited Realism: Most examples appear toy-like or low-resolution. It remains unclear whether the proposed approach generalizes well to realistic, lifelike 3D objects.\n2. Marginal Novelty: Aside from the construction of a forward-facing 3D object dataset, the work offers limited novelty. \n3. Insertion Quality: The insertion results often resemble a simple copy-paste of the forward-aligned object into a scene, lacking evidence of sophisticated integration or context-aware placement. So I can see no special thing done here.\n\nQUESTIONS:\n1. The paper states that each 3D object is rendered from four horizontal viewpoints—front, back, left, and right. \nHowever, many objects in Objaverse exhibit misalignment or arbitrary canonical orientation, meaning these standard views may not accurately capture the true forward-facing direction. As a result, there's a risk that none of the rendered views actually correspond to the perceptual front of the object. \nFurthermore, while it's mentioned that Gemini successfully identifies the front view in ~50% of cases, this raises concerns about the remaining objects—does the method fail because no true front-facing view was included?\n\n2. Did you randomize the input render positions during fine-tuning?\n\nLIMITATIONS:\n1. Limited Realism: Most examples appear toy-like or low-resolution. It remains unclear whether the proposed approach generalizes well to realistic, lifelike 3D objects.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposed a novel task on top of image-to-3D generation, i.e. orientation-aligned 3D object generation through data curation and image-to-3D model finetuning (e.g. Wonder3D and Trellis). The proposed method benefits several downstreaming tasks such as pose estimation from a single image and arrow-based object insertion. Extensive experimental results demonstrate the effectiveness of the proposed framework.\n\nSTRENGTHS AND WEAKNESSES:\n** Strengths**\n1. The methodology and data curation are well-detailed and easy to follow. \n2. The experiments are thorough and comprehensive, including both qualitative and quantitative results.\n3. The paper implies that an orientation-aligned image-to-3D object generation model benefits single-image pose estimation, which is an interesting insight.\n\n** Weaknesses**\n1. The motivation behind the proposed method is not entirely convincing, as 3D orientation does not appear to be a problem in 3D object generation as object pose is typically tied to camera pose during rendering and can be easily adjusted in tools such as Blender. That said, object orientation is arguably more objective than subjective, as it often depends on the inherent functionality of the objects themselves. For example, rendering results of both Wonder3D and Wonder3D-OA are good to me as they are simply rendered from different viewpoints relative to the objects (I know the rendered camera views are actually fixed though).\n2. As for object insertion, placing generated objects into a scene layout, such as an indoor room, is typically straightforward with mesh representations. Moreover, relighting the objects along with the scene as a whole is also relatively easy. In contrast, arrow-based object insertion does not seem to support this capability, which may result in inconsistencies in the appearance of inserted objects under novel lighting conditions compared to the original scene layout. Additionally, the global illumination appears to change significantly before and after object insertion in the supplementary video, why is that?\n\nQUESTIONS:\n1. How accurate is the proposed method's pose estimation in scenarios involving multiple objects (e.g. some examples shown in orient-anything paper)?\n2. In Table 1, why does the vanilla Wonder3D with the normal map branch have a higher Chamfer Distance than Wonder3D-OA? Is this due to the replacement of the 3D representation with LGM, or is it attributed to the orientation-aware design introduced in the paper?\n\nLIMITATIONS:\nNo potential negative societal impact can be found in this submission. My concern lies primarily in the technical aspects, and I question whether the problem setting itself is truly meaningful.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper first introduces new annotations for a subset of the Objaverse dataset, where objects are orientation-aligned to a pre-defined coordinate system.\n\nThen, two existing 3D generative models, a multi-view generative model, as well as a direct image-to-3D generative model, are fine-tuned using the proposed dataset to generate orientation-aligned 3D objects.\n\nThe authors propose estimating the orientation of 3D objects as a downstream application. Using fine-tuned 3D generative models makes an oriented CAD reference model unnecessary. To this end, they adapted and slightly modified a template-based method.\nFinally, the authors propose a practical downstream application in augmented/virtual reality scenarios, where object insertion can be steered more easily with the generated coordinate-frame aligned objects as compared to non-aligned objects.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\nThe paper tackles an important problem which is well motivated by the downstream applications shown in the paper, both in robotics and AR/VR applications.\nThe paper is generally well-written and easy to follow.\nThe paper introduces new orientation annotations for a subset of the Objaverse dataset, which will be made public and can thus be used in other projects as well.\n\n**Weaknesses:**\n\nThe authors state that wrongly annotated orientations by the VLM are manually corrected. As examples, stick-like objects and objects with ambiguous orientation or unclear front view are mentioned. \nWhile \"canonicalising\" the orientation of ambiguous objects might make sense as a data pre-processing step, it is dangerous to claim that these annotations are \"ground truth\" from now on.\nFor example, I am personally unable to tell the \"up\", \"front\" or \"right\" direction for a spoon.\n\nThe same problem also is relevant when looking at the comparison against prior work for orientation estimation in Tab.3. The authors argue that their method outperforms prior art for stick-like objects but again it is questionable whether a clear ground truth can be defined in such cases. Maybe, in these cases it would be better to measure whether the principal axes of the object are aligned with any of the coordinate frame axes.\n\nThe authors write that \"Experiments show that the pre-trained 3D generative models can further improve geometry quality after fine-tuning on our dataset\". (l.141)\nHowever, when inspecting the qualitative results in Fig.5, it rather looks like the geometric and visual appearance quality decreases for the generations of the fine-tuned model.\n\nThe paper is missing important details on how the experiments are evaluated. To be more specific, the LPIPS (and potentially CLIP) scores need to be computed in comparison against some ground-truth renderings. \nIf these renderings are e.g. at the six canonical views that were used when training Wonder3D-OA, the comparison in Tab.1 is unfair:\nAs the LGM (which is used to generate the 3D object) is conditioned on exactly these views for Wonder3D-OA but not for Wonder3D, the renderings at the evaluation viewing angles will include more artefacts caused by the LGM for Wonder3D compared to the OA version.\n\nThe presented downstream application (insertion of objects in 3D scenes) using a forward-facing arrow is not taking care of the additional degree of freedom (rotation around the forward facing arrow), which might be desired.\nThis is, however, only a minor weakness, as the alignment with the coordinate axes still makes manipulation easier than with tilted / rotated objects.\n\nThe paper should further include more details on the fine-tuning / introduced changes to prior methods.\nE.g., the pixel injector (l.196), not using depth input (l.220), or DINO pose selection stage (l.222) could be explained more clearly.\n\nQUESTIONS:\nPlease explain the evaluation procedure for computing LPIPS and CLIP scores. Which angles are used? What is the CLIP image embedding compared to? E.g., text prompts (\"front view\") or the CLIP embedding from the target shape rendering?\n\nThe baseline qualitative results look different for the same shape in Fig. 6 (see yellow bus for Trellis + VLM vs. Trellis + Orient Anything). Why? Shouldn't only their rotation change?\n\nIn the proposed DINO pose selection stage, measuring feature alignment with the L2 distance should likely be replaced with the cosine similarity. What exactly is compared here? Patch-level or CLS token?\n\nPlease feel free to further clarify any misunderstandings given my assessment of strengths and weaknesses of the paper.\n\nLIMITATIONS:\nThe paper should discuss the ambiguity of orientation annotations for certain object classes more clearly and how this affects evaluation and comparison with prior works.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles the challenge of inconsistent orientation in 3D generative models by introducing Objaverse-OA, a large-scale dataset of 14,832 orientation-aligned 3D models across 1,008 categories. Leveraging this dataset, the authors fine-tune two existing 3D generative models (Trellis and Wonder3D) to produce orientation-aligned outputs. The aligned models enable two compelling downstream applications: zero-shot object orientation estimation and efficient arrow-based object insertion.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The paper is well written.\n2. Making 3D Generative Models Orientation-Aligned is a nice and important task.\n3. I appreciate the significant amount of work presented here: constructing the data, tackling an important problem for the first time, providing nice downstream tasks, creating baselines and a user-friendly interface for object insertion, as well as great figures, supplementary material, etc.\n\nWeaknesses: \nI have a hard time deciding whether there are significant theoretical/algorithmic contributions. Essentially, the authors used the exact same method and pipeline from Orient Anything [1] to annotate a large-scale dataset (which is valuable but not novel, and this paper is not in the NeurIPS dataset submission track) and fine-tuned two generative models on the data. I do value the modifications the authors made to Wonder3D and the training ablation for Trellis, but it’s still a relatively small algorithmic contribution compared to the\nlevel expected at this conference. That said, the work is still significant.\n\n\n\n[1] Wang, Zehan and Zhang, Ziang and Pang, Tianyu and Du, Chao and Zhao,\nHengshuang and Zhao, Zhou, Orient Anything: Learning Robust Object Orientation\nEstimation from Rendering 3D Models, ICML 2025\n\n\n\n\nMinor typo:\nLine 265: double period “Tesla A100 GPUs..”\n\nQUESTIONS:\n1. I suggest that the authors include some evaluation for their dataset construction\npipeline with more metrics (like GEC/GC, ICC), as done in [2], [3], and [5], using a\ndataset with ground truth like ShapeNet [4] or similar datasets.\n2. A concurrent work that created the Canonical Objaverse Dataset [5] is worthy of\nmention if the paper is accepted, but you do not need to compare to it in this\nsubmission.\n\n[2] Rohith Agaram, Shaurya Dewan, Rahul Sajnani, Adrien Poulenard, Madhava Krishna,\nSrinath Sridhar, Canonical Fields: Self-Supervised Learning of Pose-Canonicalized Neural\nFields, CVPR 2023\n[3] Rahul Sajnani, Adrien Poulenard, Jivitesh Jain, Radhika Dua, Leonidas J. Guibas,\nSrinath Sridhar, ConDor: Self-Supervised Canonicalization of 3D Pose for Partial Shapes,\nCVPR 2022\n[4] Angel X. Chang et al., ShapeNet: An Information-Rich 3D Model Repository\n[5] Jin, Li and Wang, Yujie and Chen, Wenzheng and Dai, Qiyu and Gao, Qingzhe and\nQin, Xueying and Chen, Baoquan, One-shot 3D Object Canonicalization based on\nGeometric and Semantic Consistency, CVPR 2025\n\nLIMITATIONS:\nNot really. \nWhile Section 7's title is \"Conclusion and Limitation\", this section hardly touches limitations.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 2, "DATE": "2025-07-01"}