Orientation Matters: Making 3D Generative Models Orientation-Aligned
Authors: Yichong Lu
Yuzhuo Tian
Zijin Jiang
Yikun Zhao
Yuanbo Yang
Hao Ouyang
Haoji Hu
Huimin Yu
Yujun Shen
Yiyi Liao
Venue: NeurIPS 2025 poster ()
Keywords: computer vision
Primary area: applications
TL;DR: Make existing 3D generative models orientation aligned for orientation estimation and efficient insertion.

Abstract: Humans intuitively perceive object shape and orientation from a single image, guided by strong priors about canonical poses. However, existing 3D generative models often produce misaligned results due to inconsistent training data, limiting their usability in downstream tasks. To address this gap, we introduce the task of orientation-aligned 3D object generation: producing 3D objects from single images with consistent orientations across categories. To facilitate this, we construct Objaverse-OA, a dataset of 14,832 orientation-aligned 3D models spanning 1,008 categories. Leveraging Objaverse-OA, we fine-tune two representative 3D generative models based on multi-view diffusion and 3D variational autoencoder frameworks to produce aligned objects that generalize well to unseen objects across various categories. Experimental results demonstrate the superiority of our method over post-hoc alignment approaches. Furthermore, we showcase downstream applications enabled by our aligned object generation, including zero-shot object orientation estimation via analysis-by-synthesis and efficient arrow-based object rotation manipulation.

Orientation Matters: Making 3D Generative Models Orientation-Aligned
Yichong Lu
Yuzhuo Tian'"
Zijin Jiang" 1"
Yikun Zhao
Yuanbo Yang 1.2 Yiyi Liao
Hao Ouyang"
Haoji Hu²
Huimin Yu-
Yujun Shen
ZZeejaag University
Ant Group
http:///kiiminngggttuoooooiinonn
Abstract
Humans intuitively perceive object shape and orientation from single image, guided by strong priors about canonical poses. However, existing 3D generative models often produce misaligned results due to inconsistent training data, limiting their usability in downstream tasks. To address this gap, introduce the task orientation-aligned 3D object generation: producing 3D objects from single images with consistent orientations across categories. To facilitate this, struct Objaverse-Ou -OA, a dataset 14,832 orientation- aliigne 3D models spanning 1,008 categories. Leveraging Objaverse-OM we fine-tune two representative 3D generative models based on multi- view diffusion and 3D variational autoencoder frameworks to produce aligned objects that generalize well to unseen objects across various categories. Experimental results demonstrate the superiority of our method over post-hoc alignment approaches. Furthermore, plications enabled by our aligned object generation, including zero-shot object orientation estimation via analysis-by-synlle and efficient arrow-based object rotation manipulation.
showcase downstream ap-
Introduction
Humans possess a remarkable ability to imagine the 3D structure of an object ("synthesis") and infer its properties, such orientation ("analysis"). from single image. We intuitively recognize the correct orientation of a car on the road, know how to grasp a cup by its handle, or decide how to place chair in room. In cognitive science, this ability is linked to the concept of object constancy-the capacity to mentally reconstruct a canonicalized object despite changes in viewpoint other variations (2. [6]. This synthesis of internal object representation enables analysis of its pose, a process often described as analysis by synthesis.
Replicating this perceptual capability has long been central pursuit in computer vision. Recent progress in generative Al has significantly advanced single-view 3D object generation, making both accessible and effective. This advancement in "synthesis" offers rapid and adaptable means of constructing 3D models from everyday images, enabling a wide range of applications AR/VR content creation, robotic simulation, etc.
Despite these strides, a critical challenge remains: orientation alignment. Existing methods often neglect this aspect, largely due to inconsistencies in large-scale 3D datasets [6] [5]. As a result, generated models frequently lack standardized canonical orientations chairs may face different directions, mugs may appear tipped over, and vehicles may be misaligned. This disparity between human perceptual consistency and the output of generative models poses a significant obstacle for
Equal contribution. 'Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Objaverse-OA
Symbent
NorrCana Ofject Cano. Ofgest
Drientation
Input images
(Ours)
Input Images Trellis-OA (Ours)
3D Object Manipulation
Efficient Arrow-baeed
Figure 1: Objaverse-OA for Orientation-Aliggned Generation. We construct new dataset named Objaverse-OA, which contains orientation-aligan 3D models across 1008 categories (top). Using Objaverse-OA, we make existing 3D generative models prientation-aligned. which can further be used for zero-shot model-free orientation estimation (bottom left) and efficient arrow-based 3D object rotation manipulation (bottom right).
downstream tasks such as mulysis-by-synthese e.g., orientation estimation. To bridge this perceptual gap. it is essential that 3D generative models not only reconstruct object geometry but also generate objects in consistent, semantically meaningful orientations.
In this paper, we introduce a novel task: rrientation-alinned 3D object generation the generation of 3D object from single image such that its orientation is consistently aligned both within and across categories, in accordance with common-sense priors. While no existing methods explicitly address this task, a possible workaround involves first generating 3D object and then applying pose estimation to align its orientation. However, achieving robust and generalizable orientation alignment across diverse object categories remains significant challenge. Most 3D pose estimation approaches [54] [2] 40] focus on predicting relative poses with respect to predefined 3D CAD models or multi-view reference images. In contrast, our task demands absolute orientation prediction aligning objects to a canonical coordinate frame that corresponds to intuitive understandings of front, top, and upright directions. While some prior works explore absolute pose estimation [47 they are typically constrained to limited set of categories, primarily due to the substantial manual effort required to curate orientation-alii alligned training data. Recent efforts to scale such methods have relied either on labor-intensive human annotations [15, 24 or on the use of Vision-Language Models (VLMs) [52). However, the former approach remains category-restricted, while the latter subject to the inherent inaccuracies of VL-M-based orientation predictions. Moreover, these strategies necessitate complex post-pos pos--processig after 3D generation, which not only introduces potential inaccuracies but also reduces user-friion seer-frienlliness and practical applicability.
To address this, propose to learn canonical 3D object generation by directly fine-tuning pre-trained 3D generative models, thereby avoiding the limitations of such two-stage pipelines. Our key insight that sufficiently diverse set of orientation-aligaa generative models from producing arbitrarily oriented outputs to generating objects with consistent, canonical orientations, while maintaining strong generalization capabilities to unseen objects across various categories. To support this, we introduce Objaverse-OA. 3D models spanning 1,008 categories, each aligned to a consistent, common-sense orientation. Leveraging Objaverse-OA OA, we fine-tune two representative 3D generative models [57] 23 to produce
3D models can effectively adapt existing
new dataset comprising 14,832

---

Trellis-OA -elliis-AAee and Wonder3D-ON allowing for generating well-aligned 3D objects across a broad spectrum of categories, including those not included in the fine-tuning set.
We further demonstrate the utility of orientation-aligned 3D generative models through two down- stream applications: zero-shot 3D object orientation estimation and efficient arrow-based object rotation manipulation. For orientation estimation, our canonically generated 3D models serve as templates estimate object poses from single images, generalizing well across categories. Moreover, we develop a user-friendly interface for object rotation manipulation in the augmented reality applica- tions and 3D software, allowing users to specify the desired orientation via drawing an arrow, thereby facilitating precise placement without tedious pose adjustments.
Our contributions as follows: 1) We introduce the novel task of orientation- aliggned 3D object generation across a wide range of categories. 2) We construct Objaverse- the largest orientation- aligned 3D dataset in terms of category coverage. 3) We fine-tune existing 3D generative models on Objaverse-OA to enable canonical object generation with robust generalization to unseen objects across various categories. Experimental results across multiple datasets demonstrate that our method achieves superior orientation alignment compared to existing baselines. 4) We showcase the practical benefits of our orientation-aligned models in two key applications: zero-shot orientation estimation and efficient object rotation manipulation via intuitive user interaction.
2 Related Work
3D Generative Models: Early 3D generation methods 555
36 1
28 typically employed
GANs [46] to model 3D distribution, while these methods generate orientation-aligner objects, they are limited to a single category. The recent breakthroughs in 2D diffusion models
provide propose to generate 3D models by distilling from a 2D text-to-image generation model. However, these methods [1] always suffer from low efficiency and multi-face problems due to per-shape optimization and lack of explicit 3D supervision. Recently, methods based on have succeeded in efficiently producing multi-view consistent images via 3D attention. More recently. [25] 664 62 [5] utilize 3D latent spaces to further improve the geometry quality of the 2D-assisted approaches. However, despite huge progress on quality and efficiency, they all produce 3D models with uncanonical orientations due to orientation misalignment in their 3D training data, like Objaverse (6) Objaverse-XL (7). and TRELLIS-5000 [57].
new solutions for 3D generation. Pioneering works DreamFusion [33 and SJC 48
and their follow-ups 118 19
42
multi-view diffusion models 23 21
49 37
41
38
44 60
Object Orientation Estimation: One feasible approach to align 3D model orientations is to render the 3D models from a fixed camera and estimate the object orientations in the renderings. Although image-based 3D object pose estimation has been widely researched, most methods [54] 29
40 17] 22
61 focus on predicting relative poses based on known 3D CAD models or reference images. Since the orientation-alignee aligned 3D CAD models and reference images are not available, they cannot estimate the object orientations aligned with common sense. Category-Level object pose estimation methods [47
9 45 address this problem by generating 3D shape priors from the input images. However, most of them are limited to category-Ievel due to heavy labor effort in constructing orientation-aligan 3D datasets 47 2 15 24. In contrast, our method can generate ceanntttio--alignnne 3D objects across a large number of categories. Concurrently, Orient Anything [52] realizes zero-shot object orientation estimation by automatically constructing large orientation-aligned 3D datasets using advanced Vision Language Models (VLMs). However, it still suffers from generalizability and accuracy due to a lack of training on the object orientation estimation task. Besides, post-processing after 3D) generation is costly compared to directly generating cecentationallinne 3D models. Concurrently, [13] proposes an intra-category object pose canonicalization method and constructs Canonical Objaverse Dataset, which contains 3D objects with canonical poses within categories. However, our work focuses on inter-category object pose canonicalization.
55 20
3 Objaverse-OA Dataset
In this section, we introduce the construction of our dataset, Objaverse-OA. Dataset diversity plays crucial role in achieving strong generalization capability. To the best of our knowledge, the existing orientation-aligen 3D dataset [24] with the largest category number includes only 200 categories
3

---

Categories with highest rates:
90
Spear (100%))
Categories
(70%-300%)
Orientation Ambiquuy:
Other Object water Taucal
10
60
fre exting.isher
seeing machine
crab fen
bunk bad
Error Ittte (5)
60
pooers structures
hermonium
seaharse
Encer Ratte -ighh
50
Categorass with Lorrect Result
30
eerning ship, batteub pajamas, dorphen, cantral jear, babson, baskethall poster, dragonfly, pickle...
20
10
100
500
700
900
Object Categories (Sorted by Wrong Rate)
Figure 2: VLM's Performance in Orientation Estimation. We utilize our manually curated dataset as ground truth (GT) and show the error rate of VLM's estimation across different categories. We observe that (1) the VLM demonstrates particular difficulty in recognizing front-facing orientations for stick-like objects, and (2) a significant portion of recognition errors occur when processing objects with inherently unclear ambiguous frontal views. These challenges highlight the necessity of our manual curation.
and fewer than 2,000 3D objects. In contrast, our Objaverse-Ou dataset contains 14,832 orientation- aligned 3D objects across 1008 categories, which will be made publicly available to the research community. To build this large-scale dataset while maintaining both efficiency and accuracy, we employ hybrid pipeline that combines Vision- Saiiun-L Model (VLM) pre-processing with manual correction.
VLM pre- pre-processing As discovered by Orient Anything [2 advanced VLMs demonstrate the ability recognize object front views without task-specific training. Since most models in Objaverse primarily vary in the horizontal (yaw) axis, we follow the strategy proposed in Orient Anything: render each 3D object from four horizontal viewpoints- ffront,t-frot,, back, left, and right- and use VLM to identify the correct front view. Based on the identified view, we then rotate the 3D model accordingly align it to canonical orientation. Our data processing begins with the Objaverse-LVIS dataset, and we use Gemini-2.0 32) as the VLM for view recognition. From a total of 46,219 3D) models, Gemini successfully identifies front views for 20,664 objects. However, we observe that VLM-based recognition, while promising, still falls short of human-level accuracy due to the absence of fine-tuning and challenges associated with ambiguous or difficult cases. We illustrate the error rate of VLM's prediction across different categories in Fig, 2 One of the challenging cases is stick-like objects, like spears, keys, and forks, since many of them are not aligned in roll and pitch angles in the Objaverse. Another challenge involves geometrically narrow or thin objects such as fish, bicycles, and water faucets. such cases, VLM struggles identify the correct orientation, as it relies solely on visible front-view features without reasoning from side-view context, unlike humans. What's 's more, objects with ambiguous front views, like teapots, extinguishers, and cups, can result in inconsistent orientation predictions. To address these issues, we introduce manual correction step described below.
Manual correction: Starting from the VLM-based alignment, we manually filter or correct the wrong recognition results of the VLM and canonicalize objects with the ambiguous front-view definition. Moreover, to preserve category diversity, we reintroduce objects that were incorrectly filtered out by the VLM, particularly in cases where a category has too few correctly aligned instances. As illustrated in the Figure we manually correct the orientations for about 600 object categories, especially the stick-like objects and objects with ambiguous orientation or unclear front view features. Note that some objects have orientation ambiguity, especially tools like spoons, mugs, and fire extinguishers. During the manual correction process, we refer the object orientations defined in prior work, specifically ImageNet3D 124), for the categories it covers. For example, for spoons,

---

Structured Latents Generator
Sparse Structure Generation
Structured Labents Generation and Decoding
Gatamioal
(A) Oriontation Alignment Framework
Heet Controler
W--Dffiasion
LGW
So Pleed
Tramuble LoKA
Mult-view Generation
Liting
(Bi Orientation Aligmment on Mulli-View Diffusion Framework
Figure 3: Trellis-OA and Wonder3D-OA. We fine-tune representative methods: Trellis [57), based on a 3D-VAE backbone (top), and Wonder3D [23), based on multi-view diffusion backbone (bottom). For the 3D-VAE, find that fine-tuning only the sparse structure generator sufficient to produce orientation-alignee objects. For the multi-view diffusion model, we adopt LoRA as lightweight domain adapter to enable the generation of ineentttion-alinen target images.
which are included in ImageNet3D, align their poses in dataset accordingly. For ambiguous objects only included in our dataset, we define canonical poses based on semantic part structures, geometric features, and common knowledge, following the principles established by ImageNet3I and our supplementary material Section What's more, we also filter objects with low geometry quality and scenes with multiple objects. Experiments show that the pre-trained 3D generative models can further improve geometry quality after fine-tuning on our dataset.
4 Orientation-Aligned 3D Object Generation
Based on our curated Objaverse-OA (Section 3). we fine-tune existing 3D generative models to generate orientation-aligned objects. To demonstrate that orientation-aligned object generation benefits a variety of architectural backbones, we implement our approach on two widely used single- view image-to-3D reconstruction frameworks: a 3D VAE-based generative model (see Section and multi-view diffusion model (see Section
4.1 3D- VAE Based Generative Model
We choose statc-of-the-art 3D- ID-VAA-based generative model, Trellis [57]. as our base model, which produce fine-grained geometry and appearance aligned with the input image.
Preliminary: As shown in the Figure Trellis [57 adopts three modules for 3D asset generation
during inference including sparse structure generator Gs. structured latents generator GL, and 3D decoder D. Given the input image I, sparse structure generator G3 produces dense binary 3D grid O (0, 1INNNNNX XNXN, O Gs(I, E3a), where N is the length of the grid and is the noise sampled from (O,,,), which further converted into active voxels (P defined as sparse structure. After that, sparse latents generator Gs is used to generate structured latents z= ((x, p & (I,I, #noised), where noised is the noised sparse structure {(E,, P.) Finally, the 3D representation M is obtained via 3D decoding: M ((z).
Trellis-OAt OA: The original Trellis model [57] is unable produce orientation-alignet 3D outputs due to the orientation inconsistencies present in its training data. To address this, we fine-tune Trellis using our Objaverse-On OA dataset, resulting in Trellis-OA, which generates 3D objects with aligned orientations. Although Trellis comprises several modules for 3D generation, we find that fine-tuning only the sparse structure generator GS is sufficient for achieving orientation alignment. This likely because Trellis inherently generates object poses randomly sampled from four orthogonal directions, and our aligned pose distribution resides within this range. As a result, both the pre-trained structured latent generator GL and the 3D decoder D remain compatible with the aligned orientations and do
5

---

not require additional fine-tuning, Specifically, our fine-tuned sparse structure generator Ss generates canonical sparse structure {(p)))== which shares aligned orientations. Afterwards, pre-trained structured latents generator 91. generates canonical structured latents and 3D decoder D produces final canonical 3D models Mo with orientation aligned. Our experiments demonstrate that we can efficiently fine-tune Trellis to an orientation-aligned one while preserving its 3D priors.
4.2 Multi-view Difflusio Model
We choose Wonder3I 23 as our base multi-view diffusion model since it is one of the most representative works based on multi-view diffusion frameworks. Note that the recipe for Wonder3I) can also be used by other multi-view diffusion based methods.
Preliminary: Multi-view diffusion models are typically fine-tuned from large-scale pre-trained text-to-image models (35) and achieve multi-view consistent via 3D-aware attention mechanisms. Specifically, given an input image the multi-view diffusion model MV generates N multi-view images {Tm TM = MV (I, e), where & E EN 1) is sampled noise vector. Here, the poses of the ground truth of the multi-view images (T N i=1 are dependent on the input image I, where m. is set to predict the input image I. After that, Wonder3D adopts optimization method based on NeuS [50] to lift ] to 3D representation M.
[3 cannot produce orientation-aligned 3D models due to its misaligned training data and input- related camera settings. To address this, we implement Wonder3D-Or leveraging orientation aliiggd Objaverse- OA dataset. Given an input image I, Wonder3D-Ok first generates orientation orientation-aligned, multi-view consistent images I which are then lifted to orientation- aligned 3D representations Mos- Our key change is in the supervision setup: instead of using unpu-viewwdepeeden ground truth, we render six canonical views (front, front-left, front-right, left, right, back) from fixed camera poses based on Objaverse-OA This provides consistent orientation references. We adopt LoRA 12] as a lightweight domain adapter to fine-tune the pre-trained Wonder3D, preserving its learned 3D priors from non-canonical data while enabling alignment. Additionally, we make architectural adjustments for improved performance. Originally, Wonder3D aligns the input image with one of the predicted views to inject local features, which fails work in our canonical camera setting. Inspired by IrageDream [4,, we instead employ a pixel injector to integrate local features into the multi-view diffusion model. Specifically, Wonder3D employs a 3D dense self- attention mechanism with a shape of (b,, 6, c, f4. wy) across six views within transformer layer, where b. is the batch size, c the number of feature channel, fú and wy are the image resolution. Our pixel controller modifies this to (b., 7, c. h1, wt). incorporating the input image an additional view. We further improve efficiency by replacing Wonder3D's s test-time optimization-ban module (NeuS) with LGM (44), a recent model tailored for sparse-view 3D reconstruction. Since LGM is trained using 4-views as input, we feed the front, left, right, and back views into LGM for 3D lifting. Finally, as our lifting module performs effectively without the need for normal maps, we omit fine-tuning the cross-domain attention module in Wonder3D, simplifying the pipeline without sacrificing quality.
Wonder3D-OA: Wonder3D
3D lifting
Downstream Applications
To further demonstrate why making the 3D generative models orientation aligned is important, we implement two downstream applications, including zero-shot model-free object orientation estimation Section 5.1 and efficient arrow-based object rotation manipulation Section 5.2)
5.1 Zero-shot Model-free Object Orientation Estimation
One popular class of solutions to 3D object orientation estimation is based on analysis-by- synthesis 47 3 10] but most of them are limited to category-level due to the difficulty in synthesizing orientation-aliee aligned 3D models across categories. Another line of methods [29|5]] estimates poses relative to given CAD model, limiting practical use since a corresponding model must be available for each input image. While one could replace CAD models with generated 3D shapes (e.g.. from Trellis), the misaligned outputs of existing generative models result in unreliable pose references. In contrast, our method generates riientation-alinne 3D shapes directly from a single image, en- abling the prediction of absolute poses in a canonical frame by treating the generated 3D shape as

---

Pose Netieemnnt
DIO
Ioput tmage
Refinement
Retrel fore
Selection
Figure 4: Zero-Shot Orientation Estimation, Our orientation-aligned 3D object acts as a template for pose estimation by rendering it from multiple views, refining each, and selecting the best-matching viewpoint. Note that we do not perform training for this downstream task, where the pose refinement module is directly from FoundationPose (54). and the pose selection module directly utilizes the pre-trained DINO feature extractor [27].
a per-object template. Specifically, we build on a state-of-the-art template-based pose estimation method, FoundationPose (54), which has strong generalizability due to training on a large synthetic dataset. renders templates from a fixed set of viewpoints, refines each pose, and then selects the best match. Although FoundationPose is trained with accurate CAD models and depth maps, find that its pose refinement module remains effective even with our generated 3D shapes and without depth input. However, its pose selection performance degrades when templates differ slightly in geometry or appearance. To address this, we retain the pose refinement module but replace the pose selection stage with DINOv2 (27) ( feature-based similarity metric, shown in Figure 4 This achieved by computing the L.2 distance between DINOv2 patch feature maps of each refined rendering and the target image, and selecting the view with the highest similarity. Note that when estimating orientations of objects in scene-level images, we need to extract objects' masks via the image segmentation method [14 before 3D generation.
5.2 Efficient Arrow-based Object Rotation Manipulation
Efficient manipulation of 3D model rotation within simulation systems is crucial, yet challenging. especially when models are initialized non-canonical poses. This difficulty arises because users typically aim orient model toward desired direction, but conventional 3D simulation systems only record the model's 's pose relative to its initial state. As a result, users must manually compensate for any misalignment in the initial pose, complicating the interaction process. In contrast, our generated orientation-alle alligned 3D models enable more direct and seer-friendly rotation manipulation approach due to their aligned initial orientations. With this alignment, users simply draw an arrow indicating the desired forward facing direction, without needing to consider the model's original pose. This arrow-based interaction paradigm enhances usability and compatible with both augmented reality (AR) applications and general-purpose 3D software. AR applications, users draw an arrow in the 2D image plane, which is subsequently lifted into 3D space using monocular depth estimation techniques (e.g., [31]). The system then rotates the model so that its forward-facing axis aligns with the specified arrow, while ensuring the object remains grounded on the background plane. Note that because our models are normalized in scale, their size must still be specified, either through large language model (LLM) direct user input. In general 3D software, users draw the arrow directly in 3D space. The system then applies Rodrigues' rotation formula to align the model's orientation with the sser-specified direction. Additional implementation details are provided in the supplementary materials Section A.3
Input Images
Wonder50
Wonder10-OA [Dun]
GT
hppuliage Images
Monder10
Wonder30 04 (0us)
Figure 5: Qualitative Results on multi view diffusion backbone, Wonder3D. For each input image, we show two views with the same index from the multi-view predictions.
7

---

GSO [
Toys4k 30
CD
LPIPS..
CLIP
CD1
LPIPS.
CLIP
Wonder3D Wonder3D + PCA
0.0894
0.2799
76.37
0.0932
0.2859
87.10
0.0788
0.2554
77.80
0.0858
0.2691
87.58
Wonder3D + VLM (Gemini-2.0) 32
0.0850
0.2752
76.30
0.0880
0.2804
87.53
Wonder3D + Orient Anything (VIT-L.)
52
0.1015
0.2600
77.50
0.1079
0.2699
88.12
Wonder3D-OA (ours w/o LGM)
0.0609
0.2300
80.22
0.0571
0.2351
91.33
Wonder3D-OA (ours)
0.0564
0.2270
80.30
0.0548
0.2317
92.09
Table 1: Quantitative Comparison of geometry and appearance Multi-view Diffusion back-
bone 23
We highlight the best, second-best and third-best scores achieved on any metrics.
6 Experiment
6.1 Implementation Details
Dataset. frientation-alignen 3D generative models Trellis-OA and Wonder3D-OA are trained on our Objaverse-OA dataset, which is curated from Objaverse-LVIS 6 The base multi-view diffusion model is trained on Objaverse [6], and the base 3D- -AE-baedd model is trained on TRELLIS- 500K [57]. To demonstrate the generalizability and accuracy of our method's orientation alignment ability, we evaluate on two unseen datasets, GSO g and Toys4k [39]. To further demonstrate the
sim-to-real generalizability, we also evaluate on the real-world dataset Imagenet3D (24).
Baselines. For the task of aligned object generation, there are no existing baselines for this task. Therefore, we design baselines that perform this task in two stages: 1) object generation with misaligned orientations, and 2) orient them to aligned poses based on pose estimation using different variants: (i) Principal Component Analysis (PCA); (ii) advanced Vision Language Model (VLM) Gemini-2.0 [32;; and (iii) zero-shot model-free orientation estimation method, Orient Anything For the task of pero-shot orientation estimation, we compare our method with Orient Anything and FSDet View [58]. Note that PSDetView doesn't 't support zero-shot estimation. Therefore, we evaluate its performance only on its supported categories.
52
Metrics. To evaluate the orientation alignment ability, we rotate reconstructed 3D models using different kinds of methods and calculate Chamfer Distance (CD), LPIPS (63) and CLIP 34 scores to measure the orientation aligmment quality. To evaluate the performance of our zero-shot orientation estimation method, we calculate @cc@30 and orientation absolute error (Abs) according to the
rotation error. We follow NOCS to calculate the rotation en defined by: en = arecos Tr(B.R' )-1 where Tr represents the trace of the matrix. Note that for stick-like objects, top and side directions typically have ambiguity. Therefore, only calculate the rotation error in the front direction.
Training and inference time. To fine-tune Trellis-AA, 30000 steps, which takes only about 10 hours on the cluster of 8 Nvidia Tesla A 100 GPUs. To fine-tune Wonder3D-OA. we use total batch size of 512 for training 40000 steps, which takes about 3 days on the cluster of 8 Nvidia Tesla A100 GPUs.. For the 3D object generation, Trellis- OA takes 32.93s and Wonder3D-Or takes 6.69s for multi-view generation, 2.53s for fused 3DGS generation, and 45.37s for mesh extraction. Given the generated 3D models, our pose refinement module takes 6.54s and the pose selection module takes 7.47s, which are comparable with the SOTA template-baser pose estimation method FoundationPose
use a total batch size of 64 for training
154
6.2 Orientation-Aligned Object Generation
Comparison with baselines: We first evaluate the performance of orientation-aligned object generation. As shown in the aad Table our fine-tuned models, Wonder3D-on and Trellis- OA, surpass most two-stage baselines in geometry and appearance on both GSO and Toys4k datasets. Note that our method is suboptimal in appearance on the GSO dataset compared to the VLM baseline. probably because the GSO dataset has more objects with irregular geometry and appearance, which is challenging for our fine-tuned generator to handle. We present more qualitative results in the Figure 5 and Figure 6 shown in Figure 6 PCA cannot handle objects with different shape features and has difficulty in distinguishing the direction of estimated principal axes. VLM
8

---

GSO 8
Toys4k 39
CD
LPIPS
CLIP
CD1
LPIPS..
CLIP
Trellis
0.0770
0.2502
82.88
0.0951
0.2722
92.31
Trellis PCA
0.0798
0.2771
78.84
0.0911
0.2758
88.09
Trellis + VLM (Gemini-2.0)
32
0.0421
0.1998
89.97
0.0564
0.2137
95.19
Trellis + Orient Anything (VTT-L)
0.0604
0,2193
88.61
0.0725
0.2313
94.81
Trellis-OA (ours small)
0.0448
0.2224
82.46
0.0465
0.2055
93.74
Trellis-OA (ours)
0.0407
0.2118
88.41
0.0393
0.1932
95.71
Table Quantitative Comparison of geometry and appearance on 3D- VAE backbone [57]
Tepat Eeagle
Tosli
THIls FCA
Trafllo VIM
Tell: Oiles feytling
TreftoOk (oun))
Figure 6: Qualitative Results based on the 3D-VAE backbone, Trellis. For each input image, we render the reconstructed object from consistent views. Note that our method consistently generates objects in their canonical space.
has trouble recognizing objects with unclear front-view features, like dolphins and boats. Orient Anything still not that accurate, and typically rotates objects into leaning poses. In contrast, our method accurately aligns the orientation of the generated 3D) models without obvious geometry and appearance degradation. Besides, our method can handle real world internet images and multi-objects images as shown in Figure 7 and Figure g Furthermore, due to the high geometry quality of our Objaverse-OA, TrellisOAA eliminates the production of plane-like geometries. Please refer to the supplementary materials Section B for more qualitative results.
Ablation Study: To assess the importance of using orientation-alignel dataset with diverse categories, we conduct an ablation study by fine-tuning 3D generative models on a reduced set of 100 categories with 5720 objects, (ours small) in Table and Table The results show that reducing the number of training categories degrades both geometric and visual quality, highlighting the value of large category diversity in our Objaverse-OA OA dataset.
Figure Qualitative Results of Trellis-OA real world internet images.
Figure 8: Qualitative Results of Trellis- OA on multi-objects images.
6.3 Zero-Shot Object Orientation Estimation
Comparison with baselines: As shown in Table without specific training, our method is already comparable with the SOTA orientation estimation method [52) with the ViT-large architecture and surpasses (52) with the ViT-small architecture by large margin. Besides, our method can handle challenging stick-like objects collected from the real-world dataset while baselines fail to work, which further demonstrates utility in coping with long-tailed situations.

---

Tays4k
Stick-like Obj. from ImigeNet3D 24
Acc@l 301
Absl
Acc 307
Abs,
Viee
(Few-shot)
20.90
84.25
Orient Anything 52
(Vit-)
42.05
52.72
2.63
81.70
Orient Anything 32
(Vi-L)
63.18
36.37
9.8
78.19
Ours (Vit-S)
51.15
53,94
60.78
40.12
Ours (VIT-L)
52.87
46.76
62.25
34,20
Table 3: Quantitative Comparison of zero-shot orientation estimation Toys4k 39 and stick-like
objects from ImageNet3D 24
Ordent Angthing (VIT-argg)
Figure 9: Orientation Estimation Comparison on tike objects from ImageNet3D 244
Figure 10: Orientation Estimation Results of our method on Toys4k [39].
6.4 Efficient Arrow-based Object Rotation Manipulation
Bil
Figure 11: Qualitative results our efficient arrow-based object rotation manipulation method in augmented reality applications.
Figure and Figure 11 illustrate arrow-based object rotation manipulation method in augmented reality applications. Given an image of an object, we generate the corresponding object in its canonical pose using Trellis-AA, and then manipulate it in the reference image with the target orientation indicated by a user-specified arrow. This allows for efficient object manipulation without adjusting the object's orientation in post-hoe fashion. We present qualitative results of method in the general 3D software in the video of supplementary materials, which demonstrate the usage of our approach.
7 Conclusion and Limitation
In this paper, we aim to align the orientations of the 3D generative models for downstream orientation estimation and efficient object rotation manipulation in 3D simulation systems. Towards this goal, we construct Objaverse-OA, a dataset covering orientation-aligned 3D models across the largest number of categories. Based on Objaverse- OA, we align the orientations of 3D generative models based on two popular 3D generation frameworks, including multi-view diffusion and 3D-VAE. Built upon the conntation-alignned 3D generative models, we develop simple but effective orientation estimation approach following the analysis-by-synther manipulation method. While our method achieves promising zero-shot orientation estimation without task-specific training, future work could further improve performance by training dedicated model that leverages the generated 3D objects as templates.
paradigm and an efficient arrow-based object
Acknowledgement
This work is supported by NSFC under grant 62441223 and 62202418, and partially by Ant Group
10

---

References
[1] Eric Chan, Connor Z. Lin, Matthew Chan, Koki Nagano, Boxiao Pan. Shalini De Mello, Orazio Gallo, Leonidas Guibas, Jonathan Tremblay, Khamis, Tero Karras, and Gordon Weizstein. Efficient geometry- aware 3d generative adversarial networks. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16102-16112, 2021.
[2] Angel X. Chang, Thomas A. Funkhouser, Leonidas J. Guibas, Pat Hanrahan, Qi-Xing Huang. Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, I.. Yi, and Fisher Yu. Shapenet: An information-rich 3d model repository. ArXiv, abs/1512.00012 2015.
[3] Xu Chen, Zijian Dong, Jie Song, Andreas Geiger, and Otmar Hilliges. Category level object pose estimation via neural muallyis-by-syntheee In European Conference on Computer Vision, 2020.
[4] Xinya Chen, Hanlei Guo, Yanrui Bin, Shangzhan Zhang, Yuanbo Yang. Yue Wang, Yujun Shen, Yiyi Liao. Learning 3d-aware gans from unposed images with template feature field. In European Conference on Computer Vision, 2024.
[5] Florinel-Alin Croitoru, Vlad Hondru, Radu Tudor Ionescu, and Mubarak Shah. Diffusion models in vision: A survey, IEEE Transactions on Puttern Analysis and Machine Intelligence, 45:10850-10869, 2022.
[6] Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. 2023 IEFF/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 13142-13153, 2022.
[7] Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram S. Voleti, Samir Yitzhak Gadre, Eli VanderBilt, Aniruddha Kembhavi, Carl Vondrick, Georgia Gkioxari, Kiana Ehsani, Ludwig Schmidt, and Ali Farhadi. Objaverse-xl: A universe of 10m+ 3d objects. ArXiv, abs/2307 abs/2307.05663, 2023.
[8] Laura Downs, Anthony Francis, Nate Koenig, Brandon Kinman, Ryan Michael Hickman, Krista Reymann, Thomas Barlow McHugh, and Vincent Vanhoucke. Google scanned objects: A high-quality dataset of 3d scanned houschold items. 2022 International Conference Robotics and Automation (ICRA), pages 2553,2560,220 2022.
[9] Zhaoxin Fan, Zhenbo Song, Jian Xu, Zhicheng Wang, Kejian Wu, Hongyan I Liu, and Jun He. Object level depth reconstruction for category level 6d object pose estimation from monocular rgb image. In European Conference on Computer Vision, 2022.
[10] Jiaxin Guo, Fangxun Zhong, Rong Xiong, Yunhui Liu, Yue Wang, and Yiyi Liao. A visual navigation perspective for category- level object pose estimation. In European Conference Computer Vision, 2022.
[11] Jonathan Ho, Ajay Jain, and P. Abbeel. Denoising diffusion probabilistic models. ArXiv, abs/2006. 11 1239, 2020.
[12] J. Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. ArXiv, 0ttn/200609968. 2021.
[13] Li Jin, Yujie Wang, Wenzheng Chen, Qiyu Dai, Qingzhe Gao. Xueying Qin. Baoquan Chen. One-shot 3d object canonicalization based on geometric and semantic consistency, 2025 IEEF/CV Conference on Computer Vision and Pattern Recognition (CVPR), pages 16850-16859, 2025.
[14] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloé Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Yen Lo, Piotr Dollár, and Ross B. Girshick. Segment anything. 2023 IEEF/CVF International Conference on Computer Vision (ICCV), pages 3992-4003, 2023.
[15] Akshay Krishnan, Abhijit Kundu, Kevis- Maninis, James Hays, and Matthew Brown. Omninoes: unified noes datasct and model for 3d lifting of 2d objects. ArXiv, abs/2407.086 00700700771111111 2024.
[16] Rebecca Lawson. Achieving visual object constancy across plane rotation and depth rotation, Acta psychologica, 102 2-3:221-45, 1999.
[17] Taeyeop Lee. Bowen Wen, Minjun Kang, Gyuree Kang, In-So Kweon, and Kuk-Jin Yoon. Any6d: Model-free 6d pose estimation of novel objects. ArXiv, abs/22503.1867. 2025.
[18] Yixun Liang, Xin Yang. Jiantao Lin, Haodong Li, Xiaogang Xu, and Yingcong Chen. Luciddreamer: Towards
ext-to-3d generation via interval score matching. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6517-6526, 2023.

---

[19] Chen-Hsuan Lin, Jun Gao, Luming Tang. Towaki Takikawa, Xiaohui Zeng. Xun Huang, Karsten Kreis. Sanja Fidler, Ming-Yi Yu Liu, and Tsung-Yi Yi Lin. Magic3d: High-resolution text-to-3d content creation. 2023 IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 300-309, 2022.
[20] Jiehong Lin, Zewei Wei, Changxing Ding, and Kui Jia. Category-leve 6d object pose and size estimation using supervid deep prior deformation networks. In European Conference on Computer Vision, 2022.
[21] Yuan Liu, Chu-Hsing Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang. Synedreamer: Generating multiview--cosisttrn images from single-view image. ArXiv, abs/2309.03453, 2023.
[22] Yiho Liu, Zhaodong Jiang. Binbin Xu, Guile Wu, Yuan Ren, Tongtong Can, Binghing Liu, Rui Heng Yang, Amir Rasouli, and Jinjun Shan. Hippo: Harnessing image-to-3e 3d priors for model-free zero-shot 6d pose estimation, ArXiv, abs/2502. 10606, 2025.
[23] Xiaoxiao Long, Yuanchen Guo, Cheng Lin, Yuan Liu, Zhiyang Dou, Lingjie Liu, Yuexin Ma, Song-Hai Hai Zhang, Mare Habermann, Christian Theobalt, and Wenping Wang. Wonder3d: Single image to 3d using cross-domain diffusion. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9970-9980, 2023.
[24] Wufei Ma, Guanning Zeng, Guofeng Zhang, Qihao Liu, Letian Zhang, Adam Kortylewski, Yaoyao Liu, and Alan L. Yuille. Imagenet3d: Towards general- purpose object-level 3d understanding. ArXiv, abs/2406.096 00914,,,,996,, 2024.
[25] Quan Meng, Lei Li. Matthias Nießner, and Angela Dai. Lt3sd: Latent trees for 3d scene diffusion. ArXiv, 056224 2024.
[26] Scott O. Murray, Daniel J. Kersten, Bruno A. Olshausen, Paul Schrater. and David L. Woods. Shape perception reduces activity in human primary visual cortex. Proceedings of the National Academy of Sciences of the United States of America, 99:15164- 15169, 2002.
[27] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Q Vo, Mare Szafranice, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaacidin El-Nouby, Mahmoud Assran, Nicolas Ballas, Wojeiech Galuba, Russ Howes, Yao (Bernie) Huang, Shang- Wen Li, Ishan Misra, Michael G. Rabbat, Vasu Sharma, Gabriel Synnaeve, Huijiao Xu, Hervé Jégou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. Dinov2: Learning robust visual features without supervision. ArXiv, ahs/2304.07199, 2023.
[28] Dario Pavllo, David Joseph Tan, Marie-Julic Rakotosaona, and Federico Tombari. Shape, pose, and appearance from single image via bootstrapped radiance field inversion. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages
2022.
[29] Sida Peng. Yuan Liu, Qi-Xing Huang, Hujun Bao, and Xiaowei Zhou. Pvnet: Pixel-wise voting network for 6dof pose estimation. 2019 JEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR). pages 4556-4565, 2018.
[30] Pakkapon Phongthawee, Worameth Chinchuthakun, Nontaphat Sinsunthithet, Amit Raj. Varun Jampani, Pramook Khungur,, and Supasorn Suwajanakom. Diffusionlight: Light probes for free by painting a chrome ball. 2024 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 98-108, 2023.
[31] Luigi Piccinelli, Yung-Hsu Yang, Christos Sakaridis, Matiia Segu, Siyuan Li, Luc van Gool, and Fisher Yu. Unidepth: Universal monocular metric depth estimation. 2024 IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10106-10116, 2024.
[32] Sundar Pichai, Demis Hassabis, and Koray Kavukcuoglu. Gemini-2.0. https:/hooggooggeelccchnooggg
google-deepmind/ggoggg.. wewp://gggggoggggiiiiioeeeee gemini- www-uudddt-ddemmeerr 2024 Accessed: 2025-05-14.
[33] Ben Poole. Ajay Jain, Jonathan T. Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. ArXiv, abs/2209. 0tbs/2099.4988. 2022.
[34] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, pages 8748 8763. PMLR, 2021.
12

---

[35] Robin Rombach, A. Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resoltion image synthesis with latent diffussin models. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10674-10685, 2021.
[36] Katja Schwarz, Yiyi Liao, Michael Niemeyer, and Andreas Geiger. Graf: Generative radiance fields for 3d-aware image synthesis. ArXiv, abs/2007.02442, 2020.
[37] Ruoxi Shi, Hansheng Chen, Zhuoyang Zhang, Minghua Liu, Chao Xu, Xinyue Wei, Linghao Chen, Chong Zeng, and Hao Su. 23++; a single image consistent multi-view diffusion base model. ArXiv, abs/2310.15110, 2023.
[38] Yichun Shi, Peng Wang, Jianglong Ye, Mai Long, Kejie Li, and X. Yang. Mvdream: Multi-view diffusion for 3d generation. ArXív, abs/2308. 2023.
[39] Stefan Stojanov, Anh Thai, and James M. Rehg. Using shape to categorize: Low-shot learning with an explicit shape bias. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1798 1808, 2021.
[40] Jiaming Sun, Zihao Wang. Siyu Zhang, Xingyi He He, Hongcheng Zhao, Guofeng Zhang, and Xiaowei Zhou. Onepose: One-shot object pose estimation without cad models. 2022 IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 6815-6824, 2022.
[41] Stanislaw Szymanowicz, C. Rupprecht, and Andrea Vedaldi, Viewset diffusion: (O-jimage-conditionell 3d generative models from 2d data. 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 8829 88399, 2023.
[42] Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. Dreamgaussian: Generative gaussian splatting for efficient 3d content creation. ArXiv, abs/2309. 1...099.666.. 6653, 2023.
[43] Junshu Tang, Tengfei Wang, Bo Zhang, Ting Zhang, Ran Yi, Lizhuang Ma, and Dong Chen. Make-it- 3d: High-fidelity 3d creation from single image with diffusion prior. 2023 JEEE/CVF International Conference on Computer Vision (ICCV), pages 22762-22772, 2023.
[44] Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu. Lgm: Large multi-view gaussian model for high-rcsolution 3d content creation. In European Conference on Computer Vision, 2024.
[45] Meng Tian, Marcelo H. Ang. and Gim Hee Lee. Shape prior deformation for categorical 6d object pose and size estimation. ArXiv, abs/2007 088454, 2020.
[46] Individualized Treat and Jinsung Yoon. Generative adversarial nets. 2018
[47] He Wang, Srinath Sridhar, Jingwei Huang, Julien P. C. Valentin, Shuran Song, and Leonidas J. Guibas. Normalized object coordinate space for category-level 6d object pose and size estimation. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2637-2646, 2019.
[48] Haochen Wang, Xiaodan Du, Jiahao Li, Raymond A. Ych, and Gregory Shakhnarovich. Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation. 2023 JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12699, 2022.
[49] Peng Wang and Yichun Shi. Imagedream: Image-prompt prompt multi-view diffusion for 3d generation. ArXiv. abs/2312.02201, 2023.
[50] Peng Wang. Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. ArXiv, abs/2106.1066 2021.
[51] Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distillation. ArXiv, abs/2305.162133 2023.
[52] Zehan Wang, Ziang Zhang, Tianyu Pang, Chao Du, Hengshuang Zhao, and Zhou Zhao. Orient anything: Learning robust object orientation estimation from rendering 3d models. ArXiv, abs/2412. 18605, 2024.
[53] Jiaxin Wei, Xibin Song, Weizhe Liu, Laurent Kneip, Hongdong Li, and Pan Ji. Rgb-based category-level object pose estimation via decoupled metrie scale recovery. 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 2036-2042, 2023.
[54] Bowen Wen, Wei Yang. Jan Kautz, and Stanley T. Birchfield. Foundationpose: Unified 6d pose estimation and tracking of novel objects. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 17868 17879, 2023.
13

---

[55] Jiajun Wu, Chengkai Zhang, Tianfan Xue, Bill Freeman, and Joshua B. Tenenbaum. Learning a probabilistic latent space of object shapes via 3d generative-adversariall modeling. In Neural Information Processing Systems, 2016.
[56] Tianhao Wu, Chuanxia Zheng. Frank Guan, Andrea Vedaldi, and Tat-Jen Cham. Amodal3r: Amodal 3d reconstruction from occluded 2d images. ArXiv, abs/2503. 2025.
[57] Jianfeng Xiang, Zelong Lv, Sicheng Xu, Yu Deng. Ruicheng Wang. Bowen Zhang, Dong Chen, Xin Tong, and Jiaolong Yang. Structured 3d latents for scalable and versatile generation. ArXiv, abs/2412.01506, 2024.
[58] Yang Xiao, Vincent Lepetit, and Renaud Marlet. Few-shot object detection and viewpoint estimation for objects in the wild. IEEE Transactions on Puttern Analyxis and Machine Intelligence, 45:3090-3106, 2020.
[59] Yinghao Xu, Hao Tan, Fujun Luan, Sai Bi, Peng Wang, Jiahao Zifan Shi, Kalyan Sunkavalli, Gor- don Wetzstein, Zexiang Xu, and Kai Zhang. Dmv3d: Denoising multi-view diffusion using 3d large reconstruction model. ArXiv, abs/2311 12//2311.092111 1.09217, 2023.
[60] Yinghao Xu, Zifan Shi, Wang Yifan, Hansheng Chen, Ceyuan Yang, Sida Peng, Yujun Shen, and Gordon Wetzstein. Grm: Large gaussian reconstruction model for efficient 3d reconstruction and generation. In European Conference Computer Vision, 2024.
[61] Yuanhong Yu, Xingyi He He, Chen Zhao, Junhao Yu. Jiaqi Yang. Ruizhen Hu, Yujun Shen, Xing Zhu, Xiaowei Zhou, and Sida Peng. Boxdreamer: Dreaming box corners for generalizable object pose estimation. 2025.
[62] Longwen Zhang, Ziyu Wang. Qixuan Zhang. Qiwei Qiu, Anqi Pang, Haoran Jiang. Wei Yang. Lan Xu, and Jingyi Yu. Clay: controllable large-scale generative model creating high-quality 3d assets. ACM Transactions on Graphics (TOG), 43:1 20, 2024.
[63] Richard Zhang, Phillip Isola, Alexei Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 586-595, 2018.
[64]
Zibo Zhao, Zeqiang Lai, Qin Lin, Yunfci Zhao, Haolin Liu, Shuhui Yang, Yifei Feng, Mingxin Yang. Sheng Zhang, Xianghui Yang, Huiwen Shi, Si- Ya Liu, Junta Wu, Yihang Lian, Fan Yang, Ruining Tang, Ze-Bao He, Xinzhou Wang, Jian Liu, Xuhui Zuo, Zhuo Chen, Biwen Lei, Haohan Weng. Jing Xu, Yi Zhu, Xinhai Liu, Lixin Xu, Chang-Ping Hu, Tianyu Huang, Lifu Wang. Jihong Zhang, Mengya Chen, Liang Dong. yong Jia, Yu-Xin Cai, Jiaao Yu, Jun Tang. Hao Zhang, Zhengfeng Ye, Peng He, Runzhou Wu, Chao Zhang, Yonghao Tan, Jie Xiao, Yang-Dan Tao, Jian-Xi Zhu, Ji Xue, Kai Liu, Chongqing Zhao, Xinming Wu, Zhi wei Hu, Lei Qin, Jian Yong Peng, Zhan Li, Minghui Chen, Xipeng Zhang. Lin Niu, Paige Wang. Yingkai Wang. Hao Kuang, Zhongyi Fan, Xu Zheng, Weihao Zhuang, Yin- Yin He, Tian-Hai Liu. Yong Yang. Wang, Yuhong Liu, Jie Jiang. Jingwei Huang, and Chunchao Guo. Hunyuan3d 2.0: Scaling diffusion models for high resolution textured assets generation, ArXiv, abs/2501. 2202, 2025.
14

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No]. or [NA].
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providedd proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
"[No] or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We claim in the abstract and Section 1
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations in Section 7
15

---

Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification:
We don't have thcoretical result.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We present our experimental results in Section 6
Guidelines:
The answer NA means that the paper does not include experiments.
16

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. w
For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification:
We don't provide code in the submission.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (httpss//nips.cc/pubicc quiddes/oodeSubmisioooooidi
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https) hhips.cc/public/guideeee Stt.auulee///eeSSSimiooooiiilili for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
17

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification:
We detail experimental setting in Section 6
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification:
Our experiment doesn't involve error bars.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide information in Section
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
18

---

The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http:///eenpp..ccuuuuiiihhcceeee
Answer: [Yes]
Justification: The research conducted in the paper conforms, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss in the appendix in the supplementary materials.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has socictal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [No]
Justification: Our paper poses such risks.
Guidelines:
19

---

The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We list the Licenses in the appendix in the supplementary materials.
Guidelines:
The answer NA means that the paper does
existing assets. The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms of the package should be provided. For popular datasets, paperswithcode. htt.://wwwwww.o//eeee/ com/dotaset. has curated licenses some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- e-pakaaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Our paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
20

---

Justification:
Our paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [Yes]
Justification:
We declare the usage of Gemini in Section
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy hftng:///eeeiiisscccccmmeeeeeees for what should or should not be described.
21

---

Appendix
In this appendix, we detail dataset curation, experiment implementations, and method implementa- tions Section[A] Besides, we provide more results our efficient arrow-based object manipulation, orientation-aligned object generation, orientation estimation, and failure cases in Section B
A Implementation Details
A.1 Dataset Curation
Data Processing Pipeline. We first apply Vision Language Model (VLM) recognition across the entire dataset, followed by manual identification and correction of failure cases. Specifically, four images were rendered for cach 3D object using an orthogonal camera setup. The front view was identified using the VLM (Gemini 2.0 (32)), and the objects were subsequently rotated to their canonical poses based on the recognition results. Note that objects recognized as without front-view orientation automatically excluded from the dataset. After this initial step, the orientations of the processed objects were manually refined using Blender software.
VLM Pre-processing. Different from Orient Anything, we further enhance the text prompt by adding more recognition rules for different kinds of objects, as shown in Figure 13 We find this operation can further improve the robustness of VLM via in-context learning.
Stick-like Objects
Unclear Front-Viee Feature
Ambiguous Front-View Definition
Figure 12: Failure cases in VLM-based front-view recognition. The four images are rendered from four orthogonal cameras following our VLM-based recognition method.
Manual Corrrection Details. Despite advancements, VLMs still face challenges in accurately recognizing objects' orientations due lack of training. Three particular object categories prone to errors, as shown in Figure 12: (1) stick-like objects (e.g.. forks, spoons), (2) objects lacking distinctive front-view features (c.g., seahorses, water faucets), and (3) objects with inherently ambiguous front-view definitions (e.g., mugs, fire extinguishers). VLMs struggle with stick-like objects since they are not aligned roll and pitch angles. For objects without clear front-view features, distinguishing the front from the back often requires side-view inference, which hasn't 't been achieved by existing VLMs. Ambiguity in defining canonical front view further complicates recognition for certain object types. To address these issues, we implemented the following corrections: stick-liks like objects were aligned parallel to the global X-axis, positioning the handle toward the negative X-axis and the functional end toward the positive X-axis. Objects without distinct front-view cues were manually rotated in Blender. For objects with ambiguous front-view definitions, we aligned the component typically associated with human interaction (c.g., handles) to the negative X-axis.
22

---

I'm going to show four images of the same object from four viewpoints in turn and label them 'A.' 'B.' 'C' 'D' Four options. The option 'E.' is No front face More than One front Face". Decide whether it have front and if yes, which front of the object after the presentation. Note that: If the object is a gun, bow and arrow, etc., please use the muzzle of the gun as the front. Please note the following points: Bicycle: Front refers to the direction of the front, back refers to the direction of the rear, and side refers to both sides of the body. \Sofa: Front refers to the side with a backrest, back refers to the side without backrest, and side refers to both ends of the sofa. \Pisoll: Front refers to the direction of the muzzle, back refers the direction of the grip, and side refers both sides of the gun body. VAnimals: with the front pointing towards the head, the back pointing towards the tail, and the side pointing towards both sides of the body. Person: Front refers to the face facing, back refers to the back facing, and side refers to the sides of the body. Car: Front refers to the direction of the front of the car, back refers to the direction of the rear of the car, and side refers to both sides of the car bodies. Watch: The front refers to the face of the dial, the back refers to the back of the watch, and the sides refer to both sides of the case. Teapot: The front refers the direction of the spout, the back refers to the direction of the handle, and the side refers to both sides of the pot body; Cups:
is the
front refers to the view when the handle is heading left; Other items: Please define the front, back, and side according the structure and characteristics of the item. Key features: For each object, please consider its unique structure and features, such as the wheels of a bicycle, the backrest of a sofa, the muzzle of a handgun, the head of an animal, the face of person, the headlights of a car, the dial of a watch, the spout of a teapot, etc. These features are usually the key to determining the orientation of an object. The reasoning process: you can simulate human thinking processes when determining the orientation of an object, such as first finding the most obvious feature on the object, then determining the orientation of that feature, and finally determining the overall orientation of the object. For some objects that difficult to determine their orientation, multiple features be combined for comprehensive judgment. Stick tools and weapons such as swords, axes, knives, and wrenches are considered to have no front. If you cannot decide or there is more than one front, you should choose 'E' If there are multiple possible choice, choose the one that has a higher confidence level of showing front view.
Figure 13: Text prompt for the Vision Language Model.
VLM Error Analysis After obtaining the orientation-alige aligned dataset, we conducted a comprehensive error analysis by comparing the VLM's predictions with our manually validated ground truth. For each 3D model, we computed the Chamfer Distance (CD) between the 3D models after VLM-bbssed recognition and our manually corrected 3D models. Each model was uniformly sampled with 10,000 surface points to facilitate CD computation. Since CD values are influenced by object geometry and do not perfectly reflect orientation accuracy, we adopted a threshold- -baed approach to quantify recognition accuracy. Through empirical analysis, we found that rotating model by 90" around its principal axis typically resulted in a CD > 0.01. This threshold, 7 = 0.01, was therefore used to flag recognition errors: any model with CD exceeding this value between VLM and ground-truth orientations was considered wrongly recognized. Notably, this threshold may produce wrong results for perfectly cylindrical cubical shapes due to their inherent symmetry. However, such cases represent a negligible minority in our dataset and were therefore excluded from our error statistics to maintain analytical integritt.
A.2 Experiment Implementations
Evaluation Data. We evaluate our method on three unseen datasets: GSO g Toys4k [39], and Imagenet3D (24). For the GSO dataset, we randomly collected 48 objects with clearly defined front views. Each object was rendered into four images, sampled from the upper hemisphere with azimuth angles uniformly distributed within [0?, 360"], polar angles within [0 60%). and rotation angles within [-30-, 30"]. For the Toys4k dataset, we randomly selected 439 objects across 47 categories. Each object was rendered into single image using the same camera configuration as applied to
23

---

Reasoning: The object is chair. The front of the chair the side with the backrest, in image A, the chair's backrestis facing the viewer directly. Therefore, the front view the chair.
Heasoning: object duck. front
Reasoning: object appears computer features of mouser the scroll wheel and buttons, Image A most likely the back of the mouse. images and show the side view. Thus, C the front of the mouse.
prominent
Figure 14: Correct VLM recognition results.
/
Reasoning:
The object appears be spoon. The front of spoon the side the bowl, so view is the best candidate tor view..
Reasoning: the object appears to faucet. I'd define "front" of faucet as side with both spout and the handleis) most clearly visible. However, Views and are essentially the same, showing spout handle, while single view. There are two face, which C. So choose
the
Therefore, there
refers the direction in the handle of the cup faces backwards. and C both showcase the handle, but the "eye' design better chttercholce observed on A than makinng A better choice as the tront.
object cup.
definition,
Figure 15: Wrong VLM recognition results.
24

---

the GSO dataset. For the Imagenet3D dataset, we focused on seven stick-like object categories, comprising total of 204 objects. Please refer Table and Table for details.
airplane
bicycle
boat
bunny
bus
car
cat
chair
9
10
9
9
5
20
chicken
cow
crab
deer moose
dinosaur
dog
dolphin
dragon
8
6
10
10
10
10
10
elephant
fish
fox
frog
giraffe
guitar
helmet
helicopter
10
10
9
10
10
10
8
horse
laptop
lion
lizard
monkey
motorcycle
mouse
panda
10
6
10
10
10
10
7
PC mouse
penguin
piano pig
radio robot
shark
sheep
3
16
10
10
8
shoe
sofa
tractor
train
truck
violin
whale
10
20
16
4
10
10
Table 4: Object numbers of each category in Toys4k
39
evaluation data.
fork
knife
pen
rifle
scissors
screwdriver
spoon
30
31
33
29
38
Table 5: Object numbers of each category in Imagenet3D
2
evaluation data.
Baselines. To implement the PCA-based baseline, we first calculate the three principal axes by digondeccoppoiing the 3D models' geometries. The object is then rotated to align these principal axes with the global coordinate system's X-, y-, and z-axes. For the VLM-based baseline, follow the same method used in the dataset curation, utilizing Gemini-2.0 32 for orientation recognition. To implement the baseline based on Orient Anything [52], we use the official checkpoint based on the ViT-large architecture and adopt its data augmentation module. The object orientation is estimated from an image rendered using a fixed camera, and the object is subsequently rotated according to predicted orientation. For the pre-trained 3D generative models based on multi-view diffusion and 3D-VAE, we utilize the official Wonder3D++ and Trellis checkpoints. For the orientation estimation baseline based on FSDetView (58), we restrict evaluation to the object categories supported by the method.
Metrics. For the calculation of Chamfer Distance, we don't perform PCA for both predicted 3D models and GT models (except the PCA baseline), in order to measure the pose canonicalization performance. As for the calculation of CLIP and LPIPS, for the 3D-VA backbone, we render four orthogonal views with the camera elevation angle of 0 for both the generated and GT 3D models, and computed the LPIPS and CLIP scores based on these renderings the Multi-view Diffusion backbone, we randomly sample views on a unit sphere for evaluation to avoid unfairness, since the orthogonal views are aligned with the camera setting of our Wonder3D-OA, not aligned with the Wonder3D baselines.
matched camera poses, while for
A.3 Method Implementations
Efficient Arrow-based Object Rotation Manipulation. For the arrow-based object rotation manipulation in the augmented reality application, we define this arrow-based interaction using a start point P start and 2D end point P Pends forming a direction vector Vtarget = Pend P stsartt Then we use Unidepth 31 to estimate the camera intrinsic and depth map. With camera intrinsic, calculate the 3D camera rays Retart and Rend for the corresponding 2D points P start and
25

---

Figure 16: More results of our efficient arrow-based object manipulation method.
Pend- With the depth map, we calculate the plane P via the least square method. After that, we calculate the 3D intersection points Pard and Pidd Pitart between the corresponding camera rays Ratart, Rend and the extracted plane P. Finally, 3D direction vector D) target can be calculated: start. Since each generated model is aligned known forward vector V init we can directly compute and apply the rotation that aligns Vodd imit with targett allowing the object be placed in the correct orientation without any manual adjustment. Note that the 3D start point PSd start is also the 3D location of the inserted 3D object, the 3D models need rotate along Sd target to ensure verticality to the ground and the object size still needs to be set via VLM or user interaction, since our generated models share normalized scale. Specifically, to automatically estimate the size of the object to insert, we can directly ask the VLM to estimate the common size of the object in the real world with the single-view image as input, since our method estimates metric depth with the scale aligned with the real world for insertion. Besides, to make the insertion results realistic, we set a plane as a shadow catcher and estimate the environment lighting using DiffusionLight (3) For the arrow-bhe aroowdbased object rotation manipulation in the generic 3D software, we define this arrow-based
3dd Päd 3d pdd
Ptrrett
interaction using a 3D start point P 3d stort and 3D end point Päd exd. forming 3D direction vector end start- The rotation transformation is calculated by aligning y inst with 3d target using Rodrigues' formula and users can further rotate the objects along 3ddd target needed.
3d target
P
3d Pidd
B More Results
B.1 Efficient Arrow-based Object Manipulation
We present more results of our efficient arrow-based object manipulation method in the Figure 16
26

---

Figure 17: More qualitative results of our Trellis-OA and orientation estimation on Toys4k 339 Note that the images correspond to the input image, two renderings of the generated 3D model, and the orientation estimation results, respectively.
Chamfer Distance L
LPIPS
CLIP 1
Trellis (Manually corrected)
0.0377
0.1723
88.33
Trellis-OA (Ours)
0.0280
0.1574
88.77
Table 6: Quantitative comparison of geometry and appearance quality between Trellis with manually corrected orientations and Trellis-OA (Ours). The results demonstrate that our orientation-aligee aligmment fine-tuning can enhance the performance the pre-trained checkpoint, likely attributable to the high quality of our Objaverse-OA dataset.
B.2
Orientation-aligned Object Generation
We present more qualitative results of our Trellis-OA and orientation estimation method in Figure I7 Figure 18 Figure [9] Figure 20) and Figure 21 We also present more qualitative comparisons between WD and our Wonder3D-OA in Figure 22 and Figure 23 Besides, we present more results of Trellis-OA and Wonder3D-OA in demo video. Please check the video for details.
To investigate whether our orientation alignment fine-tuning affects the performance the original pre-trained checkpoint, randomly select 46 objects across 46 categories in the Toys4k (39] dataset. As shown in Table 6, our method not only maintains performance but may even enhance the quality of the pre-traine ireitraieed 3D generative models. This improvement is likely due to the high fidelity of our manually corrected Objaverse-OA dataset.
B.3 Orientation Estimation
We detail orientation estimation results of baselines and method at the category level in Table 7 and Table g
B.4 Impact of Occlusion
As illustrated in Figure 24 our object generation module is capable of producing reasonable results under partial occlusion, with minimal performance degradation. However, in scenarios involving severe occlusion, recommend using the method proposed in [56), which is specifically designed
27

---

Figure 18: More qualitative results of our Trellis-OA and orientation estimation on Toys4k (39
28

---

Figure 19: More qualitative results of our Trellis-OA and orientation estimation on Toys4k (39
29

---

Figure 20: More qualitative results of our Trellis-OA and orientation estimation on Toys4k (39
30

---

sirplane
bicycle
bunny
bus
FSDerView 58
-/-
40.00/ 58.52
11.11/ 114,89
-/-
20.00/75.71
40.00 58.07
Orient Anything
(Vit-S)
30.00/ 70.48
20.00/ 63.91
44.44 / 73.69
33.33 47.79
100.00 11.64
80.00 29.49
Orient Anything +
(Vi-1.)
80.00 39.03
40.00 774.3
44.44 87.91
88.89/
15.63
80.00 / 20.32
80.00 18.73
Ours (VI-S)
66.67 55.99
80.00 44,288
55.56/ 66.21
44.44 50.77
100.00 / 99.52
60.00 74.66
(VTTL)
77.78 /.3496
18.76 chair
58.47
/5.000
100.00
chicken
cow
crab
doer moose
Viee 58
+/-
19.00/96.30
-/-
-/-
-7-
Orient Anything
(Vit-S)
55.56 35.79
55 05.00//5 //399.55
25.00/ 59:75
33.33/ 61.73
10.00 /5.044
50.00 89.89
Orient Anything
52
(Vit-L)
77.78 19.83
90.00/ 15.53
50.00 34.62
50.00
40.90
50.00/44..89
100,00/ / 17.54
(ViT-S)
22.22
/3.00//5 59.091
/3..32
33.33/
30.04
Ours (VIT-I)
33,33 65.48
40.00 49.99
75.00/ 28.66
33.33/ 63.09
60.00 /6.77
60.00 34.98
dinosaur
dog
dolphia
dragon
clophant
fish
VVDetView 58
-/-
-/-
-/-
-/-
Orient Anything
(Vit-S)
50.00 /3.332
22. 53.95
30.00/ 86.84
20.00/
66.955 35
50.00 98
20.00 59.59
Orient Anything 52 (Vi-L)
70.00 /22.11
16.71
30.00/ 50.37
50.00
48.06
/ 28.76
40,00/ /4.336
Oars (VIT-)
10.00 54.34
55.56 47.04
70.00/ 34.27
10.00/ 76.13
50,00 16.19
20.00 82.77
Ours (VI-L)
40.00 38.01
44.44 /4.888
60.00 34.52
20.00 59.41
70.00 35.56
20.00 89.96
fox
giraffe
guitar
helicopter
helmet
FSDetView 5
-/-
-/-
-/-
0.00 110.48
-/-
28.57
Orient Anything
77.78 /.1.87
0.00 51.62
/6..0//7..3
101.22
14.29 53.71
Orient Anything 5
(Vit-L)
66.67 22.15
90.00/21 /21.53 53
80.00/ 24.69
50.00 35.85
50,00/744
14,29 54.06
Ours (VI--S)
55.56 /47.99
50.00 /3.22
50.00/ 32.03
40.00 39.72
25.00 111.61
57.14 70.63
Ours (VT-L)
44,44 39.19
60.00 66..88
60.00/ / 32.85
30.00 64.50
25,00/
66.30
42,86 62.73
homse
laptop
lion
lizard
monkey
molorcycle
FSDerView 5
62.50 42.29
-/-
Orient Anything 52 (Vi--)
20.00 73.80
75.00/ 19.83
50.00 / 36.90
10.00 56.94
90.00/ 19.91
25.00/ 101.14
Orient Anything (Vit-L)
80.00 36.
50.00
770009
70.00 28.20
40.00 38.71
70.00/21..
50.00/51.77
Ours (VIT-S)
70.00 0..05/3..55 31.55
50.00 72.41
30.00/ / /55778
50.00 45.07
60.00 40,70
50.00 72.01
Ours (VIT-I)
60.00 / 74
33.00/33..11 32.11
40.00 / 33.39
50.00 36.63
60.00
/49.30
62.50/ 68.29
mouse
panda
peaguin
piano
pig
Viee 58
10.00 80.55
-7-
-/-
-7-
33.33/ 102.13
-7-
Orient Anything
(Vit-S)
20.00 /43.97
57.14/30.07
0.00/ 116.94
50.00
10.00/49.00 49.04
50.00 / 64.67
30.00 50.62
Orient Anything 52
(Vi-L)
40.00 29.71
71.43/ 14.63
0.00 89.58
56.25 33,90
50,00 / 611228
80.00 30.68
Ours (V/T-S)
20.00 73.00
57.14 //9
/3.33/69.44
56.25 / 58.99 98
50.00 / 68.89
20.00 /2.001
Ours (VIT-L)
40.00/51.85
57.14/55.52
0.00 74.86
31,25/57.58
50.00 56.13
40.00
radio
robot
shark
sheep
shoe
sola
FSDerView
/9.00
15:00/ 107.20
Orient Anything g
(Vit-S)
50.00 66.47
62.50/63.46
20.00/ 50.26
75:00/
75.00/34.54
34.54
50,00 62.30
75,00 33.12
Orient Anything 9
(V
50.00 //.666
87. /3.44/
40.00 45.30
100.00
16.06 16.06
50.00 32.59
65.00 39.84
Ours (VIT-S)
50.00 26.10
37.50 48.90
60.00
/4641
75.00/
23.73
60.00 / /3.92
85.00
Ours (VTT-L)
0.00/94.70
37.50/ 75.79
70.00 24.69
100.00/ / 18.32
80.00
/4.55
85.00 33.35
tractor
truck
violin
whale
FSDetView 58
-/-
0.00
/93.15
-/-
-/-
-/-
Orient Anything
(Vit-S)
75.00 32.67
0.00 / 117.58
80.00/
/3.95
0.00 0900/95.34
44.44 /7.662
Orient Anything 5 (Vit-)
75.00 /
50.00 70.34
100.00 11.94
30.00/ 45.55 89
44.44 /3.144
Ours (VIT-S)
87,50 31.52
25.00/119.20
/0.00/23..00
50.00 69.62
44.44 57.50
Ours (V/T-L)
68.75 34.80
50.00 /76.18
100.00 14.03
30.00/65..0 65.17
44.44 :4.880
Table 7: Category level quantitative results of orientation estimation Toys4k of Acc@ 30 1 and Abs L.
33
dataset in terms
31

---

Figure 21: More qualitative results of our Trellis-OA and orientation estimation on Imagenet3D 24 dataset.
fork
knife
pen
rifle
scissors
FSDetView 58
9.09 99.66
4.76 / 81.40
0.00/81.72
54.84 40.24
0.00 / 93.31
Orient Anything
22
(Vit-S)
4.55 83.23
0.00 / 78.44
6.67 74.79
3.23 / 87.56
3.03 74.11
Orient Anything (it-L)
9.09 78.51
0.00 78.95
13.33 / 76.79
6.45 81.73
3.03 85.85
Ours (VIT-S)
54.55/37..0
37.50 56.61
50.00 59.02
61.29 33.89
48.48 /47.15
Ours (ViT-I.)
54.55 34.98
54.17 29.82
56.67 52.50
54.84 27.90
57.55
screwdriver
spoon
FSDetView 38
9.00/
0.00 103.08
Orient Anything 22
(Vit-S)
1.00/90.50
2.63 76.98
Orient Anything
52
(Vit-)
17.24 70.88
15.79 68.49
Ours (VIT-S)
79.31 /21.9
78.95 / 26.04
Ours (VI--L)
68.97 18 76.32 / 30.10
Table 8: Category level quantitative results of orientation estimation stick-like objects from Imagenet3D 24 dataset in terms of Acc@ 30 f and Abs 4.
for occlusion-robuseee robustness,s,s as the pre-trained checkpoint prior to applying our crientation-alignmenn fine-tuning. It is also important note that our orientation estimation approach has not been designed to handle occluded inputs. Further research is necessary to extend its applicability to occlusion-prom scenarios.
B.5
Failure Cases
Our method may fail in certain scenarios, as illustrated in Figure 25] For instance, when the input image captured from rear of an object, the synthesized 3D model may exhibit suboptimal results in the front view (first row). Additionally, when applied to unseen categories, the generated 3D model may incorrectly incorporate features from other categories present in the training dataset (second row).
32

---

Viewpoint
Viewpoint
(a) Wonder3D
Viewpoint
Viewpoint
(b) Wonder3D-OA
Figure 22: More qualitative comparison between (a) Wonder3D 23 and (b) our Wonder3D-OA.
33

---

Viewpoint
Viewpoint
(a) Wonder3D
Viewpoint
Viewpoint
(b) Wonder3D-OA
Figure 23: More qualitative comparison between (a) Wonder3D 23 and (b) our Wonder3D-OA.
34

---

Input Images
Generated 3D Models
Input Images
Generated 3D Models
Figure 24: Impact of occlusion
Input Images
Generated 3D Models
Figure 25:
Failure cases.
C License and Border Impact
The licensing details are provided in Table Our method is suitable for generating orientation-aligned 3D objects for downstream applications in 3D perception and augmented reality, which may benefit both scientific research and commercial development. However, is important to acknowledge potential risks: the method could be misused to generate hazardous 3D objects, such weapons, which may lead to societal concerns.
35

---

Assets
License
URI.
Blender 4.2.8
GNU General Public License (GPL)
texthtpss/wwww.heeddrrogg
Objaverse
ODC-By v1.0 license
cextttpps///bjuuvrree..llleeoeoo org/
Trellis
MIT License
Wonder3D
MIT License:
PoundationPose
NVIDIA Source Code License
LGM
44
MIT License
Table 9; License
36


Reviews:
[{"review_id": "AmadBzNAgO", "reviewer": "Reviewer_yWsi", "summary": "This paper tackles the challenge of inconsistent orientation in 3D generative models by introducing Objaverse-OA, a large-scale dataset of 14,832 orientation-aligned 3D models across 1,008 categories. Leveraging this dataset, the authors fine-tune two existing 3D generative models (Trellis and Wonder3D) to produce orientation-aligned outputs. The aligned models enable two compelling downstream applications: zero-shot object orientation estimation and efficient arrow-based object insertion.", "questions": "1. I suggest that the authors include some evaluation for their dataset construction\npipeline with more metrics (like GEC/GC, ICC), as done in [2], [3], and [5], using a\ndataset with ground truth like ShapeNet [4] or similar datasets.\n2. A concurrent work that created the Canonical Objaverse Dataset [5] is worthy of\nmention if the paper is accepted, but you do not need to compare to it in this\nsubmission.\n\n[2] Rohith Agaram, Shaurya Dewan, Rahul Sajnani, Adrien Poulenard, Madhava Krishna,\nSrinath Sridhar, Canonical Fields: Self-Supervised Learning of Pose-Canonicalized Neural\nFields, CVPR 2023\n[3] Rahul Sajnani, Adrien Poulenard, Jivitesh Jain, Radhika Dua, Leonidas J. Guibas,\nSrinath Sridhar, ConDor: Self-Supervised Canonicalization of 3D Pose for Partial Shapes,\nCVPR 2022\n[4] Angel X. Chang et al., ShapeNet: An Information-Rich 3D Model Repository\n[5] Jin, Li and Wang, Yujie and Chen, Wenzheng and Dai, Qiyu and Gao, Qingzhe and\nQin, Xueying and Chen, Baoquan, One-shot 3D Object Canonicalization based on\nGeometric and Semantic Consistency, CVPR 2025", "limitations": "Not really. \nWhile Section 7's title is \"Conclusion and Limitation\", this section hardly touches limitations.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The paper is well written.\n2. Making 3D Generative Models Orientation-Aligned is a nice and important task.\n3. I appreciate the significant amount of work presented here: constructing the data, tackling an important problem for the first time, providing nice downstream tasks, creating baselines and a user-friendly interface for object insertion, as well as great figures, supplementary material, etc.\n\nWeaknesses: \nI have a hard time deciding whether there are significant theoretical/algorithmic contributions. Essentially, the authors used the exact same method and pipeline from Orient Anything [1] to annotate a large-scale dataset (which is valuable but not novel, and this paper is not in the NeurIPS dataset submission track) and fine-tuned two generative models on the data. I do value the modifications the authors made to Wonder3D and the training ablation for Trellis, but it’s still a relatively small algorithmic contribution compared to the\nlevel expected at this conference. That said, the work is still significant.\n\n\n\n[1] Wang, Zehan and Zhang, Ziang and Pang, Tianyu and Du, Chao and Zhao,\nHengshuang and Zhao, Zhou, Orient Anything: Learning Robust Object Orientation\nEstimation from Rendering 3D Models, ICML 2025\n\n\n\n\nMinor typo:\nLine 265: double period “Tesla A100 GPUs..”", "was_revised": true, "final_justification": "The rebuttal has largely addressed my concerns. While I'm still not 100% concinced about the level of novelty, I think this paper still deserves to be accepted.", "rebuttal": "Thank you, Reviewer yWsi, for recognizing the significance of our work. Here we aim to address the concerns raised in the review.\n\n**W1. Limited theoretical/algorithmic contributions.**\n\nThe main contributions of our work are that we are the first to identify the important but overlooked task, orientation-aligned 3D generation, via in-depth research on relevant works in the field of 3D generation. We demonstrate that the most effective solution is creating a new dataset and directly fine-tuning existing 3D generative models rather than commonly used post-processing methods, including PCA, Vision Language Model (VLM), and Orient Anything [1]. We also prove the importance of orientation-aligned 3D generation by firstly applying it to vital downstream applications. \n\nBesides, we also make several other innovations compared to relevant works. For the dataset curation, while the initial stage of our data curation pipeline is inspired by Orient Anything [1], we investigate and address the limitations of the VLM through manual inspection and correction, significantly improving dataset quality. For the orientation estimation application, we are the first to utilize an orientation-aligned 3D generative model for zero-shot model-free orientation estimation. This approach enhances the orientation awareness of SOTA 6D object pose estimation methods like Any6D [2], which use orientation-misaligned generative models for template creation. Our method also exhibits superior generalizability compared to SOTA orientation estimators like Orient Anything [1]. This is because we leverage the comprehensive 3D priors from powerful 3D generative models trained on larger orientation-misaligned datasets, in addition to the rich number of categories in our own dataset. For the object insertion application, we discover that 3D orientation alignment improves the efficiency of object orientation manipulation in simulation systems like the augmented reality application.\n\n[1] Wang, Zehan et al. \"Orient Anything: Learning Robust Object Orientation Estimation from Rendering 3D Models.\" ICML 2025.\n\n[2] Lee, Taeyeop et al. \"Any6D: Model-free 6D Pose Estimation of Novel Objects.\" CVPR 2025.\n\n**W2. Minor typo.**\n\nThank you for pointing out our minor typo in Line 265. We will fix it in our final version.\n\n**Q1. More evaluation for their dataset construction pipeline.**\n\nThank you for mentioning metrics utilized in other category-level pose canonicalization methods [3,4,5], including Instance-Level Consistency (IC), Category-Level Consistency (CC), Ground Truth Consistency (GC), and Ground Truth Equivariance Consistency (GEC). In fact, the evaluation illustrated in Figure 2 of our main paper is similar to calculating the IC metric used in [3,4,5] on datasets with ground truth.\n\nSpecifically, our dataset curation pipeline comprises two stages: VLM recognition and subsequent manual correction. Consequently, the final curated dataset, Objaverse-OA, can be considered a dataset with manually annotated ground truth. In Figure 2, we evaluate the consistency between the VLM recognition results and our manually corrected Objaverse-OA results. Owing to its extensive number of categories, our Objaverse-OA is more suitable than other datasets with ground truth, like ShapeNet [6], for evaluating VLM performance on long-tailed categories.\n\nWe now provide a detailed explanation of the minor difference between the IC metric and our evaluation metric. The IC metric calculates two-way Chamfer Distance between the canonicalized versions of the shapes (with superscript $^c$) before and after applying a set of random rotations $\\mathbf{R}$ for shapes in the dataset $\\mathcal{X}$:\n\n$$\n\\mathrm{IC}:=\\frac{1}{|\\mathcal{X}||\\mathbf{R}|}\\sum_{X_i\\in\\mathcal{X}}\\sum_{R_j\\in\\mathbf{R}}\\mathrm{CD}[(R_jX_i)^c,X_i^c]\n$$\n\nOur metric also calculates the Chamfer Difference in the same way, but further applies a threshold of 0.01 to determine the correctness of recognition, as elaborated in the supplementary material section A.1: \n\n$$\n\\mathrm{OurMetirc}:=\\frac{1}{|\\mathcal{X'}|}\\sum_{X_i\\in\\mathcal{X'}} Thre(\\mathrm{CD}[(R_iX_i)^c,X_i^c])\n$$\n\n$$\nThre(CD) = \\left\\\\{\n\\begin{aligned}\n1, \\quad & \\text{if } CD \\leq 0.01 \\\\\\\\\n0, \\quad & \\text{if } CD > 0.01\n\\end{aligned}\n\\right.\n$$\n\nAs can be seen, the main difference between our metric and the IC metric is that we calculate recognition accuracy by thresholding the Chamfer Distance used in the IC metric. This is because the Chamfer Distance can be influenced by the mean shape of a category, potentially leading to an unfair comparison of VLM performance across different categories. Besides, instead of applying a set of user-defined random rotations $R$, our metric applies the original rotation $R_i$ in the Objaverse-LVIS dataset for each object $X_i$, which can lead to better evaluation of VLM's ability on the real misaligned dataset.\n\nGiven the reviewer's suggestions, we looked into the other suggested metrics, CC, GC, and GEC. We believe the CC metric is not very relevant as it assesses the consistency of canonicalization across different instances within the same category, whereas our focus is on evaluating the VLM's cross-category canonicalization ability. The GC/GEC metric is also not essential since it measures canonicalization performance compared to manual labels, which is equivalent to our metric due to evaluation on our manually annotated dataset.\n\n[3] Agaram, Rohith et al. \"Canonical Fields: Self-Supervised Learning of Pose-Canonicalized Neural Fields.\" CVPR 2023.\n\n[4] Sajnani, Rahul et al. \"ConDor: Self-Supervised Canonicalization of 3D Pose for Partial Shapes.\" CVPR 2022.\n\n[5] Jin, Li et al. \"One-shot 3D Object Canonicalization based on Geometric and Semantic Consistency.\" CVPR 2025.\n\n[6] Chang, Angel X. et al. “ShapeNet: An Information-Rich 3D Model Repository.” ArXiv 2015.\n\n**Q2: Concurrent work.**\n\nThank you for mentioning the concurrent work that created the Canonical Objaverse Dataset [5]. We will cite this excellent work in our final version. The difference between our work and theirs lies in the alignment objective: our work aims to construct a cross-category aligned 3D dataset, whereas their focus is on creating an intra-category aligned 3D dataset."}, {"review_id": "xQH8JThSWm", "reviewer": "Reviewer_y6ZQ", "summary": "The paper first introduces new annotations for a subset of the Objaverse dataset, where objects are orientation-aligned to a pre-defined coordinate system.\n\nThen, two existing 3D generative models, a multi-view generative model, as well as a direct image-to-3D generative model, are fine-tuned using the proposed dataset to generate orientation-aligned 3D objects.\n\nThe authors propose estimating the orientation of 3D objects as a downstream application. Using fine-tuned 3D generative models makes an oriented CAD reference model unnecessary. To this end, they adapted and slightly modified a template-based method.\nFinally, the authors propose a practical downstream application in augmented/virtual reality scenarios, where object insertion can be steered more easily with the generated coordinate-frame aligned objects as compared to non-aligned objects.", "questions": "Please explain the evaluation procedure for computing LPIPS and CLIP scores. Which angles are used? What is the CLIP image embedding compared to? E.g., text prompts (\"front view\") or the CLIP embedding from the target shape rendering?\n\nThe baseline qualitative results look different for the same shape in Fig. 6 (see yellow bus for Trellis + VLM vs. Trellis + Orient Anything). Why? Shouldn't only their rotation change?\n\nIn the proposed DINO pose selection stage, measuring feature alignment with the L2 distance should likely be replaced with the cosine similarity. What exactly is compared here? Patch-level or CLS token?\n\nPlease feel free to further clarify any misunderstandings given my assessment of strengths and weaknesses of the paper.", "limitations": "The paper should discuss the ambiguity of orientation annotations for certain object classes more clearly and how this affects evaluation and comparison with prior works.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths:**\n\nThe paper tackles an important problem which is well motivated by the downstream applications shown in the paper, both in robotics and AR/VR applications.\nThe paper is generally well-written and easy to follow.\nThe paper introduces new orientation annotations for a subset of the Objaverse dataset, which will be made public and can thus be used in other projects as well.\n\n**Weaknesses:**\n\nThe authors state that wrongly annotated orientations by the VLM are manually corrected. As examples, stick-like objects and objects with ambiguous orientation or unclear front view are mentioned. \nWhile \"canonicalising\" the orientation of ambiguous objects might make sense as a data pre-processing step, it is dangerous to claim that these annotations are \"ground truth\" from now on.\nFor example, I am personally unable to tell the \"up\", \"front\" or \"right\" direction for a spoon.\n\nThe same problem also is relevant when looking at the comparison against prior work for orientation estimation in Tab.3. The authors argue that their method outperforms prior art for stick-like objects but again it is questionable whether a clear ground truth can be defined in such cases. Maybe, in these cases it would be better to measure whether the principal axes of the object are aligned with any of the coordinate frame axes.\n\nThe authors write that \"Experiments show that the pre-trained 3D generative models can further improve geometry quality after fine-tuning on our dataset\". (l.141)\nHowever, when inspecting the qualitative results in Fig.5, it rather looks like the geometric and visual appearance quality decreases for the generations of the fine-tuned model.\n\nThe paper is missing important details on how the experiments are evaluated. To be more specific, the LPIPS (and potentially CLIP) scores need to be computed in comparison against some ground-truth renderings. \nIf these renderings are e.g. at the six canonical views that were used when training Wonder3D-OA, the comparison in Tab.1 is unfair:\nAs the LGM (which is used to generate the 3D object) is conditioned on exactly these views for Wonder3D-OA but not for Wonder3D, the renderings at the evaluation viewing angles will include more artefacts caused by the LGM for Wonder3D compared to the OA version.\n\nThe presented downstream application (insertion of objects in 3D scenes) using a forward-facing arrow is not taking care of the additional degree of freedom (rotation around the forward facing arrow), which might be desired.\nThis is, however, only a minor weakness, as the alignment with the coordinate axes still makes manipulation easier than with tilted / rotated objects.\n\nThe paper should further include more details on the fine-tuning / introduced changes to prior methods.\nE.g., the pixel injector (l.196), not using depth input (l.220), or DINO pose selection stage (l.222) could be explained more clearly.", "was_revised": true, "final_justification": "The authors addressed my concerns and questions in the rebuttal. While I agree with reviewer yWsi that the practical novelty of the presented work is limited, I believe that the dataset with canonicalized 3D objects will be of help for further research. The paper should be accepted but the authors should update their manuscript following their rebuttal and discussion, especially for the evaluation of multi-view diffusion models.", "rebuttal": "Thank you for recognizing that our work addresses an important problem, well motivated by the downstream applications. Here we aim to address the concerns raised in the review.\n\n**W1. & W2. Ambiguity of object orientation.** \n\nIt's true that orientation ambiguity does exist for some objects, especially tools like spoons, mugs, fire extinguishers, and similar items. However, this ambiguity does not prevent the establishment of consistent conventions to define their canonical orientations. It is important to acknowledge that canonical poses may not align perfectly with every individual's intuitive expectations due to inherent ambiguity. However, once a reasonable definition is adopted and agreed upon by the community, it effectively becomes a form of shared common sense. This agreement is crucial for enabling important tasks, such as orientation-aligned 3D generation, object orientation estimation, and efficient object orientation manipulation. \n\nIn practice, during the manual correction process, we refer to the object orientations defined in prior work, specifically ImageNet3D [1], for the categories it covers. For example, for spoons, which are included in ImageNet3D, we align their poses in our dataset accordingly. For ambiguous objects only included in our dataset, we define canonical poses based on semantic part structures, geometric features, and common knowledge, following the principles established by ImageNet3D and our supplementary material section A.1.\n\nDue to our pose alignment with ImageNet3D, its human annotation can serve as appropriate ground truth for evaluating our orientation estimation method. Furthermore, relying solely on whether an object's principal axes align with the coordinate frame axes is not a sufficient metric. A model might perform well on such a metric while still failing to distinguish between the front and back of an object. For instance, a robot using an orientation estimation algorithm incapable of identifying the head versus the handle of a spoon would struggle to use it properly.\n\n[1] Ma, Wufei et al. \"ImageNet3D: Towards General-Purpose Object-Level 3D Understanding.\" NeurIPS 2024.\n\n**W3. Qualitative results decrease.**\n\nIt is true that the visual geometric and appearance quality of multi-view outputs (Figure 5) exhibits a slight decrease. However, these results are direct outputs of the multi-view diffusion model. The final 3D quality is more strongly influenced by whether the multi-view images are correctly aligned with their corresponding camera views and the 3D lifting method. Since the main paper Table 1 is intended to measure the orientation canonicalization ability, to demonstrate the quality improvement effectively, we canonicalize the poses of Wonder3D with the GT camera poses of the input images and compare its quality to our Wonder3D-OA on the Toys4k dataset:\n\n|  |Chamfer Distance $\\downarrow$|LPIPS* $\\downarrow$|CLIP* $\\uparrow$|\n|-|:-:|:-:|:-:|\n|Wonder3D w/ GT canonicalization|0.0653|0.2460|88.70|\n|Wonder3D-OA (Ours)|0.0548|0.2317|92.09|\n\nNote that the Wonder3D adopts the same multi-view generation and 3D lifting method as we present in the main paper, with only canonicalized with GT poses. The quantitative results demonstrate that our Wonder3D-OA actually has quality improvement over Wonder3D. Note that the LPIPS* and CLIP* are the updated metrics described in the answer to W4 & Q1. Besides, in the supplementary materials (Figure 8, Figure 9, and video 2:18), we also present numerous renderings of the reconstructed 3D models.\n\nAdditionally, we also provided similar quantitative results for Trellis-OA in the supplementary materials Table 3. There, we manually rotate the 3D models produced by Trellis and compare the quantitative results to those from our Trellis-OA variant. The results demonstrate that after fine-tuning on our Objaverse-OA, Trellis exhibits better quality performance. Finally, more examples in the supplementary video (e.g., objects shown at 1:42, bottom row; 1:53, row 5, columns 1 and 3; row 6, columns 5, 8, and 10) clearly demonstrate that our Trellis-OA produces higher-quality outputs compared to the original Trellis model.\n\n**W4 & Q1. Details and unfairness in experiment evaluation.**\n\nIn our experiments, we originally rendered four orthogonal views with the camera elevation angle of 0 for both the generated and GT 3D models, and computed the LPIPS and CLIP scores based on these renderings at matched camera poses. We acknowledge that this setup may introduce bias, as it aligns with the multi-view configurations used by Wonder3D-OA. To ensure a fairer evaluation, we have updated the rendering protocol by randomly sampling views on a unit sphere for all evaluation objects. We then computed updated metrics denoted as LPIPS* and CLIP* under this revised setup:\n\n|  | LPIPS* on GSO $\\downarrow$ | CLIP* on GSO $\\uparrow$ | LPIPS* on Toys4k $\\downarrow$ | CLIP* on Toys4k $\\\\uparrow$ |\n|-|:-:|:-:|:-:|:-:|\n| Wonder3D | 0.2799 | 76.37 | 0.2859 | 87.10 |\n| Wonder3D + PCA | 0.2554 | 77.80 | 0.2691 | 87.58 |\n| Wonder3D + Orient Anything | 0.2600 | 77.50 | 0.2699 | 88.12 |\n| Wonder3D + VLM (Gemini-2.0) | 0.2752 | 76.30 | 0.2804 | 87.53 |\n| Wonder3D-OA (Ours) | **0.2270** | **80.30** | **0.2317** | **92.09** |\n\nThese updated results confirm that our method continues to outperform the baselines. We will update the results in our final version.\n\nIn terms of the reconstruction method applied to the generated images, we used the Instant-NGP [2] adopted in the Wonder3D official implementation to generate the 3D models for all results in the main paper, as we shared the same concern that LGM [3] may be more compatible with Wonder3D-OA than with the original Wonder3D framework.\n\nThere is another potential question mentioned by Reviewer MH83 that you may also have: what actually lead to the quality differences? The replacement of the 3D representation with LGM or the orientation-aware design? To address this concern, we further make an ablation study on the LGM design, where we replace the LGM with the Instant-NGP:\n\n|  |CD on GSO $\\downarrow$|LPIPS* on GSO $\\downarrow$|CLIP* on GSO $\\uparrow$|CD on Toys4k $\\downarrow$|LPIPS* on Toys4k $\\downarrow$|CLIP* on Toys4k $\\uparrow$|\n|-|:-:| :-:| :-:| :-:|:-:|:-:|\n| Wonder3D w/ Instant-NGP | 0.0894 | 0.2799 | 76.37 | 0.0932 | 0.2859 | 87.10 |\n| Wonder3D-OA w/ Instant-NGP | 0.0609 | 0.2300 | 80.22 | 0.0571 | 0.2351 | 91.33 |\n| Wonder3D-OA w/ LGM (Ours) | 0.0564 | 0.2270 | 80.30 | 0.0548 | 0.2317 | 92.09|\n\nThese results indicate that while LGM [3] contributes to improvements, the primary performance gains over the baselines in the main paper Table 1 are attributed to our orientation-aligned design.\n\n[2] Müller, Thomas et al. \"Instant neural graphics primitives with a multiresolution hash encoding.\" TOG 2022.\n\n[3] Tang, Jiaxiang et al. \"LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation.\" ECCV 2024.\n\n**W5. Without taking care of the additional degree of freedom during object insertion.**\n\nWe agree that using a forward-facing arrow alone does not account for the rotational degree of freedom around that axis. However, most real-world objects have a well-defined \"up\" axis due to their typical placement, which resolves this ambiguity. That said, certain objects, especially hand-held objects, actually lack a clearly defined vertical orientation. For such cases, our pipeline can be further extended to support rotational freedom around the forward-facing axis. Even with this flexibility, our arrow-based method still facilitates more efficient and effective object placement, as you rightly noted.\n\n**W6 & Q3. More details of the introduced changes to prior methods.**\n\nThanks for pointing out the lack of some details of our method. We will add the following details in our final version:\n\nFor the pixel injector (l.196), we follow the design in ImageDream [4] by concatenating the input image with the outputted noisy images for self-attention during training and inference to preserve the local feature in the input image. Specifically, Wonder3D employs a 3D dense self-attention mechanism with a shape of ($b _ z$, 6, $c$, $h _ l$, $w _ l$) across six views within a transformer layer, where $b _ z$ is the batch size, $c$ is the number of feature channels, $h _ l$ and $w _ l$ are the image resolution. Our pixel controller modifies this to ($b _ z$, 7, $c$, $h _ l$, $w _ l$), incorporating the input image as an additional view.\n\nFor not using depth input during orientation estimation (l.220), we replace the depth map input of FoundationPose [5] with an all-zero tensor of the same shape, thereby removing any reliance on depth input.\n\nFor the DINO pose selection stage (l.222), we obtain the patch feature maps from DINOv2 [6] for both input images and renderings of generated 3D models. Because the best-matching rendering should closely align with the input in both feature directions and norms, we use L2 distance (rather than cosine similarity), as it captures differences in both feature direction and magnitude.\n\n[4] Wang, Peng and Yichun Shi. \"ImageDream: Image-Prompt Multi-view Diffusion for 3D Generation.\" ArXiv 2023.\n\n[5] Wen, Bowen et al. \"FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects.\" CVPR 2024.\n\n[6] Oquab, Maxime et al. \"DINOv2: Learning Robust Visual Features without Supervision.\" ArXiv 2023.\n\n**Q2. Difference in qualitative results of two baselines.**\n\nThe yellow buses for Trellis + VLM vs. Trellis + Orient Anything in Fig.6 are actually the same 3D model posed in different orientations. The perceived differences likely arise from lighting variations, as we use 3 point lights for illumination following the Trellis [7] official implementation. Variations in object orientation result in differing shadow and highlight patterns, which can create the appearance differences.\n\n[7] Xiang, Jianfeng et al. \"Structured 3D Latents for Scalable and Versatile 3D Generation.\" CVPR 2025."}, {"review_id": "G4Mva4dPce", "reviewer": "Reviewer_MH83", "summary": "The paper proposed a novel task on top of image-to-3D generation, i.e. orientation-aligned 3D object generation through data curation and image-to-3D model finetuning (e.g. Wonder3D and Trellis). The proposed method benefits several downstreaming tasks such as pose estimation from a single image and arrow-based object insertion. Extensive experimental results demonstrate the effectiveness of the proposed framework.", "questions": "1. How accurate is the proposed method's pose estimation in scenarios involving multiple objects (e.g. some examples shown in orient-anything paper)?\n2. In Table 1, why does the vanilla Wonder3D with the normal map branch have a higher Chamfer Distance than Wonder3D-OA? Is this due to the replacement of the 3D representation with LGM, or is it attributed to the orientation-aware design introduced in the paper?", "limitations": "No potential negative societal impact can be found in this submission. My concern lies primarily in the technical aspects, and I question whether the problem setting itself is truly meaningful.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "** Strengths**\n1. The methodology and data curation are well-detailed and easy to follow. \n2. The experiments are thorough and comprehensive, including both qualitative and quantitative results.\n3. The paper implies that an orientation-aligned image-to-3D object generation model benefits single-image pose estimation, which is an interesting insight.\n\n** Weaknesses**\n1. The motivation behind the proposed method is not entirely convincing, as 3D orientation does not appear to be a problem in 3D object generation as object pose is typically tied to camera pose during rendering and can be easily adjusted in tools such as Blender. That said, object orientation is arguably more objective than subjective, as it often depends on the inherent functionality of the objects themselves. For example, rendering results of both Wonder3D and Wonder3D-OA are good to me as they are simply rendered from different viewpoints relative to the objects (I know the rendered camera views are actually fixed though).\n2. As for object insertion, placing generated objects into a scene layout, such as an indoor room, is typically straightforward with mesh representations. Moreover, relighting the objects along with the scene as a whole is also relatively easy. In contrast, arrow-based object insertion does not seem to support this capability, which may result in inconsistencies in the appearance of inserted objects under novel lighting conditions compared to the original scene layout. Additionally, the global illumination appears to change significantly before and after object insertion in the supplementary video, why is that?", "was_revised": true, "final_justification": "I appreciate the authors' responses, including the extensive experimental results. I would like to raise my score as most of my concerns were faithfully addressed.", "rebuttal": "Thank you, Reviewer MH83, for acknowledging that our experiments are thorough and comprehensive, and for recognizing that applying our method to single-view pose estimation presents an interesting insight. Here we aim to address the concerns raised in the review.\n\n**W1. The motivation is not entirely convincing.**\n\nWe remark that orientation-aligned 3D generation is an important and well-motivated problem, which is also acknowledged by Reviewer yWsi and Reviewer y6ZQ. Next, I will address your concerns one by one.\n\n1. Object orientation in camera coordinate is also fundamentally influenced by the object's orientation in the world coordinate system, although it is related to the camera pose, since it can be formulated as: $R _ {ob-in-cam} = R _ {ob-in-world} R^{-1} _ {cam-in-world}$, where $R _ {ob-in-cam}$ is the object pose in the camera coordinate system, $R _ {ob-in-world}$ is the object 3D pose in the world coordinate system, and $R _ {cam-in-world}$ is the camera 3D pose in the world coordinate system. Our orientation-aligned 3D generation actually aims to generate 3D models with aligned poses in the world coordinate system. This relationship also explains that the orientation difference in rendering results between Wonder3D and Wonder3D-OA from the same camera pose will lead to a discrepancy in global orientation after lifting them to 3D space. \n\n2. Manual object pose adjustment in Blender is time-consuming. As you mentioned, to edit the orientation of a 3D model, a commonly used method is manually adjusting its pose in Blender. However, manually rotating a randomly oriented model to its canonical pose or a desired orientation in Blender is time-consuming, taking approximately 20 seconds according to our experiment. For complex scenes with hundreds of objects, this becomes prohibitively inefficient.\n\n3. Our orientation-aligned 3D generation task is strongly motivated by practical downstream applications, as acknowledged by you and other reviewers. First, 3D models lacking orientation alignment are unsuitable for orientation-related tasks. We demonstrate this with quantitative results for orientation estimation, comparing the usage of orientation-misaligned Trellis to our Trellis-OA during orientation estimation:\n\n|  | ACC@30 on Toys4k $\\uparrow$ | Abs on Toys4k $\\downarrow$ | ACC@30 on ImageNet3D $\\uparrow$ | Abs on ImageNet3D $\\downarrow$ |\n|----|:----:|:----:|:----:|:----:|\n| Trellis | 6.65 | 101.94 | 13.73 | 86.49 |\n| Trellis-OA (Ours) | **52.87** | **46.76** | **62.25** | **34.20** |\n\nSecond, as mentioned above, misaligned 3D models significantly hinder efficiency in practical use. Manually rotating randomly oriented models is time-consuming, especially for complex scenes. In contrast, our orientation-aligned 3D models offer a more scalable and user-friendly solution, which is highly beneficial for downstream systems, such as the augmented reality applications presented in our paper. \n\n4. Humans' common sense of object's orientation is already built upon the functionality of the objects: e.g., the front of a television or computer is the viewing direction, while the front of a cabinet is defined by the direction of access. For objects with ambiguous front-view definitions, as also noted by Reviewer y6ZQ (Weakness 1), we adopt the conventions from ImageNet3D [1] for existing categories and follow its principles-semantic part analysis, shape features, and common knowledge-for new categories in our dataset. Please refer to our earlier response to Reviewer y6ZQ for more on this point.\n\n[1] Ma, Wufei et al. \"ImageNet3D: Towards General-Purpose Object-Level 3D Understanding.\" NeurIPS 2024.\n\n**W2. Novelty and illumination problem of the object insertion method.**\n\nWhile inserting mesh-based 3D objects into a scene and applying relighting techniques may be technically straightforward, controlling object orientation efficiently remains a challenging problem-especially for misaligned models, as previously discussed. Our proposed arrow-based object insertion framework addresses this challenge by providing a simple yet effective mechanism for controlling orientation in augmented reality scenarios. This is made possible due to our orientation-aligned 3D generation pipeline.\n\nRegarding illumination, as stated in Supplementary Material (Line 74), we employ DiffusionLight [2], an environment lighting estimation method, to ensure consistency between inserted objects and the background scene. The global illumination change shown in the supplementary video is solely intended to enhance the visibility of our arrow-based control interface and does not affect the background image itself. Specifically, we add a semi-transparent white background to the background image before insertion to make the input images and arrows more distinct. The background image remains unchanged before and after insertion (the background image after insertion is the background image we use throughout the applications).\n\n[2] Phongthawee, Pakkapon et al. \"DiffusionLight: Light Probes for Free by Painting a Chrome Ball.\" CVPR 2024.\n\n**Q1. Pose estimation in scenarios involving multiple objects.**\n\nTo extend our pose estimation method to multi-object scenarios, we can incorporate an object segmentation module, such as SAM [3] or GroundSAM [4], to segment the input scene into individual object regions. These segments are then processed independently by our pose estimation pipeline.\n\nOur method performs well in various multi-object scenarios. However, due to NeurIPS 2025 rebuttal policies, we are unable to include qualitative results for such cases.\n\nBesides, we acknowledge that highly crowded scenes with severe occlusion may still degrade performance. Addressing this issue is part of our planned future work through occlusion-aware training for orientation estimation.\n\n[3] Kirillov, Alexander et al. \"Segment Anything.\" ICCV 2023.\n\n[4] Ren, Tianhe et al. \"Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks.\" ArXiv 2024.\n\n**Q2. The reason why vanilla Wonder3D has a larger Chamfer Distance than Wonder3D-OA.**\n\nTo ensure a fair comparison, we clarify that vanilla Wonder3D does not use the normal map during 3D lifting, consistent with Wonder3D-OA. To investigate further, we present additional results that include the use of normal maps with Wonder3D:\n\n|  | Chamfer Distance on GSO $\\downarrow$ | Chamfer Distance on Toys4k $\\downarrow$ |\n|--|:---:|:---:|\n| Wonder3D w/o normal map | 0.0894 | 0.0932 |\n| Wonder3D w/ normal map | 0.0972 | 0.1018 |\n\n\nInterestingly, the use of normal maps in Wonder3D may even lead to a worse Chamfer Distance. It is because the Chamfer Distance is sensitive to the pose alignment between the evaluated and the GT 3D models. In our experiment, this metric is primarily used to measure pose canonicalization performance across different methods. Since the 3D lifting method may also influence the Chamfer Distance, we replace the LGM [5] with Instant-NGP [6] used for the Wonder3D baseline shown below. The results confirm that the improvements arise predominantly from our orientation-aware design.\n\n|  | Chamfer Distance on GSO $\\downarrow$ | Chamfer Distance on Toys4k $\\downarrow$ |\n|--|:---:|:---:|\n| Wonder3D-OA w/ Instant-NGP | 0.0609 | 0.0571 |\n| Wonder3D-OA w/ LGM (Ours) | 0.0564 | 0.0548 |\n\nBesides, in our rebuttal to Reviewer y6ZQ W3, we present more quantitative results that only measure the generation quality, disentangled from the pose canonicalization ability, by canonicalizing the 3D models generated by the Wonder3D and Trellis with the GT or manual operation. You can refer to this rebuttal if you have the same concern.\n\n[5] Tang, Jiaxiang et al. \"LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation.\" ECCV 2024.\n\n[6] Müller, Thomas et al. \"Instant neural graphics primitives with a multiresolution hash encoding.\" TOG 2022."}, {"review_id": "ML2gq1YpYH", "reviewer": "Reviewer_kr5a", "summary": "In many datasets, objects are often presented in arbitrarily oriented canonical positions, which may not align with human perception of their natural orientation. \nThis paper addresses the challenge of such non-oriented objects by constructing a large dataset of forward-facing objects, defined as orientations consistent with typical human perception. \nThe authors train two 3D generative models that, given one or more renderings of an object, predict its corresponding forward-facing orientation. \nAdditionally, they propose and train two downstream tasks that explicitly involve object orientation.", "questions": "1. The paper states that each 3D object is rendered from four horizontal viewpoints—front, back, left, and right. \nHowever, many objects in Objaverse exhibit misalignment or arbitrary canonical orientation, meaning these standard views may not accurately capture the true forward-facing direction. As a result, there's a risk that none of the rendered views actually correspond to the perceptual front of the object. \nFurthermore, while it's mentioned that Gemini successfully identifies the front view in ~50% of cases, this raises concerns about the remaining objects—does the method fail because no true front-facing view was included?\n\n2. Did you randomize the input render positions during fine-tuning?", "limitations": "1. Limited Realism: Most examples appear toy-like or low-resolution. It remains unclear whether the proposed approach generalizes well to realistic, lifelike 3D objects.", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. Dataset Creation: Construct a large-scale dataset of forward-facing objects, where each object's orientation aligns with human perception.\n2. Model Generalization: Demonstrate that fine-tuning a 3D generative model on this dataset enables generalization across diverse object categories.\n3. Qualitative Results: Present high-quality visualizations of correctly oriented objects across diverse categories, showcasing the effectiveness of the approach on Objaverse samples.\n\nWeaknesses\n1. Limited Realism: Most examples appear toy-like or low-resolution. It remains unclear whether the proposed approach generalizes well to realistic, lifelike 3D objects.\n2. Marginal Novelty: Aside from the construction of a forward-facing 3D object dataset, the work offers limited novelty. \n3. Insertion Quality: The insertion results often resemble a simple copy-paste of the forward-aligned object into a scene, lacking evidence of sophisticated integration or context-aware placement. So I can see no special thing done here.", "was_revised": true, "final_justification": "the whole paper is built on making a dataset and train on it\nit is not the place for datasets", "rebuttal": "Thank you, Reviewer kr5a, for acknowledging our contributions to dataset creation and the strength of our experimental results. Here we aim to address the concerns raised in the review.\n\n**W1.Limited Realism.**\n\nOur model can effectively generate 3D models from realistic, lifelike images. We have presented some qualitative results on the real-world dataset ImageNet3D [1] in the supplementary Figure 7. Unfortunately, due to this year NeurIPS's policy, we cannot include more qualitative results. However, Table 3 and Figure 7 of the main paper demonstrate that our orientation estimation method generalizes well to real-world images from ImageNet3D, indicating our strong performance beyond synthetic data. \n\nAs for the realism of the generated 3D models, this aspect is largely dependent on the capability of the underlying 3D generative model. Since our experiments demonstrate that the fine-tuned 3D generative models can still preserve the 3D priors of the backbone, fine-tuning more advanced 3D generative models like Hunyuan3D 2.5 [2] can effectively improve the quality of the generated 3D models.\n\n[1] Ma, Wufei et al. \"ImageNet3D: Towards General-Purpose Object-Level 3D Understanding.\" NeurIPS 2024.\n\n[2] Lai, Zeqiang, et al. \"Hunyuan3D 2.5: Towards High-Fidelity 3D Assets Generation with Ultimate Details.\" Arxiv 2025.\n\n**W2. Marginal Novelty.**\n\nThe main contributions of our work are that we are the first to identify the important but overlooked task, orientation-aligned 3D generation, via in-depth research on relevant works in the field of 3D generation. We demonstrate that the most effective solution is creating a new dataset and directly fine-tuning existing 3D generative models rather than commonly used post-processing methods, including PCA, Vision Language Model (VLM), and Orient Anything [3]. We also prove the importance of orientation-aligned 3D generation by firstly applying it to vital downstream applications.\n\nBesides, we also make several other innovations compared to relevant works. For the dataset curation, while the initial stage of our data curation pipeline is inspired by Orient Anything [3], we investigate and address the limitations of the VLM through manual inspection and correction, significantly improving dataset quality. For the orientation estimation application, we are the first to utilize an orientation-aligned 3D generative model for zero-shot model-free orientation estimation. This approach enhances the orientation awareness of SOTA 6D object pose estimation methods like Any6D [4], which use orientation-misaligned generative models for template creation. Our method also exhibits superior generalizability compared to SOTA orientation estimators like Orient Anything [3]. This is because we leverage the comprehensive 3D priors from powerful 3D generative models trained on larger orientation-misaligned datasets, in addition to the rich number of categories in our own dataset. For the object insertion application, we discover that 3D orientation alignment improves the efficiency of object orientation manipulation in simulation systems like the augmented reality application.\n\n[3] Wang, Zehan et al. \"Orient Anything: Learning Robust Object Orientation Estimation from Rendering 3D Models.\" ICML 2025.\n\n[4] Lee, Taeyeop et al. \"Any6D: Model-free 6D Pose Estimation of Novel Objects.\" CVPR 2025.\n\n**W3. Insertion quality and method novelty.**\n\nOur object insertion method primarily focuses on enabling efficient control of 3D object orientation within simulation systems such as augmented reality applications, rather than optimizing for insertion quality. Manually adjusting the orientation of a misaligned 3D model is both time-consuming and labor-intensive, especially in large-scale scenes, which takes approximately 20 seconds for a skilled user to manually rotate a randomly oriented object to the desired orientation or canonical orientation. To address this, we propose an arrow-based object insertion method that allows for intuitive and effective orientation control, made possible by our orientation-aligned 3D generation pipeline.\n\nAs described in the Supplementary Material (Line 74), we employ DiffusionLight [5], an environment lighting estimation method, to ensure visual consistency between the inserted objects and the background image. This lighting strategy helps maintain realistic integration without compromising the focus on orientation control. You can check the middle row of the supplementary Figure 2 to see the shadow caused by our environmental lighting.\n\n[5] Phongthawee, Pakkapon et al. \"DiffusionLight: Light Probes for Free by Painting a Chrome Ball.\" CVPR 2024.\n\n**Q1. Reason for the VLM failure.**\n\nAs detailed in Supplementary Section A.1, some VLM (Vision-Language Model) failures are indeed attributed to pose misalignment, particularly in the case of stick-like objects where none of the four horizontally sampled viewpoints clearly captures a true front-facing view.\n\nHowever, as shown by Orient Anything [3], most objects in the Objaverse dataset exhibit primary variation along the horizontal (yaw) axis. To further understand pose misalignment, we conducted an analysis on the Objaverse-LVIS dataset.\n\nWe used Principal Component Analysis (PCA) to determine the principal axes of the objects and evaluated whether these axes aligned with the global coordinate system. Due to variations in object shape, some canonical poses yielded uncanonical principal axes under PCA. To mitigate this, we first identified objects in the Objaverse-OA dataset with principal axes aligned to the coordinate system. We then examined the same objects within the Objaverse-LVIS dataset.\n\nThe analysis reveals that only 16.83% of objects exhibit arbitrary poses in Objaverse-LVIS. Therefore, while pose misalignment contributes to VLM errors, it is not the sole cause. As we detailed in the supplementary section A.1, the VLM recognition errors can also be caused by other reasons, like unclear front-view feature and ambiguous front-view definition.\n\n**Q2. Randomization of the input rendering positions during fine-tuning.**\n\nYes, during fine-tuning, we randomly sample camera positions on the unit sphere for rendering input images and also randomize the camera intrinsic, following the official operation adopted by Trellis [6], to ensure the fine-tuned model generalizes well across a diverse range of input views and camera configurations.\n\n[6] Xiang, Jianfeng et al. \"Structured 3D Latents for Scalable and Versatile 3D Generation.\" CVPR 2025."}]

Author rebuttal:

Meta-review: (a) This paper introduces Objaverse-OA, a large-scale dataset of 14,832 3D models annotated with orientation-aligned canonical poses across 1,008 categories. Using this dataset, the authors fine-tune two existing 3D generative models (Trellis and Wonder3D) to produce orientation-aligned outputs. These aligned models are then applied to two downstream tasks: 1) zero-shot object orientation estimation without requiring oriented CAD references, and 2) arrow-based object insertion into AR/VR environments. Extensive experimental results demonstrate the effectiveness of the proposed framework.

(b) The main contribution is a large and curated dataset of orientation-aligned 3D models that will likely be useful for future research. The paper is found to be easy to follow, with clear figures and useful supplementary materials. The reviewers appreciate the significance of the orientation alignment task and see the direct implications for robotics, AR/VR, and 3D content creation. The empirical validation demonstrates benefits for both orientation estimation and interactive object insertion.

(c) The main concerns are on the novelty. Since the main contribution is a dataset curation using an existing pipeline (Orient Anything), with limited algorithmic innovation beyond modest model modifications. Several reviewers view this as below the expected level of methodological novelty for NeurIPS. The task of estimating the orientation is not well-defined for all objects (e.g., spoons, sticks). There were concerns on the evaluation methodology (eg, there may be bias on the choice of viewpoints). There were also concerns on the data quality (eg, some generated objects appear toy-like or low-resolution). Reviewers found the downstream applications to have limitations (eg, insertion examples are more copy-paste rather than full integration).

(d) The dataset is substantial, well-annotated, and has clear practical value. The paper convincingly shows that orientation alignment improves downstream usability. The work will likely serve as a foundation for future research, particularly for robotics and AR/VR.
However, it is not considered for a spotlight nor an oral because the novelty is low (primarily data curation with light fine-tuning). Also, there are some fundamental issues with the task itself (orientation ground-truth is inherently ambiguous for some object classes). The technical contributions are not at the level of major algorithmic advance expected for a spotlight/oral.

(e) In the discussion and rebuttal three reviewers very overall satisfied with the clarifications, although the concerns on novelty and the ambiguities in the task largely remained. kr5a was the most negative reviewer (borderline reject). kr5a viewed the paper as primarily a dataset paper, but with limited realism and novelty.
In conclusion, the rebuttal addressed clarity and evaluation details but did not fully resolve concerns on novelty or orientation ambiguity. Nonetheless, three reviewers support acceptance, with consensus that the dataset and aligned models will have community value.
Decision: Accept (poster)