Title: Orient Anything V2: Unifying Orientation and Rotation Understanding
Venue: NeurIPS 2025
Accepted: yes
Keywords: Orientation Estimation

6DoF Pose Estimation
Abstract: This work presents Orient Anything V2, an enhanced foundation model for unified understanding of object 3D orientation and rotation from single or paired images. Building upon Orient Anything V1, which defines orientation via a single unique front face, V2 extends this capability to handle objects with diverse rotational symmetries and directly estimate relative rotations. These improvements are enabled by four key innovations: 1) Scalable 3D assets synthesized by generative models, ensuring broad category coverage and balanced data distribution; 2) An efficient, model-in-the-loop annotation system that robustly identifies 0 to N valid front faces for each object; 3) A symmetry-aware, periodic distribution fitting objective that captures all plausible front-facing orientations, effectively modeling object rotational symmetry; 4) A multi-frame architecture that directly predicts relative object rotations. Extensive experiments show that Orient Anything V2 achieves state-of-the-art zero-shot performance on orientation estimation, 6DoF pose estimation, and object symmetry recognition across 11 widely used benchmarks. The model demonstrates strong generalization, significantly broadening the applicability of orientation estimation in diverse downstream tasks.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents Orient Anything V2, an upgraded foundation model for understanding 3D object orientation and rotation from single or paired images. Compared to its previous version, V2 claims: a new pipeline for generating high-quality synthetic 3D assets, a robust method to identify 0 to N valid front-facing directions per object, a symmetry-aware learning objective that directly models rotational symmetry, and a multi-frame architecture that supports relative rotation estimation. The proposed model achieves strong generalization across multiple real-world benchmarks for orientation estimation, 3D pose estimation, and rotational symmetry recognition.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The problem addressed in this paper, orientation estimation of arbitrary objects, is well motivated.\n- The paper is overall clear and easy to follow.\n\nWeaknesses:\n- The paper overlooks several important related works that address similar tasks (pose estimation from a single image and orientation distribution prediction) such as RelPose [1] and NOPE [2].\n- In Table 1 and its accompanying analysis, the authors claim that “Orient Anything V2 significantly improves upon V1, benefiting from diverse synthetic data and robust ensemble annotation“. This suggests that the improvement primarily stems from the dataset, rather than from the method design changes. This is an important point, as the model architecture and training losses appear very similar between V1 and V2. If true, the contribution of this work would lie mostly on the data side.\n- In Table 4, the performance drops significantly when the model is not initialized with VGGT pre-trained weights, even falling below the baseline. This raises concerns about how much of the performance gain can truly be attributed to the proposed method itself.\n\nQUESTIONS:\nWhat is the best performance of Orient Anything V2 without using the VGGT pre-trained weights? Is it still comparable to the baseline?\n\nLIMITATIONS:\nSimilar to its predecessor, Orient Anything V2 does not estimate object translation, which is a critical component for many real-world applications, such as robotics and AR/VR.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presets Orient Anything V2 - a model for estimating the absolute 3D orientation (the pose of an object defined by its front face) and relative rotation (3D transform between pose in two images) of objects in images. The main distinctions to Orient Anything V1 are the enabling direct relative pose estimation and handling (rather than ignoring) objects with symmetry. \n\nTo achieve this, the paper makes two significant contributions. First, a data engine is developed to generate a large amount of high quality 3D assets (ImageNet21K class label -> object description from LLM -> image generated by generative model -> 3D mesh generated by Hunyan 3D). These assets are then pseudo-labeled using the V1 model and these predictions are aggregated to determine symmetry and the number of front faces with a final manual consistency check. Given this large set of labeled assets, a transformer-based model is trained to jointly predict an orientation distribution relative to a world coordinate frame, and relative rotation distribution. The proposed approach achievers state-of-the-art absolute orientation estimation, relative rotation estimation, and symmetry recognition.\n\nSTRENGTHS AND WEAKNESSES:\n**Strenghts**\n\nQuality\n- Extensive ablation studies - the paper ablates data scaling, model initialization, which are the main design components. As expected, rotation estimation benefits more from higher data scale.\n- Extensive evaluation and strong performance - to justify the claim of referring to their model as a foundation model, the authors demonstrate its zero-shot generalization on orientation estimation, relative pose estimation, and symmetry recognition. Successfully unifying these tasks and obtaining strong performance is a significant achievement. Compared with prior work, the model has strong performance even on views with a large angle gap between the frame pairs.\n- Model design - The architecture is simple and effective. Following the design of VGG-T and predicting a distribution over possible poses while handling periodicity for symmetry is elegant.\n\nClarity\n- The paper is well written with high quality figures and easy to follow. Some technical and implementation details are missing that should be included (see weaknesses)\n\nOriginality\n- Data engine - it is exciting to see that a combination of language models and image and 3D generative models can be thoughtfully combined to synthesize a dataset that is actually useful. Looking toward the future, leveraging generative models for data synthesis will be essential, and this work will inform future works attempting to do this.\n\nSignificance\n- The significance is clear -- this work unifies three pose understanding tasks in a single model with good zero-shot performance. There are clear downstream applications in robotics or 3D content generation.\n\n**Weaknesses**\n\nQuality\n- Understanding the effect of the data engine would greatly strengthen the paper. How much would performance reduce if instead of the data engine, Objaverse was used off-the-shelf?\n- Given the significant reliance on synthetic data, this warrants an explicit discussion on the visual domain gap between synthetic data and test data, as well as what can be effective mitigation strategies.\n\nClarity\n- How is the world coordinate system defined and does that affect generalization? This is an important detail that's not discussed in the main text or supplement. Given this omission, it should also be clarified how the origin points of the arrows indicating the various axes are determined for in the wild data visualizations and how these arrows are projected from 2D to 3D.\n\nMinor\nL287 - Fine grain -> Fine grained\n\nQUESTIONS:\n- What is the effect of using synthesized 3D data rather than Objeverse? Is the 3D engine worth the effort?\n- With respect to the sim2real domain gap, could the authors describe scenarios where the proposed approach may have critical failures?\n- Could the authors describe, in as much detail as possible, how the world coordinate frame is chosen and why (is it the center of mass of the object, 3D bounding box center?) and how this is handled at inference time (for an arbitrary 2D image, how can we know were the point is in 3D for the coordinate system that we learned to predict in?)? What is the process to draw the arrows on the in-the-wild images?\n\nLIMITATIONS:\nThe authors adequately address the weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on the problem of estimating object orientation from real world images. It builds on Orient Anything V1 and makes multiple improvements to the data collection and training pipeline to enhance its capabilities.\n\nFirst, they use recent 3D generative models to collect a diverse dataset of objects much beyond the previous work on Orient Anything V1. Additionally, they also use some heuristics to annotate multiple front faces on the horizontal plane for objects which are symmetric. \n\nThey then train a model to predict multiple plausible orientations of the object using this dataset, which is a significant advance compared to Orient Anything V1 which predicted only one possible orientation. Further, unlike Orient Anything V1 the same model is also capable of estimating the relative rotation between a pair of frames.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n1. The paper demonstrates impressive results across multiple tasks. For single-image absolute orientation estimation, the model outperforms Orient Anything V1 by a large margin. It also introduces two additional tasks—relative rotation estimation and horizontal-plane symmetry detection—and shows superior performance compared to strong baselines.\n\n2. I found Orient Anything V1 to be quite limited in applicability due to its inability to handle symmetric objects or those with no clear front face (as illustrated by the skateboard example in Figure 1). This work addresses that limitation effectively by modeling all plausible front faces for such objects.\n\n3. A particularly interesting contribution is the emergence of multiple front-face detection via simple statistical analysis of the predicted orientation distribution from a pretrained VLM trained on absolute orientation tasks. Despite the ill-posed nature of the problem, analyzing predicted pose distributions for each class over the dataset allows the model to detect multiple plausible front faces. This is a significant and impactful result, as there has been no clear method in the field for automatically identifying all valid orientations of arbitrary objects.\n\n4. The paper also employs some clever design choices, such as pretraining VGGT and leveraging the camera pose token—which contains related information—for orientation prediction. This results in a highly capable and versatile model.\n\n**Weaknesses:**\n\n1. In Figure 1, the tasks depicted in the third column are difficult to follow. Specifically, the overlay of the two coordinate systems in the third row is hard to interpret, and it is unclear that different tasks are being illustrated.\n\n2. Lines 23–24 are confusing; it’s not clear what it means for a model to have an intrinsic link to orientation yet not understand orientation. This could be clarified.\n\n3. Line 46—“and sets new records”—sounds overly celebratory. A more neutral phrasing would be more appropriate.\n\nQUESTIONS:\n1. Can the method be extended to work on scenes with multiple objects? It would be interesting to see some results on this and whether the model would generalize. The test datasets contain only one object per scene.\n\n2. I’m also curious if the method can work on arbitrary object classes which don’t exist in the training set. Is there a way to evaluate this?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a new model for single-view and multi-view viewpoint estimation and symmetry detection. The model builds on the existing Orient Anything model and extends that approach through proposing a novel data engine that leverage generative and VLM models. The data engine uses LLMs to generate detailed object descriptions for a large number of object types, text2mesh models to generate the 3D meshes, and VLMs to identify the front view (through a process of multiview generation, voting, and human calibration at the end). The proposed model starts with the VGGT model and trains it with the generated assets and annotation to estimate the object orientation and possible symmetries. The proposed approach achieves impressive performance on several datasets. The paper also details several ablations to evaluate impact of scale, weight initialization, and asset type.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- The paper is very well written. I especially appreciated the nuance and care taken in explaining the challenges with rotation symmetry and what examples don't make sense, as well as the limitations of prior work. I also appreciated the discussion of the limitations of real data and what generative models can provide. \n- The data engine is very well motivated and explained. The figures were especially helpful to understand what's going on. The manual verification and calibration also seems like a lot of work (L162-166). \n- The experimental section is nicely presented and the ablations done make a lot of sense. \n\n**Weaknesses**\n- The paper mentioned that Orient Anything V1 struggled in the relative rotation estimation task (L108, L114), yet, it's not evaluated on the relative rotation estimation task.\n- Similar to the previous point, VGGT (which is the basis for the proposed model) and RelPose [ref1] both seem like relevant comparisons for the relative pose estimation task. VGGT is incredibly relevant given that the results in Table 4 frame as a major contributor to the model's performance. \n- The paper makes several claims about limitations in the literature that ignore existing work on rotation symmetry and relative pose estimation especially regarding lack of open-world scenarios and limitation to category-specific methods. There exists several papers that tackled those problems in the past. For example, Ref2 predicted a probability distribution for single image rotation that capture rotation symmetries. Similarly, Ref1 predicted a distribution for pairwise views that also accounted for rotation symmetries. In terms of relative rotation estimation, Ref1 has been applied in more open-world settings. Furthermore, Ref3-4 were earlier works for relative pose estimation that did not make category/object specific assumptions. \n\n\nReferences:\n- [Ref1] Zhang et al. \"Relpose: Predicting probabilistic relative rotation for single objects in the wild.\" European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022.\n- [Ref2] Murphy et al. \"Implicit-PDF: Non-Parametric Representation of Probability Distributions on the Rotation Manifold.\" International Conference on Machine Learning. PMLR, 2021.\n- [Ref3] El Banani et al. \"Novel Object Viewpoint Estimation Through Reconstruction Alignment.\" 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2020.\n- [Ref4] Park et al. \"Latentfusion: End-to-end differentiable reconstruction and rendering for unseen object pose estimation.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020.\n\nQUESTIONS:\n- How well does Orient Anything V1, VGGT, and RelPose perform on relative rotation estimation? While I think all 3 comparisons are important, VGGT seems to be the crucial one give the results shown in Table 4. \n- The paper proposes a novel data engine and the resulted generated dataset. However, it's unclear from the text of the paper whether there are plans to release the data engine (code to generate this data) or the generated artifacts. The checklist asks the reader to refer to Sec 6, but it was unclear from Sec 6 what the plans are. Could you please clarify this? As the guidelines indicate, this is not a requirement but it does impact the contribution of the work to the community. \n\n\nThe following questions are less important and more of curiosities about how the model would perform under different edge cases. I think discussing them would improve the paper. \n- The paper appears to make an assumption regarding rotation symmetries only existing around the vertical (or up) vector (L66). This assumption doesn't seem to always hold; eg, a strafish's front view is likely facing the star shape, resulting in the rotation being relative to the front vector. I am curious how you think the model would react to something like this? Similarly for a wheel or a log, both of which will have rotation symmetries around right/left or front/back vectors.\n- Another example I am curious about is a sphere which would have 0 front faces (as well as no well defined up vector). This is discussed briefly in L70 as having no meaningful direction, but I am curious how the model would perform on such a case.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-07"}