Source-Free and Image-Only Unsupervised Domain Adaptation for Category Level Object Pose Estimation
Authors: Prakhar Kaushik
Aayush Mishra
Adam Kortylewski
Alan Yuille
Venue: ICLR 2024 poster (2024)
Keywords: 3D Pose Estimation
Unsupervised Learning
Neural Rendering
Analysis-by-Synthesis
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: We propose the first method to do UDA for 3D pose estimation using only images from target domain by incremental and selective vertex feature updates of a source neural mesh model of a feature level render and compare model.

Abstract: We consider the problem of source-free unsupervised category-level 3D pose estimation from only RGB images to an non-annotated and unlabelled target domain without any access to source domain data or annotations during adaptation. Collecting and annotating real world 3D data and corresponding images is laborious, expensive yet unavoidable process since even 3D pose domain adaptation methods require 3D data in the target domain. We introduce a method which is capable of adapting to a nuisance ridden target domain without any 3D data or annotations. We represent object categories as simple cuboid meshes, and harness a generative model of neural feature activations modeled as a von Mises Fisher distribution at each mesh vertex learnt using differential rendering. We focus on individual mesh vertex features and iteratively update them based on their proximity to corresponding features in the target domain. Our key insight stems from the observation that specific object subparts remain stable across out-of-domain (OOD) scenarios, enabling strategic utilization of these invariant subcomponents for effective model updates. Our model is then trained in an EM fashion alternating between updating the vertex features and feature extractor. We show that our method simulates fine-tuning on a global-pseudo labelled dataset under mild assumptions which converges to the target domain asymptotically. Through extensive empirical validation, we demonstrate the potency of our simple approach in addressing the domain shift challenge and significantly enhancing pose estimation accuracy. By accentuating robust and less changed object subcomponents, our framework contributes to the evolution of UDA techniques in the context of 3D pose estimation using only images from the target domain.

Published as a conference paper at ICLR 2024
SOURCE-FREE AND IMAGE-ONLY UNSUPERVISED DOMAIN ADAPTATION FOR CATEGORY LEVEL OBJECT POSE ESTIMATION
Prakhar Kaushik Aayush Mishra
Adam Kortylewski!
Alan Yuille
Johns Hopkins University
'University of Freiburg and Max-Planck-Insttinn for Informatics fpkaushi1, ami 2hr24, ayuillel e je.j...ii takortyle@mii-inf mmg.iiinif.g.........mmmmmm de
ABSTRACT
We consider the problem source-free unsupervised category-level pose estima- tion with only images to target domain without any access to source domain data or 3D annotations during adaptation. Collecting and annotating real-world 3D data and corresponding images is laborious, expensive, yet unavoidable pro- cess, since even 3D pose domain adaptation methods require 3D data in the target domain. We introduce 3DUDA, a method capable of adapting nuisance ridden target domain without 3D or depth data. Our key insight stems from the observa- tion that specific object subparts remain stable across out-of-domain (OOD) sce- narios, enabling strategic utilization of these invariant subcomponents for effective model updates. We represent object categories as simple cuboid meshes, and har- ness a generative model of neural feature activations modeled at each mesh vertex learnt using differential rendering. We focus individual locally robust mesh ver- tex features and iteratively update them based on their proximity to corresponding features in the target domain even when the global pose not correct. Our model is then trained in an EM fashion, alternating between updating the vertex features and the feature extractor. We show that our method simulates fine-tuning on a global pseudo-labeled dataset under mild assumptions, which converges to the target domain asymptotically. Through extensive empirical validation, including complex extreme UDA setup which combines real nuisances, synthetic noise, and occlusion, we demonstrate the potency of our simple approach in addressing the domain shift challenge and significantly improving pose estimation accuracy.
INTRODUCTION
In recent years, object pose estimation has witnessed remarkable progress, revolutionizing applica- tions ranging from robotics (Du et al., 2019; Wang et al., 2019a; Wong et al., 2017; Zeng et al., 2017) and augmented reality (Marchand et al., 2016; Marder-Fpppstein, 2016; Runz al., 2018) human- computer interaction. There have been works for 3D) and 6D pose estimation that have focused pri- marily on instance-level (He et al., 2021: 2020: Park et al.. 2019; Peng et al., 2019; Tremblay al., 80:8; Wang et al., 201 9a; Xiang et al., 2018) pose estimation methods. However, these methods re- quire object-specific 3D CAD models or instance-specific depth information and often unable to estimate object pose without given restance-specifii 3D priors. Category- methods (Chen et al., 2020a; Chen & Dou, 2021; Lin et al., 2021; Tian et al., 2020; Wang al., 2019b; 2021b) are more efficient, but often still require some 3D information, such as the ground truth depth map (Wang al., 2019b; Lin et al., 2021: Lee al., 2022) or point clouds (Lee al., 2023). Acquiring such labeled 3D data across different domains often formidable challenge, impeding the performance of these models when deployed real-world scenarios. Few recent attempts (Lee et al., 2022; 2023) to perform category level pose estimation in semi-supervised manner also require ground truth depth map (Lee et al., 2023) or point cloud (Lee et al., 2022) for every instance.
In this paper, we focus ameliorating the aforementioned drawbacks in UDA for 3D pose estima- tion. We design a model that is capable adapting to a target domain in an unsupervised manner without requiring any kind of 3D data and using only RGB images in the target domain. Our

---

Published as a conference paper at ICLR 2024
Before Adaptation
Mar Mcattation
(a) Local Pose Ambiguity
(b) Local Part Robustness
Our method utilizes two key observations- (a) Local Pose Ambiguity, i.e. the inherent pose ambiguity that occurs when we can only see part of the object. We utilize this ambiguity to update the local vertex features which roughly correspond object parts, even when the global pose of the object may be incorrectly estimated. (b) Local Part Robustness refers to the fact that certain parts (e.g. headlights in a car) are less affected OOD data, which verified by the (azimuth) polar histogram representing the percentage of robustly detected vertex features per image in target domain (OOD-CV (Zhao et al., 2023)) using the source model (Before Adaptation). Even before adaptation, there are few vertices which can be detected robustly and therefore are leveraged by our method to adapt to the target domain as seen by the increased robust vertex ratio After Adaptation.
Figure 1:
model does not require source data during adaptation and as shown later, is capable adapt real world OOD nuisances (Zhao et al., 2023) without requiring any synthetic data or augmentations.
Our source model is based the idea of generative modeling of neural network features (Ko- rtylewski et al., 2020; Wang et al., 2021a; 2023; Ma et al., 2022) that have been used to perform category-level 3D and 6D pose estimation. However, all of these methods are fully supervised and cannot be trivially adapted to an OOD target. We extend these neural feature-level render- and-compare methods' capabilities for source-free unsupervised learning, which can be utilized in real-world OOD scenarios. Our method, 3DUDA, is based on the observation that certain object subparts remain stable and invariant across out-of-domain (OOD) scenarios, as seen in Fig- ure 1, thereby offering robust foundation for model updates. We utilize this ensemble of less modified object local subparts and their inherent pose ambiguity in the nuisance-rideen target do- main images to adapt the source model. This allows us to ignore noise-ridden global object pose and still obtain relevant information from more robust local sub-components of the object. We focus on individual mesh vertex features, iteratively updating them based their proximity to the cor- responding features in the target domain. Our experiments show that this simple idea allows us to perform robust pose estimation in OOD scenarios with only images from the target domain.
In summary, we make several important contributions in this paper.
1. We introduce 3DUDA which is (in knowledge) the first method to do image only, source free unsupervised domain adaptation for category level 3D pose estimation.
2. 3DUDA utilizes local pose ambiguity and their relative robustness to adapt to nuisance ridden domains without access to any 3D or synthetic data. We present theoretical analysis for this insight, which motivates our method.
3. We evaluate our model real world nuisances like shape, texture, occlusion, etc. well as image corruptions and show that our model is able to adapt robustly in such scenarios. Our method performs exceedingly well in extreme UDA setups where multiple nuisance factors such as real-world nuisance, synthetic noise, and partial occlusion are combined.
2 RELATED WORKS
Category-level 3D pose estimation is a task for estimating the 3D pose of unseen instances but in known category. Current pose estimation approaches can be divided into keypoint approaches, which utilize semantic keypoints on 3D objects to predict 3D pose (Pavlakos et al., 2017a; Zhou et al., 2018) and render and compare methods which predict pose by fitting a 3D rigid transformation at the image level (Chen et al., 2020b; Wang et al., 2019b) or feature level (Wang et al., 2021a).
Feature-level render-and-compe between predicted and rendered (e.g. from a 3D mesh and a corresponding pose) object repre- sentations. Such optimization often helps in avoiding complex loss landscapes that arise by doing
methods predict 3D pose by minimizing reconstruction error
2

---

Published as a conference paper at ICLR 2024
Optimize
Mesh Rendered Feature Map
Robust Neural Vertices
OOD Data
CNN
Feature Map
Cr
Selective Vertex Feature Update
Robust Parts
Optimize
Figure 2: Overview of Our Method (3DUDA)
(a) We extract neural features from source model CNN backbone fi ow (Xtr and render feature maps from the source mesh model (Wss) (using vertex fcatures C+) and the pose estimate is optimized using render-and- compare (b) For this incorrectly estimated global pose, we measure similarity of every individual visible vertex feature with the corresponding image feature vector in fi independently (Equaation 3) and update individual values for batch of images (Equation 4). (c) The mesh model is
vertex features using average feature then updated using these changed vertices and backbone optimized using optimized neural mesh.
cender-and-comprre at the image pixel level. (Wang et al., 2019b) predict the pose of the ob- ject by solving rigid transformation between the 3D model D1 and the NOCS maps with the Umeyama algorithm (Pavlakos et al., 2017b). (Iwase et al., 2021) learned features using differen- tiable Wwrenberg-arqqueee contrastive features for the 3D model 2N and utilized a similar render-and-con comoparr setup.
optimization, whereas (Wang et al., 2021a; Ma et al., 2022) learned
Unsupervised Domain Adaptation for 3D pose estimation Unfortumately, all the methods men- tioned above are fully supervised. However, there are some semi supervsee methods like (Fu & Wang, 2022; Peng et al., 2022) that often require labeled target domain image and 3D data to work. Even methods like (Lee et al., 2022; 2023) require instance depth data, point cloud and segmentation labels during test-time inference. Other methods like (Yang et al., 2023) create synthetic data and mix them with some amount of annotated real data in order to do Synthetic to Real semi-supervised domain adaptation. To the best of knowledge, there is no previous work on unsupervised 3D pose estimation which is source-free free and requires only images for adaptation. Additionally, there also dearth of work 3D pose estimation which performs UDA for real-world nuisances like changes in texture, weather, etc. and in the presence of problems like occlusion.
METHODOLOGY
An overview of our unsupervised domain adaptation method, 3DUDA can be found in Figure 2. After defining the notation and setup, review our feature-level neural render-and-compare source model in Section 3.1 before describing our method in detail in Section 3.2.
Notation For each object category we define three sets of parameters: CNN backbone $.... neural mesh MR and clutter model B. We denote neural feature representation an input image N' xx, Fa #x xd Where is the output of layer of deep convolutional neural network backbone 2w. with being the number of channels in layer a. f E Rd a feature vector in F at position i on the 2D lattice P of the feature map. We drop the superscript in subsequent sections for notational simplicity. We represent our supervised source domain model with subscript S and our unsupervised target domain model with subscript T. For more details, A.1.
3.1 SOURCE MODEL: POSE-DEPENDENT FEATURE LEVEL RENDER AND COMPARE
Our source model is similar to previous work like Wang et al. (2021a; 2023); Ma et al. (2022) category-level pose estimation using neural feature level render and compare. These methods themselves are 3D extensions of feature generative models such as Kortylewski et al. (2020). Our source model defines probabilistic generative model of normalized real-valued feature activations
3

---

Published as a conference paper at ICLR 2024
F conditioned 3D neural mesh representation W. The neural mesh model aims to capture the 3D information of the foreground objects. For each object category y, the source model defines neural mesh 2g as {V,C), where V (V, E R is the set of vertices of the mesh and is the set of learnable features, i.e. neural features. r denotes the index of the vertices. R is the total number of vertices. We also define a clutter model B {Bn) ) describe the backgrounds. .N is a prefixed hyperparameter. For a given object pose or camera viewpoint g. we can render the neural mesh model Dis into feature map using (differentiable) rasterization (Kato et al., 2020). We can compute the object likelihood of a target feature map F E R RRx x as
C = {C, E ]==
R
p(F) B)
II iEFG
p WT 9)
II KEBG
B),
where FG and BG denote the foreground and background pixels, respectively. FG is set of all the positions in the 2D lattice P covered by the mesh M1 and BG are the positions that are not. We define P(f g) = Z [xp(] exp( fs nr,ce as a von Mises Fisher (vMF) distribution with mean G, and concentration parameter Ki.. For more details, please refer to the Appendix A.7.1.
3.2
3DUDA: UNSUPERVISED LEARNING OF 3D POSE USING NEURAL FEATURE SYNTHESIS AND SELECTIVE VERTEX FEATURE UPDATE
Given only the source model S (and no source data X')) and some non-annotated target domain RGB images, we adapt S to be able to perform well on the target domain. Figure 3 (NeMo column) shows examples of the performance of the source model in an OOD scenario. As expected, the estimated pose has diverged significantly from the ground truth pose in the target domain, indicating that the feature generative model parameterized by the neural mesh model My is no longer an adequate representative of the same object as a whole.
However, crucial observation, as seen in Figure 1, is that although the neural mesh model may not be a good global model for an object y, there is still a subset of robust vertices that corresponds to parts of objects that have undergone less changes in the new domain. Intuitively, this can be under- stood as some parts of an object changing less across domains. The number of such vertices and the threshold within which their shift contained are functions of the domain nuisance variables and the object itself. This property is intuitively leveraged by humans regularly to adapt their pre- vious knowledge understand new, unseen objects. For example, a car that may undergo changes involving shape, context, and texture will still have parts such as wheels, headlights, and windshield that change less or none at all across these domain shifts. We leverage this observation adapt the source model in an unsupervised manner.
NEURAL FEATURE SYNTHESIS WITH MULTI POSE INITIALIZATION
A fundamental benefit of Neural Mesh Models like our source model is that they are generative at the level of neural feature activations. This makes the overall reconstruction loss very smooth compared to related works that are generative the pixel level (Wang al., 2021a; 2023). Therefore, the source model can be optimized w.r.t. the pose parameters with standard stochastic gradient descent and contains one clear loss global optimum. This no longer true an OOD scenario.
Neural Feature Rendering For inference with the source model, we can infer the 3D pose g of the object y by minimizing the negative log likelihood of the model. Specifically, we first extract the neural features of the image F = T (X)) from the source CNN backbone. We define initial pose ginit using random initialization or by pre-rendering and comparing some pose samples. Using the initial pose, render the neural mesh 27 into a feature map F' E R'xx"x The projected feature map is divided into FG and Bg, depending on which pixels in the feature map are covered by the projected mesh features. We compare the rendered feature map and the image feature map position-wis wise. Given that the feature vectors are normalized and considering a constant k. the loss can be refactored as simple reconstruction loss. The dot products are normalized accordingly.
Cree = 1 In In g, B) CEieaah Σjebçf B)
(2)
The pose Ginit is optimized by minimizing Equation 2 using stochastic gradient descent.
4

---

Published as a conference paper at ICLR 2024
Multi-Pose Initialization Figure 6 shows different render-and compare optimized pose estimates by the source model that are optimized from different initial 3D pose in the target domain. This hap- pens because optimization is stuck in different local optima in the OOD loss landscape. Therefore, instead of random pose initialization, we pre-render a uniform sampling of poses from the neural mesh model and compare them with the image features. 1 5 initial poses chosen dependent how similar they are to the feature map and how far away they are from each other. We then optimize these initial poses using Equation 2, and may end up with multiple final rendered feature maps and estimated poses as shown in Figure 6. Even though the estimated object poses may be incorrect, we can still utilize these rendered maps for our Selective Vertex Feature Adaptation.
3.2.1 1
PROGRESSIVE SELECTIVE LOCAL VERTEX FEATURE ADAPTATION
Local Vertex-Feature Similarity We define the similarity between an individual rendered neural mesh Di vertex feature C, and its corresponding CNN feature / (shown as f(-r) for a pose g given renderer R as function of the parametric vMF score/likelhhood (Du et al., 2022);
Csim (fitr, C,) = Z[K,] exp (Kg fi
E EGG C, = R(N, g)
(3)
Similar similarity measures have been shown to be robust OOD detectors in earlier works like (Du et al., 2022). We define a rejection criteria dependent on a threshold ó such that all individual feature vectors from multiple data samples that correspond if (f Cr) < 5, and are not used to update these features. We can choose the threshold 6, st. majority (90-95%) of source domain features lie within the likelihood score.
specific vertex feature C, considered OOD
Selective Vertex Adaptation (SVA) from Rendered Neural Features For batch (size n) of target domain images Ny. we obtain their neural features from the finetuned CNN backbone ow and we render neural feature maps F; from our mesh model Ms using differentiable render and compare from initial pose estimates givst- We then spatially match the similarity of every rendered vertex feature with its corresponding image feature independently. For every vertex feature Cr, we average the corresponding image features f at position a in the 2D lattice which are above a threshold hyperparameter d using Equation 3, we update neural vertex feature C,- as follows:
C++11 +- aC, + (1 (1- n (I-
VS (Crm ((rr) > 5,
where Ct is the current vertex feature at timestep and C++1 is the updated vertex feature. a is a moving average hyperparameter. This can be done for the entire target domain adaptation data or in a batched manner. Subsequently, the estimated pose g' is recalculated with the updated neural mesh model, and the CNN backbone is updated by gradient descent iteratively with the following loss;
Z ZINipee" //irle=
L
Σ log TER.
(5)
EIERIEN, Z/riees/--ii + En Z|nile"'/i-..
where Rs denotes all visible vertices for the input image X. Nr denotes the vertices near r. We iteratively update subsets of vertex features and finetune the CNN backbone till convergence in a EM type manner. In practice, to avoid false positives and encourage better convergence, we establish a few conditions on our selective vertex feature adaptation process. To save computational overhead, we can fix for the loss calculation. We fix a hyperparameter eh that controls the least number of local vertices detected be similar (5- 10% of visible vertices). We also drop samples with low global similarity values during the backbone update. K, can also be recalculated in each time step t using the updated CP for a more robust measurement of similarity.
Unsupervised 3D pose estimation using SVA Figure 2 gives an intuition for our method, 3DUDA. It works as follows: (1) Extract neural features from source model CNN backbone f: = (X'7 (2) Pre-render feature maps from the source mesh (Wis) (using vertex features C,) and compare with image features (3) Choose top-3 similar rendered feature maps and calculate the optimized pose using gradient descent with reconstruction loss w.r.t fa (Equation 2). (4) For an opti- mized pose, measure the similarity of every individual visible vertex feature with the corresponding vector in fi independently (Equation 3). (5) Update individual vertex features using average fea- ture vector values for a batch of images (Equation 4). (6) Finetune CNN backbone using rendered feature maps obtained from Steps 1-3 using the updated vertex features (7) Continue steps and 6 iteratively until convergence. (8) Post adaptation, evaluate 3D object pose of target test data using steps 1-3.
5

---

Published as a conference paper at ICLR 2024
3.3 THEORETICAL RESULTS
Prior UDA works attempt to adapt in the target domain using a global pseudo-labeling setup. In global pseudo-labeling, a subset of images from the target domain are identified for which a majority (typically > R& for some Ω E [o, I]. We use Ω = for analysis.) of the visible vertex features satisty (im(firrr C+) 8,. These images are used to fine-tune the model. However, depending on how different the target domain is from the source domain, very small fraction of the target domain images (even none sometimes) usually satisfies this property in real datasets. This leads to negligible domain adaptation. However, upon careful inspection, we observed a robust subset of vertices in the neural meshes produced by the source-trained model. And as the features corresponding to these robust vertices are assumed independent, they can be used to fine-tune the model. In this section, we present conditions under which SVA simulates fine-tuning on a global pseudo-labeles dataset. This also reveals that the standard global pseudo-Jabeling setup is a special case of this formulation.
Let the distribution of a rendered vertex feature C, be denoted by P". Let the joint distributions of these rendered vertex features in the source and target domain be denoted by Ps II, P3 and Pr- = II, Pf. respectively. They are written as products of the individual marginal distributions because of the standard independence assumption between the vertices. Note that the distribution Ps is an approximation of the true underlying distribution Pg. This approximate distribution is achieved by training the source model on finite i.i.d. sample Xss from Pš and their corresponding labels (ground- frround-truth poses) Vss Similarly. the i.i.d. sample Xy elicits an approximate distribution Pr of the true Pt. Adapting for Pr is challenging because we do not have the corresponding VT.
Definition 3.1 (Vertex K K-partition) A vertex K K-partition is defined as a partition of the set of vertices (indexed by r E (1,2.... Rt) into K non-empty mutually disjoint subsets (indexed by E (1,2...,, K)). Let the set of vertices in each partitioned subset be denoted by Iv.
A given vertex K K-partition would split the joint distribution Ps into K independent joint distribu- tions (denoted by P = IIren P3) such that Ps = II PS.. The same extends for the correspond- ing target domain distributions.
Definition 3.2 (kō-subset) For a given sample X' and vertex K K-partition, kô-subset defined as 2 C X such that, Crim(Srer, Cx) > 8, Vr E It. The corresponding approximation of pla under 266 is denoted by plvó.
Assumption 3.3 (Piece-wise Support Overlap) There exists a vertex K K-partition such that the kg- subset of the target sample X7 satisfies,
12 kg
#0KK E {1,2, ..., K}
and as |27 k8 + x;, II p| Pr
This assumption requires the joint distribution of partitioned vertex subsets under 2 asymptot- ically approximate the corresponding true distributions. Intuitively, this translates to having enough support the target domain such that samples satisfying the similarity constraint (Equation 3) in each k tub-etss approximates the true target distribution of that vertex partition set.
Theorem 3.4 A target domain Xy satisfying assumption 3.3, elicits another target domain Xy" such that each sample in Xx" satisfies the global-pseude labelling constraint (Lsim (frerr Cr) > 8, Vr E 21, R)). Asymptotically with the size of the domain, X7" - Xx.
The proof of this theorem by construction of the set X7 for any Xr, and has been defered to the appendix A.3. It is easy to see that -lsbal- labelleling setup is the special case of this formulation when the vertex K --partition in assumption 3.3 is the trivial partition with K = 1. is also noteworthy that in the asymptotic case, even the global labelling setup would yield the same adaptability as SVA; but in practice, SVA yields much more data for adaptation than the former (see figure 5). Although the elicited target domain Xy does not represent the true target distribution precisely, it does yield more adaptability than global-pseudo labelling for a finite-sample and under assumption 3.3, asymptotically adapts the true distribution (Figure 4).

---

Published as a conference paper at ICLR 2024
Ground Truth
NeMo
Ours
Weather
Context Occlusion
Occlusion + Glass Blur
Weather
Figure 3: Qualitative Results of 3DUDA compared to ground truth and NeMo (Wang et al., 2021a). 3DUDA adapts to real world OOD target domains consisting of nuisances like weather and occlusion an unsupervised manner and produces robust 3D object pose estimates. The CAD objects are for representation only and are taken from ShapeNet (Chang et al., 2015).
EXPERIMENTS
Data We evaluate model on OOD-CV (Zhao et al., 2023) and Imagenet-C (Hendrycks & Dietterich, 2019) corrupted Pascal3D+ dataset (Xiang et al., 2014). OOD-CV is a benchmark in- troduced to evaluate the robustness of the model in OOD scenarios. It includes OOD examples of 10 categories that cover unseen variations of nuisances including pose, shape, texture, context, and weather. The source model is trained IID samples while the model is adapted and evaluated on OOD data for individual and combined nuisances. For SerrpteddPascalBBee tation and evaluation data with synthetic corruptions like shot noise, elastic deformation, fog, etc. from Imagenet-C. PASCAL3D+ dataset contains objects from 12 man-made categories, and each object annotated with 3D pose, 2D centroid, and object distance. During adaptation and infer- ence, only RGB images are provided. For harder set-up of real-world nuisances combined with partial occlusion, we test our algorithm on Occluded OOD-CV dataset which has been created in manner similar Wang et al. (2020) with levels of object occlusion (L1(20-40%) (2(40-60%)
we corrupt the adap-
We also evaluate our model on two extreme UDA setups of (1) Real + Synthetic corruptions. We add Imagenet-C corruptions to the OOD-CV dataset and expect models to adapt from clean IID source data to these data in an unsupervised manner. (2) Real + Synthetic Corruption Partial Occlusion. These are very difficult scenarios for UDA which, to our knowledge, have not been attempted before semi-supervised or unsupervised 3D pose estimation.
Metrics 3D pose estimation aims to recover the 3D rotation parameterized by azimuth, elevation, and in-plane rotation of the viewing camera. We follow previous works like Zhou al. (201 8); Wang et al. (2021a); Ma et al. (2022) and evaluate the error between the predicted rotation matrix and the ground truth rotation matrix: A A(Rpeed; Rgg) [[oogm(R Ro We report the accuracy of the pose estimation under common thresholds, End and alongwith median error.
Implementation Details An Imagenet pretrained Resnet50 is used feature extractor for source model. The cuboid mesh defined for each category and ensures that the majority of object area covered by it. The source model is trained for 800 epochs with batch size of 32 using

---

Published as a conference paper at ICLR 2024
an Adam optimizer in fully supervised manner. For every adaptation step, require minimum batch size 32 images for selective vertex and feature extractor update. We can also set the batch size for step adaptively by requiring enough samples s.t. 80% of vertices can be updated. For fixed K, our local vertex feature similarity threshold is .8. We train our model by switching between (in EM fashion) selective vertex update and feature extractor training for about 100 epochs. Our adaptation model is implemented in Py PyTochh (with Py PyTorch3D for differential rasterization) and takes around hours train A5000 GPUs.
Baseline Models We evaluate NeMo (Wang et al., 2021a), DMNT (Wang et al., 2023), Synthet- icP3D (Yang al., 2023) and standard Resnet-50. Wang et al. (2021a; 2023); Yang al. (2023) are pose estimation methods which utilize similar feature level render and compare methodology as our source model and have been shown to be robust and efficient. (Res50-General) is a ResNet50 classifier that formulates the pose estimation task for all categories one single classification task. Note that all models are evaluated on image only unsupervised learning setup. Although annotated target domain images are not provided to models, data augmentation synthetic data is allowed for our baseline models.
4.1 UNSUPERVISED 3D POSE ESTIMATION: RESULTS AND ANALYSIS
OOD-CV Table shows the Unsupervised 3D pose estimation results on OOD-CV (Zhao et al., 2023) dataset. This is our primary result, which shows our model efficacy in real-world scenarios. A qualitative comparison with Wang et al. (2021a) can be seen in Figure 3. All compared methods suffer equally in the real OOD scenarios. Surprisingly, the general Resnet50 model performs quite well relative to more complex models like NeMo and DMNT, suggesting that the additional category data is helpful in OOD scenarios. However, our method clearly outperforms all the models and is able to significantly bridge the domain gap.
Table I: Unsupervised 3D Pose Estimation for OOD-CV (Zhao et al., 2023) dataset
Accuracyf texture
Nuisance
Combined
shape
pose
context
weather
Res50-General
51.8
50.5
34.5
61.6
57.8
60.0
NeMo (Wang al., 2021a)
48.1
49.6
35.5
57.5
50.3
52.3
MaskRCNN (He et al., 2018)
39.4
40.3
18.6
53.3
43.6
47.7
DMNT (Wang al., 2023)
50.0
51.5
38.0
56.8
52.4
54.5
P3D (Yang et al., 2023)
48.2
52.3
45.8
51.0
54.6
44.5
Ours
94.0
93.7
95.1
97.0
95.5
83.1
JT. Accuracyy
18
Res50-General
18.1
15.7
12.6
22.3
15.5
23.4
NeMo (Wang al., 2021a)
21.7
19.3
7.1
33.6
21.5
30.3
MaskRCNN (He al., 2018)
15.3
15.6
1.6
24.3
13.8
22.9
DMNT (Wang et al., 2023)
23.6
20.7
12.6
32.6
16.6
33.5
P3D (Yang et al., 2023)
14.8
16.1
12.3
16.6
12.1
16.3
Ours
87.8
82.1
69.5
92.6
89.3
90.7
Table 2 shows the Unsupervised 3D pose estimation results on this setup for multiple corruptions. As expected, the drop model performance is largely depen- dent the type of corruption and its severity. Our method still performs significantly better when dealing with synthetic corruptions.
Pascal3D Corrrupted-Pascall33
Occluded-OOD-CY Table 3 (OccL1/L.2) shows the Unsupervised 3D pose estimation results on Eceluded-OOD-CA dataset at 2 levels of partial occlusion. This is a harder setup in which real nuisances are combined with occlusion. Our method is able to perform exceedingly well even in such a complex target domain with upto 67% improvement in accuracy. This is because our selective vertex adaptation focuses independently able to ignore occluded vertices for adaptation. Notably, Wang et al. (2021a) has been shown to be robust occlusion but suffers when it is combined with real world nuisances.
adapting individual neural vertices (and the model) and
8

---

Published as a conference paper at ICLR 2024
Table 2:
Unsupervised 3D pose estimation results for Pascalãd+ - Corrupted-Paase P
(Metrics: # #/66 Accuracy ( æ 18 Accuracy (is) Median Error (Er))
Er
Er
Er
Er
18
18
Gaussian Noise
Shot Noise
Impulse Noise
Defocus Blur
NeMo Ours
43.7
21.3
42.1
50.6
25.3
35.0
45.4
22.2
39.4
72.9
41.8
16.0
84.3
59.1
9.8
85,9
62.0
9.0
84.0
58.5
10.1
87.8
64.6
8.0
Glass Blur
Motion Blur
Zoom Blur
Snow
NeMo Ours
56.7
27.0
33.8
69.7
39.2
18.7
69.0
39.7
19.1
69.9
40.1
18.9
86.7
62.4
8.6
88.0
63.8
8.3
87.9
65.1
8.1
87.7
64.0
8.2
Frost
Fog
Contrast
Elastic Transform
NeMo Ours
73.3
44.1
16.4
85.5
59.0
9.5
74.5
43.8
14.7
77.4
50.3
13.8
86.3
62.5
8.6
88.7
65.6
7.8
88.8
66.7
7.6
88.2
64.4
8.1
Pixelate
Speckle Noise
Gaussian Blur
Spatter
NeMo Ours
77.5
53.0
13.0
67.9
38.3
20.8
68.3
36.5
18.7
72.4
44.2
17.2
88.7
65.4
7.8
87.7
64.2
8.0
87.7
63.8
8.3
87.5
63.9
8.4
NeMo (Wang et al., 2021a).
Extreme UDA: Real+Synthetic Corruption Table 3 (OOD+SN/GB) show the results the extreme UDA setup combining real and synthetic corruptions from the OOD-CV (Combined) and Imagenet-C datasets. We again see significant improvements compared to Wang al. (2021a) which have been shown previously to be robust under this challenging setup.
Table 3: Unsupervised 3D pose estimation results for Occlusion and Extreme UDA setup
(a) OceL1/L2: Real Nuisance (OOD-CV (Combined)) Occlusion (Levell/Level2) (b) OOD+SN/GB: Real Nuisance (OOD-CV) + Synthetic Noise (Speckle Noise/Glass Blur) (c) L1/L2+Spec: Real Nuisance (OOD-
CV) + Occlusion (L1/L2) Synthetic Noise (Speckle Noise)
OceL1
OceL2
OOD+SN
OOD+GB
L1+Spec
L2+Spec
18
18
0
18
18
6
18
18
NeMo
30.6
10.2
24.1
6.6
32.7
10.2
29.6
9.5
18.6
3.4
15.1
2.7
Ours
84.6
77.1
78.7
70.4
80.5
63.0
77.7
65.9
69.4
50.4
60.6
38.9
Extreme UDA: Real+Synthetic Corruption+Occlusionn Table 3 (LI/L2+Specc show the results on the extreme UDA setup combining real and synthetic corruptions along with partial occlusion from Occluded-OOD-CV and Imagenet-C datasets. This is an extremely challenging setup where three different kinds of nuisances/domain differences combined and is reflected in NeMo's sults. Our model is still able to adapt to such a target domain, showing our method's 's efficacy.
Further experimental and ablation analysis is deferred to the Appendix due to limited space.
5 CONCLUSION, LIMITATIONS AND FUTURE WORK
In this work, we attempt solve the previously unaddressed problem of unsupervised image-only source-free domain adaptation for 3D pose estimation. We focus our efforts on real world data with real world nuisances like weather, shape, texture, etc. and show that our method achieves sig- nificant success. Our method has limitations as it relies on the source model and cannot be trivially extended to articulated objects. requires multiple pre-rendered samples for pose estimation. Like many other pose estimation methods, inference requires optimization using render-and-compar methodology for optimal pose estimation. In the future, we want to extend our method to unsuper- vised 6D pose estimation and to unseen object setups. Furthermore, the importance of the concen- tration parameter k needs more research, as we believe that crucial uncertainty marker that may be relevant in more difficult domain transfer settings.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGEMENTS
This research has been supported by Army Research Laboratory award INF2320008 and ONR with N00014-21-1-2888 Adam Kortylewski acknowledges support via his Emmy Noether Research Group funded by the German Science Foundation (DFG) under Grant No. 468670075.
REFERENCES
Angel X. Chang, Thomas Funkhouser, Leonidas Guibas, Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu. Shapenet: An information-rich 3d model repository, 2015.
Dengsheng Chen, Jun Li, Zheng Wang, and Kai Xu. Learning canonical shape space for category- level 6d object pose and size estimation. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2020a. doi: 10. 109////p44200002000000 200.0200000000009999999 URL http:
//dx. doi. org/ 10. 1109/ 2021199cvvpp cvpr42660 2020 01199.
Kai Chen and Qi Dou. Sgpa: Structure-guided prior adaptation for category-level 6d object pose estimation. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 2753- 2762, 2021. URL http:///pw...aaaaat..ettoooooooo
http://p...seaaticcccoooooo www.chhww.....oooooooooooeo.eeee :4 244129110.
Xu Chen, Zijian Dong, Jie Song, Andreas Geiger, and Otmar Hilliges. Category level object pose estimation via neural analysis-by-se bysis-byyyytthsiss........ Lecture Notes in Computer Science, pp. 139-156, 2020b. ISSN 161 1611-3349, doi: 10. 0007/97--3-030-5885 10.1007/976----03--555-----
URL http: /dxx www.xxdd..oooo.. .org/
Guoguang Du, Kai Wang, Shiguo Lian, and Kaiyong Zhao. Vision-based robotic grasping from object localization, object pose estimation grasp estimation for parallel grippers: A review, 2019.
Xuefeng Du, Gabriel Gozum, Yifei Ming, and Yixuan Li. Siren: Shaping rep- resentations for detecting out-of-distribution objects. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Infor- mation Processing Systems, volume 35, pp. 20434-20449. Curran Associates, Inc.. 2022. URL https::/prroceeddingg
http:///proccecin..e.eeerereeeee newreiirdddnnnn..nnn.ne...eeeeee ca/ http:/pperrllllppppee/eee//////e file htt.://ccccabbeeeedffffeeccccaee Ponereee .ddf.
Yang Fu and Xiaolong Wang. Category-level 6d object pose estimation in the wild: A semi- supervised learning approach and new dataset, 2022.
Walter Goodwin, Sagar Vaze, loannis Havoutis, and Ingmar Posner. Zero-shot ceve object pose estimation, 2022.
Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn, 2018.
Yisheng He, Wei Sun. Haibin Huang, Jianran Liu, Haoqiang Fan, and Jian Sun. Pvn3d: A deep point-wise 3d keypoints voting network for 6dof pose estimation. 2020 IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2020. doi: 10. (0.1109/cvp442600220200 01165. URL hwthhhtp///. //tp doi www.dooo.oooooo... 10. 1ttp://..111//////1///
202...116...
Yisheng He, Haibin Huang, Haoqiang Fan, Qifeng Chen, and Jian Sun. Ffb6d: A full flow bidi- rectional fusion network for 6d pose estimation. 202/ IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), Jun 2021. doi: 2..1109/cYpr44447.221100000 http:/ //tp:/dxx.. doi. www.ooog/......... 11t.:...1111//111. 6437 2222.00300
URL
Yisheng He, Haoqiang Fan, Haibin Huang, Qifeng Chen, and Jian Sun. Towards self-supervised category-level object pose and size estimation, 2022.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common cor- ruptions and perturbations, 2019.
10

---

Published as a conference paper at ICLR 2024
Shun Iwase, Xingyu Liu, Rawal Khirodkar, Rio Yokota, and Kris M. Kitani. Repose: Fast 6d object pose refinement via deep texture rendering. 2021 IEEE/CVE International Conference on Computer Vision (ICCV), Oct 2021. doi: (0.11000ccc449222210000003 doi. doi.org//i /0. 1099 ICCVV/C92C2222200000 2021 00329.
URL http://dx.
Hiroharu Kato, Deniz Beker, Mihai Morariu, Takahiro Ando, Toru Matsuoka, Wadim Kehl, and Adrien Gaidon. Differentiable rendering: A survey, 2020.
Adam Kortylewski, He, Qing Liu, and Alan Yuille. Compositional convolutional neural networks: A deep architecture with innate robustness partial occlusion, 2020.
Taeyeop Lee, Byeong-Uke Lee, Inkyu Shin, Jaesung Choe, Ukcheol Shin, In So Kweon, and Kuk- Jin Yoon. Uda-cope: Unsupervised domain adaptation for category-level object pose estimation, 2022.
Taeyeop Lee, Jonathan Tremblay, Valts Blukis, Bowen Wen, Byeong-Uk Lee, Inkyu Shin, Stan Birchfield, In So Kweon, and Kuk-Jin Yoon. Tta-cope: Test-time adaptation for category-level object pose estimation, 2023.
Jiehong Lin, Zewei Wei, Zhihao Li, Songcen Xu. Kui Jia, and Yuanqing Li. Dualposenet: Category-level 6d object pose and size estimation using dual pose network with refined learn- ing of pose consistency. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Oct 2021. doi: 10.1 1009ficc44992..221..000 ICCV48922 2021. 00354.
URL htttp://dx. del.x.ddi.ooo www.dorgo........ooooo 1109/
Jichong Lin, Zewei Wei, Changxing Ding, and Kui Jia. Category-level 6d object pose and size estimation using supe-super deep prior deformation networks, 2022.
Mohammad Reza Loghmani,, Luca Robbiano, Mirco Planamente, Kiru Park, Barbara Caputo, and Markus Vincze. Unsupervised domain adaptation through inter-modal rotation for rgb-d object recognition. IEEE Robotics and Automation Letters, 5(4):6631 Oct 2020. ISSN 2377- 3774. doi: 10. 30C7092.
URL http:hhtp///// doi org/ LRA. 2020.
Wufei Ma, Angtian Wang, Alan Yuille, and Adam Kortylewski. Robust category-level 6d pose estimation with coarse-to-fine rendering neural features, 2022.
Eric Marchand, Hideaki Uchiyama, and Fabien Spindler. Pose estimation for augmented reality: A hands-on survey. IEFE Transactions Visualization and Computer Graphics, 22, 01 2016. doi:
20.11000/VCG..01..5555
Eitan Marder-Eppstein. Project tango. In ACM SIGGRAPH 2016 Real-Time Live!, SIGGRAPH 116 New York, NY, USA, 2016. Association for Computing Machinery. ISBN 9781450343787. doi: 001.5/2933340.29335555 URL https: //oi. org/10. 000....111/2233400110 2933550.
K.V. Mardia and P.E. Jupp. Directional Statistics. Wiley Series in Probability and Statistics. Wiley, 2009. ISBN 9780470317815. URL betps::/bookee google com/books ?id=
PTNi Cm40-MCO
Kiru Park, Timothy Patten, and Markus Vineze. Pix2pose: Pixel-wise coordinate regression of objects for 6d pose estimation. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Oct 2019. doi: 10.11999/c..22019.00776 URL http: www.:ddi...........ooooo 2CCV.2019.00. 00776.
Georgios Pavlakos, Xiaowei Zhou, Aaron Chan, Konstantinos G. Derpanis, and Kostas Daniilidis. 6-dof object pose from semantic keypoints. 2017 IEEE International Conference on Roboties and Automation (ICRA), May 2017a. doi: 10.1 10.1109/rra..2117999999 doi.org/10 1111/1099CCRR 2017. 7989233.
URL http://dx.
Georgios Pavlakos, Xiaowei Zhou, Aaron Chan, Konstantinos G. Derpanis, and Kostas Daniilidis. 6-dof object pose from semantic keypoints. 2017 IEEE International Conference Robotics and Automation (ICRA), May 2017b. doi: 10. (0.11199ficr222077779993333 URL http:/ attp://dx: /xxx doi doi.org//0.. (00rg//00.119/TCR.. 2017, 7989233.

---

Published as a conference paper at ICLR 2024
Sida Peng, Yuan Liu, Qixing Huang, Xiaowei Zhou, and Hujun Bao. Pvnet: Pixel-wise voting network for 6dof pose estimation. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2019. doi: 10.1 00.10999cvp22211.04499.. URL http: /dxx//xx//////////// doi. org/ 10. 10.1119/CVPR. 201 9. 00469.
Wanli Peng, Jianhang Yan, Hongtao Wen, and Yi Sun. Sclf-supervised category-level 6d object pose estimation with deep implicit shape representation. In Proceedings of the AAAI Conference Artificial Intelligence, volume 36, pp. 2082-2090, 2022.
Martin Runz, Maud Buffier, and Lourdes Agapito. Maskfusion: Real-time recognition, tracking and reconstruction of multiple moving objects. 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Oct 2018. doi: 10.1 10.11099/smar2211800002 URL http: / /xx. doi. 10. 1101/1111110 IISARRSSARR.... 00024..
Suvrit Sra. A short note on parameter approximation for von mises-fisher distributions: And fast implementation of i s(x). Computational Statistics, 27:177-190, 03 2012. doi: 10.1007/ 100800011-0222--1
Meng Tian, Marcelo H. Ang, and Gim Hee Lee. Shape prior deformation for categorical 6d ob- ject pose and size estimation. Lecture Notes in Computer Science, pp. 530-546, 2020, ISSN 161 doi: 10. 978-3-030-58589 32.
URL http::/dx..t. http:://xx..ci..ooo http://xx..i..0//1111110.00
Jonathan Tremblay, Thang To, Balakumar Sundaralingam, Xiang, Dieter Fox, and Stan Birch- field. Deep object pose estimation for semantic robotic grasping of household objects, 2018.
Angtian Wang, Yihong Sun, Adam Kortylewski. and Alan L Yuille. Robust object detection under occlusion with context-aware compositionallets. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12645-12654, 2020.
Angtian Wang, Adam Kortylewski, and Alan Yuille. Nemo: Neural mesh models of contrastive features for robust 3d pose estimation, 2021a.
Angtian Wang, Wufei Ma, Alan Yuille, and Adam Kortylewski. Neural textured deformable meshes for robust maalsis--y--ynthees. 2023.
Chen Wang, Danfei Xu, Yuke Zhu, Roberto Martin-Martin, Cewu Lu, Li Fei-Fei, and Silvio Savarese. Densefusion: 6d object pose estimation by iterative dense fusion. 2019 IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), Jun 2019a, doi: 10.1109/cvpr.
2019.00336. URL http::/tt//// /xx./xxxo/o// doi. www.:oii.ooooooooooooooo.o 0 http///..111////111 1119//VPRR.... 2019. 2019.00346.
He Wang. Srinath Sridhar, Jingwei Huang. Julien Valentin, Shuran Song, and Leonidas J. Guibas. Normalized object coordinate space for category-level 6d object pose and size estimation. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2019b. doi: 10. 10.11099cxp::201900000 9.0075. URL http://dss www.:dooo..gg////// www.ollg............ 1109 111111///V/ CVVR. 201 9. 00275.
Jiaze Wang, Kai Chen, and Qi Dou. Category-level 6d object pose estimation via cascaded relation
and recurrent reconstruction networks. 2021 IEEE/RS/ International Conference Intelligent
Robots and Systems (IROS), Sep 2021b. doi: 10.1109/iros51 168 01.....22..66666 URL http:
//dx. doi .rrg 10.1000000 11 09/ TROS51 68 2021 00022..96366 9636212.
Jay M. Wong, Vincent Kee, Tiffany Le, Syler Wagner, Gian-Luca Mariottini, Abraham Schneider, Lei Hamilton, Rahul Chipalkatty, Mitchell Hebert, David M.S. Johnson, Jimmy Wu, Bolei Zhou, and Antonio Torralba. Segicp: Integrated deep semantic segmentation and pose estimation. 2017 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Sep 2017, doi: 10. 2tt.:/....2....220-777. URL http: / dx doi org/ 100 1109 IROS. 2017 8206470.
Yu Xiang, Roozbeh Mottaghi, and Silvio Savarese. Beyond pascal: A benchmark for 3d object detection in the wild. In IEEE Winter Conference on Applications of Computer Vision, pp. 75-82, 2014. doi: 10. 109/WACV.0014.633 6..66601...
12

---

Published as a conference paper at ICLR 2024
Yu Xiang, Tanner Schmidt, Venkatraman Narayanan, and Dieter Fox. posecnn: A convolutional neural network for 6d object pose estimation in cluttered scenes. Robotics: Science and Systems X/V, Jun 2018. doi: 00.15600//ss.201..xv.0111, URL http: //tp::///x//// 1ttp:/xx..01..000000.5.0100 RSS. 2018 XTV. 01 9.
Jiahao Yang, Wufei Ma, Angtian Wang, Xiaoding Yuan, Alan Yuille, and Adam Kortylewski. Robust category-level 3d pose estimation from synthetic data, 2023.
Andy Zeng, Kuan- Yu, Shuran Song, Daniel Suo, Ed Walker, Alberto Rodriguez, and Jianx- iong Xiao. Multi-view self-supervised deep learning for 6d pose estimation in the amazon picking challenge. 2017 JEEE International Conference on Robotics and Automation (ICRA), May 2017. doi: 20..1099 2017.7989151
http://dd/x doi. wwt.:ooo.ooo....omm 10..0.1109IIR...
Kaifeng Zhang, Yang Fu, Shubhankar Borse, Hong Cai, Fatih Porikli, and Xiaolong Wang. Self- supervised geometric correspondence for category-level 6d object pose estimation in the wild, 2022.
Bingchen Zhao, Jiahao Wang, Wufei Ma, Artur Jesslen, Siwei Yang. Shaozuo Yu, Oliver Zendel, Christian Theobalt, Alan Yuille, and Adam Kortylewski. Ood-cv-v2: An extended benchmark for robustness to out-of-distribution shifts of individual nuisances in natural images, 2023.
Xingyi Zhou, Arjun Karpur, Linjie Luo, and Qixing Huang. Starmap for category-agnosto keypoint and viewpoint estimation, 2018
13

---

Published as a conference paper at ICLR 2024
A APPENDIX
A.1 PROBLEM SETUP
Our problem is set up as Source-Free, Image Only Unsupervised Domain Adaptation for Category Level 3D Pose Estimation. We have source 3D pose estimation model which has been trained annotated source domain data (considered in-domain or IID). However, during adaptation, we do not have the source data. During the adaptation and evaluation time, we only have images of the OOD larget domain. We don't tt have any 3D CAD model, LIDAR data, depth data or keypoint annotations for the OOD target domain data. Models are expected to adapt the source model in an unsupervised manner with the target images only. Our image data and pre- pre-processing setup follow previous 3D) pose estimation works like Zhou et al. (2018); Wang et al. (2021a).
Previous Semi-supervised 3D Unsupervised Domain Adaptation Works. There have been a few attempts in solving the domain adaptation problem in the field 3D pose estimation; however, most domain adaptation models utilize some kind of 3D data like depth map, point cloud, or CAD models in the target domain. Some works have also tried to do semi-supervised learning using synhetic data (Yang et al., 2023; Loghmani et al., 2020) (Synthetic to Real), but in addition to requiring ground truth 3D amnotations in the target domain or some target domain annotated data, they may not be applicable to real world OOD scenarios (Lee et al., 2022). Such method also seems to be partially successful since they may adapt from a specific Synthetic to Real domain but cannot trivially extend to realistic nuisance-ridden real world data.
A.2 QUALITATIVE RESULTS
Figure shows qualitative results from our method as compared ground truth and another fea- ture level generative model (Wang et al., 2021a) in different kinds of real world nuisances, partial occlusion and synthetic corruptions.
Selective Vertex Adaptation and advantages w.r.t. Occlusion The presence of partial occlusion in addition to real synthetic nuisance factors in the target domain further complicates the already difficult UDA problem. Methods like global pseudo-labeling which utilize the entire neural mesh model will suffer immensely since now the parts of the object are individually affected with some obscured and others visible. Our selective vertex adaptation is helpful in this complex UDA scenario, we focus independently shown in our experiments 4.1 and Figure 3 performs exceedingly well even in the presence of partial occlusion in the noise-ridden target domain.
the adaptation of individual neural vertices (and the model) and, as
A.3 PROOF OF THEOREM 1
We first restate the theorem,
Theorem A.1 A target domain Xy satisfying assumption 3.3, elicits another target domain Xy" such that each sample in X7" satisfies the global-pseudo labelling constraint (f Cr) > 8, Vr E (1,2,.., Asymptotically with the size of the domain, N7 -> Nx.
Proof: The CNN feature corresponding to vertex feature is fi-r. Given a vertex K K-partition, for each subset of vertex features Lx. let f denote the pre-image of these vertex features. As vertex feature distributions are independent, there exists such pre-image for each vertex feature subset. There may be uneven support in the individual pre-image sets f-h but the pre-image sets can be sampled such that the resulting distribution of vertex features matches pls for given domain X. The same pre-image argument extends to CNN features and the corresponding inputs that produce them. Let -1 denote the pre-image of the required CNN features which elicit the distribution plx over each vertex feature subset Ig-
For given target domain X'T satisfying assumption 3.3 and its corresponding vertex K- K-partition, we compute the ko-subsets. Let the pre-images of vertex features for these kö-subsets be denoted by f-k and the corresponding pre-image of the CNN features be denoted by Xy By construction, this pre-image is the target domain X7 such that each vertex feature C, produced for samples in
14

---

Published as a conference paper at ICLR 2024
this domain satisfies the global labell-g constraint. This hypothetical domain may not be easy to access but as asymptotically II, pf- converges to Pr. this pre-image Xy the given By itself.
converges to
PXTT
adapiatioo Approsimated
Figure 4: The elicited target distribution P(Xç.)) found by SVA may not be precisely the same the true target distribution (( but asymptotically (shown by arrows) it tends to the true distribution and the same happens to the adapted source model.
A.4 METHOD MOTIVATION
Figure gives an intuition into our method motivation. Our method is intuitively motivated by two simple, yet crucial intuitions. Local Part Pose Ambiguity and Local Part Robustness. Local part pose ambiguity refers to the pose ambiguity that occurs when we can see only part of an object. This is reflected in Figure car object. If we could only see the right headlight (in the red box) of the car, there are many possible global poses that the car could be in. This is often the problem is 3D pose estimation, but we utilize this pose ambiguity to our advantage. We realize that since the object parts may be visible from numerous object poses, we can still update our model parameters, which correspond to these parts even when the global pose estimation is incorrect. Our second observation deals with Robust Local parts. We observe and provide evidence in Figures 1, 5, 8 and 9 of the polar histograms of some vertex features (which roughly correspond to parts of an object) being less affected or robust in the OOD data. As is trivially observed in the real world, there are some parts of objects that are less affected or even affected by certain domain changes, and we can leverage this local robustness. For example, the right headlight in Figure is part of an object which may not change much even when encountering domain nuisances like texture, weather, and context changes headlights in the source domain.
and will still be fairly similar to features corresponding
Therefore, we are able to utilize the aforementioned observations to adapt our source model. Given local robust parts that will have high similarity to source counterparts and the fact that local parts are visible for large number of global object poses, we can focus on these individual local parts and utilize them to update our model.
A.5 ABLATION STUDY AND METHOD ANALYSIS
Selective Vertex Feature Update We can ablate the efficacy of our selective vertex feature update method and justify our assumption regarding robust parts of an object by comparing the (azimuth) polar histograms in Figure 5. The upper histogram represents the ratio of vertex feature activated (over visible vertex features) within high similarity threshold on the OOD target data with source model. It shows that even though they may not be enough robust vertex feature adaptation for global pseudo labeling set-up, there are still individual vertex that correspond to parts of an object which can still be detected robustly. This is simple observation in the real world as well, where humans identify objects new circumstances or context by drawing upon their knowledge and recognition of individual parts which have not changed much.
False Positives in Vertex-Featurr Similarity In our experiments, false positives were not major issue. For example, the average ratio of false positives in Figure 5 is 2% and has an insignificant affect on the adaptation. We believe this may be a number of reasons: (1) Our CNN backbone is trained and adapted in contrastive manner where all unique vertex features are encouraged to be different from one another and from the background clutter features (2) While adaptation, we constrain the similarity measure with high thresholds as well require lower bound of vertices being activated. (3) Pretrained CNN like Resnet50 have shown to have some invariance to 3D pose and other nuisances, which also contributes the robustness of the similarity measure.
15

---

Published as a conference paper at ICLR 2024
335
45*
tampues
40
125
Ra
280
26
Pose Error degrees
270"
235
350-
300
220
313"
50 100 125 150
375
Figure 5: (Upper left) Azimuth Polar histogram represents the ratio of visible neural mesh vertices which are detected (for OOD-CV bus category), with high similarity threshold(= 0.9) to feature vectors, by the source model. As we hypothesize in this work, the average number of robust vertices in an image varies from + 20%. The ratio of false positives 2% per vertex feature negligent and its effect is minimized due to the sampling of features from multiple image feature maps. (Top right) The cumulative pose error histogram for the source model evaluated on the OOD-CV (Com- bined Nuisance) target domain. (Lower left) Polar histogram of robust neural mesh vertices for our unsupervised adapted model which shows the major increase in adapted and robust vertex features. (Lower right) shows histogram of pose errors for our adapted model and shows how the pose error has been minimized for the target domain.
Multi-Pose Initialization Figure 6 shows the estimated pose by our source model when evaluated on sample from OOD-CV (Zhao et al., 2023) dataset. We have three highly likely solutions learned by optimizing the reconstruction loss between the rendered feature map and the image fea- ture map when the initial pose were randomly initialized.
Quality of Vertex Features after Selective Adaptation Figure 5 shows before and after results of the average ratio of correctly matched visible vertices per image in the target domain. The polar histogram is defined over the azimuth of the object pose in the target dataset. We can significant improvements where the correctly detected ratio of vertices increases from 5 20% to 50 70% which is enough estimate the pose of an object in image. This improvement is reflected in the pose error histograms in Figure 5. Similar improvements for other classes (in OOD-CV) can be seen Figures
Ablation Analysis for Clutter Model Hyperparameter N As can be observed from Table 4, our method is not sensitive to the change in the number of clutter models. This could be attributed to the fact that our similarity threshold can be construed as a clutter model with uniform distribution. We do see small improvements as the number clutter models drastically increased, which comes at high computational cost. Our default value of this hyperparameter is 5.
16

---

Published as a conference paper at ICLR 2024
Estimated Poses by source model from initialization
Ground Truth Pose
Figure 6:
Neural Sender-and-Compee optimized pose estimates for single image from target do- main (OOD-CV) with different initialization for the source model
Table Ablation
Analysis clutter model number hyperparameter
N (Clufter)
Median Error
T/18 Acuracy
x/6 Accuracy
3.01
90.1
95.1
3.03
91.1
94.4
20
2.99
92.3
95.5
35
2.79
91.7
94.5
A.6 ADDITIONAL RELATED WORKS
Although there has not been any recent work on source free and image-only unsupervised domain adaptation for 3D pose estimation, there have been few works which have worked on self-supervised, zero-shot or few shot learning with some 3D data or multi-view availability. Goodwin et al. (2022) utilize a pretrained DINO Vision Transformer, large-scale pretraining, depth estimates and multiple views of the target object to do zero shot category level pose estimation using the idea of semantic keypoint correspondence between pairs of images. Zhang et al. (2022); He et (2022) and Lin al. (2022) all utilize mesh model priors for self-supervised approach involving 3d-2d correspondence with the mesh model or point cloud, image registration and deformation. All of these methods rely on local and global geometrie constraints and correspondence to varying extents.
A.7 TRAINING AND IMPLEMENTATION DETAILS
taken from their official codebase. We utilize the imple- mentation of Ma et al. (2022) for the Resnet50-Genent trained with data from all categories at the same time, which could be factor for better OOD bustness relative to other baseline models. Yang et al. (2023) is semi-supervised setup where they create synthetic data enable pose estimation with real data. The synthetic data creation code publicly available; we utilize their reported OOD-CV (Zhao et al., 2023) results in our experimental results.
Our baseline model implementations
model. The general Resnet50 model is
During inference, it takes our method 0.33 seconds to evaluate one sample on NVIDIA RTX 2060 GPU.
A.7.1
SOURCE MODEL
Our source model similar to previous works like Wang et al. (2021a; 2023); Ma et al. (2022) category-level pose estimation using neural feature level render and compare. These methods
17

---

Published as a conference paper at ICLR 2024
Background
Loss Landscape
Outter Festure
CAN baskbnn
Alle Declusion 2
BACH 21 Biovutio
Loss
Final Pose Prediction
03
Render
Neural Mash Model
Foreground Score
m: cameeaa parameter
Backward
Figure 7: Source Model: Figure taken from Wang et al. (2021a) to reflect the training of our source model
are themselves 3D extensions of feature generative models such as Kortylewski al. (2020). Our source model defines a probabilistic generative model of normalized real-valued feature activations F conditioned 3D neural mesh representation Dt. The neural mesh model aims to capture the 3D information of the foreground objects. For each object category y, the source model defines neural mesh M.s (V,C), where V {V, E R is the set of vertices of the mesh and = (C, E R is the set of learnable features, i.e. neural features. denotes the index of the vertices. R is the total number of vertices. We also define clutter model B {B.,) >. describe the backgrounds. N is a prefixed hyperparameter. For a given object pose or camera viewpoint g. we can render the neural mesh model Wis into feature map using (differentiable) rasterization (Kato et al., 2020). We can compute the object likelihood of a target feature map F E R f2 as
p(F) B)
II p(f 20, #EFG
II p(fs BB) FEBG
where FG and BG denote the foreground and background pixels, respectively. FG is set of all the positions in the 2D lattice P covered by the mesh 21 and BG are the positions that are not. Unlike (Ma et al., 2022; Wang et al., 2023), we define (V,. a von Mises Fisher (vMF) distribution with mean C, and concentration parameter Hi, Z[K,] is
= 2[Kr] exp(frr,ff nr,e) as
normalization constant and is defined (2m)d/233
/2/221
(x) (Mardia & Jupp, 2009) where J denotes
the modified Bessel function of the first kind of order and d is the dimension. P(filb) is the background distribution, also defined as a vMF distribution. The correspondence between the image feature and the vertex features C, calculated using rendering assuming perspective projection.
The concentration parameter can be calculated using the simple approximation = R(d-R") (Sra,
2012) where R |||||| and F = + Σn for series of N independent unit vectors However, for our source model, we find that fixing i. to constant is sufficient learn a good supervised model. The training objective function of the source model is Luunce Lneurat + Letuteer + Leuntrastive- Lneural maximizes the likelihood between the neural features C, and the corresponding CNN features fi for each vertex Lelater maximizes the likelihood between the B and the CNN features fv- Lcomtrasivee defines two constrastive terms which encourage vertex features to be different from one another and the background.
Figure gives an outline our source model which is similar to the feature-level neural render and compare methodology introduced in (Wang al., 2021a; Ma al., 2022). The training objective function of the source model defined in (Wang et al., 2021a; Ma et al., 2022) as follows:
Lsoune Lncural + Letateer + Lcontrastive
R Σ r=1
Lacural
log P(fr-rnC.)) Celutter
Σ PEM
log P(fe1)),
(8)
18

---

Published as a conference paper at ICLR 2024
R
R N
conttrastive = Σ log P(fi-r/C)) + Σ Σ
log P(
(9)
Lecural maximizes the likelihood between the neural features C, and the corresponding CNN fea- tures fi-tr for each vertex r. Leluter maximizes the likelihood between the B and the CNN features fe. Ccontrasivee defines two constrastive terms which encourage vertex features to be different from one another and the background. The first term pertains to the contrastive learning between differ- ent vertices, while the second term focuses on the contrastive learning between vertex and the background. Notably, we can do inference with the source model with only RGB image which one the major reasons to utilize these class of models as our source model. However, despite earlier attempts (Yang et al., 2023; Wang et al., 2023) these models cannot be trivially extended to source-free free unsupervised learning.
19

---

Published as a conference paper at ICLR 2024
St 62
S.
223
313
222
broycle (Befors Adaptat on)
bicycle (Aler Adeptation)
2707
boal (Buliew Aclaptation)
boat (Al Au Aution)
R3
R.
1007
1204
1254
235 115
255
15+
2707
2127
Figure 8: Part Azimuth Polar histograms representing the ratio visible neural mesh vertices which are robustly detected for different categories of OOD-CV (Zhao et al., 2023) dataset before and after adaptation using our method. As explained in 5, (left column) we can see the ratio of robustly detected vertices in the target domain using the source model which provides strong evidence towards our hypothesis regarding locally robust neural vertex features irrespective of the correctness of the global object pose. Our method, 3DUDA, leverages these locally robust parts and adapts the model in an unsupervised manner. The right column reflects the drastic increase of ratio of robustly detected vertex features post adaptation.
20

---

Published as a conference paper at ICLR 2024
(Datore Adactation)
Char Ulter Acagpation)
220
20
100
19
30"
Tran (Aller Adkotatict
122
Uan CAtre AJ HAAM!
desnglatic IAN AdAILUEN
sala Alaplctunt
100
229
irttariita (Botore Adaclation
mntorbtia (6ttr Adaghatonn
205
2717
Figure 9: Part 2: Azimuth Polar histograms representing the ratio of visible neural mesh vertices which are robustly detected for different categories of OOD-CV (Zhao al., 2023) dataset before and after adaptation using our method.
21

---

Published as a conference paper at ICLR 2024
A.8 CATEGORY- WISE EXPERIMENTAL RESULTS
We list the results of our unsupervised pose estimation results with category-wise separation in this section. Due the sheer number of experimental results, focus main comparison NeMo (Wang et al., 2021a) which has been shown to be robust to certain OOD scenarios and occlu- sion (Wang et al., 2023).
A.9 EXTENDED MAIN RESULTS
Table 5: Unsupervised 3D pose estimation results for Pascal3d+ - Corrupted-Pascal3DD
(a)
(Metrics #'6 Accuracy (E), 1),18 Accuracy (fx) Median Error (Er))
Er
Er
Er
18
Shot Noise
Impulse Noise
Defocus Blur
Geneeale
39.3
8.9
52.1
36.6
8.1
55.7
45.6
10.3
44.6
ViT-b-16
39.1
8.8
55.4
36.7
8.4
54.9
45.2
13.1
41.9
DMNT
51.2
21.1
31.1
50.1
25.3
33.3
72.1
44.5
16.1
P3D
48.9
19.7
33.7
48.6
18.9
37.9
71.6
40.7
17.8
NeMo
50.6
25.3
35.0
45.4
22.2
39.4
72.9
41.8
16.0
Ours
85.9
62.0
9.0
84.0
58.5
10.1
87.8
64.6
8.0
Motion Blur
Zoom Blur
Snow
Res50-General
42.3
9.2
49.9
59.6
18.4
41.0
55.4
16.2
40.4
ViT-b-16
46.1
8.7
49.8
53.9
12.1
40.7
54.3
12.8
40.3
DMNT
70.7
41.1
16.9
70.9
42.1
16.6
72.2
44.6
16.6
P3D
69.9
37.6
17.9
70.0
40.0
18.6
72.0
41.9
17.8
NeMo
69.7
39.2
18.7
69.0
39.7
19.1
69.9
40.1
18.9
Ours
88.0
63.8
8.3
87.9
65.1
8.1
87.7
64.0
8.2
Fog
Contrast
Elastic Transform
Res50-General
65.3
19.6
31.5
61.5
19.3
38.1
45.6
8.6
43.7
ViT-b-16
65.7
21.8
29.9
61.9
21.5
37.7
41.3
6.3
47.9
DMNT
86.0
60.1
9.7
77.7
45.9
12.9
77.9
51.1
13.7
P3D
85.9
58.4
9.9
76.5
44.8
13.6
76.1
48.9
14.3
NeMo
85.5
59.0
9.5
74.5
43.8
14.7
77.4
50.3
13.8
Ours
88.7
65.6
7.8
88.8
66.7
7.6
88.2
64.4
8.1
A.9.1 OOD-CV (ZHAO AL., 2023)
Tables 7 15 16 10 11 12 13 14 17 18 show per category results for Unsupervised Domain Adaptation for the OOD-CV (Zhao et al., 2023) dataset, where we have real world nuisances in the target domain.
A.9.2 PASCAL3D+ CORRUPTED-CORRUPTED PASCAL3D+
Tables 19-50 show category wise results of Pascal3D+ Corrupted Pascal3D+ setup where add synthetic noise like glass blur, elastic transform, rain, fog, gaussian noise, etc. to the evaluation data.
22

---

Published as a conference paper at ICLR 2024
Table 6: Unsupervised 3D pose estimation results for Occlusion and Extreme UDA setup
(a) OccL1/L2: Real Nuisanee (OOD-CV (Combined)) Occlusion (Level1/Level2) (b) 0OD+SN/GB: Real
Nuisance (OOD-CV)+ Synthetic
Syynthhci Noise (Speckic Noise/Glass Blur) L./L224Speckle: Real Nuisance (OOD- CV) + Occlusion (L1/L2) Synthetic Noise (Speckle Noise)
OccL1
OceL2
OOD+SN
0 Acc.
1 Acc.
Acc.
Acc.
Acc.
18 Acc.
18
Res50-General
30.1
10.9
18.9
4.9
30.1
8.8
MaskRCNN
28.9
9.8
17.7
4.5
31.2
9.7
DMNT
32.4
11.9
25.3
7.1
35.4
10.2
P3D
29.6
11.4
22.1
6.9
33.1
11.1
NeMo
30.6
10.2
24.1
6.6
32.7
10.2
Ours
84.6
77.1
78.7
70.4
80.5
63.0
OOD+GB
L1+Speckle
L2+Speckle
Res5O-General
30.3
10.1
16.7
3.3
11.8
1.5
MaskRCNN
28.9
9.9
15.4
3.1
12.5
1.8
DMNT
32.3
10.2
18.9
3.9
16.8
3.1
P3D
30.1
9.3
17.7
3.4
15.9
2.6
NeMo
29.6
9.5
18.6
3.4
15.1
2.7
Ours
77.7
65.9
69.4
50.4
60.6
38.9
Table 7: Unsupervised 3D pose estimation results for OOD-CV (combined) Part
aeroplane
bicycle
boat
bus
car
Method
Metric
Wang al. (2021a)
7/ Acc.
45.6
48.5
39.2
66.0
68.8
x 18 Acc.
25.0
20.8
14.7
47.4
52.7
Median Error
36.35
31.71
48.0
11.6
9.22
Ours
Acc.
98.6
90.5
86.8
92.4
97.1
x 18 Acc.
94.3
88.2
75.0
88.0
92.9
Median Error
1.45
1.44
3.6
1.75
1.82
Table Unsupervised 3D pose estimation results for OOD-CV (combined) Part
chair
diningtable
motorbike
sofa
train
Method
Metric
Wang et al. (2021a)
6 Acc.
42.6
57.6
57.2
73.2
77.8
118 Acc.
14.8
19.0
23.5
26.1
60.5
Median Error
42.63
23,65
24.16
17.89
7.06
Ours
T Acc.
93.0
96.5
96.0
97.7
92.9
18 Acc.
80.8
88.2
89.2
91.6
89.5
Median Error
3.3
1.82
1.73
1.7
2.59
Table 9; Unsupervised 3D pose estimation results for OOD-CV (shape) Part 1
aeroplane
bicycle
boat
bus
car
Method
Metric
Wang et (2021a)
n 6 Acc.
41.4
58.6
52.8
76.7
76.9
n 18 Acc.
14.1
36.9
13.9
33.3
50.0
Median Error
36.0
15.77
26.34
17.08
10.95
Ours
6 Acc.
96.1
91.9
94.4
90.0
100.0
" 18 Acc.
92.2
91.0
83.3
83.3
84.6
Median Error
1.72
1.43
2.35
2.34
3.21
23

---

Published as a conference paper at ICLR 2024
Table 10: Unsupervised 3D pose estimation results for OOD-CV (shape) Part 2
chair
diningtable
motorbike
sofa
train
Method
Metric
Wang et al. (2021a)
π 1,6 Acc.
48.3
52.8
51.2
78.9
50.0
11 Acc.
15.2
12.4
25.6
26.3
22.2
Median Error
33.59
27.61
28.87
16.31
26.83
Ours
T 66 Acc.
92.7
93.3
93.0
96.8
77.8
t 18 Acc.
74.8
70.8
76.7
83.2
72.2
Median Error
3.58
5.62
2.95
2.42
3.73
Table 11: Unsupervised 3D pose estimation results for OOD-CV (weather) Part
aeroplane
boat
bus
Method
Metric
Wang et al. (2021a)
x Acc.
54.0
45.8
64.9
68.6
x 18 Acc.
38.8
22.3
50.6
56.5
Median Error
21.52
35.39
8.5
7.68
Ours
x 16 Acc.
99.3
85.7
97.4
98.8
x 18 Acc.
96.9
76.9
89.6
96.6
Median Error
2.48
2.29
1.35
1.49
Table 12: Unsupervised 3D pose estimation results for OOD-CV (weather) Part 2
chair
motorbike
sofa
train
Method
Metric
Wang al. (2021a)
T Acc.
44.1
63.9
87.5
81.4
π T\,1 11 Acc.
17.6
23,9
62.5
68.3
Median Error
48.9
20.68
5.42
5.62
Ours
T Acc.
97.1
97.8
100.0
89.8
m 18 Acc.
91.2
94.4
87.5
85.6
Median Error
2.7
1.65
1.62
1.51
Table 13: Unsupervised 3D pose estimation results for OOD-CV (texture) Part
aeroplane
bicycle
boat
bus
car
Method
Metric
Wang et al. (2021a)
6 Acc.
54.5
57.1
45.3
76.9
77.1
18 Acc.
28.7
13.1
14.0
57.9
56.2
Median Error
24.3
24.15
44.11
8.01
7.94
Ours
6 Acc.
99.0
97.6
88.4
98.3
100.0
18 Acc.
98.0
94.0
76.7
95.9
97.9
Median Error
1.68
1.68
2.75
1.68
2.61
Table 14: Unsupervised 3D pose estimation results for OOD-CV (texture) Part
chair
diningtable
motorbike
sofa
train
Method
Metric
Wang et al. (2021a)
m Acc.
36.2
65,4
59.3
67.0
72.3
18 Acc.
12.1
24.1
28.7
25.3
56.3
Median Error
62.55
18.07
22.35
18.77
7.52
Ours
T' Acc.
94.8
99.0
97.2
96.7
97.5
m r Acc.
86.2
94.8
92.6
92.3
95.8
Median Error
3.01
1.66
1.99
2.16
1.61
24

---

Published as a conference paper at ICLR 2024
Table 15: Unsupervised 3D pose estimation results for OOD-CV (context) Part
aeroplane
boat
bus
Method
Metric
Wang et al. (2021a)
16 Acc.
34.1
34.3
65.4
69.4
Acc.
12.1
10.1
50.7
45.8
Median Error
47.11
49.27
9.61
11.38
Ours
T 6 Ace.
98.8
89.9
91.9
97.2
T 18 Acc.
94.8
76.9
87.5
91.7
Median Error
1.61
3.64
1.67
2.41
Table 16: Unsupervised 3D pose estimation results for OOD-CV (context) Part 2
chair
diningtable
motorbike
sofa
train
Method
Metric
Wang et al. (2021a)
T Acc.
43.7
49.0
55.2
73.1
84.9
18 Acc.
18.3
18.3
21.8
25.4
57.0
Median Error
35.34
30.4
23.36
18.5
7.52
Ours
Acc.
94.4
98.1
94.3
99.5
94.2
18 Acc.
90.1
90.4
82.8
96.4
91.9
Median Error
3.62
2.12
1.9
1.56
1.57
Table 17: Unsupervised 3D pose estimation results for OOD-CV (pose) Part 1
aeroplane
bicycle
boat
bus
Method
Metric
Wang et al. (2021a)
T T/ Acc.
7.1
15.1
12.7
33.3
53.8
t 18 Acc.
0.0
1.6
0.0
11.1
32.7
Median Error
107.37
81.41
113.72
48.94
17.36
Ours
t 6 Acc.
96.4
73.8
79.4
80.0
92.3
t 18 Acc.
64.3
69.8
58.7
71.1
71.2
Median Error
8.74
3.59
7.39
3.01
Table 18: Unsupervised 3D pose estimation results for OOD-CV (pose) Part 2
chair
diningtable
motorbike
sofa
train
Method
Metric
Wang et al. (2021a)
6 Acc.
23.1
20.0
25.0
0.0
65.0
18 Acc.
7.7
0.0
7.1
0.0
45.0
Median Error
119.05
66.66
76.21
88.57
10.8
Ours
6 Acc.
76.9
80.0
100.0
100.0
95.0
a 18 Acc.
53.8
80.0
78.6
100.0
90.0
Median Error
6.54
3.55
2.89
1.49
2.6
25

---

Published as a conference paper at ICLR 2024
Table 19: Unsupervised 3D pose estimation for Corrupted--ascal3DD (gaussian noise) Part
plane
bicycle
boat
bottle
bus
car
Method
Metric
(Wang al., 20)
Accuracy
16.7
29.9
13.1
17.1
47.2
80.3
18 Accuracy
1.7
3.3
1.9
0.7
23.5
64.0
Median Error
84.23
60.37
86.0
45.71
37.68
7.4
Ours
Accuracy
79.8
71.9
59.9
86.9
95.7
97.9
Accuracy
46.3
16.6
32.1
48.1
91.0
93.8
Is
Median Error
11.04
20.2
20.08
10.43
3.22
3,47
Table 20: Unsupervised 3D pose estimation for Corrupted-Pascal33 (gaussian noise) Part
chair
table
bike
sofa
train
tv
Method
Metric
(Wang et al., 2021a)
Accuracy
42.3
28.3
31.3
34.5
46,6
69.1
Accuracy
8.5
2.5
3.9
6.4
18.6
19.8
Median Error
35.04
48.16
49.55
50.1
34.37
21.17
Ours
Accuracy
85.8
78.1
77.6
89.9
89.0
79.1
Accuracy
50.4
55.9
23.6
53.7
75.5
39.3
Median Error
9.92
8.63
17.16
9.22
5.44
12.86
Table 21: Unsupervised 3D pose estimation results for Corrupted-aascalDDD (shot noise) Part
plane
bicycle
boat
bottle
bus
Method
Metric
(Wang al., 2021a)
Accuracy
23.4
33.5
22.0
62.6
86.9
I8 Accuracy
5.1
5,3
3.5
2.0
37.4
71.4
Median Error
69.94
49.79
75.4
47.59
15.39
6.91
Ours
Accuracy
79.6
74.9
63.5
87.3
94.9
98.2
G
I3 Accuracy
49.7
23.6
35.5
48,9
90.8
94.3
Median Error
10.1
17.4
17.98
10.61
2.94
3.33
26

---

Published as a conference paper at ICLR 2024
Table 22: Unsupervised 3D pose estimation results for Serruteed-Pascl3DD. (shot noise) Part 2
chair
table
bike
sofa
train
tv
Method
Metric
(Wang et al., 2021a)
Accuracy
42.9
39.1
39.9
43.1
53.4
71.5
Accuracy
10.1
6.3
3.9
8.5
21.7
19.5
1R
Median Error
35.53
39.29
37.74
34.38
25.59
20.39
Ours
Accuracy
88.1
80.6
83.7
92.0
89.8
82.0
&
Accuracy
53.4
62.3
32.2
54.8
77.6
42.0
18
Median Error
9.53
7.35
14.39
9.06
5.22
12.28
Table 23: Unsupervised 3D pose estimation results for Corrupted-Pascal3D++ (impulse noise) Part
plane
bicycle
boat
bottle
bus
car
Method
Metric
(Wang et al., 2021a)
Accuracy
17.6
32.7
12.9
21.7
50.6
82.5
Accuracy
2.4
3.6
1.6
1.1
25.8
66.9
Median Error
80.9
50.67
83.49
49.34
29.27
6.94
Ours
Accuracy
80.7
73.0
57.4
85.1
93.4
97.7
Accuracy
46.7
19.1
29.2
44.6
88.0
93.1
Median Error
11.05
20.03
21.96
11.35
3.25
3.46
Table 24: Unsupervised 3D pose estimation results for Corrupted-Pascal3D-- (impulse noise) Part
chair
table
bike
sofa
train
tv
Method
Metric
(Wang et al., 2021a)
Accuracy
40.3
25.9
30.8
43.1
51.2
69.2
Accuracy
7.5
2.2
3.7
8.0
19.9
18.0
Median Error
37.0
47.17
48.24
36.48
28.19
20.42
Ours
Accuracy
88.5
76.8
79.0
89.1
88.4
81.0
&
IS Accuracy
53.0
55.3
24.2
52.2
74.4
40.4
Median Error
9.37
8.96
16.51
9.6
5.44
12.82
27

---

Published as a conference paper at ICLR 2024
Table 25: Unsupervised 3D pose estimation results for Coruptedd-aascal344 (defocus blur) Part
plane
bicycle
boat
bottle
bus
car
Method
Metric
(Wang al., 202
Accuracy
52.4
60.9
44.6
83.0
69.2
89.7
18 Accuracy
15.9
16.9
11.1
40.7
46.8
74.9
Median Error
28,41
22.27
35.9
12.14
11.25
6.57
Ours
Accuracy
84.8
81.6
71.8
88.4
94.5
98.6
Accuracy
51.9
31.8
43.3
53.0
91.5
94.6
Is
Median Error
9.67
15.72
12.11
9.52
3.06
3.18
Table 26: Unsupervised 3D pose estimation results for Corrupted-Paseee Pasiedd3- (defocus blur) Part
chair
table
bike
sofa
train
tv
Method
Metric
(Wang et al., 2021a)
Accuracy
82.6
71.6
64.0
91.3
57.6
82.9
T3 Accuracy
43.5
44.4
14.6
46.3
33.7
37.0
Median Error
10.99
11.98
22.52
10.73
18.51
13.62
Ours
Accuracy
89.7
79.5
84.3
94.4
89.1
83.8
Accuracy
60.3
62.3
32.2
61.4
78.1
43.3
Median Error
8.63
7.24
13.89
8.12
4.9
11.82
Table 27: Unsupervised 3D pose estimation results for Corrupted-Paseal3DD> (glass blur) Part 1
plane
bicycle
boat
bottle
bus
car
Method
Metric
(Wang al., 2021a)
Accuracy
24.2
31.9
28.5
78.2
53.6
75.8
I8 Accuracy
4.1
4.0
6.0
31.1
22.0
52.4
Median Error
63.83
54.77
59.95
15.43
25.27
9.43
Ours
Accuracy
80.4
75.7
69.0
88.2
93.6
97.7
G
I3 Accuracy
47.7
24.3
39.2
49.1
89.1
93.8
Median Error
10.67
17.37
14.06
10.24
3.34
3.39
28

---

Published as a conference paper at ICLR 2024
Table 28: Unsupervised 3D pose estimation results for Corrupted-Pascal3DD (glass blur) Part 2
chair
table
bike
sofa
train
tv
Method
Metric
(Wang et al., 2021a)
Accuracy
70.0
57.2
30.8
87.2
36.0
78.7
9.
Accuracy
29.8
28.5
4.2
42.5
14.4
24.2
13
Median Error
15.75
21.99
46.19
11.66
116.73
17.54
Ours
Accuracy
88.7
84.0
81.8
95.2
86.8
82.3
Accuracy
58.1
63.7
29.6
62.8
71.0
44.1
18
Median Error
8.86
7.26
15.76
8.17
5.75
11.79
Table 29: Unsupervised 3D pose estimation results for Corrupeed-aaseee Pascal3D+ (motion blur) Part
plane
bicycle
boat
bottle
bus
car
Method
Metric
(Wang et al., 2021a)
Accuracy
54.6
44.7
39.0
77.4
69.9
92.4
Accuracy
18.6
10.2
10.4
19.8
40.6
78.5
Median Error
26.85
35.07
41.92
17.82
13.07
5.73
Ours
Accuracy
85.1
80.2
71.3
87.4
94.5
98.7
Accuracy
52.6
29.1
43.9
46.3
90.6
94.8
Median Error
9.43
16.32
12.26
11.02
3.03
3.34
Table 30: Unsupervised 3D pose estimation results for Cormpted-Pascaa3DDD (motion blur) Part 2
chair
table
bike
sofa
train
Method
Metric
(Wang et al., 2021a)
Accuracy
80.2
64.4
42.1
90.4
59.8
79.7
T8 Accuracy
40.7
35.0
7.2
41.7
39.0
35.3
Median Error
12.08
15.89
37.85
11.36
16.12
14.19
Ours
G Accuracy
89.5
82.5
82.0
94.9
90.2
82.9
IS Accuracy
53.8
65.3
29.3
59.8
77.6
42.2
Median Error
9.24
7.32
15.13
8.26
5.14
11.74
29

---

Published as a conference paper at ICLR 2024
Table 31: Unsupervised 3D pose estimation results for Corrupted- Pasheed-- (zoom blur) Part
plane
bicycle
boat
bottle
bus
car
Method
Metric
(Wang al., 202
Accuracy
58.9
40.3
40.6
74.8
70.7
92.0
18 Accuracy
22.8
10.1
16.2
27.3
45.3
76.9
Median Error
22.51
40.54
42.25
16.8
11.24
6.15
Ours
Accuracy
86.1
80.2
66.6
87.8
94.0
98.5
Accuracy
57.1
29.3
40.2
54.2
91.5
94.8
Is
Median Error
8.56
16.52
13.6
9.21
2.97
3.25
Table 32: Unsupervised 3D pose estimation results for Corupted-Pasceee Pascal3D+ (zoom blur) Part
chair
table
bike
sofa
train
tv
Method
Metric
(Wang et al., 2021a)
Accuracy
78.9
62.5
36.0
86.7
63.0
77.3
T3 Accuracy
37.2
36.0
7.1
38.9
38.4
26.6
Median Error
12.59
17.87
41.47
12.17
14.46
16.68
Ours
Accuracy
91.3
84.0
82.8
94.6
91.8
82.3
Accuracy
58.5
66.3
30.1
59.9
80.6
43.0
Median Error
8.55
6.84
15.3
8.02
5.01
12.09
Table 33: Unsupervised 31) pose estimation results for Corrupted-Pascal. (snow) Part 1
plane
bicycle
boat
bottle
bus
Method
Metric
(Wang et al., 2021a)
Accuracy
52.6
61.9
34.7
70.0
82.3
94.6
Accuracy
20.4
12.9
10.1
21.8
63.9
83.4
Median Error
27.74
24.37
51.79
18.92
7.01
5.33
Ours
Accuracy
86.6
80.9
69.0
87.1
94.9
98.7
&
IS Accuracy
55.3
29.9
40.4
47.3
91.2
95.0
Median Error
8.97
16.29
13.45
10.62
2.73
3.19
30

---

Published as a conference paper at ICLR 2024
Table 34: Unsupervised 3D pose estimation results for Corrupted- P (snow) Part 2
chair
table
bike
sofa
train
tv
Method
Metric
(Wang al., 2021a)
Accuracy
72.3
58.3
57.4
72.3
70.7
75.2
Ts Accuracy
26.7
27.1
14.0
28.5
50.5
23.7
Median Error
16.36
21.16
25.11
15.83
9.71
17.64
Ours
Accuracy
90.1
81.5
84.0
92.9
88.8
82.0
IR Accuracy
55.3
62.4
34.7
58.2
77.3
44.6
Median Error
9.02
7.24
13.87
8.67
4.69
11.59
Table 35: Unsupervised 3D pose estimation results for Corrupted- Pascal3D+ (froost) Part
plane
bicycle
boat
bottle
bus
car
Method
Metric
(Wang et al., 2021a)
Accuracy
57.6
65.9
39.0
77.0
86.8
95.8
Accuracy
23.1
17.8
12.6
30.7
70.9
85.6
Median Error
22.75
21.52
43.77
15.29
5.84
5.08
Ours
Accuracy
80.7
79.8
65.3
88.0
94.0
98.6
Accuracy
51.9
27.8
38.7
49.9
89.7
94.4
Median Error
9.62
16.07
15.22
10.02
3.02
3.26
Table 36: Unsupervised 3D pose estimation results for Corrupted- PaspteedddPaca3311++t (frost) Part 2
chair
table
bike
sofa
train
Method
Metric
(Wang et al., 2021a)
Accuracy
73.1
66.8
58.8
62.4
75.7
T8 Accuracy
31.6
38.8
12.5
35,6
47.5
24.6
Median Error
15.99
14.94
24.66
12.51
11.38
18.29
Ours
G Accuracy
87.9
80.1
81.5
89.7
90.5
82.9
IS Accuracy
52.6
60.1
30.8
53.5
78.7
42.5
Median Error
9.68
7.77
14.52
9.36
5.07
13.04
31

---

Published as a conference paper at ICLR 2024
Table 37: Unsupervised 3D pose estimation results for Corrupted-PascalDD. (fog) Part
plane
bicycle
boat
bottle
bus
car
Method
Metric
(Wang al., 2021a)
Accuracy
80.4
80.2
65.9
87.4
93.6
98.3
TK Accuracy
46.4
27.4
36.3
47.0
89.5
94.2
Median Error
10.82
16.62
15.8
10.52
3.24
3.63
Ours
Accuracy
85.5
81.6
70.6
89.3
95.9
99.2
Accuracy
55.7
29.6
42.2
54.2
92.7
95.6
Median Error
8.79
15.16
13.08
8.98
2.69
3.12
Table 38: Unsupervised 3D pose estimation results for Corrupted-Parse Pascal3D+ (fog) Part 2
chair
table
bike
sofa
train
tv
Method
Metric
(Wang et al., 2021a)
Accuracy
86.2
80.1
73.6
92.6
87.6
78.7
Accuracy
45.5
55.9
21.0
55.0
69.9
31.1
Median Error
10.6
8.45
18.8
9.14
5.71
15.86
Ours
Accuracy
90.7
82.9
87.2
94.2
90.4
82.4
Accuracy
58.5
65.0
34.7
61.1
79.0
44.1
Median Error
8.73
6.97
13.26
8.16
4.74
11.72
Table 39: Unsupervised 3DD pose estimation results for Cormupted-Pascal31 (contrast) Part
plane
bicycle
boat
bottle
bus
Method
Metric
(Wang et al., 2021a)
Accuracy
56.4
67.0
56.6
82.5
81.4
92.4
Accuracy
18.2
14.9
25.6
42.3
61.1
80.3
Median Error
24.48
22.18
23.49
11.96
7.22
5.55
Ours
Accuracy
87.7
81.9
70.4
89.2
94.0
99.2
&
IS Accuracy
58.7
32.4
42.6
52.5
90.8
96.0
Median Error
8.24
15.18
12.66
9.57
2.64
3.07
32

---

Published as a conference paper at ICLR 2024
Table 40: Unsupervised 3D pose estimation results for Corrupted-Pascal.e (contrast) Part 2
chair
table
bike
sofa
train
tv
Method
Metric
(Wang et al., 2021a)
Accuracy
69.4
69.1
49.0
88.1
71.6
74.7
T8 Accuracy
26.1
38.2
7.4
45.7
45.5
28.5
Median Error
15.8
14.94
30.86
10.62
11.29
17.95
Ours
Accuracy
90.5
83.0
86.4
95.2
91.0
81.8
Accuracy
57.7
67.4
37.4
61.7
79.5
50.1
Median Error
8.82
6.51
13.28
8.03
4.63
9.98
Table 41: Unsupervised 3D pose estimation for Corrupted-Pascee Pascal3D++ (elastic transform) Part
plane
bicycle
boat
bottle
bus
car
Method
Metric
(Wang et al., 2021a)
Accuracy
76.0
48.4
51.6
84.5
89.1
97.7
Accuracy
39.9
10.2
19.8
36.9
81.8
92.7
Median Error
13.68
30.82
27.66
12.87
4.64
4.02
Ours
Accuracy
84.9
81.6
71.6
88.8
94.5
98.9
Accuracy
55.3
31.6
42.4
51.8
91.7
95.2
Median Error
8.92
16.42
12.75
9.56
3.01
3.17
Table 42: Unsupervised 3D pose estimation for Corrupted- P (elastic transform) Part 2
chair
table
bike
sofa
train
tv
Method
Metric
(Wang et al., 2021a)
Accuracy
63.2
72.8
42.9
92.0
87.6
74.1
Accuracy
16.6
46.5
4.7
54.3
66.3
21.1
Median Error
20.67
10.98
33.62
9.24
6.76
17.28
Ours
Accuracy
90.9
82.5
85.9
94.1
87.7
81.6
G
IS Accuracy
55.1
63.2
33.3
61.7
69.7
44.3
Median Error
9.32
7.13
14.3
8.05
6.3
11.65
33

---

Published as a conference paper at ICLR 2024
Table 43: Unsupervised 3D pose estimation results for Corrupted-Pascal333 (pixelate) Part
plane
bicycle
boat
bottle
bus
car
Method
Metric
(Wang al., 2021a)
Accuracy
60.1
52.2
56.1
82.7
94.0
95.1
TK Accuracy
25.9
9.5
28.4
39.1
88.0
90.1
Median Error
20.61
28.77
23.4
12.37
4.12
4.03
Ours
Accuracy
86.0
82.6
73.2
88.4
94.9
98.9
Accuracy
55.5
30.7
45.6
50.2
91.2
95.6
Median Error
8.58
16.1
11.72
9.95
2.82
3.1
Table 44: Unsupervised 3D pose estimation results for Corrupted-Pascei Pascal3D+ (pixelate) Part
chair
table
bike
sofa
train
tv
Method
Metric
(Wang et al., 2021a)
Accuracy
82.8
78.9
43.4
93.9
75.5
81.2
Accuracy
51.2
58.7
8.8
55.1
52.5
38.2
Median Error
9.66
8.01
34.47
9.14
9.45
13.2
Ours
T Accuracy
88.9
82.7
86.9
95.4
90.4
81.5
Accuracy
56.9
65.2
34.2
59.8
79.2
44.0
Median Error
9.03
6.9
13.23
8.43
5.07
11.38
Table 45: Unsupervised 3D pose estimation results for Corrupted-Pascalaee Pascal3D+ (speckle noise) Part
plane
bicycle
boat
bottle
bus
Method
Metric
(Wang et al., 2021a)
Accuracy
44.8
59.1
37.3
47.9
81.6
95.7
Accuracy
15.7
8.8
13.4
12.6
64.7
85.0
Median Error
35.14
26.09
49.24
31.02
7.15
5.21
Ours
Accuracy
84.1
81.9
69.7
87.7
94.7
98.8
&
IS Accuracy
53.5
27.9
42,2
51.7
91.2
95.1
Median Error
9.22
15.93
13.24
9.79
2.93
3.22
34

---

Published as a conference paper at ICLR 2024
Table 46: Unsupervised 3D pose estimation results for Caarupted-Pascal333 (speckle noise) Part 2
chair
table
bike
sofa
train
tv
Method
Metric
(Wang et al., 2021a)
Accuracy
63.8
56.5
66.2
71.8
70.5
75.7
Accuracy
19.0
22.5
15.2
29.6
36.2
29.1
1R
Median Error
21.65
23.82
21.6
15.58
13.99
16.64
Ours
Accuracy
90.9
81.2
83.2
92.1
89.1
82.8
&
Accuracy
52.6
62.9
38.0
55.0
76.7
46.7
18
Median Error
9.47
7.28
12.66
8,84
4.83
10.66
Table 47: Unsupervised 3D pose estimation results for Corrupted-Pascall Pascal3D+ (gaussian blur) Part
plane
bicycle
boat
bottle
bus
car
Method
Metric
(Wang et al., 2021a)
Accuracy
42.2
55.5
39.7
81.3
62.0
86.0
Accuracy
10.7
14.1
7.8
38.3
32.0
68.1
Median Error
37.05
25.81
40.23
12.78
15.19
7.17
Ours
Accuracy
84.4
77.2
70.6
88.1
93.8
98.2
Accuracy
51.7
28.4
41.1
47.8
90.0
94.2
Median Error
9.62
16.62
13.38
10.51
3.08
3.31
Table 48: Unsupervised 3D pose estimation results for Cormpted-Pascal3DDD (gaussian blur) Part 2
chair
table
bike
sofa
train
Method
Metric
(Wang al., 2021a)
Accuracy
79.6
67.6
57.7
89.4
54.3
78.4
Accuracy
39.7
38.1
11.6
42.8
28.6
32.5
Median Error
12.45
14.38
25.99
11.47
23.2
15.19
Ours
é Accuracy
88.9
83.7
83.0
95.0
89.6
82.3
Is Accuracy
56.7
65.8
33.3
58.7
77.5
43.5
Median Error
9.12
6.96
14.31
8.64
5.22
11.45
35

---

Published as a conference paper at ICLR 2024
Table 49: Unsupervised 3D pose estimation results for Corrupted-PascaD33 (spatter) Part 1
plane
bicycle
boat
bottle
bus
car
Method
Metric
(Wang et al., 2021a)
Accuracy
58.3
64.0
41.6
64.5
88.3
96.0
Accuracy
25.6
14.3
15.9
23.0
79.5
87.8
Median Error
23.54
23.43
41.0
21.41
4.97
4.69
Ours
& Accuracy
84.4
79.4
65.8
87.6
94.4
99.0
Accuracy
52.9
28.4
38.3
50.5
91.0
95.2
Median Error
9.44
16.77
15.38
9.96
3.03
3.25
Table 50: Unsupervised 3D pose estimation results for Corrupted--ascallDD+ (spatter) Part 2
chair
table
bike
sofa
train
Method
Metric
(Wang et al., 2021a)
Accuracy
73.1
56.6
63.5
78.0
82.1
70.4
Accuracy
32.0
29.4
14.0
32.7
61.2
18.0
18
Median Error
15.0
21.87
22.05
14.3
7.39
21.21
Ours
Accuracy
90.5
82.1
87.0
94.2
89.0
82.0
Accuracy
56.5
62.8
34.0
59.5
76.9
44.9
Median Error
8.81
7.45
13.68
8.24
5.28
11.27
36


Reviews:
[{"review_id": "4Vvnkf9NHC", "reviewer": "Reviewer_KkCS", "summary": "This paper study the problem of source-free unsupervised category-level pose estimation from only RGB images to a target domain without any access to source domain data or 3D annotations during adaptation. The author propose a new pipline which focus on the  individual local mesh vertex features and utilize their pose ambiguity to iteratively update them based on their prox- imity to corresponding features in the target domain even when the global pose is not correct. The proposed method shows good results.", "questions": "Please refer to the weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The proposed method outperform previous approaches by a large margin\n2. The author evaluate their model on real world nuisances like shape, texture, occlusion, etc. as well as image corruptions and show the robustness of proposed method", "weaknesses": "1. As mentioned in the article, an observation is that global information is noisy, but some local details are robust. I hope there is rigorous explanation and quantitative analysis here to support this hypothesis.\n2. Ablation is not enough, (e.g., Ablation on top-n \n3. I think this method is similar to the iterative optimization often used in instance-level post-processing. It is unfair to compare this method with other single forward methods.\n4. I'm not sure if it's right to say \"normalized real-valued feature activations\" ? (\nThe fourth line from the bottom of the third page)\n5. The figures in this article are very rough and difficult to understand.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pLGn1kDCFh", "reviewer": "Reviewer_xesL", "summary": "This paper addresses the challenge of unsupervised category-level pose estimation in a target domain using only RGB images, without access to source domain data or 3D annotations.\nThe authors introduce a method that adapts to a target domain, even when it is complicated by nuisances, without requiring 3D or depth data.\nThey represent object categories with simple cuboid meshes and use a generative model of neural feature activations at each mesh vertex.\nThey focus on updating local mesh vertex features based on their proximity to corresponding features in the target domain, even when the global pose is incorrect.\nThe key insight is the stability of specific object sub-parts across different scenarios, which allows for effective model updates.", "questions": "1. Baseline methods and other datasets\nExcept OOD-CV results, only previous baseline is NeMo. Can the authors explain why there can't be other methods in this comparison? Being better than only one baseline is not enough to claim the superiority of the proposed method.\nAlso, while I acknowledge that the paper mainly focuses on data corruption scenarios, is it possible to compare this type of approach in conventional category-level object pose benchmarks like CAMERA and REAL275 datasets provided by NOCS? I believe it would strengthen the authors' motivation if it can be generally applied to conventional syn-to-real UDA scenarios.\n\n2. GT reliability\nGround truth pose illustrated in Figure 1 and 5 seems to be not perfectly aligned with the image. Can the authors explain how these GTs are obtained, and how they are utilized? Are they only used for evaluation?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 1, "contribution": 2, "strengths": "1. This paper suggests effective render-and-compare adaptation pipeline for unsupervised domain adaptation for category-level object pose estimation task.\n\n2. Its proposed method shows the state-of-the-art performance in various corruption scenarios compared to some of the previous methods.\n\n3. The methodology part is intuitive and easy to follow text-wise.", "weaknesses": "1. Poor presentation.\nI think this paper holds good insights and corresponding technological contributions.\nYet, the presentation of the whole paper is relatively poor, making it hard to follow the overall message.\nFigure placement is not well-aligned with the text context.\nFigure 3 seems to be a very important description of explaining one of the main contributions of this paper to match sub-vertices, while there is no mention in the main manuscript referring this.\nMoreover, the main manuscript refers to figures in appendix very often, which is very inconvenient to read, while some of these figures seem important enough to be contained in the main paper for effective description. (ex, Figure 5)\nI believe that ablation studies regarding several design choices of the proposed method should be contained in the main manuscript as well, since they can effectively validate the authors claim.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FEd8suAgTA", "reviewer": "Reviewer_PE77", "summary": "This paper addresses the task of category-level 3D pose estimation within the setting of source-free and image-only unsupervised domain adaptation. The authors introduce a novel method called 3DUDA, which is developed based on the observation of the invariance of object local parts and is supported by theoretical insights. 3DUDA utilizes a learnable cuboid feature matrix to represent an object category, and assesses the accuracy of a pose by comparing the feature map of the test image with the one rendered from the categorical cuboid feature matrix using the pose. This render and compare approach enables domain adaptation by iteratively updating the categorical feature matrix and fine-tuning the source model. Experimental results demonstrate the effectiveness of the proposed method.", "questions": "- How does 3DUDA address the issue of minimizing the impact of translation and object size on neural feature rendering?\n- What values are set for the number $R$ of vertices of the mesh and the hyperparameter of $N$ of the clutter model? How do they  impact the performance of the method?\n- The format of citations in Table 1 does not align with the citation format used in the rest of the paper.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The proposed method, 3DUDA, is developed based on the common observation that certain object parts exhibit invariance across out-of-distribution (OOD) scenarios, and utilizes the categorical learnable cuboid meshes to effectively capture and store the part features at each vertex.\n- To achieve domain adaptation, 3DUDA employs an iterative process that involves updating the features of categorical meshes and fine-tuning the source model through feature-level render and compare optimization.\n- The paper is well-written and presents its ideas in a clear and understandable manner. It is accompanied by comprehensive supplementary materials and theoretical results, which greatly enhance the persuasiveness of the paper.", "weaknesses": "- It is recommended to evaluate the proposed method on commonly used datasets for category-level pose estimation, such as REAL275 [1] or Wild6D [2].\n\n- It would be beneficial to include relevant works [3,4,5,6] in the paper to provide a comprehensive overview of the existing literature in the field.\n\n[1] Wang et al., Normalized object coordinate space for category-level 6d object pose and size estimation. CVPR2019.\n\n[2] Fu et al., Category-Level 6D Object Pose Estimation in the Wild: A Semi-Supervised Learning Approach and A New Dataset.\n\n[3] Lin et al., Category-level 6D object pose and size estimation using self-supervised deep prior deformation networks. ECCV2022.\n\n[4] He et al., Towards Self-Supervised Category-Level Object Pose and Size Estimation.\n\n[5] Zhang et al., Self-Supervised Geometric Correspondence for Category-Level 6D Object Pose Estimation in the Wild. ICLR2023.\n\n[6] Goodwin et al., Zero-Shot Category-Level Object Pose Estimation. ECCV2022.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XwECcwt6Qa", "reviewer": "Reviewer_wu5J", "summary": "This paper introduces a new domain adaptation method for category-level object pose estimation. The method uses only RGB images, without relying on source domain data or 3D annotations in the target domain. The authors represent an object that belongs to a known category as a cuboid mesh and utilize an existing method to learn the vertex features. The features are iteratively updated in the target domain based on their proximity to corresponding image features. The experiments show much better performance in the target domain compared with the competitors without domain adaptation.", "questions": "•\tTo my understanding, in the experiments, the baseline models are evaluated in the target domain without domain adaptation. In this context, it is reasonable that those methods cannot generalize well in the target domain. I was wondering if there are some existing domain adaption approaches that can be applied to the baseline models. The evaluation would be more convincing, comparing NeMo + 3DUDA with a competitor such as NeMo + another domain adaption method.\n\n•\tThe pre-render feature maps are generated from the source mesh. As the vertex features are not updated here, how to make sure those feature maps are reliable in the target domain? Are there situations in which the method might be stuck in local optima or even diverge? Conducting an ablation study on the pre-rendered feature maps would be valuable.\n\n•\tIn Sec.3.2.1, the parameter $\\delta$ seems crucial for effectively updating vertex features. The authors mentioned they chose a $\\delta$ such that the majority of source domain features lie within the likelihood score. How to set $\\delta$ in practice? Is it constant? Intuitively, as the vertex features are updated, one would expect the similarity between vertex features and image features to increase. Does it make more sense to update $\\delta$ accordingly?\n\n•\tIt seems the method is time-consuming due to the iterations and pre-rendering. However, the actual time consumption during testing is unclear.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "•\tThe authors handle the problem that neither depth information nor 3D annotations are available in the target domain, which is challenging yet important in real applications.\n\n•\tThe intuition behind the presented domain adaptation method is that the local features, which represent specific parts of the object, are more robust than the global features in the target domain, which makes sense to me.\n\n•\tThe presented method achieves impressive object pose estimation results in multiple datasets with different kinds of nuisance.", "weaknesses": "The majority of the figures in this paper such as Fig.2, Fig.2, and Fig.10, exhibit low quality. It would be better if the authors could consider revising them to enhance the clarity and resolution.\n\nI am not familiar with the topic of domain adaptation, so I would not judge the novelty of this paper. Please refer to “Questions” for my concerns.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)