EAGLE: Efficient Adaptive Geometry-based Learning in Cross-view Understanding
Authors: Thanh-Dat Truong
Utsav Prabhu
Dongyi Wang
Bhiksha Raj
Susan Gauch
Jeyamkondan Subbiah
Khoa Luu
Venue: NeurIPS 2024 poster (2024)
Keywords: Unsupervised Cross-view Adaptation
Cross-view Geometric Constraint
Semantic Segmentation
Geodesic Flow-based Correlation Metric
Primary area: machine_vision

Abstract: Unsupervised Domain Adaptation has been an efficient approach to transferring the semantic segmentation model across data distributions. Meanwhile, the recent Open-vocabulary Semantic Scene understanding based on large-scale vision language models is effective in open-set settings because it can learn diverse concepts and categories. However, these prior methods fail to generalize across different camera views due to the lack of cross-view geometric modeling. At present, there are limited studies analyzing cross-view learning. To address this problem, we introduce a novel Unsupervised Cross-view Adaptation Learning approach to modeling the geometric structural change across views in Semantic Scene Understanding. First, we introduce a novel Cross-view Geometric Constraint on Unpaired Data to model structural changes in images and segmentation masks across cameras. Second, we present a new Geodesic Flow-based Correlation Metric to efficiently measure the geometric structural changes across camera views. Third, we introduce a novel view-condition prompting mechanism to enhance the view-information modeling of the open-vocabulary segmentation network in cross-view adaptation learning. The experiments on different cross-view adaptation benchmarks have shown the effectiveness of our approach in cross-view modeling, demonstrating that we achieve State-of-the-Art (SOTA) performance compared to prior unsupervised domain adaptation and open-vocabulary semantic segmentation methods.

EAGLE Efficient Adaptive Geometry-based Learning in Cross-view Understanding
Thanh-Dat Truong', Utsav Prabhu", Dongyi Wang" Bhiksha Raj Susan Gauch,, Jeyamkondan Subbiah' Khoa Luu
CCVUU Lab, University of Arkansas, USA Goooge DeepMind, USA
DDp. of BAEG, University of Arkansas, USA *Careegi Mellon University, USA
MMoammmed bin Zayed University of AI, UAE
Dep. of EECS, University of Arkansas, USA "Dep. of FDSC, University of Arkansas, USA (tt032, dongyiv, sgauch, jaubbiah, khoalum)@uark..dd bhikshadcs. cmu edu, uteayprabhudgoogle.ccom https://uark cviu, github. Eittuub.io/projett//EAA.C
Abstract
Unsupervised Domain Adaptation has been an efficient approach to transferring the semantic segmentation model across data distributions. Meanwhile, the recent ppcn-vocabulary Semantic Scene understanding based on large-scale vision lan- guage models is effective in open-set settings because it can learn diverse concepts and categories. However, these prior methods fail to generalize across different camera views due to the lack of cross-view -vess geometric modeling. At present, there are limited studies analyzing cross view learning. To address this problem, we intro- duce novel Unsupervised Cross-view Adaptation Learning approach to modeling the geometric structural change across views in Semantic Scene Understanding. First, we introduce novel Cross-view Geometric Constraint on Unpaired Data to model structural changes in images and segmentation masks across cameras. Second, we present a new Geodesic Flow-based Correlation Metric to efficiently measure the geometric structural changes across camera views. Third, we introduce a novel view-condiile modeling of the open- vocabulary segmentation network in cross-view adaptation learning. The experiments on different cross-view adaptation benchmarks have shown the effectiveness of our approach in cross-view modeling, demonstrating that we achieve State-of-the-AAe (SOTA) performance compared to prior unsupervised domain adaptation and open-vocabulary semantic segmentation methods.
prompting mechanism to enhance the view-information
Introduction
cane prrmeca
Modern segmentation models 3 4
X have
achieved remarkable results on the close-set training with a set of pre-defined categories and concepts. To work towards human-level perception where the scenes are interpreted with diverse categories and concepts, the open- vocabulary (open- (open-vocab) perception model [38] [4] based on the power of large vision-language models 30 39] has been introduced to address the limitations of close-set training. By using the power of language supervision, the large- scale vision language model is able to learn the more powerful representations where languages offer better reasoning mechanisms and open- word concept representations compared to tradi- tional close-set training methods 3 63 9]
CARS
Cross View Adapcation
Glall vere,
Figure 1: Our Proposed Cross-view Adaptation Learning Approach. Prior models, e.g., FreeSeg (38. DenseCLIP [40], trained on the car view do not perform well on the drone-view images. Mean- while, our cross- view adaptation approach is able generalize well from the to drone view.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Recent work is inspired by the success of large vision-languaag models [39 27 that are able to learn informative feature representations of both visual and textual inputs from large-scale image-text pairs. These have been adopted to further develop open-vocab semantic segmen- 40
Ground Truth
Fredittion
Source View Domain
that can work well in open-world environments. However, the open-vocab perception models remain unable to generalize across camera viewpoints. As shown in Fig. | the open-voue open-vocab model trained on views is not able to perform well on the images captured from unmanned aerial vehicles (UAVs) or drones. While this issue can be improved by training the segmentation model drone-view data, the annotation process of high-resolution UAV data is costly and time-consuming. At present, there exist many large-scale datasets with dense labels captured from camera views on the ground, e.g., car views (SYNTHIA [.), GTA 4 Cityscapes [T. 3DDD00O (68). They have been widely adopted to develop robust perception models. Since these car view and drone view datasets have many common objects of interest, incorporating knowledge from car views with drone views benefits the learning process by reusing large-scale annotations and saving efforts of manually labeling UAV is one of the potential approaches to transfer the knowledge from the car view (i.e., source domain) to the drone view (i.e., target domain). While UDA approaches have shown their effectiveness in transferring knowledge across domains, e.g., environment changes or geographical domain shifts, these methods remain limited in the cases of changing camera viewpoints. Indeed, the changes in camera positions, e.g., from the ground of cars to the high positions of drones, bring a significant difference in structures and topological layouts of scenes and objects (Fig. 2 Therefore, UDA is not complete solution to this problem due to its lack of cross- view structural modeling. Additionally, although the open vocab segmentation models have introduced several prompting mechanisms, e.g., context-aware prompting 40 or adaptive prompting (38] to improve context learning across various spendworld concepts, they are unable model the cross-view structure due the lack of view-condition information in prompts and geometric modeling. To the best of our knowledge, there are limited studies that have exploited this cross- view learning. These limitations motivate develop new adaptation learning paradigm, i.e., Unsupervised Cross-view Adaptation, that addresses prior methods to improve the performance of semantic segmentation models across views.
tation models [38
31 29
Iegun Image
Freditionn
View Adaptotion
Target View Domain
Figure 2: An Example of Illustration of Cross- View Adaptation From Car View to Drone View.
images. Unsupervised domain adaptation (UDA) [58 23
I 51555 53
This work introduces novel Efficient Adaptive Geometry-based Learning (EAGLE) to Unsupervised Cross-view Adaptation that adaptively learn and improve the performance of semantic segmentation models across camera viewpoints. First, by analyzing the geometric correlations across views, introduce novel cross-view geometric constraint unpaired data of structural changes in images and segmentation masks. Second, to efficiently model cross-view Geodesic Flow-based Metric to measure the
Contributions:
geometric structural changes, we introduce structural changes across views via their manifold structures. In addition, to further improve the prompting mechanism of the open-vocab segmentation network in cross- cross-view adaptation learning, we introduce a new view-condition prompting. Then, our cross-view geometric constraint is also imposed its feature representations of view-condition prompts to leverage its geometric knowledge embedded in our prompting mechanism. Our proposed method holds a promise to be an effective approach to addressing the problem of cross-view learning and contributes to improving UDA and open-vocaa segmentation in cross-view learning. Thus, it increases the generalizability of the segmentation models across camera views. Finally, our experiments on three presented cross-view adaptation benchmarks, i.e., SYNTHIA > UAVID, GTA - UAVID, BDD - UAVID, illustrate the effectiveness of our approach in cross-view modeling and our State-of-the- Art (SOTA) performance.
2 Related Work
and self-supervised learning 69 23 14 are common approaches to UDA in semantic segmentation. The adversarial learning approaches are typically simultaneously trained on source and target data 7 Chen al. 7 first introduced an adversarial framework to domain adaptation. Later, several approaches improved
Unsupervised Domain Adaptation Adversarial learning 6 58 59
1

---

adversarial learning by utilizing generative models
74
34 21,, using additional labels 228 59
incorporating with entropy minimization [58
65
52
or adopting the curriculum training 37
Recently, the self-supervised approaches [1 69
2
14 have achieved outstanding performance.
Araslanov et al. first proposed self-supervised augmentation consistency framework for UDA. Hoyer et al. [33 utilized Transformers to improve the UDA performance. Later, this approach was further improved by utilizing multi-resolution cropped images [24] and masked image consistency strategy [25] to enhance contextual learning. Recent studies improved the self-supervised approach by aligning both output and attention levels via the cross-domain prediction consistency framework [60], using a prototypical representation 669]. learning the cross- cross-model consistency via depths [67]. improving the class-relevant fairness [53 55 56], or exploring the relations of pseudo- labels [71]. Fashes et al.
[5 introduced prompt-based feature augmentation method to zero-shot UDA. Gong et al. 18] introduced a geodesic flow kernel to model the manifold structure between domains. Later, 47 designed distillation loss by the geodesic flow path.
Simon et al.
LisguaeLanguage and Open-Vocah Segmentation By pre-training on a large-scale vision-language
dataset 339 27 the vision-language models can learn various visual concepts and can further be transferred to other vision problems through "prompting" [[7 38
31 35], e.g., open-vocab
[3 Li et al. [29] first introduced the language-driving approach to semantic segmentation. Rao et al. 40 represented a -awree prompting mechanism for dense prediction tasks. Ghiasi et al. [17] proposed an OpenSeg framework that learns the visual- -esaalsemantic alignments. Qin et al. 38 presented a unified, universal, and open- segmentation network based on Mask2Former [ with adaptive prompting mechanism. Xu et al. [64] proposed two-stage open-vocab segmentation framework using the mask proposal generator and the pre-trained CL.IP model. Ding al. [33 decoupled the zero-shot semantic segmentation class-agnostic segmentation and segment-level zero-shot classification. Liang et al. [1] improved the two-stage open-vocab segmentation model by further fine-tuning CLIP on masked image regions and corresponding descriptions.
segmentation [64 |
Cross-view Learning The early studies exploited cross-view learning in geo-localization by using a polar transform across views [46 45] or generative networks to cross-view images 44 49]. Mcanwhile, Zhu et al. [75] exploited the correlation between street- and aerial-view data via self- attention. In semantic segmentation. Coors et al. [IO] first introduced a cross-view adaptation approach utilizing the depth labels and the cross-view transformation between car and truck views. However, this change of views in [0] is not as big a hurdle as the change of views in our problem, i... car view to drone view. Ren et al. 42 presented an adaptation approach across viewpoints using the 3D models of scenes to create pairs of cross-view images. Vidit et al. [57 modeled the geometric shift in cross FoV setting for object detection by learning position-invarian homography transform. Di Mauro et al. 112] introduced an adversarial method trained on a multi-view synthetic dataset where images are captured from different pitch and yaw angles the same altitudes of the camera positions. Meanwhile, in our problem, the camera views could be placed at different altitudes (c.g., the car and the drone), which reveals large structural differences between the images. Truong et al. [50] [4] first introduced a simple approach to model the relation across views. CROVIA (50) measures the cross-view structural changes by measuring the distribution shift and only focuses on the cross-view
adaptation setting in semantic segmentation. However, these methods [50] (54] lack theory and mechanism for cross-view geometric structural change modeling. To the best of our knowledge, there are limited studies exploiting cross-view adaptation in semantic segmentation. Therefore, our work presents a new approach to model the geometric correlation across views.
The Proposed EAGLE Approach
In this paper, we consider cross-view adaptation learning as UDA where the images of the source and target domains are captured from different camera positions (Fig. 2). Formally, let x,, x, be the input images in the source and target domains, p., Pe be the the corresponding prompts, and y.. Yr be the segmentation masks of x,; Xy. Then, the open-vocab segmentation model F maps the input x and the prompt p to the corresponding output = F(x. p). It should be noted that in the case of traditional semantic segmentation, the prompt p will be ignored, i.c., y F(x) The cross-view a adaptation learning can be formulated as Eqn. (.).
arg miu [ox, p. S.. (Ms
8o) ())+ + Exs. Ladape
(I)
3

---

where 0 is the parameters of F, y. is the ground truth, Lask is the supervised lopen-vocaby segmentation loss with ground truths, and Addppt is unsupervised adaptation loss from the source to the target domain. In the open- vocab setting, we adopt the design of Open- Vocab Fork2Former 8 38 to our network F. Prior UDA methods defined the adaptation loss Adaptt via the adversarial loss (28 5] entropy loss 58), or self-supervised loss [23] 25 Although these prior results have
illustrated their effectiveness UDA, these losses remain limited in cross-view adaptation setup.
Indeed, the adaptation setting in prior studies [58 1 23
IS is typically deployed in the context of
environmental changes (e.g., simulation to real 58
59 15 day to night 25 [5]. etc) where the
camera positions between domains remain similar. Meanwhile, in cross- view adaptation, the camera position of the source and target domain remains largely different (as shown in Fig. (2). This change in camera positions leads significant differences in the geometric layout and topological structures between the source and target domains. As result, direct adoption of prior UDA approaches to cross-view adaptation would be ineffective due to the lack of cross-vier geometric correlation modeling. To effectively address tress-view adaptation, the adaptation loss Adapt should be able to model (1) the geometric correlation between two views of source and target domains and (2) the structural changes across domains.
3.1 Cross- View Geometric Modeling
To efficiently address the cross-view adaptation learning task. it is essential to explicitly model cross-view geometric correlations by analyzing the relation between two camera views. Therefore. we first re-reconsider the cross-view geometric correlation. In particular, let X, be the corresponding image of x, captured from the target view, Ys and y: be the semantic segmentation outputs of source image x, and target image Xt, Pa be the corresponding prompt of p. in target view, respectively. Formally, the images captured from the source and the target views can be modeled as Eqn. 2
X. R(K., [R., t.). θ), X = R(K,, RR, tı]. θ)
where R is the rendering function, K, and K2 are the intrinsic matrices, [R.,, tx) and [R,, te] are the extrinsic matrices, and é represents the capturing scene. In addition, as the camera parameters of both source and target views are represented by matrices, there should exist linear transformations of camera parameters between two views as follow,
K. Tk x K., [R,, t.] = TR: x [R., t.]
where TK and TR:. are the transformation matrices.
Remark 1: The Geometric Transformation Between Camera Views. From Eqn. ( and Eqn. (3). we argue that there should exist geometric transformation T of images between two camera views as: & T(x.; TK, TRi).
Remark 2:
The Equivalent Transformation Between Image and Segmentation Output. As RGB images and segmentation maps are pixel-wised corresponding, the same geometric transformation T in the image space can be adopted for segmentation space as: y. = T (y.: Tk. TR.)
Remarks 1-2 have depicted that the geometric transformation of both image and segmentation from the source to the target view can be represented by the shared transformation TT with the camera transformation matrices Tk, TRt. Let D, x,) and (y v.) be the metrics the measure the cross-view structures changes of images and segmentation maps from the source to target domains.
We argue that the cross-view geometric correlation in the image space, i.e., (x,,,, 8c), is the- oretically proportional the one in the segmentation space, i.e., (y.. Since the camera transformations between the two views are linear (Eqn. (3) and the images x and outputs are pixel-wised corresponding, we hypothesize that the cross-view geometric correlation in the image space D, (x,, Xx) and the segmentation space (y, J.) can be modeled by a linear relation with linear scale a as follows:
(x x D, + (x., x.) = DD,(y.,F.)
(4)
3.2 Cross-view Geometric Learning on Unpaired Data
Eqn. defines a necessary condition to explicitly model the cross-view geometric correlation. Therefore, cross-view adaptation learning in Eqn. ① can be re-formed as follows:
arg min Ex..P... Craasi(ys p.. 5.) + Ex. p.lPa.. [Dx,(x, )) aDD 3:)l]
(5)

---

is the cross-view geoemtric adaptation loss, II-I| is the mean squared error loss. However, in practice, the pair data between source and target views are inaccessible as data from these two views are often collected independently. Thus, optimizing Eqn. (5) without cross-view pairs of data remains an iol-posed problem. To address this limitation, instead of learning Eqn. (5) on paired data, we proposed to model this correlation on unpaired data. Instead of solving the cross-view geometric constraint of Eqn. (5) on pair data, let us consider all cross-view unpaired samples (x,. Formally, learning the Cross-view Geometric Constraint between unpaired samples can be formulated as in Eqn. (6)
where, Adappt (y., Vr) = -
0' arg min Caa.ss.Maassse (y,, Pa, + Ex..Pa.x.. I| (x., aD (y..
(6)
where x, and x, are unpaired data, and Adont (Y ye) [D, xx) (2 yt)l is the Cross- view Geometric Adaptation loss on unpaired data. Intuitively, although the cross-view pair samples not available, the cross-view geometric constraints on paired samples between two views can be indirectly imposed by modeling the cross-view geometric structural constraint among unpaired samples. Then, by modeling the cross-view structural changes in the image and segmentation spaces, the structural change on images of unpaired data could be considered as the reference for the cross-view structural change in the segmentation space during the optimization process. This action promotes the structures of segmentation that can be effectively adapted from the source view to the target view. Importantly, the cross-vies view geometric constraint imposed on unpaired data can be mathematically proved as an upper bound of the cross-vies view constraint on paired data as follows:
[D=(x.,) x.) - DD 3l)l = O (Da Xx, x:) oD yx)l
(7)
where O is the Big O notation. The upper bound in Eqn. (19) can be proved by using the properties of triangle incquality and our correlation metrics D₂ and D, (Sec. 3.3). The detailed proof is provided in the appendix. Eqn. (9) has illustrated that by minimizing the cross-view geometric constraint on unpaired samples in Eqn. 66. the cross-view constraint on paired samples in Eqn. (5) is also maintained due to the upper bound. Therefore, our proposed Cross-view Geometric Constraint loss does NOT require the pair data between source and target views during training. Fig. iillustrates our cross-view adaptation learning framework.
3.3 Cross-view Structural Change Modeling via Geodesic Flow Path
Modeling the correlation metries 2 and D, is an important task in our ap- proach. Indeed, the metrics should be able to model the structure changes from the source to the target view. In- tuitively, the changes from the source to the target view are essentially the geodesic flow between two subspaces on the Grassmann manifold. Then, the images (or segmentation) of two views can be projected along the geodesic flow path to capture the cross-view structural changes. There- fore, to model D2 and Dy, we adopt the Geodesic Flow path to measure the cross-view structural changes by modeling the geometry in the latent space.
Textual Features
Provpet: find oer, drone view
Pretrniied CLP Text Encoder
Encoder
Decoder
Inage Space
Langvage
Seperenitton
Space
Spare
Geodest Frow
View
Decoder
#zomp:: find Cas, OAZ 12.0
Protained CUP Encoder
Textual
Figure 3: Our Cross- View Learning Framework.
Remark 3: Grassmann Manifold is the set of N --dimensional linear subspaces of (R < N < D). L.. G(N, D). A matrix with orthonormal columns P IR DxN define a subspace of G(N, D.. i.c. P G) = P = Iy where IN is the N N identity matrix.
For simplicity, we present our approach to model the cross-view structural change D, the image space. Formally, let P. and P, be the basis of the source and target domains. These bases can be obtained by the PCA algorithm. The geodesic flow between P. and P, in the manifold can be defined via the function II E E [0..1] + I[v),, where II()) G(N, D) is the subspace lying
5

---

the geodesic flow path from the source to the target view:
(I(}) P. (((U, -U2I((]]'
(8)
where R E RDX(D-N) is the orthogonal complement of P.. ie.. R P. 0. r(v) and (vv) are the diagonal matrices whose diagonal element at row i can be defined as " cos(ww;) and T; = sin(vw)). The list of is the principal angles between source and target subspaces, i.e.,
S w1 S I WN I g. U1 and U2 are the orthonormal matrices obtained by the following pair of SVDs:
2
(9) Since P.P. and R P. share the same singular vectors V, we adopt the generalized Singular Value Decomposition (SVD) 18 47] to decompose the matrices. In our approach, we model the cross-view structural changes D, by modeling the cosine similarity between projections along the geodesic flow (T)... In particular, given subspace [I()) the geodesie flow path from the source the target view, the cross-view geometric correlation of images between the source and target views can formulated by the inner product (II(U) xx) along the geodesic flow [I( as follows:
P. -
R -
x(x,,,))= [(' (X,,
L." x. ((( x,dv x {f ((ID))) de) x, xx=x00 Qx
(10)
where Q= So (P) dv. Intuitively, the matrix Q represents the manifold structure between the source to the target view. Then, Eqn. (TO) measures the cross-view structural changes between the source and the target domain based on their manifold structures. The matrix Q can be obtained in a closed form [18 47 as follows:
A A2
A2
MUP.
U R
PPUUU RU2]
(11)
Aa
where A₁. A2, and Ag are the diagonal matrices, whose diagonal elements at row i can be defined as:
A1 sin (2ws)
λ2, cos(2w:) -
As,1 sin()
(12)
2w
2wh
2w
In practice, we model the cross-view structural changes D, via the cosine similarity along the geodesic flows. Finally, the cross-view structural changes D2 can be formulate as:
x, Qx
Pz(x,, ||Q//x,, | 112x:
(13)
Similarly, we can model the cross-view geometric correlation of segmentation D, via Geodesic Flow.
3.4 View-Condition Prompting to Cross-Vies Learning
View-Condition Prompting Previous efforts 40
38
16
73
in open-vocab segmentation have
shown that better prompting mechanism can provide more meaningful textual and visual knowledge. Prior work in open-vocab segmentation designed the prompt via the class names [64, [3 38), e.g.,
"classj, class], classx" Meanwhile, other methods improve the prompting mechanism by introducing the learnable variables into the prompt [40 or adding the task information [8]] This
action helps to improve the context learning of the -siounlanguuag model. In our approach, we also exploit the effectiveness of designing prompting to cross-view learning. In particular, describing the view information can further improve the visual context learning, e.g., "class,, class), class captured from the [domain] view", where [domain] could be car (source domain) or drone (target domain). Therefore, we introduce a view-condition prompting mechanism by introducing the view information, i.e., captured from the [domain] view". into the prompt. Our view-condition prompt offers the context specific to visual learning, thus providing better transfcrability in cross-view segmentation.
Cross-view Correlation View-Condition Prompts We hypothesize that the correlation of the input prompts across domains also provides the cross-view geometric correlation in their deep representations. In particular,
fe and f. be the deep textual embeddings of view-condition prompts p. and p., and D, be metric measuring the correlation between f? and f/" In addition, since the textual encoder has been pre-trained on large-scale vision-language data [39 27], the
visual and the textual representations have been well aligned. Then, we argue that the correlation of

---

Table I: Effectiveness of Our Cross-view Adaptation Losses and Prompting Mechanism.
With
Cross View
View Condition
SYNTHIA + CAVID
GTA - UAVID
Prompt
Adapt
Road
Building
Car Tree
Person
mloU
Road
Building
Car
Tree
Terrain
Person
mloU
8.1
19.1
7.4 30.3
1.3
13.2
7.5
13.0
2.7 26.8
26.6
1.0
12.9
31.4
75.1
57.5
59.2
19.5
48.6
22.9
64.6
37.8 52.8
48.5
13.8
40.1
Supervised
75.5
91.6
79.1
77.7
42.1
73.2
76.8
91.8
81.1 77.6
67.8
43.4
73.1
15.7
27.8
15.7
34.1
7.7
20.2
16.6
26.8
7.2 300
21.7
6.0
18.1
X
36.8
75.5
61.3 60.8
21.2
51.1
27.3
66.8
42.3
55.5
47.1
25.1
44.0
38.4
76.1
62.8
62.1
21.8
52.2
29.2
67.1
45.2
56.6
48.5
27.9
45.7
Supervised
79.8
92.6
82.9
79.1
48.0
76.5
80.5
93.3
82.7
79.2
71.3
49.9
76.1
textual feature representations across views, i.e., (fL, f,"), also provides the cross-view geometric correlation due to the embedded view information in the deep representation of prompts aligned with visual representations. Therefore, similar to Eqn. 44. we hypothesize the cross-view correlation of segmentation masks and textual features can be modeled as a linear relation with scale factor y as:
(f f?) x (y.,., (f
L f() (D.(y...
(14)
Then, learning the view adaptation with view-condition prompts can be formulated as follows: argiijn P3x, 8, Carash (V., 8x) Ex, pr ael (Ay [IP, (x., xe) s.) + PP, (P, x01]
(15)
where A and Ap are the balanced-weige wwight of losses. Similar to metrics Dr and Dy, we also adopt the geodesic flow path model the cross-viee view correlation metric Dp-
4 Experiments
4.1 Datasets, Benchmarks, and Implementation
To efficiently evaluate cross-view adaptation, the cross-view benchmarks are set up from the car to the drone view. Following common practices in UDA (23 58), we choose SYNTHIA 44] GTA [3]. and BDD [68] as the source domains while UAVID B3 is chosen as the target domain. We chose to adopt these datasets because they share a class of interests and are commonly used in UDA
and segmentation benchmarks 22 61
SYNTHIA -> UAVID Benchmark SYNTHIA and UAVID share five classes of interest, i.e., Road, Building. Car, Tree, and Person. Since the UAVID dataset annotated cars, trucks, and buses as class of Car, we collapse these classes in SYNTHIA into single class of Car.
GTA + UAVID Benchmark consists of five classes in the SYNTHIA + UAVID benchmark and includes one more class of Terrain. Therefore, the GTA UAVID benchmark has six classes of interest, i.e., Road, Building. Car, Tree, Terrain, and Person.
BDD - UAVID Benchmark is real-to-real cross-view view adaptation setting. Similar to GTA UAVID benchmark, there are six classes of interest between BDDI00K and UAVID. In our experi- ments, adopt the mean Intersection over Union (mloU) metric to measure the performance.
Implementation We adopt Former g (ResNet 101) with Semantic Context Interaction of FreeSeg [38] and pre-trained text encoder of CLIP (39] for our open-vocab segmentation networks.
Our balanced weights of losses are set to \ 1.0 and Xp 0.5. Further details of our networks and hyper-parameter are provided in the appendix.
Table 2: Effectiveness of Backbones and Cross- view Metrics.
4.2 Ablation Study
Effectiveness of Cross-view Adaptation and Prompting Mechanisms Table I analyzes the effectiveness of prompting mechanisms, i.e., i.e. with and without Prompting, with and with- out Cross-view Adaptation (in Eqn. (6), with and without View-Condition Prompting (in Eqn. (5)). For supervised results, we train two dif- ferent models on UAVID with and without the Terrain class on two benchmarks. As in Table [ the cross-view adaptation loss in Eqn. 6 signifi-
SYNTHIA + UAVID
Network
Building Car Tree Terrin Person
Baelidean
23.7
31.2
33.2 36.7
11.5
27.2
ResNet
Geadesic
38.4
76.1
62.8 62.1
21.8
52.2
Baclidean
24.7
31.9
41.2 39.7
14.1
30.3
Swin
Geodesic
40.8
76.4
65.8 62.7
27.9
54.7
GTA + UAVID
Baclidean
21.7
30.0
26.2 39,7
31.7
9.5
26.5
ResNer
Geadesic
29.2
67.1
45.2 56.6
48.5
27.9
45.7
Swin
Baclidean
24.3
33.7
28.5
40.1
32.8
9.7
28.2
Geodesic
31.0
67.1
46.8
56,9
48.7
31.9
47.1
cantly improve the performance of segmentation models. With prompting and cross-view adaptation,
7

---

the mloU performance is further boosted, i.e,, the mloU performance achieves 48.6% and 40.1% 1% on two benchmarks. Additionally, by further using the view-condition prompting mechanism with our cross-view loss in Eqn. (15). the mloU results are slightly improved by +1.1% and +1.7% on two benchmarks compared to the one without view-condition prompting. Our results have closed the gap with the aoper-bound results where the models are trained on UAVID with labels.
Effectiveness of Cross-view Correlation Metries and Network Backbones Table Itudiess the impact of choosing metrics and network backbones. We consider two options, i.e., Euclidean Metric and our Geodesic Flow-based Metric, for correlation metrics D.. Dy. and Dp. As shown in Table 2 our Geodesic Flow-based metrics significantly improve the performance of our cross-view adaptation. It has shown that our approach able to measure the structural changes across views better than using the Euclidean metrics. In addition, by using the more powerful backbone (Swin), the performance of cross-view adaptation is further improved.
Effectiveness of Cross-view Learning Parameters Table 3 illustrates the impact of the lin- ear scaling factors a and 3. As in Table 3 the mIoU performance has been majorly affected by the relation between images and seg-
Table 3: Effectiveness of Linear Scale Factors, i... a and y and Subspace dimension D.
Factor
Read
Building Car Tree Person
SYNTHIA
LAVID
mlnU Road Building Car Tree Teraain
GTA
LAVID
Person mloU
0.5
35.9
13.5
39.637.2
20.8
49.4
25.4
63.9
40.344.6
46.3
25.6
41.1
a=
37.8
75.8
60.066.77
21.6
51.4
26.9
64.3
41.8 48.0
47.2
26.3
42.4
a= 1.5
38.4
76.1
62.8 62.1
218
52.2
29.2
67.1
45.2
48.5
27.9
45.7
2.0
36.9
74.8
60.7
59.4
21.2
50.6
28.1
66.0
44.2518
48.1
27.3
44.2
mentation. The best performance is gained at the optimal value of
-.0.
37.6
75.5
60.66600
A1
31.1
278
65.1
42. 51.7
47.7
26.8
43.6
=000 38.4
76.1
62.8
2188
522
29.2
67.1
45,2
48.5
27.9
45.7
9-1.5 36.2
75.3
20.5
50.4
28.5
66.0
44.2.4.77
47.9
27.5
44.8
1.5. Since the variation of
2=20 36.0
74,2
60.0 58.
20.7
49.8
26.8
64.5
42.8
47.0
26.8
43.4
D= 36.6
67.6
21.3
49.6
42.2
46.5
27.0
42.3
RGB images is higher than the segmentation, the small value a
D= 37.1
72.7
61.3 58.9
21.4
50.3
27.7
62.4
43.0
47.1
27.3
43.4
D= 256 38.4
76.1
62.8
21.8
52.
29.2
67.1
46,2
48.5
27,9
45,7
51 37,9
62.4611
21.4
51.7
28.2
64.9
44.2.4.1
47.9
27.6
44.5
could not correctly scale the rela- tion between images and segmentation while the higher value of a exaggerates the structural change of segmentation masks. Additionally, the change of slightly affects the mIoU performance. Since the textual features are well-aligned with the image, the performance of segmentation models when changing 7 also behaves similarly to the changes of a. However, the linear scale factor is more sensitive to mloU results since the images play more important role in the segmentation results due to the pixel-wiss corresponding of images and segmentation.
Effectiveness of Subspace Dimension in Geodesic Flow Table reveals the importance of choosing the subspace dimension. The cross- view geometric structural change is better modeled by increasing the dimension of the subspaces. As in T the performance is improved when the dimension is increased from 96 to 256. However, beyond that point, the mIoU performance tends to be dropped. We have observed that low dimensionality cannot model the structural changes across views since it captures small variations in structural changes. Conversely, higher dimensionality includes more noise in the cross- cross-view structural changes and increases the computational cost. We also study the impact of batch size in our appendix.
Qualitative Results. To further illustrate the effectiveness of our proposed, we visualize the results produced by our model. In the model without prompting, Figure [ illustrates the results of our cross- view adaptation compared to those without adaptation. As shown in the results, our approach can effectively segment the objects in the drone view. We also compare with the prior ProDA [69] and CROVIA (50) methods. Our qualitative results remain better than the prior adaptation method. For the model with prompting, Figure 5 illustrrss the effectiveness of our approach in three cases:
Input
Without Adaptation
Adaptation
Cross-View
CROVIA
ProDA
Input
Adoptation
Without
Cross-View
Adeptation
CROVIA
ProDA
Figure 4: The Qualitative Results of Cross-View Adaptation (Without Prompt).
8

---

Input
Without Adaptation
Cross-Visw Adaptation
Cross-Visw Adaptation
Input
Without Adaptation
Cross-View Adaptation
Adaptation
Cross-View
Condition
View Candition
Figure 5: The Qualitative Results of Cross- View Adaptation (With Prompt).
without adaptation, with cross- vies adaptation, and with view-condition prompting. As shown in the results, our cross-view adaptation can efficiently model the segmentation of the view. By using the view-condition prompting, our model can further improve the segmentation of persons and vehicles.
4.3 Comparisons with Prior UDA Methods
SYNTHIA- - UAVID As shown
Table 4: Comparisons with Domain Adaptation Approaches (With-
in Table 4 our EAGLE has
out Prompting).
achieved SOTA results and out-
SYNTHIA
CAVID
performs prior view transfor- mation (i.e., Polar Transform (45) UDA methods by a large margin. For fair comparisons, we adopt the DeepLab [3] and DAFormer 23 for the segmen- tation network. In particular, our mloU results using Lee and DAFormer are 45.2% and 50.8%. In the DAFormer backbone, the mloU results of our approach are higher than CROVIA 50 and MIC [25 by +4.8% and +9.0%.
4.3
50.0
Trute
20.5
22.6
19.3
20,7
2.5
DADA
10.7
63.1
32.9
50.0
162
34.5
BiMel
51
5.4
62.
34.8
96.
12.3
33.1
1.3
64.
10.1
05.2
20:0
10.9
21.7
Docptab
SAC
13.9
54.0
18.7
46.0
15.6
32.0
4.5
36.9
7.5
47.0
44.1
7.8
24.8
Pholah
10.0
54.7
16.1
48.5
17:0
34.3
25.5
18.7
4.5
CROVEA
55.6
43.1
5,0
Superitsed
7.3
31.7 45.
15.1
374
51.5
11.5 21.
35.5
4.0
X5.5
DARemmer
NIC [X
CRONTA [0
10.0
16.3
76.4
75.1
53.3
59.6
50.
52.
160
19.1
41.
45.0
31.7
20.5
51.9
56.1
57.6
133.55 55.2
50.7
44.3
45.3
10.0
9.3
31.5
36.8
KAGLE
75.3
89.7
63.1
25.5
58.8
23.9
65.8
38.5
53.5
09.3
14.1
46.7
Superstuol
01.2
17.5
0813
74.1
76.4
Mask2Foomer
Superstust
EAGGL
314
37.8
13.8
The IoU result of each class also consistently outperformed the prior methods. Highlighted that although our approach does NOT use depth labels, our results still outperform the one using depths, i... DADA [59]. has emphasized that our approach is able to better capture the cross-view structural changes compared to prior methods. Figure illustrates our qualitative results compared to ProDA (69 and CROVIA [50].
GTA + UAVID As shown in Table our effectiveness outperforms prior polar view transformation 45 and domain adaptation approaches when measured by both mloU performance and the IoU accuracy of each class. In particular, our mloU performance using DeepLab and DAFormer network achieves 36.7% and 40.7%, respectively. Our results have substantially closed the performance gap with the supervised results. By using the better segmentation-basse network, i.e., Farke with ResNet, the performance of our approach further improved 40.1% compared to DeepLab.
4.4 Comparisons with Open-vocab Segmentation
We compare EAGLE with the prior open- ppcab segmentation methods, i.c.. DenseCLIP 40 and an adaptive prompting FreeSeg (38] with four settings, i.e., Source Only, with AdvEnt (58), and with SAC MI and our Cross-Vice Adaptation in Eqn. (6) (without view-condition).
Open-vocab Semantic Segmentation As in Table the mloU performance of our proposed approach with cross-view adaptation outperforms prior DenseCLIP by a large margin on SYNTHIA - UAVID. By using our cross-view geometric adaptation loss, the performance DenseCLIP and FreeSeg is further enhanced, i.e., higher than DenseCLIP and FreeSeg with SAC by +3.7% and +5.0%. While FreeSeg (38 with our cross-view adaptation slightly outperforms EAGLE due to its adaptive prompting, our EAGLE approach with the better view-condition prompting achieves higher mloU

---

Table 5: Comparisons with Open-voeab Semantic Segmenta- Table 6: Comparisons with Open vocab Segmentation on Seen (mloUS) and Un-
tion.
Metsol
S01
TEN/IHA
LG
Lix
1.3
DITA HAWE
seen (mloU*) Classes.
DenscClP
LSYNTHIA
HAATD
WAAID
DesoituPP + Addien
2013
02.1
08.6
46.2
163
76.3
26.1
19.4
286
41.4
36.7
16.0
70.1
DesutiLar a Vien
32.4
42.0
15.3
196
86.7
359
354
300
-11
303
DessecLip
2535
153
113
113
17:1
25.3
53.1
25.3
63.
1934
BenoClPP a Conse
56.5
35.1
-6.3
DenscCLPP e SAc
26:1
07.4
516
464
175
421
17.3
53.0
125
52.5
43.7
15.5
33.4
Dinostul1 - AERtE
15x
3122
-1.
317
DenoCLUP Fioe View
31.
714
53
m11
214
214
155
454
16.
DemoC11P Conso
590
Ficdka Ad
500
11.1
54.9
453
180
453
506
15.
42.3
44.7
16.5
5677
Fanbag 201227
113
311
003
353
Dessag SAC
713
164
191
16.1
16.6
Trmbieg Cress Viar
68.2
0.3
-53
10.5
7.3
0.7
00
100
EAGEL
636
33.
305
57.5
Cuuditin
26.1
638
621
218
452
46.7
Patly Sapariee
851
6330
81.9
06.3
performance. Similarly, our proposed cross-view loss consistently improves the performance of DenseCLIP and FreeSeg on GTA - UAVID. The mloU results of DenseCLIP and FreeSeg using our cross-view loss achieve 37.3% and 44.4%. By further using the view-condition prompting mechanism. our mloU result is considerably higher than FreeSeg with our cross-view adaptation by +1.3%. Figure o visualizes our qualitative results of our proposed approach.
Open-vocab Segmentation on Unseen Classes T illustrates the experimental results of our cross-view adap- tation approach on unseen classes. In this experiment, consider classes of Tree and Person the unseen classes. As shown in the results, our cross-view adaptation approach with a view-condition prompting mechanism has achieved the best mloU performance on unseen classes on both benchmarks, i.e., 39.3% and 39.6% on two bench- marks. Our experimental results have further confirmed the effectivvness and the generalizability of our cross-view geometric modeling and view condition prompting ap- proach to the open-vocah segmentation across views.
Figure 6: Results of Segmenting Cars,
Trees, Persons. (A) Input, (B)
FreeSeg [38, and (C) Our EAGLE.
Real-to-Real Cross-view Adaptation Setting We evaluated approach in the real-to-real setting, i.c., BDD + UAVID. Our approach is evaluated in two different settings, i.e., Unsu- pervised Domain Adaptation and Open-Vocab Semantic Segmentation. As shown in Table 7 our results have shown a significant improve- ment in our approach in real-to-real settings in both unsupervised domain adaptation and open- vocab semantic segmentation. While the results of prior unsupervised domain adaptation, i.e.,
Table Comparison with Prior Adaptation Meth- ods and Open- Vocab Segmentation on Real-to- Real Cross-View Setting.
Setting
Mathad
RDD LIAVID
Teatin
Unsupervised
Mapution
Diesii
B:Mal.
Polar
EAGLE (Desplah)
DAFumez
23
[5
25.8
24,0
19.5
65.4
53.8
38.7
38
54.5
52.2
51.3
48.3
18.6
14.8
16.9
41.3
38,0
15.4
EAGL (DDlsmmer
20,0
66.1
41.5
566
53.1
21.5
44.5
Open-Macah Macab
Seg
EreSicg CCoos View
EARLE
Direet [1P= mio-Viww
21.9
35.4
33.6
M63
67.3
475
16.1
51
15.3
47.1
51.7
50.3
37.2
159
40.3
479
BiMaL. [5T) and DAFormer (23). gain limited performance due to their limits in cross-view learning, our method outperforms other methods these prior methods by large margin.
Conclusions
This paper has presented novel unsupervised cross-vier view adaptation approach that models the geo- metric correlation across views. We have introduced the Geodesic Flow-based metric to better model geometric structural changes across camera views. In addition, a view-condition prompting mechanism has been presented to further improve the cross-view modeling. Through our theoretical analysis and SOTA performance on both unsupervised cross-view adaptation and open-vocab seg- mentation, our approach has shown its effectiveness in cross-view modeling and improved robustness of segmentation models across views.
Limitations Our study has selected a set of learning and experiments. However, this work can potentially contain several limitations related to learning parameters and linear relation hypothesis in Eqn. (4) The details of the limitations are discussed in the appendix. We believe that these limitations will motivate future studies to improve unsupervised cross-view adaptation learning approach.
to support our hypothesis
Acknowledgment This work is partly supported by NSF Data Science, Data Analytics that are Robust and Trusted (DART), NSF SBİR Phase 2, and Arkansas Biosciences Institute (ABI) grants. We also acknowledge the Arkansas Perfooeerooe Computing Center for providing GPUs.
10

---

References
[1] N. Araslanov, and S. Roth. Self-supervised augmentation consistency for adapting seman- tic segmentation. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2021.
[2] M. Armando, S. Galaaoui, F. Baradel, T. Lucas, V. Leroy, R. Brégier, P. Weinzaepfel, and G. Rogez. Cross-view and cross pross-pose completion for 3d human understanding. arXiv preprint arXiv:331.09114. 2023.
[3] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. TPAMI, 2018.
[4] L.-C. Chen, G. Papandreou, Schroff, and H. Adam. Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv: 705500057.078 2017.
[5] Y. Chen, W. Li, C. Sakaridis, D. Dai, and L.. Van Gool. Domain adaptive faster r-cnn for object detection in the wild. In CVPR, 2018.
[6] Y. Chen, W. Li, and L.. Van Gool. Road: Reality oriented adaptation for semantic segmentation of urban scenes. In CVPR, 2018.
[7] Y.-H. Chen, W.-Y. Chen, Y.-T. Chen, B.-C. Tsai, Y.-.. F. Wang, and M. Sun. No more discrimination: Cross city adaptation of road scene segmenters. In /CCV, 2017.
[8] B. Cheng, I. Misra, A. G. Schwing, A. Kirillov, and R. Girdhar. Masked-attention mask transformer for universal image segmentation. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 1290-1299, 2022.
[9] B. Cheng, A. G. Schwing, and A. Kirillov. Per-pixel classification not all you need for semantic segmentation. In NeurIPS, 2021.
[10] B. Coors, A. P. Condurache, and A. Geiger. Nova: Learning see in novel viewpoints and domains. In 2019 International Conference on 3D Vision (3DV), pages 116-125, 2019.
[11] M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele. The Cityscapes dataset for semantic urban scene understanding. In CVPR, 2016.
[12] D. Di Mauro, A. Furnari, G. Patanè, S. Battiato, and G. M. Farinella. Sceneadapt: Scene-based domain adaptation for semantic segmentation using adversarial learning. Pattern Recognition Letters, 136:175-10 172-182, 2020.
[13] Ding, N. Xue, G.-S. Xia, and D. Dai. Decoupling zero-shot semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11582-11922, 2022.
[14] S. Ettedgui, S. Abu-Hussein, and Giryes. Proest: Boosting semantic segmentation using progressive cyclic style-transfer, 2022.
[15] M. Fahes, T.-H. Vu, A. Bursuc, P. Pérez, and R. de Charette. Poda: Prompt-driven zero-shot domain adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 16623-1863,, 2023.
[16] P. Gao, S. Geng, R. Zhang. T. Ma, R. Fang, Y. Zhang, H. Li, and Y. Qiao. Clip-adapter: Better vision-language models with feature adapters. International Journal of Computer Vision, pages 1-15, 2023.
[17) G. Ghiasi, X. Gu, Y. Cui, and T.-Y. Lin. Scaling open-vocabulary image segmentation with image-level labels. In European Conference on Computer Vision, pages 540-557. Springer, 2022.
[18] B. Gong, Y. Shi, F. Sha, and K. Grauman. Geodesic flow kernel for unsupervised domain adaptation. In 2012 IEEE Conference on Computer Vision and Pottern Recognition, pages 2066 2066-2073, 2012.
11

---

[19] G. Goyal, N. Noceti, and F. Odone. Cross-view action recognition with small-scale datasets. Image and Vision Compuring, 120:104403, 2022.
[20] K. He, X. Zhang, S. Ren, and Sun. Deep residual learning for image recognition. In CVPR, 2016.
[21] J. Hoffman, F. Tzeng, T. Park, J.-Y. Zhu, P. Isola, K. Saenko, A. Efros, and T. Darrell, CyCADA: Cycle-consistent adversarial domain adaptation. In ICML, 2018.
[22] C.-A. Hou, Y.-R. Yeh, and Y.-.. F. Wang. An unsupervised domain adaptation approach for cross-domain visual classification. In 2015 12th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pages 1-6. IEEE, 2015.
[23] L. Hoyer, D. Dai, and L. Van Gool. DAFormer: Improving network architectures and training strategies for domain-
semantic segmentation. In CVPR, 2022.
[24] L. Hoyer, D. Dai, and L. Van Gool. HRDA: Context-aware high-resolution domain-adaptive semantic segmentation. In Proceedings of the European Conference on Computer Vision (ECCV), 2022.
[25] L. Hoyer, D. Dai, H. Wang, and L. Van Gool. Mic: Masked image consistency for context- enhanced domain adaptation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 11721-11732, 2023.
[26] J. Huang, D. Guan, A. Xiao, and Lu. Cross-view regularization for domain adaptive panoptic segmentation. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pages 10133-10144, 2021.
[27] C. Jia, Yang, Y. Xia, Y.-T. Chen, Z. Parekh, H. Pham, Q. Le, Y.-H. Sung, Z. Li, and T. Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In International conference on machine learning, pages 4904-4916. PMLR, 2021.
[28] K.-H. Lee, G. Ros, Li, and A. Gaidon. SPIGAN: Privileged adversarial learning from simulation. In ICLR, 2019.
[29] B. Li, K. Q. Weinberger, S. Belongie, V. Koltun, and R. Ranftl. Language-drive semantic segmentation. In International Conference on Learning Representattinss 2022.
[30] L. H. Li, P. Zhang, H. Zhang, J. Yang, C. Li, Y. Zhong, L. Wang, L. Yuan, L. Zhang, J.-N. Hwang, K.-W. Chang, and J. Gao. Grounded language-image pre-training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10965-10975, 2022.
[31] F. Liang, B. Wu, X. Dai, K. Li, Y. Zhao, H. Zhang, P. Zhang, P. Vajda, and D. Marculescu. Open- vocabulary semantic segmentation with mask-adapted clip. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7061-7070, 2023.
[32] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEF/CVF international conference on computer vision, pages 10012-10022, 2021.
[33] Y. Lyu, G. Vosselman, G.-S. Xia, A. Yilmaz, and M. Y. Yang. Uavid: A semantic segmentation dataset for uav imagery. ISPRS Journal of Photogrammetry and Remote Sensing, 165:108 2020.
[34] Z. Murez, S. Kolouri, D. Kriegman, R. Ramamoorthi, and K. Kim. Image to image translation for domain adaptation. In CVPR, 2018
[35] H.-Q. Nguyen, T.-D. Truong, X. B. Nguyen, A. Dowling, X. Li, and K. Luu. Insect-foundation: foundation model and large-scale Im dataset for visual insect understanding. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 21945-21955. 2024.
[36] B. Pan, Sun, H. T. Leung, A. Andonian, and B. Zhou. Cross-view semantic segmentation for sensing surroundings. IEEE Robotics and Automation Letters, 58)44867-488,, 2020.
12

---

[37] F. Pan, I. Shin. F. Rameau, S. Lee. and I. S. Kweon. Unsupervised intra-domain adaptation for semantic segmentation through self-supervision. In CVPR. 2020.
[38] J. Qin, J. Wu, P. Yan, M. Li, R. Yuxi, X. Xiao, Y. Wang. R. Wang, S. Wen, X. Pan, et al. Freeseg: Unified, universal and vven-vocabulary image segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19446-19455, 2023.
[39] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748- 8763. PMLR, 2021.
[40] Y. Rao, W. Zhao, G. Chen, Y. Tang, Z. Zhu, G. Huang, J. Zhou, and J. Lu. Denseclip: Language- guided dense prediction with context-aware prompting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
[41] K. Regmi and M. Shah. Bridging the domain gap for ground-to-aerial image matching. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 470-479, 2019.
[42] H. Ren. Y. Yang, H. Wang, B. Shen, Q. Fan, Y. Zheng, C. K. Liu, and L. Guibas. Adela: Automatic dense labeling with attention for viewpoint shift in semantic segmentation. In 2022 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 2022.
[43] R. Richter, V. Vineet, S. Roth, and V. Koltun. Playing for data: Ground truth from computer games. In ECCV, 2016.
[44] G. Ros, L. Sellart, J. Materzynska, D. Vazquez, and A. M. Lopez. The SYNTHIA dataset: A large collection of synthetic images for semantic segmentation of urban scenes. In CVPR, 2016.
[45] Y. Shi, L. Liu, X. Yu, and H. Li. Spatial-aware feature aggregation for image based cross-view geo-localization. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alch' e-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32, pages 10090-10100. Curran Associates, Inc., 2019.
[46] Y. Shi, X. Yu, D. Campbell, and H. Li. Where am looking at? joint location and orientation estimation by cross-view matching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020.
[47] C. Simon, P. Koniusz, and M. Harandi. On learning the geodesic path for incremental learning. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pages 1591-1600, 2021.
[48] N. Sun, Q. Lu, W. Zheng, Liu, and G. Han. Unsupervised cross-view facial expression image generation and recognition. JEEE Transactions on Affective Computing, 2020.
[49] A. Toker, Q. Zhou, M. Maximov, and L. Leax-Taixe. Coming down to earth: Satellite-to-stred view synthesis for reo-localization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6488 6497, June 2021.
[50] T.-D. Truong, C. N. Duong, A. Dowling, S. L. Phung, J. Cothren, and K. Luu. Crovia: Seeing drone scenes from car perspective via cross-view adaptation, 2023.
[51] T.-D. Truong, C. N. Duong, N. Le, S. L. Phung, C. Rainwater, and K. Luu. Bimal: Bijective maximum likelihood approach to domain adaptation in semantic scene segmentation. In IEEE International Conference on Computer Vision (ICCV), pages 8548-8557, 2021.
[52] T.-D. Truong, P. Helton, A. Moustafa, J. D. Cothren, and K. Luu. Conda: Continual unsupervised domain adaptation learning in visual perception for self-driving cars. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages (652-5650,15 2024.
[53] T.-D. Truong, N. Le, B. Raj, J. Cothren, and K. Luu. Fredom: Fairnsss domain adaptation approach to semantic scene understanding. In IEFE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
13

---

[54] T.-D. Truong and K. Luu. Cross-view action recognition understanding from exocentric to egocentric perspective. arXiv preprint arXiv:2305.15699 2023.
[55] T.-D. Truong, H.-Q. Nguyen, B. Raj, and K. Luu. Fairesss continual learning approach to semantic scene understanding open-world environments. Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS), 2023.
[56] T.-D. Truong, U. Prabhu, Raj, J. Cothren, and Luu, Falcon: Fairness learning via contrastive attention approach to continual semantic scene understanding in open world. arXiv preprint arXiv:2311. 15965, 2023.
[57] V. Vidit, M. Engilberge, and M. Salzmann. Learning transformations to reduce the geometric shift in object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17441-17450, 2023.
[58] T.-H. Vu, H. Jain, M. Bucher, M. Cord, and P. Pérez. Advent: Adversarial entropy minimization for domain adaptation in semantic segmentation. In CVPR, 2019.
[59] T.-H. Vu, H. Jain, M. Bucher, M. Cord, and P. Pérez. Dada: Depth-aware domain adaptation in semantic segmentation. In ICCV, 2019.
[60] K. Wang, D. Kim, R. Feris, and M. Betke. Cdac: Cross-domain attention consistency in transformer for domain adaptive semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11519-11529 2023.
[61] L. Wang, R. Li, C. Zhang, S. Fang, C. Duan, X. Meng, and P. M. Atkinson. Unetformer: A unet-like transformer for efficient semantic segmentation of remote sensing urban scene imagery. ISPRS Journal of Photogrammetry and Remote Sensing, 190:196-22 224, 2022.
[62] Q. Wang, W. Min, Q- Han, Z. Yang, X. Xiong, M. Zhu, and H. Zhao. Viewpoint adaptation learning with cross-view distance metric for robust vehicle re-identification. Information Sciences, 564:71-88 11,7711847777 2021.
[63] E. Xie, W. Wang. Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo. Segformer: Simple and efficient design for semantic segmentation with transformers. In NeurIPS, 2021.
[64] M. Xu, Z. Zhang, F. Wei, Y. Lin, Y. Cao, H. Hu, and X. Bai. A simple baseline for open- vocabulary semantic segmentation with pre-trained vision-language model. In European Conference Computer Vision, pages 736-753. Springer, 2022.
[65] Z. Yan, X. Yu, Y. Qin, Y. Wu, X. Han, and S. Cui. Pixel-level intra-domain adaptation for semantic segmentation. In Proceedings of the 29th ACM International Conference on Multimedia, pages 404 413, 2021.
[66] Y. Yao and H. S. Park. Multiview cross-supervision for semantic segmentation. arXiv preprint arXiv: 1812.01738, 2018.
[67] Y. Yin, W. Hu, Liu, G. Wang, S. Xiang, and Zimmermann. Crossmatch: Source-free domain adaptive semantic segmentation via cross-modal consistency training. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 21786-21796 2023.
[68] F. Yu, H. Chen, X. Wang, W. Xian, Y. Chen, F. Liu, V. Madhavan, and T. Darrell. Bdd100k: A diverse driving dataset for heterogeneous multitask learning. Proceedings of the IEEE/CVF conference computer vixion and pattern recognition, pages 2636-2644 2020.
[69] Zhang, Zhang, T. Zhang, D. Chen, Y. Wang, and Wen. Prototypical pseudo label denoising and target structure learning for domain adaptive semantic segmentation. arXiv preprint arXiv:2101. 10979, 2021.
[70] Q. Zhang, W. Lin, and A. B. Chan. Cross-view cross-scene multi-view crowd counting. In Proceedings of the JEFF/CVF Conference on Computer Vision and Pattern Recognition, pages 557-567, 2021.
14

---

[71] D. Zhao, S. Wang. Q. Zang. D. Quan, X. Ye, R. Yang. and L. Jiao. Learning pseudo-relations for cross-domain semantic segmentation. Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 19191-19203, 2023.
[72] B. Zhou and P. Krähenbuhh.. Cross-view transformers for real-time map-view semantic segmen- tation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13760-13769, 2022.
[73] K. Zhou, J. Yang, C. C. Loy, and Z. Liu. Learning to prompt for vision-language models. International Journal of Computer Vision, 13009):2337-2348, 2022.
[74] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In 2017.
[75] S. Zhu, M. Shah, and C. Chen. Transgeo: Transformer all you need for cross-view image geo-localization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1162-1171, 2022.
15

---

Appendix
Proof of Eqn. (9)
As shown in our Eqn. (16), Geodesic Flow-based metrics have the upper bound as follows:
x, Qxx
Vx,, Xe: D2(X., x.) = ||Q Q
S
Q1/2x
xx
(16)
y's Qy
Vy Dy(y,,ye) = |QQ//y Q yyyll
2
In addition, as Dr is the distance metric, this metric should satisfy the following triangular inequality as follows:
(x &x) & (x(x,, + Dx,,,,
(17)
Similarly, D should satisfy the following triangular inequality follows:
Dy St) + Dy( 2 Ve)
Dy (y,, 51) Dy) - y...)
(18)
+ - aDy (yt. v -a (Dy(y., - Dy(J,,y.))
Then, from Eqn. 16 and Eqn. 17 above, we can further derive as follows:
Dr(Xe, - (P)
< Dx(Xx, x1) (xi,,, x.) - - (f,
5 Da (x., - (D,, + Da (x,, (
(19)
< Da Dx xx) - aD, +(1
Constant
Since a is the constant linear scale value, therefore, we can further derive as follows:
[x,, x,) Je)(
(20)
=
aD, (y.,
2
Implementation
We follow the implementation of Mask2Former 8 and FreeSeg 38 with ResNet (0) and Swin backbones [32] for our segmentation network. In particular, we adopt Mask2Former with Semantic Context Interaction of FreeSeg (38) for our open-vocab segmentation network. We use the pre-trained text encoder of CLIP 39). The textual features f!' and f.' are obtained by the CLIP textual encoder. Following common practices [38 31], we adopt the open-vocab segmentation loss of FreeSeg (38) to our supervised loss LMskk- For experiments without prompting, we use the Mask2Former network. Following the UAV protocol of [61]. the image size is set to 512 x 512. The linear scale factors and y are
1.5 and y 1.0, respectively. For the Geodesic Flow modeling, we adopt the implementation of generalized SVD decomposition [18 47 in the framework. The subspace dimension our geodesic flow-based metrics set to learning rate in our experiments are set to 16 and 2.5 x 10-1 The balanced weights of losses in our experiments are set to ' = 1.0 and Ap = 0.5. During training, the classes in the prompts are generated similarly for both view images.
256. The batch size and the base
In our Geodesic Flow-based metries, the subspaces of images and ground-truth segmentation of the source domain are pre-compueed on the entire data. For the language space, we compute the subspaces of each view based on the textual feature representations of all possible prompts in each
domain. Meanwhile, the subspaces of the segmentation on the target domain are computed based on the current batch training. For the implementation of DenseCLIP 40 and FreeSeg (38 with
AdvEnt [58), we perform the adaptation process on the mask predictions. Meanwhile, we adopt the pseudo labels and the self-supervised framework of SAC[I] for the implementation of DenseCL.IP
40 and FreeSeg [38] with SAC [I]
16

---

Table 8: Effectiveness of Batch Size.
Batch Size
SYNTHIA + UAVID
GTA + UAVID
Road
Building
Car
Tree
Person
mloU
Road
Building
Car
Tree
Terrain
Person
mloU
25.5
58.9
46.5
29.2
15.9
35.2
17.6
50.6
29.7
26.0
41.1
22.4
31.2
35.3
68.9
57.6
54.4
20.9
47.4
25.1
57.6
39.3
45.5
44.9
26.2
39.8
38.4
76.1
62.8
62.1
21.8
52.2
29.2
67.1
45.2
56.6
48.5
27.9
45.7
3 Ablation Study
Effectiveness of Batch Size Table 88 illlstrates the impact of the batch size the performance of cross-vies view domain adaptation. By increasing the batch size, the mloU performance also increased accordingly on both benchmarks. This result has illustrated that the small batch size could not have enough samples to approximate the subspace that represents geometric structures. Meanwhile, the subspace created from the large batch size will be ale to capture the geometric structure of drone-view scenes. However, due to the limitation of GPU resources, we could not evaluate the cross-view adaptation model with larger batch size.
Subspace Representation of Geodesie Flow-based Metries To illustrate the ability of structural learning of our geodesic flow-based metrics, we use subset of images of the cir-view and the drene-view dataset to visualize the base structure of subspaces obtained from the PCA algorithm. Fig. 7 visualizes the mean structures of car-vies car-view and drone-view images. As shown in Fig. 7 The subspaces of car-view images represent the geometric structures of car-view data, i.e., the road in the middle, buildings, trees on two sides, etc. Meanwhile, the geometric structures of the drone view have also been illustrated the figure with structures and topological distributions objects (e.g., the road in the middle and trees and buildings geometric structures of the car-view and the drone-view data. Then, by modeling the geodesic flow path across two subspaces, our metric is able measure the cross-view geometric structural changes (i.e., the change of structures and topological layouts of the scene) from the car view to the drone view. Our experimental results in other ablation studies have further confirmed our effectiveness in geometric structural modeling across views. Figure X illustrates the feature distributions with and without our proposed approach. As shown in Figure 8] our approach can help improve the feature representations of classes, and the cluster of each class is more compact, especially in classes of car, tree, and person.
the sides) the scenes. The results have illustrated the base
4 Discussion of Limitations and Broader Impact
Limitations. In our paper, we have specified a set of hyper-paramedt parameters and network designs to support our hypothesis and theoretical analysis. However, our proposed approach could potentially consist of several limitations. First, our work focuses on studying the impact of cross-view geometric adaptation loss and view-condition prompting mechanisms on the segmentation models across views. The balanced weights among weights, i.e., '5 and Xp, have not been fully exploited. We leave this investigation as our future experiments. Second, although the datasets and benchmarks used in our
Structure Car-View Image
Structure Drone-View Image
Figure 7: The Structures of Subspaces of Car- View and Drone- View Dataset Learned From Subset of Images.
17

---

Without Adaptation
Cross-view Adaptation
Road
Building
15-
Tree Person
10-
10
-10-
-10-
-40
-20
20
40
-7.5
-5.0
-2.5
0.0
2.5
5.0
7.5
10.0
12.5
Figure 8: The Feature Distribution Classes in SYNTHIA - UAVID Experiments.
experiments have sufficiently illustrated the effectiveness of our proposed cross-view adaptation learning approach, the lack of diverse classes and categories in datasets is also a potential limitation. Third, the hypothesis of the linear relations across views of images and segmentation mask, i.c., a, and textual representations and segmentation masks, i.e., 7, could limit the performance of the relation. The non-trivial relations across views should be deeply exploited in future research. Also, while the implementation of Fask2Former and FreeSeg is adopted to develop our approach, the experiments with other open-vocab segmentation networks should be considered in subsequent research studies. These aforementioned limitations will motivate new studies to further improve the methodology, datasets, and benchmarks the cross-view view adaptation learning paradigm.
Broader Impact. Our paper could bring significant potential for various applications that require learning across camera viewpoints. Our approach enables generalizability across camera views. thus enhancing the robustness of the segmentation model across views. In addition, our approach helps to reuse off -hf-lff large-scale data while reducing the effort of manually labeling data of new camera views.
5 Other Related Work
While the important and closely related work to our approach has been presented in our main paper, we also would like to review some other research studies that related to our method follows. In particular, Brady et al. (22 presented a cross-view transformer that learns the camera-aware positional embeddings. Although the views captured from left and right angles, the camera positions in the approach remain at the same altitude. Similarly, Pan et al. [36] present a View Parsing Network to accumulate features across first-view observations with multiple angles. Yao et al. [66] proposed semi-supervised learning approach to learn the segmentation model from multiple views of an image. Huang al. (26) cross-style regularization for domain adaptation in panoptic segmentation by imposing the consistency of the segmentation between the target images and stylized target images. Wang al. (62) proposed a viewpoint adaptation framework for the person re-identification problem by using the generative model to generate training data across various viewpoints. Hou et al. [22] presented matching cross-domain data approach domain adaptation in visual classification. Sun et al. [48] proposed cross-view facial expression adaptation framework to parallel synthesize and recognize cross-view facial expressions. Goyal et [19 introduced cross-view view action recognition approach to transferring the feature representations to different views. Zhang et al. [70] proposed multi-view crowd counting approach that adaptively chooses and aggregates multi-cameras and noise view regularization. Armando et al. (2) proposed self-supervised pre-training approach to human understanding learned on pairs of images captured from different viewpoints. Then, the pre-trained models are later used for various downstream human-centric tasks. In summary, these prior cross-view methods could require either a pair of cross-view images [2] or images captured at the same altitude with different angles [26) 72 [2) In addition, the cross-view geometric correlation
modeling has not been exploited in these prior studies [26] 72 22
18

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility. transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and precede the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justifiiaton right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]", ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS paper checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: The claims declared in the abstract match with the contributions, experimental results, and scope of the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
19

---

Justification:
The limitations of the paper are discussed in the last section of the paper.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approxinations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification:
The proof of formula is provided in the supplcmentary.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: The details datasets and implementations are presented in the experimental sections.
Guidelines:
20

---

The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome deseribe the particular way they provide for reproducibilityy In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [NA]
Justification:
The code will be published may the paper be accepted.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
stteeeeuubmissioooooooooo for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https c/iips.cc/public//uue
CodeeSumiiii wwabbbmmsssiinii ssionPolicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
21

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification:
The details of training and testing are presented in the experimental section.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [NA]
Justification: Following the standard evaluation of semantic segmentation, we evaluate our model by the standard mloU metries instead of the statistical tests.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split. initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еттог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The computational resources used in our experiments are presented in the experimental section.
Guidelines:
The answer NA means that the paper does not include experiments.
22

---

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: httpp::/mmm http://///.ooossscc////////aaaaa
Answer: [Yes]
Justification:
The content of the paper and datasets strictly follows the NeurlPS Code of
Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification; The paper does not have negative societal impact. The broader impact is discussed the appendix.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
23

---

Answer: [NA]
Justification: The paper does not have a risk. The released models will be available may the paper be accepted.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-us should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The paper provides all the references code, data, and models used in the paper.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, pappeswii thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not introduce the new dataset. The code the paper will be published may the paper be accepted.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/mmddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.

---

14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The research in this paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The research in this paper does not involve crowdsourcing nor research with human subjects. Thus, there is no requirement for IRB.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
25


Reviews:
[{"review_id": "9kHmJDDadh", "reviewer": "Reviewer_upJj", "summary": "The paper tackles a problem called cross-view semantic segmentation by using unsupervised domain adaptation methods. The cross-view means from the front-view to top-down view, ie, from car to drone. It recognizes the limitations of existing unsupervised domain adaptation and open-vocabulary semantic segmentation methods in handling geometric variations across different camera views. The paper demonstrates the effectiveness of the proposed approach through extensive experiments on cross-view adaptation benchmarks, achieving state-of-the-art performance compared to existing UDA methods.", "questions": "How is the difference of UDA settings between CROVIA and the proposed in this paper?\n\n\nIn the introduction, the authors claim that “However, the open-vocab perception models remain unable to generalize across camera viewpoints.” It would be better to show the limitation or performance along with this observation. For example, the model with and without using a cross-view setting. \n\n\nFrom eq 3 to 5, for the purpose of geometric adaptation on unpaired data, the problem is that the matrices between source and target domain data are not available. How about using estimated camera matrices for the geometric correlation in eq 3 and 4?\n\n\nThe cross-view learning framework is based on the geodesic flow. How about the comparison with other view transformation methods? For example, in the birds-eye-view semantic segmentation domain, there are different transformation methods, such as: \n[1] Pan, B., Sun, J., Leung, H. Y. T., Andonian, A., & Zhou, B. (2020). Cross-view semantic segmentation for sensing surroundings. IEEE Robotics and Automation Letters, 5(3), 4867-4873.\n\n\nIn Table 4, how about the comparison for BDD→ UAVID? Also, how about applying tye cross-view setting for other DA methods that are proposed for front-view? \n\n\nHow about using the state-of-the-art open-vocab methods? For example, x-decoder, cat-seg, clip-as-RNN, to name a few. \n\n[2] Zou, X., Dou, Z. Y., Yang, J., Gan, Z., Li, L., Li, C., ... & Gao, J. (2023). Generalized decoding for pixel, image, and language. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 15116-15127).\n\n[3] Cho, S., Shin, H., Hong, S., Arnab, A., Seo, P. H., & Kim, S. (2024). Cat-seg: Cost aggregation for open-vocabulary semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 4113-4123).\n\n[4] Sun, S., Li, R., Torr, P., Gu, X., & Li, S. (2024). Clip as rnn: Segment countless visual concepts without training endeavor. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 13171-13182).\n\n[5] Wysoczańska, M., Siméoni, O., Ramamonjisoa, M., Bursuc, A., Trzciński, T., & Pérez, P. (2023). Clip-dinoiser: Teaching clip a few dino tricks. arXiv preprint arXiv:2312.12359.\n\n\nIn Table 4, what is the reason that the method with DAFormer outperforms the one with Mask2Former. It is also very interesting why applying a larger or advanced model cannot obtain improvements. \n\n\nIn Remark 3, the grassmannn manifold is presented for the cross-view modeling method, how about the two subspaces of source and target domain? What is the difference of the feature distributions before and after domain adaptation, such as t-sne visualization? \n\n\nIt would be suggested to showcase the training process and the computational complexity of the proposed methods individually, such as in the ablation study. Apart from performance, training and runtime are also important for UAVs application.", "limitations": "The target domain is limited in only one dataset. There are different UVA datasets public available. But this is not a very large concern, because the experiments are conducted with three different source domains.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The authors propose a novel unsupervised cross-view adaptation approach. This approach includes a cross-view geometric constraint to model structural changes between images and segmentation masks, a geodesic flow-based correlation metric for efficient geometric comparison, and a view-condition prompting mechanism to enhance the capabilities.\n\n\nThe proposed method has significant gains as compared to the initial method CROVIA for car-to-drone scene segmentation UDA. Although this is not a new setting, the improvement compared to the baseline is good. \n\nThe experiments are conducted in three settings of the car-to-drone UDA, and it also includes ablation study to verify effect of different components.", "weaknesses": "The comparison between the other view transformation methods and the proposed cross-view learning method is not included. \n\nMost of the UDA methods are tailored for the front-view street scenes, like AdvEnt, ProDA, SAC. More recent and advanced open-vocab semantic segmentation methods should be included in the experiment section. For example, x-decoder, cat-seg, clip-as-RNN, etc.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer upJj,\n\n\nWe greatly appreciate your insightful review and valuable feedback.\nWe are very happy you think ***our proposed approach is efficient and achieves significant experimental results***.\nWe appreciate your constructive comments and would like to address these points as follows.\n\n\n[Q1] **Comparison with other view transformation**\n\n\n[A1] The method in [1] requires depth for training which is unfaired compared to our approach. Please refer to **[KP1]** for our comparison with other view transformation methods.\n\n\n[Q2] **Results of Advanced Open-vocab Semantic Segmentation**\n\n\n[A2] The training code of clip-as-RNN [2] is not available. Please refer to **[KP2]** for our results using Cat-Seg [3]. Due to the time limitation of the rebuttal period, we leave the investigation of other open-vocab semantic segmentation [4, 5] in our future work.\n\n\n\n\n[Q3] **Difference of UDA settings between CROVIA and EAGLE**\n\n\n[A3] For the unsupervised cross-view adaptation settings, for fair comparison, we follow similar evaluation settings. However, the CROVIA [6] paper only focuses on the cross-view adaptation setting in semantic segmentation. Our work also considers the cross-view adaptation setting on open-vocab segmentation (Tables 5-7).\n\n\n[Q4] **Results of models with and without using cross-view setting**\n\n\n[A4] In Table 2, we have illustrated the performance with and without cross-view adaptation of open-vocab segmentation. In Table 5, we have also reported the performance of DenseCLIP and FreSeg without cross-view (the first row of each group in Table). Please refer to **[KP2]** for additional results of CatSeg.\n\n\n[Q5] **Using estimated camera matrices for the geometric correlation in Eqns (3) and (4)**\n\n\n[A5] Thank you very much for your suggestion. First, we clarify our cross-view modeling approach. From Eqn (3) to (4), it illustrates our approach to cross-view modeling by considering the transformation between two camera views. Then, Eqn (4) illustrates the necessary condition to explicitly model the cross-view geometric correlation under our analysis and assumption mentioned in L182-187. However, we acknowledge that using estimated camera matrices for the geometric correlation in Enq (3) and (4) could be a potential direction for further improvement. Due to the scope of our paper, we leave this investigation as our future work.\n\n\n\n\n[Q6] **Results of other domain adaptation methods on BDD $\\to$ UAVID**\n\n\n[A6] In Table 7, we have reported the experimental results of cross-view adaptation setting on prior adaptation methods on BDD $\\to$ UAVID, including BiMaL and DAFormer. Due to the time limitation of the rebuttal period, we leave the investigation of other DA methods on BDD $\\to$ UAVID as our future work.\n\n\n[Q7] **Reason of DAFormer outperforming Mask2Former.**\n\n\n[A7] Our investigation reveals two reasons that could lead to the lower performance of Mask2Former compared to DAFormer. First, the network backbone of DAFormer is Transformer, while the backbone of Mask2Former in our experiments is ResNet 101. As shown in Table 1, if we use Swin as the backbone of Mask2Former, the results are improved compared to ResNet-101. Second, DAFormer adopts the convolution-based decoder. Meanwhile, Mask2Former adopted a Transformer for the decoder which could require more data for better generalization. Therefore, DAFormer in our experiments performs better than Mask2Former.\n\n\n\n\n[Q8] **Two subspaces of source and target domain and feature distributions before and after adaptation**\n\n\n[A8]  In Remark 3, the Grassmannn manifold is presented and will be used to model the Cross-view Structural Change where the source domain is car-view and the target domain is drone-view. The two subspaces of source and target domain are obtained via the PCA algorithms. The details of subspaces are presented in our Implementation in the appendix (L638-L645). To illustrate the feature distributions, we use the features of the last layer before the classifier in the SYNTHIA $\\to$ UAVID experiments. We compare our approach without and with cross-view adaptation. As shown in Figure 3 in the rebuttal pdf, our approach can help to improve the feature representations of classes, and the cluster of each class is more compact, especially in classes of car, tree, and person.\n\n\n\n\n[Q9] **Computational Complexity**\n\n\n[A9] In our proposed approach, the cross-view adaptation loss is only performed during the training. The computational time of our geodesic loss is approximately 0.0195 seconds per iteration. Meanwhile, in the practical deployment, the computational cost of our model relies on the segmentation network. The computational testing cost only relies on the segmentation network (e.g., DeepLab: ~776.2 GFLOPS and DAFormer: ~1447.6 GFLOPS).\n\n\n[Q10] **Target domain is limited in only one dataset**\n\n\n[A10] We have mentioned this limitation in our discussion. Please refer to **[KP3]** for results of an additional target domain.\n\n\n\n\nReferences\n\n\n[1]  Pan, B., et al. Cross-view semantic segmentation for sensing surroundings. IEEE Robotics and Automation Letters, 2020.\n\n\n[2] Sun, S. et al. Clip as rnn: Segment countless visual concepts without training endeavor. CVPR, 2024.\n\n\n[3] Cho, S., et al. Cat-seg: Cost aggregation for open-vocabulary semantic segmentation. CVPR 2024.\n\n\n[4] Zou, X., et al. Generalized decoding for pixel, image, and language. CVPR, 2023.\n\n\n[5] Wysoczańska, M., et al. Clip-dinoiser: Teaching clip a few dino tricks. arXiv, 2023.\n\n[6] T.D. Truong, et al. CROVIA: Seeing Drone Scenes from Car Perspective via Cross-View Adaptation, arXiv 2024.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vlpNEXAhdg", "reviewer": "Reviewer_8Etw", "summary": "This work proposed a novel unsupervised adaptation method for modeling structural change across different views. Additionally, the paper introduced a new metric for cross-view changes and a new prompting mechanism for cross-view open vocabulary segmentation. Through extensive experiments, the paper shows SoTA performance compared to previous unsupervised methods in cross-view semantic scene understanding.", "questions": "1.The performance of the building category in open-vocab segmentation on unseen classes is intriguing.\n2.The notation \"ν ∈ [0..1] → Π(ν)\" in line 241 is misleading.", "limitations": "The authors discussed the limitation of this paper in the appendix, including the choice of hyperparameters, lack of more diverse class labels and flaws in their original mathematical hypothesis.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.The proposed method can be trained on unpaired data, enhancing its potential for practical applications.\n2.The paper proposed a new cross-view change modeling method via geodesic flow path, which effectively models the cross-view segmentation correlation.\n3.The newly introduced method demonstrated SoTA performance across various cross-view adaptation benchmarks.\n4.The paper provides thorough mathematical derivations and theoretical analysis, along with well-designed experiments.", "weaknesses": "1.The visualization results in this paper are limited. It is necessary to supplement more visualizations of segmentation results compared with the existing methods in the paper, which will help support the experimental conclusions.\n2.Generalizing the experiments to include more UAD datasets, such as SynDrone[1], UDD[2], and ICG Drone[3], would enhance the robustness and credibility of the results.\n3.The motivation behind the design of the adaptation loss should be explained more clearly.\n4.The performance improvement in open-vocab segmentation is not very significant.\n\n[1] SynDrone – Multi-modal UAV Dataset for Urban Scenarios. \n[2] Large-scale structure from motion with semantic constraints of aerial images.\n[3] ICG Drone Dataset.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer 8Etw,\n\n\nWe are grateful for your careful reading and constructive feedback.\nWe appreciate your highlighting that ***our proposed approach is efficient and well-designed, our problem is practical, and the method achieves solid performance***.\nWe appreciate your constructive comments and would like to address these points as follows.\n\n\n[Q1] **Additional Visualization Results**\n\n\n[A1] To further illustrate our qualitative results, we add more visualization results (Figure 1 and Figure 2) in the pdf file of our rebuttal. In addition, we will release more qualitative results in the supplementary of our paper. We will also release our implementation for the reproducibility of both quantitative and qualitative results.\n\n\n[Q2] **Generalizing the experiments to include more UAV datasets**\n\n\n[A2]  We chose UAVID since these datasets have a great overlap with the source dataset (GTA, SYNTHIA, and BDD). We acknowledge the suggestion of the reviewer. Please refer to **[KP3]** for additional results of cross-view adaptation on UDD [1]. Due to the time limitation of the rebuttal period, we leave the experiments and investigation of other benchmarks of SynDrone [2] and ICG Drone [3] in our future work. It should be noted that we have also discussed the limitations of the dataset in the appendix.\n\n\n\n\n[Q3] **Performance of the building category in open-vocab segmentation on unseen classes**\n\n\n[A3] For clarification, in Table 5, all classes are used during training. In Table 6, we consider the classes of Tree and Person as the unseen classes. Therefore, the performance of building categories in our approach is valid and reasonable due to the effectiveness of our proposed method.\n\n\n[Q4] **Motivation behind the design of the adaptation loss**\n\n\n[A4] In our approach, we propose the adaptation loss by modeling the Cross-view Structural Change between source (car-view) and target (drone-view) domains. In particular, we first analyze the cross-view geometric correlation between two domains by analyzing the change of camera views and the equivalent transformation between image and segmentation output. Then, we propose to model the cross-view structural change via the geodesic flow. In particular, our loss measures the cross-view structural changes between the source and the target domains based on their manifold structures via the geodesic flow.\n\n\n[Q5] **The performance improvement in open-vocab segmentation is not very significant**\n\n\n[A5] We respectfully but strongly disagree with the reviewer on this point. As shown in Table 5, the performance of our open-cab experiment remains significant. For example, when we use our cross-view learning loss integrated with DenseCLIP (ViT) and FreSeg, the performance is up to 45.8% and  51.3 % on SYNTHIA $\\to$ UAVID. Meanwhile, without our cross-view learning, the performance is only 21.8% and 23.4%.\nOur results still outperform prior adaptation methods (i.e., AdvEnt and SAC). Performance is even further improved when using our view-condition prompting.\n\n\n[Q4] **Misleading Notation**\n\n\n[A4] Thank you very much for your feedback. We will update our notation promptly.\n\n\nReferences\n\n\n[1] Y Chen, et al. Large-scale structure from motion with semantic constraints of aerial images. PRCV, 2018.\n\n\n[2] G Rizzoli, et al. SynDrone – Multi-modal UAV Dataset for Urban Scenarios. ICCVW, 2023.\n\n\n[3]  ICG Drone Dataset. http://dronedataset.icg.tugraz.at/.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kCHI8hCkyF", "reviewer": "Reviewer_JhPV", "summary": "The paper introduces a novel method for Unsupervised Domain Adaption to adapt an open-vocabulary segmentation model across different views. To achieve this, the authors introduce a cross-view geometric constraint that captures structural changes between different views. Further, a Geodesic Flow-based Metric is introduced to measure the structural changes across scenes. They also adapt the prompting scheme to take into account the change in viewpoints.", "questions": "Personally, I would appreciate if the authors could further improve the presentation of the paper. While the technical contributions are solid and the writing is nice, the visualizations and tables would benefit from being larger and better integrated with the text. For example, by moving Table 6 to the supplementary, the authors could free up space for the very relevant Table 5. I would appreciate an improved presentation, but also understand that this can be challenging with the given space constraints.", "limitations": "The authors have sufficiently addressed the limitations of their approach.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "- The paper feature a good range of contributions, which are intuitive and interesting\n- The proposed task of unsupervised cross-view adaptation is novel and seems useful\n- The idea is well motivated \n- The paper is well written and clearly explains the proposed concepts\n- Table 2 nicely outlines the impact of the individual technical contributions", "weaknesses": "- The layout of the paper seems a bit cramped. It would have been nice to have larger visualizations. Also, many of the tables, especially those in the ablations, seem super cramped and the reader has to zoom in significantly. I understand the authors wanted to put as much information as possible into the paper, but this led to a presentation that is not top notch.\n- Table 3 has no visual separation between the results for different classes, and overall mIoU. This visual separation (e.g. vertical line) would be nice to have.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer JhPV,\n\n\nWe greatly appreciate your insightful review and valuable feedback.\nWe are very happy you encourage that ***our paper is well-written and features a good range of contributions, our problem is meaningful, and our idea is well-motivated***.\nWe appreciate your constructive comments and would like to address these points as follows.\n\n\n[Q1] **Layout of Paper**\n\n\n[A1] Thank you very much for your feedback. We will update the layout organization of our paper for better readability.\n\n\n[Q2] **Visual Separation in Tables**\n\n\n[A2] Thank you very much for your feedback. Per your suggestions, we will add more space to the tables for better readability.\n\n\n[Q3] **Free Up Space and Improve Presentation**\n\n\n[A3] Thank you very much for your feedback. We will update our paper according to your suggestions to improve our presentation and the readability.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1jkhUH7h59", "reviewer": "Reviewer_29y1", "summary": "This paper proposes a domain adaptive semantic segmentation method under the cross-view (front view to top view) setting. It addresses this using vision-language models for additional supervision. It proposes a cross-view geometric constraint to model the structural changes and similarities between two vastly different viewpoint. The paper presents detailed quantitative results, with comparisons and ablations", "questions": "Since it is not possible to provide qualitative results in the rebuttal, and since it not possible to properly validate the effectiveness of the proposed semantic segmentation method without qualitative results, I am rejecting the paper at this stage.", "limitations": "Yes, in page 9", "rating": 3, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper addresses an important and practical problem.\n\n2. The method proposed in the paper is well-tailored for the problem at hand, and appears to be solid. It seems intuitive and logical.\n\n3. The quantitative results presented clearly show the efficacy of the method.", "weaknesses": "My main problem with the paper is the lack of qualitative results. The main paper barely has any qualitative results. The supplementary video shows just one specific video. I am unable to judge the performance of the method without good qualitative analysis.\n\nAs in all semantic segmentation papers, it would be good to see where prior work fails, and how each of the components presented in the method help the case. Essentially, it is important to provide qualitative results for the comparisons, the ablations, and as well as the method.\n\nGiven that the performance improvement as depicted by the quantitative results is large, the qualitative improvements should be clearly visible too, and hence it is beneficial to add those results.\n\nAlso, one main motivation for using a VLM is the open set setting (as described in the abstract). While described as a key point in the paper, none of the experimental benchmarks deal with that.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer 29y1,\n\n\nWe would like to express our gratitude for your careful reading and valuable feedback.\nWe are very happy you encourage that ***our paper addresses an important and practical problem, our proposed approach is well-tailored for the problem and appears to be solid and logical, and our approach also achieves solid quantitative performance***.\nIn addition, we would like to emphasize that other reviewers have also encouraged ***our quantitative results in our experiments and ablation studies are well-designed to illustrate the effectiveness of our proposed approach*** (Reviewer 8Etw, upJj, and JhPV). We appreciate your constructive comments and would like to address these points as follows.\n\n\n[Q1] **Lack of qualitative results**\n\n\n[A1] We have included the qualitative results of our ablation study in the pdf file of our rebuttal. In the model without prompting, Figure 1 illustrates the results of our cross-view adaptation compared to without adaptation. As shown in the results, our approach can effectively segment the objects in the drone view. We also compare with the prior ProDA [1] method in this figure. Our qualitative results remain better than the prior adaptation method. For the model with prompting, Figure 2 illustrates the effectiveness of our approach in 3 cases, without adaptation, with cross-view adaptation, and with view-condition prompting. As shown in the results, our cross-view adaptation can efficiently model the segmentation of the view. By using the view-condition prompting, our model can further improve the segmentation of persons and vehicles. The images are better viewed in color and 2x zoom. In addition to the quantitative results in Table 2, our qualitative results further confirm the effectiveness of our proposed approach. **We will release our implementation for the reproducibility of both quantitative and qualitative results.** We will release more qualitative results and comparisons in the final version of our paper.\n\n\n\n\n\n\n[Q2] **Using a VLM is the open-set setting**\n\n\n[A2] In Tables 5-6 in our paper, we report the performance of open-vocab segmentation (i.e. DenseCLIP, FreSeg). Specifically, The DenseCLIP model is developed based on the vision-language model (i.e., CLIP). These experimental results show the limited performance of these methods in the cross-view settings. The results also illustrate the effectiveness of our proposed approach in an open-vocab segmentation setting.\nIn Table 6, we also present our experimental results in the open-set setting where classes of 'Tree' and 'Person' are considered as the unseen classes (which are not used during training).\nWe would like to highlight that other reviewers encourage ***our experiments and ablation studies are well-designed for contributions*** (Reviewer 8Etw, upJj, and JhPV) and ***our experimental results are solid*** (Reviewers upJj, 8Etw, and JhPV).\n\nReferences\n\n[1] P Zhang, et al. Prototypical pseudo label denoising and target structure learning for domain adaptive semantic segmentation. CVPR, 2021.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
## Global Response


We would like to thank all the reviewers for their careful reading and invaluable feedback.
Reviewer JhPV ***accepts*** our paper;
Reviewer upJj and Reviewer 8Etw ***weakly accept***;
and Reviewer 29y1 consider a ***reject*** at this stage.
We appreciate the reviewers encouraged that
***our problem is useful and practical*** (Reviewers 8Etw, JhPV, and 29y1),
***our proposed approach is intuitive and logical, appears to be solid*** (Reviewer 8Etw, JhPV, and 29y1).
and ***our experimental results are solid*** (Reviewers upJj, 8Etw, KhPV, and 29y1).
We have also updated our typos and suggested references in our paper.


On the constructive side, Reviewer 29y1 and Reviewer upJj suggest additional visualization to confirm the effectiveness of the proposed approach. We have included a rebuttal Figure PDF, including a visualization of our qualitative results and feature distributions. In addition, as suggested by Reviewers upJj and 8Etw, to further illustrate the effectiveness of our proposed approach, we conduct additional experiments as follows:


**[KP1] Comparison with Other View Transformation.** To further illustrate our effectiveness compared to other view transformation methods, we compare the results of our approach with another view transformation using DeepLab, i.e., Polar Transformation in [1, 2].  As shown in the table below, our approach outperforms polar transformation.


|  | Road | Building | Car | Tree | Terrain | Person | mIoU |
|---|---|---|---|---|---|---|---|
| BDD $\to$ UAVID |  |  |  |  |  |  |  |
| Polar Transform | 21.1 | 9.6 | 36.4 | 24.1 | 14.6 | 4.6 | 18.4 |
| **EAGLE** | **24.0** | **53.8** | **39.0** | **52.2** | **48.3** | **16.9** | **39.0** |
| SYNTHIA $\to$ UAVID |  |  |  |  |  |  |  |
| Polar Transform | 20.5 | 10.9 | 38.2 | 22.6 | - | 4.3 | 19.3 |
| **EAGLE** | **29.9** | **65.7** | **55.5** | **56.8** | **-** | **18.3** | **45.2** |
| GTA $\to$ UAVID |  |  |  |  |  |  |  |
| Polar Transform | 19.4 | 9.1 | 37.8 | 20.7 | 15.6 | 2.5 | 17.5 |
| **EAGLE** | **20.5** | **53.0** | **37.6** | **50.7** | **45.3** | **13.0** | **36.7** |


**[KP2] Results of Advanced Open-vocab Semantic Segmentation.** We reproduce the results of Cat-Seg [3] as shown in the following table. Overall, our cross-view adaptation can significantly improve the performance of open-vocab semantic segmentation models.


|  | Road | Building | Car | Tree | Person | mIoU |
|---|---|---|---|---|---|---|
| CatSeg | 19.9 | 27.3 | 22.8 | 33.1 | 10.7 | 22.8 |
| CatSeg+CrossView | 37.2 | 73.0 | 61.8 | 61.0 | 20.1 | 50.6 |
| EAGLE | 36.8 | 75.5 | 61.3 | 60.8 | 21.2 | 51.1 |
| EAGLE+ViewCondition | **38.4** | **76.1** | **62.8** | **62.1** | **21.8** | **52.2** |


**[KP3] Results of Additional Target Dataset**. To further illustrate the effectiveness of our approach, we conduct an additional cross-view adaptation using the UDD dataset [4], i.e., SYNTHIA $\to$ UDD, with 4 classes of 'Tree', 'Building', 'Road', and 'Vehicle'. We adopt the DeepLab segmentation network in our experiments. As shown in our experiments, our approach outperforms the prior adaptation method and closes the gap with supervised learning.




|  | Tree | Building | Road | Vehicle | mIoU |
|---|---|---|---|---|---|
| No Adapt | 19.66 | 13.50 | 9.66 | 6.55 | 12.34 |
| BiMaL | 35.61 | 29.71 | 21.17 | 19.05 | 26.38 |
| **EAGLE** | **48.27** | **44.07** | **41.39** | **38.60** | **43.08** |
| Upper Bound | 70.37 | 77.79 | 78.42 | 68.56 | 73.79 |




In addition, we have provided comprehensive responses to queries raised by the reviewers. We hope our explanations can effectively address the reviewers' concerns. Once again, we want to thank all the reviewers for their valuable and insightful feedback. We sincerely hope that our efforts will result in a favorable reconsideration of the scores by the reviewers.


References


[1] Y. Shi, et al. Spatial-aware feature aggregation for image based cross-view geo-localization. NeurIPS, 2019.


[2] Y. Shi, et al. Where am I looking at? joint location and orientation estimation by cross-view matching. CVPR, 2020.


[3]  Cho, S., et al. Cat-seg: Cost aggregation for open-vocabulary semantic segmentation. CVPR 2024.


[4] Y Chen, et al. Large-scale structure from motion with semantic constraints of aerial images. PRCV, 2018.

Meta-review: Two of the reviewers recommended weak accept, and the others accept and reject. While 29y1 recommended rejection, there comments during the discussion after reading the rebuttal indicated that their only remaining concern was regarding the lack of qualitative comparisons to more recent methods. 

Given the above, this AC agrees with the reviewers and recommends that the paper is accepted. The authors are strongly encouraged to take the reviewer recommendations and suggestions into account when revising the paper for the camera ready version. Specifically, they should focus on the following:  
* Update the text to fix the cases where the text/descriptions/notation were not clear to the reviewers.   
* The paper is very cramped, use the extra page in the camera ready version to space it out more so that it is more readable.
* Add the comparisons to other view transformation methods provided to upJj.  
* Add the additional qualitative comparisons to newer methods (i.e. not just the older methods displayed in the rebuttal) requested by 29y1 and 8Etw. Fig. 3 from the rebuttal PDF is fine for the appendix, but not necessary to add it to the main paper. 
* Update the related work text so that the difference to Crovia by Truong et al. is more clearly defined.  

Minor comments:
* Fig. 1 caption “could not perform well” -> “do not perform well”
* Fig. 2 It is not clear what method is being shown here. Improve the caption.
Decision: Accept (poster)