EAGLE: Efficient Adaptive Geometry-based Learning in Cross-view Understanding

Authors: thanh-dat  truong
utsav  prabhu
dongyi  wang
bhiksha  raj
susan  gauch
jeyamkondan  subbiah
khoa  luu
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-13 07:58:50

Abstract: Unsupervised Domain Adaptation has been an efficient approach to transferring the semantic segmentation model across data distributions. Meanwhile, the recent Open-vocabulary Semantic Scene understanding based on large-scale vision language models is effective in open-set settings because it can learn diverse concepts and categories. However, these prior methods fail to generalize across different camera views due to the lack of cross-view geometric modeling. At present, there are limited studies analyzing cross-view learning. To address this problem, we introduce a novel Unsupervised Cross-view Adaptation Learning approach to modeling the geometric structural change across views in Semantic Scene Understanding. First, we introduce a novel Cross-view Geometric Constraint on Unpaired Data to model structural changes in images and segmentation masks across cameras. Second, we present a new Geodesic Flow-based Correlation Metric to efficiently measure the geometric structural changes across camera views. Third, we introduce a novel view-condition prompting mechanism to enhance the view-information modeling of the open-vocabulary segmentation network in cross-view adaptation learning. The experiments on different cross-view adaptation benchmarks have shown the effectiveness of our approach in cross-view modeling, demonstrating that we achieve State-of-the-Art (SOTA) performance compared to prior unsupervised domain adaptation and open-vocabulary semantic segmentation methods.

Main paper: 
 introduction: 
 Figure 1 : Our Proposed Cross-view Adaptation Learning Approach. Prior models, e.g., FreeSeg [38] , DenseCLIP [40] , trained on the car view do not perform well on the drone-view images. Meanwhile, our cross-view adaptation approach is able to generalize well from the car to drone view.Modern segmentation models [3, 4, 63] have achieved remarkable results on the close-set training with a set of pre-defined categories and concepts. To work towards human-level perception where the scenes are interpreted with diverse categories and concepts, the openvocabulary (open-vocab) perception model [38, 40] based on the power of large vision-language models [30, 39] has been introduced to address the limitations of close-set training. By using the power of language as supervision, the largescale vision language model is able to learn the more powerful representations where languages offer better reasoning mechanisms and openword concept representations compared to traditional close-set training methods [3, 63, 9] . Recent work is inspired by the success of large vision-language models [39, 27] that are able to learn informative feature representations of both visual and textual inputs from large-scale image-text pairs. These have been adopted to further develop open-vocab semantic segmentation models [38, 40, 31, 29] that can work well in open-world environments. However, the open-vocab perception models remain unable to generalize across camera viewpoints. As shown in Fig. 1 , the open-vocab model trained on car views is not able to perform well on the images captured from unmanned aerial vehicles (UAVs) or drones. While this issue can be improved by training the segmentation model on drone-view data, the annotation process of high-resolution UAV data is costly and time-consuming. At present, there exist many large-scale datasets with dense labels captured from camera views on the ground, e.g., car views (SYNTHIA [44] , GTA [43] , Cityscapes [11] , BDD100K [68] ). They have been widely adopted to develop robust perception models. Since these car view and drone view datasets have many common objects of interest, incorporating knowledge from car views with drone views benefits the learning process by reusing large-scale annotations and saving efforts of manually labeling UAV images. Unsupervised domain adaptation (UDA) [58, 23, 1, 51, 53 ] is one of the potential approaches to transfer the knowledge from the car view (i.e., source domain) to the drone view (i.e., target domain). While UDA approaches have shown their effectiveness in transferring knowledge across domains, e.g., environment changes or geographical domain shifts, these methods remain limited in the cases of changing camera viewpoints. Indeed, the changes in camera positions, e.g., from the ground of cars to the high positions of drones, bring a significant difference in structures and topological layouts of scenes and objects (Fig. 2 ). Therefore, UDA is not a complete solution to this problem due to its lack of cross-view structural modeling. Additionally, although the open-vocab segmentation models have introduced several prompting mechanisms, e.g., context-aware prompting [40] or adaptive prompting [38] to improve context learning across various open-world concepts, they are unable to model the cross-view structure due to the lack of view-condition information in prompts and geometric modeling. To the best of our knowledge, there are limited studies that have exploited this cross-view learning. These limitations motivate us to develop a new adaptation learning paradigm, i.e., Unsupervised Cross-view Adaptation, that addresses prior methods to improve the performance of semantic segmentation models across views.Contributions: This work introduces a novel Efficient Adaptive Geometry-based Learning (EAGLE) to Unsupervised Cross-view Adaptation that can adaptively learn and improve the performance of semantic segmentation models across camera viewpoints. First, by analyzing the geometric correlations across views, we introduce a novel cross-view geometric constraint on unpaired data of structural changes in images and segmentation masks. Second, to efficiently model cross-view geometric structural changes, we introduce a new Geodesic Flow-based Metric to measure the structural changes across views via their manifold structures. In addition, to further improve the prompting mechanism of the open-vocab segmentation network in cross-view adaptation learning, we introduce a new view-condition prompting. Then, our cross-view geometric constraint is also imposed on its feature representations of view-condition prompts to leverage its geometric knowledge embedded in our prompting mechanism. Our proposed method holds a promise to be an effective approach to addressing the problem of cross-view learning and contributes to improving UDA and open-vocab segmentation in cross-view learning. Thus, it increases the generalizability of the segmentation models across camera views. Finally, our experiments on three presented cross-view adaptation benchmarks, i.e., SYNTHIA → UAVID, GTA → UAVID, BDD → UAVID, illustrate the effectiveness of our approach in cross-view modeling and our State-of-the-Art (SOTA) performance. 
related work: 
 Unsupervised Domain Adaptation Adversarial learning [6, 58, 59 ] and self-supervised learning [1, 69, 23, 14] are common approaches to UDA in semantic segmentation. The adversarial learning approaches are typically simultaneously trained on source and target data [58, 7, 6] . Chen et al. [7] first introduced an adversarial framework to domain adaptation. Later, several approaches improved adversarial learning by utilizing generative models [74, 34, 21] , using additional labels [28, 59] , incorporating with entropy minimization [58, 65, 51, 52] , or adopting the curriculum training [37] . Recently, the self-supervised approaches [1, 69, 23, 14] have achieved outstanding performance. Araslanov et al. [1] first proposed a self-supervised augmentation consistency framework for UDA. Hoyer et al. [23] utilized Transformers to improve the UDA performance. Later, this approach was further improved by utilizing multi-resolution cropped images [24] and masked image consistency strategy [25] to enhance contextual learning. Recent studies improved the self-supervised approach by aligning both output and attention levels via the cross-domain prediction consistency framework [60] , using a prototypical representation [69] , learning the cross-model consistency via depths [67] , improving the class-relevant fairness [53, 55, 56] , or exploring the relations of pseudo-labels [71] . Fashes et al. [15] introduced a prompt-based feature augmentation method to zero-shot UDA. Gong et al. [18] introduced a geodesic flow kernel to model the manifold structure between domains. Later, Simon et al. [47] designed distillation loss by the geodesic flow path.Vision-Language and Open-Vocab Segmentation By pre-training on a large-scale vision-language dataset [39, 27] , the vision-language models can learn various visual concepts and can further be transferred to other vision problems through "prompting" [17, 38, 31, 35] , e.g., open-vocab segmentation [64, 13, 38] . Li et al. [29] first introduced the language-driving approach to semantic segmentation. Rao et al. [40] represented a context-aware prompting mechanism for dense prediction tasks. Ghiasi et al. [17] proposed an OpenSeg framework that learns the visual-semantic alignments. Qin et al. [38] presented a unified, universal, and open-vocab segmentation network based on Mask2Former [8] with an adaptive prompting mechanism. Xu et al. [64] proposed a two-stage open-vocab segmentation framework using the mask proposal generator and the pre-trained CLIP model. Ding et al. [13] decoupled the zero-shot semantic segmentation to class-agnostic segmentation and segment-level zero-shot classification. Liang et al. [31] improved the two-stage open-vocab segmentation model by further fine-tuning CLIP on masked image regions and corresponding descriptions.The early studies exploited cross-view learning in geo-localization by using a polar transform across views [46, 45] or generative networks to cross-view images [41, 49] . Meanwhile, Zhu et al. [75] exploited the correlation between street-and aerial-view data via selfattention. In semantic segmentation, Coors et al. [10] first introduced a cross-view adaptation approach utilizing the depth labels and the cross-view transformation between car and truck views. However, this change of views in [10] is not as big a hurdle as the change of views in our problem, i.e., car view to drone view. Ren et al. [42] presented an adaptation approach across viewpoints using the 3D models of scenes to create pairs of cross-view images. Vidit et al. [57] modeled the geometric shift in cross FoV setting for object detection by learning position-invariant homography transform. Di Mauro et al. [12] introduced an adversarial method trained on a multi-view synthetic dataset where images are captured from different pitch and yaw angles at the same altitudes of the camera positions. Meanwhile, in our problem, the camera views could be placed at different altitudes (e.g., the car and the drone), which reveals large structural differences between the images. Truong et al. [50, 54] first introduced a simple approach to model the relation across views. CROVIA [50] measures the cross-view structural changes by measuring the distribution shift and only focuses on the cross-view adaptation setting in semantic segmentation. However, these methods [50, 54] lack a theory and a mechanism for cross-view geometric structural change modeling. To the best of our knowledge, there are limited studies exploiting cross-view adaptation in semantic segmentation. Therefore, our work presents a new approach to model the geometric correlation across views.We follow the implementation of Mask2Former [8] and FreeSeg [38] with ResNet [20] and Swin backbones [32] for our segmentation network. In particular, we adopt Mask2Former with Semantic Context Interaction of FreeSeg [38] for our open-vocab segmentation network. We use the pre-trained text encoder of CLIP [39] . The textual features f p s and f p t are obtained by the CLIP textual encoder. Following common practices [38, 31] , we adopt the open-vocab segmentation loss of FreeSeg [38] to our supervised loss L M ask . For experiments without prompting, we use the Mask2Former network. Following the UAV protocol of [61] , the image size is set to 512 × 512. The linear scale factors α and γ are set to α = 1.5 and γ = 1.0, respectively. For the Geodesic Flow modeling, we adopt the implementation of generalized SVD decomposition [18, 47] in the framework. The subspace dimension in our geodesic flow-based metrics is set to D = 256. The batch size and the base learning rate in our experiments are set to 16 and 2.5 × 10 -4 . The balanced weights of losses in our experiments are set to λ I = 1.0 and λ P = 0.5. During training, the classes in the prompts are generated similarly for both view images.In our Geodesic Flow-based metrics, the subspaces of images and ground-truth segmentation of the source domain are pre-computed on the entire data. For the language space, we compute the subspaces of each view based on the textual feature representations of all possible prompts in each domain. Meanwhile, the subspaces of the segmentation on the target domain are computed based on the current batch of training. For the implementation of DenseCLIP [40] and FreeSeg [38] with AdvEnt [58] , we perform the adaptation process on the mask predictions. Meanwhile, we adopt the pseudo labels and the self-supervised framework of SAC [1] for the implementation of DenseCLIP [40] and FreeSeg [38] with SAC [1] .  
the proposed eagle approach: 
 In this paper, we consider cross-view adaptation learning as UDA where the images of the source and target domains are captured from different camera positions (Fig. 2 ). Formally, let x s , x t be the input images in the source and target domains, p s , p t be the the corresponding prompts, and y s , y t be the segmentation masks of x s , x t . Then, the open-vocab segmentation model F maps the input x and the prompt p to the corresponding output y = F (x, p). It should be noted that in the case of traditional semantic segmentation, the prompt p will be ignored, i.e., y = F (x) The cross-view adaptation learning can be formulated as Eqn. (1) .EQUATIONwhere θ is the parameters of F , ŷs is the ground truth, L M ask is the supervised (open-vocab) segmentation loss with ground truths, and L Adapt is unsupervised adaptation loss from the source to the target domain. In the open-vocab setting, we adopt the design of Open-Vocab Mask2Former [8, 38] to our network F . Prior UDA methods defined the adaptation loss L Adapt via the adversarial loss [28, 5] , entropy loss [51, 58] , or self-supervised loss [23, 25] . Although these prior results have illustrated their effectiveness in UDA, these losses remain limited in cross-view adaptation setup. Indeed, the adaptation setting in prior studies [58, 1, 23, 15] is typically deployed in the context of environmental changes (e.g., simulation to real [58, 59, 15] , day to night [25, 15] , etc) where the camera positions between domains remain similar. Meanwhile, in cross-view adaptation, the camera position of the source and target domain remains largely different (as shown in Fig. 2 ). This change in camera positions leads to significant differences in the geometric layout and topological structures between the source and target domains. As a result, direct adoption of prior UDA approaches to cross-view adaptation would be ineffective due to the lack of cross-view geometric correlation modeling. To effectively address cross-view adaptation, the adaptation loss L Adapt should be able to model (1) the geometric correlation between two views of source and target domains and (2) the structural changes across domains.To efficiently address the cross-view adaptation learning task, it is essential to explicitly model cross-view geometric correlations by analyzing the relation between two camera views. Therefore, we first re-reconsider the cross-view geometric correlation. In particular, let xt be the corresponding image of x s captured from the target view, y s and ȳt be the semantic segmentation outputs of source image x s and target image xt , pt be the corresponding prompt of p s in target view, respectively. Formally, the images captured from the source and the target views can be modeled as Eqn. (2) .EQUATIONwhere R is the rendering function, K s and K t are the intrinsic matrices, [R s , t s ] and [R t , t t ] are the extrinsic matrices, and Θ represents the capturing scene. In addition, as the camera parameters of both source and target views are represented by matrices, there should exist linear transformations of camera parameters between two views as follow,EQUATIONwhere T K and T Rt are the transformation matrices.Remark 1: The Geometric Transformation Between Camera Views. From Eqn. (2) and Eqn. (3), we argue that there should exist a geometric transformation T of images between two camera views as: xt = T (x s ; T K , T Rt ).Remark 2: The Equivalent Transformation Between Image and Segmentation Output. As RGB images and segmentation maps are pixel-wised corresponding, the same geometric transformation T in the image space can be adopted for segmentation space as: ȳt = T (y s ; T K , T Rt )Remarks 1-2 have depicted that the geometric transformation of both image and segmentation from the source to the target view can be represented by the shared transformation T with the camera transformation matrices T K , T Rt . Let D x (x s , xt ) and D y (y s , ȳt ) be the metrics the measure the cross-view structures changes of images and segmentation maps from the source to target domains.We argue that the cross-view geometric correlation in the image space, i.e., D x (x s , xt ), is theoretically proportional to the one in the segmentation space, i.e., D y (y s , ȳt ). Since the camera transformations between the two views are linear (Eqn. (3)) and the images x and outputs y are pixel-wised corresponding, we hypothesize that the cross-view geometric correlation in the image space D x (x s , xt ) and the segmentation space D y (y s , ȳt ) can be modeled by a linear relation with linear scale α as follows: where x s and x t are unpaired data, and L Adapt (y s , y t ) = ||D x (x s , x t ) -αD y (y s , y t )|| is the Crossview Geometric Adaptation loss on unpaired data. Intuitively, although the cross-view pair samples are not available, the cross-view geometric constraints on paired samples between two views can be indirectly imposed by modeling the cross-view geometric structural constraint among unpaired samples. Then, by modeling the cross-view structural changes in the image and segmentation spaces, the structural change on images of unpaired data could be considered as the reference for the cross-view structural change in the segmentation space during the optimization process. This action promotes the structures of segmentation that can be effectively adapted from the source view to the target view. Importantly, the cross-view geometric constraint imposed on unpaired data can be mathematically proved as an upper bound of the cross-view constraint on paired data as follows:EQUATION)where O is the Big O notation. The upper bound in Eqn. ( 19) can be proved by using the properties of triangle inequality and our correlation metrics D x and D y (Sec. 3.3). The detailed proof is provided in the appendix. Eqn. ( 19) has illustrated that by minimizing the cross-view geometric constraint on unpaired samples in Eqn. ( 6), the cross-view constraint on paired samples in Eqn. ( 5) is also maintained due to the upper bound. Therefore, our proposed Cross-view Geometric Constraint loss does NOT require the pair data between source and target views during training. Fig. 3 illustrates our cross-view adaptation learning framework. Modeling the correlation metrics D x and D y is an important task in our approach. Indeed, the metrics should be able to model the structure changes from the source to the target view. Intuitively, the changes from the source to the target view are essentially the geodesic flow between two subspaces on the Grassmann manifold. Then, the images (or segmentation) of two views can be projected along the geodesic flow path to capture the cross-view structural changes. Therefore, to model D x and D y , we adopt the Geodesic Flow path to measure the cross-view structural changes by modeling the geometry in the latent space. For simplicity, we present our approach to model the cross-view structural change D x in the image space. Formally, let P s and P t be the basis of the source and target domains. These bases can be obtained by the PCA algorithm. The geodesic flow between P s and P t in the manifold can be defined via the function Π : ν ∈ [0..1] → Π(ν), where Π(ν) ∈ G(N, D) is the subspace lying on the geodesic flow path from the source to the target view:EQUATIONwhere R ∈ R D×(D-N ) is the orthogonal complement of P s , i.e., R ⊤ P s = 0. Γ(ν) and Σ(ν) are the diagonal matrices whose diagonal element at row i can be defined as γ i = cos(νω i ) and σ i = sin(νω i ). The list of ω i is the principal angles between source and target subspaces, i.e., 0 ≤ ω 1 ≤ ... ≤ ω N ≤ π 2 . U 1 and U 2 are the orthonormal matrices obtained by the following pair of SVDs:EQUATIONSince P ⊤ s P t and R ⊤ P t share the same singular vectors V, we adopt the generalized Singular Value Decomposition (SVD) [18, 47] to decompose the matrices. In our approach, we model the cross-view structural changes D x by modeling the cosine similarity between projections along the geodesic flow Π(ν). In particular, given a subspace Π(ν) on the geodesic flow path from the source to the target view, the cross-view geometric correlation of images between the source and target views can formulated by the inner product g Π(ν) (x s , x t ) along the geodesic flow Π(ν) as follows:EQUATIONwhereQ = 1 0 Π(ν)Π(ν) ⊤ dν.Intuitively, the matrix Q represents the manifold structure between the source to the target view. Then, Eqn. ( 10) measures the cross-view structural changes between the source and the target domain based on their manifold structures. The matrix Q can be obtained in a closed form [18, 47] as follows:EQUATIONwhere Λ 1 , Λ 2 , and Λ 3 are the diagonal matrices, whose diagonal elements at row i can be defined as:EQUATIONIn practice, we model the cross-view structural changes D x via the cosine similarity along the geodesic flows. Finally, the cross-view structural changes D x can be formulate as:EQUATIONSimilarly, we can model the cross-view geometric correlation of segmentation D y via Geodesic Flow.View-Condition Prompting Previous efforts [40, 38, 16, 73] in open-vocab segmentation have shown that a better prompting mechanism can provide more meaningful textual and visual knowledge.Prior work in open-vocab segmentation designed the prompt via the class names [64, 13, 38] , e.g., "class 1 , class 1 , ..., class K ". Meanwhile, other methods improve the prompting mechanism by introducing the learnable variables into the prompt [40] or adding the task information [38] . This action helps to improve the context learning of the vision-language model. In our approach, we also exploit the effectiveness of designing prompting to cross-view learning. In particular, describing the view information can further improve the visual context learning, e.g., "class 1 , class 1 , ..., class K captured from the [domain] view", where [domain] could be car (source domain) or drone (target domain). Therefore, we introduce a view-condition prompting mechanism by introducing the view information, i.e., captured from the [domain] view", into the prompt. Our view-condition prompt offers the context specific to visual learning, thus providing better transferability in cross-view segmentation.We hypothesize that the correlation of the input prompts across domains also provides the cross-view geometric correlation in their deep representations. In particular, let f p s and f p t be the deep textual embeddings of view-condition prompts p s and p t , and D p be metric measuring the correlation between f p s and f p t . In addition, since the textual encoder has been pre-trained on large-scale vision-language data [39, 27] , the visual and the textual representations have been well aligned. Then, we argue that the correlation of textual feature representations across views, i.e., D p (f p s , f p t ), also provides the cross-view geometric correlation due to the embedded view information in the deep representation of prompts aligned with visual representations. Therefore, similar to Eqn. (4), we hypothesize the cross-view correlation of segmentation masks and textual features can be modeled as a linear relation with a scale factor γ as:EQUATIONThen, learning the cross-view adaptation with view-condition prompts can be formulated as follows: where λ I and λ P are the balanced-weight of losses. Similar to metrics D x and D y , we also adopt the geodesic flow path to model the cross-view correlation metric D p .θ * =Effectiveness of Batch Size Table 8 illustrates the impact of the batch size on the performance of cross-view domain adaptation. By increasing the batch size, the mIoU performance is also increased accordingly on both benchmarks. This result has illustrated that the small batch size could not have enough samples to approximate the subspace that represents geometric structures. Meanwhile, the subspace created from the large batch size will be ale to capture the geometric structure of drone-view scenes. However, due to the limitation of GPU resources, we could not evaluate the cross-view adaptation model with larger batch size. 
datasets, benchmarks, and implementation: 
 To efficiently evaluate cross-view adaptation, the cross-view benchmarks are set up from the car to the drone view. Following common practices in UDA [23, 58] , we choose SYNTHIA [44] , GTA [43] , and BDD100K [68] as the source domains while UAVID [33] is chosen as the target domain. We chose to adopt these datasets because they share a class of interests and are commonly used in UDA and segmentation benchmarks [23, 61] .SYNTHIA → UAVID Benchmark SYNTHIA and UAVID share five classes of interest, i.e., Road, Building, Car, Tree, and Person. Since the UAVID dataset annotated cars, trucks, and buses as a class of Car, we collapse these classes in SYNTHIA into a single class of Car.GTA → UAVID Benchmark consists of five classes in the SYNTHIA → UAVID benchmark and includes one more class of Terrain. Therefore, the GTA → UAVID benchmark has six classes of interest, i.e., Road, Building, Car, Tree, Terrain, and Person.BDD → UAVID Benchmark is a real-to-real cross-view adaptation setting. Similar to GTA → UAVID benchmark, there are six classes of interest between BDD100K and UAVID. In our experiments, we adopt the mean Intersection over Union (mIoU) metric to measure the performance.Implementation We adopt Mask2Former [8] (ResNet 101) with Semantic Context Interaction of FreeSeg [38] and pre-trained text encoder of CLIP [39] for our open-vocab segmentation networks. Our balanced weights of losses are set to λ I = 1.0 and λ P = 0.5. Further details of our networks and hyper-parameters are provided in the appendix. 15)). For supervised results, we train two different models on UAVID with and without the Terrain class on two benchmarks. As in Table 1 , the cross-view adaptation loss in Eqn. (6) significantly improve the performance of segmentation models. With prompting and cross-view adaptation, the mIoU performance is further boosted, i.e., the mIoU performance achieves 48.6% and 40.1% on two benchmarks. Additionally, by further using the view-condition prompting mechanism with our cross-view loss in Eqn. (15) , the mIoU results are slightly improved by +1.1% and +1.7% on two benchmarks compared to the one without view-condition prompting. Our results have closed the gap with the upper-bound results where the models are trained on UAVID with labels.Effectiveness of Cross-view Correlation Metrics and Network Backbones Table 2 studies the impact of choosing metrics and network backbones. We consider two options, i.e., Euclidean Metric and our Geodesic Flow-based Metric, for correlation metrics D x , D y , and D p . As shown in Table 2 , our Geodesic Flow-based metrics significantly improve the performance of our cross-view adaptation. It has shown that our approach is able to measure the structural changes across views better than using the Euclidean metrics. In addition, by using the more powerful backbone (Swin), the performance of cross-view adaptation is further improved. Effectiveness of Cross-view Learning Parameters Table 3 illustrates the impact of the linear scaling factors α and β. As in Table 3 , the mIoU performance has been majorly affected by the relation between images and segmentation. The best performance is gained at the optimal value of α = 1.5. Since the variation of RGB images is higher than the segmentation, the small value α could not correctly scale the relation between images and segmentation while the higher value of α exaggerates the structural change of segmentation masks. Additionally, the change of γ slightly affects the mIoU performance. Since the textual features are well-aligned with the image, the performance of segmentation models when changing γ also behaves similarly to the changes of α. However, the linear scale factor α is more sensitive to mIoU results since the images play a more important role in the segmentation results due to the pixel-wise corresponding of images and segmentation. 3 reveals the importance of choosing the subspace dimension. The cross-view geometric structural change is better modeled by increasing the dimension of the subspaces. As in Table 3 , the performance is improved when the dimension is increased from 96 to 256. However, beyond that point, the mIoU performance tends to be dropped. We have observed that low dimensionality cannot model the structural changes across views since it captures small variations in structural changes. Conversely, higher dimensionality includes more noise in the cross-view structural changes and increases the computational cost. We also study the impact of batch size in our appendix.Qualitative Results. To further illustrate the effectiveness of our proposed, we visualize the results produced by our model. In the model without prompting, Figure 4 illustrates the results of our cross-view adaptation compared to those without adaptation. As shown in the results, our approach can effectively segment the objects in the drone view. We also compare with the prior ProDA [69] and CROVIA [50] methods. Our qualitative results remain better than the prior adaptation method. For the model with prompting, Figure 5 illustrates the effectiveness of our approach in three cases: without adaptation, with cross-view adaptation, and with view-condition prompting. As shown in the results, our cross-view adaptation can efficiently model the segmentation of the view. By using the view-condition prompting, our model can further improve the segmentation of persons and vehicles. SYNTHIA → UAVID As shown in Table 4 , our EAGLE has achieved SOTA results and outperforms prior view transformation (i.e., Polar Transform [45] ) UDA methods by a large margin. For fair comparisons, we adopt the DeepLab [3] and DAFormer [23] for the segmentation network. In particular, our mIoU results using DeepLab and DAFormer are 45.2% and 50.8%.In the DAFormer backbone, the mIoU results of our approach are higher than CROVIA [50] and MIC [25] by +4.8% and +9.0%.The IoU result of each class also consistently outperformed the prior methods. Highlighted that although our approach does NOT use depth labels, our results still outperform the one using depths, i.e., DADA [59] . It has emphasized that our approach is able to better capture the cross-view structural changes compared to prior methods. Figure 4 illustrates our qualitative results compared to ProDA [69] and CROVIA [50] .GTA → UAVID As shown in Table 4 , our effectiveness outperforms prior polar view transformation [45] and domain adaptation approaches when measured by both mIoU performance and the IoU accuracy of each class. In particular, our mIoU performance using DeepLab and DAFormer network achieves 36.7% and 40.7%, respectively. Our results have substantially closed the performance gap with the supervised results. By using the better segmentation-based network, i.e., Mask2Former with ResNet, the performance of our approach is further improved to 40.1% compared to DeepLab.We compare EAGLE with the prior open-vocab segmentation methods, i.e., DenseCLIP [40] and an adaptive prompting FreeSeg [38] with four settings, i.e., Source Only, with AdvEnt [58] , and with SAC [1] , and our Cross-View Adaptation in Eqn. (6) (without view-condition).Open-vocab Semantic Segmentation As in Table 5 , the mIoU performance of our proposed approach with cross-view adaptation outperforms prior DenseCLIP by a large margin on SYNTHIA → UAVID. By using our cross-view geometric adaptation loss, the performance of DenseCLIP and FreeSeg is further enhanced, i.e., higher than DenseCLIP and FreeSeg with SAC by +3.7% and +5.0%.While FreeSeg [38] with our cross-view adaptation slightly outperforms EAGLE due to its adaptive prompting, our EAGLE approach with the better view-condition prompting achieves higher mIoU performance. Similarly, our proposed cross-view loss consistently improves the performance of DenseCLIP and FreeSeg on GTA → UAVID. The mIoU results of DenseCLIP and FreeSeg using our cross-view loss achieve 37.3% and 44.4%. By further using the view-condition prompting mechanism, our mIoU result is considerably higher than FreeSeg with our cross-view adaptation by +1.3%. Figure 6 visualizes our qualitative results of our proposed approach. (A) Input, (B) FreeSeg [38] , and (C) Our EAGLE.Open-vocab Segmentation on Unseen Classes Table 6 illustrates the experimental results of our cross-view adaptation approach on unseen classes. In this experiment, we consider classes of Tree and Person as the unseen classes. As shown in the results, our cross-view adaptation approach with a view-condition prompting mechanism has achieved the best mIoU performance on unseen classes on both benchmarks, i.e., 39.3% and 39.6% on two benchmarks. Our experimental results have further confirmed the effectiveness and the generalizability of our cross-view geometric modeling and view-condition prompting approach to the open-vocab segmentation across views. Real-to-Real Cross-view Adaptation Setting We evaluated our approach in the real-to-real setting, i.e., BDD → UAVID. Our approach is evaluated in two different settings, i.e., Unsupervised Domain Adaptation and Open-Vocab Semantic Segmentation. As shown in Table 7 , our results have shown a significant improvement in our approach in real-to-real settings in both unsupervised domain adaptation and openvocab semantic segmentation. While the results of prior unsupervised domain adaptation, i.e., BiMaL [51] and DAFormer [23] , gain limited performance due to their limits in cross-view learning, our method outperforms other methods these prior methods by a large margin.Limitations. In our paper, we have specified a set of hyper-parameters and network designs to support our hypothesis and theoretical analysis. However, our proposed approach could potentially consist of several limitations. First, our work focuses on studying the impact of cross-view geometric adaptation loss and view-condition prompting mechanisms on the segmentation models across views.The balanced weights among weights, i.e., λ I and λ P , have not been fully exploited. We leave this investigation as our future experiments. Second, although the datasets and benchmarks used in our experiments have sufficiently illustrated the effectiveness of our proposed cross-view adaptation learning approach, the lack of diverse classes and categories in datasets is also a potential limitation. Third, the hypothesis of the linear relations across views of images and segmentation mask, i.e., α, and textual representations and segmentation masks, i.e., γ, could limit the performance of the relation. The non-trivial relations across views should be deeply exploited in future research. Also, while the implementation of Mask2Former and FreeSeg is adopted to develop our approach, the experiments with other open-vocab segmentation networks should be considered in subsequent research studies. These aforementioned limitations will motivate new studies to further improve the methodology, datasets, and benchmarks of the cross-view adaptation learning paradigm.Broader Impact. Our paper could bring significant potential for various applications that require learning across camera viewpoints. Our approach enables generalizability across camera views, thus enhancing the robustness of the segmentation model across views. In addition, our approach helps to reuse off-the-shelf large-scale data while reducing the effort of manually labeling data of new camera views. 
conclusions: 
 This paper has presented a novel unsupervised cross-view adaptation approach that models the geometric correlation across views. We have introduced the Geodesic Flow-based metric to better model geometric structural changes across camera views. In addition, a new view-condition prompting mechanism has been presented to further improve the cross-view modeling. Through our theoretical analysis and SOTA performance on both unsupervised cross-view adaptation and open-vocab segmentation, our approach has shown its effectiveness in cross-view modeling and improved robustness of segmentation models across views.Limitations Our study has selected a set of learning hyper-parameters to support our hypothesis and experiments. However, this work can potentially contain several limitations related to learning parameters and linear relation hypothesis in Eqn. (4) . The details of the limitations are discussed in the appendix. We believe that these limitations will motivate future studies to improve our unsupervised cross-view adaptation learning approach.While the important and closely related work to our approach has been presented in our main paper, we also would like to review some other research studies that are related to our method as follows. In particular, Brady et al. [72] presented a cross-view transformer that learns the camera-aware positional embeddings. Although the views are captured from left and right angles, the camera positions in the approach remain at the same altitude. Similarly, Pan et al. [36] present a View Parsing Network to accumulate features across first-view observations with multiple angles. Yao et al. [66] proposed a semi-supervised learning approach to learn the segmentation model from multiple views of an image. Huang et al. [26] a cross-style regularization for domain adaptation in panoptic segmentation by imposing the consistency of the segmentation between the target images and stylized target images. Wang et al. [62] proposed a viewpoint adaptation framework for the person re-identification problem by using the generative model to generate training data across various viewpoints. Hou et al. [22] presented a matching cross-domain data approach to domain adaptation in visual classification. Sun et al. [48] proposed a cross-view facial expression adaptation framework to parallel synthesize and recognize cross-view facial expressions. Goyal et al. [19] introduced a cross-view action recognition approach to transferring the feature representations to different views. Zhang et al. [70] proposed a multi-view crowd counting approach that adaptively chooses and aggregates multi-cameras and a noise view regularization. Armando et al. [2] proposed a self-supervised pre-training approach to human understanding learned on pairs of images captured from different viewpoints. Then, the pre-trained models are later used for various downstream human-centric tasks. In summary, these prior cross-view methods could require either a pair of cross-view images [2] or images captured at the same altitude with different angles [26, 72, 22] . In addition, the cross-view geometric correlation modeling has not been exploited in these prior studies [26, 72, 22, 2] • The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. , with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [NA] Justification: The code will be published may the paper be accepted. Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes]Justification: The details of training and testing are presented in the experimental section.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [NA]Justification: Following the standard evaluation of semantic segmentation, we evaluate our model by the standard mIoU metrics instead of the statistical tests.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes]Justification: The computational resources used in our experiments are presented in the experimental section.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes]Justification: The paper does not have a negative societal impact. The broader impact is discussed in the appendix.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: As shown in our Eqn. ( 16), our Geodesic Flow-based metrics have the upper bound as follows:EQUATIONIn addition, as D x is the distance metric, this metric should satisfy the following triangular inequality as follows:EQUATIONSimilarly, D y should satisfy the following triangular inequality as follows:D y (y t , ȳt ) + D y (ȳ t , y s ) ≥ D y (y s , y t ) ⇔ D y (y t , ȳt ) ≥ D y (y s , y t ) -D y (ȳ t , y s ) ⇔ -αD y (y t , ȳt ) ≤ -α (D y (y s , y t ) -D y (ȳ t , y s )) (18)Then, from Eqn. ( 16) and Eqn. ( 17) above, we can further derive as follows:EQUATION)Since α is the constant linear scale value, therefore, we can further derive as follows:EQUATIONTo illustrate the ability of structural learning of our geodesic flow-based metrics, we use a subset of images of the car-view and the drone-view dataset to visualize the base structure of subspaces obtained from the PCA algorithm. Fig. 7 visualizes the mean structures of car-view and drone-view images. As shown in Fig. 7 , The subspaces of car-view images represent the geometric structures of car-view data, i.e., the road in the middle, buildings, trees on two sides, etc. Meanwhile, the geometric structures of the drone view have also been illustrated in the figure with structures and topological distributions of objects (e.g., the road in the middle and trees and buildings on the sides) on the scenes. The results have illustrated the base geometric structures of the car-view and the drone-view data. Then, by modeling the geodesic flow path across two subspaces, our metric is able to measure the cross-view geometric structural changes (i.e., the change of structures and topological layouts of the scene) from the car view to the drone view. Our experimental results in other ablation studies have further confirmed our effectiveness in geometric structural modeling across views. Figure 8 illustrates the feature distributions with and without our proposed approach. As shown in Figure 8 , our approach can help to improve the feature representations of classes, and the cluster of each class is more compact, especially in classes of car, tree, and person. 


Decision: Two of the reviewers recommended weak accept, and the others accept and reject. While 29y1 recommended rejection, there comments during the discussion after reading the rebuttal indicated that their only remaining concern was regarding the lack of qualitative comparisons to more recent methods. 

Given the above, this AC agrees with the reviewers and recommends that the paper is accepted. The authors are strongly encouraged to take the reviewer recommendations and suggestions into account when revising the paper for the camera ready version. Specifically, they should focus on the following:  
* Update the text to fix the cases where the text/descriptions/notation were not clear to the reviewers.   
* The paper is very cramped, use the extra page in the camera ready version to space it out more so that it is more readable.
* Add the comparisons to other view transformation methods provided to upJj.  
* Add the additional qualitative comparisons to newer methods (i.e. not just the older methods displayed in the rebuttal) requested by 29y1 and 8Etw. Fig. 3 from the rebuttal PDF is fine for the appendix, but not necessary to add it to the main paper. 
* Update the related work text so that the difference to Crovia by Truong et al. is more clearly defined.  

Minor comments:
* Fig. 1 caption “could not perform well” -> “do not perform well”
* Fig. 2 It is not clear what method is being shown here. Improve the caption.

Reviews: review: paper_summary: paper_summary: The paper tackles a problem called cross-view semantic segmentation by using unsupervised domain adaptation methods. The cross-view means from the front-view to top-down view, ie, from car to drone. It recognizes the limitations of existing unsupervised domain adaptation and open-vocabulary semantic segmentation methods in handling geometric variations across different camera views. The paper demonstrates the effectiveness of the proposed approach through extensive experiments on cross-view adaptation benchmarks, achieving state-of-the-art performance compared to existing UDA methods.
strength_weakness: strength_weakness: The comparison between the other view transformation methods and the proposed cross-view learning method is not included. 

Most of the UDA methods are tailored for the front-view street scenes, like AdvEnt, ProDA, SAC. More recent and advanced open-vocab semantic segmentation methods should be included in the experiment section. For example, x-decoder, cat-seg, clip-as-RNN, etc.
questions: questions: How is the difference of UDA settings between CROVIA and the proposed in this paper?


In the introduction, the authors claim that “However, the open-vocab perception models remain unable to generalize across camera viewpoints.” It would be better to show the limitation or performance along with this observation. For example, the model with and without using a cross-view setting. 


From eq 3 to 5, for the purpose of geometric adaptation on unpaired data, the problem is that the matrices between source and target domain data are not available. How about using estimated camera matrices for the geometric correlation in eq 3 and 4?


The cross-view learning framework is based on the geodesic flow. How about the comparison with other view transformation methods? For example, in the birds-eye-view semantic segmentation domain, there are different transformation methods, such as: 
[1] Pan, B., Sun, J., Leung, H. Y. T., Andonian, A., & Zhou, B. (2020). Cross-view semantic segmentation for sensing surroundings. IEEE Robotics and Automation Letters, 5(3), 4867-4873.


In Table 4, how about the comparison for BDD→ UAVID? Also, how about applying tye cross-view setting for other DA methods that are proposed for front-view? 


How about using the state-of-the-art open-vocab methods? For example, x-decoder, cat-seg, clip-as-RNN, to name a few. 

[2] Zou, X., Dou, Z. Y., Yang, J., Gan, Z., Li, L., Li, C., ... & Gao, J. (2023). Generalized decoding for pixel, image, and language. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 15116-15127).

[3] Cho, S., Shin, H., Hong, S., Arnab, A., Seo, P. H., & Kim, S. (2024). Cat-seg: Cost aggregation for open-vocabulary semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 4113-4123).

[4] Sun, S., Li, R., Torr, P., Gu, X., & Li, S. (2024). Clip as rnn: Segment countless visual concepts without training endeavor. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 13171-13182).

[5] Wysoczańska, M., Siméoni, O., Ramamonjisoa, M., Bursuc, A., Trzciński, T., & Pérez, P. (2023). Clip-dinoiser: Teaching clip a few dino tricks. arXiv preprint arXiv:2312.12359.


In Table 4, what is the reason that the method with DAFormer outperforms the one with Mask2Former. It is also very interesting why applying a larger or advanced model cannot obtain improvements. 


In Remark 3, the grassmannn manifold is presented for the cross-view modeling method, how about the two subspaces of source and target domain? What is the difference of the feature distributions before and after domain adaptation, such as t-sne visualization? 


It would be suggested to showcase the training process and the computational complexity of the proposed methods individually, such as in the ablation study. Apart from performance, training and runtime are also important for UAVs application.
limitations: limitations: The target domain is limited in only one dataset. There are different UVA datasets public available. But this is not a very large concern, because the experiments are conducted with three different source domains.
paper_summary: paper_summary: This work proposed a novel unsupervised adaptation method for modeling structural change across different views. Additionally, the paper introduced a new metric for cross-view changes and a new prompting mechanism for cross-view open vocabulary segmentation. Through extensive experiments, the paper shows SoTA performance compared to previous unsupervised methods in cross-view semantic scene understanding.
strength_weakness: strength_weakness: 1.The visualization results in this paper are limited. It is necessary to supplement more visualizations of segmentation results compared with the existing methods in the paper, which will help support the experimental conclusions.
2.Generalizing the experiments to include more UAD datasets, such as SynDrone[1], UDD[2], and ICG Drone[3], would enhance the robustness and credibility of the results.
3.The motivation behind the design of the adaptation loss should be explained more clearly.
4.The performance improvement in open-vocab segmentation is not very significant.

[1] SynDrone – Multi-modal UAV Dataset for Urban Scenarios. 
[2] Large-scale structure from motion with semantic constraints of aerial images.
[3] ICG Drone Dataset.
questions: questions: 1.The performance of the building category in open-vocab segmentation on unseen classes is intriguing.
2.The notation "ν ∈ [0..1] → Π(ν)" in line 241 is misleading.
limitations: limitations: The authors discussed the limitation of this paper in the appendix, including the choice of hyperparameters, lack of more diverse class labels and flaws in their original mathematical hypothesis.
paper_summary: paper_summary: The paper introduces a novel method for Unsupervised Domain Adaption to adapt an open-vocabulary segmentation model across different views. To achieve this, the authors introduce a cross-view geometric constraint that captures structural changes between different views. Further, a Geodesic Flow-based Metric is introduced to measure the structural changes across scenes. They also adapt the prompting scheme to take into account the change in viewpoints.
strength_weakness: strength_weakness: - The layout of the paper seems a bit cramped. It would have been nice to have larger visualizations. Also, many of the tables, especially those in the ablations, seem super cramped and the reader has to zoom in significantly. I understand the authors wanted to put as much information as possible into the paper, but this led to a presentation that is not top notch.
- Table 3 has no visual separation between the results for different classes, and overall mIoU. This visual separation (e.g. vertical line) would be nice to have.
questions: questions: Personally, I would appreciate if the authors could further improve the presentation of the paper. While the technical contributions are solid and the writing is nice, the visualizations and tables would benefit from being larger and better integrated with the text. For example, by moving Table 6 to the supplementary, the authors could free up space for the very relevant Table 5. I would appreciate an improved presentation, but also understand that this can be challenging with the given space constraints.
limitations: limitations: The authors have sufficiently addressed the limitations of their approach.
paper_summary: paper_summary: This paper proposes a domain adaptive semantic segmentation method under the cross-view (front view to top view) setting. It addresses this using vision-language models for additional supervision. It proposes a cross-view geometric constraint to model the structural changes and similarities between two vastly different viewpoint. The paper presents detailed quantitative results, with comparisons and ablations
strength_weakness: strength_weakness: My main problem with the paper is the lack of qualitative results. The main paper barely has any qualitative results. The supplementary video shows just one specific video. I am unable to judge the performance of the method without good qualitative analysis.

As in all semantic segmentation papers, it would be good to see where prior work fails, and how each of the components presented in the method help the case. Essentially, it is important to provide qualitative results for the comparisons, the ablations, and as well as the method.

Given that the performance improvement as depicted by the quantitative results is large, the qualitative improvements should be clearly visible too, and hence it is beneficial to add those results.

Also, one main motivation for using a VLM is the open set setting (as described in the abstract). While described as a key point in the paper, none of the experimental benchmarks deal with that.
questions: questions: Since it is not possible to provide qualitative results in the rebuttal, and since it not possible to properly validate the effectiveness of the proposed semantic segmentation method without qualitative results, I am rejecting the paper at this stage.
limitations: limitations: Yes, in page 9

Comments: title: Feedback to Reviewer Response
Rebuttal Follow Up
Rebuttal Follow Up
Response to Reviewer Feedback
Response to rebuttal
Response to Reviewer Feedback
Response to Reviewer's Feedback
Response to rebuttal
comment: Dear Reviewer 29y1,


As shown in the following table (the detailed comparison can be found in Table 4, page 8 in the paper), our EAGLE has outperformed CROVIA and ProDA. We believe the improvement compared to CROVIA is also illustrated in the visualization (as our comparison with ProDA) due to our reproduced results.  **In our revised paper, we will add more qualitative comparisons between our method and CROVIA and release our implementation for research reproducibility (both quantitative and qualitative results).**

| SYNTHIA to UAVID | Road | Building | Car | Tree | Person | mIoU |
|---|---|---|---|---|---|---|
| ProDA | 10.6 | 64.7 | 34.1 | 44.5 | 17.0 | 34.2 |
| CROVIA | 10.6 | 65.7 | 51.7 | 55.6 | 17.0 | 40.0 |
| **EAGLE** | **29.9** | **65.7** | **55.5** | **56.8** | **18.3** | **45.2** |

If you have any other concerns, please do not hesitate to raise your questions. We are happy to address your questions.


Thank you very much,

Authors
Dear authors,

Thank you for your responses.

I believe it'll be great to see qualitative comparisons against CROVIA. 

I feel these comparisons will be significant in demonstrating the superiority of the method given the large quantitative improvements. It is unclear why these comparisons were omitted in the rebuttal pdf, especially considering that ProDA is a 2021 paper. Additionally, the NeurIPS submission provided further quantitative results on CROVIA (2023) beyond those in the original CROVIA paper, suggesting that an implementation was available.

Thanks,
Reviewer
Dear Reviewer upJj,

Thank you so much for your positive rating and insightful feedback!

As the reviewer-author discussion is approaching the deadline, we are reaching out to you to ensure that our rebuttal effectively addresses your concerns. If you have any further questions, please let us know. We appreciate your invaluable input.

Thank you very much,

Authors
Dear Reviewer 29y1,

Thank you very much for your insightful feedback.

The reviewer-author discussion deadline is nearing. We have not received your final response to our final rebuttal. Therefore, we are reaching out to you to ensure that our rebuttal effectively addresses your concerns. If you have any further questions, please let us know. We appreciate your invaluable input.

Thank you very much,

Authors
Dear Reviewer JhPV,

Thank you for your invaluable feedback and positive rating. We're pleased that our rebuttal has addressed your concerns. We are dedicated to updating our paper based on your suggestion to improve our paper's presentation.

Thank you very much,

Authors
Dear authors,
Thanks for addressing my concerns.
I will keep my rating as I remain convinced that this paper is a candidate for acceptance.

Best regards
Dear Reviewer 8Etw,

We would like to thank you for your invaluable feedback and positive rating. We are glad that our rebuttal has addressed your concerns. We are committed to revising our paper according to your recommendations to improve the quality of our paper.

Thank you very much,

Authors
I appreciate the authors for providing clarifications and additional experimental results, which have addressed some of my concerns. I will keep my score.
Dear Reviewer 29y1,

We are glad that our rebuttal has addressed your concerns related to open-set settings. 

For clarification of our visualization results, we prefer the ProDA method for comparison in our rebuttal since ProDA is officially published in CVPR 2021. Meanwhile, CROVIA is only available in a preprint in arXiv. We would like to thank you for your suggestions. We will add more qualitative comparisons between our method and CROVIA in our revised paper.

For clarification of performance improvement compared to CROVIA, our EAGLE approach models the cross-view geometric structural changes via the geodesic flow path. Meanwhile, CROVIA measures the cross-view structural changes by measuring the distribution shift. Thus, our approach can efficiently measure structural changes across views via their manifold structure between two views, more intuitive compared to measuring the distribution shift used in CROVIA. For example, as shown in Figure 5 (page 9) in the CROVIA paper (https://arxiv.org/pdf/2304.07199), we have seen that the segmentation of the class “Person” is not as good as our EAGLE approach. Our quantitative performance in Table 4 also illustrates that our approach performs better than CROVIA.

If you have any other concerns, please do not hesitate to raise your questions. We are happy to address your questions.

Thank you very much,

Authors
Thanks a lot for the detailed rebuttal, I really appreciate your time and efforts in this. 

I went through the other reviews, as well as the detailed responses. 

Thank you for the clarification on the open-set setting. 

Regarding qualitative results, is there any particular reason why comparisons are shown against PRODA (2021) and not CROVIA (2023)? 

I see in Table 4 that the improvement of the method in the paper over CROVIA is 5%, so it would be nice to see where exactly the improvement is coming from. I understand that the code of CROVIA is not open-source, but since the NeurIPS submission has additional results on CROVIA (over what is reported in the CROVIA paper at https://arxiv.org/pdf/2304.07199), I believe you have an implementation, it'll be nice to see those qualitative comparisons.