Controllable Satellite-to-Street-View Synthesis with Precise Pose Alignment and Zero-Shot Environmental Control
Authors: Xianghui Ze
Zhenbo Song
Qiwei Wang
Jianfeng Lu
Yujiao Shi
Venue: ICLR 2025 Poster (2025)
Keywords: Satellite to street-view synthesis
diffusion model
controllable
precise pose Alignment
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Generating street-view images from satellite imagery is a challenging task, particularly in maintaining accurate pose alignment and incorporating diverse environmental conditions. While diffusion models have shown promise in generative tasks, their ability to maintain strict pose alignment throughout the diffusion process is limited. In this paper, we propose a novel Iterative Homography Adjustment (IHA) scheme applied during the denoising process, which effectively addresses pose misalignment and ensures spatial consistency in the generated street-view images. Additionally, currently, available datasets for satellite-to-street-view generation are limited in their diversity of illumination and weather conditions, thereby restricting the generalizability of the generated outputs. To mitigate this, we introduce a text-guided illumination and weather-controlled sampling strategy that enables fine-grained control over the environmental factors. Extensive quantitative and qualitative evaluations demonstrate that our approach significantly improves pose accuracy and enhances the diversity and realism of generated street-view images, setting a new benchmark for satellite-to-street-view generation tasks.

Published as a conference paper at ICLR 2025
CONTROLLABLE SATELLITE-T-SSTREET VIEW SYN- THESIS WITH PRECISE POSE ALIGNMENT AND ZERO- SHOT ENVIRONMENTAL CONTROL
Xianghui Ze!; Zhenbo Song', Qiwei Wang", Jianfeng Lu', Yujiao Shi
Naajjing University of Science and Technology, "ShanghaiTech University {zexh, songzb, L edu. on. en. (wanggw2024,
edu.en
ABSTRACT
Generating street-view images from satellite imagery is a challenging task, par- ticularly in maintaining accurate pose alignment and incorporating diverse envi- ronmental conditions. While diffusion models have shown promise in generative tasks, their ability to maintain strict pose alignment throughout the diffusion pro- cess limited. In this paper, we propose a novel Iterative Homography Adjust- ment (IHA) scheme applied during the denoising process, which effectively ad- dresses pose misalignment and ensures spatial consistency in the generated street- view images. Additionally, currently, available datasets for generation are limited in their diversity of illumination and weather conditions, thereby restricting the generalizability of the generated outputs. To mitigate this, we introduce a rext-guided illumination and weather-controller sampling strategy that enables fine-ge one-grained control over the environmental factors. Extensive quan- titative and qualitative evaluations demonstrate that our approach significantly im- proves pose accuracy and enhances the diversity and realism of generated street- view images, setting new benchmark for satellitetto-stree-viee generation tasks.
INTRODUCTION
This paper tackles the problem of satellite-to wew-ttoo-sttt--ieeeeee synthesis, aiming to generate street-view images that are geometrically consistent with satellite imagery under pre-determined relative pose and diverse environmental conditions. Synthesizing such images has critical applications in urban modeling, geospatial analysis, and virtual reality. While satellite images provide comprehensive global coverage at low cost, capturing ground-level data resource-intensive and lacks access to varied weather and seasonal conditions. Therefore, generating geometrically accurate street-view images with controllable environmental diversity is a challenging but essential task.
The difficulties in this task stem from two key aspects: geometric consistency and environmental di- versity. Geometric consistency is challenging due to the vast perspective difference between satellite and ground views where satellite images provide top-down perspective, street-view images show the scene from a lateral angle, resulting in minimal visual overlap. Establishing a reliable mapping between these viewpoints requires precise handling of geometric information. Additionally, environmental diversity is crucial for practical applications, as street-view images under different weather and lighting conditions are needed to simulate real-world scenarios. Existing datasets for satellite-tee satellite-to-stredd voestreet-view tasks offer limited variation in environmental conditions, further complicat- ing the generation of diverse and realistic outputs.
Recent works have made progress in this domain by using geometric priors to bridge the gap between satellite and ground perspectives. For example, [Shi al. (2022a) used multi-plane images to infer depth maps from satellite data and generate geound-level images, while Qian et al. (2023) rendered panoramas from satellite images based on density fields. However, these approaches are prone to emror due to their reliance on approximate 3D priors. Meanwhile, methods focusing generating images with environmental diversity (Assion et al. (2024)) are often limited to in-domain editing and lack the ability to generate diverse scenes from satellite inputs.
"Tii work was done during a visit at ShanghaiTech University. Corresponding author.

---

Published as a conference paper at ICLR 2025
Diffusion
IHA
ZoEC
uutumn
Sunriser
Sat map
Geometric distortion
Geometric correction
Diverse conditions
Ground truth
Figure I: Our method synthesizes ground-level images from satellite maps by integrating Iterative Homogra- phy Adjustment (IHA) to refine geometric alignment and Text-guided Zero-shot Environmental Control (ZoEC) provide flexible environmental control, enabling precise pose alignment and diverse scene generation.
As shown in Fig. T this paper proposes novel framework for satellite-to-street view synthesis that addresses two key challenges: ensuring geometric alignment and enabling environmental control. Our approach is based on the classical stable diffusion model (Rombach et al. (2022b)), which has shown strong performance in image synthesis but faces significant challenges in maintaining precise pose information during generation, often leading to misalignment between generated street-view images and satellite inputs.
To overcome these issues, we propose two key innovations. First, unlike standard diffusion models where conditions are implicitly encoded within the denoising U-Net, introduce cross-view conditioning mechanism that incorporates geometric information. This mechanism ensures that the generated street-view panoramas maintain consistent spatial layout, aligning scene objects with the satellite images. Second, we propose an Iterative Homography Adjustment scheme that operates during the diffusion sampling phase. This process corrects pose misalignment by iteratively adjusting the intermediate output based on the relative pose difference between the generated image and the satellite view, ensuring geometric consistency throughout the generation process.
Furthermore, real-world environments exhibit a wide range of illumination and weather conditions, which are rarely reflected existing seweliie-an--tteet---eee-e- the model's ability to generate realistic strect-view images under different environmental settings. To overcome this limitation, we introduce a Zero-Shot Environmental Control strategy, which uses text prompts to guide the generation of street-vis view images under varying illumination and weather conditions without retraining the model.
datasets. This lack of diversity limits
Finally, since satellite images lack crucial details like the sky and the sides of buildings that present in ground-level views, traditional metrics such as RMSE are inadequate for fair comparisons across different methods. To provide metrics that assess both semantic and geometric consistency between generated and ground truth images, well as the alignment between the generated images, pose, and environmental conditions.
comprehensive evaluation,
introduce
evaluation
Our contributions are summarized as follows:
We introduce a novel cross-view conditioning mechanism that incorporates geometric in- formation, ensuring precise spatial alignment between the generated street-view panoramas and satellite images.
We propose an Iterative Homography Adjustment scheme during the diffusion sampling process, addressing pose misalignment by iteratively correcting the generated images for geometric consistency.
We demonstrate the effectiveness of our framework in generating diverse street-view im- ages with controllable environmental conditions (e.g., weather and lighting) in a zero-shot manner.
We design new evaluation metries that measure both semantic and geometric consistency between the generated and ground truth images, as well as the alignment between the gen- erated outputs and the controllable pose and environmental conditions.
2

---

Published as a conference paper at ICLR 2025
2 RELATED WORK
Cross-view ground scene generation. In the study by Zhai al. (2017) the authors first at- tempted to align the semantic features satellite images onto ground-level perspectives. In Wu al. (2022), GANs were employed to generate ground images. Strong geometric relationships were introduced in the task ground image generation by Lu et al. (2020); Shi et al. (2022a); Qian et al. (2023). Li et al. (2021 2024b) explicitly constructed 31D point cloud representation of the scene, and then transformed it into a scene representation in feed-forward manner. [Gao et al. (2024a); Xuu & Qin (024))
et al. (2024a) advocate for generating ground images from ground-to to-ground scene segmentation images. Among these, Gao et al. (2024a) specifically highlights the impact of various noises on the generated results and innovatively proposes a noise-prior-basee
solution. However, previous methods have predominantly relied on coarse scene priors, leading to compounded errors the results. We propose GCA and IHA ensure geometric consistency between ground images and satellite views. The cross-view generation work targeting single objects is also highly inspir- ing. [iu et al. (2023) overlays camera position encoding for scene transformation, while Poole et al.) (202)) utilizes diffusion to optimize the Nerf representation of scenes. Melas-Kyy Kyriazi et al. (2024) Gao Gao et al. (2024b) generate continuous frame data based on video diffusion. These approaches reconstruction, especially when dealing with significant perspective differences between satellite and ground images, which is the issue we are dedicated to addressing.
often fail in large-scale
Text-controlled image generation. In text generation, multitude of solutions have emerged over time leveraging Generative Adversarial Networks (GANs) (Dash al. (2017) Regmi Borji 2018): Ruan et al.] 202T): Tao et al. (2022). However, with the introduction of diffusion (Song et al. 22200 Rombach et al. (2022b)), its exceptional generation capability has evolved into a potent tool for creating images. Significant strides have been taken in text-driven image synthesis through dif- fusion by AAvrahami et al. (2022); Li et al, (2023b); [Brooks al. (2023). Li al, (2023a); Gao et al.) (224c) propose method that generates ground images based on text conditions and BEV segmen- tation images. However, this strategy is hampered by the limitations of expressive capabilities in scene segmentation, leading arbitrary results in scene synthesis. In this paper, employ satellite images with enhanced representational capabilities for ground synthesis and introduce a novel text- guided mechanism to ensure both the reliability of scene generation and the diversity of generated results.
METHOD OVERVIEW
Our task is to generate realistic street-view images based on a given satellite image, a specified relative pose, and an environmental factor such as illumination or weather. We use latent diffusion model framework to conditionally synthesize these street-view images while ensuring control over both geometric and environmental aspects. The geometric condition is encoded via the satellite image and relative pose, while environmental control is applied during inference to achieve diverse visual effects.
3.1 DIFFUSION MODEL FRAMEWORK
To generate street-view images with controlled conditions, we leverage a latent diffusion model (Rombach al. (2022a)). Let the target street-view image be denoted To and its cor- responding latent embedding as 2p. Using an image decoder D(-), the target image can be recon- structed as xo D(zo). The forward diffusion process adds Gaussian noise the latent embedding Zo progressively, resulting in 2 at each time step E [0, 7",, where the noise level is defined by 3e (0.
Varzo + VI ā;E
(I)
where (00 I), Cr. = - B. and 0z I d; The denoising U-Net, 60(21, I, c). is used to predict the added noise, with condition e representing the posed embedding of the satellite image.
The reverse diffusion process is based on DDIM sampling:
p(2-11 (N== ())
(2)
3

---

Published as a conference paper at ICLR 2025
X (
Pose Correction
Env Control
Carmatton
Central
D
Zr-1
"ummerr
G Geometric
Cross-A Attesion
Sat Map
R.T
Xo
H
"Summer"
Text Encoder
Ve
2,,0
Warp
Homography
A Adjustment
Puse
Warp
Diff
R
D
each patch
Image Encoder
Pose Correction
ZzD
Env Control
Overview of our framework. Our approach aims utilize satellite images generate corresponding ground images. We leverage geometric relationships extensively and have the capability to alter the features of the generated images based on different text prompts.
Figure 2:
where o, controls the sampling's stochasticity, and ue (21 t, c) is calculated as:
2t
V1 ā. colzt, t,) + / 1 ā.- 1--ā
po(ze,t,)) V
(3)
Var
3.2 ENVIRONMENTAL GUIDANCE AND POSE ALIGNMENT
Due to the lack labeled environmental conditions (e.g., variations weather and lighting) in the existing cross view image datasets, using classifier-free guidance for the control, like satellite image features and the relative pose, during the training process impossible. To address this issue, we propose zero-shot environmental guidance approach using text prompts by leveraging classifier guidance strategy. Specifically, we employ off-the-shelf CLIP text embeddings as environmental control, enabling the guidance of environmental factors like lighting and weather during inference. Furthermore, implicitly leveraging satellite image features as a condition often leads to pose mis- alignment generated street-view images, as demonstrated Fig To further enhance the pose alignment, we introduce an Iterative Homography Adjustment within the DDIM sampling process. Specifically, we denote environmental and pose guidance as Ilex and Jpos. respectively. shown in Appendix A.4] Using classifier-gus -usiiiiirr- (Dhariwal & Nichol 202 )) the denoising process with environmental and pose guidance defined as:
- = (z 1, c) + AV=, logp(gpowe |z,) + VV log loge(ttellzz) orE
(4)
where hyperparameters 7 and À control the strength of environmental and pose conditioning, respec- tively, ensuring that the generated image meets both environmental and geometric requirements.
In the following section, we discuss how conditions c of the posed satellite image are encoded, along with further technical details enforcing pose alignment through Vz, log(p(0pmm (=)) and applying environmental guidance via Vz, (og(p()e))22z) during the inference process.
THE PROPOSED FRAMEWORK
This paper aims to generate ground-level images that accurately align with given satellite im- age, relative pose, and environmental conditions. To achieve precise pose control, as illustrated Fig. 2 we propose embedding satellite features and the relative pose information into the diffu- sion model through two complementary mechanisms: a Geometric Cross-Attention module within the denoising network and an iterative pose enhancement strategy during the inference denoising stage. Considering diverse weather and illumination data are absent in the training set, we design Zero-Shot Environmental Control strategy that allows flexible control over scene variations during inference without requiring additional training data.
4

---

Published as a conference paper at ICLR 2025
each pose
feature refinement
cosine similarity
V V
InfoNCE
G2S
feature refinement
Z₁,0
R, T
Figure 3:
Homography Adjustment. We compare ground-level images with satellite images, calculating the loss by assessing the relative pose of the ground images at time against a predefined pose. Ground images are then adjusted based this comparison.
4.1 CROSS-VIEW CONDITIONING MECHANISM
Instead of implicitly encoding global feature vectors of posed satellite images, we propose Geo- metric Cross-Attention (GCA) mechanism that explicitly constructs geometric correspondences.
Given a pixel coordinate (ug. vg)) in the target ground image, its corresponding 3D scene point can be determined if the depth d is known. Since acquiring accurate depth map for the ground view impractical, we instead hypothesize a set of N reference height values {h, J.V. (relative to the ground plane) for each pixel, capturing potential depth variations across different regions. For each reference height, GCA estimates an offset Ah; based on the current feature state and calculates an attention weight A, for each height hypothesis to evaluate its reliability. The GCA mechanism then aggregates features from the satellite image as follows:
N GCA(Q.V) Σ Ai (V & P(ug: Ug he Ahu))
(5)
where Q and V represent the ground and satellite features, respectively. Here, A, and Ad, are computed from the ground features Q, and the A, is the result of applying softmax to the reference heights. P(ug, V/ h; + Ah;) maps the ground pixel (ug. 12y) at height + Ah, to the corresponding satellite pixel coordinates based on relative pose. The symbol @ denotes a sampling operation that extracts features from V according the projected satellite coordinates.
Unlike previous approaches that rely strictly on predefined 3D priors (Li al. (2024a)), our method allows for iterative refinement of satelliteto-groound correspondences. This iterative adjustment enables the model to gradually correct initial misalignments and improve spatial consistency over time. By focusing on relevant regions through projected coordinates, GCA also reduces computa- tional load compared to traditional global attention mechanisms, effectively enhancing geometric alignment between satellite images and generating ground-level views.
4.2 ITERATIVE POSE ALIGNMENT DURING INFERENCE
The inherent randomness of the diffusion process and implicit reasoning can often result in generated images that deviate from the specified pose, causing positional shifts, as shown in Experiment [53 To address this issue, we propose an iterative position correction mechanism that adjusts each gen- erated result to better match the specified pose throughout inference. Due to the complex scene depth, pose discrepancies can lead to substantial misalignments, making pixel-wise color or flow corrections difficult to learn. Instead, we introduce an Iterative Homography Adjustment mecha- nism that applies a Homography transformation matrix to correct the latent representation 2; at each inference step, i.e., 2 & gHid(H), where grid(H) represents coordinates generated by H. The matrix enables transformations such as scaling, translation, and rotation, corresponding to the spatial alignment of the camera. Since achieving precise correction with single H is challenging, we perform iterative pose adjustments in parallel with DDIM denoising steps, gradually guiding the output toward the target pose.
The process for adjusting H is illustrated in the bottom left of Fig. 2 We first compute an initial latent 2n, from 2 using the denoising Unet's output timestep 1. We initialize H as an identity
5

---

Published as a conference paper at ICLR 2025
Algorithm Iterative Homography Adjustment for Pose Refinement.
Input:
diffusion steps t, the noisy image 2,, satellite image conditions I, and the rotation R and translation T relative to the satellite image. for all /. from T to 1 do
Zt +- 2, & grid(H) H) I H a diagonal matrix of ones
2t,0, 2111 DDIM t, R. T.I.) // climinate noise to obtain 21,0, 2t-1
- V Pose 2t R, T,),)) adjust the Homography matrix
2t-1 - @ grid(H) H)
end for
matrix, applying it to 20.t. which then project an overhead view using a ground plane sumption, as in Shi et al. (2025). To ensure robust pose alignment, as illustrated in Fig. 3 we sample multiple candidate ground camera poses in addition to the specified pose, cropping and rotating the corresponding regions from the satellite image feature maps, which are extracted from the condi- tioning branch of the denoising Unet. We introduce Feature Refinement module in each branch to map the ground and satellite features to a shared representation space. These modules share the same architecture but have separate weights. Next, we compute an alignment score, (Sk Ik=1 using cosine similarity between the ground and satellite feature maps at K candidate poses. To quantify alignment, we use the InfoNCE loss (Oord et al. 2018)):
Sk-
Cpose - log En eS//T'
where T is temperature hyperparameter. The InfoNCE loss reaches its minimum when the ground and satellite feature maps achieve the highest similarity at the ground truth (GT) relative pose k". Using this loss, we compute the gradient with respect H and update it to improve alignment.
Our Homography Adjustment module applies transformations directly to the latent vector 20,e rather than mapping it back to image space, avoiding the need for heavy ground-to-satelle network which operates on the original image resolution. Instead, we employ lightweight Feature Refinement module on f and the satellite features from the denoising Unet's conditioning branch, significantly reducing the computational cost.
localization
Training the Homography Adjustment network follows the initial training of the denoising Unet within the diffusion framework. After obtaining 2,,0 from the Unet, then train the Homography Adjustment network using the InfoNCE loss defined in Eq. 6 During the inference stage, both the denoising Unet and the Homography Adjustment network are trained. We then use it to refine H in the stochastic denoising sampling process in DDIM to improve pose alignment iteratively.
4.3 TEXT-GUIDED ZERO-SHOT ENVIRONMENTAL CONTROL
Many autonomous driving datasets such CVUSA (hai al. (207)) lack detailed textual anno- tations. As shown in Experiment 5.4] we discovered that when using descriptions generated by large multimodal model along with detailed satellite images as condition, the generative model tends to prioritize accurately representing the satellite image conditions, significantly diminishing the in- fluence of textual prompts during the training process. Therefore, we separated the text prompt. Inspired by Dhariwal & Nichol (202T). we utilized text prompts to guide the direction of reasoning during the inference stage. We replace e, with ε
Ez(,,, = <, (, + V Cient Creat)
The gradient term Vz, Lteat (Zt, Ctext) guides the diffusion process towards the desired direction, thereby ensuring that the generated results consistent with the textual conditions.
To robustly obtain the loss between the generated results and the text prompt, we randomly partition 2 into multiple patches and calculate the similarity with the text for each patch. We then use CLIP (Radford et al. (202)) to extract features from the processed patches and the text, and calculate the cosine similarity between these features obtain the final loss:
Ltezt(zs, (text) = 1
sim (CLPP(()))))) CCII(UEE)
(8)
N

---

Published as a conference paper at ICLR 2025
(d) ControlNet Figure 4: Qualitative comparison with previous work on the CVUSA dataset, our framework able to main- tain hetter geometric relationships.
(a) Sat
(b) S2S
(c) Sat2Den
(e) Ours
(1) GT
Table 1:
Quantitative comparison with existing algorithms on CVUSA dataset. The best results are highlighted
in orange and the second-best in blue.
Structural similarity| 1SSIM
Perceptual similarity
Semantic Similarity JDINO
Pixel Similarity JRMSE 1PSNR
Method
!Depth
[Self sim
Pequeeze Patez JFID
JSegAny
1SD
Pix2Pix
0.32
0.21
0.35
0.51
44.51
5.24
0.38
55.84
13,20
12.08
21.85
XFork
0.29
S2S
0.35
0.19
0.32
0.48
29.49
5.11
0.39
54.57
13.40
12.30
21.05
Sa12Dnnsity
0.33
0.19
0.31
0.46
47.85
4.95
0.38
54.23
13.46
12.27
19.83
ControlNet
0.32
0.20
0,33
0.49
22.55
5.21
0.38
58,25
12.83
12.08
21.78
CrossDiff 0.37
23.67
12.00
Ours 0.38
0.18
0.30
0.45
21.30
4.81
0.36
52.92
13.67
12.33
19.58
where 2,,0 is the result of removing noise from 22. z1.0 is the L-th patch of 21,0. Ctext is the textual condition, and sim represents cosine similarity. Similar to the Iterative Pose Alignment illustrated in Sec. 4.2, this environmenta control strategy is also applied during the inference stage, iteratively modifying the noisy latent during the denoising process.
4.4 EVALUATION METRICS
There is a limited overlap between satellite images and ground truth images, further complicated by varying weather conditions and seasons, which makes it challenging to synthesize target view images that perfectly match the ground truth image provided in the dataset. As result, pixel- level metrics like RMSE and PSNR are inadequate for this task (Zhang et al. (2018); [Shi et al.) (222aa)) Instead of focusing on color discrepancies between the generated street view images and the actual ground truth images, our emphasis should be on whether they represent the same locations. Therefore, use structural similarity, perceptual similarity, semantic similarity, and depth similarity for performance evaluation. Furthermore, since satellite images do not capture sky information, advocate for cropping the sky portion during evaluation.
The evaluation of Structural Similarity is derived from SSIM and self-similarity structures based on DINO features (Caron et al. (2021) Shechtman & Irani (2007); Tumanyan et al. (2022)) Per- ceptual Similarity is evaluated based on the FID metric and compares the similarity of features ex- tracted from AlexNet (Krizhevsky et al. (2017) and SqueezeNet (landola (2016)). Semantic Sim- ilarity is proposed for evaluating high-level semantic features. We employ widely acknowledged DINO (Caron al. (202T)) and Segment Anything (Kirillov et al. (2023) for feature extraction to compare the semantic consistency of images. Additionally, given that satellite images contain depth information of ground scenes, evaluating the depth of generated ground images is crucial. We use DepthAnything (Yang et al. (2024)) to assess the depth differences between real ground truth images and generated images. For a fair comparison, we also report commonly used metrics such as RMSE, PSNR, and SD to evaluate Pixel Similarity. These metrics are more forgiving for assessing blurred images, as clear images tend to exhibit increased pixel differences due to the introduction of details such seasons, weather, and shadows. This discrepancy conflicts with the intended outcome of our generated images, so we strongly recommend utilizing alternative metrics over pixel similarity eval- uation metrics. Finally, for text similarity, we compute the similarity of Clip Radford et al. (2021)

---

Published as a conference paper at ICLR 2025
(a) Sat
(b) S2S
(c) Sat2Den
(d) ControlNet
(e) Ours
(I) GT
Figure 5: Qualitative comparison with previous work on VIGOR dataset, our model can effectively capture road surface information from satellite images, resulting clearer and more distinct road lines.
Table Quantitative comparison with existing algorithms on VIGOR dataset. The best results are highlighted in orange and the second-best in blue.
Structural similarity ISSIM
Perceptual similarity
Semantic Similarity JDINO
Pixel Similarity [RMSE
Method
Depth
JSelf sim
Pequuese
Patez
LFID
JSegAny
PPSNR
SSD
Pix2Pix
0.37
0.17
0.33
0.45
67.96
4.72
0.38
55.00
13.33
12.93
8.65
S2S
0.33
0.18
0.37
0.49
121.10
5.03
0.40
62.94
12.16
12.31
10.87
Sat2Density
0.40
0.16
0.28
0.39
54.49
4.41
0.36
50,23
14.14
12.90
8.05
ControlNet
0.34
0.17
0.34
0.46
23.68
4.95
0.39
63.98
12.02
12.59
10.01
Ours
0.42
0.16
0.27
0.38
28.01
4.34
0.35
52.16
13.80
13.07
7.10
features for evaluation and utilize Blip Li al. (0022) rate of answers that align with the textual descriptions.
describe the images, assessing the recall
5 EXPERIMENTS
5.1 IMPLEMENTATION DETAILS
Experimental setup. We take 256 x 256 satellite images as input to predict 128 x 512 ground images, following the same setup as in Shi al. (2022a) for fair comparison. Our model is finetuned based on the Stable Diffusion 1.5 model (Rombach et al. (2022a)), with the Cross-Attention of diffusion replaced by Geometric Cross-Attention, and satellite image conditions processed through a simple VIT network for feature extraction. During inference, we employ DDIM sampling with 50 sampling steps, applying the Homography Adjustment scheme in the first 40 sampling steps and Zero-Shot Environmental Control throughout the entire sampling process. In Geometric Cross- Attention, we utilize 8 sampling heights of [-3, -2, -1, 2, 3, 4, 5]. This constitutes an empirical setup. Training process on three GPUs with batch size of 32 for 200 epochs.
Datasets. We adopt three cross-view datasets: CVUSA (Zhai et al. (2017). KITTI (Geiger et al. (Si & Li (2022)) ), and VIGOR (Zhu et al. (2021); Lentsch et al. (2022)). These datasets com- prise pairs of cross-view data, combining gevund-level images with their corresponding satellites. The ground-level images in CVUSA and VIGOR panoramic, while the ground-level images in KITTI have a limited horizontal field of view (HFoV). CVUSA comprises 35,532 pairs of satellite and street view images for training and 8,884 pairs for testing. Following the setup of the cross-view localization task (Shi & Li 2002);; Xia et al. (2023). KITTI includes 19,655 pairs in the training data and 3,773 pairs in the testing data. VIGOR gathers data from New York, Seattle, San Francisco, and Chicago, dividing the data from each city into 52,609 pairs for the training set and 52,605 pairs for the test set.
5.2 COMPARISON WITH EXISTING METHODS
In this section, we compare Pix2Pix (Isola et al. (007)) and XFork (Regmi & Borji (018)) are GAN-based methods that ex- tract implicit features from satellite images to generate ground images. S2S (Shi et al. (2022a)) and Sat2Den (Qian et al. (2023) introduce explicit geometric information into the network and project satellite images ground view based on height priors. CrossDiff (Li et al. (2024a)) is diffusion
approach with previous ground map generation methods.
8

---

Published as a conference paper at ICLR 2025
(a) Map
(b) LDM
(c) GT
(d) LDM WIIHA
Figure 6: Qualitative ablation experiments of IHA on KITTI and CVUSA datasets. IHA can correct pose offsets in the inference process effectively.
Table Ablation study on the KITTI dataset. We compared the positional accuracy of generated images using a cross-view localization model.
Distance Javerage Jmedian Javerage Jmedian
Angle
Lateral
Longitudinal fd=l 'd=3
Azimuth 10=1 10=3
fd=1
1d=3
LDM
10.74
6.92
17.60
6.64
43.97
67.74
15.35 36.79
7.87 24.04
LDM W. THA
10.67
6.85
17.46
6.73
44.08
67.77
15.77
36.95
8.08 23.78
LDM W.GCA W.IHA
10.51
6.66
17.14
6.64
45.40 68.67 16.78 38.22
8.31 24.22
method that relies high prior knowledge. Furthermore, noticing the robust capabilities of Control- Net (Zhang et al. (2023)) ), we also compared it with our method. The implementation of ControlNet mirrors the successful case (Sastry al. (2024), ). ControlNet receives textual conditions and ground map contour conditions. The textual conditions are generated using LLAVA (Liu et al. (2024)) and randomly masked out with probability of 0.5 during training. The ground map contour conditions are derived from satellite maps projected based on prior ground height assumptions.
Quantitative results, as shown in Table and Table demonstrate that the quality of ground image generation produced by our method sigmificantly surpasses that of other approaches. particularly maintaining geometric consistency. This performance stems from the Cross-Vice Conditioning Mechanism we employ and the Iterative Pose Alignment conducted during inference. In the visual representation, as depicted Fig. 4 our method adheres to the geometric cues from satellite im- ages, generating ground images that align with the scenes and outperform previous algorithms in representing pathways. While ControlNet outperforms us in the FID metric on the VIGOR dataset, note that its other quantitative metries notably lower, indicating that ControlNet accurately captures dataset distributions but struggles to faithfully translate satellite image hints into improved ground image generation, point supported by Fig. S On the VIGOR dataset, method excels in ground feature identification and architectural representation compared to other algorithms.
5.3 THE EFFECTIVENESS OF POSE ALIGNMENT
Compared to other datasets, KITTI's data is collected along the same route and divided into training and test sets. Due the high similarity satellite image conditions between adjacent points along the path, the implicit condition diffusion model fails to distinguish effectively, leading to the gener- ation of ground images with positional offsets quite easily. To demonstrate the effectiveness of the proposed pose alignment approaches in this paper, employ the original implicit diffusion model as our baseline and gradually add the proposed modules, i.c., the IHA and the GCA. To better eval- uate the results of Pose Alignment, pose estimation method, to evaluate the consistency of the relative pose between generated images and the satellite images with respect the conditioning pose, in addition image-level similarities.
employ CCVPE (Xia et al. (2023)
powerful cross-view

---

Published as a conference paper at ICLR 2025
ControlNat
(a) No prompt
(b) Spring
(c) Summer
(d) Autum
(e) Winter
Figure 7: Qualitative comparison with ControlNet on CVUSA. ControlNet trained under weak textual condi- tions fails effectively adjust image information based text prompts. Our Zero-shot Environmental Control can adjust image information based on text prompts while preserving the spatial structure of the image.
Table The similarity between generated images and environmental text prompt.
Clip Score
Blip Score
spring
summer
autumn
winter
spring
summer
autumn
winter
LDM ControlNet
0.21
0.21
0.20
0.18
5.14
60.87
7.44
1.78
0.21
0.21
0.20
0.20
12.89
44.63
5.50
13.06
Ours
0.26
0.24
0.24
0.23
69.74
82.05
78.62
47.67
The quantitative comparison is shown in Table 3 It can be seen that both IHA and GCA improve the generated image quality in terms image-level similarity and pose consistency.
Fig. 6 provides examples of Pose Alignment KITTI and CVUSA datasets. Both the images before and after correction are generated with the same noise, the only difference being whether Pose Alignment is enabled. We can observe the significant role played by Pose Aligmment in successfully correcting pose deformities in the generated results. Furthermore, the method is equally applicable to both FoV images and panoramic images, demonstrating high level of universality.
5.4 THE EFFECTIVENESS OF TEXT-GUIDED ENVIRONMENTAL CONTROL
To evaluate the effectiveness of the Text-guided Environmental Control mechanism, we conduct comparative experiments with LDM and ControlNet on the CVUSA dataset, generating images rep- resenting the four seasons: spring, summer, autumn, and winter. Notably, our environmental control mechanism is applied during inference in a zero-shot manner, while for LDM and ControlNet, the environmental control incorporated network. To train ControlNet's 's text condition branch, we used LLAVA to generate text annotations for the ground images.
additional text condition embedded within the denoising
As shown in Table 4 the quantitative results indicate that our method produces outputs that align more effectively with the provided textual prompts. One challenge encountered was the inconsis- tency the quality of LLAVAA-geeratted text descriptions, as they were produced without manual annotations. During ControlNet's training, the model tended to prioritize well-expressed satellite image prompts, diminishing the impact of textual prompts over time. This behavior led to Control- Net's reduced sensitivity diverse text prompts, as illustrated in Fig. Environmental Control (ZoEC) mechanism exhibited greater robustness in handling unsupervised scenarios, maintaining stronger adherence to the given text conditions.
contrast, our Zero-shot
CONCLUSION
In this paper, we have presented novel approach for generating ground-level images from satellite imagery, addressing the dual challenges of geometric alignment and environmental diversity. Our method introduced two key innovations: the Iterative Homography Adjustment (IHA) mechanism, which ensures accurate pose alignment between the satellite and generated ground-level views. and the Text-guided Zero-shot Environmental Control (ZoEC), which allows flexible control over light- ing, weather, and seasonal variations without requiring additional training data. By incorporating geometric tross-attention in the diffusion process, further improved the consistency between satellite and street-level views. Extensive experiments demonstrated that our method outperforms existing approaches in both geometric accuracy and environmental adaptability.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
The authors are grateful for the valuable comments and suggestions by the reviewers and AC. This work was supported by NSFC (62406194, 62302220), Shanghai Frontiers Science Center of Human- Centered Artificial Intelligence (ShangHAD, MoE Key Laboratory of Intelligent Perception and Human-Mace Machine Collaboration (KLIP- HuMaoo, A part of the experiments of this work were sup- ported by the core facility Platform of Computer Science and Communication, SIST, ShanghaiTech University.
REFERENCES
Felix Assion, Florens Gressner, Nitin Augustine, Jona Klemenc, Ahmed Hammam, Alexandre Krat- tinger, Holger Trittenbach, and Sascha Riemer. A-bdd: Leveraging data augmentations for safe autonomous driving in adverse weather and lighting. arXiv preprint arXiv:2408.0660 2024.
Omri Avrahami, Dani Lischinski, and Ohad Fried. Blended diffusion for text-driven editing of natural images. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 18208-182) 18218, 2022.
Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18402, 2023.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 9650-9660, 2021.
Ayushman Dash, John Cristian Borges Gamboa, Sheraz Ahmed, Marcus Liwicki, and Muham- mad Zeshan Afzal. Tac-gan-text conditioned auxiliary classifier generative adversarial network. arXiv preprint
2017.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8794, 2021.
Huan-ang Gao, Mingju Gao, Jiaju Li, Wenyi Li, Rong Zhi, Hao Tang, and Hao Zhao. Scp- diff: Photo-realistic semantic image synthesis with spatial-categoricn joint prior. arXiv preprint 203V:22003.09638, 2024a.
Ruiqi Gao, Aleksander Holynski, Philipp Henzler, Arthur Brussee, Ricardo Martin- Martin-Brualla, Pratul Srinivasan, Jonathan Barron, and Ben Poole. Cat3d: Create anything in 3d with multi-view diffusion models. arXiv preprint arXiv:2405.. 10314, 2024b.
Ruiyuan Gao, Kai Chen, Enze Xie, Lanqing Hong, Zhenguo Li, Dit-Yan Yeung, and Qiang Xu. MagicDrive: Street view generation with diverse 3d geometry control. In International Confer- ence on Learning Representations, 2024c.
Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The kitti dataset. The International Journal of Robotics Research, 32(11):123-11237,00 2013.
Forrest N landola. Squeezenet: Alexnet-level accuracy with 50x fewer parameters and; 0.5 mb model size. arXiv preprint arXiv: 2016.
Phillip Isola, Jun Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. CVPR, 2017.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceed- ings of the IEEE/CVF International Conference on Computer Vision, pp.
2023.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convo- lutional neural networks. Communications of the ACM, 60(6):84-0,, 2017.
Ted de Vries Lentsch, Zimin Xia, Holger Caesar, and Julian FP Kooij. Slicematch: Geometry-guide aggregation for cross-view pose estimation. arXiv preprint 2XXi::221.11465: 2022.
11

---

Published as a conference paper at ICLR 2025
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language- image pre- training for unified lasounla understanding and generation. In International conference on PMLR, 2022.
machine learning, pp.
Weijia Li, Jun He, Junyan Ye, Huaping Zhong. Zhimeng Zheng, Zilong Huang. Dahua Lin, and Conghui He. Crossviewdiff:: A cross-view diffusion model for satellite-t--street view synthesis. arXiv preprint arXiv:2408. 14765, 2024a.
Xiaofan Li, Yifu Zhang, and Xiaoqing Ye. Drivingdiffusion: Layout-guided multi-view driving scene video generation with latent diffusion model. arXiv preprint arXiv:2310.07771, 2023a.
Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jianwei Yang. Jianfeng Gao, Chunyuan Li, and Yong Jae Lee. Gligen: Open-set grounded texl-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22511 125221, 2023b.
Zuoyue Li, Zhenqiang Li, Zhaopeng Cui, Rongjun Qin, Marc Pollefeys, and Martin R Oswald. Sat2vid: Street-view panoramic video synthesis from single satellite image. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 12436-12445, 2021.
Zuoyue Li, Zhenqiang Li, Zhaopeng Cui, Mare Pollefeys, and Martin R Oswald. Sat2scene: 3d urban scene generation from satellite images with diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7141-7150, 2024b.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024.
Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl Vondrick. Zero-I-to-3; Zero-shot one image to 3d object. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 9298 9309, 2023.
Xiaohu Lu, Zuoyue Li. Zhaopeng Cui, Martin R Oswald, Mare Pollefeys, and Rongjun Qin. Geometry-aware muellite-to-groume image synthesis for urban areas. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 859-867, 2020.
Luke Melas-Kyriazi, Iro Laina, Christian Rupprecht, Natalia Neverova, Andrea Vedaldi, Oran Gafni, and Filippos Kokkinos. 1m-3d: Iterative multiview diffusion and reconstruction for high-quality 3d generation. arXiv preprint arXiv:2402. 2024.
Aaron den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding, arXiv preprint 18Xiv:1807.0377 2018.
Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffuuion. arXiv preprint urXiv:2209.14988 2022.
Ming Qian, Jincheng Xiong, Gui-Song Xia, and Nan Xue. Sat2density: Faithful density learning from satellite satellite-gruund image pairs. In Proceedings of the IEEE/CVF international Conference on Computer Vision pp. 3683-3692, 2023.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pp- 8748-8763. PMLR, 2021.
Krishna Regmi and Ali Borji. Cross-view image synthesis using conditional gans. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 3501-3510, 2018.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR), pp. 10684-10695, June 2022a.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022h.
12

---

Published as a conference paper at ICLR 2025
Shulan Ruan, Yong Zhang. Kun Zhang, Yanbo Fan, Fan Tang, Qi Liu, and Enhong Chen. Dae- gan: Dynamic aspect-aware gan for text-to-image synthesis. In Proceedings of the IEEE/CVF international conference computer vision, pp.
2021.
Srikumar Sastry, Subash Khanal, Aayush Dhakal, and Nathan Jacobs. Geosynth: Contextually- aware high-resolution satellite image synthesis. In Proceedings of the IEEE/CVE Conference Computer Vision and Pattern Recognition, pp. 460-470, 2024.
Eli Shechtman and Michal Irani. Matching local self-similarities across images and videos. In 2007 IEEE conference on computer vision and patlern recognition, pp. 1-8. IEEE, 2007.
Yujiao Shi and Hongdong Li. Beyond cross-view image retrieval: Highly accurate vehicle localiza- tion using satellite image. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 17010-17020, 2022.
Yujiao Shi, Liu Liu, Xin Yu, and Hongdong Li. Spatial-aware feature aggregation for image based view geo-localization. Advances Neural Information Processing Systems, 32, 2019.
Yujiao Shi, Xin Yu, Dylan Campbell, and Hongdong Li. Where am looking at? joint location and orientation estimation by cross-view matching. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp.
2020a.
Yujiao Shi, Xin Yu, Liu Liu, Tong Zhang, and Hongdong Li. Optimal feature transport for cross- view image geo-localization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 11990-11999 2020b.
Yujiao Shi, Dylan Campbell, Xin Yu, and Hongdong Li. Geometry-guided street-view panorama synthesis from satellite imagery. IEEE Transactions on Pattern Analysis and Machine Intelli- gence, 44(12):10009 1002200022200 2022a.
Yujiao Shi, Xin Yu, Liu Liu, Dylan Campbell, Piotr Koniusz, and Hongdong Li. Accurate 3-dof camera geo-localization via ground-to-satellite image matching. IEEE transactions pattern analysis and machine intelligence, 45(3):2682-28) 2697, 2022b.
Yujiao Shi, Xin Yu, Shan Wang, and Hongdong Li. Cvlnet: Cross-view semantic correspondence learning for video-based camera localization. In Asian Conference on Computer Vision, pp. 123- 141. Springer, 2022c.
Yujiao Shi, Fei Wu, Akhil Perincherry, Ankit Vora, and Hongdong Li. Boosting 3-dof ground-to- satellite camera localization accuracy via geometry-guided cross-view transformer. In Proceed- ings of the IEEE/CVF International Conference on Computer Vision, pp. 21516-21526, 2023.
Yujiao Shi, Hongdong Li, Akhil Perincherry, and Ankit Vora. Weakly-supervised camera localiza- tion by ground-to-satelliee image registration. In European Conference Computer Vision, pp. 39-57. Springer, 2025.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. basre-based generative modeling through stochastic differential equations. arXiv preprint 1.Xiv:20011.13450 2020.
Zhenbo Song, Jianfeng Lu, Yujiao Shi, al. Learning dense flow field for highly-accurate cross- view camera localization. Advances in Neural Information Processing Systems, 36, 2024.
Ming Tao, Hao Tang, Fei Wu, Xiao- Yuan Jing, Bing-Kun Bao, and Changsheng Xu. Df-gan: A sim- plc and effective bascline for text-to-image synthesis. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 16515-16525, 2022.
Narek Tumanyan, Omer Bar- Tal, Shai Bagon, and Tali Dekel. Splicing vit features for semantic appearance transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10748-10757, 2022.
Songsong Wu, Hao Tang. Xiao Yuan Jing, Haifeng Zhao, Jianjun Qian, Nicu Sebe, and Yan Yan. Cross-view panorama image synthesis. IEEE Transactions Multimedia, 25:3546-3559, 2022.
13

---

Published as a conference paper at ICLR 2025
Zimin Xia, Olaf Booij, Marco Manfredi, and Julian FP Kooij. Visual cross-view metric localization with dense uncertainty estimates. In European Conference on Computer Vision, pp. 90-106. Springer, 2022.
Zimin Xia, Olaf Booij, and Julian FP Kooij. Convolutional cross-view pose estimation. IEEE Transactions on Pultern Analysis and Machine Intelligence, 2023.
Zimin Xia, Yujiao Shi, Hongdong Li, and Julian FP Kooij. Adapting fine-grained cross-view lo- calization areas without fine ground truth. In European Conference on Computer Vision, pp- 397-415. Springer, 2025.
Ningli Xu and Rongjun Qin. Geospecific view gencraton-geoneety-cooeeetttt aware high-resolution ground view inference from satellite views. arXiv preprint arxiv:2407.00066,, 2024.
An Yang. Junshu Pan, Junyang Lin, Rui Men, Yichang Zhang, Jingren Zhou, and Chang Zhou. Chi- nese clip: Contrastive vision-language pretraining chinese. arXiv preprint vrXiv:2211.01335. 2022.
Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pp. 10371-10381, 2024.
Menghua Zhai, Zachary Bessinger, Scott Workman, and Nathan Jacobs. Predicting ground-level scene layout from aerial imagery. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 867-875, 2017.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control text-to-image diffusion models. In Proceedings the IEEE/CVF International Conference on Computer Vision, pp. 3836-3847, 2023.
Richard Zhang, Phillip Isola, Alexei Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features perceptual metric. In Proceedings the IEEE conference computer vision and pattern recognition, pp. 586-595, 2018.
Sijie Zhu, Taojiannan Yang, and Chen Chen. Vigor: Cross-view image geo-localization beyond one- to-one retrieval. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 3640-3649, 2021.
14

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.1 THE ROBUSTNESS OF IHA IN THE FACE OF NOISE.
1FF
IHA
Sample
Sample
Sample
GT
Figure 8: In the first row, we demonstrate that ground images generated from satellite maps exhibit varying offsets under different initial noise conditions. The second row illustrates the pose correction results of the first row using the Iterative Homography Adjustment (IHA).
As shown in Fig. 8 when using different initial noises, the reverse diffusion process often pro- duces unpredictable outcomes. In the first row, diverse noises result in different view perspectives. However, the IHA module adeptly corrects these deviations and gets consistent results. This case shows the remarkable robustness of IHA for identifying and rectifying the accurate pose of generated ground images.
A.2 DATA AUGMENTATION FOR CROSS-VIEW LOCALIZATION
Table 5: Using generated street data for data augmentation. We report the localization accuracy of Shi ct al.
(2025) on the KITTI dataset.
TestI
Distance
Angle
Lateral
Longitudinal
Azimuth 10=1 10=3
Javerage Imedian
Javerage
Imedian
td=1
td=3
[d=1
1d=3
Wo.augmentation
11.11
7.646
0.1811
0.1492
|56.98
87.23
10.68
31.57
99.66 100.0
Pix2Pix
11.72
9.020
0.1810
0.1491
54.31
86.03
9.833
27.54
99.66 100.0
W Waaugmentation
ControlNet
11.15
7,688
0.1810
0.1491
54.17
87.36
11.61
31.27
99.66 100.0
Ours
10.88
7.167
0.1810
0.1491
57.57 87.70
11.95 32.65
99.66 100.0
Test2
Distance Javerage Imedian
Angle
Lateral
Longitudinal
Azimuth
Javerage
Imedian
td=1
1d=3
[d=1
1d=3
10=1 10=3
Wo.augmentation
14.07
10.61
0.1570
0.1305
660.32
84.90) 8 8.632
25.14
100.0 100.0
Pix2Pix
14.44
11.29
0.1569
0.1305
57.44
80.63
8.101 23.89
100.0 100.0
Wauugmenttionn
ControlNet
14.31
10.91
0.1569
0.1305
55.99
82.39
8.260
25.05
100.0 100.0
Ours
13.94
9.885
0.1569
0.1305
62.87 85.51 8.923 26.92
100.0 100.0
The cross-view positioning algorithm has achieved high level of accuracy after a long period of development (Shi et al. (2019) 2020a b) 2022b c 2023); [Song et al. (2024): Xia et al. (2022) 2025 ).
To verify that the proposed generative model can assist in vehicle localization in autonomous driv- ing scenarios, we utilize generated street data for data augmentation in training localization model.
The experiments are conducted on the KITTI dataset (Geiger al. (2013): Shi Li (2022) ). Geiger
et al.] (2013) provides ground images captured by vehicles, while Shi & Li (2022) collects corre- sponding satellite images for each ground image. The dataset divided into Training, Test1, and Test2 subsets. The images in Testl come from the same area as the images in the training set, while the images in Test2 come from different areas. In our data augmentation experiment, we use ground map resolution of 128x512 and satellite map resolution of 512x512. For each satellite image in the training set of the KITTI dataset, we utilize the proposed framework to generate street scene images under various season conditions. During the training of the cross-view localization algorithm Shi et al. (2025), , both generated and real data are employed with a 50% probability each to augment the KITTI dataset. In Table present the localization accuracy with and without the
15

---

Published as a conference paper at ICLR 2025
data augmentation. Both of them trained for 15 epochs. In the Wauugmentation experiments, compare the data generated by Pix2Pix, ControlNet, and our own method. The data augmentation with generated data significantly improves cross-view localization performance, demonstrating the usefulness of our approach to autonomous driving tasks. Compared to our generation strategy method, the data generated by GAN-based Pix2Píx is very blurry. Adding the generated data from Pix2Pix actually deteriorates the model's perception of details. The data generated by ControlNet, although clear, lacks strong positional constraints. The pose offset leads incorrect matches in the localization model during training. Compared to other methods, our approach can ensure geomet- rically consistent generation and text-guided diverse environment creation, thereby achieving the purpose of data augmentation and yielding outstanding results.
A.3 MORE ANALYSIS OF THE IHA.
A.3.1 THE HOMOGRAPHY TRANSFORMATION IN IHA.
Figure 9: The relationship between Homography Adjustment and the camera position satellite imagery. The corresponding satellite position and ground images are labeled with arrows and borders using the same colors.
The Homography transformation is defined by 3x3 matrix and can map a plane an image to another plane. It allows for image manipulations including rotation, translation, scaling, shearing, and perspective mansformation, As depicted in Fig. 9 adjusting the ground map through Homogra- phy enables correspondence with different perspectives in satellite image. Translating the ground image corresponds to perpendicular movements to the satellite image, scaling represents horizon- tal (front-back) movements within the satellite image, and the projection transformation signifies changes in yaw angle.
H
H=
1.00e
1663--5
1.95e-6
100ee
139e-
1677-1
1100e"
761e
4.23e
1966
1.00
5.45e
-
100.a
9,50g
3.12
1.00e
-2.17
- 6.688
1.00e
-3,63e
-6. -6.10e
1.00e"
121e
-1.75e*
100e
IHA Figure 10: The Homography Adjustment operation in the DDIM process. The visual results of each step stem from (2x.0)
2-
25
10
Due to variations in initial noise and uncertainties in the implicit learning of features, diffusion does always yield favorable outcomes, depicted in Fig. 6 many ground images may become dis- torted, indicating displacement in the corresponding satellite image coordinates. Motivated by this, we employ satellite image-guided guided supervision in the DDIM generation process. Within the DDIM process, we utilize satellite image conditions, the result 2 at step f,, and the expected location R, of the ground image in the satellite image. We supervise the positioning algorithm illustrated in Fig. 3 to adjust the location of the generated 2,-1 based on the Homography matrix for the ground image. Given the substantial difficulty in achieving correct results through a single Homography Ad- justment, adopt iterative approach, progressively executing Algorithm I to guide the ground image pixels toward the correct coordinates. Fig. 10 we illustrate the intermediate results and step-by-step changes in the denoising process using IHA.
16

---

Published as a conference paper at ICLR 2025
A.3.2 THE COMPUTATIONAL WORKLOAD IN IHA
Table 6: The analysis of computational efficiency of IHA.
Memory
Time Cost
IHA
20126MB
5,406s
W.IHA
21022MB
5.513s
The IHA operates in the latent space using low-resolution feature maps. Since the IHA calculation relies on the pixel coordinates, does not impose significant computational increase. We conduct the efficiency tests with batch size of I, and the results are presented in Table When comparing memory usage, the baseline model without IHA (Wo.IHA) requires 20, 126MB, while the inclusion of IHA (WIHA) increases memory usage slightly to 21,022MB (896MB increase) due to the ad- dition of lightweight localization network. terms of time cost, the baseline model (Wo.IHA) requires 5.406 seconds per image, while the model with IHA (W.IHA) increases this slightly to 5.513 seconds per image-an additional cost of only 0.107 seconds per image. We believe that IHA strikes favorable balance between performance and computational efficiency, offering promising practical value.
A.4 DERIVATION OF FORMULAE.
Based on the score-based formulation of a diffusion mode[Song al. (2020), and the introduction of conditional guidance Spose 9tezt, our objective is to learn
é = Cee V 21 log p Stewt)
Using the Bayes' formula, obtain:
plze Sposes Itext) Plosee 9tezt (bezz)Z))p(zz)
(10)
P ppsses Itent)
In the formula, p(9pose: Itent) can be considered a constant, and we denote C. Utilizing the independence of environmental and positional conditions, express the formula as:
p(2-119posee 9text) Cp(9pose: 9text (2)))(())
(11)
Cpppppppoose /2 ((ttett (2 (2)())))
We substitute Eq into Eq 9 to obtain:
6 04 V log p(z)( Spose: Stezt)
(12)
=- (V logp(zz) + 2., log p(gpasell)) V., log p(gtent)))))
From the score-based formula E vI ai v, logp(z)), we can derive:
& = c v1 04 V 2, log p(Opose 2)) v1 a v., log ((9texe/2))
(13)
Substituting the new & into Eq. in place of co t, c). we obtain:
2 = no (2,, + V loge(g (2) + AV 2. plg (2,) orE
(14)
The hyperparameters and À control the influence of pose and environmental conditioning, where higher values strengthen the alignment with desired conditions.
A.5 QUALITATIVE ANALYSIS OF RURAL SCENES PERTAINING TO CVUSA.
Fig. [] presents additional visual results using the CVUSA dataset, mostly collected in rural cas. Notably, S2S and Sat2Den struggle to capture the geometric features of rural scenes, produc- ing blurry and indistinct images. ControlNet also shows noticeable geometric misalignments. In contrast, our method, leveraging the proposed Geometric Cross-Attention (GCA) mechanism and Iterative Homography Adjustment (IHA), demonstrates superior recovery of road structures.
For the CVUSA dataset, another characteristic is that buildings often occupy only a small region in the target souund-veww images. The latent embedding of conditioning satellite images can hardly
17

---

Published as a conference paper at ICLR 2025
(a) Sat
S2S
Sa2Den
(c) CorbrolNeet
(e) Ouns
(f) GT
Figure 11: Qualitative visualization for generated images using different methods in complex scenes.
encode the buildings due to few occupant pixels. Owing to the geometric reasoning capabilities of GCA and IHA, our method excels at mining deep semantic features at the correct coordinates. As result, it accurately reconstructs buildings in the appropriate regions of the ground-view images and synthesizes realistic facade appearances, as illustrated in the third example.
In summary, method not only retains strong structural consistency but also exhibits promising ability to generate small elements in satellite images.
A.6 MORE ANALYSIS OF THE GCA.
A.6.1 VISUAL ANALYSIS OF THE GCA MECHANISM.
Sat map
Sat feature (V)
Ours
GT
Sampled feature maps according to different height candidates
A ttention map for each candidate height map Height candidate
Height candidate
Height candidate 3
Height candidate 4
Sampled feature maps according to beight candidate
Generating ground feature Before step2
Step1
Step2
After step2
Figure 12: Visualization results of the GCA mechanism. The feature maps visualized using PCA, while the attention maps displayed with jet color mapping. In the middle row, we present ground feature maps sampled from satellite features according to four candidate height map candidates alongside their corresponding attention maps. On the bottom left, we illustrate ground feature maps sampled from satellite features according to the same height plane but at different GCA steps, with feature map differences reflecting variations in height information. The bottom right shows the aggregated features obtained from the learned height candidates across different GCA steps, demonstrating the progressive refinement achieved by the GCA mechanism.
18

---

Published as a conference paper at ICLR 2025
In this section, present visualizations of intermediate features of satellite images under varying height assumptions and the corresponding attention maps A,. The progressive ground features using the GCA mechanism are also revealed.
During the GCA process, the inputs include the current ground map features (Q), satellite map features (V), and the relative poses R and T. We eight height assumptions (h {-3, 1-, 2,, 3, 4, in experiments. Here, h represents the relative height of the satellite im- age to the camera. When the scene lies below the camera, the lower part of the camera captures the scene while the camera's ss upper part works for scenes above the camera. This behavior is illustrated in the second row of Fig. [2
As defined in Eq. 5 the GCA mechanism infers offsets Ah, and attention weights A, for each height candidate based on the current ground map features (Q). The attention weights A are normalized via a softmax function across the eight height planes, determining the confidence associated with each plane.
In the middle of Fig. 12 the top row shows the ground feature maps sampled from satellite features at different candidate heights (h, + Ah;), while the bottom row visualizes the corresponding attention maps {A).y In the first GCA iteration (step 1), the projection plane samples features based on the initial height assumption, and the attention evenly distributed across various regions, providing a baseline solution. By the second GCA iteration (step 2), the inferred offisets Ah, introduce positional shifts in the projection plane, as highlighted in the red box. These shifts enable the projection to capture more details of ground images, such pathways. The GCA iteration also gradually refines the representations of other scene elements.
As the GCA iterations progress, the attention maps evolve to focus salient regions. For planes where < O, the attention primarily targets the ground, while for planes where h > 0, shifts towards elevated elements like trees. The attention shifting illustrates that GCA effectively enhances feature representations of the entire scene in a progressive manner.
A.6.2 COMPARISON WITH NAIVE GROSS-ATTENTE
The proposed GCA module offers the following advantages over naive cross-attention mechanism.
1. Flexibility in view-related image generation. By leveraging the relative pose between satellite and ground images, the proposed GCA enables the generation of ground images at arbitrary locations and from arbitrary views on the same satellite map. In contrast, the simple cross-attention mechanism cannot handle view changes and requires additional modules process the relative pose information (e.g., Zero-1-to-3 Liu et al. 2023)). Our method is more flexible and can handle various relative pose differences.
2. Avoid redundancy information. The proposed GCA limits attention to regions likely to corre- spond geometrically. Each ground-view pixel only attends to satellite image areas along its camera ray. This focused attention minimizes noise from irrelevant regions, unlike naive cross-attention, which indiscriminately considers the entire image.
3. Improved computational efficiency and reduced GPU memory usage. GCA achieves significant reductions in computational complexity by employing sparse sampling. For satellite image features of size S S and ground image features of size H * W, the complexity of naive cross-attention is O(S * S * H * W). In contrast, our algorithm samples N planes (N=8), with the complexity of sampling being O(N * H W), computing horizontal and vertical coordinate offsets at O(2N H W), and attention calculation at ONN H * W). Overall, our complexity is significantly reduced to 0 N H W) compared to naive vrossaattention. In practical experiments, for inference one example, native cross-attention consumes 250MB of GPU memory, whereas using GCA reduces this to only 184MB.
A.7 DISCUSSING THE REASONS FOR THE FAILURE OF THE CFG SCHEME IN CONTROLLING THE ENVIRONMENT.
In Tab 4 employ the LDM and ControlNet models using Classifier-Free Guidance (CFG) ap- proach for multi-condition generation. Both LDM and ControlNet take two conditions as inputs: satellite images and image descriptions generated by LLAVA. Through training. observed that
19

---

Published as a conference paper at ICLR 2025
The image captures a scenic view of two-lane mad trees on both sides. Th road is surmunneed by forest, giving it serene and peaceful atmosphere. The road appears to be empty, with no cars or people visible in the scene. The sky can be soen in the background, adding to the picturesque setting.
LLAVA
Prompt: "D escribe the contents of the image"
One to many
One to one
Figure 13: Using LLAVA-generated text for ground images correspond multiple ground pictures, while satellite images correspond one-to-one with ground images.
the weight assigned to the satellite image condition far exceeded that of the environmental descrip- tion condition. Furthermore, depicted in Fig- [3] image description generated by LLAVA could correspond to multiple ground images, and text captions significantly lack geometric informa- satellite image. which only encapsulates
tion. Each ground image corresponds specifically geometric descriptions but also texture representations. Consequently, the LDM and ControlNet tend prioritize the satellite images over the ground image descriptions, leading the degraded control of euvironmental conditions.
A.8 THE PERFORMANCE OF TEXT-GUIDED ZERO-SHOT ENVIRONMENTAL CONTROL IN A MULTILINGUAL SETTING.
"####""
"#E
"#"*""
Controllable generation in the Chinese context. This figure displays generated images of spring. autumn, and winter from top bottom.
Figure 14:
In order to ensure compatibility across linguistic contexts, our framework provides a flexible inter- face that can integrate with various multimodal large language models. In the paper implementation, we utilize a frozen CLIP model to extract textual and visual features and compute losses. This ap- proach eliminates additional training and enables straightforward adaptation other multimodal
20

---

Published as a conference paper at ICLR 2025
LLMs. For instance, support Chinese text prompts, Chinese- CLIP Yang al. (2022) is incorpo- rated. By computing losses between Chinese text and image features, the generating process can be effectively guided to align with the input language.
A.9 DISCUSSION AND LIMITATIONS
The primary issue revolves around the accurate restoration of building details, which is common limitation in the task of ground image generation from satellite data. Due to the significant discrep- ancy between satellite and ground views, satellite images lack the texture details of building facades. Therefore, is difficult to faithfully reconstruct the building details from the ground view. The other limitation is the lane marking generation. There inconsistencies between the generated lanes and the real-world lanes. These differences arise largely due to misalignment of the capture time between the satellite maps and ground images. Despite these limitations, our method demonstrates significant improvements in terms of geometric alignment and environmental control in generating diverse ground-view view images.
A.10
THE RESULTS OF TEXT-GUIDED ZERO-SHOT ENVIRONMENTAL CONTROL.
(a) No text prompt Figure 15: Generating under different textual conditions to obtain text-guided image results while maintaining the structure of the images
(b) Spring
(c) Summer
(d) Autumn
(e) Winter
A.11 MORE RESULTS FROM POSE ALIGNMENT.
(a) Sat
(b) LDM Figure 16: Results of IHA on the KITTI dataset.
GT
(d) LDM W. IHA
21

---

Published as a conference paper at ICLR 2025
(a) Sat
(b) LDM
(c) GT
(d) LDM W. IHA
Figure 17: Results of IHA CVUSA dataset.
22


Reviews:
[{"review_id": "d7AOgN2WIs", "reviewer": "Reviewer_7YpL", "summary": "This paper focuses on generating street-view images from satellite imagery while preserving geometric consistency and capturing diverse environmental conditions.\n\nTo achieve precise geometric control, the authors propose a novel cross-view attention mechanism paired with an Iterative Homography Adjustment scheme. To address environmental variability, they introduce a Zero-Shot Environmental Control strategy.\n\nFurthermore, the paper introduces new evaluation metrics to assess both semantic and geometric consistency between the generated images and corresponding ground truth images.\n\nExtensive experiments validate the effectiveness of the proposed approach in producing diverse, realistic street-view images.", "questions": "see weakness box.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper clearly defines the problem by identifying two key challenges in Satellite-to-Street-View Synthesis: geometric consistency and environmental diversity. It effectively addresses these challenges by proposing corresponding solutions.\n\nThe innovative Iterative Homography Adjustment scheme stands out as a significant contribution. By iteratively refining the intermediate output based on relative pose differences, this approach corrects pose misalignments. I believe the \"iterative adjustment\" mechanism not only plays a crucial role in the Satellite-to-Street-View Synthesis task but also holds potential for broader applications in other controllable image synthesis tasks.\n\nThe introduction of the Zero-Shot Environmental Control (ZoEC) mechanism is another noteworthy highlight. Compared to existing methods like ControlNet, ZoEC demonstrates superior performance in synthesizing environmental variations through text-based control, enhancing the flexibility and realism of the generated outputs.\n\nThe paper is well-written and well-structured, making it easy to follow and understand. The clarity of presentation further strengthens the overall impact of the work.", "weaknesses": "Please provide an analysis of how the introduction of the Iterative Homography Adjustment (IHA) affects inference speed. Understanding the trade-off between accuracy and computational efficiency would enhance the practical value of the method.\n\nThe paper lacks an ablation study on the proposed Geometric Cross-Attention (GCA) mechanism. Including this analysis would help isolate and highlight the specific contribution of GCA to the overall performance.\n\nWhile the rationale for introducing a new evaluation metric is understandable, for a fair comparison, it is important to also report results using the evaluation pipeline and metrics from previous works, such as RMSE, PSNR, and SD. This would provide a more comprehensive assessment of the method's performance relative to existing approaches.\n\nThe scholarship could be improved by referencing other relevant works in conditional street-view synthesis, such as SCP-Diff: Photo-Realistic Semantic Image Synthesis with Spatial-Categorical Joint Prior [ECCV 2024]. This would situate the proposed method more effectively within the broader research landscape.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1cAAzqKa4g", "reviewer": "Reviewer_jsaR", "summary": "This paper presents a new method for satellite-to-street view synthesis. This paper mainly achieves this goal by integrating Iterative Homography Adjustment to refine geometric alignment and Text-guided Zero-shot Environmental Control to provide flexible environmental control.", "questions": "1. Why choose the affine transformation to describe relative pose shift?\n2. Is Clip Score enough it measure text-based generation ability?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The proposed method is motivated and effective.", "weaknesses": "1. Many important works related to generation are missing in this paper, such as DreamFusion[1], Zero-1-to-3[2],  IM-3D[3] and CAT3D[4]. I strongly suggest that the author reorganize Related Works and make a detailed review and introduction to the development of the generation field.\n2. The experimental results do not show notable advantages over previous baselines as reported in Table 1 and Table 2.\n\n\n[1] DreamFusion: Text-to-3D using 2D diffusion,  Poole et. al.\n\n[2] Zero-1-to-3: Zero-shot One Image to 3D Object, Liu et. al.\n\n[3] IM-3D: Iterative Multiview Diffusion and Reconstruction for High-Quality 3D Generation, Melas-Kyriazi et. al.\n\n[4] CAT3D: Create Anything in 3D with Multi-View Diffusion Models, Gao et. al.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KUx8JrFTV9", "reviewer": "Reviewer_vmd3", "summary": "This paper proposes an innovative approach for synthesizing street-view images from satellite imagery, addressing the challenges of geometric alignment and environmental diversity. The authors introduce a Geometric Cross-Attention mechanism (GCA) and Iterative Homography Adjustment (IHA) to ensure geometric consistency between the generated images and satellite views. Additionally, Zero-shot Environmental Control (ZoEC) is employed to flexibly adjust the environmental conditions of the generated images without the need for additional training data. Experimental results show that this method outperforms existing techniques in both geometric accuracy and environmental diversity, demonstrating potential for applications in urban modeling and virtual reality.", "questions": "1. The authors should discuss how the Zero-shot Environmental Control (ZoEC) performs under different languages or diverse text descriptions. Can the model maintain environmental adjustment accuracy in multilingual scenarios?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The Iterative Homography Adjustment mechanism iteratively adjusts the pose of the image during the generation process, significantly improving the geometric consistency between the generated image and the satellite image and ensuring the correct spatial alignment of objects in the scene.\n2. The Zero-shot Environmental Control mechanism applies text prompts during inference to control environmental conditions without requiring additional training data. At the same time, ZoEC maintains the spatial structure of the image, ensuring that changes to environmental conditions do not affect the original geometric and semantic consistency of the generated image.\n3. Extensive experiments have demonstrated the effectiveness of the proposed methods in the paper.", "weaknesses": "1. The English expression is inconsistent, and the organization of the paper needs improvement. Some academic terms are used inconsistently, such as \"pose correction\" and \"pose alignment,\" which should be uniformly applied throughout the paper.\n2. There are some errors in the manuscript's presentation, for example: Figures 1–3 are not properly referenced or explained in the main text; Tables 9–10 lack corresponding table header explanations; and the parameter τ in Equation 9 is not adequately explained, making it difficult to read.\n3. As far as I know, the CVUSA dataset contains numerous images taken in rural scenes, with many street-view images that include roads or buildings. The author’s displayed images seem to lack discussion on reconstruction results related to these elements.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TKKOVsYyuK", "reviewer": "Reviewer_KCsS", "summary": "Overall, this paper propose a method for satellite-to-street-view synthesis. The authors propose “CROSS-VIEW CONDITIONING MECHANISM” for better geometry consistency, and propose “ITERATIVE POSE ALIGNMENT DURING INFERENCE” at inference stage to further improve the results by utilizing the gradient values between the projected generated image and satellite image. Besides, “TEXT-GUIDED ZERO-SHOT ENVIRONMENTAL CONTROL” utilized the gradient values of CLIP alignment loss to enhance the text controlling.", "questions": "1. In Equation (4) and (5), could you please provide a comprehensive derivative process? Due to space limitations, I suggest that the detailed process can be placed in the supplementary materials. Besides, it is recommended for the authors to provide more details for the meaning of the derived results. What benefits does this derivation bring to subsequent model design? How do model design and formula derivation relate to each other?\n\n2. In Method, the proposed “CROSS-VIEW CONDITIONING MECHANISM” injects the controlling information into the diffusion model. Actually, it is like to introduce a query to extract the features from the satellite image. However, the authors’ statement makes me a little confused. Firstly, the authors state that \\mathbf{A} represents the confidence. \\delta{h} represents the height offset, but there is no evidence shown here. If they are indeed what the author says they are, at least some visualization should be displayed or some reasons should be explained. Besides, the detail of sampling operation should be given. Secondly, this module is named Geometric Cross-Attention (GCA), but there is no ‘attention’ mechanism here, equation (7) even does not contain a softmax symbol. Thirdly, what is the benefit of this mechanism brings as compared to the naive cross-attention? \n\n3. In “ITERATIVE POSE ALIGNMENT DURING INFERENCE”, equation (8) and Line-277 also have the sampling operation defined in Line-251, please elaborate it. Why the symbol in equation (8) is different in Line-273? \n\n4. In “TEXT-GUIDED ZERO-SHOT ENVIRONMENTAL CONTROL”, why choosing “\\epsilon” to controlling the generative precess with respect\tto text prompt? It is weird for “potent satellite image prompts significantly undermine the influence of text prompts during training”. Have you applied the CFG scheme?", "limitations": "", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "This paper is well-organized and the proposed approach can improve the geometric consistency and textual controlling ability of generative method.", "weaknesses": "1. This article lacks a strong motivation and fails to clearly convey the significance of achieving geometric consistency and more accurate street views from satellite images. It doesn't specify under what circumstances a model with enhanced geometric consistency would be preferred. I believe this paper is quite similar to a series of works such as BEVGen[1] and BEVControl[2], which tell the story of geometric consistency with respect to the data augmentation. The author may benefit from expanding this point.\n[1]. BEVGen: Street-View Image Generation from a Bird's-Eye View Layout\n[2]. BEVControl: Accurately Controlling Street-view Elements with Multi-perspective Consistency via BEV Sketch Layout\n\n2. Some typos: There is a blank line before almost every line of Equation. They should be deleted. In Line-459, it should be Table 1.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)