Title: WeatherPrompt: Multi-modality Representation Learning for All-Weather Drone Visual Geo-Localization
Venue: NeurIPS 2025
Accepted: yes
Keywords: Training-free

Geolocalization

Text Guidance

Drone Navigation
Abstract: Visual geo-localization for drones faces critical degradation under weather perturbations, \eg, rain and fog, where existing methods struggle with two inherent limitations: 1) Heavy reliance on limited weather categories that constrain generalization, and 2) Suboptimal disentanglement of entangled scene-weather features through pseudo weather categories.
We present WeatherPrompt, a multi-modality learning paradigm that establishes weather-invariant representations through fusing the image embedding with the text context. 
Our framework introduces two key contributions: First, a Training-free Weather Reasoning mechanism that employs off-the-shelf large multi-modality models to synthesize multi-weather textual descriptions through human-like reasoning. It improves the scalability to unseen or complex weather, and could reflect different weather strength. 
Second, to better disentangle the scene and weather features, we propose a multi-modality framework with the dynamic gating mechanism driven by the text embedding to adaptively reweight and fuse visual features across modalities. The framework is further optimized by the cross-modal objectives, including image-text contrastive learning and image-text matching, which maps the same scene with different weather conditions closer in the representation space. 
Extensive experiments validate that, under diverse weather conditions, our method achieves competitive recall rates compared to state-of-the-art drone geo-localization methods. Notably, it improves Recall@1 by 13.37\% under night conditions and by 18.69\% under fog and snow conditions. Our code is available at https://github.com/Jahawn-Wen/WeatherPrompt.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes WeatherPrompt, a novel multi-modal learning framework for robust drone visual geo-localization under diverse weather conditions. The key innovation lies in leveraging CoT prompting with LVLMs to generate structured weather-aware textual descriptions without manual labeling, enabling scalable training data creation. The framework further introduces a text-driven dynamic gating mechanism to disentangle scene and weather features adaptively, enhanced by multi-modal contrastive and matching losses. Experiments on University-1652 and SUES-200 datasets demonstrate significant improvements over state-of-the-art methods, particularly in challenging conditions (e.g., +18.69% Recall@1 for fog/snow).\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The method is intuitive. Currently, it is a trend to use large language models (LLMs) for scene description. However, direct image description by LLMs may lead to hallucination, while using chain-of-thought (CoT) can guide the output of LLMs to meet expectations.\n\n2. The training-free description pipeline generalizes to unseen or mixed weather conditions, addressing a key limitation of closed-set methods. Currently, many methods utilize learning from synthetic data to generalize to the real world.\n\n3. Comprehensive benchmarks across 10+ weather types and ablation studies validate the design choices\n\nWeaknesses:\n\n1. The method is intuitive but overly simplistic. Regarding the proposed scene description method, although it is training-free, its corresponding contribution points are much fewer. Moreover, the multimodal alignment framework also lacks key innovative points.\n\n2. I still have some questions to ask you, all of which are included in Questions.\n\nQUESTIONS:\n(1) The article points out that large vision-language models usually generate weather labels or descriptions in an intuitive manner. However, how can you ensure that the descriptions guided by CoT prompts are free from factual biases?\nThe current approach only improves the accuracy of the descriptions provided by LLMs, but it cannot supervise whether these descriptions are accurate.\n\n(2) Expansion of (1): The 6-step CoT prompt is central to description quality. Was this structure empirically optimized? A comparison with simpler prompts (e.g., direct QA) would help justify the complexity.\n\n(3)Regarding the second challenge in the intro: (2) Scene–Weather Feature Entanglement is somewhat ambiguous.\nThe entanglement of scene and weather features here is not clearly explained. Both the text description and the image features should contain scene and weather information. The disentanglement here should be targeted at the scene and weather in the two modalities.\n\n(4) You report strong performance on Dark+Rain+Fog, but the dataset’s synthetic weather may not capture real-world complexity. How does the model handle natural weather transitions without explicit CoT supervision during inference?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-19"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nWeatherPrompt introduces a training-free, text-guided multimodal framework for drone geo-localization under diverse weather conditions by leveraging large vision–language models and chain-of-thought prompting to generate structured weather and spatial descriptions. Through a dynamic gating mechanism and contrastive alignment between text and image features, the model achieves robust performance across unseen and adverse weather scenarios, outperforming prior state-of-the-art methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The issue studied in the paper is interesting. It is largely overlooked by existing methods.\n2. The abs and intro sections are well-structured and easy to follow.\n\nWeaknesses:\n1. For Figure 3, how to get the box ( $I$, $I_1$, ... )?\n2. Line 178, the authors claim that their method is training-free, but why is it training-free? There are learnable parameters in the pipeline.\n3. Line 181, it's confused between Figure 3 and the illustration here. The cross-modal fusion module is not reflected in Figure 3.\n4. Lines 185-198 are not clearly written. It is ambiguous whether $T$ refers to the different colored segments in Figure 3 or the entire weather descriptions. Simply referring to \"text embeddings\" is too vague.\n5. For line 193, the samples, What is $C$?\n6. Line 211, MPL should be MLP.\n7. In Table 1, the normal part, the performance drop seems too large. Would training Sample4Geo on the augmented data directly lead to good localization results under different weather conditions?\n8. A very important set of experiments is missing: Why are these four loss functions necessary? Do all of them actually contribute positively? Do they have the same impact under different weather conditions?\n9. Additionally, there is no analysis of hyperparameters. Do the weights of these losses have to be equal?\n10. Why this paper is under \"Reinforcement learning\" primary area?\n11. The code is not open released.\n\nQUESTIONS:\nPlease refer to the weaknesses section.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a multimodal learning framework named WeatherPrompt, which aims to address the performance degradation of drone-based visual geo-localization under complex weather conditions. This framework establishes weather-invariant representations by fusing image embeddings with textual context to enhance generalization capability across diverse weather scenarios. \nThe main innovations include: 1) A training-free weather reasoning mechanism that automatically generates high-quality weather descriptions using large multimodal models; 2) A text-driven multimodal architecture that disentangles scene and weather features through dynamic gating mechanisms; 3) Comprehensive validation demonstrating superior performance across multiple benchmark datasets.\n\nSTRENGTHS AND WEAKNESSES:\n1. Strengths: The methodology section is comprehensive, with clear explanations of the training-free weather reasoning mechanism and the multi-modality alignment model. The use of chain-of-thought prompting is well-justified, and the reasoning steps are clearly outlined. Besides, the approach leverages the power of large multi-modality models in a novel way, providing a scalable solution for generating high-quality weather descriptions. This could inspire further research on using large language models for vision-language alignment tasks. Thirdly, the paper highlights the importance of disentangling scene and weather features, which is a critical step towards more robust and generalizable geo-localization systems. The proposed techniques could be applied to other computer vision tasks that require weather-invariant representations.\n2. Weaknesses: The paper focuses primarily on drone geo-localization and does not explore the broader applicability of the proposed framework to other vision-language tasks. A discussion on potential extensions or limitations in other domains would strengthen the significance of the work. Besides, the generated weather and spatial descriptions inherit the biases of the pre-trained language model, which may affect the accuracy of the alignment process. Although the model achieves excellent performance on the existing dataset, the limited geographic and weather diversity may hinder its generalization to rare or region-specific weather phenomena on a global scale.\n\nQUESTIONS:\n1. On p.2, line 74: Add “@” between “Recall” and “1” (typo correction).\n2. How do the authors plan to further validate the generalization capability of WeatherPrompt on real-world datasets with more diverse and complex weather conditions? Are there any ongoing efforts to collect and test on additional real-world drone geo-localization datasets? Related rating: significance.\n3. Given the use of large pre-trained models like Qwen2.5-VL-32B, what are the computational requirements and potential bottlenecks for deploying WeatherPrompt in real-time applications, especially on resource-constrained platforms? It is important to note that the inference time reported in the paper (0.024 seconds per query) does not include the time required to generate the text descriptions, which is a necessary step for each inference in practical applications. How does this additional time impact the feasibility of real-time deployment, and what strategies can be employed to mitigate this potential bottleneck? Related rating: quality.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes to enhance the models’ robustness under different weather conditions by combining the text information prompted from Large Vision Language Models (LVLMs) and the original image feature at the inference stage. In the proposed “training-free” pipeline, a weather-driven channel gating module, an image-text contrastive loss, and an image-text matching loss are proposed to promote the feature alignment at different granularities. Experiments on two drone-view geo-localization datasets demonstrate the robustness of the proposed pipeline across different weather conditions.\n\nSTRENGTHS AND WEAKNESSES:\nPros:\n- Compared to scene representation using images, text information is inherently robust to the variance of weathers or image degradations. Therefore, it is an interesting and promising idea to leverage the text information to enhance the model’s robustness to different weathers.\n- Experiments on two datasets demonstrate the significant improvement from the proposed method. Also, the results on real world videos show that the proposed method can be appliable to real-world scenarios.\n\nCons:\n- My major concern is about the significance of proposing a so-called \"training-free\" pipeline for different weather conditions, especially for the drone-based geo-localization task. In other words, the model’s robustness can be simply enhanced by training under a random combination of different weather conditions. Since the main results only compare the proposed model with baseline models pre-trained on clean images, it is potentially unfair for these models which have no prior information about weather, and it is hard to see how baselines perform when the model has been exposed to different weather-induced distortions during training.\n- The paper claims generalizability to unseen weather conditions, while the generalization experiments seem to be only performed on different combinations of distortions that have been seen during training. A more comprehensive investigation about the model’s extrapolation ability would further enhance the performance of the model.\n- Although the proposed method can enhance the robustness under various weather conditions, there are instances where performance drops under standard (clean) conditions. This trade-off could be more explicitly analyzed and discussed.\n- The method fuses image features with text embeddings at inference time, but the contribution of textual information to robustness is not clearly analyzed. An ablation study isolating the effect of text-only input would help clarify its role in enhancing robustness under various weather scenarios.\n\nQUESTIONS:\nSee my comments, I could increase my score if these weaknesses can be properly addressed.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper works on improving geolocalization for drones under diverse weather conditions. The paper argues that there are fundamentally two issues with current methods for drone visual geo-localization: 1) current models take into account pre-defined fixed set of weather labels but real-world weather scenarios are combinatorial and more complex, and 2) the weather and visual representation from existing models are often entangled leading to poor representations on unseen weather situations. To alleviate these issues, the authors first propose a pipeline to generate robust weather prompts by leveraging exisitng pre-trained VLMs. Second, the authors propose a representation learning pipeline that aligns image and text representations and directly predicts the geographic location.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n- The motivation for the paper is clear. The sharp degradation in performance of geolocalization models under adverse weather conditions suggest that there is a strong need to design systems that are explicitly conditioned to handle this issue.\n\n## Weaknesses \nMajor:\n- Ln 144 mentions that the authors use \"imgaug\" to generate \"realistic weather variations\". This pipeline needs to be explained and evaluated in detail. Without understanding the design for generating these weather variations, it is hard to judge how good/realistic the synthetic data is. If there is significant difference in the data distribution of the synthetically introduced weather variations, and real weather variations, any downstream model trained on such data will likely overfit to specific patterns in the synthetic data and not generalize well to real-world conditions. Some of the examples shown in figure 2, specially ones with snow, look quite different from a plausibe \"real\" counterpart.\n- There are several components that are missing or atleast confusing, when reading the paper. For example, how inference is done is not clear when reading the paper. The framework itself has a classification module that predicts geolocation, which could be a concievable way to geolocalize. Ln (232-233) mentions that Euclidean distance between query and candidate embeddings is used but it is not clear how the candidate embeddings are generated.\n\nMinor:\n- In Table 2, AP is incorrectly bolded on \"ours\" when Sample4Geo has a higher AP\n- Ln 144: Gramatically incorrect sentence\n- Ln 211: Is it supposed to be MLP?\n- The primary area for this paper is selected as Reinforcement Learning. This paper is much more aligned to Representation Learning than Reinforcement Learning.\n\nQUESTIONS:\n1. Please adress some of the comments in the weakness section\n2. This is text from the paper: \"Existing methods directly inject coarse pseudo-weather labels (e.g., rain, fog) into visual representations during training, leading to a severe entanglement between scene semantics and weather disturbances. Consequently, the model learns suboptimal representations under mixed or unseen weather conditions, fails to disentangle scene content from weather noise, and is severely limited in cross-weather generalization.\" Could you please elaborate on why the first sentence leads to the outcome described in the second sentence.\n\nLIMITATIONS:\nThe model is primarily evaluated on the synthetic data generated by authors which makes it difficult to understand the general applicability of the model in real-world scenarios. While there is an experiment on real-world data (Table 3c), the data is very small with limited details, making it unclear if those results will be reproducible in other settings.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}