GenAI4ELab/papercli-papers huggingface dataset:
Vision-Based Landing Guidance through Tracking and Orientation Estimation
Authors: João P. K. Ferreira; João P. Pinto; Júlia Moura; Yi Li; Cristiano L. Castro; Plamen Angelov
Venue: WACV 2025 (main)

Abstract: Fixed-wing aerial vehicles are equipped with functionalities such as ILS (instrument landing system) PAR (precision approach radar) and DGPS (differential global positioning system) enabling fully automated landings. However these systems impose significant costs on airport operations due to high installation and maintenance requirements. Moreover since these navigation parameters come from ground or satellite signals they are vulnerable to interference. A more cost-effective and independent alternative for guiding landing is a vision-based system that detects the runway and aligns the aircraft reducing the pilot's cognitive load. This paper proposes a novel framework that addresses three key challenges in developing autonomous vision-based landing systems. Firstly to overcome the lack of aerial front-view video data we created high-quality videos simulating landing approaches through the generator code available in the LARD (landing approach runway detection dataset) repository. Secondly in contrast to former studies focusing on object detection for finding the runway we chose the state-of-the-art model LoRAT to track runways within bounding boxes in each video frame. Thirdly to align the aircraft with the designated landing runway we extract runway keypoints from the resulting LoRAT frames and estimate the camera relative pose via the Perspective-n-Point algorithm. Our experimental results over a dataset of generated videos and original images from the LARD dataset consistently demonstrate the proposed framework's highly accurate tracking and alignment capabilities. Our approach source code and the LoRAT model pre-trained with LARD videos are available at https://github.com/jpklock2/vision-based-landing-guidance
A Video is Worth 10000 Words: Training and Benchmarking with Diverse Captions for Better Long Video Retrieval
Authors: Matthew Gwilliam; Michael Cogswell; Meng Ye; Karan Sikka; Abhinav Shrivastava; Ajay Divakaran
Venue: WACV 2025 (main)

Abstract: Existing long video retrieval systems are trained and tested in the paragraph-to-video retrieval regime where every long video is described by a single long paragraph. This neglects the richness and variety of possible valid descriptions of a video which could range anywhere from moment-by-moment detail to a single phrase summary. To provide a more thorough evaluation of the capabilities of long video retrieval systems we propose a pipeline that leverages state-of-the-art large language models to carefully generate a diverse set of synthetic captions for long videos. We validate this pipeline's fidelity via rigorous human inspection. We use synthetic captions from this pipeline to perform a benchmark of a representative set of video language models using long video datasets and show that the models struggle on shorter captions. We show that finetuning on this data can both mitigate these issues (+2.8% R@1 over SOTA on ActivityNet with diverse captions) and even improve performance on standard paragraph-to-video retrieval (+1.0% R@1 on ActivityNet). We also use synthetic data from our pipeline as query expansion in the zero-shot setting (+3.4% R@1 on ActivityNet). We derive insights by analyzing failure cases for retrieval with short captions.
WeedsGalore: A Multispectral and Multitemporal UAV-Based Dataset for Crop and Weed Segmentation in Agricultural Maize Fields
Authors: Ekin Celikkan; Timo Kunzmann; Yertay Yeskaliyev; Sibylle Itzerott; Nadja Klein; Martin Herold
Venue: WACV 2025 (main)

Abstract: Weeds are one of the major reasons for crop yield loss but current weeding practices fail to manage weeds in an efficient and targeted manner. Effective weed management is especially important for crops with high worldwide production such as maize to maximize crop yield for meeting increasing global demands. Advances in near-sensing and computer vision enable the development of new tools for weed management. Specifically state-of-the-art segmentation models coupled with novel sensing technologies can facilitate timely and accurate weeding and monitoring systems. However learning-based approaches require annotated data and show a lack of generalization to aerial imaging for different crops. We present a novel dataset for semantic and instance segmentation of crops and weeds in agricultural maize fields. The multispectral UAV-based dataset contains images with RGB red-edge and near-infrared bands a large number of plant instances dense annotations for maize and four weed classes and is multitemporal. We provide extensive baseline results for both tasks including probabilistic methods to quantify prediction uncertainty improve model calibration and demonstrate the approach's applicability to out-of-distribution data. The results show the effectiveness of the two additional bands compared to RGB only and better performance in our target domain than models trained on existing datasets. We hope our dataset advances research on methods and operational systems for fine-grained weed identification enhancing the robustness and applicability of UAV-based weed management. The dataset and code are available at https://github.com/GFZ/weedsgalore.
Generalist YOLO: Towards Real-Time End-to-End Multi-Task Visual Language Models
Authors: Hung-Shuo Chang; Chien-Yao Wang; Richard Robert Wang; Gene Chou; Hong-Yuan Mark Liao
Venue: WACV 2025 (main)

Abstract: Generalist models capable of handling multiple modalities and tasks simultaneously are currently one of the hottest research topics. However due to interference between different tasks during the training process existing generalist models require a very large decoder to achieve good results in various tasks which makes real-time prediction difficult for current generalist models. This paper introduces Generalist YOLO which takes a significant step towards real-time prediction systems for visual language generalist models. The proposed Generalist YOLO uses a unified encoder to reduce conflicts between different tasks thereby decreasing the complexity required by the decoder. It also introduces a primary-secondary co-attention mechanism that allows different tasks to learn together more effectively achieving high efficiency and high accuracy. We propose a semantically consistent asymmetric training strategy allowing various tasks to benefit from performance improvements brought by the latest research results in various fields. The proposed Generalist YOLO achieves excellent results on various vision and language tasks based on MS COCO. While maintaining high accuracy across all tasks it is 135 times faster than existing generalist models. The source code is released on GitHub at https://github.com/WongKinYiu/GeneralistYOLO.
Event-Guided Low-Light Video Semantic Segmentation
Authors: Zhen Yao; Mooi Choo Chuah
Venue: WACV 2025 (main)

Abstract: Recent video semantic segmentation (VSS) methods have demonstrated promising results in well-lit environments. However their performance significantly drops in low-light scenarios due to limited visibility and reduced contextual details. In addition unfavorable low-light conditions make it harder to incorporate temporal consistency across video frames and thus lead to video flickering effects. Compared with conventional cameras event cameras can capture motion dynamics filter out temporal-redundant information and are robust to lighting conditions. To this end we propose EVSNet a lightweight framework that leverages event modality to guide the learning of a unified illumination-invariant representation. Specifically we leverage a Motion Extraction Module to extract short-term and long-term temporal motions from event modality and a Motion Fusion Module to integrate image features and motion features adaptively. Furthermore we use a Temporal Decoder to exploit video contexts and generate segmentation predictions. Such designs in EVSNet result in a lightweight architecture while achieving SOTA performance. Experimental results on 3 large-scale datasets demonstrate our proposed EVSNet outperforms SOTA methods with up to 11x higher parameter efficiency.
VADet: Multi-Frame LiDAR 3D Object Detection using Variable Aggregation
Authors: Chengjie Huang; Vahdat Abdelzad; Sean Sedwards; Krzysztof Czarnecki
Venue: WACV 2025 (main)

Abstract: Input aggregation is a simple technique used by state-of-the-art LiDAR 3D object detectors to improve detection. However increasing aggregation is known to have diminishing returns and even performance degradation due to objects responding differently to the number of aggregated frames. To address this limitation we propose an efficient adaptive method which we call Variable Aggregation Detection (VADet). Instead of aggregating the entire scene using a fixed number of frames VADet performs aggregation per object with the number of frames determined by an object's observed properties such as speed and point density. VADet thus reduces the inherent trade-offs of fixed aggregation and is not architecture specific. To demonstrate its benefits we apply VADet to three popular single-stage detectors and achieve state-of-the-art performance on the Waymo dataset.
CL-Cross VQA: A Continual Learning Benchmark for Cross-Domain Visual Question Answering
Authors: Yao Zhang; Haokun Chen; Ahmed Frikha; Denis Krompass; Gengyuan Zhang; Jindong Gu; Volker Tresp
Venue: WACV 2025 (main)

Abstract: Visual Question Answering (VQA) systems witnessed a significant advance in recent years due to the development of large-scale Vision-Language Pre-trained Models (VLPMs). As the application scenario and user demand change over time an advanced VQA system is expected to be capable of continuously expanding its knowledge and capabilities over time not only to handle new tasks (i.e. new question types or visual scenes) but also to answer questions in new specialized domains without forgetting previously acquired knowledge and skills. Existing works studying CL on VQA tasks primarily consider answer- and question-type incremental learning or scene- and function-incremental learning whereas how VQA systems perform when they encounter new domains and increasing user demands has not been studied. Motivated by this we introduce CL-CrossVQA a rigorous Continual Learning benchmark for Cross-domain Visual Question Answering through which we conduct extensive experiments on 4 VLPMs 5 CL approaches and 5 VQA datasets from different domains. In addition by probing the forgetting phenomenon of the intermediate layers we provide insights into how model architecture affects CL performance why CL approaches can help mitigate forgetting in VLPMs and how to design CL approaches suitable for VLPMs in this challenging continual learning environment. To facilitate future work on developing an advanced All-in-One VQA system we will release our datasets and code.
LogicNet: A Logical Consistency Embedded Face Attribute Learning Network
Authors: Haiyu Wu; Sicong Tian; Huayu Li; Kevin W. Bowyer
Venue: WACV 2025 (main)

Abstract: Ensuring logical consistency in predictions is a crucial yet overlooked aspect in face attribute classification. We explore the potential reasons for this oversight and introduce two pressing challenges to the field: 1) How can we ensure that a model when trained with data checked for logical consistency yields predictions that are logically consistent? 2) How can we achieve the same with training data that hasn't undergone logical consistency checks? Minimizing manual effort is also essential for enhancing automation. To address these challenges we introduce two datasets FH41K and CelebA-logic and propose LogicNet which combines adversarial learning and label poisoning to learn the logical relationship between attributes without the need for post-processing steps. The accuracy of LogicNet surpasses that of the next-best approach by 13.36% 9.96% and 1.01% on FH37K FH41K and CelebA-logic respectively. In real-world case analysis our approach can achieve a reduction of more than 50% in the average number of failed cases (logically inconsistent attributes) compared to other methods. Code link: https://github.com/HaiyuWu/LogicNet.
EgoPoints: Advancing Point Tracking for Egocentric Videos
Authors: Ahmad Darkhalil; Rhodri Guerrier; Adam W. Harley; Dima Damen
Venue: WACV 2025 (main)

Abstract: We introduce EgoPoints a benchmark for point tracking in egocentric videos. We annotate 4.7K challenging tracks in egocentric sequences. Compared to the popular TAP-Vid-DAVIS evaluation benchmark we include 9x more points that go out-of-view and 59x more points that require re-identification (ReID) after returning to view. To measure the performance of models on these challenging points we introduce evaluation metrics that specifically monitor tracking performance on points in-view out-of-view and points that require re-identification. We then propose a pipeline to create semi-real sequences with automatic ground truth. We generate 11K such sequences by combining dynamic Kubric objects with scene points from EPIC Fields. When fine-tuning point tracking methods on these sequences and evaluating on our annotated EgoPoints sequences we improve CoTracker across all metrics including the tracking accuracy d^*_avg by 2.7 percentage points and accuracy on ReID sequences (ReIDd_avg) by 2.4 points. We also improve d^*_avg and ReIDd_avg of PIPs++ by 0.3 and 2.8 respectively.
Optimizing Vision-Language Model for Road Crossing Intention Estimation
Authors: Roy Uziel; Oded Bialer
Venue: WACV 2025 (main)

Abstract: Identifying a pedestrian's intention to cross the road is crucial for autonomous driving as it alerts the system to stop or slow down. However determining crossing intention from video is challenging due to the need for extracting complex high-level semantics. This paper introduces ClipCross a novel classification framework optimized to extract high-level semantic features using the vision-language model CLIP for determining crossing intention. Existing CLIP-based methods perform poorly in this task as CLIP's image and text encoders fail to capture the nuanced semantic distinctions between crossing and non-crossing intention images. ClipCross addresses this by optimizing a set of CLIP text embeddings to extract high-level semantic features which a multi-layer perceptron uses to distinguish between crossing and non-crossing intentions. ClipCross achieves state-of-the-art performance on crossing intention estimation benchmark datasets: PIE PSI and JAAD.
Loose Social-Interaction Recognition in Real-World Therapy Scenarios
Authors: Abid Ali; Rui Dai; Ashish Marisetty; Guillaume Astruc; Monique Thonnat; Jean-Marc Odobez; Susanne Thummler; Francois Bremond
Venue: WACV 2025 (main)

Abstract: The computer vision community has explored dyadic interactions for atomic actions such as pushing carrying-object etc. However with the advancement in deep learning models there is a need to explore more complex dyadic situations such as loose interactions. These are interactions where two people perform certain atomic activities to complete a global action irrespective of temporal synchronisation and physical engagement like cooking-together for example. Analysing these types of dyadic-interactions has several useful applications in the medical domain for social-skills development and mental health diagnosis. To achieve this we propose a novel dual-path architecture to capture the loose interaction between two individuals. Our model learns global abstract features from each stream via a CNNs backbone and fuses them using a new Global-Layer-Attention module based on a cross-attention strategy. We evaluate our model on real-world autism diagnoses such as our Loose-Interaction dataset and the publicly available Autism dataset for loose interactions. Our network achieves baseline results on the Loose-Interaction and SOTA results on the Autism datasets. Moreover we study different social interactions by experimenting on a publicly available dataset i.e. NTU-RGB+D (interactive classes from both NTU-60 and NTU-120). We have found that different interactions require different network designs. We also compare a slightly different version of our method (details in Section 3.6) by incorporating time information to address tight interactions achieving SOTA results.
Semantic Segmentation Method for Automated Indoor 3D Reconstruction Based on Architectural-Knowledge-Aware Features
Authors: Yahan Chen; Wenzheng Liu; Xiaowei Luo
Venue: WACV 2025 (main)

Abstract: 3D point cloud semantic segmentation is an important step for 3D indoors reconstruction. In recent years many outstanding deep learning models have been proposed for semantic segmentation which can achieve remarkable performance. However it is found the index indicating semantic prediction accuracy in terms of structural components (e.g. columns beams etc.) in buildings is far from satisfying lacking sufficient information for subsequent 3D reconstruction. For better segmenting and identifying structural components this work proposes Architectural-Knowledge-Aware features (AKAFs) i.e. F1 and F2 which are strategically incorporated into a developed two-stage training framework wherein outstanding semantic segmentation models are adopted as backbones. By incorporating F1 which formalizes the position distribution pattern of building structural components semantic information can be explored preliminary in the explicit stage (i.e. the first stage). The second stage is the implicit stage where F2 is derived based on the Semantic and Relative Position Fusing Module (SRPFM) which implicitly introduces more relative position information of building components for semantic segmentation. Extensive experiments have been conducted on the S3DIS dataset adopting three outstanding backbones. Results demonstrate that the proposed AKAFs can increase the accuracy of segmentation for structural components by more than 5%. As a result the overall semantic segmentation performance increases by 2%. Even on the current SOTA model (PT-v3) the proposed AKAFs show promising semantic segmentation promotion. Portable and well-compatible with most point-based semantic segmentation models AKAFs can effectively perceive more architectural characteristics hidden in indoor point clouds especially prominent for points of structural components.
AdQuestA: Knowledge-Guided Visual Question Answer Framework for Advertisements
Authors: Neha Choudhary; Poonam Goyal; Devashish Siwatch; Atharva Chandak; Harsh Mahajan; Varun Khurana; Yaman Kumar
Venue: WACV 2025 (main)

Abstract: In the rapidly evolving landscape of digital marketing effective customer engagement through advertisements is crucial for brands. Thus computational understanding of ads is pivotal for recommendation authoring and customer behaviour simulation. Despite advancements in knowledge-guided visual-question-answering (VQA) models existing frameworks often lack domain-specific responses and suffer from a dearth of benchmark datasets for advertisements. To address this gap we introduce ADVQA the first dataset for ad-related VQA sourced from Facebook and X (twitter) which facilitates further research in ad comprehension. It comprises open-ended questions and detailed context obtained automatically from web articles. Moreover we present AdQuestA a novel multimodal framework for knowledge-guided open-ended question-answering tailored to advertisements. AdQuestA leverages a Retrieval Augmented Generation (RAG) to obtain question-aware ad context as explicit knowledge and image-grounded implicit knowledge effectively exploiting inherent relationships for reasoning. Extensive experiments corroborate its efficacy yielding state-of-the-art performance on the ADVQA dataset even surpassing 10X larger models such as GPT-4 on this task. Our framework not only enhances understanding of ad content but also advances the broader landscape of knowledge-guided VQA models.
Generating Long-Take Videos via Effective Keyframes and Guidance
Authors: Hsin-Ping Huang; Yu-Chuan Su; Ming-Hsuan Yang
Venue: WACV 2025 (main)

Abstract: We tackle the challenge of generating long-take videos encompassing multiple non-repetitive yet coherent events. Existing approaches generate long videos conditioned on single input guidance often leading to repetitive content. To address this problem we develop a framework that uses multiple guidance sources to enhance long video generation. The main idea of our approach is to decouple video generation into keyframe generation and frame interpolation. In this process keyframe generation focuses on creating multiple coherent events while the frame interpolation stage generates smooth intermediate frames between keyframes using existing video generation models. A novel mask attention module is further introduced to improve coherence and efficiency. Experiments on challenging real-world videos demonstrate that the proposed method outperforms prior methods by up to 9.5% in objective metrics.
RGB-D Video Mirror Detection
Authors: Mingchen Xu; Peter Herbert; Yu-Kun Lai; Ze Ji; Jing Wu
Venue: WACV 2025 (main)

Abstract: Mirror detection aims to identify mirror areas in a scene with recent methods either integrating depth information (RGB-D) or making use of temporal information (video). However utilizing both data is still under-explored due to the lack of a high-quality dataset and an effective method for the RGB-D Video Mirror Detection (DVMD) problem. To the best of our knowledge this is the first work to address the DVMD problem. To exploit depth and temporal information in mirror segmentation we first construct a large-scale RGB-D Video Mirror Detection Dataset (DVMD-D) which contains 17977 RGB-D images from 273 diverse videos. We further develop a novel model named DVMDNet which can first locate the mirrors based on triple consistencies: local consistency cross-modality consistency and global consistency and then refine the mirror boundaries through content discontinuity taking the temporal information within videos into account. We conduct a comparative study on the DVMD dataset evaluating 12 state-of-the art models (including single-image mirror detection single-image glass detection RGB-D mirror detection video shadow detection video glass detection and video mirror detection methods). Code is available from https://github.com/UpChen/2025_DVMDNet.
PostoMETRO: Pose Token Enhanced Mesh Transformer for Robust 3D Human Mesh Recovery
Authors: Wendi Yang; Zi-Hang Jiang; Shang Zhao; S. Kevin Zhou
Venue: WACV 2025 (main)

Abstract: With the recent advancements in single-image-based 3D human pose and shape estimation (3DHPSE) there is a growing amount of works that can achieve good results on standard benchmarks but struggle to yield accurate human mesh in extreme scenarios like occlusion. Previous works propose to leverage 2D poses to help 3D HPSE model improve performance under occlusion but usually rely on manual design to integrate 2D poses and only aim for specific kinds of occlusion. In this paper we present PostoMETRO (POSe TOken enhanced MEsh TRansfOrmer) which integrates 2D pose prior knowledge as tokens into transformers to improve model's performance under occlusion. Using a VQ-VAE-based pose tokenizer we efficiently represent 2D poses as tokens and feed them to transformers together with image tokens. Subsequently these tokens are queried by vertex and joint tokens to decode 3D coordinates of mesh vertices and human joints. Our proposed 2D poses integration strategy is manual-design-free and suitable for various kinds of occlusion. Experiments on both standard and occlusion-specific benchmarks demonstrate the effectiveness of PostoMETRO.