GenAI4ELab/papercli-papers huggingface dataset:
CAMEL: Confidence-Aware Multi-Task Ensemble Learning with Spatial Information for Retina OCT Image Classification and Segmentation
Authors: Juho Jung; Migyeong Yang; Hyunseon Won; Jiwon Kim; Jeong Mo Han; Joon Seo Hwang; Daniel Duck-Jin Hwang; Jinyoung Han
Venue: WACV 2025 (main)

Abstract: Precise retina Optical Coherence Tomography (OCT) image classification and segmentation are important for diagnosing various retinal diseases and identifying specific regions. Alongside comprehensive lesion identification reducing the predictive uncertainty of models is crucial for improving reliability in clinical retinal practice. However existing methods have primarily focused on a limited set of regions identified in OCT images and have often faced challenges due to aleatoric and epistemic uncertainty. To address these issues we propose CAMEL (Confidence-Aware Multi-task Ensemble Learning) a novel framework designed to reduce task-specific uncertainty in multi-task learning. CAMEL achieves this by estimating model confidence at both pixel and image levels and leveraging confidence-aware ensemble learning to minimize the uncertainty inherent in single-model predictions. CAMEL demonstrates state-of-the-art performance on a comprehensive retinal OCT image dataset containing annotations for nine distinct retinal regions and nine retinal diseases. Furthermore extensive experiments highlight the clinical utility of CAMEL especially in scenarios with minimal regions significant class imbalances and diverse regions and diseases. Our code is publicly available at: https://github.com/DSAIL-SKKU/CAMEL.
ERM++: An Improved Baseline for Domain Generalization
Authors: Piotr Teterwak; Kuniaki Saito; Theodoros Tsiligkaridis; Kate Saenko; Bryan Plummer
Venue: WACV 2025 (main)

Abstract: Domain Generalization (DG) aims to develop classifiers that can generalize to new unseen data distributions a critical capability when collecting new domain-specific data is impractical. A common DG baseline minimizes the empirical risk on the source domains. Recent studies have shown that this approach known as Empirical Risk Minimization (ERM) can outperform most more complex DG methods when properly tuned. However these studies have primarily focused on a narrow set of hyperparameters neglecting other factors that can enhance robustness and prevent overfitting and catastrophic forgetting properties which are critical for strong DG performance. In our investigation of training data utilization (i.e. duration and setting validation splits) initialization and additional regularizers we find that tuning these previously overlooked factors significantly improves model generalization across diverse datasets without adding much complexity. We call this improved yet simple baseline ERM++. Despite its ease of implementation ERM++ improves DG performance by over 5% compared to prior ERM baselines on a standard benchmark of 5 datasets with a ResNet-50 and over 15% with a ViT-B/16. It also outperforms all state-of-the-art methods on DomainBed datasets with both architectures. Importantly ERM++ is easy to integrate into existing frameworks like DomainBed making it a practical and powerful tool for researchers and practitioners. Overall ERM++ challenges the need for more complex DG methods by providing a stronger more reliable baseline that maintains simplicity and ease of use. Code is available at https://github.com/piotr-teterwak/erm_plusplus.
WAFFLE: Multimodal Floorplan Understanding in the Wild
Authors: Keren Ganon; Morris Alper; Rachel Mikulinsky; Hadar Averbuch-Elor
Venue: WACV 2025 (main)

Abstract: Buildings are a central feature of human culture and are increasingly being analyzed with computational methods. However recent works on computational building understanding have largely focused on natural imagery of buildings neglecting the fundamental element defining a building's structure - its floorplan. Conversely existing works on floorplan understanding are extremely limited in scope often focusing on floorplans of a single semantic category and region (e.g. floorplans of apartments from a single country). In this work we introduce WAFFLE a novel multimodal floorplan understanding dataset of nearly 20K floorplan images and metadata curated from Internet data spanning diverse building types locations and data formats. By using a large language model and multimodal foundation models we curate and extract semantic information from these images and their accompanying noisy metadata. We show that WAFFLE enables progress on new building understanding tasks both discriminative and generative which were not feasible using prior datasets. We will publicly release WAFFLE along with our code and trained models providing the research community with a new foundation for learning the semantics of buildings.
Test-Time Low Rank Adaptation via Confidence Maximization for Zero-Shot Generalization of Vision-Language Models
Authors: Raza Imam; Hanan Gani; Muhammad Huzaifa; Karthik Nandakumar
Venue: WACV 2025 (main)

Abstract: The conventional modus operandi for adapting pre-trained vision-language models (VLMs) during test-time involves tuning learnable prompts i.e. test-time prompt tuning. This paper introduces Test-Time Low-rank adaptation (TTL) as an alternative to prompt tuning for zero-shot generalization of large-scale VLMs. Taking inspiration from recent advancements in efficiently fine-tuning large language models TTL offers a test-time parameter-efficient adaptation approach that updates the attention weights of the transformer encoder by maximizing prediction confidence. The self-supervised confidence maximization objective is specified using a weighted entropy loss that enforces consistency among predictions of augmented samples. TTL introduces only a small amount of trainable parameters for low-rank adapters in the model space while keeping the prompts and backbone frozen. Extensive experiments on a variety of natural distribution and cross-domain tasks show that TTL can outperform other techniques for test-time optimization of VLMs in strict zero-shot settings. Specifically TTL outperforms test-time prompt tuning baselines with a significant improvement on average. Our code is available at https://github.com/Razaimam45/TTL-Test-Time-Low-Rank-Adaptation.
ANTHROPOS-V: Benchmarking the Novel Task of Crowd Volume Estimation
Authors: Luca Collorone; Stefano Darrigo; Massimiliano Pappa; Guido M. Damely di Melendugno; Giovanni Ficarra; Fabio Galasso
Venue: WACV 2025 (main)

Abstract: We introduce the novel task of Crowd Volume Estimation (CVE) defined as the process of estimating the collective body volume of crowds using only RGB images. Besides event management and public safety CVE can be instrumental in approximating body weight unlocking weight-sensitive applications such as infrastructure stress assessment and assuring even weight balance. We propose the first benchmark for CVE comprising ANTHROPOS-V a synthetic photorealistic video dataset featuring crowds in diverse urban environments. Its annotations include each person's volume SMPL shape parameters and keypoints. Also we explore metrics pertinent to CVE define baseline models adapted from Human Mesh Recovery and Crowd Counting domains and propose a CVE-specific methodology that surpasses baselines. Although synthetic the weights and heights of individuals are aligned with the real-world population distribution across genders and they transfer to the downstream task of CVE from real images. Benchmark and code are available at github.com/colloroneluca/Crowd-Volume-Estimation.
HybridDepth: Robust Metric Depth Fusion by Leveraging Depth from Focus and Single-Image Priors
Authors: Ashkan Ganj; Hang Su; Tian Guo
Venue: WACV 2025 (main)

Abstract: We propose HYBRIDDEPTH a robust depth estimation pipeline that addresses key challenges in depth estimation including scale ambiguity hardware heterogeneity and generalizability. HYBRIDDEPTH leverages focal stack data conveniently accessible in common mobile devices to produce accurate metric depth maps. By incorporating depth priors afforded by recent advances in single-image depth estimation our model achieves a higher level of structural detail compared to existing methods. We test our pipeline as an end-to-end system with a newly developed mobile client to capture focal stacks which are then sent to a GPU-powered server for depth estimation. Comprehensive quantitative and qualitative analyses demonstrate that HYBRIDDEPTH outperforms state-of-the-art (SOTA) models on common datasets such as DDFF12 and NYU Depth V2. HYBRIDDEPTH also shows strong zero-shot generalization. When trained on NYU Depth V2 HYBRIDDEPTH surpasses SOTA models in zero-shot performance on ARKitScenes and delivers more structurally accurate depth maps on Mobile Depth. The code is available at https://github.com/cake-lab/HybridDepth/.
MFTIQ: Multi-Flow Tracker with Independent Matching Quality Estimation
Authors: Jonas Serych; Michal Neoral; Jiri Matas
Venue: WACV 2025 (main)

Abstract: In this work we present MFTIQ a novel dense long-term tracking model that advances the Multi-Flow Tracker (MFT) framework to address challenges in point-level visual tracking in video sequences. MFTIQ builds upon the flow-chaining concepts of MFT integrating an Independent Quality (IQ) module that separates correspondence quality estimation from optical flow computations. This decoupling significantly enhances the accuracy and flexibility of the tracking process allowing MFTIQ to maintain reliable trajectory predictions even in scenarios of prolonged occlusions and complex dynamics. Designed to be "plug-and-play" MFTIQ can be employed with any off-the-shelf optical flow method without the need for fine-tuning or architectural modifications. Experimental validations on the TAP-Vid Davis dataset show that MFTIQ with RoMa optical flow not only surpasses MFT but also performs comparably to state-of-the-art trackers while having substantially faster processing speed. Code and models available at https://github.com/serycjon/MFTIQ.
VHS: High-Resolution Iterative Stereo Matching with Visual Hull Priors
Authors: Markus Plack; Hannah Dröge; Leif Van Holland; Matthias B. Hullin
Venue: WACV 2025 (main)

Abstract: We present a stereo-matching method for depth estimation from high-resolution images using visual hulls as priors and a memory-efficient technique for the correlation computation. Our method uses object masks extracted from supplementary views of the scene to guide the disparity estimation effectively reducing the search space for matches. This approach is specifically tailored to stereo rigs in volumetric capture systems where an accurate depth plays a key role in the downstream reconstruction task. To enable training and regression at high resolutions targeted by recent systems our approach extends a sparse correlation computation into a hybrid sparse-dense scheme suitable for application in leading recurrent network architectures. We evaluate the performance-efficiency trade-off of our method compared to state-of-the-art approaches and demonstrate the efficacy of the visual hull guidance. In addition we propose a training scheme for a further reduction of memory requirements during optimization facilitating training on high-resolution data.
A Mamba-Based Siamese Network for Remote Sensing Change Detection
Authors: Jay N. Paranjape; Celso de Melo; Vishal M. Patel
Venue: WACV 2025 (main)

Abstract: Change detection in remote sensing images is an essential tool for analyzing a region at different times. It finds varied applications in monitoring environmental changes man-made changes as well as corresponding decision-making and prediction of future trends. Deep learning methods like Convolutional Neural Networks (CNNs) and Transformers have achieved remarkable success in detecting significant changes given two images at different times. In this paper we propose a Mamba-based Change Detector (M-CD) that segments out the regions of interest even better. Mamba-based architectures demonstrate linear-time training capabilities and an improved receptive field over transformers. Our experiments on four widely used change detection datasets demonstrate significant improvements over existing state-of-the-art (SOTA) methods. Code: https://github.com/JayParanjape/M-CD
Language-Guided Instance-Aware Domain-Adaptive Panoptic Segmentation
Authors: Elham Amin Mansour; Ozan Unal; Suman Saha; Benjamin Bejar; Luc Van Gool
Venue: WACV 2025 (main)

Abstract: The increasing relevance of panoptic segmentation is tied to the advancements in autonomous driving and AR/VR applications. However the deployment of such models has been limited due to the expensive nature of dense data annotation giving rise to unsupervised domain adaptation (UDA). A key challenge in panoptic UDA is reducing the domain gap between a labeled source and an unlabeled target domain while harmonizing the subtasks of semantic and instance segmentation to limit catastrophic interference. While considerable progress has been achieved existing approaches mainly focus on the adaptation of semantic segmentation. In this work we focus on incorporating instance-level adaptation via a novel instance-aware cross-domain mixing strategy IMix. IMix significantly enhances the panoptic quality by improving instance segmentation performance. Specifically we propose inserting high-confidence predicted instances from the target domain onto source images retaining the exhaustiveness of the resulting pseudo-labels while reducing the injected confirmation bias. Nevertheless such an enhancement comes at the cost of degraded semantic performance attributed to catastrophic forgetting. To mitigate this issue we regularize our semantic branch by employing CLIP-based domain alignment (CDA) exploiting the domain-robustness of natural language prompts. Finally we present an end-to-end model incorporating these two mechanisms called LIDAPS achieving state-of-the-art results on all popular panoptic UDA benchmarks. https://github.com/elhamAm/LIDAPS
Bit-Flip Induced Latency Attacks in Object Detection
Authors: Manojna Sistla; Yu Wen; Aamir Bader Shah; Chenpei Huang; Lening Wang; Xuqing Wu; Jiefu Chen; Miao Pan; Xin Fu
Venue: WACV 2025 (main)

Abstract: Deep learning and computer vision have experienced significant advancements particularly in critical applications such as autonomous driving and real-time surveillance where object detection (OD) plays a pivotal role. Ensuring the accuracy and speed of these systems is paramount to prevent accidents or failures. Recently latency-based attacks have emerged as a new threat driven by the essential need for real-time performance in various applications. These attacks target model responsiveness to disrupt system performance without necessarily compromising accuracy. Our preliminary experiments show that introducing just a few bit flips to key parameters in OD models can significantly increase latency degrading performance. Meanwhile recent advancements in memory-based attacks such as Row Hammer demonstrate the ability to conveniently introduce bit flips at desired locations without physical hardware interaction. Based on the observations we propose a novel attack on OD models that leverages row-hammer to introduce bit-flips via side channels targeting the non-maximum suppression (NMS) filter and significantly increasing latency. Unlike previous methods that modify input data our technique ensures efficiency by minimizing bit-flips through critical path exploitation and achieves practical applicability with only a subset of validation data. Experiments across various datasets and models validate our approach demonstrating latency increases up to 71.6 ms (20.4x) with just 31 bit-flips.
Foundation Models and Adaptive Feature Selection: A Synergistic Approach to Video Question Answering
Authors: Sai Bhargav Rongali; Mohamad Hassan N C; Ankit Jha; Neha Bhargava; Saurabh Prasad; Biplab Banerjee
Venue: WACV 2025 (main)

Abstract: This paper tackles the intricate challenge of video question-answering (VideoQA). Despite notable progress current methods fall short of effectively integrating questions with video frames and semantic object-level abstractions to create question-aware video representations. We introduce Local - Global Question Aware Video Embedding (LGQAVE) which incorporates three major innovations to integrate multi-modal knowledge better and emphasize semantic visual concepts relevant to specific questions. LGQAVE moves beyond traditional ad-hoc frame sampling by utilizing a cross-attention mechanism that precisely identifies the most relevant frames concerning the questions. It captures the dynamics of objects within these frames using distinct graphs grounding them in question semantics with the miniGPT model. These graphs are processed by a question-aware dynamic graph transformer (Q-DGT) which refines the outputs to develop nuanced global and local video representations. An additional cross-attention module integrates these local and global embeddings to generate the final video embeddings which a language model uses to generate answers. Extensive evaluations across multiple benchmarks demonstrate that LGQAVE significantly outperforms existing models in delivering accurate multi-choice and open-ended answers.
LLM-RSPF: Large Language Model-Based Robotic System Planning Framework for Domain Specific Use-Cases
Authors: Chandan Kumar Singh; Devesh Kumar; Vipul Sanap; Rajesh Sinha
Venue: WACV 2025 (main)

Abstract: The employment of large language models (LLMs) for task planning and reasoning has emerged as a focal point of interest within the robotics research community. However directly applying LLMs even with large token-sized prompts does not achieve the task planning performance required for an industrial-grade domain-specific use-case (DSU). This work aims to overcome the obstacles of a robotic task planner for DSUs by introducing a novel planning framework LLM-RSPF (Large Language Model-based Robotic System Planning Framework). Central to the LLM-RSPF is a novel robotic system ontology that organizes the components of the robotic system in a coherent and a systematic manner. The ontology empowers the LLM-RSPF to efficiently capture a contextual representation of the DSU using the LLMs. Subsequently the research introduces a LLM-tuning regimen referred as chain of hierarchical thought (CoHT) specifically crafted to complement the proposed system ontology. Integrating these two components the LLM-RSPF aims to enhance the accuracy robustness and throughput of a robotic system in a cost-effective manner. In addition the research presents an empirical methodology to generate the LLM-tuning dataset size for a guaranteed performance. The LLM-RSPF is validated on a retail order-fulfillment use-case thereby illustrating the efficacy of the framework. Through rigorous evaluation the LLM-RSPF demonstrates exceptional performance on the generated dataset effectively meeting the DSU objectives.
MemFusionMap: Working Memory Fusion for Online Vectorized HD Map Construction
Authors: Jingyu Song; Xudong Chen; Liupei Lu; Jie Li; Katherine A. Skinner
Venue: WACV 2025 (main)

Abstract: High-definition (HD) maps provide environmental information for autonomous driving systems and are essential for safe planning. While existing methods with single-frame input achieve impressive performance for online vectorized HD map construction they still struggle with complex scenarios and occlusions. We propose MemFusionMap a novel temporal fusion model with enhanced temporal reasoning capabilities for online HD map construction. Specifically we contribute a working memory fusion module that improves the model's memory capacity to reason across a history of frames. We also design a novel temporal overlap heatmap to explicitly inform the model about the temporal overlap information and vehicle trajectory in the Bird's Eye View space. By integrating these two designs MemFusionMap significantly outperforms existing methods while also maintaining a versatile design for scalability. We conduct extensive evaluation on open-source benchmarks and demonstrate a maximum improvement of 5.4% in mAP over state-of-the-art methods. The project page for MemFusionMap is https://song-jingyu.github.io/MemFusionMap.
DragonTrack: Transformer-Enhanced Graphical Multi-Person Tracking in Complex Scenarios
Authors: Bishoy Galoaa; Somaieh Amraee; Sarah Ostadabbas
Venue: WACV 2025 (main)

Abstract: This paper introduces the dynamic robust adaptive graph-based tracker (DragonTrack) as a novel end-to-end framework for multi-person tracking (MPT) by integrating a detection transformer model for object detection and feature extraction with a graph convolutional network for re-identification. DragonTrack leverages encoded features from the transformer for precise subject matching and track maintenance while the graphical component processes these features alongside geometric data to predict subsequent positions of tracked people. This methodology aims to enhance tracking accuracy and reliability as evidenced by improvements in key metrics such as higher order tracking accuracy (HOTA) and multiple object tracking accuracy (MOTA). We quantitatively compare DragonTrack with state-of-the-art methods on MOT17 MOT20 and DanceTrack datasets in which DragonTrack outperforms other methods. In challenging scenarios such as DanceTrack DragonTrack achieves an impressive MOTA score of 93.4 significantly higher than the second-best SOTA method ByteTrack which achieves only 89.6. Similarly on MOT17 DragonTrack scores 82.0 in MOTA surpassing the closest competitor with a score of 80.3. On MOT20 DragonTrack attains a HOTA score of 63.2 outperforming the next best method scoring 62.6
Improving Detail in Pluralistic Image Inpainting with Feature Dequantization
Authors: Kyungri Park; Woohwan Jung
Venue: WACV 2025 (main)

Abstract: Pluralistic Image Inpainting (PII) offers multiple plausible solutions for restoring missing parts of images and has been successfully applied to various applications including image editing and object removal. Recently VQGAN-based methods have been proposed and have shown that they significantly improve the structural integrity in the generated images. Nevertheless the state-of-the-art VQGAN-based model PUT faces a critical challenge: degradation of detail quality in output images due to feature quantization. Feature quantization restricts the latent space and causes information loss which negatively affects the detail quality essential for image inpainting. To tackle the problem we propose the FDM (Feature Dequantization Module) specifically designed to restore the detail quality of images by compensating for the information loss. Furthermore we develop an efficient training method for FDM which drastically reduces training costs. We empirically demonstrate that our method significantly enhances the detail quality of the generated images with negligible training and inference overheads. The code is available at https://github.com/hyudsl/FDM