GenAI4ELab/papercli-papers huggingface dataset:
Q-TempFusion: Quantization-Aware Temporal Multi-Sensor Fusion on Bird's-Eye View Representation
Authors: Pinrui Yu; Zhenglun Kong; Pu Zhao; Peiyan Dong; Hao Tang; Fei Sun; Xue Lin; Yanzhi Wang
Venue: WACV 2025 (main)

Abstract: Recent advancements in bird's-eye view (BEV) perception models have highlighted the superior performance of LiDAR-camera fusion systems over single-modality approaches garnering considerable interest in the field. Despite the progress the integration of temporal information a technique that has considerably benefitted camera-only BEV models remains underexplored for LiDAR-camera fusion. This paper presents Q-TempFusion a novel approach for temporal multi-sensor fusion designed to enhance the BEV model's inference speed while keeping high predictive performance compared with the current state-of-the-art. Moreover we are the first to make the multi-modality BEV model profiling on hardware devices. To address the challenges of substantial memory demands and non-trivial latency that hinder deployment in on-vehicle systems particularly when temporal dynamics are incorporated into complex multi-sensor models we introduce an activation-aware quantization framework to generate the fully 8-bit quantized Q-TempFusion model based on the profiling result which can be directly deployed to target devices with negligible detection performance degradation. Our experiments show that our Q-TempFusion (8-bit) achieves 70.3% mAP and 72.7% NDS with 3x - 18x FPS improvement over leading multi-modality baselines and the Q-TempFusion (32-bit) achieves 72.1% mAP and 74.8% NDS comparable to SOTA multi-modality approaches. The results suggest that Q-TempFusion is a promising step toward real-time multi-sensor BEV applications setting a new benchmark for efficient and reliable perception.
RD-DPP: Rate-Distortion Theory Meets Determinantal Point Process to Diversify Learning Data Samples
Authors: Xiwen Chen; Huayu Li; Peijie Qiu; Wenhui Zhu; Rahul Amin; Abolfazl Razi
Venue: WACV 2025 (main)

Abstract: Selecting representative samples plays an indispensable role in many machine learning and computer vision applications under limited resources (e.g. limited communication bandwidth and computational power). Determinantal Point Process (DPP) is a widely used method for selecting the most diverse representative samples that can summarize a dataset. However its adaptability to different tasks remains an open challenge as it is challenging for DPP to perform task-specific tuning. In contrast Rate-Distortion (RD) theory provides a way to measure task-specific diversity. However optimizing RD for a data selection problem remains challenging because the quantity that needs to be optimized is the index set of the selected samples. To tackle these challenges we first draw an inherent relationship between DPP and RD theory. Our theoretical derivation paves the way to take advantage of both RD and DPP for a task-specific data selection. To this end we propose a novel method for task-specific data selection for multi-level classification tasks named RD-DPP. Empirical studies on seven different datasets using five benchmark models demonstrate the effectiveness of the proposed RD-DPP method. Our method also outperforms recent strong competing methods while exhibiting high generalizability to a variety of learning tasks. The source code is available on https://github.com/xiwenc1/RD-DPP
D2FP: Learning Implicit Prior for Human Parsing
Authors: Junyoung Hong; Hyeri Yang; Ye Ju Kim; Haerim Kim; Shinwoong Kim; Euna Shim; Kyungjae Lee
Venue: WACV 2025 (main)

Abstract: Human parsing aims to segment human images into fine-grained semantic parts. Considering the underlying structure of the human body state-of-the-art methods typically depend on prior assumptions to represent intrinsic body relationships. However leveraging the same structural prior knowledge across various scenarios poses challenges in achieving stable prediction and requires additional network design efforts. To address these issues we introduce a novel method the Dynamic Dual Transformer for Parsing (D2FP) which dynamically learns the implicit prior structures of the human body. Specifically we derive input-dependent prior features from the learnable semantics of human images generating prior-embedded object queries accordingly before feeding them into the Transformer decoders. Our model includes three major components to effectively learn prior object queries: a prior extraction module a prior embedding module and a multi-scale dual Transformer decoder. Furthermore a novel prior enhancement strategy is introduced where the final decoded object queries provide structural clues to enhance initial prior features. Experimental results demonstrate the superiority and effectiveness of the proposed method across two well-known human parsing benchmarks: LIP and CIHP. Code and models are available at https://github.com/cvlab-yongin/D2FP.
ELBA: Learning by Asking for Embodied Visual Navigation and Task Completion
Authors: Ying Shen; Daniel Bis; Cynthia Lu; Ismini Lourentzou
Venue: WACV 2025 (main)

Abstract: The research community has shown increasing interest in designing intelligent embodied agents that can assist humans in accomplishing tasks. Although there have been significant advancements in related vision-language benchmarks most prior work has focused on building agents that follow instructions rather than endowing agents the ability to ask questions to actively resolve ambiguities arising naturally in embodied environments. To address this gap we propose an Embodied Learning-By-Asking (ELBA) model that learns when and what questions to ask to dynamically acquire additional information for completing the task. We evaluate ELBA on the TEACh vision-dialog navigation and task completion dataset. Experimental results show that the proposed method achieves improved task performance compared to baseline models without question-answering capabilities. Code is available at https://github.com/PLAN-Lab/ELBA.
DisFlowEm : One-Shot Emotional Talking Head Generation using Disentangled Pose and Expression Flow-Guidance
Authors: Sanjana Sinha; Brojeshwar Bhowmick; Lokender Tiwari; Sushovan Chanda
Venue: WACV 2025 (main)

Abstract: Generating realistic one-shot emotional talking head animation on arbitrary faces is a challenging problem as it requires realistic emotions head movements identity preservation and accurate lip sync. Existing emotional talking face generation methods either fail to retain the identity information of arbitrary subjects owing to the limited variability of existing emotional datasets or they fail to capture emotions accurately even if they preserve identity of arbitrary faces. Moreover most of the methods rely on additional input videos for driving poses and/or or expressions on the generated video. For practical applications it is infeasible to obtain driving videos of the same or different subject with variations in head pose expressions etc. In this paper we propose a novel approach for Audio-driven Emotional Talking Head generation from a single image with emotion-controllable head pose generation. Unlike existing methods our method does not require a driving video either for pose or emotions and can generate different emotions and diverse head pose variations from input speech and a single image of an arbitrary subject in neutral emotion. Our method overcomes the limitations of existing emotional audio-visual datasets by learning a disentangled approach for optical flow computation approach for pose and expression. Using our proposed method of independently computing pose-driven and expression-driven optical flow our image generation network can be pre-trained on a large dataset with greater pose variability but lacking emotion annotations. The expression flow generation branch is fine-tuned on a smaller emotional dataset to accurately capture different emotions not present in the original dataset while retaining the pose variability from the original dataset. We present extensive experiments to demonstrate the superiority of our proposed method in generating talking head animation with accurate emotions diverse head movements and generalization to arbitrary faces.
Modality-Incremental Learning with Disjoint Relevance Mapping Networks for Image-Based Semantic Segmentation
Authors: Niharika Hegde; Shishir Muralidhara; René Schuster; Didier Stricker
Venue: WACV 2025 (main)

Abstract: In autonomous driving environment perception has significantly advanced with the utilization of deep learning techniques for diverse sensors such as cameras depth sensors or infrared sensors. The diversity in the sensor stack increases the safety and contributes to robustness against adverse weather and lighting conditions. However the variance in data acquired from different sensors poses challenges. In the context of continual learning (CL) incremental learning is especially challenging for considerably large domain shifts e.g. different sensor modalities. This amplifies the problem of catastrophic forgetting. To address this issue we formulate the concept of modality-incremental learning and examine its necessity by contrasting it with existing incremental learning paradigms. We propose the use of a modified Relevance Mapping Network (RMN) to incrementally learn new modalities while preserving performance on previously learned modalities in which relevance maps are disjoint. Experimental results demonstrate that the prevention of shared connections in this approach helps alleviate the problem of forgetting within the constraints of a strict continual learning framework.
Uncertainty-Guided Cross Attention Ensemble Mean Teacher for Semi-Supervised Medical Image Segmentation
Authors: Meghana Karri; Amit Soni Arya; Koushik Biswas; Nicolo Gennaro; Vedat Cicek; Gorkem Durak; Yury S. Velichko; Ulas Bagci
Venue: WACV 2025 (main)

Abstract: This work proposes a novel framework Uncertainty-Guided Cross Attention Ensemble Mean Teacher (UG-CEMT) for achieving state-of-the-art performance in semi-supervised medical image segmentation. UG-CEMT leverages the strengths of co-training and knowledge distillation by combining a Cross-attention Ensemble Mean Teacher framework (CEMT) inspired by Vision Transformers (ViT) with uncertainty-guided consistency regularization and Sharpness-Aware Minimization emphasizing uncertainty. UG-CEMT improves semi-supervised performance while maintaining a consistent network architecture and task setting by fostering high disparity between sub-networks. Experiments demonstrate significant advantages over existing methods like Mean Teacher and Cross-pseudo Supervision in terms of disparity domain generalization and medical image segmentation performance. UG-CEMT achieves state-of-the-art results on multi-center prostate MRI and cardiac MRI datasets where object segmentation is particularly challenging. Our results show that using only 10% labeled data UG-CEMT approaches the performance of fully supervised methods demonstrating its effectiveness in exploiting unlabeled data for robust medical image segmentation. The code is publicly available at https://github.com/Meghnak13/UG-CEMT
FUN-AD: Fully Unsupervised Learning for Anomaly Detection with Noisy Training Data
Authors: Jiin Im; Yongho Son; Je Hyeong Hong
Venue: WACV 2025 (main)

Abstract: While the mainstream research in anomaly detection has mainly followed the one-class classification practical industrial environments often incur noisy training data due to annotation errors or lack of labels for new or refurbished products. To address these issues we propose a novel learning-based approach for fully unsupervised anomaly detection with unlabeled and potentially contaminated training data. Our method is motivated by two observations that i) the pairwise feature distances between the normal samples are on average likely to be smaller than those between the anomaly samples or heterogeneous samples and ii) pairs of features mutually closest to each other are likely to be homogeneous pairs which hold if the normal data has smaller variance than the anomaly data. Building on the first observation that nearest-neighbor distances can distinguish between confident normal samples and anomalies we propose a pseudo-labeling strategy using an iteratively reconstructed memory bank (IRMB). The second observation is utilized as a new loss function to promote class-homogeneity between mutually closest pairs thereby reducing the ill-posedness of the task. Experimental results on two public industrial anomaly benchmarks and semantic anomaly examples validate the effectiveness of FUN-AD across different scenarios and anomaly-to-normal ratios. Our code is available at https://github.com/HY-Vision-Lab/FUNAD.
Predicting Event Memorability using Personalized Federated Learning
Authors: Sourasekhar Banerjee; Debaditya Roy; Vigneshwaran Subbaraju; Monowar Bhuyan
Venue: WACV 2025 (main)

Abstract: Lifelog images are very useful as memory cues for recalling past events. Estimating the level of event memory recall induced by a given lifelog image (event memorability) is useful for selecting images for cognitive interventions. Previous works for predicting event memorability follow a centralized model training paradigm that requires several users to share their lifelog images. This risks violating the privacy of individual lifeloggers. Alternatively a personal model trained with a lifelogger's own data guarantees privacy. However it imposes significant effort on the lifelogger to provide a large enough sample of self-rated images to develop a well-performing model for event memorability. Therefore we propose a clustered personalized federated learning setup FedMEM that avoids sharing raw images but still enables collaborative learning via model sharing. For an enhanced learning performance in the presence of data heterogeneity FedMEM evaluates similarity among users to group them into clusters. We demonstrate that our approach furnishes high-performing personalized models compared to the state-of-the-art.
Contrastive Learning of Image Representations Guided by Spatial Relations
Authors: Logan Servant; Michaël Clément; Laurent Wendling; Camille Kurtz
Venue: WACV 2025 (main)

Abstract: The spatial information contained in images is of critical importance for many computer vision tasks. Current state-of-the-art approaches dealing with spatially-related tasks such as spatial relationship recognition are typically trained in a supervised manner with semantic information carried by annotations. However datasets containing spatial relations (such as VisualGenome and SpatialSense) contain many errors or ambiguities at the label level (e.g. polysemy of the relations use of different reference frames across relations) which might deteriorate the representation learning step. The representations and image embeddings obtained from this training setup carry poor spatial information as they are entangled with other modalities such as semantic information. To deal with this issue we introduce C-SIP (Contrastive Spatial-Image Pre-training) an approach aiming to learn better spatially-aware image representations more in agreement with human perception of a scene where spatial information is structuring. This training strategy focuses on the alignment of the embeddings of an image encoder and a spatial encoder optimized from the image content in a self-supervised manner. We showcase that training a model with spatial information at its core thanks to C-SIP allows for better spatially-aware image representations on three downstream tasks. These representations can be used in a zero-shot setting such as image retrieval or fine-tuned on semantic tasks such as visual question answering and provide better results compared with supervised counterparts. Our source code can be found at https://github.com/Logan-wilson/CSIP.
TAM-VT: Transformation-Aware Multi-Scale Video Transformer for Segmentation and Tracking
Authors: Raghav Goyal; Wan-Cyuan Fan; Mennatullah Siam; Leonid Sigal
Venue: WACV 2025 (main)

Abstract: Video Object Segmentation (VOS) has emerged as an increasingly important problem with availability of larger datasets and more complex and realistic settings which involve long videos with global motion (e.g. in egocentric settings) depicting small objects undergoing both rigid and non-rigid (including state) deformations. While a number of recent approaches have been explored for this task these data characteristics still present challenges. In this work we propose a novel clip-based DETR-style encoder-decoder architecture which focuses on systematically analyzing and addressing aforementioned challenges. Specifically we propose a novel transformation-aware loss that focuses learning on portions of the video where an object undergoes significant deformations - a form of "soft" hard examples mining. Further we propose a multiplicative time-coded memory beyond vanilla additive positional encoding which helps propagate context across long videos. Finally we incorporate these in our proposed holistic multi-scale video transformer for tracking via multi-scale memory matching and decoding to ensure sensitivity and accuracy for long videos and small objects. Our model enables on-line inference with long videos in a windowed fashion by breaking the video into clips and propagating context among them. We illustrate that short clip length and longer memory with learned time-coding are important design choices for improved performance. Collectively these technical contributions enable our model to achieve new state-of-the-art (SoTA) performance on two complex egocentric datasets - VISOR [13] and VOST [44] while achieving comparable to SoTA results on the conventional VOS benchmark DAVIS'17 [38]. Detailed ablations vali date our design choices and provide insights into the impor tance of parameter choices and impact on performance.
V-MIND: Building Versatile Monocular Indoor 3D Detector with Diverse 2D Annotations
Authors: Jin-Cheng Jhang; Tao Tu; Fu-En Wang; Ke Zhang; Min Sun; Cheng-Hao Kuo
Venue: WACV 2025 (main)

Abstract: The field of indoor monocular 3D object detection is gaining significant attention fueled by the increasing demand in VR/AR and robotic applications. However its advancement is impeded by the limited availability and diversity of 3D training data owing to the labor-intensive nature of 3D data collection and annotation processes. In this paper we present V-MIND (Versatile Monocular INdoor Detector) which enhances the performance of indoor 3D detectors across a diverse set of object classes by harnessing publicly available large-scale 2D datasets. By leveraging well-established monocular depth estimation techniques and camera intrinsic predictors we can generate 3D training data by converting large-scale 2D images into 3D point clouds and subsequently deriving pseudo 3D bounding boxes. To mitigate distance errors inherent in the converted point clouds we introduce a novel 3D self-calibration loss for refining the pseudo 3D bounding boxes during training. Additionally we propose a novel ambiguity loss to address the ambiguity that arises when introducing new classes from 2D datasets. Finally through joint training with existing 3D datasets and pseudo 3D bounding boxes derived from 2D datasets V-MIND achieves state-of-the-art object detection performance across a wide range of classes on the Omni3D indoor dataset.
Multi-Surrogate-Teacher Assistance for Representation Alignment in Fingerprint-Based Indoor Localization
Authors: Son Minh Nguyen; Linh Duy Tran; Duc Le; Paul Havinga
Venue: WACV 2025 (main)

Abstract: Despite remarkable progress in knowledge transfer across visual and textual domains extending these achievements to indoor localization particularly for learning transferable representations among Received Signal Strength (RSS) fingerprint datasets remains a challenge. This is due to inherent discrepancies among these RSS datasets largely including variations in building structure the input number and disposition of WiFi anchors. Accordingly specialized networks which were deprived of the ability to discern transferable representations readily incorporate environment-sensitive clues into the learning process hence limiting their potential when applied to specific RSS datasets. Initially we design an Expert Training phase which features multiple surrogate generative teachers all serving as a global adapter that homogenizes the input disparities among independent source RSS datasets while preserving their unique characteristics. In a subsequent Expert Distilling phase we continue introducing a triplet of underlying constraints that requires minimizing the differences in essential knowledge between the specialized network and surrogate teachers through refining its representation learning on the target dataset. This process implicitly fosters a representational alignment in such a way that is less sensitive to specific environmental dynamics. Extensive experiments conducted on three benchmark WiFi RSS fingerprint datasets underscore the effectiveness of the framework that significantly exerts the full potential of specialized networks in localization
FALCON: Fair Face Recognition via Local Optimal Feature Normalization
Authors: Rouqaiah Al-Refai; Philipp Hempel; Clara Biagi; Philipp Terhörst
Venue: WACV 2025 (main)

Abstract: Face recognition systems are widely used for identity verification in various fields. However recent studies have highlighted bias issues related to demographic and non-demographic attributes such as accessories haircolor ethnicity or gender. These biases lead to higher error rates for specific attribute subgroups. This is especially problematic in critical areas like forensics where these systems are deployed. Addressing this issue requires a solution that reduces bias without compromising accuracy. Existing methods focus on learning less biased face representations but they are often difficult to integrate into current systems or negatively impact overall recognition performance. This work introduces FALCON (Fair Adaptation Through Local Optimal Normalization) an effective method to increase fairness in face recognition systems. FALCON operates in an unsupervised manner addressing bias without requiring demographic labels and can be easily integrated as a post-processing step. It treats individuals with similar traits similarly reducing bias in face recognition by processing each image individually. The proposed method is rigorously tested across various face recognition models and datasets and compared with four other fairness post-processing methods. Results show that FALCON significantly enhances both fairness and accuracy. Unlike other methods it allows seamlessly adjusting the fairness-accuracy trade-off while effectively addressing bias.
Self-Aligning Depth-Regularized Radiance Fields for Asynchronous RGB-D Sequences
Authors: Yuxin Huang; Andong Yang; Yuantao Chen; Runyi Yang; Zhenxin Zhu; Chao Hou; Hao Zhao; Guyue Zhou
Venue: WACV 2025 (main)

Abstract: It has been shown that learning radiance fields with depth rendering and depth supervision can effectively promote the quality and convergence of view synthesis. However this paradigm requires input RGB-D sequences to be synchronized. In the UAV city modeling scenario there exists asynchrony between RGB images and depth images due to the different frequencies of the solid-state LiDAR and RGB sensors. To synthesize high-quality views in such a scenario we propose a novel time-pose function which is an implicit network that maps timestamps to SE(3) elements. To train this function we also design a joint optimization scheme to jointly learn the large-scale depth-regularized radiance fields and the time-pose function. Furthermore we propose a large synthetic dataset with diverse controlled mismatches and ground truth to evaluate this new problem setting systematically. The proposed approach has been evaluated on both datasets and in a real drone. To evaluate the impact of view density each algorithm was test on three different trajectories with different view densities. Compared to state-of-the-art baseline methods the proposed approach reduces reconstruction error by 35.26% in city modeling scenarios. Our code is available at github.com/saythe17/AsyncNeRF.