GenAI4ELab/papercli-papers huggingface dataset:
Utilizing Uncertainty in 2D Pose Detectors for Probabilistic 3D Human Mesh Recovery
Authors: Tom Wehrbein; Marco Rudolph; Bodo Rosenhahn; Bastian Wandt
Venue: WACV 2025 (main)

Abstract: Monocular 3D human pose and shape estimation is an inherently ill-posed problem due to depth ambiguities occlusions and truncations. Recent probabilistic approaches learn a distribution over plausible 3D human meshes by maximizing the likelihood of the ground-truth pose given an image. We show that this objective function alone is not sufficient to best capture the full distributions. Instead we propose to additionally supervise the learned distributions by minimizing the distance to distributions encoded in heatmaps of a 2D pose detector. Moreover we reveal that current methods often generate incorrect hypotheses for invisible joints which is not detected by the evaluation protocols. We demonstrate that person segmentation masks can be utilized during training to significantly decrease the number of invalid samples and introduce two metrics to evaluate it. Our normalizing flow-based approach predicts plausible 3D human mesh hypotheses that are consistent with the image evidence while maintaining high diversity for ambiguous body parts. Experiments on 3DPW and EMDB show that we outperform other state-of-the-art probabilistic methods. Code is available for research purposes at https://github.com/twehrbein/humr.
Comparative Evaluation of 3D Reconstruction Methods for Object Pose Estimation
Authors: Varun Burde; Assia Benbihi; Pavel Burget; Torsten Sattler
Venue: WACV 2025 (main)

Abstract: Current generalizable object pose estimators i.e. approaches that do not need to be trained per object rely on accurate 3D models. Predominantly CAD models are used which can be hard to obtain in practice. At the same time it is often possible to acquire images of an object. Naturally this leads to the question of whether 3D models reconstructed from images are sufficient to facilitate accurate object pose estimation. We aim to answer this question by proposing a novel benchmark for measuring the impact of 3D reconstruction quality on pose estimation accuracy. Our benchmark provides calibrated images suitable for reconstruction and registered with the test images of the YCB-V dataset for pose evaluation under the BOP benchmark format. Detailed experiments with multiple state-of-the-art 3D reconstruction and object pose estimation approaches show that the geometry produced by modern reconstruction methods is often sufficient for accurate pose estimation. Our experiments lead to interesting observations: (1) Standard metrics for measuring 3D reconstruction quality are not necessarily indicative of pose estimation accuracy which shows the need for dedicated benchmarks such as ours. (2) Classical non-learning-based approaches can perform on par with modern learning-based reconstruction techniques and can even offer a better reconstruction time-pose accuracy tradeoff. (3) There is still a sizable gap between performance with reconstructed and with CAD models. To foster research on closing this gap the benchmark is made available at https://github.com/VarunBurde/reconstruction_pose_benchmark.
Re-Identifying People in Video via Learned Temporal Attention and Multi-Modal Foundation Models
Authors: Cole Hill; Florence Yellin; Krishna Regmi; Dawei Du; Scott McCloskey
Venue: WACV 2025 (main)

Abstract: Biometric recognition from security camera video is a challenging problem when the individuals change clothes or when they are partly occluded. Others have recently demonstrated that CLIP's visual encoder performs well in this domain but existing methods fail to make use of the model's text encoder or temporal information available in video. In this paper we present VCLIP a method for person identification in videos captured in challenging poses and with changes to a person's clothing. Harnessing the power of pre-trained vision-language models we jointly train a temporal fusion network while fine-tuning the visual encoder. To leverage the cross-modal embedding space we use learned biometric pedestrian attribute features to further enhance our model's person re-identification (Re-ID) ability. We demonstrate significant performance improvements via experiments with the MEVID and CCVID datasets particularly in the more challenging clothes-changing conditions. In support of this and future methods that use textual attributes for Re-ID with multimodal models we release a dataset of annotated pedestrian attributes for the popular MEVID dataset.
GET-UP: GEomeTric-Aware Depth Estimation with Radar Points UPsampling
Authors: Huawei Sun; Zixu Wang; Hao Feng; Julius Ott; Lorenzo Servadei; Robert Wille
Venue: WACV 2025 (main)

Abstract: Depth estimation plays a pivotal role in autonomous driving facilitating a comprehensive understanding of the vehicle's 3D surroundings. Radar with its robustness to adverse weather conditions and capability to measure distances has drawn significant interest for radar-camera depth estimation. However existing algorithms process the inherently noisy and sparse radar data by projecting 3D points onto the image plane for pixel-level feature extraction overlooking the valuable geometric information contained within the radar point cloud. To address this gap we propose GET-UP leveraging attention-enhanced Graph Neural Networks (GNN) to exchange and aggregate both 2D and 3D information from radar data. This approach effectively enriches the feature representation by incorporating spatial relationships compared to traditional methods that rely only on 2D feature extraction. Furthermore we incorporate a point cloud upsampling task to densify the radar point cloud rectify point positions and derive additional 3D features under the guidance of lidar data. Finally we fuse radar and camera features during the decoding phase for depth estimation. We benchmark our proposed GET-UP on the nuScenes dataset achieving state-of-the-art performance with a 15.3% and 14.7% improvement in MAE and RMSE over the previously best-performing model. Code: https://github.com/ harborsarah/GET-UP
Detecting Origin Attribution for Text-to-Image Diffusion Models
Authors: Katherine Xu; Lingzhi Zhang; Jianbo Shi
Venue: WACV 2025 (main)

Abstract: Modern text-to-image (T2I) diffusion models can generate images with remarkable realism and creativity. These advancements have sparked research in fake image detection and attribution yet prior studies have not fully explored the practical and scientific dimensions of this task. In addition to attributing images to 12 state-of-the-art T2I generators we provide extensive analyses on what inference stage hyperparameters and image modifications are discernible. Our experiments reveal that initialization seeds are highly detectable along with other subtle variations in the image generation process to some extent. We further investigate what visual traces are leveraged in image attribution by perturbing high-frequency details and employing mid-level representations of image style and structure. Notably altering high-frequency information causes only slight reductions in accuracy and training an attributor on style representations outperforms training on RGB images. Our analyses underscore that fake images are detectable and attributable at various levels of visual granularity.
SALVE: A 3D Reconstruction Benchmark of Wounds from Consumer-Grade Videos
Authors: Remi Chierchia; Leo Lebrat; David Ahmedt-Aristizabal; Olivier Salvado; Clinton Fookes; Rodrigo Santa Cruz
Venue: WACV 2025 (main)

Abstract: Managing chronic wounds is a global challenge that can be alleviated by the adoption of automatic systems for clinical wound assessment from consumer-grade videos. While 2D image analysis approaches are insufficient for handling the 3D features of wounds existing approaches utilizing 3D reconstruction methods have not been thoroughly evaluated. To address this gap this paper presents a comprehensive study on 3D wound reconstruction from consumer-grade videos. Specifically we introduce the SALVE dataset comprising video recordings of realistic wound phantoms captured with different cameras. Using this dataset we assess the accuracy and precision of state-of-the-art methods for 3D reconstruction ranging from traditional photogrammetry pipelines to advanced neural rendering approaches. In our experiments we observe that photogrammetry approaches do not provide smooth surfaces suitable for precise clinical measurements of wounds. Neural rendering approaches show promise in addressing this issue advancing the use of this technology in wound care practices.
Hierarchical Light Transformer Ensembles for Multimodal Trajectory Forecasting
Authors: Adrien Lafage; Mathieu Barbier; Gianni Franchi; David Filliat
Venue: WACV 2025 (main)

Abstract: Accurate trajectory forecasting is crucial for the performance of various systems such as advanced driver-assistance systems and self-driving vehicles. These forecasts allow us to anticipate events that lead to collisions and therefore to mitigate them. Deep Neural Networks have excelled in motion forecasting but overconfidence and weak uncertainty quantification persist. Deep Ensembles address these concerns yet applying them to multimodal distributions remains challenging. In this paper we propose a novel approach named Hierarchical Light Transformer Ensembles (HLT-Ens) aimed at efficiently training an ensemble of Transformer architectures using a novel hierarchical loss function. HLT-Ens leverages grouped fully connected layers inspired by grouped convolution techniques to capture multimodal distributions effectively. We demonstrate that HLT-Ens achieves state-of-the-art performance levels through extensive experimentation offering a promising avenue for improving trajectory forecasting techniques. We make our code available at github.com/alafage/hlt-ens.
Compositional Segmentation of Cardiac Images Leveraging Metadata
Authors: Abbas Khan; Muhammad Asad; Martin Benning; Caroline Roney; Gregory Slabaugh
Venue: WACV 2025 (main)

Abstract: Cardiac image segmentation is essential for automated cardiac function assessment and monitoring of changes in cardiac structures over time. Inspired by coarse-to-fine approaches in image analysis we propose a novel multitask compositional segmentation approach that can simultaneously localize the heart in a cardiac image and perform part-based segmentation of different regions of interest. We demonstrate that this compositional approach achieves better results than direct segmentation of the anatomies. Further we propose a novel Cross-Modal Feature Integration (CMFI) module to leverage the metadata related to cardiac imaging collected during image acquisition. We perform experiments on two different modalities MRI and ultrasound using public datasets Multi-Disease Multi-View and Multi-Centre (M&Ms-2) and Multi-structure Ultrasound Segmentation (CAMUS) data to showcase the efficiency of the proposed compositional segmentation method and Cross-Modal Feature Integration module incorporating metadata within the proposed compositional segmentation network. The source code is available: https://github.com/kabbas570/CompSeg-MetaData.
Instructive3D: Editing Large Reconstruction Models with Text Instructions
Authors: Kunal Kathare; Ankit Dhiman; K Vikas Gowda; Siddharth Aravindan; Shubham Monga; Basavaraja Shanthappa Vandrotti; Lokesh R Boregowda
Venue: WACV 2025 (main)

Abstract: Transformer based methods have enabled users to create modify and comprehend text and image data. Recently proposed Large Reconstruction Models (LRMs) further extend this by providing the ability to generate high-quality 3D models with the help of a single object image. These models however lack the ability to manipulate or edit the finer details such as adding standard design patterns or changing the color and reflectance of the generated objects thus lacking fine-grained control that may be very helpful in domains such as augmented reality animation and gaming. Naively training LRMs for this purpose would require generating precisely edited images and 3D object pairs which is computationally expensive. In this paper we propose Instructive3D a novel LRM based model that integrates generation and fine-grained editing through user text prompts of 3D objects into a single model. We accomplish this by adding an adapter that performs a diffusion process conditioned on a text prompt specifying edits in the triplane latent space representation of 3D object models. Our method does not require the generation of edited 3D objects. Additionally Instructive3D allows us to perform geometrically consistent modifications as the edits done through user-defined text prompts are applied to the triplane latent representation thus enhancing the versatility and precision of 3D objects generated. We compare the objects generated by Instructive3D and a baseline that first generates the 3D object meshes using a standard LRM model and then edits these 3D objects using text prompts when images are provided from the Objaverse LVIS dataset. We find that Instructive3D produces qualitatively superior 3D objects with the properties specified by the edit prompts.
PVP: Polar Representation Boost for 3D Semantic Occupancy Prediction
Authors: Yujing Xue; Jiaxiang Liu; Jiawei Du; Joey Tianyi Zhou
Venue: WACV 2025 (main)

Abstract: Recently representations based on polar coordinates have exhibited promising characteristics for 3D perceptual tasks. In addition to Cartesian-based methods representing surrounding spaces through polar grids offers a compelling alternative in these tasks. This approach is advantageous for its ability to represent larger areas while preserving greater detail of nearby spaces. However polar-based methods are inherently challenged by the issue of feature distortion due to the non-uniform division inherent to polar representation. To harness the advantages of polar representation while addressing its challenges we propose Polar Voxel Occupancy Predictor (PVP) a novel 3D multi-modal occupancy predictor operating in polar coordinates. PVP mitigates the issues of feature distortion and misalignment across different modalities with the following two design elements: 1) Global Represent Propagation (GRP) module which incorporates global spatial information into the intermediate 3D volume taking into account the prior spatial structure. It then employs Global Decomposed Attention to accurately propagate features to their correct locations. 2) Plane Decomposed Convolution (PD-Conv) which simplifies 3D distortions in polar coordinates by replacing 3D convolution with a series of 2D convolutions. With these straightforward yet impactful modifications our PVP surpasses state-of-the-art works by significant margins--improving by 1.9% mIoU and 2.9% IoU over LiDAR-only methods and by 7.9% mIoU and 6.8% IoU over multimodal methods on the OpenOccupancy dataset.
TempA-VLP: Temporal-Aware Vision-Language Pretraining for Longitudinal Exploration in Chest X-ray Image
Authors: Zhuoyi Yang; Liyue Shen
Venue: WACV 2025 (main)

Abstract: Longitudinal medical image processing is a significant task to understand the dynamic changes of disease by taking and comparing image series over time providing insights into how conditions evolve and enabling more accurate diagnosis and treatment planning. While recent advancements in biomedical Vision-Language Pre-training (VLP) have enabled label-efficient representation learning with paired medical images and reports existing methods primarily pair a single image with the corresponding textual report limiting their ability to capture temporal relationships. To address this limitation it is essential to learn temporal-aware cross-modal representations from sequential medical images and text reports that highlight the temporal changes occurring between examinations. Specifically we introduce TempA-VLP a temporal-aware vision language pre-training framework with a cross-exam encoder to integrate the information from both prior and current examinations. This approach enables the model to capture dynamic representations that reflect disease progression over time which allows us to (i) achieve state-of-the-art performance in disease progression classification (ii) localize dynamic progression regions across consecutive examinations as demonstrated in our new task dynamic phrase grounding on the Chest-Imagenome Gold dataset and (iii) highlight progression localized regions often relevant to lesion areas which in turn improves disease classification tasks on a single image.
SpotDiffusion: A Fast Approach for Seamless Panorama Generation Over Time
Authors: Stanislav Frolov; Brian B. Moser; Andreas Dengel
Venue: WACV 2025 (main)

Abstract: Generating high-resolution images with generative models has recently been made widely accessible by leveraging diffusion models pre-trained on large-scale datasets. Various techniques such as MultiDiffusion and SyncDiffusion have further pushed image generation beyond training resolutions i.e. from square images to panorama by merging multiple overlapping diffusion paths or employing gradient descent to maintain perceptual coherence. However these methods suffer from significant computational inefficiencies due to generating and averaging numerous predictions which is required in practice to produce high-quality and seamless images. This work addresses this limitation and presents a novel approach that eliminates the need to generate and average numerous overlapping denoising predictions. Our method shifts non-overlapping denoising windows over time ensuring that seams in one timestep are corrected in the next. This results in coherent high-resolution images with fewer overall steps. We demonstrate the effectiveness of our approach through qualitative and quantitative evaluations comparing it with MultiDiffusion SyncDiffusion and StitchDiffusion. Our method offers several key benefits including improved computational efficiency and faster inference times while producing comparable or better image quality.
OccLoff: Learning Optimized Feature Fusion for 3D Occupancy Prediction
Authors: Ji Zhang; Yiran Ding; Zixin Liu
Venue: WACV 2025 (main)

Abstract: 3D semantic occupancy prediction is crucial for finely representing the surrounding environment which is essential for ensuring the safety in autonomous driving. Existing fusion-based occupancy methods typically involve performing a 2D-to-3D view transformation on image features followed by computationally intensive 3D operations to fuse these with LiDAR features leading to high computational costs and reduced accuracy. Moreover current research on occupancy prediction predominantly focuses on designing specific network architectures often tailored to particular models with limited attention given to the more fundamental aspect of semantic feature learning. This gap hinders the development of more transferable methods that could enhance the performance of various occupancy models. To address these challenges we propose OccLoff a framework that Learns to Optimize Feature Fusion for 3D occupancy prediction. Specifically we introduce a sparse fusion encoder with entropy masks that directly fuses 3D and 2D features improving model accuracy while reducing computational overhead. Additionally we propose a transferable proxy-based loss function and an adaptive hard sample weighting algorithm which enhance the performance of several state-of-the-art methods. Extensive evaluations on the nuScenes and SemanticKITTI benchmarks demonstrate the superiority of our framework and ablation studies confirm the effectiveness of each proposed module.
Beyond Spatial Explanations: Explainable Face Recognition in the Frequency Domain
Authors: Marco Huber; Naser Damer
Venue: WACV 2025 (main)

Abstract: The need for more transparent face recognition (FR) along with other visual-based decision-making systems has recently attracted more attention in research society and industry. The reasons why two face images are matched or not matched by a deep learning-based face recognition system are not obvious due to the high number of parameters and the complexity of the models. However it is important for users operators and developers to ensure trust and accountability of the system and to analyze drawbacks such as biased behavior. While many previous works use spatial semantic maps to highlight the regions that have a significant influence on the decision of the face recognition system frequency components which are also considered by CNNs are neglected. In this work we take a step forward and investigate explainable face recognition in the unexplored frequency domain. This makes this work the first to propose explainability of verification-based decisions in the frequency domain thus explaining the relative influence of the frequency components of each input toward the obtained outcome. To achieve this we manipulate face images in the spatial frequency domain and investigate the impact on verification outcomes. In extensive quantitative experiments along with investigating two special scenarios cases cross-resolution FR and morphing attacks (the latter in supplementary material) we observe the applicability of our proposed frequency-based explanations.
A Conflict-Guided Evidential Multimodal Fusion for Semantic Segmentation
Authors: Lucas Deregnaucourt; Hind Laghmara; Alexis Lechervy; Samia Ainouz
Venue: WACV 2025 (main)

Abstract: This article presents a novel and robust approach to semantic segmentation based on the fusion of different image modalities (conventional and non-conventional images). The robustness of fusion methods and their ability to tolerate sensor failures are crucial challenges for their deployment in real-world environments. It is essential to develop unique fusion models that can operate even in the absence of certain modalities during inference. However current fusion methods have a strong dependence on the RGB branch resulting in significant performance losses in case of its unavailability. To address this issue we propose ECoLaF (Evidential Conflict-guided Late Fusion) a late fusion method based on Dempster-Shafer theory. This method adaptively reduces the output of each modality according to their conflicts before fusing them. Experimental results show that our approach outperforms state-of-the-art methods in terms of robustness on the MCubeS and DeLiVER datasets especially when the RGB sensor is not operational. This study offers new perspectives for improving the robustness of semantic segmentation in multimodal contexts. Code is available at https://github.com/deregnaucourtlucas/ECoLaF.
Elucidating Optimal Reward-Diversity Tradeoffs in Text-to-Image Diffusion Models
Authors: Rohit Jena; Ali Taghibakhshi; Sahil Jain; Gerald Shen; Nima Tajbakhsh; Arash Vahdat
Venue: WACV 2025 (main)

Abstract: Text-to-image (T2I) diffusion models have become prominent tools for generating high-fidelity images from text prompts. However when trained on unfiltered internet data these models can produce unsafe incorrect or stylistically undesirable images that are not aligned with human preferences. To address this recent approaches have incorporated human preference datasets to fine-tune T2I models or to optimize reward functions that capture these preferences. Although effective these methods are vulnerable to reward hacking where the model overfits to the reward function leading to a loss of diversity in the generated images. In this paper we prove the inevitability of reward hacking and study natural regularization techniques like KL divergence and LoRA scaling and their limitations for diffusion models. We also introduce Annealed Importance Guidance (AIG) an inference-time regularization inspired by Annealed Importance Sampling which retains the diversity of the base model while achieving Pareto-Optimal reward-diversity tradeoffs. Our experiments demonstrate the benefits of AIG for Stable Diffusion models striking the optimal balance between reward optimization and image diversity. Furthermore a user study confirms that AIG improves diversity and quality of generated images across different model architectures and reward functions.