GenAI4ELab/papercli-papers huggingface dataset:
CT to PET Translation: A Large-Scale Dataset and Domain-Knowledge-Guided Diffusion Approach
Authors: Dac Thai Nguyen; Trung Thanh Nguyen; Huu Tien Nguyen; Thanh Trung Nguyen; Huy Hieu Pham; Thanh Hung Nguyen; Thao Nguyen Truong; Phi Le Nguyen
Venue: WACV 2025 (main)

Abstract: Positron Emission Tomography (PET) and Computed Tomography (CT) are essential for diagnosing staging and monitoring various diseases particularly cancer. Despite their importance the use of PET/CT systems is limited by the necessity for radioactive materials the scarcity of PET scanners and the high cost associated with PET imaging. In contrast CT scanners are more widely available and significantly less expensive. In response to these challenges our study addresses the issue of generating PET images from CT images aiming to reduce both the medical examination cost and the associated health risks for patients. Our contributions are twofold: First we introduce a conditional diffusion model named CPDM which to our knowledge is one of the initial attempts to employ a diffusion model for translating from CT to PET images. Second we provide the largest CT-PET dataset to date comprising 2028628 paired CT-PET images which facilitates the training and evaluation of CT-to-PET translation models. For the CPDM model we incorporate domain knowledge to develop two conditional maps: the Attention map and the Attenuation map. The former helps the diffusion process focus on areas of interest while the latter improves PET data correction and ensures accurate diagnostic information. Experimental evaluations across various benchmarks demonstrate that CPDM surpasses existing methods in generating high-quality PET images in terms of multiple metrics. The source code and data samples are available at https://github.com/thanhhff/CPDM.
GTA-HDR: A Large-Scale Synthetic Dataset for HDR Image Reconstruction
Authors: Hrishav Bakul Barua; Kalin Stefanov; KokSheik Wong; Abhinav Dhall; Ganesh Krishnasamy
Venue: WACV 2025 (main)

Abstract: High Dynamic Range (HDR) content (i.e. images and videos) has a broad range of applications. However capturing HDR content from real-world scenes is expensive and time-consuming. Therefore the challenging task of reconstructing visually accurate HDR images from their Low Dynamic Range (LDR) counterparts is gaining attention in the vision research community. A major challenge is the lack of datasets which capture diverse scene conditions (e.g. lighting weather locations) and various image features (e.g. color contrast saturation). To address this gap we introduce GTA-HDR a large-scale synthetic dataset of photo-realistic HDR images sampled from the GTA-V video game. We perform thorough evaluation of the proposed dataset which enables significant qualitative and quantitative improvements of the state-of-the-art HDR image reconstruction methods. Furthermore we demonstrate the effectiveness of the proposed dataset and its impact on additional computer vision tasks including 3D human pose estimation human body part segmentation and holistic scene segmentation. The dataset data collection pipeline and evaluation code are available at: https://github.com/HrishavBakulBarua/GTA-HDR.
HOPE: A Memory-Based and Composition-Aware Framework for Zero-Shot Learning with Hopfield Network and Soft Mixture of Experts
Authors: Do Huu Dat; Po-Yuan Mao; Tien Hoang Nguyen; Wray Buntine; Mohammed Bennamoun
Venue: WACV 2025 (main)

Abstract: Compositional Zero-Shot Learning (CZSL) has emerged as an essential paradigm in machine learning aiming to overcome the constraints of traditional zero-shot learning by incorporating compositional thinking into its methodology. Conventional zero-shot learning has difficulty managing unfamiliar combinations of seen and unseen classes because it depends on pre-defined class embeddings. In contrast Compositional Zero-Shot Learning leverages the inherent hierarchies and structural connections among classes creating new class representations by combining attributes components or other semantic elements. In our paper we propose a novel framework that for the first time combines the Modern Hopfield Network with a Mixture of Experts (HOPE) to classify the compositions of previously unseen objects. Specifically the Modern Hopfield Network creates a memory that stores label prototypes and identifies relevant labels for a given input image. Subsequently the Mixture of Expert models integrates the image with the appropriate prototype to produce the final composition classification. Our approach achieves SOTA performance on several benchmarks including MIT-States and UT-Zappos. We also examine how each component contributes to improved generalization.
IRIS-VIS: A New Dataset for Visibility Estimation in an Industrial Environment
Authors: Flavien Armangeon; Thibaud Ehret; Enric Meinhardt-Llopis; Rafael Grompone von Gioi; Guillaume Thibault; Marc Petit; Gabriele Facciolo
Venue: WACV 2025 (main)

Abstract: Point cloud visibility estimation is fundamental as it is useful for many computer vision applications including surface reconstruction 3D segmentation from paired images and point densification. Previous works showed outstanding results on simple object and outdoor datasets. However unlike the previously studied scenes the most challenging environments are those providing a high amount of object points in the same direction typically in complex indoor scenes. In this kind of environments due to the lack of real data ground truth quantitative analysis are either missing or based on simulated data. In this work we present IRIS-VIS (Industrial Room In Saclay - VISibility) a new dataset for point visibility estimation in an indoor environment. It is a high complexity scene due to the large variety in the shape size and orientation of the objects. To our knowledge this is the first dataset on real indoor data providing a dense LiDAR station-based point cloud along with a well-fitted CAD model. The latter is useful to compute automatically quickly and accurately the visibility from any given viewpoint enabling evaluations under infinite conditions. We propose new metrics for the visibility estimation task and evaluate state-of-the-art methods in both sparse and dense conditions with the proposed dataset.
PivotAlign: Improve Semi-Supervised Learning by Learning Intra-Class Heterogeneity and Aligning with Pivots
Authors: Lingjie Yi; Tao Sun; Yikai Zhang; Songzhu Zheng; Weimin Lyu; Haibin Ling; Chao Chen
Venue: WACV 2025 (main)

Abstract: Self-supervised learning plays an important role in current state-of-the-art semi-supervised learning (SSL) methods. These methods learn inter-class heterogeneity among data and generate pseudo-labels based on class level representations. However they often neglect intra-class heterogeneity resulting in the under-exploitation of finer-grained semantic relationships within classes. To address this limitation we introduce PivotAlign a novel SSL approach that aims to 1) learn hierarchical representations to detect both inter-class and intra-class semantic relationships and 2) refine pseudo-labels based on learned representations with a class-debiasing strategy. Specifically we first learn a set of pivots as sub-prototypes of classes. We then train representations so that features align with the assigned pivot and are hierarchically grouped based on both inter-class and intra-class heterogeneity. This allows us to capture both inter-class and intra-class semantic relationships among data and leverage them to better assign and refine pseudo-labels. Additionally since SSL methods are prone to bias toward classes that are easier to learn we further re-balance class predictions to alleviate this class bias. We demonstrate the effectiveness of PivotAlign on various SSL benchmarks where PivotAlign achieves state-of-the-art performances. The source code will be released upon publication of the work.
SpectFormer: Frequency and Attention is What You Need in a Vision Transformer
Authors: Badri N. Patro; Vinay P. Namboodiri; Vijay S. Agneeswaran
Venue: WACV 2025 (main)

Abstract: Vision transformers have been applied successfully for image recognition tasks. There have been either multiheaded self-attention based (ViT [12] DeIT [54]) similar to the original work in textual models or more recently based on spectral layers (Fnet [29] GFNet [46] AFNO [15]). We hypothesize that spectral layers capture high-frequency information such as lines and edges while attention layers capture token interactions. We investigate this hypothesis through this work and observe that indeed mixing spectral and multi-headed attention layers provides a better transformer architecture. We thus propose the novel Spectformer architecture for vision transformers that has initial spectral and deeper multi-headed attention layers. We believe that the resulting representation allows the transformer to capture the feature representation appropriately and it yields improved performance over other transformer representations. For instance it improves the top-1 accuracy by 2% on ImageNet compared to both GFNet-H and LiT. SpectFormer-H-S reaches 84.25% top-1 accuracy on ImageNet-1K (state of the art for small version). Further Spectformer-H-L achieves 85.7% which is the state of the art for the comparable base version of the transformers. We further validated the SpectFormer performance in other scenarios such as transfer learning on standard datasets such as CIFAR-10 CIFAR-100 Oxford- IIIT-flower and Standford Car datasets. We then investigate its use in downstream tasks such as object detection and instance segmentation on the MS-COCO dataset and observe that Spectformer shows consistent performance that is comparable to the best backbones and can be further optimized and improved. The source code is available on this website https://github.com/badripatro/SpectFormers.
EgoCast: Forecasting Egocentric Human Pose in the Wild
Authors: Maria Escobar; Juanita Puentes; Cristhian Forigua; Jordi Pont-Tuset; Kevis-Kokitsi Maninis; Pablo Arbelaez
Venue: WACV 2025 (main)

Abstract: Accurately estimating and forecasting human body pose is important for enhancing the user's sense of immersion in Augmented Reality. Addressing this need our paper introduces EgoCast a bimodal method for 3D human pose forecasting using egocentric videos and proprioceptive data. We study the task of human pose forecasting in a realistic setting extending the boundaries of temporal forecasting in dynamic scenes and building on the current framework for current pose estimation in the wild. We introduce a current-frame estimation module that generates pseudo-groundtruth poses for inference eliminating the need for past groundtruth poses typically required by current methods during forecasting. Our experimental results on the recent Ego-Exo4D and Aria Digital Twin datasets validate EgoCast for real-life motion estimation. On the Ego-Exo4D Body Pose 2024 Challenge our method significantly outperforms the state-of-the-art approaches laying the groundwork for future research in human pose estimation and forecasting in unscripted activities with egocentric inputs.
ReEdit: Multimodal Exemplar-Based Image Editing
Authors: Ashutosh Srivastava; Tarun Ram Menta; Abhinav Java; Avadhoot Gorakh Jadhav; Silky Singh; Surgan Jandial; Balaji Krishnamurthy
Venue: WACV 2025 (main)

Abstract: Modern Text-to-Image (T2I) Diffusion models have revolutionized image editing by enabling the generation of high-quality photorealistic images. While the de-facto method for performing edits with T2I models is through text instructions this approach is non-trivial due to the complex many-to-many mapping between natural language and images. In this work we address exemplar-based image editing - the task of transferring an edit from an exemplar pair to a content image(s). We propose ReEdit a modular and efficient end-to-end framework that captures edits in both text and image modalities while ensuring the fidelity of the edited image. We validate the effectiveness of ReEdit through extensive comparisons with state-of-the-art baselines and sensitivity analyses of key design choices. Our results demonstrate that ReEdit consistently outperforms contemporary approaches both qualitatively and quantitatively. Additionally ReEdit boasts high practical applicability as it does not require any task-specific optimization and is 4 times faster than the existing state-of-the-art. The code and data for our work is available at https://reedit-diffusion.github.io/.
PGRID: Power Grid Reconstruction in Informal Developments using High-Resolution Aerial Imagery
Authors: Simone Fobi Nsutezo; Amrita Gupta; Duncan Kebut; Seema Iyer; Luana Marotti; Rahul Dodhia; Juan M. Lavista Ferres; Anthony Ortiz
Venue: WACV 2025 (main)

Abstract: As of 2023 a record 117 million people have been displaced worldwide more than double the number from a decade ago [22]. Of these 32 million are refugees under the UNHCR mandate with 8.7 million residing in refugee camps. A critical issue faced by these populations is the lack of access to electricity with 80% of the 8.7 million refugees and displaced persons in camps globally relying on traditional biomass for cooking and lacking reliable power for essential tasks such as cooking and charging phones. Often the burden of collecting firewood falls on women and children who frequently travel up to 20 kilometers into dangerous areas increasing their vulnerability. [7] Electricity access could significantly alleviate these challenges but a major obstacle is the lack of accurate power grid infrastructure maps particularly in resource-constrained environments like refugee camps needed for energy access planning. Existing power grid maps are often outdated incomplete or dependent on costly complex technologies limiting their practicality. To address this issue PGRID is a novel application-based approach which utilizes high-resolution aerial imagery to detect electrical poles and segment electrical lines creating precise power grid maps. PGRID was tested in the Turkana region of Kenya specifically the Kakuma and Kalobeyei Camps covering 84 km2 and housing over 200000 residents. Our findings show that PGRID delivers high-fidelity power grid maps especially in unplanned settlements with F1-scores of 0.71 and 0.82 for pole detection and line segmentation respectively. This study highlights a practical application for leveraging open data and limited labels to improve power grid mapping in unplanned settlements where the growing number of displaced persons urgently need sustainable energy infrastructure solutions.
Enhancing Predictive Imaging Biomarker Discovery through Treatment Effect Analysis
Authors: Shuhan Xiao; Lukas Klein; Jens Petersen; Philipp Vollmuth; Paul F. Jaeger; Klaus H. Maier-Hein
Venue: WACV 2025 (main)

Abstract: Identifying predictive covariates which forecast individual treatment effectiveness is crucial for decision-making across different disciplines such as personalized medicine. These covariates referred to as biomarkers are extracted from pre-treatment data often within randomized controlled trials and should be distinguished from prognostic biomarkers which are independent of treatment assignment. Our study focuses on discovering predictive imaging biomarkers specific image features by leveraging pre-treatment images to uncover new causal relationships. Unlike labor-intensive approaches relying on handcrafted features prone to bias we present a novel task of directly learning predictive features from images. We propose an evaluation protocol to assess a model's ability to identify predictive imaging biomarkers and differentiate them from purely prognostic ones by employing statistical testing and a comprehensive analysis of image feature attribution. We explore the suitability of deep learning models originally developed for estimating the conditional average treatment effect (CATE) for this task which have been assessed primarily for their precision of CATE estimation while overlooking the evaluation of imaging biomarker discovery. Our proof-of-concept analysis demonstrates the feasibility and potential of our approach in discovering and validating predictive imaging biomarkers from synthetic outcomes and real-world image datasets. Our code is available at https://github.com/MIC-DKFZ/predictive_image_biomarker_analysis.
DeepCA: Deep Learning-Based 3D Coronary Artery Tree Reconstruction from Two 2D Non-Simultaneous X-ray Angiography Projections
Authors: Yiying Wang; Abhirup Banerjee; Robin P. Choudhury; Vicente Grau
Venue: WACV 2025 (main)

Abstract: Cardiovascular diseases (CVDs) are the most common cause of death worldwide. Invasive x-ray coronary angiography (ICA) is one of the most important imaging modalities for the diagnosis of CVDs. ICA typically acquires only two 2D projections which makes the 3D geometry of coronary vessels difficult to interpret thus requiring 3D coronary artery tree reconstruction from two projections. State-of-the-art approaches require significant manual interactions and cannot correct the non-rigid cardiac and respiratory motions between non-simultaneous projections. In this study we propose a novel deep learning pipeline named DeepCA. We leverage the Wasserstein conditional generative adversarial network with gradient penalty latent convolutional transformer layers and a dynamic snake convolutional critic to implicitly compensate for the non-rigid motion and provide 3D coronary artery tree reconstruction. Through simulating projections from coronary computed tomography angiography (CCTA) we achieve the generalisation of 3D coronary tree reconstruction on real non-simultaneous ICA projections. We incorporate an application-specific evaluation metric to validate our proposed model on both a CCTA dataset and a real ICA dataset together with Chamfer l_2 distance. The results demonstrate promising performance of our DeepCA model in vessel topology preservation recovery of missing features and generalisation ability to real ICA data. To the best of our knowledge this is the first study that leverages deep learning to achieve 3D coronary tree reconstruction from two real non-simultaneous x-ray angiographic projections. The implementation of this work is available at: https://github.com/WangStephen/DeepCA.
Precise Integral in NeRFs: Overcoming the Approximation Errors of Numerical Quadrature
Authors: Boyuan Zhang; Zhenliang He; Meina Kan; Shiguang Shan
Venue: WACV 2025 (main)

Abstract: Neural Radiance Fields (NeRFs) use neural networks to translate spatial coordinates to corresponding volume density and directional radiance enabling realistic novel view synthesis through volume rendering. Rendering new viewpoints involves computing volume rendering integrals along rays usually approximated by numerical quadrature because of lacking closed-form solutions. In this paper utilizing Taylor expansion we demonstrate that numerical quadrature causes inevitable approximation error in NeRF integrals due to ignoring the parameter associated with the Lagrange remainder. To mitigate the approximation error we propose a novel neural field with segment representation as input to implicitly model the remainder parameter. In theory our proposed method is proven to possess the potential to achieve fully precise rendering integral as demonstrated by comprehensive experiments on several commonly used datasets with state-of-the-art results.
FitDiff: Robust Monocular 3D Facial Shape and Reflectance Estimation using Diffusion Models
Authors: Stathis Galanakis; Alexandros Lattas; Stylianos Moschoglou; Stefanos Zafeiriou
Venue: WACV 2025 (main)

Abstract: The remarkable progress in 3D face reconstruction has resulted in high-detail and photorealistic facial representations. Recently Diffusion Models have revolutionized the capabilities of generative methods by surpassing the performance of GANs. In this work we present FitDiff a diffusion-based 3D facial avatar generative model. Leveraging diffusion principles our model accurately generates relightable facial avatars utilizing an identity embedding extracted from an "in-the-wild" 2D facial image. The introduced multi-modal diffusion model concurrently outputs facial reflectance maps (diffuse and specular albedo and normals) and shapes showcasing great generalization capabilities. It is solely trained on an annotated subset of a public facial dataset paired with 3D reconstructions. We revisit the typical 3D facial fitting approach by guiding a reverse diffusion process using perceptual and face recognition losses. Being the first LDM conditioned on face recognition embeddings FitDiff reconstructs relightable human avatars that can be used as-is in common rendering engines starting only from an unconstrained facial image and achieving state-of-the-art performance.
Differential Privacy Mechanisms in Neural Tangent Kernel Regression
Authors: Jiuxiang Gu; Yingyu Liang; Zhizhou Sha; Zhenmei Shi; Zhao Song
Venue: WACV 2025 (main)

Abstract: Training data privacy is a fundamental problem in modern Artificial Intelligence (AI) applications such as face recognition recommendation systems language generation and many others as it may contain sensitive user information related to legal issues. To fundamentally understand how privacy mechanisms work in AI applications we study differential privacy (DP) in the Neural Tangent Kernel (NTK) regression setting where DP is one of the most powerful tools for measuring privacy under statistical learning and NTK is one of the most popular analysis frameworks for studying the learning mechanisms of deep neural networks. In our work we can show provable guarantees for both differential privacy and test accuracy of our NTK regression. Furthermore we conduct experiments on the basic image classification dataset CIFAR10 to demonstrate that NTK regression can preserve good accuracy under a modest privacy budget supporting the validity of our analysis. To our knowledge this is the first work to provide a DP guarantee for NTK regression.
DiL: An Explainable and Practical Metric for Abnormal Uncertainty in Object Detection
Authors: Amit Giloni; Omer Hofman; Ikuya Morikawa; Toshiya Shimizu; Yuval Elovici; Asaf Shabtai
Venue: WACV 2025 (main)

Abstract: Although object detection models are widely used their predictive performance has been shown to deteriorate when faced with abnormal scenes. Such abnormalities can occur naturally (by partially occluded or out-of-distribution objects) or deliberately (in the case of an adversarial attack). Existing uncertainty quantification methods such as object detection evaluation metrics and label-uncertainty quantification techniques do not consider the abnormalities' effect on the model's internal decision-making process. Furthermore practical methods that consider the effects of abnormalities (such as abnormality detection and mitigation) are designed to deal with one type of abnormality. We present distinctive localization (DiL) an unsupervised practical and explainable metric that quantitatively interprets any type of abnormality and can be leveraged for preventive purposes. By utilizing XAI techniques (saliency maps) DiL maps the objectness of a given scene and captures the model's inner uncertainty regarding the identified (and missed) objects. DiL was evaluated across nine use cases including partially occluded and out-of-distribution objects as well as adversarial patches in both physical and digital spaces on benchmark datasets and our newly E-PO dataset (generated with DALL-E 2). Our results show that DiL: i) successfully interprets and quantifies an abnormality's effect on the model's decision-making process regardless of the abnormality type; and ii) can be leveraged to detect and mitigate this effect.