GenAI4ELab/papercli-papers huggingface dataset:
Deep Geometric Moments Promote Shape Consistency in Text-to-3D Generation
Authors: Utkarsh Nath; Rajeev Goel; Eun Som Jeon; Changhoon Kim; Kyle Min; Yezhou Yang; Yingzhen Yang; Pavan Turaga
Venue: WACV 2025 (main)

Abstract: To address the data scarcity associated with 3D assets 2D-lifting techniques such as Score Distillation Sampling (SDS) have become a widely adopted practice in text-to-3D generation pipelines. However the diffusion models used in these techniques are prone to viewpoint bias and thus lead to geometric inconsistencies such as the Janus problem. To counter this we introduce MT3D a text-to-3D generative model that leverages a high-fidelity 3D object to overcome viewpoint bias and explicitly infuse geometric understanding into the generation pipeline. Firstly we employ depth maps derived from a high-quality 3D model as control signals to guarantee that the generated 2D images preserve the fundamental shape and structure thereby reducing the inherent viewpoint bias. Next we utilize deep geometric moments to ensure geometric consistency in the 3D representation explicitly. By incorporating geometric details from a 3D asset MT3D enables the creation of diverse and geometrically consistent objects thereby improving the quality and usability of our 3D representations. Project page and code: https://moment-3d.github.io/
Dropout the High-Rate Downsampling: A Novel Design Paradigm for UHD Image Restoration
Authors: Chen Wu; Ling Wang; Long Peng; Dianjie Lu; Zhuoran Zheng
Venue: WACV 2025 (main)

Abstract: With the popularization of high-end mobile devices Ultra-high-definition (UHD) images have become ubiquitous in our lives. The restoration of UHD images is a highly challenging problem due to the exaggerated pixel count which often leads to memory overflow during processing. Existing methods either downsample UHD images at a high rate before processing or split them into multiple patches for separate processing. However high-rate downsampling leads to significant information loss while patch-based approaches inevitably introduce boundary artifacts. In this paper we propose a novel design paradigm to solve the UHD image restoration problem called D2Net. D2Net enables direct full-resolution inference on UHD images without the need for high-rate downsampling or dividing the images into several patches. Specifically we ingeniously utilize the characteristics of the frequency domain to establish long-range dependencies of features. Taking into account the richer local patterns in UHD images we also design a multi-scale convolutional group to capture local features. Additionally during the decoding stage we dynamically incorporate features from the encoding stage to reduce the flow of irrelevant information. Extensive experiments on three UHD image restoration tasks including low-light image enhancement image dehazing and image deblurring show that our model achieves better quantitative and qualitative results than state-of-the-art methods.
Multimodal Fusion Learning with Dual Attention for Medical Imaging
Authors: Joy Dhar; Nayyar Zaidi; Maryam Haghighat; Sudipta Roy; Puneet Goyal; Azadeh Alavi; Vikas Kumar
Venue: WACV 2025 (main)

Abstract: Multimodal fusion learning has shown significant promise in classifying various diseases such as skin cancer and brain tumors. However existing methods face three key limitations. First they often lack generalizability to other diagnosis tasks due to their focus on a particular disease. Second they do not fully leverage multiple health records from diverse modalities to learn robust complementary information. And finally they typically rely on a single attention mechanism missing the benefits of multiple attention strategies within and across various modalities. To address these issues this paper proposes a dual robust information fusion attention mechanism (DRIFA) that leverages two attention modules - i.e. multi-branch fusion attention module and the multimodal information fusion attention module. DRIFA can be integrated with any deep neural network forming a multi modal fusion learning framework denoted as DRIFA-Net. We show that the multi-branch fusion attention of DRIFA learns enhanced representations for each modality such as dermoscopy pap smear MRI and CT-scan whereas multimodal information fusion attention module learns more refined multimodal shared representations - improving the network's generalization across multiple tasks and enhancing overall performance. Additionally to estimate the uncertainty of DRIFA-Net predictions we have employed an ensemble Monte Carlo dropout strategy. Extensive experiments on five publicly available datasets with diverse modalities demonstrate that our approach consistently outperforms state-of-the-art methods. The code is available at https://github.com/misti1203/DRIFA-Net.
Sign Language Recognition: A Large-Scale Multi-View Dataset and Comprehensive Evaluation
Authors: Nguyen Son Dinh; Tuan Dung Nguyen; Duc Tri Tran; Nguyen Dang Huy Pham; Thuan Hieu Tran; Ngoc Anh Tong; Quang Huy Hoang; Phi Le Nguyen
Venue: WACV 2025 (main)

Abstract: Vision-based sign language recognition is an extensively researched problem aimed at advancing communication between deaf and hearing individuals. Numerous Sign Language Recognition (SLR) datasets have been introduced to promote research in this field spanning multiple languages vocabulary sizes and signers. However most existing popular datasets focus predominantly on the frontal view of signers neglecting visual information from other perspectives. In practice many sign languages contain words that have similar hand movements and expressions making it challenging to differentiate between them from a single frontal view. Although a few studies have proposed sign language datasets using multi-view data these datasets remain limited in vocabulary size and scale hindering their generalizability and practicality. To address this issue we introduce a new large-scale multi-view sign language recognition dataset spanning 1000 glosses and 30 signers resulting in over 84000 multi-view videos. To the best of our knowledge this is the first multi-view sign language recognition dataset of this scale. In conjunction with offering a comprehensive dataset we perform extensive experiments to assess the performance of state-of-the-art Sign Language Recognition models utilizing on our dataset. The findings indicate that utilizing multi-view data substantially enhances model accuracy across all models with a maximum performance improvement of up to 19.75% compared to models trained on single-view data. Our dataset and baseline models are publicly accessible on GitHub.
Enhancing Skin Disease Diagnosis: Interpretable Visual Concept Discovery with SAM
Authors: Xin Hu; Janet Wang; Jihun Hamm; Rie R Yotsu; Zhengming Ding
Venue: WACV 2025 (main)

Abstract: Current AI-assisted skin image diagnosis has achieved dermatologist-level performance in classifying skin cancer driven by rapid advancements in deep learning architectures. However unlike traditional vision tasks skin images in general present unique challenges due to the limited availability of well-annotated datasets complex variations in conditions and the necessity for detailed interpretations to ensure patient safety. Previous segmentation methods have sought to reduce image noise and enhance diagnostic performance but these techniques require fine-grained pixel-level ground truth masks for training. In contrast with the rise of foundation models the Segment Anything Model (SAM) has been introduced to facilitate promptable segmentation enabling the automation of the segmentation process with simple yet effective prompts. Efforts applying SAM predominantly focus on dermatoscopy images which present more easily identifiable lesion boundaries than clinical photos taken with smartphones. This limitation constrains the practicality of these approaches to real-world applications. To overcome the challenges posed by noisy clinical photos acquired via non-standardized protocols and to improve diagnostic accessibility we propose a novel Cross-Attentive Fusion framework for interpretable skin lesion diagnosis. Our method leverages SAM to generate visual concepts for skin diseases using prompts integrating local visual concepts with global image features to enhance model performance. Extensive evaluation on two skin disease datasets demonstrates our proposed method's effectiveness on lesion diagnosis and interpretability.
uLayout: Unified Room Layout Estimation for Perspective and Panoramic Images
Authors: Jonathan Lee; Bolivar E Solarte; Chin-Hsuan Wu; Jin-Cheng Jhang; Fu-En Wang; Yi-Hsuan Tsai; Min Sun
Venue: WACV 2025 (main)

Abstract: We present uLayout a unified model for estimating room layout geometries from both perspective and panoramic images whereas traditional solutions require different model designs for each image type. The key idea of our solution is to unify both domains into the equirectangular projection particularly allocating perspective images into the most suitable latitude coordinate to effectively exploit both domains seamlessly. To address the Field-of-View (FoV) difference between the input domains we design uLayout with a shared feature extractor with an extra 1D-Convolution layer to condition each domain input differently. This conditioning allows us to efficiently formulate a column-wise feature regression problem regardless of the FoV input. This simple yet effective approach achieves competitive performance with current state-of-the-art solutions and shows for the first time a single end-to-end model for both domains. Extensive experiments in the real-world datasets LSUN Matterport3D PanoContext and Stanford 2D-3D evidence the contribution of our approach. Code is available at https://github.com/JonathanLee112/uLayout
RayGauss: Volumetric Gaussian-Based Ray Casting for Photorealistic Novel View Synthesis
Authors: Hugo Blanc; Jean-Emmanuel Deschaud; Alexis Paljic
Venue: WACV 2025 (main)

Abstract: Differentiable volumetric rendering-based methods made significant progress in novel view synthesis. On one hand innovative methods have replaced the Neural Radiance Fields (NeRF) network with locally parameterized structures enabling high-quality renderings in a reasonable time. On the other hand approaches have used differentiable splatting instead of NeRF's ray casting to optimize radiance fields rapidly using Gaussian kernels allowing for fine adaptation to the scene. However differentiable ray casting of irregularly spaced kernels has been scarcely explored while splatting despite enabling fast rendering times is susceptible to clearly visible artifacts. Our work closes this gap by providing a physically consistent formulation of the emitted radiance and density decomposed with Gaussian functions associated with Spherical Gaussians/Harmonics for all-frequency colorimetric representation. We also introduce a method enabling differentiable ray casting of irregularly distributed Gaussians using an algorithm that integrates radiance fields slab by slab and leverages a BVH structure. This allows our approach to finely adapt to the scene while avoiding splatting artifacts. As a result we achieve superior rendering quality compared to the state-of-the-art while maintaining reasonable training times and achieving inference speeds of 25 FPS on the Blender dataset. The associated code is available at: github.com/hugobl1/ray gauss.
To Ask or Not to Ask? Detecting Absence of Information in Vision and Language Navigation
Authors: Savitha Sam Abraham; Sourav Garg; Feras Dayoub
Venue: WACV 2025 (main)

Abstract: Recent research in Vision Language Navigation (VLN) has overlooked the development of agents' inquisitive abilities which allow them to ask clarifying questions when instructions are incomplete. This paper addresses how agents can recognize "when" they lack sufficient information without focusing on "what" is missing particularly in VLN tasks with vague instructions. Equipping agents with this ability enhances efficiency by reducing potential digressions and seeking timely assistance. The challenge in identifying such uncertain points is balancing between being overly cautious (high recall) and overly confident (high precision). We propose an attention-based instruction-vagueness estimation module that learns associations between instructions and the agent's trajectory. By leveraging instruction-to-path alignment information during training the module's vagueness estimation performance improves by around 52% in terms of precision-recall balance. In our ablative experiments we also demonstrate the effectiveness of incorporating this additional instruction-to-path attention network alongside the cross-modal attention networks within the navigator module. Our results show that the attention scores from the instruction-to-path attention network serve as better indicators for estimating vagueness.
F2former: When Fractional Fourier Meets Deep Wiener Deconvolution and Selective Frequency Transformer for Image Deblurring
Authors: Subhajit Paul; Sahil Kumawat; Ashutosh Gupta; Deepak Mishra
Venue: WACV 2025 (main)

Abstract: Recent progress in image deblurring techniques focuses mainly on operating in both frequency and spatial domains using the Fourier transform (FT) properties. However their performance is limited due to the dependency of FT on stationary signals and its lack of capability to extract spatial-frequency properties. In this paper we propose a novel approach based on the Fractional Fourier Transform (FRFT) a unified spatial-frequency representation leveraging both spatial and frequency components simultaneously making it ideal for processing non-stationary signals like images. Specifically we introduce a Fractional Fourier Transformer (F2former) where we combine the classical fractional Fourier based Wiener deconvolution (F2WD) as well as a multi-branch encoder-decoder transformer based on a new fractional frequency aware transformer block (F2TB). We design F2TB consisting of a fractional frequency aware self-attention (F2SA) to estimate element-wise product attention based on important frequency components and a novel feed-forward network based on frequency division multiplexing (FM-FFN) to refine high and low frequency features separately for efficient latent clear image restoration. Experimental results for the cases of both motion deblurring as well as defocus deblurring show that the performance of our proposed method is superior to other state-of-the-art (SOTA) approaches.
A Simple-but-Effective Baseline for Training-Free Class-Agnostic Counting
Authors: Yuhao Lin; Haiming Xu; Lingqiao Liu; Javen Qinfeng Shi
Venue: WACV 2025 (main)

Abstract: Class-Agnostic Counting (CAC) seeks to accurately count objects in a given image with only a few reference examples. While previous methods achieving this relied on additional training recent efforts have shown that it's possible to accomplish this without training by utilizing pre-existing foundation models particularly the Segment Anything Model (SAM) for counting via instance-level segmentation. Although promising current training-free methods still lag behind their training-based counterparts in terms of performance. In this research we present a straightforward training-free solution that effectively bridges this performance gap serving as a strong baseline. The primary contribution of our work lies in the discovery of four key technologies that can enhance performance. Specifically we suggest employing a superpixel algorithm to generate more precise initial point prompts utilizing an image encoder with richer semantic knowledge to replace the SAM encoder for representing candidate objects and adopting a multiscale mechanism and a transductive prototype scheme to update the representation of reference examples. By combining these four technologies our approach achieves significant improvements over existing training-free methods and delivers performance on par with training-based ones.
DiffuCE: Expert-Level CBCT Image Enhancement using a Novel Conditional Denoising Diffusion Model with Latent Alignment
Authors: Fang-Yi Su; Tzu-Hung Chang; Jung-Hsien Chiang
Venue: WACV 2025 (main)

Abstract: Cone-Beam Computed Tomography (CBCT) has garnered significant attention due to lower radiation dosage and faster scanning time which has been widely used in clinical applications for decades. However its poor image quality is always challenging to clinical experts. To address this problem we propose our work DiffuCE a Diffusion model framework for CBCT Enhancement. The main contributions of our work are three-fold: (1) Increased Generalizability: Our training data exclusively comprises pixel space data eliminating the necessity for additional imaging machine settings. This emphasizes the model's ability to generalize effectively across diverse conditions. (2) Efficient Training: Rather than starting from scratch our approach fine-tunes from a well-established foundation model. This illustrates the viability of efficient training strategies for medical image restoration tasks optimizing resource utilization. (3) Competitive Performance: DiffuCE exhibits outstanding performance excelling in FID and LPIPS with 0.01 and 36.99 ahead of the second place in the private set. In the public dataset DiffuCE has a competitive performance compared to other SOTAs. Moreover in expert assessments DiffuCE achieves the highest score of 7.06 for overall satisfaction which is 1.38 ahead of the second place affirming its performance from a clinical standpoint. Codes are available at https://github.com/lzh107u/DiffuCE
CoVLA: Comprehensive Vision-Language-Action Dataset for Autonomous Driving
Authors: Hidehisa Arai; Keita Miwa; Kento Sasaki; Kohei Watanabe; Yu Yamaguchi; Shunsuke Aoki; Issei Yamamoto
Venue: WACV 2025 (main)

Abstract: Autonomous driving particularly navigating complex and unanticipated scenarios demands sophisticated reasoning and planning capabilities. While Multi-modal Large Language Models (MLLMs) offer a promising avenue for this their use has been largely confined to understanding complex environmental contexts or generating high-level driving commands with few studies extending their application to end-to-end path planning. A major research bottleneck is the lack of large-scale annotated datasets encompassing vision language and action. To address this issue we propose CoVLA (Comprehensive Vision-Language-Action) Dataset an extensive dataset comprising real-world driving videos spanning more than 80 hours. This dataset leverages a novel scalable approach based on automated data processing and a caption generation pipeline to generate accurate driving trajectories paired with detailed natural language descriptions of driving environments and maneuvers. This approach utilizes raw in-vehicle sensor data allowing it to surpass existing datasets in scale and annotation richness. Using CoVLA we investigate the driving capabilities of MLLMs that can handle vision language and action in a variety of driving scenarios. Our results illustrate the strong proficiency of our model in generating coherent language and action outputs emphasizing the potential of Vision-Language-Action (VLA) models in the field of autonomous driving. This dataset establishes a framework for robust interpretable and data-driven autonomous driving systems by providing a comprehensive platform for training and evaluating VLA models contributing to safer and more reliable self-driving vehicles. The dataset is released for academic purpose.
Reducing the Content Bias for AI-Generated Image Detection
Authors: Seoyeon Gye; Junwon Ko; Hyounguk Shon; Minchan Kwon; Junmo Kim
Venue: WACV 2025 (main)

Abstract: Identifying AI-generated content is critical for the safe and ethical use of generative AI. Recent research has focused on developing detectors that generalize to unknown generators with popular methods relying either on high-level features or low-level fingerprints. However these methods have clear limitations: biased towards unseen content or vulnerable to common image degradations such as JPEG compression. To address these issues we propose a novel approach SFLD which incorporates PatchShuffle to integrate high-level semantic and low-level textural information. SFLD applies PatchShuffle at multiple levels improving robustness and generalization across various generative models. Additionally current benchmarks face challenges such as low image quality insufficient content preservation and limited class diversity. In response we introduce TwinSynths a new benchmark generation methodology that constructs visually near-identical pairs of real and synthetic images to ensure high quality and content preservation. Our extensive experiments and analysis show that SFLD outperforms existing methods on detecting a wide variety of fake images sourced from GANs diffusion models and TwinSynths demonstrating the state-of-the-art performance and generalization capabilities to novel generative models.
Structure-Aware Human Body Reshaping with Adaptive Affinity-Graph Network
Authors: Qiwen Deng; Yangcen Liu
Venue: WACV 2025 (main)

Abstract: Given a source portrait the automatic human body reshaping task aims at editing it to an aesthetic body shape. As the technology has been widely used in media several methods have been proposed mainly focusing on generating optical flow to warp the body shape. However those previous works only consider the local transformation of different body parts (arms torso and legs) ignoring the global affinity and limiting the capacity to ensure consistency and quality across the entire body. In this paper we propose a novel Adaptive Affinity-Graph Network (AAGN) which extracts the global affinity between different body parts to enhance the quality of the generated optical flow. Specifically our AAGN primarily introduces the following designs: (1) we propose an Adaptive Affinity-Graph (AAG) Block that leverages the characteristic of a fully connected graph. AAG represents different body parts as nodes in an adaptive fully connected graph and captures all the affinities between nodes to obtain a global affinity map. The design could better improve the consistency between body parts. (2) Besides for high-frequency details are crucial for photo aesthetics a Body Shape Discriminator (BSD) is designed to extract information from both high-frequency and spatial domains. Particularly an SRM filter is utilized to extract high-frequency details which are combined with spatial features as input to the BSD. With this design BSD guides the Flow Generator (FG) to pay attention to various fine details rather than rigid pixel-level fitting. Extensive experiments conducted on the BR-5K dataset demonstrate that our framework significantly enhances the aesthetic appeal of reshaped photos surpassing all previous work to achieve state-of-the-art in all evaluation metrics. Our code is available at https://github.com/Randle-Github/AGGN.
UW-GS: Distractor-Aware 3D Gaussian Splatting for Enhanced Underwater Scene Reconstruction
Authors: Haoran Wang; Nantheera Anantrasirichai; Fan Zhang; David Bull
Venue: WACV 2025 (main)

Abstract: 3D Gaussian splatting (3DGS) offers the capability to achieve real-time high quality 3D scene rendering. However 3DGS assumes that the scene is in a clear medium environment and struggles to generate satisfactory representations in underwater scenes where light absorption and scattering are prevalent and moving objects are involved. To overcome these we introduce a novel Gaussian Splatting-based method UW-GS designed specifically for underwater applications. It introduces a color appearance that models distance-dependent color variation employs a new physics-based density control strategy to enhance clarity for distant objects and uses a binary motion mask to handle dynamic content. Optimized with a well-designed loss function supporting for scattering media and strengthened by pseudo-depth maps UW-GS outperforms existing methods with PSNR gains up to 1.26dB. To fully verify the effectiveness of the model we also developed a new underwater dataset S-UW with dynamic object masks. The code of UW-GS and S-UW will be available at https://github.com/WangHaoran16/UW-GS.