GenAI4ELab/papercli-papers huggingface dataset:
ReFu: Recursive Fusion for Exemplar-Free 3D Class-Incremental Learning
Authors: Yi Yang; Lei Zhong; Huiping Zhuang
Venue: WACV 2025 (main)

Abstract: We introduce a novel Recursive Fusion model dubbed ReFu designed to integrate point clouds and meshes for exemplar-free 3D Class-Incremental Learning where the model learns new 3D classes while retaining knowledge of previously learned ones. Unlike existing methods that either rely on storing historical data to mitigate forgetting or focus on single data modalities ReFu eliminates the need for exemplar storage while utilizing the complementary strengths of both point clouds and meshes. To achieve this we introduce a recursive method which continuously accumulates knowledge by updating the regularized auto-correlation matrix. Furthermore we propose a fusion module featuring a Pointcloud-guided Mesh Attention Layer that learns correlations between the two modalities. This mechanism effectively integrates point cloud and mesh features leading to more robust and stable continual learning. Experiments across various datasets demonstrate that our proposed framework outperforms existing methods in 3D class-incremental learning. Project Page: https://arlo-yang.github.io/ReFu/
Talking Head Anime 4: Distillation for Real-Time Performance
Authors: Pramook Khungurn
Venue: WACV 2025 (main)

Abstract: We study the problem of creating a character model that can be controlled in real time from a single image of an anime character. A solution would greatly reduce the cost of creating avatars computer games and other interactive applications. Talking Head Anime 3 (THA3) is an open source project that attempts to directly address the problem. It takes as input (1) an image of an anime character's upper body and (2) a 45-dimensional pose vector and outputs a new image of the same character taking the specified pose. The range of possible movements is expressive enough for personal avatars and certain types of game characters. THA3's main limitation is its speed. It can achieve interactive frame rates (approximately 20 FPS) only if it is run on a very powerful GPU (Nvidia Titan RTX or better). Based on the insight that avatars and game characters do not need to change their appearance every so often we propose a technique to distill the system into a small student neural network (< 2 MB) specific to a particular character. The student model can generate 512x512 animation frames in real time (no more than 30 FPS) using consumer gaming GPUs while preserving the image quality of the teacher model. For the first time our technique makes the whole system practical for real-time applications.
NeRFs are Mirror Detectors: using Structural Similarity for Multi-View Mirror Scene Reconstruction with 3D Surface Primitives
Authors: Leif Van Holland; Michael Weinmann; Jan U. Müller; Patrick Stotko; Reinhard Klein
Venue: WACV 2025 (main)

Abstract: While neural radiance fields (NeRF) led to a breakthrough in photorealistic novel view synthesis handling mirroring surfaces still denotes a particular challenge as they introduce severe inconsistencies in the scene representation. Previous attempts either focus on reconstructing single reflective objects or rely on strong supervision guidance in terms of additional user-provided annotations of visible image regions of the mirrors thereby limiting the practical usability. In contrast in this paper we present NeRF-MD a method which shows that NeRFs can be considered as mirror detectors and which is capable of reconstructing neural radiance fields of scenes containing mirroring surfaces without the need for prior annotations. To this end we first compute an initial estimate of the scene geometry by training a standard NeRF using a depth reprojection loss. Our key insight lies in the fact that parts of the scene corresponding to a mirroring surface will still exhibit a significant photometric inconsistency whereas the remaining parts are already reconstructed in a plausible manner. This allows us to detect mirror surfaces by fitting geometric primitives to such inconsistent regions in this initial stage of the training. Using this information we then jointly optimize the radiance field and mirror geometry in a second training stage to refine their quality. We demonstrate the capability of our method to allow the faithful detection of mirrors in the scene as well as the reconstruction of a single consistent scene representation and demonstrate its potential in comparison to baseline and mirror-aware approaches.
VortSDF: 3D Modeling with Centroidal Voronoi Tesselation on Signed Distance Field
Authors: Diego Thomas; Briac Toussaint; Jean-Sebastien Franco; Edmond Boyer
Venue: WACV 2025 (main)

Abstract: Volumetric shape representations have become ubiquitous in multi-view reconstruction tasks. They often build on regular voxel grids as discrete representations of 3D shape functions such as SDF or radiance fields either as the full shape model or as sampled instantiations of continuous representations as with neural networks. Despite their proven efficiency voxel representations come with the precision versus complexity trade-off. This inherent limitation can significantly impact performance when moving away from simple and uncluttered scenes. In this paper we investigate an alternative discretization strategy with the Centroidal Voronoi Tessellation (CVT). CVTs allow to better partition the observation space with respect to shape occupancy and to focus the discretization around shape surfaces. To leverage this discretization strategy for multi-view reconstruction we introduce a volumetric optimization framework that combines explicit SDF fields with a shallow color network in order to estimate 3D shape properties over tetrahedral grids. Experimental results with Chamfer statistics validate this approach with unprecedented reconstruction quality on various scenarios such as objects open scenes or human.
ARTIST: Improving the Generation of Text-Rich Images with Disentangled Diffusion Models and Large Language Models
Authors: Jianyi Zhang; Yufan Zhou; Jiuxiang Gu; Curtis Wigington; Tong Yu; Yiran Chen; Tong Sun; Ruiyi Zhang
Venue: WACV 2025 (main)

Abstract: Diffusion models have demonstrated exceptional capabilities in generating a broad spectrum of visual content yet their proficiency in rendering text is still limited: they often generate inaccurate characters or words that fail to blend well with the underlying image. To address these shortcomings we introduce a novel framework named ARTIST which incorporates a dedicated textual diffusion model to focus on the learning of text structures specifically. Initially we pretrain this textual model to capture the intricacies of text representation. Subsequently we finetune a visual diffusion model enabling it to assimilate textual structure information from the pretrained textual model. This disentangled architecture design and training strategy significantly enhance the text rendering ability of the diffusion models for text-rich image generation. Additionally we leverage the capabilities of pretrained large language models to interpret user intentions better contributing to improved generation quality. Empirical results on the MARIO-Eval benchmark underscore the effectiveness of the proposed method showing an improvement of up to 15% in various metrics.
Advancing Weight and Channel Sparsification with Enhanced Saliency
Authors: Xinglong Sun; Maying Shen; Hongxu Yin; Lei Mao; Pavlo Molchanov; Jose M. Alvarez
Venue: WACV 2025 (main)

Abstract: Pruning aims to accelerate and compress models by removing redundant parameters identified by specifically designed importance scores which are usually imperfect. This removal is irreversible often leading to subpar performance in pruned models. Dynamic sparse training while attempting to adjust sparse structures during training for continual reassessment and refinement has several limitations including criterion inconsistency between pruning and growth unsuitability for structured sparsity and short-sighted growth strategies. Our paper introduces an efficient innovative paradigm to enhance a given importance criterion for either unstructured or structured sparsity. Our method separates the model into an active structure for exploitation and an exploration space for potential updates. During exploitation we optimize the active structure whereas in exploration we reevaluate and reintegrate parameters from the exploration space through a pruning and growing step consistently guided by the same given importance criterion. To prepare for exploration we briefly "reactivate" all parameters in the exploration space and train them for a few iterations while keeping the active part frozen offering a preview of the potential performance gains from reintegrating these parameters. We show on various datasets and configurations that existing importance criterion even simple as magnitude can be enhanced with ours to achieve state-of-the-art performance and training cost reductions. Notably on ImageNet with ResNet50 ours achieves an +1.3 increase in Top-1 accuracy over prior art at 90% ERKsparsity. Compared with the SOTA latency pruning method HALP we reduced its training cost by over 70% while attaining a faster and more accurate pruned model.
GMT: Guided Mask Transformer for Leaf Instance Segmentation
Authors: Feng Chen; Sotirios A. Tsaftaris; Mario Valerio Giuffrida
Venue: WACV 2025 (main)

Abstract: Leaf instance segmentation is a challenging multi-instance segmentation task aiming to separate and delineate each leaf in an image of a plant. Accurate segmentation of each leaf is crucial for plant-related applications such as the fine-grained monitoring of plant growth and crop yield estimation. This task is challenging because of the high similarity (in shape and colour) great size variation and heavy occlusions among leaf instances. Furthermore the typically small size of annotated leaf datasets makes it more difficult to learn the distinctive features needed for precise segmentation. We hypothesise that the key to overcoming the these challenges lies in the specific spatial patterns of leaf distribution. In this paper we propose the Guided Mask Transformer (GMT) which leverages and integrates leaf spatial distribution priors into a Transformer-based segmentor. These spatial priors are embedded in a set of guide functions that map leaves at different positions into a more separable embedding space. Our GMT consistently outperforms the state-of-the-art on three public plant datasets. Our code is available at https://github.com/vios-s/gmt-leaf-ins-seg.
Realistic and Efficient Face Swapping: A Unified Approach with Diffusion Models
Authors: Sanoojan Baliah; Qinliang Lin; Shengcai Liao; Xiaodan Liang; Muhammad Haris Khan
Venue: WACV 2025 (main)

Abstract: Despite promising progress in face swapping task realistic swapped images remain elusive often marred by artifacts particularly in scenarios involving high pose variation color differences and occlusion. To address these issues we propose a novel approach that better harnesses diffusion models for face-swapping by making following core contributions. (a) We propose to re-frame the face-swapping task as a self-supervised train-time inpainting problem enhancing the identity transfer while blending with the target image. (b) We introduce a multi-step Denoising Diffusion Implicit Model (DDIM) sampling during training reinforcing identity and perceptual similarities. (c) Third we introduce CLIP feature disentanglement to extract pose expression and lighting information from the target image improving fidelity. (d) Further we introduce a mask shuffling technique during inpainting training which allows us to create a so-called universal model for swapping with an additional feature of head swapping. Ours can swap hair and even accessories beyond traditional face swapping. Unlike prior works reliant on multiple off-the-shelf models ours is a relatively unified approach and so it is resilient to errors in other off-the-shelf models. Extensive experiments on FFHQ and CelebA datasets validate the efficacy and robustness of our approach showcasing high-fidelity realistic face-swapping with minimal inference time. Our code is available at https://github.com/Sanoojan/REFace.
Enhancing Visual Classification using Comparative Descriptors
Authors: Hankyeol Lee; Gawon Seo; Wonseok Choi; Geunyoung Jung; Kyungwoo Song; Jiyoung Jung
Venue: WACV 2025 (main)

Abstract: The performance of vision-language models (VLMs) such as CLIP in visual classification tasks has been enhanced by leveraging semantic knowledge from large language models (LLMs) including GPT. Recent studies have shown that in zero-shot classification tasks descriptors incorporating additional cues high-level concepts or even random characters often outperform those using only category names. In many classification tasks while the top-1 accuracy may be relatively low the top-5 accuracy is often significantly higher. This gap implies that most misclassifications occur among a few similar classes highlighting the model's difficulty in distinguishing between classes with subtle differences. To address this challenge we introduce a novel concept of comparative descriptors. These descriptors emphasize the unique features of a target class against its most similar classes enhancing differentiation. By generating and integrating these comparative descriptors into the classification framework we refine the semantic focus and improve classification accuracy. An additional filtering process ensures that these descriptors are closer to the image embeddings in the CLIP space further enhancing performance. Our approach demonstrates improved accuracy and robustness in visual classification tasks by addressing the specific challenge of subtle inter-class differences. Code is available at https://github.com/hk1ee/Comparative-CLIP
Dense Depth from Event Focal Stack
Authors: Kenta Horikawa; Mariko Isogawa; Hideo Saito; Shohei Mori
Venue: WACV 2025 (main)

Abstract: We propose a method for dense depth estimation from an event stream generated when sweeping the focal plane of the driving lens attached to an event camera. In this method a depth map is inferred from an "event focal stack" composed of the event stream using a convolutional neural network trained with synthesized event focal stacks. The synthesized event stream is created from a focal stack generated by Blender for any arbitrary 3D scene. This allows for training on scenes with diverse structures. Additionally we explored methods to eliminate the domain gap between real event streams and synthetic event streams. Our method demonstrates superior performance over a depth-from-defocus method in the image domain on synthetic and real datasets.
Elucidating the Solution Space of Extended Reverse-Time SDE for Diffusion Models
Authors: Qinpeng Cui; Xinyi Zhang; Qiqi Bao; Qingmin Liao
Venue: WACV 2025 (main)

Abstract: Sampling from Diffusion Models can alternatively be seen as solving differential equations where there is a challenge in balancing speed and image visual quality. ODE-based samplers offer rapid sampling time but reach a performance limit whereas SDE-based samplers achieve superior quality albeit with longer iterations. In this work we formulate the sampling process as an Extended Reverse-Time SDE (ER SDE) unifying prior explorations into ODEs and SDEs. Theoretically leveraging the semi-linear structure of ER SDE solutions we offer exact solutions and approximate solutions for VP SDE and VE SDE respectively. Based on the approximate solution space of the ER SDE referred to as one-step prediction errors we yield mathematical insights elucidating the rapid sampling capability of ODE solvers and the high-quality sampling ability of SDE solvers. Additionally we unveil that VP SDE solvers stand on par with their VE SDE counterparts. Based on these findings leveraging the dual advantages of ODE solvers and SDE solvers we devise efficient high-quality samplers namely ER-SDE-Solvers. Experimental results demonstrate that ER-SDE-Solvers achieve state-of-the-art performance across all stochastic samplers while maintaining efficiency of deterministic samplers. Specifically on the ImageNet 128 x 128 dataset ER-SDE-Solvers obtain 8.33 FID in only 20 function evaluations. Code is available at https://github.com/QinpengCui/ER-SDE-Solver
AnomalyDINO: Boosting Patch-Based Few-Shot Anomaly Detection with DINOv2
Authors: Simon Damm; Mike Laszkiewicz; Johannes Lederer; Asja Fischer
Venue: WACV 2025 (main)

Abstract: Recent advances in multimodal foundation models have set new standards in few-shot anomaly detection. This paper explores whether high-quality visual features alone are sufficient to rival existing state-of-the-art vision-language models. We affirm this by adapting DINOv2 for one-shot and few-shot anomaly detection with a focus on industrial applications. We show that this approach does not only rival existing techniques but can even outmatch them in many settings. Our proposed vision-only approach AnomalyDINO follows the well-established patch-level deep nearest neighbor paradigm and enables both image-level anomaly prediction and pixel-level anomaly segmentation. The approach is methodologically simple and training-free and thus does not require any additional data for fine-tuning or meta-learning. Despite its simplicity AnomalyDINO achieves state-of-the-art results in one- and few-shot anomaly detection (e.g. pushing the one-shot performance on MVTec-AD from an AUROC of 93.1% to 96.6%). The reduced overhead coupled with its outstanding few-shot performance makes AnomalyDINO a strong candidate for fast deployment e.g. in industrial contexts.
Tuned Contrastive Learning
Authors: Chaitanya Animesh; Manmohan Chandraker
Venue: WACV 2025 (main)

Abstract: In recent times contrastive learning based loss functions have become increasingly popular for visual self-supervised representation learning owing to their state-of-the-art (SOTA) performance. Most of the modern contrastive learning methods generalize only to one positive and multiple negatives per anchor in a batch. A recent state-of-the-art contrastive loss called supervised contrastive (SupCon) loss extends self-supervised contrastive learning to supervised setting by generalizing to multiple positives and negatives in a batch and improves upon the cross-entropy loss. In this paper we propose a novel contrastive loss function -- Tuned Contrastive Learning (TCL) loss that generalizes to multiple positives and negatives in a batch and offers parameters to tune and improve the gradient responses from hard positives and hard negatives. We provide theoretical analysis of our loss function's gradient response and show mathematically how it is better than that of SupCon loss. We empirically compare our loss function with SupCon loss and cross-entropy loss in supervised setting on multiple classification-task datasets to show its effectiveness. We also show the stability of our loss function to a range of hyper-parameter settings. Unlike SupCon loss which is only applied to supervised setting we show how to extend TCL to self-supervised setting and empirically compare it with various SOTA self-supervised learning methods. Hence we show that TCL loss achieves performance on par with SOTA methods in both supervised and self-supervised settings.
EdgeGaussians - 3D Edge Mapping via Gaussian Splatting
Authors: Kunal Chelani; Assia Benbihi; Torsten Sattler; Fredrik Kahl
Venue: WACV 2025 (main)

Abstract: With their meaningful geometry and omnipresence in the 3D world edges are extremely useful primitives in computer vision. Methods for 3D edge reconstruction have 1) either focused on reconstructing 3D edges by triangulating tracks of 2D line segments across images or 2) more recently learning a 3D edge distance field from multi-view images. The triangulation-based methods struggle to repeatedly detect and robustly match line segments resulting in noisy and incomplete reconstructions in many cases. Methods in the latter class rely on sampling edge points from the learnt implicit field which is limited by the spatial resolution of the voxel grid used for sampling resulting in imprecise points that require refinement. Further such methods require a long training that scales poorly with the size of the scene. In this paper we propose a method that explicitly learns 3D edge points with a 3D Gaussian Splatting representation trained from edge images. Backed by efficient training the proposed method produces results better than or at-par with the current state-of-the-art methods while being an order of magnitude faster. The code is released at https://github.com/kunalchelani/EdgeGaussians.
Guess Future Anomalies from Normalcy: Forecasting Abnormal Behavior in Real-World Videos
Authors: Snehashis Majhi; Mohammed Guermal; Antitza Dantcheva; Quan Kong; Lorenzo Garattoni; Gianpiero Francesca; François Brémond
Venue: WACV 2025 (main)

Abstract: Forecasting Abnormal Human Behavior (AHB) aims to predict unusual behavior in advance by analyzing early patterns of normal human interactions. Unlike typical action prediction methods this task focuses on observing only normal interactions to predict both short and long term future abnormal behavior. Despite its affirmative impact on society AHB prediction remains under-explored in current research. This is primarily due to the challenges involved in anticipating complex human behaviors and interactions with surrounding agents in real-world situations. Further there exists an underlying uncertainty between the early normal patterns and the future abnormal behavior thereby making the prediction harder. To address these challenges we introduce a novel transformer model that improves early interaction modeling by accounting for uncertainties in both observations and future outcomes. To the best of our knowledge we are the first to explore the task. Therefore we present a new comprehensive dataset referred to as AHB-F which features real-world scenarios with complex human interactions. The AHB-F has a deterministic evaluation protocol that ensures only normal frames to be observed for long and short term future prediction. We extensively evaluate and compare competitive action anticipation methods on our benchmark. Our results show that our method consistently outperforms existing action anticipation approaches both in quantitative and qualitative evaluations.
Single-Layer Distillation with Fourier Convolutions for Texture Anomaly Detection
Authors: Simon Thomine; Hichem Snoussi
Venue: WACV 2025 (main)

Abstract: In industrial quality control detecting anomalies in visual textures is essential for ensuring product quality and operational efficiency. Early identification of defects prevents faulty items from reaching consumers reduces waste and maintains high standards of production. Numerous unsupervised anomaly detection methods heavily depend on the integration of multiple layers from various pre-trained models a selection often made through empirical means. We propose SingleNet an innovative knowledge distillation approach tailored for fast unsupervised texture anomaly detection using a single layer from a compact pre-trained model. Contrary to the previous knowledge distillation approaches our network leverages fast Fourier convolutions (FFC) to reconstruct a degraded version of the teacher extracted features. At test time we employed a frequency-aware filtering mechanism to reduce reconstruction artifacts caused by discrepancies between teacher and student architectures. Empirical results demonstrate the efficacy of our approach attaining state-of-the-art performance across evaluated datasets coupled with expedited high-speed inference.