GenAI4ELab/papercli-papers huggingface dataset:
Improving Uncertainty Estimation with Confidence-Aware Training Data
Authors: Sergey Korchagin; Ekaterina Zaychenkova; Aleksei Khalin; Aleksandr Yugay; Alexey Zaytsev; Egor Ershov
Venue: WACV 2025 (main)

Abstract: AI-driven second-opinion systems play a crucial role in decision-making especially in medicine where accurate predictions guide clinicians. However quantifying uncertainty in deep learning is challenging as current methods often rely on hard class labels which do not reflect true prediction confidence. This often results in overconfident predictions and slow convergence to true probabilities. To address this we suggest a new method that separates uncertainty into two types: epistemic and aleatoric. We estimate these uncertainties using hard and soft confidence labels with experts providing confidence levels that indicate the likelihood of misclassification. We release an updated blood typing dataset consisting of 3139 images with soft labels of uncertainty annotations from six experts and hard labels collected from medical records. Proposed approach improves SotA uncertainty estimation quality by two times for blood typing (classification) and by 62% for histology (segmentation). The code is available at: https://github.com/createcolor/confidence-aware-uncertainty.
Sli2Vol+: Segmenting 3D Medical Images Based on an Object Estimation Guided Correspondence Flow Network
Authors: Delin An; Pengfei Gu; Milan Sonka; Chaoli Wang; Danny Z. Chen
Venue: WACV 2025 (main)

Abstract: Deep learning (DL) methods have shown remarkable successes in medical image segmentation often using large amounts of annotated data for model training. However acquiring a large number of diverse labeled 3D medical image datasets is highly difficult and expensive. Recently mask propagation DL methods were developed to reduce the annotation burden on 3D medical images. For example Sli2Vol [59] proposed a self-supervised framework (SSF) to learn correspondences by matching neighboring slices via slice reconstruction in the training stage; the learned correspondences were then used to propagate a labeled slice to other slices in the test stage. But these methods are still prone to error accumulation due to the inter-slice propagation of reconstruction errors. Also they do not handle discontinuities well which can occur between consecutive slices in 3D images as they emphasize exploiting object continuity. To address these challenges in this work we propose a new SSF called Sli2Vol+ for segmenting any anatomical structures in 3D medical images using only a single annotated slice per training and testing volume. Specifically in the training stage we first propagate an annotated 2D slice of a training volume to the other slices generating pseudo-labels (PLs). Then we develop a novel Object Estimation Guided Correspondence Flow Network to learn reliable correspondences between consecutive slices and corresponding PLs in a self-supervised manner. In the test stage such correspondences are utilized to propagate a single annotated slice to the other slices of a test volume. We demonstrate the effectiveness of our method on various medical image segmentation tasks with different datasets showing better generalizability across different organs modalities and modals. Code is available at https://github.com/adlsn/Sli2Volplus
LIME: Localized Image Editing via Attention Regularization in Diffusion Models
Authors: Enis Simsar; Alessio Tonioni; Yongqin Xian; Thomas Hofmann; Federico Tombari
Venue: WACV 2025 (main)

Abstract: Diffusion models (DMs) have gained prominence due to their ability to generate high-quality varied images with recent advancements in text-to-image generation. The research focus is now shifting towards the controllability of DMs. A significant challenge within this domain is localized editing where specific areas of an image are modified without affecting the rest of the content. This paper introduces LIME for localized image editing in diffusion models. LIME does not require user-specified regions of interest (RoI) or additional text input but rather employs features from pre-trained methods and a straightforward clustering method to obtain precise editing mask. Then by leveraging cross-attention maps it refines these segments for finding regions to obtain localized edits. Finally we propose a novel cross-attention regularization technique that penalizes unrelated cross-attention scores in the RoI during the denoising steps ensuring localized edits. Our approach without re-training fine-tuning and additional user inputs consistently improves the performance of existing methods in various editing benchmarks. The project page can be found at https://enisimsar.github.io/LIME/.
Sifting through the Haystack - Efficiently Finding Rare Animal Behaviors in Large-Scale Datasets
Authors: Shir Bar; Or Hirschorn; Roi Holzman; Shai Avidan
Venue: WACV 2025 (main)

Abstract: In the study of animal behavior researchers often record long continuous videos accumulating into large-scale datasets. However the behaviors of interest are often rare compared to routine behaviors. This incurs a heavy cost on manual annotation forcing users to sift through many samples before finding their needles. We propose a pipeline to efficiently sample rare behaviors from large datasets enabling the creation of training datasets for rare behavior classifiers. Our method only needs an unlabeled animal pose or acceleration dataset as input and makes no assumptions regarding the type number or characteristics of the rare behaviors. Our pipeline is based on a recent graph-based anomaly detection model for human behavior which we apply to this new data domain. It leverages anomaly scores to automatically label normal samples while directing human annotation efforts toward anomalies. In research data anomalies may come from many different sources (e.g. signal noise versus true rare instances). Hence the entire labeling budget is focused on the abnormal classes letting the user review and label samples according to their needs. We tested our approach on three datasets of freely-moving animals acquired in the laboratory and the field. We found that graph-based models are particularly useful when studying motion-based behaviors in animals yielding good results while using a small labeling budget. Our method consistently outperformed traditional random sampling offering an average improvement of 70% in performance and creating datasets even when the behavior of interest was only 0.02% of the data. Even when the performance gain was minor (e.g. when the behavior is not rare) our method still reduced the annotation effort by half.
DualCIR: Enhancing Training-Free Composed Image Retrieval via Dual-Directional Descriptions
Authors: Jingjiao Zhao; Jiaju Li; Dongze Lian; Liguo Sun; Pin Lv
Venue: WACV 2025 (main)

Abstract: Integrating language and images for Composed Image Retrieval (CIR) allows for a flexible representation of search intent making it a focal point in multi-modal research. Traditional CIR methods train models on complex triplet datasets. In contrast Zero-Shot Composed Image Retrieval (ZS-CIR) eliminates the need for constructing datasets and training models for each specific task attracting significant attention from researchers. CIReVL is a training-free method that translates visual content into textual representations employing large language models (LLMs) to perform robust text inference and visual language models (VLMs) to ensure multi-modal alignment during retrieval. Although this approach achieves high performance without training and offers strong interpretability relying solely on single descriptive texts for retrieval has limitations in handling complex search demands. To address this issue we propose a DualCIR framework which utilizes an LLM to generate refined dual-directional descriptions--both positive and negative--alongside generating holistic target descriptions. We score retrieval data separately via these descriptions merge the scores perform ranking and ultimately achieve retrieval. Extensive experiments across three datasets in the natural image and fashion domains show that our approach maintains low inference costs while noticeably enhancing retrieval effectiveness. Compared to existing training-free methods our DualCIR achieves state-of-the-art performance and is even competitive with the training-based methods.
ReMP: Reusable Motion Prior for Multi-Domain 3D Human Pose Estimation and Motion Inbetweening
Authors: Hojun Jang; Young Min Kim
Venue: WACV 2025 (main)

Abstract: We present Reusable Motion prior (ReMP) an effective motion prior that can accurately track the temporal evolution of motion in various downstream tasks. Inspired by the success of foundation models we argue that a robust spatio-temporal motion prior can encapsulate underlying 3D dynamics applicable to various sensor modalities. We learn the rich motion prior from a sequence of complete parametric models of posed human body shape. Our prior can easily estimate poses in missing frames or noisy measurements despite significant occlusion by employing a temporal attention mechanism. More interestingly our prior can guide the system with incomplete and challenging input measurements to quickly extract critical information to estimate the sequence of poses significantly improving the training efficiency for mesh sequence recovery. ReMP consistently outperforms the baseline method on diverse and practical 3D motion data including depth point clouds LiDAR scans and IMU sensor data. Project page is available in https://hojunjang17.github.io/ReMP.
Decomposed Distribution Matching in Dataset Condensation
Authors: Sahar Rahimi Malakshan; Mohammad Saeed Ebrahimi Saadabadi; Ali Dabouei; Nasser Nasrabadi
Venue: WACV 2025 (main)

Abstract: Dataset Condensation (DC) aims to reduce deep neural networks training efforts by synthesizing a small dataset such that it will be as effective as the original large dataset. Conventionally DC relies on a costly bi-level optimization which prohibits its practicality. Recent research formulates DC as a distribution matching problem which circumvents the costly bi-level optimization. However this efficiency sacrifices the DC performance. To investigate this performance degradation we decomposed the dataset distribution into content and style. Our observations indicate two major shortcomings of: 1) style discrepancy between original and condensed data and 2) limited intra-class diversity of condensed dataset. We present a simple yet effective method to match the style information between original and condensed data employing statistical moments of feature maps as well-established style indicators. Moreover we enhance the intra-class diversity by maximizing the Kullback-Leibler divergence within each synthetic class i.e. content. We demonstrate the efficacy of our method through experiments on diverse datasets of varying size and resolution achieving improvements of up to 4.1% on CIFAR10 4.2% on CIFAR100 4.3% on TinyImageNet 2.0% on ImageNet-1K 3.3% on ImageWoof 2.5% on ImageNette and 5.5% in continual learning accuracy.
Reviving Poor Object Segmentations in OOD Medical Images using Variational-Deep-PCA Modeling on Segmentation Maps with Sampling-Free Learning
Authors: Jimut B. Pal; Shantanu Welling; Himali Saini; Suyash P. Awate
Venue: WACV 2025 (main)

Abstract: For object segmentation in medical images deep neural networks (DNNs) typically perform poorly on out-of-distribution (OOD) images stemming from the large variability in image-acquisition equipment and protocols across sites. However compared to such variability in the acquired medical images we observe that the variability in the underlying object-segmentation maps is far lower. Thus we propose a novel DNN framework to model this variability in segmentation maps and leverage it to revive poor segmentations produced by existing DNNs on OOD images. Our DNN framework (i) learns the principal modes of variation in a class of segmentation maps (ii) models each segmentation map using a low-dimensional mixture-of-modes latent representation on a simplex (iii) enables sampling-free variational learning and uncertainty estimation and (iv) trains using small in-distribution image sets. When OOD-image segmentations are extremely poor we propose a novel human-in-the-loop method needing minuscule human intervention. Results using 6 publicly-available datasets and 8 existing DNN segmenters show the benefits of our framework in OOD-image object segmentation.
Training-Free Medical Image Inverses via Bi-Level Guided Diffusion Models
Authors: Hossein Askari; Fred Roosta; Hongfu Sun
Venue: WACV 2025 (main)

Abstract: In medical imaging inverse problems aim to infer high-fidelity images from incomplete noisy measurements minimizing expenses and risks to patients in clinical settings. Diffusion models have recently emerged as a promising solution to such practical challenges proving particularly useful for the training-free inference of images from partially acquired measurements in Magnetic Resonance Imaging (MRI) and Computed Tomography (CT). A central challenge however is how to guide an unconditional denoised estimate to conform to the measurement information. Existing methods often employ either deficient projection or inefficient posterior approximation leading to suboptimal performance. In this paper we propose Bi-level Guided Diffusion Models (BGDM) a zero-shot imaging framework that efficiently steers the image generation process through a bi-level guidance strategy. Specifically BGDM first approximates an inner-level conditional posterior mean to establish an initial measurement-consistent prediction and then solves an outer-level proximal optimization objective to reinforce the measurement consistency. Our experimental findings leveraging publicly available MRI and CT datasets indicate that BGDM is more effective and efficient compared to baseline methods consistently generating high-fidelity medical images and significantly reducing hallucinatory artifacts in cases of sparse measurements. Code https://github.com/hosseinaskari-cs/BGDM.
Fine-Tuning Image-Conditional Diffusion Models is Easier than You Think
Authors: Gonzalo Martin Garcia; Karim Abou Zeid; Christian Schmidt; Daan de Geus; Alexander Hermans; Bastian Leibe
Venue: WACV 2025 (main)

Abstract: Recent work showed that large diffusion models can be reused as highly precise monocular depth estimators by casting depth estimation as an image-conditional image generation task. While the proposed model achieved state-of-the-art results high computational demands due to multi-step inference limited its use in many scenarios. In this paper we show that the perceived inefficiency was caused by a flaw in the inference pipeline that has so far gone unnoticed. The fixed model performs comparably to the best previously reported configuration while being more than 200x faster. To optimize for downstream task performance we perform end-to-end fine-tuning on top of the single-step model with task-specific losses and get a deterministic model that outperforms all other diffusion-based depth and normal estimation models on common zero-shot benchmarks. We surprisingly find that this fine-tuning protocol also works directly on Stable Diffusion and achieves comparable performance to current state-of-the-art diffusion-based depth and normal estimation models calling into question some of the conclusions drawn from prior works.
Feature Fusion Transferability Aware Transformer for Unsupervised Domain Adaptation
Authors: Xiaowei Yu; Zhe Huang; Zao Zhang
Venue: WACV 2025 (main)

Abstract: Unsupervised domain adaptation (UDA) aims to leverage the knowledge learned from labeled source domains to improve performance on the unlabeled target domains. While Convolutional Neural Networks (CNNs) have been dominant in previous UDA methods recent research has shown promise in applying Vision Transformers (ViTs) to this task. In this study we propose a novel Feature Fusion Transferability Aware Transformer (FFTAT) to enhance ViT performance in UDA tasks. Our method introduces two key innovations: First we introduce a patch discriminator to evaluate the transferability of patches generating a transferability matrix. We integrate this matrix into self-attention directing the model to focus on transferable patches. Second we propose a feature fusion technique to fuse embeddings in the latent space enabling each embedding to incorporate information from all others thereby improving generalization. These two components work in synergy to enhance feature representation learning. Extensive experiments on widely used benchmarks demonstrate that our method significantly improves UDA performance achieving state-of-the-art (SOTA) results.
Secrets of Edge-Informed Contrast Maximization for Event-Based Vision
Authors: Pritam P. Karmokar; Quan H. Nguyen; William J. Beksi
Venue: WACV 2025 (main)

Abstract: Event cameras capture the motion of intensity gradients (edges) in the image plane in the form of rapid asynchronous events. When accumulated in 2D histograms these events depict overlays of the edges in motion consequently obscuring the spatial structure of the generating edges. Contrast maximization (CM) is an optimization framework that can reverse this effect and produce sharp spatial structures that resemble the moving intensity gradients by estimating the motion trajectories of the events. Nonetheless CM is still an underexplored area of research with avenues for improvement. In this paper we propose a novel hybrid approach that extends CM from uni-modal (events only) to bi-modal (events and edges). We leverage the underpinning concept that given a reference time optimally warped events produce sharp gradients consistent with the moving edge at that time. Specifically we formalize a correlation-based objective to aid CM and provide key insights into the incorporation of multiscale and multireference techniques. Moreover our edge-informed CM method yields superior sharpness scores and establishes new state-of-the-art event optical flow benchmarks on the MVSEC DSEC and ECD datasets.
Explicit Guidance for Robust Video Frame Interpolation against Discontinuous Motions
Authors: JaeHyun Park; Nam Ik Cho
Venue: WACV 2025 (main)

Abstract: Nowadays many videos contain graphic elements such as logos subtitles and user interfaces. These overlayed elements exhibit discontinuous motions characterized by static or instantaneous motions that are neither spatially nor temporally coherent. As existing Video Frame Interpolation (VFI) methods rely on motion-compensation techniques they work best for videos with continuous motions but face limitations against videos with discontinuous motion. In this paper we propose a simple framework to enhance the robustness of existing VFI models against discontinuous motion. We first identify key properties that distinguish discontinuous from continuous motion. They are then leveraged by the Discontinuity map (D-map) estimator to explicitly guide the classification of continuous and discontinuous areas through a coherence mask and an additional supervisory signal. Our framework separately interpolates the predicted continuous and discontinuous regions to achieve state-of-the-art performance against synthetic discontinuous motions while also generalizing well to real-world discontinuous motions. Moreover our framework's `plug-and-play' design enables easy application to existing VFI models without the need for retraining and maintains strong performance on continuous motion.
Towards Secure and Usable 3D Assets: A Novel Framework for Automatic Visible Watermarking
Authors: Gursimran Singh; Tianxi Hu; Mohammad Akbari; Qiang Tang; Yong Zhang
Venue: WACV 2025 (main)

Abstract: 3D models particularly AI-generated ones have witnessed a recent surge across various industries such as entertainment. Hence there is an alarming need to protect the intellectual property and avoid the misuse of these valuable assets. As a viable solution to address these concerns we rigorously define the novel task of automated 3D visible watermarking in terms of two competing aspects: watermark quality and asset utility. Moreover we propose a method of embedding visible watermarks that automatically determines the right location orientation and number of watermarks to be placed on arbitrary 3D assets for high watermark quality and asset utility. Our method is based on a novel rigid-body optimization that uses back-propagation to automatically learn transforms for ideal watermark placement. In addition we propose a novel curvature-matching method for fusing the watermark into the 3D model that further improves readability and security. Finally we provide a detailed experimental analysis on two benchmark 3D datasets validating the superior performance of our approach in comparison to baselines. Code and demo are available here.
SensorFlow: Sensor and Image Fused Video Stabilization
Authors: Jiyang Yu; Tianhao Zhang; Fuhao Shi; Lei He; Chia-Kai Liang
Venue: WACV 2025 (main)

Abstract: We present SensorFlow a novel image and sensor fusion framework for robust high-quality video stabilization. We start with sensor-based pre-stabilization that smooths out large-scale camera motion. A new angular velocity domain optimization has been introduced to achieve frame rate invariance. We then feed the stabilized optical flows into an occlusion-aware 3D CNN that infers dense warp fields to remove residual translation and jitter. To further avoid distortion we propose a novel masking scheme to determine the disoccluded and dynamic regions in optical flow and inpaint them with spatially smooth flow vectors. Our method is appealing as it shares both the dense warping field's flexibility to correct complex motions and the robustness of sensor data for arbitrarily challenging scenes. We have validated its effectiveness and demonstrated our solution outperforms state-of-the-art alternatives via extensive ablation studies and quantitative comparisons.
TrackDiffusion: Tracklet-Conditioned Video Generation via Diffusion Models
Authors: Pengxiang Li; Kai Chen; Zhili Liu; Ruiyuan Gao; Lanqing Hong; Dit-Yan Yeung; Huchuan Lu; Xu Jia
Venue: WACV 2025 (main)

Abstract: Despite remarkable achievements in video synthesis achieving granular control over complex dynamics such as nuanced movement among multiple interacting objects still presents a significant hurdle for dynamic world modeling compounded by the necessity to manage appearance and disappearance drastic scale changes and ensure consistency for instances across frames. These challenges hinder the development of video generation that can faithfully mimic real-world complexity limiting utility for applications requiring high-level realism and controllability including advanced scene simulation and training of perception systems. To address that we propose TrackDiffusion a novel video generation framework affording fine-grained trajectory-conditioned motion control via diffusion models which facilitates the precise manipulation of the object trajectories and interactions overcoming the prevalent limitation of scale and continuity disruptions. A pivotal component of TrackDiffusion is the instance enhancer which explicitly ensures inter-frame consistency of multiple objects a critical factor overlooked in the current literature. Moreover we demonstrate that generated video sequences by our TrackDiffusion can be used as training data for visual perception models. To the best of our knowledge this is the first work to apply video diffusion models with tracklet conditions and demonstrate that generated frames can be beneficial for improving the performance of object trackers.