GenAI4ELab/papercli-papers huggingface dataset:
NCAP: Scene Text Image Super-Resolution with Non-CAtegorical Prior
Authors: Dongwoo Park; Suk Pil Ko
Venue: WACV 2025 (main)

Abstract: Scene text image super-resolution (STISR) enhances the resolution and quality of low-resolution images. Unlike previous studies that treated scene text images as natural images recent methods using a text prior (TP) extracted from a pre-trained text recognizer have shown strong performance. However two major issues emerge: (1) Explicit categorical priors like TP can negatively impact STISR if incorrect. We reveal that these explicit priors are unstable and propose replacing them with Non-CAtegorical Prior (NCAP) using penultimate layer representations. (2) Pre-trained recognizers used to generate TP struggle with low-resolution images. To address this most studies jointly train the recognizer with the STISR network to bridge the domain gap between low- and high-resolution images but this can cause an overconfidence phenomenon in the prior modality. We highlight this issue and propose a method to mitigate it by mixing hard and soft labels. Experiments on the TextZoom dataset demonstrate an improvement by 3.5% while our method significantly enhances generalization performance by 14.8% across four text recognition datasets. Our method generalizes to all TP-guided STISR networks.
Image-Level Regression for Uncertainty-Aware Retinal Image Segmentation
Authors: Trung D. Q. Dang; Huy Hoang Nguyen; Aleksei Tiulpin
Venue: WACV 2025 (main)

Abstract: Accurate retinal vessel (RV) segmentation is a crucial step in the quantitative assessment of retinal vasculature which is needed for the early detection of retinal diseases and other conditions. Numerous studies have been conducted to tackle the problem of segmenting vessels automatically using a pixel-wise classification approach. The common practice of creating ground truth labels is to categorize pixels as foreground and background. This approach is however biased and it ignores the uncertainty of a human annotator when it comes to annotating e.g. thin vessels. In this work we propose a simple and effective method that casts the RV segmentation task as an image-level regression. For this purpose we first introduce a novel Segmentation Annotation Uncertainty-Aware (SAUNA) transform which adds pixel uncertainty to the ground truth using the pixel's closeness to the annotation boundary and vessel thickness. To train our model with soft labels we generalize the earlier proposed Jaccard metric loss to arbitrary hypercubes for soft Jaccard index (Intersection-over-Union) optimization. Additionally we employ a stable version of the Focal-L1 loss for pixel-wise regression. We conduct thorough experiments and compare our method to a diverse set of baselines across 5 retinal image datasets. Our empirical results indicate that the integration of the SAUNA transform and these segmentation losses led to significant performance boosts for different segmentation models. Particularly our methodology enables UNet-like architectures to substantially outperform computational-intensive baselines. Our implementation is available at https://github.com/Oulu-IMEDS/SAUNA.
Test-Time Adaptation of 3D Point Clouds via Denoising Diffusion Models
Authors: Hamidreza Dastmalchi; Aijun An; Ali Cheraghian; Shafin Rahman; Sameera Ramasinghe
Venue: WACV 2025 (main)

Abstract: Test-time adaptation (TTA) of 3D point clouds is crucial for mitigating discrepancies between training and testing samples in real-world scenarios particularly when handling corrupted point clouds. LiDAR data for instance can be affected by sensor failures or environmental factors causing domain gaps. Adapting models to these distribution shifts online is crucial as training for every possible variation is impractical. Existing methods often focus on fine-tuning pre-trained models based on self-supervised learning or pseudo-labeling which can lead to forgetting valuable source domain knowledge over time and reduce generalization on future tests. In this paper we introduce a novel 3D test-time adaptation method termed 3DD-TTA which stands for 3D Denoising Diffusion Test-Time Adaptation. This method uses a diffusion strategy that adapts input point cloud samples to the source domain while keeping the source model parameters intact. The approach uses a Variational Autoencoder (VAE) to encode the corrupted point cloud into a shape latent and latent points. These latent points are corrupted with Gaussian noise and subjected to a denoising diffusion process. During this process both the shape latent and latent points are updated to preserve fidelity guiding the denoising toward generating consistent samples that align more closely with the source domain. We conduct extensive experiments on the ShapeNet dataset and investigate its generalizability on ModelNet40 and ScanObjectNN achieving state-of-the-art results. The code has been released at https://github.com/hamidreza-dastmalchi/3DD-TTA.
Task Configuration Impacts Annotation Quality and Model Training Performance in Crowdsourced Image Segmentation
Authors: Benjamin R Bauchwitz; Mary Cummings
Venue: WACV 2025 (main)

Abstract: Many industrial image segmentation systems require training on large annotated datasets but there is little standardization for producing training annotations. Data is often obtained via crowdsourcing but many labeling task configurations are set by convenience and may have unintended effects on data quality. In this work we (1) demonstrate a new software tool for running crowdsourced image segmentation experiments (2) present a dataset capturing variation in segmentation annotations produced under different task configurations and (3) experimentally evaluate the quality of the annotations produced by these different configurations. We show annotation quality can be significantly improved by paying annotators per annotated object rather than per image and by leveraging paintbrush-style drawing tools rather than polygon or curve drag tools. We also show that some task complexity is required to maintain annotator engagement and sufficient task performance. Finally we show that many configuration-related annotation errors degrade model training performance but that models can tolerate error error patterns that are common across crowdsourced annotation schemes.
Hausdorff Distance Matching with Adaptive Query Denoising for Rotated Detection Transformer
Authors: Hakjin Lee; MinKi Song; Jamyoung Koo; Junghoon Seo
Venue: WACV 2025 (main)

Abstract: Detection Transformers (DETR) have recently set new benchmarks in object detection. However their performance in detecting rotated objects lags behind established oriented object detectors. Our analysis identifies a key observation: the boundary discontinuity and square-like problem in bipartite matching poses an issue with assigning appropriate ground truths to predictions leading to du plicate low-confidence predictions. To address this we introduce a Hausdorff distance-based cost for bipartite matching which more accurately quantifies the discrepancy between predictions and ground truths. Additionally we find that a static denoising approach impedes the training of rotated DETR especially as the quality of the detector's predictions begins to exceed that of the noised ground truths. To overcome this we propose an adaptive query denoising method that employs bipartite matching to selectively eliminate noised queries that detract from model improvement. When compared to models adopting a ResNet-50 backbone our proposed model yields remarkable improvements achieving +4.18 AP50 +4.59 AP50 and +4.99 AP50 on DOTA-v2.0 DOTA-v1.5 and DIOR-R respectively.
3D Shape Completion using Multi-Resolution Spectral Encoding
Authors: Pallabjyoti Deka; Saumik Bhattacharya; Debashis Sen; Prabir Kumar Biswas
Venue: WACV 2025 (main)

Abstract: Reconstruction of intricate local patterns and large missing regions during 3D shape completion has the contradictory requirements of computation over a wider context and operations for finer detail restoration. To this end we propose a multi-resolution spectral encoding based 3D shape completion approach to work on truncated Signed Distance Field (SDF) based shape representations. Our novelty lies in judiciously integrating multi-resolution 3D convolutional blocks that encode the input shape and a spectral module (SM) that captures the shape-wide context thus addressing the contradictory requirements. SM acts on the features extracted from both partial input scans and shape priors using the multi-resolution convolutional blocks. Our SM contains a 3D convolutional block placed between fast Fourier transform (FFT) and inverse FFT operations which results in the expansion of the receptive field for the appropriate context computation. Our approach has an attention-based encoder-decoder architecture where the encoding of a partial scan is acted upon by shape prior encodings to produce attention maps. These attention maps are leveraged differently in pretraining and in the later training and inference stages of our approach to produce the reconstructed 3D shape. A surface gradient-based loss function is used in addition to the L1 loss both in the pretraining and training stages for emphasizing the differences in minute details. These along with an attention refinement operation often leads to complete reconstruction while restoring finer details. Experiments using standard synthetic and real datasets demonstrate the superiority of our approach over the state-of-the-art.
Enhancing Embodied Object Detection with Spatial Feature Memory
Authors: Nicolas Harvey Chapman; Christopher Lehnert; Will Browne; Feras Dayoub
Venue: WACV 2025 (main)

Abstract: Deep-learning and large scale language-image training have produced image object detectors that generalise well to diverse environments and semantic classes. However existing object detection paradigms are not optimally tailored for the embodied conditions inherent in robotics where the same objects are repeatedly observed over time. In this setting detectors that operate on single images or short sequences are likely to produce inconsistent predictions. Motivated by this we explore if the embodiment of the detector can be utilised to generate more consistent and reliable detections during repeat observation of a scene. We propose a novel framework that incrementally updates a spatial feature memory while using it as a prior to perform image object detection. By leveraging the embodiment of the robot in this way raw object detection performance is enhanced by up to 4.12 mAP and downstream robotic tasks such as semantic mapping and object recall are improved. We also investigate the structure this spatial memory should take leading to an implementation that aggregates features from the shared language-image embedding space. This approach allows the detector to effectively balance the use of memory and image features while ensuring that the benefits of language-image pre-training can be enjoyed alongside our spatial memory.
ROSA: Reconstructing Object Shape and Appearance Textures by Adaptive Detail Transfer
Authors: Julian Kaltheuner; Patrick Stotko; Reinhard Klein
Venue: WACV 2025 (main)

Abstract: Reconstructing an object's shape and appearance in terms of a mesh textured by a spatially-varying bidirectional reflectance distribution function (SVBRDF) from a limited set of images captured under collocated light is an ill-posed problem. Previous state-of-the-art approaches either aim to reconstruct the appearance directly on the geometry or additionally use texture normals as part of the appearance features. However this requires detailed but inefficiently large meshes that would have to be simplified in a post-processing step or suffers from well-known limitations of normal maps such as missing shadows or incorrect silhouettes. Another limiting factor is the fixed and typically low resolution of the texture estimation resulting in loss of important surface details. To overcome these problems we present ROSA an inverse rendering method that directly optimizes mesh geometry with spatially adaptive mesh resolution solely based on the image data. In particular we refine the mesh and locally condition the surface smoothness based on the estimated normal texture and mesh curvature. In addition we enable the reconstruction of fine appearance details in high-resolution textures through a pioneering tile-based method that operates on a single pre-trained decoder network but is not limited by the network output resolution.
Interactive Object Detection for Tiny Objects in Large Remotely Sensed Images
Authors: Marvin Burges; Sebastian Zambanini; Robert Sablatnig
Venue: WACV 2025 (main)

Abstract: This paper highlights the potential of a Human-In-the-Loop (HIL) in interactive object detection methods. Although automation in computer vision is advancing rapidly certain critical tasks such as detecting UneXploded Ordnance (UXO) space/marine debris or the generation of new datasets require 100% recall and near-perfect precision. These tasks are often performed manually since automatic methods do not achieve the necessary accuracy. However interactive object detection frameworks can potentially enhance annotation speed while maintaining the recall and accuracy of manual annotation. We propose IRTDETR an interactive and real-time object detection method for very large imagery to address this. Using either point or bounding box annotations provided by a HIL it globally relates the full image with the annotator inputs via a cross-attention-like mechanism employs an attention loss to maximize the classification score based on similarity and reuses portions of the network outputs during iterative refinements to conserve resources. We conduct experiments on five different datasets (Tiny-DOTA CHAI AITOD SarDET and COCO) to verify the efficacy of our approach. Our method surpasses existing interactive annotation approaches achieving a higher mean Average Precision (mAP) with the same number of clicks. Additionally we validate the annotation efficiency of our method in a user study demonstrating it is 2.46x quicker and asks for only 72% of the task load (NASA-TLX) compared to fully manual annotation. The code will is available under https://github.com/mburges-cvl/WACV_IAODF.
Cross-Task Affinity Learning for Multitask Dense Scene Predictions
Authors: Dimitrios Sinodinos; Narges Armanfard
Venue: WACV 2025 (main)

Abstract: Multitask learning (MTL) has become prominent for its ability to predict multiple tasks jointly achieving better per-task performance with fewer parameters than single-task learning. Recently decoder-focused architectures have significantly improved multitask performance by refining task predictions using features from related tasks. However most refinement methods struggle to efficiently capture both local and long-range dependencies between task-specific representations and cross-task patterns. In this paper we introduce the Cross-Task Affinity Learning (CTAL) module a lightweight framework that enhances task refinement in multitask networks. CTAL effectively captures local and long-range cross-task interactions by optimizing task affinity matrices for parameter-efficient grouped convolutions without concern for information loss. Our results demonstrate state-of-the-art MTL performance for both CNN and transformer backbones using significantly fewer parameters than single-task learning.
LORD: Large Models Based Opposite Reward Design for Autonomous Driving
Authors: Xin Ye; Feng Tao; Abhirup Mallik; Burhaneddin Yaman; Liu Ren
Venue: WACV 2025 (main)

Abstract: Reinforcement learning (RL) based autonomous driving has emerged as a promising alternative to data-driven imitation learning approaches. However crafting effective reward functions for RL poses challenges due to the complexity of defining and quantifying good driving behaviors across diverse scenarios. Recently large pretrained models have gained significant attention as zero-shot reward models for tasks specified with desired linguistic goals. However the desired linguistic goals for autonomous driving such as "drive safely" are ambiguous and incomprehensible by pretrained models. On the other hand undesired linguistic goals like "collision" are more concrete and tractable. In this work we introduce LORD a novel large models based opposite reward design through undesired linguistic goals to enable the efficient use of large pretrained models as zero-shot reward models. Through extensive experiments our proposed framework shows its efficiency in leveraging the power of large pretrained models for achieving safe and enhanced autonomous driving. Moreover the proposed approach shows improved generalization capabilities as it outperforms counterpart methods across diverse and challenging driving scenarios.
Moment of Untruth: Dealing with Negative Queries in Video Moment Retrieval
Authors: Kevin Flanagan; Dima Damen; Michael Wray
Venue: WACV 2025 (main)

Abstract: Video Moment Retrieval is a common task to evaluate the performance of visual-language models - it involves localising start and end times of moments in videos from query sentences. The current task formulation assumes that the queried moment is present in the video resulting in false positive moment predictions when irrelevant query sentences are provided. In this paper we propose the task of Negative-Aware Video Moment Retrieval (NA-VMR) which considers both moment retrieval accuracy and negative query rejection accuracy. We make the distinction between In-Domain and Out-of-Domain negative queries and provide new evaluation benchmarks for two popular video moment retrieval datasets: QVHighlights and Charades-STA. We analyse the ability of current SOTA video moment retrieval approaches to adapt to Negative-Aware Video Moment Retrieval and propose UniVTG-NA an adaptation of UniVTG designed to tackle NA-VMR. UniVTG-NA achieves high negative rejection accuracy (avg. 98.4%) scores while retaining moment retrieval scores to within 3.87% Recall@1. Dataset splits are available at https://github.com/keflanagan/MomentofUntruth
Sparse-View 3D Reconstruction of Clothed Humans via Normal Maps
Authors: Jane Wu; Diego Thomas; Ronald Fedkiw
Venue: WACV 2025 (main)

Abstract: We present a novel deep learning-based approach to the 3D reconstruction of clothed humans using weak supervision via 2D normal maps. Given a single RGB image or multiview images our network is optimized to infer a person-specific signed distance function (SDF) discretized on a tetrahedral mesh surrounding the body in a rest pose. Subsequently estimated human pose and camera parameters are used to generate a normal map from the SDF. A key aspect of our approach is the direct use of the Marching Tetrahedra algorithm in end-to-end optimization and in order to do so we derive analytical gradients to facilitate straightforward differentiation (and thus backpropagation). Additionally predicted normal maps allow us to leverage pretrained image-to-normal networks in order to minimize a surface error instead of a photometric error. We demonstrate the efficacy of our approach on both labeled and in-the-wild data in the context of existing clothed human reconstruction methods.
Optimizing Neural Network Effectiveness via Non-Monotonicity Refinement
Authors: Koushik Biswas; Amit Reza; Meghana Karri; Debesh Jha; Hongyi Pan; Nikhil Tomar; Aliza Subedi; Smriti Regmi; Ulas Bagci
Venue: WACV 2025 (main)

Abstract: Activation functions play a crucial role in artificial neural networks by introducing non-linearities that enable networks to learn complex patterns in data. An appropriate choice of an activation function plays a crucial role in the training dynamics of a neural network which can boost network performance significantly. Rectified Linear Unit (ReLU) and its variants like leaky ReLU and parametric ReLU have emerged as the most popular activations due to their ability to enable faster training and generalization in deep neural networks despite having some significant issues like vanishing gradient problems. In this paper we have proposed smooth functions which we call the AMSU family which are smooth approximations of the maximum function. We derive three activations from the AMSU family namely AMSU-1 AMSU-2 & AMSU-3 and show their effectiveness in different deep learning problems. By simply replacing the ReLU function Top-1 accuracy improves by 5.88% 5.96% and 5.32% on the CIFAR100 dataset on the ShuffleNet V2 model. Also replacing ReLU with AMSU-1 AMSU-2 and AMSU-3 Top-1 accuracy improves by 8.50% 8.29% and 7.70% on the CIFAR100 dataset on the ShuffleNet V2 model with FGSM attack. Also Replacing ReLU with AMSU-1 AMSU-2 and AMSU-3 on ImageNet-1K data we got 3%-5% improvement on ShuffleNet and MobileNet models. The source code is publicly available at https://github.com/koushik313/AMSU.
STRIDE: Single-Video Based Temporally Continuous Occlusion-Robust 3D Pose Estimation
Authors: Rohit Lal; Saketh Bachu; Yash Garg; Arindam Dutta; Calvin-Khang Ta; Hannah Dela Cruz; Dripta S. Raychaudhuri; M. Salman Asif; Amit Roy-Chowdhury
Venue: WACV 2025 (main)

Abstract: The capability to accurately estimate 3D human poses is crucial for diverse fields such as action recognition gait recognition and virtual/augmented reality. However a persistent and significant challenge within this field is the accurate prediction of human poses under conditions of severe occlusion. Traditional image-based estimators struggle with heavy occlusions due to a lack of temporal context resulting in inconsistent predictions. While video-based models benefit from processing temporal data they encounter limitations when faced with prolonged occlusions that extend over multiple frames. This challenge arises because these models struggle to generalize beyond their training datasets and the variety of occlusions is hard to capture in the training data. Addressing these challenges we propose STRIDE (Single-video based TempoRally contInuous occlusion Robust 3D Pose Estimation) a novel Test-Time Training (TTT) approach to fit a human motion prior for each video. This approach specifically handles occlusions that were not encountered during the model's training. By employing STRIDE we can refine a sequence of noisy initial pose estimates into accurate temporally coherent poses during test time effectively overcoming the limitations of prior methods. Our framework demonstrates flexibility by being model-agnostic allowing us to use any off-the-shelf 3D pose estimation method for improving robustness and temporal consistency. We validate STRIDE's efficacy through comprehensive experiments on challenging datasets like Occluded Human3.6M Human3.6M and OCMotion where it not only outperforms existing single-image and video-based pose estimation models but also showcases superior handling of substantial occlusions achieving fast robust accurate and temporally consistent 3D pose estimates. Code is made publicly available at https://github.com/take2rohit/stride