GenAI4ELab/papercli-papers huggingface dataset:
MOOSS: Mask-Enhanced Temporal Contrastive Learning for Smooth State Evolution in Visual Reinforcement Learning
Authors: Jiarui Sun; M. Ugur Akcal; Girish Chowdhary; Wei Zhang
Venue: WACV 2025 (main)

Abstract: In visual Reinforcement Learning (RL) learning from pixel-based observations poses significant challenges on sample efficiency primarily due to the complexity of extracting informative state representations from high-dimensional data. Previous methods such as contrastive-based approaches have made strides in improving sample efficiency but fall short in modeling the nuanced evolution of states. To address this we introduce MOOSS a novel framework that leverages a temporal contrastive objective with the help of graph-based spatial-temporal masking to explicitly model state evolution in visual RL. Specifically we propose a self-supervised dual-component strategy that integrates (1) a graph construction of pixel-based observations for spatial-temporal masking coupled with (2) a multi-level contrastive learning mechanism that enriches state representations by emphasizing temporal continuity and change of states. MOOSS advances the understanding of state dynamics by disrupting and learning from spatial-temporal correlations which facilitates policy learning. Our comprehensive evaluation on multiple continuous and discrete control benchmarks shows that MOOSS outperforms previous state-of-the-art visual RL methods in terms of sample efficiency demonstrating the effectiveness of our method.
Removing Geometric Bias in One-Class Anomaly Detection with Adaptive Feature Perturbation
Authors: Romain Hermary; Vincent Gaudilliere; Abd El Rahman Shabayek; Djamila Aouada
Venue: WACV 2025 (main)

Abstract: One-class anomaly detection aims to detect objects that do not belong to a predefined normal class. In practice training data lack those anomalous samples; hence state-of-the-art methods are trained to discriminate between normal and synthetically-generated pseudo-anomalous data. Most methods use data augmentation techniques on normal images to simulate anomalies. However the best-performing ones implicitly leverage a geometric bias present in the benchmarking datasets. This limits their usability in more general conditions. Others are relying on basic noising schemes that may be suboptimal in capturing the underlying structure of normal data. In addition most still favour the image domain to generate pseudo-anomalies training models end-to-end from only the normal class and overlooking richer representations of the information. To overcome these limitations we consider frozen yet rich feature spaces given by pretrained models and create pseudo-anomalous features with a novel adaptive linear feature perturbation technique. It adapts the noise distribution to each sample applies decaying linear perturbations to feature vectors and further guides the classification process using a contrastive learning objective. Experimental evaluation conducted on both standard and geometric bias-free datasets demonstrates the superiority of our approach with respect to comparable baselines. The codebase is accessible via our public repository.
CryoMAE: Few-Shot Cryo-EM Particle Picking with Masked Autoencoders
Authors: Chentianye Xu; Xueying Zhan; Min Xu
Venue: WACV 2025 (main)

Abstract: Cryo-electron microscopy (cryo-EM) emerges as a pivotal technology for determining the architecture of cells viruses and protein assemblies at near-atomic resolution. Traditional particle picking a key step in cryo-EM struggles with manual effort and automated methods' sensitivity to low signal-to-noise ratio (SNR) and varied particle orientations. Furthermore existing neural network (NN)-based approaches often require extensive labeled datasets limiting their practicality. To overcome these obstacles we introduce cryoMAE a novel approach based on few-shot learning that harnesses the capabilities of Masked Autoencoders (MAE) to enable efficient selection of single particles in cryo-EM images. Contrary to conventional NN-based techniques cryoMAE requires only a minimal set of positive particle images for training yet demonstrates high performance in particle detection. Furthermore the implementation of a self-cross similarity loss ensures distinct features for particle and background regions thereby enhancing the discrimination capability of cryoMAE. Experiments on large-scale cryo-EM datasets show that cryoMAE outperforms existing state-of-the-art (SOTA) methods improving 3D reconstruction resolution by up to 22.4%. Our code is available at: https://github.com/xulabs/aitom.
BroadTrack: Broadcast Camera Tracking for Soccer
Authors: Floriane Magera; Thomas Hoyoux; Olivier Barnich; Marc Van Droogenbroeck
Venue: WACV 2025 (main)

Abstract: Camera calibration and localization sometimes simply named camera calibration enables many applications in the context of soccer broadcasting for instance regarding the interpretation and analysis of the game or the insertion of augmented reality graphics for storytelling or refereeing purposes. To contribute to such applications the research community has typically focused on single-view calibration methods leveraging the near-omnipresence of soccer field markings in wide-angle broadcast views but leaving all temporal aspects if considered at all to general-purpose tracking or filtering techniques. Only a few contributions have been made to leverage any domain-specific knowledge for this tracking task and as a result there lacks a truly performant and off-the-shelf camera tracking system tailored for soccer broadcasting specifically for elevated tripod-mounted cameras around the stadium. In this work we present such a system capable of addressing the task of soccer broadcast camera tracking efficiently robustly and accurately outperforming by far the most precise methods of the state-of-the-art. By combining the available open-source soccer field detectors with carefully designed camera and tripod models our tracking system BroadTrack halves the mean reprojection error rate and gains more than 15% in terms of Jaccard index for camera calibration on the SoccerNet dataset. Furthermore as the SoccerNet dataset videos are relatively short (30 seconds) we also present qualitative results on a 20-minute broadcast clip to showcase the robustness and the soundness of our system.
Negative-Prompt Inversion: Fast Image Inversion for Editing with Text-Guided Diffusion Models
Authors: Daiki Miyake; Akihiro Iohara; Yu Saito; Toshiyuki Tanaka
Venue: WACV 2025 (main)

Abstract: In image editing employing diffusion models it is crucial to preserve the reconstruction fidelity to the original image while changing its style. Although existing methods ensure reconstruction fidelity through optimization a drawback of these is the significant amount of time required for optimization. In this paper we propose negative-prompt inversion a method capable of achieving equivalent reconstruction solely through forward propagation without optimization thereby enabling ultrafast editing processes. We experimentally demonstrate that the reconstruction fidelity of our method is comparable to that of existing methods allowing for inversion at a resolution of 512 pixels and with 50 sampling steps within approximately 5 seconds which is more than 30 times faster than null-text inversion. Reduction of the computation time by the proposed method further allows us to use a larger number of sampling steps in diffusion models to improve the reconstruction fidelity with a moderate increase in computation time.
EvoCL: Continual Learning over Evolving Domains
Authors: Vishnuprasadh Kumaravelu; P.K. Srijith; Sunil Gupta
Venue: WACV 2025 (main)

Abstract: Continual Learning aspires to build models capable of learning new tasks without forgetting previously learnt tasks. In real-world settings the distributions underlying the tasks are prone to shift. This necessitates a model capable of observing how the task distributions drift with time and adapt proactively. We present a novel framework of continual learning under evolving domains. Our approach employs a hypernetwork with separate embeddings conditioned on both domain and task to address this problem. The hypernetwork generates customised classifier weights corresponding to any domain-task pair. We employ a separate network that is trained end to end along with the hypernetwork to predict the next domain embedding which in turn helps to generate classifier parameters corresponding to the next future domain in the evolution. We conduct extensive experiments on various datasets with a wide variety of distribution shifts to demonstrate the efficacy of our model in generalizing to future domains across all the tasks.
Adaptive and Temporally Consistent Gaussian Surfels for Multi-View Dynamic Reconstruction
Authors: Decai Chen; Brianne Oberson; Ingo Feldmann; Oliver Schreer; Anna Hilsmann; Peter Eisert
Venue: WACV 2025 (main)

Abstract: 3D Gaussian Splatting has recently achieved notable success in novel view synthesis for dynamic scenes and geometry reconstruction in static scenes. Building on these advancements early methods have been developed for dynamic surface reconstruction by globally optimizing entire sequences. However reconstructing dynamic scenes with significant topology changes emerging or disappearing objects and rapid movements remains a substantial challenge particularly for long sequences. To address these issues we propose AT-GS a novel method for reconstructing high-quality dynamic surfaces from multi-view videos through per-frame incremental optimization. To avoid local minima across frames we introduce a unified and adaptive gradient-aware densification strategy that integrates the strengths of conventional cloning and splitting techniques. Additionally we reduce temporal jittering in dynamic surfaces by ensuring consistency in curvature maps across consecutive frames. Our method achieves superior accuracy and temporal coherence in dynamic surface reconstruction delivering high-fidelity space-time novel view synthesis even in complex and challenging scenes. Extensive experiments on diverse multi-view video datasets demonstrate the effectiveness of our approach showing clear advantages over baseline methods.
ZeroComp: Zero-Shot Object Compositing from Image Intrinsics via Diffusion
Authors: Zitian Zhang; Frédéric Fortier-Chouinard; Mathieu Garon; Anand Bhattad; Jean-François Lalonde
Venue: WACV 2025 (main)

Abstract: We present ZeroComp an effective zero-shot 3D object compositing approach that does not require paired composite-scene images during training. Our method leverages ControlNet to condition from intrinsic images and combines it with a Stable Diffusion model to utilize its scene priors together operating as an effective rendering engine. During training ZeroComp uses intrinsic images based on geometry albedo and masked shading all without the need for paired images of scenes with and without composite objects. Once trained it seamlessly integrates virtual 3D objects into scenes adjusting shading to create realistic composites. We develop a high-quality evaluation dataset and demonstrate that ZeroComp outperforms methods using explicit lighting estimations and generative techniques in quantitative and human perception benchmarks. Additionally ZeroComp extends to real and outdoor image compositing even when trained solely on synthetic indoor data showcasing its effectiveness in image compositing.
Needles & Haystacks: Dataset and Benchmark for Domain-Agnostic Image-Based Rigid Slice-to-Volume Registration
Authors: Anton Frolov; Florian Kleiner; Christiane Rößler; Volker Rodehorst
Venue: WACV 2025 (main)

Abstract: We address domain-agnostic slice-to-volume (S2V) registration the alignment of 2D sliced/tomographic images into 3D volumes without prior knowledge of structure shape or orientation. While S2V registration is well-studied in medical imaging which often relies on auxiliary information (e.g. landmarks segmentation masks pre-defined orientations canonical/atlas volumes) applications such as micro-structure characterization in materials science lack such domain-specific aids. This leaves the task inherently ill-posed due to noise unstructured regions repetitive patterns rotational and translational symmetries. To address this challenge we present "Needles & Haystacks" a novel multi-domain algorithm development dataset with 158436 unique registration problems and ground-truth solutions based on diverse and openly licensed real-world volumetric data. Additionally we provide an online platform with 8461 test problems for reproducible evaluation of competing methods. We also propose strong baseline solutions with public implementations and highlight opportunities for further algorithmic advancements.
Retrieval Augmented Recipe Generation
Authors: Guoshan Liu; Hailong Yin; Bin Zhu; Jingjing Chen; Chong-Wah Ngo; Yu-Gang Jiang
Venue: WACV 2025 (main)

Abstract: The growing interest in generating recipes from food images has drawn substantial research attention in recent years. Existing works for recipe generation primarily utilize a two-stage training method--first predicting ingredients from a food image and then generating instructions from both the image and ingredients. Large Multi-modal Models (LMMs) which have achieved notable success across a variety of vision and language tasks shed light on generating both ingredients and instructions directly from images. Nevertheless LMMs still face the common issue of hallucinations during recipe generation leading to suboptimal performance. To tackle this issue we propose a retrieval augmented large multimodal model for recipe generation. We first introduce Stochastic Diversified Retrieval Augmentation (SDRA) to retrieve recipes semantically related to the image from an existing datastore as a supplement integrating them into the prompt to add diverse and rich context to the input image. Additionally Self-Consistency Ensemble Voting mechanism is proposed to determine the most confident prediction recipes as the final output. It calculates the consistency among generated recipe candidates which use different retrieval recipes as context for generation. Extensive experiments validate the effectiveness of our proposed method which demonstrates state-of-the-art (SOTA) performance in recipe generation on the Recipe1M dataset.
Enriching Local Patterns with Multi-Token Attention for Broad-Sight Neural Networks
Authors: Hankyul Kang; Jongbin Ryu
Venue: WACV 2025 (main)

Abstract: In neural networks recognizing visual patterns is challenging because global average pooling disregards local patterns and solely relies on over-concentrated activation. Global average pooling enforces the network to learn objects regardless of their location so features tend to be activated only in specific regions. To support this claim we provide a novel analysis of the problems that over-concentration brings about in networks with extensive experiments. We analyze the over-concentration through problems arising from feature variance and dead neurons that are not activated. Based on our analysis we introduce a multi-token attention pooling layer to alleviate the over-concentration problem. Our attention-pooling layer captures broad-sight local patterns by learning multiple tokens with the proposed distillation algorithm. It resolves the high bias and high variance errors of learned multi-tokens which is crucial when aggregating local patterns with multi-tokens. Our method applies to various vision tasks and network architectures such as CNN ViT and MLP-Mixer. The proposed method improves baselines with few extra resources and a network employing our pooling method works favorably against state-of-the-art networks. We open-source the code at https://github.com/Lab-LVM/imagenet-models.
Data-Efficient 3D Visual Grounding via Order-Aware Referring
Authors: Tung-Yu Wu; Sheng-Yu Huang; Yu-Chiang Frank Wang
Venue: WACV 2025 (main)

Abstract: 3D visual grounding aims to identify the target object within a 3D point cloud scene referred to by a natural language description. Previous works usually require significant data relating to point color and their descriptions to exploit the corresponding complicated verbo-visual relations. In our work we introduce Vigor a novel Data-Efficient 3D Visual Grounding framework via Order-aware Referring. Vigor leverages LLM to produce a desirable referential order from the input description for 3D visual grounding. With the proposed stacked object-referring blocks the predicted anchor objects in the above order allow one to locate the target object progressively without supervision on the identities of anchor objects or exact relations between anchor/target objects. We also present an order-aware warm-up training strategy which augments referential orders for pre-training the visual grounding framework allowing us to better capture the complex verbo-visual relations and benefit the desirable data-efficient learning scheme. Experimental results on the NR3D and ScanRefer datasets demonstrate our superiority in low-resource scenarios. In particular Vigor surpasses current state-of-the-art frameworks by 9.3% and 7.6% grounding accuracy under 1% data and 10% data settings on the NR3D dataset respectively.
Boosting Semi-Supervised Video Action Detection with Temporal Context
Authors: Donghyeon Kwon; Inho Kim; Suha Kwak
Venue: WACV 2025 (main)

Abstract: This paper studies semi-supervised learning of video action detection (VAD) which assumes that only a small portion of training videos are labeled and the others remain unlabeled. The existing semi-supervised methods for VAD mainly focus on leveraging spatial context of unlabeled video lacking its exploration of temporal context. To resolve this we present a novel semi-supervised learning framework that effectively incorporates spatio-temporal context during training. We first introduce a new augmentation strategy called temporal cross-view augmentation to achieve robust representation across clips depicting the same action but not aligned on the time axis. We also propose a new context fusion method called global-local context fusion that effectively utilizes the spatio-temporal context of videos to enhances the features of each frame by incorporating those of other frames within a clip; this method aids in actively leveraging spatio-temporal context of video leading to significant performance improvement. Our framework was evaluated on UCF101-24 and JHMDB-21 where it outperformed all existing methods in every evaluation setting.
CCASeg: Decoding Multi-Scale Context with Convolutional Cross-Attention for Semantic Segmentation
Authors: Jiwon Yoo; Dami Ko; Gyeonghwan Kim
Venue: WACV 2025 (main)

Abstract: Capturing multi-scale context within feature maps is crucial for semantic segmentation. With the success of the Vision Transformer (ViT) recent models have been designed with transformer decoders to capture it. However these models face limitations in utilizing diverse contextual information due to the inherent nature of the attention mechanism and structural constraints. Typically multi-head attention which leads to similar receptive fields for each token feture is achieved at the expense of significantly increased computational cost. The nature of the structure can cause inconsistent combination of the information across different levels. To address this issue in this paper we propose a novel and effective decoding scheme CCASeg which is based on convolutional cross-attention (CCA). The proposed CCA along with the decoding structure is devised not only to capture both local and global context through convolutional kernels of various sizes but also to achieve high efficiency by effective utilization of the cheap convolution operations. Moreover the decoding structure which ensures the successive combination of information across various levels facilitates understanding of diverse contexts. Consequently this novel decoding scheme enables feature maps to effectively learn the relationships between objects of different sizes. In this way our proposed CCASeg outperforms previous state-of-the-art methods on popular semantic segmentation benchmarks including ADE20K Cityscapes COCO-stuff and iSAID.
STLight: A Fully Convolutional Approach for Efficient Predictive Learning by Spatio-Temporal Joint Processing
Authors: Andrea Alfarano; Alberto Alfarano; Linda Friso; Andrea Bacciu; Irene Amerini; Fabrizio Silvestri
Venue: WACV 2025 (main)

Abstract: Spatio-Temporal predictive Learning is a self-supervised learning paradigm that enables models to identify spatial and temporal patterns by predicting future frames based on past frames. Traditional methods which use recurrent neural networks to capture temporal patterns have proven their effectiveness but come with high system complexity and computational demand. Convolutions could offer a more efficient alternative but are limited by their characteristic of treating all previous frames equally resulting in poor temporal characterization and by their local receptive field limiting the capacity to capture distant correlations among frames. In this paper we propose STLight a novel method for spatio-temporal learning that relies solely on channel-wise and depth-wise convolutions as learnable layers. STLight overcomes the limitations of traditional convolutional approaches by rearranging spatial and temporal dimensions together using a single convolution to mix both types of features into a comprehensive spatio-temporal patch representation. This representation is then processed in a purely convolutional framework capable of focusing simultaneously on the interaction among near and distant patches and subsequently allowing for efficient reconstruction of the predicted frames. Our architecture achieves state-of-the-art performance on STL benchmarks across different datasets and settings while significantly improving computational efficiency in terms of parameters and computational FLOPs. The code is publicly available
Active Learning for Image Segmentation with Binary User Feedback
Authors: Debanjan Goswami; Shayok Chakraborty
Venue: WACV 2025 (main)

Abstract: Deep learning algorithms have depicted commendable performance in a variety of computer vision applications. However training a robust deep neural network necessitates a large amount of labeled training data which is time-consuming and labor-intensive to acquire. This problem is even more serious for an application like image segmentation as the human oracle has to hand-annotate each and every pixel in a given training image which is extremely laborious. Active learning algorithms automatically identify the salient and exemplar samples from large amounts of unlabeled data and tremendously reduce human annotation effort in inducing a machine learning model. In this paper we propose a novel active learning algorithm for image segmentation with the goal of further reducing the labeling burden on the human oracles. Our framework identifies a batch of informative images together with a list of semantic classes for each and the human annotator merely needs to answer whether a given semantic class is present or absent in a given image. To the best of our knowledge this is the first research effort to develop an active learning framework for image segmentation which poses only binary (yes/no) queries to the users. We pose the image and class selection as a constrained optimization problem and derive a linear programming relaxation to select a batch of (image-class) pairs which are maximally informative to the underlying deep neural network. Our extensive empirical studies on three challenging datasets corroborate the potential of our method in substantially reducing human annotation effort for real-world image segmentation applications.