GenAI4ELab/papercli-papers huggingface dataset:
Improving Shift Invariance in Convolutional Neural Networks with Translation Invariant Polyphase Sampling
Authors: Sourajit Saha; Tejas Gokhale
Venue: WACV 2025 (main)

Abstract: Convolutional neural networks (CNNs) widely deployed in several applications contain downsampling operators in their pooling layers which have been observed to be sensitive to pixel-level shift affecting the robustness of CNNs. We study shift invariance through the lens of maximum sampling bias (MSB) and find MSB to be negatively correlated with shift invariance. Based on this insight we propose a learnable pooling operator called Translation Invariant Polyphase Sampling (TIPS) to reduce MSB and learn translation-invariant representations. TIPS results in consistent performance gains on multiple benchmarks for image classification object detection and semantic segmentation in terms of accuracy shift consistency shift fidelity as well as improvements in adversarial and distributional robustness. TIPS results in the lowest MSB compared to all previous methods thus explaining the strong empirical results. TIPS can be integrated into any CNN and can be trained end-to-end with marginal computational overhead. Code: https://github.com/sourajitcs/tips/
ELMGS: Enhancing Memory and Computation Scalability through Compression for 3D Gaussian Splatting
Authors: Muhammad Salman Ali; Sung-Ho Bae; Enzo Tartaglione
Venue: WACV 2025 (main)

Abstract: 3D models have recently been popularized by the potentiality of end-to-end training offered first by Neural Radiance Fields and most recently by 3D Gaussian Splatting models. The latter has the big advantage of naturally providing fast training convergence and high editability. However as the research around these is still in its infancy there is still a gap in the literature regarding the model's scalability. In this work we propose an approach enabling both memory and computation scalability of such models. More specifically we propose an iterative pruning strategy that removes redundant information encoded in the model. We also enhance compressibility for the model by including a differentiable quantization and entropy coding estimator in the optimization strategy. Our results on popular benchmarks showcase the effectiveness of the proposed approach and open the road to the broad deployability of such a solution even on resource-constrained devices.
Revisiting Disparity from Dual-Pixel Images: Physics-Informed Lightweight Depth Estimation
Authors: Teppei Kurita; Yuhi Kondo; Legong Sun; Takayuki Sasaki; Sho Nitta; Yasuhiro Hashimoto; Yoshinori Muramatsu; Yusuke Moriuchi
Venue: WACV 2025 (main)

Abstract: In this study we propose a high-performance disparity (depth) estimation method using dual-pixel (DP) images with few parameters. Conventional end-to-end deep-learning methods have many parameters but do not fully exploit disparity constraints which limits their performance. Therefore we propose a lightweight disparity estimation method based on a completion-based network that explicitly constrains disparity and learns the physical and systemic disparity properties of DP. By modeling the DP-specific disparity error parametrically and using it for sampling during training the network acquires the unique properties of DP and enhances robustness. This learning also allows us to use a common RGB-D dataset for training without a DP dataset which is labor-intensive to acquire. Furthermore we propose a non-learning-based refinement framework that efficiently handles inherent disparity expansion errors by appropriately refining the confidence map of the network output. As a result the proposed method achieved state-of-the-art results while reducing the overall system size to 1/5 of that of the conventional method even without using the DP dataset for training thereby demonstrating its effectiveness. The code and dataset are available on our project site.
Optimizing Dense Visual Predictions Through Multi-Task Coherence and Prioritization
Authors: Maxime Fontana; Michael Spratling; Miaojing Shi
Venue: WACV 2025 (main)

Abstract: Multi-Task Learning (MTL) involves the concurrent training of multiple tasks offering notable advantages for dense prediction tasks in computer vision. MTL not only reduces training and inference time as opposed to having multiple single-task models but also enhances task accuracy through the interaction of multiple tasks. However existing methods face limitations. They often rely on suboptimal cross-task interactions resulting in task-specific predictions with poor geometric and predictive coherence. In addition many approaches use inadequate loss weighting strategies which do not address the inherent variability in task evolution during training. To overcome these challenges we propose an advanced MTL model specifically designed for dense vision tasks. Our model leverages state-of-the-art vision transformers with task-specific decoders. To enhance cross-task coherence we introduce a trace-back method that improves both cross-task geometric and predictive features. Furthermore we present a novel dynamic task balancing approach that projects task losses onto a common scale and prioritizes more challenging tasks during training. Extensive experiments demonstrate the superiority of our method establishing new state-of-the-art performance across two benchmark datasets. The code is available at: https://github.com/Klodivio355/MT-CP.
InvisMark: Invisible and Robust Watermarking for AI-Generated Image Provenance
Authors: Rui Xu; Mengya Hu; Deren Lei; Yaxi Li; David Lowe; Alex Gorevski; Mingyu Wang; Emily Ching; Alex Deng
Venue: WACV 2025 (main)

Abstract: The proliferation of AI-generated images has intensified the need for robust content authentication methods. We present InvisMark a novel watermarking technique designed for high-resolution AI-generated images. Our approach leverages advanced neural network architectures and training strategies to embed imperceptible yet highly robust watermarks. InvisMark achieves state-of-the-art performance in imperceptibility (PSNR 51 SSIM   0.998) while maintaining over 97% bit accuracy across various image manipulations. Notably we demonstrate the successful encoding of 256-bit watermarks significantly expanding payload capacity while preserving image quality. This enables the embedding of UUIDs with error correction codes achieving near-perfect decoding success rates even under challenging image distortions. We also address potential vulnerabilities against advanced attacks and propose mitigation strategies. By combining high imperceptibility extended payload capacity and resilience to manipulations InvisMark provides a robust foundation for ensuring media provenance in an era of increasingly sophisticated AI-generated content.
PocoLoco: A Point Cloud Diffusion Model of Human Shape in Loose Clothing
Authors: Siddharth Seth; Rishabh Dabral; Diogo C Luvizon; Marc Habermann; Ming-Hsuan Yang; Christian Theobalt; Adam Kortylewski
Venue: WACV 2025 (main)

Abstract: Modeling a human avatar that can plausibly deform to articulations is an active area of research. We present PocoLoco - the first template-free point-based pose-conditioned generative model for 3D humans in loose clothing. We motivate our work by noting that most methods require a parametric model of the human body to ground pose-dependent deformations. Consequently they are restricted to modeling clothing that is topologically similar to the naked body and do not extend well to loose clothing. The few methods that attempt to model loose clothing typically require either canonicalization or a UV-parameterization and need to address the challenging problem of explicitly estimating correspondences for the deforming clothes. In this work we formulate avatar clothing deformation as a conditional point-cloud generation task within the denoising diffusion framework. Crucially our framework operates directly on unordered point clouds eliminating the need for a parametric model or a clothing template. This also enables a variety of practical applications such as point-cloud completion and pose-based editing - important features for virtual human animation. As current datasets for human avatars in loose clothing are far too small for training diffusion models we release a dataset of two subjects performing various poses in loose clothing with a total of 75K point clouds. By contributing towards tackling the challenging task of effectively modeling loose clothing and expanding the available data for training these models we aim to set the stage for further innovation in digital humans. The source code is available at https: //github.com/sidsunny/pocoloco.
NeuManifold: Neural Watertight Manifold Reconstruction with Efficient and High-Quality Rendering Support
Authors: Xinyue Wei; Fanbo Xiang; Sai Bi; Anpei Chen; Kalyan Sunkavalli; Zexiang Xu; Hao Su
Venue: WACV 2025 (main)

Abstract: While existing volumetric rendering approaches provide photorealistic results extracting high-quality meshes from optimized neural field representations is challenging. Conversely existing differentiable rasterization-based methods are typically sensitive to initialization and suffer from poor mesh rendering quality. In this paper we introduce NeuManifold a novel method for reconstructing watertight manifold meshes with high-quality textures from multi-view input images. NeuManifold overcomes the limitations of existing approaches by first learning a neural volumetric field and then refining it through differentiable mesh extraction and surface rendering. To eliminate artifacts and preserve mesh properties during iso-surface extraction we introduce a novel differentiable marching cubes method. Instead of traditional textures we use neural textures to enhance rendering quality. To integrate with modern graphics rendering pipelines we also provide customized GLSL shader support for neural textures. Extensive experiments demonstrate that NeuManifold outperforms existing mesh-based reconstruction methods in both mesh quality and rendering metrics achieving comparable or superior rendering quality to prior volume-rendering-based methods. The generated results enable real-time high-quality rendering and seamlessly support numerous graphics pipelines and applications requiring high-quality meshes such as 3D printing and physical simulation. https://sarahweiii.github.io/neumanifold/.
Learning Visual-Semantic Hierarchical Attribute Space for Interpretable Open-Set Recognition
Authors: Zhuo Xu; Xiang Xiang
Venue: WACV 2025 (main)

Abstract: In the field of open-set recognition conventional models often focus on addressing challenges within a single hierarchical category and these methods frequently lack interpretability. In this paper we propose a novel solution that utilizes attributes and hierarchical relationships to achieve interpretable open-set recognition. Our method is centered around the visual-semantic attribute space. By leveraging hierarchy division we can decompose the attributes into more granular components thereby yielding additional performance improvements. When confronted with an unfamiliar object our method not only classifies it as an unknown category but also provides insights into the broader category and its associated attributes. This capability enhances interpretability by offering valuable information regarding the potential category and characteristics of the object. Experimental results demonstrate great performance improvements compared to existing methods.
Channel Propagation Networks for Refreshable Vision Transformer
Authors: Junhyeong Go; Jongbin Ryu
Venue: WACV 2025 (main)

Abstract: In this paper we introduce the Channel Propagation method which aims to increase the channels of the Vision Transformer systematically. Skip connections are commonly acknowledged as a propagation approach that improves the stability of the performance in Vision Transformers. Nevertheless it is important to note that these skip connections may give rise to the problem of over-smoothing wherein similar features are represented in multiple layers. To tackle this matter our proposed approach for Channel Propagation in Vision Transformers retains the present signal information while concurrently propagating location-specific signals in a newly introduced channel dimension. On the other hand the proposed Channel Propagation method effectively maintains the integrity of identity representation while simultaneously including patch-wise location-specific supervision by introducing a new channel dimension. The inclusion of this approach in Vision Transformers mitigates the issue of over-smoothing while also improving the performance of visual recognition tasks. In our experiments we confirm that the proposed method is effective for various visual recognition tasks. Specifically our method demonstrates enhanced performance when implemented on Vision Transformer models; the classification accuracy is increased considerably for plain and hierarchical architectures on the ImageNet dataset.
Swin-: Gradient-Based Image Restoration from Image Sequences using Video Swin-Transformers
Authors: Monika Kwiatkowski; Simon Matern; Olaf Hellwich
Venue: WACV 2025 (main)

Abstract: Most deep-learning models for vision tasks rely on RGB images as their primary input layer assuming the model inherently discovers an optimal representation. In this work we challenge this assumption and show that image gradients offer a straightforward yet robust representation for multi-frame image restoration. We demonstrate that clusters naturally emerge within gradient patches indicating improved estimation of the underlying signal. We develop a Video Swin-Transformer model operating in the gradient domain facilitated by the implementation of two differentiable gradient modules. One module computes image gradients using convolutions with gradient filters while the other reconstructs an RGB image from its gradient representation using deconvolution in the frequency domain. Additionally we employ a composite training loss that measures the error both in the color domain and its gradient counterpart. Applied to a multi-frame image restoration task involving the removal of lighting shadows and occlusions our model consistently outperforms RGB-based counterparts without introducing additional parameters thanks to its gradient regularization. We further apply our framework to various restoration tasks discussing its advantages and limitations. Qualitative results highlight the model's improved generalization to real-world video scenarios demonstrating successful adaptation from synthetic image training to real video data deployment.
Knockoff Branch: Model Stealing Attack via Adding Neurons in the Pre-Trained Model
Authors: Li-Ying Hung; Cooper Cheng-Yuan Ku
Venue: WACV 2025 (main)

Abstract: We introduce Knockoff Branch: adding few neurons as a knockoff container for learning stolen features. Model stealing attacks extract the functionality from the victim model by querying APIs. Prior work substantially enhanced transferability and improved query efficiency between the adversary model and the victim model. However there is still a limited understanding of the knockoff itself. For knockoff the model is either compared to the same type but with different structures or different types and capacities. For this reason we propose a framework to analyze the knockoff quality for a single model specifically reinvestigating transformer-based extraction. We observed that 1) when the adversary can access the public pretrained model full fine-tuning is not necessary. This allows a knockoff to require only about 0.5% of trainable parameters and 20 epochs. 2) Although querying by out-of-distribution datasets leads to a sub-optimal knockoff this issue can be mitigated by scaling branch features even without using complicated sampling strategies. Our proposed method is lightweight and achieves high accuracy at most similar to white-box knowledge distillation (higher performance than the victim model). https://github.com/onlyin-hung/knockoff-branch.
CLFace: A Scalable and Resource-Efficient Continual Learning Framework for Lifelong Face Recognition
Authors: Mahedi Hasan; Shoaib Meraj Sami; Nasser Nasrabadi
Venue: WACV 2025 (main)

Abstract: An important aspect of deploying face recognition (FR) algorithms in real-world applications is their ability to learn new face identities from a continuous data stream. However the online training of existing deep neural network-based FR algorithms which are pre-trained offline on large-scale stationary datasets encounter two major challenges: 1) catastrophic forgetting of previously learned identities and 2) the need to store past data for complete retraining from scratch leading to significant storage constraints and privacy concerns. In this paper we introduce CLFace a continual learning framework designed to preserve and incrementally extend the learned knowledge. CLFace eliminates the classification layer resulting in a resource-efficient FR model that remains fixed throughout lifelong learning and provides label-free supervision to a student model making it suitable for open-set face recognition during incremental steps. We introduce an objective function that employs feature-level distillation to reduce drift between feature maps of the student and teacher models across multiple stages. Additionally it incorporates a geometry-preserving distillation scheme to maintain the orientation of the teacher model's feature embedding. Furthermore a contrastive knowledge distillation is incorporated to continually enhance the discriminative power of the feature representation by matching similarities between new identities. Experiments on several benchmark FR datasets demonstrate that CLFace outperforms baseline approaches and state-of-the-art methods on unseen identities using both in-domain and out-of-domain datasets.
CardioSyntax: End-to-End SYNTAX Score Prediction - Dataset Benchmark and Method
Authors: Alexander Ponomarchuk; Ivan Kruzhilov; Gleb Mazanov; Ruslan Utegenov; Artem Shadrin; Galina Zubkova; Ivan Bessonov; Pavel Blinov
Venue: WACV 2025 (main)

Abstract: The SYNTAX score has become a widely used measure of coronary disease severity crucial in selecting the optimal mode of the revascularization procedure. This paper introduces a new medical regression and classification problem -- automatically estimating SYNTAX score from coronary angiography. Our study presents a comprehensive CardioSYNTAX dataset of 3018 patients for the SYNTAX score estimation and coronary dominance classification. The dataset features a balanced distribution of individuals with zero and non-zero scores. This dataset includes a first-of-its-kind complete coronary angiography samples captured through a multi-view X-ray video allowing one to observe coronary arteries from multiple perspectives. Furthermore we present a novel fully automatic end-to-end method for estimating the SYNTAX. For such a difficult task we have achieved a solid coefficient of determination R2 of 0.51 in score value prediction and 77.3% accuracy for zero score classification.
GroundingMate: Aiding Object Grounding for Goal-Oriented Vision-and-Language Navigation
Authors: Qianyi Liu; Siqi Zhang; Yanyuan Qiao; Junyou Zhu; Xiang Li; Longteng Guo; Qunbo Wang; Xingjian He; Qi Wu; Jing Liu
Venue: WACV 2025 (main)

Abstract: Goal-Oriented Vision-and-Language Navigation (VLN) aims to enable agents to navigate to specified locations and identify designated target objects following natural language instruction. This approach has gained popularity due to its close alignment with real-world scenarios. However existing studies have predominantly focused on enhancing navigation performance neglecting the ability to locate objects at the navigation endpoint. This oversight has resulted in a significant discrepancy between the success rates of navigation and object grounding. The challenge is compounded by the complex reasoning required by the instructions and the necessity to synthesize multi-perspective images of objects which overwhelms traditional object grounding methods. We leverage the Multi-Modal Large Language Model (MLLM) to bridge this gap allowing agents to seek assistance from these models when struggling to locate the target object. The agent conducts a multi-stage evaluation to discern the cause of its confusion and promptly extracts and updates the most relevant information for MLLM to assess. Our method is plug-and-play and model-agnostic facilitating integration with numerous existing VLN strategies without the need for retraining. Implementing our approach across four distinct methods has improved performance on the REVERIE and SOON datasets demonstrating the effectiveness and generalizability of our technique.
AH-OCDA: Amplitude-Based Curriculum Learning and Hopfield Segmentation Model for Open Compound Domain Adaptation
Authors: Jaehyun Choi; Junwon Ko; Dong-Jae Lee; Junmo Kim
Venue: WACV 2025 (main)

Abstract: Open compound domain adaptation (OCDA) is a practical domain adaptation problem that consists of a source domain target compound domain and unseen open domain. In this problem the absence of domain labels and pixel-level segmentation labels for both compound and open domains poses challenges to the direct application of existing domain adaptation and generalization methods. To address this issue we propose Amplitude-based curriculum learning and a Hopfield segmentation model for Open Compound Domain Adaptation (AH-OCDA). Our method comprises two complementary components: 1) amplitude-based curriculum learning and 2) Hopfield segmentation model. Without prior knowledge of target domains within the compound domains amplitude-based curriculum learning gradually induces the semantic segmentation model to adapt from the near-source compound domain to the far-source compound domain by ranking unlabeled compound domain images through Fast Fourier Transform (FFT). Additionally the Hopfield segmentation model maps segmentation feature distributions from arbitrary domains to the feature distributions of the source domain. AH-OCDA achieves state-of-the-art performance on two OCDA benchmarks and extended open domains demonstrating its adaptability to continuously changing compound domains and unseen open domains.
Image Adaptation for Colour Vision Deficient Viewers using Vision Transformers
Authors: Thomas Gillooly; Jean-Baptiste Thomas; Jon Y. Hardeberg; Giuseppe Claudio Guarnera
Venue: WACV 2025 (main)

Abstract: Colour Vision Deficiency (CVD) occurs when anomalous retinal cone spectral responses impact the ability to distinguish between certain colours. To enhance image quality and viewing experience recolouring algorithms seek to modify pixel values so that this does not lead to a loss of detail or image quality. Recent approaches to recolouring for CVD viewers employ neural models which exploit higher order features to direct colour adaptation. In this work we build upon the idea that visual neural models exhibit emergent behaviour which mimics the human visual system. We make use of these learned behaviours to guide the colour adaptation process by considering regions of the image that are the most semantically meaningful for a non-CVD viewer and compensate for them appropriately if they are absent or distorted in a CVD-simulated version of the image. We find that a minimal algorithm built atop a pre-trained model produces results that substantially boost contrast and salience for viewers affected by CVD. We also investigate a few cases where modifications are absent indicating that a neurally guided salience-based model may also provide a means of determining when recolouring is not necessary. Additionally we introduce a novel metric that quantifies the contrast increase or decrease under changes in image colour.