pwc-archive/papers-with-abstracts huggingface dataset:
TransVFC: A Transformable Video Feature Compression Framework for Machines
Authors: Yuxiao Sun, Yao Zhao, Meiqin Liu, Chao Yao, Huihui Bai, Chunyu Lin, Weisi Lin
Date: 2025-03-31 00:00:00
Tasks: Feature Compression

Nowadays, more and more video transmissions primarily aim at downstream machine vision tasks rather than humans. While widely deployed Human Visual System (HVS) oriented video coding standards like H.265/HEVC and H.264/AVC are efficient, they are not the optimal approaches for Video Coding for Machines (VCM) scenarios, leading to unnecessary bitrate expenditure. The academic and technical exploration within the VCM domain has led to the development of several strategies, and yet, conspicuous limitations remain in their adaptability for multi-task scenarios. To address the challenge, we propose a Transformable Video Feature Compression (TransVFC) framework. It offers a compress-then-transfer solution and includes a video feature codec and Feature Space Transform (FST) modules. In particular, the temporal redundancy of video features is squeezed by the codec through the scheme-based inter-prediction module. Then, the codec implements perception-guided conditional coding to minimize spatial redundancy and help the reconstructed features align with downstream machine perception.After that, the reconstructed features are transferred to new feature spaces for diverse downstream tasks by FST modules. To accommodate a new downstream task, it only requires training one lightweight FST module, avoiding retraining and redeploying the upstream codec and downstream task networks. Experiments show that TransVFC achieves high rate-task performance for diverse tasks of different granularities. We expect our work can provide valuable insights for video feature compression in multi-task scenarios. The codes are at https://github.com/Ws-Syx/TransVFC.
OncoReg: Medical Image Registration for Oncological Challenges
Authors: Wiebke Heyer, Yannic Elser, Lennart Berkel, Xinrui Song, Xuanang Xu, Pingkun Yan, Xi Jia, Jinming Duan, Zi Li, Tony C. W. Mok, Bowen Li, Christian Staackmann, Christoph Großbröhmer, Lasse Hansen, Alessa Hering, Malte M. Sieren, Mattias P. Heinrich
Date: 2025-03-29 00:00:00
Tasks: Image Registration, Medical Image Registration

In modern cancer research, the vast volume of medical data generated is often underutilised due to challenges related to patient privacy. The OncoReg Challenge addresses this issue by enabling researchers to develop and validate image registration methods through a two-phase framework that ensures patient privacy while fostering the development of more generalisable AI models. Phase one involves working with a publicly available dataset, while phase two focuses on training models on a private dataset within secure hospital networks. OncoReg builds upon the foundation established by the Learn2Reg Challenge by incorporating the registration of interventional cone-beam computed tomography (CBCT) with standard planning fan-beam CT (FBCT) images in radiotherapy. Accurate image registration is crucial in oncology, particularly for dynamic treatment adjustments in image-guided radiotherapy, where precise alignment is necessary to minimise radiation exposure to healthy tissues while effectively targeting tumours. This work details the methodology and data behind the OncoReg Challenge and provides a comprehensive analysis of the competition entries and results. Findings reveal that feature extraction plays a pivotal role in this registration task. A new method emerging from this challenge demonstrated its versatility, while established approaches continue to perform comparably to newer techniques. Both deep learning and classical approaches still play significant roles in image registration, with the combination of methods - particularly in feature extraction - proving most effective.
Improved Motion Plane Adaptive 360-Degree Video Compression Using Affine Motion Models
Authors: Marina Ritthaler, Andy Regensky, André Kaup
Date: 2025-03-29 00:00:00
Tasks: Motion Compensation, Motion Estimation, Video Compression

Efficient compression of 360-degree video content requires the application of advanced motion models for interframe prediction. The Motion Plane Adaptive (MPA) motion model projects the frames on multiple perspective planes in the 3D space. It improves the motion compensation by estimating the motion on those planes with a translational diamond search. In this work, we enhance this motion model with an affine parameterization and motion estimation method. Thereby, we find a feasible trade-off between the quality of the reconstructed frames and the computational cost. The affine motion estimation is hereby done with the inverse compositional Lucas-Kanade algorithm. With the proposed method, it is possible to improve the motion compensation significantly, so that the motion compensated frame has a Weighted-to-Spherically-uniform Peak Signal-to-Noise Ratio (WS-PSNR) which is about 1.6 dB higher than with the conventional MPA. In a basic video codec, the improved inter prediction can lead to Bj{\o}ntegaard Delta (BD) rate savings between 9 % and 35 % depending on the block size (BS) and number of motion parameters.
Towards Interpretable Counterfactual Generation via Multimodal Autoregression
Authors: Chenglong Ma, Yuanfeng Ji, Jin Ye, Lu Zhang, Ying Chen, Tianbin Li, Mingjie Li, Junjun He, Hongming Shan
Date: 2025-03-29 00:00:00
Tasks: counterfactual, Decision Making, Image Generation, Medical Image Generation

Counterfactual medical image generation enables clinicians to explore clinical hypotheses, such as predicting disease progression, facilitating their decision-making. While existing methods can generate visually plausible images from disease progression prompts, they produce silent predictions that lack interpretation to verify how the generation reflects the hypothesized progression -- a critical gap for medical applications that require traceable reasoning. In this paper, we propose Interpretable Counterfactual Generation (ICG), a novel task requiring the joint generation of counterfactual images that reflect the clinical hypothesis and interpretation texts that outline the visual changes induced by the hypothesis. To enable ICG, we present ICG-CXR, the first dataset pairing longitudinal medical images with hypothetical progression prompts and textual interpretations. We further introduce ProgEmu, an autoregressive model that unifies the generation of counterfactual images and textual interpretations. We demonstrate the superiority of ProgEmu in generating progression-aligned counterfactuals and interpretations, showing significant potential in enhancing clinical decision support and medical education. Project page: https://progemu.github.io.
ShiftLIC: Lightweight Learned Image Compression with Spatial-Channel Shift Operations
Authors: Youneng Bao, Wen Tan, Chuanmin Jia, Mu Li, Yongsheng Liang, Yonghong Tian
Date: 2025-03-29 00:00:00
Tasks: Image Compression

Learned Image Compression (LIC) has attracted considerable attention due to their outstanding rate-distortion (R-D) performance and flexibility. However, the substantial computational cost poses challenges for practical deployment. The issue of feature redundancy in LIC is rarely addressed. Our findings indicate that many features within the LIC backbone network exhibit similarities. This paper introduces ShiftLIC, a novel and efficient LIC framework that employs parameter-free shift operations to replace large-kernel convolutions, significantly reducing the model's computational burden and parameter count. Specifically, we propose the Spatial Shift Block (SSB), which combines shift operations with small-kernel convolutions to replace large-kernel. This approach maintains feature extraction efficiency while reducing both computational complexity and model size. To further enhance the representation capability in the channel dimension, we propose a channel attention module based on recursive feature fusion. This module enhances feature interaction while minimizing computational overhead. Additionally, we introduce an improved entropy model integrated with the SSB module, making the entropy estimation process more lightweight and thereby comprehensively reducing computational costs. Experimental results demonstrate that ShiftLIC outperforms leading compression methods, such as VVC Intra and GMM, in terms of computational cost, parameter count, and decoding latency. Additionally, ShiftLIC sets a new SOTA benchmark with a BD-rate gain per MACs/pixel of -102.6\%, showcasing its potential for practical deployment in resource-constrained environments. The code is released at https://github.com/baoyu2020/ShiftLIC.
Deep learning-enabled prediction of surgical errors during cataract surgery: from simulation to real-world application
Authors: Maxime Faure, Pierre-Henri Conze, Béatrice Cochener, Anas-Alexis Benyoussef, Mathieu Lamard, Gwenolé Quellec
Date: 2025-03-28 00:00:00
Tasks: Domain Adaptation, Prediction, Unsupervised Domain Adaptation

Real-time prediction of technical errors from cataract surgical videos can be highly beneficial, particularly for telementoring, which involves remote guidance and mentoring through digital platforms. However, the rarity of surgical errors makes their detection and analysis challenging using artificial intelligence. To tackle this issue, we leveraged videos from the EyeSi Surgical cataract surgery simulator to learn to predict errors and transfer the acquired knowledge to real-world surgical contexts. By employing deep learning models, we demonstrated the feasibility of making real-time predictions using simulator data with a very short temporal history, enabling on-the-fly computations. We then transferred these insights to real-world settings through unsupervised domain adaptation, without relying on labeled videos from real surgeries for training, which are limited. This was achieved by aligning video clips from the simulator with real-world footage and pre-training the models using pretext tasks on both simulated and real surgical data. For a 1-second prediction window on the simulator, we achieved an overall AUC of 0.820 for error prediction using 600$\times$600 pixel images, and 0.784 using smaller 299$\times$299 pixel images. In real-world settings, we obtained an AUC of up to 0.663 with domain adaptation, marking an improvement over direct model application without adaptation, which yielded an AUC of 0.578. To our knowledge, this is the first work to address the tasks of learning surgical error prediction on a simulator using video data only and transferring this knowledge to real-world cataract surgery.
Coupled Video Frame Interpolation and Encoding with Hybrid Event Cameras for Low-Power High-Framerate Video
Authors: Hidekazu Takahashi, Takefumi Nagumo, Kensei Jo, Aumiller Andreas, Saeed Rad, Rodrigo Caye Daudt, Yoshitaka Miyatani, Hayato Wakabayashi, Christian Brandli
Date: 2025-03-28 00:00:00
Tasks: Event-based vision, Video Frame Interpolation

Every generation of mobile devices strives to capture video at higher resolution and frame rate than previous ones. This quality increase also requires additional power and computation to capture and encode high-quality media. We propose a method to reduce the overall power consumption for capturing high-quality videos in mobile devices. Using video frame interpolation (VFI), sensors can be driven at lower frame rate, which reduces sensor power consumption. With modern RGB hybrid event-based vision sensors (EVS), event data can be used to guide the interpolation, leading to results of much higher quality. If applied naively, interpolation methods can be expensive and lead to large amounts of intermediate data before video is encoded. This paper proposes a video encoder that generates a bitstream for high frame rate video without explicit interpolation. The proposed method estimates encoded video data (notably motion vectors) rather than frames. Thus, an encoded video file can be produced directly without explicitly producing intermediate frames.
Benchmarking Deep Learning-Based Methods for Irradiance Nowcasting with Sky Images
Authors: Lorenzo F. C. Varaschin, Danilo Silva
Date: 2025-03-27 00:00:00
Tasks: Benchmarking

To address the high levels of uncertainty associated with photovoltaic energy, an increasing number of studies focusing on short-term solar forecasting have been published. Most of these studies use deep learning-based models to directly forecast a solar irradiance or photovoltaic power value given an input of sky image sequences. Recently, however, advances in generative modeling have led to approaches that divide the forecasting problem into two sub-problems: 1) future event prediction, i.e. generating future sky images; and 2) solar irradiance or photovoltaic power nowcasting, i.e. predicting the concurrent value from a single image. One such approach is the SkyGPT model, where they show that the potential for improvement is much larger for the nowcasting model than for the generative model. Thus, in this paper, we focus on the solar irradiance nowcasting problem and conduct an extensive benchmark of deep learning architectures across the widely-used Folsom, SIRTA and NREL datasets. Moreover, we perform ablation experiments on different training configurations and data processing techniques, including the choice of the target variable used for training and adjustments of the timestamp alignment between images and irradiance measurements. In particular, we draw attention to a potential error associated with the sky image timestamps in the Folsom dataset and a possible fix is discussed. All our results are reported in terms of both the root mean squared error and the mean absolute error and, by leveraging the three datasets, we demonstrate that our findings are consistent across different solar stations.
Implicit neural representations for end-to-end PET reconstruction
Authors: Younès Moussaoui, Diana Mateus, Nasrin Taheri, Saïd Moussaoui, Thomas Carlier, Simon Stute
Date: 2025-03-26 00:00:00
Tasks: Denoising, Image Reconstruction

Implicit neural representations (INRs) have demonstrated strong capabilities in various medical imaging tasks, such as denoising, registration, and segmentation, by representing images as continuous functions, allowing complex details to be captured. For image reconstruction problems, INRs can also reduce artifacts typically introduced by conventional reconstruction algorithms. However, to the best of our knowledge, INRs have not been studied in the context of PET reconstruction. In this paper, we propose an unsupervised PET image reconstruction method based on the implicit SIREN neural network architecture using sinusoidal activation functions. Our method incorporates a forward projection model and a loss function adapted to perform PET image reconstruction directly from sinograms, without the need for large training datasets. The performance of the proposed approach was compared with that of conventional penalized likelihood methods and deep image prior (DIP) based reconstruction using brain phantom data and realistically simulated sinograms. The results show that the INR-based approach can reconstruct high-quality images with a simpler, more efficient model, offering improvements in PET image reconstruction, particularly in terms of contrast, activity recovery, and relative bias.
UFM: Unified Feature Matching Pre-training with Multi-Modal Image Assistants
Authors: Yide Di, Yun Liao, Hao Zhou, Kaijun Zhu, Qing Duan, Junhui Liu, Mingyu Lu
Date: 2025-03-26 00:00:00
Tasks: Data Augmentation

Image feature matching, a foundational task in computer vision, remains challenging for multimodal image applications, often necessitating intricate training on specific datasets. In this paper, we introduce a Unified Feature Matching pre-trained model (UFM) designed to address feature matching challenges across a wide spectrum of modal images. We present Multimodal Image Assistant (MIA) transformers, finely tunable structures adept at handling diverse feature matching problems. UFM exhibits versatility in addressing both feature matching tasks within the same modal and those across different modals. Additionally, we propose a data augmentation algorithm and a staged pre-training strategy to effectively tackle challenges arising from sparse data in specific modals and imbalanced modal datasets. Experimental results demonstrate that UFM excels in generalization and performance across various feature matching tasks. The code will be released at:https://github.com/LiaoYun0x0/UFM.
Deep Learning-Based Quantitative Assessment of Renal Chronicity Indices in Lupus Nephritis
Authors: Tianqi Tu, Hui Wang, Jiangbo Pei, Xiaojuan Yu, Aidong Men, Suxia Wang, Qingchao Chen, Ying Tan, Feng Yu, Minghui Zhao
Date: 2025-03-26 00:00:00

Background: Renal chronicity indices (CI) have been identified as strong predictors of long-term outcomes in lupus nephritis (LN) patients. However, assessment by pathologists is hindered by challenges such as substantial time requirements, high interobserver variation, and susceptibility to fatigue. This study aims to develop an effective deep learning (DL) pipeline that automates the assessment of CI and provides valuable prognostic insights from a disease-specific perspective. Methods: We curated a dataset comprising 282 slides obtained from 141 patients across two independent cohorts with a complete 10-years follow-up. Our DL pipeline was developed on 60 slides (22,410 patch images) from 30 patients in the training cohort and evaluated on both an internal testing set (148 slides, 77,605 patch images) and an external testing set (74 slides, 27,522 patch images). Results: The study included two cohorts with slight demographic differences, particularly in age and hemoglobin levels. The DL pipeline showed high segmentation performance across tissue compartments and histopathologic lesions, outperforming state-of-the-art methods. The DL pipeline also demonstrated a strong correlation with pathologists in assessing CI, significantly improving interobserver agreement. Additionally, the DL pipeline enhanced prognostic accuracy, particularly in outcome prediction, when combined with clinical parameters and pathologist-assessed CIs Conclusions: The DL pipeline demonstrated accuracy and efficiency in assessing CI in LN, showing promise in improving interobserver agreement among pathologists. It also exhibited significant value in prognostic analysis and enhancing outcome prediction in LN patients, offering a valuable tool for clinical decision-making.
Impact of Synchronization Offsets and CSI Feedback Delay in Distributed MIMO Systems
Authors: Kumar Sai Bondada, Daniel Jakubisin, R. Michael Buehrer
Date: 2025-03-31 00:00:00

The main challenges of distributed MIMO systems lie in achieving highly accurate synchronization and ensuring the availability of accurate channel state information (CSI) at distributed nodes. This paper analytically examines the effects of synchronization offsets and CSI feedback delays on system capacity, providing insights into how these affect the coherent joint transmission gain. The capacity expressions are first derived under ideal conditions, and the effects of synchronization offsets and feedback delays are subsequently incorporated. This analysis can be applied to any distributed MIMO architecture. A comprehensive study, including system models and simulations evaluating the analytical expressions, is presented to quantify the capacity degradation caused by these factors. This study provides valuable insights into the design and performance of distributed MIMO systems. The analysis shows that time and frequency offsets, along with CSI feedback delay, cause inter-layer interference. Additionally, time offsets result in inter-symbol interference.
Deep Learning-Based Data Fusion of 6G Sensing and Inertial Information for Target Positioning: Experimental Validation
Authors: Karthik Muthineni, Alexander Artemenko, Artjom Grudnitsky, Josep Vidal, Montse Najar
Date: 2025-03-31 00:00:00
Tasks: Integrated sensing and communication, ISAC

The sixth-generation (6G) cellular technology will be deployed with a key feature of Integrated Sensing and Communication (ISAC), allowing the cellular network to map the environment through radar sensing on top of providing communication services. In this regard, the entire network can be considered as a sensor with a broader Field of View (FoV) of the environment, assisting in both the positioning of active and detection of passive targets. On the other hand, the non-3GPP sensors available on the target can provide additional information specific to the target that can be beneficially combined with ISAC sensing information to enhance the overall achievable positioning accuracy. In this paper, we first study the performance of the ISAC system in terms of its achievable accuracy in positioning the mobile target in an indoor scenario. Second, we study the performance gain achieved in the ISAC positioning accuracy after fusing the information from the target's non-3GPP sensors. To this end, we propose a novel data fusion solution based on the deep learning framework to fuse the information from ISAC and non-3GPP sensors. We validate our proposed data fusion and positioning solution with a real-world ISAC Proof-of-Concept (PoC) as the wireless infrastructure, an Automated Guided Vehicle (AGV) as the target, and the Inertial Measurement Unit (IMU) sensor on the target as the non-3GPP sensor. The experimental results show that our proposed solution achieves an average positioning error of $3~\textrm{cm}$, outperforming the considered baselines.
Reinforcing Localization Credibility Through Convex Optimization
Authors: Slavisa Tomic, Marko Beko, Yakubu Tsado, Bamidele Adebisi, Abiola Oladipo
Date: 2025-03-31 00:00:00

This work proposes a novel approach to reinforce localization security in wireless networks in the presence of malicious nodes that are able to manipulate (spoof) radio measurements. It substitutes the original measurement model by another one containing an auxiliary variance dilation parameter that disguises corrupted radio links into ones with large noise variances. This allows for relaxing the non-convex maximum likelihood estimator (MLE) into a semidefinite programming (SDP) problem by applying convex-concave programming (CCP) procedure. The proposed SDP solution simultaneously outputs target location and attacker detection estimates, eliminating the need for further application of sophisticated detectors. Numerical results corroborate excellent performance of the proposed method in terms of localization accuracy and show that its detection rates are highly competitive with the state of the art.
Net 3.2 Tbps 225 Gbaud PAM4 O-Band IM/DD 2 km Transmission Using FR8 and DR8 with a CMOS 3 nm SerDes and TFLN Modulators
Authors: Charles St-Arnault, Santiago Bernal, Derek Kita, Ross Dickson, Mariam Yehia Abdelaziz, Aleksandar Nikic, Benton Qiu, Benjamin Krueger, Fabio Pittalà, Christian Reimer, Bruce Beggs, Naim Ben-Hamida, David V. Plant
Date: 2025-03-31 00:00:00

We report the first 3.2 and 4.2 Tbps (8 x 225Gbaud PAM4-8), IM/DD transmission system using FR8 and DR8 configurations with TFLN modulators driven by a 3nm SerDes under the HD-FEC threshold.