pwc-archive/papers-with-abstracts huggingface dataset:
RMFA-Net: A Neural ISP for Real RAW to RGB Image Reconstruction
Authors: Fei Li, Wenbo Hou, Peng Jia
Date: 2024-06-17 00:00:00
Tasks: Image Reconstruction, Tone Mapping

Deep learning-based ISP algorithms have demonstrated significant potential in raw2rgb reconstruction. However, existing networks have not fully considered the specific characteristics of raw data, such as black level and CFA, which can negatively impact texture and color if mishandled. Moreover, uneven exposure in raw data is also not considered carefully, leading to adverse effects on contrast and brightness. In this paper, we introduce RMFA-Net to tackle these problems. We perform implicit black level correction to mitigate color shifts in dim scenes. To preserve high-frequency information and prevent misalignment, we propose a novel Three-Channel-Split mode. To address the issue of uneven exposure, we designed an explicit tone mapping module based on the Retinex theory. We train and evaluate our models using the dataset released by the Mobile AI 2022 Learned Smartphone ISP Challenge. It is demonstrated that RMFA-Net outperforms previous algorithms, achieving a PSNR score of over 25 dB, surpassing the state-of-the-art by +1 dB. Furthermore, we developed a lightweight version, RMFANet-tiny, for engineering deployment while still maintaining strong performance, surpassing the SOTA by +0.5 dB.
Multispectral Snapshot Image Registration Using Learned Cross Spectral Disparity Estimation and a Deep Guided Occlusion Reconstruction Network
Authors: Frank Sippel, Jürgen Seiler, André Kaup
Date: 2024-06-17 00:00:00
Tasks: CPU, Data Augmentation, Disparity Estimation, GPU, Image Registration

Multispectral imaging aims at recording images in different spectral bands. This is extremely beneficial in diverse discrimination applications, for example in agriculture, recycling or healthcare. One approach for snapshot multispectral imaging, which is capable of recording multispectral videos, is by using camera arrays, where each camera records a different spectral band. Since the cameras are at different spatial positions, a registration procedure is necessary to map every camera to the same view. In this paper, we present a multispectral snapshot image registration with three novel components. First, a cross spectral disparity estimation network is introduced, which is trained on a popular stereo database using pseudo spectral data augmentation. Subsequently, this disparity estimation is used to accurately detect occlusions by warping the disparity map in a layer-wise manner. Finally, these detected occlusions are reconstructed by a learned deep guided neural network, which leverages the structure from other spectral components. It is shown that each element of this registration process as well as the final result is superior to the current state of the art. In terms of PSNR, our registration achieves an improvement of over 3 dB. At the same time, the runtime is decreased by a factor of over 3 on a CPU. Additionally, the registration is executable on a GPU, where the runtime can be decreased by a factor of 111. The source code and the data is available at https://github.com/FAU-LMS/MSIR.
On Efficient Neural Network Architectures for Image Compression
Authors: Yichi Zhang, Zhihao Duan, Fengqing Zhu
Date: 2024-06-14 00:00:00
Tasks: Efficient Neural Network, Image Compression

Recent advances in learning-based image compression typically come at the cost of high complexity. Designing computationally efficient architectures remains an open challenge. In this paper, we empirically investigate the impact of different network designs in terms of rate-distortion performance and computational complexity. Our experiments involve testing various transforms, including convolutional neural networks and transformers, as well as various context models, including hierarchical, channel-wise, and space-channel context models. Based on the results, we present a series of efficient models, the final model of which has comparable performance to recent best-performing methods but with significantly lower complexity. Extensive experiments provide insights into the design of architectures for learned image compression and potential direction for future research. The code is available at \url{https://gitlab.com/viper-purdue/efficient-compression}.
An accurate detection of micro-collapse during the lyophilisation of a 5% w/v lactose solution using a combination of novel techniques: intelligent laser speckle imaging (ILSI) and through-vial impedance spectroscopy (TVIS)
Authors: Ahmet Orun, Anand Vadesa, Geoff Smith
Date: 2024-06-11 00:00:00
Tasks: Texture Classification

Context: In a freeze drying (FD) process, an accurate observation and control of the process parameters at critical stages are at high importance. Particularly accurate and timely identification of the critical temperature (TC) at the end of primary drying phase would lead to heating energy cost reduction and product waste elimination by ending the process before the product's micro-collapse stage. Aim: Within this work, a combination of novel techniques optical technique called Intelligent Laser Speckle Imaging (ILSI) in association with product image texture analysis, coupled to an electrical impedance technique called through vial impedance spectroscopy (TVIS) has been applied onto the real-time product states observation process to identify the specific structural product surface/subsurface characteristics and hence the micro-collapse stage. Method: 2 cycles - one providing a profile for standard approach (non-collapsed) and another with a temperature ramp through TC to micro-collapse, TVIS provides an assessment of the onset of micro-collapse through the assessment of the acceleration in drying rate whereas the ILSI with pattern recognition detects the change in microstructure
Lightening Anything in Medical Images
Authors: Ben Fei, Yixuan Li, Weidong Yang, Hengjun Gao, Jingyi Xu, Lipeng Ma, Yatian Yang, Pinghong Zhou
Date: 2024-06-01 00:00:00
Tasks: Decision Making, Diagnostic, Image Enhancement, Medical Image Enhancement

The development of medical imaging techniques has made a significant contribution to clinical decision-making. However, the existence of suboptimal imaging quality, as indicated by irregular illumination or imbalanced intensity, presents significant obstacles in automating disease screening, analysis, and diagnosis. Existing approaches for natural image enhancement are mostly trained with numerous paired images, presenting challenges in data collection and training costs, all while lacking the ability to generalize effectively. Here, we introduce a pioneering training-free Diffusion Model for Universal Medical Image Enhancement, named UniMIE. UniMIE demonstrates its unsupervised enhancement capabilities across various medical image modalities without the need for any fine-tuning. It accomplishes this by relying solely on a single pre-trained model from ImageNet. We conduct a comprehensive evaluation on 13 imaging modalities and over 15 medical types, demonstrating better qualities, robustness, and accuracy than other modality-specific and data-inefficient models. By delivering high-quality enhancement and corresponding accuracy downstream tasks across a wide range of tasks, UniMIE exhibits considerable potential to accelerate the advancement of diagnostic tools and customized treatment plans.
Feasibility of Non-Line-of-Sight Integrated Sensing and Communication at mmWave
Authors: Paolo Tosi, Marcus Henninger, Lucas Giroto de Oliveira, Silvio Mandelli
Date: 2024-06-18 00:00:00
Tasks: Integrated sensing and communication, Intrusion Detection, ISAC

One rarely addressed direction in the context of Integrated Sensing and Communication (ISAC) is non-line-of-sight (NLOS) sensing, with the potential to enable use cases like intrusion detection and to increase the value that wireless networks can bring. However, ISAC networks impose challenges for sensing due to their communication-oriented design. For instance, time division duplex transmission creates spectral holes in time, resulting in spectral replicas in the radar image. To counteract this, we evaluate different channel state information processing strategies and discuss their tradeoffs. We further propose an ensemble of techniques to detect targets in NLOS conditions. Our approaches are validated with experiments using a millimeter wave ISAC proof of concept in a factory-like environment. The results show that target detection in NLOS is generally possible with ISAC.
Beyond Near-Field: Far-Field Location Division Multiple Access in Downlink MIMO Systems
Authors: Haoyan Liu, Caijian Jie, Min Yang, Chengguang Li
Date: 2024-06-18 00:00:00

Exploring channel dimensions has been the driving force behind breakthroughs in successive generations of mobile communication systems. In 5G, space division multiple access (SDMA) leveraging massive MIMO has been crucial in enhancing system capacity through spatial differentiation of users. However, SDMA can only finely distinguish users at adjacent angles in ultra-dense networks by extremely large-scale antenna arrays. For a long time, most research has focused on the angle domain of the space, overlooking the potential of the distance domain. Near-field location division multiple access (LDMA) was proposed based on the beam-focusing effect yielded by near-field spherical propagation model, partitioning channel resources by both angle and distance. To achieve a similar idea in the far-field region, this paper introduces a far-field LDMA scheme for wideband systems based on orthogonal frequency division multiplexing (OFDM). Benefiting from frequency diverse arrays (FDA), it becomes possible to manipulate beams in the distance domain. Combined with OFDM, the inherent cyclic prefix ensures a complete OFDM symbol can be received without losing distance information, while the matched filter of OFDM helps eliminate the time-variance of FDA steering vectors. Theoretical and simulation results show that LDMA can fully exploit the additional degrees of freedom in the distance domain to significantly improve spectral efficiency, especially in narrow sector multiple access (MA) scenarios. Moreover, LDMA can maintain independence between array elements even in single-path channels, making it stand out in MA schemes at millimeter-wave and higher frequency bands.
MSE Minimization in RIS-Aided MU-MIMO with Discrete Phase Shifts and Fronthaul Quantization
Authors: Parisa Ramezani, Yasaman Khorsandmanesh, Emil Björnson
Date: 2024-06-18 00:00:00
Tasks: Quantization

In this paper, we consider a downlink multi-user multiple-input multiple-output (MU-MIMO) communication assisted by a reconfigurable intelligent surface (RIS) and study the precoding and RIS configuration design under practical system constraints. These constraints include the limited-capacity fronthaul at the transmitter side and the finite resolution of RIS elements. We investigate the sum mean squared error (MSE) minimization problem and propose an algorithm based on the block coordinate descent method to optimize the precoding, RIS configuration, and receiver gains. We compute the precoding vectors and RIS configuration using the Schnorr-Euchner sphere decoding (SESD) method which delivers the optimal MSE-minimizing solution. We numerically evaluate the performance of the proposed SESD-based methods and corroborate their effectiveness in improving the system performance.
Hybrid Beamforming Design for Near-Field ISAC with Modular XL-MIMO
Authors: Chunwei Meng, Dingyou Ma, Zhaolin Wang, Yuanwei Liu, Zhiqing Wei, Zhiyong Feng
Date: 2024-06-18 00:00:00
Tasks: Integrated sensing and communication, ISAC

A novel modular extremely large-scale multiple-input-multiple-output (XL-MIMO) integrated sensing and communication (ISAC) framework is proposed in this paper. We consider a downlink ISAC scenario and exploit the modular array architecture to enhance the communication spectral efficiency and sensing resolution while reducing the channel modeling complexity by employing the hybrid spherical and planar wavefront model. Considering the hybrid digital-analog structure inherent to modular arrays, we formulate a joint analog-digital beamforming design problem based on the communication spectral efficiency and sensing signal-to-clutter-plus-noise ratio (SCNR). By exploring the structural similarity of the communication and sensing channels, it is proved that the optimal transmit covariance matrix lies in the subspace spanned by the subarray response vectors, yielding a closed-form solution for the optimal analog beamformer. Consequently, the joint design problem is transformed into a low-dimensional rank-constrained digital beamformer optimization. We first propose a manifold optimization method that directly optimizes the digital beamformer on the rank-constrained Stiefel manifold. Additionally, we develop an semidefinite relaxation (SDR)-based approach that relaxes the rank constraint and employ the randomization technique to obtain a near-optimal solution. Simulation results demonstrate the effectiveness of the proposed modular XL-MIMO ISAC framework and algorithms.
Advancing Ubiquitous Wireless Connectivity through Channel Twinning
Authors: Yashuai Cao, Linglong Dai, Jingbo Tan, Jintao Wang, Tianyue Zheng, Wei Ni, Ekram Hossain, Dusit Niyato
Date: 2024-06-18 00:00:00
Tasks: Scene Recognition

As an emerging trend in channel acquisition (CA), the concept of channel twinning (CT) has been proposed as a powerful enabler of ubiquitous connectivity in next-generation (xG) wireless systems. By fusing multimodal sensor data, CT advocates a high-fidelity and low-overhead CA paradigm, which is promising to provide accurate channel prediction in cross-domain and high-mobility scenarios of ubiquitous xG networks. However, existing literature lacks a universal CT architecture to address the challenges of heterogeneous scenarios, data, and resources in xG networks, which hinders the widespread deployment and applications of CT. This article discusses a new modularized CT architecture to bridge scene recognition, cooperative sensing, and decentralized training, comprising versatile model configuration, multimodal cooperative sensing, and lightweight twin modeling modules. Additionally, this article presents a detailed concept, technical features, and case studies of CT, outlines mainstream trends of realization methods, followed by potential applications of CT-empowered ubiquitous connectivity, and issues requiring future investigations.
Model-Based Deep Learning for Music Information Research
Authors: Gael Richard, Vincent Lostanlen, Yi-Hsuan Yang, Meinard Müller
Date: 2024-06-17 00:00:00
Tasks: Deep Learning, model

In this article, we investigate the notion of model-based deep learning in the realm of music information research (MIR). Loosely speaking, we refer to the term model-based deep learning for approaches that combine traditional knowledge-based methods with data-driven techniques, especially those based on deep learning, within a diff erentiable computing framework. In music, prior knowledge for instance related to sound production, music perception or music composition theory can be incorporated into the design of neural networks and associated loss functions. We outline three specifi c scenarios to illustrate the application of model-based deep learning in MIR, demonstrating the implementation of such concepts and their potential.
An Approximate Wave-Number Domain Expression for Near-Field XL-array Channel
Authors: Hongbo Xing, Yuxiang Zhang, Jianhua Zhang, Huixin Xu, Guangyi Liu, Qixing Wang
Date: 2024-06-17 00:00:00
Tasks: Form, parameter estimation

As Extremely large-scale array (XL-array) technology advances and carrier frequency rises, the near-field effects in communication are intensifying. In near-field conditions, channels exhibit a diffusion phenomenon in the angular domain, existing research indicates that this phenomenon can be leveraged for efficient parameter estimation and beam training. However, the channel model in angular domain lacks closed-form analysis, making the time complexity of the corresponding algorithm high. To address this issue, this paper analyzes the near-field diffusion effect in the wave-number domain, where the wave-number domain can be viewed as the continuous form of the angular domain. A closed-form approximate wave-number domain expression is proposed, based on the Principle of Stationary Phase. Subsequently, we derive a simplified expression for the case where the user distance is much larger than the array aperture, which is more concise. Subsequently, we verify the accuracy of the proposed approximate expression through simulations and demonstrate its effectiveness using a beam training example. Results indicate that the beam training scheme, improved by the wave-number domain approximation model, can effectively estimate near-field user parameters and perform beam training using far-field DFT codebooks. Moreover, its performance surpasses that of existing DFT codebook-based beam training methods.
STAR-RIS-based Pulse-Doppler Radars
Authors: Emanuele Grossi, Hedieh Taremizadeh, Luca Venturino
Date: 2024-06-17 00:00:00

In this study, we consider a pulse-Doppler radar relying on a simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) for scanning a given volume; the radar receiver is collocated with the STAR-RIS and aims to detect moving targets and estimate their radial velocity in the presence of clutter. To separate the echoes received from the transmissive and reflective half-spaces, the STAR-RIS superimposes a different slow-time modulation on the pulses redirected in each half-space, while the radar detector employs a decision rule based on a generalized information criterion (GIC). Two scanning policies are introduced, namely, simultaneous and sequential scanning, with different tradeoffs in terms of radial velocity estimation accuracy and complexity of the radar detector.
No Analog Combiner TTD-based Hybrid Precoding for Multi-User Sub-THz Communications
Authors: Dang Qua Nguyen, Alexei Ashikhmin, Hong Yang, Taejoon Kim
Date: 2024-06-17 00:00:00

We address the design and optimization of real-world-suitable hybrid precoders for multi-user wideband sub-terahertz (sub-THz) communications. We note that the conventional fully connected true-time delay (TTD)-based architecture is impractical because there is no room for the required large number of analog signal combiners in the circuit board. Additionally, analog signal combiners incur significant signal power loss. These limitations are often overlooked in sub-THz research. To overcome these issues, we study a non-overlapping subarray architecture that eliminates the need for analog combiners. We extend the conventional single-user assumption by formulating an optimization problem to maximize the minimum data rate for simultaneously served users. This complex optimization problem is divided into two sub-problems. The first sub-problem aims to ensure a fair subarray allocation for all users and is solved via a continuous domain relaxation technique. The second sub-problem deals with practical TTD device constraints on range and resolution to maximize the subarray gain and is resolved by shifting to the phase domain. Our simulation results highlight significant performance gain for our real-world-ready TTD-based hybrid precoders.
Explainable Bayesian Recurrent Neural Smoother to Capture Global State Evolutionary Correlations
Authors: Shi Yan, Yan Liang, Huayu Zhang, Le Zheng, Difan Zou, Binglu Wang
Date: 2024-06-17 00:00:00
Tasks: Computational Efficiency

Through integrating the evolutionary correlations across global states in the bidirectional recursion, an explainable Bayesian recurrent neural smoother (EBRNS) is proposed for offline data-assisted fixed-interval state smoothing. At first, the proposed model, containing global states in the evolutionary interval, is transformed into an equivalent model with bidirectional memory. This transformation incorporates crucial global state information with support for bi-directional recursive computation. For the transformed model, the joint state-memory-trend Bayesian filtering and smoothing frameworks are derived by introducing the bidirectional memory iteration mechanism and offline data into Bayesian estimation theory. The derived frameworks are implemented using the Gaussian approximation to ensure analytical properties and computational efficiency. Finally, the neural network modules within EBRNS and its two-stage training scheme are designed. Unlike most existing approaches that artificially combine deep learning and model-based estimation, the bidirectional recursion and internal gated structures of EBRNS are naturally derived from Bayesian estimation theory, explainably integrating prior model knowledge, online measurement, and offline data. Experiments on representative real-world datasets demonstrate that the high smoothing accuracy of EBRNS is accompanied by data efficiency and a lightweight parameter scale.
Near-Field Localization and Sensing with Large-Aperture Arrays: From Signal Modeling to Processing
Authors: Zhaolin Wang, Parisa Ramezani, Yuanwei Liu, Emil Björnson
Date: 2024-06-16 00:00:00

The signal processing community is currently witnessing a growing interest in near-field signal processing, driven by the trend towards the use of large aperture arrays with high spatial resolution in the fields of communication, localization, sensing, imaging, etc. From the perspective of localization and sensing, this trend breaks the basic far-field assumptions that have dominated the array signal processing research in the past, presenting new challenges and promising opportunities.
Modulo Sampling in Shift-Invariant Spaces: Recovery and Stability Enhancement
Authors: Yhonatan Kvich, Yonina C. Eldar
Date: 2024-06-16 00:00:00

Sampling shift-invariant (SI) signals with a high dynamic range poses a notable challenge in the domain of analog-to-digital conversion (ADC). It is essential for the ADC's dynamic range to exceed that of the incoming analog signal to ensure no vital information is lost during the conversion process. Modulo sampling, an approach initially explored with bandlimited (BL) signals, offers a promising solution to overcome the constraints of dynamic range. In this paper, we expand on the recent advancements in modulo sampling to encompass a broader range of SI signals. Our proposed strategy incorporates analog preprocessing, including the use of a mixer and a low-pass filter (LPF), to transform the signal into a bandlimited one. This BL signal can be accurately reconstructed from its modulo samples if sampled at slightly above its Nyquist frequency. The recovery of the original SI signal from this BL representation is then achieved through suitable filtering. We also examine the efficacy of this system across various noise conditions. Careful choice of the mixer plays a pivotal role in enhancing the method's reliability, especially with generators prone to instability. Our approach thus broadens the framework of modulo sampling's utility in efficiently recovering SI signals, pushing its boundaries beyond BL signals while sampling only slightly above the rate needed for a SI signal.