pwc-archive/papers-with-abstracts huggingface dataset:
Unsupervised categorization of similarity measures
Authors: Yoshiyuki Ohmura, Wataru Shimaya, Yasuo Kuniyoshi
Date: 2025-02-12 00:00:00
Tasks: Representation Learning

In general, objects can be distinguished on the basis of their features, such as color or shape. In particular, it is assumed that similarity judgments about such features can be processed independently in different metric spaces. However, the unsupervised categorization mechanism of metric spaces corresponding to object features remains unknown. Here, we show that the artificial neural network system can autonomously categorize metric spaces through representation learning to satisfy the algebraic independence between neural networks, and project sensory information onto multiple high-dimensional metric spaces to independently evaluate the differences and similarities between features. Conventional methods often constrain the axes of the latent space to be mutually independent or orthogonal. However, the independent axes are not suitable for categorizing metric spaces. High-dimensional metric spaces that are independent of each other are not uniquely determined by the mutually independent axes, because any combination of independent axes can form mutually independent spaces. In other words, the mutually independent axes cannot be used to naturally categorize different feature spaces, such as color space and shape space. Therefore, constraining the axes to be mutually independent makes it difficult to categorize high-dimensional metric spaces. To overcome this problem, we developed a method to constrain only the spaces to be mutually independent and not the composed axes to be independent. Our theory provides general conditions for the unsupervised categorization of independent metric spaces, thus advancing the mathematical theory of functional differentiation of neural networks.
Extract-QD Framework: A Generic Approach for Quality-Diversity in Noisy, Stochastic or Uncertain Domains
Authors: Manon Flageat, Johann Huber, François Helenon, Stephane Doncieux, Antoine Cully
Date: 2025-02-10 00:00:00
Tasks: Diversity

Quality-Diversity (QD) has demonstrated potential in discovering collections of diverse solutions to optimisation problems. Originally designed for deterministic environments, QD has been extended to noisy, stochastic, or uncertain domains through various Uncertain-QD (UQD) methods. However, the large number of UQD methods, each with unique constraints, makes selecting the most suitable one challenging. To remedy this situation, we present two contributions: first, the Extract-QD Framework (EQD Framework), and second, Extract-ME (EME), a new method derived from it. The EQD Framework unifies existing approaches within a modular view, and facilitates developing novel methods by interchanging modules. We use it to derive EME, a novel method that consistently outperforms or matches the best existing methods on standard benchmarks, while previous methods show varying performance. In a second experiment, we show how our EQD Framework can be used to augment existing QD algorithms and in particular the well-established Policy-Gradient-Assisted-MAP-Elites method, and demonstrate improved performance in uncertain domains at no additional evaluation cost. For any new uncertain task, our contributions now provide EME as a reliable "first guess" method, and the EQD Framework as a tool for developing task-specific approaches. Together, these contributions aim to lower the cost of adopting UQD insights in QD applications.
D2SA: Dual-Stage Distribution and Slice Adaptation for Efficient Test-Time Adaptation in MRI Reconstruction
Authors: Lipei Zhang, Rui Sun, Zhongying Deng, Yanqi Cheng, Carola-Bibiane Schönlieb, Angelica I Aviles-Rivero
Date: 2025-03-25 00:00:00
Tasks: MRI Reconstruction, Self-Supervised Learning, Test-time Adaptation

Variations in Magnetic resonance imaging (MRI) scanners and acquisition protocols cause distribution shifts that degrade reconstruction performance on unseen data. Test-time adaptation (TTA) offers a promising solution to address this discrepancies. However, previous single-shot TTA approaches are inefficient due to repeated training and suboptimal distributional models. Self-supervised learning methods are also limited by scarce date scenarios. To address these challenges, we propose a novel Dual-Stage Distribution and Slice Adaptation (D2SA) via MRI implicit neural representation (MR-INR) to improve MRI reconstruction performance and efficiency, which features two stages. In the first stage, an MR-INR branch performs patient-wise distribution adaptation by learning shared representations across slices and modelling patient-specific shifts with mean and variance adjustments. In the second stage, single-slice adaptation refines the output from frozen convolutional layers with a learnable anisotropic diffusion module, preventing over-smoothing and reducing computation. Experiments across four MRI distribution shifts demonstrate that our method can integrate well with various self-supervised learning (SSL) framework, improving performance and accelerating convergence under diverse conditions.
Utilizing Novelty-based Evolution Strategies to Train Transformers in Reinforcement Learning
Authors: Matyáš Lorenc
Date: 2025-02-10 00:00:00

In this paper, we experiment with novelty-based variants of OpenAI-ES, the NS-ES and NSR-ES algorithms, and evaluate their effectiveness in training complex, transformer-based architectures designed for the problem of reinforcement learning such as Decision Transformers. We also test if we can accelerate the novelty-based training of these larger models by seeding the training by a pretrained models. By this, we build on our previous work, where we tested the ability of evolution strategies - specifically the aforementioned OpenAI-ES - to train the Decision Transformer architecture. The results were mixed. NS-ES showed progress, but it would clearly need many more iterations for it to yield interesting results. NSR-ES, on the other hand, proved quite capable of being straightforwardly used on larger models, since its performance appears as similar between the feed-forward model and Decision Transformer, as it was for the OpenAI-ES in our previous work.
A Hybrid Tabu Scatter Search Algorithm for Simulation-Based Optimization of Multi-Objective Runway Operations Scheduling
Authors: Bulent Soykan
Date: 2025-02-08 00:00:00
Tasks: Fairness, Scheduling

This dissertation addresses the growing challenge of air traffic flow management by proposing a simulation-based optimization (SbO) approach for multi-objective runway operations scheduling. The goal is to optimize airport capacity utilization while minimizing delays, fuel consumption, and environmental impacts. Given the NP-Hard complexity of the problem, traditional analytical methods often rely on oversimplifications and fail to account for real-world uncertainties, limiting their practical applicability. The proposed SbO framework integrates a discrete-event simulation model to handle stochastic conditions and a hybrid Tabu-Scatter Search algorithm to identify Pareto-optimal solutions, explicitly incorporating uncertainty and fairness among aircraft as key objectives. Computational experiments using real-world data from a major U.S. airport demonstrate the approach's effectiveness and tractability, outperforming traditional methods such as First-Come-First-Served (FCFS) and deterministic approaches while maintaining schedule fairness. The algorithm's ability to generate trade-off solutions between competing objectives makes it a promising decision support tool for air traffic controllers managing complex runway operations.
Leveraging Line-of-Sight Propagation for Near-Field Beamfocusing in Cell-Free Networks
Authors: Georgios Mylonopoulos, Giovanni Interdonato, Stefano Buzzi, Pei Liu
Date: 2025-03-27 00:00:00

Cell-free (CF) massive multiple-input multiple-output (MIMO) is a promising approach for next-generation wireless networks, enabling scalable deployments of multiple small access points (APs) to enhance coverage and service for multiple user equipments (UEs). While most existing research focuses on low-frequency bands with Rayleigh fading models, emerging 5G trends are shifting toward higher frequencies, where geometric channel models and line-of-sight (LoS) propagation become more relevant. In this work, we explore how distributed massive MIMO in the LoS regime can achieve near-field-like conditions by forming artificially large arrays through coordinated AP deployments. We investigate centralized and decentralized CF architectures, leveraging structured channel estimation (SCE) techniques that exploit the line-of-sight properties of geometric channels. Our results demonstrate that dense distributed AP deployments significantly improve system performance w.r.t. the case of a co-located array, even in highly populated UE scenarios, while SCE approaches the performance of perfect CSI.
Abnormal Mutations: Evolution Strategies Don't Require Gaussianity
Authors: Jacob de Nobel, Diederick Vermetten, Hao Wang, Anna V. Kononova, Günter Rudolph, Thomas Bäck
Date: 2025-02-05 00:00:00

The mutation process in evolution strategies has been interlinked with the normal distribution since its inception. Many lines of reasoning have been given for this strong dependency, ranging from maximum entropy arguments to the need for isotropy. However, some theoretical results suggest that other distributions might lead to similar local convergence properties. This paper empirically shows that a wide range of evolutionary strategies, from the (1+1)-ES to CMA-ES, show comparable optimization performance when using a mutation distribution other than the standard Gaussian. Replacing it with, e.g., uniformly distributed mutations, does not deteriorate the performance of ES, when using the default adaptation mechanism for the strategy parameters. We observe that these results hold not only for the sphere model but also for a wider range of benchmark problems.
Shape Adaptive Reconfigurable Holographic Surfaces
Authors: Jalal Jalali, Mostafa Darabi, Rodrigo C. de Lamare
Date: 2025-03-27 00:00:00

Reconfigurable Intelligent Surfaces (RIS) have emerged as a key solution to dynamically adjust wireless propagation by tuning the reflection coefficients of large arrays of passive elements. Reconfigurable Holographic Surfaces (RHS) build on the same foundation as RIS but extend it by employing holographic principles for finer-grained wave manipulation | that is, applying higher spatial control over the reflected signals for more precise beam steering. In this paper, we investigate shape-adaptive RHS deployments in a multi-user network. Rather than treating each RHS as a uniform reflecting surface, we propose a selective element activation strategy that dynamically adapts the spatial arrangement of deployed RHS regions to a subset of predefined shapes. In particular, we formulate a system throughput maximization problem that optimizes the shape of the selected RHS elements, active beamforming at the access point (AP), and passive beamforming at the RHS to enhance coverage and mitigate signal blockage. The resulting problem is non-convex and becomes even more challenging to solve as the number of RHS and users increases; to tackle this, we introduce an alternating optimization (AO) approach that efficiently finds near-optimal solutions irrespective of the number or spatial configuration of RHS. Numerical results demonstrate that shape adaptation enables more efficient resource distribution, enhancing the effectiveness of multi-RHS deployments as the network scales.
Polarization-Aware Antenna Selection for Joint Radar and Communication in XL-MIMO Systems
Authors: Ahmed Naeem, Liza Afeef, Huseyin Arslan
Date: 2025-03-27 00:00:00

A key challenge in dual-polarized multiplexing for joint radar and communication (JRC) systems is cross-polarization (cross-pol) leakage caused by depolarization. In conventional MIMO systems, depolarization arises solely from the channel; however, in XL-MIMO systems, non-stationary properties of the array cause additional polarization shifts at each antenna element, further degrading JRC performance. This paper introduces a channel model incorporating polarization shifts due to the propagation channel and antenna elements in the near-field. We also propose an antenna selection (AS) scheme that dynamically chooses antennas based on polarization imbalance and cross-pol leakage, enhancing spectral efficiency, symbol error rate, and radar detection probability. Simulations show that the proposed AS significantly outperforms traditional methods, providing scalable benefits for XL-MIMO JRC systems.
Accelerating Linear Recurrent Neural Networks for the Edge with Unstructured Sparsity
Authors: Alessandro Pierro, Steven Abreu, Jonathan Timcheck, Philipp Stratmann, Andreas Wild, Sumit Bam Shrestha
Date: 2025-02-03 00:00:00
Tasks: Audio Denoising, Denoising, GPU, Model Compression

Linear recurrent neural networks enable powerful long-range sequence modeling with constant memory usage and time-per-token during inference. These architectures hold promise for streaming applications at the edge, but deployment in resource-constrained environments requires hardware-aware optimizations to minimize latency and energy consumption. Unstructured sparsity offers a compelling solution, enabling substantial reductions in compute and memory requirements--when accelerated by compatible hardware platforms. In this paper, we conduct a scaling study to investigate the Pareto front of performance and efficiency across inference compute budgets. We find that highly sparse linear RNNs consistently achieve better efficiency-performance trade-offs than dense baselines, with 2x less compute and 36% less memory at iso-accuracy. Our models achieve state-of-the-art results on a real-time streaming task for audio denoising. By quantizing our sparse models to fixed-point arithmetic and deploying them on the Intel Loihi 2 neuromorphic chip for real-time processing, we translate model compression into tangible gains of 42x lower latency and 149x lower energy consumption compared to a dense model on an edge GPU. Our findings showcase the transformative potential of unstructured sparsity, paving the way for highly efficient recurrent neural networks in real-world, resource-constrained environments.
Brain Age Group Classification Based on Resting State Functional Connectivity Metrics
Authors: Prerna Singh, Kuldeep Singh Yadav, Lalan Kumar, Tapan Kumar Gandhi
Date: 2025-03-27 00:00:00
Tasks: Clustering, Functional Connectivity

This study investigated age-related changes in functional connectivity using resting-state fMRI and explored the efficacy of traditional deep learning for classifying brain developmental stages (BDS). Functional connectivity was assessed using Seed-Based Phase Synchronization (SBPS) and Pearson correlation across 160 ROIs. Clustering was performed using t-SNE, and network topology was analyzed through graph-theoretic metrics. Adaptive learning was implemented to classify the age group by extracting bottleneck features through mobileNetV2. These deep features were embedded and classified using Random Forest and PCA. Results showed a shift in phase synchronization patterns from sensory-driven networks in youth to more distributed networks with aging. t-SNE revealed that SBPS provided the most distinct clustering of BDS. Global efficiency and participation coefficient followed an inverted U-shaped trajectory, while clustering coefficient and modularity exhibited a U-shaped pattern. MobileNet outperformed other models, achieving the highest classification accuracy for BDS. Aging was associated with reduced global integration and increased local connectivity, indicating functional network reorganization. While this study focused solely on functional connectivity from resting-state fMRI and a limited set of connectivity features, deep learning demonstrated superior classification performance, highlighting its potential for characterizing age-related brain changes.
G{é}n{é}ration de Matrices de Corr{é}lation avec des Structures de Graphe par Optimisation Convexe
Authors: Ali Fahkar, Kévin Polisano, Irène Gannaz, Sophie Achard
Date: 2025-03-27 00:00:00

This work deals with the generation of theoretical correlation matrices with specific sparsity patterns, associated to graph structures. We present a novel approach based on convex optimization, offering greater flexibility compared to existing techniques, notably by controlling the mean of the entry distribution in the generated correlation matrices. This allows for the generation of correlation matrices that better represent realistic data and can be used to benchmark statistical methods for graph inference.
Understanding Why Adam Outperforms SGD: Gradient Heterogeneity in Transformers
Authors: Akiyoshi Tomihari, Issei Sato
Date: 2025-01-31 00:00:00

Transformers are challenging to optimize with SGD and typically require adaptive optimizers such as Adam. However, the reasons behind the superior performance of Adam over SGD remain unclear. In this study, we investigate the optimization of transformers by focusing on gradient heterogeneity, defined as the disparity in gradient norms among parameters. Our analysis shows that gradient heterogeneity hinders gradient-based optimization, including SGD, while sign-based optimization, a simplified variant of Adam, is less affected. We further examine gradient heterogeneity in transformers and show that it is influenced by the placement of layer normalization. Experimental results from fine-tuning transformers in both NLP and vision domains validate our theoretical analyses. This study provides insights into the optimization challenges of transformers and offers guidance for designing future optimization algorithms. Code is available at https://github.com/tom4649/gradient-heterogeneity.
SAGRAD: A Program for Neural Network Training with Simulated Annealing and the Conjugate Gradient Method
Authors: Javier Bernal, Jose Torres-Jimenez
Date: 2025-01-31 00:00:00

SAGRAD (Simulated Annealing GRADient), a Fortran 77 program for computing neural networks for classification using batch learning, is discussed. Neural network training in SAGRAD is based on a combination of simulated annealing and M{\o}ller's scaled conjugate gradient algorithm, the latter a variation of the traditional conjugate gradient method, better suited for the nonquadratic nature of neural networks. Different aspects of the implementation of the training process in SAGRAD are discussed, such as the efficient computation of gradients and multiplication of vectors by Hessian matrices that are required by M{\o}ller's algorithm; the (re)initialization of weights with simulated annealing required to (re)start M{\o}ller's algorithm the first time and each time thereafter that it shows insufficient progress in reaching a possibly local minimum; and the use of simulated annealing when M{\o}ller's algorithm, after possibly making considerable progress, becomes stuck at a local minimum or flat area of weight space. Outlines of the scaled conjugate gradient algorithm, the simulated annealing procedure and the training process used in SAGRAD are presented together with results from running SAGRAD on two examples of training data.
Low-Cost Phase Precoding for Short-Reach Fiber Links with Direct Detection
Authors: Ulrike Höfler, Daniel Plabst, Norbert Hanik
Date: 2025-03-27 00:00:00

Low-cost analog phase precoding is used to compensate chromatic dispersion (CD) in fibers with intensity modulation and direct detection (IM/DD). In contrast to conventional precoding with an in-phase and quadrature (IQ) Mach-Zehnder modulator (MZM), only a single additional phase modulator (PM) is required at the transmitter. Depending on the CD, the PM generates a periodic phase modulation that is modelled by a Fourier series and optimized via a mean squared error (MSE) cost criterion. Numerical results compare achievable information rates (AIRs) for 4- and 6-PAM. With the additional PM, energy gains of up to 3 dB are achieved for moderate fiber lengths.
SpikingRTNH: Spiking Neural Network for 4D Radar Object Detection
Authors: Dong-Hee Paek, Seung-Hyun Kong
Date: 2025-01-31 00:00:00
Tasks: 3D Object Detection, Autonomous Driving, Autonomous Vehicles, Object, object-detection, Object Detection, Radar Object Detection

Recently, 4D Radar has emerged as a crucial sensor for 3D object detection in autonomous vehicles, offering both stable perception in adverse weather and high-density point clouds for object shape recognition. However, processing such high-density data demands substantial computational resources and energy consumption. We propose SpikingRTNH, the first spiking neural network (SNN) for 3D object detection using 4D Radar data. By replacing conventional ReLU activation functions with leaky integrate-and-fire (LIF) spiking neurons, SpikingRTNH achieves significant energy efficiency gains. Furthermore, inspired by human cognitive processes, we introduce biological top-down inference (BTI), which processes point clouds sequentially from higher to lower densities. This approach effectively utilizes points with lower noise and higher importance for detection. Experiments on K-Radar dataset demonstrate that SpikingRTNH with BTI significantly reduces energy consumption by 78% while achieving comparable detection performance to its ANN counterpart (51.1% AP 3D, 57.0% AP BEV). These results establish the viability of SNNs for energy-efficient 4D Radar-based object detection in autonomous driving systems. All codes are available at https://github.com/kaist-avelab/k-radar.
The Optimal Tradeoff Between PAPR and Ambiguity Functions for Generalized OFDM Waveform Set in ISAC Systems
Authors: Bichai Wang, Xiuhong Wei, Xueru Li, Yongxing Zhou
Date: 2025-03-27 00:00:00
Tasks: ISAC

Integrated sensing and communications (ISAC) has been identified as one of the six usage scenarios for IMT-2030. Compared with communication performance, sensing performance is much more vulnerable to interference, and the received backscattered sensing signal with target information is usually too weak to be detected. It is interesting to understand the optimal tradeoff between interference rejection and signal strength improvement for the best sensing performance, but unfortunately it still remains unknown. In this paper, the trinity of auto-ambiguity function (AF), cross-AF and peak-to-average-power ratio (PAPR) is proposed to describe the interference and coverage related aspects for ISAC systems where multi-carrier waveform is usually assumed. We extend the existing orthogonal frequency division multiplexing (OFDM) waveforms in 5G to a generalized OFDM waveform set with some new members and a unified parametric representation. Then the optimal Pareto tradeoff between PAPR, auto-AF and cross-AF (i.e., the union bound) is developed for the generalized OFDM waveform set. To achieve the optimal Pareto union bound with reasonable computational complexity, we further propose a framework to optimize waveform parameters and sequences jointly. Finally, some practical design examples are provided and numerical results reveal that significant improvements can be achieved compared to the state-of-the-art 5G waveforms and sequences.