pwc-archive/papers-with-abstracts huggingface dataset:
Optuna vs Code Llama: Are LLMs a New Paradigm for Hyperparameter Tuning?
Authors: Roman Kochnev, Arash Torabi Goodarzi, Zofia Antonina Bentyn, Dmitry Ignatov, Radu Timofte
Date: 2025-04-08 00:00:00
Tasks: Computational Efficiency, Hyperparameter Optimization

Optimal hyperparameter selection is critical for maximizing neural network performance, especially as models grow in complexity. This work investigates the viability of leveraging large language models (LLMs) for hyperparameter optimization by fine-tuning a parameter-efficient version of Code Llama using LoRA. The adapted LLM is capable of generating accurate and efficient hyperparameter recommendations tailored to diverse neural network architectures. Unlike traditional approaches such as Optuna, which rely on computationally intensive trial-and-error procedures, our method achieves competitive or superior results in terms of Root Mean Square Error (RMSE) while significantly reducing computational overhead. Our findings demonstrate that LLM-based optimization not only matches the performance of state-of-the-art techniques like Tree-structured Parzen Estimators (TPE) but also substantially accelerates the tuning process. This positions LLMs as a promising alternative for rapid experimentation, particularly in resource-constrained environments such as edge devices and mobile platforms, where computational efficiency is essential. In addition to improved efficiency, the method offers time savings and consistent performance across various tasks, highlighting its robustness and generalizability. All generated hyperparameters are included in the LEMUR Neural Network (NN) Dataset, which is publicly available and serves as an open-source benchmark for hyperparameter optimization research.
Towards Combinatorial Interpretability of Neural Computation
Authors: Micah Adler, Dan Alistarh, Nir Shavit
Date: 2025-04-10 00:00:00

We introduce combinatorial interpretability, a methodology for understanding neural computation by analyzing the combinatorial structures in the sign-based categorization of a network's weights and biases. We demonstrate its power through feature channel coding, a theory that explains how neural networks compute Boolean expressions and potentially underlies other categories of neural network computation. According to this theory, features are computed via feature channels: unique cross-neuron encodings shared among the inputs the feature operates on. Because different feature channels share neurons, the neurons are polysemantic and the channels interfere with one another, making the computation appear inscrutable. We show how to decipher these computations by analyzing a network's feature channel coding, offering complete mechanistic interpretations of several small neural networks that were trained with gradient descent. Crucially, this is achieved via static combinatorial analysis of the weight matrices, without examining activations or training new autoencoding networks. Feature channel coding reframes the superposition hypothesis, shifting the focus from neuron activation directionality in high-dimensional space to the combinatorial structure of codes. It also allows us for the first time to exactly quantify and explain the relationship between a network's parameter size and its computational capacity (i.e. the set of features it can compute with low error), a relationship that is implicitly at the core of many modern scaling laws. Though our initial studies of feature channel coding are restricted to Boolean functions, we believe they provide a rich, controlled, and informative research space, and that the path we propose for combinatorial interpretation of neural computation can provide a basis for understanding both artificial and biological neural circuits.
Learning in Spiking Neural Networks with a Calcium-based Hebbian Rule for Spike-timing-dependent Plasticity
Authors: Willian Soares Girão, Nicoletta Risi, Elisabetta Chicca
Date: 2025-04-09 00:00:00
Tasks: Edge-computing

Understanding how biological neural networks are shaped via local plasticity mechanisms can lead to energy-efficient and self-adaptive information processing systems, which promises to mitigate some of the current roadblocks in edge computing systems. While biology makes use of spikes to seamless use both spike timing and mean firing rate to modulate synaptic strength, most models focus on one of the two. In this work, we present a Hebbian local learning rule that models synaptic modification as a function of calcium traces tracking neuronal activity. We show how the rule reproduces results from spike time and spike rate protocols from neuroscientific studies. Moreover, we use the model to train spiking neural networks on MNIST digit recognition to show and explain what sort of mechanisms are needed to learn real-world patterns. We show how our model is sensitive to correlated spiking activity and how this enables it to modulate the learning rate of the network without altering the mean firing rate of the neurons nor the hyparameters of the learning rule. To the best of our knowledge, this is the first work that showcases how spike timing and rate can be complementary in their role of shaping the connectivity of spiking neural networks.
The Method for Storing Patterns in Neural Networks-Memorization and Recall of QR code Patterns-
Authors: Hiroshi Inazawa
Date: 2025-04-09 00:00:00
Tasks: Language Modeling, Language Modelling, Large Language Model, Memorization

In this paper, we propose a mechanism for storing complex patterns within a neural network and subsequently recalling them. This model is based on our work published in 2018(Inazawa, 2018), which we have refined and extended in this work. With the recent advancements in deep learning and large language model (LLM)-based AI technologies (generative AI), it can be considered that methodologies for the learning are becoming increasingly well-established. In the future, we expect to see further research on memory using models based on Transformers (Vaswani, et. al., 2017, Rae, et. al., 2020), but in this paper we propose a simpler and more powerful model of memory and recall in neural networks. The advantage of storing patterns in a neural network lies in its ability to recall the original pattern even when an incomplete version is presented. The patterns we have produced for use in this study have been QR code (DENSO WAVE, 1994), which has become widely used as an information transmission tool in recent years.
Accelerating Vehicle Routing via AI-Initialized Genetic Algorithms
Authors: Ido Greenberg, Piotr Sielski, Hugo Linsenmaier, Rajesh Gandham, Shie Mannor, Alex Fender, Gal Chechik, Eli Meirom
Date: 2025-04-08 00:00:00
Tasks: Combinatorial Optimization, reinforcement-learning, Reinforcement Learning

Vehicle Routing Problems (VRP) are an extension of the Traveling Salesperson Problem and are a fundamental NP-hard challenge in combinatorial optimization. Solving VRP in real-time at large scale has become critical in numerous applications, from growing markets like last-mile delivery to emerging use-cases like interactive logistics planning. Such applications involve solving similar problem instances repeatedly, yet current state-of-the-art solvers treat each instance on its own without leveraging previous examples. We introduce a novel optimization framework that uses a reinforcement learning agent - trained on prior instances - to quickly generate initial solutions, which are then further optimized by genetic algorithms. Our framework, Evolutionary Algorithm with Reinforcement Learning Initialization (EARLI), consistently outperforms current state-of-the-art solvers across various time scales. For example, EARLI handles vehicle routing with 500 locations within 1s, 10x faster than current solvers for the same solution quality, enabling applications like real-time and interactive routing. EARLI can generalize to new data, as demonstrated on real e-commerce delivery data of a previously unseen city. Our hybrid framework presents a new way to combine reinforcement learning and genetic algorithms, paving the road for closer interdisciplinary collaboration between AI and optimization communities towards real-time optimization in diverse domains.
On the Dynamics of Mating Preferences in Genetic Programming
Authors: José Maria Simões, Nuno Lourenço, Penousal Machado
Date: 2025-04-08 00:00:00
Tasks: Diversity, Evolutionary Algorithms

Several mating restriction techniques have been implemented in Evolutionary Algorithms to promote diversity. From similarity-based selection to niche preservation, the general goal is to avoid premature convergence by not having fitness pressure as the single evolutionary force. In a way, such methods can resemble the mechanisms involved in Sexual Selection, although generally assuming a simplified approach. Recently, a selection method called mating Preferences as Ideal Mating Partners (PIMP) has been applied to GP, providing promising results both in performance and diversity maintenance. The method mimics Mate Choice through the unbounded evolution of personal preferences rather than having a single set of rules to shape parent selection. As such, PIMP allows ideal mate representations to evolve freely, thus potentially taking advantage of Sexual Selection as a dynamic secondary force to fitness pressure. However, it is still unclear how mating preferences affect the overall population and how dependent they are on set-up choices. In this work, we tracked the evolution of individual preferences through different mutation types, searching for patterns and evidence of self-reinforcement. Results suggest that mating preferences do not stand on their own, relying on subtree mutation to avoid convergence to single-node trees. Nevertheless, they consistently promote smaller and more balanced solutions depth-wise than a standard tournament selection, reducing the impact of bloat. Furthermore, when coupled with subtree mutation it also results in more solution diversity with statistically significant results.
Interpretable Non-linear Survival Analysis with Evolutionary Symbolic Regression
Authors: Luigi Rovito, Marco Virgolin
Date: 2025-04-08 00:00:00
Tasks: regression, Survival Analysis, Symbolic Regression

Survival Regression (SuR) is a key technique for modeling time to event in important applications such as clinical trials and semiconductor manufacturing. Currently, SuR algorithms belong to one of three classes: non-linear black-box -- allowing adaptability to many datasets but offering limited interpretability (e.g., tree ensembles); linear glass-box -- being easier to interpret but limited to modeling only linear interactions (e.g., Cox proportional hazards); and non-linear glass-box -- allowing adaptability and interpretability, but empirically found to have several limitations (e.g., explainable boosting machines, survival trees). In this work, we investigate whether Symbolic Regression (SR), i.e., the automated search of mathematical expressions from data, can lead to non-linear glass-box survival models that are interpretable and accurate. We propose an evolutionary, multi-objective, and multi-expression implementation of SR adapted to SuR. Our empirical results on five real-world datasets show that SR consistently outperforms traditional glass-box methods for SuR in terms of accuracy per number of dimensions in the model, while exhibiting comparable accuracy with black-box methods. Furthermore, we offer qualitative examples to assess the interpretability potential of SR models for SuR. Code at: https://github.com/lurovi/SurvivalMultiTree-pyNSGP.
Dynamic Neural Field Modeling of Visual Contrast for Perceiving Incoherent Looming
Authors: Ziyan Qin, Qinbing Fu, Jigen Peng, Shigang Yue
Date: 2025-04-06 00:00:00

Amari's Dynamic Neural Field (DNF) framework provides a brain-inspired approach to modeling the average activation of neuronal groups. Leveraging a single field, DNF has become a promising foundation for low-energy looming perception module in robotic applications. However, the previous DNF methods face significant challenges in detecting incoherent or inconsistent looming features--conditions commonly encountered in real-world scenarios, such as collision detection in rainy weather. Insights from the visual systems of fruit flies and locusts reveal encoding ON/OFF visual contrast plays a critical role in enhancing looming selectivity. Additionally, lateral excitation mechanism potentially refines the responses of loom-sensitive neurons to both coherent and incoherent stimuli. Together, these offer valuable guidance for improving looming perception models. Building on these biological evidence, we extend the previous single-field DNF framework by incorporating the modeling of ON/OFF visual contrast, each governed by a dedicated DNF. Lateral excitation within each ON/OFF-contrast field is formulated using a normalized Gaussian kernel, and their outputs are integrated in the Summation field to generate collision alerts. Experimental evaluations show that the proposed model effectively addresses incoherent looming detection challenges and significantly outperforms state-of-the-art locust-inspired models. It demonstrates robust performance across diverse stimuli, including synthetic rain effects, underscoring its potential for reliable looming perception in complex, noisy environments with inconsistent visual cues.
Memetic Search for Green Vehicle Routing Problem with Private Capacitated Refueling Stations
Authors: Rui Xu, Xing Fan, Shengcai Liu, Wenjie Chen, Ke Tang
Date: 2025-04-06 00:00:00
Tasks: Efficient Exploration

The green vehicle routing problem with private capacitated alternative fuel stations (GVRP-PCAFS) extends the traditional green vehicle routing problem by considering refueling stations limited capacity, where a limited number of vehicles can refuel simultaneously with additional vehicles must wait. This feature presents new challenges for route planning, as waiting times at stations must be managed while keeping route durations within limits and reducing total travel distance. This article presents METS, a novel memetic algorithm (MA) with separate constraint-based tour segmentation (SCTS) and efficient local search (ELS) for solving GVRP-PCAFS. METS combines global and local search effectively through three novelties. For global search, the SCTS strategy splits giant tours to generate diverse solutions, and the search process is guided by a comprehensive fitness evaluation function to dynamically control feasibility and diversity to produce solutions that are both diverse and near-feasible. For local search, ELS incorporates tailored move operators with constant-time move evaluation mechanisms, enabling efficient exploration of large solution neighborhoods. Experimental results demonstrate that METS discovers 31 new best-known solutions out of 40 instances in existing benchmark sets, achieving substantial improvements over current state-of-the-art methods. Additionally, a new large-scale benchmark set based on real-world logistics data is introduced to facilitate future research.
Attention-Driven LPLC2 Neural Ensemble Model for Multi-Target Looming Detection and Localization
Authors: Renyuan Liu, Qinbing Fu
Date: 2025-04-06 00:00:00

Lobula plate/lobula columnar, type 2 (LPLC2) visual projection neurons in the fly's visual system possess highly looming-selective properties, making them ideal for developing artificial collision detection systems. The four dendritic branches of individual LPLC2 neurons, each tuned to specific directional motion, enhance the robustness of looming detection by utilizing radial motion opponency. Existing models of LPLC2 neurons either concentrate on individual cells to detect centroid-focused expansion or utilize population-voting strategies to obtain global collision information. However, their potential for addressing multi-target collision scenarios remains largely untapped. In this study, we propose a numerical model for LPLC2 populations, leveraging a bottom-up attention mechanism driven by motion-sensitive neural pathways to generate attention fields (AFs). This integration of AFs with highly nonlinear LPLC2 responses enables precise and continuous detection of multiple looming objects emanating from any region of the visual field. We began by conducting comparative experiments to evaluate the proposed model against two related models, highlighting its unique characteristics. Next, we tested its ability to detect multiple targets in dynamic natural scenarios. Finally, we validated the model using real-world video data collected by aerial robots. Experimental results demonstrate that the proposed model excels in detecting, distinguishing, and tracking multiple looming targets with remarkable speed and accuracy. This advanced ability to detect and localize looming objects, especially in complex and dynamic environments, holds great promise for overcoming collision-detection challenges in mobile intelligent machines.
Structuring Multiple Simple Cycle Reservoirs with Particle Swarm Optimization
Authors: Ziqiang Li, Robert Simon Fong, Kantaro Fujiwara, Kazuyuki Aihara, Gouhei Tanaka
Date: 2025-04-06 00:00:00
Tasks: Time Series Prediction

Reservoir Computing (RC) is a time-efficient computational paradigm derived from Recurrent Neural Networks (RNNs). The Simple Cycle Reservoir (SCR) is an RC model that stands out for its minimalistic design, offering extremely low construction complexity and proven capability of universally approximating time-invariant causal fading memory filters, even in the linear dynamics regime. This paper introduces Multiple Simple Cycle Reservoirs (MSCRs), a multi-reservoir framework that extends Echo State Networks (ESNs) by replacing a single large reservoir with multiple interconnected SCRs. We demonstrate that optimizing MSCR using Particle Swarm Optimization (PSO) outperforms existing multi-reservoir models, achieving competitive predictive performance with a lower-dimensional state space. By modeling interconnections as a weighted Directed Acyclic Graph (DAG), our approach enables flexible, task-specific network topology adaptation. Numerical simulations on three benchmark time-series prediction tasks confirm these advantages over rival algorithms. These findings highlight the potential of MSCR-PSO as a promising framework for optimizing multi-reservoir systems, providing a foundation for further advancements and applications of interconnected SCRs for developing efficient AI devices.
Three-Factor Learning in Spiking Neural Networks: An Overview of Methods and Trends from a Machine Learning Perspective
Authors: Szymon Mazurek, Jakub Caputa, Jan K. Argasiński, Maciej Wielgosz
Date: 2025-04-06 00:00:00

Three-factor learning rules in Spiking Neural Networks (SNNs) have emerged as a crucial extension to traditional Hebbian learning and Spike-Timing-Dependent Plasticity (STDP), incorporating neuromodulatory signals to improve adaptation and learning efficiency. These mechanisms enhance biological plausibility and facilitate improved credit assignment in artificial neural systems. This paper takes a view on this topic from a machine learning perspective, providing an overview of recent advances in three-factor learning, discusses theoretical foundations, algorithmic implementations, and their relevance to reinforcement learning and neuromorphic computing. In addition, we explore interdisciplinary approaches, scalability challenges, and potential applications in robotics, cognitive modeling, and AI systems. Finally, we highlight key research gaps and propose future directions for bridging the gap between neuroscience and artificial intelligence.
Decision SpikeFormer: Spike-Driven Transformer for Decision Making
Authors: Wei Huang, Qinying Gu, Nanyang Ye
Date: 2025-04-04 00:00:00
Proceeding: CVPR 2025 1
Tasks: D4RL, Decision Making, Offline RL, Reinforcement Learning (RL)

Offline reinforcement learning (RL) enables policy training solely on pre-collected data, avoiding direct environment interaction - a crucial benefit for energy-constrained embodied AI applications. Although Artificial Neural Networks (ANN)-based methods perform well in offline RL, their high computational and energy demands motivate exploration of more efficient alternatives. Spiking Neural Networks (SNNs) show promise for such tasks, given their low power consumption. In this work, we introduce DSFormer, the first spike-driven transformer model designed to tackle offline RL via sequence modeling. Unlike existing SNN transformers focused on spatial dimensions for vision tasks, we develop Temporal Spiking Self-Attention (TSSA) and Positional Spiking Self-Attention (PSSA) in DSFormer to capture the temporal and positional dependencies essential for sequence modeling in RL. Additionally, we propose Progressive Threshold-dependent Batch Normalization (PTBN), which combines the benefits of LayerNorm and BatchNorm to preserve temporal dependencies while maintaining the spiking nature of SNNs. Comprehensive results in the D4RL benchmark show DSFormer's superiority over both SNN and ANN counterparts, achieving 78.4% energy savings, highlighting DSFormer's advantages not only in energy efficiency but also in competitive performance. Code and models are public at https://wei-nijuan.github.io/DecisionSpikeFormer.
QSViT: A Methodology for Quantizing Spiking Vision Transformers
Authors: Rachmad Vidya Wicaksana Putra, Saad Iftikhar, Muhammad Shafique
Date: 2025-04-01 00:00:00
Tasks: Quantization

Vision Transformer (ViT)-based models have shown state-of-the-art performance (e.g., accuracy) in vision-based AI tasks. However, realizing their capability in resource-constrained embedded AI systems is challenging due to their inherent large memory footprints and complex computations, thereby incurring high power/energy consumption. Recently, Spiking Vision Transformer (SViT)-based models have emerged as alternate low-power ViT networks. However, their large memory footprints still hinder their applicability for resource-constrained embedded AI systems. Therefore, there is a need for a methodology to compress SViT models without degrading the accuracy significantly. To address this, we propose QSViT, a novel design methodology to compress the SViT models through a systematic quantization strategy across different network layers. To do this, our QSViT employs several key steps: (1) investigating the impact of different precision levels in different network layers, (2) identifying the appropriate base quantization settings for guiding bit precision reduction, (3) performing a guided quantization strategy based on the base settings to select the appropriate quantization setting, and (4) developing an efficient quantized network based on the selected quantization setting. The experimental results demonstrate that, our QSViT methodology achieves 22.75% memory saving and 21.33% power saving, while also maintaining high accuracy within 2.1% from that of the original non-quantized SViT model on the ImageNet dataset. These results highlight the potential of QSViT methodology to pave the way toward the efficient SViT deployments on resource-constrained embedded AI systems.