pwc-archive/papers-with-abstracts huggingface dataset:
Case-Based Reasoning Approach for Solving Financial Question Answering
Authors: Yikyung Kim, Jay-Yoon Lee
Date: 2024-05-18 00:00:00
Tasks: Question Answering

Measuring a machine's understanding of human language often involves assessing its reasoning skills, i.e. logical process of deriving answers to questions. While recent language models have shown remarkable proficiency in text based tasks, their efficacy in complex reasoning problems involving heterogeneous information such as text, tables, and numbers remain uncertain. Addressing this gap, FinQA introduced a numerical reasoning dataset for financial documents and simultaneously proposed a program generation approach . Our investigation reveals that half of the errors (48%) stem from incorrect operations being generated. To address this issue, we propose a novel approach to tackle numerical reasoning problems using case based reasoning (CBR), an artificial intelligence paradigm that provides problem solving guidance by offering similar cases (i.e. similar questions and corresponding logical programs). Our model retrieves relevant cases to address a given question, and then generates an answer based on the retrieved cases and contextual information. Through experiments on the FinQA dataset, we demonstrate competitive performance of our approach and additionally show that by expanding case repository, we can help solving complex multi step programs which FinQA showed weakness of.
Domain Wall Magnetic Tunnel Junction Reliable Integrate and Fire Neuron
Authors: Can Cui1, Sam Liu, Jaesuk Kwon, Jean Anne C. Incorvia
Date: 2024-05-23 00:00:00
Tasks: image-classification, Image Classification

In spiking neural networks, neuron dynamics are described by the biologically realistic integrate-and-fire model that captures membrane potential accumulation and above-threshold firing behaviors. Among the hardware implementations of integrate-and-fire neuron devices, one important feature, reset, has been largely ignored. Here, we present the design and fabrication of a magnetic domain wall and magnetic tunnel junction based artificial integrate-and-fire neuron device that achieves reliable reset at the end of the integrate-fire cycle. We demonstrate the domain propagation in the domain wall racetrack (integration), reading using a magnetic tunnel junction (fire), and reset as the domain is ejected from the racetrack, showing the artificial neuron can be operated continuously over 100 integrate-fire-reset cycles. Both pulse amplitude and pulse number encoding is demonstrated. The device data is applied on an image classification task using a spiking neural network and shown to have comparable performance to an ideal leaky, integrate-and-fire neural network. These results achieve the first demonstration of reliable integrate-fire-reset in domain wall-magnetic tunnel junction-based neuron devices and shows the promise of spintronics for neuromorphic computing.
Time Cell Inspired Temporal Codebook in Spiking Neural Networks for Enhanced Image Generation
Authors: Linghao Feng, Dongcheng Zhao, Sicheng Shen, Yiting Dong, Guobin Shen, Yi Zeng
Date: 2024-05-23 00:00:00
Tasks: Image Generation

This paper presents a novel approach leveraging Spiking Neural Networks (SNNs) to construct a Variational Quantized Autoencoder (VQ-VAE) with a temporal codebook inspired by hippocampal time cells. This design captures and utilizes temporal dependencies, significantly enhancing the generative capabilities of SNNs. Neuroscientific research has identified hippocampal "time cells" that fire sequentially during temporally structured experiences. Our temporal codebook emulates this behavior by triggering the activation of time cell populations based on similarity measures as input stimuli pass through it. We conducted extensive experiments on standard benchmark datasets, including MNIST, FashionMNIST, CIFAR10, CelebA, and downsampled LSUN Bedroom, to validate our model's performance. Furthermore, we evaluated the effectiveness of the temporal codebook on neuromorphic datasets NMNIST and DVS-CIFAR10, and demonstrated the model's capability with high-resolution datasets such as CelebA-HQ, LSUN Bedroom, and LSUN Church. The experimental results indicate that our method consistently outperforms existing SNN-based generative models across multiple datasets, achieving state-of-the-art performance. Notably, our approach excels in generating high-resolution and temporally consistent data, underscoring the crucial role of temporal information in SNN-based generative modeling.
Advancing Spiking Neural Networks for Sequential Modeling with Central Pattern Generators
Authors: Changze Lv, Dongqi Han, Yansen Wang, Xiaoqing Zheng, Xuanjing Huang, Dongsheng Li
Date: 2024-05-23 00:00:00
Tasks: image-classification, Image Classification, text-classification, Text Classification, Time Series, Time Series Forecasting

Spiking neural networks (SNNs) represent a promising approach to developing artificial neural networks that are both energy-efficient and biologically plausible. However, applying SNNs to sequential tasks, such as text classification and time-series forecasting, has been hindered by the challenge of creating an effective and hardware-friendly spike-form positional encoding (PE) strategy. Drawing inspiration from the central pattern generators (CPGs) in the human brain, which produce rhythmic patterned outputs without requiring rhythmic inputs, we propose a novel PE technique for SNNs, termed CPG-PE. We demonstrate that the commonly used sinusoidal PE is mathematically a specific solution to the membrane potential dynamics of a particular CPG. Moreover, extensive experiments across various domains, including time-series forecasting, natural language processing, and image classification, show that SNNs with CPG-PE outperform their conventional counterparts. Additionally, we perform analysis experiments to elucidate the mechanism through which SNNs encode positional information and to explore the function of CPGs in the human brain. This investigation may offer valuable insights into the fundamental principles of neural computation. Our code is available at https://github.com/microsoft/SeqSNN.
A Memetic NSGA-III for Green Flexible Production with Real-Time Energy Costs & Emissions
Authors: Sascha C Burmeister
Date: 2024-05-23 00:00:00
Tasks: Scheduling

The use of renewable energies strengthens decarbonization strategies. To integrate volatile renewable sources, energy systems require grid expansion, storage capabilities, or flexible consumption. This study focuses on industries that adapt production to real-time energy markets, offering flexible consumption to the grid. Flexible production considers not only traditional goals like minimizing production time, but also minimizing energy costs and emissions, thereby enhancing the sustainability of businesses. However, existing research focuses on single goals, neglects the combination of makespan, energy costs, and emissions, or assumes constant or periodic tariffs instead of a dynamic energy market. We present a novel memetic NSGA-III to minimize makespan, energy cost, and emissions, integrating real energy market data, and allowing manufacturers to adapt energy consumption to current grid conditions. Evaluating it with benchmark instances from literature and real energy market data, we explore the trade-offs between objectives, showcasing potential savings in energy costs and emissions on estimated Pareto fronts.
Automated Optimal Layout Generator for Animal Shelters: A framework based on Genetic Algorithm, TOPSIS and Graph Theory
Authors: Arghavan Jalayer, Masoud Jalayer, Mehdi Khakzand, Mohsen Faizi
Date: 2024-05-23 00:00:00

Overpopulation in animal shelters contributes to increased disease spread and higher expenses on animal healthcare, leading to fewer adoptions and more shelter deaths. Additionally, one of the greatest challenges that shelters face is the noise level in the dog kennel area, which is physically and physiologically hazardous for both animals and staff. This paper proposes a multi-criteria optimization framework to automatically design cage layouts that maximize shelter capacity, minimize tension in the dog kennel area by reducing the number of cages facing each other, and ensure accessibility for staff and visitors. The proposed framework uses a Genetic Algorithm (GA) to systematically generate and improve layouts. A novel graph theory-based algorithm is introduced to process solutions and calculate fitness values. Additionally, the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) is used to rank and sort the layouts in each iteration. The graph-based algorithm calculates variables such as cage accessibility and shortest paths to access points. Furthermore, a heuristic algorithm is developed to calculate layout scores based on the number of cages facing each other. This framework provides animal shelter management with a flexible decision-support system that allows for different strategies by assigning various weights to the TOPSIS criteria. Results from cats' and dogs' kennel areas show that the proposed framework can suggest optimal layouts that respect different priorities within acceptable runtimes.
Illustrating the Efficiency of Popular Evolutionary Multi-Objective Algorithms Using Runtime Analysis
Authors: Duc-Cuong Dang, Andre Opris, Dirk Sudholt
Date: 2024-05-22 00:00:00
Tasks: Diversity

Runtime analysis has recently been applied to popular evolutionary multi-objective (EMO) algorithms like NSGA-II in order to establish a rigorous theoretical foundation. However, most analyses showed that these algorithms have the same performance guarantee as the simple (G)SEMO algorithm. To our knowledge, there are no runtime analyses showing an advantage of a popular EMO algorithm over the simple algorithm for deterministic problems. We propose such a problem and use it to showcase the superiority of popular EMO algorithms over (G)SEMO: OneTrapZeroTrap is a straightforward generalization of the well-known Trap function to two objectives. We prove that, while GSEMO requires at least $n^n$ expected fitness evaluations to optimise OneTrapZeroTrap, popular EMO algorithms NSGA-II, NSGA-III and SMS-EMOA, all enhanced with a mild diversity mechanism of avoiding genotype duplication, only require $O(n \log n)$ expected fitness evaluations. Our analysis reveals the importance of the key components in each of these sophisticated algorithms and contributes to a better understanding of their capabilities.
Towards Exploratory Quality Diversity Landscape Analysis
Authors: Kyriacos Mosphilis, Vassilis Vassiliades
Date: 2024-05-22 00:00:00
Tasks: Diversity

This work is a preliminary study on using Exploratory Landscape Analysis (ELA) for Quality Diversity (QD) problems. We seek to understand whether ELA features can potentially be used to characterise QD problems paving the way for automating QD algorithm selection. Our results demonstrate that ELA features are affected by QD optimisation differently than random sampling, and more specifically, by the choice of variation operator, behaviour function, archive size and problem dimensionality.
An Efficient Approach for Solving Expensive Constrained Multiobjective Optimization Problems
Authors: Kamrul Hasan Rahi
Date: 2024-05-22 00:00:00
Tasks: Evolutionary Algorithms, Multiobjective Optimization

To solve real-world expensive constrained multi-objective optimization problems (ECMOPs), surrogate/approximation models are commonly incorporated in evolutionary algorithms to pre-select promising candidate solutions for evaluation. However, the performance of existing approaches are highly dependent on the relative position of unconstrained and constrained Pareto fronts (UPF and CPF, respectively). In addition, the uncertainty information of surrogate models is often ignored, which can misguide the search. To mitigate these key issues (among others), an efficient probabilistic selection based constrained multi-objective EA is proposed, referred to as PSCMOEA. It comprises novel elements such as (a) an adaptive search bound identification scheme based on the feasibility and convergence status of evaluated solutions (b) a probabilistic selection method backed by theoretical formulations of model mean and uncertainties to conduct search in the predicted space to identify promising solutions (c) an efficient single infill sampling approach to balance feasibility, convergence and diversity across different stages of the search and (d) an adaptive switch to unconstrained search based on certain search conditions. Numerical experiments are conducted on an extensive range of challenging constrained problems using low evaluation budgets to simulate ECMOPs. The performance of PSCMOEA is benchmarked against five competitive state-of-the-art algorithms, to demonstrate its competitive and consistent performance.
Combining Relevance and Magnitude for Resource-Aware DNN Pruning
Authors: Carla Fabiana Chiasserini, Francesco Malandrino, Nuria Molner, Zhiqiang Zhao
Date: 2024-05-21 00:00:00

Pruning neural networks, i.e., removing some of their parameters whilst retaining their accuracy, is one of the main ways to reduce the latency of a machine learning pipeline, especially in resource- and/or bandwidth-constrained scenarios. In this context, the pruning technique, i.e., how to choose the parameters to remove, is critical to the system performance. In this paper, we propose a novel pruning approach, called FlexRel and predicated upon combining training-time and inference-time information, namely, parameter magnitude and relevance, in order to improve the resulting accuracy whilst saving both computational resources and bandwidth. Our performance evaluation shows that FlexRel is able to achieve higher pruning factors, saving over 35% bandwidth for typical accuracy targets.
Zero-inflation in the Multivariate Poisson Lognormal Family
Authors: Bastien Batardière, Julien Chiquet, François Gindraud, Mahendra Mariadassou
Date: 2024-05-23 00:00:00
Tasks: Variational Inference

Analyzing high-dimensional count data is a challenge and statistical model-based approaches provide an adequate and efficient framework that preserves explainability. The (multivariate) Poisson-Log-Normal (PLN) model is one such model: it assumes count data are driven by an underlying structured latent Gaussian variable, so that the dependencies between counts solely stems from the latent dependencies. However PLN doesn't account for zero-inflation, a feature frequently observed in real-world datasets. Here we introduce the Zero-Inflated PLN (ZIPLN) model, adding a multivariate zero-inflated component to the model, as an additional Bernoulli latent variable. The Zero-Inflation can be fixed, site-specific, feature-specific or depends on covariates. We estimate model parameters using variational inference that scales up to datasets with a few thousands variables and compare two approximations: (i) independent Gaussian and Bernoulli variational distributions or (ii) Gaussian variational distribution conditioned on the Bernoulli one. The method is assessed on synthetic data and the efficiency of ZIPLN is established even when zero-inflation concerns up to $90\%$ of the observed counts. We then apply both ZIPLN and PLN to a cow microbiome dataset, containing $90.6\%$ of zeroes. Accounting for zero-inflation significantly increases log-likelihood and reduces dispersion in the latent space, thus leading to improved group discrimination.
Prediction of cancer dynamics under treatment using Bayesian neural networks: A simulated study
Authors: Even Moa Myklebust, Arnoldo Frigessi, Fredrik Schjesvold, Jasmine Foo, Kevin Leder, Alvaro Köhn-Luque
Date: 2024-05-23 00:00:00
Tasks: Time Series Prediction

Predicting cancer dynamics under treatment is challenging due to high inter-patient heterogeneity, lack of predictive biomarkers, and sparse and noisy longitudinal data. Mathematical models can summarize cancer dynamics by a few interpretable parameters per patient. Machine learning methods can then be trained to predict the model parameters from baseline covariates, but do not account for uncertainty in the parameter estimates. Instead, hierarchical Bayesian modeling can model the relationship between baseline covariates to longitudinal measurements via mechanistic parameters while accounting for uncertainty in every part of the model. The mapping from baseline covariates to model parameters can be modeled in several ways. A linear mapping simplifies inference but fails to capture nonlinear covariate effects and scale poorly for interaction modeling when the number of covariates is large. In contrast, Bayesian neural networks can potentially discover interactions between covariates automatically, but at a substantial cost in computational complexity. In this work, we develop a hierarchical Bayesian model of subpopulation dynamics that uses baseline covariate information to predict cancer dynamics under treatment, inspired by cancer dynamics in multiple myeloma (MM), where serum M protein is a well-known proxy of tumor burden. As a working example, we apply the model to a simulated dataset and compare its ability to predict M protein trajectories to a model with linear covariate effects. Our results show that the Bayesian neural network covariate effect model predicts cancer dynamics more accurately than a linear covariate effect model when covariate interactions are present. The framework can also be applied to other types of cancer or other time series prediction problems that can be described with a parametric model.
Semi-Discrete Optimal Transport: Nearly Minimax Estimation With Stochastic Gradient Descent and Adaptive Entropic Regularization
Authors: Ferdinand Genans, Antoine Godichon-Baggioni, François-Xavier Vialard, Olivier Wintenberger
Date: 2024-05-23 00:00:00

Optimal Transport (OT) based distances are powerful tools for machine learning to compare probability measures and manipulate them using OT maps. In this field, a setting of interest is semi-discrete OT, where the source measure $\mu$ is continuous, while the target $\nu$ is discrete. Recent works have shown that the minimax rate for the OT map is $\mathcal{O}(t^{-1/2})$ when using $t$ i.i.d. subsamples from each measure (two-sample setting). An open question is whether a better convergence rate can be achieved when the full information of the discrete measure $\nu$ is known (one-sample setting). In this work, we answer positively to this question by (i) proving an $\mathcal{O}(t^{-1})$ lower bound rate for the OT map, using the similarity between Laguerre cells estimation and density support estimation, and (ii) proposing a Stochastic Gradient Descent (SGD) algorithm with adaptive entropic regularization and averaging acceleration. To nearly achieve the desired fast rate, characteristic of non-regular parametric problems, we design an entropic regularization scheme decreasing with the number of samples. Another key step in our algorithm consists of using a projection step that permits to leverage the local strong convexity of the regularized OT problem. Our convergence analysis integrates online convex optimization and stochastic gradient techniques, complemented by the specificities of the OT semi-dual. Moreover, while being as computationally and memory efficient as vanilla SGD, our algorithm achieves the unusual fast rates of our theory in numerical experiments.
Causal Inference with Cocycles
Authors: Hugh Dance, Benjamin Bloem-Reddy
Date: 2024-05-22 00:00:00
Tasks: Causal Inference, counterfactual

Many interventions in causal inference can be represented as transformations. We identify a local symmetry property satisfied by a large class of causal models under such interventions. Where present, this symmetry can be characterized by a type of map called a cocycle, an object that is central to dynamical systems theory. We show that such cocycles exist under general conditions and are sufficient to identify interventional and counterfactual distributions. We use these results to derive cocycle-based estimators for causal estimands and show they achieve semiparametric efficiency under typical conditions. Since (infinitely) many distributions can share the same cocycle, these estimators make causal inference robust to mis-specification by sidestepping superfluous modelling assumptions. We demonstrate both robustness and state-of-the-art performance in several simulations, and apply our method to estimate the effects of 401(k) pension plan eligibility on asset accumulation using a real dataset.
Neural Networks-based Random Vortex Methods for Modelling Incompressible Flows
Authors: Vladislav Cherepanov, Sebastian W. Ertel
Date: 2024-05-22 00:00:00

In this paper we introduce a novel Neural Networks-based approach for approximating solutions to the (2D) incompressible Navier--Stokes equations, which is an extension of so called Deep Random Vortex Methods (DRVM), that does not require the knowledge of the Biot--Savart kernel associated to the computational domain. Our algorithm uses a Neural Network (NN), that approximates the vorticity based on a loss function that uses a computationally efficient formulation of the Random Vortex Dynamics. The neural vorticity estimator is then combined with traditional numerical PDE-solvers, which can be considered as a final implicit linear layer of the network, for the Poisson equation to compute the velocity field. The main advantage of our method compared to the standard DRVM and other NN-based numerical algorithms is that it strictly enforces physical properties, such as incompressibility or (no slip) boundary conditions, which might be hard to guarantee otherwise. The approximation abilities of our algorithm, and its capability for incorporating measurement data, are validated by several numerical experiments.
Exact Gradients for Stochastic Spiking Neural Networks Driven by Rough Signals
Authors: Christian Holberg, Cristopher Salvi
Date: 2024-05-22 00:00:00

We introduce a mathematically rigorous framework based on rough path theory to model stochastic spiking neural networks (SSNNs) as stochastic differential equations with event discontinuities (Event SDEs) and driven by c\`adl\`ag rough paths. Our formalism is general enough to allow for potential jumps to be present both in the solution trajectories as well as in the driving noise. We then identify a set of sufficient conditions ensuring the existence of pathwise gradients of solution trajectories and event times with respect to the network's parameters and show how these gradients satisfy a recursive relation. Furthermore, we introduce a general-purpose loss function defined by means of a new class of signature kernels indexed on c\`adl\`ag rough paths and use it to train SSNNs as generative models. We provide an end-to-end autodifferentiable solver for Event SDEs and make its implementation available as part of the $\texttt{diffrax}$ library. Our framework is, to our knowledge, the first enabling gradient-based training of SSNNs with noise affecting both the spike timing and the network's dynamics.