pwc-archive/papers-with-abstracts huggingface dataset:
Empirical analysis of PGA-MAP-Elites for Neuroevolution in Uncertain Domains
Authors: Manon Flageat, Felix Chalumeau, Antoine Cully
Date: 2022-10-24 00:00:00
Tasks: Deep Reinforcement Learning, Diversity

Quality-Diversity algorithms, among which MAP-Elites, have emerged as powerful alternatives to performance-only optimisation approaches as they enable generating collections of diverse and high-performing solutions to an optimisation problem. However, they are often limited to low-dimensional search spaces and deterministic environments. The recently introduced Policy Gradient Assisted MAP-Elites (PGA-MAP-Elites) algorithm overcomes this limitation by pairing the traditional Genetic operator of MAP-Elites with a gradient-based operator inspired by Deep Reinforcement Learning. This new operator guides mutations toward high-performing solutions using policy-gradients. In this work, we propose an in-depth study of PGA-MAP-Elites. We demonstrate the benefits of policy-gradients on the performance of the algorithm and the reproducibility of the generated solutions when considering uncertain domains. We first prove that PGA-MAP-Elites is highly performant in both deterministic and uncertain high-dimensional environments, decorrelating the two challenges it tackles. Secondly, we show that in addition to outperforming all the considered baselines, the collections of solutions generated by PGA-MAP-Elites are highly reproducible in uncertain environments, approaching the reproducibility of solutions found by Quality-Diversity approaches built specifically for uncertain applications. Finally, we propose an ablation and in-depth analysis of the dynamic of the policy-gradients-based variation. We demonstrate that the policy-gradient variation operator is determinant to guarantee the performance of PGA-MAP-Elites but is only essential during the early stage of the process, where it finds high-performing regions of the search space.
Towards Energy-Efficient, Low-Latency and Accurate Spiking LSTMs
Authors: Gourav Datta, Haoqin Deng, Robert Aviles, Peter A. Beerel
Date: 2022-10-23 00:00:00

Spiking Neural Networks (SNNs) have emerged as an attractive spatio-temporal computing paradigm for complex vision tasks. However, most existing works yield models that require many time steps and do not leverage the inherent temporal dynamics of spiking neural networks, even for sequential tasks. Motivated by this observation, we propose an \rev{optimized spiking long short-term memory networks (LSTM) training framework that involves a novel ANN-to-SNN conversion framework, followed by SNN training}. In particular, we propose novel activation functions in the source LSTM architecture and judiciously select a subset of them for conversion to integrate-and-fire (IF) activations with optimal bias shifts. Additionally, we derive the leaky-integrate-and-fire (LIF) activation functions converted from their non-spiking LSTM counterparts which justifies the need to jointly optimize the weights, threshold, and leak parameter. We also propose a pipelined parallel processing scheme which hides the SNN time steps, significantly improving system latency, especially for long sequences. The resulting SNNs have high activation sparsity and require only accumulate operations (AC), in contrast to expensive multiply-and-accumulates (MAC) needed for ANNs, except for the input layer when using direct encoding, yielding significant improvements in energy efficiency. We evaluate our framework on sequential learning tasks including temporal MNIST, Google Speech Commands (GSC), and UCI Smartphone datasets on different LSTM architectures. We obtain test accuracy of 94.75% with only 2 time steps with direct encoding on the GSC dataset with 4.1x lower energy than an iso-architecture standard LSTM.
A Trainable Sequence Learner that Learns and Recognizes Two-Input Sequence Patterns
Authors: Jan Hohenheim, Zhaoyu Devon Liu, Tommaso Stecconi, Pietro Palopoli
Date: 2022-10-21 00:00:00

We present two designs for an analog circuit that can learn to detect a temporal sequence of two inputs. The training phase is done by feeding the circuit with the desired sequence and, after the training is completed, each time the trained sequence is encountered again the circuit will emit a signal of correct recognition. Sequences are in the order of tens of nanoseconds. The first design can reset the trained sequence on runtime but assumes very strict timing of the inputs. The second design can only be trained once but is lenient in the input's timing.
A Dimension-Augmented Physics-Informed Neural Network (DaPINN) with High Level Accuracy and Efficiency
Authors: Weilong Guan, Kaihan Yang, Yinsheng Chen, Zhong Guan
Date: 2022-10-19 00:00:00
Tasks: Inductive Bias

Physics-informed neural networks (PINNs) have been widely applied in different fields due to their effectiveness in solving partial differential equations (PDEs). However, the accuracy and efficiency of PINNs need to be considerably improved for scientific and commercial use. To address this issue, we systematically propose a novel dimension-augmented physics-informed neural network (DaPINN), which simultaneously and significantly improves the accuracy and efficiency of the PINN. In the DaPINN model, we introduce inductive bias in the neural network to enhance network generalizability by adding a special regularization term to the loss function. Furthermore, we manipulate the network input dimension by inserting additional sample features and incorporating the expanded dimensionality in the loss function. Moreover, we verify the effectiveness of power series augmentation, Fourier series augmentation and replica augmentation, in both forward and backward problems. In most experiments, the error of DaPINN is 1$\sim$2 orders of magnitude lower than that of PINN. The results show that the DaPINN outperforms the original PINN in terms of both accuracy and efficiency with a reduced dependence on the number of sample points. We also discuss the complexity of the DaPINN and its compatibility with other methods.
Artificial Life using a Book and Bookmarker
Authors: Keishu Utimula
Date: 2022-10-07 00:00:00
Tasks: Artificial Life

Reproduction, development, and individual interactions are essential topics in artificial life. The cellular automata, which can handle these in a composite way, is highly restricted in its form and behavior because it represents life as a pattern of cells. In contrast, the virtual creatures proposed by Karl Sims have a very high degree of freedom in terms of morphology and behavior. However, they have limited expressive capacity in terms of those viewpoints. This study carefully extracts the characteristics of the cellular automata and Sims models to propose a new artificial life model that can simulate reproduction, development, and individual interactions while exhibiting high expressive power for morphology and behavior. The simulation was performed by sequentially reading a book with genetic information and repeatedly executing four actions: expansion, connection, disconnection, and transition. The virtual creatures in the proposed model exhibit unique survival strategies and lifestyles and acquire interesting properties in reproduction, development, and individual interactions while having freedom in morphology and behavior.
Protocols for classically training quantum generative models on probability distributions
Authors: Sachin Kasture, Oleksandr Kyriienko, Vincent E. Elfving
Date: 2022-10-24 00:00:00

Quantum Generative Modelling (QGM) relies on preparing quantum states and generating samples from these states as hidden - or known - probability distributions. As distributions from some classes of quantum states (circuits) are inherently hard to sample classically, QGM represents an excellent testbed for quantum supremacy experiments. Furthermore, generative tasks are increasingly relevant for industrial machine learning applications, and thus QGM is a strong candidate for demonstrating a practical quantum advantage. However, this requires that quantum circuits are trained to represent industrially relevant distributions, and the corresponding training stage has an extensive training cost for current quantum hardware in practice. In this work, we propose protocols for classical training of QGMs based on circuits of the specific type that admit an efficient gradient computation, while remaining hard to sample. In particular, we consider Instantaneous Quantum Polynomial (IQP) circuits and their extensions. Showing their classical simulability in terms of the time complexity, sparsity and anti-concentration properties, we develop a classically tractable way of simulating their output probability distributions, allowing classical training to a target probability distribution. The corresponding quantum sampling from IQPs can be performed efficiently, unlike when using classical sampling. We numerically demonstrate the end-to-end training of IQP circuits using probability distributions for up to 30 qubits on a regular desktop computer. When applied to industrially relevant distributions this combination of classical training with quantum sampling represents an avenue for reaching advantage in the NISQ era.
Contraction of Locally Differentially Private Mechanisms
Authors: Shahab Asoodeh, Huanyu Zhang
Date: 2022-10-24 00:00:00
Tasks: Density Estimation

We investigate the contraction properties of locally differentially private mechanisms. More specifically, we derive tight upper bounds on the divergence between $PK$ and $QK$ output distributions of an $\epsilon$-LDP mechanism $K$ in terms of a divergence between the corresponding input distributions $P$ and $Q$, respectively. Our first main technical result presents a sharp upper bound on the $\chi^2$-divergence $\chi^2(PK}\|QK)$ in terms of $\chi^2(P\|Q)$ and $\varepsilon$. We also show that the same result holds for a large family of divergences, including KL-divergence and squared Hellinger distance. The second main technical result gives an upper bound on $\chi^2(PK\|QK)$ in terms of total variation distance $\mathsf{TV}(P, Q)$ and $\epsilon$. We then utilize these bounds to establish locally private versions of the van Trees inequality, Le Cam's, Assouad's, and the mutual information methods, which are powerful tools for bounding minimax estimation risks. These results are shown to lead to better privacy analyses than the state-of-the-arts in several statistical problems such as entropy and discrete distribution estimation, non-parametric density estimation, and hypothesis testing.
Applications of Machine Learning in Pharmacogenomics: Clustering Plasma Concentration-Time Curves
Authors: Jackson P. Lautier, Stella Grosser, Jessica Kim, Hyewon Kim, Junghi Kim
Date: 2022-10-24 00:00:00
Tasks: Clustering, Data Visualization, Descriptive, Dynamic Time Warping, Time Series, Time Series Analysis

Pharmaceutical researchers are continually searching for techniques to improve both drug development processes and patient outcomes. An area of recent interest is the potential for machine learning (ML) applications within pharmacology. One such application not yet given close study is the unsupervised clustering of plasma concentration-time curves, hereafter, pharmacokinetic (PK) curves. In this paper, we present our findings on how to cluster PK curves by their similarity. Specifically, we find clustering to be effective at identifying similar-shaped PK curves and informative for understanding patterns within each cluster of PK curves. Because PK curves are time series data objects, our approach utilizes the extensive body of research related to the clustering of time series data as a starting point. As such, we examine many dissimilarity measures between time series data objects to find those most suitable for PK curves. We identify Euclidean distance as generally most appropriate for clustering PK curves, and we further show that dynamic time warping, Fr\'{e}chet, and structure-based measures of dissimilarity like correlation may produce unexpected results. As an illustration, we apply these methods in a case study with 250 PK curves used in a previous pharmacogenomic study. Our case study finds that an unsupervised ML clustering with Euclidean distance, without any subject genetic information, is able to independently validate the same conclusions as the reference pharmacogenomic results. To our knowledge, this is the first such demonstration. Further, the case study demonstrates how the clustering of PK curves may generate insights that could be difficult to perceive solely with population level summary statistics of PK metrics.
Optimal Discriminant Analysis in High-Dimensional Latent Factor Models
Authors: Xin Bing, Marten Wegkamp
Date: 2022-10-23 00:00:00
Tasks: valid, Vocal Bursts Intensity Prediction

In high-dimensional classification problems, a commonly used approach is to first project the high-dimensional features into a lower dimensional space, and base the classification on the resulting lower dimensional projections. In this paper, we formulate a latent-variable model with a hidden low-dimensional structure to justify this two-step procedure and to guide which projection to choose. We propose a computationally efficient classifier that takes certain principal components (PCs) of the observed features as projections, with the number of retained PCs selected in a data-driven way. A general theory is established for analyzing such two-step classifiers based on any projections. We derive explicit rates of convergence of the excess risk of the proposed PC-based classifier. The obtained rates are further shown to be optimal up to logarithmic factors in the minimax sense. Our theory allows the lower-dimension to grow with the sample size and is also valid even when the feature dimension (greatly) exceeds the sample size. Extensive simulations corroborate our theoretical findings. The proposed method also performs favorably relative to other existing discriminant methods on three real data examples.
Transport Reversible Jump Proposals
Authors: Laurence Davies, Robert Salomone, Matthew Sutton, Christopher Drovandi
Date: 2022-10-22 00:00:00
Tasks: Density Estimation

Reversible jump Markov chain Monte Carlo (RJMCMC) proposals that achieve reasonable acceptance rates and mixing are notoriously difficult to design in most applications. Inspired by recent advances in deep neural network-based normalizing flows and density estimation, we demonstrate an approach to enhance the efficiency of RJMCMC sampling by performing transdimensional jumps involving reference distributions. In contrast to other RJMCMC proposals, the proposed method is the first to apply a non-linear transport-based approach to construct efficient proposals between models with complicated dependency structures. It is shown that, in the setting where exact transports are used, our RJMCMC proposals have the desirable property that the acceptance probability depends only on the model probabilities. Numerical experiments demonstrate the efficacy of the approach.
Online Information Retrieval Evaluation using the STELLA Framework
Authors: Timo Breuer, Narges Tavakolpoursaleh, Johann Schaible, Daniel Hienert, Philipp Schaer, Leyla Jael Castro
Date: 2022-10-24 00:00:00
Tasks: Information Retrieval, Recommendation Systems, Retrieval

Involving users in early phases of software development has become a common strategy as it enables developers to consider user needs from the beginning. Once a system is in production, new opportunities to observe, evaluate and learn from users emerge as more information becomes available. Gathering information from users to continuously evaluate their behavior is a common practice for commercial software, while the Cranfield paradigm remains the preferred option for Information Retrieval (IR) and recommendation systems in the academic world. Here we introduce the Infrastructures for Living Labs STELLA project which aims to create an evaluation infrastructure allowing experimental systems to run along production web-based academic search systems with real users. STELLA combines user interactions and log files analyses to enable large-scale A/B experiments for academic search.
Clarinet: A Music Retrieval System
Authors: Kshitij Alwadhi, Rohan Sharma, Siddhant Sharma
Date: 2022-10-23 00:00:00
Tasks: Melody Extraction, Retrieval

A MIDI based approach for music recognition is proposed and implemented in this paper. Our Clarinet music retrieval system is designed to search piano MIDI files with high recall and speed. We design a novel melody extraction algorithm that improves recall results by more than 10%. We also implement 3 algorithms for retrieval-two self designed (RSA Note and RSA Time), and a modified version of the Mongeau Sankoff Algorithm. Algorithms to achieve tempo and scale invariance are also discussed in this paper. The paper also contains detailed experimentation and benchmarks with four different metrics. Clarinet achieves recall scores of more than 94%.
Aircraft Classification Based on PCA and Feature Fusion Techniques in Convolutional Neural Network
Authors: F. Azam, A. Rizvi, W. Z. Khan, M. Y. Aalsalem, H. Yu and Y. B. Zikria
Date: 2021-12-01 00:00:00
Proceeding: IEEE Access, vol. 9, pp 2021 12
Tasks: feature selection

The characterization of aircraft in remote sensing satellite imagery has many armed and
civil applications. For civil purposes, such as in tragedy and emergency aircraft searching, airport scrutiny
and aircraft identification from satellite images are very important. This study presents an automated
methodology based on handcrafted and deep convolutional neural network (DCNN) features. The presented
aircraft classification technique consists of the following steps. The handcrafted features achieved from
a local binary pattern (LBP) and DCNN are fused by feature fusion techniques. The DCNN features are
extracted from Alexnet and Inception V3. Then we adopted a feature selection technique called principal
component analysis (PCA). PCA removes the redundant and irrelevant information and improves the
classification performance. Then, Famous supervised methodologies categorize these selected features.
We chose the best classifier based on its highest accuracy. The proposed technique is executed on the
multi-type aircraft remote sensing images (MTARSI) dataset, and the overall highest accuracy that we
achieved from our proposed method is 96.8% by the linear support vector machine (SVM) classifier.
Contrastive Prototypical Network with Wasserstein Confidence Penalty
Authors: Haoqing Wang, Zhi-Hong Deng
Date: 2022-10-21 00:00:00
Proceeding: European Conference on Computer Vision 2022 10
Tasks: Contrastive Learning, Few-Shot Learning, Inductive Bias, Unsupervised Few-Shot Image Classification, Unsupervised Few-Shot Learning

Unsupervised few-shot learning aims to learn the inductive bias from unlabeled dataset for solving the novel few-shot tasks. The existing unsupervised few-shot learning models and the contrastive learning models follow a unified paradigm. Therefore, we conduct empirical study under this paradigm and find that pairwise contrast, meta losses and large batch size are the important design factors. This results in our CPN (Contrastive Prototypical Network) model, which combines the prototypical loss with pairwise contrast and outperforms the existing models from this paradigm with modestly large batch size. Furthermore, the one-hot prediction target in CPN could lead to learning the sample-specific information. To this end, we propose Wasserstein Confidence Penalty which can impose appropriate penalty on overconfident predictions based on the semantic relationships among pseudo classes. Our full model, CPNWCP (Contrastive Prototypical Network with Wasserstein Confidence Penalty), achieves state-of-the-art performance on miniImageNet and tieredImageNet under unsupervised setting. Our code is available at https://github.com/Haoqing-Wang/CPNWCP.
LungAttn: advanced lung sound classification using attention mechanism with dual TQWT and triple STFT spectrogram
Authors: Jizuo Li, Jiajun Yuan, Hansong Wang, Shijian Liu, Qianyu Guo, Yi Ma, Yongfu Li, Liebin Zhao, Guoxing Wang
Date: 2021-10-29 00:00:00
Proceeding: Physiological Measurement 2021 10
Tasks: Audio Classification, Lung Sound Classification, Sound Classification, Specificity

Objective. Auscultation of lung sound plays an important role in the early diagnosis of lung diseases. This work aims to develop an automated adventitious lung sound detection method to reduce the workload of physicians.Approach. We propose a deep learning architecture, LungAttn, which incorporates augmented attention convolution into ResNet block to improve the classification accuracy of lung sound. We adopt a feature extraction method based on dual tunable Q-factor wavelet transform and triple short-time Fourier transform to obtain a multi-channel spectrogram. Mixup method is introduced to augment adventitious lung sound recordings to address the imbalance dataset problem.Main results. Based on the ICBHI 2017 challenge dataset, we implement our framework and compare with the state-of-the-art works. Experimental results show that LungAttn has achieved the Sensitivity, Se, Specificity, Sp and Score of 36.36%, 71.44% and 53.90%, respectively. Of which, our work has improved the Scoreby 1.69% compared to the state-of-the-art models based on the official ICBHI 2017 dataset splitting method.Significance. Multi-channel spectrogram based on different oscillatory behavior of adventitious lung sound provides necessary information of lung sound recordings. Attention mechanism is introduced to lung sound classification methods and has proved to be effective. The proposed LungAttn model can potentially improve the speed and accuracy of lung sound classification in clinical practice.
Localization and Clustering Based on Swarm Intelligence in UAV Networks for Emergency Communications
Authors: Muhammad Yeasir Arafat, Sangman Moh, Member, IEEE
Date: 2019-07-28 00:00:00
Proceeding: IEEE Internet of Things Journal 2019 7
Tasks: Clustering

Abstract— In recent years, unmanned aerial vehicle (UAV) 
networks have been a focus area of the academic and industrial 
research community. They have been used in many military and 
civilian applications. Emergency communication is one of the 
essential requirements for first responders and victims in the 
aftermath of natural disasters. In such scenarios, UAVs may 
configure ad hoc wireless networks to cover a large area. In UAV 
networks, however, localization and routing are challenging tasks 
owing to the high mobility, unstable links, dynamic topology, and 
limited energy of UAVs. Here, we propose 
swarm-intelligence-based localization and clustering schemes in
UAV networks for emergency communications. First, we propose 
a new three-dimensional (3D) swarm-intelligence-based 
localization (SIL) algorithm based on particle swarm optimization 
(PSO) that exploits the particle search space in a limited boundary 
by using the bounding box method. In the 3D search space, anchor 
UAV nodes are randomly distributed and the SIL algorithm 
measures the distance to existing anchor nodes for estimating the 
location of the target UAV nodes. Convergence time and 
localization accuracy are improved with lower computational cost. 
Second, we propose an energy-efficient swarm-intelligence-based
clustering (SIC) algorithm based on PSO, in which the particle 
fitness function is exploited for inter-cluster distance, intra-cluster 
distance, residual energy, and geographic location. For 
energy-efficient clustering, cluster heads are selected based on 
improved particle optimization. The proposed SIC outperforms 
five typical routing protocols regarding packet delivery ratio,
average end-to-end delay, and routing overhead. Moreover, SIC 
consumes less energy and prolongs network lifetime.