pwc-archive/papers-with-abstracts huggingface dataset:
Machine Learning assisted Chimera and Solitary states in Networks
Authors: Niraj Kushwaha, Naveen Kumar Mendola, Saptarshi Ghosh, Ajay Deep Kachhvah, Sarika Jalan
Date: 2020-11-02 00:00:00
Tasks: BIG-bench Machine Learning, Specificity

Chimera and Solitary states have captivated scientists and engineers due to their peculiar dynamical states corresponding to the co-existence of coherent and incoherent dynamical evolution in coupled units in various natural and artificial systems. It has been further demonstrated that such states can be engineered in systems of coupled oscillators by the suitable implementation of communication delays. Here, using supervised machine learning, we predict (a) the precise value of delay which is sufficient for engineering chimera and solitary states for a given set of system parameters, as well as (b) the intensity of incoherence for such engineered states. The results are demonstrated for two different examples consisting of single layer and multi layer networks. First, the chimera states (solitary states) are engineered by establishing delays in the neighboring links of a node (the interlayer links) in a 2-D lattice (multiplex network) of oscillators. Then, different machine learning classifiers, KNN, SVM and MLP-Neural Network are employed by feeding the data obtained from the network models. Once a machine learning model is trained using a limited amount of data, it makes predictions for a given unknown systems parameter values. Testing accuracy, sensitivity, and specificity analysis reveal that MLP-NN classifier is better suited than Knn or SVM classifier for the predictions of parameters values for engineered chimera and solitary states. The technique provides an easy methodology to predict critical delay values as well as the intensity of incoherence for designing an experimental setup to create solitary and chimera states.
Dynamic Community Detection into Analyzing of Wildfires Events
Authors: Alessandra Marli, Didier A Vega-Oliveros, Moshé Cotacallapa, Leonardo N Ferreira, Elbert EN Macau, Marcos G Quiles
Date: 2020-11-02 00:00:00
Tasks: Community Detection, Dynamic Community Detection

The study and comprehension of complex systems are crucial intellectual and scientific challenges of the 21st century. In this scenario, network science has emerged as a mathematical tool to support the study of such systems. Examples include environmental processes such as wildfires, which are known for their considerable impact on human life. However, there is a considerable lack of studies of wildfire from a network science perspective. Here, employing the chronological network concept -- a temporal network where nodes are linked if two consecutive events occur between them -- we investigate the information that dynamic community structures reveal about the wildfires' dynamics. Particularly, we explore a two-phase dynamic community detection approach, i.e., we applied the Louvain algorithm on a series of snapshots. Then we used the Jaccard similarity coefficient to match communities across adjacent snapshots. Experiments with the MODIS dataset of fire events in the Amazon basing were conducted. Our results show that the dynamic communities can reveal wildfire patterns observed throughout the year.
Multi-IRS-assisted Multi-Cell Uplink MIMO Communications under Imperfect CSI: A Deep Reinforcement Learning Approach
Authors: JungHoon Kim, Seyyedali Hosseinalipour, Taejoon Kim, David J. Love, Christopher G. Brinton
Date: 2020-11-02 00:00:00
Tasks: Decision Making, Deep Reinforcement Learning, Sequential Decision Making

Applications of intelligent reflecting surfaces (IRSs) in wireless networks have attracted significant attention recently. Most of the relevant literature is focused on the single cell setting where a single IRS is deployed and perfect channel state information (CSI) is assumed. In this work, we develop a novel methodology for multi-IRS-assisted multi-cell networks in the uplink. We consider the scenario in which (i) channels are dynamic and (ii) only partial CSI is available at each base station (BS); specifically, scalar effective channel powers from only a subset of user equipments (UE). We formulate the sum-rate maximization problem aiming to jointly optimize the IRS reflect beamformers, BS combiners, and UE transmit powers. In casting this as a sequential decision making problem, we propose a multi-agent deep reinforcement learning algorithm to solve it, where each BS acts as an independent agent in charge of tuning the local UE transmit powers, the local IRS reflect beamformer, and its combiners. We introduce an efficient information-sharing scheme that requires limited information exchange among neighboring BSs to cope with the non-stationarity caused by the coupling of actions taken by multiple BSs. Our numerical results show that our method obtains substantial improvement in average data rate compared to baseline approaches, e.g., fixed UE transmit power and maximum ratio combining.
Improved Max-value Entropy Search for Multi-objective Bayesian Optimization with Constraints
Authors: Daniel Fernández-Sánchez, Eduardo C. Garrido-Merchán, Daniel Hernández-Lobato
Date: 2020-11-02 00:00:00
Tasks: Bayesian Optimization

We present MESMOC+, an improved version of Max-value Entropy search for Multi-Objective Bayesian optimization with Constraints (MESMOC). MESMOC+ can be used to solve constrained multi-objective problems when the objectives and the constraints are expensive to evaluate. MESMOC+ works by minimizing the entropy of the solution of the optimization problem in function space, i.e., the Pareto frontier, to guide the search for the optimum. The cost of MESMOC+ is linear in the number of objectives and constraints. Furthermore, it is often significantly smaller than the cost of alternative methods based on minimizing the entropy of the Pareto set. The reason for this is that it is easier to approximate the required computations in MESMOC+. Moreover, MESMOC+'s acquisition function is expressed as the sum of one acquisition per each black-box (objective or constraint). Thus, it can be used in a decoupled evaluation setting in which one chooses not only the next input location to evaluate, but also which black-box to evaluate there. We compare MESMOC+ with related methods in synthetic and real optimization problems. These experiments show that the entropy estimation provided by MESMOC+ is more accurate than that of previous methods. This leads to better optimization results. MESMOC+ is also competitive with other information-based methods for constrained multi-objective Bayesian optimization, but it is significantly faster.
Optimize what matters: Training DNN-HMM Keyword Spotting Model Using End Metric
Authors: Ashish Shrivastava, Arnav Kundu, Chandra Dhir, Devang Naik, Oncel Tuzel
Date: 2020-11-02 00:00:00
Tasks: Decoder, Keyword Spotting

Deep Neural Network--Hidden Markov Model (DNN-HMM) based methods have been successfully used for many always-on keyword spotting algorithms that detect a wake word to trigger a device. The DNN predicts the state probabilities of a given speech frame, while HMM decoder combines the DNN predictions of multiple speech frames to compute the keyword detection score. The DNN, in prior methods, is trained independent of the HMM parameters to minimize the cross-entropy loss between the predicted and the ground-truth state probabilities. The mis-match between the DNN training loss (cross-entropy) and the end metric (detection score) is the main source of sub-optimal performance for the keyword spotting task. We address this loss-metric mismatch with a novel end-to-end training strategy that learns the DNN parameters by optimizing for the detection score. To this end, we make the HMM decoder (dynamic programming) differentiable and back-propagate through it to maximize the score for the keyword and minimize the scores for non-keyword speech segments. Our method does not require any change in the model architecture or the inference framework; therefore, there is no overhead in run-time memory or compute requirements. Moreover, we show significant reduction in false rejection rate (FRR) at the same false trigger experience (> 70% over independent DNN training).
Learning to Maximize Speech Quality Directly Using MOS Prediction for Neural Text-to-Speech
Authors: Yeunju Choi, Youngmoon Jung, Youngjoo Suh, Hoirin Kim
Date: 2020-11-02 00:00:00
Tasks: Knowledge Distillation, Speech Synthesis, text-to-speech, Text to Speech, Voice Conversion

Although recent neural text-to-speech (TTS) systems have achieved high-quality speech synthesis, there are cases where a TTS system generates low-quality speech, mainly caused by limited training data or information loss during knowledge distillation. Therefore, we propose a novel method to improve speech quality by training a TTS model under the supervision of perceptual loss, which measures the distance between the maximum possible speech quality score and the predicted one. We first pre-train a mean opinion score (MOS) prediction model and then train a TTS model to maximize the MOS of synthesized speech using the pre-trained MOS prediction model. The proposed method can be applied independently regardless of the TTS model architecture or the cause of speech quality degradation and efficiently without increasing the inference time or model complexity. The evaluation results for the MOS and phone error rate demonstrate that our proposed approach improves previous models in terms of both naturalness and intelligibility.
Adversarial Examples in Constrained Domains
Authors: Ryan Sheatsley, Nicolas Papernot, Michael Weisman, Gunjan Verma, Patrick McDaniel
Date: 2020-11-02 00:00:00
Tasks: Intrusion Detection, Network Intrusion Detection

Machine learning algorithms have been shown to be vulnerable to adversarial manipulation through systematic modification of inputs (e.g., adversarial examples) in domains such as image recognition. Under the default threat model, the adversary exploits the unconstrained nature of images; each feature (pixel) is fully under control of the adversary. However, it is not clear how these attacks translate to constrained domains that limit which and how features can be modified by the adversary (e.g., network intrusion detection). In this paper, we explore whether constrained domains are less vulnerable than unconstrained domains to adversarial example generation algorithms. We create an algorithm for generating adversarial sketches: targeted universal perturbation vectors which encode feature saliency within the envelope of domain constraints. To assess how these algorithms perform, we evaluate them in constrained (e.g., network intrusion detection) and unconstrained (e.g., image recognition) domains. The results demonstrate that our approaches generate misclassification rates in constrained domains that were comparable to those of unconstrained domains (greater than 95%). Our investigation shows that the narrow attack surface exposed by constrained domains is still sufficiently large to craft successful adversarial examples; and thus, constraints do not appear to make a domain robust. Indeed, with as little as five randomly selected features, one can still generate adversarial examples.
Focus on the present: a regularization method for the ASR source-target attention layer
Authors: Nanxin Chen, Piotr Żelasko, Jesús Villalba, Najim Dehak
Date: 2020-11-02 00:00:00
Tasks: Decoder, speech-recognition, Speech Recognition

This paper introduces a novel method to diagnose the source-target attention in state-of-the-art end-to-end speech recognition models with joint connectionist temporal classification (CTC) and attention training. Our method is based on the fact that both, CTC and source-target attention, are acting on the same encoder representations. To understand the functionality of the attention, CTC is applied to compute the token posteriors given the attention outputs. We found that the source-target attention heads are able to predict several tokens ahead of the current one. Inspired by the observation, a new regularization method is proposed which leverages CTC to make source-target attention more focused on the frames corresponding to the output token being predicted by the decoder. Experiments reveal stable improvements up to 7\% and 13\% relatively with the proposed regularization on TED-LIUM 2 and LibriSpeech.
Strongly universally consistent nonparametric regression and classification with privatised data
Authors: Thomas Berrett, László Györfi, Harro Walk
Date: 2020-10-31 00:00:00
Tasks: Binary Classification, General Classification, regression

In this paper we revisit the classical problem of nonparametric regression, but impose local differential privacy constraints. Under such constraints, the raw data $(X_1,Y_1),\ldots,(X_n,Y_n)$, taking values in $\mathbb{R}^d \times \mathbb{R}$, cannot be directly observed, and all estimators are functions of the randomised output from a suitable privacy mechanism. The statistician is free to choose the form of the privacy mechanism, and here we add Laplace distributed noise to a discretisation of the location of a feature vector $X_i$ and to the value of its response variable $Y_i$. Based on this randomised data, we design a novel estimator of the regression function, which can be viewed as a privatised version of the well-studied partitioning regression estimator. The main result is that the estimator is strongly universally consistent. Our methods and analysis also give rise to a strongly universally consistent binary classification rule for locally differentially private data.
Fast Network Community Detection with Profile-Pseudo Likelihood Methods
Authors: Jiangzhou Wang, Jingfei Zhang, Binghui Liu, Ji Zhu, Jianhua Guo
Date: 2020-11-01 00:00:00
Tasks: Community Detection, Stochastic Block Model

The stochastic block model is one of the most studied network models for community detection. It is well-known that most algorithms proposed for fitting the stochastic block model likelihood function cannot scale to large-scale networks. One prominent work that overcomes this computational challenge is Amini et al.(2013), which proposed a fast pseudo-likelihood approach for fitting stochastic block models to large sparse networks. However, this approach does not have convergence guarantee, and is not well suited for small- or medium- scale networks. In this article, we propose a novel likelihood based approach that decouples row and column labels in the likelihood function, which enables a fast alternating maximization; the new method is computationally efficient, performs well for both small and large scale networks, and has provable convergence guarantee. We show that our method provides strongly consistent estimates of the communities in a stochastic block model. As demonstrated in simulation studies, the proposed method outperforms the pseudo-likelihood approach in terms of both estimation accuracy and computation efficiency, especially for large sparse networks. We further consider extensions of our proposed method to handle networks with degree heterogeneity and bipartite properties.
c-lasso -- a Python package for constrained sparse and robust regression and classification
Authors: Léo Simpson, Patrick L. Combettes, Christian L. Müller
Date: 2020-11-02 00:00:00
Tasks: General Classification, regression

We introduce c-lasso, a Python package that enables sparse and robust linear regression and classification with linear equality constraints. The underlying statistical forward model is assumed to be of the following form: \[ y = X \beta + \sigma \epsilon \qquad \textrm{subject to} \qquad C\beta=0 \] Here, $X \in \mathbb{R}^{n\times d}$is a given design matrix and the vector $y \in \mathbb{R}^{n}$ is a continuous or binary response vector. The matrix $C$ is a general constraint matrix. The vector $\beta \in \mathbb{R}^{d}$ contains the unknown coefficients and $\sigma$ an unknown scale. Prominent use cases are (sparse) log-contrast regression with compositional data $X$, requiring the constraint $1_d^T \beta = 0$ (Aitchion and Bacon-Shone 1984) and the Generalized Lasso which is a special case of the described problem (see, e.g, (James, Paulson, and Rusmevichientong 2020), Example 3). The c-lasso package provides estimators for inferring unknown coefficients and scale (i.e., perspective M-estimators (Combettes and M\"uller 2020a)) of the form \[ \min_{\beta \in \mathbb{R}^d, \sigma \in \mathbb{R}_{0}} f\left(X\beta - y,{\sigma} \right) + \lambda \left\lVert \beta\right\rVert_1 \qquad \textrm{subject to} \qquad C\beta = 0 \] for several convex loss functions $f(\cdot,\cdot)$. This includes the constrained Lasso, the constrained scaled Lasso, and sparse Huber M-estimators with linear equality constraints.
Sampling Algorithms, from Survey Sampling to Monte Carlo Methods: Tutorial and Literature Review
Authors: Benyamin Ghojogh, Hadi Nekoei, Aydin Ghojogh, Fakhri Karray, Mark Crowley
Date: 2020-11-02 00:00:00
Tasks: Survey Sampling

This paper is a tutorial and literature review on sampling algorithms. We have two main types of sampling in statistics. The first type is survey sampling which draws samples from a set or population. The second type is sampling from probability distribution where we have a probability density or mass function. In this paper, we cover both types of sampling. First, we review some required background on mean squared error, variance, bias, maximum likelihood estimation, Bernoulli, Binomial, and Hypergeometric distributions, the Horvitz-Thompson estimator, and the Markov property. Then, we explain the theory of simple random sampling, bootstrapping, stratified sampling, and cluster sampling. We also briefly introduce multistage sampling, network sampling, and snowball sampling. Afterwards, we switch to sampling from distribution. We explain sampling from cumulative distribution function, Monte Carlo approximation, simple Monte Carlo methods, and Markov Chain Monte Carlo (MCMC) methods. For simple Monte Carlo methods, whose iterations are independent, we cover importance sampling and rejection sampling. For MCMC methods, we cover Metropolis algorithm, Metropolis-Hastings algorithm, Gibbs sampling, and slice sampling. Then, we explain the random walk behaviour of Monte Carlo methods and more efficient Monte Carlo methods, including Hamiltonian (or hybrid) Monte Carlo, Adler's overrelaxation, and ordered overrelaxation. Finally, we summarize the characteristics, pros, and cons of sampling methods compared to each other. This paper can be useful for different fields of statistics, machine learning, reinforcement learning, and computational physics.
LRA: an accelerated rough set framework based on local redundancy of attribute for feature selection
Authors: Shuyin Xia, Wenhua Li, Guoyin Wang, Xinbo Gao, Changqing Zhang, Elisabeth Giem
Date: 2020-10-31 00:00:00
Tasks: Attribute, feature selection

In this paper, we propose and prove the theorem regarding the stability of attributes in a decision system. Based on the theorem, we propose the LRA framework for accelerating rough set algorithms. It is a general-purpose framework which can be applied to almost all rough set methods significantly . Theoretical analysis guarantees high efficiency. Note that the enhancement of efficiency will not lead to any decrease of the classification accuracy. Besides, we provide a simpler prove for the positive approximation acceleration framework.
A Curious New Result of Resolution Strategies in Negation-Limited Inverters Problem
Authors: Ruo Ando, Yoshiyasu Takefuji
Date: 2020-11-02 00:00:00
Tasks: Automated Theorem Proving, Negation

Generally, negation-limited inverters problem is known as a puzzle of constructing an inverter with AND gates and OR gates and a few inverters. In this paper, we introduce a curious new result about the effectiveness of two powerful ATP (Automated Theorem Proving) strategies on tackling negation limited inverter problem. Two resolution strategies are UR (Unit Resulting) resolution and hyper-resolution. In experiment, we come two kinds of automated circuit construction: 3 input/output inverters and 4 input/output BCD Counter Circuit. Both circuits are constructed with a few limited inverters. Curiously, it has been turned out that UR resolution is drastically faster than hyper-resolution in the measurement of the size of SOS (Set of Support). Besides, we discuss the syntactic and semantic criteria which might causes considerable difference of computation cost between UR resolution and hyper-resolution.
Domain-specific Knowledge Graphs: A survey
Authors: Bilal Abu-Salih
Date: 2020-10-31 00:00:00
Tasks: Knowledge Graphs, Survey

Knowledge Graphs (KGs) have made a qualitative leap and effected a real revolution in knowledge representation. This is leveraged by the underlying structure of the KG which underpins a better comprehension, reasoning and interpretation of knowledge for both human and machine. Therefore, KGs continue to be used as the main means of tackling a plethora of real-life problems in various domains. However, there is no consensus in regard to a plausible and inclusive definition of a domain-specific KG. Further, in conjunction with several limitations and deficiencies, various domain-specific KG construction approaches are far from perfect. This survey is the first to offer a comprehensive definition of a domain-specific KG. Also, the paper presents a thorough review of the state-of-the-art approaches drawn from academic works relevant to seven domains of knowledge. An examination of current approaches reveals a range of limitations and deficiencies. At the same time, uncharted territories on the research map are highlighted to tackle extant issues in the literature and point to directions for future research.
Beelines: Motion Prediction Metrics for Self-Driving Safety and Comfort
Authors: Skanda Shridhar, Yuhang Ma, Tara Stentz, Zhengdi Shen, Galen Clark Haynes, Neil Traft
Date: 2020-11-01 00:00:00
Tasks: motion prediction, Prediction

The commonly used metrics for motion prediction do not correlate well with a self-driving vehicle's system-level performance. The most common metrics are average displacement error (ADE) and final displacement error (FDE), which omit many features, making them poor self-driving performance indicators. Since high-fidelity simulations and track testing can be resource-intensive, the use of prediction metrics better correlated with full-system behavior allows for swifter iteration cycles. In this paper, we offer a conceptual framework for prediction evaluation highly specific to self-driving. We propose two complementary metrics that quantify the effects of motion prediction on safety (related to recall) and comfort (related to precision). Using a simulator, we demonstrate that our safety metric has a significantly better signal-to-noise ratio than displacement error in identifying unsafe events.
Efficient Learning of Control Policies for Robust Quadruped Bounding using Pretrained Neural Networks
Authors: Zhicheng Wang, Anqiao Li, Yixiao Zheng, Anhuan Xie, Zhibin Li, Jun Wu, Qiuguo Zhu
Date: 2020-11-01 00:00:00
Tasks: Deep Reinforcement Learning, Feature Engineering

Bounding is one of the important gaits in quadrupedal locomotion for negotiating obstacles. The authors proposed an effective approach that can learn robust bounding gaits more efficiently despite its large variation in dynamic body movements. The authors first pretrained the neural network (NN) based on data from a robot operated by conventional model based controllers, and then further optimised the pretrained NN via deep reinforcement learning (DRL). In particular, the authors designed a reward function considering contact points and phases to enforce the gait symmetry and periodicity, which improved the bounding performance. The NN based feedback controller was learned in the simulation and directly deployed on the real quadruped robot Jueying Mini successfully. A variety of environments are presented both indoors and outdoors with the authors approach. The authors approach shows efficient computing and good locomotion results by the Jueying Mini quadrupedal robot bounding over uneven terrain.