pwc-archive/papers-with-abstracts huggingface dataset:
Mathematical decisions and non-causal elements of explainable AI
Authors: Atoosa Kasirzadeh
Date: 2019-10-30 00:00:00
Tasks: Decision Making, Ethics

The social implications of algorithmic decision-making in sensitive contexts have generated lively debates among multiple stakeholders, such as moral and political philosophers, computer scientists, and the public. Yet, the lack of a common language and a conceptual framework for an appropriate bridging of the moral, technical, and political aspects of the debate prevents the discussion to be as effective as it can be. Social scientists and psychologists are contributing to this debate by gathering a wealth of empirical data, yet a philosophical analysis of the social implications of algorithmic decision-making remains comparatively impoverished. In attempting to address this lacuna, this paper argues that a hierarchy of different types of explanations for why and how an algorithmic decision outcome is achieved can establish the relevant connection between the moral and technical aspects of algorithmic decision-making. In particular, I offer a multi-faceted conceptual framework for the explanations and the interpretations of algorithmic decisions, and I claim that this framework can lay the groundwork for a focused discussion among multiple stakeholders about the social implications of algorithmic decision-making, as well as AI governance and ethics more generally.
An Augmented Transformer Architecture for Natural Language Generation Tasks
Authors: Hailiang Li, Adele Y. C. Wang, Yang Liu, Du Tang, Zhibin Lei, Wenye Li
Date: 2019-10-30 00:00:00
Tasks: Part-Of-Speech Tagging, POS, POS Tagging, Text Generation, Translation

The Transformer based neural networks have been showing significant advantages on most evaluations of various natural language processing and other sequence-to-sequence tasks due to its inherent architecture based superiorities. Although the main architecture of the Transformer has been continuously being explored, little attention was paid to the positional encoding module. In this paper, we enhance the sinusoidal positional encoding algorithm by maximizing the variances between encoded consecutive positions to obtain additional promotion. Furthermore, we propose an augmented Transformer architecture encoded with additional linguistic knowledge, such as the Part-of-Speech (POS) tagging, to boost the performance on some natural language generation tasks, e.g., the automatic translation and summarization tasks. Experiments show that the proposed architecture attains constantly superior results compared to the vanilla Transformer.
Thompson Sampling via Local Uncertainty
Authors: Zhendong Wang, Mingyuan Zhou
Date: 2019-10-30 00:00:00
Proceeding: ICML 2020 1
Tasks: Decision Making, Multi-Armed Bandits, Sequential Decision Making, Thompson Sampling, Variational Inference

Thompson sampling is an efficient algorithm for sequential decision making, which exploits the posterior uncertainty to address the exploration-exploitation dilemma. There has been significant recent interest in integrating Bayesian neural networks into Thompson sampling. Most of these methods rely on global variable uncertainty for exploration. In this paper, we propose a new probabilistic modeling framework for Thompson sampling, where local latent variable uncertainty is used to sample the mean reward. Variational inference is used to approximate the posterior of the local variable, and semi-implicit structure is further introduced to enhance its expressiveness. Our experimental results on eight contextual bandit benchmark datasets show that Thompson sampling guided by local uncertainty achieves state-of-the-art performance while having low computational complexity.
Software defect prediction with zero-inflated Poisson models
Authors: Daniel Rodriguez, Javier Dolado, Javier Tuya, Dietmar Pfahl
Date: 2019-10-30 00:00:00
Tasks: Prediction

In this work we apply several Poisson and zero-inflated models for software defect prediction. We apply different functions from several R packages such as pscl, MASS, R2Jags and the recent glmmTMB. We test the functions using the Equinox dataset. The results show that Zero-inflated models, fitted with either maximum likelihood estimation or with Bayesian approach, are slightly better than other models, using the AIC as selection criterion.
Metric Learning with Background Noise Class for Few-shot Detection of Rare Sound Events
Authors: Kazuki Shimada, Yuichiro Koyama, Akira Inoue
Date: 2019-10-30 00:00:00
Tasks: Few-Shot Learning, Metric Learning

Few-shot learning systems for sound event recognition have gained interests since they require only a few examples to adapt to new target classes without fine-tuning. However, such systems have only been applied to chunks of sounds for classification or verification. In this paper, we aim to achieve few-shot detection of rare sound events, from query sequence that contain not only the target events but also the other events and background noise. Therefore, it is required to prevent false positive reactions to both the other events and background noise. We propose metric learning with background noise class for the few-shot detection. The contribution is to present the explicit inclusion of background noise as an independent class, a suitable loss function that emphasizes this additional class, and a corresponding sampling strategy that assists training. It provides a feature space where the event classes and the background noise class are sufficiently separated. Evaluations on few-shot detection tasks, using DCASE 2017 task2 and ESC-50, show that our proposed method outperforms metric learning without considering the background noise class. The few-shot detection performance is also comparable to that of the DCASE 2017 task2 baseline system, which requires huge amount of annotated audio data.
Unifying mirror descent and dual averaging
Authors: Anatoli Juditsky, Joon Kwon, Éric Moulines
Date: 2019-10-30 00:00:00

We introduce and analyze a new family of first-order optimization algorithms which generalizes and unifies both mirror descent and dual averaging. Within the framework of this family, we define new algorithms for constrained optimization that combines the advantages of mirror descent and dual averaging. Our preliminary simulation study shows that these new algorithms significantly outperform available methods in some situations.
Machine Learning for Geometrically-Consistent Angular Spread Function Estimation in Massive MIMO
Authors: Yi Song, Mahdi Barzegar Khalilsarai, Saeid Haghighatshoar, Giuseppe Caire
Date: 2019-10-30 00:00:00
Tasks: BIG-bench Machine Learning

In the spatial channel models used in multi-antenna wireless communications, the propagation from a single-antenna transmitter (e.g., a user) to an M-antenna receiver (e.g., a Base Station) occurs through scattering clusters located in the far field of the receiving antenna array. The Angular Spread Function (ASF) of the corresponding M-dim channel vector describes the angular density of the received signal power at the array. The modern literature on massive MIMO has recognized that the knowledge of covariance matrix of user channel vectors is very useful for various applications such as hybrid digital analog beamforming, pilot decontamination, etc. Therefore, most literature has focused on the estimation of such channel covariance matrices. However, in some applications such as uplink-downlink covariance transformation (for FDD massive MIMO precoding) and channel sounding some form of ASF estimation is required either implicitly or explicitly. It turns out that while covariance estimation is well-known and well-conditioned, the ASF estimation is a much harder problem and is in general ill-posed. In this paper, we show that under additional geometrically-consistent group-sparsity structure on the ASF, which is prevalent in almost all wireless propagation scenarios, one is able to estimate ASF properly. We propose sparse dictionary-based algorithms that promote this group-sparsity structure via suitable regularizations. Since generally it is difficult to capture the notion of group-sparsity through proper regularization, we propose another algorithm based on Deep Neural Networks (DNNs) that learns this structure. We provide numerical simulations to assess the performance of our proposed algorithms. We also compare the results with that of other methods in the literature, where we re-frame those methods in the context of ASF estimation in massive MIMO.
Multimodal Learning For Classroom Activity Detection
Authors: Hang Li, Yu Kang, Wenbiao Ding, Song Yang, Songfan Yang, Gale Yan Huang, Zitao Liu
Date: 2019-10-22 00:00:00
Tasks: Action Detection, Activity Detection

Classroom activity detection (CAD) focuses on accurately classifying whether the teacher or student is speaking and recording both the length of individual utterances during a class. A CAD solution helps teachers get instant feedback on their pedagogical instructions. This greatly improves educators' teaching skills and hence leads to students' achievement. However, CAD is very challenging because (1) the CAD model needs to be generalized well enough for different teachers and students; (2) data from both vocal and language modalities has to be wisely fused so that they can be complementary; and (3) the solution shouldn't heavily rely on additional recording device. In this paper, we address the above challenges by using a novel attention based neural framework. Our framework not only extracts both speech and language information, but utilizes attention mechanism to capture long-term semantic dependence. Our framework is device-free and is able to take any classroom recording as input. The proposed CAD learning framework is evaluated in two real-world education applications. The experimental results demonstrate the benefits of our approach on learning attention based neural network from classroom data with different modalities, and show our approach is able to outperform state-of-the-art baselines in terms of various evaluation metrics.
Unsupervised Representation Learning with Future Observation Prediction for Speech Emotion Recognition
Authors: Zheng Lian, Jian-Hua Tao, Bin Liu, Jian Huang
Date: 2019-10-24 00:00:00
Tasks: Emotion Recognition, Representation Learning, Speech Emotion Recognition, Transfer Learning

Prior works on speech emotion recognition utilize various unsupervised learning approaches to deal with low-resource samples. However, these methods pay less attention to modeling the long-term dynamic dependency, which is important for speech emotion recognition. To deal with this problem, this paper combines the unsupervised representation learning strategy -- Future Observation Prediction (FOP), with transfer learning approaches (such as Fine-tuning and Hypercolumns). To verify the effectiveness of the proposed method, we conduct experiments on the IEMOCAP database. Experimental results demonstrate that our method is superior to currently advanced unsupervised learning strategies.
Domain adversarial learning for emotion recognition
Authors: Zheng Lian, Jian-Hua Tao, Bin Liu, Jian Huang
Date: 2019-10-24 00:00:00
Tasks: Emotion Recognition

In practical applications for emotion recognition, users do not always exist in the training corpus. The mismatch between training speakers and testing speakers affects the performance of the trained model. To deal with this problem, we need our model to focus on emotion-related information, while ignoring the difference between speaker identities. In this paper, we look into the use of the domain adversarial neural network (DANN) to extract a common representation between different speakers. The primary task is to predict emotion labels. The secondary task is to learn a common representation where speaker identities can not be distinguished. By using the gradient reversal layer, the gradients coming from the secondary task are used to bring the representations for different speakers closer. To verify the effectiveness of the proposed method, we conduct experiments on the IEMOCAP database. Experimental results demonstrate that the proposed framework shows an absolute improvement of 3.48% over state-of-the-art strategies.
How Coupled are Mass Spectrometry and Capillary Electrophoresis?
Authors: Caroline Ceribeli, Henrique F. de Arruda, Luciano da F. Costa
Date: 2019-10-18 00:00:00
Tasks: Articles

The understanding of how science works can contribute to making scientific development more effective. In this paper, we report an analysis of the organization and interconnection between two important issues in chemistry, namely mass spectrometry (MS) and capillary electrophoresis (CE). For that purpose, we employed science of science techniques based on complex networks. More specifically, we considered a citation network in which the nodes and connections represent papers and citations, respectively. Interesting results were found, including a good separation between some clusters of articles devoted to instrumentation techniques and applications. However, the papers that describe CE-MS did not lead to a well-defined cluster. In order to better understand the organization of the citation network, we considered a multi-scale analysis, in which we used the information regarding sub-clusters. Firstly, we analyzed the sub-cluster of the first article devoted to the coupling between CE and MS, which was found to be a good representation of its sub-cluster. The second analysis was about the sub-cluster of a seminal paper known to be the first that dealt with proteins by using CE-MS. By considering the proposed methodologies, our paper paves the way for researchers working with both techniques, since it elucidates the knowledge organization and can therefore lead to better literature reviews.
Learning pairwise Markov network structures using correlation neighborhoods
Authors: Juri Kuronen, Jukka Corander, Johan Pensar
Date: 2019-10-30 00:00:00

Markov networks are widely studied and used throughout multivariate statistics and computer science. In particular, the problem of learning the structure of Markov networks from data without invoking chordality assumptions in order to retain expressiveness of the model class has been given a considerable attention in the recent literature, where numerous constraint-based or score-based methods have been introduced. Here we develop a new search algorithm for the network score-optimization that has several computational advantages and scales well to high-dimensional data sets. The key observation behind the algorithm is that the neighborhood of a variable can be efficiently captured using local penalized likelihood ratio (PLR) tests by exploiting an exponential decay of correlations across the neighborhood with an increasing graph-theoretic distance from the focus node. The candidate neighborhoods are then processed by a two-stage hill-climbing (HC) algorithm. Our approach, termed fully as PLRHC-BIC$_{0.5}$, compares favorably against the state-of-the-art methods in all our experiments spanning both low- and high-dimensional networks and a wide range of sample sizes. An efficient implementation of PLRHC-BIC$_{0.5}$ is freely available from the URL: https://github.com/jurikuronen/plrhc.
Linear Speedup in Saddle-Point Escape for Decentralized Non-Convex Optimization
Authors: Stefan Vlaski, Ali H. Sayed
Date: 2019-10-30 00:00:00
Tasks: Stochastic Optimization

Under appropriate cooperation protocols and parameter choices, fully decentralized solutions for stochastic optimization have been shown to match the performance of centralized solutions and result in linear speedup (in the number of agents) relative to non-cooperative approaches in the strongly-convex setting. More recently, these results have been extended to the pursuit of first-order stationary points in non-convex environments. In this work, we examine in detail the dependence of second-order convergence guarantees on the spectral properties of the combination policy for non-convex multi agent optimization. We establish linear speedup in saddle-point escape time in the number of agents for symmetric combination policies and study the potential for further improvement by employing asymmetric combination weights. The results imply that a linear speedup can be expected in the pursuit of second-order stationary points, which exclude local maxima as well as strict saddle-points and correspond to local or even global minima in many important learning settings.
UniXGrad: A Universal, Adaptive Algorithm with Optimal Guarantees for Constrained Optimization
Authors: Ali Kavis, Kfir. Y. Levy, Francis Bach, Volkan Cevher
Date: 2019-10-30 00:00:00
Proceeding: NeurIPS 2019 12

We propose a novel adaptive, accelerated algorithm for the stochastic constrained convex optimization setting. Our method, which is inspired by the Mirror-Prox method, \emph{simultaneously} achieves the optimal rates for smooth/non-smooth problems with either deterministic/stochastic first-order oracles. This is done without any prior knowledge of the smoothness nor the noise properties of the problem. To the best of our knowledge, this is the first adaptive, unified algorithm that achieves the optimal rates in the constrained setting. We demonstrate the practical performance of our framework through extensive numerical experiments.
Fault Tolerance of Neural Networks in Adversarial Settings
Authors: Vasisht Duddu, N. Rajesh Pillai, D. Vijay Rao, Valentina E. Balas
Date: 2019-10-30 00:00:00
Tasks: Adversarial Robustness, Fairness

Artificial Intelligence systems require a through assessment of different pillars of trust, namely, fairness, interpretability, data and model privacy, reliability (safety) and robustness against against adversarial attacks. While these research problems have been extensively studied in isolation, an understanding of the trade-off between different pillars of trust is lacking. To this extent, the trade-off between fault tolerance, privacy and adversarial robustness is evaluated for the specific case of Deep Neural Networks, by considering two adversarial settings under a security and a privacy threat model. Specifically, this work studies the impact of the fault tolerance of the Neural Network on training the model by adding noise to the input (Adversarial Robustness) and noise to the gradients (Differential Privacy). While training models with noise to inputs, gradients or weights enhances fault tolerance, it is observed that adversarial robustness and fault tolerance are at odds with each other. On the other hand, ($\epsilon,\delta$)-Differentially Private models enhance the fault tolerance, measured using generalisation error, theoretically has an upper bound of $e^{\epsilon} - 1 + \delta$. This novel study of the trade-off between different elements of trust is pivotal for training a model which satisfies the requirements for different pillars of trust simultaneously.
Lightweight and Efficient End-to-End Speech Recognition Using Low-Rank Transformer
Authors: Genta Indra Winata, Samuel Cahyawijaya, Zhaojiang Lin, Zihan Liu, Pascale Fung
Date: 2019-10-30 00:00:00
Tasks: Language Modeling, Language Modelling, speech-recognition, Speech Recognition

Highly performing deep neural networks come at the cost of computational complexity that limits their practicality for deployment on portable devices. We propose the low-rank transformer (LRT), a memory-efficient and fast neural architecture that significantly reduces the parameters and boosts the speed of training and inference for end-to-end speech recognition. Our approach reduces the number of parameters of the network by more than 50% and speeds up the inference time by around 1.35x compared to the baseline transformer model. The experiments show that our LRT model generalizes better and yields lower error rates on both validation and test sets compared to an uncompressed transformer model. The LRT model outperforms those from existing works on several datasets in an end-to-end setting without using an external language model or acoustic data.
Distilling Black-Box Travel Mode Choice Model for Behavioral Interpretation
Authors: Xilei Zhao, Zhengze Zhou, Xiang Yan, Pascal Van Hentenryck
Date: 2019-10-30 00:00:00
Tasks: BIG-bench Machine Learning

Machine learning has proved to be very successful for making predictions in travel behavior modeling. However, most machine-learning models have complex model structures and offer little or no explanation as to how they arrive at these predictions. Interpretations about travel behavior models are essential for decision makers to understand travelers' preferences and plan policy interventions accordingly. Therefore, this paper proposes to apply and extend the model distillation approach, a model-agnostic machine-learning interpretation method, to explain how a black-box travel mode choice model makes predictions for the entire population and subpopulations of interest. Model distillation aims at compressing knowledge from a complex model (teacher) into an understandable and interpretable model (student). In particular, the paper integrates model distillation with market segmentation to generate more insights by accounting for heterogeneity. Furthermore, the paper provides a comprehensive comparison of student models with the benchmark model (decision tree) and the teacher model (gradient boosting trees) to quantify the fidelity and accuracy of the students' interpretations.
Nonembeddability of Persistence Diagrams with $p>2$ Wasserstein Metric
Authors: Alexander Wagner
Date: 2019-10-30 00:00:00

Persistence diagrams do not admit an inner product structure compatible with any Wasserstein metric. Hence, when applying kernel methods to persistence diagrams, the underlying feature map necessarily causes distortion. We prove persistence diagrams with the p-Wasserstein metric do not admit a coarse embedding into a Hilbert space when p > 2.
A Statistical Learning Approach to Reactive Power Control in Distribution Systems
Authors: Qiuling Yang, Alireza Sadeghi, Gang Wang, Georgios B. Giannakis, Jian Sun
Date: 2019-10-25 00:00:00
Tasks: Computational Efficiency

Pronounced variability due to the growth of renewable energy sources, flexible loads, and distributed generation is challenging residential distribution systems. This context, motivates well fast, efficient, and robust reactive power control. Real-time optimal reactive power control is possible in theory by solving a non-convex optimization problem based on the exact model of distribution flow. However, lack of high-precision instrumentation and reliable communications, as well as the heavy computational burden of non-convex optimization solvers render computing and implementing the optimal control challenging in practice. Taking a statistical learning viewpoint, the input-output relationship between each grid state and the corresponding optimal reactive power control is parameterized in the present work by a deep neural network, whose unknown weights are learned offline by minimizing the power loss over a number of historical and simulated training pairs. In the inference phase, one just feeds the real-time state vector into the learned neural network to obtain the `optimal' reactive power control with only several matrix-vector multiplications. The merits of this novel statistical learning approach are computational efficiency as well as robustness to random input perturbations. Numerical tests on a 47-bus distribution network using real data corroborate these practical merits.