pwc-archive/papers-with-abstracts huggingface dataset:
Improving EEG Decoding via Clustering-based Multi-task Feature Learning
Authors: Yu Zhang, Tao Zhou, Wei Wu, Hua Xie, Hongru Zhu, Guoxu Zhou, Andrzej Cichocki
Date: 2020-12-12 00:00:00
Tasks: Brain Computer Interface, Clustering, EEG, Eeg Decoding, Electroencephalogram (EEG), Multi-Task Learning

Accurate electroencephalogram (EEG) pattern decoding for specific mental tasks is one of the key steps for the development of brain-computer interface (BCI), which is quite challenging due to the considerably low signal-to-noise ratio of EEG collected at the brain scalp. Machine learning provides a promising technique to optimize EEG patterns toward better decoding accuracy. However, existing algorithms do not effectively explore the underlying data structure capturing the true EEG sample distribution, and hence can only yield a suboptimal decoding accuracy. To uncover the intrinsic distribution structure of EEG data, we propose a clustering-based multi-task feature learning algorithm for improved EEG pattern decoding. Specifically, we perform affinity propagation-based clustering to explore the subclasses (i.e., clusters) in each of the original classes, and then assign each subclass a unique label based on a one-versus-all encoding strategy. With the encoded label matrix, we devise a novel multi-task learning algorithm by exploiting the subclass relationship to jointly optimize the EEG pattern features from the uncovered subclasses. We then train a linear support vector machine with the optimized features for EEG pattern decoding. Extensive experimental studies are conducted on three EEG datasets to validate the effectiveness of our algorithm in comparison with other state-of-the-art approaches. The improved experimental results demonstrate the outstanding superiority of our algorithm, suggesting its prominent performance for EEG pattern decoding in BCI applications.
An improved mixture of probabilistic PCA for nonlinear data-driven process monitoring
Authors: Jingxin Zhang, Hao Chen, Songhang Chen, Xia Hong
Date: 2020-12-12 00:00:00
Tasks: Fault Detection

An improved mixture of probabilistic principal component analysis (PPCA) has been introduced for nonlinear data-driven process monitoring in this paper. To realize this purpose, the technique of a mixture of probabilistic principal component analysers is utilized to establish the model of the underlying nonlinear process with local PPCA models, where a novel composite monitoring statistic is proposed based on the integration of two monitoring statistics in modified PPCA-based fault detection approach. Besides, the weighted mean of the monitoring statistics aforementioned is utilised as a metrics to detect potential abnormalities. The virtues of the proposed algorithm have been discussed in comparison with several unsupervised algorithms. Finally, Tennessee Eastman process and an autosuspension model are employed to demonstrate the effectiveness of the proposed scheme further.
A unified framework for closed-form nonparametric regression, classification, preference and mixed problems with Skew Gaussian Processes
Authors: Alessio Benavoli, Dario Azzimonti, Dario Piga
Date: 2020-12-12 00:00:00
Tasks: Active Learning, Binary Classification, Form, Gaussian Processes, General Classification

Skew-Gaussian processes (SkewGPs) extend the multivariate Unified Skew-Normal distributions over finite dimensional vectors to distribution over functions. SkewGPs are more general and flexible than Gaussian processes, as SkewGPs may also represent asymmetric distributions. In a recent contribution we showed that SkewGP and probit likelihood are conjugate, which allows us to compute the exact posterior for non-parametric binary classification and preference learning. In this paper, we generalize previous results and we prove that SkewGP is conjugate with both the normal and affine probit likelihood, and more in general, with their product. This allows us to (i) handle classification, preference, numeric and ordinal regression, and mixed problems in a unified framework; (ii) derive closed-form expression for the corresponding posterior distributions. We show empirically that the proposed framework based on SkewGP provides better performance than Gaussian processes in active learning and Bayesian (constrained) optimization. These two tasks are fundamental for design of experiments and in Data Science.
Tutoring Reinforcement Learning via Feedback Control
Authors: Francesco De Lellis, Giovanni Russo, Mario di Bernardo
Date: 2020-12-12 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We introduce a control-tutored reinforcement learning (CTRL) algorithm. The idea is to enhance tabular learning algorithms by means of a control strategy with limited knowledge of the system model. By tutoring the learning process, the learning rate can be substantially reduced. We use the classical problem of stabilizing an inverted pendulum as a benchmark to numerically illustrate the advantages and disadvantages of the approach.
VoxSRC 2020: The Second VoxCeleb Speaker Recognition Challenge
Authors: Arsha Nagrani, Joon Son Chung, Jaesung Huh, Andrew Brown, Ernesto Coto, Weidi Xie, Mitchell McLaren, Douglas A Reynolds, Andrew Zisserman
Date: 2020-12-12 00:00:00
Tasks: Speaker Recognition

We held the second installment of the VoxCeleb Speaker Recognition Challenge in conjunction with Interspeech 2020. The goal of this challenge was to assess how well current speaker recognition technology is able to diarise and recognize speakers in unconstrained or `in the wild' data. It consisted of: (i) a publicly available speaker recognition and diarisation dataset from YouTube videos together with ground truth annotation and standardised evaluation software; and (ii) a virtual public challenge and workshop held at Interspeech 2020. This paper outlines the challenge, and describes the baselines, methods used, and results. We conclude with a discussion of the progress over the first installment of the challenge.
Learning over no-Preferred and Preferred Sequence of items for Robust Recommendation
Authors: Aleksandra Burashnikova, Marianne Clausel, Charlotte Laclau, Frack Iutzeller, Yury Maximov, Massih-Reza Amini
Date: 2020-12-12 00:00:00
Tasks: Recommendation Systems

In this paper, we propose a theoretically founded sequential strategy for training large-scale Recommender Systems (RS) over implicit feedback, mainly in the form of clicks. The proposed approach consists in minimizing pairwise ranking loss over blocks of consecutive items constituted by a sequence of non-clicked items followed by a clicked one for each user. We present two variants of this strategy where model parameters are updated using either the momentum method or a gradient-based approach. To prevent from updating the parameters for an abnormally high number of clicks over some targeted items (mainly due to bots), we introduce an upper and a lower threshold on the number of updates for each user. These thresholds are estimated over the distribution of the number of blocks in the training set. The thresholds affect the decision of RS and imply a shift over the distribution of items that are shown to the users. Furthermore, we provide a convergence analysis of both algorithms and demonstrate their practical efficiency over six large-scale collections, both regarding different ranking measures and computational time.
Concept Drift Monitoring and Diagnostics of Supervised Learning Models via Score Vectors
Authors: Kungang Zhang, Anh T. Bui, Daniel W. Apley
Date: 2020-12-12 00:00:00
Tasks: Diagnostic

Supervised learning models are one of the most fundamental classes of models. Viewing supervised learning from a probabilistic perspective, the set of training data to which the model is fitted is usually assumed to follow a stationary distribution. However, this stationarity assumption is often violated in a phenomenon called concept drift, which refers to changes over time in the predictive relationship between covariates $\mathbf{X}$ and a response variable $Y$ and can render trained models suboptimal or obsolete. We develop a comprehensive and computationally efficient framework for detecting, monitoring, and diagnosing concept drift. Specifically, we monitor the Fisher score vector, defined as the gradient of the log-likelihood for the fitted model, using a form of multivariate exponentially weighted moving average, which monitors for general changes in the mean of a random vector. In spite of the substantial performance advantages that we demonstrate over popular error-based methods, a score-based approach has not been previously considered for concept drift monitoring. Advantages of the proposed score-based framework include applicability to any parametric model, more powerful detection of changes as shown in theory and experiments, and inherent diagnostic capabilities for helping to identify the nature of the changes.
A Refined SVD Algorithm for Collaborative Filtering
Authors: Marko Kabić, Gabriel Duque López, Daniel Keller
Date: 2020-12-13 00:00:00
Tasks: Clustering, Collaborative Filtering

Collaborative filtering tries to predict the ratings of a user over some items based on opinions of other users with similar taste. The ratings are usually given in the form of a sparse matrix, the goal being to find the missing entries (i.e. ratings). Various approaches to collaborative filtering exist, some of the most popular ones being the Singular Value Decomposition (SVD) and K-means clustering. One of the challenges in the SVD approach is finding a good initialization of the unknown ratings. A possible initialization is suggested by [1]. In this paper we explain how K-means approach can be used to achieve the further refinement of this initialization for SVD. We show that our technique outperforms both initialization techniques used separately.
$k$-Variance: A Clustered Notion of Variance
Authors: Justin Solomon, Kristjan Greenewald, Haikady N. Nagaraja
Date: 2020-12-13 00:00:00

We introduce $k$-variance, a generalization of variance built on the machinery of random bipartite matchings. $K$-variance measures the expected cost of matching two sets of $k$ samples from a distribution to each other, capturing local rather than global information about a measure as $k$ increases; it is easily approximated stochastically using sampling and linear programming. In addition to defining $k$-variance and proving its basic properties, we provide in-depth analysis of this quantity in several key cases, including one-dimensional measures, clustered measures, and measures concentrated on low-dimensional subsets of $\mathbb R^n$. We conclude with experiments and open problems motivated by this new way to summarize distributional shape.
Predicting Generalization in Deep Learning via Local Measures of Distortion
Authors: Abhejit Rajagopal, Vamshi C. Madala, Shivkumar Chandrasekaran, Peder E. Z. Larson
Date: 2020-12-13 00:00:00
Tasks: Deep Learning, Quantization

We study generalization in deep learning by appealing to complexity measures originally developed in approximation and information theory. While these concepts are challenged by the high-dimensional and data-defined nature of deep learning, we show that simple vector quantization approaches such as PCA, GMMs, and SVMs capture their spirit when applied layer-wise to deep extracted features giving rise to relatively inexpensive complexity measures that correlate well with generalization performance. We discuss our results in 2020 NeurIPS PGDL challenge.
fMRI-Kernel Regression: A Kernel-based Method for Pointwise Statistical Analysis of rs-fMRI for Population Studies
Authors: Anand A. Joshi, Soyoung Choi, Haleh Akrami, Richard M. Leahy
Date: 2020-12-13 00:00:00
Tasks: Functional Connectivity, Time Series Analysis

Due to the spontaneous nature of resting-state fMRI (rs-fMRI) signals, cross-subject comparison and therefore, group studies of rs-fMRI are challenging. Most existing group comparison methods use features extracted from the fMRI time series, such as connectivity features, independent component analysis (ICA), and functional connectivity density (FCD) methods. However, in group studies, especially in the case of spectrum disorders, distances to a single atlas or a representative subject do not fully reflect the differences between subjects that may lie on a multi-dimensional spectrum. Moreover, there may not exist an individual subject or even an average atlas in such cases that is representative of all subjects. Here we describe an approach that measures pairwise distances between the synchronized rs-fMRI signals of pairs of subjects instead of to a single reference point. We also present a method for fMRI data comparison that leverages this generated pairwise feature to establish a radial basis function kernel matrix. This kernel matrix is used in turn to perform kernel regression of rs-fMRI to a clinical variable such as a cognitive or neurophysiological performance score of interest. This method opens a new pointwise analysis paradigm for fMRI data. We demonstrate the application of this method by performing a pointwise analysis on the cortical surface using rs-fMRI data to identify cortical regions associated with variability in ADHD index. While pointwise analysis methods are common in anatomical studies such as cortical thickness analysis and voxel- and tensor-based morphometry and its variants, such a method is lacking for rs-fMRI and could improve the utility of rs-fMRI for group studies. The method presented in this paper is aimed at filling this gap.
Radial Deformation Emplacement in Power Transformers Using Long Short-Term Memory Networks
Authors: Arash Moradzadeh, Kazem Pourhossein, Behnam Mohammadi-Ivatloo, Tohid Khalili, Ali Bidram
Date: 2020-12-13 00:00:00

A power transformer winding is usually subject to mechanical stress and tension because of improper transportation or operation. Radial deformation (RD) is an example of mechanical stress that can impact power transformer operation through short circuit faults and insulation damages. Frequency response analysis (FRA) is a well-known method to diagnose mechanical defects in transformers. Despite the precision of FRA, the interpretation of the calculated frequency response curves is not straightforward and requires complex calculations. In this paper, a deep learning algorithm called long short-term memory (LSTM) is used as a feature extraction technique to locate RD faults in their early stages. The experimental results verify the effectiveness of the proposed method in the diagnosis and locating of RD defects.
Using Restricted Boltzmann Machines to Model Molecular Geometries
Authors: Peter Nekrasov, Jessica Freeze, Victor Batista
Date: 2020-12-13 00:00:00

Precise physical descriptions of molecules can be obtained by solving the Schrodinger equation; however, these calculations are intractable and even approximations can be cumbersome. Force fields, which estimate interatomic potentials based on empirical data, are also time-consuming. This paper proposes a new methodology for modeling a set of physical parameters by taking advantage of the restricted Boltzmann machine's fast learning capacity and representational power. By training the machine on ab initio data, we can predict new data in the distribution of molecular configurations matching the ab initio distribution. In this paper we introduce a new RBM based on the Tanh activation function, and conduct a comparison of RBMs with different activation functions, including sigmoid, Gaussian, and (Leaky) ReLU. Finally we demonstrate the ability of Gaussian RBMs to model small molecules such as water and ethane.
DeepSweep: An Evaluation Framework for Mitigating DNN Backdoor Attacks using Data Augmentation
Authors: Han Qiu, Yi Zeng, Shangwei Guo, Tianwei Zhang, Meikang Qiu, Bhavani Thuraisingham
Date: 2020-12-13 00:00:00
Tasks: Backdoor Attack, Data Augmentation

Public resources and services (e.g., datasets, training platforms, pre-trained models) have been widely adopted to ease the development of Deep Learning-based applications. However, if the third-party providers are untrusted, they can inject poisoned samples into the datasets or embed backdoors in those models. Such an integrity breach can cause severe consequences, especially in safety- and security-critical applications. Various backdoor attack techniques have been proposed for higher effectiveness and stealthiness. Unfortunately, existing defense solutions are not practical to thwart those attacks in a comprehensive way. In this paper, we investigate the effectiveness of data augmentation techniques in mitigating backdoor attacks and enhancing DL models' robustness. An evaluation framework is introduced to achieve this goal. Specifically, we consider a unified defense solution, which (1) adopts a data augmentation policy to fine-tune the infected model and eliminate the effects of the embedded backdoor; (2) uses another augmentation policy to preprocess input samples and invalidate the triggers during inference. We propose a systematic approach to discover the optimal policies for defending against different backdoor attacks by comprehensively evaluating 71 state-of-the-art data augmentation functions. Extensive experiments show that our identified policy can effectively mitigate eight different kinds of backdoor attacks and outperform five existing defense methods. We envision this framework can be a good benchmark tool to advance future DNN backdoor studies.
Process monitoring based on orthogonal locality preserving projection with maximum likelihood estimation
Authors: Jingxin Zhang, Maoyin Chen, Hao Chen, Xia Hong, Donghua Zhou
Date: 2020-12-13 00:00:00
Tasks: Density Estimation, Dimensionality Reduction, Fault Detection, Fault Diagnosis

By integrating two powerful methods of density reduction and intrinsic dimensionality estimation, a new data-driven method, referred to as OLPP-MLE (orthogonal locality preserving projection-maximum likelihood estimation), is introduced for process monitoring. OLPP is utilized for dimensionality reduction, which provides better locality preserving power than locality preserving projection. Then, the MLE is adopted to estimate intrinsic dimensionality of OLPP. Within the proposed OLPP-MLE, two new static measures for fault detection $T_{\scriptscriptstyle {OLPP}}^2$ and ${\rm SPE}_{\scriptscriptstyle {OLPP}}$ are defined. In order to reduce algorithm complexity and ignore data distribution, kernel density estimation is employed to compute thresholds for fault diagnosis. The effectiveness of the proposed method is demonstrated by three case studies.
Neural network approaches to point lattice decoding
Authors: Vincent Corlay, Joseph J. Boutros, Philippe Ciblat, Loïc Brunel
Date: 2020-12-13 00:00:00

We characterize the complexity of the lattice decoding problem from a neural network perspective. The notion of Voronoi-reduced basis is introduced to restrict the space of solutions to a binary set. On the one hand, this problem is shown to be equivalent to computing a continuous piecewise linear (CPWL) function restricted to the fundamental parallelotope. On the other hand, it is known that any function computed by a ReLU feed-forward neural network is CPWL. As a result, we count the number of affine pieces in the CPWL decoding function to characterize the complexity of the decoding problem. It is exponential in the space dimension $n$, which induces shallow neural networks of exponential size. For structured lattices we show that folding, a technique equivalent to using a deep neural network, enables to reduce this complexity from exponential in $n$ to polynomial in $n$. Regarding unstructured MIMO lattices, in contrary to dense lattices many pieces in the CPWL decoding function can be neglected for quasi-optimal decoding on the Gaussian channel. This makes the decoding problem easier and it explains why shallow neural networks of reasonable size are more efficient with this category of lattices (in low to moderate dimensions).
Monitoring multimode processes: a modified PCA algorithm with continual learning ability
Authors: Jingxin Zhang, Donghua Zhou, Maoyin Chen
Date: 2020-12-13 00:00:00
Tasks: Continual Learning

For multimode processes, one generally establishes local monitoring models corresponding to local modes. However, the significant features of previous modes may be catastrophically forgotten when a monitoring model for the current mode is built. It would result in an abrupt performance decrease. It could be an effective manner to make local monitoring model remember the features of previous modes. Choosing the principal component analysis (PCA) as a basic monitoring model, we try to resolve this problem. A modified PCA algorithm is built with continual learning ability for monitoring multimode processes, which adopts elastic weight consolidation (EWC) to overcome catastrophic forgetting of PCA for successive modes. It is called PCA-EWC, where the significant features of previous modes are preserved when a PCA model is established for the current mode. The optimal parameters are acquired by differences of convex functions. Moreover, the proposed PCA-EWC is extended to general multimode processes and the procedure is presented. The computational complexity and key parameters are discussed to further understand the relationship between PCA and the proposed algorithm. Potential limitations and relevant solutions are pointed to understand the algorithm further. Numerical case study and a practical industrial system in China are employed to illustrate the effectiveness of the proposed algorithm.
Adaptive and Oblivious Randomized Subspace Methods for High-Dimensional Optimization: Sharp Analysis and Lower Bounds
Authors: Jonathan Lacotte, Mert Pilanci
Date: 2020-12-13 00:00:00
Tasks: subspace methods

We propose novel randomized optimization methods for high-dimensional convex problems based on restrictions of variables to random subspaces. We consider oblivious and data-adaptive subspaces and study their approximation properties via convex duality and Fenchel conjugates. A suitable adaptive subspace can be generated by sampling a correlated random matrix whose second order statistics mirror the input data. We illustrate that the adaptive strategy can significantly outperform the standard oblivious sampling method, which is widely used in the recent literature. We show that the relative error of the randomized approximations can be tightly characterized in terms of the spectrum of the data matrix and Gaussian width of the dual tangent cone at optimum. We develop lower bounds for both optimization and statistical error measures based on concentration of measure and Fano's inequality. We then present the consequences of our theory with data matrices of varying spectral decay profiles. Experimental results show that the proposed approach enables significant speed ups in a wide variety of machine learning and optimization problems including logistic regression, kernel classification with random convolution layers and shallow neural networks with rectified linear units.
Improving the Classification of Rare Chords with Unlabeled Data
Authors: Marcelo Bortolozzo, Rodrigo Schramm, Claudio R. Jung
Date: 2020-12-13 00:00:00
Tasks: Chord Recognition, Classification, General Classification, Self-Learning

In this work, we explore techniques to improve performance for rare classes in the task of Automatic Chord Recognition (ACR). We first explored the use of the focal loss in the context of ACR, which was originally proposed to improve the classification of hard samples. In parallel, we adapted a self-learning technique originally designed for image recognition to the musical domain. Our experiments show that both approaches individually (and their combination) improve the recognition of rare chords, but using only self-learning with noise addition yields the best results.