pwc-archive/papers-with-abstracts huggingface dataset:
The value of big data for analyzing growth dynamics of technology based new ventures
Authors: Maksim Malyy, Zeljko Tekic, Tatiana Podladchikova
Date: 2021-04-07 00:00:00

This study demonstrates that web-search traffic information, in particular, Google Trends data, is a credible novel source of high-quality and easy-to-access data for analyzing technology-based new ventures (TBNVs) growth trajectories. Utilizing the diverse sample of 241 US-based TBNVs, we comparatively analyze the relationship between companies' evolution curves represented by search activity on the one hand and by valuations achieved through rounds of venture investments on another. The results suggest that TBNV's growth dynamics are positively and strongly correlated with its web search traffic across the sample. This correlation is more robust when a company is a) more successful (in terms of valuation achieved) - especially if it is a "unicorn"; b) consumer-oriented (i.e., b2c); and 3) develops products in the form of a digital platform. Further analysis based on fuzzy-set Qualitative Comparative Analysis (fsQCA) shows that for the most successful companies ("unicorns") and consumer-oriented digital platforms (i.e., b2c digital platform companies) proposed approach may be extremely reliable, while for other high-growth TBNVs it is useful for analyzing their growth dynamics, albeit to a more limited degree. The proposed methodological approach opens a wide range of possibilities for analyzing, researching and predicting the growth of recently formed growth-oriented companies, in practice and academia.
Bayesian adversarial multi-node bandit for optimal smart grid protection against cyber attacks
Authors: Jianyu Xu, Bin Liu, Huadong Mo, Daoyi Dong
Date: 2021-02-20 00:00:00

The cybersecurity of smart grids has become one of key problems in developing reliable modern power and energy systems. This paper introduces a non-stationary adversarial cost with a variation constraint for smart grids and enables us to investigate the problem of optimal smart grid protection against cyber attacks in a relatively practical scenario. In particular, a Bayesian multi-node bandit (MNB) model with adversarial costs is constructed and a new regret function is defined for this model. An algorithm called Thompson-Hedge algorithm is presented to solve the problem and the superior performance of the proposed algorithm is proven in terms of the convergence rate of the regret function. The applicability of the algorithm to real smart grid scenarios is verified and the performance of the algorithm is also demonstrated by numerical examples.
Coupling a generative model with a discriminative learning framework for speaker verification
Authors: Xugang Lu, Peng Shen, Yu Tsao, Hisashi Kawai
Date: 2021-01-09 00:00:00
Tasks: Decision Making, feature selection, Speaker Verification

The speaker verification (SV) task is to decide whether an utterance is spoken by a target or an imposter speaker. For most studies, a log-likelihood ratio (LLR) score is estimated based on a generative probability model on speaker features and compared with a threshold for making a decision. However, the generative model usually focuses on individual feature distributions, does not have the discriminative feature selection ability, and is easy to be distracted by nuisance features. The SV could be formulated as a binary discrimination task where neural network-based discriminative learning could be applied. In discriminative learning, the nuisance features could be removed with the help of label supervision. However, discriminative learning pays more attention to classification boundaries and is prone to overfitting to a training set which may result in bad generalization on a test set. Thus, we propose a hybrid learning framework, i.e., coupling a joint Bayesian (JB) generative model structure and parameters with a neural discriminative learning framework for SV. A two-branch Siamese neural network is built with dense layers that are coupled with factorized affine transforms as used in the JB model. The LLR score estimation in the JB model is formulated according to the distance metric in the discriminative learning framework. By initializing the two-branch neural network with the generatively learned model parameters of the JB model, we train the model parameters with the pairwise samples as a binary discrimination task. Moreover, a direct evaluation metric in SV based on minimum empirical Bayes risk is designed and integrated as an objective function in discriminative learning. We carried out SV experiments on Speakers in the wild and Voxceleb. Experimental results showed that our proposed model improved the performance with a large margin compared with state-of-art models for SV.
Building a Scalable and Interpretable Bayesian Deep Learning Framework for Quality Control of Free Form Surfaces
Authors: Sumit Sinha, Pasquale Franciosa, Dariusz Ceglarek
Date: 1994-04-07 00:00:00
Tasks: Active Learning, Benchmarking, Continual Learning, Decision Making, Deep Learning, Diagnostic, Form

Deep learning has demonstrated high accuracy for 3D object shape error modeling necessary to estimate dimensional and geometric quality defects in multi-station assembly systems (MAS). Increasingly, deep learning-driven Root Cause Analysis (RCA) is used for decision-making when planning corrective action of quality defects. However, given the current absence of scalability enabling models, training deep learning models for each individual MAS is exceedingly time-consuming as it requires large amounts of labelled data and multiple computational cycles. Additionally, understanding and interpreting how deep learning produces final predictions while quantifying various uncertainties also remains a fundamental challenge. In an effort to address these gaps, a novel closed-loop in-process (CLIP) diagnostic framework underpinned algorithm portfolio is proposed which simultaneously enhances scalability and interpretability of the current Bayesian deep learning approach, Object Shape Error Response (OSER), to isolate root cause(s) of quality defects in MAS. The OSER-MAS leverages a Bayesian 3D U-Net architecture integrated with Computer-Aided Engineering simulations to estimate root causes. The CLIP diagnostic framework shortens OSER-MAS model training time by developing: (i) closed-loop training to enable faster convergence for a single MAS by leveraging uncertainty estimates of the Bayesian 3D U-net model; and, (ii) transfer/continual learning-based scalability model to transmit meta-knowledge from the trained model to a new MAS resulting in convergence using comparatively less training samples. Additionally, CLIP increases the transparency for quality-related root cause predictions by developing interpretability model which is based on 3D Gradient-based Class Activation Maps (3D Grad-CAMs) and entails: (a) linking elements of MAS model with functional elements of the U-Net architecture; and, (b) relating features extracted by the architecture with elements of the MAS model and further with the object shape error patterns for root cause(s) that occur in MAS. Benchmarking studies are conducted using six automotive-MAS with varying complexities. Results highlight a reduction in training samples of up to 56% with a loss in performance of up to 2.1%.
Dynamics in cortical activity revealed by resting-state MEG rhythms
Authors: J. Mendoza-Ruiz, C. E. Alonso-Malaver, M. Valderrama, O. A. Rosso, J. H. Martínez
Date: 2020-07-09 00:00:00

The brain may be thought of as a many-body architecture with a spatio-temporal dynamics described by neuronal structures. The oscillatory nature of brain activity allows these structures (nodes) to be described as a set of coupled oscillators forming a network where the node dynamics, and that of the network topology can be studied. Quantifying its dynamics at various scales is an issue that claims to be explored for several brain activities, e.g., activity at rest. The resting-state associates the underlying brain dynamics of healthy subjects that are not actively compromised with sensory or cognitive processes. Studying its dynamics is highly non-trivial but opens the door to understand the general principles of brain functioning. We hypothesize about how could be the spatio-temporal dynamics of cortical fluctuations for healthy subjects at resting-state. We retrieve the alphabet that reconstructs the dynamics (entropy/complexity) of magnetoencephalograpy signals. We assemble the cortical connectivity to elicit the network's dynamics. We depict an order relation between entropy/complexity for frequency bands. We unveiled that the posterior cortex conglomerates nodes with both stronger dynamics and high clustering for {\alpha} band. The existence of these order relations suggests an emergent phenomenon of each band. Interestingly, we find that the posterior cortex plays a cardinal role in both the dynamics and structure regarding the resting-state. To the best of our knowledge, this is the first study with magnetoencephalograpy involving information theory and network science to better understand the dynamics and structure of brain activity at rest for different bands and scales.
Generalized Approach to Matched Filtering using Neural Networks
Authors: Jingkai Yan, Mariam Avagyan, Robert E. Colgan, Doğa Veske, Imre Bartos, John Wright, Zsuzsa Márka, Szabolcs Márka
Date: 2021-04-08 00:00:00
Tasks: Gravitational Wave Detection

Gravitational wave science is a pioneering field with rapidly evolving data analysis methodology currently assimilating and inventing deep learning techniques. The bulk of the sophisticated flagship searches of the field rely on the time-tested matched filtering principle within their core. In this paper, we make a key observation on the relationship between the emerging deep learning and the traditional techniques: matched filtering is formally equivalent to a particular neural network. This means that a neural network can be constructed analytically to exactly implement matched filtering, and can be further trained on data or boosted with additional complexity for improved performance. Moreover, we show that the proposed neural network architecture can outperform matched filtering, both with or without knowledge of a prior on the parameter distribution. When a prior is given, the proposed neural network can approach the statistically optimal performance. We also propose and investigate two different neural network architectures MNet-Shallow and MNet-Deep, both of which implement matched filtering at initialization and can be trained on data. MNet-Shallow has simpler structure, while MNet-Deep is more flexible and can deal with a wider range of distributions. Our theoretical findings are corroborated by experiments using real LIGO data and synthetic injections, where our proposed methods significantly outperform matched filtering at false positive rates above $5\times 10^{-3}\%$. The fundamental equivalence between matched filtering and neural networks allows us to define a "complexity standard candle" to characterize the relative complexity of the different approaches to gravitational wave signal searches in a common framework. Finally, our results suggest new perspectives on the role of deep learning in gravitational wave detection.
Handwriting Transformers
Authors: Ankan Kumar Bhunia, Salman Khan, Hisham Cholakkal, Rao Muhammad Anwer, Fahad Shahbaz Khan, Mubarak Shah
Date: 2021-04-08 00:00:00
Proceeding: ICCV 2021 10
Tasks: Decoder, Image Generation, Text Generation

We propose a novel transformer-based styled handwritten text image generation approach, HWT, that strives to learn both style-content entanglement as well as global and local writing style patterns. The proposed HWT captures the long and short range relationships within the style examples through a self-attention mechanism, thereby encoding both global and local style patterns. Further, the proposed transformer-based HWT comprises an encoder-decoder attention that enables style-content entanglement by gathering the style representation of each query character. To the best of our knowledge, we are the first to introduce a transformer-based generative network for styled handwritten text generation. Our proposed HWT generates realistic styled handwritten text images and significantly outperforms the state-of-the-art demonstrated through extensive qualitative, quantitative and human-based evaluations. The proposed HWT can handle arbitrary length of text and any desired writing style in a few-shot setting. Further, our HWT generalizes well to the challenging scenario where both words and writing style are unseen during training, generating realistic styled handwritten text images.
Generating Multi-type Temporal Sequences to Mitigate Class-imbalanced Problem
Authors: Lun Jiang, Nima Salehi Sadghiani, Zhuo Tao, Andrew Cohen
Date: 2021-04-07 00:00:00
Tasks: BIG-bench Machine Learning, Click-Through Rate Prediction, Fraud Detection, Reinforcement Learning (RL), Temporal Sequences, Vocal Bursts Type Prediction

From the ad network standpoint, a user's activity is a multi-type sequence of temporal events consisting of event types and time intervals. Understanding user patterns in ad networks has received increasing attention from the machine learning community. Particularly, the problems of fraud detection, Conversion Rate (CVR), and Click-Through Rate (CTR) prediction are of interest. However, the class imbalance between major and minor classes in these tasks can bias a machine learning model leading to poor performance. This study proposes using two multi-type (continuous and discrete) training approaches for GANs to deal with the limitations of traditional GANs in passing the gradient updates for discrete tokens. First, we used the Reinforcement Learning (RL)-based training approach and then, an approximation of the multinomial distribution parameterized in terms of the softmax function (Gumble-Softmax). Our extensive experiments based on synthetic data have shown the trained generator can generate sequences with desired properties measured by multiple criteria.
Universal Spectral Adversarial Attacks for Deformable Shapes
Authors: Arianna Rampini, Franco Pestarini, Luca Cosmo, Simone Melzi, Emanuele Rodolà
Date: 2021-04-07 00:00:00
Proceeding: CVPR 2021 1

Machine learning models are known to be vulnerable to adversarial attacks, namely perturbations of the data that lead to wrong predictions despite being imperceptible. However, the existence of "universal" attacks (i.e., unique perturbations that transfer across different data points) has only been demonstrated for images to date. Part of the reason lies in the lack of a common domain, for geometric data such as graphs, meshes, and point clouds, where a universal perturbation can be defined. In this paper, we offer a change in perspective and demonstrate the existence of universal attacks for geometric data (shapes). We introduce a computational procedure that operates entirely in the spectral domain, where the attacks take the form of small perturbations to short eigenvalue sequences; the resulting geometry is then synthesized via shape-from-spectrum recovery. Our attacks are universal, in that they transfer across different shapes, different representations (meshes and point clouds), and generalize to previously unseen data.
Triplot: model agnostic measures and visualisations for variable importance in predictive models that take into account the hierarchical correlation structure
Authors: Katarzyna Pekala, Katarzyna Woznica, Przemyslaw Biecek
Date: 2021-04-07 00:00:00
Tasks: Explainable artificial intelligence, Interpretable Machine Learning

One of the key elements of explanatory analysis of a predictive model is to assess the importance of individual variables. Rapid development of the area of predictive model exploration (also called explainable artificial intelligence or interpretable machine learning) has led to the popularization of methods for local (instance level) and global (dataset level) methods, such as Permutational Variable Importance, Shapley Values (SHAP), Local Interpretable Model Explanations (LIME), Break Down and so on. However, these methods do not use information about the correlation between features which significantly reduce the explainability of the model behaviour. In this work, we propose new methods to support model analysis by exploiting the information about the correlation between variables. The dataset level aspect importance measure is inspired by the block permutations procedure, while the instance level aspect importance measure is inspired by the LIME method. We show how to analyze groups of variables (aspects) both when they are proposed by the user and when they should be determined automatically based on the hierarchical structure of correlations between variables. Additionally, we present the new type of model visualisation, triplot, which exploits a hierarchical structure of variable grouping to produce a high information density model visualisation. This visualisation provides a consistent illustration for either local or global model and data exploration. We also show an example of real-world data with 5k instances and 37 features in which a significant correlation between variables affects the interpretation of the effect of variable importance. The proposed method is, to our knowledge, the first to allow direct use of the correlation between variables in exploratory model analysis.
Rethinking the Backdoor Attacks' Triggers: A Frequency Perspective
Authors: Yi Zeng, Won Park, Z. Morley Mao, Ruoxi Jia
Date: 2021-04-07 00:00:00
Proceeding: ICCV 2021 10

Backdoor attacks have been considered a severe security threat to deep learning. Such attacks can make models perform abnormally on inputs with predefined triggers and still retain state-of-the-art performance on clean data. While backdoor attacks have been thoroughly investigated in the image domain from both attackers' and defenders' sides, an analysis in the frequency domain has been missing thus far. This paper first revisits existing backdoor triggers from a frequency perspective and performs a comprehensive analysis. Our results show that many current backdoor attacks exhibit severe high-frequency artifacts, which persist across different datasets and resolutions. We further demonstrate these high-frequency artifacts enable a simple way to detect existing backdoor triggers at a detection rate of 98.50% without prior knowledge of the attack details and the target model. Acknowledging previous attacks' weaknesses, we propose a practical way to create smooth backdoor triggers without high-frequency artifacts and study their detectability. We show that existing defense works can benefit by incorporating these smooth triggers into their design consideration. Moreover, we show that the detector tuned over stronger smooth triggers can generalize well to unseen weak smooth triggers. In short, our work emphasizes the importance of considering frequency analysis when designing both backdoor attacks and defenses in deep learning.
Semi-supervised on-device neural network adaptation for remote and portable laser-induced breakdown spectroscopy
Authors: Kshitij Bhardwaj, Maya Gokhale
Date: 2021-04-08 00:00:00

Laser-induced breakdown spectroscopy (LIBS) is a popular, fast elemental analysis technique used to determine the chemical composition of target samples, such as in industrial analysis of metals or in space exploration. Recently, there has been a rise in the use of machine learning (ML) techniques for LIBS data processing. However, ML for LIBS is challenging as: (i) the predictive models must be lightweight since they need to be deployed in highly resource-constrained and battery-operated portable LIBS systems; and (ii) since these systems can be remote, the models must be able to self-adapt to any domain shift in input distributions which could be due to the lack of different types of inputs in training data or dynamic environmental/sensor noise. This on-device retraining of model should not only be fast but also unsupervised due to the absence of new labeled data in remote LIBS systems. We introduce a lightweight multi-layer perceptron (MLP) model for LIBS that can be adapted on-device without requiring labels for new input data. It shows 89.3% average accuracy during data streaming, and up to 2.1% better accuracy compared to an MLP model that does not support adaptation. Finally, we also characterize the inference and retraining performance of our model on Google Pixel2 phone.
Learning Graph Structures with Transformer for Multivariate Time Series Anomaly Detection in IoT
Authors: Zekai Chen, Dingshuo Chen, Xiao Zhang, Zixuan Yuan, Xiuzhen Cheng
Date: 2021-04-08 00:00:00
Tasks: Anomaly Detection, Time Series, Time Series Analysis, Time Series Anomaly Detection

Many real-world IoT systems, which include a variety of internet-connected sensory devices, produce substantial amounts of multivariate time series data. Meanwhile, vital IoT infrastructures like smart power grids and water distribution networks are frequently targeted by cyber-attacks, making anomaly detection an important study topic. Modeling such relatedness is, nevertheless, unavoidable for any efficient and effective anomaly detection system, given the intricate topological and nonlinear connections that are originally unknown among sensors. Furthermore, detecting anomalies in multivariate time series is difficult due to their temporal dependency and stochasticity. This paper presented GTA, a new framework for multivariate time series anomaly detection that involves automatically learning a graph structure, graph convolution, and modeling temporal dependency using a Transformer-based architecture. The connection learning policy, which is based on the Gumbel-softmax sampling approach to learn bi-directed links among sensors directly, is at the heart of learning graph structure. To describe the anomaly information flow between network nodes, we introduced a new graph convolution called Influence Propagation convolution. In addition, to tackle the quadratic complexity barrier, we suggested a multi-branch attention mechanism to replace the original multi-head self-attention method. Extensive experiments on four publicly available anomaly detection benchmarks further demonstrate the superiority of our approach over alternative state-of-the-arts. Codes are available at https://github.com/ZEKAICHEN/GTA.
Gi and Pal Scores: Deep Neural Network Generalization Statistics
Authors: Yair Schiff, Brian Quanz, Payel Das, Pin-Yu Chen
Date: 2021-04-08 00:00:00
Tasks: regression

The field of Deep Learning is rich with empirical evidence of human-like performance on a variety of regression, classification, and control tasks. However, despite these successes, the field lacks strong theoretical error bounds and consistent measures of network generalization and learned invariances. In this work, we introduce two new measures, the Gi-score and Pal-score, that capture a deep neural network's generalization capabilities. Inspired by the Gini coefficient and Palma ratio, measures of income inequality, our statistics are robust measures of a network's invariance to perturbations that accurately predict generalization gaps, i.e., the difference between accuracy on training and test sets.