pwc-archive/papers-with-abstracts huggingface dataset:
Defensive Approximation: Securing CNNs using Approximate Computing
Authors: Amira Guesmi, Ihsen Alouani, Khaled Khasawneh, Mouna Baklouti, Tarek Frikha, Mohamed Abid, Nael Abu-Ghazaleh
Date: 2020-06-13 00:00:00

In the past few years, an increasing number of machine-learning and deep learning structures, such as Convolutional Neural Networks (CNNs), have been applied to solving a wide range of real-life problems. However, these architectures are vulnerable to adversarial attacks. In this paper, we propose for the first time to use hardware-supported approximate computing to improve the robustness of machine learning classifiers. We show that our approximate computing implementation achieves robustness across a wide range of attack scenarios. Specifically, for black-box and grey-box attack scenarios, we show that successful adversarial attacks against the exact classifier have poor transferability to the approximate implementation. Surprisingly, the robustness advantages also apply to white-box attacks where the attacker has access to the internal implementation of the approximate classifier. We explain some of the possible reasons for this robustness through analysis of the internal operation of the approximate implementation. Furthermore, our approximate computing model maintains the same level in terms of classification accuracy, does not require retraining, and reduces resource utilization and energy consumption of the CNN. We conducted extensive experiments on a set of strong adversarial attacks; We empirically show that the proposed implementation increases the robustness of a LeNet-5 and an Alexnet CNNs by up to 99% and 87%, respectively for strong grey-box adversarial attacks along with up to 67% saving in energy consumption due to the simpler nature of the approximate logic. We also show that a white-box attack requires a remarkably higher noise budget to fool the approximate classifier, causing an average of 4db degradation of the PSNR of the input image relative to the images that succeed in fooling the exact classifier
Transferring Monolingual Model to Low-Resource Language: The Case of Tigrinya
Authors: Abrhalei Tela, Abraham Woubie, Ville Hautamaki
Date: 2020-06-13 00:00:00
Tasks: Language Modeling, Language Modelling, Sentiment Analysis, Transfer Learning

In recent years, transformer models have achieved great success in natural language processing (NLP) tasks. Most of the current state-of-the-art NLP results are achieved by using monolingual transformer models, where the model is pre-trained using a single language unlabelled text corpus. Then, the model is fine-tuned to the specific downstream task. However, the cost of pre-training a new transformer model is high for most languages. In this work, we propose a cost-effective transfer learning method to adopt a strong source language model, trained from a large monolingual corpus to a low-resource language. Thus, using XLNet language model, we demonstrate competitive performance with mBERT and a pre-trained target language model on the cross-lingual sentiment (CLS) dataset and on a new sentiment analysis dataset for low-resourced language Tigrinya. With only 10k examples of the given Tigrinya sentiment analysis dataset, English XLNet has achieved 78.88% F1-Score outperforming BERT and mBERT by 10% and 7%, respectively. More interestingly, fine-tuning (English) XLNet model on the CLS dataset has promising results compared to mBERT and even outperformed mBERT for one dataset of the Japanese language.
Learning Sparse Graphons and the Generalized Kesten-Stigum Threshold
Authors: Emmanuel Abbe, Shuangping Li, Allan Sly
Date: 2020-06-13 00:00:00

The problem of learning graphons has attracted considerable attention across several scientific communities, with significant progress over the recent years in sparser regimes. Yet, the current techniques still require diverging degrees in order to succeed with efficient algorithms in the challenging cases where the local structure of the graph is homogeneous. This paper provides an efficient algorithm to learn graphons in the constant expected degree regime. The algorithm is shown to succeed in estimating the rank-$k$ projection of a graphon in the $L_2$ metric if the top $k$ eigenvalues of the graphon satisfy a generalized Kesten-Stigum condition.
Sensorless Freehand 3D Ultrasound Reconstruction via Deep Contextual Learning
Authors: Hengtao Guo, Sheng Xu, Bradford Wood, Pingkun Yan
Date: 2020-06-13 00:00:00

Transrectal ultrasound (US) is the most commonly used imaging modality to guide prostate biopsy and its 3D volume provides even richer context information. Current methods for 3D volume reconstruction from freehand US scans require external tracking devices to provide spatial position for every frame. In this paper, we propose a deep contextual learning network (DCL-Net), which can efficiently exploit the image feature relationship between US frames and reconstruct 3D US volumes without any tracking device. The proposed DCL-Net utilizes 3D convolutions over a US video segment for feature extraction. An embedded self-attention module makes the network focus on the speckle-rich areas for better spatial movement prediction. We also propose a novel case-wise correlation loss to stabilize the training process for improved accuracy. Highly promising results have been obtained by using the developed method. The experiments with ablation studies demonstrate superior performance of the proposed method by comparing against other state-of-the-art methods. Source code of this work is publicly available at https://github.com/DIAL-RPI/FreehandUSRecon.
A New Algorithm for Tessellated Kernel Learning
Authors: Brendon K. Colbert, Matthew M. Peet
Date: 2020-06-13 00:00:00
Tasks: regression

The accuracy and complexity of machine learning algorithms based on kernel optimization are limited by the set of kernels over which they are able to optimize. An ideal set of kernels should: admit a linear parameterization (for tractability); be dense in the set of all kernels (for robustness); be universal (for accuracy). The recently proposed Tesselated Kernels (TKs) is currently the only known class which meets all three criteria. However, previous algorithms for optimizing TKs were limited to classification and relied on Semidefinite Programming (SDP) - limiting them to relatively small datasets. By contrast, the 2-step algorithm proposed here scales to 10,000 data points and extends to the regression problem. Furthermore, when applied to benchmark data, the algorithm demonstrates significant improvement in performance over Neural Nets and SimpleMKL with similar computation time.
Synthetic Interventions
Authors: Anish Agarwal, Devavrat Shah, Dennis Shen
Date: 2020-06-13 00:00:00

The synthetic controls (SC) methodology is a prominent tool for policy evaluation in panel data applications. Researchers commonly justify the SC framework with a low-rank matrix factor model that assumes the potential outcomes are described by low-dimensional unit and time specific latent factors. In the recent work of [Abadie '20], one of the pioneering authors of the SC method posed the question of how the SC framework can be extended to multiple treatments. This article offers one resolution to this open question that we call synthetic interventions (SI). Fundamental to the SI framework is a low-rank tensor factor model, which extends the matrix factor model by including a latent factorization over treatments. Under this model, we propose a generalization of the standard SC-based estimators. We prove the consistency for one instantiation of our approach and provide conditions under which it is asymptotically normal. Moreover, we conduct a representative simulation to study its prediction performance and revisit the canonical SC case study of [Abadie-Diamond-Hainmueller '10] on the impact of anti-tobacco legislations by exploring related questions not previously investigated.
Faster MCMC for Gaussian Latent Position Network Models
Authors: Neil A. Spencer, Brian Junker, Tracy M. Sweet
Date: 2020-06-13 00:00:00
Tasks: Bayesian Inference, Clustering, Position

Latent position network models are a versatile tool in network science; applications include clustering entities, controlling for causal confounders, and defining priors over unobserved graphs. Estimating each node's latent position is typically framed as a Bayesian inference problem, with Metropolis within Gibbs being the most popular tool for approximating the posterior distribution. However, it is well-known that Metropolis within Gibbs is inefficient for large networks; the acceptance ratios are expensive to compute, and the resultant posterior draws are highly correlated. In this article, we propose an alternative Markov chain Monte Carlo strategy -- defined using a combination of split Hamiltonian Monte Carlo and Firefly Monte Carlo -- that leverages the posterior distribution's functional form for more efficient posterior computation. We demonstrate that these strategies outperform Metropolis within Gibbs and other algorithms on synthetic networks, as well as on real information-sharing networks of teachers and staff in a school district.
Predictive modeling approaches in laser-based material processing
Authors: Maria Christina Velli, George D. Tsibidis, Alexandros Mimidis, Evangelos Skoulas, Yannis Pantazis, Emmanuel Stratakis
Date: 2020-06-13 00:00:00

Predictive modelling represents an emerging field that combines existing and novel methodologies aimed to rapidly understand physical mechanisms and concurrently develop new materials, processes and structures. In the current study, previously-unexplored predictive modelling in a key-enabled technology, the laser-based manufacturing, aims to automate and forecast the effect of laser processing on material structures. The focus is centred on the performance of representative statistical and machine learning algorithms in predicting the outcome of laser processing on a range of materials. Results on experimental data showed that predictive models were able to satisfactorily learn the mapping between the laser input variables and the observed material structure. These results are further integrated with simulation data aiming to elucidate the multiscale physical processes upon laser-material interaction. As a consequence, we augmented the adjusted simulated data to the experimental and substantially improved the predictive performance, due to the availability of increased number of sampling points. In parallel, a metric to identify and quantify the regions with high predictive uncertainty, is presented, revealing that high uncertainty occurs around the transition boundaries. Our results can set the basis for a systematic methodology towards reducing material design, testing and production cost via the replacement of expensive trial-and-error based manufacturing procedure with a precise pre-fabrication predictive tool.
Rethinking Clustering for Robustness
Authors: Motasem Alfarra, Juan C. Pérez, Adel Bibi, Ali Thabet, Pablo Arbeláez, Bernard Ghanem
Date: 2020-06-13 00:00:00
Tasks: Clustering

This paper studies how encouraging semantically-aligned features during deep neural network training can increase network robustness. Recent works observed that Adversarial Training leads to robust models, whose learnt features appear to correlate with human perception. Inspired by this connection from robustness to semantics, we study the complementary connection: from semantics to robustness. To do so, we provide a robustness certificate for distance-based classification models (clustering-based classifiers). Moreover, we show that this certificate is tight, and we leverage it to propose ClusTR (Clustering Training for Robustness), a clustering-based and adversary-free training framework to learn robust models. Interestingly, \textit{ClusTR} outperforms adversarially-trained networks by up to $4\%$ under strong PGD attacks.
High-Dimensional Similarity Search with Quantum-Assisted Variational Autoencoder
Authors: Nicholas Gao, Max Wilson, Thomas Vandal, Walter Vinci, Ramakrishna Nemani, Eleanor Rieffel
Date: 2020-06-13 00:00:00
Tasks: Quantum Machine Learning, Vocal Bursts Intensity Prediction

Recent progress in quantum algorithms and hardware indicates the potential importance of quantum computing in the near future. However, finding suitable application areas remains an active area of research. Quantum machine learning is touted as a potential approach to demonstrate quantum advantage within both the gate-model and the adiabatic schemes. For instance, the Quantum-assisted Variational Autoencoder has been proposed as a quantum enhancement to the discrete VAE. We extend on previous work and study the real-world applicability of a QVAE by presenting a proof-of-concept for similarity search in large-scale high-dimensional datasets. While exact and fast similarity search algorithms are available for low dimensional datasets, scaling to high-dimensional data is non-trivial. We show how to construct a space-efficient search index based on the latent space representation of a QVAE. Our experiments show a correlation between the Hamming distance in the embedded space and the Euclidean distance in the original space on the Moderate Resolution Imaging Spectroradiometer (MODIS) dataset. Further, we find real-world speedups compared to linear search and demonstrate memory-efficient scaling to half a billion data points.
Collegial Ensembles
Authors: Etai Littwin, Ben Myara, Sima Sabah, Joshua Susskind, Shuangfei Zhai, Oren Golan
Date: 2020-06-13 00:00:00
Proceeding: NeurIPS 2020 12

Modern neural network performance typically improves as model size increases. A recent line of research on the Neural Tangent Kernel (NTK) of over-parameterized networks indicates that the improvement with size increase is a product of a better conditioned loss landscape. In this work, we investigate a form of over-parameterization achieved through ensembling, where we define collegial ensembles (CE) as the aggregation of multiple independent models with identical architectures, trained as a single model. We show that the optimization dynamics of CE simplify dramatically when the number of models in the ensemble is large, resembling the dynamics of wide models, yet scale much more favorably. We use recent theoretical results on the finite width corrections of the NTK to perform efficient architecture search in a space of finite width CE that aims to either minimize capacity, or maximize trainability under a set of constraints. The resulting ensembles can be efficiently implemented in practical architectures using group convolutions and block diagonal layers. Finally, we show how our framework can be used to analytically derive optimal group convolution modules originally found using expensive grid searches, without having to train a single model.
Words ranking and Hirsch index for identifying the core of the hapaxes in political texts
Authors: Valerio Ficcadenti, Roy Cerqueti, Marcel Ausloos, Gurjeet Dhesi
Date: 2020-06-13 00:00:00

This paper deals with a quantitative analysis of the content of official political speeches. We study a set of about one thousand talks pronounced by the US Presidents, ranging from Washington to Trump. In particular, we search for the relevance of the rare words, i.e. those said only once in each speech -- the so-called hapaxes. We implement a rank-size procedure of Zipf-Mandelbrot type for discussing the hapaxes' frequencies regularity over the overall set of speeches. Starting from the obtained rank-size law, we define and detect the core of the hapaxes set by means of a procedure based on an Hirsch index variant. We discuss the resulting list of words in the light of the overall US Presidents' speeches. We further show that this core of hapaxes itself can be well fitted through a Zipf-Mandelbrot law and that contains elements producing deviations at the low ranks between scatter plots and fitted curve -- the so-called king and vice-roy effect. Some socio-political insights are derived from the obtained findings about the US Presidents messages.
Uncertainty-aware Score Distribution Learning for Action Quality Assessment
Authors: Yansong Tang, Zanlin Ni, Jiahuan Zhou, Danyang Zhang, Jiwen Lu, Ying Wu, Jie zhou
Date: 2020-06-13 00:00:00
Proceeding: CVPR 2020 6
Tasks: Action Quality Assessment

Assessing action quality from videos has attracted growing attention in recent years. Most existing approaches usually tackle this problem based on regression algorithms, which ignore the intrinsic ambiguity in the score labels caused by multiple judges or their subjective appraisals. To address this issue, we propose an uncertainty-aware score distribution learning (USDL) approach for action quality assessment (AQA). Specifically, we regard an action as an instance associated with a score distribution, which describes the probability of different evaluated scores. Moreover, under the circumstance where fine-grained score labels are available (e.g., difficulty degree of an action or multiple scores from different judges), we further devise a multi-path uncertainty-aware score distributions learning (MUSDL) method to explore the disentangled components of a score. We conduct experiments on three AQA datasets containing various Olympic actions and surgical activities, where our approaches set new state-of-the-arts under the Spearman's Rank Correlation.
Automate Obstructive Sleep Apnea Diagnosis Using Convolutional Neural Networks
Authors: Longlong Feng, Xu Wang
Date: 2020-06-13 00:00:00
Tasks: Classification, General Classification

Identifying sleep problem severity from overnight polysomnography (PSG) recordings plays an important role in diagnosing and treating sleep disorders such as the Obstructive Sleep Apnea (OSA). This analysis traditionally is done by specialists manually through visual inspections, which can be tedious, time-consuming, and is prone to subjective errors. One of the solutions is to use Convolutional Neural Networks (CNN) where the convolutional and pooling layers behave as feature extractors and some fully-connected (FCN) layers are used for making final predictions for the OSA severity. In this paper, a CNN architecture with 1D convolutional and FCN layers for classification is presented. The PSG data for this project are from the Cleveland Children's Sleep and Health Study database and classification results confirm the effectiveness of the proposed CNN method. The proposed 1D CNN model achieves excellent classification results without manually preprocesssing PSG signals such as feature extraction and feature reduction.
Convolutional Generation of Textured 3D Meshes
Authors: Dario Pavllo, Graham Spinks, Thomas Hofmann, Marie-Francine Moens, Aurelien Lucchi
Date: 2020-06-13 00:00:00
Proceeding: NeurIPS 2020 12

While recent generative models for 2D images achieve impressive visual results, they clearly lack the ability to perform 3D reasoning. This heavily restricts the degree of control over generated objects as well as the possible applications of such models. In this work, we bridge this gap by leveraging recent advances in differentiable rendering. We design a framework that can generate triangle meshes and associated high-resolution texture maps, using only 2D supervision from single-view natural images. A key contribution of our work is the encoding of the mesh and texture as 2D representations, which are semantically aligned and can be easily modeled by a 2D convolutional GAN. We demonstrate the efficacy of our method on Pascal3D+ Cars and CUB, both in an unconditional setting and in settings where the model is conditioned on class labels, attributes, and text. Finally, we propose an evaluation methodology that assesses the mesh and texture quality separately.
Top influencers can be identified universally by combining classical centralities
Authors: Doina Bucur
Date: 2020-06-13 00:00:00

Information flow, opinion, and epidemics spread over structured networks. When using individual node centrality indicators to predict which nodes will be among the top influencers or spreaders in a large network, no single centrality has consistently good ranking power. We show that statistical classifiers using two or more centralities as input are instead consistently predictive over many diverse, static real-world topologies. Certain pairs of centralities cooperate particularly well in statistically drawing the boundary between the top spreaders and the rest: local centralities measuring the size of a node's neighbourhood benefit from the addition of a global centrality such as the eigenvector centrality, closeness, or the core number. This is, intuitively, because a local centrality may rank highly some nodes which are located in dense, but peripheral regions of the network---a situation in which an additional global centrality indicator can help by prioritising nodes located more centrally. The nodes selected as superspreaders will usually jointly maximise the values of both centralities. As a result of the interplay between centrality indicators, training classifiers with seven classical indicators leads to a nearly maximum average precision function (0.995) across the networks in this study.
Horseshoe Prior Bayesian Quantile Regression
Authors: David Kohns, Tibor Szendrei
Date: 2020-06-13 00:00:00
Tasks: quantile regression, regression

This paper extends the horseshoe prior of Carvalho et al. (2010) to Bayesian quantile regression (HS-BQR) and provides a fast sampling algorithm for computation in high dimensions. The performance of the proposed HS-BQR is evaluated on Monte Carlo simulations and a high dimensional Growth-at-Risk (GaR) forecasting application for the U.S. The Monte Carlo design considers several sparsity and error structures. Compared to alternative shrinkage priors, the proposed HS-BQR yields better (or at worst similar) performance in coefficient bias and forecast error. The HS-BQR is particularly potent in sparse designs and in estimating extreme quantiles. As expected, the simulations also highlight that identifying quantile specific location and scale effects for individual regressors in dense DGPs requires substantial data. In the GaR application, we forecast tail risks as well as complete forecast densities using the McCracken and Ng (2020) database. Quantile specific and density calibration score functions show that the HS-BQR provides the best performance, especially at short and medium run horizons. The ability to produce well calibrated density forecasts and accurate downside risk measures in large data contexts makes the HS-BQR a promising tool for nowcasting applications and recession modelling.
Structure learning for CTBN's via penalized maximum likelihood methods
Authors: Maryia Shpak, Błażej Miasojedow, Wojciech Rejchel
Date: 2020-06-13 00:00:00

The continuous-time Bayesian networks (CTBNs) represent a class of stochastic processes, which can be used to model complex phenomena, for instance, they can describe interactions occurring in living processes, in social science models or in medicine. The literature on this topic is usually focused on the case when the dependence structure of a system is known and we are to determine conditional transition intensities (parameters of the network). In the paper, we study the structure learning problem, which is a more challenging task and the existing research on this topic is limited. The approach, which we propose, is based on a penalized likelihood method. We prove that our algorithm, under mild regularity conditions, recognizes the dependence structure of the graph with high probability. We also investigate the properties of the procedure in numerical studies to demonstrate its effectiveness.