pwc-archive/papers-with-abstracts huggingface dataset:
Bayes-TrEx: a Bayesian Sampling Approach to Model Transparency by Example
Authors: Serena Booth, Yilun Zhou, Ankit Shah, Julie Shah
Date: 2020-02-19 00:00:00
Tasks: Domain Adaptation

Post-hoc explanation methods are gaining popularity for interpreting, understanding, and debugging neural networks. Most analyses using such methods explain decisions in response to inputs drawn from the test set. However, the test set may have few examples that trigger some model behaviors, such as high-confidence failures or ambiguous classifications. To address these challenges, we introduce a flexible model inspection framework: Bayes-TrEx. Given a data distribution, Bayes-TrEx finds in-distribution examples with a specified prediction confidence. We demonstrate several use cases of Bayes-TrEx, including revealing highly confident (mis)classifications, visualizing class boundaries via ambiguous examples, understanding novel-class extrapolation behavior, and exposing neural network overconfidence. We use Bayes-TrEx to study classifiers trained on CLEVR, MNIST, and Fashion-MNIST, and we show that this framework enables more flexible holistic model analysis than just inspecting the test set. Code is available at https://github.com/serenabooth/Bayes-TrEx.
Network Clustering Via Kernel-ARMA Modeling and the Grassmannian The Brain-Network Case
Authors: Cong Ye, Konstantinos Slavakis, Pratik V. Patil, Johan Nakuci, Sarah F. Muldoon, John Medaglia
Date: 2020-02-18 00:00:00
Tasks: Clustering, Community Detection, Node Clustering, Time Series, Time Series Analysis

This paper introduces a clustering framework for networks with nodes annotated with time-series data. The framework addresses all types of network-clustering problems: State clustering, node clustering within states (a.k.a. topology identification or community detection), and even subnetwork-state-sequence identification/tracking. Via a bottom-up approach, features are first extracted from the raw nodal time-series data by kernel autoregressive-moving-average modeling to reveal non-linear dependencies and low-rank representations, and then mapped onto the Grassmann manifold (Grassmannian). All clustering tasks are performed by leveraging the underlying Riemannian geometry of the Grassmannian in a novel way. To validate the proposed framework, brain-network clustering is considered, where extensive numerical tests on synthetic and real functional magnetic resonance imaging (fMRI) data demonstrate that the advocated learning framework compares favorably versus several state-of-the-art clustering schemes.
On the Estimation of Complex Circuits Functional Failure Rate by Machine Learning Techniques
Authors: Thomas Lange, Aneesh Balakrishnan, Maximilien Glorieux, Dan Alexandrescu, Luca Sterpone
Date: 2020-02-18 00:00:00
Tasks: BIG-bench Machine Learning

De-Rating or Vulnerability Factors are a major feature of failure analysis efforts mandated by today's Functional Safety requirements. Determining the Functional De-Rating of sequential logic cells typically requires computationally intensive fault-injection simulation campaigns. In this paper a new approach is proposed which uses Machine Learning to estimate the Functional De-Rating of individual flip-flops and thus, optimising and enhancing fault injection efforts. Therefore, first, a set of per-instance features is described and extracted through an analysis approach combining static elements (cell properties, circuit structure, synthesis attributes) and dynamic elements (signal activity). Second, reference data is obtained through first-principles fault simulation approaches. Finally, one part of the reference dataset is used to train the Machine Learning algorithm and the remaining is used to validate and benchmark the accuracy of the trained tool. The intended goal is to obtain a trained model able to provide accurate per-instance Functional De-Rating data for the full list of circuit instances, an objective that is difficult to reach using classical methods. The presented methodology is accompanied by a practical example to determine the performance of various Machine Learning models for different training sizes.
TensorShield: Tensor-based Defense Against Adversarial Attacks on Images
Authors: Negin Entezari, Evangelos E. Papalexakis
Date: 2020-02-18 00:00:00
Tasks: BIG-bench Machine Learning, image-classification, Image Classification, Tensor Decomposition

Recent studies have demonstrated that machine learning approaches like deep neural networks (DNNs) are easily fooled by adversarial attacks. Subtle and imperceptible perturbations of the data are able to change the result of deep neural networks. Leveraging vulnerable machine learning methods raises many concerns especially in domains where security is an important factor. Therefore, it is crucial to design defense mechanisms against adversarial attacks. For the task of image classification, unnoticeable perturbations mostly occur in the high-frequency spectrum of the image. In this paper, we utilize tensor decomposition techniques as a preprocessing step to find a low-rank approximation of images which can significantly discard high-frequency perturbations. Recently a defense framework called Shield could "vaccinate" Convolutional Neural Networks (CNN) against adversarial examples by performing random-quality JPEG compressions on local patches of images on the ImageNet dataset. Our tensor-based defense mechanism outperforms the SLQ method from Shield by 14% against FastGradient Descent (FGSM) adversarial attacks, while maintaining comparable speed.
Empirical Study on Airline Delay Analysis and Prediction
Authors: Ripon Patgiri, Sajid Hussain, Aditya Nongmeikapam
Date: 2020-02-17 00:00:00
Tasks: Decision Making, L2 Regularization, Prediction

The Big Data analytics are a logical analysis of very large scale datasets. The data analysis enhances an organization and improve the decision making process. In this article, we present Airline Delay Analysis and Prediction to analyze airline datasets with the combination of weather dataset. In this research work, we consider various attributes to analyze flight delay, for example, day-wise, airline-wise, cloud cover, temperature, etc. Moreover, we present rigorous experiments on various machine learning model to predict correctly the delay of a flight, namely, logistic regression with L2 regularization, Gaussian Naive Bayes, K-Nearest Neighbors, Decision Tree classifier and Random forest model. The accuracy of the Random Forest model is 82% with a delay threshold of 15 minutes of flight delay. The analysis is carried out using dataset from 1987 to 2008, the training is conducted with dataset from 2000 to 2007 and validated prediction result using 2008 data. Moreover, we have got recall 99% in the Random Forest model.
ArcText: A Unified Text Approach to Describing Convolutional Neural Network Architectures
Authors: Yanan Sun, Ziyao Ren, Gary G. Yen, Bing Xue, Mengjie Zhang, Jiancheng Lv
Date: 2020-02-16 00:00:00

The superiority of Convolutional Neural Networks (CNNs) largely relies on their architectures that are often manually crafted with extensive human expertise. Unfortunately, such kind of domain knowledge is not necessarily owned by each of the users interested. Data mining on existing CNN can discover useful patterns and fundamental sub-comments from their architectures, providing researchers with strong prior knowledge to design proper CNN architectures when they have no expertise in CNNs. There have been various state-of-the-art data mining algorithms at hand, while there is only rare work that has been done for the mining. One of the main reasons is the gap between CNN architectures and data mining algorithms. Specifically, the current CNN architecture descriptions cannot be exactly vectorized to the input of data mining algorithms. In this paper, we propose a unified approach, named ArcText, to describing CNN architectures based on text. Particularly, four different units and an ordering method have been elaborately designed in ArcText, to uniquely describe the same architecture with sufficient information. Also, the resulted description can be exactly converted back to the corresponding CNN architecture. ArcText bridges the gap between CNN architectures and data mining researchers, and has the potentiality to be utilized to wider scenarios.
The Archimedean trap: Why traditional reinforcement learning will probably not yield AGI
Authors: Samuel Allen Alexander
Date: 2020-02-15 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

After generalizing the Archimedean property of real numbers in such a way as to make it adaptable to non-numeric structures, we demonstrate that the real numbers cannot be used to accurately measure non-Archimedean structures. We argue that, since an agent with Artificial General Intelligence (AGI) should have no problem engaging in tasks that inherently involve non-Archimedean rewards, and since traditional reinforcement learning rewards are real numbers, therefore traditional reinforcement learning probably will not lead to AGI. We indicate two possible ways traditional reinforcement learning could be altered to remove this roadblock.
DeepPlume: Very High Resolution Real-Time Air Quality Mapping
Authors: Grégoire Jauvion, Thibaut Cassard, Boris Quennehen, David Lissmyr
Date: 2020-02-14 00:00:00
Tasks: Vocal Bursts Intensity Prediction

This paper presents an engine able to predict jointly the real-time concentration of the main pollutants harming people's health: nitrogen dioxyde (NO2), ozone (O3) and particulate matter (PM2.5 and PM10, which are respectively the particles whose size are below 2.5 um and 10 um). The engine covers a large part of the world and is fed with real-time official stations measures, atmospheric models' forecasts, land cover data, road networks and traffic estimates to produce predictions with a very high resolution in the range of a few dozens of meters. This resolution makes the engine adapted to very innovative applications like street-level air quality mapping or air quality adjusted routing. Plume Labs has deployed a similar prediction engine to build several products aiming at providing air quality data to individuals and businesses. For the sake of clarity and reproducibility, the engine presented here has been built specifically for this paper and differs quite significantly from the one used in Plume Labs' products. A major difference is in the data sources feeding the engine: in particular, this prediction engine does not include mobile sensors measurements.
Dynamic Systems Simulation and Control Using Consecutive Recurrent Neural Networks
Authors: Srikanth Chandar, Harsha Sunder
Date: 2020-02-14 00:00:00

In this paper, we introduce a novel architecture to connecting adaptive learning and neural networks into an arbitrary machine's control system paradigm. Two consecutive Recurrent Neural Networks (RNNs) are used together to accurately model the dynamic characteristics of electromechanical systems that include controllers, actuators and motors. The age-old method of achieving control with the use of the- Proportional, Integral and Derivative constants is well understood as a simplified method that does not capture the complexities of the inherent nonlinearities of complex control systems. In the context of controlling and simulating electromechanical systems, we propose an alternative to PID controllers, employing a sequence of two Recurrent Neural Networks. The first RNN emulates the behavior of the controller, and the second the actuator/motor. The second RNN when used in isolation, potentially serves as an advantageous alternative to extant testing methods of electromechanical systems.
Boosting rare benthic macroinvertebrates taxa identification with one-class classification
Authors: Fahad Sohrab, Jenni Raitoharju
Date: 2020-02-12 00:00:00
Tasks: Classification, General Classification, One-Class Classification

Insect monitoring is crucial for understanding the consequences of rapid ecological changes, but taxa identification currently requires tedious manual expert work and cannot be scaled-up efficiently. Deep convolutional neural networks (CNNs), provide a viable way to significantly increase the biomonitoring volumes. However, taxa abundances are typically very imbalanced and the amounts of training images for the rarest classes are simply too low for deep CNNs. As a result, the samples from the rare classes are often completely missed, while detecting them has biological importance. In this paper, we propose combining the trained deep CNN with one-class classifiers to improve the rare species identification. One-class classification models are traditionally trained with much fewer samples and they can provide a mechanism to indicate samples potentially belonging to the rare classes for human inspection. Our experiments confirm that the proposed approach may indeed support moving towards partial automation of the taxa identification task.
Uncertainty based Class Activation Maps for Visual Question Answering
Authors: Badri N. Patro, Mayank Lunayach, Vinay P. Namboodiri
Date: 2020-01-23 00:00:00
Tasks: Deep Learning, Probabilistic Deep Learning, Question Answering, Visual Question Answering, Visual Question Answering (VQA)

Understanding and explaining deep learning models is an imperative task. Towards this, we propose a method that obtains gradient-based certainty estimates that also provide visual attention maps. Particularly, we solve for visual question answering task. We incorporate modern probabilistic deep learning methods that we further improve by using the gradients for these estimates. These have two-fold benefits: a) improvement in obtaining the certainty estimates that correlate better with misclassified samples and b) improved attention maps that provide state-of-the-art results in terms of correlation with human attention regions. The improved attention maps result in consistent improvement for various methods for visual question answering. Therefore, the proposed technique can be thought of as a recipe for obtaining improved certainty estimates and explanations for deep learning models. We provide detailed empirical analysis for the visual question answering task on all standard benchmarks and comparison with state of the art methods.
Word Embeddings Inherently Recover the Conceptual Organization of the Human Mind
Authors: Victor Swift
Date: 2020-02-06 00:00:00
Tasks: BIG-bench Machine Learning, Word Embeddings

Machine learning is a means to uncover deep patterns from rich sources of data. Here, we find that machine learning can recover the conceptual organization of the human mind when applied to the natural language use of millions of people. Utilizing text from billions of webpages, we recover most of the concepts contained in English, Dutch, and Japanese, as represented in large scale Word Association networks. Our results justify machine learning as a means to probe the human mind, at a depth and scale that has been unattainable using self-report and observational methods. Beyond direct psychological applications, our methods may prove useful for projects concerned with defining, assessing, relating, or uncovering concepts in any scientific field.
Deep RL-based Trajectory Planning for AoI Minimization in UAV-assisted IoT
Authors: Conghao Zhou, Hongli He, Peng Yang, Feng Lyu, WenWu, Nan Cheng, and Xuemin (Sherman) Shen
Date: 2019-12-09 00:00:00
Proceeding: 2019 11th International Conference on Wireless Communications and Signal Processing (WCSP) 2019 12
Tasks: Decision Making, Deep Reinforcement Learning, Reinforcement Learning, Reinforcement Learning (RL), Trajectory Planning

Due to the flexibility and low deployment cost,
unmanned aerial vehicles (UAVs) have been widely used to assist
cellular networks in providing extended coverage for Internet
of Things (IoT) networks. Existing throughput or delay-based
UAV trajectory planning methods cannot meet the requirement
of collecting fresh information from IoT devices. In this paper,
by taking age-of-information (AoI) as a measure of information
freshness, we investigate AoI-based UAV trajectory planning for
fresh data collection. To model the complicated association and
interaction pattern between UAV and IoT devices, the UAV
trajectory planning problem is formulated as a Markov decision
process (MDP) to capture the dynamics of UAV locations. As net-
work topology and traffic generation pattern are unknown ahead,
we propose an AoI-based trajectory planning (A-TP) algorithm
using deep reinforcement learning (RL) technique. To accelerate
the learning process during online decision making, the off-line
pre-training of deep neural networks is performed. Extensive
simulation results demonstrate that the proposed algorithm can
significantly reduce the AoI of collected IoT data, as compared
to other benchmark approaches.
A Fast Projected Fixed-Point Algorithm for Large Graph Matching
Authors: Yao Lu, Kai-Zhu Huang, Cheng-Lin Liu
Date: 2012-07-03 00:00:00
Tasks: Graph Matching

We propose a fast approximate algorithm for large graph matching. A new projected fixed-point method is defined and a new doubly stochastic projection is adopted to derive the algorithm. Previous graph matching algorithms suffer from high computational complexity and therefore do not have good scalability with respect to graph size. For matching two weighted graphs of $n$ nodes, our algorithm has time complexity only $O(n^3)$ per iteration and space complexity $O(n^2)$. In addition to its scalability, our algorithm is easy to implement, robust, and able to match undirected weighted attributed graphs of different sizes. While the convergence rate of previous iterative graph matching algorithms is unknown, our algorithm is theoretically guaranteed to converge at a linear rate. Extensive experiments on large synthetic and real graphs (more than 1,000 nodes) were conducted to evaluate the performance of various algorithms. Results show that in most cases our proposed algorithm achieves better performance than previous state-of-the-art algorithms in terms of both speed and accuracy in large graph matching. In particular, with high accuracy, our algorithm takes only a few seconds (in a PC) to match two graphs of 1,000 nodes.
MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
Authors: Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, Ming Zhou
Date: 2020-02-25 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: Sentence Retrieval, Zero-shot Text Search

Pre-trained language models (e.g., BERT (Devlin et al., 2018) and its variants) have achieved remarkable success in varieties of NLP tasks. However, these models usually consist of hundreds of millions of parameters which brings challenges for fine-tuning and online serving in real-life applications due to latency and capacity constraints. In this work, we present a simple and effective approach to compress large Transformer (Vaswani et al., 2017) based pre-trained models, termed as deep self-attention distillation. The small model (student) is trained by deeply mimicking the self-attention module, which plays a vital role in Transformer networks, of the large model (teacher). Specifically, we propose distilling the self-attention module of the last Transformer layer of the teacher, which is effective and flexible for the student. Furthermore, we introduce the scaled dot-product between values in the self-attention module as the new deep self-attention knowledge, in addition to the attention distributions (i.e., the scaled dot-product of queries and keys) that have been used in existing works. Moreover, we show that introducing a teacher assistant (Mirzadeh et al., 2019) also helps the distillation of large pre-trained Transformer models. Experimental results demonstrate that our monolingual model outperforms state-of-the-art baselines in different parameter size of student models. In particular, it retains more than 99% accuracy on SQuAD 2.0 and several GLUE benchmark tasks using 50% of the Transformer parameters and computations of the teacher model. We also obtain competitive results in applying deep self-attention distillation to multilingual pre-trained models.
On Feature Normalization and Data Augmentation
Authors: Boyi Li, Felix Wu, Ser-Nam Lim, Serge Belongie, Kilian Q. Weinberger
Date: 2020-02-25 00:00:00
Proceeding: CVPR 2021 1
Tasks: Data Augmentation, Domain Generalization, Image Classification, Image Generation

The moments (a.k.a., mean and standard deviation) of latent features are often removed as noise when training image recognition models, to increase stability and reduce training time. However, in the field of image generation, the moments play a much more central role. Studies have shown that the moments extracted from instance normalization and positional normalization can roughly capture style and shape information of an image. Instead of being discarded, these moments are instrumental to the generation process. In this paper we propose Moment Exchange, an implicit data augmentation method that encourages the model to utilize the moment information also for recognition models. Specifically, we replace the moments of the learned features of one training image by those of another, and also interpolate the target labels -- forcing the model to extract training signal from the moments in addition to the normalized features. As our approach is fast, operates entirely in feature space, and mixes different signals than prior methods, one can effectively combine it with existing augmentation approaches. We demonstrate its efficacy across several recognition benchmark data sets where it improves the generalization capability of highly competitive baseline networks with remarkable consistency.
A General Method for Robust Learning from Batches
Authors: Ayush Jain, Alon Orlitsky
Date: 2020-02-25 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: Classification, General Classification

In many applications, data is collected in batches, some of which are corrupt or even adversarial. Recent work derived optimal robust algorithms for estimating discrete distributions in this setting. We consider a general framework of robust learning from batches, and determine the limits of both classification and distribution estimation over arbitrary, including continuous, domains. Building on these results, we derive the first robust agnostic computationally-efficient learning algorithms for piecewise-interval classification, and for piecewise-polynomial, monotone, log-concave, and gaussian-mixture distribution estimation.
Toward fast and accurate human pose estimation via soft-gated skip connections
Authors: Adrian Bulat, Jean Kossaifi, Georgios Tzimiropoulos, Maja Pantic
Date: 2020-02-25 00:00:00
Tasks: Pose Estimation

This paper is on highly accurate and highly efficient human pose estimation. Recent works based on Fully Convolutional Networks (FCNs) have demonstrated excellent results for this difficult problem. While residual connections within FCNs have proved to be quintessential for achieving high accuracy, we re-analyze this design choice in the context of improving both the accuracy and the efficiency over the state-of-the-art. In particular, we make the following contributions: (a) We propose gated skip connections with per-channel learnable parameters to control the data flow for each channel within the module within the macro-module. (b) We introduce a hybrid network that combines the HourGlass and U-Net architectures which minimizes the number of identity connections within the network and increases the performance for the same parameter budget. Our model achieves state-of-the-art results on the MPII and LSP datasets. In addition, with a reduction of 3x in model size and complexity, we show no decrease in performance when compared to the original HourGlass network.