pwc-archive/papers-with-abstracts huggingface dataset:
Daedalus: Breaking Non-Maximum Suppression in Object Detection via Adversarial Examples
Authors: Derui Wang, Chaoran Li, Sheng Wen, Qing-Long Han, Surya Nepal, Xiangyu Zhang, Yang Xiang
Date: 2019-02-06 00:00:00
Tasks: Autonomous Vehicles, object-detection, Object Detection

This paper demonstrates that Non-Maximum Suppression (NMS), which is commonly used in Object Detection (OD) tasks to filter redundant detection results, is no longer secure. Considering that NMS has been an integral part of OD systems, thwarting the functionality of NMS can result in unexpected or even lethal consequences for such systems. In this paper, an adversarial example attack which triggers malfunctioning of NMS in end-to-end OD models is proposed. The attack, namely \texttt{Daedalus}, compresses the dimensions of detection boxes to evade NMS. As a result, the final detection output contains extremely dense false positives. This can be fatal for many OD applications such as autonomous vehicles and surveillance systems. The attack can be generalised to different end-to-end OD models, such that the attack cripples various OD applications. Furthermore, a way to craft robust adversarial examples is developed by using an ensemble of popular detection models as the substitutes. Considering the pervasive nature of model reusing in real-world OD scenarios, Daedalus examples crafted based on an \textit{ensemble of substitutes} can launch attacks without knowing the parameters of the victim models. Experimental results demonstrate that the attack effectively stops NMS from filtering redundant bounding boxes. As the evaluation results suggest, Daedalus increases the false positive rate in detection results to $99.9\%$ and reduces the mean average precision scores to $0$, while maintaining a low cost of distortion on the original inputs. It is also demonstrated that the attack can be practically launched against real-world OD systems via printed posters.
On ADMM in Deep Learning: Convergence and Saturation-Avoidance
Authors: Jinshan Zeng, Shao-Bo Lin, Yuan YAO, Ding-Xuan Zhou
Date: 2019-02-06 00:00:00
Tasks: Deep Learning

In this paper, we develop an alternating direction method of multipliers (ADMM) for deep neural networks training with sigmoid-type activation functions (called \textit{sigmoid-ADMM pair}), mainly motivated by the gradient-free nature of ADMM in avoiding the saturation of sigmoid-type activations and the advantages of deep neural networks with sigmoid-type activations (called deep sigmoid nets) over their rectified linear unit (ReLU) counterparts (called deep ReLU nets) in terms of approximation. In particular, we prove that the approximation capability of deep sigmoid nets is not worse than that of deep ReLU nets by showing that ReLU activation function can be well approximated by deep sigmoid nets with two hidden layers and finitely many free parameters but not vice-verse. We also establish the global convergence of the proposed ADMM for the nonlinearly constrained formulation of the deep sigmoid nets training from arbitrary initial points to a Karush-Kuhn-Tucker (KKT) point at a rate of order ${\cal O}(1/k)$. Besides sigmoid activation, such a convergence theorem holds for a general class of smooth activations. Compared with the widely used stochastic gradient descent (SGD) algorithm for the deep ReLU nets training (called ReLU-SGD pair), the proposed sigmoid-ADMM pair is practically stable with respect to the algorithmic hyperparameters including the learning rate, initial schemes and the pro-processing of the input data. Moreover, we find that to approximate and learn simple but important functions the proposed sigmoid-ADMM pair numerically outperforms the ReLU-SGD pair.
Content-based image retrieval system with most relevant features among wavelet and color features
Authors: Abdolreza Rashno, Elyas Rashno
Date: 2019-02-06 00:00:00
Tasks: Content-Based Image Retrieval, feature selection, Image Retrieval, Management, Retrieval

Content-based image retrieval (CBIR) has become one of the most important
research directions in the domain of digital data management. In this paper, a
new feature extraction schema including the norm of low frequency components in
wavelet transformation and color features in RGB and HSV domains are proposed
as representative feature vector for images in database followed by appropriate
similarity measure for each feature type. In CBIR systems, retrieving results
are so sensitive to image features. We address this problem with selection of
most relevant features among complete feature set by ant colony optimization
(ACO)-based feature selection which minimize the number of features as well as
maximize F-measure in CBIR system. To evaluate the performance of our proposed
CBIR system, it has been compared with three older proposed systems. Results
show that the precision and recall of our proposed system are higher than older
ones for the majority of image categories in Corel database.
Un modèle Bayésien de co-clustering de données mixtes
Authors: Aichetou Bouchareb, Marc Boullé, Fabrice Rossi, Fabrice Clérot
Date: 2019-02-06 00:00:00
Tasks: Clustering, Model Selection

We propose a MAP Bayesian approach to perform and evaluate a co-clustering of
mixed-type data tables. The proposed model infers an optimal segmentation of
all variables then performs a co-clustering by minimizing a Bayesian model
selection cost function. One advantage of this approach is that it is user
parameter-free. Another main advantage is the proposed criterion which gives an
exact measure of the model quality, measured by probability of fitting it to
the data. Continuous optimization of this criterion ensures finding better and
better models while avoiding data over-fitting. The experiments conducted on
real data show the interest of this co-clustering approach in exploratory data
analysis of large data sets.
Fooling Neural Network Interpretations via Adversarial Model Manipulation
Authors: Juyeon Heo, Sunghwan Joo, Taesup Moon
Date: 2019-02-06 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: model, Network Interpretation

We ask whether the neural network interpretation methods can be fooled via adversarial model manipulation, which is defined as a model fine-tuning step that aims to radically alter the explanations without hurting the accuracy of the original models, e.g., VGG19, ResNet50, and DenseNet121. By incorporating the interpretation results directly in the penalty term of the objective function for fine-tuning, we show that the state-of-the-art saliency map based interpreters, e.g., LRP, Grad-CAM, and SimpleGrad, can be easily fooled with our model manipulation. We propose two types of fooling, Passive and Active, and demonstrate such foolings generalize well to the entire validation set as well as transfer to other interpretation methods. Our results are validated by both visually showing the fooled explanations and reporting quantitative metrics that measure the deviations from the original explanations. We claim that the stability of neural network interpretation method with respect to our adversarial model manipulation is an important criterion to check for developing robust and reliable neural network interpretation method.
Bidirectional Inference Networks: A Class of Deep Bayesian Networks for Health Profiling
Authors: Hao Wang, Chengzhi Mao, Hao He, Ming-Min Zhao, Tommi S. Jaakkola, Dina Katabi
Date: 2019-02-06 00:00:00
Tasks: Computational Efficiency, EEG, Electroencephalogram (EEG), Sleep Quality, Structured Prediction

We consider the problem of inferring the values of an arbitrary set of
variables (e.g., risk of diseases) given other observed variables (e.g.,
symptoms and diagnosed diseases) and high-dimensional signals (e.g., MRI images
or EEG). This is a common problem in healthcare since variables of interest
often differ for different patients. Existing methods including Bayesian
networks and structured prediction either do not incorporate high-dimensional
signals or fail to model conditional dependencies among variables. To address
these issues, we propose bidirectional inference networks (BIN), which stich
together multiple probabilistic neural networks, each modeling a conditional
dependency. Predictions are then made via iteratively updating variables using
backpropagation (BP) to maximize corresponding posterior probability.
Furthermore, we extend BIN to composite BIN (CBIN), which involves the
iterative prediction process in the training stage and improves both accuracy
and computational efficiency by adaptively smoothing the optimization
landscape. Experiments on synthetic and real-world datasets (a sleep study and
a dermatology dataset) show that CBIN is a single model that can achieve
state-of-the-art performance and obtain better accuracy in most inference tasks
than multiple models each specifically trained for a different task.
Robust Matrix Completion State Estimation in Distribution Systems
Authors: Bo Liu, Hongyu Wu, Yingchen Zhang, Rui Yang, Andrey Bernstein
Date: 2019-02-06 00:00:00
Tasks: Matrix Completion, State Estimation

Due to the insufficient measurements in the distribution system state estimation (DSSE), full observability and redundant measurements are difficult to achieve without using the pseudo measurements. The matrix completion state estimation (MCSE) combines the matrix completion and power system model to estimate voltage by exploring the low-rank characteristics of the matrix. This paper proposes a robust matrix completion state estimation (RMCSE) to estimate the voltage in a distribution system under a low-observability condition. Tradition state estimation weighted least squares (WLS) method requires full observability to calculate the states and needs redundant measurements to proceed a bad data detection. The proposed method improves the robustness of the MCSE to bad data by minimizing the rank of the matrix and measurements residual with different weights. It can estimate the system state in a low-observability system and has robust estimates without the bad data detection process in the face of multiple bad data. The method is numerically evaluated on the IEEE 33-node radial distribution system. The estimation performance and robustness of RMCSE are compared with the WLS with the largest normalized residual bad data identification (WLS-LNR), and the MCSE.
Neural Fictitious Self-Play on ELF Mini-RTS
Authors: Keigo Kawamura, Yoshimasa Tsuruoka
Date: 2019-02-06 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Despite the notable successes in video games such as Atari 2600, current AI
is yet to defeat human champions in the domain of real-time strategy (RTS)
games. One of the reasons is that an RTS game is a multi-agent game, in which
single-agent reinforcement learning methods cannot simply be applied because
the environment is not a stationary Markov Decision Process. In this paper, we
present a first step toward finding a game-theoretic solution to RTS games by
applying Neural Fictitious Self-Play (NFSP), a game-theoretic approach for
finding Nash equilibria, to Mini-RTS, a small but nontrivial RTS game provided
on the ELF platform. More specifically, we show that NFSP can be effectively
combined with policy gradient reinforcement learning and be applied to
Mini-RTS. Experimental results also show that the scalability of NFSP can be
substantially improved by pretraining the models with simple self-play using
policy gradients, which by itself gives a strong strategy despite its lack of
theoretical guarantee of convergence.
Testing Markov Chains without Hitting
Authors: Yeshwanth Cherapanamjeri, Peter L. Bartlett
Date: 2019-02-06 00:00:00

We study the problem of identity testing of markov chains. In this setting,
we are given access to a single trajectory from a markov chain with unknown
transition matrix $Q$ and the goal is to determine whether $Q = P$ for some
known matrix $P$ or $\text{Dist}(P, Q) \geq \epsilon$ where $\text{Dist}$ is
suitably defined. In recent work by Daskalakis, Dikkala and Gravin, 2018, it
was shown that it is possible to distinguish between the two cases provided the
length of the observed trajectory is at least super-linear in the hitting time
of $P$ which may be arbitrarily large.
  In this paper, we propose an algorithm that avoids this dependence on hitting
time thus enabling efficient testing of markov chains even in cases where it is
infeasible to observe every state in the chain. Our algorithm is based on
combining classical ideas from approximation algorithms with techniques for the
spectral analysis of markov chains.
Fast Mean Estimation with Sub-Gaussian Rates
Authors: Yeshwanth Cherapanamjeri, Nicolas Flammarion, Peter L. Bartlett
Date: 2019-02-06 00:00:00

We propose an estimator for the mean of a random vector in $\mathbb{R}^d$
that can be computed in time $O(n^4+n^2d)$ for $n$ i.i.d.~samples and that has
error bounds matching the sub-Gaussian case. The only assumptions we make about
the data distribution are that it has finite mean and covariance; in
particular, we make no assumptions about higher-order moments. Like the
polynomial time estimator introduced by Hopkins, 2018, which is based on the
sum-of-squares hierarchy, our estimator achieves optimal statistical efficiency
in this challenging setting, but it has a significantly faster runtime and a
simpler analysis.
Are All Layers Created Equal?
Authors: Chiyuan Zhang, Samy Bengio, Yoram Singer
Date: 2019-02-06 00:00:00
Proceeding: ICML Workshop Deep_Phenomen 2019 6
Tasks: All

Understanding deep neural networks is a major research objective with notable experimental and theoretical attention in recent years. The practical success of excessively large networks underscores the need for better theoretical analyses and justifications. In this paper we focus on layer-wise functional structure and behavior in overparameterized deep models. To do so, we study empirically the layers' robustness to post-training re-initialization and re-randomization of the parameters. We provide experimental results which give evidence for the heterogeneity of layers. Morally, layers of large deep neural networks can be categorized as either "robust" or "critical". Resetting the robust layers to their initial values does not result in adverse decline in performance. In many cases, robust layers hardly change throughout training. In contrast, re-initializing critical layers vastly degrades the performance of the network with test error essentially dropping to random guesses. Our study provides further evidence that mere parameter counting or norm calculations are too coarse in studying generalization of deep models, and "flatness" and robustness analysis of trained models need to be examined while taking into account the respective network architectures.
An Automated Spectral Clustering for Multi-scale Data
Authors: Milad Afzalan, Farrokh Jazizadeh
Date: 2019-02-06 00:00:00
Tasks: Clustering

Spectral clustering algorithms typically require a priori selection of input
parameters such as the number of clusters, a scaling parameter for the affinity
measure, or ranges of these values for parameter tuning. Despite efforts for
automating the process of spectral clustering, the task of grouping data in
multi-scale and higher dimensional spaces is yet to be explored. This study
presents a spectral clustering heuristic algorithm that obviates the need for
an input by estimating the parameters from the data itself. Specifically, it
introduces the heuristic of iterative eigengap search with (1) global scaling
and (2) local scaling. These approaches estimate the scaling parameter and
implement iterative eigengap quantification along a search tree to reveal
dissimilarities at different scales of a feature space and identify clusters.
The performance of these approaches has been tested on various real-world
datasets of power variation with multi-scale nature and gene expression. Our
findings show that iterative eigengap search with a PCA-based global scaling
scheme can discover different patterns with an accuracy of higher than 90% in
most cases without asking for a priori input information.
CodedReduce: A Fast and Robust Framework for Gradient Aggregation in Distributed Learning
Authors: Amirhossein Reisizadeh, Saurav Prakash, Ramtin Pedarsani, Amir Salman Avestimehr
Date: 2019-02-06 00:00:00

We focus on the commonly used synchronous Gradient Descent paradigm for large-scale distributed learning, for which there has been a growing interest to develop efficient and robust gradient aggregation strategies that overcome two key system bottlenecks: communication bandwidth and stragglers' delays. In particular, Ring-AllReduce (RAR) design has been proposed to avoid bandwidth bottleneck at any particular node by allowing each worker to only communicate with its neighbors that are arranged in a logical ring. On the other hand, Gradient Coding (GC) has been recently proposed to mitigate stragglers in a master-worker topology by allowing carefully designed redundant allocation of the data set to the workers. We propose a joint communication topology design and data set allocation strategy, named CodedReduce (CR), that combines the best of both RAR and GC. That is, it parallelizes the communications over a tree topology leading to efficient bandwidth utilization, and carefully designs a redundant data set allocation and coding strategy at the nodes to make the proposed gradient aggregation scheme robust to stragglers. In particular, we quantify the communication parallelization gain and resiliency of the proposed CR scheme, and prove its optimality when the communication topology is a regular tree. Moreover, we characterize the expected run-time of CR and show order-wise speedups compared to the benchmark schemes. Finally, we empirically evaluate the performance of our proposed CR design over Amazon EC2 and demonstrate that it achieves speedups of up to 27.2x and 7.0x, respectively over the benchmarks GC and RAR.
Semi-supervised learning via Feedforward-Designed Convolutional Neural Networks
Authors: Yueru Chen, Yijing Yang, Min Zhang, C. -C. Jay Kuo
Date: 2019-02-06 00:00:00
Tasks: Benchmarking, General Classification, image-classification, Image Classification

A semi-supervised learning framework using the feedforward-designed
convolutional neural networks (FF-CNNs) is proposed for image classification in
this work. One unique property of FF-CNNs is that no backpropagation is used in
model parameters determination. Since unlabeled data may not always enhance
semi-supervised learning, we define an effective quality score and use it to
select a subset of unlabeled data in the training process. We conduct
experiments on the MNIST, SVHN, and CIFAR-10 datasets, and show that the
proposed semi-supervised FF-CNN solution outperforms the CNN trained by
backpropagation (BP-CNN) when the amount of labeled data is reduced.
Furthermore, we develop an ensemble system that combines the output decision
vectors of different semi-supervised FF-CNNs to boost classification accuracy.
The ensemble systems can achieve further performance gains on all three
benchmarking datasets.
Technical Considerations for Semantic Segmentation in MRI using Convolutional Neural Networks
Authors: Arjun D. Desai, Garry E. Gold, Brian A. Hargreaves, Akshay S. Chaudhari
Date: 2019-02-05 00:00:00
Tasks: Deep Learning, Segmentation, Semantic Segmentation

High-fidelity semantic segmentation of magnetic resonance volumes is critical
for estimating tissue morphometry and relaxation parameters in both clinical
and research applications. While manual segmentation is accepted as the
gold-standard, recent advances in deep learning and convolutional neural
networks (CNNs) have shown promise for efficient automatic segmentation of soft
tissues. However, due to the stochastic nature of deep learning and the
multitude of hyperparameters in training networks, predicting network behavior
is challenging. In this paper, we quantify the impact of three factors
associated with CNN segmentation performance: network architecture, training
loss functions, and training data characteristics. We evaluate the impact of
these variations on the segmentation of femoral cartilage and propose potential
modifications to CNN architectures and training protocols to train these models
with confidence.
Polyphonic Music Composition with LSTM Neural Networks and Reinforcement Learning
Authors: Harish Kumar, Balaraman Ravindran
Date: 2019-02-05 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In the domain of algorithmic music composition, machine learning-driven
systems eliminate the need for carefully hand-crafting rules for composition.
In particular, the capability of recurrent neural networks to learn complex
temporal patterns lends itself well to the musical domain. Promising results
have been observed across a number of recent attempts at music composition
using deep RNNs. These approaches generally aim at first training neural
networks to reproduce subsequences drawn from existing songs. Subsequently,
they are used to compose music either at the audio sample-level or at the
note-level. We designed a representation that divides polyphonic music into a
small number of monophonic streams. This representation greatly reduces the
complexity of the problem and eliminates an exponential number of probably poor
compositions. On top of our LSTM neural network that learnt musical sequences
in this representation, we built an RL agent that learnt to find combinations
of songs whose joint dominance produced pleasant compositions. We present
Amadeus, an algorithmic music composition system that composes music that
consists of intricate melodies, basic chords, and even occasional contrapuntal
sequences.
Exchangeable Generative Models with Flow Scans
Authors: Christopher Bender, Kevin O'Connor, Yang Li, Juan Jose Garcia, Manzil Zaheer, Junier Oliva
Date: 2019-02-05 00:00:00
Tasks: Density Estimation

In this work, we develop a new approach to generative density estimation for exchangeable, non-i.i.d. data. The proposed framework, FlowScan, combines invertible flow transformations with a sorted scan to flexibly model the data while preserving exchangeability. Unlike most existing methods, FlowScan exploits the intradependencies within sets to learn both global and local structure. FlowScan represents the first approach that is able to apply sequential methods to exchangeable density estimation without resorting to averaging over all possible permutations. We achieve new state-of-the-art performance on point cloud and image set modeling.
A General Theory for Structured Prediction with Smooth Convex Surrogates
Authors: Alex Nowak-Vila, Francis Bach, Alessandro Rudi
Date: 2019-02-05 00:00:00
Tasks: General Classification, Graph Matching, regression, Structured Prediction

In this work we provide a theoretical framework for structured prediction
that generalizes the existing theory of surrogate methods for binary and
multiclass classification based on estimating conditional probabilities with
smooth convex surrogates (e.g. logistic regression). The theory relies on a
natural characterization of structural properties of the task loss and allows
to derive statistical guarantees for many widely used methods in the context of
multilabeling, ranking, ordinal regression and graph matching. In particular,
we characterize the smooth convex surrogates compatible with a given task loss
in terms of a suitable Bregman divergence composed with a link function. This
allows to derive tight bounds for the calibration function and to obtain novel
results on existing surrogate frameworks for structured prediction such as
conditional random fields and quadratic surrogates.