pwc-archive/papers-with-abstracts huggingface dataset:
Large Scale Clustering with Variational EM for Gaussian Mixture Models
Authors: Florian Hirschberger, Dennis Forster, Jörg Lücke
Date: 2018-10-01 00:00:00
Tasks: Benchmarking, Clustering, Quantization

This paper represents a preliminary (pre-reviewing) version of a sublinear variational algorithm for isotropic Gaussian mixture models (GMMs). Further developments of the algorithm for GMMs with diagonal covariance matrices (instead of isotropic clusters) and their corresponding benchmarking results have been published by TPAMI (doi:10.1109/TPAMI.2021.3133763) in the paper "A Variational EM Acceleration for Efficient Clustering at Very Large Scales". We kindly refer the reader to the TPAMI paper instead of this much earlier arXiv version (the TPAMI paper is also open access). Publicly available source code accompanies the paper (see https://github.com/variational-sublinear-clustering). Please note that the TPAMI paper does not contain the benchmark on the 80 Million Tiny Images dataset anymore because we followed the call of the dataset creators to discontinue the use of that dataset. The aim of the project (which resulted in this arXiv version and the later TPAMI paper) is the exploration of the current efficiency and large-scale limits in fitting a parametric model for clustering to data distributions. To reduce computational complexity, we used a clustering objective based on truncated variational EM (which reduces complexity for many clusters) in combination with coreset objectives (which reduce complexity for many data points). We used efficient coreset construction and efficient seeding to translate the theoretical sublinear complexity gains into an efficient algorithm. In applications to standard large-scale benchmarks for clustering, we then observed substantial wall-clock speedups compared to already highly efficient clustering approaches. To demonstrate that the observed efficiency enables applications previously considered unfeasible, we clustered the entire and unscaled 80 Million Tiny Images dataset into up to 32,000 clusters.
Graph Diffusion-Embedding Networks
Authors: Bo Jiang, Doudou Lin, Jin Tang
Date: 2018-10-01 00:00:00

We present a novel graph diffusion-embedding networks (GDEN) for graph
structured data. GDEN is motivated by our closed-form formulation on
regularized feature diffusion on graph. GDEN integrates both regularized
feature diffusion and low-dimensional embedding simultaneously in a unified
network model. Moreover, based on GDEN, we can naturally deal with structured
data with multiple graph structures. Experiments on semi-supervised learning
tasks on several benchmark datasets demonstrate the better performance of the
proposed GDEN when comparing with the traditional GCN models.
On Theory for BART
Authors: Veronika Rockova, Enakshi Saha
Date: 2018-10-01 00:00:00
Tasks: Ensemble Learning

Ensemble learning is a statistical paradigm built on the premise that many
weak learners can perform exceptionally well when deployed collectively. The
BART method of Chipman et al. (2010) is a prominent example of Bayesian
ensemble learning, where each learner is a tree. Due to its impressive
performance, BART has received a lot of attention from practitioners. Despite
its wide popularity, however, theoretical studies of BART have begun emerging
only very recently. Laying the foundations for the theoretical analysis of
Bayesian forests, Rockova and van der Pas (2017) showed optimal posterior
concentration under conditionally uniform tree priors. These priors deviate
from the actual priors implemented in BART. Here, we study the exact BART prior
and propose a simple modification so that it also enjoys optimality properties.
To this end, we dive into branching process theory. We obtain tail bounds for
the distribution of total progeny under heterogeneous Galton-Watson (GW)
processes exploiting their connection to random walks. We conclude with a
result stating the optimal rate of posterior convergence for BART.
The Profiling Machine: Active Generalization over Knowledge
Authors: Filip Ilievski, Eduard Hovy, Qizhe Xie, Piek Vossen
Date: 2018-10-01 00:00:00
Tasks: Anomaly Detection, Management

The human mind is a powerful multifunctional knowledge storage and management
system that performs generalization, type inference, anomaly detection,
stereotyping, and other tasks. A dynamic KR system that appropriately profiles
over sparse inputs to provide complete expectations for unknown facets can help
with all these tasks. In this paper, we introduce the task of profiling,
inspired by theories and findings in social psychology about the potential of
profiles for reasoning and information processing. We describe two generic
state-of-the-art neural architectures that can be easily instantiated as
profiling machines to generate expectations and applied to any kind of
knowledge to fill gaps. We evaluate these methods against Wikidata and crowd
expectations, and compare the results to gain insight in the nature of
knowledge captured by various profiling methods. We make all code and data
available to facilitate future research.
Geometric Constellation Shaping for Fiber Optic Communication Systems via End-to-end Learning
Authors: Rasmus T. Jones, Tobias A. Eriksson, Metodi P. Yankov, Benjamin J. Puttnam, Georg Rademacher, Ruben S. Luis, Darko Zibar
Date: 2018-10-01 00:00:00
Tasks: BIG-bench Machine Learning

In this paper, an unsupervised machine learning method for geometric
constellation shaping is investigated. By embedding a differentiable fiber
channel model within two neural networks, the learning algorithm is optimizing
for a geometric constellation shape. The learned constellations yield improved
performance to state-of-the-art geometrically shaped constellations, and
include an implicit trade-off between amplification noise and nonlinear
effects. Further, the method allows joint optimization of system parameters,
such as the optimal launch power, simultaneously with the constellation shape.
An experimental demonstration validates the findings. Improved performances are
reported, up to 0.13 bit/4D in simulation and experimentally up to 0.12 bit/4D.
Riemannian Adaptive Optimization Methods
Authors: Gary Bécigneul, Octavian-Eugen Ganea
Date: 2018-10-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Riemannian optimization, Stochastic Optimization

Several first order stochastic optimization methods commonly used in the
Euclidean domain such as stochastic gradient descent (SGD), accelerated
gradient descent or variance reduced methods have already been adapted to
certain Riemannian settings. However, some of the most popular of these
optimization tools - namely Adam , Adagrad and the more recent Amsgrad - remain
to be generalized to Riemannian manifolds. We discuss the difficulty of
generalizing such adaptive schemes to the most agnostic Riemannian setting, and
then provide algorithms and convergence proofs for geodesically convex
objectives in the particular case of a product of Riemannian manifolds, in
which adaptivity is implemented across manifolds in the cartesian product. Our
generalization is tight in the sense that choosing the Euclidean space as
Riemannian manifold yields the same algorithms and regret bounds as those that
were already known for the standard algorithms. Experimentally, we show faster
convergence and to a lower train loss value for Riemannian adaptive methods
over their corresponding baselines on the realistic task of embedding the
WordNet taxonomy in the Poincare ball.
Bayesian Prediction of Future Street Scenes using Synthetic Likelihoods
Authors: Apratim Bhattacharyya, Mario Fritz, Bernt Schiele
Date: 2018-10-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Bayesian Inference, Diversity, Precipitation Forecasting

For autonomous agents to successfully operate in the real world, the ability
to anticipate future scene states is a key competence. In real-world scenarios,
future states become increasingly uncertain and multi-modal, particularly on
long time horizons. Dropout based Bayesian inference provides a computationally
tractable, theoretically well grounded approach to learn likely
hypotheses/models to deal with uncertain futures and make predictions that
correspond well to observations -- are well calibrated. However, it turns out
that such approaches fall short to capture complex real-world scenes, even
falling behind in accuracy when compared to the plain deterministic approaches.
This is because the used log-likelihood estimate discourages diversity. In this
work, we propose a novel Bayesian formulation for anticipating future scene
states which leverages synthetic likelihoods that encourage the learning of
diverse models to accurately capture the multi-modal nature of future scene
states. We show that our approach achieves accurate state-of-the-art
predictions and calibrated probabilities through extensive experiments for
scene anticipation on Cityscapes dataset. Moreover, we show that our approach
generalizes across diverse tasks such as digit generation and precipitation
forecasting.
Improving the Generalization of Adversarial Training with Domain Adaptation
Authors: Chuanbiao Song, Kun He, Li-Wei Wang, John E. Hopcroft
Date: 2018-10-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Adversarial Attack, Domain Adaptation

By injecting adversarial examples into training data, adversarial training is
promising for improving the robustness of deep learning models. However, most
existing adversarial training approaches are based on a specific type of
adversarial attack. It may not provide sufficiently representative samples from
the adversarial domain, leading to a weak generalization ability on adversarial
examples from other attacks. Moreover, during the adversarial training,
adversarial perturbations on inputs are usually crafted by fast single-step
adversaries so as to scale to large datasets. This work is mainly focused on
the adversarial training yet efficient FGSM adversary. In this scenario, it is
difficult to train a model with great generalization due to the lack of
representative adversarial samples, aka the samples are unable to accurately
reflect the adversarial domain. To alleviate this problem, we propose a novel
Adversarial Training with Domain Adaptation (ATDA) method. Our intuition is to
regard the adversarial training on FGSM adversary as a domain adaption task
with limited number of target domain samples. The main idea is to learn a
representation that is semantically meaningful and domain invariant on the
clean domain as well as the adversarial domain. Empirical evaluations on
Fashion-MNIST, SVHN, CIFAR-10 and CIFAR-100 demonstrate that ATDA can greatly
improve the generalization of adversarial training and the smoothness of the
learned models, and outperforms state-of-the-art methods on standard benchmark
datasets. To show the transfer ability of our method, we also extend ATDA to
the adversarial training on iterative attacks such as PGD-Adversial Training
(PAT) and the defense performance is improved considerably.
Risk-Averse Stochastic Convex Bandit
Authors: Adrian Rivera Cardoso, Huan Xu
Date: 2018-10-01 00:00:00

Motivated by applications in clinical trials and finance, we study the
problem of online convex optimization (with bandit feedback) where the decision
maker is risk-averse. We provide two algorithms to solve this problem. The
first one is a descent-type algorithm which is easy to implement. The second
algorithm, which combines the ellipsoid method and a center point device,
achieves (almost) optimal regret bounds with respect to the number of rounds.
To the best of our knowledge this is the first attempt to address risk-aversion
in the online convex bandit problem.
Benchmark Analysis of Representative Deep Neural Network Architectures
Authors: Simone Bianco, Remi Cadene, Luigi Celona, Paolo Napoletano
Date: 2018-10-01 00:00:00

This work presents an in-depth analysis of the majority of the deep neural
networks (DNNs) proposed in the state of the art for image recognition. For
each DNN multiple performance indices are observed, such as recognition
accuracy, model complexity, computational complexity, memory usage, and
inference time. The behavior of such performance indices and some combinations
of them are analyzed and discussed. To measure the indices we experiment the
use of DNNs on two different computer architectures, a workstation equipped
with a NVIDIA Titan X Pascal and an embedded system based on a NVIDIA Jetson
TX1 board. This experimentation allows a direct comparison between DNNs running
on machines with very different computational capacity. This study is useful
for researchers to have a complete view of what solutions have been explored so
far and in which research directions are worth exploring in the future; and for
practitioners to select the DNN architecture(s) that better fit the resource
constraints of practical deployments and applications. To complete this work,
all the DNNs, as well as the software used for the analysis, are available
online.
Classification Using Link Prediction
Authors: Seyed Amin Fadaee, Maryam Amir Haeri
Date: 2018-10-01 00:00:00
Tasks: Classification, General Classification, Link Prediction, Prediction

Link prediction in a graph is the problem of detecting the missing links that
would be formed in the near future. Using a graph representation of the data,
we can convert the problem of classification to the problem of link prediction
which aims at finding the missing links between the unlabeled data (unlabeled
nodes) and their classes. To our knowledge, despite the fact that numerous
algorithms use the graph representation of the data for classification, none
are using link prediction as the heart of their classifying procedure. In this
work, we propose a novel algorithm called CULP (Classification Using Link
Prediction) which uses a new structure namely Label Embedded Graph or LEG and a
link predictor to find the class of the unlabeled data. Different link
predictors along with Compatibility Score - a new link predictor we proposed
that is designed specifically for our settings - has been used and showed
promising results for classifying different datasets. This paper further
improved CULP by designing an extension called CULM which uses a majority vote
(hence the M in the acronym) procedure with weights proportional to the
predictions' confidences to use the predictive power of multiple link
predictors and also exploits the low level features of the data. Extensive
experimental evaluations shows that both CULP and CULM are highly accurate and
competitive with the cutting edge graph classifiers and general classifiers.
Data-driven Discovery of Cyber-Physical Systems
Authors: Ye Yuan, Xiuchuan Tang, Wei Pan, Xiuting Li, Wei Zhou, Hai-Tao Zhang, Han Ding, Jorge Goncalves
Date: 2018-10-01 00:00:00

Cyber-physical systems (CPSs) embed software into the physical world. They
appear in a wide range of applications such as smart grids, robotics,
intelligent manufacture and medical monitoring. CPSs have proved resistant to
modeling due to their intrinsic complexity arising from the combination of
physical components and cyber components and the interaction between them. This
study proposes a general framework for reverse engineering CPSs directly from
data. The method involves the identification of physical systems as well as the
inference of transition logic. It has been applied successfully to a number of
real-world examples ranging from mechanical and electrical systems to medical
applications. The novel framework seeks to enable researchers to make
predictions concerning the trajectory of CPSs based on the discovered model.
Such information has been proven essential for the assessment of the
performance of CPS, the design of failure-proof CPS and the creation of design
guidelines for new CPSs.
Counterfactually Fair Prediction Using Multiple Causal Models
Authors: Fabio Massimo Zennaro, Magdalena Ivanovska
Date: 2018-10-01 00:00:00
Tasks: counterfactual, Fairness, Prediction

In this paper we study the problem of making predictions using multiple
structural casual models defined by different agents, under the constraint that
the prediction satisfies the criterion of counterfactual fairness. Relying on
the frameworks of causality, fairness and opinion pooling, we build upon and
extend previous work focusing on the qualitative aggregation of causal Bayesian
networks and causal models. In order to complement previous qualitative
results, we devise a method based on Monte Carlo simulations. This method
enables a decision-maker to aggregate the outputs of the causal models provided
by different experts while guaranteeing the counterfactual fairness of the
result. We demonstrate our approach on a simple, yet illustrative, toy case
study.
Part-Level Convolutional Neural Networks for Pedestrian Detection Using Saliency and Boundary Box Alignment
Authors: Inyong Yun, Cheolkon Jung, Xinran Wang, Alfred O. Hero, Joongkyu Kim
Date: 2018-10-01 00:00:00
Tasks: Pedestrian Detection

Pedestrians in videos have a wide range of appearances such as body poses,
occlusions, and complex backgrounds, and there exists the proposal shift
problem in pedestrian detection that causes the loss of body parts such as head
and legs. To address it, we propose part-level convolutional neural networks
(CNN) for pedestrian detection using saliency and boundary box alignment in
this paper. The proposed network consists of two sub-networks: detection and
alignment. We use saliency in the detection sub-network to remove false
positives such as lamp posts and trees. We adopt bounding box alignment on
detection proposals in the alignment sub-network to address the proposal shift
problem. First, we combine FCN and CAM to extract deep features for pedestrian
detection. Then, we perform part-level CNN to recall the lost body parts.
Experimental results on various datasets demonstrate that the proposed method
remarkably improves accuracy in pedestrian detection and outperforms existing
state-of-the-arts in terms of log average miss rate at false position per image
(FPPI).
Perfect Match: A Simple Method for Learning Representations For Counterfactual Inference With Neural Networks
Authors: Patrick Schwab, Lorenz Linhardt, Walter Karlen
Date: 2018-10-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: counterfactual, Counterfactual Inference

Learning representations for counterfactual inference from observational data is of high practical relevance for many domains, such as healthcare, public policy and economics. Counterfactual inference enables one to answer "What if...?" questions, such as "What would be the outcome if we gave this patient treatment $t_1$?". However, current methods for training neural networks for counterfactual inference on observational data are either overly complex, limited to settings with only two available treatments, or both. Here, we present Perfect Match (PM), a method for training neural networks for counterfactual inference that is easy to implement, compatible with any architecture, does not add computational complexity or hyperparameters, and extends to any number of treatments. PM is based on the idea of augmenting samples within a minibatch with their propensity-matched nearest neighbours. Our experiments demonstrate that PM outperforms a number of more complex state-of-the-art methods in inferring counterfactual outcomes across several benchmarks, particularly in settings with many treatments.
SmartChoices: Hybridizing Programming and Machine Learning
Authors: Victor Carbune, Thierry Coppey, Alexander Daryin, Thomas Deselaers, Nikhil Sarda, Jay Yagnik
Date: 2018-10-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: BIG-bench Machine Learning, Reinforcement Learning, Reinforcement Learning (RL)

We present SmartChoices, an approach to making machine learning (ML) a first class citizen in programming languages which we see as one way to lower the entrance cost to applying ML to problems in new domains. There is a growing divide in approaches to building systems: on the one hand, programming leverages human experts to define a system while on the other hand behavior is learned from data in machine learning. We propose to hybridize these two by providing a 3-call API which we expose through an object called SmartChoice. We describe the SmartChoices-interface, how it can be used in programming with minimal code changes, and demonstrate that it is an easy to use but still powerful tool by demonstrating improvements over not using ML at all on three algorithmic problems: binary search, QuickSort, and caches. In these three examples, we replace the commonly used heuristics with an ML model entirely encapsulated within a SmartChoice and thus requiring minimal code changes. As opposed to previous work applying ML to algorithmic problems, our proposed approach does not require to drop existing implementations but seamlessly integrates into the standard software development workflow and gives full control to the software developer over how ML methods are applied. Our implementation relies on standard Reinforcement Learning (RL) methods. To learn faster, we use the heuristic function, which they are replacing, as an initial function. We show how this initial function can be used to speed up and stabilize learning while providing a safety net that prevents performance to become substantially worse -- allowing for a safe deployment in critical applications in real life.
One-Click Annotation with Guided Hierarchical Object Detection
Authors: Adithya Subramanian, Anbumani Subramanian
Date: 2018-10-01 00:00:00
Tasks: Object, object-detection, Object Detection

The increase in data collection has made data annotation an interesting and
valuable task in the contemporary world. This paper presents a new methodology
for quickly annotating data using click-supervision and hierarchical object
detection. The proposed work is semi-automatic in nature where the task of
annotations is split between the human and a neural network. We show that our
improved method of annotation reduces the time, cost and mental stress on a
human annotator. The research also highlights how our method performs better
than the current approach in different circumstances such as variation in
number of objects, object size and different datasets. Our approach also
proposes a new method of using object detectors making it suitable for data
annotation task. The experiment conducted on PASCAL VOC dataset revealed that
annotation created from our approach achieves a mAP of 0.995 and a recall of
0.903. The Our Approach has shown an overall improvement by 8.5%, 18.6% in mean
average precision and recall score for KITTI and 69.6%, 36% for CITYSCAPES
dataset. The proposed framework is 3-4 times faster as compared to the standard
annotation method.
Privado: Practical and Secure DNN Inference with Enclaves
Authors: Karan Grover, Shruti Tople, Shweta Shinde, Ranjita Bhagwan, Ramachandran Ramjee
Date: 2018-10-01 00:00:00

Cloud providers are extending support for trusted hardware primitives such as Intel SGX. Simultaneously, the field of deep learning is seeing enormous innovation as well as an increase in adoption. In this paper, we ask a timely question: "Can third-party cloud services use Intel SGX enclaves to provide practical, yet secure DNN Inference-as-a-service?" We first demonstrate that DNN models executing inside enclaves are vulnerable to access pattern based attacks. We show that by simply observing access patterns, an attacker can classify encrypted inputs with 97% and 71% attack accuracy for MNIST and CIFAR10 datasets on models trained to achieve 99% and 79% original accuracy respectively. This motivates the need for PRIVADO, a system we have designed for secure, easy-to-use, and performance efficient inference-as-a-service. PRIVADO is input-oblivious: it transforms any deep learning framework that is written in C/C++ to be free of input-dependent access patterns thus eliminating the leakage. PRIVADO is fully-automated and has a low TCB: with zero developer effort, given an ONNX description of a model, it generates compact and enclave-compatible code which can be deployed on an SGX cloud platform. PRIVADO incurs low performance overhead: we use PRIVADO with Torch framework and show its overhead to be 17.18% on average on 11 different contemporary neural networks.