pwc-archive/papers-with-abstracts huggingface dataset:
Global Convergence of EM Algorithm for Mixtures of Two Component Linear Regression
Authors: Jeongyeol Kwon, Wei Qian, Constantine Caramanis, Yudong Chen, Damek Davis
Date: 2018-10-12 00:00:00
Tasks: regression

The Expectation-Maximization algorithm is perhaps the most broadly used algorithm for inference of latent variable problems. A theoretical understanding of its performance, however, largely remains lacking. Recent results established that EM enjoys global convergence for Gaussian Mixture Models. For Mixed Linear Regression, however, only local convergence results have been established, and those only for the high SNR regime. We show here that EM converges for mixed linear regression with two components (it is known that it may fail to converge for three or more), and moreover that this convergence holds for random initialization. Our analysis reveals that EM exhibits very different behavior in Mixed Linear Regression from its behavior in Gaussian Mixture Models, and hence our proofs require the development of several new ideas.
Policy Transfer with Strategy Optimization
Authors: Wenhao Yu, C. Karen Liu, Greg Turk
Date: 2018-10-12 00:00:00
Proceeding: ICLR 2019 5
Tasks: Transfer Learning

Computer simulation provides an automatic and safe way for training robotic
control policies to achieve complex tasks such as locomotion. However, a policy
trained in simulation usually does not transfer directly to the real hardware
due to the differences between the two environments. Transfer learning using
domain randomization is a promising approach, but it usually assumes that the
target environment is close to the distribution of the training environments,
thus relying heavily on accurate system identification. In this paper, we
present a different approach that leverages domain randomization for
transferring control policies to unknown environments. The key idea that,
instead of learning a single policy in the simulation, we simultaneously learn
a family of policies that exhibit different behaviors. When tested in the
target environment, we directly search for the best policy in the family based
on the task performance, without the need to identify the dynamic parameters.
We evaluate our method on five simulated robotic control problems with
different discrepancies in the training and testing environment and demonstrate
that our method can overcome larger modeling errors compared to training a
robust policy or an adaptive policy.
Graph HyperNetworks for Neural Architecture Search
Authors: Chris Zhang, Mengye Ren, Raquel Urtasun
Date: 2018-10-12 00:00:00
Proceeding: ICLR 2019 5
Tasks: Graph Neural Network, Neural Architecture Search

Neural architecture search (NAS) automatically finds the best task-specific neural network topology, outperforming many manual architecture designs. However, it can be prohibitively expensive as the search requires training thousands of different networks, while each can last for hours. In this work, we propose the Graph HyperNetwork (GHN) to amortize the search cost: given an architecture, it directly generates the weights by running inference on a graph neural network. GHNs model the topology of an architecture and therefore can predict network performance more accurately than regular hypernetworks and premature early stopping. To perform NAS, we randomly sample architectures and use the validation accuracy of networks with GHN generated weights as the surrogate search signal. GHNs are fast -- they can search nearly 10 times faster than other random search methods on CIFAR-10 and ImageNet. GHNs can be further extended to the anytime prediction setting, where they have found networks with better speed-accuracy tradeoff than the state-of-the-art manual designs.
Explaining Black Boxes on Sequential Data using Weighted Automata
Authors: Stephane Ayache, Remi Eyraud, Noe Goudian
Date: 2018-10-12 00:00:00

Understanding how a learned black box works is of crucial interest for the
future of Machine Learning. In this paper, we pioneer the question of the
global interpretability of learned black box models that assign numerical
values to symbolic sequential data. To tackle that task, we propose a spectral
algorithm for the extraction of weighted automata (WA) from such black boxes.
This algorithm does not require the access to a dataset or to the inner
representation of the black box: the inferred model can be obtained solely by
querying the black box, feeding it with inputs and analyzing its outputs.
Experiments using Recurrent Neural Networks (RNN) trained on a wide collection
of 48 synthetic datasets and 2 real datasets show that the obtained
approximation is of great quality.
Estimating Information Flow in Deep Neural Networks
Authors: Ziv Goldfeld, Ewout van den Berg, Kristjan Greenewald, Igor Melnyk, Nam Nguyen, Brian Kingsbury, Yury Polyanskiy
Date: 2018-10-12 00:00:00
Tasks: Clustering

We study the flow of information and the evolution of internal representations during deep neural network (DNN) training, aiming to demystify the compression aspect of the information bottleneck theory. The theory suggests that DNN training comprises a rapid fitting phase followed by a slower compression phase, in which the mutual information $I(X;T)$ between the input $X$ and internal representations $T$ decreases. Several papers observe compression of estimated mutual information on different DNN models, but the true $I(X;T)$ over these networks is provably either constant (discrete $X$) or infinite (continuous $X$). This work explains the discrepancy between theory and experiments, and clarifies what was actually measured by these past works. To this end, we introduce an auxiliary (noisy) DNN framework for which $I(X;T)$ is a meaningful quantity that depends on the network's parameters. This noisy framework is shown to be a good proxy for the original (deterministic) DNN both in terms of performance and the learned representations. We then develop a rigorous estimator for $I(X;T)$ in noisy DNNs and observe compression in various models. By relating $I(X;T)$ in the noisy DNN to an information-theoretic communication problem, we show that compression is driven by the progressive clustering of hidden representations of inputs from the same class. Several methods to directly monitor clustering of hidden representations, both in noisy and deterministic DNNs, are used to show that meaningful clusters form in the $T$ space. Finally, we return to the estimator of $I(X;T)$ employed in past works, and demonstrate that while it fails to capture the true (vacuous) mutual information, it does serve as a measure for clustering. This clarifies the past observations of compression and isolates the geometric clustering of hidden representations as the true phenomenon of interest.
Does Haze Removal Help CNN-based Image Classification?
Authors: Yanting Pei, Yaping Huang, Qi Zou, Yuhang Lu, Song Wang
Date: 2018-10-12 00:00:00
Proceeding: ECCV 2018 9
Tasks: Classification, General Classification, image-classification, Image Classification, Image Dehazing, SSIM

Hazy images are common in real scenarios and many dehazing methods have been
developed to automatically remove the haze from images. Typically, the goal of
image dehazing is to produce clearer images from which human vision can better
identify the object and structural details present in the images. When the
ground-truth haze-free image is available for a hazy image, quantitative
evaluation of image dehazing is usually based on objective metrics, such as
Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM). However, in
many applications, large-scale images are collected not for visual examination
by human. Instead, they are used for many high-level vision tasks, such as
automatic classification, recognition and categorization. One fundamental
problem here is whether various dehazing methods can produce clearer images
that can help improve the performance of the high-level tasks. In this paper,
we empirically study this problem in the important task of image classification
by using both synthetic and real hazy image datasets. From the experimental
results, we find that the existing image-dehazing methods cannot improve much
the image-classification performance and sometimes even reduce the
image-classification performance.
Improving Generalization of Sequence Encoder-Decoder Networks for Inverse Imaging of Cardiac Transmembrane Potential
Authors: Sandesh Ghimire, Prashnna Kumar Gyawali, John L. Sapp, Milan Horacek, Linwei Wang
Date: 2018-10-12 00:00:00
Tasks: Decoder, Learning Theory

Deep learning models have shown state-of-the-art performance in many inverse
reconstruction problems. However, it is not well understood what properties of
the latent representation may improve the generalization ability of the
network. Furthermore, limited models have been presented for inverse
reconstructions over time sequences. In this paper, we study the generalization
ability of a sequence encoder decoder model for solving inverse reconstructions
on time sequences. Our central hypothesis is that the generalization ability of
the network can be improved by 1) constrained stochasticity and 2) global
aggregation of temporal information in the latent space. First, drawing from
analytical learning theory, we theoretically show that a stochastic latent
space will lead to an improved generalization ability. Second, we consider an
LSTM encoder-decoder architecture that compresses a global latent vector from
all last-layer units in the LSTM encoder. This model is compared with
alternative LSTM encoder-decoder architectures, each in deterministic and
stochastic versions. The results demonstrate that the generalization ability of
an inverse reconstruction network can be improved by constrained stochasticity
combined with global aggregation of temporal information in the latent space.
Faster k-Medoids Clustering: Improving the PAM, CLARA, and CLARANS Algorithms
Authors: Erich Schubert, Peter J. Rousseeuw
Date: 2018-10-12 00:00:00
Tasks: Clustering

Clustering non-Euclidean data is difficult, and one of the most used algorithms besides hierarchical clustering is the popular algorithm Partitioning Around Medoids (PAM), also simply referred to as k-medoids. In Euclidean geometry the mean-as used in k-means-is a good estimator for the cluster center, but this does not hold for arbitrary dissimilarities. PAM uses the medoid instead, the object with the smallest dissimilarity to all others in the cluster. This notion of centrality can be used with any (dis-)similarity, and thus is of high relevance to many domains such as biology that require the use of Jaccard, Gower, or more complex distances. A key issue with PAM is its high run time cost. We propose modifications to the PAM algorithm to achieve an O(k)-fold speedup in the second SWAP phase of the algorithm, but will still find the same results as the original PAM algorithm. If we slightly relax the choice of swaps performed (at comparable quality), we can further accelerate the algorithm by performing up to k swaps in each iteration. With the substantially faster SWAP, we can now also explore alternative strategies for choosing the initial medoids. We also show how the CLARA and CLARANS algorithms benefit from these modifications. It can easily be combined with earlier approaches to use PAM and CLARA on big data (some of which use PAM as a subroutine, hence can immediately benefit from these improvements), where the performance with high k becomes increasingly important. In experiments on real data with k=100, we observed a 200-fold speedup compared to the original PAM SWAP algorithm, making PAM applicable to larger data sets as long as we can afford to compute a distance matrix, and in particular to higher k (at k=2, the new SWAP was only 1.5 times faster, as the speedup is expected to increase with k).
Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience
Authors: Yevgen Chebotar, Ankur Handa, Viktor Makoviychuk, Miles Macklin, Jan Issac, Nathan Ratliff, Dieter Fox
Date: 2018-10-12 00:00:00

We consider the problem of transferring policies to the real world by
training on a distribution of simulated scenarios. Rather than manually tuning
the randomization of simulations, we adapt the simulation parameter
distribution using a few real world roll-outs interleaved with policy training.
In doing so, we are able to change the distribution of simulations to improve
the policy transfer by matching the policy behavior in simulation and the real
world. We show that policies trained with our method are able to reliably
transfer to different robots in two real world tasks: swing-peg-in-hole and
opening a cabinet drawer. The video of our experiments can be found at
https://sites.google.com/view/simopt
Building Dynamic Knowledge Graphs from Text using Machine Reading Comprehension
Authors: Rajarshi Das, Tsendsuren Munkhdalai, Xingdi Yuan, Adam Trischler, Andrew McCallum
Date: 2018-10-12 00:00:00
Proceeding: ICLR 2019 5
Tasks: Knowledge Graphs, Machine Reading Comprehension, Procedural Text Understanding, Question Answering, Reading Comprehension

We propose a neural machine-reading model that constructs dynamic knowledge
graphs from procedural text. It builds these graphs recurrently for each step
of the described procedure, and uses them to track the evolving states of
participant entities. We harness and extend a recently proposed machine reading
comprehension (MRC) model to query for entity states, since these states are
generally communicated in spans of text and MRC models perform well in
extracting entity-centric spans. The explicit, structured, and evolving
knowledge graph representations that our model constructs can be used in
downstream question answering tasks to improve machine comprehension of text,
as we demonstrate empirically. On two comprehension tasks from the recently
proposed PROPARA dataset (Dalvi et al., 2018), our model achieves
state-of-the-art results. We further show that our model is competitive on the
RECIPES dataset (Kiddon et al., 2015), suggesting it may be generally
applicable. We present some evidence that the model's knowledge graphs help it
to impose commonsense constraints on its predictions.
SmartPM: Automatic Adaptation of Dynamic Processes at Run-Time
Authors: Andrea Marrella
Date: 2018-10-12 00:00:00
Tasks: Management

The research activity outlined in this PhD thesis is devoted to define a
general approach, a concrete architecture and a prototype Process Management
System (PMS) for the automated adaptation of dynamic processes at run-time, on
the basis of a declarative specification of process tasks and relying on
well-established reasoning about actions and planning techniques. The purpose
is to demonstrate that the combination of procedural and imperative models with
declarative elements, along with the exploitation of techniques from the field
of artificial intelligence (AI), such as Situation Calculus, IndiGolog and
automated planning, can increase the ability of existing PMSs of supporting
dynamic processes. To this end, a prototype PMS named SmartPM, which is
specifically tailored for supporting collaborative work of process participants
during pervasive scenarios, has been developed. The adaptation mechanism
deployed on SmartPM is based on execution monitoring for detecting failures at
run-time, which does not require the definition of the adaptation strategy in
the process itself (as most of the current approaches do), and on automatic
planning techniques for the synthesis of the recovery procedure.
Temporal Convolutional Memory Networks for Remaining Useful Life Estimation of Industrial Machinery
Authors: Lahiru Jayasinghe, Tharaka Samarasinghe, Chau Yuen, Jenny Chen Ni Low, Shuzhi Sam Ge
Date: 2018-10-12 00:00:00
Tasks: Data Augmentation

Accurately estimating the remaining useful life (RUL) of industrial machinery
is beneficial in many real-world applications. Estimation techniques have
mainly utilized linear models or neural network based approaches with a focus
on short term time dependencies. This paper, introduces a system model that
incorporates temporal convolutions with both long term and short term time
dependencies. The proposed network learns salient features and complex temporal
variations in sensor values, and predicts the RUL. A data augmentation method
is used for increased accuracy. The proposed method is compared with several
state-of-the-art algorithms on publicly available datasets. It demonstrates
promising results, with superior results for datasets obtained from complex
environments.
The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systems
Authors: Robert Krajewski, Julian Bock, Laurent Kloeker, Lutz Eckstein
Date: 2018-10-11 00:00:00
Tasks: Trajectory Prediction

Scenario-based testing for the safety validation of highly automated vehicles
is a promising approach that is being examined in research and industry. This
approach heavily relies on data from real-world scenarios to derive the
necessary scenario information for testing. Measurement data should be
collected at a reasonable effort, contain naturalistic behavior of road users
and include all data relevant for a description of the identified scenarios in
sufficient quality. However, the current measurement methods fail to meet at
least one of the requirements. Thus, we propose a novel method to measure data
from an aerial perspective for scenario-based validation fulfilling the
mentioned requirements. Furthermore, we provide a large-scale naturalistic
vehicle trajectory dataset from German highways called highD. We evaluate the
data in terms of quantity, variety and contained scenarios. Our dataset
consists of 16.5 hours of measurements from six locations with 110 000
vehicles, a total driven distance of 45 000 km and 5600 recorded complete lane
changes. The highD dataset is available online at: http://www.highD-dataset.com
Inventory Balancing with Online Learning
Authors: Wang Chi Cheung, Will Ma, David Simchi-Levi, Xinshang Wang
Date: 2018-10-11 00:00:00

We study a general problem of allocating limited resources to heterogeneous customers over time under model uncertainty. Each type of customer can be serviced using different actions, each of which stochastically consumes some combination of resources, and returns different rewards for the resources consumed. We consider a general model where the resource consumption distribution associated with each (customer type, action)-combination is not known, but is consistent and can be learned over time. In addition, the sequence of customer types to arrive over time is arbitrary and completely unknown. We overcome both the challenges of model uncertainty and customer heterogeneity by judiciously synthesizing two algorithmic frameworks from the literature: inventory balancing, which "reserves" a portion of each resource for high-reward customer types which could later arrive, and online learning, which shows how to "explore" the resource consumption distributions of each customer type under different actions. We define an auxiliary problem, which allows for existing competitive ratio and regret bounds to be seamlessly integrated. Furthermore, we show that the performance guarantee generated by our framework is tight, that is, we provide an information-theoretic lower bound which shows that both the loss from competitive ratio and the loss for regret are relevant in the combined problem. Finally, we demonstrate the efficacy of our algorithms on a publicly available hotel data set. Our framework is highly practical in that it requires no historical data (no fitted customer choice models, nor forecasting of customer arrival patterns) and can be used to initialize allocation strategies in fast-changing environments.
Bottom-up Attention, Models of
Authors: Ali Borji, Hamed R. -Tavakoli, Zoya Bylinskii
Date: 2018-10-11 00:00:00
Tasks: Saliency Prediction

In this review, we examine the recent progress in saliency prediction and
proposed several avenues for future research. In spite of tremendous efforts
and huge progress, there is still room for improvement in terms finer-grained
analysis of deep saliency models, evaluation measures, datasets, annotation
methods, cognitive studies, and new applications. This chapter will appear in
Encyclopedia of Computational Neuroscience.
InfiNet: Fully Convolutional Networks for Infant Brain MRI Segmentation
Authors: Shubham Kumar, Sailesh Conjeti, Abhijit Guha Roy, Christian Wachinger, Nassir Navab
Date: 2018-10-11 00:00:00
Tasks: Decoder, Infant Brain Mri Segmentation, MRI segmentation, Segmentation, Semantic Segmentation

We present a novel, parameter-efficient and practical fully convolutional
neural network architecture, termed InfiNet, aimed at voxel-wise semantic
segmentation of infant brain MRI images at iso-intense stage, which can be
easily extended for other segmentation tasks involving multi-modalities.
InfiNet consists of double encoder arms for T1 and T2 input scans that feed
into a joint-decoder arm that terminates in the classification layer. The
novelty of InfiNet lies in the manner in which the decoder upsamples lower
resolution input feature map(s) from multiple encoder arms. Specifically, the
pooled indices computed in the max-pooling layers of each of the encoder blocks
are related to the corresponding decoder block to perform non-linear
learning-free upsampling. The sparse maps are concatenated with intermediate
encoder representations (skip connections) and convolved with trainable filters
to produce dense feature maps. InfiNet is trained end-to-end to optimize for
the Generalized Dice Loss, which is well-suited for high class imbalance.
InfiNet achieves the whole-volume segmentation in under 50 seconds and we
demonstrate competitive performance against multiple state-of-the art deep
architectures and their multi-modal variants.
Learning Optimal Deep Projection of $^{18}$F-FDG PET Imaging for Early Differential Diagnosis of Parkinsonian Syndromes
Authors: Shubham Kumar, Abhijit Guha Roy, Ping Wu, Sailesh Conjeti, R. S. Anand, Jian Wang, Igor Yakushev, Stefan Förster, Markus Schwaiger, Sung-Cheng Huang, Axel Rominger, Chuantao Zuo, Kuangyu Shi
Date: 2018-10-11 00:00:00
Tasks: Diagnostic, Dimensionality Reduction, General Classification

Several diseases of parkinsonian syndromes present similar symptoms at early
stage and no objective widely used diagnostic methods have been approved until
now. Positron emission tomography (PET) with $^{18}$F-FDG was shown to be able
to assess early neuronal dysfunction of synucleinopathies and tauopathies.
Tensor factorization (TF) based approaches have been applied to identify
characteristic metabolic patterns for differential diagnosis. However, these
conventional dimension-reduction strategies assume linear or multi-linear
relationships inside data, and are therefore insufficient to distinguish
nonlinear metabolic differences between various parkinsonian syndromes. In this
paper, we propose a Deep Projection Neural Network (DPNN) to identify
characteristic metabolic pattern for early differential diagnosis of
parkinsonian syndromes. We draw our inspiration from the existing TF methods.
The network consists of a (i) compression part: which uses a deep network to
learn optimal 2D projections of 3D scans, and a (ii) classification part: which
maps the 2D projections to labels. The compression part can be pre-trained
using surplus unlabelled datasets. Also, as the classification part operates on
these 2D projections, it can be trained end-to-end effectively with limited
labelled data, in contrast to 3D approaches. We show that DPNN is more
effective in comparison to existing state-of-the-art and plausible baselines.