pwc-archive/papers-with-abstracts huggingface dataset:
Unsupervised Trajectory Segmentation and Promoting of Multi-Modal Surgical Demonstrations
Authors: Zhenzhou Shao, Hongfa Zhao, Jiexin Xie, Ying Qu, Yong Guan, Jindong Tan
Date: 2018-10-01 00:00:00
Tasks: Segmentation

To improve the efficiency of surgical trajectory segmentation for robot learning in robot-assisted minimally invasive surgery, this paper presents a fast unsupervised method using video and kinematic data, followed by a promoting procedure to address the over-segmentation issue. Unsupervised deep learning network, stacking convolutional auto-encoder, is employed to extract more discriminative features from videos in an effective way. To further improve the accuracy of segmentation, on one hand, wavelet transform is used to filter out the noises existed in the features from video and kinematic data. On the other hand, the segmentation result is promoted by identifying the adjacent segments with no state transition based on the predefined similarity measurements. Extensive experiments on a public dataset JIGSAWS show that our method achieves much higher accuracy of segmentation than state-of-the-art methods in the shorter time.
Taming VAEs
Authors: Danilo Jimenez Rezende, Fabio Viola
Date: 2018-10-01 00:00:00

In spite of remarkable progress in deep latent variable generative modeling,
training still remains a challenge due to a combination of optimization and
generalization issues. In practice, a combination of heuristic algorithms (such
as hand-crafted annealing of KL-terms) is often used in order to achieve the
desired results, but such solutions are not robust to changes in model
architecture or dataset. The best settings can often vary dramatically from one
problem to another, which requires doing expensive parameter sweeps for each
new case. Here we develop on the idea of training VAEs with additional
constraints as a way to control their behaviour. We first present a detailed
theoretical analysis of constrained VAEs, expanding our understanding of how
these models work. We then introduce and analyze a practical algorithm termed
Generalized ELBO with Constrained Optimization, GECO. The main advantage of
GECO for the machine learning practitioner is a more intuitive, yet principled,
process of tuning the loss. This involves defining of a set of constraints,
which typically have an explicit relation to the desired model performance, in
contrast to tweaking abstract hyper-parameters which implicitly affect the
model behavior. Encouraging experimental results in several standard datasets
indicate that GECO is a very robust and effective tool to balance
reconstruction and compression constraints.
Detecting Satire in the News with Machine Learning
Authors: Andreas Stöckl
Date: 2018-10-01 00:00:00
Tasks: Articles, BIG-bench Machine Learning, regression

We built models with Logistic Regression and linear Support Vector Machines
on a large dataset consisting of regular news articles and news from satirical
websites, and showed that such linear classifiers on a corpus with about 60,000
articles can perform with a precision of 98.7% and a recall of 95.2% on a
random test set of the news. On the other hand, when testing the classifier on
"publication sources" which are completely unknown during training, only an
accuracy of 88.2% and an F1-score of 76.3% are achieved. As another result, we
showed that the same algorithm can distinguish between news written by the news
agency itself and paid articles from customers. Here the results had an
accuracy of 99%.
Elastic Neural Networks for Classification
Authors: Yi Zhou, Yue Bai, Shuvra S. Bhattacharyya, Heikki Huttunen
Date: 2018-10-01 00:00:00
Tasks: Classification, General Classification

In this work we propose a framework for improving the performance of any deep neural network that may suffer from vanishing gradients. To address the vanishing gradient issue, we study a framework, where we insert an intermediate output branch after each layer in the computational graph and use the corresponding prediction loss for feeding the gradient to the early layers. The framework - which we name Elastic network - is tested with several well-known networks on CIFAR10 and CIFAR100 datasets, and the experimental results show that the proposed framework improves the accuracy on both shallow networks (e.g., MobileNet) and deep convolutional neural networks (e.g., DenseNet). We also identify the types of networks where the framework does not improve the performance and discuss the reasons. Finally, as a side product, the computational complexity of the resulting networks can be adjusted in an elastic manner by selecting the output branch according to current computational budget.
Probabilistic Meta-Representations Of Neural Networks
Authors: Theofanis Karaletsos, Peter Dayan, Zoubin Ghahramani
Date: 2018-10-01 00:00:00

Existing Bayesian treatments of neural networks are typically characterized
by weak prior and approximate posterior distributions according to which all
the weights are drawn independently. Here, we consider a richer prior
distribution in which units in the network are represented by latent variables,
and the weights between units are drawn conditionally on the values of the
collection of those variables. This allows rich correlations between related
weights, and can be seen as realizing a function prior with a Bayesian
complexity regularizer ensuring simple solutions. We illustrate the resulting
meta-representations and representations, elucidating the power of this prior.
Optimal Adaptive and Accelerated Stochastic Gradient Descent
Authors: Qi Deng, Yi Cheng, Guanghui Lan
Date: 2018-10-01 00:00:00
Tasks: BIG-bench Machine Learning, Stochastic Optimization

Stochastic gradient descent (\textsc{Sgd}) methods are the most powerful
optimization tools in training machine learning and deep learning models.
Moreover, acceleration (a.k.a. momentum) methods and diagonal scaling (a.k.a.
adaptive gradient) methods are the two main techniques to improve the slow
convergence of \textsc{Sgd}. While empirical studies have demonstrated
potential advantages of combining these two techniques, it remains unknown
whether these methods can achieve the optimal rate of convergence for
stochastic optimization. In this paper, we present a new class of adaptive and
accelerated stochastic gradient descent methods and show that they exhibit the
optimal sampling and iteration complexity for stochastic optimization. More
specifically, we show that diagonal scaling, initially designed to improve
vanilla stochastic gradient, can be incorporated into accelerated stochastic
gradient descent to achieve the optimal rate of convergence for smooth
stochastic optimization. We also show that momentum, apart from being known to
speed up the convergence rate of deterministic optimization, also provides us
new ways of designing non-uniform and aggressive moving average schemes in
stochastic optimization. Finally, we present some heuristics that help to
implement adaptive accelerated stochastic gradient descent methods and to
further improve their practical performance for machine learning and deep
learning.
Generative Adversarial Network for Medical Images (MI-GAN)
Authors: Talha Iqbal, Hazrat Ali
Date: 2018-10-01 00:00:00
Tasks: Generative Adversarial Network

Deep learning algorithms produces state-of-the-art results for different
machine learning and computer vision tasks. To perform well on a given task,
these algorithms require large dataset for training. However, deep learning
algorithms lack generalization and suffer from over-fitting whenever trained on
small dataset, especially when one is dealing with medical images. For
supervised image analysis in medical imaging, having image data along with
their corresponding annotated ground-truths is costly as well as time consuming
since annotations of the data is done by medical experts manually. In this
paper, we propose a new Generative Adversarial Network for Medical Imaging
(MI-GAN). The MI-GAN generates synthetic medical images and their segmented
masks, which can then be used for the application of supervised analysis of
medical images. Particularly, we present MI-GAN for synthesis of retinal
images. The proposed method generates precise segmented images better than the
existing techniques. The proposed model achieves a dice coefficient of 0.837 on
STARE dataset and 0.832 on DRIVE dataset which is state-of-the-art performance
on both the datasets.
Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network
Authors: Xuanqing Liu, Yao Li, Chongruo wu, Cho-Jui Hsieh
Date: 2018-10-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Adversarial Defense

We present a new algorithm to train a robust neural network against adversarial attacks. Our algorithm is motivated by the following two ideas. First, although recent work has demonstrated that fusing randomness can improve the robustness of neural networks (Liu 2017), we noticed that adding noise blindly to all the layers is not the optimal way to incorporate randomness. Instead, we model randomness under the framework of Bayesian Neural Network (BNN) to formally learn the posterior distribution of models in a scalable way. Second, we formulate the mini-max problem in BNN to learn the best model distribution under adversarial attacks, leading to an adversarial-trained Bayesian neural net. Experiment results demonstrate that the proposed algorithm achieves state-of-the-art performance under strong attacks. On CIFAR-10 with VGG network, our model leads to 14\% accuracy improvement compared with adversarial training (Madry 2017) and random self-ensemble (Liu 2017) under PGD attack with $0.035$ distortion, and the gap becomes even larger on a subset of ImageNet.
Learnable Pooling Methods for Video Classification
Authors: Sebastian Kmiec, Juhan Bae, Ruijian An
Date: 2018-10-01 00:00:00
Tasks: Classification, General Classification, Video Classification, Video Understanding

We introduce modifications to state-of-the-art approaches to aggregating
local video descriptors by using attention mechanisms and function
approximations. Rather than using ensembles of existing architectures, we
provide an insight on creating new architectures. We demonstrate our solutions
in the "The 2nd YouTube-8M Video Understanding Challenge", by using frame-level
video and audio descriptors. We obtain testing accuracy similar to the state of
the art, while meeting budget constraints, and touch upon strategies to improve
the state of the art. Model implementations are available in
https://github.com/pomonam/LearnablePoolingMethods.
End-To-End Alzheimer's Disease Diagnosis and Biomarker Identification
Authors: Soheil Esmaeilzadeh, Dimitrios Ioannis Belivanis, Kilian M. Pohl, Ehsan Adeli
Date: 2018-10-01 00:00:00
Tasks: Deep Learning

As shown in computer vision, the power of deep learning lies in automatically
learning relevant and powerful features for any perdition task, which is made
possible through end-to-end architectures. However, deep learning approaches
applied for classifying medical images do not adhere to this architecture as
they rely on several pre- and post-processing steps. This shortcoming can be
explained by the relatively small number of available labeled subjects, the
high dimensionality of neuroimaging data, and difficulties in interpreting the
results of deep learning methods. In this paper, we propose a simple 3D
Convolutional Neural Networks and exploit its model parameters to tailor the
end-to-end architecture for the diagnosis of Alzheimer's disease (AD). Our
model can diagnose AD with an accuracy of 94.1\% on the popular ADNI dataset
using only MRI data, which outperforms the previous state-of-the-art. Based on
the learned model, we identify the disease biomarkers, the results of which
were in accordance with the literature. We further transfer the learned model
to diagnose mild cognitive impairment (MCI), the prodromal stage of AD, which
yield better results compared to other methods.
A Simple Machine Learning Method for Commonsense Reasoning? A Short Commentary on Trinh & Le (2018)
Authors: Walid S. Saba
Date: 2018-10-01 00:00:00
Tasks: BIG-bench Machine Learning, Natural Language Understanding

This is a short Commentary on Trinh & Le (2018) ("A Simple Method for
Commonsense Reasoning") that outlines three serious flaws in the cited paper
and discusses why data-driven approaches cannot be considered as serious models
for the commonsense reasoning needed in natural language understanding in
general, and in reference resolution, in particular.
FIRE-DES++: Enhanced Online Pruning of Base Classifiers for Dynamic Ensemble Selection
Authors: Rafael M. O. Cruz, Dayvid V. R. Oliveira, George D. C. Cavalcanti, Robert Sabourin
Date: 2018-10-01 00:00:00
Tasks: Classification, General Classification

Despite being very effective in several classification tasks, Dynamic
Ensemble Selection (DES) techniques can select classifiers that classify all
samples in the region of competence as being from the same class. The Frienemy
Indecision REgion DES (FIRE-DES) tackles this problem by pre-selecting
classifiers that correctly classify at least one pair of samples from different
classes in the region of competence of the test sample. However, FIRE-DES
applies the pre-selection for the classification of a test sample if and only
if its region of competence is composed of samples from different classes
(indecision region), even though this criterion is not reliable for determining
if a test sample is located close to the borders of classes (true indecision
region) when the region of competence is obtained using classical nearest
neighbors approach. Because of that, FIRE-DES mistakes noisy regions for true
indecision regions, leading to the pre-selection of incompetent classifiers,
and mistakes true indecision regions for safe regions, leaving samples in such
regions without any pre-selection. To tackle these issues, we propose the
FIRE-DES++, an enhanced FIRE-DES that removes noise and reduces the overlap of
classes in the validation set; and defines the region of competence using an
equal number of samples of each class, avoiding selecting a region of
competence with samples of a single class. Experiments are conducted using
FIRE-DES++ with 8 different dynamic selection techniques on 64 classification
datasets. Experimental results show that FIRE-DES++ increases the
classification performance of all DES techniques considered in this work,
outperforming FIRE-DES with 7 out of the 8 DES techniques, and outperforming
state-of-the-art DES frameworks.
Layer-compensated Pruning for Resource-constrained Convolutional Neural Networks
Authors: Ting-Wu Chin, Cha Zhang, Diana Marculescu
Date: 2018-10-01 00:00:00
Tasks: Meta-Learning, Scheduling

Resource-efficient convolution neural networks enable not only the
intelligence on edge devices but also opportunities in system-level
optimization such as scheduling. In this work, we aim to improve the
performance of resource-constrained filter pruning by merging two sub-problems
commonly considered, i.e., (i) how many filters to prune for each layer and
(ii) which filters to prune given a per-layer pruning budget, into a global
filter ranking problem. Our framework entails a novel algorithm, dubbed
layer-compensated pruning, where meta-learning is involved to determine better
solutions. We show empirically that the proposed algorithm is superior to prior
art in both effectiveness and efficiency. Specifically, we reduce the accuracy
gap between the pruned and original networks from 0.9% to 0.7% with 8x
reduction in time needed for meta-learning, i.e., from 1 hour down to 7
minutes. To this end, we demonstrate the effectiveness of our algorithm using
ResNet and MobileNetV2 networks under CIFAR-10, ImageNet, and Bird-200
datasets.
Interactive Agent Modeling by Learning to Probe
Authors: Tianmin Shu, Caiming Xiong, Ying Nian Wu, Song-Chun Zhu
Date: 2018-10-01 00:00:00
Tasks: Imitation Learning, Reinforcement Learning

The ability of modeling the other agents, such as understanding their
intentions and skills, is essential to an agent's interactions with other
agents. Conventional agent modeling relies on passive observation from
demonstrations. In this work, we propose an interactive agent modeling scheme
enabled by encouraging an agent to learn to probe. In particular, the probing
agent (i.e. a learner) learns to interact with the environment and with a
target agent (i.e., a demonstrator) to maximize the change in the observed
behaviors of that agent. Through probing, rich behaviors can be observed and
are used for enhancing the agent modeling to learn a more accurate mind model
of the target agent. Our framework consists of two learning processes: i)
imitation learning for an approximated agent model and ii) pure
curiosity-driven reinforcement learning for an efficient probing policy to
discover new behaviors that otherwise can not be observed. We have validated
our approach in four different tasks. The experimental results suggest that the
agent model learned by our approach i) generalizes better in novel scenarios
than the ones learned by passive observation, random probing, and other
curiosity-driven approaches do, and ii) can be used for enhancing performance
in multiple applications including distilling optimal planning to a policy net,
collaboration, and competition. A video demo is available at
https://www.dropbox.com/s/8mz6rd3349tso67/Probing_Demo.mov?dl=0
Simple and Fast Algorithms for Interactive Machine Learning with Random Counter-examples
Authors: Jagdeep Bhatia
Date: 2018-10-01 00:00:00
Tasks: BIG-bench Machine Learning, Clustering, PAC learning

This work describes simple and efficient algorithms for interactively learning non-binary concepts in the learning from random counter-examples (LRC) model. Here, learning takes place from random counter-examples that the learner receives in response to their proper equivalence queries. In this context, the learning time is defined as the number of counter-examples needed by the learner to identify the target concept. Such learning is particularly suited for online ranking, classification, clustering, etc., where machine learning models must be used before they are fully trained. We provide two simple LRC algorithms, deterministic and randomized, for exactly learning non-binary target concepts for any concept class $H$. We show that both of these algorithms have an $\mathcal{O}(\log{}|H|)$ asymptotically optimal average learning time. This solves an open problem on the existence of an efficient LRC randomized algorithm while simplifying and generalizing previous results. We also show that the expected learning time of any arbitrary LRC algorithm can be upper bounded by $\mathcal{O}(\frac{1}{\epsilon}\log{\frac{|H|}{\delta}})$, where $\epsilon$ and $\delta$ are the allowed learning error and failure probability respectively. This shows that LRC interactive learning is at least as efficient as non-interactive Probably Approximately Correct (PAC) learning. Our simulations show that in practice, these algorithms outperform their theoretical bounds.
One Network to Solve All ROIs: Deep Learning CT for Any ROI using Differentiated Backprojection
Authors: Yoseob Han, Jong Chul Ye
Date: 2018-10-01 00:00:00
Tasks: All, CT Reconstruction

Computed tomography for region-of-interest (ROI) reconstruction has advantages of reducing X-ray radiation dose and using a small detector. However, standard analytic reconstruction methods suffer from severe cupping artifacts, and existing model-based iterative reconstruction methods require extensive computations. Recently, we proposed a deep neural network to learn the cupping artifact, but the network is not well generalized for different ROIs due to the singularities in the corrupted images. Therefore, there is an increasing demand for a neural network that works well for any ROI sizes. In this paper, two types of neural networks are designed. The first type learns ROI size-specific cupping artifacts from the analytic reconstruction images, whereas the second type network is to learn to invert the finite Hilbert transform from the truncated differentiated backprojection (DBP) data. Their generalizability for any ROI sizes is then examined. Experimental results show that the new type of neural network significantly outperforms the existing iterative methods for any ROI size in spite of significantly reduced run-time complexity. Since the proposed method consistently surpasses existing methods for any ROIs, it can be used as a general CT reconstruction engine for many practical applications without compromising possible detector truncation.
Ranking Paragraphs for Improving Answer Recall in Open-Domain Question Answering
Authors: Jinhyuk Lee, Seongjun Yun, Hyunjae Kim, Miyoung Ko, Jaewoo Kang
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Information Retrieval, Open-Domain Question Answering, Question Answering, Reading Comprehension, Retrieval

Recently, open-domain question answering (QA) has been combined with machine
comprehension models to find answers in a large knowledge source. As
open-domain QA requires retrieving relevant documents from text corpora to
answer questions, its performance largely depends on the performance of
document retrievers. However, since traditional information retrieval systems
are not effective in obtaining documents with a high probability of containing
answers, they lower the performance of QA systems. Simply extracting more
documents increases the number of irrelevant documents, which also degrades the
performance of QA systems. In this paper, we introduce Paragraph Ranker which
ranks paragraphs of retrieved documents for a higher answer recall with less
noise. We show that ranking paragraphs and aggregating answers using Paragraph
Ranker improves performance of open-domain QA pipeline on the four open-domain
QA datasets by 7.8% on average.
Learning Deep Representations from Clinical Data for Chronic Kidney Disease
Authors: Duc Thanh Anh Luong, Varun Chandola
Date: 2018-10-01 00:00:00
Tasks: Time Series Analysis

We study the behavior of a Time-Aware Long Short-Term Memory Autoencoder, a
state-of-the-art method, in the context of learning latent representations from
irregularly sampled patient data. We identify a key issue in the way such
recurrent neural network models are being currently used and show that the
solution of the issue leads to significant improvements in the learnt
representations on both synthetic and real datasets. A detailed analysis of the
improved methodology for representing patients suffering from Chronic Kidney
Disease (CKD) using clinical data is provided. Experimental results show that
the proposed T-LSTM model is able to capture the long-term trends in the data,
while effectively handling the noise in the signal. Finally, we show that by
using the latent representations of the CKD patients obtained from the T-LSTM
autoencoder, one can identify unusual patient profiles from the target
population.