pwc-archive/papers-with-abstracts huggingface dataset:
A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction
Authors: Jo Schlemper, Jose Caballero, Joseph V. Hajnal, Anthony Price, Daniel Rueckert
Date: 2017-04-08 00:00:00
Tasks: compressed sensing, Dictionary Learning, Image Reconstruction

Inspired by recent advances in deep learning, we propose a framework for
reconstructing dynamic sequences of 2D cardiac magnetic resonance (MR) images
from undersampled data using a deep cascade of convolutional neural networks
(CNNs) to accelerate the data acquisition process. In particular, we address
the case where data is acquired using aggressive Cartesian undersampling.
Firstly, we show that when each 2D image frame is reconstructed independently,
the proposed method outperforms state-of-the-art 2D compressed sensing
approaches such as dictionary learning-based MR image reconstruction, in terms
of reconstruction error and reconstruction speed. Secondly, when reconstructing
the frames of the sequences jointly, we demonstrate that CNNs can learn
spatio-temporal correlations efficiently by combining convolution and data
sharing approaches. We show that the proposed method consistently outperforms
state-of-the-art methods and is capable of preserving anatomical structure more
faithfully up to 11-fold undersampling. Moreover, reconstruction is very fast:
each complete dynamic sequence can be reconstructed in less than 10s and, for
the 2D case, each image frame can be reconstructed in 23ms, enabling real-time
applications.
Robust Visual SLAM with Point and Line Features
Authors: Xingxing Zuo, Xiaojia Xie, Yong liu, Guoquan Huang
Date: 2017-11-23 00:00:00
Tasks: Stereo Matching, Stereo Matching Hand

In this paper, we develop a robust efficient visual SLAM system that utilizes
heterogeneous point and line features. By leveraging ORB-SLAM [1], the proposed
system consists of stereo matching, frame tracking, local mapping, loop
detection, and bundle adjustment of both point and line features. In
particular, as the main theoretical contributions of this paper, we, for the
first time, employ the orthonormal representation as the minimal
parameterization to model line features along with point features in visual
SLAM and analytically derive the Jacobians of the re-projection errors with
respect to the line parameters, which significantly improves the SLAM solution.
The proposed SLAM has been extensively tested in both synthetic and real-world
experiments whose results demonstrate that the proposed system outperforms the
state-of-the-art methods in various scenarios.
Sensor Selection and Random Field Reconstruction for Robust and Cost-effective Heterogeneous Weather Sensor Networks for the Developing World
Authors: Pengfei Zhang, Ido Nevat, Gareth W. Peters, Wolfgang Fruehwirt, Yongchao Huang, Ivonne Anders, Michael Osborne
Date: 2017-11-12 00:00:00
Tasks: Combinatorial Optimization

We address the two fundamental problems of spatial field reconstruction and
sensor selection in heterogeneous sensor networks: (i) how to efficiently
perform spatial field reconstruction based on measurements obtained
simultaneously from networks with both high and low quality sensors; and (ii)
how to perform query based sensor set selection with predictive MSE performance
guarantee. For the first problem, we developed a low complexity algorithm based
on the spatial best linear unbiased estimator (S-BLUE). Next, building on the
S-BLUE, we address the second problem, and develop an efficient algorithm for
query based sensor set selection with performance guarantee. Our algorithm is
based on the Cross Entropy method which solves the combinatorial optimization
problem in an efficient manner.
Gland Instance Segmentation Using Deep Multichannel Neural Networks
Authors: Yan Xu, Yang Li, Yipei Wang, Mingyuan Liu, Yubo Fan, Maode Lai, Eric I-Chao Chang
Date: 2016-11-21 00:00:00
Tasks: Instance Segmentation, Segmentation, Semantic Segmentation

Objective: A new image instance segmentation method is proposed to segment
individual glands (instances) in colon histology images. This process is
challenging since the glands not only need to be segmented from a complex
background, they must also be individually identified. Methods: We leverage the
idea of image-to-image prediction in recent deep learning by designing an
algorithm that automatically exploits and fuses complex multichannel
information - regional, location, and boundary cues - in gland histology
images. Our proposed algorithm, a deep multichannel framework, alleviates heavy
feature design due to the use of convolutional neural networks and is able to
meet multifarious requirements by altering channels. Results: Compared with
methods reported in the 2015 MICCAI Gland Segmentation Challenge and other
currently prevalent instance segmentation methods, we observe state-of-the-art
results based on the evaluation metrics. Conclusion: The proposed deep
multichannel algorithm is an effective method for gland instance segmentation.
Significance: The generalization ability of our model not only enable the
algorithm to solve gland instance segmentation problems, but the channel is
also alternative that can be replaced for a specific task.
IVE-GAN: Invariant Encoding Generative Adversarial Networks
Authors: Robin Winter, Djork-Arné Clevert
Date: 2017-11-23 00:00:00
Proceeding: ICLR 2018 1
Tasks: Image Generation

Generative adversarial networks (GANs) are a powerful framework for
generative tasks. However, they are difficult to train and tend to miss modes
of the true data generation process. Although GANs can learn a rich
representation of the covered modes of the data in their latent space, the
framework misses an inverse mapping from data to this latent space. We propose
Invariant Encoding Generative Adversarial Networks (IVE-GANs), a novel GAN
framework that introduces such a mapping for individual samples from the data
by utilizing features in the data which are invariant to certain
transformations. Since the model maps individual samples to the latent space,
it naturally encourages the generator to cover all modes. We demonstrate the
effectiveness of our approach in terms of generative performance and learning
rich representations on several datasets including common benchmark image
generation tasks.
BBQ-Networks: Efficient Exploration in Deep Reinforcement Learning for Task-Oriented Dialogue Systems
Authors: Zachary C. Lipton, Xiujun Li, Jianfeng Gao, Lihong Li, Faisal Ahmed, Li Deng
Date: 2016-08-17 00:00:00
Tasks: Deep Reinforcement Learning, Efficient Exploration, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Task-Oriented Dialogue Systems, Thompson Sampling

We present a new algorithm that significantly improves the efficiency of
exploration for deep Q-learning agents in dialogue systems. Our agents explore
via Thompson sampling, drawing Monte Carlo samples from a Bayes-by-Backprop
neural network. Our algorithm learns much faster than common exploration
strategies such as $\epsilon$-greedy, Boltzmann, bootstrapping, and
intrinsic-reward-based ones. Additionally, we show that spiking the replay
buffer with experiences from just a few successful episodes can make Q-learning
feasible when it might otherwise fail.
Estimating Mixed Memberships with Sharp Eigenvector Deviations
Authors: Xueyu Mao, Purnamrita Sarkar, Deepayan Chakrabarti
Date: 2017-09-01 00:00:00
Tasks: Community Detection

We consider the problem of estimating community memberships of nodes in a network, where every node is associated with a vector determining its degree of membership in each community. Existing provably consistent algorithms often require strong assumptions about the population, are computationally expensive, and only provide an overall error bound for the whole community membership matrix. This paper provides uniform rates of convergence for the inferred community membership vector of each node in a network generated from the Mixed Membership Stochastic Blockmodel (MMSB); to our knowledge, this is the first work to establish per-node rates for overlapping community detection in networks. We achieve this by establishing sharp row-wise eigenvector deviation bounds for MMSB. Based on the simplex structure inherent in the eigen-decomposition of the population matrix, we build on established corner-finding algorithms from the optimization community to infer the community membership vectors. Our results hold over a broad parameter regime where the average degree only grows poly-logarithmically with the number of nodes. Using experiments with simulated and real datasets, we show that our method achieves better error with lower variability over competing methods, and processes real world networks of up to 100,000 nodes within tens of seconds.
Minimum Energy Quantized Neural Networks
Authors: Bert Moons, Koen Goetschalckx, Nick Van Berckelaer, Marian Verhelst
Date: 2017-11-01 00:00:00

This work targets the automated minimum-energy optimization of Quantized
Neural Networks (QNNs) - networks using low precision weights and activations.
These networks are trained from scratch at an arbitrary fixed point precision.
At iso-accuracy, QNNs using fewer bits require deeper and wider network
architectures than networks using higher precision operators, while they
require less complex arithmetic and less bits per weights. This fundamental
trade-off is analyzed and quantified to find the minimum energy QNN for any
benchmark and hence optimize energy-efficiency. To this end, the energy
consumption of inference is modeled for a generic hardware platform. This
allows drawing several conclusions across different benchmarks. First, energy
consumption varies orders of magnitude at iso-accuracy depending on the number
of bits used in the QNN. Second, in a typical system, BinaryNets or int4
implementations lead to the minimum energy solution, outperforming int8
networks up to 2-10x at iso-accuracy. All code used for QNN training is
available from https://github.com/BertMoons.
Net-Trim: Convex Pruning of Deep Neural Networks with Performance Guarantee
Authors: Alireza Aghasi, Afshin Abdi, Nam Nguyen, Justin Romberg
Date: 2016-11-16 00:00:00
Proceeding: NeurIPS 2017 12

We introduce and analyze a new technique for model reduction for deep neural
networks. While large networks are theoretically capable of learning
arbitrarily complex models, overfitting and model redundancy negatively affects
the prediction accuracy and model variance. Our Net-Trim algorithm prunes
(sparsifies) a trained network layer-wise, removing connections at each layer
by solving a convex optimization program. This program seeks a sparse set of
weights at each layer that keeps the layer inputs and outputs consistent with
the originally trained model. The algorithms and associated analysis are
applicable to neural networks operating with the rectified linear unit (ReLU)
as the nonlinear activation. We present both parallel and cascade versions of
the algorithm. While the latter can achieve slightly simpler models with the
same generalization performance, the former can be computed in a distributed
manner. In both cases, Net-Trim significantly reduces the number of connections
in the network, while also providing enough regularization to slightly reduce
the generalization error. We also provide a mathematical analysis of the
consistency between the initial network and the retrained model. To analyze the
model sample complexity, we derive the general sufficient conditions for the
recovery of a sparse transform matrix. For a single layer taking independent
Gaussian random vectors of length $N$ as inputs, we show that if the network
response can be described using a maximum number of $s$ non-zero weights per
node, these weights can be learned from $\mathcal{O}(s\log N)$ samples.
Self-Reinforced Cascaded Regression for Face Alignment
Authors: Xin Fan, Risheng Liu, Kang Huyan, Yuyao Feng, Zhongxuan Luo
Date: 2017-11-23 00:00:00
Tasks: Face Alignment, Philosophy, regression

Cascaded regression is prevailing in face alignment thanks to its accuracy
and robustness, but typically demands manually annotated examples having low
discrepancy between shape-indexed features and shape updates. In this paper, we
propose a self-reinforced strategy that iteratively expands the quantity and
improves the quality of training examples, thus upgrading the performance of
cascaded regression itself. The reinforced term evaluates the example quality
upon the consistence on both local appearance and global geometry of human
faces, and constitutes the example evolution by the philosophy of "survival of
the fittest". We train a set of discriminative classifiers, each associated
with one landmark label, to prune those examples with inconsistent local
appearance, and further validate the geometric relationship among groups of
labeled landmarks against the common global geometry derived from a projective
invariant. We embed this generic strategy into typical cascaded regressions,
and the alignment results on several benchmark data sets demonstrate its
effectiveness to predict good examples starting from a small subset.
Improving the Accuracy of Pre-trained Word Embeddings for Sentiment Analysis
Authors: Seyed Mahdi Rezaeinia, Ali Ghodsi, Rouhollah Rahmani
Date: 2017-11-23 00:00:00
Tasks: Marketing, Part-Of-Speech Tagging, POS, POS Tagging, Sentiment Analysis, Sentiment Classification, Word Embeddings

Sentiment analysis is one of the well-known tasks and fast growing research
areas in natural language processing (NLP) and text classifications. This
technique has become an essential part of a wide range of applications
including politics, business, advertising and marketing. There are various
techniques for sentiment analysis, but recently word embeddings methods have
been widely used in sentiment classification tasks. Word2Vec and GloVe are
currently among the most accurate and usable word embedding methods which can
convert words into meaningful vectors. However, these methods ignore sentiment
information of texts and need a huge corpus of texts for training and
generating exact vectors which are used as inputs of deep learning models. As a
result, because of the small size of some corpuses, researcher often have to
use pre-trained word embeddings which were trained on other large text corpus
such as Google News with about 100 billion words. The increasing accuracy of
pre-trained word embeddings has a great impact on sentiment analysis research.
In this paper we propose a novel method, Improved Word Vectors (IWV), which
increases the accuracy of pre-trained word embeddings in sentiment analysis.
Our method is based on Part-of-Speech (POS) tagging techniques, lexicon-based
approaches and Word2Vec/GloVe methods. We tested the accuracy of our method via
different deep learning models and sentiment datasets. Our experiment results
show that Improved Word Vectors (IWV) are very effective for sentiment
analysis.
Indirect Supervision for Relation Extraction using Question-Answer Pairs
Authors: Zeqiu Wu, Xiang Ren, Frank F. Xu, Ji Li, Jiawei Han
Date: 2017-10-30 00:00:00
Tasks: Question Answering, Relation, Relation Extraction

Automatic relation extraction (RE) for types of interest is of great
importance for interpreting massive text corpora in an efficient manner.
Traditional RE models have heavily relied on human-annotated corpus for
training, which can be costly in generating labeled data and become obstacles
when dealing with more relation types. Thus, more RE extraction systems have
shifted to be built upon training data automatically acquired by linking to
knowledge bases (distant supervision). However, due to the incompleteness of
knowledge bases and the context-agnostic labeling, the training data collected
via distant supervision (DS) can be very noisy. In recent years, as increasing
attention has been brought to tackling question-answering (QA) tasks, user
feedback or datasets of such tasks become more accessible. In this paper, we
propose a novel framework, ReQuest, to leverage question-answer pairs as an
indirect source of supervision for relation extraction, and study how to use
such supervision to reduce noise induced from DS. Our model jointly embeds
relation mentions, types, QA entity mention pairs and text features in two
low-dimensional spaces (RE and QA), where objects with same relation types or
semantically similar question-answer pairs have similar representations. Shared
features connect these two spaces, carrying clearer semantic knowledge from
both sources. ReQuest, then use these learned embeddings to estimate the types
of test relation mentions. We formulate a global objective function and adopt a
novel margin-based QA loss to reduce noise in DS by exploiting semantic
evidence from the QA dataset. Our experimental results achieve an average of
11% improvement in F1 score on two public RE datasets combined with TREC QA
dataset.
An Improved Training Procedure for Neural Autoregressive Data Completion
Authors: Maxime Voisin, Daniel Ritchie
Date: 2017-11-23 00:00:00

Neural autoregressive models are explicit density estimators that achieve
state-of-the-art likelihoods for generative modeling. The D-dimensional data
distribution is factorized into an autoregressive product of one-dimensional
conditional distributions according to the chain rule. Data completion is a
more involved task than data generation: the model must infer missing variables
for any partially observed input vector. Previous work introduced an
order-agnostic training procedure for data completion with autoregressive
models. Missing variables in any partially observed input vector can be imputed
efficiently by choosing an ordering where observed dimensions precede
unobserved ones and by computing the autoregressive product in this order. In
this paper, we provide evidence that the order-agnostic (OA) training procedure
is suboptimal for data completion. We propose an alternative procedure (OA++)
that reaches better performance in fewer computations. It can handle all data
completion queries while training fewer one-dimensional conditional
distributions than the OA procedure. In addition, these one-dimensional
conditional distributions are trained proportionally to their expected usage at
inference time, reducing overfitting. Finally, our OA++ procedure can exploit
prior knowledge about the distribution of inference completion queries, as
opposed to OA. We support these claims with quantitative experiments on
standard datasets used to evaluate autoregressive generative models.
Online Clustering of Contextual Cascading Bandits
Authors: Shuai Li
Date: 2017-11-23 00:00:00
Tasks: Clustering, Online Clustering

We consider a new setting of online clustering of contextual cascading
bandits, an online learning problem where the underlying cluster structure over
users is unknown and needs to be learned from a random prefix feedback. More
precisely, a learning agent recommends an ordered list of items to a user, who
checks the list and stops at the first satisfactory item, if any. We propose an
algorithm of CLUB-cascade for this setting and prove a $T$-step regret bound of
order $\tilde{O}(\sqrt{T})$. Previous work corresponds to the degenerate case
of only one cluster, and our general regret bound in this special case also
significantly improves theirs. We conduct experiments on both synthetic and
real data, and demonstrate the effectiveness of our algorithm and the advantage
of incorporating online clustering method.
Regularization of Deep Neural Networks with Spectral Dropout
Authors: Salman Khan, Munawar Hayat, Fatih Porikli
Date: 2017-11-23 00:00:00

The big breakthrough on the ImageNet challenge in 2012 was partially due to
the `dropout' technique used to avoid overfitting. Here, we introduce a new
approach called `Spectral Dropout' to improve the generalization ability of
deep neural networks. We cast the proposed approach in the form of regular
Convolutional Neural Network (CNN) weight layers using a decorrelation
transform with fixed basis functions. Our spectral dropout method prevents
overfitting by eliminating weak and `noisy' Fourier domain coefficients of the
neural network activations, leading to remarkably better results than the
current regularization methods. Furthermore, the proposed is very efficient due
to the fixed basis functions used for spectral transformation. In particular,
compared to Dropout and Drop-Connect, our method significantly speeds up the
network convergence rate during the training process (roughly x2), with
considerably higher neuron pruning rates (an increase of ~ 30%). We demonstrate
that the spectral dropout can also be used in conjunction with other
regularization approaches resulting in additional performance gains.
End-to-End Supervised Product Quantization for Image Search and Retrieval
Authors: Benjamin Klein, Lior Wolf
Date: 2017-11-23 00:00:00
Proceeding: CVPR 2019 6
Tasks: General Classification, Image Retrieval, Quantization, Retrieval

Product Quantization, a dictionary based hashing method, is one of the leading unsupervised hashing techniques. While it ignores the labels, it harnesses the features to construct look up tables that can approximate the feature space. In recent years, several works have achieved state of the art results on hashing benchmarks by learning binary representations in a supervised manner. This work presents Deep Product Quantization (DPQ), a technique that leads to more accurate retrieval and classification than the latest state of the art methods, while having similar computational complexity and memory footprint as the Product Quantization method. To our knowledge, this is the first work to introduce a dictionary-based representation that is inspired by Product Quantization and which is learned end-to-end, and thus benefits from the supervised signal. DPQ explicitly learns soft and hard representations to enable an efficient and accurate asymmetric search, by using a straight-through estimator. Our method obtains state of the art results on an extensive array of retrieval and classification experiments.
Insensitive Stochastic Gradient Twin Support Vector Machine for Large Scale Problems
Authors: Zhen Wang, Yuan-Hai Shao, Lan Bai, Li-Ming Liu, Nai-Yang Deng
Date: 2017-04-19 00:00:00
Tasks: General Classification

Stochastic gradient descent algorithm has been successfully applied on
support vector machines (called PEGASOS) for many classification problems. In
this paper, stochastic gradient descent algorithm is investigated to twin
support vector machines for classification. Compared with PEGASOS, the proposed
stochastic gradient twin support vector machines (SGTSVM) is insensitive on
stochastic sampling for stochastic gradient descent algorithm. In theory, we
prove the convergence of SGTSVM instead of almost sure convergence of PEGASOS.
For uniformly sampling, the approximation between SGTSVM and twin support
vector machines is also given, while PEGASOS only has an opportunity to obtain
an approximation of support vector machines. In addition, the nonlinear SGTSVM
is derived directly from its linear case. Experimental results on both
artificial datasets and large scale problems show the stable performance of
SGTSVM with a fast learning speed.