pwc-archive/papers-with-abstracts huggingface dataset:
A General Neural Network Hardware Architecture on FPGA
Authors: Yufeng Hao
Date: 2017-11-06 00:00:00
Tasks: Autonomous Driving, BIG-bench Machine Learning

Field Programmable Gate Arrays (FPGAs) plays an increasingly important role
in data sampling and processing industries due to its highly parallel
architecture, low power consumption, and flexibility in custom algorithms.
Especially, in the artificial intelligence field, for training and implement
the neural networks and machine learning algorithms, high energy efficiency
hardware implement and massively parallel computing capacity are heavily
demanded. Therefore, many global companies have applied FPGAs into AI and
Machine learning fields such as autonomous driving and Automatic Spoken
Language Recognition (Baidu) [1] [2] and Bing search (Microsoft) [3].
Considering the FPGAs great potential in these fields, we tend to implement a
general neural network hardware architecture on XILINX ZU9CG System On Chip
(SOC) platform [4], which contains abundant hardware resource and powerful
processing capacity. The general neural network architecture on the FPGA SOC
platform can perform forward and backward algorithms in deep neural networks
(DNN) with high performance and easily be adjusted according to the type and
scale of the neural networks.
Hierarchical Surface Prediction for 3D Object Reconstruction
Authors: Christian Häne, Shubham Tulsiani, Jitendra Malik
Date: 2017-04-03 00:00:00
Tasks: 3D geometry, 3D Geometry Prediction, 3D Object Reconstruction, Object, Object Reconstruction, Prediction

Recently, Convolutional Neural Networks have shown promising results for 3D
geometry prediction. They can make predictions from very little input data such
as a single color image. A major limitation of such approaches is that they
only predict a coarse resolution voxel grid, which does not capture the surface
of the objects well. We propose a general framework, called hierarchical
surface prediction (HSP), which facilitates prediction of high resolution voxel
grids. The main insight is that it is sufficient to predict high resolution
voxels around the predicted surfaces. The exterior and interior of the objects
can be represented with coarse resolution voxels. Our approach is not dependent
on a specific input type. We show results for geometry prediction from color
images, depth images and shape completion from partial voxel grids. Our
analysis shows that our high resolution predictions are more accurate than low
resolution predictions.
Information-theoretic analysis of generalization capability of learning algorithms
Authors: Aolin Xu, Maxim Raginsky
Date: 2017-05-22 00:00:00
Proceeding: NeurIPS 2017 12

We derive upper bounds on the generalization error of a learning algorithm in
terms of the mutual information between its input and output. The bounds
provide an information-theoretic understanding of generalization in learning
problems, and give theoretical guidelines for striking the right balance
between data fit and generalization by controlling the input-output mutual
information. We propose a number of methods for this purpose, among which are
algorithms that regularize the ERM algorithm with relative entropy or with
random noise. Our work extends and leads to nontrivial improvements on the
recent results of Russo and Zou.
End-to-end Lung Nodule Detection in Computed Tomography
Authors: Dufan Wu, Kyungsang Kim, Bin Dong, Georges El Fakhri, Quanzheng Li
Date: 2017-11-06 00:00:00
Tasks: Computed Tomography (CT), Diagnostic, Lung Nodule Detection

Computer aided diagnostic (CAD) system is crucial for modern med-ical
imaging. But almost all CAD systems operate on reconstructed images, which were
optimized for radiologists. Computer vision can capture features that is subtle
to human observers, so it is desirable to design a CAD system op-erating on the
raw data. In this paper, we proposed a deep-neural-network-based detection
system for lung nodule detection in computed tomography (CT). A
primal-dual-type deep reconstruction network was applied first to convert the
raw data to the image space, followed by a 3-dimensional convolutional neural
network (3D-CNN) for the nodule detection. For efficient network training, the
deep reconstruction network and the CNN detector was trained sequentially
first, then followed by one epoch of end-to-end fine tuning. The method was
evaluated on the Lung Image Database Consortium image collection (LIDC-IDRI)
with simulated forward projections. With 144 multi-slice fanbeam pro-jections,
the proposed end-to-end detector could achieve comparable sensitivity with the
reference detector, which was trained and applied on the fully-sampled image
data. It also demonstrated superior detection performance compared to detectors
trained on the reconstructed images. The proposed method is general and could
be expanded to most detection tasks in medical imaging.
From Multimodal to Unimodal Webpages for Developing Countries
Authors: Vidyapu Sandeep, V Vijaya Saradhi, Samit Bhattacharya
Date: 2017-11-06 00:00:00

The multimodal web elements such as text and images are associated with
inherent memory costs to store and transfer over the Internet. With the limited
network connectivity in developing countries, webpage rendering gets delayed in
the presence of high-memory demanding elements such as images (relative to
text). To overcome this limitation, we propose a Canonical Correlation Analysis
(CCA) based computational approach to replace high-cost modality with an
equivalent low-cost modality. Our model learns a common subspace for low-cost
and high-cost modalities that maximizes the correlation between their visual
features. The obtained common subspace is used for determining the low-cost
(text) element of a given high-cost (image) element for the replacement. We
analyze the cost-saving performance of the proposed approach through an
eye-tracking experiment conducted on real-world webpages. Our approach reduces
the memory-cost by at least 83.35% by replacing images with text.
REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models
Authors: George Tucker, andriy mnih, Chris J. Maddison, Dieterich Lawson, Jascha Sohl-Dickstein
Date: 2017-03-21 00:00:00
Proceeding: NeurIPS 2017 12

Learning in models with discrete latent variables is challenging due to high
variance gradient estimators. Generally, approaches have relied on control
variates to reduce the variance of the REINFORCE estimator. Recent work (Jang
et al. 2016, Maddison et al. 2016) has taken a different approach, introducing
a continuous relaxation of discrete variables to produce low-variance, but
biased, gradient estimates. In this work, we combine the two approaches through
a novel control variate that produces low-variance, \emph{unbiased} gradient
estimates. Then, we introduce a modification to the continuous relaxation and
show that the tightness of the relaxation can be adapted online, removing it as
a hyperparameter. We show state-of-the-art variance reduction on several
benchmark generative modeling tasks, generally leading to faster convergence to
a better final log-likelihood.
An efficient quantum algorithm for generative machine learning
Authors: Xun Gao, Zhengyu Zhang, Lu-Ming Duan
Date: 2017-11-06 00:00:00
Tasks: BIG-bench Machine Learning, Quantum Machine Learning

A central task in the field of quantum computing is to find applications
where quantum computer could provide exponential speedup over any classical
computer. Machine learning represents an important field with broad
applications where quantum computer may offer significant speedup. Several
quantum algorithms for discriminative machine learning have been found based on
efficient solving of linear algebraic problems, with potential exponential
speedup in runtime under the assumption of effective input from a quantum
random access memory. In machine learning, generative models represent another
large class which is widely used for both supervised and unsupervised learning.
Here, we propose an efficient quantum algorithm for machine learning based on a
quantum generative model. We prove that our proposed model is exponentially
more powerful to represent probability distributions compared with classical
generative models and has exponential speedup in training and inference at
least for some instances under a reasonable assumption in computational
complexity theory. Our result opens a new direction for quantum machine
learning and offers a remarkable example in which a quantum algorithm shows
exponential improvement over any classical algorithm in an important
application field.
Maximum Entropy Distributions: Bit Complexity and Stability
Authors: Damian Straszak, Nisheeth K. Vishnoi
Date: 2017-11-06 00:00:00

Maximum entropy distributions with discrete support in $m$ dimensions arise in machine learning, statistics, information theory, and theoretical computer science. While structural and computational properties of max-entropy distributions have been extensively studied, basic questions such as: Do max-entropy distributions over a large support (e.g., $2^m$) with a specified marginal vector have succinct descriptions (polynomial-size in the input description)? and: Are entropy maximizing distributions "stable" under the perturbation of the marginal vector? have resisted a rigorous resolution. Here we show that these questions are related and resolve both of them. Our main result shows a ${\rm poly}(m, \log 1/\varepsilon)$ bound on the bit complexity of $\varepsilon$-optimal dual solutions to the maximum entropy convex program -- for very general support sets and with no restriction on the marginal vector. Applications of this result include polynomial time algorithms to compute max-entropy distributions over several new and old polytopes for any marginal vector in a unified manner, a polynomial time algorithm to compute the Brascamp-Lieb constant in the rank-1 case. The proof of this result allows us to show that changing the marginal vector by $\delta$ changes the max-entropy distribution in the total variation distance roughly by a factor of ${\rm poly}(m, \log 1/\delta)\sqrt{\delta}$ -- even when the size of the support set is exponential. Together, our results put max-entropy distributions on a mathematically sound footing -- these distributions are robust and computationally feasible models for data.
Estimating Cosmological Parameters from the Dark Matter Distribution
Authors: Siamak Ravanbakhsh, Junier Oliva, Sebastien Fromenteau, Layne C. Price, Shirley Ho, Jeff Schneider, Barnabas Poczos
Date: 2017-11-06 00:00:00

A grand challenge of the 21st century cosmology is to accurately estimate the
cosmological parameters of our Universe. A major approach to estimating the
cosmological parameters is to use the large-scale matter distribution of the
Universe. Galaxy surveys provide the means to map out cosmic large-scale
structure in three dimensions. Information about galaxy locations is typically
summarized in a "single" function of scale, such as the galaxy correlation
function or power-spectrum. We show that it is possible to estimate these
cosmological parameters directly from the distribution of matter. This paper
presents the application of deep 3D convolutional networks to volumetric
representation of dark-matter simulations as well as the results obtained using
a recently proposed distribution regression framework, showing that machine
learning techniques are comparable to, and can sometimes outperform,
maximum-likelihood point estimates using "cosmological models". This opens the
way to estimating the parameters of our Universe with higher accuracy.
Accelerated Parallel Non-conjugate Sampling for Bayesian Non-parametric Models
Authors: Michael Minyi Zhang, Sinead A. Williamson, Fernando Perez-Cruz
Date: 2017-05-19 00:00:00
Tasks: Bayesian Inference, valid

Inference of latent feature models in the Bayesian nonparametric setting is generally difficult, especially in high dimensional settings, because it usually requires proposing features from some prior distribution. In special cases, where the integration is tractable, we can sample new feature assignments according to a predictive likelihood. We present a novel method to accelerate the mixing of latent variable model inference by proposing feature locations based on the data, as opposed to the prior. First, we introduce an accelerated feature proposal mechanism that we show is a valid MCMC algorithm for posterior inference. Next, we propose an approximate inference strategy to perform accelerated inference in parallel. A two-stage algorithm that combines the two approaches provides a computationally attractive method that can quickly reach local convergence to the posterior distribution of our model, while allowing us to exploit parallelization.
Artificial Generation of Big Data for Improving Image Classification: A Generative Adversarial Network Approach on SAR Data
Authors: Dimitrios Marmanis, Wei Yao, Fathalrahman Adam, Mihai Datcu, Peter Reinartz, Konrad Schindler, Jan Dirk Wegner, Uwe Stilla
Date: 2017-11-06 00:00:00
Tasks: Classification, General Classification, Generative Adversarial Network, image-classification, Image Classification, Image Generation

Very High Spatial Resolution (VHSR) large-scale SAR image databases are still
an unresolved issue in the Remote Sensing field. In this work, we propose such
a dataset and use it to explore patch-based classification in urban and
periurban areas, considering 7 distinct semantic classes. In this context, we
investigate the accuracy of large CNN classification models and pre-trained
networks for SAR imaging systems. Furthermore, we propose a Generative
Adversarial Network (GAN) for SAR image generation and test, whether the
synthetic data can actually improve classification accuracy.
Beyond normality: Learning sparse probabilistic graphical models in the non-Gaussian setting
Authors: Rebecca E. Morrison, Ricardo Baptista, Youssef Marzouk
Date: 2017-11-02 00:00:00
Proceeding: NeurIPS 2017 12

We present an algorithm to identify sparse dependence structure in continuous
and non-Gaussian probability distributions, given a corresponding set of data.
The conditional independence structure of an arbitrary distribution can be
represented as an undirected graph (or Markov random field), but most
algorithms for learning this structure are restricted to the discrete or
Gaussian cases. Our new approach allows for more realistic and accurate
descriptions of the distribution in question, and in turn better estimates of
its sparse Markov structure. Sparsity in the graph is of interest as it can
accelerate inference, improve sampling methods, and reveal important
dependencies between variables. The algorithm relies on exploiting the
connection between the sparsity of the graph and the sparsity of transport
maps, which deterministically couple one probability measure to another.
PersonRank: Detecting Important People in Images
Authors: Wei-Hong Li, Benchao Li, Wei-Shi Zheng
Date: 2017-11-06 00:00:00

Always, some individuals in images are more important/attractive than others
in some events such as presentation, basketball game or speech. However, it is
challenging to find important people among all individuals in images directly
based on their spatial or appearance information due to the existence of
diverse variations of pose, action, appearance of persons and various changes
of occasions. We overcome this difficulty by constructing a multiple
Hyper-Interaction Graph to treat each individual in an image as a node and
inferring the most active node referring to interactions estimated by various
types of clews. We model pairwise interactions between persons as the edge
message communicated between nodes, resulting in a bidirectional
pairwise-interaction graph. To enrich the personperson interaction estimation,
we further introduce a unidirectional hyper-interaction graph that models the
consensus of interaction between a focal person and any person in a local
region around. Finally, we modify the PageRank algorithm to infer the
activeness of persons on the multiple Hybrid-Interaction Graph (HIG), the union
of the pairwise-interaction and hyperinteraction graphs, and we call our
algorithm the PersonRank. In order to provide publicable datasets for
evaluation, we have contributed a new dataset called Multi-scene Important
People Image Dataset and gathered a NCAA Basketball Image Dataset from sports
game sequences. We have demonstrated that the proposed PersonRank outperforms
related methods clearly and substantially.
Speaking the Same Language: Matching Machine to Human Captions by Adversarial Training
Authors: Rakshith Shetty, Marcus Rohrbach, Lisa Anne Hendricks, Mario Fritz, Bernt Schiele
Date: 2017-03-30 00:00:00
Proceeding: ICCV 2017 10
Tasks: Image Captioning

While strong progress has been made in image captioning over the last years,
machine and human captions are still quite distinct. A closer look reveals that
this is due to the deficiencies in the generated word distribution, vocabulary
size, and strong bias in the generators towards frequent captions. Furthermore,
humans -- rightfully so -- generate multiple, diverse captions, due to the
inherent ambiguity in the captioning task which is not considered in today's
systems.
  To address these challenges, we change the training objective of the caption
generator from reproducing groundtruth captions to generating a set of captions
that is indistinguishable from human generated captions. Instead of
handcrafting such a learning target, we employ adversarial training in
combination with an approximate Gumbel sampler to implicitly match the
generated distribution to the human one. While our method achieves comparable
performance to the state-of-the-art in terms of the correctness of the
captions, we generate a set of diverse captions, that are significantly less
biased and match the word statistics better in several aspects.
Style Transfer from Non-Parallel Text by Cross-Alignment
Authors: Tianxiao Shen, Tao Lei, Regina Barzilay, Tommi Jaakkola
Date: 2017-05-26 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Decipherment, Machine Translation, Style Transfer, Text Style Transfer, Translation

This paper focuses on style transfer on the basis of non-parallel text. This
is an instance of a broad family of problems including machine translation,
decipherment, and sentiment modification. The key challenge is to separate the
content from other aspects such as style. We assume a shared latent content
distribution across different text corpora, and propose a method that leverages
refined alignment of latent representations to perform style transfer. The
transferred sentences from one style should match example sentences from the
other style as a population. We demonstrate the effectiveness of this
cross-alignment method on three tasks: sentiment modification, decipherment of
word substitution ciphers, and recovery of word order.
It Takes (Only) Two: Adversarial Generator-Encoder Networks
Authors: Dmitry Ulyanov, Andrea Vedaldi, Victor Lempitsky
Date: 2017-04-07 00:00:00
Tasks: Vocal Bursts Valence Prediction

We present a new autoencoder-type architecture that is trainable in an
unsupervised mode, sustains both generation and inference, and has the quality
of conditional and unconditional samples boosted by adversarial learning.
Unlike previous hybrids of autoencoders and adversarial networks, the
adversarial game in our approach is set up directly between the encoder and the
generator, and no external mappings are trained in the process of learning. The
game objective compares the divergences of each of the real and the generated
data distributions with the prior distribution in the latent space. We show
that direct generator-vs-encoder game leads to a tight coupling of the two
components, resulting in samples and reconstructions of a comparable quality to
some recently-proposed more complex architectures.
Revisiting Perceptron: Efficient and Label-Optimal Learning of Halfspaces
Authors: Songbai Yan, Chicheng Zhang
Date: 2017-02-18 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Active Learning

It has been a long-standing problem to efficiently learn a halfspace using as
few labels as possible in the presence of noise. In this work, we propose an
efficient Perceptron-based algorithm for actively learning homogeneous
halfspaces under the uniform distribution over the unit sphere. Under the
bounded noise condition~\cite{MN06}, where each label is flipped with
probability at most $\eta < \frac 1 2$, our algorithm achieves a near-optimal
label complexity of
$\tilde{O}\left(\frac{d}{(1-2\eta)^2}\ln\frac{1}{\epsilon}\right)$ in time
$\tilde{O}\left(\frac{d^2}{\epsilon(1-2\eta)^3}\right)$. Under the adversarial
noise condition~\cite{ABL14, KLS09, KKMS08}, where at most a $\tilde
\Omega(\epsilon)$ fraction of labels can be flipped, our algorithm achieves a
near-optimal label complexity of $\tilde{O}\left(d\ln\frac{1}{\epsilon}\right)$
in time $\tilde{O}\left(\frac{d^2}{\epsilon}\right)$. Furthermore, we show that
our active learning algorithm can be converted to an efficient passive learning
algorithm that has near-optimal sample complexities with respect to $\epsilon$
and $d$.
Learning Multiple Tasks with Multilinear Relationship Networks
Authors: Mingsheng Long, Zhangjie Cao, Jian-Min Wang, Philip S. Yu
Date: 2015-06-06 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Multi-Task Learning

Deep networks trained on large-scale data can learn transferable features to
promote learning multiple tasks. Since deep features eventually transition from
general to specific along deep networks, a fundamental problem of multi-task
learning is how to exploit the task relatedness underlying parameter tensors
and improve feature transferability in the multiple task-specific layers. This
paper presents Multilinear Relationship Networks (MRN) that discover the task
relationships based on novel tensor normal priors over parameter tensors of
multiple task-specific layers in deep convolutional networks. By jointly
learning transferable features and multilinear relationships of tasks and
features, MRN is able to alleviate the dilemma of negative-transfer in the
feature layers and under-transfer in the classifier layer. Experiments show
that MRN yields state-of-the-art results on three multi-task learning datasets.
Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesis
Authors: Dmitry Ulyanov, Andrea Vedaldi, Victor Lempitsky
Date: 2017-01-09 00:00:00
Proceeding: CVPR 2017 7
Tasks: Diversity, Image Generation, Image Stylization, Texture Synthesis

The recent work of Gatys et al., who characterized the style of an image by
the statistics of convolutional neural network filters, ignited a renewed
interest in the texture generation and image stylization problems. While their
image generation technique uses a slow optimization process, recently several
authors have proposed to learn generator neural networks that can produce
similar outputs in one quick forward pass. While generator networks are
promising, they are still inferior in visual quality and diversity compared to
generation-by-optimization. In this work, we advance them in two significant
ways. First, we introduce an instance normalization module to replace batch
normalization with significant improvements to the quality of image
stylization. Second, we improve diversity by introducing a new learning
formulation that encourages generators to sample unbiasedly from the Julesz
texture ensemble, which is the equivalence class of all images characterized by
certain filter responses. Together, these two improvements take feed forward
texture synthesis and image stylization much closer to the quality of
generation-via-optimization, while retaining the speed advantage.