pwc-archive/papers-with-abstracts huggingface dataset:
Gang of GANs: Generative Adversarial Networks with Maximum Margin Ranking
Authors: Felix Juefei-Xu, Vishnu Naresh Boddeti, Marios Savvides
Date: 2017-04-17 00:00:00

Traditional generative adversarial networks (GAN) and many of its variants
are trained by minimizing the KL or JS-divergence loss that measures how close
the generated data distribution is from the true data distribution. A recent
advance called the WGAN based on Wasserstein distance can improve on the KL and
JS-divergence based GANs, and alleviate the gradient vanishing, instability,
and mode collapse issues that are common in the GAN training. In this work, we
aim at improving on the WGAN by first generalizing its discriminator loss to a
margin-based one, which leads to a better discriminator, and in turn a better
generator, and then carrying out a progressive training paradigm involving
multiple GANs to contribute to the maximum margin ranking loss so that the GAN
at later stages will improve upon early stages. We call this method Gang of
GANs (GoGAN). We have shown theoretically that the proposed GoGAN can reduce
the gap between the true data distribution and the generated data distribution
by at least half in an optimally trained WGAN. We have also proposed a new way
of measuring GAN quality which is based on image completion tasks. We have
evaluated our method on four visual datasets: CelebA, LSUN Bedroom, CIFAR-10,
and 50K-SSFF, and have seen both visual and quantitative improvement over
baseline WGAN.
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Authors: Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, Hartwig Adam
Date: 2017-04-17 00:00:00
Tasks: geo-localization, Image Stitching, Object Detection

We present a class of efficient models called MobileNets for mobile and
embedded vision applications. MobileNets are based on a streamlined
architecture that uses depth-wise separable convolutions to build light weight
deep neural networks. We introduce two simple global hyper-parameters that
efficiently trade off between latency and accuracy. These hyper-parameters
allow the model builder to choose the right sized model for their application
based on the constraints of the problem. We present extensive experiments on
resource and accuracy tradeoffs and show strong performance compared to other
popular models on ImageNet classification. We then demonstrate the
effectiveness of MobileNets across a wide range of applications and use cases
including object detection, finegrain classification, face attributes and large
scale geo-localization.
A Neural Architecture for Generating Natural Language Descriptions from Source Code Changes
Authors: Pablo Loyola, Edison Marrese-Taylor, Yutaka Matsuo
Date: 2017-04-17 00:00:00
Proceeding: ACL 2017 7
Tasks: Decoder

We propose a model to automatically describe changes introduced in the source
code of a program using natural language. Our method receives as input a set of
code commits, which contains both the modifications and message introduced by
an user. These two modalities are used to train an encoder-decoder
architecture. We evaluated our approach on twelve real world open source
projects from four different programming languages. Quantitative and
qualitative results showed that the proposed approach can generate feasible and
semantically sound descriptions not only in standard in-project settings, but
also in a cross-project setting.
First Efficient Convergence for Streaming k-PCA: a Global, Gap-Free, and Near-Optimal Rate
Authors: Zeyuan Allen-Zhu, Yuanzhi Li
Date: 2016-07-26 00:00:00

We study streaming principal component analysis (PCA), that is to find, in
$O(dk)$ space, the top $k$ eigenvectors of a $d\times d$ hidden matrix $\bf
\Sigma$ with online vectors drawn from covariance matrix $\bf \Sigma$.
  We provide $\textit{global}$ convergence for Oja's algorithm which is
popularly used in practice but lacks theoretical understanding for $k>1$. We
also provide a modified variant $\mathsf{Oja}^{++}$ that runs $\textit{even
faster}$ than Oja's. Our results match the information theoretic lower bound in
terms of dependency on error, on eigengap, on rank $k$, and on dimension $d$,
up to poly-log factors. In addition, our convergence rate can be made gap-free,
that is proportional to the approximation error and independent of the
eigengap.
  In contrast, for general rank $k$, before our work (1) it was open to design
any algorithm with efficient global convergence rate; and (2) it was open to
design any algorithm with (even local) gap-free convergence rate in $O(dk)$
space.
Quantum Mechanical Approach to Modelling Reliability of Sensor Reports
Authors: Zichang He, Wen Jiang
Date: 2017-04-17 00:00:00

Dempster-Shafer evidence theory is wildly applied in multi-sensor data
fusion. However, lots of uncertainty and interference exist in practical
situation, especially in the battle field. It is still an open issue to model
the reliability of sensor reports. Many methods are proposed based on the
relationship among collected data. In this letter, we proposed a quantum
mechanical approach to evaluate the reliability of sensor reports, which is
based on the properties of a sensor itself. The proposed method is used to
modify the combining of evidences.
Mixture modeling on related samples by $ψ$-stick breaking and kernel perturbation
Authors: Jacopo Soriano, Li Ma
Date: 2017-04-17 00:00:00
Tasks: Bayesian Inference

There has been great interest recently in applying nonparametric kernel
mixtures in a hierarchical manner to model multiple related data samples
jointly. In such settings several data features are commonly present: (i) the
related samples often share some, if not all, of the mixture components but
with differing weights, (ii) only some, not all, of the mixture components vary
across the samples, and (iii) often the shared mixture components across
samples are not aligned perfectly in terms of their location and spread, but
rather display small misalignments either due to systematic cross-sample
difference or more often due to uncontrolled, extraneous causes. Properly
incorporating these features in mixture modeling will enhance the efficiency of
inference, whereas ignoring them not only reduces efficiency but can jeopardize
the validity of the inference due to issues such as confounding. We introduce
two techniques for incorporating these features in modeling related data
samples using kernel mixtures. The first technique, called $\psi$-stick
breaking, is a joint generative process for the mixing weights through the
breaking of both a stick shared by all the samples for the components that do
not vary in size across samples and an idiosyncratic stick for each sample for
those components that do vary in size. The second technique is to imbue random
perturbation into the kernels, thereby accounting for cross-sample
misalignment. These techniques can be used either separately or together in
both parametric and nonparametric kernel mixtures. We derive efficient Bayesian
inference recipes based on MCMC sampling for models featuring these techniques,
and illustrate their work through both simulated data and a real flow cytometry
data set in prediction/estimation, cross-sample calibration, and testing
multi-sample differences.
Boosting with Structural Sparsity: A Differential Inclusion Approach
Authors: Chendi Huang, Xinwei Sun, Jiechao Xiong, Yuan YAO
Date: 2017-04-16 00:00:00
Tasks: Denoising, Image Denoising, Model Selection

Boosting as gradient descent algorithms is one popular method in machine
learning. In this paper a novel Boosting-type algorithm is proposed based on
restricted gradient descent with structural sparsity control whose underlying
dynamics are governed by differential inclusions. In particular, we present an
iterative regularization path with structural sparsity where the parameter is
sparse under some linear transforms, based on variable splitting and the
Linearized Bregman Iteration. Hence it is called \emph{Split LBI}. Despite its
simplicity, Split LBI outperforms the popular generalized Lasso in both theory
and experiments. A theory of path consistency is presented that equipped with a
proper early stopping, Split LBI may achieve model selection consistency under
a family of Irrepresentable Conditions which can be weaker than the necessary
and sufficient condition for generalized Lasso. Furthermore, some $\ell_2$
error bounds are also given at the minimax optimal rates. The utility and
benefit of the algorithm are illustrated by several applications including
image denoising, partial order ranking of sport teams, and world university
grouping with crowdsourced ranking data.
CT Image Reconstruction in a Low Dimensional Manifold
Authors: Wenxiang Cong, Ge Wang, Qingsong Yang, Jiang Hsieh, Jia Li, Rongjie Lai
Date: 2017-04-16 00:00:00
Tasks: Image Reconstruction

Regularization methods are commonly used in X-ray CT image reconstruction.
Different regularization methods reflect the characterization of different
prior knowledge of images. In a recent work, a new regularization method called
a low-dimensional manifold model (LDMM) is investigated to characterize the
low-dimensional patch manifold structure of natural images, where the manifold
dimensionality characterizes structural information of an image. In this paper,
we propose a CT image reconstruction method based on the prior knowledge of the
low-dimensional manifold of CT image. Using the clinical raw projection data
from GE clinic, we conduct comparisons for the CT image reconstruction among
the proposed method, the simultaneous algebraic reconstruction technique (SART)
with the total variation (TV) regularization, and the filtered back projection
(FBP) method. Results show that the proposed method can successfully recover
structural details of an imaging object, and achieve higher spatial and
contrast resolution of the reconstructed image than counterparts of FBP and
SART with TV.
Synergy of all-purpose static solver and temporal reasoning tools in dynamic integrated expert systems
Authors: Galina Rybina, Alexey Mozgachev, Dmitry Demidov
Date: 2017-04-16 00:00:00
Tasks: All

The paper discusses scientific and technological problems of dynamic
integrated expert systems development. Extensions of problem-oriented
methodology for dynamic integrated expert systems development are considered.
Attention is paid to the temporal knowledge representation and processing.
A Novel Experimental Platform for In-Vessel Multi-Chemical Molecular Communications
Authors: Nariman Farsad, David Pan, Andrea Goldsmith
Date: 2017-04-16 00:00:00

This work presents a new multi-chemical experimental platform for molecular
communication where the transmitter can release different chemicals. This
platform is designed to be inexpensive and accessible, and it can be expanded
to simulate different environments including the cardiovascular system and
complex network of pipes in industrial complexes and city infrastructures. To
demonstrate the capabilities of the platform, we implement a time-slotted
binary communication system where a bit-0 is represented by an acid pulse, a
bit-1 by a base pulse, and information is carried via pH signals. The channel
model for this system, which is nonlinear and has long memories, is unknown.
Therefore, we devise novel detection algorithms that use techniques from
machine learning and deep learning to train a maximum-likelihood detector.
Using these algorithms the bit error rate improves by an order of magnitude
relative to the approach used in previous works. Moreover, our system achieves
a data rate that is an order of magnitude higher than any of the previous
molecular communication platforms.
Random Walk Sampling for Big Data over Networks
Authors: Saeed Basirian, Alexander Jung
Date: 2017-04-16 00:00:00

It has been shown recently that graph signals with small total variation can
be accurately recovered from only few samples if the sampling set satisfies a
certain condition, referred to as the network nullspace property. Based on this
recovery condition, we propose a sampling strategy for smooth graph signals
based on random walks. Numerical experiments demonstrate the effectiveness of
this approach for graph signals obtained from a synthetic random graph model as
well as a real-world dataset.
Harvesting Multiple Views for Marker-less 3D Human Pose Annotations
Authors: Georgios Pavlakos, Xiaowei Zhou, Konstantinos G. Derpanis, Kostas Daniilidis
Date: 2017-04-16 00:00:00
Proceeding: CVPR 2017 7
Tasks: 3D geometry, 3D Human Pose Estimation, Pose Prediction, Weakly-supervised 3D Human Pose Estimation

Recent advances with Convolutional Networks (ConvNets) have shifted the
bottleneck for many computer vision tasks to annotated data collection. In this
paper, we present a geometry-driven approach to automatically collect
annotations for human pose prediction tasks. Starting from a generic ConvNet
for 2D human pose, and assuming a multi-view setup, we describe an automatic
way to collect accurate 3D human pose annotations. We capitalize on constraints
offered by the 3D geometry of the camera setup and the 3D structure of the
human body to probabilistically combine per view 2D ConvNet predictions into a
globally optimal 3D pose. This 3D pose is used as the basis for harvesting
annotations. The benefit of the annotations produced automatically with our
approach is demonstrated in two challenging settings: (i) fine-tuning a generic
ConvNet-based 2D pose predictor to capture the discriminative aspects of a
subject's appearance (i.e.,"personalization"), and (ii) training a ConvNet from
scratch for single view 3D human pose prediction without leveraging 3D pose
groundtruth. The proposed multi-view pose estimator achieves state-of-the-art
results on standard benchmarks, demonstrating the effectiveness of our method
in exploiting the available multi-view information.
A Security Monitoring Framework For Virtualization Based HEP Infrastructures
Authors: A. Gomez Ramirez, M. Martinez Pedreira, C. Grigoras, L. Betev, C. Lara, U. Kebschull for the ALICE Collaboration
Date: 2017-04-16 00:00:00
Tasks: BIG-bench Machine Learning, CPU, Distributed Computing

High Energy Physics (HEP) distributed computing infrastructures require
automatic tools to monitor, analyze and react to potential security incidents.
These tools should collect and inspect data such as resource consumption, logs
and sequence of system calls for detecting anomalies that indicate the presence
of a malicious agent. They should also be able to perform automated reactions
to attacks without administrator intervention. We describe a novel framework
that accomplishes these requirements, with a proof of concept implementation
for the ALICE experiment at CERN. We show how we achieve a fully virtualized
environment that improves the security by isolating services and Jobs without a
significant performance impact. We also describe a collected dataset for
Machine Learning based Intrusion Prevention and Detection Systems on Grid
computing. This dataset is composed of resource consumption measurements (such
as CPU, RAM and network traffic), logfiles from operating system services, and
system call data collected from production Jobs running in an ALICE Grid test
site and a big set of malware. This malware was collected from security
research sites. Based on this dataset, we will proceed to develop Machine
Learning algorithms able to detect malicious Jobs.
A Hybrid ACO Algorithm for the Next Release Problem
Authors: He Jiang, Jingyuan Zhang, Jifeng Xuan, Zhilei Ren, Yan Hu
Date: 2017-04-16 00:00:00

In this paper, we propose a Hybrid Ant Colony Optimization algorithm (HACO)
for Next Release Problem (NRP). NRP, a NP-hard problem in requirement
engineering, is to balance customer requests, resource constraints, and
requirement dependencies by requirement selection. Inspired by the successes of
Ant Colony Optimization algorithms (ACO) for solving NP-hard problems, we
design our HACO to approximately solve NRP. Similar to traditional ACO
algorithms, multiple artificial ants are employed to construct new solutions.
During the solution construction phase, both pheromone trails and neighborhood
information will be taken to determine the choices of every ant. In addition, a
local search (first found hill climbing) is incorporated into HACO to improve
the solution quality. Extensively wide experiments on typical NRP test
instances show that HACO outperforms the existing algorithms (GRASP and
simulated annealing) in terms of both solution uality and running time.
Approximating the Backbone in the Weighted Maximum Satisfiability Problem
Authors: He Jiang, Jifeng Xuan, Yan Hu
Date: 2017-04-16 00:00:00

The weighted Maximum Satisfiability problem (weighted MAX-SAT) is a NP-hard
problem with numerous applications arising in artificial intelligence. As an
efficient tool for heuristic design, the backbone has been applied to
heuristics design for many NP-hard problems. In this paper, we investigated the
computational complexity for retrieving the backbone in weighted MAX-SAT and
developed a new algorithm for solving this problem. We showed that it is
intractable to retrieve the full backbone under the assumption that . Moreover,
it is intractable to retrieve a fixed fraction of the backbone as well. And
then we presented a backbone guided local search (BGLS) with Walksat operator
for weighted MAX-SAT. BGLS consists of two phases: the first phase samples the
backbone information from local optima and the backbone phase conducts local
search under the guideline of backbone. Extensive experimental results on the
benchmark showed that BGLS outperforms the existing heuristics in both solution
quality and runtime.
Average Stability is Invariant to Data Preconditioning. Implications to Exp-concave Empirical Risk Minimization
Authors: Alon Gonen, Shai Shalev-Shwartz
Date: 2016-01-15 00:00:00

We show that the average stability notion introduced by
\cite{kearns1999algorithmic, bousquet2002stability} is invariant to data
preconditioning, for a wide class of generalized linear models that includes
most of the known exp-concave losses. In other words, when analyzing the
stability rate of a given algorithm, we may assume the optimal preconditioning
of the data. This implies that, at least from a statistical perspective,
explicit regularization is not required in order to compensate for
ill-conditioned data, which stands in contrast to a widely common approach that
includes a regularization for analyzing the sample complexity of generalized
linear models. Several important implications of our findings include: a) We
demonstrate that the excess risk of empirical risk minimization (ERM) is
controlled by the preconditioned stability rate. This immediately yields a
relatively short and elegant proof for the fast rates attained by ERM in our
context. b) We strengthen the recent bounds of \cite{hardt2015train} on the
stability rate of the Stochastic Gradient Descent algorithm.
In-Datacenter Performance Analysis of a Tensor Processing Unit
Authors: Norman P. Jouppi, Cliff Young, Nishant Patil, David Patterson, Gaurav Agrawal, Raminder Bajwa, Sarah Bates, Suresh Bhatia, Nan Boden, Al Borchers, Rick Boyle, Pierre-luc Cantin, Clifford Chao, Chris Clark, Jeremy Coriell, Mike Daley, Matt Dau, Jeffrey Dean, Ben Gelb, Tara Vazir Ghaemmaghami, Rajendra Gottipati, William Gulland, Robert Hagmann, C. Richard Ho, Doug Hogberg, John Hu, Robert Hundt, Dan Hurt, Julian Ibarz, Aaron Jaffey, Alek Jaworski, Alexander Kaplan, Harshit Khaitan, Andy Koch, Naveen Kumar, Steve Lacy, James Laudon, James Law, Diemthu Le, Chris Leary, Zhuyuan Liu, Kyle Lucke, Alan Lundin, Gordon MacKean, Adriana Maggiore, Maire Mahony, Kieran Miller, Rahul Nagarajan, Ravi Narayanaswami, Ray Ni, Kathy Nix, Thomas Norrie, Mark Omernick, Narayana Penukonda, Andy Phelps, Jonathan Ross, Matt Ross, Amir Salek, Emad Samadiani, Chris Severn, Gregory Sizikov, Matthew Snelham, Jed Souter, Dan Steinberg, Andy Swing, Mercedes Tan, Gregory Thorson, Bo Tian, Horia Toma, Erick Tuttle, Vijay Vasudevan, Richard Walter, Walter Wang, Eric Wilcox, Doe Hyun Yoon
Date: 2017-04-16 00:00:00
Tasks: CPU, GPU

Many architects believe that major improvements in cost-energy-performance
must now come from domain-specific hardware. This paper evaluates a custom
ASIC---called a Tensor Processing Unit (TPU)---deployed in datacenters since
2015 that accelerates the inference phase of neural networks (NN). The heart of
the TPU is a 65,536 8-bit MAC matrix multiply unit that offers a peak
throughput of 92 TeraOps/second (TOPS) and a large (28 MiB) software-managed
on-chip memory. The TPU's deterministic execution model is a better match to
the 99th-percentile response-time requirement of our NN applications than are
the time-varying optimizations of CPUs and GPUs (caches, out-of-order
execution, multithreading, multiprocessing, prefetching, ...) that help average
throughput more than guaranteed latency. The lack of such features helps
explain why, despite having myriad MACs and a big memory, the TPU is relatively
small and low power. We compare the TPU to a server-class Intel Haswell CPU and
an Nvidia K80 GPU, which are contemporaries deployed in the same datacenters.
Our workload, written in the high-level TensorFlow framework, uses production
NN applications (MLPs, CNNs, and LSTMs) that represent 95% of our datacenters'
NN inference demand. Despite low utilization for some applications, the TPU is
on average about 15X - 30X faster than its contemporary GPU or CPU, with
TOPS/Watt about 30X - 80X higher. Moreover, using the GPU's GDDR5 memory in the
TPU would triple achieved TOPS and raise TOPS/Watt to nearly 70X the GPU and
200X the CPU.
AnchorNet: A Weakly Supervised Network to Learn Geometry-sensitive Features For Semantic Matching
Authors: David Novotny, Diane Larlus, Andrea Vedaldi
Date: 2017-04-16 00:00:00
Proceeding: CVPR 2017 7
Tasks: Object

Despite significant progress of deep learning in recent years,
state-of-the-art semantic matching methods still rely on legacy features such
as SIFT or HoG. We argue that the strong invariance properties that are key to
the success of recent deep architectures on the classification task make them
unfit for dense correspondence tasks, unless a large amount of supervision is
used. In this work, we propose a deep network, termed AnchorNet, that produces
image representations that are well-suited for semantic matching. It relies on
a set of filters whose response is geometrically consistent across different
object instances, even in the presence of strong intra-class, scale, or
viewpoint variations. Trained only with weak image-level labels, the final
representation successfully captures information about the object structure and
improves results of state-of-the-art semantic matching methods such as the
deformable spatial pyramid or the proposal flow methods. We show positive
results on the cross-instance matching task where different instances of the
same object category are matched as well as on a new cross-category semantic
matching task aligning pairs of instances each from a different object class.