pwc-archive/papers-with-abstracts huggingface dataset:
A Semantic Model for Historical Manuscripts
Authors: Sahar Aljalbout, Gilles Falquet
Date: 2018-01-31 00:00:00
Tasks: model

The study and publication of historical scientific manuscripts are com- plex
tasks that involve, among others, the explicit representation of the text mean-
ings and reasoning on temporal entities. In this paper we present the first
results of an interdisciplinary project dedicated to the study of Saussure's
manuscripts. These results aim to fulfill requirements elaborated with
Saussurean humanists. They comprise a model for the representation of
time-varying statements and time-varying domain knowledge (in particular
terminologies) as well as imple- mentation techniques for the semantic indexing
of manuscripts and for temporal reasoning on knowledge extracted from the
manuscripts.
A Comprehensive Survey of Graph Embedding: Problems, Techniques and Applications
Authors: Hongyun Cai, Vincent W. Zheng, Kevin Chen-Chuan Chang
Date: 2017-09-22 00:00:00
Tasks: Diversity, Graph Embedding, Link Prediction, Node Classification, Survey

Graph is an important data representation which appears in a wide diversity
of real-world scenarios. Effective graph analytics provides users a deeper
understanding of what is behind the data, and thus can benefit a lot of useful
applications such as node classification, node recommendation, link prediction,
etc. However, most graph analytics methods suffer the high computation and
space cost. Graph embedding is an effective yet efficient way to solve the
graph analytics problem. It converts the graph data into a low dimensional
space in which the graph structural information and graph properties are
maximally preserved. In this survey, we conduct a comprehensive review of the
literature in graph embedding. We first introduce the formal definition of
graph embedding as well as the related concepts. After that, we propose two
taxonomies of graph embedding which correspond to what challenges exist in
different graph embedding problem settings and how the existing work address
these challenges in their solutions. Finally, we summarize the applications
that graph embedding enables and suggest four promising future research
directions in terms of computation efficiency, problem settings, techniques and
application scenarios.
An Instability in Variational Inference for Topic Models
Authors: Behrooz Ghorbani, Hamid Javadi, Andrea Montanari
Date: 2018-02-02 00:00:00
Tasks: Articles, Topic Models, Variational Inference

Topic models are Bayesian models that are frequently used to capture the
latent structure of certain corpora of documents or images. Each data element
in such a corpus (for instance each item in a collection of scientific
articles) is regarded as a convex combination of a small number of vectors
corresponding to `topics' or `components'. The weights are assumed to have a
Dirichlet prior distribution. The standard approach towards approximating the
posterior is to use variational inference algorithms, and in particular a mean
field approximation.
  We show that this approach suffers from an instability that can produce
misleading conclusions. Namely, for certain regimes of the model parameters,
variational inference outputs a non-trivial decomposition into topics. However
--for the same parameter values-- the data contain no actual information about
the true decomposition, and hence the output of the algorithm is uncorrelated
with the true topic decomposition. Among other consequences, the estimated
posterior mean is significantly wrong, and estimated Bayesian credible regions
do not achieve the nominal coverage. We discuss how this instability is
remedied by more accurate mean field approximations.
Interpretable Deep Convolutional Neural Networks via Meta-learning
Authors: Xuan Liu, Xiaoguang Wang, Stan Matwin
Date: 2018-02-02 00:00:00
Tasks: Clustering, Fairness, Meta-Learning

Model interpretability is a requirement in many applications in which crucial
decisions are made by users relying on a model's outputs. The recent movement
for "algorithmic fairness" also stipulates explainability, and therefore
interpretability of learning models. And yet the most successful contemporary
Machine Learning approaches, the Deep Neural Networks, produce models that are
highly non-interpretable. We attempt to address this challenge by proposing a
technique called CNN-INTE to interpret deep Convolutional Neural Networks (CNN)
via meta-learning. In this work, we interpret a specific hidden layer of the
deep CNN model on the MNIST image dataset. We use a clustering algorithm in a
two-level structure to find the meta-level training data and Random Forest as
base learning algorithms to generate the meta-level test data. The
interpretation results are displayed visually via diagrams, which clearly
indicates how a specific test instance is classified. Our method achieves
global interpretation for all the test instances without sacrificing the
accuracy obtained by the original deep CNN model. This means our model is
faithful to the deep CNN model, which leads to reliable interpretations.
Road Damage Detection Using Deep Neural Networks with Images Captured Through a Smartphone
Authors: Hiroya Maeda, Yoshihide Sekimoto, Toshikazu Seto, Takehiro Kashiyama, Hiroshi Omata
Date: 2018-01-29 00:00:00
Tasks: GPU, object-detection, Object Detection, Road Damage Detection

Research on damage detection of road surfaces using image processing
techniques has been actively conducted, achieving considerably high detection
accuracies. Many studies only focus on the detection of the presence or absence
of damage. However, in a real-world scenario, when the road managers from a
governing body need to repair such damage, they need to clearly understand the
type of damage in order to take effective action. In addition, in many of these
previous studies, the researchers acquire their own data using different
methods. Hence, there is no uniform road damage dataset available openly,
leading to the absence of a benchmark for road damage detection. This study
makes three contributions to address these issues. First, to the best of our
knowledge, for the first time, a large-scale road damage dataset is prepared.
This dataset is composed of 9,053 road damage images captured with a smartphone
installed on a car, with 15,435 instances of road surface damage included in
these road images. In order to generate this dataset, we cooperated with 7
municipalities in Japan and acquired road images for more than 40 hours. These
images were captured in a wide variety of weather and illuminance conditions.
In each image, we annotated the bounding box representing the location and type
of damage. Next, we used a state-of-the-art object detection method using
convolutional neural networks to train the damage detection model with our
dataset, and compared the accuracy and runtime speed on both, using a GPU
server and a smartphone. Finally, we demonstrate that the type of damage can be
classified into eight types with high accuracy by applying the proposed object
detection method. The road damage dataset, our experimental results, and the
developed smartphone application used in this study are publicly available
(https://github.com/sekilab/RoadDamageDetector/).
ExpNet: Landmark-Free, Deep, 3D Facial Expressions
Authors: Feng-Ju Chang, Anh Tuan Tran, Tal Hassner, Iacopo Masi, Ram Nevatia, Gerard Medioni
Date: 2018-02-02 00:00:00
Tasks: 3D Face Reconstruction, 3D Facial Expression Recognition, Emotion Recognition, Facial Landmark Detection

We describe a deep learning based method for estimating 3D facial expression
coefficients. Unlike previous work, our process does not relay on facial
landmark detection methods as a proxy step. Recent methods have shown that a
CNN can be trained to regress accurate and discriminative 3D morphable model
(3DMM) representations, directly from image intensities. By foregoing facial
landmark detection, these methods were able to estimate shapes for occluded
faces appearing in unprecedented in-the-wild viewing conditions. We build on
those methods by showing that facial expressions can also be estimated by a
robust, deep, landmark-free approach. Our ExpNet CNN is applied directly to the
intensities of a face image and regresses a 29D vector of 3D expression
coefficients. We propose a unique method for collecting data to train this
network, leveraging on the robustness of deep networks to training label noise.
We further offer a novel means of evaluating the accuracy of estimated
expression coefficients: by measuring how well they capture facial emotions on
the CK+ and EmotiW-17 emotion recognition benchmarks. We show that our ExpNet
produces expression coefficients which better discriminate between facial
emotions than those obtained using state of the art, facial landmark detection
techniques. Moreover, this advantage grows as image scales drop, demonstrating
that our ExpNet is more robust to scale changes than landmark detection
methods. Finally, at the same level of accuracy, our ExpNet is orders of
magnitude faster than its alternatives.
Causal Learning and Explanation of Deep Neural Networks via Autoencoded Activations
Authors: Michael Harradon, Jeff Druce, Brian Ruttenberg
Date: 2018-02-02 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification

Deep neural networks are complex and opaque. As they enter application in a
variety of important and safety critical domains, users seek methods to explain
their output predictions. We develop an approach to explaining deep neural
networks by constructing causal models on salient concepts contained in a CNN.
We develop methods to extract salient concepts throughout a target network by
using autoencoders trained to extract human-understandable representations of
network activations. We then build a bayesian causal model using these
extracted concepts as variables in order to explain image classification.
Finally, we use this causal model to identify and visualize features with
significant causal influence on final classification.
DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
Authors: Shuchang Zhou, Yuxin Wu, Zekun Ni, Xinyu Zhou, He Wen, Yuheng Zou
Date: 2016-06-20 00:00:00
Tasks: CPU, GPU, Quantization

We propose DoReFa-Net, a method to train convolutional neural networks that
have low bitwidth weights and activations using low bitwidth parameter
gradients. In particular, during backward pass, parameter gradients are
stochastically quantized to low bitwidth numbers before being propagated to
convolutional layers. As convolutions during forward/backward passes can now
operate on low bitwidth weights and activations/gradients respectively,
DoReFa-Net can use bit convolution kernels to accelerate both training and
inference. Moreover, as bit convolutions can be efficiently implemented on CPU,
FPGA, ASIC and GPU, DoReFa-Net opens the way to accelerate training of low
bitwidth neural network on these hardware. Our experiments on SVHN and ImageNet
datasets prove that DoReFa-Net can achieve comparable prediction accuracy as
32-bit counterparts. For example, a DoReFa-Net derived from AlexNet that has
1-bit weights, 2-bit activations, can be trained from scratch using 6-bit
gradients to get 46.1\% top-1 accuracy on ImageNet validation set. The
DoReFa-Net AlexNet model is released publicly.
Scalable Lévy Process Priors for Spectral Kernel Learning
Authors: Phillip A. Jang, Andrew E. Loeb, Matthew B. Davidow, Andrew Gordon Wilson
Date: 2018-02-02 00:00:00
Tasks: Gaussian Processes

Gaussian processes are rich distributions over functions, with generalization
properties determined by a kernel function. When used for long-range
extrapolation, predictions are particularly sensitive to the choice of kernel
parameters. It is therefore critical to account for kernel uncertainty in our
predictive distributions. We propose a distribution over kernels formed by
modelling a spectral mixture density with a L\'evy process. The resulting
distribution has support for all stationary covariances--including the popular
RBF, periodic, and Mat\'ern kernels--combined with inductive biases which
enable automatic and data efficient learning, long-range extrapolation, and
state of the art predictive performance. The proposed model also presents an
approach to spectral regularization, as the L\'evy process introduces a
sparsity-inducing prior over mixture components, allowing automatic selection
over model order and pruning of extraneous components. We exploit the algebraic
structure of the proposed process for $\mathcal{O}(n)$ training and
$\mathcal{O}(1)$ predictions. We perform extrapolations having reasonable
uncertainty estimates on several benchmarks, show that the proposed model can
recover flexible ground truth covariances and that it is robust to errors in
initialization.
Mobile Machine Learning Hardware at ARM: A Systems-on-Chip (SoC) Perspective
Authors: Yuhao Zhu, Matthew Mattina, Paul Whatmough
Date: 2018-01-19 00:00:00
Tasks: BIG-bench Machine Learning

Machine learning is playing an increasingly significant role in emerging
mobile application domains such as AR/VR, ADAS, etc. Accordingly, hardware
architects have designed customized hardware for machine learning algorithms,
especially neural networks, to improve compute efficiency. However, machine
learning is typically just one processing stage in complex end-to-end
applications, involving multiple components in a mobile Systems-on-a-chip
(SoC). Focusing only on ML accelerators loses bigger optimization opportunity
at the system (SoC) level. This paper argues that hardware architects should
expand the optimization scope to the entire SoC. We demonstrate one particular
case-study in the domain of continuous computer vision where camera sensor,
image signal processor (ISP), memory, and NN accelerator are synergistically
co-designed to achieve optimal system-level efficiency.
Analysis of Fast Alternating Minimization for Structured Dictionary Learning
Authors: Saiprasad Ravishankar, Anna Ma, Deanna Needell
Date: 2018-02-01 00:00:00
Tasks: Denoising, Dictionary Learning, Operator learning

Methods exploiting sparsity have been popular in imaging and signal
processing applications including compression, denoising, and imaging inverse
problems. Data-driven approaches such as dictionary learning and transform
learning enable one to discover complex image features from datasets and
provide promising performance over analytical models. Alternating minimization
algorithms have been particularly popular in dictionary or transform learning.
In this work, we study the properties of alternating minimization for
structured (unitary) sparsifying operator learning. While the algorithm
converges to the stationary points of the non-convex problem in general, we
prove rapid local linear convergence to the underlying generative model under
mild assumptions. Our experiments show that the unitary operator learning
algorithm is robust to initialization.
Reconstructing Video from Interferometric Measurements of Time-Varying Sources
Authors: Katherine L. Bouman, Michael D. Johnson, Adrian V. Dalca, Andrew A. Chael, Freek Roelofs, Sheperd S. Doeleman, William T. Freeman
Date: 2017-11-03 00:00:00
Tasks: Image Imputation, Radio Interferometry

Very long baseline interferometry (VLBI) makes it possible to recover images
of astronomical sources with extremely high angular resolution. Most recently,
the Event Horizon Telescope (EHT) has extended VLBI to short millimeter
wavelengths with a goal of achieving angular resolution sufficient for imaging
the event horizons of nearby supermassive black holes. VLBI provides
measurements related to the underlying source image through a sparse set
spatial frequencies. An image can then be recovered from these measurements by
making assumptions about the underlying image. One of the most important
assumptions made by conventional imaging methods is that over the course of a
night's observation the image is static. However, for quickly evolving sources,
such as the galactic center's supermassive black hole (Sgr A*) targeted by the
EHT, this assumption is violated and these conventional imaging approaches
fail. In this work we propose a new way to model VLBI measurements that allows
us to recover both the appearance and dynamics of an evolving source by
reconstructing a video rather than a static image. By modeling VLBI
measurements using a Gaussian Markov Model, we are able to propagate
information across observations in time to reconstruct a video, while
simultaneously learning about the dynamics of the source's emission region. We
demonstrate our proposed Expectation-Maximization (EM) algorithm, StarWarps, on
realistic synthetic observations of black holes, and show how it substantially
improves results compared to conventional imaging algorithms. Additionally, we
demonstrate StarWarps on real VLBI data of the M87 Jet from the VLBA.
Fast Incremental SVDD Learning Algorithm with the Gaussian Kernel
Authors: Hansi Jiang, Haoyu Wang, Wenhao Hu, Deovrat Kakde, Arin Chaudhuri
Date: 2017-09-01 00:00:00
Tasks: Incremental Learning, Outlier Detection

Support vector data description (SVDD) is a machine learning technique that
is used for single-class classification and outlier detection. The idea of SVDD
is to find a set of support vectors that defines a boundary around data. When
dealing with online or large data, existing batch SVDD methods have to be rerun
in each iteration. We propose an incremental learning algorithm for SVDD that
uses the Gaussian kernel. This algorithm builds on the observation that all
support vectors on the boundary have the same distance to the center of sphere
in a higher-dimensional feature space as mapped by the Gaussian kernel
function. Each iteration involves only the existing support vectors and the new
data point. Moreover, the algorithm is based solely on matrix manipulations;
the support vectors and their corresponding Lagrange multiplier $\alpha_i$'s
are automatically selected and determined in each iteration. It can be seen
that the complexity of our algorithm in each iteration is only $O(k^2)$, where
$k$ is the number of support vectors. Experimental results on some real data
sets indicate that FISVDD demonstrates significant gains in efficiency with
almost no loss in either outlier detection accuracy or objective function
value.
Multi-level Residual Networks from Dynamical Systems View
Authors: Bo Chang, Lili Meng, Eldad Haber, Frederick Tung, David Begert
Date: 2017-10-27 00:00:00
Proceeding: ICLR 2018 1
Tasks: General Classification, image-classification, Image Classification

Deep residual networks (ResNets) and their variants are widely used in many
computer vision applications and natural language processing tasks. However,
the theoretical principles for designing and training ResNets are still not
fully understood. Recently, several points of view have emerged to try to
interpret ResNet theoretically, such as unraveled view, unrolled iterative
estimation and dynamical systems view. In this paper, we adopt the dynamical
systems point of view, and analyze the lesioning properties of ResNet both
theoretically and experimentally. Based on these analyses, we additionally
propose a novel method for accelerating ResNet training. We apply the proposed
method to train ResNets and Wide ResNets for three image classification
benchmarks, reducing training time by more than 40% with superior or on-par
accuracy.
Kernelized Hashcode Representations for Relation Extraction
Authors: Sahil Garg, Aram Galstyan, Greg Ver Steeg, Irina Rish, Guillermo Cecchi, Shuyang Gao
Date: 2017-11-10 00:00:00
Tasks: General Classification, Relation, Relation Extraction

Kernel methods have produced state-of-the-art results for a number of NLP tasks such as relation extraction, but suffer from poor scalability due to the high cost of computing kernel similarities between natural language structures. A recently proposed technique, kernelized locality-sensitive hashing (KLSH), can significantly reduce the computational cost, but is only applicable to classifiers operating on kNN graphs. Here we propose to use random subspaces of KLSH codes for efficiently constructing an explicit representation of NLP structures suitable for general classification methods. Further, we propose an approach for optimizing the KLSH model for classification problems by maximizing an approximation of mutual information between the KLSH codes (feature vectors) and the class labels. We evaluate the proposed approach on biomedical relation extraction datasets, and observe significant and robust improvements in accuracy w.r.t. state-of-the-art classifiers, along with drastic (orders-of-magnitude) speedup compared to conventional kernel methods.
Adaptive Memory Networks
Authors: Daniel Li, Asim Kadav
Date: 2018-02-01 00:00:00
Proceeding: ICLR 2018 1
Tasks: Decision Making, Question Answering

We present Adaptive Memory Networks (AMN) that processes input-question pairs
to dynamically construct a network architecture optimized for lower inference
times for Question Answering (QA) tasks. AMN processes the input story to
extract entities and stores them in memory banks. Starting from a single bank,
as the number of input entities increases, AMN learns to create new banks as
the entropy in a single bank becomes too high. Hence, after processing an
input-question(s) pair, the resulting network represents a hierarchical
structure where entities are stored in different banks, distanced by question
relevance. At inference, one or few banks are used, creating a tradeoff between
accuracy and performance. AMN is enabled by dynamic networks that allow input
dependent network creation and efficiency in dynamic mini-batching as well as
our novel bank controller that allows learning discrete decision making with
high accuracy. In our results, we demonstrate that AMN learns to create
variable depth networks depending on task complexity and reduces inference
times for QA tasks.
Learning Semantic Segmentation with Diverse Supervision
Authors: Linwei Ye, Zhi Liu, Yang Wang
Date: 2018-02-01 00:00:00
Tasks: object-detection, Object Detection, Segmentation, Semantic Segmentation

Models based on deep convolutional neural networks (CNN) have significantly
improved the performance of semantic segmentation. However, learning these
models requires a large amount of training images with pixel-level labels,
which are very costly and time-consuming to collect. In this paper, we propose
a method for learning CNN-based semantic segmentation models from images with
several types of annotations that are available for various computer vision
tasks, including image-level labels for classification, box-level labels for
object detection and pixel-level labels for semantic segmentation. The proposed
method is flexible and can be used together with any existing CNN-based
semantic segmentation networks. Experimental evaluation on the challenging
PASCAL VOC 2012 and SIFT-flow benchmarks demonstrate that the proposed method
can effectively make use of diverse training data to improve the performance of
the learned models.