pwc-archive/papers-with-abstracts huggingface dataset:
Know-Evolve: Deep Temporal Reasoning for Dynamic Knowledge Graphs
Authors: Rakshit Trivedi, Hanjun Dai, Yichen Wang, Le Song
Date: 2017-05-16 00:00:00
Proceeding: ICML 2017 8
Tasks: Entity Embeddings, Knowledge Graphs, Relational Reasoning

The availability of large scale event data with time stamps has given rise to
dynamically evolving knowledge graphs that contain temporal information for
each edge. Reasoning over time in such dynamic knowledge graphs is not yet well
understood. To this end, we present Know-Evolve, a novel deep evolutionary
knowledge network that learns non-linearly evolving entity representations over
time. The occurrence of a fact (edge) is modeled as a multivariate point
process whose intensity function is modulated by the score for that fact
computed based on the learned entity embeddings. We demonstrate significantly
improved performance over various relational learning approaches on two large
scale real-world datasets. Further, our method effectively predicts occurrence
or recurrence time of a fact which is novel compared to prior reasoning
approaches in multi-relational setting.
A Fixed-Point Model for Pancreas Segmentation in Abdominal CT Scans
Authors: Yuyin Zhou, Lingxi Xie, Wei Shen, Yan Wang, Elliot K. Fishman, Alan L. Yuille
Date: 2016-12-25 00:00:00
Tasks: Organ Segmentation, Pancreas Segmentation, Segmentation

Deep neural networks have been widely adopted for automatic organ
segmentation from abdominal CT scans. However, the segmentation accuracy of
some small organs (e.g., the pancreas) is sometimes below satisfaction,
arguably because deep networks are easily disrupted by the complex and variable
background regions which occupies a large fraction of the input volume. In this
paper, we formulate this problem into a fixed-point model which uses a
predicted segmentation mask to shrink the input region. This is motivated by
the fact that a smaller input region often leads to more accurate segmentation.
In the training process, we use the ground-truth annotation to generate
accurate input regions and optimize network weights. On the testing stage, we
fix the network parameters and update the segmentation results in an iterative
manner. We evaluate our approach on the NIH pancreas segmentation dataset, and
outperform the state-of-the-art by more than 4%, measured by the average
Dice-S{\o}rensen Coefficient (DSC). In addition, we report 62.43% DSC in the
worst case, which guarantees the reliability of our approach in clinical
applications.
Dual Iterative Hard Thresholding: From Non-convex Sparse Minimization to Non-smooth Concave Maximization
Authors: Bo Liu, Xiao-Tong Yuan, Lezi Wang, Qingshan Liu, Dimitris N. Metaxas
Date: 2017-03-01 00:00:00
Proceeding: ICML 2017 8
Tasks: Computational Efficiency, Stochastic Optimization

Iterative Hard Thresholding (IHT) is a class of projected gradient descent
methods for optimizing sparsity-constrained minimization models, with the best
known efficiency and scalability in practice. As far as we know, the existing
IHT-style methods are designed for sparse minimization in primal form. It
remains open to explore duality theory and algorithms in such a non-convex and
NP-hard problem setting. In this paper, we bridge this gap by establishing a
duality theory for sparsity-constrained minimization with $\ell_2$-regularized
loss function and proposing an IHT-style algorithm for dual maximization. Our
sparse duality theory provides a set of sufficient and necessary conditions
under which the original NP-hard/non-convex problem can be equivalently solved
in a dual formulation. The proposed dual IHT algorithm is a super-gradient
method for maximizing the non-smooth dual objective. An interesting finding is
that the sparse recovery performance of dual IHT is invariant to the Restricted
Isometry Property (RIP), which is required by virtually all the existing primal
IHT algorithms without sparsity relaxation. Moreover, a stochastic variant of
dual IHT is proposed for large-scale stochastic optimization. Numerical results
demonstrate the superiority of dual IHT algorithms to the state-of-the-art
primal IHT-style algorithms in model estimation accuracy and computational
efficiency.
Deep Learning Autoencoder Approach for Handwritten Arabic Digits Recognition
Authors: Mohamed Loey, Ahmed El-Sawy, Hazem EL-Bakry
Date: 2017-06-21 00:00:00
Tasks: Deep Learning, General Classification

This paper presents a new unsupervised learning approach with stacked
autoencoder (SAE) for Arabic handwritten digits categorization. Recently,
Arabic handwritten digits recognition has been an important area due to its
applications in several fields. This work is focusing on the recognition part
of handwritten Arabic digits recognition that face several challenges,
including the unlimited variation in human handwriting and the large public
databases. Arabic digits contains ten numbers that were descended from the
Indian digits system. Stacked autoencoder (SAE) tested and trained the MADBase
database (Arabic handwritten digits images) that contain 10000 testing images
and 60000 training images. We show that the use of SAE leads to significant
improvements across different machine-learning classification algorithms. SAE
is giving an average accuracy of 98.5%.
Multi-Modal Trip Hazard Affordance Detection On Construction Sites
Authors: Sean McMahon, Niko Sünderhauf, Ben Upcroft, Michael Milford
Date: 2017-06-21 00:00:00
Tasks: Affordance Detection

Trip hazards are a significant contributor to accidents on construction and
manufacturing sites, where over a third of Australian workplace injuries occur
[1]. Current safety inspections are labour intensive and limited by human
fallibility,making automation of trip hazard detection appealing from both a
safety and economic perspective. Trip hazards present an interesting challenge
to modern learning techniques because they are defined as much by affordance as
by object type; for example wires on a table are not a trip hazard, but can be
if lying on the ground. To address these challenges, we conduct a comprehensive
investigation into the performance characteristics of 11 different colour and
depth fusion approaches, including 4 fusion and one non fusion approach; using
colour and two types of depth images. Trained and tested on over 600 labelled
trip hazards over 4 floors and 2000m$\mathrm{^{2}}$ in an active construction
site,this approach was able to differentiate between identical objects in
different physical configurations (see Figure 1). Outperforming a colour-only
detector, our multi-modal trip detector fuses colour and depth information to
achieve a 4% absolute improvement in F1-score. These investigative results and
the extensive publicly available dataset moves us one step closer to assistive
or fully automated safety inspection systems on construction sites.
Inverted Bilingual Topic Models for Lexicon Extraction from Non-parallel Data
Authors: Tengfei Ma, Tetsuya Nasukawa
Date: 2016-12-21 00:00:00
Tasks: Topic Models, Translation

Topic models have been successfully applied in lexicon extraction. However,
most previous methods are limited to document-aligned data. In this paper, we
try to address two challenges of applying topic models to lexicon extraction in
non-parallel data: 1) hard to model the word relationship and 2) noisy seed
dictionary. To solve these two challenges, we propose two new bilingual topic
models to better capture the semantic information of each word while
discriminating the multiple translations in a noisy seed dictionary. We extend
the scope of topic models by inverting the roles of "word" and "document". In
addition, to solve the problem of noise in seed dictionary, we incorporate the
probability of translation selection in our models. Moreover, we also propose
an effective measure to evaluate the similarity of words in different languages
and select the optimal translation pairs. Experimental results using real world
data demonstrate the utility and efficacy of the proposed models.
Dance Dance Convolution
Authors: Chris Donahue, Zachary C. Lipton, Julian McAuley
Date: 2017-03-20 00:00:00
Proceeding: ICML 2017 8
Tasks: Rhythm

Dance Dance Revolution (DDR) is a popular rhythm-based video game. Players
perform steps on a dance platform in synchronization with music as directed by
on-screen step charts. While many step charts are available in standardized
packs, players may grow tired of existing charts, or wish to dance to a song
for which no chart exists. We introduce the task of learning to choreograph.
Given a raw audio track, the goal is to produce a new step chart. This task
decomposes naturally into two subtasks: deciding when to place steps and
deciding which steps to select. For the step placement task, we combine
recurrent and convolutional neural networks to ingest spectrograms of low-level
audio features to predict steps, conditioned on chart difficulty. For step
selection, we present a conditional LSTM generative model that substantially
outperforms n-gram and fixed-window approaches.
Compact Tensor Pooling for Visual Question Answering
Authors: Yang Shi, Tommaso Furlanello, Anima Anandkumar
Date: 2017-06-20 00:00:00
Tasks: Question Answering, Visual Question Answering, Visual Question Answering (VQA)

Performing high level cognitive tasks requires the integration of feature
maps with drastically different structure. In Visual Question Answering (VQA)
image descriptors have spatial structures, while lexical inputs inherently
follow a temporal sequence. The recently proposed Multimodal Compact Bilinear
pooling (MCB) forms the outer products, via count-sketch approximation, of the
visual and textual representation at each spatial location. While this
procedure preserves spatial information locally, outer-products are taken
independently for each fiber of the activation tensor, and therefore do not
include spatial context. In this work, we introduce multi-dimensional sketch
({MD-sketch}), a novel extension of count-sketch to tensors. Using this new
formulation, we propose Multimodal Compact Tensor Pooling (MCT) to fully
exploit the global spatial context during bilinear pooling operations.
Contrarily to MCB, our approach preserves spatial context by directly
convolving the MD-sketch from the visual tensor features with the text vector
feature using higher order FFT. Furthermore we apply MCT incrementally at each
step of the question embedding and accumulate the multi-modal vectors with a
second LSTM layer before the final answer is chosen.
Using Convolutional Neural Networks in Robots with Limited Computational Resources: Detecting NAO Robots while Playing Soccer
Authors: Nicolás Cruz, Kenzo Lobos-Tsunekawa, Javier Ruiz-del-Solar
Date: 2017-06-20 00:00:00

The main goal of this paper is to analyze the general problem of using
Convolutional Neural Networks (CNNs) in robots with limited computational
capabilities, and to propose general design guidelines for their use. In
addition, two different CNN based NAO robot detectors that are able to run in
real-time while playing soccer are proposed. One of the detectors is based on
the XNOR-Net and the other on the SqueezeNet. Each detector is able to process
a robot object-proposal in ~1ms, with an average number of 1.5 proposals per
frame obtained by the upper camera of the NAO. The obtained detection rate is
~97%.
Toward Real-Time Decentralized Reinforcement Learning using Finite Support Basis Functions
Authors: Kenzo Lobos-Tsunekawa, David L. Leottau, Javier Ruiz-del-Solar
Date: 2017-06-20 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

This paper addresses the design and implementation of complex Reinforcement
Learning (RL) behaviors where multi-dimensional action spaces are involved, as
well as the need to execute the behaviors in real-time using robotic platforms
with limited computational resources and training times. For this purpose, we
propose the use of decentralized RL, in combination with finite support basis
functions as alternatives to Gaussian RBF, in order to alleviate the effects of
the curse of dimensionality on the action and state spaces respectively, and to
reduce the computation time. As testbed, a RL based controller for the in-walk
kick in NAO robots, a challenging and critical problem for soccer robotics, is
used. The reported experiments show empirically that our solution saves up to
99.94% of execution time and 98.82% of memory consumption during execution,
without diminishing performance compared to classical approaches.
Recognition of Grasp Points for Clothes Manipulation under unconstrained Conditions
Authors: Luz María Martínez, Javier Ruiz-del-Solar
Date: 2017-06-20 00:00:00

In this work a system for recognizing grasp points in RGB-D images is
proposed. This system is intended to be used by a domestic robot when deploying
clothes lying at a random position on a table. By taking into consideration
that the grasp points are usually near key parts of clothing, such as the waist
of pants or the neck of a shirt. The proposed system attempts to detect these
key parts first, using a local multivariate contour that adapts its shape
accordingly. Then, the proposed system applies the Vessel Enhancement filter to
identify wrinkles in the clothes, allowing to compute a roughness index for the
clothes. Finally, by mixing (i) the key part contours and (ii) the roughness
information obtained by the vessel filter, the system is able to recognize
grasp points for unfolding a piece of clothing. The recognition system is
validated using realistic RGB-D images of different cloth types.
Highly curved image sensors: a practical approach for improved optical performance
Authors: Brian Guenter, Neel Joshi, Richard Stoakley, Andrew Keefe, Kevin Geary, Ryan Freeman, Jake Hundley, Pamela Patterson, David Hammon, Guillermo Herrera, Elena Sherman, Andrew Nowak, Randall Schubert, Peter Brewer, Louis Yang, Russell Mott, Geoff McKnight
Date: 2017-06-20 00:00:00
Tasks: Friction

The significant optical and size benefits of using a curved focal surface for
imaging systems have been well studied yet never brought to market for lack of
a high-quality, mass-producible, curved image sensor. In this work we
demonstrate that commercial silicon CMOS image sensors can be thinned and
formed into accurate, highly curved optical surfaces with undiminished
functionality. Our key development is a pneumatic forming process that avoids
rigid mechanical constraints and suppresses wrinkling instabilities. A
combination of forming-mold design, pressure membrane elastic properties, and
controlled friction forces enables us to gradually contact the die at the
corners and smoothly press the sensor into a spherical shape. Allowing the die
to slide into the concave target shape enables a threefold increase in the
spherical curvature over prior approaches having mechanical constraints that
resist deformation, and create a high-stress, stretch-dominated state. Our
process creates a bridge between the high precision and low-cost but planar
CMOS process, and ideal non-planar component shapes such as spherical imagers
for improved optical systems. We demonstrate these curved sensors in prototype
cameras with custom lenses, measuring exceptional resolution of 3220
line-widths per picture height at an aperture of f/1.2 and nearly 100% relative
illumination across the field. Though we use a 1/2.3" format image sensor in
this report, we also show this process is generally compatible with many state
of the art imaging sensor formats. By example, we report photogrammetry test
data for an APS-C sized silicon die formed to a 30$^\circ$ subtended spherical
angle. These gains in sharpness and relative illumination enable a new
generation of ultra-high performance, manufacturable, digital imaging systems
for scientific, industrial, and artistic use.
Interpretable Predictions of Tree-based Ensembles via Actionable Feature Tweaking
Authors: Gabriele Tolomei, Fabrizio Silvestri, Andrew Haines, Mounia Lalmas
Date: 2017-06-20 00:00:00
Tasks: Feature Engineering

Machine-learned models are often described as "black boxes". In many
real-world applications however, models may have to sacrifice predictive power
in favour of human-interpretability. When this is the case, feature engineering
becomes a crucial task, which requires significant and time-consuming human
effort. Whilst some features are inherently static, representing properties
that cannot be influenced (e.g., the age of an individual), others capture
characteristics that could be adjusted (e.g., the daily amount of carbohydrates
taken). Nonetheless, once a model is learned from the data, each prediction it
makes on new instances is irreversible - assuming every instance to be a static
point located in the chosen feature space. There are many circumstances however
where it is important to understand (i) why a model outputs a certain
prediction on a given instance, (ii) which adjustable features of that instance
should be modified, and finally (iii) how to alter such a prediction when the
mutated instance is input back to the model. In this paper, we present a
technique that exploits the internals of a tree-based ensemble classifier to
offer recommendations for transforming true negative instances into positively
predicted ones. We demonstrate the validity of our approach using an online
advertising application. First, we design a Random Forest classifier that
effectively separates between two types of ads: low (negative) and high
(positive) quality ads (instances). Then, we introduce an algorithm that
provides recommendations that aim to transform a low quality ad (negative
instance) into a high quality one (positive instance). Finally, we evaluate our
approach on a subset of the active inventory of a large ad network, Yahoo
Gemini.
Medical Concept Representation Learning from Electronic Health Records and its Application on Heart Failure Prediction
Authors: Edward Choi, Andy Schuetz, Walter F. Stewart, Jimeng Sun
Date: 2016-02-11 00:00:00
Tasks: Representation Learning

Objective: To transform heterogeneous clinical data from electronic health
records into clinically meaningful constructed features using data driven
method that rely, in part, on temporal relations among data. Materials and
Methods: The clinically meaningful representations of medical concepts and
patients are the key for health analytic applications. Most of existing
approaches directly construct features mapped to raw data (e.g., ICD or CPT
codes), or utilize some ontology mapping such as SNOMED codes. However, none of
the existing approaches leverage EHR data directly for learning such concept
representation. We propose a new way to represent heterogeneous medical
concepts (e.g., diagnoses, medications and procedures) based on co-occurrence
patterns in longitudinal electronic health records. The intuition behind the
method is to map medical concepts that are co-occuring closely in time to
similar concept vectors so that their distance will be small. We also derive a
simple method to construct patient vectors from the related medical concept
vectors. Results: For qualitative evaluation, we study similar medical concepts
across diagnosis, medication and procedure. In quantitative evaluation, our
proposed representation significantly improves the predictive modeling
performance for onset of heart failure (HF), where classification methods (e.g.
logistic regression, neural network, support vector machine and K-nearest
neighbors) achieve up to 23% improvement in area under the ROC curve (AUC)
using this proposed representation. Conclusion: We proposed an effective method
for patient and medical concept representation learning. The resulting
representation can map relevant concepts together and also improves predictive
modeling performance.
Chemception: A Deep Neural Network with Minimal Chemistry Knowledge Matches the Performance of Expert-developed QSAR/QSPR Models
Authors: Garrett B. Goh, Charles Siegel, Abhinav Vishnu, Nathan O. Hodas, Nathan Baker
Date: 2017-06-20 00:00:00
Tasks: Computational chemistry, Deep Learning, Feature Engineering, image-classification, Image Classification, Prediction, speech-recognition, Speech Recognition

In the last few years, we have seen the transformative impact of deep
learning in many applications, particularly in speech recognition and computer
vision. Inspired by Google's Inception-ResNet deep convolutional neural network
(CNN) for image classification, we have developed "Chemception", a deep CNN for
the prediction of chemical properties, using just the images of 2D drawings of
molecules. We develop Chemception without providing any additional explicit
chemistry knowledge, such as basic concepts like periodicity, or advanced
features like molecular descriptors and fingerprints. We then show how
Chemception can serve as a general-purpose neural network architecture for
predicting toxicity, activity, and solvation properties when trained on a
modest database of 600 to 40,000 compounds. When compared to multi-layer
perceptron (MLP) deep neural networks trained with ECFP fingerprints,
Chemception slightly outperforms in activity and solvation prediction and
slightly underperforms in toxicity prediction. Having matched the performance
of expert-developed QSAR/QSPR deep learning models, our work demonstrates the
plausibility of using deep neural networks to assist in computational chemistry
research, where the feature engineering process is performed primarily by a
deep learning algorithm.
Arrays of (locality-sensitive) Count Estimators (ACE): High-Speed Anomaly Detection via Cache Lookups
Authors: Chen Luo, Anshumali Shrivastava
Date: 2017-06-20 00:00:00
Tasks: Anomaly Detection, Unsupervised Anomaly Detection

Anomaly detection is one of the frequent and important subroutines deployed
in large-scale data processing systems. Even being a well-studied topic,
existing techniques for unsupervised anomaly detection require storing
significant amounts of data, which is prohibitive from memory and latency
perspective. In the big-data world existing methods fail to address the new set
of memory and latency constraints. In this paper, we propose ACE (Arrays of
(locality-sensitive) Count Estimators) algorithm that can be 60x faster than
the ELKI package~\cite{DBLP:conf/ssd/AchtertBKSZ09}, which has the fastest
implementation of the unsupervised anomaly detection algorithms. ACE algorithm
requires less than $4MB$ memory, to dynamically compress the full data
information into a set of count arrays. These tiny $4MB$ arrays of counts are
sufficient for unsupervised anomaly detection. At the core of the ACE
algorithm, there is a novel statistical estimator which is derived from the
sampling view of Locality Sensitive Hashing(LSH). This view is significantly
different and efficient than the widely popular view of LSH for near-neighbor
search. We show the superiority of ACE algorithm over 11 popular baselines on 3
benchmark datasets, including the KDD-Cup99 data which is the largest available
benchmark comprising of more than half a million entries with ground truth
anomaly labels.