pwc-archive/papers-with-abstracts huggingface dataset:
ConnNet: A Long-Range Relation-Aware Pixel-Connectivity Network for Salient Segmentation
Authors: Michael Kampffmeyer, Nanqing Dong, Xiaodan Liang, Yu-jia Zhang, Eric P. Xing
Date: 2018-04-20 00:00:00
Tasks: Relation, Segmentation, Semantic Segmentation

Salient segmentation aims to segment out attention-grabbing regions, a
critical yet challenging task and the foundation of many high-level computer
vision applications. It requires semantic-aware grouping of pixels into salient
regions and benefits from the utilization of global multi-scale contexts to
achieve good local reasoning. Previous works often address it as two-class
segmentation problems utilizing complicated multi-step procedures including
refinement networks and complex graphical models. We argue that semantic
salient segmentation can instead be effectively resolved by reformulating it as
a simple yet intuitive pixel-pair based connectivity prediction task. Following
the intuition that salient objects can be naturally grouped via semantic-aware
connectivity between neighboring pixels, we propose a pure Connectivity Net
(ConnNet). ConnNet predicts connectivity probabilities of each pixel with its
neighboring pixels by leveraging multi-level cascade contexts embedded in the
image and long-range pixel relations. We investigate our approach on two tasks,
namely salient object segmentation and salient instance-level segmentation, and
illustrate that consistent improvements can be obtained by modeling these tasks
as connectivity instead of binary segmentation tasks for a variety of network
architectures. We achieve state-of-the-art performance, outperforming or being
comparable to existing approaches while reducing inference time due to our less
complex approach.
Direct Network Transfer: Transfer Learning of Sentence Embeddings for Semantic Similarity
Authors: Li Zhang, Steven R. Wilson, Rada Mihalcea
Date: 2018-04-20 00:00:00
Tasks: Natural Language Understanding, Semantic Similarity, Semantic Textual Similarity, Sentence, Sentence Embeddings, text similarity, Transfer Learning

Sentence encoders, which produce sentence embeddings using neural networks,
are typically evaluated by how well they transfer to downstream tasks. This
includes semantic similarity, an important task in natural language
understanding. Although there has been much work dedicated to building sentence
encoders, the accompanying transfer learning techniques have received
relatively little attention. In this paper, we propose a transfer learning
setting specialized for semantic similarity, which we refer to as direct
network transfer. Through experiments on several standard text similarity
datasets, we show that applying direct network transfer to existing encoders
can lead to state-of-the-art performance. Additionally, we compare several
approaches to transfer sentence encoders to semantic similarity tasks, showing
that the choice of transfer learning setting greatly affects the performance in
many cases, and differs by encoder and dataset.
HandyNet: A One-stop Solution to Detect, Segment, Localize & Analyze Driver Hands
Authors: Akshay Rangesh, Mohan M. Trivedi
Date: 2018-04-20 00:00:00
Tasks: Autonomous Vehicles

Tasks related to human hands have long been part of the computer vision
community. Hands being the primary actuators for humans, convey a lot about
activities and intents, in addition to being an alternative form of
communication/interaction with other humans and machines. In this study, we
focus on training a single feedforward convolutional neural network (CNN)
capable of executing many hand related tasks that may be of use in autonomous
and semi-autonomous vehicles of the future. The resulting network, which we
refer to as HandyNet, is capable of detecting, segmenting and localizing (in
3D) driver hands inside a vehicle cabin. The network is additionally trained to
identify handheld objects that the driver may be interacting with. To meet the
data requirements to train such a network, we propose a method for cheap
annotation based on chroma-keying, thereby bypassing weeks of human effort
required to label such data. This process can generate thousands of labeled
training samples in an efficient manner, and may be replicated in new
environments with relative ease.
An Unsupervised Learning Model for Deformable Medical Image Registration
Authors: Guha Balakrishnan, Amy Zhao, Mert R. Sabuncu, John Guttag, Adrian V. Dalca
Date: 2018-02-07 00:00:00
Proceeding: CVPR 2018 6
Tasks: Deformable Medical Image Registration, Image Registration, Medical Image Analysis, Medical Image Registration

We present a fast learning-based algorithm for deformable, pairwise 3D
medical image registration. Current registration methods optimize an objective
function independently for each pair of images, which can be time-consuming for
large data. We define registration as a parametric function, and optimize its
parameters given a set of images from a collection of interest. Given a new
pair of scans, we can quickly compute a registration field by directly
evaluating the function using the learned parameters. We model this function
using a convolutional neural network (CNN), and use a spatial transform layer
to reconstruct one image from another while imposing smoothness constraints on
the registration field. The proposed method does not require supervised
information such as ground truth registration fields or anatomical landmarks.
We demonstrate registration accuracy comparable to state-of-the-art 3D image
registration, while operating orders of magnitude faster in practice. Our
method promises to significantly speed up medical image analysis and processing
pipelines, while facilitating novel directions in learning-based registration
and its applications. Our code is available at
https://github.com/balakg/voxelmorph .
Efficient Contextualized Representation: Language Model Pruning for Sequence Labeling
Authors: Liyuan Liu, Xiang Ren, Jingbo Shang, Jian Peng, Jiawei Han
Date: 2018-04-20 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Language Modeling, Language Modelling, Named Entity Recognition (NER)

Many efforts have been made to facilitate natural language processing tasks
with pre-trained language models (LMs), and brought significant improvements to
various applications. To fully leverage the nearly unlimited corpora and
capture linguistic information of multifarious levels, large-size LMs are
required; but for a specific task, only parts of these information are useful.
Such large-sized LMs, even in the inference stage, may cause heavy computation
workloads, making them too time-consuming for large-scale applications. Here we
propose to compress bulky LMs while preserving useful information with regard
to a specific task. As different layers of the model keep different
information, we develop a layer selection method for model pruning using
sparsity-inducing regularization. By introducing the dense connectivity, we can
detach any layer without affecting others, and stretch shallow and wide LMs to
be deep and narrow. In model training, LMs are learned with layer-wise dropouts
for better robustness. Experiments on two benchmark datasets demonstrate the
effectiveness of our method.
Autotune: A Derivative-free Optimization Framework for Hyperparameter Tuning
Authors: Patrick Koch, Oleg Golovidov, Steven Gardner, Brett Wujek, Joshua Griffin, Yan Xu
Date: 2018-04-20 00:00:00
Tasks: BIG-bench Machine Learning

Machine learning applications often require hyperparameter tuning. The
hyperparameters usually drive both the efficiency of the model training process
and the resulting model quality. For hyperparameter tuning, machine learning
algorithms are complex black-boxes. This creates a class of challenging
optimization problems, whose objective functions tend to be nonsmooth,
discontinuous, unpredictably varying in computational expense, and include
continuous, categorical, and/or integer variables. Further, function
evaluations can fail for a variety of reasons including numerical difficulties
or hardware failures. Additionally, not all hyperparameter value combinations
are compatible, which creates so called hidden constraints. Robust and
efficient optimization algorithms are needed for hyperparameter tuning. In this
paper we present an automated parallel derivative-free optimization framework
called \textbf{Autotune}, which combines a number of specialized sampling and
search methods that are very effective in tuning machine learning models
despite these challenges. Autotune provides significantly improved models over
using default hyperparameter settings with minimal user interaction on
real-world applications. Given the inherent expense of training numerous
candidate models, we demonstrate the effectiveness of Autotune's search methods
and the efficient distributed and parallel paradigms for training and tuning
models, and also discuss the resource trade-offs associated with the ability to
both distribute the training process and parallelize the tuning process.
An Aggregated Multicolumn Dilated Convolution Network for Perspective-Free Counting
Authors: Diptodip Deb, Jonathan Ventura
Date: 2018-04-20 00:00:00

We propose the use of dilated filters to construct an aggregation module in a
multicolumn convolutional neural network for perspective-free counting.
Counting is a common problem in computer vision (e.g. traffic on the street or
pedestrians in a crowd). Modern approaches to the counting problem involve the
production of a density map via regression whose integral is equal to the
number of objects in the image. However, objects in the image can occur at
different scales (e.g. due to perspective effects) which can make it difficult
for a learning agent to learn the proper density map. While the use of multiple
columns to extract multiscale information from images has been shown before,
our approach aggregates the multiscale information gathered by the multicolumn
convolutional neural network to improve performance. Our experiments show that
our proposed network outperforms the state-of-the-art on many benchmark
datasets, and also that using our aggregation module in combination with a
higher number of columns is beneficial for multiscale counting.
Understanding AI Data Repositories with Automatic Query Generation
Authors: Erik Altman
Date: 2018-04-20 00:00:00

We describe a set of techniques to generate queries automatically based on
one or more ingested, input corpuses. These queries require no a priori domain
knowledge, and hence no human domain experts. Thus, these auto-generated
queries help address the epistemological question of how we know what we know,
or more precisely in this case, how an AI system with ingested data knows what
it knows. These auto-generated queries can also be used to identify and remedy
problem areas in ingested material -- areas for which the knowledge of the AI
system is incomplete or even erroneous. Similarly, the proposed techniques
facilitate tests of AI capability -- both in terms of coverage and accuracy. By
removing humans from the main learning loop, our approach also allows more
effective scaling of AI and cognitive capabilities to provide (1) broader
coverage in a single domain such as health or geology; and (2) more rapid
deployment to new domains. The proposed techniques also allow ingested
knowledge to be extended naturally. Our investigations are early, and this
paper provides a description of the techniques. Assessment of their efficacy is
our next step for future work.
A Deep Face Identification Network Enhanced by Facial Attributes Prediction
Authors: Fariborz Taherkhani, Nasser M. Nasrabadi, Jeremy Dawson
Date: 2018-04-20 00:00:00
Tasks: Attribute, Face Identification, Gender Prediction, Prediction

In this paper, we propose a new deep framework which predicts facial
attributes and leverage it as a soft modality to improve face identification
performance. Our model is an end to end framework which consists of a
convolutional neural network (CNN) whose output is fanned out into two separate
branches; the first branch predicts facial attributes while the second branch
identifies face images. Contrary to the existing multi-task methods which only
use a shared CNN feature space to train these two tasks jointly, we fuse the
predicted attributes with the features from the face modality in order to
improve the face identification performance. Experimental results show that our
model brings benefits to both face identification as well as facial attribute
prediction performance, especially in the case of identity facial attributes
such as gender prediction. We tested our model on two standard datasets
annotated by identities and face attributes. Experimental results indicate that
the proposed model outperforms most of the current existing face identification
and attribute prediction methods.
Learning Neural Networks with Two Nonlinear Layers in Polynomial Time
Authors: Surbhi Goel, Adam Klivans
Date: 2017-09-18 00:00:00
Tasks: Learning Theory, PAC learning, Vocal Bursts Valence Prediction

We give a polynomial-time algorithm for learning neural networks with one
layer of sigmoids feeding into any Lipschitz, monotone activation function
(e.g., sigmoid or ReLU). We make no assumptions on the structure of the
network, and the algorithm succeeds with respect to {\em any} distribution on
the unit ball in $n$ dimensions (hidden weight vectors also have unit norm).
This is the first assumption-free, provably efficient algorithm for learning
neural networks with two nonlinear layers.
  Our algorithm-- {\em Alphatron}-- is a simple, iterative update rule that
combines isotonic regression with kernel methods. It outputs a hypothesis that
yields efficient oracle access to interpretable features. It also suggests a
new approach to Boolean learning problems via real-valued conditional-mean
functions, sidestepping traditional hardness results from computational
learning theory.
  Along these lines, we subsume and improve many longstanding results for PAC
learning Boolean functions to the more general, real-valued setting of {\em
probabilistic concepts}, a model that (unlike PAC learning) requires non-i.i.d.
noise-tolerance.
Adversarial Training Versus Weight Decay
Authors: Angus Galloway, Thomas Tanay, Graham W. Taylor
Date: 2018-04-10 00:00:00

Performance-critical machine learning models should be robust to input
perturbations not seen during training. Adversarial training is a method for
improving a model's robustness to some perturbations by including them in the
training process, but this tends to exacerbate other vulnerabilities of the
model. The adversarial training framework has the effect of translating the
data with respect to the cost function, while weight decay has a scaling
effect. Although weight decay could be considered a crude regularization
technique, it appears superior to adversarial training as it remains stable
over a broader range of regimes and reduces all generalization errors. Equipped
with these abstractions, we provide key baseline results and methodology for
characterizing robustness. The two approaches can be combined to yield one
small model that demonstrates good robustness to several white-box attacks
associated with different metrics.
Deep Learning on Operational Facility Data Related to Large-Scale Distributed Area Scientific Workflows
Authors: Alok Singh, Eric Stephan, Malachi Schram, Ilkay Altintas
Date: 2018-04-17 00:00:00
Tasks: Anomaly Detection, Distributed Computing

Distributed computing platforms provide a robust mechanism to perform
large-scale computations by splitting the task and data among multiple
locations, possibly located thousands of miles apart geographically. Although
such distribution of resources can lead to benefits, it also comes with its
associated problems such as rampant duplication of file transfers increasing
congestion, long job completion times, unexpected site crashing, suboptimal
data transfer rates, unpredictable reliability in a time range, and suboptimal
usage of storage elements. In addition, each sub-system becomes a potential
failure node that can trigger system wide disruptions. In this vision paper, we
outline our approach to leveraging Deep Learning algorithms to discover
solutions to unique problems that arise in a system with computational
infrastructure that is spread over a wide area. The presented vision, motivated
by a real scientific use case from Belle II experiments, is to develop
multilayer neural networks to tackle forecasting, anomaly detection and
optimization challenges in a complex and distributed data movement environment.
Through this vision based on Deep Learning principles, we aim to achieve
reduced congestion events, faster file transfer rates, and enhanced site
reliability.
Parametric t-Distributed Stochastic Exemplar-centered Embedding
Authors: Martin Renqiang Min, Hongyu Guo, Dinghan Shen
Date: 2017-10-14 00:00:00
Tasks: Data Visualization

Parametric embedding methods such as parametric t-SNE (pt-SNE) have been
widely adopted for data visualization and out-of-sample data embedding without
further computationally expensive optimization or approximation. However, the
performance of pt-SNE is highly sensitive to the hyper-parameter batch size due
to conflicting optimization goals, and often produces dramatically different
embeddings with different choices of user-defined perplexities. To effectively
solve these issues, we present parametric t-distributed stochastic
exemplar-centered embedding methods. Our strategy learns embedding parameters
by comparing given data only with precomputed exemplars, resulting in a cost
function with linear computational and memory complexity, which is further
reduced by noise contrastive samples. Moreover, we propose a shallow embedding
network with high-order feature interactions for data visualization, which is
much easier to tune but produces comparable performance in contrast to a deep
neural network employed by pt-SNE. We empirically demonstrate, using several
benchmark datasets, that our proposed methods significantly outperform pt-SNE
in terms of robustness, visual effects, and quantitative evaluations.
Disease-Atlas: Navigating Disease Trajectories with Deep Learning
Authors: Bryan Lim, Mihaela van der Schaar
Date: 2018-03-27 00:00:00
Tasks: Deep Learning

Joint models for longitudinal and time-to-event data are commonly used in
longitudinal studies to forecast disease trajectories over time. While there
are many advantages to joint modeling, the standard forms suffer from
limitations that arise from a fixed model specification, and computational
difficulties when applied to high-dimensional datasets. In this paper, we
propose a deep learning approach to address these limitations, enhancing
existing methods with the inherent flexibility and scalability of deep neural
networks, while retaining the benefits of joint modeling. Using longitudinal
data from a real-world medical dataset, we demonstrate improvements in
performance and scalability, as well as robustness in the presence of
irregularly sampled data.
Inseparability and Conservative Extensions of Description Logic Ontologies: A Survey
Authors: Elena Botoeva, Boris Konev, Carsten Lutz, Vladislav Ryzhikov, Frank Wolter, Michael Zakharyaschev
Date: 2018-04-20 00:00:00
Tasks: Survey

The question whether an ontology can safely be replaced by another, possibly
simpler, one is fundamental for many ontology engineering and maintenance
tasks. It underpins, for example, ontology versioning, ontology modularization,
forgetting, and knowledge exchange. What safe replacement means depends on the
intended application of the ontology. If, for example, it is used to query
data, then the answers to any relevant ontology-mediated query should be the
same over any relevant data set; if, in contrast, the ontology is used for
conceptual reasoning, then the entailed subsumptions between concept
expressions should coincide. This gives rise to different notions of ontology
inseparability such as query inseparability and concept inseparability, which
generalize corresponding notions of conservative extensions. We survey results
on various notions of inseparability in the context of description logic
ontologies, discussing their applications, useful model-theoretic
characterizations, algorithms for determining whether two ontologies are
inseparable (and, sometimes, for computing the difference between them if they
are not), and the computational complexity of this problem.
Value-aware Quantization for Training and Inference of Neural Networks
Authors: Eunhyeok Park, Sungjoo Yoo, Peter Vajda
Date: 2018-04-20 00:00:00
Proceeding: ECCV 2018 9
Tasks: Quantization

We propose a novel value-aware quantization which applies aggressively
reduced precision to the majority of data while separately handling a small
amount of large data in high precision, which reduces total quantization errors
under very low precision. We present new techniques to apply the proposed
quantization to training and inference. The experiments show that our method
with 3-bit activations (with 2% of large ones) can give the same training
accuracy as full-precision one while offering significant (41.6% and 53.7%)
reductions in the memory cost of activations in ResNet-152 and Inception-v3
compared with the state-of-the-art method. Our experiments also show that deep
networks such as Inception-v3, ResNet-101 and DenseNet-121 can be quantized for
inference with 4-bit weights and activations (with 1% 16-bit data) within 1%
top-1 accuracy drop.
Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning
Authors: Felipe Petroski Such, Vashisht Madhavan, Edoardo Conti, Joel Lehman, Kenneth O. Stanley, Jeff Clune
Date: 2017-12-18 00:00:00
Tasks: Deep Reinforcement Learning, Evolutionary Algorithms, Q-Learning, Reinforcement Learning, Reinforcement Learning (RL)

Deep artificial neural networks (DNNs) are typically trained via
gradient-based learning algorithms, namely backpropagation. Evolution
strategies (ES) can rival backprop-based algorithms such as Q-learning and
policy gradients on challenging deep reinforcement learning (RL) problems.
However, ES can be considered a gradient-based algorithm because it performs
stochastic gradient descent via an operation similar to a finite-difference
approximation of the gradient. That raises the question of whether
non-gradient-based evolutionary algorithms can work at DNN scales. Here we
demonstrate they can: we evolve the weights of a DNN with a simple,
gradient-free, population-based genetic algorithm (GA) and it performs well on
hard deep RL problems, including Atari and humanoid locomotion. The Deep GA
successfully evolves networks with over four million free parameters, the
largest neural networks ever evolved with a traditional evolutionary algorithm.
These results (1) expand our sense of the scale at which GAs can operate, (2)
suggest intriguingly that in some cases following the gradient is not the best
choice for optimizing performance, and (3) make immediately available the
multitude of neuroevolution techniques that improve performance. We demonstrate
the latter by showing that combining DNNs with novelty search, which encourages
exploration on tasks with deceptive or sparse reward functions, can solve a
high-dimensional problem on which reward-maximizing algorithms (e.g.\ DQN, A3C,
ES, and the GA) fail. Additionally, the Deep GA is faster than ES, A3C, and DQN
(it can train Atari in ${\raise.17ex\hbox{$\scriptstyle\sim$}}$4 hours on one
desktop or ${\raise.17ex\hbox{$\scriptstyle\sim$}}$1 hour distributed on 720
cores), and enables a state-of-the-art, up to 10,000-fold compact encoding
technique.