pwc-archive/papers-with-abstracts huggingface dataset:
GossipGraD: Scalable Deep Learning using Gossip Communication based Asynchronous Gradient Descent
Authors: Jeff Daily, Abhinav Vishnu, Charles Siegel, Thomas Warfel, Vinay Amatya
Date: 2018-03-15 00:00:00
Tasks: CPU, Deep Learning, GPU

In this paper, we present GossipGraD - a gossip communication protocol based
Stochastic Gradient Descent (SGD) algorithm for scaling Deep Learning (DL)
algorithms on large-scale systems. The salient features of GossipGraD are: 1)
reduction in overall communication complexity from {\Theta}(log(p)) for p
compute nodes in well-studied SGD to O(1), 2) model diffusion such that compute
nodes exchange their updates (gradients) indirectly after every log(p) steps,
3) rotation of communication partners for facilitating direct diffusion of
gradients, 4) asynchronous distributed shuffle of samples during the
feedforward phase in SGD to prevent over-fitting, 5) asynchronous communication
of gradients for further reducing the communication cost of SGD and GossipGraD.
We implement GossipGraD for GPU and CPU clusters and use NVIDIA GPUs (Pascal
P100) connected with InfiniBand, and Intel Knights Landing (KNL) connected with
Aries network. We evaluate GossipGraD using well-studied dataset ImageNet-1K
(~250GB), and widely studied neural network topologies such as GoogLeNet and
ResNet50 (current winner of ImageNet Large Scale Visualization Research
Challenge (ILSVRC)). Our performance evaluation using both KNL and Pascal GPUs
indicates that GossipGraD can achieve perfect efficiency for these datasets and
their associated neural network topologies. Specifically, for ResNet50,
GossipGraD is able to achieve ~100% compute efficiency using 128 NVIDIA Pascal
P100 GPUs - while matching the top-1 classification accuracy published in
literature.
Virtual CNN Branching: Efficient Feature Ensemble for Person Re-Identification
Authors: Albert Gong, Qiang Qiu, Guillermo Sapiro
Date: 2018-03-15 00:00:00
Tasks: Person Re-Identification

In this paper we introduce an ensemble method for convolutional neural
network (CNN), called "virtual branching," which can be implemented with nearly
no additional parameters and computation on top of standard CNNs. We propose
our method in the context of person re-identification (re-ID). Our CNN model
consists of shared bottom layers, followed by "virtual" branches, where neurons
from a block of regular convolutional and fully-connected layers are
partitioned into multiple sets. Each virtual branch is trained with different
data to specialize in different aspects, e.g., a specific body region or pose
orientation. In this way, robust ensemble representations are obtained against
human body misalignment, deformations, or variations in viewing angles, at
nearly no any additional cost. The proposed method achieves competitive
performance on multiple person re-ID benchmark datasets, including Market-1501,
CUHK03, and DukeMTMC-reID.
Capturing Structure Implicitly from Time-Series having Limited Data
Authors: Daniel Emaasit, Matthew Johnson
Date: 2018-03-15 00:00:00
Tasks: scientific discovery, Small Data Image Classification, Time Series, Time Series Analysis

Scientific fields such as insider-threat detection and highway-safety
planning often lack sufficient amounts of time-series data to estimate
statistical models for the purpose of scientific discovery. Moreover, the
available limited data are quite noisy. This presents a major challenge when
estimating time-series models that are robust to overfitting and have
well-calibrated uncertainty estimates. Most of the current literature in these
fields involve visualizing the time-series for noticeable structure and hard
coding them into pre-specified parametric functions. This approach is
associated with two limitations. First, given that such trends may not be
easily noticeable in small data, it is difficult to explicitly incorporate
expressive structure into the models during formulation. Second, it is
difficult to know $\textit{a priori}$ the most appropriate functional form to
use. To address these limitations, a nonparametric Bayesian approach was
proposed to implicitly capture hidden structure from time series having limited
data. The proposed model, a Gaussian process with a spectral mixture kernel,
precludes the need to pre-specify a functional form and hard code trends, is
robust to overfitting and has well-calibrated uncertainty estimates.
Multi-Task Learning by Deep Collaboration and Application in Facial Landmark Detection
Authors: Ludovic Trottier, Philippe Giguère, Brahim Chaib-Draa
Date: 2017-10-28 00:00:00
Proceeding: ICLR 2018 1
Tasks: Facial Landmark Detection, Multi-Task Learning

Convolutional neural networks (CNNs) have become the most successful approach
in many vision-related domains. However, they are limited to domains where data
is abundant. Recent works have looked at multi-task learning (MTL) to mitigate
data scarcity by leveraging domain-specific information from related tasks. In
this paper, we present a novel soft-parameter sharing mechanism for CNNs in a
MTL setting, which we refer to as Deep Collaboration. We propose taking into
account the notion that task relevance depends on depth by using lateral
transformation blocs with skip connections. This allows extracting
task-specific features at various depth without sacrificing features relevant
to all tasks. We show that CNNs connected with our Deep Collaboration obtain
better accuracy on facial landmark detection with related tasks. We finally
verify that our approach effectively allows knowledge sharing by showing
depth-specific influence of tasks that we know are related.
Spatially-Adaptive Filter Units for Deep Neural Networks
Authors: Domen Tabernik, Matej Kristan, Aleš Leonardis
Date: 2017-11-30 00:00:00
Proceeding: CVPR 2018 6
Tasks: Image Classification, Semantic Segmentation

Classical deep convolutional networks increase receptive field size by either
gradual resolution reduction or application of hand-crafted dilated
convolutions to prevent increase in the number of parameters. In this paper we
propose a novel displaced aggregation unit (DAU) that does not require
hand-crafting. In contrast to classical filters with units (pixels) placed on a
fixed regular grid, the displacement of the DAUs are learned, which enables
filters to spatially-adapt their receptive field to a given problem. We
extensively demonstrate the strength of DAUs on a classification and semantic
segmentation tasks. Compared to ConvNets with regular filter, ConvNets with
DAUs achieve comparable performance at faster convergence and up to 3-times
reduction in parameters. Furthermore, DAUs allow us to study deep networks from
novel perspectives. We study spatial distributions of DAU filters and analyze
the number of parameters allocated for spatial coverage in a filter.
Enriching Frame Representations with Distributionally Induced Senses
Authors: Stefano Faralli, Alexander Panchenko, Chris Biemann, Simone Paolo Ponzetto
Date: 2018-03-15 00:00:00
Proceeding: LREC 2018 5

We introduce a new lexical resource that enriches the Framester knowledge
graph, which links Framnet, WordNet, VerbNet and other resources, with semantic
features from text corpora. These features are extracted from distributionally
induced sense inventories and subsequently linked to the manually-constructed
frame representations to boost the performance of frame disambiguation in
context. Since Framester is a frame-based knowledge graph, which enables
full-fledged OWL querying and reasoning, our resource paves the way for the
development of novel, deeper semantic-aware applications that could benefit
from the combination of knowledge from text and complex symbolic
representations of events and participants. Together with the resource we also
provide the software we developed for the evaluation in the task of Word Frame
Disambiguation (WFD).
Local Spectral Graph Convolution for Point Set Feature Learning
Authors: Chu Wang, Babak Samari, Kaleem Siddiqi
Date: 2018-03-15 00:00:00
Proceeding: ECCV 2018 9
Tasks: 3D Point Cloud Classification, Clustering

Feature learning on point clouds has shown great promise, with the
introduction of effective and generalizable deep learning frameworks such as
pointnet++. Thus far, however, point features have been abstracted in an
independent and isolated manner, ignoring the relative layout of neighboring
points as well as their features. In the present article, we propose to
overcome this limitation by using spectral graph convolution on a local graph,
combined with a novel graph pooling strategy. In our approach, graph
convolution is carried out on a nearest neighbor graph constructed from a
point's neighborhood, such that features are jointly learned. We replace the
standard max pooling step with a recursive clustering and pooling strategy,
devised to aggregate information from within clusters of nodes that are close
to one another in their spectral coordinates, leading to richer overall feature
descriptors. Through extensive experiments on diverse datasets, we show a
consistent demonstrable advantage for the tasks of both point set
classification and segmentation.
RUSSE: The First Workshop on Russian Semantic Similarity
Authors: Alexander Panchenko, Natalia Loukachevitch, Dmitry Ustalov, Denis Paperno, Christian Meyer, Natalia Konstantinova
Date: 2018-03-15 00:00:00
Tasks: Semantic Similarity, Semantic Textual Similarity

The paper gives an overview of the Russian Semantic Similarity Evaluation
(RUSSE) shared task held in conjunction with the Dialogue 2015 conference.
There exist a lot of comparative studies on semantic similarity, yet no
analysis of such measures was ever performed for the Russian language.
Exploring this problem for the Russian language is even more interesting,
because this language has features, such as rich morphology and free word
order, which make it significantly different from English, German, and other
well-studied languages. We attempt to bridge this gap by proposing a shared
task on the semantic similarity of Russian nouns. Our key contribution is an
evaluation methodology based on four novel benchmark datasets for the Russian
language. Our analysis of the 105 submissions from 19 teams reveals that
successful approaches for English, such as distributional and skip-gram models,
are directly applicable to Russian as well. On the one hand, the best results
in the contest were obtained by sophisticated supervised models that combine
evidence from different sources. On the other hand, completely unsupervised
approaches, such as a skip-gram model estimated on a large-scale corpus, were
able score among the top 5 systems.
2D Reconstruction of Small Intestine's Interior Wall
Authors: Rahman Attar, Xiang Xie, Zhihua Wang, Shigang Yue
Date: 2018-03-15 00:00:00
Tasks: Image Registration, Image Stitching

Examining and interpreting of a large number of wireless endoscopic images
from the gastrointestinal tract is a tiresome task for physicians. A practical
solution is to automatically construct a two dimensional representation of the
gastrointestinal tract for easy inspection. However, little has been done on
wireless endoscopic image stitching, let alone systematic investigation. The
proposed new wireless endoscopic image stitching method consists of two main
steps to improve the accuracy and efficiency of image registration. First, the
keypoints are extracted by Principle Component Analysis and Scale Invariant
Feature Transform (PCA-SIFT) algorithm and refined with Maximum Likelihood
Estimation SAmple Consensus (MLESAC) outlier removal to find the most reliable
keypoints. Second, the optimal transformation parameters obtained from first
step are fed to the Normalised Mutual Information (NMI) algorithm as an initial
solution. With modified Marquardt-Levenberg search strategy in a multiscale
framework, the NMI can find the optimal transformation parameters in the
shortest time. The proposed methodology has been tested on two different
datasets - one with real wireless endoscopic images and another with images
obtained from Micro-Ball (a new wireless cubic endoscopy system with six image
sensors). The results have demonstrated the accuracy and robustness of the
proposed methodology both visually and quantitatively.
Compressed Sensing MRI Reconstruction using a Generative Adversarial Network with a Cyclic Loss
Authors: Tran Minh Quan, Thanh Nguyen-Duc, Won-Ki Jeong
Date: 2017-09-03 00:00:00
Tasks: compressed sensing, Generative Adversarial Network, MRI Reconstruction

Compressed Sensing MRI (CS-MRI) has provided theoretical foundations upon
which the time-consuming MRI acquisition process can be accelerated. However,
it primarily relies on iterative numerical solvers which still hinders their
adaptation in time-critical applications. In addition, recent advances in deep
neural networks have shown their potential in computer vision and image
processing, but their adaptation to MRI reconstruction is still in an early
stage. In this paper, we propose a novel deep learning-based generative
adversarial model, RefineGAN, for fast and accurate CS-MRI reconstruction. The
proposed model is a variant of fully-residual convolutional autoencoder and
generative adversarial networks (GANs), specifically designed for CS-MRI
formulation; it employs deeper generator and discriminator networks with cyclic
data consistency loss for faithful interpolation in the given under-sampled
k-space data. In addition, our solution leverages a chained network to further
enhance the reconstruction quality. RefineGAN is fast and accurate -- the
reconstruction process is extremely rapid, as low as tens of milliseconds for
reconstruction of a 256x256 image, because it is one-way deployment on a
feed-forward network, and the image quality is superior even for extremely low
sampling rate (as low as 10%) due to the data-driven nature of the method. We
demonstrate that RefineGAN outperforms the state-of-the-art CS-MRI methods by a
large margin in terms of both running time and image quality via evaluation
using several open-source MRI databases.
Theory and Algorithms for Forecasting Time Series
Authors: Vitaly Kuznetsov, Mehryar Mohri
Date: 2018-03-15 00:00:00
Tasks: Time Series, Time Series Analysis, Time Series Forecasting

We present data-dependent learning bounds for the general scenario of
non-stationary non-mixing stochastic processes. Our learning guarantees are
expressed in terms of a data-dependent measure of sequential complexity and a
discrepancy measure that can be estimated from data under some mild
assumptions. We also also provide novel analysis of stable time series
forecasting algorithm using this new notion of discrepancy that we introduce.
We use our learning bounds to devise new algorithms for non-stationary time
series forecasting for which we report some preliminary experimental results.
An End-to-End Goal-Oriented Dialog System with a Generative Natural Language Response Generation
Authors: Stefan Constantin, Jan Niehues, Alex Waibel
Date: 2018-03-06 00:00:00
Tasks: Goal-Oriented Dialog, Response Generation

Recently advancements in deep learning allowed the development of end-to-end
trained goal-oriented dialog systems. Although these systems already achieve
good performance, some simplifications limit their usage in real-life
scenarios.
  In this work, we address two of these limitations: ignoring positional
information and a fixed number of possible response candidates. We propose to
use positional encodings in the input to model the word order of the user
utterances. Furthermore, by using a feedforward neural network, we are able to
generate the output word by word and are no longer restricted to a fixed number
of possible response candidates. Using the positional encoding, we were able to
achieve better accuracies in the Dialog bAbI Tasks and using the feedforward
neural network for generating the response, we were able to save computation
time and space consumption.
HENet:A Highly Efficient Convolutional Neural Networks Optimized for Accuracy, Speed and Storage
Authors: Qiuyu Zhu, Ruixin Zhang
Date: 2018-03-07 00:00:00

In order to enhance the real-time performance of convolutional neural
networks(CNNs), more and more researchers are focusing on improving the
efficiency of CNN. Based on the analysis of some CNN architectures, such as
ResNet, DenseNet, ShuffleNet and so on, we combined their advantages and
proposed a very efficient model called Highly Efficient Networks(HENet). The
new architecture uses an unusual way to combine group convolution and channel
shuffle which was mentioned in ShuffleNet. Inspired by ResNet and DenseNet, we
also proposed a new way to use element-wise addition and concatenation
connection with each block. In order to make greater use of feature maps,
pooling operations are removed from HENet. The experiments show that our
model's efficiency is more than 1 times higher than ShuffleNet on many open
source datasets, such as CIFAR-10/100 and SVHN.
Deep Architectures for Learning Context-dependent Ranking Functions
Authors: Karlson Pfannschmidt, Pritha Gupta, Eyke Hüllermeier
Date: 2018-03-15 00:00:00
Tasks: Object

Object ranking is an important problem in the realm of preference learning.
On the basis of training data in the form of a set of rankings of objects,
which are typically represented as feature vectors, the goal is to learn a
ranking function that predicts a linear order of any new set of objects.
Current approaches commonly focus on ranking by scoring, i.e., on learning an
underlying latent utility function that seeks to capture the inherent utility
of each object. These approaches, however, are not able to take possible
effects of context-dependence into account, where context-dependence means that
the utility or usefulness of an object may also depend on what other objects
are available as alternatives. In this paper, we formalize the problem of
context-dependent ranking and present two general approaches based on two
natural representations of context-dependent ranking functions. Both approaches
are instantiated by means of appropriate neural network architectures, which
are evaluated on suitable benchmark task.
Aggregated Sparse Attention for Steering Angle Prediction
Authors: Sen He, Dmitry Kangin, Yang Mi, Nicolas Pugeault
Date: 2018-03-15 00:00:00
Tasks: Autonomous Driving, Prediction

In this paper, we apply the attention mechanism to autonomous driving for
steering angle prediction. We propose the first model, applying the recently
introduced sparse attention mechanism to visual domain, as well as the
aggregated extension for this model. We show the improvement of the proposed
method, comparing to no attention as well as to different types of attention.
Minimax optimal rates for Mondrian trees and forests
Authors: Jaouad Mourtada, Stéphane Gaïffas, Erwan Scornet
Date: 2018-03-15 00:00:00

Introduced by Breiman, Random Forests are widely used classification and
regression algorithms. While being initially designed as batch algorithms,
several variants have been proposed to handle online learning. One particular
instance of such forests is the \emph{Mondrian Forest}, whose trees are built
using the so-called Mondrian process, therefore allowing to easily update their
construction in a streaming fashion. In this paper, we provide a thorough
theoretical study of Mondrian Forests in a batch learning setting, based on new
results about Mondrian partitions. Our results include consistency and
convergence rates for Mondrian Trees and Forests, that turn out to be minimax
optimal on the set of $s$-H\"older function with $s \in (0,1]$ (for trees and
forests) and $s \in (1,2]$ (for forests only), assuming a proper tuning of
their complexity parameter in both cases. Furthermore, we prove that an
adaptive procedure (to the unknown $s \in (0, 2]$) can be constructed by
combining Mondrian Forests with a standard model aggregation algorithm. These
results are the first demonstrating that some particular random forests achieve
minimax rates \textit{in arbitrary dimension}. Owing to their remarkably simple
distributional properties, which lead to minimax rates, Mondrian trees are a
promising basis for more sophisticated yet theoretically sound random forests
variants.
Tick: a Python library for statistical learning, with a particular emphasis on time-dependent modelling
Authors: Emmanuel Bacry, Martin Bompaire, Stéphane Gaïffas, Soren Poulsen
Date: 2017-07-10 00:00:00
Tasks: Point Processes, Survival Analysis

Tick is a statistical learning library for Python~3, with a particular
emphasis on time-dependent models, such as point processes, and tools for
generalized linear models and survival analysis. The core of the library is an
optimization module providing model computational classes, solvers and proximal
operators for regularization. tick relies on a C++ implementation and
state-of-the-art optimization algorithms to provide very fast computations in a
single node multi-core setting. Source code and documentation can be downloaded
from https://github.com/X-DataInitiative/tick
Salient Region Segmentation
Authors: Sen He, Nicolas Pugeault
Date: 2018-03-15 00:00:00
Tasks: Gaze Prediction, Prediction, regression, Saliency Prediction, Segmentation

Saliency prediction is a well studied problem in computer vision. Early
saliency models were based on low-level hand-crafted feature derived from
insights gained in neuroscience and psychophysics. In the wake of deep learning
breakthrough, a new cohort of models were proposed based on neural network
architectures, allowing significantly higher gaze prediction than previous
shallow models, on all metrics.
  However, most models treat the saliency prediction as a \textit{regression}
problem, and accurate regression of high-dimensional data is known to be a hard
problem. Furthermore, it is unclear that intermediate levels of saliency (ie,
neither very high, nor very low) are meaningful: Something is either salient,
or it is not.
  Drawing from those two observations, we reformulate the saliency prediction
problem as a salient region \textit{segmentation} problem. We demonstrate that
the reformulation allows for faster convergence than the classical regression
problem, while performance is comparable to state-of-the-art.
  We also visualise the general features learned by the model, which are showed
to be consistent with insights from psychophysics.
Rearrangement with Nonprehensile Manipulation Using Deep Reinforcement Learning
Authors: Weihao Yuan, Johannes A. Stork, Danica Kragic, Michael Y. Wang, Kaiyu Hang
Date: 2018-03-15 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Rearranging objects on a tabletop surface by means of nonprehensile
manipulation is a task which requires skillful interaction with the physical
world. Usually, this is achieved by precisely modeling physical properties of
the objects, robot, and the environment for explicit planning. In contrast, as
explicitly modeling the physical environment is not always feasible and
involves various uncertainties, we learn a nonprehensile rearrangement strategy
with deep reinforcement learning based on only visual feedback. For this, we
model the task with rewards and train a deep Q-network. Our potential
field-based heuristic exploration strategy reduces the amount of collisions
which lead to suboptimal outcomes and we actively balance the training set to
avoid bias towards poor examples. Our training process leads to quicker
learning and better performance on the task as compared to uniform exploration
and standard experience replay. We demonstrate empirical evidence from
simulation that our method leads to a success rate of 85%, show that our system
can cope with sudden changes of the environment, and compare our performance
with human level performance.