pwc-archive/papers-with-abstracts huggingface dataset:
Conditional Similarity Networks
Authors: Andreas Veit, Serge Belongie, Theofanis Karaletsos
Date: 2016-03-25 00:00:00
Proceeding: CVPR 2017 7
Tasks: Triplet

What makes images similar? To measure the similarity between images, they are
typically embedded in a feature-vector space, in which their distance preserve
the relative dissimilarity. However, when learning such similarity embeddings
the simplifying assumption is commonly made that images are only compared to
one unique measure of similarity. A main reason for this is that contradicting
notions of similarities cannot be captured in a single space. To address this
shortcoming, we propose Conditional Similarity Networks (CSNs) that learn
embeddings differentiated into semantically distinct subspaces that capture the
different notions of similarities. CSNs jointly learn a disentangled embedding
where features for different similarities are encoded in separate dimensions as
well as masks that select and reweight relevant dimensions to induce a subspace
that encodes a specific similarity notion. We show that our approach learns
interpretable image representations with visually relevant semantic subspaces.
Further, when evaluating on triplet questions from multiple similarity notions
our model even outperforms the accuracy obtained by training individual
specialized networks for each notion separately.
Continuously heterogeneous hyper-objects in cryo-EM and 3-D movies of many temporal dimensions
Authors: Roy R. Lederman, Amit Singer
Date: 2017-04-10 00:00:00

Single particle cryo-electron microscopy (EM) is an increasingly popular
method for determining the 3-D structure of macromolecules from noisy 2-D
images of single macromolecules whose orientations and positions are random and
unknown. One of the great opportunities in cryo-EM is to recover the structure
of macromolecules in heterogeneous samples, where multiple types or multiple
conformations are mixed together. Indeed, in recent years, many tools have been
introduced for the analysis of multiple discrete classes of molecules mixed
together in a cryo-EM experiment. However, many interesting structures have a
continuum of conformations which do not fit discrete models nicely; the
analysis of such continuously heterogeneous models has remained a more elusive
goal. In this manuscript, we propose to represent heterogeneous molecules and
similar structures as higher dimensional objects. We generalize the basic
operations used in many existing reconstruction algorithms, making our approach
generic in the sense that, in principle, existing algorithms can be adapted to
reconstruct those higher dimensional objects. As proof of concept, we present a
prototype of a new algorithm which we use to solve simulated reconstruction
problems.
ActionVLAD: Learning spatio-temporal aggregation for action classification
Authors: Rohit Girdhar, Deva Ramanan, Abhinav Gupta, Josef Sivic, Bryan Russell
Date: 2017-04-10 00:00:00
Proceeding: CVPR 2017 7
Tasks: Action Classification, Classification, General Classification, Long-video Activity Recognition, Video Classification

In this work, we introduce a new video representation for action
classification that aggregates local convolutional features across the entire
spatio-temporal extent of the video. We do so by integrating state-of-the-art
two-stream networks with learnable spatio-temporal feature aggregation. The
resulting architecture is end-to-end trainable for whole-video classification.
We investigate different strategies for pooling across space and time and
combining signals from the different streams. We find that: (i) it is important
to pool jointly across space and time, but (ii) appearance and motion streams
are best aggregated into their own separate representations. Finally, we show
that our representation outperforms the two-stream base architecture by a large
margin (13% relative) as well as out-performs other baselines with comparable
base architectures on HMDB51, UCF101, and Charades video classification
benchmarks.
Field of Groves: An Energy-Efficient Random Forest
Authors: Zafar Takhirov, Joseph Wang, Marcia S. Louis, Venkatesh Saligrama, Ajay Joshi
Date: 2017-04-10 00:00:00
Tasks: General Classification

Machine Learning (ML) algorithms, like Convolutional Neural Networks (CNN),
Support Vector Machines (SVM), etc. have become widespread and can achieve high
statistical performance. However their accuracy decreases significantly in
energy-constrained mobile and embedded systems space, where all computations
need to be completed under a tight energy budget. In this work, we present a
field of groves (FoG) implementation of random forests (RF) that achieves an
accuracy comparable to CNNs and SVMs under tight energy budgets. Evaluation of
the FoG shows that at comparable accuracy it consumes ~1.48x, ~24x, ~2.5x, and
~34.7x lower energy per classification compared to conventional RF, SVM_RBF ,
MLP, and CNN, respectively. FoG is ~6.5x less energy efficient than SVM_LR, but
achieves 18% higher accuracy on average across all considered datasets.
Expecting the Unexpected: Training Detectors for Unusual Pedestrians with Adversarial Imposters
Authors: Shiyu Huang, Deva Ramanan
Date: 2017-03-18 00:00:00
Proceeding: CVPR 2017 7
Tasks: Autonomous Vehicles, Pedestrian Detection

As autonomous vehicles become an every-day reality, high-accuracy pedestrian
detection is of paramount practical importance. Pedestrian detection is a
highly researched topic with mature methods, but most datasets focus on common
scenes of people engaged in typical walking poses on sidewalks. But performance
is most crucial for dangerous scenarios, such as children playing in the street
or people using bicycles/skateboards in unexpected ways. Such "in-the-tail"
data is notoriously hard to observe, making both training and testing
difficult. To analyze this problem, we have collected a novel annotated dataset
of dangerous scenarios called the Precarious Pedestrian dataset. Even given a
dedicated collection effort, it is relatively small by contemporary standards
(around 1000 images). To allow for large-scale data-driven learning, we explore
the use of synthetic data generated by a game engine. A significant challenge
is selected the right "priors" or parameters for synthesis: we would like
realistic data with poses and object configurations that mimic true Precarious
Pedestrians. Inspired by Generative Adversarial Networks (GANs), we generate a
massive amount of synthetic data and train a discriminative classifier to
select a realistic subset, which we deem the Adversarial Imposters. We
demonstrate that this simple pipeline allows one to synthesize realistic
training data by making use of rendering/animation engines within a GAN
framework. Interestingly, we also demonstrate that such data can be used to
rank algorithms, suggesting that Adversarial Imposters can also be used for
"in-the-tail" validation at test-time, a notoriously difficult challenge for
real-world deployment.
Deep Learning and Quantum Entanglement: Fundamental Connections with Implications to Network Design
Authors: Yoav Levine, David Yakira, Nadav Cohen, Amnon Shashua
Date: 2017-04-05 00:00:00
Proceeding: ICLR 2018 1
Tasks: Inductive Bias

Deep convolutional networks have witnessed unprecedented success in various
machine learning applications. Formal understanding on what makes these
networks so successful is gradually unfolding, but for the most part there are
still significant mysteries to unravel. The inductive bias, which reflects
prior knowledge embedded in the network architecture, is one of them. In this
work, we establish a fundamental connection between the fields of quantum
physics and deep learning. We use this connection for asserting novel
theoretical observations regarding the role that the number of channels in each
layer of the convolutional network fulfills in the overall inductive bias.
Specifically, we show an equivalence between the function realized by a deep
convolutional arithmetic circuit (ConvAC) and a quantum many-body wave
function, which relies on their common underlying tensorial structure. This
facilitates the use of quantum entanglement measures as well-defined
quantifiers of a deep network's expressive ability to model intricate
correlation structures of its inputs. Most importantly, the construction of a
deep ConvAC in terms of a Tensor Network is made available. This description
enables us to carry a graph-theoretic analysis of a convolutional network, with
which we demonstrate a direct control over the inductive bias of the deep
network via its channel numbers, that are related to the min-cut in the
underlying graph. This result is relevant to any practitioner designing a
network for a specific task. We theoretically analyze ConvACs, and empirically
validate our findings on more common ConvNets which involve ReLU activations
and max pooling. Beyond the results described above, the description of a deep
convolutional network in well-defined graph-theoretic tools and the formal
connection to quantum entanglement, are two interdisciplinary bridges that are
brought forth by this work.
An Incremental Parser for Abstract Meaning Representation
Authors: Marco Damonte, Shay B. Cohen, Giorgio Satta
Date: 2016-08-22 00:00:00
Proceeding: EACL 2017 4
Tasks: Abstract Meaning Representation, AMR Parsing, Named Entity Recognition, Named Entity Recognition (NER), Semantic Role Labeling, Word Sense Disambiguation

Meaning Representation (AMR) is a semantic representation for natural
language that embeds annotations related to traditional tasks such as named
entity recognition, semantic role labeling, word sense disambiguation and
co-reference resolution. We describe a transition-based parser for AMR that
parses sentences left-to-right, in linear time. We further propose a test-suite
that assesses specific subtasks that are helpful in comparing AMR parsers, and
show that our parser is competitive with the state of the art on the LDC2015E86
dataset and that it outperforms state-of-the-art parsers for recovering named
entities and handling polarity.
Deep Neural Network Based Precursor microRNA Prediction on Eleven Species
Authors: Jaya Thomas, Lee Sael
Date: 2017-04-10 00:00:00
Tasks: General Classification

MicroRNA (miRNA) are small non-coding RNAs that regulates the gene expression
at the post-transcriptional level. Determining whether a sequence segment is
miRNA is experimentally challenging. Also, experimental results are sensitive
to the experimental environment. These limitations inspire the development of
computational methods for predicting the miRNAs. We propose a deep learning
based classification model, called DP-miRNA, for predicting precursor miRNA
sequence that contains the miRNA sequence. The feature set based Restricted
Boltzmann Machine method, which we call DP-miRNA, uses 58 features that are
categorized into four groups: sequence features, folding measures, stem-loop
features and statistical feature. We evaluate the performance of the DP-miRNA
on eleven twelve data sets of varying species, including the human. The deep
neural network based classification outperformed support vector machine, neural
network, naive Baye's classifiers, k-nearest neighbors, random forests, and a
hybrid system combining support vector machine and genetic algorithm.
Hierarchical Boundary-Aware Neural Encoder for Video Captioning
Authors: Lorenzo Baraldi, Costantino Grana, Rita Cucchiara
Date: 2016-11-28 00:00:00
Proceeding: CVPR 2017 7
Tasks: Decoder, Video Captioning, Video Description

The use of Recurrent Neural Networks for video captioning has recently gained
a lot of attention, since they can be used both to encode the input video and
to generate the corresponding description. In this paper, we present a
recurrent video encoding scheme which can discover and leverage the
hierarchical structure of the video. Unlike the classical encoder-decoder
approach, in which a video is encoded continuously by a recurrent layer, we
propose a novel LSTM cell, which can identify discontinuity points between
frames or segments and modify the temporal connections of the encoding layer
accordingly. We evaluate our approach on three large-scale datasets: the
Montreal Video Annotation dataset, the MPII Movie Description dataset and the
Microsoft Video Description Corpus. Experiments show that our approach can
discover appropriate hierarchical representations of input videos and improve
the state of the art results on movie description datasets.
Differentially Private Variational Inference for Non-conjugate Models
Authors: Joonas Jälkö, Onur Dikmen, Antti Honkela
Date: 2016-10-27 00:00:00
Tasks: Bayesian Inference, Variational Inference

Many machine learning applications are based on data collected from people,
such as their tastes and behaviour as well as biological traits and genetic
data. Regardless of how important the application might be, one has to make
sure individuals' identities or the privacy of the data are not compromised in
the analysis. Differential privacy constitutes a powerful framework that
prevents breaching of data subject privacy from the output of a computation.
Differentially private versions of many important Bayesian inference methods
have been proposed, but there is a lack of an efficient unified approach
applicable to arbitrary models. In this contribution, we propose a
differentially private variational inference method with a very wide
applicability. It is built on top of doubly stochastic variational inference, a
recent advance which provides a variational solution to a large class of
models. We add differential privacy into doubly stochastic variational
inference by clipping and perturbing the gradients. The algorithm is made more
efficient through privacy amplification from subsampling. We demonstrate the
method can reach an accuracy close to non-private level under reasonably strong
privacy guarantees, clearly improving over previous sampling-based alternatives
especially in the strong privacy regime.
Nonnegative Matrix Underapproximation for Robust Multiple Model Fitting
Authors: Mariano Tepper, Guillermo Sapiro
Date: 2016-11-04 00:00:00
Proceeding: CVPR 2017 7
Tasks: model

In this work, we introduce a highly efficient algorithm to address the
nonnegative matrix underapproximation (NMU) problem, i.e., nonnegative matrix
factorization (NMF) with an additional underapproximation constraint. NMU
results are interesting as, compared to traditional NMF, they present
additional sparsity and part-based behavior, explaining unique data features.
To show these features in practice, we first present an application to the
analysis of climate data. We then present an NMU-based algorithm to robustly
fit multiple parametric models to a dataset. The proposed approach delivers
state-of-the-art results for the estimation of multiple fundamental matrices
and homographies, outperforming other alternatives in the literature and
exemplifying the use of efficient NMU computations.
Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation
Authors: Jose Caballero, Christian Ledig, Andrew Aitken, Alejandro Acosta, Johannes Totz, Zehan Wang, Wenzhe Shi
Date: 2016-11-16 00:00:00
Proceeding: CVPR 2017 7
Tasks: Motion Compensation, Video Super-Resolution

Convolutional neural networks have enabled accurate image super-resolution in
real-time. However, recent attempts to benefit from temporal correlations in
video super-resolution have been limited to naive or inefficient architectures.
In this paper, we introduce spatio-temporal sub-pixel convolution networks that
effectively exploit temporal redundancies and improve reconstruction accuracy
while maintaining real-time speed. Specifically, we discuss the use of early
fusion, slow fusion and 3D convolutions for the joint processing of multiple
consecutive video frames. We also propose a novel joint motion compensation and
video super-resolution algorithm that is orders of magnitude more efficient
than competing methods, relying on a fast multi-resolution spatial transformer
module that is end-to-end trainable. These contributions provide both higher
accuracy and temporally more consistent videos, which we confirm qualitatively
and quantitatively. Relative to single-frame models, spatio-temporal networks
can either reduce the computational cost by 30% whilst maintaining the same
quality or provide a 0.2dB gain for a similar computational cost. Results on
publicly available datasets demonstrate that the proposed algorithms surpass
current state-of-the-art performance in both accuracy and efficiency.
Character-Word LSTM Language Models
Authors: Lyan Verwimp, Joris Pelemans, Hugo Van hamme, Patrick Wambacq
Date: 2017-04-10 00:00:00
Proceeding: EACL 2017 4
Tasks: Language Modeling, Language Modelling

We present a Character-Word Long Short-Term Memory Language Model which both
reduces the perplexity with respect to a baseline word-level language model and
reduces the number of parameters of the model. Character information can reveal
structural (dis)similarities between words and can even be used when a word is
out-of-vocabulary, thus improving the modeling of infrequent and unknown words.
By concatenating word and character embeddings, we achieve up to 2.77% relative
improvement on English compared to a baseline model with a similar amount of
parameters and 4.57% on Dutch. Moreover, we also outperform baseline word-level
models with a larger number of parameters.
R-Clustering for Egocentric Video Segmentation
Authors: Estefania Talavera, Mariella Dimiccoli, Marc Bolaños, Maedeh Aghaei, Petia Radeva
Date: 2017-04-10 00:00:00
Tasks: Clustering, Drift Detection, Segmentation, Video Segmentation, Video Semantic Segmentation

In this paper, we present a new method for egocentric video temporal
segmentation based on integrating a statistical mean change detector and
agglomerative clustering(AC) within an energy-minimization framework. Given the
tendency of most AC methods to oversegment video sequences when clustering
their frames, we combine the clustering with a concept drift detection
technique (ADWIN) that has rigorous guarantee of performances. ADWIN serves as
a statistical upper bound for the clustering-based video segmentation. We
integrate both techniques in an energy-minimization framework that serves to
disambiguate the decision of both techniques and to complete the segmentation
taking into account the temporal continuity of video frames descriptors. We
present experiments over egocentric sets of more than 13.000 images acquired
with different wearable cameras, showing that our method outperforms
state-of-the-art clustering methods.
Learning Motion Patterns in Videos
Authors: Pavel Tokmakov, Karteek Alahari, Cordelia Schmid
Date: 2016-12-21 00:00:00
Proceeding: CVPR 2017 7
Tasks: Decoder, Motion Segmentation, Optical Flow Estimation, Segmentation, Semantic Segmentation, Unsupervised Video Object Segmentation

The problem of determining whether an object is in motion, irrespective of
camera motion, is far from being solved. We address this challenging task by
learning motion patterns in videos. The core of our approach is a fully
convolutional network, which is learned entirely from synthetic video
sequences, and their ground-truth optical flow and motion segmentation. This
encoder-decoder style architecture first learns a coarse representation of the
optical flow field features, and then refines it iteratively to produce motion
labels at the original high-resolution. We further improve this labeling with
an objectness map and a conditional random field, to account for errors in
optical flow, and also to focus on moving "things" rather than "stuff". The
output label of each pixel denotes whether it has undergone independent motion,
i.e., irrespective of camera motion. We demonstrate the benefits of this
learning framework on the moving object segmentation task, where the goal is to
segment all objects in motion. Our approach outperforms the top method on the
recently released DAVIS benchmark dataset, comprising real-world sequences, by
5.6%. We also evaluate on the Berkeley motion segmentation database, achieving
state-of-the-art results.
Spatio-Temporal Modeling of Users' Check-ins in Location-Based Social Networks
Authors: Ali Zarezade, Sina Jafarzadeh, Hamid R. Rabiee
Date: 2016-11-23 00:00:00

Social networks are getting closer to our real physical world. People share
the exact location and time of their check-ins and are influenced by their
friends. Modeling the spatio-temporal behavior of users in social networks is
of great importance for predicting the future behavior of users, controlling
the users' movements, and finding the latent influence network. It is observed
that users have periodic patterns in their movements. Also, they are influenced
by the locations that their close friends recently visited. Leveraging these
two observations, we propose a probabilistic model based on a doubly stochastic
point process with a periodic decaying kernel for the time of check-ins and a
time-varying multinomial distribution for the location of check-ins of users in
the location-based social networks. We learn the model parameters using an
efficient EM algorithm, which distributes over the users. Experiments on
synthetic and real data gathered from Foursquare show that the proposed
inference algorithm learns the parameters efficiently and our model outperforms
the other alternatives in the prediction of time and location of check-ins.
SMASH: Physics-guided Reconstruction of Collisions from Videos
Authors: Aron Monszpart, Nils Thuerey, Niloy J. Mitra
Date: 2016-03-29 00:00:00
Tasks: 3D Reconstruction, valid

Collision sequences are commonly used in games and entertainment to add drama
and excitement. Authoring even two body collisions in the real world can be
difficult, as one has to get timing and the object trajectories to be correctly
synchronized. After tedious trial-and-error iterations, when objects can
actually be made to collide, then they are difficult to capture in 3D. In
contrast, synthetically generating plausible collisions is difficult as it
requires adjusting different collision parameters (e.g., object mass ratio,
coefficient of restitution, etc.) and appropriate initial parameters. We
present SMASH to directly read off appropriate collision parameters directly
from raw input video recordings. Technically we enable this by utilizing laws
of rigid body collision to regularize the problem of lifting 2D trajectories to
a physically valid 3D reconstruction of the collision. The reconstructed
sequences can then be modified and combined to easily author novel and
plausible collisions. We evaluate our system on a range of synthetic scenes and
demonstrate the effectiveness of our method by accurately reconstructing
several complex real world collision events.