pwc-archive/papers-with-abstracts huggingface dataset:
On the stable recovery of deep structured linear networks under sparsity constraints
Authors: Francois Malgouyres
Date: 2017-05-31 00:00:00

We consider a deep structured linear network under sparsity constraints. We study sharp conditions guaranteeing the stability of the optimal parameters defining the network. More precisely, we provide sharp conditions on the network architecture and the sample under which the error on the parameters defining the network scales linearly with the reconstruction error (i.e. the risk). Therefore, under these conditions, the weights obtained with a successful algorithms are well defined and only depend on the architecture of the network and the sample. The features in the latent spaces are stably defined. The stability property is required in order to interpret the features defined in the latent spaces. It can also lead to a guarantee on the statistical risk. This is what motivates this study. The analysis is based on the recently proposed Tensorial Lifting. The particularity of this paper is to consider a sparsity prior. This leads to a better stability constant. As an illustration, we detail the analysis and provide sharp stability guarantees for convolutional linear network under sparsity prior. In this analysis, we distinguish the role of the network architecture and the sample input. This highlights the requirements on the data in connection to parameter stability.
Memetic Graph Clustering
Authors: Sonja Biedermann, Monika Henzinger, Christian Schulz, Bernhard Schuster
Date: 2018-02-20 00:00:00
Tasks: Clustering, Graph Clustering

It is common knowledge that there is no single best strategy for graph
clustering, which justifies a plethora of existing approaches. In this paper,
we present a general memetic algorithm, VieClus, to tackle the graph clustering
problem. This algorithm can be adapted to optimize different objective
functions. A key component of our contribution are natural recombine operators
that employ ensemble clusterings as well as multi-level techniques. Lastly, we
combine these techniques with a scalable communication protocol, producing a
system that is able to compute high-quality solutions in a short amount of
time. We instantiate our scheme with local search for modularity and show that
our algorithm successfully improves or reproduces all entries of the 10th
DIMACS implementation~challenge under consideration using a small amount of
time.
Self-ensembling for visual domain adaptation
Authors: Geoffrey French, Michal Mackiewicz, Mark Fisher
Date: 2017-06-16 00:00:00
Proceeding: ICLR 2018 1
Tasks: Domain Adaptation

This paper explores the use of self-ensembling for visual domain adaptation
problems. Our technique is derived from the mean teacher variant (Tarvainen et
al., 2017) of temporal ensembling (Laine et al;, 2017), a technique that
achieved state of the art results in the area of semi-supervised learning. We
introduce a number of modifications to their approach for challenging domain
adaptation scenarios and evaluate its effectiveness. Our approach achieves
state of the art results in a variety of benchmarks, including our winning
entry in the VISDA-2017 visual domain adaptation challenge. In small image
benchmarks, our algorithm not only outperforms prior art, but can also achieve
accuracy that is close to that of a classifier trained in a supervised fashion.
Fusing Video and Inertial Sensor Data for Walking Person Identification
Authors: Yuehong Huang, Yu-Chee Tseng
Date: 2018-02-20 00:00:00
Tasks: Person Identification

An autonomous computer system (such as a robot) typically needs to identify,
locate, and track persons appearing in its sight. However, most solutions have
their limitations regarding efficiency, practicability, or environmental
constraints. In this paper, we propose an effective and practical system which
combines video and inertial sensors for person identification (PID). Persons
who do different activities are easy to identify. To show the robustness and
potential of our system, we propose a walking person identification (WPID)
method to identify persons walking at the same time. By comparing features
derived from both video and inertial sensor data, we can associate sensors in
smartphones with human objects in videos. Results show that the correctly
identified rate of our WPID method can up to 76% in 2 seconds.
Segmentation hiérarchique faiblement supervisée
Authors: Amin Fehri, Santiago Velasco-Forero, Fernand Meyer
Date: 2018-02-20 00:00:00
Tasks: Image Segmentation, Position, Segmentation, Semantic Segmentation, Weakly supervised segmentation

Image segmentation is the process of partitioning an image into a set of
meaningful regions according to some criteria. Hierarchical segmentation has
emerged as a major trend in this regard as it favors the emergence of important
regions at different scales. On the other hand, many methods allow us to have
prior information on the position of structures of interest in the images. In
this paper, we present a versatile hierarchical segmentation method that takes
into account any prior spatial information and outputs a hierarchical
segmentation that emphasizes the contours or regions of interest while
preserving the important structures in the image. An application of this method
to the weakly-supervised segmentation problem is presented.
Traffic Graph Convolutional Recurrent Neural Network: A Deep Learning Framework for Network-Scale Traffic Learning and Forecasting
Authors: Zhiyong Cui, Kristian Henrickson, Ruimin Ke, Ziyuan Pu, Yinhai Wang
Date: 2018-02-20 00:00:00
Tasks: Traffic Prediction

Traffic forecasting is a particularly challenging application of spatiotemporal forecasting, due to the time-varying traffic patterns and the complicated spatial dependencies on road networks. To address this challenge, we learn the traffic network as a graph and propose a novel deep learning framework, Traffic Graph Convolutional Long Short-Term Memory Neural Network (TGC-LSTM), to learn the interactions between roadways in the traffic network and forecast the network-wide traffic state. We define the traffic graph convolution based on the physical network topology. The relationship between the proposed traffic graph convolution and the spectral graph convolution is also discussed. An L1-norm on graph convolution weights and an L2-norm on graph convolution features are added to the model's loss function to enhance the interpretability of the proposed model. Experimental results show that the proposed model outperforms baseline methods on two real-world traffic state datasets. The visualization of the graph convolution weights indicates that the proposed framework can recognize the most influential road segments in real-world traffic networks.
Recent Trends in Deep Learning Based Natural Language Processing
Authors: Tom Young, Devamanyu Hazarika, Soujanya Poria, Erik Cambria
Date: 2017-08-09 00:00:00
Tasks: Deep Learning

Deep learning methods employ multiple processing layers to learn hierarchical
representations of data and have produced state-of-the-art results in many
domains. Recently, a variety of model designs and methods have blossomed in the
context of natural language processing (NLP). In this paper, we review
significant deep learning related models and methods that have been employed
for numerous NLP tasks and provide a walk-through of their evolution. We also
summarize, compare and contrast the various models and put forward a detailed
understanding of the past, present and future of deep learning in NLP.
Feature-Guided Black-Box Safety Testing of Deep Neural Networks
Authors: Matthew Wicker, Xiaowei Huang, Marta Kwiatkowska
Date: 2017-10-21 00:00:00
Tasks: object-detection, Object Detection, Self-Driving Cars, Traffic Sign Recognition

Despite the improved accuracy of deep neural networks, the discovery of
adversarial examples has raised serious safety concerns. Most existing
approaches for crafting adversarial examples necessitate some knowledge
(architecture, parameters, etc.) of the network at hand. In this paper, we
focus on image classifiers and propose a feature-guided black-box approach to
test the safety of deep neural networks that requires no such knowledge. Our
algorithm employs object detection techniques such as SIFT (Scale Invariant
Feature Transform) to extract features from an image. These features are
converted into a mutable saliency distribution, where high probability is
assigned to pixels that affect the composition of the image with respect to the
human visual system. We formulate the crafting of adversarial examples as a
two-player turn-based stochastic game, where the first player's objective is to
minimise the distance to an adversarial example by manipulating the features,
and the second player can be cooperative, adversarial, or random. We show that,
theoretically, the two-player game can con- verge to the optimal strategy, and
that the optimal strategy represents a globally minimal adversarial image. For
Lipschitz networks, we also identify conditions that provide safety guarantees
that no adversarial examples exist. Using Monte Carlo tree search we gradually
explore the game state space to search for adversarial examples. Our
experiments show that, despite the black-box setting, manipulations guided by a
perception-based saliency distribution are competitive with state-of-the-art
methods that rely on white-box saliency matrices or sophisticated optimization
procedures. Finally, we show how our method can be used to evaluate robustness
of neural networks in safety-critical applications such as traffic sign
recognition in self-driving cars.
A Machine Learning Approach to Air Traffic Route Choice Modelling
Authors: Rodrigo Marcos, Oliva García-Cantú, Ricardo Herranz
Date: 2018-02-19 00:00:00
Tasks: BIG-bench Machine Learning, Decision Making, Management, Trajectory Prediction

Air Traffic Flow and Capacity Management (ATFCM) is one of the constituent
parts of Air Traffic Management (ATM). The goal of ATFCM is to make airport and
airspace capacity meet traffic demand and, when capacity opportunities are
exhausted, optimise traffic flows to meet the available capacity. One of the
key enablers of ATFCM is the accurate estimation of future traffic demand. The
available information (schedules, flight plans, etc.) and its associated level
of uncertainty differ across the different ATFCM planning phases, leading to
qualitative differences between the types of forecasting that are feasible at
each time horizon. While abundant research has been conducted on tactical
trajectory prediction (i.e., during the day of operations), trajectory
prediction in the pre-tactical phase, when few or no flight plans are
available, has received much less attention. As a consequence, the methods
currently in use for pre-tactical traffic forecast are still rather
rudimentary, often resulting in suboptimal ATFCM decision making. This paper
proposes a machine learning approach for the prediction of airlines route
choices between two airports as a function of route characteristics, such as
flight efficiency, air navigation charges and expected level of congestion.
Different predictive models based on multinomial logistic regression and
decision trees are formulated and calibrated with historical traffic data, and
a critical evaluation of each model is conducted. We analyse the predictive
power of each model in terms of its ability to forecast traffic volumes at the
level of charging zones, proving significant potential to enhance pre-tactical
traffic forecast. We conclude by discussing the limitations and room for
improvement of the proposed approach, as well as the future developments
required to produce reliable traffic forecasts at a higher spatial and temporal
resolution.
Evaluating Machine Translation Performance on Chinese Idioms with a Blacklist Method
Authors: Yutong Shao, Rico Sennrich, Bonnie Webber, Federico Fancellu
Date: 2017-11-21 00:00:00
Proceeding: LREC 2018 5
Tasks: Machine Translation, Translation, Word Translation

Idiom translation is a challenging problem in machine translation because the
meaning of idioms is non-compositional, and a literal (word-by-word)
translation is likely to be wrong. In this paper, we focus on evaluating the
quality of idiom translation of MT systems. We introduce a new evaluation
method based on an idiom-specific blacklist of literal translations, based on
the insight that the occurrence of any blacklisted words in the translation
output indicates a likely translation error. We introduce a dataset, CIBB
(Chinese Idioms Blacklists Bank), and perform an evaluation of a
state-of-the-art Chinese-English neural MT system. Our evaluation confirms that
a sizable number of idioms in our test set are mistranslated (46.1%), that
literal translation error is a common error type, and that our blacklist method
is effective at identifying literal translation errors.
Randomness in Deconvolutional Networks for Visual Representation
Authors: Kun He, Jingbo Wang, Haochuan Li, Yao Shu, Mengxiao Zhang, Man Zhu, Li-Wei Wang, John E. Hopcroft
Date: 2017-04-02 00:00:00
Tasks: General Classification, Image Reconstruction

Toward a deeper understanding on the inner work of deep neural networks, we
investigate CNN (convolutional neural network) using DCN (deconvolutional
network) and randomization technique, and gain new insights for the intrinsic
property of this network architecture. For the random representations of an
untrained CNN, we train the corresponding DCN to reconstruct the input images.
Compared with the image inversion on pre-trained CNN, our training converges
faster and the yielding network exhibits higher quality for image
reconstruction. It indicates there is rich information encoded in the random
features; the pre-trained CNN may discard information irrelevant for
classification and encode relevant features in a way favorable for
classification but harder for reconstruction. We further explore the property
of the overall random CNN-DCN architecture. Surprisingly, images can be
inverted with satisfactory quality. Extensive empirical evidence as well as
theoretical analysis are provided.
Long-Term Online Smoothing Prediction Using Expert Advice
Authors: Alexander Korotin, Vladimir V'yugin, Evgeny Burnaev
Date: 2017-11-08 00:00:00
Tasks: Prediction, Time Series, Time Series Analysis, Time Series Prediction

For the prediction with experts' advice setting, we construct forecasting
algorithms that suffer loss not much more than any expert in the pool. In
contrast to the standard approach, we investigate the case of long-term
forecasting of time series and consider two scenarios. In the first one, at
each step $t$ the learner has to combine the point forecasts of the experts
issued for the time interval $[t+1, t+d]$ ahead. Our approach implies that at
each time step experts issue point forecasts for arbitrary many steps ahead and
then the learner (algorithm) combines these forecasts and the forecasts made
earlier into one vector forecast for steps $[t+1,t+d]$. By combining past and
the current long-term forecasts we obtain a smoothing mechanism that protects
our algorithm from temporary trend changes, noise and outliers. In the second
scenario, at each step $t$ experts issue a prediction function, and the learner
has to combine these functions into the single one, which will be used for
long-term time-series prediction. For each scenario, we develop an algorithm
for combining experts forecasts and prove $O(\ln T)$ adversarial regret upper
bound for both algorithms.
Fitting New Speakers Based on a Short Untranscribed Sample
Authors: Eliya Nachmani, Adam Polyak, Yaniv Taigman, Lior Wolf
Date: 2018-02-20 00:00:00
Proceeding: ICML 2018 7
Tasks: Speech Synthesis, text-to-speech, Text to Speech

Learning-based Text To Speech systems have the potential to generalize from
one speaker to the next and thus require a relatively short sample of any new
voice. However, this promise is currently largely unrealized. We present a
method that is designed to capture a new speaker from a short untranscribed
audio sample. This is done by employing an additional network that given an
audio sample, places the speaker in the embedding space. This network is
trained as part of the speech synthesis system using various consistency
losses. Our results demonstrate a greatly improved performance on both the
dataset speakers, and, more importantly, when fitting new voices, even from
very short samples.
Unsupervised Band Selection of Hyperspectral Images via Multi-dictionary Sparse Representation
Authors: Fei Li, Pingping Zhang, Huchuan Lu
Date: 2018-02-20 00:00:00
Tasks: Dictionary Learning, General Classification, Hyperspectral Image Classification, image-classification, Image Classification

Hyperspectral images have far more spectral bands than ordinary multispectral
images. Rich band information provides more favorable conditions for the
tremendous applications. However, significant increase in the dimensionality of
spectral bands may lead to the curse of dimensionality, especially for
classification applications. Furthermore, there are a large amount of redundant
information among the raw image cubes due to water absorptions, sensor noises
and other influence factors. Band selection is a direct and effective method to
remove redundant information and reduce the spectral dimension for decreasing
computational complexity and avoiding the curse of dimensionality. In this
paper, we present a novel learning framework for band selection based on the
idea of sparse representation. More specifically, first each band is
approximately represented by the linear combination of other bands, then the
original band image can be represented by a multi-dictionary learning
mechanism. As a result, a group of weights can be obtained by sparse
optimization for all bands. Finally, the specific bands will be selected, if
they get higher weights than other bands in the representation of the original
image. Experimental results on three widely used hyperspectral datasets show
that our proposed algorithm achieves better performance in hyperspectral image
classification, when compared with other state-of-art band selection methods.
Output Reachable Set Estimation and Verification for Multi-Layer Neural Networks
Authors: Weiming Xiang, Hoang-Dung Tran, Taylor T. Johnson
Date: 2017-08-09 00:00:00
Tasks: Sensitivity

In this paper, the output reachable estimation and safety verification
problems for multi-layer perceptron neural networks are addressed. First, a
conception called maximum sensitivity in introduced and, for a class of
multi-layer perceptrons whose activation functions are monotonic functions, the
maximum sensitivity can be computed via solving convex optimization problems.
Then, using a simulation-based method, the output reachable set estimation
problem for neural networks is formulated into a chain of optimization
problems. Finally, an automated safety verification is developed based on the
output reachable set estimation result. An application to the safety
verification for a robotic arm model with two joints is presented to show the
effectiveness of proposed approaches.
Robust positioning of drones for land use monitoring in strong terrain relief using vision-based navigation
Authors: Oleg Kupervasser, Vitalii Sarychev, Alexander Rubinstein, Roman Yavich
Date: 2018-02-20 00:00:00
Tasks: Position

For land use monitoring, the main problems are robust positioning in urban
canyons and strong terrain reliefs with the use of GPS system only. Indeed,
satellite signal reflection and shielding in urban canyons and strong terrain
relief results in problems with correct positioning. Using GNSS-RTK does not
solve the problem completely because in some complex situations the whole
satellite's system works incorrectly. We transform the weakness (urban canyons
and strong terrain relief) to an advantage. It is a vision-based navigation
using a map of the terrain relief. We investigate and demonstrate the
effectiveness of this technology in Chinese region Xiaoshan. The accuracy of
the vision-based navigation system corresponds to the expected for these
conditions. . It was concluded that the maximum position error based on
vision-based navigation is 20 m and the maximum angle Euler error based on
vision-based navigation is 0.83 degree. In case of camera movement, the maximum
position error based on vision-based navigation is 30m and the maximum Euler
angle error based on vision-based navigation is 2.2 degrees.
Heterogeneous Knowledge Transfer in Video Emotion Recognition, Attribution and Summarization
Authors: Baohan Xu, Yanwei Fu, Yu-Gang Jiang, Boyang Li, Leonid Sigal
Date: 2015-11-16 00:00:00
Tasks: Emotion Recognition, Transfer Learning, Video Emotion Recognition, Zero-Shot Learning

Emotion is a key element in user-generated videos. However, it is difficult
to understand emotions conveyed in such videos due to the complex and
unstructured nature of user-generated content and the sparsity of video frames
expressing emotion. In this paper, for the first time, we study the problem of
transferring knowledge from heterogeneous external sources, including image and
textual data, to facilitate three related tasks in understanding video emotion:
emotion recognition, emotion attribution and emotion-oriented summarization.
Specifically, our framework (1) learns a video encoding from an auxiliary
emotional image dataset in order to improve supervised video emotion
recognition, and (2) transfers knowledge from an auxiliary textual corpora for
zero-shot recognition of emotion classes unseen during training. The proposed
technique for knowledge transfer facilitates novel applications of emotion
attribution and emotion-oriented summarization. A comprehensive set of
experiments on multiple datasets demonstrate the effectiveness of our
framework.
Differentiable Dynamic Programming for Structured Prediction and Attention
Authors: Arthur Mensch, Mathieu Blondel
Date: 2018-02-11 00:00:00
Proceeding: ICML 2018 7
Tasks: Machine Translation, Prediction, Structured Prediction, Time Series, Time Series Alignment, Time Series Analysis, Translation

Dynamic programming (DP) solves a variety of structured combinatorial
problems by iteratively breaking them down into smaller subproblems. In spite
of their versatility, DP algorithms are usually non-differentiable, which
hampers their use as a layer in neural networks trained by backpropagation. To
address this issue, we propose to smooth the max operator in the dynamic
programming recursion, using a strongly convex regularizer. This allows to
relax both the optimal value and solution of the original combinatorial
problem, and turns a broad class of DP algorithms into differentiable
operators. Theoretically, we provide a new probabilistic perspective on
backpropagating through these DP operators, and relate them to inference in
graphical models. We derive two particular instantiations of our framework, a
smoothed Viterbi algorithm for sequence prediction and a smoothed DTW algorithm
for time-series alignment. We showcase these instantiations on two structured
prediction tasks and on structured and sparse attention for neural machine
translation.