pwc-archive/papers-with-abstracts huggingface dataset:
Enhancing Time Series Momentum Strategies Using Deep Neural Networks
Authors: Bryan Lim, Stefan Zohren, Stephen Roberts
Date: 2019-04-09 00:00:00
Tasks: Position, Time Series, Time Series Analysis

While time series momentum is a well-studied phenomenon in finance, common strategies require the explicit definition of both a trend estimator and a position sizing rule. In this paper, we introduce Deep Momentum Networks -- a hybrid approach which injects deep learning based trading rules into the volatility scaling framework of time series momentum. The model also simultaneously learns both trend estimation and position sizing in a data-driven manner, with networks directly trained by optimising the Sharpe ratio of the signal. Backtesting on a portfolio of 88 continuous futures contracts, we demonstrate that the Sharpe-optimised LSTM improved traditional methods by more than two times in the absence of transactions costs, and continue outperforming when considering transaction costs up to 2-3 basis points. To account for more illiquid assets, we also propose a turnover regularisation term which trains the network to factor in costs at run-time.
Data Selection with Cluster-Based Language Difference Models and Cynical Selection
Authors: Lucía Santamaría, Amittai Axelrod
Date: 2019-04-09 00:00:00
Proceeding: IWSLT 2017 12
Tasks: Machine Translation, POS, Sentence, Translation

We present and apply two methods for addressing the problem of selecting
relevant training data out of a general pool for use in tasks such as machine
translation. Building on existing work on class-based language difference
models, we first introduce a cluster-based method that uses Brown clusters to
condense the vocabulary of the corpora. Secondly, we implement the cynical data
selection method, which incrementally constructs a training corpus to
efficiently model the task corpus. Both the cluster-based and the cynical data
selection approaches are used for the first time within a machine translation
system, and we perform a head-to-head comparison. Our intrinsic evaluations
show that both new methods outperform the standard Moore-Lewis approach
(cross-entropy difference), in terms of better perplexity and OOV rates on
in-domain data. The cynical approach converges much quicker, covering nearly
all of the in-domain vocabulary with 84% less data than the other methods.
Furthermore, the new approaches can be used to select machine translation
training data for training better systems. Our results confirm that class-based
selection using Brown clusters is a viable alternative to POS-based class-based
methods, and removes the reliance on a part-of-speech tagger. Additionally, we
are able to validate the recently proposed cynical data selection method,
showing that its performance in SMT models surpasses that of traditional
cross-entropy difference methods and more closely matches the sentence length
of the task corpus.
Performance Monitoring for End-to-End Speech Recognition
Authors: Ruizhi Li, Gregory Sell, Hynek Hermansky
Date: 2019-04-09 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Decoder, speech-recognition, Speech Recognition

Measuring performance of an automatic speech recognition (ASR) system without
ground-truth could be beneficial in many scenarios, especially with data from
unseen domains, where performance can be highly inconsistent. In conventional
ASR systems, several performance monitoring (PM) techniques have been
well-developed to monitor performance by looking at tri-phone posteriors or
pre-softmax activations from neural network acoustic modeling. However,
strategies for monitoring more recently developed end-to-end ASR systems have
not yet been explored, and so that is the focus of this paper. We adapt
previous PM measures (Entropy, M-measure and Auto-encoder) and apply our
proposed RNN predictor in the end-to-end setting. These measures utilize the
decoder output layer and attention probability vectors, and their predictive
power is measured with simple linear models. Our findings suggest that
decoder-level features are more feasible and informative than attention-level
probabilities for PM measures, and that M-measure on the decoder posteriors
achieves the best overall predictive performance with an average prediction
error 8.8%. Entropy measures and RNN-based prediction also show competitive
predictability, especially for unseen conditions.
Contextual Attention for Hand Detection in the Wild
Authors: Supreeth Narasimhaswamy, Zhengwei Wei, Yang Wang, Justin Zhang, Minh Hoai
Date: 2019-04-09 00:00:00
Proceeding: ICCV 2019 10
Tasks: Hand Detection, object-detection, Object Detection

We present Hand-CNN, a novel convolutional network architecture for detecting
hand masks and predicting hand orientations in unconstrained images. Hand-CNN
extends MaskRCNN with a novel attention mechanism to incorporate contextual
cues in the detection process. This attention mechanism can be implemented as
an efficient network module that captures non-local dependencies between
features. This network module can be inserted at different stages of an object
detection network, and the entire detector can be trained end-to-end.
  We also introduce a large-scale annotated hand dataset containing hands in
unconstrained images for training and evaluation. We show that Hand-CNN
outperforms existing methods on several datasets, including our hand detection
benchmark and the publicly available PASCAL VOC human layout challenge. We also
conduct ablation studies on hand detection to show the effectiveness of the
proposed contextual attention module.
Non-Lambertian Surface Shape and Reflectance Reconstruction Using Concentric Multi-Spectral Light Field
Authors: Mingyuan Zhou, Yu Ji, Yuqi Ding, Jinwei Ye, S. Susan Young, Jingyi Yu
Date: 2019-04-09 00:00:00
Tasks: Depth Estimation

Recovering the shape and reflectance of non-Lambertian surfaces remains a
challenging problem in computer vision since the view-dependent appearance
invalidates traditional photo-consistency constraint. In this paper, we
introduce a novel concentric multi-spectral light field (CMSLF) design that is
able to recover the shape and reflectance of surfaces with arbitrary material
in one shot. Our CMSLF system consists of an array of cameras arranged on
concentric circles where each ring captures a specific spectrum. Coupled with a
multi-spectral ring light, we are able to sample viewpoint and lighting
variations in a single shot via spectral multiplexing. We further show that
such concentric camera/light setting results in a unique pattern of specular
changes across views that enables robust depth estimation. We formulate a
physical-based reflectance model on CMSLF to estimate depth and multi-spectral
reflectance map without imposing any surface prior. Extensive synthetic and
real experiments show that our method outperforms state-of-the-art light
field-based techniques, especially in non-Lambertian scenes.
Knowledge Distillation for Human Action Anticipation
Authors: Vinh Tran, Yang Wang, Minh Hoai
Date: 2019-04-09 00:00:00
Tasks: Action Anticipation, Action Recognition, Knowledge Distillation, Self-Supervised Learning, Temporal Action Localization

We consider the task of training a neural network to anticipate human actions in video. This task is challenging given the complexity of video data, the stochastic nature of the future, and the limited amount of annotated training data. In this paper, we propose a novel knowledge distillation framework that uses an action recognition network to supervise the training of an action anticipation network, guiding the latter to attend to the relevant information needed for correctly anticipating the future actions. This framework is possible thanks to a novel loss function to account for positional shifts of semantic concepts in a dynamic video. The knowledge distillation framework is a form of self-supervised learning, and it takes advantage of unlabeled data. Experimental results on JHMDB and EPIC-KITCHENS dataset show the effectiveness of our approach.
Black-Box Complexity of the Binary Value Function
Authors: Nina Bulanova, Maxim Buzdalov
Date: 2019-04-09 00:00:00

The binary value function, or BinVal, has appeared in several studies in
theory of evolutionary computation as one of the extreme examples of linear
pseudo-Boolean functions. Its unbiased black-box complexity was previously
shown to be at most $\lceil \log_2 n \rceil + 2$, where $n$ is the problem
size. We augment it with an upper bound of $\log_2 n + 2.42141558 - o(1)$,
which is more precise for many values of $n$. We also present a lower bound of
$\log_2 n + 1.1186406 - o(1)$. Additionally, we prove that BinVal is an easiest
function among all unimodal pseudo-Boolean functions at least for unbiased
algorithms.
Characterizing the impact of geometric properties of word embeddings on task performance
Authors: Brendan Whitaker, Denis Newman-Griffis, Aparajita Haldar, Hakan Ferhatosmanoglu, Eric Fosler-Lussier
Date: 2019-04-09 00:00:00
Proceeding: WS 2019 6
Tasks: Position, Word Embeddings

Analysis of word embedding properties to inform their use in downstream NLP
tasks has largely been studied by assessing nearest neighbors. However,
geometric properties of the continuous feature space contribute directly to the
use of embedding features in downstream models, and are largely unexplored. We
consider four properties of word embedding geometry, namely: position relative
to the origin, distribution of features in the vector space, global pairwise
distances, and local pairwise distances. We define a sequence of
transformations to generate new embeddings that expose subsets of these
properties to downstream models and evaluate change in task performance to
understand the contribution of each property to NLP models. We transform
publicly available pretrained embeddings from three popular toolkits (word2vec,
GloVe, and FastText) and evaluate on a variety of intrinsic tasks, which model
linguistic information in the vector space, and extrinsic tasks, which use
vectors as input to machine learning models. We find that intrinsic evaluations
are highly sensitive to absolute position, while extrinsic tasks rely primarily
on local similarity. Our findings suggest that future embedding models and
post-processing techniques should focus primarily on similarity to nearby
points in vector space.
SWNet: Small-World Neural Networks and Rapid Convergence
Authors: Mojan Javaheripi, Bita Darvish Rouhani, Farinaz Koushanfar
Date: 2019-04-09 00:00:00
Tasks: General Classification, image-classification, Image Classification

Training large and highly accurate deep learning (DL) models is
computationally costly. This cost is in great part due to the excessive number
of trained parameters, which are well-known to be redundant and compressible
for the execution phase. This paper proposes a novel transformation which
changes the topology of the DL architecture such that it reaches an optimal
cross-layer connectivity. This transformation leverages our important
observation that for a set level of accuracy, convergence is fastest when
network topology reaches the boundary of a Small-World Network. Small-world
graphs are known to possess a specific connectivity structure that enables
enhanced signal propagation among nodes. Our small-world models, called SWNets,
provide several intriguing benefits: they facilitate data (gradient) flow
within the network, enable feature-map reuse by adding long-range connections
and accommodate various network architectures/datasets. Compared to densely
connected networks (e.g., DenseNets), SWNets require a substantially fewer
number of training parameters while maintaining a similar level of
classification accuracy. We evaluate our networks on various DL model
architectures and image classification datasets, namely, CIFAR10, CIFAR100, and
ILSVRC (ImageNet). Our experiments demonstrate an average of ~2.1x improvement
in convergence speed to the desired accuracy
Universal Lipschitz Approximation in Bounded Depth Neural Networks
Authors: Jeremy E. J. Cohen, Todd Huster, Ra Cohen
Date: 2019-04-09 00:00:00
Tasks: BIG-bench Machine Learning

Adversarial attacks against machine learning models are a rather hefty
obstacle to our increasing reliance on these models. Due to this, provably
robust (certified) machine learning models are a major topic of interest.
Lipschitz continuous models present a promising approach to solving this
problem. By leveraging the expressive power of a variant of neural networks
which maintain low Lipschitz constants, we prove that three layer neural
networks using the FullSort activation function are Universal Lipschitz
function Approximators (ULAs). This both explains experimental results and
paves the way for the creation of better certified models going forward. We
conclude by presenting experimental results that suggest that ULAs are a not
just a novelty, but a competitive approach to providing certified classifiers,
using these results to motivate several potential topics of further research.
POSEAMM: A Unified Framework for Solving Pose Problems using an Alternating Minimization Method
Authors: Joao Campos, Joao R. Cardoso, Pedro Miraldo
Date: 2019-04-09 00:00:00
Tasks: Pose Estimation, Translation

Pose estimation is one of the most important problems in computer vision. It
can be divided in two different categories -- absolute and relative -- and may
involve two different types of camera models: central and non-central.
State-of-the-art methods have been designed to solve separately these problems.
This paper presents a unified framework that is able to solve any pose problem
by alternating optimization techniques between two set of parameters, rotation
and translation. In order to make this possible, it is necessary to define an
objective function that captures the problem at hand. Since the objective
function will depend on the rotation and translation it is not possible to
solve it as a simple minimization problem. Hence the use of Alternating
Minimization methods, in which the function will be alternatively minimized
with respect to the rotation and the translation. We show how to use our
framework in three distinct pose problems. Our methods are then benchmarked
with both synthetic and real data, showing their better balance between
computational time and accuracy.
3D Object Instance Recognition and Pose Estimation Using Triplet Loss with Dynamic Margin
Authors: Sergey Zakharov, Wadim Kehl, Benjamin Planche, Andreas Hutter, Slobodan Ilic
Date: 2019-04-09 00:00:00
Tasks: Pose Estimation, Triplet

In this paper, we address the problem of 3D object instance recognition and
pose estimation of localized objects in cluttered environments using
convolutional neural networks. Inspired by the descriptor learning approach of
Wohlhart et al., we propose a method that introduces the dynamic margin in the
manifold learning triplet loss function. Such a loss function is designed to
map images of different objects under different poses to a lower-dimensional,
similarity-preserving descriptor space on which efficient nearest neighbor
search algorithms can be applied. Introducing the dynamic margin allows for
faster training times and better accuracy of the resulting low-dimensional
manifolds. Furthermore, we contribute the following: adding in-plane rotations
(ignored by the baseline method) to the training, proposing new background
noise types that help to better mimic realistic scenarios and improve accuracy
with respect to clutter, adding surface normals as another powerful image
modality representing an object surface leading to better performance than
merely depth, and finally implementing an efficient online batch generation
that allows for better variability during the training phase. We perform an
exhaustive evaluation to demonstrate the effects of our contributions.
Additionally, we assess the performance of the algorithm on the large BigBIRD
dataset to demonstrate good scalability properties of the pipeline with respect
to the number of models.
A Non-linear Differential CNN-Rendering Module for 3D Data Enhancement
Authors: Yonatan Svirsky, Andrei Sharf
Date: 2019-04-09 00:00:00
Tasks: General Classification

In this work we introduce a differential rendering module which allows neural
networks to efficiently process cluttered data. The module is composed of
continuous piecewise differentiable functions defined as a sensor array of
cells embedded in 3D space. Our module is learnable and can be easily
integrated into neural networks allowing to optimize data rendering towards
specific learning tasks using gradient based methods in an end-to-end fashion.
Essentially, the module's sensor cells are allowed to transform independently
and locally focus and sense different parts of the 3D data. Thus, through their
optimization process, cells learn to focus on important parts of the data,
bypassing occlusions, clutter and noise. Since sensor cells originally lie on a
grid, this equals to a highly non-linear rendering of the scene into a 2D
image. Our module performs especially well in presence of clutter and
occlusions. Similarly, it deals well with non-linear deformations and improves
classification accuracy through proper rendering of the data. In our
experiments, we apply our module to demonstrate efficient localization and
classification tasks in cluttered data both 2D and 3D.
Just Jump: Dynamic Neighborhood Aggregation in Graph Neural Networks
Authors: Matthias Fey
Date: 2019-04-09 00:00:00
Tasks: General Classification, Node Classification, Representation Learning

We propose a dynamic neighborhood aggregation (DNA) procedure guided by
(multi-head) attention for representation learning on graphs. In contrast to
current graph neural networks which follow a simple neighborhood aggregation
scheme, our DNA procedure allows for a selective and node-adaptive aggregation
of neighboring embeddings of potentially differing locality. In order to avoid
overfitting, we propose to control the channel-wise connections between input
and output by making use of grouped linear projections. In a number of
transductive node-classification experiments, we demonstrate the effectiveness
of our approach.
Accelerated Nuclear Magnetic Resonance Spectroscopy with Deep Learning
Authors: Xiaobo Qu, Yihui Huang, Hengfa Lu, Tianyu Qiu, Di Guo, Tatiana Agback, Vladislav Orekhov, Zhong Chen
Date: 2019-04-09 00:00:00
Tasks: Deep Learning

Nuclear magnetic resonance (NMR) spectroscopy serves as an indispensable tool in chemistry and biology but often suffers from long experimental time. We present a proof-of-concept of application of deep learning and neural network for high-quality, reliable, and very fast NMR spectra reconstruction from limited experimental data. We show that the neural network training can be achieved using solely synthetic NMR signal, which lifts the prohibiting demand for a large volume of realistic training data usually required in the deep learning approach.
Data assimilation in a nonlinear time-delayed dynamical system
Authors: Tullio Traverso, Luca Magri
Date: 2019-04-09 00:00:00
Tasks: State Estimation

When the heat released by a flame is sufficiently in phase with the acoustic
pressure, a self-excited thermoacoustic oscillation can arise. These nonlinear
oscillations are one of the biggest challenges faced in the design of safe and
reliable gas turbines and rocket motors. In the worst-case scenario,
uncontrolled thermoacoustic oscillations can shake an engine apart.
Reduced-order thermoacoustic models, which are nonlinear and time-delayed, can
only qualitatively predict thermoacoustic oscillations. To make reduced-order
models quantitatively predictive, we develop a data assimilation framework for
state estimation. We numerically estimate the most likely nonlinear state of a
Galerkin-discretized time delayed model of a horizontal Rijke tube, which is a
prototypical combustor. Data assimilation is an optimal blending of
observations with previous state estimates (background) to produce optimal
initial conditions. A cost functional is defined to measure the statistical
distance between the model output and the measurements from experiments; and
the distance between the initial conditions and the background knowledge. Its
minimum corresponds to the optimal state, which is computed by Lagrangian
optimization with the aid of adjoint equations. We study the influence of the
number of Galerkin modes, which are the natural acoustic modes of the duct,
with which the model is discretized. We show that decomposing the measured
pressure signal in a finite number of modes is an effective way to enhance
state estimation, especially when nonlinear modal interactions occur during the
assimilation window. This work represents the first application of data
assimilation to nonlinear thermoacoustics, which opens up new possibilities for
real-time calibration of reduced-order models with experimental measurements.
ContactGrasp: Functional Multi-finger Grasp Synthesis from Contact
Authors: Samarth Brahmbhatt, Ankur Handa, James Hays, Dieter Fox
Date: 2019-04-07 00:00:00
Tasks: Object

Grasping and manipulating objects is an important human skill. Since most objects are designed to be manipulated by human hands, anthropomorphic hands can enable richer human-robot interaction. Desirable grasps are not only stable, but also functional: they enable post-grasp actions with the object. However, functional grasp synthesis for high degree-of-freedom anthropomorphic hands from object shape alone is challenging because of the large optimization space. We present ContactGrasp, a framework for functional grasp synthesis from object shape and contact on the object surface. Contact can be manually specified or obtained through demonstrations. Our contact representation is object-centric and allows functional grasp synthesis even for hand models different than the one used for demonstration. Using a dataset of contact demonstrations from humans grasping diverse household objects, we synthesize functional grasps for three hand models and two functional intents. The project webpage is https://contactdb.cc.gatech.edu/contactgrasp.html.
Modeling a Hidden Dynamical System Using Energy Minimization and Kernel Density Estimates
Authors: Trevor K. Karn, Steven Petrone, Christopher Griffin
Date: 2019-04-08 00:00:00
Tasks: Density Estimation, Trajectory Modeling

In this paper we develop a kernel density estimation (KDE) approach to modeling and forecasting recurrent trajectories on a compact manifold. For the purposes of this paper, a trajectory is a sequence of coordinates in a phase space defined by an underlying hidden dynamical system. Our work is inspired by earlier work on the use of KDE to detect shipping anomalies using high-density, high-quality automated information system (AIS) data as well as our own earlier work in trajectory modeling. We focus specifically on the sparse, noisy trajectory reconstruction problem in which the data are (i) sparsely sampled and (ii) subject to an imperfect observer that introduces noise. Under certain regularity assumptions, we show that the constructed estimator minimizes a specific energy function defined over the trajectory as the number of samples obtained grows.
DeepSphere: towards an equivariant graph-based spherical CNN
Authors: Michaël Defferrard, Nathanaël Perraudin, Tomasz Kacprzak, Raphael Sgier
Date: 2019-04-08 00:00:00
Tasks: Graph Neural Network

Spherical data is found in many applications. By modeling the discretized
sphere as a graph, we can accommodate non-uniformly distributed, partial, and
changing samplings. Moreover, graph convolutions are computationally more
efficient than spherical convolutions. As equivariance is desired to exploit
rotational symmetries, we discuss how to approach rotation equivariance using
the graph neural network introduced in Defferrard et al. (2016). Experiments
show good performance on rotation-invariant learning problems. Code and
examples are available at https://github.com/SwissDataScienceCenter/DeepSphere