pwc-archive/papers-with-abstracts huggingface dataset:
Analysis of the Generalization Error: Empirical Risk Minimization over Deep Artificial Neural Networks Overcomes the Curse of Dimensionality in the Numerical Approximation of Black-Scholes Partial Differential Equations
Authors: Julius Berner, Philipp Grohs, Arnulf Jentzen
Date: 2018-09-09 00:00:00

The development of new classification and regression algorithms based on empirical risk minimization (ERM) over deep neural network hypothesis classes, coined deep learning, revolutionized the area of artificial intelligence, machine learning, and data analysis. In particular, these methods have been applied to the numerical solution of high-dimensional partial differential equations with great success. Recent simulations indicate that deep learning-based algorithms are capable of overcoming the curse of dimensionality for the numerical solution of Kolmogorov equations, which are widely used in models from engineering, finance, and the natural sciences. The present paper considers under which conditions ERM over a deep neural network hypothesis class approximates the solution of a $d$-dimensional Kolmogorov equation with affine drift and diffusion coefficients and typical initial values arising from problems in computational finance up to error $\varepsilon$. We establish that, with high probability over draws of training samples, such an approximation can be achieved with both the size of the hypothesis class and the number of training samples scaling only polynomially in $d$ and $\varepsilon^{-1}$. It can be concluded that ERM over deep neural network hypothesis classes overcomes the curse of dimensionality for the numerical solution of linear Kolmogorov equations with affine coefficients.
Active Inverse Reward Design
Authors: Sören Mindermann, Rohin Shah, Adam Gleave, Dylan Hadfield-Menell
Date: 2018-09-09 00:00:00
Tasks: Informativeness

Designers of AI agents often iterate on the reward function in a trial-and-error process until they get the desired behavior, but this only guarantees good behavior in the training environment. We propose structuring this process as a series of queries asking the user to compare between different reward functions. Thus we can actively select queries for maximum informativeness about the true reward. In contrast to approaches asking the designer for optimal behavior, this allows us to gather additional information by eliciting preferences between suboptimal behaviors. After each query, we need to update the posterior over the true reward function from observing the proxy reward function chosen by the designer. The recently proposed Inverse Reward Design (IRD) enables this. Our approach substantially outperforms IRD in test environments. In particular, it can query the designer about interpretable, linear reward functions and still infer non-linear ones.
SHOMA at Parseme Shared Task on Automatic Identification of VMWEs: Neural Multiword Expression Tagging with High Generalisation
Authors: Shiva Taslimipoor, Omid Rohanian
Date: 2018-09-09 00:00:00
Tasks: Word Embeddings

This paper presents a language-independent deep learning architecture adapted
to the task of multiword expression (MWE) identification. We employ a neural
architecture comprising of convolutional and recurrent layers with the addition
of an optional CRF layer at the top. This system participated in the open track
of the Parseme shared task on automatic identification of verbal MWEs due to
the use of pre-trained wikipedia word embeddings. It outperformed all
participating systems in both open and closed tracks with the overall
macro-average MWE-based F1 score of 58.09 averaged among all languages. A
particular strength of the system is its superior performance on unseen data
entries.
LDW-SCSA: Logistic Dynamic Weight based Sine Cosine Search Algorithm for Numerical Functions Optimization
Authors: Turker Tuncer
Date: 2018-09-09 00:00:00

Particle swarm optimization (PSO) and Sine Cosine algorithm (SCA) have been
widely used optimization methods but these methods have some disadvantages such
as trapped local optimum point. In order to solve this problem and obtain more
successful results than others, a novel logistic dynamic weight based sine
cosine search algorithm (LDW-SCSA) is presented in this paper. In the LDW-SCSA
method, logistic map is used as dynamic weight generator. Logistic map is one
of the famous and widely used chaotic map in the literature. Search process of
SCA is modified in the LDW-SCSA. To evaluate performance of the LDW-SCSA, the
widely used numerical benchmark functions were utilized as test suite and other
swarm optimization methods were used to obtain the comparison results. Superior
performances of the LDW-SCSA are proved success of this method.
SEGA: Variance Reduction via Gradient Sketching
Authors: Filip Hanzely, Konstantin Mishchenko, Peter Richtarik
Date: 2018-09-09 00:00:00
Proceeding: NeurIPS 2018 12

We propose a randomized first order optimization method--SEGA (SkEtched
GrAdient method)-- which progressively throughout its iterations builds a
variance-reduced estimate of the gradient from random linear measurements
(sketches) of the gradient obtained from an oracle. In each iteration, SEGA
updates the current estimate of the gradient through a sketch-and-project
operation using the information provided by the latest sketch, and this is
subsequently used to compute an unbiased estimate of the true gradient through
a random relaxation procedure. This unbiased estimate is then used to perform a
gradient step. Unlike standard subspace descent methods, such as coordinate
descent, SEGA can be used for optimization problems with a non-separable
proximal term. We provide a general convergence analysis and prove linear
convergence for strongly convex objectives. In the special case of coordinate
sketches, SEGA can be enhanced with various techniques such as importance
sampling, minibatching and acceleration, and its rate is up to a small constant
factor identical to the best-known rate of coordinate descent.
Attentional Multi-Reading Sarcasm Detection
Authors: Reza Ghaeini, Xiaoli Z. Fern, Prasad Tadepalli
Date: 2018-09-09 00:00:00
Tasks: Sarcasm Detection

Recognizing sarcasm often requires a deep understanding of multiple sources
of information, including the utterance, the conversational context, and real
world facts. Most of the current sarcasm detection systems consider only the
utterance in isolation. There are some limited attempts toward taking into
account the conversational context. In this paper, we propose an interpretable
end-to-end model that combines information from both the utterance and the
conversational context to detect sarcasm, and demonstrate its effectiveness
through empirical evaluations. We also study the behavior of the proposed model
to provide explanations for the model's decisions. Importantly, our model is
capable of determining the impact of utterance and conversational context on
the model's decisions. Finally, we provide an ablation study to illustrate the
impact of different components of the proposed model.
TextContourNet: a Flexible and Effective Framework for Improving Scene Text Detection Architecture with a Multi-task Cascade
Authors: Dafang He, Xiao Yang, Daniel Kifer, C. Lee Giles
Date: 2018-09-09 00:00:00
Tasks: Scene Text Detection, Text Detection

We study the problem of extracting text instance contour information from
images and use it to assist scene text detection. We propose a novel and
effective framework for this and experimentally demonstrate that: (1) A CNN
that can be effectively used to extract instance-level text contour from
natural images. (2) The extracted contour information can be used for better
scene text detection. We propose two ways for learning the contour task
together with the scene text detection: (1) as an auxiliary task and (2) as
multi-task cascade. Extensive experiments with different benchmark datasets
demonstrate that both designs improve the performance of a state-of-the-art
scene text detector and that a multi-task cascade design achieves the best
performance.
Distance preserving model order reduction of graph-Laplacians and cluster analysis
Authors: Vladimir Druskin, Alexander V. Mamonov, Mikhail Zaslavsky
Date: 2018-09-09 00:00:00
Tasks: Clustering, Dimensionality Reduction, Graph Clustering

Graph-Laplacians and their spectral embeddings play an important role in multiple areas of machine learning. This paper is focused on graph-Laplacian dimension reduction for the spectral clustering of data as a primary application. Spectral embedding provides a low-dimensional parametrization of the data manifold which makes the subsequent task (e.g., clustering) much easier. However, despite reducing the dimensionality of data, the overall computational cost may still be prohibitive for large data sets due to two factors. First, computing the partial eigendecomposition of the graph-Laplacian typically requires a large Krylov subspace. Second, after the spectral embedding is complete, one still has to operate with the same number of data points. For example, clustering of the embedded data is typically performed with various relaxations of k-means which computational cost scales poorly with respect to the size of data set. In this work, we switch the focus from the entire data set to a subset of graph vertices (target subset). We develop two novel algorithms for such low-dimensional representation of the original graph that preserves important global distances between the nodes of the target subset. In particular, it allows to ensure that target subset clustering is consistent with the spectral clustering of the full data set if one would perform such. That is achieved by a properly parametrized reduced-order model (ROM) of the graph-Laplacian that approximates accurately the diffusion transfer function of the original graph for inputs and outputs restricted to the target subset. Working with a small target subset reduces greatly the required dimension of Krylov subspace and allows to exploit the conventional algorithms (like approximations of k-means) in the regimes when they are most robust and efficient.
Randomized Iterative Algorithms for Fisher Discriminant Analysis
Authors: Agniva Chowdhury, Jiasen Yang, Petros Drineas
Date: 2018-09-09 00:00:00
Tasks: Dimensionality Reduction

Fisher discriminant analysis (FDA) is a widely used method for classification
and dimensionality reduction. When the number of predictor variables greatly
exceeds the number of observations, one of the alternatives for conventional
FDA is regularized Fisher discriminant analysis (RFDA). In this paper, we
present a simple, iterative, sketching-based algorithm for RFDA that comes with
provable accuracy guarantees when compared to the conventional approach. Our
analysis builds upon two simple structural results that boil down to randomized
matrix multiplication, a fundamental and well-understood primitive of
randomized linear algebra. We analyze the behavior of RFDA when the ridge
leverage and the standard leverage scores are used to select predictor
variables and we prove that accurate approximations can be achieved by a sample
whose size depends on the effective degrees of freedom of the RFDA problem. Our
results yield significant improvements over existing approaches and our
empirical evaluations support our theoretical analyses.
How clever is the FiLM model, and how clever can it be?
Authors: Alexander Kuhnle, Huiyuan Xie, Ann Copestake
Date: 2018-09-09 00:00:00
Tasks: Diagnostic

The FiLM model achieves close-to-perfect performance on the diagnostic CLEVR
dataset and is distinguished from other such models by having a comparatively
simple and easily transferable architecture. In this paper, we investigate in
more detail the ability of FiLM to learn various linguistic constructions. Our
main results show that (a) FiLM is not able to learn relational statements
straight away except for very simple instances, (b) training on a broader set
of instances as well as pretraining on simpler instance types can help
alleviate these learning difficulties, (c) mixing is less robust than
pretraining and very sensitive to the compositional structure of the dataset.
Overall, our results suggest that the approach of big all-encompassing datasets
and the paradigm of "the effectiveness of data" may have fundamental
limitations.
An iterative method for classification of binary data
Authors: Denali Molitor, Deanna Needell
Date: 2018-09-09 00:00:00
Tasks: Classification, Data Compression, General Classification

In today's data driven world, storing, processing, and gleaning insights from
large-scale data are major challenges. Data compression is often required in
order to store large amounts of high-dimensional data, and thus, efficient
inference methods for analyzing compressed data are necessary. Building on a
recently designed simple framework for classification using binary data, we
demonstrate that one can improve classification accuracy of this approach
through iterative applications whose output serves as input to the next
application. As a side consequence, we show that the original framework can be
used as a data preprocessing step to improve the performance of other methods,
such as support vector machines. For several simple settings, we showcase the
ability to obtain theoretical guarantees for the accuracy of the iterative
classification method. The simplicity of the underlying classification
framework makes it amenable to theoretical analysis and studying this approach
will hopefully serve as a step toward developing theory for more sophisticated
deep learning technologies.
A Neural Temporal Model for Human Motion Prediction
Authors: Anand Gopalakrishnan, Ankur Mali, Dan Kifer, C. Lee Giles, Alexander G. Ororbia
Date: 2018-09-09 00:00:00
Proceeding: CVPR 2019 6
Tasks: Human motion prediction, model, motion prediction, Motion Synthesis, Prediction

We propose novel neural temporal models for predicting and synthesizing human motion, achieving state-of-the-art in modeling long-term motion trajectories while being competitive with prior work in short-term prediction and requiring significantly less computation. Key aspects of our proposed system include: 1) a novel, two-level processing architecture that aids in generating planned trajectories, 2) a simple set of easily computable features that integrate derivative information, and 3) a novel multi-objective loss function that helps the model to slowly progress from simple next-step prediction to the harder task of multi-step, closed-loop prediction. Our results demonstrate that these innovations improve the modeling of long-term motion trajectories. Finally, we propose a novel metric, called Normalized Power Spectrum Similarity (NPSS), to evaluate the long-term predictive ability of motion synthesis models, complementing the popular mean-squared error (MSE) measure of Euler joint angles over time. We conduct a user study to determine if the proposed NPSS correlates with human evaluation of long-term motion more strongly than MSE and find that it indeed does. We release code and additional results (visualizations) for this paper at: https://github.com/cr7anand/neural_temporal_models
Stochastic Gradient Descent Learns State Equations with Nonlinear Activations
Authors: Samet Oymak
Date: 2018-09-09 00:00:00
Proceeding: ICLR 2019 5

We study discrete time dynamical systems governed by the state equation
$h_{t+1}=\phi(Ah_t+Bu_t)$. Here $A,B$ are weight matrices, $\phi$ is an
activation function, and $u_t$ is the input data. This relation is the backbone
of recurrent neural networks (e.g. LSTMs) which have broad applications in
sequential learning tasks. We utilize stochastic gradient descent to learn the
weight matrices from a finite input/state trajectory $(u_t,h_t)_{t=0}^N$. We
prove that SGD estimate linearly converges to the ground truth weights while
using near-optimal sample size. Our results apply to increasing activations
whose derivatives are bounded away from zero. The analysis is based on i) a
novel SGD convergence result with nonlinear activations and ii) careful
statistical characterization of the state vector. Numerical experiments verify
the fast convergence of SGD on ReLU and leaky ReLU in consistence with our
theory.
Leveraging Elastic Demand for Forecasting
Authors: Houtao Deng, Ganesh Krishnan, Ji Chen, Dong Liang
Date: 2018-09-09 00:00:00

Demand variance can result in a mismatch between planned supply and actual
demand. Demand shaping strategies such as pricing can be used to shift elastic
demand to reduce the imbalance. In this work, we propose to consider elastic
demand in the forecasting phase. We present a method to reallocate the
historical elastic demand to reduce variance, thus making forecasting and
supply planning more effective.
Fingertip Detection and Tracking for Recognition of Air-Writing in Videos
Authors: Sohom Mukherjee, Arif Ahmed, Debi Prosad Dogra, Samarjit Kar, Partha Pratim Roy
Date: 2018-09-09 00:00:00
Tasks: Fingertip Detection, Hand Detection, Hand Segmentation, object-detection, Object Detection

Air-writing is the process of writing characters or words in free space using
finger or hand movements without the aid of any hand-held device. In this work,
we address the problem of mid-air finger writing using web-cam video as input.
In spite of recent advances in object detection and tracking, accurate and
robust detection and tracking of the fingertip remains a challenging task,
primarily due to small dimension of the fingertip. Moreover, the initialization
and termination of mid-air finger writing is also challenging due to the
absence of any standard delimiting criterion. To solve these problems, we
propose a new writing hand pose detection algorithm for initialization of
air-writing using the Faster R-CNN framework for accurate hand detection
followed by hand segmentation and finally counting the number of raised fingers
based on geometrical properties of the hand. Further, we propose a robust
fingertip detection and tracking approach using a new signature function called
distance-weighted curvature entropy. Finally, a fingertip velocity-based
termination criterion is used as a delimiter to mark the completion of the
air-writing gesture. Experiments show the superiority of the proposed fingertip
detection and tracking algorithm over state-of-the-art approaches giving a mean
precision of 73.1 % while achieving real-time performance at 18.5 fps, a
condition which is of vital importance to air-writing. Character recognition
experiments give a mean accuracy of 96.11 % using the proposed air-writing
system, a result which is comparable to that of existing handwritten character
recognition systems.
Can Neural Generators for Dialogue Learn Sentence Planning and Discourse Structuring?
Authors: Lena Reed, Shereen Oraby, Marilyn Walker
Date: 2018-09-09 00:00:00
Proceeding: WS 2018 11
Tasks: Relation, Sentence, Task-Oriented Dialogue Systems

Responses in task-oriented dialogue systems often realize multiple
propositions whose ultimate form depends on the use of sentence planning and
discourse structuring operations. For example a recommendation may consist of
an explicitly evaluative utterance e.g. Chanpen Thai is the best option, along
with content related by the justification discourse relation, e.g. It has great
food and service, that combines multiple propositions into a single phrase.
While neural generation methods integrate sentence planning and surface
realization in one end-to-end learning framework, previous work has not shown
that neural generators can: (1) perform common sentence planning and discourse
structuring operations; (2) make decisions as to whether to realize content in
a single sentence or over multiple sentences; (3) generalize sentence planning
and discourse relation operations beyond what was seen in training. We
systematically create large training corpora that exhibit particular sentence
planning operations and then test neural models to see what they learn. We
compare models without explicit latent variables for sentence planning with
ones that provide explicit supervision during training. We show that only the
models with additional supervision can reproduce sentence planing and discourse
operations and generalize to situations unseen in training.
Training for Faster Adversarial Robustness Verification via Inducing ReLU Stability
Authors: Kai Y. Xiao, Vincent Tjeng, Nur Muhammad Shafiullah, Aleksander Madry
Date: 2018-09-09 00:00:00
Proceeding: ICLR 2019 5
Tasks: Adversarial Robustness

We explore the concept of co-design in the context of neural network
verification. Specifically, we aim to train deep neural networks that not only
are robust to adversarial perturbations but also whose robustness can be
verified more easily. To this end, we identify two properties of network models
- weight sparsity and so-called ReLU stability - that turn out to significantly
impact the complexity of the corresponding verification task. We demonstrate
that improving weight sparsity alone already enables us to turn computationally
intractable verification problems into tractable ones. Then, improving ReLU
stability leads to an additional 4-13x speedup in verification times. An
important feature of our methodology is its "universality," in the sense that
it can be used with a broad range of training procedures and verification
approaches.
Performance Metrics (Error Measures) in Machine Learning Regression, Forecasting and Prognostics: Properties and Typology
Authors: Alexei Botchkarev
Date: 2018-09-09 00:00:00
Tasks: BIG-bench Machine Learning, General Classification, regression

Performance metrics (error measures) are vital components of the evaluation
frameworks in various fields. The intention of this study was to overview of a
variety of performance metrics and approaches to their classification. The main
goal of the study was to develop a typology that will help to improve our
knowledge and understanding of metrics and facilitate their selection in
machine learning regression, forecasting and prognostics. Based on the analysis
of the structure of numerous performance metrics, we propose a framework of
metrics which includes four (4) categories: primary metrics, extended metrics,
composite metrics, and hybrid sets of metrics. The paper identified three (3)
key components (dimensions) that determine the structure and properties of
primary metrics: method of determining point distance, method of normalization,
method of aggregation of point distances over a data set.
Evidence-based lean logic profiles for conceptual data modelling languages
Authors: Pablo Rubén Fillottrani, C. Maria Keet
Date: 2018-09-09 00:00:00

Multiple logic-based reconstructions of conceptual data modelling languages such as EER, UML Class Diagrams, and ORM exist. They mainly cover various fragments of the languages and none are formalised such that the logic applies simultaneously for all three modelling language families as unifying mechanism. This hampers interchangeability, interoperability, and tooling support. In addition, due to the lack of a systematic design process of the logic used for the formalisation, hidden choices permeate the formalisations that have rendered them incompatible. We aim to address these problems, first, by structuring the logic design process in a methodological way. We generalise and extend the DSL design process to apply to logic language design more generally and, in particular, by incorporating an ontological analysis of language features in the process. Second, we specify minimal logic profiles availing of this extended process, including the ontological commitments embedded in the languages, of evidence gathered of language feature usage, and of computational complexity insights from Description Logics (DL). The profiles characterise the essential logic structure needed to handle the semantics of conceptual models, therewith enabling the development of interoperability tools. There is no known DL language that matches exactly the features of those profiles and the common core is small (in the tractable DL $\mathcal{ALNI}$). Although hardly any inconsistencies can be derived with the profiles, it is promising for scalable runtime use of conceptual data models.