pwc-archive/papers-with-abstracts huggingface dataset:
Deep calibration of rough stochastic volatility models
Authors: Christian Bayer, Benjamin Stemper
Date: 2018-10-08 00:00:00

Sparked by Al\`os, Le\'on, and Vives (2007); Fukasawa (2011, 2017); Gatheral,
Jaisson, and Rosenbaum (2018), so-called rough stochastic volatility models
such as the rough Bergomi model by Bayer, Friz, and Gatheral (2016) constitute
the latest evolution in option price modeling. Unlike standard bivariate
diffusion models such as Heston (1993), these non-Markovian models with
fractional volatility drivers allow to parsimoniously recover key stylized
facts of market implied volatility surfaces such as the exploding power-law
behaviour of the at-the-money volatility skew as time to maturity goes to zero.
Standard model calibration routines rely on the repetitive evaluation of the
map from model parameters to Black-Scholes implied volatility, rendering
calibration of many (rough) stochastic volatility models prohibitively
expensive since there the map can often only be approximated by costly Monte
Carlo (MC) simulations (Bennedsen, Lunde, & Pakkanen, 2017; McCrickerd &
Pakkanen, 2018; Bayer et al., 2016; Horvath, Jacquier, & Muguruza, 2017). As a
remedy, we propose to combine a standard Levenberg-Marquardt calibration
routine with neural network regression, replacing expensive MC simulations with
cheap forward runs of a neural network trained to approximate the implied
volatility map. Numerical experiments confirm the high accuracy and speed of
our approach.
Hierarchical clustering that takes advantage of both density-peak and density-connectivity
Authors: Ye Zhu, Kai Ming Ting, Yuan Jin, Maia Angelova
Date: 2018-10-08 00:00:00
Tasks: Clustering

This paper focuses on density-based clustering, particularly the Density Peak (DP) algorithm and the one based on density-connectivity DBSCAN; and proposes a new method which takes advantage of the individual strengths of these two methods to yield a density-based hierarchical clustering algorithm. Our investigation begins with formally defining the types of clusters DP and DBSCAN are designed to detect; and then identifies the kinds of distributions that DP and DBSCAN individually fail to detect all clusters in a dataset. These identified weaknesses inspire us to formally define a new kind of clusters and propose a new method called DC-HDP to overcome these weaknesses to identify clusters with arbitrary shapes and varied densities. In addition, the new method produces a richer clustering result in terms of hierarchy or dendrogram for better cluster structures understanding. Our empirical evaluation results show that DC-HDP produces the best clustering results on 14 datasets in comparison with 7 state-of-the-art clustering algorithms.
Rethinking Breiman's Dilemma in Neural Networks: Phase Transitions of Margin Dynamics
Authors: Weizhi Zhu, Yifei HUANG, Yuan YAO
Date: 2018-10-08 00:00:00
Tasks: Generalization Bounds

Margin enlargement over training data has been an important strategy since perceptrons in machine learning for the purpose of boosting the robustness of classifiers toward a good generalization ability. Yet Breiman (1999) showed a dilemma that a uniform improvement on margin distribution does NOT necessarily reduces generalization errors. In this paper, we revisit Breiman's dilemma in deep neural networks with recently proposed spectrally normalized margins, from a novel perspective based on phase transitions of normalized margin distributions in training dynamics. Normalized margin distribution of a classifier over the data, can be divided into two parts: low/small margins such as some negative margins for misclassified samples vs. high/large margins for high confident correctly classified samples, that often behave differently during the training process. Low margins for training and test datasets are often effectively reduced in training, along with reductions of training and test errors; while high margins may exhibit different dynamics, reflecting the trade-off between expressive power of models and complexity of data. When data complexity is comparable to the model expressiveness, high margin distributions for both training and test data undergo similar decrease-increase phase transitions during training. In such cases, one can predict the trend of generalization or test error by margin-based generalization bounds with restricted Rademacher complexities, shown in two ways in this paper with early stopping time exploiting such phase transitions. On the other hand, over-expressive models may have both low and high training margins undergoing uniform improvements, with a distinct phase transition in test margin dynamics. This reconfirms the Breiman's dilemma associated with overparameterized neural networks where margins fail to predict overfitting.
Training Passive Photonic Reservoirs with Integrated Optical Readout
Authors: Matthias Freiberger, Andrew Katumba, Peter Bienstman, Joni Dambre
Date: 2018-10-08 00:00:00

As Moore's law comes to an end, neuromorphic approaches to computing are on
the rise. One of these, passive photonic reservoir computing, is a strong
candidate for computing at high bitrates (> 10 Gbps) and with low energy
consumption. Currently though, both benefits are limited by the necessity to
perform training and readout operations in the electrical domain. Thus, efforts
are currently underway in the photonic community to design an integrated
optical readout, which allows to perform all operations in the optical domain.
In addition to the technological challenge of designing such a readout, new
algorithms have to be designed in order to train it. Foremost, suitable
algorithms need to be able to deal with the fact that the actual on-chip
reservoir states are not directly observable. In this work, we investigate
several options for such a training algorithm and propose a solution in which
the complex states of the reservoir can be observed by appropriately setting
the readout weights, while iterating over a predefined input sequence. We
perform numerical simulations in order to compare our method with an ideal
baseline requiring full observability as well as with an established black-box
optimization approach (CMA-ES).
Detecting Memorization in ReLU Networks
Authors: Edo Collins, Siavash Arjomand Bigdeli, Sabine Süsstrunk
Date: 2018-10-08 00:00:00
Proceeding: ICLR 2019 5
Tasks: Memorization

We propose a new notion of `non-linearity' of a network layer with respect to
an input batch that is based on its proximity to a linear system, which is
reflected in the non-negative rank of the activation matrix. We measure this
non-linearity by applying non-negative factorization to the activation matrix.
Considering batches of similar samples, we find that high non-linearity in deep
layers is indicative of memorization. Furthermore, by applying our approach
layer-by-layer, we find that the mechanism for memorization consists of
distinct phases. We perform experiments on fully-connected and convolutional
neural networks trained on several image and audio datasets. Our results
demonstrate that as an indicator for memorization, our technique can be used to
perform early stopping.
Empirical Bounds on Linear Regions of Deep Rectifier Networks
Authors: Thiago Serra, Srikumar Ramalingam
Date: 2018-10-08 00:00:00
Proceeding: ICLR 2019 5

We can compare the expressiveness of neural networks that use rectified linear units (ReLUs) by the number of linear regions, which reflect the number of pieces of the piecewise linear functions modeled by such networks. However, enumerating these regions is prohibitive and the known analytical bounds are identical for networks with same dimensions. In this work, we approximate the number of linear regions through empirical bounds based on features of the trained network and probabilistic inference. Our first contribution is a method to sample the activation patterns defined by ReLUs using universal hash functions. This method is based on a Mixed-Integer Linear Programming (MILP) formulation of the network and an algorithm for probabilistic lower bounds of MILP solution sets that we call MIPBound, which is considerably faster than exact counting and reaches values in similar orders of magnitude. Our second contribution is a tighter activation-based bound for the maximum number of linear regions, which is particularly stronger in networks with narrow layers. Combined, these bounds yield a fast proxy for the number of linear regions of a deep neural network.
A Survey on Periocular Biometrics Research
Authors: Fernando Alonso-Fernandez, Josef Bigun
Date: 2018-10-08 00:00:00
Tasks: Survey

Periocular refers to the facial region in the vicinity of the eye, including
eyelids, lashes and eyebrows. While face and irises have been extensively
studied, the periocular region has emerged as a promising trait for
unconstrained biometrics, following demands for increased robustness of face or
iris systems. With a surprisingly high discrimination ability, this region can
be easily obtained with existing setups for face and iris, and the requirement
of user cooperation can be relaxed, thus facilitating the interaction with
biometric systems. It is also available over a wide range of distances even
when the iris texture cannot be reliably obtained (low resolution) or under
partial face occlusion (close distances). Here, we review the state of the art
in periocular biometrics research. A number of aspects are described,
including: i) existing databases, ii) algorithms for periocular detection
and/or segmentation, iii) features employed for recognition, iv) identification
of the most discriminative regions of the periocular area, v) comparison with
iris and face modalities, vi) soft-biometrics (gender/ethnicity
classification), and vii) impact of gender transformation and plastic surgery
on the recognition accuracy. This work is expected to provide an insight of the
most relevant issues in periocular biometrics, giving a comprehensive coverage
of the existing literature and current state of the art.
Multi-Task Learning for Domain-General Spoken Disfluency Detection in Dialogue Systems
Authors: Igor Shalyminov, Arash Eshghi, Oliver Lemon
Date: 2018-10-08 00:00:00
Tasks: Multi-Task Learning

Spontaneous spoken dialogue is often disfluent, containing pauses,
hesitations, self-corrections and false starts. Processing such phenomena is
essential in understanding a speaker's intended meaning and controlling the
flow of the conversation. Furthermore, this processing needs to be word-by-word
incremental to allow further downstream processing to begin as early as
possible in order to handle real spontaneous human conversational behaviour.
  In addition, from a developer's point of view, it is highly desirable to be
able to develop systems which can be trained from `clean' examples while also
able to generalise to the very diverse disfluent variations on the same data --
thereby enhancing both data-efficiency and robustness. In this paper, we
present a multi-task LSTM-based model for incremental detection of disfluency
structure, which can be hooked up to any component for incremental
interpretation (e.g. an incremental semantic parser), or else simply used to
`clean up' the current utterance as it is being produced.
  We train the system on the Switchboard Dialogue Acts (SWDA) corpus and
present its accuracy on this dataset. Our model outperforms prior neural
network-based incremental approaches by about 10 percentage points on SWDA
while employing a simpler architecture. To test the model's generalisation
potential, we evaluate the same model on the bAbI+ dataset, without any
additional training. bAbI+ is a dataset of synthesised goal-oriented dialogues
where we control the distribution of disfluencies and their types. This shows
that our approach has good generalisation potential, and sheds more light on
which types of disfluency might be amenable to domain-general processing.
Local Explanation Methods for Deep Neural Networks Lack Sensitivity to Parameter Values
Authors: Julius Adebayo, Justin Gilmer, Ian Goodfellow, Been Kim
Date: 2018-10-08 00:00:00
Tasks: BIG-bench Machine Learning, Sensitivity

Explaining the output of a complicated machine learning model like a deep
neural network (DNN) is a central challenge in machine learning. Several
proposed local explanation methods address this issue by identifying what
dimensions of a single input are most responsible for a DNN's output. The goal
of this work is to assess the sensitivity of local explanations to DNN
parameter values. Somewhat surprisingly, we find that DNNs with
randomly-initialized weights produce explanations that are both visually and
quantitatively similar to those produced by DNNs with learned weights. Our
conjecture is that this phenomenon occurs because these explanations are
dominated by the lower level features of a DNN, and that a DNN's architecture
provides a strong prior which significantly affects the representations learned
at these lower layers. NOTE: This work is now subsumed by our recent
manuscript, Sanity Checks for Saliency Maps (to appear NIPS 2018), where we
expand on findings and address concerns raised in Sundararajan et. al. (2018).
Sanity Checks for Saliency Maps
Authors: Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, Been Kim
Date: 2018-10-08 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Edge Detection, Saliency Detection

Saliency methods have emerged as a popular tool to highlight features in an input deemed relevant for the prediction of a learned model. Several saliency methods have been proposed, often guided by visual appeal on image data. In this work, we propose an actionable methodology to evaluate what kinds of explanations a given method can and cannot provide. We find that reliance, solely, on visual assessment can be misleading. Through extensive experiments we show that some existing saliency methods are independent both of the model and of the data generating process. Consequently, methods that fail the proposed tests are inadequate for tasks that are sensitive to either data or model, such as, finding outliers in the data, explaining the relationship between inputs and outputs that the model learned, and debugging the model. We interpret our findings through an analogy with edge detection in images, a technique that requires neither training data nor model. Theory in the case of a linear model and a single-layer convolutional neural network supports our experimental findings.
Guiding Intelligent Surveillance System by learning-by-synthesis gaze estimation
Authors: Tongtong Zhao, Yuxiao Yan, Jinjia Peng, Zetian Mi, Xianping Fu
Date: 2018-10-08 00:00:00
Tasks: Gaze Estimation

We describe a novel learning-by-synthesis method for estimating gaze
direction of an automated intelligent surveillance system. Recently, progress
in learning-by-synthesis has proposed training models on synthetic images,
which can effectively reduce the cost of manpower and material resources.
However, learning from synthetic images still fails to achieve the desired
performance compared to naturalistic images due to the different distribution
of synthetic images. In an attempt to address this issue, previous method is to
improve the realism of synthetic images by learning a model. However, the
disadvantage of the method is that the distortion has not been improved and the
authenticity level is unstable. To solve this problem, we put forward a new
structure to improve synthetic images, via the reference to the idea of style
transformation, through which we can efficiently reduce the distortion of
pictures and minimize the need of real data annotation. We estimate that this
enables generation of highly realistic images, which we demonstrate both
qualitatively and with a user study. We quantitatively evaluate the generated
images by training models for gaze estimation. We show a significant
improvement over using synthetic images, and achieve state-of-the-art results
on various datasets including MPIIGaze dataset.
Optimizing Waiting Thresholds Within A State Machine
Authors: Rohit Pandey, Yifan Chang, Cameron White, Gaurav Jagtiani, Aerin Young Kim, Gil Lapid Shafriri, Sathya Singh
Date: 2018-10-08 00:00:00

Azure (the cloud service provided by Microsoft) is composed of physical
computing units which are called nodes. These nodes are controlled by a
software component called Fabric Controller (FC), which can consider the nodes
to be in one of many different states such as Ready, Unhealthy, Booting, etc.
Some of these states correspond to a node being unresponsive to FCs requests.
When a node goes unresponsive for more than a set threshold, FC intervenes and
reboots the node. We minimized the downtime caused by the intervention
threshold when a node switches to the Unhealthy state by fitting various
heavy-tail probability distributions. We consider using features of the node to
customize the organic recovery model to the individual nodes that go unhealthy.
This regression approach allows us to use information about the node like
hardware, software versions, historical performance indicators, etc. to inform
the organic recovery model and hence the optimal threshold. In another
direction, we consider generalizing this to an arbitrary number of thresholds
within the node state machine (or Markov chain). When the states become
intertwined in ways that different thresholds start affecting each other, we
can't simply optimize each of them in isolation. For best results, we must
consider this as an optimization problem in many variables (the number of
thresholds). We no longer have a nice closed form solution for this more
complex problem like we did with one threshold, but we can still use numerical
techniques (gradient descent) to solve it.
TV-regularized CT Reconstruction and Metal Artifact Reduction Using Inequality Constraints with Preconditioning
Authors: Clemens Schiffer
Date: 2018-10-08 00:00:00
Tasks: Computed Tomography (CT), CT Reconstruction, Metal Artifact Reduction

Total variation(TV) regularization is applied to X-Ray computed
tomography(CT) in an effort to reduce metal artifacts. Tikhonov regularization
with $L^2$ data fidelity term and total variation regularization is augmented
in this novel model by inequality constraints on sinogram data affected by
metal to model errors caused by metal. The formulated problem is discretized
and solved using the Chambolle-Pock algorithm. Faster convergence is achieved
using preconditioning in a Douglas-Rachford spitting method as well as Advanced
Direction Method of Multipliers(ADMM). The methods are applied to real and
synthetic data demonstrating feasibility of the model to reduce metal
artifacts. Technical details of CT data used and its processing are given in
the appendix.
Query Tracking for E-commerce Conversational Search: A Machine Comprehension Perspective
Authors: Yunlun Yang, Yu Gong, Xi Chen
Date: 2018-10-08 00:00:00
Tasks: Conversational Search, Natural Language Understanding, Reading Comprehension, slot-filling, Slot Filling

With the development of dialog techniques, conversational search has
attracted more and more attention as it enables users to interact with the
search engine in a natural and efficient manner. However, comparing with the
natural language understanding in traditional task-oriented dialog which
focuses on slot filling and tracking, the query understanding in E-commerce
conversational search is quite different and more challenging due to more
diverse user expressions and complex intentions. In this work, we define the
real-world problem of query tracking in E-commerce conversational search, in
which the goal is to update the internal query after each round of interaction.
We also propose a self attention based neural network to handle the task in a
machine comprehension perspective. Further more we build a novel E-commerce
query tracking dataset from an operational E-commerce Search Engine, and
experimental results on this dataset suggest that our proposed model
outperforms several baseline methods by a substantial gain for Exact Match
accuracy and F1 score, showing the potential of machine comprehension like
model for this task.
Light-Weight RefineNet for Real-Time Semantic Segmentation
Authors: Vladimir Nekrasov, Chunhua Shen, Ian Reid
Date: 2018-10-08 00:00:00
Tasks: GPU, Image Segmentation, Real-Time Semantic Segmentation, Segmentation, Semantic Segmentation

We consider an important task of effective and efficient semantic image
segmentation. In particular, we adapt a powerful semantic segmentation
architecture, called RefineNet, into the more compact one, suitable even for
tasks requiring real-time performance on high-resolution inputs. To this end,
we identify computationally expensive blocks in the original setup, and propose
two modifications aimed to decrease the number of parameters and floating point
operations. By doing that, we achieve more than twofold model reduction, while
keeping the performance levels almost intact. Our fastest model undergoes a
significant speed-up boost from 20 FPS to 55 FPS on a generic GPU card on
512x512 inputs with solid 81.1% mean iou performance on the test set of PASCAL
VOC, while our slowest model with 32 FPS (from original 17 FPS) shows 82.7%
mean iou on the same dataset. Alternatively, we showcase that our approach is
easily mixable with light-weight classification networks: we attain 79.2% mean
iou on PASCAL VOC using a model that contains only 3.3M parameters and performs
only 9.3B floating point operations.
Patient-Specific 3D Volumetric Reconstruction of Bioresorbable Stents: A Method to Generate 3D Geometries for Computational Analysis of Coronaries Treated with Bioresorbable Stents
Authors: Boyi Yang, Marina Piccinelli, Gaetano Esposito, Tianli Han, Yasir Bouchi, Bill Gogas, Don Giddens, Habib Samady, Alessandro Veneziani
Date: 2018-10-08 00:00:00
Tasks: 3D geometry, 3D Volumetric Reconstruction

As experts continue to debate the optimal surgery practice for coronary
disease - percutaneous coronary intervention (PCI) or coronary aortic bypass
graft (CABG) - computational tools may provide a quantitative assessment of
each option. Computational fluid dynamics (CFD) has been used to assess the
interplay between hemodynamics and stent struts; it is of particular interest
in Bioresorbable Vascular Stents (BVS), since their thicker struts may result
in impacted flow patterns and possible pathological consequences. Many proofs
of concept are presented in the literature; however, a practical method for
extracting patient-specific stented coronary artery geometries from images over
a large number of patients remains an open problem.
  This work provides a possible pipeline for the reconstruction of the BVS.
Using Optical Coherence Tomographies (OCT) and Invasive Coronary Angiographies
(ICA), we can reconstruct the 3D geometry of deployed BVS in vivo. We
illustrate the stent reconstruction process: (i) automatic strut detection,
(ii) identification of stent components, (iii) 3D registration of stent
curvature, and (iv) final stent volume reconstruction. The methodology is
designed for use on clinical OCT images, as opposed to approaches that relied
on a small number of virtually deployed stents.
  The proposed reconstruction process is validated with a virtual phantom
stent, providing quantitative assessment of the methodology, and with selected
clinical cases, confirming feasibility. Using multimodality image analysis, we
obtain reliable reconstructions within a reasonable timeframe. This work is the
first step toward a fully automated reconstruction and simulation procedure
aiming at an extensive quantitative analysis of the impact of BVS struts on
hemodynamics via CFD in clinical trials, going beyond the proof-of-concept
stage.