pwc-archive/papers-with-abstracts huggingface dataset:
SQG-Differential Evolution for difficult optimization problems under a tight function evaluation budget
Authors: Ramses Sala, Niccolo Baldanzini, Marco Pierini
Date: 2017-10-18 00:00:00

In the context of industrial engineering, it is important to integrate
efficient computational optimization methods in the product development
process. Some of the most challenging simulation-based engineering design
optimization problems are characterized by: a large number of design variables,
the absence of analytical gradients, highly non-linear objectives and a limited
function evaluation budget. Although a huge variety of different optimization
algorithms is available, the development and selection of efficient algorithms
for problems with these industrial relevant characteristics, remains a
challenge. In this communication, a hybrid variant of Differential Evolution
(DE) is introduced which combines aspects of Stochastic Quasi-Gradient (SQG)
methods within the framework of DE, in order to improve optimization efficiency
on problems with the previously mentioned characteristics. The performance of
the resulting derivative-free algorithm is compared with other state-of-the-art
DE variants on 25 commonly used benchmark functions, under tight function
evaluation budget constraints of 1000 evaluations. The experimental results
indicate that the new algorithm performs excellent on the 'difficult' (high
dimensional, multi-modal, inseparable) test functions. The operations used in
the proposed mutation scheme, are computationally inexpensive, and can be
easily implemented in existing differential evolution variants or other
population-based optimization algorithms by a few lines of program code as an
non-invasive optional setting. Besides the applicability of the presented
algorithm by itself, the described concepts can serve as a useful and
interesting addition to the algorithmic operators in the frameworks of
heuristics and evolutionary optimization and computing.
A Memristor-Based Optimization Framework for AI Applications
Authors: Sijia Liu, Yanzhi Wang, Makan Fardad, Pramod K. Varshney
Date: 2017-10-18 00:00:00
Tasks: BIG-bench Machine Learning

Memristors have recently received significant attention as ubiquitous
device-level components for building a novel generation of computing systems.
These devices have many promising features, such as non-volatility, low power
consumption, high density, and excellent scalability. The ability to control
and modify biasing voltages at the two terminals of memristors make them
promising candidates to perform matrix-vector multiplications and solve systems
of linear equations. In this article, we discuss how networks of memristors
arranged in crossbar arrays can be used for efficiently solving optimization
and machine learning problems. We introduce a new memristor-based optimization
framework that combines the computational merit of memristor crossbars with the
advantages of an operator splitting method, alternating direction method of
multipliers (ADMM). Here, ADMM helps in splitting a complex optimization
problem into subproblems that involve the solution of systems of linear
equations. The capability of this framework is shown by applying it to linear
programming, quadratic programming, and sparse optimization. In addition to
ADMM, implementation of a customized power iteration (PI) method for
eigenvalue/eigenvector computation using memristor crossbars is discussed. The
memristor-based PI method can further be applied to principal component
analysis (PCA). The use of memristor crossbars yields a significant speed-up in
computation, and thus, we believe, has the potential to advance optimization
and machine learning research in artificial intelligence (AI).
Phase Transitions in the Pooled Data Problem
Authors: Jonathan Scarlett, Volkan Cevher
Date: 2017-10-18 00:00:00
Proceeding: NeurIPS 2017 12

In this paper, we study the pooled data problem of identifying the labels
associated with a large collection of items, based on a sequence of pooled
tests revealing the counts of each label within the pool. In the noiseless
setting, we identify an exact asymptotic threshold on the required number of
tests with optimal decoding, and prove a phase transition between complete
success and complete failure. In addition, we present a novel noisy variation
of the problem, and provide an information-theoretic framework for
characterizing the required number of tests for general random noise models.
Our results reveal that noise can make the problem considerably more difficult,
with strict increases in the scaling laws even at low noise levels. Finally, we
demonstrate similar behavior in an approximate recovery setting, where a given
number of errors is allowed in the decoded labels.
A complete characterization of optimal dictionaries for least squares representation
Authors: Mohammed Rayyan Sheriff, Debasish Chatterjee
Date: 2017-10-18 00:00:00

Dictionaries are collections of vectors used for representations of elements
in Euclidean spaces. While recent research on optimal dictionaries is focussed
on providing sparse (i.e., $\ell_0$-optimal,) representations, here we consider
the problem of finding optimal dictionaries such that representations of
samples of a random vector are optimal in an $\ell_2$-sense. For us, optimality
of representation is equivalent to minimization of the average $\ell_2$-norm of
the coefficients used to represent the random vector, with the lengths of the
dictionary vectors being specified a priori. With the help of recent results on
rank-$1$ decompositions of symmetric positive semidefinite matrices and the
theory of majorization, we provide a complete characterization of
$\ell_2$-optimal dictionaries. Our results are accompanied by polynomial time
algorithms that construct $\ell_2$-optimal dictionaries from given data.
DNN and CNN with Weighted and Multi-task Loss Functions for Audio Event Detection
Authors: Huy Phan, Martin Krawczyk-Becker, Timo Gerkmann, Alfred Mertins
Date: 2017-08-10 00:00:00
Tasks: Event Detection, Task 2

This report presents our audio event detection system submitted for Task 2,
"Detection of rare sound events", of DCASE 2017 challenge. The proposed system
is based on convolutional neural networks (CNNs) and deep neural networks
(DNNs) coupled with novel weighted and multi-task loss functions and
state-of-the-art phase-aware signal enhancement. The loss functions are
tailored for audio event detection in audio streams. The weighted loss is
designed to tackle the common issue of imbalanced data in background/foreground
classification while the multi-task loss enables the networks to simultaneously
model the class distribution and the temporal structures of the target events
for recognition. Our proposed systems significantly outperform the challenge
baseline, improving F-score from 72.7% to 90.0% and reducing detection error
rate from 0.53 to 0.18 on average on the development data. On the evaluation
data, our submission obtains an average F1-score of 88.3% and an error rate of
0.22 which are significantly better than those obtained by the DCASE baseline
(i.e. an F1-score of 64.1% and an error rate of 0.64).
Cross-view Asymmetric Metric Learning for Unsupervised Person Re-identification
Authors: Hong-Xing Yu, An-Cong Wu, Wei-Shi Zheng
Date: 2017-08-27 00:00:00
Proceeding: ICCV 2017 10
Tasks: Clustering, Metric Learning, Person Re-Identification, Unsupervised Person Re-Identification

While metric learning is important for Person re-identification (RE-ID), a
significant problem in visual surveillance for cross-view pedestrian matching,
existing metric models for RE-ID are mostly based on supervised learning that
requires quantities of labeled samples in all pairs of camera views for
training. However, this limits their scalabilities to realistic applications,
in which a large amount of data over multiple disjoint camera views is
available but not labelled. To overcome the problem, we propose unsupervised
asymmetric metric learning for unsupervised RE-ID. Our model aims to learn an
asymmetric metric, i.e., specific projection for each view, based on asymmetric
clustering on cross-view person images. Our model finds a shared space where
view-specific bias is alleviated and thus better matching performance can be
achieved. Extensive experiments have been conducted on a baseline and five
large-scale RE-ID datasets to demonstrate the effectiveness of the proposed
model. Through the comparison, we show that our model works much more suitable
for unsupervised RE-ID compared to classical unsupervised metric learning
models. We also compare with existing unsupervised RE-ID methods, and our model
outperforms them with notable margins. Specifically, we report the results on
large-scale unlabelled RE-ID dataset, which is important but unfortunately less
concerned in literatures.
Probabilistic Integration: A Role in Statistical Computation?
Authors: François-Xavier Briol, Chris. J. Oates, Mark Girolami, Michael A. Osborne, Dino Sejdinovic
Date: 2015-12-03 00:00:00
Tasks: Numerical Integration

A research frontier has emerged in scientific computation, wherein numerical
error is regarded as a source of epistemic uncertainty that can be modelled.
This raises several statistical challenges, including the design of statistical
methods that enable the coherent propagation of probabilities through a
(possibly deterministic) computational work-flow. This paper examines the case
for probabilistic numerical methods in routine statistical computation. Our
focus is on numerical integration, where a probabilistic integrator is equipped
with a full distribution over its output that reflects the presence of an
unknown numerical error. Our main technical contribution is to establish, for
the first time, rates of posterior contraction for these methods. These show
that probabilistic integrators can in principle enjoy the "best of both
worlds", leveraging the sampling efficiency of Monte Carlo methods whilst
providing a principled route to assess the impact of numerical error on
scientific conclusions. Several substantial applications are provided for
illustration and critical evaluation, including examples from statistical
modelling, computer graphics and a computer model for an oil reservoir.
Query-based Attention CNN for Text Similarity Map
Authors: Tzu-Chien Liu, Yu-Hsueh Wu, Hung-Yi Lee
Date: 2017-09-15 00:00:00
Tasks: Question Answering, Sentence, text similarity

In this paper, we introduce Query-based Attention CNN(QACNN) for Text
Similarity Map, an end-to-end neural network for question answering. This
network is composed of compare mechanism, two-staged CNN architecture with
attention mechanism, and a prediction layer. First, the compare mechanism
compares between the given passage, query, and multiple answer choices to build
similarity maps. Then, the two-staged CNN architecture extracts features
through word-level and sentence-level. At the same time, attention mechanism
helps CNN focus more on the important part of the passage based on the query
information. Finally, the prediction layer find out the most possible answer
choice. We conduct this model on the MovieQA dataset using Plot Synopses only,
and achieve 79.99% accuracy which is the state of the art on the dataset.
Case Study: Explaining Diabetic Retinopathy Detection Deep CNNs via Integrated Gradients
Authors: Linyi Li, Matt Fredrikson, Shayak Sen, Anupam Datta
Date: 2017-09-27 00:00:00
Tasks: Diabetic Retinopathy Detection

In this report, we applied integrated gradients to explaining a neural
network for diabetic retinopathy detection. The integrated gradient is an
attribution method which measures the contributions of input to the quantity of
interest. We explored some new ways for applying this method such as explaining
intermediate layers, filtering out unimportant units by their attribution value
and generating contrary samples. Moreover, the visualization results extend the
use of diabetic retinopathy detection model from merely predicting to assisting
finding potential lesions.
Feature-Fused SSD: Fast Detection for Small Objects
Authors: Guimei Cao, Xuemei Xie, Wenzhe Yang, Quan Liao, Guangming Shi, Jinjian Wu
Date: 2017-09-15 00:00:00
Tasks: object-detection, Object Detection, Small Object Detection

Small objects detection is a challenging task in computer vision due to its
limited resolution and information. In order to solve this problem, the
majority of existing methods sacrifice speed for improvement in accuracy. In
this paper, we aim to detect small objects at a fast speed, using the best
object detector Single Shot Multibox Detector (SSD) with respect to
accuracy-vs-speed trade-off as base architecture. We propose a multi-level
feature fusion method for introducing contextual information in SSD, in order
to improve the accuracy for small objects. In detailed fusion operation, we
design two feature fusion modules, concatenation module and element-sum module,
different in the way of adding contextual information. Experimental results
show that these two fusion modules obtain higher mAP on PASCALVOC2007 than
baseline SSD by 1.6 and 1.7 points respectively, especially with 2-3 points
improvement on some smallobjects categories. The testing speed of them is 43
and 40 FPS respectively, superior to the state of the art Deconvolutional
single shot detector (DSSD) by 29.4 and 26.4 FPS. Code is available at
https://github.com/wnzhyee/Feature-Fused-SSD. Keywords: small object detection,
feature fusion, real-time, single shot multi-box detector
Function Norms and Regularization in Deep Networks
Authors: Amal Rannen Triki, Maxim Berman, Matthew B. Blaschko
Date: 2017-10-18 00:00:00
Tasks: Image Segmentation, Learning Theory, Semantic Segmentation, Stochastic Optimization

Deep neural networks (DNNs) have become increasingly important due to their
excellent empirical performance on a wide range of problems. However,
regularization is generally achieved by indirect means, largely due to the
complex set of functions defined by a network and the difficulty in measuring
function complexity. There exists no method in the literature for additive
regularization based on a norm of the function, as is classically considered in
statistical learning theory. In this work, we propose sampling-based
approximations to weighted function norms as regularizers for deep neural
networks. We provide, to the best of our knowledge, the first proof in the
literature of the NP-hardness of computing function norms of DNNs, motivating
the necessity of an approximate approach. We then derive a generalization bound
for functions trained with weighted norms and prove that a natural stochastic
optimization strategy minimizes the bound. Finally, we empirically validate the
improved performance of the proposed regularization strategies for both convex
function sets as well as DNNs on real-world classification and image
segmentation tasks demonstrating improved performance over weight decay,
dropout, and batch normalization. Source code will be released at the time of
publication.
Build Fast and Accurate Lemmatization for Arabic
Authors: Hamdy Mubarak
Date: 2017-10-18 00:00:00
Proceeding: LREC 2018 5
Tasks: Information Retrieval, Lemmatization, Retrieval

In this paper we describe the complexity of building a lemmatizer for Arabic
which has a rich and complex derivational morphology, and we discuss the need
for a fast and accurate lammatization to enhance Arabic Information Retrieval
(IR) results. We also introduce a new data set that can be used to test
lemmatization accuracy, and an efficient lemmatization algorithm that
outperforms state-of-the-art Arabic lemmatization in terms of accuracy and
speed. We share the data set and the code for public.
Learning weakly supervised multimodal phoneme embeddings
Authors: Rahma Chaabouni, Ewan Dunbar, Neil Zeghidour, Emmanuel Dupoux
Date: 2017-04-23 00:00:00
Tasks: Multi-Task Learning

Recent works have explored deep architectures for learning multimodal speech
representation (e.g. audio and images, articulation and audio) in a supervised
way. Here we investigate the role of combining different speech modalities,
i.e. audio and visual information representing the lips movements, in a weakly
supervised way using Siamese networks and lexical same-different side
information. In particular, we ask whether one modality can benefit from the
other to provide a richer representation for phone recognition in a weakly
supervised setting. We introduce mono-task and multi-task methods for merging
speech and visual modalities for phone recognition. The mono-task learning
consists in applying a Siamese network on the concatenation of the two
modalities, while the multi-task learning receives several different
combinations of modalities at train time. We show that multi-task learning
enhances discriminability for visual and multimodal inputs while minimally
impacting auditory inputs. Furthermore, we present a qualitative analysis of
the obtained phone embeddings, and show that cross-modal visual input can
improve the discriminability of phonological features which are visually
discernable (rounding, open/close, labial place of articulation), resulting in
representations that are closer to abstract linguistic features than those
based on audio only.
Generating Reflectance Curves from sRGB Triplets
Authors: Scott Allen Burns
Date: 2017-10-11 00:00:00
Tasks: Metamerism, Triplet

The color sensation evoked by an object depends on both the spectral power distribution of the illumination and the reflectance properties of the object being illuminated. The color sensation can be characterized by three color-space values, such as XYZ, RGB, HSV, L*a*b*, etc. It is straightforward to compute the three values given the illuminant and reflectance curves. The converse process of computing a reflectance curve given the color-space values and the illuminant is complicated by the fact that an infinite number of different reflectance curves can give rise to a single set of color-space values (metamerism). This paper presents five algorithms for generating a reflectance curve from a specified sRGB triplet, written for a general audience. The algorithms are designed to generate reflectance curves that are similar to those found with naturally occurring colored objects. The computed reflectance curves are compared to a database of thousands of reflectance curves measured from paints and pigments available both commercially and in nature, and the similarity is quantified. One particularly useful application of these algorithms is in the field of computer graphics, where modeling color transformations sometimes requires wavelength-specific information, such as when modeling subtractive color mixture.
The Quality of the Covariance Selection Through Detection Problem and AUC Bounds
Authors: Navid Tafaghodi Khajavi, Anthony Kuh
Date: 2016-05-18 00:00:00
Tasks: Model Selection

We consider the problem of quantifying the quality of a model selection
problem for a graphical model. We discuss this by formulating the problem as a
detection problem. Model selection problems usually minimize a distance between
the original distribution and the model distribution. For the special case of
Gaussian distributions, the model selection problem simplifies to the
covariance selection problem which is widely discussed in literature by
Dempster [2] where the likelihood criterion is maximized or equivalently the
Kullback-Leibler (KL) divergence is minimized to compute the model covariance
matrix. While this solution is optimal for Gaussian distributions in the sense
of the KL divergence, it is not optimal when compared with other information
divergences and criteria such as Area Under the Curve (AUC).
  In this paper, we analytically compute upper and lower bounds for the AUC and
discuss the quality of model selection problem using the AUC and its bounds as
an accuracy measure in detection problem. We define the correlation
approximation matrix (CAM) and show that analytical computation of the KL
divergence, the AUC and its bounds only depend on the eigenvalues of CAM. We
also show the relationship between the AUC, the KL divergence and the ROC curve
by optimizing with respect to the ROC curve. In the examples provided, we pick
tree structures as the simplest graphical models. We perform simulations on
fully-connected graphs and compute the tree structured models by applying the
widely used Chow-Liu algorithm [3]. Examples show that the quality of tree
approximation models are not good in general based on information divergences,
the AUC and its bounds when the number of nodes in the graphical model is
large. We show both analytically and by simulations that the 1-AUC for the tree
approximation model decays exponentially as the dimension of graphical model
increases.
A Survey Of Cross-lingual Word Embedding Models
Authors: Sebastian Ruder, Ivan Vulić, Anders Søgaard
Date: 2017-06-15 00:00:00
Tasks: Cross-Lingual Transfer, Cross-Lingual Word Embeddings, Survey, Word Embeddings

Cross-lingual representations of words enable us to reason about word meaning in multilingual contexts and are a key facilitator of cross-lingual transfer when developing natural language processing models for low-resource languages. In this survey, we provide a comprehensive typology of cross-lingual word embedding models. We compare their data requirements and objective functions. The recurring theme of the survey is that many of the models presented in the literature optimize for the same objectives, and that seemingly different models are often equivalent modulo optimization strategies, hyper-parameters, and such. We also discuss the different ways cross-lingual word embeddings are evaluated, as well as future challenges and research horizons.
Simultaneous Recognition and Pose Estimation of Instruments in Minimally Invasive Surgery
Authors: Thomas Kurmann, Pablo Marquez Neila, Xiaofei Du, Pascal Fua, Danail Stoyanov, Sebastian Wolf, Raphael Sznitman
Date: 2017-10-18 00:00:00
Tasks: Pose Estimation

Detection of surgical instruments plays a key role in ensuring patient safety
in minimally invasive surgery. In this paper, we present a novel method for 2D
vision-based recognition and pose estimation of surgical instruments that
generalizes to different surgical applications. At its core, we propose a novel
scene model in order to simultaneously recognize multiple instruments as well
as their parts. We use a Convolutional Neural Network architecture to embody
our model and show that the cross-entropy loss is well suited to optimize its
parameters which can be trained in an end-to-end fashion. An additional
advantage of our approach is that instrument detection at test time is achieved
while avoiding the need for scale-dependent sliding window evaluation. This
allows our approach to be relatively parameter free at test time and shows good
performance for both instrument detection and tracking. We show that our
approach surpasses state-of-the-art results on in-vivo retinal microsurgery
image data, as well as ex-vivo laparoscopic sequences.
Deceased Organ Matching in Australia
Authors: Toby Walsh
Date: 2017-10-18 00:00:00

Despite efforts to increase the supply of organs from living donors, most
kidney transplants performed in Australia still come from deceased donors. The
age of these donated organs has increased substantially in recent decades as
the rate of fatal accidents on roads has fallen. The Organ and Tissue Authority
in Australia is therefore looking to design a new mechanism that better matches
the age of the organ to the age of the patient. I discuss the design,
axiomatics and performance of several candidate mechanisms that respect the
special online nature of this fair division problem.