pwc-archive/papers-with-abstracts huggingface dataset:
Deep learning long-range information in undirected graphs with wave networks
Authors: Matthew K. Matlock, Arghya Datta, Na Le Dang, Kevin Jiang, S. Joshua Swamidass
Date: 2018-10-29 00:00:00

Graph algorithms are key tools in many fields of science and technology. Some
of these algorithms depend on propagating information between distant nodes in
a graph. Recently, there have been a number of deep learning architectures
proposed to learn on undirected graphs. However, most of these architectures
aggregate information in the local neighborhood of a node, and therefore they
may not be capable of efficiently propagating long-range information. To solve
this problem we examine a recently proposed architecture, wave, which
propagates information back and forth across an undirected graph in waves of
nonlinear computation. We compare wave to graph convolution, an architecture
based on local aggregation, and find that wave learns three different
graph-based tasks with greater efficiency and accuracy. These three tasks
include (1) labeling a path connecting two nodes in a graph, (2) solving a maze
presented as an image, and (3) computing voltages in a circuit. These tasks
range from trivial to very difficult, but wave can extrapolate from small
training examples to much larger testing examples. These results show that wave
may be able to efficiently solve a wide range of problems that require
long-range information propagation across undirected graphs. An implementation
of the wave network, and example code for the maze problem are included in the
tflon deep learning toolkit (https://bitbucket.org/mkmatlock/tflon).
Imagination Based Sample Construction for Zero-Shot Learning
Authors: Gang Yang, Jinlu Liu, Xirong Li
Date: 2018-10-29 00:00:00
Tasks: Attribute, Image Retrieval, Retrieval, Zero-Shot Learning

Zero-shot learning (ZSL) which aims to recognize unseen classes with no
labeled training sample, efficiently tackles the problem of missing labeled
data in image retrieval. Nowadays there are mainly two types of popular methods
for ZSL to recognize images of unseen classes: probabilistic reasoning and
feature projection. Different from these existing types of methods, we propose
a new method: sample construction to deal with the problem of ZSL. Our proposed
method, called Imagination Based Sample Construction (IBSC), innovatively
constructs image samples of target classes in feature space by mimicking human
associative cognition process. Based on an association between attribute and
feature, target samples are constructed from different parts of various
samples. Furthermore, dissimilarity representation is employed to select
high-quality constructed samples which are used as labeled data to train a
specific classifier for those unseen classes. In this way, zero-shot learning
is turned into a supervised learning problem. As far as we know, it is the
first work to construct samples for ZSL thus, our work is viewed as a baseline
for future sample construction methods. Experiments on four benchmark datasets
show the superiority of our proposed method.
Unsupervised Data Selection for Supervised Learning
Authors: Gabriele Valvano, Andrea Leo, Daniele Della Latta, Nicola Martini, Gianmarco Santini, Dante Chiappino, Emiliano Ricciardi
Date: 2018-10-29 00:00:00

Recent research put a big effort in the development of deep learning
architectures and optimizers obtaining impressive results in areas ranging from
vision to language processing. However little attention has been addressed to
the need of a methodological process of data collection. In this work we
hypothesize that high quality data for supervised learning can be selected in
an unsupervised manner and that by doing so one can obtain models capable to
generalize better than in the case of random training set construction.
However, preliminary results are not robust and further studies on the subject
should be carried out.
A Convergence indicator for Multi-Objective Optimisation Algorithms
Authors: Thiago Santos, Sebastiao Xavier
Date: 2018-10-29 00:00:00
Tasks: Diversity

The algorithms of multi-objective optimisation had a relative growth in the
last years. Thereby, it's requires some way of comparing the results of these.
In this sense, performance measures play a key role. In general, it's
considered some properties of these algorithms such as capacity, convergence,
diversity or convergence-diversity. There are some known measures such as
generational distance (GD), inverted generational distance (IGD), hypervolume
(HV), Spread($\Delta$), Averaged Hausdorff distance ($\Delta_p$), R2-indicator,
among others. In this paper, we focuses on proposing a new indicator to measure
convergence based on the traditional formula for Shannon entropy. The main
features about this measure are: 1) It does not require tho know the true
Pareto set and 2) Medium computational cost when compared with Hypervolume.
Audio inpainting of music by means of neural networks
Authors: Andrés Marafioti, Nicki Holighaus, Piotr Majdak, Nathanaël Perraudin
Date: 2018-10-29 00:00:00
Tasks: Audio Generation, Audio inpainting

We studied the ability of deep neural networks (DNNs) to restore missing audio content based on its context, a process usually referred to as audio inpainting. We focused on gaps in the range of tens of milliseconds. The proposed DNN structure was trained on audio signals containing music and musical instruments, separately, with 64-ms long gaps. The input to the DNN was the context, i.e., the signal surrounding the gap, transformed into time-frequency (TF) coefficients. Our results were compared to those obtained from a reference method based on linear predictive coding (LPC). For music, our DNN significantly outperformed the reference method, demonstrating a generally good usability of the proposed DNN structure for inpainting complex audio signals like music.
Phase Harmonic Correlations and Convolutional Neural Networks
Authors: Stéphane Mallat, Sixin Zhang, Gaspar Rochette
Date: 2018-10-29 00:00:00
Tasks: Time Series, Time Series Analysis

A major issue in harmonic analysis is to capture the phase dependence of frequency representations, which carries important signal properties. It seems that convolutional neural networks have found a way. Over time-series and images, convolutional networks often learn a first layer of filters which are well localized in the frequency domain, with different phases. We show that a rectifier then acts as a filter on the phase of the resulting coefficients. It computes signal descriptors which are local in space, frequency and phase. The non-linear phase filter becomes a multiplicative operator over phase harmonics computed with a Fourier transform along the phase. We prove that it defines a bi-Lipschitz and invertible representation. The correlations of phase harmonics coefficients characterise coherent structures from their phase dependence across frequencies. For wavelet filters, we show numerically that signals having sparse wavelet coefficients can be recovered from few phase harmonic correlations, which provide a compressive representation
ActionXPose: A Novel 2D Multi-view Pose-based Algorithm for Real-time Human Action Recognition
Authors: Federico Angelini, Zeyu Fu, Yang Long, Ling Shao, Syed Mohsen Naqvi
Date: 2018-10-29 00:00:00
Tasks: Action Recognition, Temporal Action Localization

We present ActionXPose, a novel 2D pose-based algorithm for posture-level
Human Action Recognition (HAR). The proposed approach exploits 2D human poses
provided by OpenPose detector from RGB videos. ActionXPose aims to process
poses data to be provided to a Long Short-Term Memory Neural Network and to a
1D Convolutional Neural Network, which solve the classification problem.
ActionXPose is one of the first algorithms that exploits 2D human poses for
HAR. The algorithm has real-time performance and it is robust to camera
movings, subject proximity changes, viewpoint changes, subject appearance
changes and provide high generalization degree. In fact, extensive simulations
show that ActionXPose can be successfully trained using different datasets at
once. State-of-the-art performance on popular datasets for posture-related HAR
problems (i3DPost, KTH) are provided and results are compared with those
obtained by other methods, including the selected ActionXPose baseline.
Moreover, we also proposed two novel datasets called MPOSE and ISLD recorded in
our Intelligent Sensing Lab, to show ActionXPose generalization performance.
Gradual Machine Learning for Entity Resolution
Authors: Boyi Hou, Qun Chen, Yanyan Wang, Youcef Nafa, Zhanhuai Li
Date: 2018-10-29 00:00:00
Tasks: BIG-bench Machine Learning, Entity Resolution, General Classification

Usually considered as a classification problem, entity resolution (ER) can be very challenging on real data due to the prevalence of dirty values. The state-of-the-art solutions for ER were built on a variety of learning models (most notably deep neural networks), which require lots of accurately labeled training data. Unfortunately, high-quality labeled data usually require expensive manual work, and are therefore not readily available in many real scenarios. In this paper, we propose a novel learning paradigm for ER, called gradual machine learning, which aims to enable effective machine labeling without the requirement for manual labeling effort. It begins with some easy instances in a task, which can be automatically labeled by the machine with high accuracy, and then gradually labels more challenging instances by iterative factor graph inference. In gradual machine learning, the hard instances in a task are gradually labeled in small stages based on the estimated evidential certainty provided by the labeled easier instances. Our extensive experiments on real data have shown that the performance of the proposed approach is considerably better than its unsupervised alternatives, and highly competitive compared to the state-of-the-art supervised techniques. Using ER as a test case, we demonstrate that gradual machine learning is a promising paradigm potentially applicable to other challenging classification tasks requiring extensive labeling effort.
Burst ranking for blind multi-image deblurring
Authors: Fidel A. Guerrero Peña, Pedro D. Marrero Fernández, Tsang Ing Ren, Jorge J. G. Leandro, Ricardo Nishihara
Date: 2018-10-29 00:00:00
Tasks: Deblurring, Image Deblurring

We propose a new incremental aggregation algorithm for multi-image deblurring
with automatic image selection. The primary motivation is that current bursts
deblurring methods do not handle well situations in which misalignment or
out-of-context frames are present in the burst. These real-life situations
result in poor reconstructions or manual selection of the images that will be
used to deblur. Automatically selecting best frames within the burst to improve
the base reconstruction is challenging because the amount of possible images
fusions is equal to the power set cardinal. Here, we approach the multi-image
deblurring problem as a two steps process. First, we successfully learn a
comparison function to rank a burst of images using a deep convolutional neural
network. Then, an incremental Fourier burst accumulation with a reconstruction
degradation mechanism is applied fusing only less blurred images that are
sufficient to maximize the reconstruction quality. Experiments with the
proposed algorithm have shown superior results when compared to other similar
approaches, outperforming other methods described in the literature in
previously described situations. We validate our findings on several synthetic
and real datasets.
Using Machine Learning to Predict the Evolution of Physics Research
Authors: Wenyuan Liu, Stanisław Saganowski, Przemysław Kazienko, Siew Ann Cheong
Date: 2018-10-29 00:00:00
Tasks: BIG-bench Machine Learning, Descriptive

The advancement of science as outlined by Popper and Kuhn is largely
qualitative, but with bibliometric data it is possible and desirable to develop
a quantitative picture of scientific progress. Furthermore it is also important
to allocate finite resources to research topics that have growth potential, to
accelerate the process from scientific breakthroughs to technological
innovations. In this paper, we address this problem of quantitative knowledge
evolution by analysing the APS publication data set from 1981 to 2010. We build
the bibliographic coupling and co-citation networks, use the Louvain method to
detect topical clusters (TCs) in each year, measure the similarity of TCs in
consecutive years, and visualize the results as alluvial diagrams. Having the
predictive features describing a given TC and its known evolution in the next
year, we can train a machine learning model to predict future changes of TCs,
i.e., their continuing, dissolving, merging and splitting. We found the number
of papers from certain journals, the degree, closeness, and betweenness to be
the most predictive features. Additionally, betweenness increases significantly
for merging events, and decreases significantly for splitting events. Our
results represent a first step from a descriptive understanding of the Science
of Science (SciSci), towards one that is ultimately prescriptive.
Learning to Teach with Dynamic Loss Functions
Authors: Lijun Wu, Fei Tian, Yingce Xia, Yang Fan, Tao Qin, Jian-Huang Lai, Tie-Yan Liu
Date: 2018-10-29 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: BIG-bench Machine Learning, image-classification, Image Classification, Machine Translation

Teaching is critical to human society: it is with teaching that prospective
students are educated and human civilization can be inherited and advanced. A
good teacher not only provides his/her students with qualified teaching
materials (e.g., textbooks), but also sets up appropriate learning objectives
(e.g., course projects and exams) considering different situations of a
student. When it comes to artificial intelligence, treating machine learning
models as students, the loss functions that are optimized act as perfect
counterparts of the learning objective set by the teacher. In this work, we
explore the possibility of imitating human teaching behaviors by dynamically
and automatically outputting appropriate loss functions to train machine
learning models. Different from typical learning settings in which the loss
function of a machine learning model is predefined and fixed, in our framework,
the loss function of a machine learning model (we call it student) is defined
by another machine learning model (we call it teacher). The ultimate goal of
teacher model is cultivating the student to have better performance measured on
development dataset. Towards that end, similar to human teaching, the teacher,
a parametric model, dynamically outputs different loss functions that will be
used and optimized by its student model at different training stages. We
develop an efficient learning method for the teacher model that makes gradient
based optimization possible, exempt of the ineffective solutions such as policy
optimization. We name our method as "learning to teach with dynamic loss
functions" (L2T-DLF for short). Extensive experiments on real world tasks
including image classification and neural machine translation demonstrate that
our method significantly improves the quality of various student models.
Marginal Replay vs Conditional Replay for Continual Learning
Authors: Timothée Lesort, Alexander Gepperth, Andrei Stoian, David Filliat
Date: 2018-10-29 00:00:00
Tasks: Classification, Continual Learning, General Classification

We present a new replay-based method of continual classification learning that we term "conditional replay" which generates samples and labels together by sampling from a distribution conditioned on the class. We compare conditional replay to another replay-based continual learning paradigm (which we term "marginal replay") that generates samples independently of their class and assigns labels in a separate step. The main improvement in conditional replay is that labels for generated samples need not be inferred, which reduces the margin for error in complex continual classification learning tasks. We demonstrate the effectiveness of this approach using novel and standard benchmarks constructed from MNIST and FashionMNIST data, and compare to the regularization-based \textit{elastic weight consolidation} (EWC) method.
On the Convergence Rate of Training Recurrent Neural Networks
Authors: Zeyuan Allen-Zhu, Yuanzhi Li, Zhao Song
Date: 2018-10-29 00:00:00
Proceeding: NeurIPS 2019 12

How can local-search methods such as stochastic gradient descent (SGD) avoid bad local minima in training multi-layer neural networks? Why can they fit random labels even given non-convex and non-smooth architectures? Most existing theory only covers networks with one hidden layer, so can we go deeper? In this paper, we focus on recurrent neural networks (RNNs) which are multi-layer networks widely used in natural language processing. They are harder to analyze than feedforward neural networks, because the $\textit{same}$ recurrent unit is repeatedly applied across the entire time horizon of length $L$, which is analogous to feedforward networks of depth $L$. We show when the number of neurons is sufficiently large, meaning polynomial in the training data size and in $L$, then SGD is capable of minimizing the regression loss in the linear convergence rate. This gives theoretical evidence of how RNNs can memorize data. More importantly, in this paper we build general toolkits to analyze multi-layer networks with ReLU activations. For instance, we prove why ReLU activations can prevent exponential gradient explosion or vanishing, and build a perturbation theory to analyze first-order approximation of multi-layer networks.
PartsNet: A Unified Deep Network for Automotive Engine Precision Parts Defect Detection
Authors: Zhenshen Qu, Jianxiong Shen, Ruikun Li, Junyu Liu, Qiuyu Guan
Date: 2018-10-29 00:00:00
Tasks: Defect Detection, Segmentation

Defect detection is a basic and essential task in automatic parts production,
especially for automotive engine precision parts. In this paper, we propose a
new idea to construct a deep convolutional network combining related knowledge
of feature processing and the representation ability of deep learning. Our
algorithm consists of a pixel-wise segmentation Deep Neural Network (DNN) and a
feature refining network. The fully convolutional DNN is presented to learn
basic features of parts defects. After that, several typical traditional
methods which are used to refine the segmentation results are transformed into
convolutional manners and integrated. We assemble these methods as a shallow
network with fixed weights and empirical thresholds. These thresholds are then
released to enhance its adaptation ability and realize end-to-end training.
Testing results on different datasets show that the proposed method has good
portability and outperforms the state-of-the-art algorithms.
Speaking style adaptation in Text-To-Speech synthesis using Sequence-to-sequence models with attention
Authors: Bajibabu Bollepalli, Lauri Juvela, Paavo Alku
Date: 2018-10-29 00:00:00
Tasks: Speech Synthesis, text-to-speech, Text to Speech, Text-To-Speech Synthesis, Transfer Learning

Currently, there are increasing interests in text-to-speech (TTS) synthesis
to use sequence-to-sequence models with attention. These models are end-to-end
meaning that they learn both co-articulation and duration properties directly
from text and speech. Since these models are entirely data-driven, they need
large amounts of data to generate synthetic speech with good quality. However,
in challenging speaking styles, such as Lombard speech, it is difficult to
record sufficiently large speech corpora. Therefore, in this study we propose a
transfer learning method to adapt a sequence-to-sequence based TTS system of
normal speaking style to Lombard style. Moreover, we experiment with a WaveNet
vocoder in synthesis of Lombard speech. We conducted subjective evaluations to
assess the performance of the adapted TTS systems. The subjective evaluation
results indicated that an adaptation system with the WaveNet vocoder clearly
outperformed the conventional deep neural network based TTS system in synthesis
of Lombard speech.
Patch-Based Sparse Representation For Bacterial Detection
Authors: Ahmed Karam Eldaly, Yoann Altmann, Ahsan Akram, Antonios Perperidis, Kevin Dhaliwal, Stephen McLaughlin
Date: 2018-10-29 00:00:00

In this paper, we propose an unsupervised approach for bacterial detection in
optical endomicroscopy images. This approach splits each image into a set of
overlapping patches and assumes that observed intensities are linear
combinations of the actual intensity values associated with background image
structures, corrupted by additive Gaussian noise and potentially by a sparse
outlier term modelling anomalies (which are considered to be candidate
bacteria). The actual intensity term representing background structures is
modelled as a linear combination of a few atoms drawn from a dictionary which
is learned from bacteria-free data and then fixed while analyzing new images.
The bacteria detection task is formulated as a minimization problem and an
alternating direction method of multipliers (ADMM) is then used to estimate the
unknown parameters. Simulations conducted using two ex vivo lung datasets show
good detection and correlation performance between bacteria counts identified
by a trained clinician and those of the proposed method.
Logit Pairing Methods Can Fool Gradient-Based Attacks
Authors: Marius Mosbach, Maksym Andriushchenko, Thomas Trost, Matthias Hein, Dietrich Klakow
Date: 2018-10-29 00:00:00
Tasks: Adversarial Robustness

Recently, Kannan et al. [2018] proposed several logit regularization methods
to improve the adversarial robustness of classifiers. We show that the
computationally fast methods they propose - Clean Logit Pairing (CLP) and Logit
Squeezing (LSQ) - just make the gradient-based optimization problem of crafting
adversarial examples harder without providing actual robustness. We find that
Adversarial Logit Pairing (ALP) may indeed provide robustness against
adversarial examples, especially when combined with adversarial training, and
we examine it in a variety of settings. However, the increase in adversarial
accuracy is much smaller than previously claimed. Finally, our results suggest
that the evaluation against an iterative PGD attack relies heavily on the
parameters used and may result in false conclusions regarding robustness of a
model.