pwc-archive/papers-with-abstracts huggingface dataset:
Diabetes Mellitus Forecasting Using Population Health Data in Ontario, Canada
Authors: Mathieu Ravaut, Hamed Sadeghi, Kin Kwan Leung, Maksims Volkovs, Laura C. Rosella
Date: 2019-04-08 00:00:00
Tasks: BIG-bench Machine Learning, Diagnostic

Leveraging health administrative data (HAD) datasets for predicting the risk
of chronic diseases including diabetes has gained a lot of attention in the
machine learning community recently. In this paper, we use the largest health
records datasets of patients in Ontario,Canada. Provided by the Institute of
Clinical Evaluative Sciences (ICES), this database is age, gender and
ethnicity-diverse. The datasets include demographics, lab measurements,drug
benefits, healthcare system interactions, ambulatory and hospitalizations
records. We perform one of the first large-scale machine learning studies with
this data to study the task of predicting diabetes in a range of 1-10 years
ahead, which requires no additional screening of individuals.In the best setup,
we reach a test AUC of 80.3 with a single-model trained on an observation
window of 5 years with a one-year buffer using all datasets. A subset of top 15
features alone (out of a total of 963) could provide a test AUC of 79.1. In
this paper, we provide extensive machine learning model performance and feature
contribution analysis, which enables us to narrow down to the most important
features useful for diabetes forecasting. Examples include chronic conditions
such as asthma and hypertension, lab results, diagnostic codes in insurance
claims, age and geographical information.
ASAP: Architecture Search, Anneal and Prune
Authors: Asaf Noy, Niv Nayman, Tal Ridnik, Nadav Zamir, Sivan Doveh, Itamar Friedman, Raja Giryes, Lihi Zelnik-Manor
Date: 2019-04-08 00:00:00
Tasks: GPU, Neural Architecture Search

Automatic methods for Neural Architecture Search (NAS) have been shown to produce state-of-the-art network models. Yet, their main drawback is the computational complexity of the search process. As some primal methods optimized over a discrete search space, thousands of days of GPU were required for convergence. A recent approach is based on constructing a differentiable search space that enables gradient-based optimization, which reduces the search time to a few days. While successful, it still includes some noncontinuous steps, e.g., the pruning of many weak connections at once. In this paper, we propose a differentiable search space that allows the annealing of architecture weights, while gradually pruning inferior operations. In this way, the search converges to a single output network in a continuous manner. Experiments on several vision datasets demonstrate the effectiveness of our method with respect to the search cost and accuracy of the achieved model. Specifically, with $0.2$ GPU search days we achieve an error rate of $1.68\%$ on CIFAR-10.
Completely Unsupervised Speech Recognition By A Generative Adversarial Network Harmonized With Iteratively Refined Hidden Markov Models
Authors: Kuan-Yu Chen, Che-Ping Tsai, Da-Rong Liu, Hung-Yi Lee, Lin-shan Lee
Date: 2019-04-08 00:00:00
Tasks: Generative Adversarial Network, speech-recognition, Speech Recognition, Unsupervised Speech Recognition

Producing a large annotated speech corpus for training ASR systems remains difficult for more than 95% of languages all over the world which are low-resourced, but collecting a relatively big unlabeled data set for such languages is more achievable. This is why some initial effort have been reported on completely unsupervised speech recognition learned from unlabeled data only, although with relatively high error rates. In this paper, we develop a Generative Adversarial Network (GAN) to achieve this purpose, in which a Generator and a Discriminator learn from each other iteratively to improve the performance. We further use a set of Hidden Markov Models (HMMs) iteratively refined from the machine generated labels to work in harmony with the GAN. The initial experiments on TIMIT data set achieve an phone error rate of 33.1%, which is 8.5% lower than the previous state-of-the-art.
Weighted Point Cloud Augmentation for Neural Network Training Data Class-Imbalance
Authors: David Griffiths, Jan Boehm
Date: 2019-04-08 00:00:00

Recent developments in the field of deep learning for 3D data have
demonstrated promising potential for end-to-end learning directly from point
clouds. However, many real-world point clouds contain a large class im-balance
due to the natural class im-balance observed in nature. For example, a 3D scan
of an urban environment will consist mostly of road and facade, whereas other
objects such as poles will be under-represented. In this paper we address this
issue by employing a weighted augmentation to increase classes that contain
fewer points. By mitigating the class im-balance present in the data we
demonstrate that a standard PointNet++ deep neural network can achieve higher
performance at inference on validation data. This was observed as an increase
of F1 score of 19% and 25% on two test benchmark datasets; ScanNet and
Semantic3D respectively where no class im-balance pre-processing had been
performed. Our networks performed better on both highly-represented and
under-represented classes, which indicates that the network is learning more
robust and meaningful features when the loss function is not overly exposed to
only a few classes.
Leveraging the Invariant Side of Generative Zero-Shot Learning
Authors: Jingjing Li, Mengmeng Jin, Ke Lu, Zhengming Ding, Lei Zhu, Zi Huang
Date: 2019-04-08 00:00:00
Proceeding: CVPR 2019 6
Tasks: Generalized Zero-Shot Learning, Zero-Shot Learning

Conventional zero-shot learning (ZSL) methods generally learn an embedding,
e.g., visual-semantic mapping, to handle the unseen visual samples via an
indirect manner. In this paper, we take the advantage of generative adversarial
networks (GANs) and propose a novel method, named leveraging invariant side GAN
(LisGAN), which can directly generate the unseen features from random noises
which are conditioned by the semantic descriptions. Specifically, we train a
conditional Wasserstein GANs in which the generator synthesizes fake unseen
features from noises and the discriminator distinguishes the fake from real via
a minimax game. Considering that one semantic description can correspond to
various synthesized visual samples, and the semantic description, figuratively,
is the soul of the generated features, we introduce soul samples as the
invariant side of generative zero-shot learning in this paper. A soul sample is
the meta-representation of one class. It visualizes the most
semantically-meaningful aspects of each sample in the same category. We
regularize that each generated sample (the varying side of generative ZSL)
should be close to at least one soul sample (the invariant side) which has the
same class label with it. At the zero-shot recognition stage, we propose to use
two classifiers, which are deployed in a cascade way, to achieve a
coarse-to-fine result. Experiments on five popular benchmarks verify that our
proposed approach can outperform state-of-the-art methods with significant
improvements.
Large Margin Multi-modal Multi-task Feature Extraction for Image Classification
Authors: Yong Luo, Yonggang Wen, DaCheng Tao, Jie Gui, Chao Xu
Date: 2019-04-08 00:00:00
Tasks: General Classification, image-classification, Image Classification

The features used in many image analysis-based applications are frequently of
very high dimension. Feature extraction offers several advantages in
high-dimensional cases, and many recent studies have used multi-task feature
extraction approaches, which often outperform single-task feature extraction
approaches. However, most of these methods are limited in that they only
consider data represented by a single type of feature, even though features
usually represent images from multiple modalities. We therefore propose a novel
large margin multi-modal multi-task feature extraction (LM3FE) framework for
handling multi-modal features for image classification. In particular, LM3FE
simultaneously learns the feature extraction matrix for each modality and the
modality combination coefficients. In this way, LM3FE not only handles
correlated and noisy features, but also utilizes the complementarity of
different modalities to further help reduce feature redundancy in each
modality. The large margin principle employed also helps to extract strongly
predictive features so that they are more suitable for prediction (e.g.,
classification). An alternating algorithm is developed for problem optimization
and each sub-problem can be efficiently solved. Experiments on two challenging
real-world image datasets demonstrate the effectiveness and superiority of the
proposed method.
ContextDesc: Local Descriptor Augmentation with Cross-Modality Context
Authors: Zixin Luo, Tianwei Shen, Lei Zhou, Jiahui Zhang, Yao Yao, Shiwei Li, Tian Fang, Long Quan
Date: 2019-04-08 00:00:00
Proceeding: CVPR 2019 6
Tasks: Geometric Matching

Most existing studies on learning local features focus on the patch-based
descriptions of individual keypoints, whereas neglecting the spatial relations
established from their keypoint locations. In this paper, we go beyond the
local detail representation by introducing context awareness to augment
off-the-shelf local feature descriptors. Specifically, we propose a unified
learning framework that leverages and aggregates the cross-modality contextual
information, including (i) visual context from high-level image representation,
and (ii) geometric context from 2D keypoint distribution. Moreover, we propose
an effective N-pair loss that eschews the empirical hyper-parameter search and
improves the convergence. The proposed augmentation scheme is lightweight
compared with the raw local feature description, meanwhile improves remarkably
on several large-scale benchmarks with diversified scenes, which demonstrates
both strong practicality and generalization ability in geometric matching
applications.
Heterogeneous Multi-task Metric Learning across Multiple Domains
Authors: Yong Luo, Yonggang Wen, DaCheng Tao
Date: 2019-04-08 00:00:00
Tasks: Metric Learning, Scene Classification, Text Categorization, Transfer Learning

Distance metric learning (DML) plays a crucial role in diverse machine
learning algorithms and applications. When the labeled information in target
domain is limited, transfer metric learning (TML) helps to learn the metric by
leveraging the sufficient information from other related domains. Multi-task
metric learning (MTML), which can be regarded as a special case of TML,
performs transfer across all related domains. Current TML tools usually assume
that the same feature representation is exploited for different domains.
However, in real-world applications, data may be drawn from heterogeneous
domains. Heterogeneous transfer learning approaches can be adopted to remedy
this drawback by deriving a metric from the learned transformation across
different domains. But they are often limited in that only two domains can be
handled. To appropriately handle multiple domains, we develop a novel
heterogeneous multi-task metric learning (HMTML) framework. In HMTML, the
metrics of all different domains are learned together. The transformations
derived from the metrics are utilized to induce a common subspace, and the
high-order covariance among the predictive structures of these domains is
maximized in this subspace. There do exist a few heterogeneous transfer
learning approaches that deal with multiple domains, but the high-order
statistics (correlation information), which can only be exploited by
simultaneously examining all domains, is ignored in these approaches. Compared
with them, the proposed HMTML can effectively explore such high-order
information, thus obtaining more reliable feature transformations and metrics.
Effectiveness of our method is validated by the extensive and intensive
experiments on text categorization, scene classification, and social image
annotation.
Transferring Knowledge Fragments for Learning Distance Metric from A Heterogeneous Domain
Authors: Yong Luo, Yonggang Wen, Tongliang Liu, DaCheng Tao
Date: 2019-04-08 00:00:00
Tasks: Metric Learning, Transfer Learning

The goal of transfer learning is to improve the performance of target
learning task by leveraging information (or transferring knowledge) from other
related tasks. In this paper, we examine the problem of transfer distance
metric learning (DML), which usually aims to mitigate the label information
deficiency issue in the target DML. Most of the current Transfer DML (TDML)
methods are not applicable to the scenario where data are drawn from
heterogeneous domains. Some existing heterogeneous transfer learning (HTL)
approaches can learn target distance metric by usually transforming the samples
of source and target domain into a common subspace. However, these approaches
lack flexibility in real-world applications, and the learned transformations
are often restricted to be linear. This motivates us to develop a general
flexible heterogeneous TDML (HTDML) framework. In particular, any
(linear/nonlinear) DML algorithms can be employed to learn the source metric
beforehand. Then the pre-learned source metric is represented as a set of
knowledge fragments to help target metric learning. We show how generalization
error in the target domain could be reduced using the proposed transfer
strategy, and develop novel algorithm to learn either linear or nonlinear
target metric. Extensive experiments on various applications demonstrate the
effectiveness of the proposed method.
Nucleus Neural Network: A Data-driven Self-organized Architecture
Authors: Jia Liu, Maoguo Gong, Haibo He
Date: 2019-04-08 00:00:00

Artificial neural networks which are inspired from the learning mechanism of brain have achieved great successes in many problems, especially those with deep layers. In this paper, we propose a nucleus neural network (NNN) and corresponding connecting architecture learning method. In a nucleus, there are no regular layers, i.e., a neuron may connect to all the neurons in the nucleus. This type of architecture gets rid of layer limitation and may lead to more powerful learning capability. It is crucial to determine the connections between them given numerous neurons. Based on the principle that more relevant input and output neuron pair deserves higher connecting density, we propose an efficient architecture learning model for the nucleus. Moreover, we improve the learning method for connecting weights and biases given the optimized architecture. We find that this novel architecture is robust to irrelevant components in test data. So we reconstruct a new dataset based on the MNIST dataset where the types of digital backgrounds in training and test sets are different. Experiments demonstrate that the proposed learner achieves significant improvement over traditional learners on the reconstructed data set.
Plant-wide fault and disturbance screening using combined transfer entropy and eigenvector centrality analysis
Authors: Simon Streicher, Carl Sandrock
Date: 2019-04-08 00:00:00
Tasks: Fault Detection

Finding the source of a disturbance or fault in complex systems such as
industrial chemical processing plants can be a difficult task and consume a
significant number of engineering hours. In many cases, a systematic
elimination procedure is considered to be the only feasible approach but can
cause undesired process upsets. Practitioners desire robust alternative
approaches.
  This paper presents an unsupervised, data-driven method for ranking process
elements according to the magnitude and novelty of their influence. Partial
bivariate transfer entropy estimation is used to infer a weighted directed
graph of process elements. Eigenvector centrality is applied to rank network
nodes according to their overall effect. As the ranking of process elements
rely on emerging properties that depend on the aggregate of many connections,
the results are robust to errors in the estimation of individual edge
properties and the inclusion of indirect connections that do not represent the
true causal structure of the process.
  A monitoring chart of continuously calculated process element importance
scores over multiple overlapping time regions can assist with incipient fault
detection. Ranking results combined with visual inspection of information
transfer networks is also useful for root cause analysis of known faults and
disturbances. A software implementation of the proposed method is available.
Evaluation of Greek Word Embeddings
Authors: Stamatis Outsios, Christos Karatsalos, Konstantinos Skianis, Michalis Vazirgiannis
Date: 2019-04-08 00:00:00
Proceeding: LREC 2020 5
Tasks: Word Embeddings

Since word embeddings have been the most popular input for many NLP tasks, evaluating their quality is of critical importance. Most research efforts are focusing on English word embeddings. This paper addresses the problem of constructing and evaluating such models for the Greek language. We created a new word analogy corpus considering the original English Word2vec word analogy corpus and some specific linguistic aspects of the Greek language as well. Moreover, we created a Greek version of WordSim353 corpora for a basic evaluation of word similarities. We tested seven word vector models and our evaluation showed that we are able to create meaningful representations. Last, we discovered that the morphological complexity of the Greek language and polysemy can influence the quality of the resulting word embeddings.
Only Relevant Information Matters: Filtering Out Noisy Samples to Boost RL
Authors: Yannis Flet-Berliac, Philippe Preux
Date: 2019-04-08 00:00:00
Tasks: continuous-control, Continuous Control, Denoising, Policy Gradient Methods, Reinforcement Learning

In reinforcement learning, policy gradient algorithms optimize the policy directly and rely on sampling efficiently an environment. Nevertheless, while most sampling procedures are based on direct policy sampling, self-performance measures could be used to improve such sampling prior to each policy update. Following this line of thought, we introduce SAUNA, a method where non-informative transitions are rejected from the gradient update. The level of information is estimated according to the fraction of variance explained by the value function: a measure of the discrepancy between V and the empirical returns. In this work, we use this metric to select samples that are useful to learn from, and we demonstrate that this selection can significantly improve the performance of policy gradient methods. In this paper: (a) We define SAUNA's metric and introduce its method to filter transitions. (b) We conduct experiments on a set of benchmark continuous control problems. SAUNA significantly improves performance. (c) We investigate how SAUNA reliably selects samples with the most positive impact on learning and study its improvement on both performance and sample efficiency.
CRAD: Clustering with Robust Autocuts and Depth
Authors: Xin Huang, Yulia R. Gel
Date: 2019-04-08 00:00:00
Tasks: Clustering, Time Series, Time Series Analysis, Time Series Clustering

We develop a new density-based clustering algorithm named CRAD which is based
on a new neighbor searching function with a robust data depth as the
dissimilarity measure. Our experiments prove that the new CRAD is highly
competitive at detecting clusters with varying densities, compared with the
existing algorithms such as DBSCAN, OPTICS and DBCA. Furthermore, a new
effective parameter selection procedure is developed to select the optimal
underlying parameter in the real-world clustering, when the ground truth is
unknown. Lastly, we suggest a new clustering framework that extends CRAD from
spatial data clustering to time series clustering without a-priori knowledge of
the true number of clusters. The performance of CRAD is evaluated through
extensive experimental studies.
On a generalization of the Jensen-Shannon divergence and the JS-symmetrization of distances relying on abstract means
Authors: Frank Nielsen
Date: 2019-04-08 00:00:00
Tasks: Clustering, Form

The Jensen-Shannon divergence is a renown bounded symmetrization of the unbounded Kullback-Leibler divergence which measures the total Kullback-Leibler divergence to the average mixture distribution. However the Jensen-Shannon divergence between Gaussian distributions is not available in closed-form. To bypass this problem, we present a generalization of the Jensen-Shannon (JS) divergence using abstract means which yields closed-form expressions when the mean is chosen according to the parametric family of distributions. More generally, we define the JS-symmetrizations of any distance using generalized statistical mixtures derived from abstract means. In particular, we first show that the geometric mean is well-suited for exponential families, and report two closed-form formula for (i) the geometric Jensen-Shannon divergence between probability densities of the same exponential family, and (ii) the geometric JS-symmetrization of the reverse Kullback-Leibler divergence. As a second illustrating example, we show that the harmonic mean is well-suited for the scale Cauchy distributions, and report a closed-form formula for the harmonic Jensen-Shannon divergence between scale Cauchy distributions. We also define generalized Jensen-Shannon divergences between matrices (e.g., quantum Jensen-Shannon divergences) and consider clustering with respect to these novel Jensen-Shannon divergences.
Early warning in egg production curves from commercial hens: A SVM approach
Authors: Iván Ramírez Morales, Daniel Rivero Cebrián, Enrique Fernández Blanco, Alejandro Pazos Sierra
Date: 2019-04-08 00:00:00
Tasks: Handwritten Text Recognition, Management, Sensitivity, Specificity, Weather Forecasting

Artificial Intelligence allows the improvement of our daily life, for
instance, speech and handwritten text recognition, real time translation and
weather forecasting are common used applications. In the livestock sector,
machine learning algorithms have the potential for early detection and warning
of problems, which represents a significant milestone in the poultry industry.
Production problems generate economic loss that could be avoided by acting in a
timely manner. In the current study, training and testing of support vector
machines are addressed, for an early detection of problems in the production
curve of commercial eggs, using farm's egg production data of 478,919 laying
hens grouped in 24 flocks. Experiments using support vector machines with a 5
k-fold cross-validation were performed at different previous time intervals, to
alert with up to 5 days of forecasting interval, whether a flock will
experience a problem in production curve. Performance metrics such as accuracy,
specificity, sensitivity, and positive predictive value were evaluated,
reaching 0-day values of 0.9874, 0.9876, 0.9783 and 0.6518 respectively on
unseen data (test-set). The optimal forecasting interval was from zero to three
days, performance metrics decreases as the forecasting interval is increased.
It should be emphasized that this technique was able to issue an alert a day in
advance, achieving an accuracy of 0.9854, a specificity of 0.9865, a
sensitivity of 0.9333 and a positive predictive value of 0.6135. This novel
application embedded in a computer system of poultry management is able to
provide significant improvements in early detection and warning of problems
related to the production curve.