pwc-archive/papers-with-abstracts huggingface dataset:
Towards Using Context-Dependent Symbols in CTC Without State-Tying Decision Trees
Authors: Jan Chorowski, Adrian Lancucki, Bartosz Kostka, Michal Zapotoczny
Date: 2019-01-14 00:00:00
Tasks: Language Modeling, Language Modelling

Deep neural acoustic models benefit from context-dependent (CD) modeling of
output symbols. We consider direct training of CTC networks with CD outputs,
and identify two issues. The first one is frame-level normalization of
probabilities in CTC, which induces strong language modeling behavior that
leads to overfitting and interference with external language models. The second
one is poor generalization in the presence of numerous lexical units like
triphones or tri-chars. We mitigate the former with utterance-level
normalization of probabilities. The latter typically requires reducing the CD
symbol inventory with state-tying decision trees, which have to be transferred
from classical GMM-HMM systems. We replace the trees with a CD symbol embedding
network, which saves parameters and ensures generalization to unseen and
undersampled CD symbols. The embedding network is trained together with the
rest of the acoustic model and removes one of the last cases in which neural
systems have to be bootstrapped from GMM-HMM ones.
iPhys: An Open Non-Contact Imaging-Based Physiological Measurement Toolbox
Authors: Daniel McDuff, Ethan Blackford
Date: 2019-01-14 00:00:00

Imaging-based, non-contact measurement of physiology (including imaging
photoplethysmography and imaging ballistocardiography) is a growing field of
research. There are several strengths of imaging methods that make them
attractive. They remove the need for uncomfortable contact sensors and can
enable spatial and concomitant measurement from a single sensor. Furthermore,
cameras are ubiquitous and often low-cost solutions for sensing. Open source
toolboxes help accelerate the progress of research by providing a means to
compare new approaches against standard implementations of the
state-of-the-art. We present an open source imaging-based physiological
measurement toolbox with implementations of many of the most frequently
employed computational methods. We hope that this toolbox will contribute to
the advancement of non-contact physiological sensing methods.
A Self-Correcting Deep Learning Approach to Predict Acute Conditions in Critical Care
Authors: Ziyuan Pan, Hao Du, Kee Yuan Ngiam, Fei Wang, Ping Shum, Mengling Feng
Date: 2019-01-14 00:00:00
Tasks: Prediction

In critical care, intensivists are required to continuously monitor high
dimensional vital signs and lab measurements to detect and diagnose acute
patient conditions. This has always been a challenging task. In this study, we
propose a novel self-correcting deep learning prediction approach to address
this challenge. We focus on an example of the prediction of acute kidney injury
(AKI). Compared with the existing models, our method has a number of distinct
features: we utilized the accumulative data of patients in ICU; we developed a
self-correcting mechanism that feeds errors from the previous predictions back
into the network; we also proposed a regularization method that takes into
account not only the model's prediction error on the label but also its
estimation errors on the input data. This mechanism is applied in both
regression and classification tasks. We compared the performance of our
proposed method with the conventional deep learning models on two real-world
clinical datasets and demonstrated that our proposed model constantly
outperforms these baseline models. In particular, the proposed model achieved
area under ROC curve at 0.893 on the MIMIC III dataset, and 0.871 on the
Philips eICU dataset.
A Distributed Synchronous SGD Algorithm with Global Top-$k$ Sparsification for Low Bandwidth Networks
Authors: Shaohuai Shi, Qiang Wang, Kaiyong Zhao, Zhenheng Tang, Yuxin Wang, Xiang Huang, Xiaowen Chu
Date: 2019-01-14 00:00:00
Tasks: GPU

Distributed synchronous stochastic gradient descent (S-SGD) has been widely
used in training large-scale deep neural networks (DNNs), but it typically
requires very high communication bandwidth between computational workers (e.g.,
GPUs) to exchange gradients iteratively. Recently, Top-$k$ sparsification
techniques have been proposed to reduce the volume of data to be exchanged
among workers. Top-$k$ sparsification can zero-out a significant portion of
gradients without impacting the model convergence. However, the sparse
gradients should be transferred with their irregular indices, which makes the
sparse gradients aggregation difficult. Current methods that use AllGather to
accumulate the sparse gradients have a communication complexity of $O(kP)$,
where $P$ is the number of workers, which is inefficient on low bandwidth
networks with a large number of workers. We observe that not all top-$k$
gradients from $P$ workers are needed for the model update, and therefore we
propose a novel global Top-$k$ (gTop-$k$) sparsification mechanism to address
the problem. Specifically, we choose global top-$k$ largest absolute values of
gradients from $P$ workers, instead of accumulating all local top-$k$ gradients
to update the model in each iteration. The gradient aggregation method based on
gTop-$k$ sparsification reduces the communication complexity from $O(kP)$ to
$O(k\log P)$. Through extensive experiments on different DNNs, we verify that
gTop-$k$ S-SGD has nearly consistent convergence performance with S-SGD, and it
has only slight degradations on generalization performance. In terms of scaling
efficiency, we evaluate gTop-$k$ on a cluster with 32 GPU machines which are
interconnected with 1 Gbps Ethernet. The experimental results show that our
method achieves $2.7-12\times$ higher scaling efficiency than S-SGD and
$1.1-1.7\times$ improvement than the existing Top-$k$ S-SGD.
Iterative Deep Learning Based Unbiased Stereology With Human-in-the-Loop
Authors: Saeed S. Alahmari, Dmitry Goldgof, Lawrence O. Hall, Palak Dave, Hady Ahmady Phoulady, Peter R. Mouton
Date: 2019-01-14 00:00:00
Tasks: BIG-bench Machine Learning, Deep Learning, Segmentation

Lack of enough labeled data is a major problem in building machine learning
based models when the manual annotation (labeling) is error-prone, expensive,
tedious, and time-consuming. In this paper, we introduce an iterative deep
learning based method to improve segmentation and counting of cells based on
unbiased stereology applied to regions of interest of extended depth of field
(EDF) images. This method uses an existing machine learning algorithm called
the adaptive segmentation algorithm (ASA) to generate masks (verified by a
user) for EDF images to train deep learning models. Then an iterative deep
learning approach is used to feed newly predicted and accepted deep learning
masks/images (verified by a user) to the training set of the deep learning
model. The error rate in unbiased stereology count of cells on an unseen test
set reduced from about 3 % to less than 1 % after 5 iterations of the iterative
deep learning based unbiased stereology process.
Exploring Transfer Learning for Low Resource Emotional TTS
Authors: Noé Tits, Kevin El Haddad, Thierry Dutoit
Date: 2019-01-14 00:00:00
Proceeding: Advances in Intelligent Systems and Computing 2019 8
Tasks: Deep Learning, Emotional Speech Synthesis, Expressive Speech Synthesis, Speech Synthesis, Text-To-Speech Synthesis, Transfer Learning

During the last few years, spoken language technologies have known a big
improvement thanks to Deep Learning. However Deep Learning-based algorithms
require amounts of data that are often difficult and costly to gather.
Particularly, modeling the variability in speech of different speakers,
different styles or different emotions with few data remains challenging. In
this paper, we investigate how to leverage fine-tuning on a pre-trained Deep
Learning-based TTS model to synthesize speech with a small dataset of another
speaker. Then we investigate the possibility to adapt this model to have
emotional TTS by fine-tuning the neutral TTS model with a small emotional
dataset.
Ordinal Monte Carlo Tree Search
Authors: Tobias Joppen, Johannes Fürnkranz
Date: 2019-01-14 00:00:00

In many problem settings, most notably in game playing, an agent receives a
possibly delayed reward for its actions. Often, those rewards are handcrafted
and not naturally given. Even simple terminal-only rewards, like winning equals
1 and losing equals -1, can not be seen as an unbiased statement, since these
values are chosen arbitrarily, and the behavior of the learner may change with
different encodings, such as setting the value of a loss to -0:5, which is
often done in practice to encourage learning. It is hard to argue about good
rewards and the performance of an agent often depends on the design of the
reward signal. In particular, in domains where states by nature only have an
ordinal ranking and where meaningful distance information between game state
values are not available, a numerical reward signal is necessarily biased. In
this paper, we take a look at Monte Carlo Tree Search (MCTS), a popular
algorithm to solve MDPs, highlight a reoccurring problem concerning its use of
rewards, and show that an ordinal treatment of the rewards overcomes this
problem. Using the General Video Game Playing framework we show a dominance of
our newly proposed ordinal MCTS algorithm over preference-based MCTS, vanilla
MCTS and various other MCTS variants.
Learning Shared Semantic Space with Correlation Alignment for Cross-modal Event Retrieval
Authors: Zhenguo Yang, Zehang Lin, Peipei Kang, Jianming Lv, Qing Li, Wenyin Liu
Date: 2019-01-14 00:00:00
Tasks: Articles, Retrieval

In this paper, we propose to learn shared semantic space with correlation alignment (${S}^{3}CA$) for multimodal data representations, which aligns nonlinear correlations of multimodal data distributions in deep neural networks designed for heterogeneous data. In the context of cross-modal (event) retrieval, we design a neural network with convolutional layers and fully-connected layers to extract features for images, including images on Flickr-like social media. Simultaneously, we exploit a fully-connected neural network to extract semantic features for texts, including news articles from news media. In particular, nonlinear correlations of layer activations in the two neural networks are aligned with correlation alignment during the joint training of the networks. Furthermore, we project the multimodal data into a shared semantic space for cross-modal (event) retrieval, where the distances between heterogeneous data samples can be measured directly. In addition, we contribute a Wiki-Flickr Event dataset, where the multimodal data samples are not describing each other in pairs like the existing paired datasets, but all of them are describing semantic events. Extensive experiments conducted on both paired and unpaired datasets manifest the effectiveness of ${S}^{3}CA$, outperforming the state-of-the-art methods.
DeepFlash: Turning a Flash Selfie into a Studio Portrait
Authors: Nicola Capece, Francesco Banterle, Paolo Cignoni, Fabio Ganovelli, Roberto Scopigno, Ugo Erra
Date: 2019-01-14 00:00:00
Tasks: 3D Depth Estimation

We present a method for turning a flash selfie taken with a smartphone into a photograph as if it was taken in a studio setting with uniform lighting. Our method uses a convolutional neural network trained on a set of pairs of photographs acquired in an ad-hoc acquisition campaign. Each pair consists of one photograph of a subject's face taken with the camera flash enabled and another one of the same subject in the same pose illuminated using a photographic studio-lighting setup. We show how our method can amend defects introduced by a close-up camera flash, such as specular highlights, shadows, skin shine, and flattened images.
NEARBY Platform for Detecting Asteroids in Astronomical Images Using Cloud-based Containerized Applications
Authors: V. Bacu, A. Sabou, T. Stefanut, D. Gorgan, O. Vaduvescu
Date: 2019-01-14 00:00:00

The continuing monitoring and surveying of the nearby space to detect Near
Earth Objects (NEOs) and Near Earth Asteroids (NEAs) are essential because of
the threats that this kind of objects impose on the future of our planet. We
need more computational resources and advanced algorithms to deal with the
exponential growth of the digital cameras' performances and to be able to
process (in near real-time) data coming from large surveys. This paper presents
a software platform called NEARBY that supports automated detection of moving
sources (asteroids) among stars from astronomical images. The detection
procedure is based on the classic "blink" detection and, after that, the system
supports visual analysis techniques to validate the moving sources, assisted by
static and dynamical presentations.
Semi-supervised Learning with Graphs: Covariance Based Superpixels For Hyperspectral Image Classification
Authors: Philip Sellars, Angelica Aviles-Rivero, Nicolas Papadakis, David Coomes, Anita Faul, Carola-Bibane Schönlieb
Date: 2019-01-14 00:00:00
Tasks: Benchmarking, Classification, General Classification, Hyperspectral Image Classification, image-classification, Image Classification, Superpixels

In this paper, we present a graph-based semi-supervised framework for hyperspectral image classification. We first introduce a novel superpixel algorithm based on the spectral covariance matrix representation of pixels to provide a better representation of our data. We then construct a superpixel graph, based on carefully considered feature vectors, before performing classification. We demonstrate, through a set of experimental results using two benchmarking datasets, that our approach outperforms three state-of-the-art classification frameworks, especially when an extremely small amount of labelled data is used.
Albanian Language Identification in Text Documents
Authors: Klesti Hoxha, Artur Baxhaku
Date: 2019-01-14 00:00:00
Tasks: Articles, General Classification, Language Identification

In this work we investigate the accuracy of standard and state-of-the-art
language identification methods in identifying Albanian in written text
documents. A dataset consisting of news articles written in Albanian has been
constructed for this purpose. We noticed a considerable decrease of accuracy
when using test documents that miss the Albanian alphabet letters " \"E " and "
\c{C} " and created a custom training corpus that solved this problem by
achieving an accuracy of more than 99%. Based on our experiments, the most
performing language identification methods for Albanian use a na\"ive Bayes
classifier and n-gram based classification features.
How does Disagreement Help Generalization against Label Corruption?
Authors: Xingrui Yu, Bo Han, Jiangchao Yao, Gang Niu, Ivor W. Tsang, Masashi Sugiyama
Date: 2019-01-14 00:00:00
Tasks: Learning with noisy labels, Memorization, Weakly-supervised Learning

Learning with noisy labels is one of the hottest problems in weakly-supervised learning. Based on memorization effects of deep neural networks, training on small-loss instances becomes very promising for handling noisy labels. This fosters the state-of-the-art approach "Co-teaching" that cross-trains two deep neural networks using the small-loss trick. However, with the increase of epochs, two networks converge to a consensus and Co-teaching reduces to the self-training MentorNet. To tackle this issue, we propose a robust learning paradigm called Co-teaching+, which bridges the "Update by Disagreement" strategy with the original Co-teaching. First, two networks feed forward and predict all data, but keep prediction disagreement data only. Then, among such disagreement data, each network selects its small-loss data, but back propagates the small-loss data from its peer network and updates its own parameters. Empirical results on benchmark datasets demonstrate that Co-teaching+ is much superior to many state-of-the-art methods in the robustness of trained models.
Edge SLAM: Edge Points Based Monocular Visual SLAM
Authors: Soumyadip Maity, Arindam Saha, Brojeshwar Bhowmick
Date: 2019-01-14 00:00:00
Tasks: Optical Flow Estimation, Point Tracking

Visual SLAM shows significant progress in recent years due to high attention
from vision community but still, challenges remain for low-textured
environments. Feature based visual SLAMs do not produce reliable camera and
structure estimates due to insufficient features in a low-textured environment.
Moreover, existing visual SLAMs produce partial reconstruction when the number
of 3D-2D correspondences is insufficient for incremental camera estimation
using bundle adjustment. This paper presents Edge SLAM, a feature based
monocular visual SLAM which mitigates the above mentioned problems. Our
proposed Edge SLAM pipeline detects edge points from images and tracks those
using optical flow for point correspondence. We further refine these point
correspondences using geometrical relationship among three views. Owing to our
edge-point tracking, we use a robust method for two-view initialization for
bundle adjustment. Our proposed SLAM also identifies the potential situations
where estimating a new camera into the existing reconstruction is becoming
unreliable and we adopt a novel method to estimate the new camera reliably
using a local optimization technique. We present an extensive evaluation of our
proposed SLAM pipeline with most popular open datasets and compare with the
state-of-the art. Experimental result indicates that our Edge SLAM is robust
and works reliably well for both textured and less-textured environment in
comparison to existing state-of-the-art SLAMs.
Multi-band Weighted $l_p$ Norm Minimization for Image Denoising
Authors: Yanchi Su, Zhanshan Li, Haihong Yu, Zeyu Wang
Date: 2019-01-14 00:00:00
Tasks: Denoising, Image Denoising

Low rank matrix approximation (LRMA) has drawn increasing attention in recent years, due to its wide range of applications in computer vision and machine learning. However, LRMA, achieved by nuclear norm minimization (NNM), tends to over-shrink the rank components with the same threshold and ignore the differences between rank components. To address this problem, we propose a flexible and precise model named multi-band weighted $l_p$ norm minimization (MBWPNM). The proposed MBWPNM not only gives more accurate approximation with a Schatten $p$-norm, but also considers the prior knowledge where different rank components have different importance. We analyze the solution of MBWPNM and prove that MBWPNM is equivalent to a non-convex $l_p$ norm subproblems under certain weight condition, whose global optimum can be solved by a generalized soft-thresholding algorithm. We then adopt the MBWPNM algorithm to color and multispectral image denoising. Extensive experiments on additive white Gaussian noise removal and realistic noise removal demonstrate that the proposed MBWPNM achieves a better performance than several state-of-art algorithms.
Proceedings of the 2nd Symposium on Problem-solving, Creativity and Spatial Reasoning in Cognitive Systems, ProSocrates 2017
Authors: Ana-Maria Olteteanu, Zoe Falomir
Date: 2019-01-14 00:00:00
Tasks: Spatial Reasoning

This book contains the accepted papers at ProSocrates 2017 Symposium:
Problem-solving,Creativity and Spatial Reasoning in Cognitive Systems.
ProSocrates 2017 symposium was held at the Hansewissenschaftkolleg (HWK) of
Advanced Studies in Delmenhorst, 20-21July 2017. This was the second edition of
this symposium which aims to bring together researchers interested in spatial
reasoning, problem solving and creativity.
Integrating Learning and Reasoning with Deep Logic Models
Authors: Giuseppe Marra, Francesco Giannini, Michelangelo Diligenti, Marco Gori
Date: 2019-01-14 00:00:00
Tasks: Deep Learning

Deep learning is very effective at jointly learning feature representations
and classification models, especially when dealing with high dimensional input
patterns. Probabilistic logic reasoning, on the other hand, is capable to take
consistent and robust decisions in complex environments. The integration of
deep learning and logic reasoning is still an open-research problem and it is
considered to be the key for the development of real intelligent agents. This
paper presents Deep Logic Models, which are deep graphical models integrating
deep learning and logic reasoning both for learning and inference. Deep Logic
Models create an end-to-end differentiable architecture, where deep learners
are embedded into a network implementing a continuous relaxation of the logic
knowledge. The learning process allows to jointly learn the weights of the deep
learners and the meta-parameters controlling the high-level reasoning. The
experimental results show that the proposed methodology overtakes the
limitations of the other approaches that have been proposed to bridge deep
learning and reasoning.
Towards Testing of Deep Learning Systems with Training Set Reduction
Authors: Helge Spieker, Arnaud Gotlieb
Date: 2019-01-14 00:00:00
Tasks: Deep Learning

Testing the implementation of deep learning systems and their training
routines is crucial to maintain a reliable code base. Modern software
development employs processes, such as Continuous Integration, in which changes
to the software are frequently integrated and tested. However, testing the
training routines requires running them and fully training a deep learning
model can be resource-intensive, when using the full data set. Using only a
subset of the training data can improve test run time, but can also reduce its
effectiveness. We evaluate different ways for training set reduction and their
ability to mimic the characteristics of model training with the original full
data set. Our results underline the usefulness of training set reduction,
especially in resource-constrained environments.
Image Based Review Text Generation with Emotional Guidance
Authors: Xuehui Sun, Zihan Zhou, Yuda Fan
Date: 2019-01-14 00:00:00
Tasks: Image Captioning, Text Generation

In the current field of computer vision, automatically generating texts from
given images has been a fully worked technique. Up till now, most works of this
area focus on image content describing, namely image-captioning. However, rare
researches focus on generating product review texts, which is ubiquitous in the
online shopping malls and is crucial for online shopping selection and
evaluation. Different from content describing, review texts include more
subjective information of customers, which may bring difference to the results.
Therefore, we aimed at a new field concerning generating review text from
customers based on images together with the ratings of online shopping
products, which appear as non-image attributes. We made several adjustments to
the existing image-captioning model to fit our task, in which we should also
take non-image features into consideration. We also did experiments based on
our model and get effective primary results.