pwc-archive/papers-with-abstracts huggingface dataset:
Causality and Bayesian network PDEs for multiscale representations of porous media
Authors: Kimoon Um, Eric Joseph Hall, Markos A. Katsoulakis, Daniel M. Tartakovsky
Date: 2019-01-06 00:00:00
Tasks: Sensitivity

Microscopic (pore-scale) properties of porous media affect and often
determine their macroscopic (continuum- or Darcy-scale) counterparts.
Understanding the relationship between processes on these two scales is
essential to both the derivation of macroscopic models of, e.g., transport
phenomena in natural porous media, and the design of novel materials, e.g., for
energy storage. Most microscopic properties exhibit complex statistical
correlations and geometric constraints, which presents challenges for the
estimation of macroscopic quantities of interest (QoIs), e.g., in the context
of global sensitivity analysis (GSA) of macroscopic QoIs with respect to
microscopic material properties. We present a systematic way of building
correlations into stochastic multiscale models through Bayesian networks. This
allows us to construct the joint probability density function (PDF) of model
parameters through causal relationships that emulate engineering processes,
e.g., the design of hierarchical nanoporous materials. Such PDFs also serve as
input for the forward propagation of parametric uncertainty; our findings
indicate that the inclusion of causal relationships impacts predictions of
macroscopic QoIs. To assess the impact of correlations and causal relationships
between microscopic parameters on macroscopic material properties, we use a
moment-independent GSA based on the differential mutual information. Our GSA
accounts for the correlated inputs and complex non-Gaussian QoIs. The global
sensitivity indices are used to rank the effect of uncertainty in microscopic
parameters on macroscopic QoIs, to quantify the impact of causality on the
multiscale model's predictions, and to provide physical interpretations of
these results for hierarchical nanoporous materials.
Named Entity Recognition in Electronic Health Records Using Transfer Learning Bootstrapped Neural Networks
Authors: Luka Gligic, Andrey Kormilitzin, Paul Goldberg, Alejo Nevado-Holgado
Date: 2019-01-06 00:00:00
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Transfer Learning

Neural networks (NNs) have become the state of the art in many machine learning applications, especially in image and sound processing [1]. The same, although to a lesser extent [2,3], could be said in natural language processing (NLP) tasks, such as named entity recognition. However, the success of NNs remains dependent on the availability of large labelled datasets, which is a significant hurdle in many important applications. One such case are electronic health records (EHRs), which are arguably the largest source of medical data, most of which lies hidden in natural text [4,5]. Data access is difficult due to data privacy concerns, and therefore annotated datasets are scarce. With scarce data, NNs will likely not be able to extract this hidden information with practical accuracy. In our study, we develop an approach that solves these problems for named entity recognition, obtaining 94.6 F1 score in I2B2 2009 Medical Extraction Challenge [6], 4.3 above the architecture that won the competition. Beyond the official I2B2 challenge, we further achieve 82.4 F1 on extracting relationships between medical terms. To reach this state-of-the-art accuracy, our approach applies transfer learning to leverage on datasets annotated for other I2B2 tasks, and designs and trains embeddings that specially benefit from such transfer.
Improving Unsupervised Word-by-Word Translation with Language Model and Denoising Autoencoder
Authors: Yunsu Kim, Jiahui Geng, Hermann Ney
Date: 2019-01-06 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Denoising, Language Modeling, Language Modelling, Translation, Word Translation

Unsupervised learning of cross-lingual word embedding offers elegant matching
of words across languages, but has fundamental limitations in translating
sentences. In this paper, we propose simple yet effective methods to improve
word-by-word translation of cross-lingual embeddings, using only monolingual
corpora but without any back-translation. We integrate a language model for
context-aware search, and use a novel denoising autoencoder to handle
reordering. Our system surpasses state-of-the-art unsupervised neural
translation systems without costly iterative training. We also analyze the
effect of vocabulary size and denoising type on the translation performance,
which provides better understanding of learning the cross-lingual word
embedding and its usage in translation.
PyOD: A Python Toolbox for Scalable Outlier Detection
Authors: Yue Zhao, Zain Nasrullah, Zheng Li
Date: 2019-01-06 00:00:00
Tasks: Anomaly Detection, Outlier Detection, outlier ensembles

PyOD is an open-source Python toolbox for performing scalable outlier detection on multivariate data. Uniquely, it provides access to a wide range of outlier detection algorithms, including established outlier ensembles and more recent neural network-based approaches, under a single, well-documented API designed for use by both practitioners and researchers. With robustness and scalability in mind, best practices such as unit testing, continuous integration, code coverage, maintainability checks, interactive examples and parallelization are emphasized as core components in the toolbox's development. PyOD is compatible with both Python 2 and 3 and can be installed through Python Package Index (PyPI) or https://github.com/yzhao062/pyod.
Solving L1-regularized SVMs and related linear programs: Revisiting the effectiveness of Column and Constraint Generation
Authors: Antoine Dedieu, Rahul Mazumder, Haoyue Wang
Date: 2019-01-06 00:00:00
Tasks: BIG-bench Machine Learning

The linear Support Vector Machine (SVM) is a classic classification technique in machine learning. Motivated by applications in modern high dimensional statistics, we consider penalized SVM problems involving the minimization of a hinge-loss function with a convex sparsity-inducing regularizer such as: the L1-norm on the coefficients, its grouped generalization and the sorted L1-penalty (aka Slope). Each problem can be expressed as a Linear Program (LP) and is computationally challenging when the number of features and/or samples is large -- the current state of algorithms for these problems is rather nascent when compared to the usual L2-regularized linear SVM. To this end, we propose new computational algorithms for these LPs by bringing together techniques from (a) classical column (and constraint) generation methods and (b) first order methods for non-smooth convex optimization -- techniques that are rarely used together for solving large scale LPs. These components have their respective strengths; and while they are found to be useful as separate entities, they have not been used together in the context of solving large scale LPs such as the ones studied herein. Our approach complements the strengths of (a) and (b) -- leading to a scheme that seems to significantly outperform commercial solvers as well as specialized implementations for these problems. We present numerical results on a series of real and synthetic datasets demonstrating the surprising effectiveness of classic column/constraint generation methods in the context of challenging LP-based machine learning tasks.
CC-Net: Image Complexity Guided Network Compression for Biomedical Image Segmentation
Authors: Suraj Mishra, Peixian Liang, Adam Czajka, Danny Z. Chen, X. Sharon Hu
Date: 2019-01-06 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

Convolutional neural networks (CNNs) for biomedical image analysis are often of very large size, resulting in high memory requirement and high latency of operations. Searching for an acceptable compressed representation of the base CNN for a specific imaging application typically involves a series of time-consuming training/validation experiments to achieve a good compromise between network size and accuracy. To address this challenge, we propose CC-Net, a new image complexity-guided CNN compression scheme for biomedical image segmentation. Given a CNN model, CC-Net predicts the final accuracy of networks of different sizes based on the average image complexity computed from the training data. It then selects a multiplicative factor for producing a desired network with acceptable network accuracy and size. Experiments show that CC-Net is effective for generating compressed segmentation networks, retaining up to 95% of the base network segmentation accuracy and utilizing only 0.1% of trainable parameters of the full-sized networks in the best case.
Unsupervised Training for Large Vocabulary Translation Using Sparse Lexicon and Word Classes
Authors: Yunsu Kim, Julian Schamper, Hermann Ney
Date: 2019-01-06 00:00:00
Proceeding: EACL 2017 4
Tasks: Translation

We address for the first time unsupervised training for a translation task
with hundreds of thousands of vocabulary words. We scale up the
expectation-maximization (EM) algorithm to learn a large translation table
without any parallel text or seed lexicon. First, we solve the memory
bottleneck and enforce the sparsity with a simple thresholding scheme for the
lexicon. Second, we initialize the lexicon training with word classes, which
efficiently boosts the performance. Our methods produced promising results on
two large-scale unsupervised translation tasks.
Learning-Free Iris Segmentation Revisited: A First Step Toward Fast Volumetric Operation Over Video Samples
Authors: Jeffery Kinnison, Mateusz Trokielewicz, Camila Carballo, Adam Czajka, Walter Scheirer
Date: 2019-01-06 00:00:00
Tasks: Iris Segmentation, Segmentation

Subject matching performance in iris biometrics is contingent upon fast,
high-quality iris segmentation. In many cases, iris biometrics acquisition
equipment takes a number of images in sequence and combines the segmentation
and matching results for each image to strengthen the result. To date,
segmentation has occurred in 2D, operating on each image individually. But such
methodologies, while powerful, do not take advantage of potential gains in
performance afforded by treating sequential images as volumetric data. As a
first step in this direction, we apply the Flexible Learning-Free
Reconstructoin of Neural Volumes (FLoRIN) framework, an open source
segmentation and reconstruction framework originally designed for neural
microscopy volumes, to volumetric segmentation of iris videos. Further, we
introduce a novel dataset of near-infrared iris videos, in which each subject's
pupil rapidly changes size due to visible-light stimuli, as a test bed for
FLoRIN. We compare the matching performance for iris masks generated by FLoRIN,
deep-learning-based (SegNet), and Daugman's (OSIRIS) iris segmentation
approaches. We show that by incorporating volumetric information, FLoRIN
achieves a factor of 3.6 to an order of magnitude increase in throughput with
only a minor drop in subject matching performance. We also demonstrate that
FLoRIN-based iris segmentation maintains this speedup on low-resource hardware,
making it suitable for embedded biometrics systems.
A Comparative Study on Vocabulary Reduction for Phrase Table Smoothing
Authors: Yunsu Kim, Andreas Guta, Joern Wuebker, Hermann Ney
Date: 2019-01-06 00:00:00
Proceeding: WS 2016 8
Tasks: Translation

This work systematically analyzes the smoothing effect of vocabulary
reduction for phrase translation models. We extensively compare various
word-level vocabularies to show that the performance of smoothing is not
significantly affected by the choice of vocabulary. This result provides
empirical evidence that the standard phrase translation model is extremely
sparse. Our experiments also reveal that vocabulary reduction is more effective
for smoothing large-scale phrase tables.
Transductive Zero-Shot Learning with Visual Structure Constraint
Authors: Zi-Yu Wan, Dong-Dong Chen, Yan Li, Xingguang Yan, Junge Zhang, Yizhou Yu, Jing Liao
Date: 2019-01-06 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Zero-Shot Learning

To recognize objects of the unseen classes, most existing Zero-Shot Learning(ZSL) methods first learn a compatible projection function between the common semantic space and the visual space based on the data of source seen classes, then directly apply it to the target unseen classes. However, in real scenarios, the data distribution between the source and target domain might not match well, thus causing the well-known \textbf{domain shift} problem. Based on the observation that visual features of test instances can be separated into different clusters, we propose a new visual structure constraint on class centers for transductive ZSL, to improve the generality of the projection function (i.e. alleviate the above domain shift problem). Specifically, three different strategies (symmetric Chamfer-distance, Bipartite matching distance, and Wasserstein distance) are adopted to align the projected unseen semantic centers and visual cluster centers of test instances. We also propose a new training strategy to handle the real cases where many unrelated images exist in the test dataset, which is not considered in previous methods. Experiments on many widely used datasets demonstrate that the proposed visual structure constraint can bring substantial performance gain consistently and achieve state-of-the-art results. The source code is available at \url{https://github.com/raywzy/VSC}.
Segmentation Guided Image-to-Image Translation with Adversarial Networks
Authors: Songyao Jiang, Zhiqiang Tao, Yun Fu
Date: 2019-01-06 00:00:00
Tasks: Image-to-Image Translation, Segmentation, Semantic Segmentation, Translation

Recently image-to-image translation has received increasing attention, which
aims to map images in one domain to another specific one. Existing methods
mainly solve this task via a deep generative model, and focus on exploring the
relationship between different domains. However, these methods neglect to
utilize higher-level and instance-specific information to guide the training
process, leading to a great deal of unrealistic generated images of low
quality. Existing methods also lack of spatial controllability during
translation. To address these challenge, we propose a novel Segmentation Guided
Generative Adversarial Networks (SGGAN), which leverages semantic segmentation
to further boost the generation performance and provide spatial mapping. In
particular, a segmentor network is designed to impose semantic information on
the generated images. Experimental results on multi-domain face image
translation task empirically demonstrate our ability of the spatial
modification and our superiority in image quality over several state-of-the-art
methods.
Learning Nonlinear Mixtures: Identifiability and Algorithm
Authors: Bo Yang, Xiao Fu, Nicholas D. Sidiropoulos, Kejun Huang
Date: 2019-01-06 00:00:00
Tasks: Clustering

Linear mixture models have proven very useful in a plethora of applications,
e.g., topic modeling, clustering, and source separation. As a critical aspect
of the linear mixture models, identifiability of the model parameters is
well-studied, under frameworks such as independent component analysis and
constrained matrix factorization. Nevertheless, when the linear mixtures are
distorted by an unknown nonlinear functions -- which is well-motivated and more
realistic in many cases -- the identifiability issues are much less studied.
This work proposes an identification criterion for a nonlinear mixture model
that is well grounded in many real-world applications, and offers
identifiability guarantees. A practical implementation based on a judiciously
designed neural network is proposed to realize the criterion, and an effective
learning algorithm is proposed. Numerical results on synthetic and real-data
corroborate effectiveness of the proposed method.
Automated Multiscale 3D Feature Learning for Vessels Segmentation in Thorax CT Images
Authors: Tomasz Konopczyński, Thorben Kröger, Lei Zheng, Christoph S. Garbe, Jürgen Hesser
Date: 2019-01-06 00:00:00
Tasks: Dictionary Learning

We address the vessel segmentation problem by building upon the multiscale
feature learning method of Kiros et al., which achieves the current top score
in the VESSEL12 MICCAI challenge. Following their idea of feature learning
instead of hand-crafted filters, we have extended the method to learn 3D
features. The features are learned in an unsupervised manner in a multi-scale
scheme using dictionary learning via least angle regression. The 3D feature
kernels are further convolved with the input volumes in order to create feature
maps. Those maps are used to train a supervised classifier with the annotated
voxels. In order to process the 3D data with a large number of filters a
parallel implementation has been developed. The algorithm has been applied on
the example scans and annotations provided by the VESSEL12 challenge. We have
compared our setup with Kiros et al. by running their implementation. Our
current results show an improvement in accuracy over the slice wise method from
96.66$\pm$1.10% to 97.24$\pm$0.90%.
Self-Expressive Subspace Clustering to Recognize Motion Dynamics of a Multi-Joint Coordination for Chronic Ankle Instability
Authors: Shaodi Qian, Sheng-Che Yen, Eric Folmar, Chun-An Chou
Date: 2019-01-06 00:00:00
Tasks: Clustering

Ankle sprains and instability are major public health concerns. Up to 70% of individuals do not fully recover from a single ankle sprain and eventually develop chronic ankle instability (CAI). The diagnosis of CAI has been mainly based on self-report rather than objective biomechanical measures. The goal of this study is to quantitatively recognize the motion pattern of a multi-joint coordination using biosensor data from bilateral hip, knee, and ankle joints, and further distinguish between CAI and healthy cohorts. We propose an analytic framework, where a nonlinear subspace clustering method is developed to learn the motion dynamic patterns from an inter-connected network of multiply joints. A support vector machine model is trained with a leave-one-subject-out cross validation to validate the learned measures compared to traditional statistical measures. The computational results showed >70% classification accuracy on average based on the dataset of 48 subjects (25 with CAI and 23 normal controls) examined in our designed experiment. It is found that CAI can be observed from other joints (e.g., hips) significantly, which reflects the fact that there are interactions in the multi-joint coordination system. The developed method presents a potential to support the decisions with motion patterns during diagnosis, treatment, rehabilitation of gait abnormality caused by physical injury (e.g., ankle sprains in this study) or even central nervous system disorders.
Unsupervised uncertainty estimation using spatiotemporal cues in video saliency detection
Authors: Tariq Alshawi, Zhiling Long, Ghassan AlRegib
Date: 2019-01-06 00:00:00
Tasks: Saliency Detection, Video Saliency Detection

In this paper, we address the problem of quantifying reliability of
computational saliency for videos, which can be used to improve saliency-based
video processing and enable more reliable performance and risk assessment of
such processing. Our approach is twofold. First, we explore spatial
correlations in both saliency map and eye-fixation map. Then, we learn
spatiotemporal correlations that define a reliable saliency map. We first study
spatiotemporal eye-fixation data from a public dataset and investigate a common
feature in human visual attention, which dictates correlation in saliency
between a pixel and its direct neighbors. Based on the study, we then develop
an algorithm that estimates a pixel-wise uncertainty map that reflects our
confidence in the associated computational saliency map by relating a pixel's
saliency to the saliency of its neighbors. To estimate such uncertainties, we
measure the divergence of a pixel, in a saliency map, from its local
neighborhood. Additionally, we propose a systematic procedure to evaluate the
estimation performance by explicitly computing uncertainty ground truth as a
function of a given saliency map and eye fixations of human subjects. In our
experiments, we explore multiple definitions of locality and neighborhoods in
spatiotemporal video signals. In addition, we examine the relationship between
the parameters of our proposed algorithm and the content of the videos. The
proposed algorithm is unsupervised, making it more suitable for generalization
to most natural videos. Also, it is computationally efficient and flexible for
customization to specific video content. Experiments using three publicly
available video datasets show that the proposed algorithm outperforms
state-of-the-art uncertainty estimation methods with improvement in accuracy up
to 63% and offers efficiency and flexibility that make it more useful in
practical situations.
An online supervised learning algorithm based on triple spikes for spiking neural networks
Authors: Guojun Chen, Xianghong Lin, Guoen Wang
Date: 2019-01-06 00:00:00

Using precise times of every spike, spiking supervised learning has more
effects on complex spatial-temporal pattern than supervised learning only
through neuronal firing rates. The purpose of spiking supervised learning after
spatial-temporal encoding is to emit desired spike trains with precise times.
Existing algorithms of spiking supervised learning have excellent performances,
but mechanisms of them still have some problems, such as the limitation of
neuronal types and complex computation. Based on an online regulative mechanism
of biological synapses, this paper proposes an online supervised learning
algorithm of multiple spike trains for spiking neural networks. The proposed
algorithm with a spatial-temporal transformation can make a simple direct
regulation of synaptic weights as soon as firing time of an output spike is
obtained. Besides, it is also not restricted by types of spiking neuron models.
Relationship among desired output, actual output and input spike trains is
firstly analyzed and synthesized to simply select a unit of pair-spike for a
direct regulation. And then a computational method is constructed based on
simple triple spikes using this direct regulation. Compared with other learning
algorithms, results of experiments show that proposed algorithm has higher
learning accuracy and efficiency.
Bandwidth Reduction using Importance Weighted Pruning on Ring AllReduce
Authors: Zehua Cheng, Zhenghua Xu
Date: 2019-01-06 00:00:00

It is inevitable to train large deep learning models on a large-scale cluster
equipped with accelerators system. Deep gradient compression would highly
increase the bandwidth utilization and speed up the training process but hard
to implement on ring structure. In this paper, we find that redundant gradient
and gradient staleness has negative effect on training. We have observed that
in different epoch and different steps, the neural networks focus on updating
different layers and different parameters. In order to save more communication
bandwidth and preserve the accuracy on ring structure, which break the restrict
as the node increase, we propose a new algorithm to measure the importance of
gradients on large-scale cluster implementing ring all-reduce based on the size
of the ratio of parameter calculation gradient to parameter value. Our
importance weighted pruning approach achieved 64X and 58.8X of gradient
compression ratio on AlexNet and ResNet50 on ImageNet. Meanwhile, in order to
maintain the sparseness of the gradient propagation, we randomly broadcast the
index of important gradients on each node. While the remaining nodes are ready
for the index gradient and perform all-reduce update. This would speed up the
convergence of the model and preserve the training accuracy.