pwc-archive/papers-with-abstracts huggingface dataset:
Cross-modality (CT-MRI) prior augmented deep learning for robust lung tumor segmentation from small MR datasets
Authors: Jue Jiang, Yu-Chi Hu, Neelam Tyagi, Pengpeng Zhang, Andreas Rimner, Joseph O. Deasy, Harini Veeraraghavan
Date: 2019-01-31 00:00:00
Tasks: Data Augmentation, Deep Learning, Segmentation, Tumor Segmentation

Lack of large expert annotated MR datasets makes training deep learning
models difficult. Therefore, a cross-modality (MR-CT) deep learning
segmentation approach that augments training data using pseudo MR images
produced by transforming expert-segmented CT images was developed. Eighty-One
T2-weighted MRI scans from 28 patients with non-small cell lung cancers were
analyzed. Cross-modality prior encoding the transformation of CT to pseudo MR
images resembling T2w MRI was learned as a generative adversarial deep learning
model. This model augmented training data arising from 6 expert-segmented T2w
MR patient scans with 377 pseudo MRI from non-small cell lung cancer CT patient
scans with obtained from the Cancer Imaging Archive. A two-dimensional Unet
implemented with batch normalization was trained to segment the tumors from T2w
MRI. This method was benchmarked against (a) standard data augmentation and two
state-of-the art cross-modality pseudo MR-based augmentation and (b) two
segmentation networks. Segmentation accuracy was computed using Dice similarity
coefficient (DSC), Hausdroff distance metrics, and volume ratio. The proposed
approach produced the lowest statistical variability in the intensity
distribution between pseudo and T2w MR images measured as Kullback-Leibler
divergence of 0.069. This method produced the highest segmentation accuracy
with a DSC of 0.75 and the lowest Hausdroff distance on the test dataset. This
approach produced highly similar estimations of tumor growth as an expert (P =
0.37). A novel deep learning MR segmentation was developed that overcomes the
limitation of learning robust models from small datasets by leveraging learned
cross-modality priors to augment training. The results show the feasibility of
the approach and the corresponding improvement over the state-of-the-art
methods.
Adding Interpretable Attention to Neural Translation Models Improves Word Alignment
Authors: Thomas Zenkel, Joern Wuebker, John DeNero
Date: 2019-01-31 00:00:00
Tasks: Machine Translation, Translation, Word Alignment

Multi-layer models with multiple attention heads per layer provide superior
translation quality compared to simpler and shallower models, but determining
what source context is most relevant to each target word is more challenging as
a result. Therefore, deriving high-accuracy word alignments from the
activations of a state-of-the-art neural machine translation model is an open
challenge. We propose a simple model extension to the Transformer architecture
that makes use of its hidden representations and is restricted to attend solely
on encoder information to predict the next word. It can be trained on bilingual
data without word-alignment information. We further introduce a novel alignment
inference procedure which applies stochastic gradient descent to directly
optimize the attention activations towards a given target word. The resulting
alignments dramatically outperform the naive approach to interpreting
Transformer attention activations, and are comparable to Giza++ on two publicly
available data sets.
Efficient Relative Pose Estimation for Cameras and Generalized Cameras in Case of Known Relative Rotation Angle
Authors: Evgeniy Martyushev, Bo Li
Date: 2019-01-31 00:00:00
Tasks: Pose Estimation

We propose two minimal solutions to the problem of relative pose estimation
of (i) a calibrated camera from four points in two views and (ii) a calibrated
generalized camera from five points in two views. In both cases, the relative
rotation angle between the views is assumed to be known. In practice, such
angle can be derived from the readings of a 3d gyroscope. We represent the
rotation part of the motion in terms of unit quaternions in order to construct
polynomial equations encoding the epipolar constraints. The Gr\"{o}bner basis
technique is then used to efficiently derive the solutions. Our first solver
for regular cameras significantly improves the existing state-of-the-art
solution. The second solver for generalized cameras is novel.
  The presented minimal solvers can be used in a hypothesize-and-test
architecture such as RANSAC for reliable pose estimation. Experiments on
synthetic and real datasets confirm that our algorithms are numerically stable,
fast and robust.
Functional Regularisation for Continual Learning with Gaussian Processes
Authors: Michalis K. Titsias, Jonathan Schwarz, Alexander G. de G. Matthews, Razvan Pascanu, Yee Whye Teh
Date: 2019-01-31 00:00:00
Proceeding: ICLR 2020 1
Tasks: Bayesian Inference, Continual Learning, Gaussian Processes, Sequential Bayesian Inference

We introduce a framework for Continual Learning (CL) based on Bayesian inference over the function space rather than the parameters of a deep neural network. This method, referred to as functional regularisation for Continual Learning, avoids forgetting a previous task by constructing and memorising an approximate posterior belief over the underlying task-specific function. To achieve this we rely on a Gaussian process obtained by treating the weights of the last layer of a neural network as random and Gaussian distributed. Then, the training algorithm sequentially encounters tasks and constructs posterior beliefs over the task-specific functions by using inducing point sparse Gaussian process methods. At each step a new task is first learnt and then a summary is constructed consisting of (i) inducing inputs -- a fixed-size subset of the task inputs selected such that it optimally represents the task -- and (ii) a posterior distribution over the function values at these inputs. This summary then regularises learning of future tasks, through Kullback-Leibler regularisation terms. Our method thus unites approaches focused on (pseudo-)rehearsal with those derived from a sequential Bayesian inference perspective in a principled way, leading to strong results on accepted benchmarks.
Deep Learning for Inverse Problems: Bounds and Regularizers
Authors: Jaweria Amjad, Zhaoyan Lyu, Miguel R. D. Rodrigues
Date: 2019-01-31 00:00:00
Tasks: Deep Learning, Image Super-Resolution, Super-Resolution

Inverse problems arise in a number of domains such as medical imaging, remote
sensing, and many more, relying on the use of advanced signal and image
processing approaches -- such as sparsity-driven techniques -- to determine
their solution. This paper instead studies the use of deep learning approaches
to approximate the solution of inverse problems. In particular, the paper
provides a new generalization bound, depending on key quantity associated with
a deep neural network -- its Jacobian matrix -- that also leads to a number of
computationally efficient regularization strategies applicable to inverse
problems. The paper also tests the proposed regularization strategies in a
number of inverse problems including image super-resolution ones. Our numerical
results conducted on various datasets show that both fully connected and
convolutional neural networks regularized using the regularization or proxy
regularization strategies originating from our theory exhibit much better
performance than deep networks regularized with standard approaches such as
weight-decay.
Semi-Supervised Ordinal Regression Based on Empirical Risk Minimization
Authors: Taira Tsuchiya, Nontawat Charoenphakdee, Issei Sato, Masashi Sugiyama
Date: 2019-01-31 00:00:00
Tasks: regression

Ordinal regression is aimed at predicting an ordinal class label. In this paper, we consider its semi-supervised formulation, in which we have unlabeled data along with ordinal-labeled data to train an ordinal regressor. There are several metrics to evaluate the performance of ordinal regression, such as the mean absolute error, mean zero-one error, and mean squared error. However, the existing studies do not take the evaluation metric into account, have a restriction on the model choice, and have no theoretical guarantee. To overcome these problems, we propose a novel generic framework for semi-supervised ordinal regression based on the empirical risk minimization principle that is applicable to optimizing all of the metrics mentioned above. Besides, our framework has flexible choices of models, surrogate losses, and optimization algorithms without the common geometric assumption on unlabeled data such as the cluster assumption or manifold assumption. We further provide an estimation error bound to show that our risk estimator is consistent. Finally, we conduct experiments to show the usefulness of our framework.
Learning Efficient Lexically-Constrained Neural Machine Translation with External Memory
Authors: Ya Li, Xinyu Liu, Dan Liu, Xueqiang Zhang, Junhua Liu
Date: 2019-01-31 00:00:00
Tasks: Machine Translation, NMT, Sentence, Translation

Recent years has witnessed dramatic progress of neural machine translation
(NMT), however, the method of manually guiding the translation procedure
remains to be better explored. Previous works proposed to handle such problem
through lexcially-constrained beam search in the decoding phase. Unfortunately,
these lexically-constrained beam search methods suffer two fatal disadvantages:
high computational complexity and hard beam search which generates unexpected
translations. In this paper, we propose to learn the ability of
lexically-constrained translation with external memory, which can overcome the
above mentioned disadvantages. For the training process, automatically
extracted phrase pairs are extracted from alignment and sentence parsing, then
further be encoded into an external memory. This memory is then used to provide
lexically-constrained information for training through a memory-attention
machanism. Various experiments are conducted on WMT Chinese to English and
English to German tasks. All the results can demonstrate the effectiveness of
our method.
Automated brain extraction of multi-sequence MRI using artificial neural networks
Authors: Fabian Isensee, Marianne Schell, Irada Tursunova, Gianluca Brugnara, David Bonekamp, Ulf Neuberger, Antje Wick, Heinz-Peter Schlemmer, Sabine Heiland, Wolfgang Wick, Martin Bendszus, Klaus Hermann Maier-Hein, Philipp Kickingereder
Date: 2019-01-31 00:00:00

Brain extraction is a critical preprocessing step in the analysis of MRI neuroimaging studies and influences the accuracy of downstream analyses. The majority of brain extraction algorithms are, however, optimized for processing healthy brains and thus frequently fail in the presence of pathologically altered brain or when applied to heterogeneous MRI datasets. Here we introduce a new, rigorously validated algorithm (termed HD-BET) relying on artificial neural networks that aims to overcome these limitations. We demonstrate that HD-BET outperforms six popular, publicly available brain extraction algorithms in several large-scale neuroimaging datasets, including one from a prospective multicentric trial in neuro-oncology, yielding state-of-the-art performance with median improvements of +1.16 to +2.11 points for the DICE coefficient and -0.66 to -2.51 mm for the Hausdorff distance. Importantly, the HD-BET algorithm shows robust performance in the presence of pathology or treatment-induced tissue alterations, is applicable to a broad range of MRI sequence types and is not influenced by variations in MRI hardware and acquisition parameters encountered in both research and clinical practice. For broader accessibility our HD-BET prediction algorithm is made freely available (http://www.neuroAI-HD.org) and may become an essential component for robust, automated, high-throughput processing of MRI neuroimaging data.
IMaT: Unsupervised Text Attribute Transfer via Iterative Matching and Translation
Authors: Zhijing Jin, Di Jin, Jonas Mueller, Nicholas Matthews, Enrico Santus
Date: 2019-01-31 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Attribute, Style Transfer, Text Attribute Transfer, Text Style Transfer, Translation

Text attribute transfer aims to automatically rewrite sentences such that they possess certain linguistic attributes, while simultaneously preserving their semantic content. This task remains challenging due to a lack of supervised parallel data. Existing approaches try to explicitly disentangle content and attribute information, but this is difficult and often results in poor content-preservation and ungrammaticality. In contrast, we propose a simpler approach, Iterative Matching and Translation (IMaT), which: (1) constructs a pseudo-parallel corpus by aligning a subset of semantically similar sentences from the source and the target corpora; (2) applies a standard sequence-to-sequence model to learn the attribute transfer; (3) iteratively improves the learned transfer function by refining imperfections in the alignment. In sentiment modification and formality transfer tasks, our method outperforms complex state-of-the-art systems by a large margin. As an auxiliary contribution, we produce a publicly-available test set with human-generated transfer references.
Optimization of the Area Under the ROC Curve using Neural Network Supervectors for Text-Dependent Speaker Verification
Authors: Victoria Mingote, Antonio Miguel, Alfonso Ortega, Eduardo Lleida
Date: 2019-01-31 00:00:00
Tasks: Speaker Verification, Text-Dependent Speaker Verification, Triplet

This paper explores two techniques to improve the performance of
text-dependent speaker verification systems based on deep neural networks.
Firstly, we propose a general alignment mechanism to keep the temporal
structure of each phrase and obtain a supervector with the speaker and phrase
information, since both are relevant for a text-dependent verification. As we
show, it is possible to use different alignment techniques to replace the
global average pooling providing significant gains in performance. Moreover, we
also present a novel back-end approach to train a neural network for detection
tasks by optimizing the Area Under the Curve (AUC) as an alternative to the
usual triplet loss function, so the system is end-to-end, with a cost function
close to our desired measure of performance. As we can see in the experimental
section, this approach improves the system performance, since our triplet
neural network based on an approximation of the AUC (aAUC) learns how to
discriminate between pairs of examples from the same identity and pairs of
different identities. The different alignment techniques to produce
supervectors in addition to the new back-end approach were tested on the
RSR2015-Part I database for text-dependent speaker verification, providing
competitive results compared to similar size networks using the global average
pooling to extract supervectors and using a simple back-end or triplet loss
training.
Generalized Dirichlet-process-means for $f$-separable distortion measures
Authors: Masahiro Kobayashi, Kazuho Watanabe
Date: 2019-01-31 00:00:00
Tasks: Clustering

DP-means clustering was obtained as an extension of $K$-means clustering. While it is implemented with a simple and efficient algorithm, it can estimate the number of clusters simultaneously. However, DP-means is specifically designed for the average distortion measure. Therefore, it is vulnerable to outliers in data, and can cause large maximum distortion in clusters. In this work, we extend the objective function of the DP-means to $f$-separable distortion measures and propose a unified learning algorithm to overcome the above problems by selecting the function $f$. Further, the influence function of the estimated cluster center is analyzed to evaluate the robustness against outliers. We demonstrate the performance of the generalized method by numerical experiments using real datasets.
AnomiGAN: Generative adversarial networks for anonymizing private medical data
Authors: Ho Bae, Dahuin Jung, Sungroh Yoon
Date: 2019-01-31 00:00:00

Typical personal medical data contains sensitive information about
individuals. Storing or sharing the personal medical data is thus often risky.
For example, a short DNA sequence can provide information that can not only
identify an individual, but also his or her relatives. Nonetheless, most
countries and researchers agree on the necessity of collecting personal medical
data. This stems from the fact that medical data, including genomic data, are
an indispensable resource for further research and development regarding
disease prevention and treatment. To prevent personal medical data from being
misused, techniques to reliably preserve sensitive information should be
developed for real world application. In this paper, we propose a framework
called anonymized generative adversarial networks (AnomiGAN), to improve the
maintenance of privacy of personal medical data, while also maintaining high
prediction performance. We compared our method to state-of-the-art techniques
and observed that our method preserves the same level of privacy as
differential privacy (DP), but had better prediction results. We also observed
that there is a trade-off between privacy and performance results depending on
the degree of preservation of the original data. Here, we provide a
mathematical overview of our proposed model and demonstrate its validation
using UCI machine learning repository datasets in order to highlight its
utility in practice. Experimentally, our approach delivers a better performance
compared to that of the DP approach.
New Tricks for Estimating Gradients of Expectations
Authors: Christian J. Walder, Paul Roussel, Richard Nock, Cheng Soon Ong, Masashi Sugiyama
Date: 2019-01-31 00:00:00

We introduce a family of pairwise stochastic gradient estimators for gradients of expectations, which are related to the log-derivative trick, but involve pairwise interactions between samples. The simplest example of our new estimator, dubbed the fundamental trick estimator, is shown to arise from either a) introducing and approximating an integral representation based on the fundamental theorem of calculus, or b) applying the reparameterisation trick to an implicit parameterisation under infinitesimal perturbation of the parameters. From the former perspective we generalise to a reproducing kernel Hilbert space representation, giving rise to a locality parameter in the pairwise interactions mentioned above, yielding our representer trick estimator. The resulting estimators are unbiased and shown to offer an independent component of useful information in comparison with the log-derivative estimator. We provide a further novel theoretical analysis which further characterises the variance reduction afforded by the new techniques. Promising analytical and numerical examples confirm the theory and intuitions behind the new estimators.
Hyperbox based machine learning algorithms: A comprehensive survey
Authors: Thanh Tung Khuat, Dymitr Ruta, Bogdan Gabrys
Date: 2019-01-31 00:00:00
Tasks: BIG-bench Machine Learning, Survey

With the rapid development of digital information, the data volume generated
by humans and machines is growing exponentially. Along with this trend, machine
learning algorithms have been formed and evolved continuously to discover new
information and knowledge from different data sources. Learning algorithms
using hyperboxes as fundamental representational and building blocks are a
branch of machine learning methods. These algorithms have enormous potential
for high scalability and online adaptation of predictors built using hyperbox
data representations to the dynamically changing environments and streaming
data. This paper aims to give a comprehensive survey of literature on
hyperbox-based machine learning models. In general, according to the
architecture and characteristic features of the resulting models, the existing
hyperbox-based learning algorithms may be grouped into three major categories:
fuzzy min-max neural networks, hyperbox-based hybrid models, and other
algorithms based on hyperbox representations. Within each of these groups, this
paper shows a brief description of the structure of models, associated learning
algorithms, and an analysis of their advantages and drawbacks. Main
applications of these hyperbox-based models to the real-world problems are also
described in this paper. Finally, we discuss some open problems and identify
potential future research directions in this field.
Robust Inference via Generative Classifiers for Handling Noisy Labels
Authors: Kimin Lee, Sukmin Yun, Kibok Lee, Honglak Lee, Bo Li, Jinwoo Shin
Date: 2019-01-31 00:00:00

Large-scale datasets may contain significant proportions of noisy (incorrect) class labels, and it is well-known that modern deep neural networks (DNNs) poorly generalize from such noisy training datasets. To mitigate the issue, we propose a novel inference method, termed Robust Generative classifier (RoG), applicable to any discriminative (e.g., softmax) neural classifier pre-trained on noisy datasets. In particular, we induce a generative classifier on top of hidden feature spaces of the pre-trained DNNs, for obtaining a more robust decision boundary. By estimating the parameters of generative classifier using the minimum covariance determinant estimator, we significantly improve the classification accuracy with neither re-training of the deep model nor changing its architectures. With the assumption of Gaussian distribution for features, we prove that RoG generalizes better than baselines under noisy labels. Finally, we propose the ensemble version of RoG to improve its performance by investigating the layer-wise characteristics of DNNs. Our extensive experimental results demonstrate the superiority of RoG given different learning models optimized by several training techniques to handle diverse scenarios of noisy labels.
Distributed Correlation-Based Feature Selection in Spark
Authors: Raul-Jose Palma-Mendoza, Luis de-Marcos, Daniel Rodriguez, Amparo Alonso-Betanzos
Date: 2019-01-31 00:00:00
Tasks: feature selection

CFS (Correlation-Based Feature Selection) is an FS algorithm that has been
successfully applied to classification problems in many domains. We describe
Distributed CFS (DiCFS) as a completely redesigned, scalable, parallel and
distributed version of the CFS algorithm, capable of dealing with the large
volumes of data typical of big data applications. Two versions of the algorithm
were implemented and compared using the Apache Spark cluster computing model,
currently gaining popularity due to its much faster processing times than
Hadoop's MapReduce model. We tested our algorithms on four publicly available
datasets, each consisting of a large number of instances and two also
consisting of a large number of features. The results show that our algorithms
were superior in terms of both time-efficiency and scalability. In leveraging a
computer cluster, they were able to handle larger datasets than the
non-distributed WEKA version while maintaining the quality of the results,
i.e., exactly the same features were returned by our algorithms when compared
to the original algorithm available in WEKA.