pwc-archive/papers-with-abstracts huggingface dataset:
Joint Weakly and Semi-Supervised Deep Learning for Localization and Classification of Masses in Breast Ultrasound Images
Authors: Seung Yeon Shin, Soochahn Lee, Il Dong Yun, Sun Mi Kim, Kyoung Mu Lee
Date: 2017-10-10 00:00:00
Tasks: General Classification

We propose a framework for localization and classification of masses in
breast ultrasound (BUS) images. We have experimentally found that training
convolutional neural network based mass detectors with large, weakly annotated
datasets presents a non-trivial problem, while overfitting may occur with those
trained with small, strongly annotated datasets. To overcome these problems, we
use a weakly annotated dataset together with a smaller strongly annotated
dataset in a hybrid manner. We propose a systematic weakly and semi-supervised
training scenario with appropriate training loss selection. Experimental
results show that the proposed method can successfully localize and classify
masses with less annotation effort. The results trained with only 10 strongly
annotated images along with weakly annotated images were comparable to results
trained from 800 strongly annotated images, with the 95% confidence interval of
difference -3.00%--5.00%, in terms of the correct localization (CorLoc)
measure, which is the ratio of images with intersection over union with ground
truth higher than 0.5. With the same number of strongly annotated images,
additional weakly annotated images can be incorporated to give a 4.5% point
increase in CorLoc, from 80.00% to 84.50% (with 95% confidence intervals
76.00%--83.75% and 81.00%--88.00%). The effects of different algorithmic
details and varied amount of data are presented through ablative analysis.
Gaussian Processes for Data-Efficient Learning in Robotics and Control
Authors: Marc Peter Deisenroth, Dieter Fox, Carl Edward Rasmussen
Date: 2015-02-10 00:00:00
Tasks: Gaussian Processes, Reinforcement Learning, Reinforcement Learning (RL)

Autonomous learning has been a promising direction in control and robotics
for more than a decade since data-driven learning allows to reduce the amount
of engineering knowledge, which is otherwise required. However, autonomous
reinforcement learning (RL) approaches typically require many interactions with
the system to learn controllers, which is a practical limitation in real
systems, such as robots, where many interactions can be impractical and time
consuming. To address this problem, current learning approaches typically
require task-specific knowledge in form of expert demonstrations, realistic
simulators, pre-shaped policies, or specific knowledge about the underlying
dynamics. In this article, we follow a different approach and speed up learning
by extracting more information from data. In particular, we learn a
probabilistic, non-parametric Gaussian process transition model of the system.
By explicitly incorporating model uncertainty into long-term planning and
controller learning our approach reduces the effects of model errors, a key
problem in model-based learning. Compared to state-of-the art RL our
model-based policy search method achieves an unprecedented speed of learning.
We demonstrate its applicability to autonomous learning in real robot and
control tasks.
DeepLesion: Automated Deep Mining, Categorization and Detection of Significant Radiology Image Findings using Large-Scale Clinical Lesion Annotations
Authors: Ke Yan, Xiaosong Wang, Le Lu, Ronald M. Summers
Date: 2017-10-04 00:00:00
Tasks: Lesion Segmentation, Medical Image Analysis

Extracting, harvesting and building large-scale annotated radiological image
datasets is a greatly important yet challenging problem. It is also the
bottleneck to designing more effective data-hungry computing paradigms (e.g.,
deep learning) for medical image analysis. Yet, vast amounts of clinical
annotations (usually associated with disease image findings and marked using
arrows, lines, lesion diameters, segmentation, etc.) have been collected over
several decades and stored in hospitals' Picture Archiving and Communication
Systems. In this paper, we mine and harvest one major type of clinical
annotation data - lesion diameters annotated on bookmarked images - to learn an
effective multi-class lesion detector via unsupervised and supervised deep
Convolutional Neural Networks (CNN). Our dataset is composed of 33,688
bookmarked radiology images from 10,825 studies of 4,477 unique patients. For
every bookmarked image, a bounding box is created to cover the target lesion
based on its measured diameters. We categorize the collection of lesions using
an unsupervised deep mining scheme to generate clustered pseudo lesion labels.
Next, we adopt a regional-CNN method to detect lesions of multiple categories,
regardless of missing annotations (normally only one lesion is annotated,
despite the presence of multiple co-existing findings). Our integrated mining,
categorization and detection framework is validated with promising empirical
results, as a scalable, universal or multi-purpose CAD paradigm built upon
abundant retrospective medical data. Furthermore, we demonstrate that detection
accuracy can be significantly improved by incorporating pseudo lesion labels
(e.g., Liver lesion/tumor, Lung nodule/tumor, Abdomen lesions, Chest lymph node
and others). This dataset will be made publicly available (under the open
science initiative).
Confidence through Attention
Authors: Matīss Rikters, Mark Fishel
Date: 2017-10-10 00:00:00
Proceeding: MTSummit 2017 9
Tasks: Machine Translation, Translation

Attention distributions of the generated translations are a useful bi-product
of attention-based recurrent neural network translation models and can be
treated as soft alignments between the input and output tokens. In this work,
we use attention distributions as a confidence metric for output translations.
We present two strategies of using the attention distributions: filtering out
bad translations from a large back-translated corpus, and selecting the best
translation in a hybrid setup of two different translation systems. While
manual evaluation indicated only a weak correlation between our confidence
score and human judgments, the use-cases showed improvements of up to 2.22 BLEU
points for filtering and 0.99 points for hybrid translation, tested on
English<->German and English<->Latvian translation.
Fast and Safe: Accelerated gradient methods with optimality certificates and underestimate sequences
Authors: Majid Jahani, Naga Venkata C. Gudapati, Chenxin Ma, Rachael Tappenden, Martin Takáč
Date: 2017-10-10 00:00:00

In this work we introduce the concept of an Underestimate Sequence (UES), which is motivated by Nesterov's estimate sequence. Our definition of a UES utilizes three sequences, one of which is a lower bound (or under-estimator) of the objective function. The question of how to construct an appropriate sequence of lower bounds is addressed, and we present lower bounds for strongly convex smooth functions and for strongly convex composite functions, which adhere to the UES framework. Further, we propose several first order methods for minimizing strongly convex functions in both the smooth and composite cases. The algorithms, based on efficiently updating lower bounds on the objective functions, have natural stopping conditions that provide the user with a certificate of optimality. Convergence of all algorithms is guaranteed through the UES framework, and we show that all presented algorithms converge linearly, with the accelerated variants enjoying the optimal linear rate of convergence.
High-dimensional dynamics of generalization error in neural networks
Authors: Madhu S. Advani, Andrew M. Saxe
Date: 2017-10-10 00:00:00
Tasks: Vocal Bursts Intensity Prediction

We perform an average case analysis of the generalization dynamics of large
neural networks trained using gradient descent. We study the
practically-relevant "high-dimensional" regime where the number of free
parameters in the network is on the order of or even larger than the number of
examples in the dataset. Using random matrix theory and exact solutions in
linear models, we derive the generalization error and training error dynamics
of learning and analyze how they depend on the dimensionality of data and
signal to noise ratio of the learning problem. We find that the dynamics of
gradient descent learning naturally protect against overtraining and
overfitting in large networks. Overtraining is worst at intermediate network
sizes, when the effective number of free parameters equals the number of
samples, and thus can be reduced by making a network smaller or larger.
Additionally, in the high-dimensional regime, low generalization error requires
starting with small initial weights. We then turn to non-linear neural
networks, and show that making networks very large does not harm their
generalization performance. On the contrary, it can in fact reduce
overtraining, even without early stopping or regularization of any sort. We
identify two novel phenomena underlying this behavior in overcomplete models:
first, there is a frozen subspace of the weights in which no learning occurs
under gradient descent; and second, the statistical properties of the
high-dimensional regime yield better-conditioned input correlations which
protect against overtraining. We demonstrate that naive application of
worst-case theories such as Rademacher complexity are inaccurate in predicting
the generalization performance of deep neural networks, and derive an
alternative bound which incorporates the frozen subspace and conditioning
effects and qualitatively matches the behavior observed in simulation.
Investigating the Parameter Space of Evolutionary Algorithms
Authors: Moshe Sipper, Weixuan Fu, Karuna Ahuja, Jason H. Moore
Date: 2017-06-13 00:00:00
Tasks: Evolutionary Algorithms

The practice of evolutionary algorithms involves the tuning of many
parameters. How big should the population be? How many generations should the
algorithm run? What is the (tournament selection) tournament size? What
probabilities should one assign to crossover and mutation? Through an extensive
series of experiments over multiple evolutionary algorithm implementations and
problems we show that parameter space tends to be rife with viable parameters,
at least for 25 the problems studied herein. We discuss the implications of
this finding in practice.
LinXGBoost: Extension of XGBoost to Generalized Local Linear Models
Authors: Laurent de Vito
Date: 2017-10-10 00:00:00
Tasks: regression

XGBoost is often presented as the algorithm that wins every ML competition.
Surprisingly, this is true even though predictions are piecewise constant. This
might be justified in high dimensional input spaces, but when the number of
features is low, a piecewise linear model is likely to perform better. XGBoost
was extended into LinXGBoost that stores at each leaf a linear model. This
extension, equivalent to piecewise regularized least-squares, is particularly
attractive for regression of functions that exhibits jumps or discontinuities.
Those functions are notoriously hard to regress. Our extension is compared to
the vanilla XGBoost and Random Forest in experiments on both synthetic and
real-world data sets.
Multilevel Modeling with Structured Penalties for Classification from Imaging Genetics data
Authors: Pascal Lu, Olivier Colliot
Date: 2017-10-10 00:00:00
Tasks: Additive models, General Classification

In this paper, we propose a framework for automatic classification of
patients from multimodal genetic and brain imaging data by optimally combining
them. Additive models with unadapted penalties (such as the classical group
lasso penalty or $L_1$-multiple kernel learning) treat all modalities in the
same manner and can result in undesirable elimination of specific modalities
when their contributions are unbalanced. To overcome this limitation, we
introduce a multilevel model that combines imaging and genetics and that
considers joint effects between these two modalities for diagnosis prediction.
Furthermore, we propose a framework allowing to combine several penalties
taking into account the structure of the different types of data, such as a
group lasso penalty over the genetic modality and a $L_2$-penalty on imaging
modalities. Finally , we propose a fast optimization algorithm, based on a
proximal gradient method. The model has been evaluated on genetic (single
nucleotide polymorphisms-SNP) and imaging (anatomical MRI measures) data from
the ADNI database, and compared to additive models. It exhibits good
performances in AD diagnosis; and at the same time, reveals relationships
between genes, brain regions and the disease status.
Optimizing Long Short-Term Memory Recurrent Neural Networks Using Ant Colony Optimization to Predict Turbine Engine Vibration
Authors: AbdElRahman ElSaid, Travis Desell, Fatima El Jamiy, James Higgins, Brandon Wild
Date: 2017-10-10 00:00:00

This article expands on research that has been done to develop a recurrent
neural network (RNN) capable of predicting aircraft engine vibrations using
long short-term memory (LSTM) neurons. LSTM RNNs can provide a more
generalizable and robust method for prediction over analytical calculations of
engine vibration, as analytical calculations must be solved iteratively based
on specific empirical engine parameters, making this approach ungeneralizable
across multiple engines. In initial work, multiple LSTM RNN architectures were
proposed, evaluated and compared. This research improves the performance of the
most effective LSTM network design proposed in the previous work by using a
promising neuroevolution method based on ant colony optimization (ACO) to
develop and enhance the LSTM cell structure of the network. A parallelized
version of the ACO neuroevolution algorithm has been developed and the evolved
LSTM RNNs were compared to the previously used fixed topology. The evolved
networks were trained on a large database of flight data records obtained from
an airline containing flights that suffered from excessive vibration. Results
were obtained using MPI (Message Passing Interface) on a high performance
computing (HPC) cluster, evolving 1000 different LSTM cell structures using 168
cores over 4 days. The new evolved LSTM cells showed an improvement of 1.35%,
reducing prediction error from 5.51% to 4.17% when predicting excessive engine
vibrations 10 seconds in the future, while at the same time dramatically
reducing the number of weights from 21,170 to 11,810.
Machine Translation Evaluation Resources and Methods: A Survey
Authors: Lifeng Han
Date: 2016-05-15 00:00:00
Tasks: Informativeness, Machine Translation, Natural Language Inference, Sentence, Survey, TAG, Translation

We introduce the Machine Translation (MT) evaluation survey that contains
both manual and automatic evaluation methods. The traditional human evaluation
criteria mainly include the intelligibility, fidelity, fluency, adequacy,
comprehension, and informativeness. The advanced human assessments include
task-oriented measures, post-editing, segment ranking, and extended criteriea,
etc. We classify the automatic evaluation methods into two categories,
including lexical similarity scenario and linguistic features application. The
lexical similarity methods contain edit distance, precision, recall, F-measure,
and word order. The linguistic features can be divided into syntactic features
and semantic features respectively. The syntactic features include part of
speech tag, phrase types and sentence structures, and the semantic features
include named entity, synonyms, textual entailment, paraphrase, semantic roles,
and language models. The deep learning models for evaluation are very newly
proposed. Subsequently, we also introduce the evaluation methods for MT
evaluation including different correlation scores, and the recent quality
estimation (QE) tasks for MT.
  This paper differs from the existing works
\cite{GALEprogram2009,EuroMatrixProject2007} from several aspects, by
introducing some recent development of MT evaluation measures, the different
classifications from manual to automatic evaluation measures, the introduction
of recent QE tasks of MT, and the concise construction of the content.
  We hope this work will be helpful for MT researchers to easily pick up some
metrics that are best suitable for their specific MT model development, and
help MT evaluation researchers to get a general clue of how MT evaluation
research developed. Furthermore, hopefully, this work can also shine some light
on other evaluation tasks, except for translation, of NLP fields.
Fast and Strong Convergence of Online Learning Algorithms
Authors: Zheng-Chu Guo, Lei Shi
Date: 2017-10-10 00:00:00

In this paper, we study the online learning algorithm without explicit
regularization terms. This algorithm is essentially a stochastic gradient
descent scheme in a reproducing kernel Hilbert space (RKHS). The polynomially
decaying step size in each iteration can play a role of regularization to
ensure the generalization ability of online learning algorithm. We develop a
novel capacity dependent analysis on the performance of the last iterate of
online learning algorithm. The contribution of this paper is two-fold. First,
our nice analysis can lead to the convergence rate in the standard mean square
distance which is the best so far. Second, we establish, for the first time,
the strong convergence of the last iterate with polynomially decaying step
sizes in the RKHS norm. We demonstrate that the theoretical analysis
established in this paper fully exploits the fine structure of the underlying
RKHS, and thus can lead to sharp error estimates of online learning algorithm.
Traffic Sign Timely Visual Recognizability Evaluation Based on 3D Measurable Point Clouds
Authors: Shanxin Zhang, Cheng Wang, Zhuang Yang, Chenglu Wen, Jonathan Li, Chenhui Yang
Date: 2017-10-10 00:00:00
Tasks: Point Cloud Segmentation

The timely provision of traffic sign information to drivers is essential for
the drivers to respond, to ensure safe driving, and to avoid traffic accidents
in a timely manner. We proposed a timely visual recognizability quantitative
evaluation method for traffic signs in large-scale transportation environments.
To achieve this goal, we first address the concept of a visibility field to
reflect the visible distribution of three-dimensional (3D) space and construct
a traffic sign Visibility Evaluation Model (VEM) to measure the traffic sign
visibility for a given viewpoint. Then, based on the VEM, we proposed the
concept of the Visual Recognizability Field (VRF) to reflect the visual
recognizability distribution in 3D space and established a Visual
Recognizability Evaluation Model (VREM) to measure a traffic sign visual
recognizability for a given viewpoint. Next, we proposed a Traffic Sign Timely
Visual Recognizability Evaluation Model (TSTVREM) by combining VREM, the actual
maximum continuous visual recognizable distance, and traffic big data to
measure a traffic sign visual recognizability in different lanes. Finally, we
presented an automatic algorithm to implement the TSTVREM model through traffic
sign and road marking detection and classification, traffic sign environment
point cloud segmentation, viewpoints calculation, and TSTVREM model
realization. The performance of our method for traffic sign timely visual
recognizability evaluation is tested on three road point clouds acquired by a
mobile laser scanning system (RIEGL VMX-450) according to Road Traffic Signs
and Markings (GB 5768-1999 in China), showing that our method is feasible and
efficient.
Contaminated speech training methods for robust DNN-HMM distant speech recognition
Authors: Mirco Ravanelli, Maurizio Omologo
Date: 2017-10-10 00:00:00
Tasks: Distant Speech Recognition, Speech Enhancement, speech-recognition, Speech Recognition

Despite the significant progress made in the last years, state-of-the-art
speech recognition technologies provide a satisfactory performance only in the
close-talking condition. Robustness of distant speech recognition in adverse
acoustic conditions, on the other hand, remains a crucial open issue for future
applications of human-machine interaction. To this end, several advances in
speech enhancement, acoustic scene analysis as well as acoustic modeling, have
recently contributed to improve the state-of-the-art in the field. One of the
most effective approaches to derive a robust acoustic modeling is based on
using contaminated speech, which proved helpful in reducing the acoustic
mismatch between training and testing conditions.
  In this paper, we revise this classical approach in the context of modern
DNN-HMM systems, and propose the adoption of three methods, namely, asymmetric
context windowing, close-talk based supervision, and close-talk based
pre-training. The experimental results, obtained using both real and simulated
data, show a significant advantage in using these three methods, overall
providing a 15% error rate reduction compared to the baseline systems. The same
trend in performance is confirmed either using a high-quality training set of
small size, and a large one.
Langevin Dynamics with Continuous Tempering for Training Deep Neural Networks
Authors: Nanyang Ye, Zhanxing Zhu, Rafal K. Mantiuk
Date: 2017-03-13 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Stochastic Optimization

Minimizing non-convex and high-dimensional objective functions is
challenging, especially when training modern deep neural networks. In this
paper, a novel approach is proposed which divides the training process into two
consecutive phases to obtain better generalization performance: Bayesian
sampling and stochastic optimization. The first phase is to explore the energy
landscape and to capture the "fat" modes; and the second one is to fine-tune
the parameter learned from the first phase. In the Bayesian learning phase, we
apply continuous tempering and stochastic approximation into the Langevin
dynamics to create an efficient and effective sampler, in which the temperature
is adjusted automatically according to the designed "temperature dynamics".
These strategies can overcome the challenge of early trapping into bad local
minima and have achieved remarkable improvements in various types of neural
networks as shown in our theoretical analysis and empirical experiments.
Semantic Segmentation of Colon Glands with Deep Convolutional Neural Networks and Total Variation Segmentation
Authors: Philipp Kainz, Michael Pfeiffer, Martin Urschler
Date: 2015-11-21 00:00:00
Tasks: General Classification, Segmentation, Semantic Segmentation

Segmentation of histopathology sections is an ubiquitous requirement in
digital pathology and due to the large variability of biological tissue,
machine learning techniques have shown superior performance over standard image
processing methods. As part of the GlaS@MICCAI2015 colon gland segmentation
challenge, we present a learning-based algorithm to segment glands in tissue of
benign and malignant colorectal cancer. Images are preprocessed according to
the Hematoxylin-Eosin staining protocol and two deep convolutional neural
networks (CNN) are trained as pixel classifiers. The CNN predictions are then
regularized using a figure-ground segmentation based on weighted total
variation to produce the final segmentation result. On two test sets, our
approach achieves a tissue classification accuracy of 98% and 94%, making use
of the inherent capability of our system to distinguish between benign and
malignant tissue.