pwc-archive/papers-with-abstracts huggingface dataset:
Multikernel activation functions: formulation and a case study
Authors: Simone Scardapane, Elena Nieddu, Donatella Firmani, Paolo Merialdo
Date: 2019-01-29 00:00:00
Tasks: Optical Character Recognition (OCR)

The design of activation functions is a growing research area in the field of
neural networks. In particular, instead of using fixed point-wise functions
(e.g., the rectified linear unit), several authors have proposed ways of
learning these functions directly from the data in a non-parametric fashion. In
this paper we focus on the kernel activation function (KAF), a recently
proposed framework wherein each function is modeled as a one-dimensional kernel
model, whose weights are adapted through standard backpropagation-based
optimization. One drawback of KAFs is the need to select a single kernel
function and its eventual hyper-parameters. To partially overcome this problem,
we motivate an extension of the KAF model, in which multiple kernels are
linearly combined at every neuron, inspired by the literature on multiple
kernel learning. We provide an application of the resulting multi-KAF on a
realistic use case, specifically handwritten Latin OCR, on a large dataset
collected in the context of the `In Codice Ratio' project. Results show that
multi-KAFs can improve the accuracy of the convolutional networks previously
developed for the task, with faster convergence, even with a smaller number of
overall parameters.
Detection of Alzheimers Disease from MRI using Convolutional Neural Networks, Exploring Transfer Learning And BellCNN
Authors: GuruRaj Awate
Date: 2019-01-29 00:00:00
Tasks: Transfer Learning

There is a need for automatic diagnosis of certain diseases from medical
images that could help medical practitioners for further assessment towards
treating the illness. Alzheimers disease is a good example of a disease that is
often misdiagnosed. Alzheimers disease (Hear after referred to as AD), is
caused by atrophy of certain brain regions and by brain cell death and is the
leading cause of dementia and memory loss [1]. MRI scans reveal this
information but atrophied regions are different for different individuals which
makes the diagnosis a bit more trickier and often gets misdiagnosed [1, 13]. We
believe that our approach to this particular problem would improve the
assessment quality by pre-flagging the images which are more likely to have AD.
We propose two solutions to this; one with transfer learning [9] and other by
BellCNN [14], a custom made Convolutional Neural Network (Hear after referred
to as CNN). Advantages and disadvantages of each approach will also be
discussed in their respective sections. The dataset used for this project is
provided by Open Access Series of Imaging Studies (Hear after referred to as
OASIS) [2, 3, 4], which contains over 400 subjects, 100 of whom have mild to
severe dementia. The dataset has labeled these subjects by two standards of
diagnosis; MiniMental State Examination (Hear after referred to as MMSE) and
Clinical Dementia Rating (Hear after referred to as CDR). These are some of the
general tools and concepts which are prerequisites to our solution; CNN [5, 6],
Neural Networks [10] (Hear after referred to as NN), Anaconda bundle for
python, Regression, Tensorflow [7]. Keywords: Alzheimers Disease, Convolutional
Neural Network, BellCNN, Image Recognition, Machine Learning, MRI, OASIS,
Tensorflow
Partially Exchangeable Networks and Architectures for Learning Summary Statistics in Approximate Bayesian Computation
Authors: Samuel Wiqvist, Pierre-Alexandre Mattei, Umberto Picchini, Jes Frellsen
Date: 2019-01-29 00:00:00
Tasks: Time Series, Time Series Analysis

We present a novel family of deep neural architectures, named partially exchangeable networks (PENs) that leverage probabilistic symmetries. By design, PENs are invariant to block-switch transformations, which characterize the partial exchangeability properties of conditionally Markovian processes. Moreover, we show that any block-switch invariant function has a PEN-like representation. The DeepSets architecture is a special case of PEN and we can therefore also target fully exchangeable data. We employ PENs to learn summary statistics in approximate Bayesian computation (ABC). When comparing PENs to previous deep learning methods for learning summary statistics, our results are highly competitive, both considering time series and static models. Indeed, PENs provide more reliable posterior samples even when using less training data.
Representation Transfer for Differentially Private Drug Sensitivity Prediction
Authors: Teppo Niinimäki, Mikko Heikkilä, Antti Honkela, Samuel Kaski
Date: 2019-01-29 00:00:00
Tasks: BIG-bench Machine Learning, Cancer type classification, Dimensionality Reduction, Prediction, Representation Learning, Sensitivity

Motivation: Human genomic datasets often contain sensitive information that
limits use and sharing of the data. In particular, simple anonymisation
strategies fail to provide sufficient level of protection for genomic data,
because the data are inherently identifiable. Differentially private machine
learning can help by guaranteeing that the published results do not leak too
much information about any individual data point. Recent research has reached
promising results on differentially private drug sensitivity prediction using
gene expression data. Differentially private learning with genomic data is
challenging because it is more difficult to guarantee the privacy in high
dimensions. Dimensionality reduction can help, but if the dimension reduction
mapping is learned from the data, then it needs to be differentially private
too, which can carry a significant privacy cost. Furthermore, the selection of
any hyperparameters (such as the target dimensionality) needs to also avoid
leaking private information.
  Results: We study an approach that uses a large public dataset of similar
type to learn a compact representation for differentially private learning. We
compare three representation learning methods: variational autoencoders, PCA
and random projection. We solve two machine learning tasks on gene expression
of cancer cell lines: cancer type classification, and drug sensitivity
prediction. The experiments demonstrate significant benefit from all
representation learning methods with variational autoencoders providing the
most accurate predictions most often. Our results significantly improve over
previous state-of-the-art in accuracy of differentially private drug
sensitivity prediction.
Revised JNLPBA Corpus: A Revised Version of Biomedical NER Corpus for Relation Extraction Task
Authors: Ming-Siang Huang, Po-Ting Lai, Richard Tzong-Han Tsai, Wen-Lian Hsu
Date: 2019-01-29 00:00:00
Tasks: Event Extraction, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Relation, Relation Extraction

The advancement of biomedical named entity recognition (BNER) and biomedical
relation extraction (BRE) researches promotes the development of text mining in
biological domains. As a cornerstone of BRE, robust BNER system is required to
identify the mentioned NEs in plain texts for further relation extraction
stage. However, the current BNER corpora, which play important roles in these
tasks, paid less attention to achieve the criteria for BRE task. In this study,
we present Revised JNLPBA corpus, the revision of JNLPBA corpus, to broaden the
applicability of a NER corpus from BNER to BRE task. We preserve the original
entity types including protein, DNA, RNA, cell line and cell type while all the
abstracts in JNLPBA corpus are manually curated by domain experts again basis
on the new annotation guideline focusing on the specific NEs instead of general
terms. Simultaneously, several imperfection issues in JNLPBA are pointed out
and made up in the new corpus. To compare the adaptability of different NER
systems in Revised JNLPBA and JNLPBA corpora, the F1-measure was measured in
three open sources NER systems including BANNER, Gimli and NERSuite. In the
same circumstance, all the systems perform average 10% better in Revised JNLPBA
than in JNLPBA. Moreover, the cross-validation test is carried out which we
train the NER systems on JNLPBA/Revised JNLPBA corpora and access the
performance in both protein-protein interaction extraction (PPIE) and
biomedical event extraction (BEE) corpora to confirm that the newly refined
Revised JNLPBA is a competent NER corpus in biomedical relation application.
The revised JNLPBA corpus is freely available at
iasl-btm.iis.sinica.edu.tw/BNER/Content/Revised_JNLPBA.zip.
A Push-Pull Layer Improves Robustness of Convolutional Neural Networks
Authors: Nicola Strisciuglio, Manuel Lopez-Antequera, Nicolai Petkov
Date: 2019-01-29 00:00:00
Tasks: General Classification, image-classification, Image Classification

We propose a new layer in Convolutional Neural Networks (CNNs) to increase
their robustness to several types of noise perturbations of the input images.
We call this a push-pull layer and compute its response as the combination of
two half-wave rectified convolutions, with kernels of opposite polarity. It is
based on a biologically-motivated non-linear model of certain neurons in the
visual system that exhibit a response suppression phenomenon, known as
push-pull inhibition. We validate our method by substituting the first
convolutional layer of the LeNet-5 and WideResNet architectures with our
push-pull layer. We train the networks on nonperturbed training images from the
MNIST, CIFAR-10 and CIFAR-100 data sets, and test on images perturbed by noise
that is unseen by the training process. We demonstrate that our push-pull
layers contribute to a considerable improvement in robustness of classification
of images perturbed by noise, while maintaining state-of-the-art performance on
the original image classification task.
Approximating Spectral Clustering via Sampling: a Review
Authors: Nicolas Tremblay, Andreas Loukas
Date: 2019-01-29 00:00:00
Tasks: Clustering

Spectral clustering refers to a family of unsupervised learning algorithms
that compute a spectral embedding of the original data based on the
eigenvectors of a similarity graph. This non-linear transformation of the data
is both the key of these algorithms' success and their Achilles heel: forming a
graph and computing its dominant eigenvectors can indeed be computationally
prohibitive when dealing with more that a few tens of thousands of points. In
this paper, we review the principal research efforts aiming to reduce this
computational cost. We focus on methods that come with a theoretical control on
the clustering performance and incorporate some form of sampling in their
operation. Such methods abound in the machine learning, numerical linear
algebra, and graph signal processing literature and, amongst others, include
Nystr\"om-approximation, landmarks, coarsening, coresets, and compressive
spectral clustering. We present the approximation guarantees available for each
and discuss practical merits and limitations. Surprisingly, despite the breadth
of the literature explored, we conclude that there is still a gap between
theory and practice: the most scalable methods are only intuitively motivated
or loosely controlled, whereas those that come with end-to-end guarantees rely
on strong assumptions or enable a limited gain of computation time.
catch22: CAnonical Time-series CHaracteristics
Authors: Carl H. Lubba, Sarab S. Sethi, Philip Knaute, Simon R Schultz, Ben D. Fulcher, Nick S. Jones
Date: 2019-01-29 00:00:00
Tasks: Classification, Clustering, Dimensionality Reduction, General Classification, Time Series, Time Series Analysis, Time Series Classification

Capturing the dynamical properties of time series concisely as interpretable
feature vectors can enable efficient clustering and classification for
time-series applications across science and industry. Selecting an appropriate
feature-based representation of time series for a given application can be
achieved through systematic comparison across a comprehensive time-series
feature library, such as those in the hctsa toolbox. However, this approach is
computationally expensive and involves evaluating many similar features,
limiting the widespread adoption of feature-based representations of time
series for real-world applications. In this work, we introduce a method to
infer small sets of time-series features that (i) exhibit strong classification
performance across a given collection of time-series problems, and (ii) are
minimally redundant. Applying our method to a set of 93 time-series
classification datasets (containing over 147000 time series) and using a
filtered version of the hctsa feature library (4791 features), we introduce a
generically useful set of 22 CAnonical Time-series CHaracteristics, catch22.
This dimensionality reduction, from 4791 to 22, is associated with an
approximately 1000-fold reduction in computation time and near linear scaling
with time-series length, despite an average reduction in classification
accuracy of just 7%. catch22 captures a diverse and interpretable signature of
time series in terms of their properties, including linear and non-linear
autocorrelation, successive differences, value distributions and outliers, and
fluctuation scaling properties. We provide an efficient implementation of
catch22, accessible from many programming environments, that facilitates
feature-based time-series analysis for scientific, industrial, financial and
medical applications using a common language of interpretable time-series
properties.
Divide and Generate: Neural Generation of Complex Sentences
Authors: Tomoya Ogata, Mamoru Komachi, Tomoya Takatani
Date: 2019-01-29 00:00:00
Tasks: Sentence

We propose a task to generate a complex sentence from a simple sentence in
order to amplify various kinds of responses in the database. We first divide a
complex sentence into a main clause and a subordinate clause to learn a
generator model of modifiers, and then use the model to generate a modifier
clause to create a complex sentence from a simple sentence. We present an
automatic evaluation metric to estimate the quality of the models and show that
a pipeline model outperforms an end-to-end model.
An Arabic Dependency Treebank in the Travel Domain
Authors: Dima Taji, Jamila El Gizuli, Nizar Habash
Date: 2019-01-29 00:00:00
Tasks: Translation

In this paper we present a dependency treebank of travel domain sentences in
Modern Standard Arabic. The text comes from a translation of the English
equivalent sentences in the Basic Traveling Expressions Corpus. The treebank
dependency representation is in the style of the Columbia Arabic Treebank. The
paper motivates the effort and discusses the construction process and
guidelines. We also present parsing results and discuss the effect of domain
and genre difference on parsing.
Towards Optimal Discrete Online Hashing with Balanced Similarity
Authors: Mingbao Lin, Rongrong Ji, Hong Liu, Xiaoshuai Sun, Yongjian Wu, Yunsheng Wu
Date: 2019-01-29 00:00:00
Tasks: Retrieval

When facing large-scale image datasets, online hashing serves as a promising
solution for online retrieval and prediction tasks. It encodes the online
streaming data into compact binary codes, and simultaneously updates the hash
functions to renew codes of the existing dataset. To this end, the existing
methods update hash functions solely based on the new data batch, without
investigating the correlation between such new data and the existing dataset.
In addition, existing works update the hash functions using a relaxation
process in its corresponding approximated continuous space. And it remains as
an open problem to directly apply discrete optimizations in online hashing. In
this paper, we propose a novel supervised online hashing method, termed
Balanced Similarity for Online Discrete Hashing (BSODH), to solve the above
problems in a unified framework. BSODH employs a well-designed hashing
algorithm to preserve the similarity between the streaming data and the
existing dataset via an asymmetric graph regularization. We further identify
the "data-imbalance" problem brought by the constructed asymmetric graph, which
restricts the application of discrete optimization in our problem. Therefore, a
novel balanced similarity is further proposed, which uses two equilibrium
factors to balance the similar and dissimilar weights and eventually enables
the usage of discrete optimizations. Extensive experiments conducted on three
widely-used benchmarks demonstrate the advantages of the proposed method over
the state-of-the-art methods.
A Modular Benchmarking Infrastructure for High-Performance and Reproducible Deep Learning
Authors: Tal Ben-Nun, Maciej Besta, Simon Huber, Alexandros Nikolaos Ziogas, Daniel Peter, Torsten Hoefler
Date: 2019-01-29 00:00:00
Tasks: Benchmarking, Deep Learning, Vocal Bursts Intensity Prediction

We introduce Deep500: the first customizable benchmarking infrastructure that enables fair comparison of the plethora of deep learning frameworks, algorithms, libraries, and techniques. The key idea behind Deep500 is its modular design, where deep learning is factorized into four distinct levels: operators, network processing, training, and distributed training. Our evaluation illustrates that Deep500 is customizable (enables combining and benchmarking different deep learning codes) and fair (uses carefully selected metrics). Moreover, Deep500 is fast (incurs negligible overheads), verifiable (offers infrastructure to analyze correctness), and reproducible. Finally, as the first distributed and reproducible benchmarking system for deep learning, Deep500 provides software infrastructure to utilize the most powerful supercomputers for extreme-scale workloads.
Generative Adversarial Networks for geometric surfaces prediction in injection molding
Authors: Pierre Nagorny, Thomas Lacombe, Hugues Favreliere, Maurice Pillet, Eric Pairel, Ronan Le Goff, Marlene Wali, Jerome Loureaux, Patrice Kiener
Date: 2019-01-29 00:00:00

Geometrical and appearance quality requirements set the limits of the current
industrial performance in injection molding. To guarantee the product's
quality, it is necessary to adjust the process settings in a closed loop. Those
adjustments cannot rely on the final quality because a part takes days to be
geometrically stable. Thus, the final part geometry must be predicted from
measurements on hot parts. In this paper, we use recent success of Generative
Adversarial Networks (GAN) with the pix2pix network architecture to predict the
final part geometry, using only hot parts thermographic images, measured right
after production. Our dataset is really small, and the GAN learns to translate
thermography to geometry. We firstly study prediction performances using
different image similarity comparison algorithms. Moreover, we introduce the
innovative use of Discrete Modal Decomposition (DMD) to analyze network
predictions. The DMD is a geometrical parameterization technique using a modal
space projection to geometrically describe surfaces. We study GAN performances
to retrieve geometrical parameterization of surfaces.
Bayes Imbalance Impact Index: A Measure of Class Imbalanced Dataset for Classification Problem
Authors: Yang Lu, Yiu-ming Cheung, Yuan Yan Tang
Date: 2019-01-29 00:00:00
Tasks: General Classification

Recent studies have shown that imbalance ratio is not the only cause of the
performance loss of a classifier in imbalanced data classification. In fact,
other data factors, such as small disjuncts, noises and overlapping, also play
the roles in tandem with imbalance ratio, which makes the problem difficult.
Thus far, the empirical studies have demonstrated the relationship between the
imbalance ratio and other data factors only. To the best of our knowledge,
there is no any measurement about the extent of influence of class imbalance on
the classification performance of imbalanced data. Further, it is also unknown
for a dataset which data factor is actually the main barrier for
classification. In this paper, we focus on Bayes optimal classifier and study
the influence of class imbalance from a theoretical perspective. Accordingly,
we propose an instance measure called Individual Bayes Imbalance Impact Index
($IBI^3$) and a data measure called Bayes Imbalance Impact Index ($BI^3$).
$IBI^3$ and $BI^3$ reflect the extent of influence purely by the factor of
imbalance in terms of each minority class sample and the whole dataset,
respectively. Therefore, $IBI^3$ can be used as an instance complexity measure
of imbalance and $BI^3$ is a criterion to show the degree of how imbalance
deteriorates the classification. As a result, we can therefore use $BI^3$ to
judge whether it is worth using imbalance recovery methods like sampling or
cost-sensitive methods to recover the performance loss of a classifier. The
experiments show that $IBI^3$ is highly consistent with the increase of
prediction score made by the imbalance recovery methods and $BI^3$ is highly
consistent with the improvement of F1 score made by the imbalance recovery
methods on both synthetic and real benchmark datasets.
Mask-RCNN and U-net Ensembled for Nuclei Segmentation
Authors: Aarno Oskar Vuola, Saad Ullah Akram, Juho Kannala
Date: 2019-01-29 00:00:00
Tasks: Segmentation

Nuclei segmentation is both an important and in some ways ideal task for
modern computer vision methods, e.g. convolutional neural networks. While
recent developments in theory and open-source software have made these tools
easier to implement, expert knowledge is still required to choose the right
model architecture and training setup. We compare two popular segmentation
frameworks, U-Net and Mask-RCNN in the nuclei segmentation task and find that
they have different strengths and failures. To get the best of both worlds, we
develop an ensemble model to combine their predictions that can outperform both
models by a significant margin and should be considered when aiming for best
nuclei segmentation performance.
Visual Rhythm Prediction with Feature-Aligning Network
Authors: Yutong Xie, Haiyang Wang, Yan Hao, Zihao Xu
Date: 2019-01-29 00:00:00
Tasks: Optical Flow Estimation, Prediction, Rhythm

In this paper, we propose a data-driven visual rhythm prediction method,
which overcomes the previous works' deficiency that predictions are made
primarily by human-crafted hard rules. In our approach, we first extract
features including original frames and their residuals, optical flow, scene
change, and body pose. These visual features will be next taken into an
end-to-end neural network as inputs. Here we observe that there are some slight
misaligning between features over the timeline and assume that this is due to
the distinctions between how different features are computed. To solve this
problem, the extracted features are aligned by an elaborately designed layer,
which can also be applied to other models suffering from mismatched features,
and boost performance. Then these aligned features are fed into sequence
labeling layers implemented with BiLSTM and CRF to predict the onsets. Due to
the lack of existing public training and evaluation set, we experiment on a
dataset constructed by ourselves based on professionally edited Music Videos
(MVs), and the F1 score of our approach reaches 79.6.