pwc-archive/papers-with-abstracts huggingface dataset:
Deep Learning Methods for Improved Decoding of Linear Codes
Authors: Eliya Nachmani, Elad Marciano, Loren Lugosch, Warren J. Gross, David Burshtein, Yair Beery
Date: 2017-06-21 00:00:00
Tasks: Decoder, Deep Learning

The problem of low complexity, close to optimal, channel decoding of linear
codes with short to moderate block length is considered. It is shown that deep
learning methods can be used to improve a standard belief propagation decoder,
despite the large example space. Similar improvements are obtained for the
min-sum algorithm. It is also shown that tying the parameters of the decoders
across iterations, so as to form a recurrent neural network architecture, can
be implemented with comparable results. The advantage is that significantly
less parameters are required. We also introduce a recurrent neural decoder
architecture based on the method of successive relaxation. Improvements over
standard belief propagation are also observed on sparser Tanner graph
representations of the codes. Furthermore, we demonstrate that the neural
belief propagation decoder can be used to improve the performance, or
alternatively reduce the computational complexity, of a close to optimal
decoder of short BCH codes.
Aggregated Channels Network for Real-Time Pedestrian Detection
Authors: Farzin Ghorban, Javier Marín, Yu Su, Alessandro Colombo, Anton Kummert
Date: 2018-01-01 00:00:00
Tasks: GPU, Pedestrian Detection

Convolutional neural networks (CNNs) have demonstrated their superiority in
numerous computer vision tasks, yet their computational cost results
prohibitive for many real-time applications such as pedestrian detection which
is usually performed on low-consumption hardware. In order to alleviate this
drawback, most strategies focus on using a two-stage cascade approach.
Essentially, in the first stage a fast method generates a significant but
reduced amount of high quality proposals that later, in the second stage, are
evaluated by the CNN. In this work, we propose a novel detection pipeline that
further benefits from the two-stage cascade strategy. More concretely, the
enriched and subsequently compressed features used in the first stage are
reused as the CNN input. As a consequence, a simpler network architecture,
adapted for such small input sizes, allows to achieve real-time performance and
obtain results close to the state-of-the-art while running significantly faster
without the use of GPU. In particular, considering that the proposed pipeline
runs in frame rate, the achieved performance is highly competitive. We
furthermore demonstrate that the proposed pipeline on itself can serve as an
effective proposal generator.
Automated image segmentation for detecting cell spreading for metastasizing assessments of cancer development
Authors: Sholpan Kauanova, Ivan Vorobjev, Alex Pappachen James
Date: 2018-01-01 00:00:00
Tasks: Boundary Detection, Image Segmentation, Segmentation, Semantic Segmentation

The automated segmentation of cells in microscopic images is an open research
problem that has important implications for studies of the developmental and
cancer processes based on in vitro models. In this paper, we present the
approach for segmentation of the DIC images of cultured cells using G-neighbor
smoothing followed by Kauwahara filtering and local standard deviation approach
for boundary detection. NIH FIJI/ImageJ tools are used to create the ground
truth dataset. The results of this work indicate that detection of cell
boundaries using segmentation approach even in the case of realistic
measurement conditions is a challenging problem.
Quality assessment metrics for edge detection and edge-aware filtering: A tutorial review
Authors: Diana Sadykova, Alex Pappachen James
Date: 2018-01-01 00:00:00
Tasks: Edge Detection, SSIM

The quality assessment of edges in an image is an important topic as it helps
to benchmark the performance of edge detectors, and edge-aware filters that are
used in a wide range of image processing tasks. The most popular image quality
metrics such as Mean squared error (MSE), Peak signal-to-noise ratio (PSNR) and
Structural similarity (SSIM) metrics for assessing and justifying the quality
of edges. However, they do not address the structural and functional accuracy
of edges in images with a wide range of natural variabilities. In this review,
we provide an overview of all the most relevant performance metrics that can be
used to benchmark the quality performance of edges in images. We identify four
major groups of metrics and also provide a critical insight into the evaluation
protocol and governing equations.
Automated rating of recorded classroom presentations using speech analysis in kazakh
Authors: Akzharkyn Izbassarova, Aidana Irmanova, A. P. James
Date: 2018-01-01 00:00:00

Effective presentation skills can help to succeed in business, career and
academy. This paper presents the design of speech assessment during the oral
presentation and the algorithm for speech evaluation based on criteria of
optimal intonation. As the pace of the speech and its optimal intonation varies
from language to language, developing an automatic identification of language
during the presentation is required. Proposed algorithm was tested with
presentations delivered in Kazakh language. For testing purposes the features
of Kazakh phonemes were extracted using MFCC and PLP methods and created a
Hidden Markov Model (HMM) [5], [5] of Kazakh phonemes. Kazakh vowel formants
were defined and the correlation between the deviation rate in fundamental
frequency and the liveliness of the speech to evaluate intonation of the
presentation was analyzed. It was established that the threshold value between
monotone and dynamic speech is 0.16 and the error for intonation evaluation is
19%.
Facial emotion recognition using min-max similarity classifier
Authors: Olga Krestinskaya, Alex Pappachen James
Date: 2018-01-01 00:00:00
Tasks: Classification, Emotion Recognition, Facial Emotion Recognition, feature selection, General Classification, Template Matching

Recognition of human emotions from the imaging templates is useful in a wide
variety of human-computer interaction and intelligent systems applications.
However, the automatic recognition of facial expressions using image template
matching techniques suffer from the natural variability with facial features
and recording conditions. In spite of the progress achieved in facial emotion
recognition in recent years, the effective and computationally simple feature
selection and classification technique for emotion recognition is still an open
problem. In this paper, we propose an efficient and straightforward facial
emotion recognition algorithm to reduce the problem of inter-class pixel
mismatch during classification. The proposed method includes the application of
pixel normalization to remove intensity offsets followed-up with a Min-Max
metric in a nearest neighbor classifier that is capable of suppressing feature
outliers. The results indicate an improvement of recognition performance from
92.85% to 98.57% for the proposed Min-Max classification method when tested on
JAFFE database. The proposed emotion recognition technique outperforms the
existing template matching methods.
Least Square Error Method Robustness of Computation: What is not usually considered and taught
Authors: Vaclav Skala
Date: 2018-01-01 00:00:00

There are many practical applications based on the Least Square Error (LSE)
approximation. It is based on a square error minimization 'on a vertical' axis.
The LSE method is simple and easy also for analytical purposes. However, if
data span is large over several magnitudes or non-linear LSE is used, severe
numerical instability can be expected. The presented contribution describes a
simple method for large span of data LSE computation. It is especially
convenient if large span of data are to be processed, when the 'standard'
pseudoinverse matrix is ill conditioned. It is actually based on a LSE solution
using orthogonal basis vectors instead of orthonormal basis vectors. The
presented approach has been used for a linear regression as well as for
approximation using radial basis functions.
Semantic Segmentation of Human Thigh Quadriceps Muscle in Magnetic Resonance Images
Authors: Ezak Ahmad, Manu Goyal, Jamie S. McPhee, Hans Degens, Moi Hoon Yap
Date: 2018-01-01 00:00:00
Tasks: Segmentation, Semantic Segmentation, Transfer Learning

This paper presents an end-to-end solution for MRI thigh quadriceps
segmentation. This is the first attempt that deep learning methods are used for
the MRI thigh segmentation task. We use the state-of-the-art Fully
Convolutional Networks with transfer learning approach for the semantic
segmentation of regions of interest in MRI thigh scans. To further improve the
performance of the segmentation, we propose a post-processing technique using
basic image processing methods. With our proposed method, we have established a
new benchmark for MRI thigh quadriceps segmentation with mean Jaccard
Similarity Index of 0.9502 and processing time of 0.117 second per image.
Train longer, generalize better: closing the generalization gap in large batch training of neural networks
Authors: Elad Hoffer, Itay Hubara, Daniel Soudry
Date: 2017-05-24 00:00:00
Proceeding: NeurIPS 2017 12

Background: Deep learning models are typically trained using stochastic
gradient descent or one of its variants. These methods update the weights using
their gradient, estimated from a small fraction of the training data. It has
been observed that when using large batch sizes there is a persistent
degradation in generalization performance - known as the "generalization gap"
phenomena. Identifying the origin of this gap and closing it had remained an
open problem.
  Contributions: We examine the initial high learning rate training phase. We
find that the weight distance from its initialization grows logarithmically
with the number of weight updates. We therefore propose a "random walk on
random landscape" statistical model which is known to exhibit similar
"ultra-slow" diffusion behavior. Following this hypothesis we conducted
experiments to show empirically that the "generalization gap" stems from the
relatively small number of updates rather than the batch size, and can be
completely eliminated by adapting the training regime used. We further
investigate different techniques to train models in the large-batch regime and
present a novel algorithm named "Ghost Batch Normalization" which enables
significant decrease in the generalization gap without increasing the number of
updates. To validate our findings we conduct several additional experiments on
MNIST, CIFAR-10, CIFAR-100 and ImageNet. Finally, we reassess common practices
and beliefs concerning training of deep models and suggest they may not be
optimal to achieve good generalization.
PDE-Net: Learning PDEs from Data
Authors: Zichao Long, Yiping Lu, Xianzhong Ma, Bin Dong
Date: 2017-10-26 00:00:00
Proceeding: ICML 2018 7

In this paper, we present an initial attempt to learn evolution PDEs from
data. Inspired by the latest development of neural network designs in deep
learning, we propose a new feed-forward deep network, called PDE-Net, to
fulfill two objectives at the same time: to accurately predict dynamics of
complex systems and to uncover the underlying hidden PDE models. The basic idea
of the proposed PDE-Net is to learn differential operators by learning
convolution kernels (filters), and apply neural networks or other machine
learning methods to approximate the unknown nonlinear responses. Comparing with
existing approaches, which either assume the form of the nonlinear response is
known or fix certain finite difference approximations of differential
operators, our approach has the most flexibility by learning both differential
operators and the nonlinear responses. A special feature of the proposed
PDE-Net is that all filters are properly constrained, which enables us to
easily identify the governing PDE models while still maintaining the expressive
and predictive power of the network. These constrains are carefully designed by
fully exploiting the relation between the orders of differential operators and
the orders of sum rules of filters (an important concept originated from
wavelet theory). We also discuss relations of the PDE-Net with some existing
networks in computer vision such as Network-In-Network (NIN) and Residual
Neural Network (ResNet). Numerical experiments show that the PDE-Net has the
potential to uncover the hidden PDE of the observed dynamics, and predict the
dynamical behavior for a relatively long time, even in a noisy environment.
Theory of Optimizing Pseudolinear Performance Measures: Application to F-measure
Authors: Shameem A Puthiya Parambath, Nicolas Usunier, Yves GRANDVALET
Date: 2015-05-01 00:00:00
Tasks: Classification, General Classification, Information Retrieval, Retrieval, valid

Non-linear performance measures are widely used for the evaluation of
learning algorithms. For example, $F$-measure is a commonly used performance
measure for classification problems in machine learning and information
retrieval community. We study the theoretical properties of a subset of
non-linear performance measures called pseudo-linear performance measures which
includes $F$-measure, \emph{Jaccard Index}, among many others. We establish
that many notions of $F$-measures and \emph{Jaccard Index} are pseudo-linear
functions of the per-class false negatives and false positives for binary,
multiclass and multilabel classification. Based on this observation, we present
a general reduction of such performance measure optimization problem to
cost-sensitive classification problem with unknown costs. We then propose an
algorithm with provable guarantees to obtain an approximately optimal
classifier for the $F$-measure by solving a series of cost-sensitive
classification problems. The strength of our analysis is to be valid on any
dataset and any class of classifiers, extending the existing theoretical
results on pseudo-linear measures, which are asymptotic in nature. We also
establish the multi-objective nature of the $F$-score maximization problem by
linking the algorithm with the weighted-sum approach used in multi-objective
optimization. We present numerical experiments to illustrate the relative
importance of cost asymmetry and thresholding when learning linear classifiers
on various $F$-measure optimization tasks.
Speeding Up MCMC by Efficient Data Subsampling
Authors: Matias Quiroz, Robert Kohn, Mattias Villani, Minh-Ngoc Tran
Date: 2014-04-16 00:00:00

We propose Subsampling MCMC, a Markov Chain Monte Carlo (MCMC) framework
where the likelihood function for $n$ observations is estimated from a random
subset of $m$ observations. We introduce a highly efficient unbiased estimator
of the log-likelihood based on control variates, such that the computing cost
is much smaller than that of the full log-likelihood in standard MCMC. The
likelihood estimate is bias-corrected and used in two dependent pseudo-marginal
algorithms to sample from a perturbed posterior, for which we derive the
asymptotic error with respect to $n$ and $m$, respectively. We propose a
practical estimator of the error and show that the error is negligible even for
a very small $m$ in our applications. We demonstrate that Subsampling MCMC is
substantially more efficient than standard MCMC in terms of sampling efficiency
for a given computational budget, and that it outperforms other subsampling
methods for MCMC proposed in the literature.
Dynamic Pricing in High-dimensions
Authors: Adel Javanmard, Hamid Nazerzadeh
Date: 2016-09-24 00:00:00
Tasks: Vocal Bursts Intensity Prediction

We study the pricing problem faced by a firm that sells a large number of
products, described via a wide range of features, to customers that arrive over
time. Customers independently make purchasing decisions according to a general
choice model that includes products features and customers' characteristics,
encoded as $d$-dimensional numerical vectors, as well as the price offered. The
parameters of the choice model are a priori unknown to the firm, but can be
learned as the (binary-valued) sales data accrues over time. The firm's
objective is to minimize the regret, i.e., the expected revenue loss against a
clairvoyant policy that knows the parameters of the choice model in advance,
and always offers the revenue-maximizing price. This setting is motivated in
part by the prevalence of online marketplaces that allow for real-time pricing.
We assume a structured choice model, parameters of which depend on $s_0$ out of
the $d$ product features. We propose a dynamic policy, called Regularized
Maximum Likelihood Pricing (RMLP) that leverages the (sparsity) structure of
the high-dimensional model and obtains a logarithmic regret in $T$. More
specifically, the regret of our algorithm is of $O(s_0 \log d \cdot \log T)$.
Furthermore, we show that no policy can obtain regret better than $O(s_0 (\log
d + \log T))$.
Beyond Word Embeddings: Learning Entity and Concept Representations from Large Scale Knowledge Bases
Authors: Walid Shalaby, Wlodek Zadrozny, Hongxia Jin
Date: 2018-01-01 00:00:00
Tasks: Semantic Parsing, Word Embeddings

Text representations using neural word embeddings have proven effective in
many NLP applications. Recent researches adapt the traditional word embedding
models to learn vectors of multiword expressions (concepts/entities). However,
these methods are limited to textual knowledge bases (e.g., Wikipedia). In this
paper, we propose a novel and simple technique for integrating the knowledge
about concepts from two large scale knowledge bases of different structure
(Wikipedia and Probase) in order to learn concept representations. We adapt the
efficient skip-gram model to seamlessly learn from the knowledge in Wikipedia
text and Probase concept graph. We evaluate our concept embedding models on two
tasks: (1) analogical reasoning, where we achieve a state-of-the-art
performance of 91% on semantic analogies, (2) concept categorization, where we
achieve a state-of-the-art performance on two benchmark datasets achieving
categorization accuracy of 100% on one and 98% on the other. Additionally, we
present a case study to evaluate our model on unsupervised argument type
identification for neural semantic parsing. We demonstrate the competitive
accuracy of our unsupervised method and its ability to better generalize to out
of vocabulary entity mentions compared to the tedious and error prone methods
which depend on gazetteers and regular expressions.
Error-Robust Multi-View Clustering
Authors: Mehrnaz Najafi, Lifang He, Philip S. Yu
Date: 2018-01-01 00:00:00
Tasks: Clustering

In the era of big data, data may come from multiple sources, known as
multi-view data. Multi-view clustering aims at generating better clusters by
exploiting complementary and consistent information from multiple views rather
than relying on the individual view. Due to inevitable system errors caused by
data-captured sensors or others, the data in each view may be erroneous.
Various types of errors behave differently and inconsistently in each view.
More precisely, error could exhibit as noise and corruptions in reality.
Unfortunately, none of the existing multi-view clustering approaches handle all
of these error types. Consequently, their clustering performance is
dramatically degraded. In this paper, we propose a novel Markov chain method
for Error-Robust Multi-View Clustering (EMVC). By decomposing each view into a
shared transition probability matrix and error matrix and imposing structured
sparsity-inducing norms on error matrices, we characterize and handle typical
types of errors explicitly. To solve the challenging optimization problem, we
propose a new efficient algorithm based on Augmented Lagrangian Multipliers and
prove its convergence rigorously. Experimental results on various synthetic and
real-world datasets show the superiority of the proposed EMVC method over the
baseline methods and its robustness against different types of errors.
Identification of Parallel Passages Across a Large Hebrew/Aramaic Corpus
Authors: Avi Shmidman, Moshe Koppel, Ely Porat
Date: 2016-02-28 00:00:00

We propose a method for efficiently finding all parallel passages in a large
corpus, even if the passages are not quite identical due to rephrasing and
orthographic variation. The key ideas are the representation of each word in
the corpus by its two most infrequent letters, finding matched pairs of strings
of four or five words that differ by at most one word and then identifying
clusters of such matched pairs. Using this method, over 4600 parallel pairs of
passages were identified in the Babylonian Talmud, a Hebrew-Aramaic corpus of
over 1.8 million words, in just over 30 seconds. Empirical comparisons on
sample data indicate that the coverage obtained by our method is essentially
the same as that obtained using slow exhaustive methods.
von Mises-Fisher Mixture Model-based Deep learning: Application to Face Verification
Authors: Md. Abul Hasnat, Julien Bohné, Jonathan Milgram, Stéphane Gentric, Liming Chen
Date: 2017-06-13 00:00:00
Tasks: Clustering, Face Verification

A number of pattern recognition tasks, \textit{e.g.}, face verification, can
be boiled down to classification or clustering of unit length directional
feature vectors whose distance can be simply computed by their angle. In this
paper, we propose the von Mises-Fisher (vMF) mixture model as the theoretical
foundation for an effective deep-learning of such directional features and
derive a novel vMF Mixture Loss and its corresponding vMF deep features. The
proposed vMF feature learning achieves the characteristics of discriminative
learning, \textit{i.e.}, compacting the instances of the same class while
increasing the distance of instances from different classes. Moreover, it
subsumes a number of popular loss functions as well as an effective method in
deep learning, namely normalization. We conduct extensive experiments on face
verification using 4 different challenging face datasets, \textit{i.e.}, LFW,
YouTube faces, CACD and IJB-A. Results show the effectiveness and excellent
generalization ability of the proposed approach as it achieves state-of-the-art
results on the LFW, YouTube faces and CACD datasets and competitive results on
the IJB-A dataset.
Estimation and Inference of Treatment Effects with $L_2$-Boosting in High-Dimensional Settings
Authors: Jannis Kueck, Ye Luo, Martin Spindler, Zigan Wang
Date: 2017-12-31 00:00:00
Tasks: valid, Variable Selection

Empirical researchers are increasingly faced with rich data sets containing many controls or instrumental variables, making it essential to choose an appropriate approach to variable selection. In this paper, we provide results for valid inference after post- or orthogonal $L_2$-Boosting is used for variable selection. We consider treatment effects after selecting among many control variables and instrumental variable models with potentially many instruments. To achieve this, we establish new results for the rate of convergence of iterated post-$L_2$-Boosting and orthogonal $L_2$-Boosting in a high-dimensional setting similar to Lasso, i.e., under approximate sparsity without assuming the beta-min condition. These results are extended to the 2SLS framework and valid inference is provided for treatment effect analysis. We give extensive simulation results for the proposed methods and compare them with Lasso. In an empirical application, we construct efficient IVs with our proposed methods to estimate the effect of pre-merger overlap of bank branch networks in the US on the post-merger stock returns of the acquirer bank.