pwc-archive/papers-with-abstracts huggingface dataset:
Ridge Regression and Provable Deterministic Ridge Leverage Score Sampling
Authors: Shannon R. McCurdy
Date: 2018-03-15 00:00:00
Proceeding: NeurIPS 2018
Tasks: regression

Ridge leverage scores provide a balance between low-rank approximation and
regularization, and are ubiquitous in randomized linear algebra and machine
learning. Deterministic algorithms are also of interest in the moderately big
data regime, because deterministic algorithms provide interpretability to the
practitioner by having no failure probability and always returning the same
results.
  We provide provable guarantees for deterministic column sampling using ridge
leverage scores. The matrix sketch returned by our algorithm is a column subset
of the original matrix, yielding additional interpretability. Like the
randomized counterparts, the deterministic algorithm provides (1 + {\epsilon})
error column subset selection, (1 + {\epsilon}) error projection-cost
preservation, and an additive-multiplicative spectral bound. We also show that
under the assumption of power-law decay of ridge leverage scores, this
deterministic algorithm is provably as accurate as randomized algorithms.
  Lastly, ridge regression is frequently used to regularize ill-posed linear
least-squares problems. While ridge regression provides shrinkage for the
regression coefficients, many of the coefficients remain small but non-zero.
Performing ridge regression with the matrix sketch returned by our algorithm
and a particular regularization parameter forces coefficients to zero and has a
provable (1 + {\epsilon}) bound on the statistical risk. As such, it is an
interesting alternative to elastic net regularization.
Efficient Hardware Realization of Convolutional Neural Networks using Intra-Kernel Regular Pruning
Authors: Maurice Yang, Mahmoud Faraj, Assem Hussein, Vincent Gaudet
Date: 2018-03-15 00:00:00

The recent trend toward increasingly deep convolutional neural networks
(CNNs) leads to a higher demand of computational power and memory storage.
Consequently, the deployment of CNNs in hardware has become more challenging.
In this paper, we propose an Intra-Kernel Regular (IKR) pruning scheme to
reduce the size and computational complexity of the CNNs by removing redundant
weights at a fine-grained level. Unlike other pruning methods such as
Fine-Grained pruning, IKR pruning maintains regular kernel structures that are
exploitable in a hardware accelerator. Experimental results demonstrate up to
10x parameter reduction and 7x computational reduction at a cost of less than
1% degradation in accuracy versus the un-pruned case.
Escaping Saddles with Stochastic Gradients
Authors: Hadi Daneshmand, Jonas Kohler, Aurelien Lucchi, Thomas Hofmann
Date: 2018-03-15 00:00:00
Proceeding: ICML 2018 7

We analyze the variance of stochastic gradients along negative curvature
directions in certain non-convex machine learning models and show that
stochastic gradients exhibit a strong component along these directions.
Furthermore, we show that - contrary to the case of isotropic noise - this
variance is proportional to the magnitude of the corresponding eigenvalues and
not decreasing in the dimensionality. Based upon this observation we propose a
new assumption under which we show that the injection of explicit, isotropic
noise usually applied to make gradient descent escape saddle points can
successfully be replaced by a simple SGD step. Additionally - and under the
same condition - we derive the first convergence rate for plain SGD to a
second-order stationary point in a number of iterations that is independent of
the problem dimension.
Local Distance Metric Learning for Nearest Neighbor Algorithm
Authors: Hossein Rajabzadeh, Mansoor Zolghadri Jahromi, Mohammad Sadegh Zare, Mostafa Fakhrahmad
Date: 2018-03-05 00:00:00
Tasks: Metric Learning

Distance metric learning is a successful way to enhance the performance of
the nearest neighbor classifier. In most cases, however, the distribution of
data does not obey a regular form and may change in different parts of the
feature space. Regarding that, this paper proposes a novel local distance
metric learning method, namely Local Mahalanobis Distance Learning (LMDL), in
order to enhance the performance of the nearest neighbor classifier. LMDL
considers the neighborhood influence and learns multiple distance metrics for a
reduced set of input samples. The reduced set is called as prototypes which try
to preserve local discriminative information as much as possible. The proposed
LMDL can be kernelized very easily, which is significantly desirable in the
case of highly nonlinear data. The quality as well as the efficiency of the
proposed method assesses through a set of different experiments on various
datasets and the obtained results show that LDML as well as the kernelized
version is superior to the other related state-of-the-art methods.
EEG machine learning with Higuchi fractal dimension and Sample Entropy as features for successful detection of depression
Authors: Milena Cukic, David Pokrajac, Miodrag Stokic, slobodan Simic, Vlada Radivojevic, Milos Ljubisavljevic
Date: 2018-03-15 00:00:00
Tasks: BIG-bench Machine Learning, EEG, Electroencephalogram (EEG)

Reliable diagnosis of depressive disorder is essential for both optimal
treatment and prevention of fatal outcomes. In this study, we aimed to
elucidate the effectiveness of two non-linear measures, Higuchi Fractal
Dimension (HFD) and Sample Entropy (SampEn), in detecting depressive disorders
when applied on EEG. HFD and SampEn of EEG signals were used as features for
seven machine learning algorithms including Multilayer Perceptron, Logistic
Regression, Support Vector Machines with the linear and polynomial kernel,
Decision Tree, Random Forest, and Naive Bayes classifier, discriminating EEG
between healthy control subjects and patients diagnosed with depression. We
confirmed earlier observations that both non-linear measures can discriminate
EEG signals of patients from healthy control subjects. The results suggest that
good classification is possible even with a small number of principal
components. Average accuracy among classifiers ranged from 90.24% to 97.56%.
Among the two measures, SampEn had better performance. Using HFD and SampEn and
a variety of machine learning techniques we can accurately discriminate
patients diagnosed with depression vs controls which can serve as a highly
sensitive, clinically relevant marker for the diagnosis of depressive
disorders.
Deep Co-Training for Semi-Supervised Image Recognition
Authors: Siyuan Qiao, Wei Shen, Zhishuai Zhang, Bo wang, Alan Yuille
Date: 2018-03-15 00:00:00
Proceeding: ECCV 2018 9

In this paper, we study the problem of semi-supervised image recognition,
which is to learn classifiers using both labeled and unlabeled images. We
present Deep Co-Training, a deep learning based method inspired by the
Co-Training framework. The original Co-Training learns two classifiers on two
views which are data from different sources that describe the same instances.
To extend this concept to deep learning, Deep Co-Training trains multiple deep
neural networks to be the different views and exploits adversarial examples to
encourage view difference, in order to prevent the networks from collapsing
into each other. As a result, the co-trained networks provide different and
complementary information about the data, which is necessary for the
Co-Training framework to achieve good results. We test our method on SVHN,
CIFAR-10/100 and ImageNet datasets, and our method outperforms the previous
state-of-the-art methods by a large margin.
Unraveling Go gaming nature by Ising Hamiltonian and common fate graphs: tactics and statistics
Authors: Didier Barradas-Bautista, Matías Alvarado
Date: 2018-03-15 00:00:00

Go gaming is a struggle between adversaries, black and white simple stones,
and aim to control the most Go board territory for success. Rules are simple
but Go game fighting is highly intricate. Stones placement and interaction on
board is random-appearance, likewise interaction phenomena among basic elements
in physics thermodynamics, chemistry, biology, or social issues. We model the
Go game dynamic employing an Ising model energy function, whose interaction
coefficients reflect the application of rules and tactics to build long-term
strategies. At any step of the game, the energy function of the model assesses
the control and strength of a player over the board. A close fit between
predictions of the model with actual game's scores is obtained. AlphaGo
computer is the current top Go player, but its behavior does not wholly reveal
the Go gaming nature. The Ising function allows for precisely model the
stochastic evolutions of Go gaming patterns, so, to advance the understanding
on Go own-dynamic -beyond the player`s abilities. The analysis of the frequency
and combination of tactics shows the formation of patterns in the groups of
stones during a game, regarding the turn of each player, or if human or
computer adversaries are confronted.
Deep Choice Model Using Pointer Networks for Airline Itinerary Prediction
Authors: Alejandro Mottini, Rodrigo Acuna-Agost
Date: 2018-03-15 00:00:00

Travel providers such as airlines and on-line travel agents are becoming more
and more interested in understanding how passengers choose among alternative
itineraries when searching for flights. This knowledge helps them better
display and adapt their offer, taking into account market conditions and
customer needs. Some common applications are not only filtering and sorting
alternatives, but also changing certain attributes in real-time (e.g., changing
the price). In this paper, we concentrate with the problem of modeling air
passenger choices of flight itineraries. This problem has historically been
tackled using classical Discrete Choice Modelling techniques. Traditional
statistical approaches, in particular the Multinomial Logit model (MNL), is
widely used in industrial applications due to its simplicity and general good
performance. However, MNL models present several shortcomings and assumptions
that might not hold in real applications. To overcome these difficulties, we
present a new choice model based on Pointer Networks. Given an input sequence,
this type of deep neural architecture combines Recurrent Neural Networks with
the Attention Mechanism to learn the conditional probability of an output whose
values correspond to positions in an input sequence. Therefore, given a
sequence of different alternatives presented to a customer, the model can learn
to point to the one most likely to be chosen by the customer. The proposed
method was evaluated on a real dataset that combines on-line user search logs
and airline flight bookings. Experimental results show that the proposed model
outperforms the traditional MNL model on several metrics.
Analyzing Roles of Classifiers and Code-Mixed factors for Sentiment Identification
Authors: Soumil Mandal, Dipankar Das
Date: 2018-01-08 00:00:00

Multilingual speakers often switch between languages to express themselves on
social communication platforms. Sometimes, the original script of the language
is preserved, while using a common script for all the languages is quite
popular as well due to convenience. On such occasions, multiple languages are
being mixed with different rules of grammar, using the same script which makes
it a challenging task for natural language processing even in case of accurate
sentiment identification. In this paper, we report results of various
experiments carried out on movie reviews dataset having this code-mixing
property of two languages, English and Bengali, both typed in Roman script. We
have tested various machine learning algorithms trained only on English
features on our code-mixed data and have achieved the maximum accuracy of
59.00% using Naive Bayes (NB) model. We have also tested various models trained
on code-mixed data, as well as English features and the highest accuracy of
72.50% was obtained by a Support Vector Machine (SVM) model. Finally, we have
analyzed the misclassified snippets and have discussed the challenges needed to
be resolved for better accuracy.
Studying Invariances of Trained Convolutional Neural Networks
Authors: Charlotte Bunne, Lukas Rahmann, Thomas Wolf
Date: 2018-03-15 00:00:00
Tasks: General Classification, image-classification, Image Classification

Convolutional Neural Networks (CNNs) define an exceptionally powerful class
of models for image classification, but the theoretical background and the
understanding of how invariances to certain transformations are learned is
limited. In a large scale screening with images modified by different affine
and nonaffine transformations of varying magnitude, we analyzed the behavior of
the CNN architectures AlexNet and ResNet. If the magnitude of different
transformations does not exceed a class- and transformation dependent
threshold, both architectures show invariant behavior. In this work we
furthermore introduce a new learnable module, the Invariant Transformer Net,
which enables us to learn differentiable parameters for a set of affine
transformations. This allows us to extract the space of transformations to
which the CNN is invariant and its class prediction robust.
Mo2Cap2: Real-time Mobile 3D Motion Capture with a Cap-mounted Fisheye Camera
Authors: Weipeng Xu, Avishek Chatterjee, Michael Zollhoefer, Helge Rhodin, Pascal Fua, Hans-Peter Seidel, Christian Theobalt
Date: 2018-03-15 00:00:00
Tasks: 3D Pose Estimation, Egocentric Pose Estimation, GPU, Pose Estimation

We propose the first real-time approach for the egocentric estimation of 3D
human body pose in a wide range of unconstrained everyday activities. This
setting has a unique set of challenges, such as mobility of the hardware setup,
and robustness to long capture sessions with fast recovery from tracking
failures. We tackle these challenges based on a novel lightweight setup that
converts a standard baseball cap to a device for high-quality pose estimation
based on a single cap-mounted fisheye camera. From the captured egocentric live
stream, our CNN based 3D pose estimation approach runs at 60Hz on a
consumer-level GPU. In addition to the novel hardware setup, our other main
contributions are: 1) a large ground truth training corpus of top-down fisheye
images and 2) a novel disentangled 3D pose estimation approach that takes the
unique properties of the egocentric viewpoint into account. As shown by our
evaluation, we achieve lower 3D joint error as well as better 2D overlay than
the existing baselines.
An Introduction to Deep Visual Explanation
Authors: Housam Khalifa Bashier Babiker, Randy Goebel
Date: 2017-11-26 00:00:00
Tasks: Deep Learning, General Classification, image-classification, Image Classification

The practical impact of deep learning on complex supervised learning problems
has been significant, so much so that almost every Artificial Intelligence
problem, or at least a portion thereof, has been somehow recast as a deep
learning problem. The applications appeal is significant, but this appeal is
increasingly challenged by what some call the challenge of explainability, or
more generally the more traditional challenge of debuggability: if the outcomes
of a deep learning process produce unexpected results (e.g., less than expected
performance of a classifier), then there is little available in the way of
theories or tools to help investigate the potential causes of such unexpected
behavior, especially when this behavior could impact people's lives. We
describe a preliminary framework to help address this issue, which we call
"deep visual explanation" (DVE). "Deep," because it is the development and
performance of deep neural network models that we want to understand. "Visual,"
because we believe that the most rapid insight into a complex multi-dimensional
model is provided by appropriate visualization techniques, and "Explanation,"
because in the spectrum from instrumentation by inserting print statements to
the abductive inference of explanatory hypotheses, we believe that the key to
understanding deep learning relies on the identification and exposure of
hypotheses about the performance behavior of a learned deep model. In the
exposition of our preliminary framework, we use relatively straightforward
image classification examples and a variety of choices on initial configuration
of a deep model building scenario. By careful but not complicated
instrumentation, we expose classification outcomes of deep models using
visualization, and also show initial results for one potential application of
interpretability.
Analog simulator of integro-differential equations with classical memristors
Authors: G. Alvarado Barrios, J. C. Retamal, E. Solano, M. Sanz
Date: 2018-03-15 00:00:00

An analog computer makes use of continuously changeable quantities of a system, such as its electrical, mechanical, or hydraulic properties, to solve a given problem. While these devices are usually computationally more powerful than their digital counterparts, they suffer from analog noise which does not allow for error control. We will focus on analog computers based on active electrical networks comprised of resistors, capacitors, and operational amplifiers which are capable of simulating any linear ordinary differential equation. However, the class of nonlinear dynamics they can solve is limited. In this work, by adding memristors to the electrical network, we show that the analog computer can simulate a large variety of linear and nonlinear integro-differential equations by carefully choosing the conductance and the dynamics of the memristor state variable. To the best of our knowledge, this is the first time that circuits based on memristors are proposed for simulations. We study the performance of these analog computers by simulating integro-differential models related to fluid dynamics, nonlinear Volterra equations for population growth, and quantum models describing non-Markovian memory effects, among others. Finally, we perform stability tests by considering imperfect analog components, obtaining robust solutions with up to $13\%$ relative error for relevant timescales.
Smartphone picture organization: A hierarchical approach
Authors: Stefan Lonn, Petia Radeva, Mariella Dimiccoli
Date: 2018-03-15 00:00:00

We live in a society where the large majority of the population has a camera-equipped smartphone. In addition, hard drives and cloud storage are getting cheaper and cheaper, leading to a tremendous growth in stored personal photos. Unlike photo collections captured by a digital camera, which typically are pre-processed by the user who organizes them into event-related folders, smartphone pictures are automatically stored in the cloud. As a consequence, photo collections captured by a smartphone are highly unstructured and because smartphones are ubiquitous, they present a larger variability compared to pictures captured by a digital camera. To solve the need of organizing large smartphone photo collections automatically, we propose here a new methodology for hierarchical photo organization into topics and topic-related categories. Our approach successfully estimates latent topics in the pictures by applying probabilistic Latent Semantic Analysis, and automatically assigns a name to each topic by relying on a lexical database. Topic-related categories are then estimated by using a set of topic-specific Convolutional Neuronal Networks. To validate our approach, we ensemble and make public a large dataset of more than 8,000 smartphone pictures from 10 persons. Experimental results demonstrate better user satisfaction with respect to state of the art solutions in terms of organization.
RankME: Reliable Human Ratings for Natural Language Generation
Authors: Jekaterina Novikova, Ondřej Dušek, Verena Rieser
Date: 2018-03-15 00:00:00
Proceeding: NAACL 2018 6
Tasks: Attribute, Experimental Design, Text Generation

Human evaluation for natural language generation (NLG) often suffers from
inconsistent user ratings. While previous research tends to attribute this
problem to individual user preferences, we show that the quality of human
judgements can also be improved by experimental design. We present a novel
rank-based magnitude estimation method (RankME), which combines the use of
continuous scales and relative assessments. We show that RankME significantly
improves the reliability and consistency of human ratings compared to
traditional evaluation methods. In addition, we show that it is possible to
evaluate NLG systems according to multiple, distinct criteria, which is
important for error analysis. Finally, we demonstrate that RankME, in
combination with Bayesian estimation of system quality, is a cost-effective
alternative for ranking multiple NLG systems.
Toolflows for Mapping Convolutional Neural Networks on FPGAs: A Survey and Future Directions
Authors: Stylianos I. Venieris, Alexandros Kouris, Christos-Savvas Bouganis
Date: 2018-03-15 00:00:00

In the past decade, Convolutional Neural Networks (CNNs) have demonstrated
state-of-the-art performance in various Artificial Intelligence tasks. To
accelerate the experimentation and development of CNNs, several software
frameworks have been released, primarily targeting power-hungry CPUs and GPUs.
In this context, reconfigurable hardware in the form of FPGAs constitutes a
potential alternative platform that can be integrated in the existing deep
learning ecosystem to provide a tunable balance between performance, power
consumption and programmability. In this paper, a survey of the existing
CNN-to-FPGA toolflows is presented, comprising a comparative study of their key
characteristics which include the supported applications, architectural
choices, design space exploration methods and achieved performance. Moreover,
major challenges and objectives introduced by the latest trends in CNN
algorithmic research are identified and presented. Finally, a uniform
evaluation methodology is proposed, aiming at the comprehensive, complete and
in-depth evaluation of CNN-to-FPGA toolflows.
Contrasting information theoretic decompositions of modulatory and arithmetic interactions in neural information processing systems
Authors: Jim W. Kay, William A. Phillips
Date: 2018-03-15 00:00:00

Biological and artificial neural systems are composed of many local
processors, and their capabilities depend upon the transfer function that
relates each local processor's outputs to its inputs. This paper uses a recent
advance in the foundations of information theory to study the properties of
local processors that use contextual input to amplify or attenuate transmission
of information about their driving inputs. This advance enables the information
transmitted by processors with two distinct inputs to be decomposed into those
components unique to each input, that shared between the two inputs, and that
which depends on both though it is in neither, i.e. synergy. The decompositions
that we report here show that contextual modulation has information processing
properties that contrast with those of all four simple arithmetic operators,
that it can take various forms, and that the form used in our previous studies
of artificial neural nets composed of local processors with both driving and
contextual inputs is particularly well-suited to provide the distinctive
capabilities of contextual modulation under a wide range of conditions. We
argue that the decompositions reported here could be compared with those
obtained from empirical neurobiological and psychophysical data under
conditions thought to reflect contextual modulation. That would then shed new
light on the underlying processes involved. Finally, we suggest that such
decompositions could aid the design of context-sensitive machine learning
algorithms.