pwc-archive/papers-with-abstracts huggingface dataset:
"Attention" for Detecting Unreliable News in the Information Age
Authors: Venkatesh Duppada
Date: 2017-11-03 00:00:00
Tasks: Articles

An Unreliable news is any piece of information which is false or misleading,
deliberately spread to promote political, ideological and financial agendas.
Recently the problem of unreliable news has got a lot of attention as the
number instances of using news and social media outlets for propaganda have
increased rapidly. This poses a serious threat to society, which calls for
technology to automatically and reliably identify unreliable news sources. This
paper is an effort made in this direction to build systems for detecting
unreliable news articles. In this paper, various NLP algorithms were built and
evaluated on Unreliable News Data 2017 dataset. Variants of hierarchical
attention networks (HAN) are presented for encoding and classifying news
articles which achieve the best results of 0.944 ROC-AUC. Finally, Attention
layer weights are visualized to understand and give insight into the decisions
made by HANs. The results obtained are very promising and encouraging to deploy
and use these systems in the real world to mitigate the problem of unreliable
news.
Decentralised firewall for malware detection
Authors: Saurabh Raje, Shyamal Vaderia, Neil Wilson, Rudrakh Panigrahi
Date: 2017-11-03 00:00:00
Tasks: General Classification, Malware Detection

This paper describes the design and development of a decentralized firewall
system powered by a novel malware detection engine. The firewall is built using
blockchain technology. The detection engine aims to classify Portable
Executable (PE) files as malicious or benign. File classification is carried
out using a deep belief neural network (DBN) as the detection engine. Our
approach is to model the files as grayscale images and use the DBN to classify
those images into the aforementioned two classes. An extensive data set of
10,000 files is used to train the DBN. Validation is carried out using 4,000
files previously unexposed to the network. The final result of whether to allow
or block a file is obtained by arriving at a proof of work based consensus in
the blockchain network.
Spectral Mixture Kernels for Multi-Output Gaussian Processes
Authors: Gabriel Parra, Felipe Tobar
Date: 2017-09-05 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Gaussian Processes

Early approaches to multiple-output Gaussian processes (MOGPs) relied on
linear combinations of independent, latent, single-output Gaussian processes
(GPs). This resulted in cross-covariance functions with limited parametric
interpretation, thus conflicting with the ability of single-output GPs to
understand lengthscales, frequencies and magnitudes to name a few. On the
contrary, current approaches to MOGP are able to better interpret the
relationship between different channels by directly modelling the
cross-covariances as a spectral mixture kernel with a phase shift. We extend
this rationale and propose a parametric family of complex-valued cross-spectral
densities and then build on Cram\'er's Theorem (the multivariate version of
Bochner's Theorem) to provide a principled approach to design multivariate
covariance functions. The so-constructed kernels are able to model delays among
channels in addition to phase differences and are thus more expressive than
previous methods, while also providing full parametric interpretation of the
relationship across channels. The proposed method is first validated on
synthetic data and then compared to existing MOGP methods on two real-world
examples.
Automatic Differentiation for Tensor Algebras
Authors: Sebastian Urban, Patrick van der Smagt
Date: 2017-11-03 00:00:00
Tasks: tensor algebra

Kjolstad et. al. proposed a tensor algebra compiler. It takes expressions
that define a tensor element-wise, such as $f_{ij}(a,b,c,d) =
\exp\left[-\sum_{k=0}^4 \left((a_{ik}+b_{jk})^2\, c_{ii} + d_{i+k}^3 \right)
\right]$, and generates the corresponding compute kernel code.
  For machine learning, especially deep learning, it is often necessary to
compute the gradient of a loss function $l(a,b,c,d)=l(f(a,b,c,d))$ with respect
to parameters $a,b,c,d$. If tensor compilers are to be applied in this field,
it is necessary to derive expressions for the derivatives of element-wise
defined tensors, i.e. expressions for $(da)_{ik}=\partial l/\partial a_{ik}$.
  When the mapping between function indices and argument indices is not 1:1,
special attention is required. For the function $f_{ij} (x) = x_i^2$, the
derivative of the loss is $(dx)_i=\partial l/\partial x_i=\sum_j
(df)_{ij}2x_i$; the sum is necessary because index $j$ does not appear in the
indices of $f$. Another example is $f_{i}(x)=x_{ii}^2$, where $x$ is a matrix;
here we have $(dx)_{ij}=\delta_{ij}(df)_i2x_{ii}$; the Kronecker delta is
necessary because the derivative is zero for off-diagonal elements. Another
indexing scheme is used by $f_{ij}(x)=\exp x_{i+j}$; here the correct
derivative is $(dx)_{k}=\sum_i (df)_{i,k-i} \exp x_{k}$, where the range of the
sum must be chosen appropriately.
  In this publication we present an algorithm that can handle any case in which
the indices of an argument are an arbitrary linear combination of the indices
of the function, thus all the above examples can be handled. Sums (and their
ranges) and Kronecker deltas are automatically inserted into the derivatives as
necessary. Additionally, the indices are transformed, if required (as in the
last example). The algorithm outputs a symbolic expression that can be
subsequently fed into a tensor algebra compiler.
  Source code is provided.
Computationally efficient cardiac views projection using 3D Convolutional Neural Networks
Authors: Matthieu Le, Jesse Lieman-Sifry, Felix Lau, Sean Sall, Albert Hsiao, Daniel Golden
Date: 2017-11-03 00:00:00
Tasks: Diagnostic

4D Flow is an MRI sequence which allows acquisition of 3D images of the
heart. The data is typically acquired volumetrically, so it must be reformatted
to generate cardiac long axis and short axis views for diagnostic
interpretation. These views may be generated by placing 6 landmarks: the left
and right ventricle apex, and the aortic, mitral, pulmonary, and tricuspid
valves. In this paper, we propose an automatic method to localize landmarks in
order to compute the cardiac views. Our approach consists of first calculating
a bounding box that tightly crops the heart, followed by a landmark
localization step within this bounded region. Both steps are based on a 3D
extension of the recently introduced ENet. We demonstrate that the long and
short axis projections computed with our automated method are of equivalent
quality to projections created with landmarks placed by an experienced cardiac
radiologist, based on a blinded test administered to a different cardiac
radiologist.
Accelerating Training of Deep Neural Networks via Sparse Edge Processing
Authors: Sourya Dey, Yinan Shao, Keith M. Chugg, Peter A. Beerel
Date: 2017-11-03 00:00:00

We propose a reconfigurable hardware architecture for deep neural networks
(DNNs) capable of online training and inference, which uses algorithmically
pre-determined, structured sparsity to significantly lower memory and
computational requirements. This novel architecture introduces the notion of
edge-processing to provide flexibility and combines junction pipelining and
operational parallelization to speed up training. The overall effect is to
reduce network complexity by factors up to 30x and training time by up to 35x
relative to GPUs, while maintaining high fidelity of inference results. This
has the potential to enable extensive parameter searches and development of the
largely unexplored theoretical foundation of DNNs. The architecture
automatically adapts itself to different network sizes given available hardware
resources. As proof of concept, we show results obtained for different bit
widths.
Generalized Linear Model Regression under Distance-to-set Penalties
Authors: Jason Xu, Eric C. Chi, Kenneth Lange
Date: 2017-11-03 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: regression

Estimation in generalized linear models (GLM) is complicated by the presence
of constraints. One can handle constraints by maximizing a penalized
log-likelihood. Penalties such as the lasso are effective in high dimensions,
but often lead to unwanted shrinkage. This paper explores instead penalizing
the squared distance to constraint sets. Distance penalties are more flexible
than algebraic and regularization penalties, and avoid the drawback of
shrinkage. To optimize distance penalized objectives, we make use of the
majorization-minimization principle. Resulting algorithms constructed within
this framework are amenable to acceleration and come with global convergence
guarantees. Applications to shape constraints, sparse regression, and
rank-restricted matrix regression on synthetic and real data showcase strong
empirical performance, even under non-convex constraints.
Fine-tuning CNN Image Retrieval with No Human Annotation
Authors: Filip Radenović, Giorgos Tolias, Ondřej Chum
Date: 2017-11-03 00:00:00
Tasks: Image Retrieval, Retrieval

Image descriptors based on activations of Convolutional Neural Networks
(CNNs) have become dominant in image retrieval due to their discriminative
power, compactness of representation, and search efficiency. Training of CNNs,
either from scratch or fine-tuning, requires a large amount of annotated data,
where a high quality of annotation is often crucial. In this work, we propose
to fine-tune CNNs for image retrieval on a large collection of unordered images
in a fully automated manner. Reconstructed 3D models obtained by the
state-of-the-art retrieval and structure-from-motion methods guide the
selection of the training data. We show that both hard-positive and
hard-negative examples, selected by exploiting the geometry and the camera
positions available from the 3D models, enhance the performance of
particular-object retrieval. CNN descriptor whitening discriminatively learned
from the same training data outperforms commonly used PCA whitening. We propose
a novel trainable Generalized-Mean (GeM) pooling layer that generalizes max and
average pooling and show that it boosts retrieval performance. Applying the
proposed method to the VGG network achieves state-of-the-art performance on the
standard benchmarks: Oxford Buildings, Paris, and Holidays datasets.
Factoring Exogenous State for Model-Free Monte Carlo
Authors: Sean McGregor, Rachel Houtman, Claire Montgomery, Ronald Metoyer, Thomas G. Dietterich
Date: 2017-03-28 00:00:00
Tasks: Management, model

Policy analysts wish to visualize a range of policies for large
simulator-defined Markov Decision Processes (MDPs). One visualization approach
is to invoke the simulator to generate on-policy trajectories and then
visualize those trajectories. When the simulator is expensive, this is not
practical, and some method is required for generating trajectories for new
policies without invoking the simulator. The method of Model-Free Monte Carlo
(MFMC) can do this by stitching together state transitions for a new policy
based on previously-sampled trajectories from other policies. This "off-policy
Monte Carlo simulation" method works well when the state space has low
dimension but fails as the dimension grows. This paper describes a method for
factoring out some of the state and action variables so that MFMC can work in
high-dimensional MDPs. The new method, MFMCi, is evaluated on a very
challenging wildfire management MDP.
Fault Detection of Broken Rotor Bar in LS-PMSM Using Random Forests
Authors: Juan C. Quiroz, Norman Mariun, Mohammad Rezazadeh Mehrjou, Mahdi Izadi, Norhisam Misron, Mohd Amran Mohd Radzi
Date: 2017-11-03 00:00:00
Tasks: Diagnostic, Fault Detection, feature selection

This paper proposes a new approach to diagnose broken rotor bar failure in a
line start-permanent magnet synchronous motor (LS-PMSM) using random forests.
The transient current signal during the motor startup was acquired from a
healthy motor and a faulty motor with a broken rotor bar fault. We extracted 13
statistical time domain features from the startup transient current signal, and
used these features to train and test a random forest to determine whether the
motor was operating under normal or faulty conditions. For feature selection,
we used the feature importances from the random forest to reduce the number of
features to two features. The results showed that the random forest classifies
the motor condition as healthy or faulty with an accuracy of 98.8% using all
features and with an accuracy of 98.4% by using only the mean-index and
impulsion features. The performance of the random forest was compared with a
decision tree, Na\"ive Bayes classifier, logistic regression, linear ridge, and
a support vector machine, with the random forest consistently having a higher
accuracy than the other algorithms. The proposed approach can be used in
industry for online monitoring and fault diagnostic of LS-PMSM motors and the
results can be helpful for the establishment of preventive maintenance plans in
factories.
Optimal Errors and Phase Transitions in High-Dimensional Generalized Linear Models
Authors: Jean Barbier, Florent Krzakala, Nicolas Macris, Léo Miolane, Lenka Zdeborová
Date: 2017-08-10 00:00:00
Tasks: compressed sensing, Vocal Bursts Intensity Prediction

Generalized linear models (GLMs) arise in high-dimensional machine learning,
statistics, communications and signal processing. In this paper we analyze GLMs
when the data matrix is random, as relevant in problems such as compressed
sensing, error-correcting codes or benchmark models in neural networks. We
evaluate the mutual information (or "free entropy") from which we deduce the
Bayes-optimal estimation and generalization errors. Our analysis applies to the
high-dimensional limit where both the number of samples and the dimension are
large and their ratio is fixed. Non-rigorous predictions for the optimal errors
existed for special cases of GLMs, e.g. for the perceptron, in the field of
statistical physics based on the so-called replica method. Our present paper
rigorously establishes those decades old conjectures and brings forward their
algorithmic interpretation in terms of performance of the generalized
approximate message-passing algorithm. Furthermore, we tightly characterize,
for many learning problems, regions of parameters for which this algorithm
achieves the optimal performance, and locate the associated sharp phase
transitions separating learnable and non-learnable regions. We believe that
this random version of GLMs can serve as a challenging benchmark for
multi-purpose algorithms. This paper is divided in two parts that can be read
independently: The first part (main part) presents the model and main results,
discusses some applications and sketches the main ideas of the proof. The
second part (supplementary informations) is much more detailed and provides
more examples as well as all the proofs.
Fisher GAN
Authors: Youssef Mroueh, Tom Sercu
Date: 2017-05-26 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: General Classification

Generative Adversarial Networks (GANs) are powerful models for learning
complex distributions. Stable training of GANs has been addressed in many
recent works which explore different metrics between distributions. In this
paper we introduce Fisher GAN which fits within the Integral Probability
Metrics (IPM) framework for training GANs. Fisher GAN defines a critic with a
data dependent constraint on its second order moments. We show in this paper
that Fisher GAN allows for stable and time efficient training that does not
compromise the capacity of the critic, and does not need data independent
constraints such as weight clipping. We analyze our Fisher IPM theoretically
and provide an algorithm based on Augmented Lagrangian for Fisher GAN. We
validate our claims on both image sample generation and semi-supervised
classification using Fisher GAN.
On Fairness and Calibration
Authors: Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, Kilian Q. Weinberger
Date: 2017-09-06 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Fairness, General Classification

The machine learning community has become increasingly concerned with the
potential for bias and discrimination in predictive models. This has motivated
a growing line of work on what it means for a classification procedure to be
"fair." In this paper, we investigate the tension between minimizing error
disparity across different population groups while maintaining calibrated
probability estimates. We show that calibration is compatible only with a
single error constraint (i.e. equal false-negatives rates across groups), and
show that any algorithm that satisfies this relaxation is no better than
randomizing a percentage of predictions for an existing classifier. These
unsettling findings, which extend and generalize existing results, are
empirically confirmed on several datasets.
Discovering More Precise Process Models from Event Logs by Filtering Out Chaotic Activities
Authors: Niek Tax, Natalia Sidorova, Wil M. P. van der Aalst
Date: 2017-11-03 00:00:00
Tasks: Management

Process Discovery is concerned with the automatic generation of a process
model that describes a business process from execution data of that business
process. Real life event logs can contain chaotic activities. These activities
are independent of the state of the process and can, therefore, happen at
rather arbitrary points in time. We show that the presence of such chaotic
activities in an event log heavily impacts the quality of the process models
that can be discovered with process discovery techniques. The current modus
operandi for filtering activities from event logs is to simply filter out
infrequent activities. We show that frequency-based filtering of activities
does not solve the problems that are caused by chaotic activities. Moreover, we
propose a novel technique to filter out chaotic activities from event logs. We
evaluate this technique on a collection of seventeen real-life event logs that
originate from both the business process management domain and the smart home
environment domain. As demonstrated, the developed activity filtering methods
enable the discovery of process models that are more behaviorally specific
compared to process models that are discovered using standard frequency-based
filtering.
Mandolin: A Knowledge Discovery Framework for the Web of Data
Authors: Tommaso Soru, Diego Esteves, Edgard Marx, Axel-Cyrille Ngonga Ngomo
Date: 2017-11-03 00:00:00
Tasks: Link Prediction, Relational Reasoning

Markov Logic Networks join probabilistic modeling with first-order logic and
have been shown to integrate well with the Semantic Web foundations. While
several approaches have been devised to tackle the subproblems of rule mining,
grounding, and inference, no comprehensive workflow has been proposed so far.
In this paper, we fill this gap by introducing a framework called Mandolin,
which implements a workflow for knowledge discovery specifically on RDF
datasets. Our framework imports knowledge from referenced graphs, creates
similarity relationships among similar literals, and relies on state-of-the-art
techniques for rule mining, grounding, and inference computation. We show that
our best configuration scales well and achieves at least comparable results
with respect to other statistical-relational-learning algorithms on link
prediction.
Distributed Unmixing of Hyperspectral Data With Sparsity Constraint
Authors: Sara Khoshsokhan, Roozbeh Rajabi, Hadi Zayyani
Date: 2017-11-03 00:00:00
Tasks: Distributed Optimization

Spectral unmixing (SU) is a data processing problem in hyperspectral remote
sensing. The significant challenge in the SU problem is how to identify
endmembers and their weights, accurately. For estimation of signature and
fractional abundance matrices in a blind problem, nonnegative matrix
factorization (NMF) and its developments are used widely in the SU problem. One
of the constraints which was added to NMF is sparsity constraint that was
regularized by L 1/2 norm. In this paper, a new algorithm based on distributed
optimization has been used for spectral unmixing. In the proposed algorithm, a
network including single-node clusters has been employed. Each pixel in
hyperspectral images considered as a node in this network. The distributed
unmixing with sparsity constraint has been optimized with diffusion LMS
strategy, and then the update equations for fractional abundance and signature
matrices are obtained. Simulation results based on defined performance metrics,
illustrate advantage of the proposed algorithm in spectral unmixing of
hyperspectral data compared with other methods. The results show that the AAD
and SAD of the proposed approach are improved respectively about 6 and 27
percent toward distributed unmixing in SNR=25dB.
Optimization by a quantum reinforcement algorithm
Authors: A. Ramezanpour
Date: 2017-06-13 00:00:00

A reinforcement algorithm solves a classical optimization problem by
introducing a feedback to the system which slowly changes the energy landscape
and converges the algorithm to an optimal solution in the configuration space.
Here, we use this strategy to concentrate (localize) preferentially the wave
function of a quantum particle, which explores the configuration space of the
problem, on an optimal configuration. We examine the method by solving
numerically the equations governing the evolution of the system, which are
similar to the nonlinear Schr\"odinger equations, for small problem sizes. In
particular, we observe that reinforcement increases the minimal energy gap of
the system in a quantum annealing algorithm. Our numerical simulations and the
latter observation show that such kind of quantum feedbacks might be helpful in
solving a computationally hard optimization problem by a quantum reinforcement
algorithm.
Good Semi-supervised Learning that Requires a Bad GAN
Authors: Zihang Dai, Zhilin Yang, Fan Yang, William W. Cohen, Ruslan Salakhutdinov
Date: 2017-05-27 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: General Classification, Semi-Supervised Image Classification

Semi-supervised learning methods based on generative adversarial networks
(GANs) obtained strong empirical results, but it is not clear 1) how the
discriminator benefits from joint training with a generator, and 2) why good
semi-supervised classification performance and a good generator cannot be
obtained at the same time. Theoretically, we show that given the discriminator
objective, good semisupervised learning indeed requires a bad generator, and
propose the definition of a preferred generator. Empirically, we derive a novel
formulation based on our analysis that substantially improves over feature
matching GANs, obtaining state-of-the-art results on multiple benchmark
datasets.