pwc-archive/papers-with-abstracts huggingface dataset:
Knowledge Engineering for Hybrid Deductive Databases
Authors: Dietmar Seipel
Date: 2017-01-03 00:00:00

Modern knowledge base systems frequently need to combine a collection of
databases in different formats: e.g., relational databases, XML databases, rule
bases, ontologies, etc. In the deductive database system DDBASE, we can manage
these different formats of knowledge and reason about them. Even the file
systems on different computers can be part of the knowledge base. Often, it is
necessary to handle different versions of a knowledge base. E.g., we might want
to find out common parts or differences of two versions of a relational
database.
  We will examine the use of abstractions of rule bases by predicate dependency
and rule predicate graphs. Also the proof trees of derived atoms can help to
compare different versions of a rule base. Moreover, it might be possible to
have derivations joining rules with other formalisms of knowledge
representation.
  Ontologies have shown their benefits in many applications of intelligent
systems, and there have been many proposals for rule languages compatible with
the semantic web stack, e.g., SWRL, the semantic web rule language. Recently,
ontologies are used in hybrid systems for specifying the provenance of the
different components.
Visualization Regularizers for Neural Network based Image Recognition
Authors: Biswajit Paria, Vikas Reddy, Anirban Santara, Pabitra Mitra
Date: 2016-04-10 00:00:00
Tasks: General Classification

The success of deep neural networks is mostly due their ability to learn
meaningful features from the data. Features learned in the hidden layers of
deep neural networks trained in computer vision tasks have been shown to be
similar to mid-level vision features. We leverage this fact in this work and
propose the visualization regularizer for image tasks. The proposed
regularization technique enforces smoothness of the features learned by hidden
nodes and turns out to be a special case of Tikhonov regularization. We achieve
higher classification accuracy as compared to existing regularizers such as the
L2 norm regularizer and dropout, on benchmark datasets without changing the
training computational complexity.
Akid: A Library for Neural Network Research and Production from a Dataism Approach
Authors: Shuai Li
Date: 2017-01-03 00:00:00
Tasks: Distributed Computing, GPU

Neural networks are a revolutionary but immature technique that is fast
evolving and heavily relies on data. To benefit from the newest development and
newly available data, we want the gap between research and production as small
as possibly. On the other hand, differing from traditional machine learning
models, neural network is not just yet another statistic model, but a model for
the natural processing engine --- the brain. In this work, we describe a neural
network library named {\texttt akid}. It provides higher level of abstraction
for entities (abstracted as blocks) in nature upon the abstraction done on
signals (abstracted as tensors) by Tensorflow, characterizing the dataism
observation that all entities in nature processes input and emit out in some
ways. It includes a full stack of software that provides abstraction to let
researchers focus on research instead of implementation, while at the same time
the developed program can also be put into production seamlessly in a
distributed environment, and be production ready. At the top application stack,
it provides out-of-box tools for neural network applications. Lower down, akid
provides a programming paradigm that lets user easily build customized models.
The distributed computing stack handles the concurrency and communication, thus
letting models be trained or deployed to a single GPU, multiple GPUs, or a
distributed environment without affecting how a model is specified in the
programming paradigm stack. Lastly, the distributed deployment stack handles
how the distributed computing is deployed, thus decoupling the research
prototype environment with the actual production environment, and is able to
dynamically allocate computing resources, so development (Devs) and operations
(Ops) could be separated. Please refer to http://akid.readthedocs.io/en/latest/
for documentation.
Towards multiple kernel principal component analysis for integrative analysis of tumor samples
Authors: Nora K. Speicher, Nico Pfeifer
Date: 2017-01-02 00:00:00
Tasks: Clustering, Data Integration

Personalized treatment of patients based on tissue-specific cancer subtypes
has strongly increased the efficacy of the chosen therapies. Even though the
amount of data measured for cancer patients has increased over the last years,
most cancer subtypes are still diagnosed based on individual data sources (e.g.
gene expression data). We propose an unsupervised data integration method based
on kernel principal component analysis. Principal component analysis is one of
the most widely used techniques in data analysis. Unfortunately, the
straight-forward multiple-kernel extension of this method leads to the use of
only one of the input matrices, which does not fit the goal of gaining
information from all data sources. Therefore, we present a scoring function to
determine the impact of each input matrix. The approach enables visualizing the
integrated data and subsequent clustering for cancer subtype identification.
Due to the nature of the method, no free parameters have to be set. We apply
the methodology to five different cancer data sets and demonstrate its
advantages in terms of results and usability.
Peak-Piloted Deep Network for Facial Expression Recognition
Authors: Xiangyun Zhao, Xiaodan Liang, Luoqi Liu, Teng Li, Yugang Han, Nuno Vasconcelos, Shuicheng Yan
Date: 2016-07-24 00:00:00
Tasks: Face Recognition, Facial Expression Recognition, Facial Expression Recognition (FER), Robust Face Recognition

Objective functions for training of deep networks for face-related
recognition tasks, such as facial expression recognition (FER), usually
consider each sample independently. In this work, we present a novel
peak-piloted deep network (PPDN) that uses a sample with peak expression (easy
sample) to supervise the intermediate feature responses for a sample of
non-peak expression (hard sample) of the same type and from the same subject.
The expression evolving process from non-peak expression to peak expression can
thus be implicitly embedded in the network to achieve the invariance to
expression intensities. A special purpose back-propagation procedure, peak
gradient suppression (PGS), is proposed for network training. It drives the
intermediate-layer feature responses of non-peak expression samples towards
those of the corresponding peak expression samples, while avoiding the inverse.
This avoids degrading the recognition capability for samples of peak expression
due to interference from their non-peak expression counterparts. Extensive
comparisons on two popular FER datasets, Oulu-CASIA and CK+, demonstrate the
superiority of the PPDN over state-ofthe-art FER methods, as well as the
advantages of both the network structure and the optimization strategy.
Moreover, it is shown that PPDN is a general architecture, extensible to other
tasks by proper definition of peak and non-peak samples. This is validated by
experiments that show state-of-the-art performance on pose-invariant face
recognition, using the Multi-PIE dataset.
Collaborative Filtering with Recurrent Neural Networks
Authors: Robin Devooght, Hugues Bersini
Date: 2016-08-26 00:00:00
Tasks: Collaborative Filtering, Movie Recommendation

We show that collaborative filtering can be viewed as a sequence prediction
problem, and that given this interpretation, recurrent neural networks offer
very competitive approach. In particular we study how the long short-term
memory (LSTM) can be applied to collaborative filtering, and how it compares to
standard nearest neighbors and matrix factorization methods on movie
recommendation. We show that the LSTM is competitive in all aspects, and
largely outperforms other methods in terms of item coverage and short term
predictions.
Deep Convolutional Neural Networks for Pairwise Causality
Authors: Karamjit Singh, Garima Gupta, Lovekesh Vig, Gautam Shroff, Puneet Agarwal
Date: 2017-01-03 00:00:00
Tasks: Attribute, Causal Discovery, counterfactual, Counterfactual Reasoning

Discovering causal models from observational and interventional data is an
important first step preceding what-if analysis or counterfactual reasoning. As
has been shown before, the direction of pairwise causal relations can, under
certain conditions, be inferred from observational data via standard
gradient-boosted classifiers (GBC) using carefully engineered statistical
features. In this paper we apply deep convolutional neural networks (CNNs) to
this problem by plotting attribute pairs as 2-D scatter plots that are fed to
the CNN as images. We evaluate our approach on the 'Cause- Effect Pairs' NIPS
2013 Data Challenge. We observe that a weighted ensemble of CNN with the
earlier GBC approach yields significant improvement. Further, we observe that
when less training data is available, our approach performs better than the GBC
based approach suggesting that CNN models pre-trained to determine the
direction of pairwise causal direction could have wider applicability in causal
discovery and enabling what-if or counterfactual analysis.
Shortcut Sequence Tagging
Authors: Huijia Wu, Jiajun Zhang, Cheng-qing Zong
Date: 2017-01-03 00:00:00
Tasks: POS, POS Tagging

Deep stacked RNNs are usually hard to train. Adding shortcut connections
across different layers is a common way to ease the training of stacked
networks. However, extra shortcuts make the recurrent step more complicated. To
simply the stacked architecture, we propose a framework called shortcut block,
which is a marriage of the gating mechanism and shortcuts, while discarding the
self-connected part in LSTM cell. We present extensive empirical experiments
showing that this design makes training easy and improves generalization. We
propose various shortcut block topologies and compositions to explore its
effectiveness. Based on this architecture, we obtain a 6% relatively
improvement over the state-of-the-art on CCGbank supertagging dataset. We also
get comparable results on POS tagging task.
Deviant Learning Algorithm: Learning Sparse Mismatch Representations through Time and Space
Authors: Emmanuel Ndidi Osegi, Vincent Ike Anireh
Date: 2016-09-06 00:00:00

Predictive coding (PDC) has recently attracted attention in the neuroscience
and computing community as a candidate unifying paradigm for neuronal studies
and artificial neural network implementations particularly targeted at
unsupervised learning systems. The Mismatch Negativity (MMN) has also recently
been studied in relation to PC and found to be a useful ingredient in neural
predictive coding systems. Backed by the behavior of living organisms, such
networks are particularly useful in forming spatio-temporal transitions and
invariant representations of the input world. However, most neural systems
still do not account for large number of synapses even though this has been
shown by a few machine learning researchers as an effective and very important
component of any neural system if such a system is to behave properly. Our
major point here is that PDC systems with the MMN effect in addition to a large
number of synapses can greatly improve any neural learning system's performance
and ability to make decisions in the machine world. In this paper, we propose a
novel bio-mimetic computational intelligence algorithm -- the Deviant Learning
Algorithm, inspired by these key ideas and functional properties of recent
brain-cognitive discoveries and theories. We also show by numerical experiments
guided by theoretical insights, how our invented bio-mimetic algorithm can
achieve competitive predictions even with very small problem specific data.
End-to-End Attention based Text-Dependent Speaker Verification
Authors: Shi-Xiong Zhang, Zhuo Chen, Yong Zhao, Jinyu Li, Yifan Gong
Date: 2017-01-03 00:00:00
Tasks: Speaker Verification, Text-Dependent Speaker Verification

A new type of End-to-End system for text-dependent speaker verification is
presented in this paper. Previously, using the phonetically
discriminative/speaker discriminative DNNs as feature extractors for speaker
verification has shown promising results. The extracted frame-level (DNN
bottleneck, posterior or d-vector) features are equally weighted and aggregated
to compute an utterance-level speaker representation (d-vector or i-vector). In
this work we use speaker discriminative CNNs to extract the noise-robust
frame-level features. These features are smartly combined to form an
utterance-level speaker vector through an attention mechanism. The proposed
attention model takes the speaker discriminative information and the phonetic
information to learn the weights. The whole system, including the CNN and
attention model, is joint optimized using an end-to-end criterion. The training
algorithm imitates exactly the evaluation process --- directly mapping a test
utterance and a few target speaker utterances into a single verification score.
The algorithm can automatically select the most similar impostor for each
target speaker to train the network. We demonstrated the effectiveness of the
proposed end-to-end system on Windows $10$ "Hey Cortana" speaker verification
task.
Robust and Real-time Deep Tracking Via Multi-Scale Domain Adaptation
Authors: Xinyu Wang, Hanxi Li, Yi Li, Fumin Shen, Fatih Porikli
Date: 2017-01-03 00:00:00
Tasks: Domain Adaptation, image-classification, Image Classification, Object Tracking, Visual Tracking

Visual tracking is a fundamental problem in computer vision. Recently, some
deep-learning-based tracking algorithms have been achieving record-breaking
performances. However, due to the high complexity of deep learning, most deep
trackers suffer from low tracking speed, and thus are impractical in many
real-world applications. Some new deep trackers with smaller network structure
achieve high efficiency while at the cost of significant decrease on precision.
In this paper, we propose to transfer the feature for image classification to
the visual tracking domain via convolutional channel reductions. The channel
reduction could be simply viewed as an additional convolutional layer with the
specific task. It not only extracts useful information for object tracking but
also significantly increases the tracking speed. To better accommodate the
useful feature of the target in different scales, the adaptation filters are
designed with different sizes. The yielded visual tracker is real-time and also
illustrates the state-of-the-art accuracies in the experiment involving two
well-adopted benchmarks with more than 100 test videos.
Linear Hypothesis Testing in Dense High-Dimensional Linear Models
Authors: Yinchu Zhu, Jelena Bradic
Date: 2016-10-10 00:00:00
Tasks: regression, Two-sample testing, valid, Vocal Bursts Intensity Prediction

We propose a methodology for testing linear hypothesis in high-dimensional
linear models. The proposed test does not impose any restriction on the size of
the model, i.e. model sparsity or the loading vector representing the
hypothesis. Providing asymptotically valid methods for testing general linear
functions of the regression parameters in high-dimensions is extremely
challenging -- especially without making restrictive or unverifiable
assumptions on the number of non-zero elements. We propose to test the moment
conditions related to the newly designed restructured regression, where the
inputs are transformed and augmented features. These new features incorporate
the structure of the null hypothesis directly. The test statistics are
constructed in such a way that lack of sparsity in the original model parameter
does not present a problem for the theoretical justification of our procedures.
We establish asymptotically exact control on Type I error without imposing any
sparsity assumptions on model parameter or the vector representing the linear
hypothesis. Our method is also shown to achieve certain optimality in detecting
deviations from the null hypothesis. We demonstrate the favorable finite-sample
performance of the proposed methods, via a number of numerical and a real data
example.
Permuted and Augmented Stick-Breaking Bayesian Multinomial Regression
Authors: Quan Zhang, Mingyuan Zhou
Date: 2016-12-30 00:00:00
Tasks: regression

To model categorical response variables given their covariates, we propose a
permuted and augmented stick-breaking (paSB) construction that one-to-one maps
the observed categories to randomly permuted latent sticks. This new
construction transforms multinomial regression into regression analysis of
stick-specific binary random variables that are mutually independent given
their covariate-dependent stick success probabilities, which are parameterized
by the regression coefficients of their corresponding categories. The paSB
construction allows transforming an arbitrary cross-entropy-loss binary
classifier into a Bayesian multinomial one. Specifically, we parameterize the
negative logarithms of the stick failure probabilities with a family of
covariate-dependent softplus functions to construct nonparametric Bayesian
multinomial softplus regression, and transform Bayesian support vector machine
(SVM) into Bayesian multinomial SVM. These Bayesian multinomial regression
models are not only capable of providing probability estimates, quantifying
uncertainty, increasing robustness, and producing nonlinear classification
decision boundaries, but also amenable to posterior simulation. Example results
demonstrate their attractive properties and performance.
Truthful Facility Location with Additive Errors
Authors: Iddan Golomb, Christos Tzamos
Date: 2017-01-02 00:00:00

We address the problem of locating facilities on the $[0,1]$ interval based
on reports from strategic agents. The cost of each agent is her distance to the
closest facility, and the global objective is to minimize either the maximum
cost of an agent or the social cost.
  As opposed to the extensive literature on facility location which considers
the multiplicative error, we focus on minimizing the worst-case additive error.
Minimizing the additive error incentivizes mechanisms to adapt to the size of
the instance. I.e., mechanisms can sacrifice little efficiency in small
instances (location profiles in which all agents are relatively close to one
another), in order to gain more [absolute] efficiency in large instances. We
argue that this measure is better suited for many manifestations of the
facility location problem in various domains.
  We present tight bounds for mechanisms locating a single facility in both
deterministic and randomized cases. We further provide several extensions for
locating multiple facilities.
Stance detection in online discussions
Authors: Peter Krejzl, Barbora Hourová, Josef Steinberger
Date: 2017-01-02 00:00:00
Tasks: Stance Detection

This paper describes our system created to detect stance in online
discussions. The goal is to identify whether the author of a comment is in
favor of the given target or against. Our approach is based on a maximum
entropy classifier, which uses surface-level, sentiment and domain-specific
features. The system was originally developed to detect stance in English
tweets. We adapted it to process Czech news commentaries.
Structured Sparse Convolutional Autoencoder
Authors: Ehsan Hosseini-Asl
Date: 2016-04-17 00:00:00

This paper aims to improve the feature learning in Convolutional Networks
(Convnet) by capturing the structure of objects. A new sparsity function is
imposed on the extracted featuremap to capture the structure and shape of the
learned object, extracting interpretable features to improve the prediction
performance. The proposed algorithm is based on organizing the activation
within and across featuremap by constraining the node activities through
$\ell_{2}$ and $\ell_{1}$ normalization in a structured form.
Sparse model selection in the highly under-sampled regime
Authors: Nicola Bulso, Matteo Marsili, Yasser Roudi
Date: 2016-03-03 00:00:00
Tasks: Model Selection

We propose a method for recovering the structure of a sparse undirected
graphical model when very few samples are available. The method decides about
the presence or absence of bonds between pairs of variable by considering one
pair at a time and using a closed form formula, analytically derived by
calculating the posterior probability for every possible model explaining a two
body system using Jeffreys prior. The approach does not rely on the
optimisation of any cost functions and consequently is much faster than
existing algorithms. Despite this time and computational advantage, numerical
results show that for several sparse topologies the algorithm is comparable to
the best existing algorithms, and is more accurate in the presence of hidden
variables. We apply this approach to the analysis of US stock market data and
to neural data, in order to show its efficiency in recovering robust
statistical dependencies in real data with non stationary correlations in time
and space.
Towards a Deep Learning Framework for Unconstrained Face Detection
Authors: Yutong Zheng, Chenchen Zhu, Khoa Luu, Chandrasekhar Bhagavatula, T. Hoang Ngan Le, Marios Savvides
Date: 2016-12-16 00:00:00
Tasks: Deep Learning, Face Detection, Face Recognition, Pose Estimation, Robust Face Recognition

Robust face detection is one of the most important pre-processing steps to
support facial expression analysis, facial landmarking, face recognition, pose
estimation, building of 3D facial models, etc. Although this topic has been
intensely studied for decades, it is still challenging due to numerous variants
of face images in real-world scenarios. In this paper, we present a novel
approach named Multiple Scale Faster Region-based Convolutional Neural Network
(MS-FRCNN) to robustly detect human facial regions from images collected under
various challenging conditions, e.g. large occlusions, extremely low
resolutions, facial expressions, strong illumination variations, etc. The
proposed approach is benchmarked on two challenging face detection databases,
i.e. the Wider Face database and the Face Detection Dataset and Benchmark
(FDDB), and compared against recent other face detection methods, e.g.
Two-stage CNN, Multi-scale Cascade CNN, Faceness, Aggregate Chanel Features,
HeadHunter, Multi-view Face Detection, Cascade CNN, etc. The experimental
results show that our proposed approach consistently achieves highly
competitive results with the state-of-the-art performance against other recent
face detection methods.