pwc-archive/papers-with-abstracts huggingface dataset:
A Neural Transfer Function for a Smooth and Differentiable Transition Between Additive and Multiplicative Interactions
Authors: Sebastian Urban, Patrick van der Smagt
Date: 2015-03-19 00:00:00

Existing approaches to combine both additive and multiplicative neural units
either use a fixed assignment of operations or require discrete optimization to
determine what function a neuron should perform. This leads either to an
inefficient distribution of computational resources or an extensive increase in
the computational complexity of the training procedure.
  We present a novel, parameterizable transfer function based on the
mathematical concept of non-integer functional iteration that allows the
operation each neuron performs to be smoothly and, most importantly,
differentiablely adjusted between addition and multiplication. This allows the
decision between addition and multiplication to be integrated into the standard
backpropagation training procedure.
Deep SimNets
Authors: Nadav Cohen, Or Sharir, Amnon Shashua
Date: 2015-06-09 00:00:00
Proceeding: CVPR 2016 6

We present a deep layered architecture that generalizes convolutional neural
networks (ConvNets). The architecture, called SimNets, is driven by two
operators: (i) a similarity function that generalizes inner-product, and (ii) a
log-mean-exp function called MEX that generalizes maximum and average. The two
operators applied in succession give rise to a standard neuron but in "feature
space". The feature spaces realized by SimNets depend on the choice of the
similarity operator. The simplest setting, which corresponds to a convolution,
realizes the feature space of the Exponential kernel, while other settings
realize feature spaces of more powerful kernels (Generalized Gaussian, which
includes as special cases RBF and Laplacian), or even dynamically learned
feature spaces (Generalized Multiple Kernel Learning). As a result, the SimNet
contains a higher abstraction level compared to a traditional ConvNet. We argue
that enhanced expressiveness is important when the networks are small due to
run-time constraints (such as those imposed by mobile applications). Empirical
evaluation validates the superior expressiveness of SimNets, showing a
significant gain in accuracy over ConvNets when computational resources at
run-time are limited. We also show that in large-scale settings, where
computational complexity is less of a concern, the additional capacity of
SimNets can be controlled with proper regularization, yielding accuracies
comparable to state of the art ConvNets.
Using Enthymemes to Fill the Gap between Logical Argumentation and Revision of Abstract Argumentation Frameworks
Authors: Jean-Guy Mailly
Date: 2016-03-29 00:00:00
Tasks: Abstract Argumentation

In this paper, we present a preliminary work on an approach to fill the gap
between logic-based argumentation and the numerous approaches to tackle the
dynamics of abstract argumentation frameworks. Our idea is that, even when
arguments and attacks are defined by means of a logical belief base, there may
be some uncertainty about how accurate is the content of an argument, and so
the presence (or absence) of attacks concerning it. We use enthymemes to
illustrate this notion of uncertainty of arguments and attacks. Indeed, as
argued in the literature, real arguments are often enthymemes instead of
completely specified deductive arguments. This means that some parts of the
pair (support, claim) may be missing because they are supposed to belong to
some "common knowledge", and then should be deduced by the agent which receives
the enthymeme. But the perception that agents have of the common knowledge may
be wrong, and then a first agent may state an enthymeme that her opponent is
not able to decode in an accurate way. It is likely that the decoding of the
enthymeme by the agent leads to mistaken attacks between this new argument and
the existing ones. In this case, the agent can receive some information about
attacks or arguments acceptance statuses which disagree with her argumentation
framework. We exemplify a way to incorporate this new piece of information by
means of existing works on the dynamics of abstract argumentation frameworks.
Machine Learning and Cloud Computing: Survey of Distributed and SaaS Solutions
Authors: Daniel Pop
Date: 2016-03-29 00:00:00
Tasks: BIG-bench Machine Learning, Cloud Computing

Applying popular machine learning algorithms to large amounts of data raised
new challenges for the ML practitioners. Traditional ML libraries does not
support well processing of huge datasets, so that new approaches were needed.
Parallelization using modern parallel computing frameworks, such as MapReduce,
CUDA, or Dryad gained in popularity and acceptance, resulting in new ML
libraries developed on top of these frameworks. We will briefly introduce the
most prominent industrial and academic outcomes, such as Apache Mahout,
GraphLab or Jubatus.
  We will investigate how cloud computing paradigm impacted the field of ML.
First direction is of popular statistics tools and libraries (R system, Python)
deployed in the cloud. A second line of products is augmenting existing tools
with plugins that allow users to create a Hadoop cluster in the cloud and run
jobs on it. Next on the list are libraries of distributed implementations for
ML algorithms, and on-premise deployments of complex systems for data analytics
and data mining. Last approach on the radar of this survey is ML as
Software-as-a-Service, several BigData start-ups (and large companies as well)
already opening their solutions to the market.
Multi-Cue Zero-Shot Learning with Strong Supervision
Authors: Zeynep Akata, Mateusz Malinowski, Mario Fritz, Bernt Schiele
Date: 2016-03-29 00:00:00
Proceeding: CVPR 2016 6
Tasks: Attribute, Form, Retrieval, Zero-Shot Learning

Scaling up visual category recognition to large numbers of classes remains
challenging. A promising research direction is zero-shot learning, which does
not require any training data to recognize new classes, but rather relies on
some form of auxiliary information describing the new classes. Ultimately, this
may allow to use textbook knowledge that humans employ to learn about new
classes by transferring knowledge from classes they know well. The most
successful zero-shot learning approaches currently require a particular type of
auxiliary information -- namely attribute annotations performed by humans --
that is not readily available for most classes. Our goal is to circumvent this
bottleneck by substituting such annotations by extracting multiple pieces of
information from multiple unstructured text sources readily available on the
web. To compensate for the weaker form of auxiliary information, we incorporate
stronger supervision in the form of semantic part annotations on the classes
from which we transfer knowledge. We achieve our goal by a joint embedding
framework that maps multiple text parts as well as multiple semantic parts into
a common space. Our results consistently and significantly improve on the
state-of-the-art in zero-short recognition and retrieval.
Multi-Band Image Fusion Based on Spectral Unmixing
Authors: Qi Wei, Jose Bioucas-Dias, Nicolas Dobigeon, Jean-Yves Tourneret, Marcus Chen, Simon Godsill
Date: 2016-03-29 00:00:00

This paper presents a multi-band image fusion algorithm based on unsupervised
spectral unmixing for combining a high-spatial low-spectral resolution image
and a low-spatial high-spectral resolution image. The widely used linear
observation model (with additive Gaussian noise) is combined with the linear
spectral mixture model to form the likelihoods of the observations. The
non-negativity and sum-to-one constraints resulting from the intrinsic physical
properties of the abundances are introduced as prior information to regularize
this ill-posed problem. The joint fusion and unmixing problem is then
formulated as maximizing the joint posterior distribution with respect to the
endmember signatures and abundance maps, This optimization problem is attacked
with an alternating optimization strategy. The two resulting sub-problems are
convex and are solved efficiently using the alternating direction method of
multipliers. Experiments are conducted for both synthetic and semi-real data.
Simulation results show that the proposed unmixing based fusion scheme improves
both the abundance and endmember estimation comparing with the state-of-the-art
joint fusion and unmixing algorithms.
Unified View of Matrix Completion under General Structural Constraints
Authors: Suriya Gunasekar, Arindam Banerjee, Joydeep Ghosh
Date: 2016-03-29 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: Matrix Completion

In this paper, we present a unified analysis of matrix completion under
general low-dimensional structural constraints induced by {\em any} norm
regularization. We consider two estimators for the general problem of
structured matrix completion, and provide unified upper bounds on the sample
complexity and the estimation error. Our analysis relies on results from
generic chaining, and we establish two intermediate results of independent
interest: (a) in characterizing the size or complexity of low dimensional
subsets in high dimensional ambient space, a certain partial complexity measure
encountered in the analysis of matrix completion problems is characterized in
terms of a well understood complexity measure of Gaussian widths, and (b) it is
shown that a form of restricted strong convexity holds for matrix completion
problems under general norm regularization. Further, we provide several
non-trivial examples of structures included in our framework, notably the
recently proposed spectral $k$-support norm.
ROOT13: Spotting Hypernyms, Co-Hyponyms and Randoms
Authors: Enrico Santus, Tin-Shing Chiu, Qin Lu, Alessandro Lenci, Chu-Ren Huang
Date: 2016-03-29 00:00:00
Tasks: Classification, General Classification

In this paper, we describe ROOT13, a supervised system for the classification
of hypernyms, co-hyponyms and random words. The system relies on a Random
Forest algorithm and 13 unsupervised corpus-based features. We evaluate it with
a 10-fold cross validation on 9,600 pairs, equally distributed among the three
classes and involving several Parts-Of-Speech (i.e. adjectives, nouns and
verbs). When all the classes are present, ROOT13 achieves an F1 score of 88.3%,
against a baseline of 57.6% (vector cosine). When the classification is binary,
ROOT13 achieves the following results: hypernyms-co-hyponyms (93.4% vs. 60.2%),
hypernymsrandom (92.3% vs. 65.5%) and co-hyponyms-random (97.3% vs. 81.5%). Our
results are competitive with stateof-the-art models.
Interpretability of Multivariate Brain Maps in Brain Decoding: Definition and Quantification
Authors: Seyed Mostafa Kia
Date: 2016-03-29 00:00:00
Tasks: Brain Decoding, Model Selection, Two-sample testing

Brain decoding is a popular multivariate approach for hypothesis testing in
neuroimaging. It is well known that the brain maps derived from weights of
linear classifiers are hard to interpret because of high correlations between
predictors, low signal to noise ratios, and the high dimensionality of
neuroimaging data. Therefore, improving the interpretability of brain decoding
approaches is of primary interest in many neuroimaging studies. Despite
extensive studies of this type, at present, there is no formal definition for
interpretability of multivariate brain maps. As a consequence, there is no
quantitative measure for evaluating the interpretability of different brain
decoding methods. In this paper, first, we present a theoretical definition of
interpretability in brain decoding; we show that the interpretability of
multivariate brain maps can be decomposed into their reproducibility and
representativeness. Second, as an application of the proposed theoretical
definition, we formalize a heuristic method for approximating the
interpretability of multivariate brain maps in a binary magnetoencephalography
(MEG) decoding scenario. Third, we propose to combine the approximated
interpretability and the performance of the brain decoding model into a new
multi-objective criterion for model selection. Our results for the MEG data
show that optimizing the hyper-parameters of the regularized linear classifier
based on the proposed criterion results in more informative multivariate brain
maps. More importantly, the presented definition provides the theoretical
background for quantitative evaluation of interpretability, and hence,
facilitates the development of more effective brain decoding algorithms in the
future.
Nine Features in a Random Forest to Learn Taxonomical Semantic Relations
Authors: Enrico Santus, Alessandro Lenci, Tin-Shing Chiu, Qin Lu, Chu-Ren Huang
Date: 2016-03-29 00:00:00
Proceeding: LREC 2016 5
Tasks: General Classification

ROOT9 is a supervised system for the classification of hypernyms, co-hyponyms
and random words that is derived from the already introduced ROOT13 (Santus et
al., 2016). It relies on a Random Forest algorithm and nine unsupervised
corpus-based features. We evaluate it with a 10-fold cross validation on 9,600
pairs, equally distributed among the three classes and involving several
Parts-Of-Speech (i.e. adjectives, nouns and verbs). When all the classes are
present, ROOT9 achieves an F1 score of 90.7%, against a baseline of 57.2%
(vector cosine). When the classification is binary, ROOT9 achieves the
following results against the baseline: hypernyms-co-hyponyms 95.7% vs. 69.8%,
hypernyms-random 91.8% vs. 64.1% and co-hyponyms-random 97.8% vs. 79.4%. In
order to compare the performance with the state-of-the-art, we have also
evaluated ROOT9 in subsets of the Weeds et al. (2014) datasets, proving that it
is in fact competitive. Finally, we investigated whether the system learns the
semantic relation or it simply learns the prototypical hypernyms, as claimed by
Levy et al. (2015). The second possibility seems to be the most likely, even
though ROOT9 can be trained on negative examples (i.e., switched hypernyms) to
drastically reduce this bias.
What a Nerd! Beating Students and Vector Cosine in the ESL and TOEFL Datasets
Authors: Enrico Santus, Tin-Shing Chiu, Qin Lu, Alessandro Lenci, Chu-Ren Huang
Date: 2016-03-29 00:00:00
Proceeding: LREC 2016 5
Tasks: Word Similarity

In this paper, we claim that Vector Cosine, which is generally considered one
of the most efficient unsupervised measures for identifying word similarity in
Vector Space Models, can be outperformed by a completely unsupervised measure
that evaluates the extent of the intersection among the most associated
contexts of two target words, weighting such intersection according to the rank
of the shared contexts in the dependency ranked lists. This claim comes from
the hypothesis that similar words do not simply occur in similar contexts, but
they share a larger portion of their most relevant contexts compared to other
related words. To prove it, we describe and evaluate APSyn, a variant of
Average Precision that, independently of the adopted parameters, outperforms
the Vector Cosine and the co-occurrence on the ESL and TOEFL test sets. In the
best setting, APSyn reaches 0.73 accuracy on the ESL dataset and 0.70 accuracy
in the TOEFL dataset, beating therefore the non-English US college applicants
(whose average, as reported in the literature, is 64.50%) and several
state-of-the-art approaches.
Instance-sensitive Fully Convolutional Networks
Authors: Jifeng Dai, Kaiming He, Yi Li, Shaoqing Ren, Jian Sun
Date: 2016-03-29 00:00:00
Tasks: Position, Semantic Segmentation

Fully convolutional networks (FCNs) have been proven very successful for
semantic segmentation, but the FCN outputs are unaware of object instances. In
this paper, we develop FCNs that are capable of proposing instance-level
segment candidates. In contrast to the previous FCN that generates one score
map, our FCN is designed to compute a small set of instance-sensitive score
maps, each of which is the outcome of a pixel-wise classifier of a relative
position to instances. On top of these instance-sensitive score maps, a simple
assembling module is able to output instance candidate at each position. In
contrast to the recent DeepMask method for segmenting instances, our method
does not have any high-dimensional layer related to the mask resolution, but
instead exploits image local coherence for estimating instances. We present
competitive results of instance segment proposal on both PASCAL VOC and MS
COCO.
Some Insights About the Small Ball Probability Factorization for Hilbert Random Elements
Authors: Enea Bongiorno, Aldo Goia
Date: 2015-01-18 00:00:00

Asymptotic factorizations for the small-ball probability (SmBP) of a Hilbert
valued random element $X$ are rigorously established and discussed. In
particular, given the first $d$ principal components (PCs) and as the radius
$\varepsilon$ of the ball tends to zero, the SmBP is asymptotically
proportional to (a) the joint density of the first $d$ PCs, (b) the volume of
the $d$-dimensional ball with radius $\varepsilon$, and (c) a correction factor
weighting the use of a truncated version of the process expansion. Moreover,
under suitable assumptions on the spectrum of the covariance operator of $X$
and as $d$ diverges to infinity when $\varepsilon$ vanishes, some
simplifications occur. In particular, the SmBP factorizes asymptotically as the
product of the joint density of the first $d$ PCs and a pure volume parameter.
All the provided factorizations allow to define a surrogate intensity of the
SmBP that, in some cases, leads to a genuine intensity. To operationalize the
stated results, a non-parametric estimator for the surrogate intensity is
introduced and it is proved that the use of estimated PCs, instead of the true
ones, does not affect the rate of convergence. Finally, as an illustration,
simulations in controlled frameworks are provided.
Flexible Multi-layer Sparse Approximations of Matrices and Applications
Authors: Luc Le Magoarou, Rémi Gribonval
Date: 2015-06-24 00:00:00
Tasks: BIG-bench Machine Learning, Denoising, Dictionary Learning, Image Denoising

The computational cost of many signal processing and machine learning
techniques is often dominated by the cost of applying certain linear operators
to high-dimensional vectors. This paper introduces an algorithm aimed at
reducing the complexity of applying linear operators in high dimension by
approximately factorizing the corresponding matrix into few sparse factors. The
approach relies on recent advances in non-convex optimization. It is first
explained and analyzed in details and then demonstrated experimentally on
various problems including dictionary learning for image denoising, and the
approximation of large matrices arising in inverse problems.
Cross-Language Domain Adaptation for Classifying Crisis-Related Short Messages
Authors: Muhammad Imran, Prasenjit Mitra, Jaideep Srivastava
Date: 2016-02-17 00:00:00
Tasks: Domain Adaptation

Rapid crisis response requires real-time analysis of messages. After a
disaster happens, volunteers attempt to classify tweets to determine needs,
e.g., supplies, infrastructure damage, etc. Given labeled data, supervised
machine learning can help classify these messages. Scarcity of labeled data
causes poor performance in machine training. Can we reuse old tweets to train
classifiers? How can we choose labeled tweets for training? Specifically, we
study the usefulness of labeled data of past events. Do labeled tweets in
different language help? We observe the performance of our classifiers trained
using different combinations of training sets obtained from past disasters. We
perform extensive experimentation on real crisis datasets and show that the
past labels are useful when both source and target events are of the same type
(e.g. both earthquakes). For similar languages (e.g., Italian and Spanish),
cross-language domain adaptation was useful, however, when for different
languages (e.g., Italian and English), the performance decreased.
Regularization and Kernelization of the Maximin Correlation Approach
Authors: Taehoon Lee, Taesup Moon, Seung Jean Kim, Sungroh Yoon
Date: 2015-02-21 00:00:00
Tasks: Classification, General Classification, Optical Character Recognition, Optical Character Recognition (OCR), Protein Function Prediction, Robust classification

Robust classification becomes challenging when each class consists of
multiple subclasses. Examples include multi-font optical character recognition
and automated protein function prediction. In correlation-based
nearest-neighbor classification, the maximin correlation approach (MCA)
provides the worst-case optimal solution by minimizing the maximum
misclassification risk through an iterative procedure. Despite the optimality,
the original MCA has drawbacks that have limited its wide applicability in
practice. That is, the MCA tends to be sensitive to outliers, cannot
effectively handle nonlinearities in datasets, and suffers from having high
computational complexity. To address these limitations, we propose an improved
solution, named regularized maximin correlation approach (R-MCA). We first
reformulate MCA as a quadratically constrained linear programming (QCLP)
problem, incorporate regularization by introducing slack variables in the
primal problem of the QCLP, and derive the corresponding Lagrangian dual. The
dual formulation enables us to apply the kernel trick to R-MCA so that it can
better handle nonlinearities. Our experimental results demonstrate that the
regularization and kernelization make the proposed R-MCA more robust and
accurate for various classification tasks than the original MCA. Furthermore,
when the data size or dimensionality grows, R-MCA runs substantially faster by
solving either the primal or dual (whichever has a smaller variable dimension)
of the QCLP.
Learning Image Matching by Simply Watching Video
Authors: Gucan Long, Laurent Kneip, Jose M. Alvarez, Hongdong Li
Date: 2016-03-19 00:00:00

This work presents an unsupervised learning based approach to the ubiquitous
computer vision problem of image matching. We start from the insight that the
problem of frame-interpolation implicitly solves for inter-frame
correspondences. This permits the application of analysis-by-synthesis: we
firstly train and apply a Convolutional Neural Network for frame-interpolation,
then obtain correspondences by inverting the learned CNN. The key benefit
behind this strategy is that the CNN for frame-interpolation can be trained in
an unsupervised manner by exploiting the temporal coherency that is naturally
contained in real-world video sequences. The present model therefore learns
image matching by simply watching videos. Besides a promise to be more
generally applicable, the presented approach achieves surprising performance
comparable to traditional empirically designed methods.
Towards an Automated Requirements-driven Development of Smart Cyber-Physical Systems
Authors: Jiri Vinarek, Petr Hnetynka
Date: 2016-03-29 00:00:00
Tasks: Translation

The Invariant Refinement Method for Self Adaptation (IRM-SA) is a design
method targeting development of smart Cyber-Physical Systems (sCPS). It allows
for a systematic translation of the system requirements into the system
architecture expressed as an ensemble-based component system (EBCS). However,
since the requirements are captured using natural language, there exists the
danger of their misinterpretation due to natural language requirements'
ambiguity, which could eventually lead to design errors. Thus, automation and
validation of the design process is desirable. In this paper, we (i) analyze
the translation process of natural language requirements into the IRM-SA model,
(ii) identify individual steps that can be automated and/or validated using
natural language processing techniques, and (iii) propose suitable methods.