pwc-archive/papers-with-abstracts huggingface dataset:
Tensor Regression Networks
Authors: Jean Kossaifi, Zachary C. Lipton, Arinbjorn Kolbeinsson, Aran Khanna, Tommaso Furlanello, Anima Anandkumar
Date: 2017-07-26 00:00:00
Tasks: regression

Convolutional neural networks typically consist of many convolutional layers followed by one or more fully connected layers. While convolutional layers map between high-order activation tensors, the fully connected layers operate on flattened activation vectors. Despite empirical success, this approach has notable drawbacks. Flattening followed by fully connected layers discards multilinear structure in the activations and requires many parameters. We address these problems by incorporating tensor algebraic operations that preserve multilinear structure at every layer. First, we introduce Tensor Contraction Layers (TCLs) that reduce the dimensionality of their input while preserving their multilinear structure using tensor contraction. Next, we introduce Tensor Regression Layers (TRLs), which express outputs through a low-rank multilinear mapping from a high-order activation tensor to an output tensor of arbitrary order. We learn the contraction and regression factors end-to-end, and produce accurate nets with fewer parameters. Additionally, our layers regularize networks by imposing low-rank constraints on the activations (TCL) and regression weights (TRL). Experiments on ImageNet show that, applied to VGG and ResNet architectures, TCLs and TRLs reduce the number of parameters compared to fully connected layers by more than 65% while maintaining or increasing accuracy. In addition to the space savings, our approach's ability to leverage topological structure can be crucial for structured data such as MRI. In particular, we demonstrate significant performance improvements over comparable architectures on three tasks associated with the UK Biobank dataset.
Detection of Block-Exchangeable Structure in Large-Scale Correlation Matrices
Authors: Samuel Perreault, Thierry Duchesne, Johanna G. Nešlehová
Date: 2017-06-19 00:00:00
Tasks: Econometrics

Correlation matrices are omnipresent in multivariate data analysis. When the
number d of variables is large, the sample estimates of correlation matrices
are typically noisy and conceal underlying dependence patterns. We consider the
case when the variables can be grouped into K clusters with exchangeable
dependence; this assumption is often made in applications, e.g., in finance and
econometrics. Under this partial exchangeability condition, the corresponding
correlation matrix has a block structure and the number of unknown parameters
is reduced from d(d-1)/2 to at most K(K+1)/2. We propose a robust algorithm
based on Kendall's rank correlation to identify the clusters without assuming
the knowledge of K a priori or anything about the margins except continuity.
The corresponding block-structured estimator performs considerably better than
the sample Kendall rank correlation matrix when K < d. The new estimator can
also be much more efficient in finite samples even in the unstructured case K =
d, although there is no gain asymptotically. When the distribution of the data
is elliptical, the results extend to linear correlation matrices and their
inverses. The procedure is illustrated on financial stock returns.
Building Machines that Learn and Think for Themselves: Commentary on Lake et al., Behavioral and Brain Sciences, 2017
Authors: M. Botvinick, D. G. T. Barrett, P. Battaglia, N. de Freitas, D. Kumaran, J. Z Leibo, T. Lillicrap, J. Modayil, S. Mohamed, N. C. Rabinowitz, D. J. Rezende, A. Santoro, T. Schaul, C. Summerfield, G. Wayne, T. Weber, D. Wierstra, S. Legg, D. Hassabis
Date: 2017-11-22 00:00:00

We agree with Lake and colleagues on their list of key ingredients for
building humanlike intelligence, including the idea that model-based reasoning
is essential. However, we favor an approach that centers on one additional
ingredient: autonomy. In particular, we aim toward agents that can both build
and exploit their own internal models, with minimal human hand-engineering. We
believe an approach centered on autonomous learning has the greatest chance of
success as we scale toward real-world complexity, tackling domains for which
ready-made formal models are not available. Here we survey several important
examples of the progress that has been made toward building autonomous agents
with humanlike abilities, and highlight some outstanding challenges.
Variational Bayesian Inference For A Scale Mixture Of Normal Distributions Handling Missing Data
Authors: G. Revillon, A. Djafari, C. Enderli
Date: 2017-11-22 00:00:00
Tasks: Bayesian Inference, Classification, Clustering, General Classification, Missing Values, Sensitivity

In this paper, a scale mixture of Normal distributions model is developed for
classification and clustering of data having outliers and missing values. The
classification method, based on a mixture model, focuses on the introduction of
latent variables that gives us the possibility to handle sensitivity of model
to outliers and to allow a less restrictive modelling of missing data.
Inference is processed through a Variational Bayesian Approximation and a
Bayesian treatment is adopted for model learning, supervised classification and
clustering.
Effective Strategies in Zero-Shot Neural Machine Translation
Authors: Thanh-Le Ha, Jan Niehues, Alexander Waibel
Date: 2017-11-21 00:00:00
Proceeding: IWSLT 2017 12
Tasks: Machine Translation, Translation

In this paper, we proposed two strategies which can be applied to a
multilingual neural machine translation system in order to better tackle
zero-shot scenarios despite not having any parallel corpus. The experiments
show that they are effective in terms of both performance and computing
resources, especially in multilingual translation of unbalanced data in real
zero-resourced condition when they alleviate the language bias problem.
ForestHash: Semantic Hashing With Shallow Random Forests and Tiny Convolutional Networks
Authors: Qiang Qiu, Jose Lezama, Alex Bronstein, Guillermo Sapiro
Date: 2017-11-22 00:00:00
Proceeding: ECCV 2018 9
Tasks: General Classification, image-classification, Image Classification, Image Retrieval, Retrieval

Hash codes are efficient data representations for coping with the ever
growing amounts of data. In this paper, we introduce a random forest semantic
hashing scheme that embeds tiny convolutional neural networks (CNN) into
shallow random forests, with near-optimal information-theoretic code
aggregation among trees. We start with a simple hashing scheme, where random
trees in a forest act as hashing functions by setting `1' for the visited tree
leaf, and `0' for the rest. We show that traditional random forests fail to
generate hashes that preserve the underlying similarity between the trees,
rendering the random forests approach to hashing challenging. To address this,
we propose to first randomly group arriving classes at each tree split node
into two groups, obtaining a significantly simplified two-class classification
problem, which can be handled using a light-weight CNN weak learner. Such
random class grouping scheme enables code uniqueness by enforcing each class to
share its code with different classes in different trees. A non-conventional
low-rank loss is further adopted for the CNN weak learners to encourage code
consistency by minimizing intra-class variations and maximizing inter-class
distance for the two random class groups. Finally, we introduce an
information-theoretic approach for aggregating codes of individual trees into a
single hash code, producing a near-optimal unique hash for each class. The
proposed approach significantly outperforms state-of-the-art hashing methods
for image retrieval tasks on large-scale public datasets, while performing at
the level of other state-of-the-art image classification techniques while
utilizing a more compact and efficient scalable representation. This work
proposes a principled and robust procedure to train and deploy in parallel an
ensemble of light-weight CNNs, instead of simply going deeper.
Dynamic classifier chains for multi-label learning
Authors: Pawel Trajdos, Marek Kurzynski
Date: 2017-10-20 00:00:00
Tasks: General Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Multi-Label Learning

In this paper, we deal with the task of building a dynamic ensemble of chain
classifiers for multi-label classification. To do so, we proposed two concepts
of classifier chains algorithms that are able to change label order of the
chain without rebuilding the entire model. Such modes allows anticipating the
instance-specific chain order without a significant increase in computational
burden. The proposed chain models are built using the Naive Bayes classifier
and nearest neighbour approach as a base single-label classifiers. To take the
benefits of the proposed algorithms, we developed a simple heuristic that
allows the system to find relatively good label order. The heuristic sort
labels according to the label-specific classification quality gained during the
validation phase. The heuristic tries to minimise the phenomenon of error
propagation in the chain. The experimental results showed that the proposed
model based on Naive Bayes classifier the above-mentioned heuristic is an
efficient tool for building dynamic chain classifiers.
Riemannian tangent space mapping and elastic net regularization for cost-effective EEG markers of brain atrophy in Alzheimer's disease
Authors: Wolfgang Fruehwirt, Matthias Gerstgrasser, Pengfei Zhang, Leonard Weydemann, Markus Waser, Reinhold Schmidt, Thomas Benke, Peter Dal-Bianco, Gerhard Ransmayr, Dieter Grossegger, Heinrich Garn, Gareth W. Peters, Stephen Roberts, Georg Dorffner
Date: 2017-11-22 00:00:00
Tasks: EEG, Electroencephalogram (EEG), regression

The diagnosis of Alzheimer's disease (AD) in routine clinical practice is
most commonly based on subjective clinical interpretations. Quantitative
electroencephalography (QEEG) measures have been shown to reflect
neurodegenerative processes in AD and might qualify as affordable and thereby
widely available markers to facilitate the objectivization of AD assessment.
Here, we present a novel framework combining Riemannian tangent space mapping
and elastic net regression for the development of brain atrophy markers. While
most AD QEEG studies are based on small sample sizes and psychological test
scores as outcome measures, here we train and test our models using data of one
of the largest prospective EEG AD trials ever conducted, including MRI
biomarkers of brain atrophy.
Feature Control as Intrinsic Motivation for Hierarchical Reinforcement Learning
Authors: Nat Dilokthanakul, Christos Kaplanis, Nick Pawlowski, Murray Shanahan
Date: 2017-05-18 00:00:00
Tasks: Hierarchical Reinforcement Learning, Montezuma's Revenge, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

The problem of sparse rewards is one of the hardest challenges in
contemporary reinforcement learning. Hierarchical reinforcement learning (HRL)
tackles this problem by using a set of temporally-extended actions, or options,
each of which has its own subgoal. These subgoals are normally handcrafted for
specific tasks. Here, though, we introduce a generic class of subgoals with
broad applicability in the visual domain. Underlying our approach (in common
with work using "auxiliary tasks") is the hypothesis that the ability to
control aspects of the environment is an inherently useful skill to have. We
incorporate such subgoals in an end-to-end hierarchical reinforcement learning
system and test two variants of our algorithm on a number of games from the
Atari suite. We highlight the advantage of our approach in one of the hardest
games -- Montezuma's revenge -- for which the ability to handle sparse rewards
is key. Our agent learns several times faster than the current state-of-the-art
HRL agent in this game, reaching a similar level of performance. UPDATE
22/11/17: We found that a standard A3C agent with a simple shaped reward, i.e.
extrinsic reward + feature control intrinsic reward, has comparable performance
to our agent in Montezuma Revenge. In light of the new experiments performed,
the advantage of our HRL approach can be attributed more to its ability to
learn useful features from intrinsic rewards rather than its ability to explore
and reuse abstracted skills with hierarchical components. This has led us to a
new conclusion about the result.
On Faster Convergence of Cyclic Block Coordinate Descent-type Methods for Strongly Convex Minimization
Authors: Xingguo Li, Tuo Zhao, Raman Arora, Han Liu, Mingyi Hong
Date: 2016-07-10 00:00:00
Tasks: regression

The cyclic block coordinate descent-type (CBCD-type) methods, which performs
iterative updates for a few coordinates (a block) simultaneously throughout the
procedure, have shown remarkable computational performance for solving strongly
convex minimization problems. Typical applications include many popular
statistical machine learning methods such as elastic-net regression, ridge
penalized logistic regression, and sparse additive regression. Existing
optimization literature has shown that for strongly convex minimization, the
CBCD-type methods attain iteration complexity of
$\mathcal{O}(p\log(1/\epsilon))$, where $\epsilon$ is a pre-specified accuracy
of the objective value, and $p$ is the number of blocks. However, such
iteration complexity explicitly depends on $p$, and therefore is at least $p$
times worse than the complexity $\mathcal{O}(\log(1/\epsilon))$ of gradient
descent (GD) methods. To bridge this theoretical gap, we propose an improved
convergence analysis for the CBCD-type methods. In particular, we first show
that for a family of quadratic minimization problems, the iteration complexity
$\mathcal{O}(\log^2(p)\cdot\log(1/\epsilon))$ of the CBCD-type methods matches
that of the GD methods in term of dependency on $p$, up to a $\log^2 p$ factor.
Thus our complexity bounds are sharper than the existing bounds by at least a
factor of $p/\log^2(p)$. We also provide a lower bound to confirm that our
improved complexity bounds are tight (up to a $\log^2 (p)$ factor), under the
assumption that the largest and smallest eigenvalues of the Hessian matrix do
not scale with $p$. Finally, we generalize our analysis to other strongly
convex minimization problems beyond quadratic ones.
A correlational analysis of multiagent sensorimotor interactions: clustering autonomous and controllable entities
Authors: M. Sánchez-Fibla, C. Moulin-Frier, X. Arsiwalla, P. Verschure
Date: 2017-11-22 00:00:00
Tasks: Clustering, TAG

A first step to reach Theory of Mind (ToM) abilities (attribution of beliefs
to others) in synthetic agents through sensorimotor interactions, would be to
tag sensory data with agent typology and action intentions: autonomous agent X
moved an object under the box. We propose a dual arm robotic setup in which ToM
could be probed. We then discuss what measures can be extracted from
sensorimotor interaction data (based on a correlation analysis) in the proposed
setup that allow to distinguish self than other and other/inanimate from
other/active with intentions. We finally discuss what elements are missing in
current cognitive architectures to be able to acquire ToM abilities in
synthetic agents from sensorimotor interactions, bottom-up from reactive agent
interaction behaviors and top-down from the optimization of social behaviour
and cooperation.
Adaptive Cardinality Estimation
Authors: Oleg Ivanov, Sergey Bartunov
Date: 2017-11-22 00:00:00

In this paper we address cardinality estimation problem which is an important
subproblem in query optimization. Query optimization is a part of every
relational DBMS responsible for finding the best way of the execution for the
given query. These ways are called plans. The execution time of different plans
may differ by several orders, so query optimizer has a great influence on the
whole DBMS performance. We consider cost-based query optimization approach as
the most popular one. It was observed that cost-based optimization quality
depends much on cardinality estimation quality. Cardinality of the plan node is
the number of tuples returned by it.
  In the paper we propose a novel cardinality estimation approach with the use
of machine learning methods. The main point of the approach is using query
execution statistics of the previously executed queries to improve cardinality
estimations. We called this approach adaptive cardinality estimation to reflect
this point. The approach is general, flexible, and easy to implement. The
experimental evaluation shows that this approach significantly increases the
quality of cardinality estimation, and therefore increases the DBMS performance
for some queries by several times or even by several dozens of times.
Evaluate the Malignancy of Pulmonary Nodules Using the 3D Deep Leaky Noisy-or Network
Authors: Fangzhou Liao, Ming Liang, Zhe Li, Xiaolin Hu, Sen Song
Date: 2017-11-22 00:00:00
Tasks: Computed Tomography (CT), Region Proposal

Automatic diagnosing lung cancer from Computed Tomography (CT) scans involves
two steps: detect all suspicious lesions (pulmonary nodules) and evaluate the
whole-lung/pulmonary malignancy. Currently, there are many studies about the
first step, but few about the second step. Since the existence of nodule does
not definitely indicate cancer, and the morphology of nodule has a complicated
relationship with cancer, the diagnosis of lung cancer demands careful
investigations on every suspicious nodule and integration of information of all
nodules. We propose a 3D deep neural network to solve this problem. The model
consists of two modules. The first one is a 3D region proposal network for
nodule detection, which outputs all suspicious nodules for a subject. The
second one selects the top five nodules based on the detection confidence,
evaluates their cancer probabilities and combines them with a leaky noisy-or
gate to obtain the probability of lung cancer for the subject. The two modules
share the same backbone network, a modified U-net. The over-fitting caused by
the shortage of training data is alleviated by training the two modules
alternately. The proposed model won the first place in the Data Science Bowl
2017 competition. The code has been made publicly available.
Three-Stream Convolutional Networks for Video-based Person Re-Identification
Authors: Zeng Yu, Tianrui Li, Ning Yu, Xun Gong, Ke Chen, Yi Pan
Date: 2017-11-22 00:00:00
Tasks: Person Re-Identification, Video-Based Person Re-Identification

This paper aims to develop a new architecture that can make full use of the
feature maps of convolutional networks. To this end, we study a number of
methods for video-based person re-identification and make the following
findings: 1) Max-pooling only focuses on the maximum value of a receptive
field, wasting a lot of information. 2) Networks with different streams even
including the one with the worst performance work better than networks with
same streams, where each one has the best performance alone. 3) A full
connection layer at the end of convolutional networks is not necessary. Based
on these studies, we propose a new convolutional architecture termed
Three-Stream Convolutional Networks (TSCN). It first uses different streams to
learn different aspects of feature maps for attentive spatio-temporal fusion of
video, and then merges them together to study some union features. To further
utilize the feature maps, two architectures are designed by using the
strategies of multi-scale and upsampling. Comparative experiments on iLIDS-VID,
PRID-2011 and MARS datasets illustrate that the proposed architectures are
significantly better for feature extraction than the state-of-the-art models.
H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation from CT Volumes
Authors: Xiaomeng Li, Hao Chen, Xiaojuan Qi, Qi Dou, Chi-Wing Fu, Pheng Ann Heng
Date: 2017-09-21 00:00:00
Tasks: Automatic Liver And Tumor Segmentation, GPU, Image Segmentation, Lesion Segmentation, Liver Segmentation, Segmentation, Semantic Segmentation, Tumor Segmentation

Liver cancer is one of the leading causes of cancer death. To assist doctors
in hepatocellular carcinoma diagnosis and treatment planning, an accurate and
automatic liver and tumor segmentation method is highly demanded in the
clinical practice. Recently, fully convolutional neural networks (FCNs),
including 2D and 3D FCNs, serve as the back-bone in many volumetric image
segmentation. However, 2D convolutions can not fully leverage the spatial
information along the third dimension while 3D convolutions suffer from high
computational cost and GPU memory consumption. To address these issues, we
propose a novel hybrid densely connected UNet (H-DenseUNet), which consists of
a 2D DenseUNet for efficiently extracting intra-slice features and a 3D
counterpart for hierarchically aggregating volumetric contexts under the spirit
of the auto-context algorithm for liver and tumor segmentation. We formulate
the learning process of H-DenseUNet in an end-to-end manner, where the
intra-slice representations and inter-slice features can be jointly optimized
through a hybrid feature fusion (HFF) layer. We extensively evaluated our
method on the dataset of MICCAI 2017 Liver Tumor Segmentation (LiTS) Challenge
and 3DIRCADb Dataset. Our method outperformed other state-of-the-arts on the
segmentation results of tumors and achieved very competitive performance for
liver segmentation even with a single model.
Qualitative Assessment of Recurrent Human Motion
Authors: Andre Ebert, Michael Till Beck, Andy Mattausch, Lenz Belzner, Claudia Linnhoff Popien
Date: 2017-03-07 00:00:00
Tasks: Time Series, Time Series Analysis

Smartphone applications designed to track human motion in combination with
wearable sensors, e.g., during physical exercising, raised huge attention
recently. Commonly, they provide quantitative services, such as personalized
training instructions or the counting of distances. But qualitative monitoring
and assessment is still missing, e.g., to detect malpositions, to prevent
injuries, or to optimize training success. We address this issue by presenting
a concept for qualitative as well as generic assessment of recurrent human
motion by processing multi-dimensional, continuous time series tracked with
motion sensors. Therefore, our segmentation procedure extracts individual
events of specific length and we propose expressive features to accomplish a
qualitative motion assessment by supervised classification. We verified our
approach within a comprehensive study encompassing 27 athletes undertaking
different body weight exercises. We are able to recognize six different
exercise types with a success rate of 100% and to assess them qualitatively
with an average success rate of 99.3%.
Neuron-level Selective Context Aggregation for Scene Segmentation
Authors: Zhenhua Wang, Fanglin Gu, Dani Lischinski, Daniel Cohen-Or, Changhe Tu, Baoquan Chen
Date: 2017-11-22 00:00:00
Tasks: Scene Segmentation, Segmentation

Contextual information provides important cues for disambiguating visually
similar pixels in scene segmentation. In this paper, we introduce a
neuron-level Selective Context Aggregation (SCA) module for scene segmentation,
comprised of a contextual dependency predictor and a context aggregation
operator. The dependency predictor is implicitly trained to infer contextual
dependencies between different image regions. The context aggregation operator
augments local representations with global context, which is aggregated
selectively at each neuron according to its on-the-fly predicted dependencies.
The proposed mechanism enables data-driven inference of contextual
dependencies, and facilitates context-aware feature learning. The proposed
method improves strong baselines built upon VGG16 on challenging scene
segmentation datasets, which demonstrates its effectiveness in modeling context
information.