pwc-archive/papers-with-abstracts huggingface dataset:
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Authors: Joao Carreira, Andrew Zisserman
Date: 2017-05-22 00:00:00
Proceeding: CVPR 2017 7
Tasks: Action Classification, Action Recognition, General Classification, image-classification, Skeleton Based Action Recognition, Video Object Tracking

The paucity of videos in current action classification datasets (UCF-101 and
HMDB-51) has made it difficult to identify good video architectures, as most
methods obtain similar performance on existing small-scale benchmarks. This
paper re-evaluates state-of-the-art architectures in light of the new Kinetics
Human Action Video dataset. Kinetics has two orders of magnitude more data,
with 400 human action classes and over 400 clips per class, and is collected
from realistic, challenging YouTube videos. We provide an analysis on how
current architectures fare on the task of action classification on this dataset
and how much performance improves on the smaller benchmark datasets after
pre-training on Kinetics.
  We also introduce a new Two-Stream Inflated 3D ConvNet (I3D) that is based on
2D ConvNet inflation: filters and pooling kernels of very deep image
classification ConvNets are expanded into 3D, making it possible to learn
seamless spatio-temporal feature extractors from video while leveraging
successful ImageNet architecture designs and even their parameters. We show
that, after pre-training on Kinetics, I3D models considerably improve upon the
state-of-the-art in action classification, reaching 80.9% on HMDB-51 and 98.0%
on UCF-101.
Bridge type classification: supervised learning on a modified NBI dataset
Authors: Achyuthan Jootoo, David Lattanzi
Date: 2018-02-12 00:00:00
Tasks: Classification, feature selection, General Classification, Vocal Bursts Type Prediction

A key phase in the bridge design process is the selection of the structural
system. Due to budget and time constraints, engineers typically rely on
engineering judgment and prior experience when selecting a structural system,
often considering a limited range of design alternatives. The objective of this
study was to explore the suitability of supervised machine learning as a
preliminary design aid that provides guidance to engineers with regards to the
statistically optimal bridge type to choose, ultimately improving the
likelihood of optimized design, design standardization, and reduced maintenance
costs. In order to devise this supervised learning system, data for over
600,000 bridges from the National Bridge Inventory database were analyzed. Key
attributes for determining the bridge structure type were identified through
three feature selection techniques. Potentially useful attributes like seismic
intensity and historic data on the cost of materials (steel and concrete) were
then added from the US Geological Survey (USGS) database and Engineering News
Record. Decision tree, Bayes network and Support Vector Machines were used for
predicting the bridge design type. Due to state-to-state variations in material
availability, material costs, and design codes, supervised learning models
based on the complete data set did not yield favorable results. Supervised
learning models were then trained and tested using 10-fold cross validation on
data for each state. Inclusion of seismic data improved the model performance
noticeably. The data was then resampled to reduce the bias of the models
towards more common design types, and the supervised learning models thus
constructed showed further improvements in performance. The average recall and
precision for the state models was 88.6% and 88.0% using Decision Trees, 84.0%
and 83.7% using Bayesian Networks, and 80.8% and 75.6% using SVM.
A graph-embedded deep feedforward network for disease outcome classification and feature selection using gene expression data
Authors: Yunchuan Kong, Tianwei Yu
Date: 2018-01-18 00:00:00
Tasks: Classification, feature selection, General Classification, Robust classification, Sparse Learning

Gene expression data represents a unique challenge in predictive model
building, because of the small number of samples $(n)$ compared to the huge
amount of features $(p)$. This "$n<<p$" property has hampered application of
deep learning techniques for disease outcome classification. Sparse learning by
incorporating external gene network information could be a potential solution
to this issue. Still, the problem is very challenging because (1) there are
tens of thousands of features and only hundreds of training samples, (2) the
scale-free structure of the gene network is unfriendly to the setup of
convolutional neural networks. To address these issues and build a robust
classification model, we propose the Graph-Embedded Deep Feedforward Networks
(GEDFN), to integrate external relational information of features into the deep
neural network architecture. The method is able to achieve sparse connection
between network layers to prevent overfitting. To validate the method's
capability, we conducted both simulation experiments and a real data analysis
using a breast cancer RNA-seq dataset from The Cancer Genome Atlas (TCGA). The
resulting high classification accuracy and easily interpretable feature
selection results suggest the method is a useful addition to the current
classification models and feature selection procedures. The method is available
at https://github.com/yunchuankong/NetworkNeuralNetwork.
Dissecting Adam: The Sign, Magnitude and Variance of Stochastic Gradients
Authors: Lukas Balles, Philipp Hennig
Date: 2017-05-22 00:00:00
Proceeding: ICML 2018 7

The ADAM optimizer is exceedingly popular in the deep learning community. Often it works very well, sometimes it doesn't. Why? We interpret ADAM as a combination of two aspects: for each weight, the update direction is determined by the sign of stochastic gradients, whereas the update magnitude is determined by an estimate of their relative variance. We disentangle these two aspects and analyze them in isolation, gaining insight into the mechanisms underlying ADAM. This analysis also extends recent results on adverse effects of ADAM on generalization, isolating the sign aspect as the problematic one. Transferring the variance adaptation to SGD gives rise to a novel method, completing the practitioner's toolbox for problems where ADAM fails.
A General Pipeline for 3D Detection of Vehicles
Authors: Xinxin Du, Marcelo H. Ang Jr., Sertac Karaman, Daniela Rus
Date: 2018-02-12 00:00:00
Tasks: 3D Object Detection, Autonomous Driving, vehicle detection

Autonomous driving requires 3D perception of vehicles and other objects in
the in environment. Much of the current methods support 2D vehicle detection.
This paper proposes a flexible pipeline to adopt any 2D detection network and
fuse it with a 3D point cloud to generate 3D information with minimum changes
of the 2D detection networks. To identify the 3D box, an effective model
fitting algorithm is developed based on generalised car models and score maps.
A two-stage convolutional neural network (CNN) is proposed to refine the
detected 3D box. This pipeline is tested on the KITTI dataset using two
different 2D detection networks. The 3D detection results based on these two
networks are similar, demonstrating the flexibility of the proposed pipeline.
The results rank second among the 3D detection algorithms, indicating its
competencies in 3D detection.
Graph Classification with 2D Convolutional Neural Networks
Authors: Antoine Jean-Pierre Tixier, Giannis Nikolentzos, Polykarpos Meladianos, Michalis Vazirgiannis
Date: 2017-07-29 00:00:00
Proceeding: ICLR 2018 1
Tasks: Classification, General Classification, Graph Classification, Graph Learning

Graph learning is currently dominated by graph kernels, which, while powerful, suffer some significant limitations. Convolutional Neural Networks (CNNs) offer a very appealing alternative, but processing graphs with CNNs is not trivial. To address this challenge, many sophisticated extensions of CNNs have recently been introduced. In this paper, we reverse the problem: rather than proposing yet another graph CNN model, we introduce a novel way to represent graphs as multi-channel image-like structures that allows them to be handled by vanilla 2D CNNs. Experiments reveal that our method is more accurate than state-of-the-art graph kernels and graph CNNs on 4 out of 6 real-world datasets (with and without continuous node attributes), and close elsewhere. Our approach is also preferable to graph kernels in terms of time complexity. Code and data are publicly available.
Robust Particle Swarm Optimizer based on Chemomimicry
Authors: Casey Kneale, Karl S. Booksh
Date: 2017-02-03 00:00:00

A particle swarm optimizer (PSO) loosely based on the phenomena of
crystallization and a chaos factor which follows the complimentary error
function is described. The method features three phases: diffusion, directed
motion, and nucleation. During the diffusion phase random walk is the only
contributor to particle motion. As the algorithm progresses the contribution
from chaos decreases and movement toward global best locations is pursued until
convergence has occurred. The algorithm was found to be more robust to local
minima in multimodal test functions than a standard PSO algorithm and is
designed for problems which feature experimental precision.
Few-shot learning of neural networks from scratch by pseudo example optimization
Authors: Akisato Kimura, Zoubin Ghahramani, Koh Takeuchi, Tomoharu Iwata, Naonori Ueda
Date: 2018-02-08 00:00:00
Tasks: Few-Shot Learning, Knowledge Distillation

In this paper, we propose a simple but effective method for training neural
networks with a limited amount of training data. Our approach inherits the idea
of knowledge distillation that transfers knowledge from a deep or wide
reference model to a shallow or narrow target model. The proposed method
employs this idea to mimic predictions of reference estimators that are more
robust against overfitting than the network we want to train. Different from
almost all the previous work for knowledge distillation that requires a large
amount of labeled training data, the proposed method requires only a small
amount of training data. Instead, we introduce pseudo training examples that
are optimized as a part of model parameters. Experimental results for several
benchmark datasets demonstrate that the proposed method outperformed all the
other baselines, such as naive training of the target model and standard
knowledge distillation.
Deep learning based supervised semantic segmentation of Electron Cryo-Subtomograms
Authors: Chang Liu, Xiangrui Zeng, Ruogu Lin, Xiaodan Liang, Zachary Freyberg, Eric Xing, Min Xu
Date: 2018-02-12 00:00:00
Tasks: Decoder, Deep Learning, Segmentation, Semantic Segmentation

Cellular Electron Cryo-Tomography (CECT) is a powerful imaging technique for
the 3D visualization of cellular structure and organization at submolecular
resolution. It enables analyzing the native structures of macromolecular
complexes and their spatial organization inside single cells. However, due to
the high degree of structural complexity and practical imaging limitations,
systematic macromolecular structural recovery inside CECT images remains
challenging. Particularly, the recovery of a macromolecule is likely to be
biased by its neighbor structures due to the high molecular crowding. To reduce
the bias, here we introduce a novel 3D convolutional neural network inspired by
Fully Convolutional Network and Encoder-Decoder Architecture for the supervised
segmentation of macromolecules of interest in subtomograms. The tests of our
models on realistically simulated CECT data demonstrate that our new approach
has significantly improved segmentation performance compared to our baseline
approach. Also, we demonstrate that the proposed model has generalization
ability to segment new structures that do not exist in training data.
The Complex Event Recognition Group
Authors: Elias Alevizos, Alexander Artikis, Nikos Katzouris, Evangelos Michelioudakis, Georgios Paliouras
Date: 2018-02-12 00:00:00
Tasks: BIG-bench Machine Learning

The Complex Event Recognition (CER) group is a research team, affiliated with
the National Centre of Scientific Research "Demokritos" in Greece. The CER
group works towards advanced and efficient methods for the recognition of
complex events in a multitude of large, heterogeneous and interdependent data
streams. Its research covers multiple aspects of complex event recognition,
from efficient detection of patterns on event streams to handling uncertainty
and noise in streams, and machine learning techniques for inferring interesting
patterns. Lately, it has expanded to methods for forecasting the occurrence of
events. It was founded in 2009 and currently hosts 3 senior researchers, 5 PhD
students and works regularly with under-graduate students.
Augmenting End-to-End Dialog Systems with Commonsense Knowledge
Authors: Tom Young, Erik Cambria, Iti Chaturvedi, Minlie Huang, Hao Zhou, Subham Biswas
Date: 2017-09-16 00:00:00
Tasks: Retrieval

Building dialog agents that can converse naturally with humans is a
challenging yet intriguing problem of artificial intelligence. In open-domain
human-computer conversation, where the conversational agent is expected to
respond to human responses in an interesting and engaging way, commonsense
knowledge has to be integrated into the model effectively. In this paper, we
investigate the impact of providing commonsense knowledge about the concepts
covered in the dialog. Our model represents the first attempt to integrating a
large commonsense knowledge base into end-to-end conversational models. In the
retrieval-based scenario, we propose the Tri-LSTM model to jointly take into
account message and commonsense for selecting an appropriate response. Our
experiments suggest that the knowledge-augmented models are superior to their
knowledge-free counterparts in automatic evaluation.
Deep Convolutional Neural Networks on Cartoon Functions
Authors: Philipp Grohs, Thomas Wiatowski, Helmut Bölcskei
Date: 2016-04-29 00:00:00
Tasks: Translation

Wiatowski and B\"olcskei, 2015, proved that deformation stability and
vertical translation invariance of deep convolutional neural network-based
feature extractors are guaranteed by the network structure per se rather than
the specific convolution kernels and non-linearities. While the translation
invariance result applies to square-integrable functions, the deformation
stability bound holds for band-limited functions only. Many signals of
practical relevance (such as natural images) exhibit, however, sharp and curved
discontinuities and are, hence, not band-limited. The main contribution of this
paper is a deformation stability result that takes these structural properties
into account. Specifically, we establish deformation stability bounds for the
class of cartoon functions introduced by Donoho, 2001.
Inferring the time-varying functional connectivity of large-scale computer networks from emitted events
Authors: Antoine Messager, George Parisis, Istvan Z Kiss, Robert Harper, Phil Tee, Luc Berthouze
Date: 2018-02-12 00:00:00
Tasks: Functional Connectivity, Time Series, Time Series Analysis

We consider the problem of inferring the functional connectivity of a
large-scale computer network from sparse time series of events emitted by its
nodes. We do so under the following three domain-specific constraints: (a)
non-stationarity of the functional connectivity due to unknown temporal changes
in the network, (b) sparsity of the time-series of events that limits the
effectiveness of classical correlation-based analysis, and (c) lack of an
explicit model describing how events propagate through the network. Under the
assumption that the probability of two nodes being functionally connected
correlates with the mean delay between their respective events, we develop an
inference method whose output is an undirected weighted network where the
weight of an edge between two nodes denotes the probability of these nodes
being functionally connected. Using a combination of windowing and convolution
to calculate at each time window a score quantifying the likelihood of a pair
of nodes emitting events in quick succession, we develop a model of
time-varying connectivity whose parameters are determined by maximising the
model's predictive power from one time window to the next. To assess the
effectiveness of our inference method, we construct synthetic data for which
ground truth is available and use these data to benchmark our approach against
three state-of-the-art inference methods. We conclude by discussing its
application to data from a real-world large-scale computer network.
Average Size of Implicational Bases
Authors: Giacomo Kahn, Alexandre Bazin
Date: 2018-02-12 00:00:00

Implicational bases are objects of interest in formal concept analysis and
its applications. Unfortunately, even the smallest base, the Duquenne-Guigues
base, has an exponential size in the worst case. In this paper, we use results
on the average number of minimal transversals in random hypergraphs to show
that the base of proper premises is, on average, of quasi-polynomial size.
Introducer Concepts in n-Dimensional Contexts
Authors: Giacomo Kahn, Alexandre Bazin
Date: 2018-02-12 00:00:00

Concept lattices are well-known conceptual structures that organise
interesting patterns-the concepts-extracted from data. In some applications,
such as software engineering or data mining, the size of the lattice can be a
problem, as it is often too large to be efficiently computed, and too complex
to be browsed. For this reason, the Galois Sub-Hierarchy, a restriction of the
concept lattice to introducer concepts, has been introduced as a smaller
alternative. In this paper, we generalise the Galois Sub-Hierarchy to
n-lattices, conceptual structures obtained from multidimensional data in the
same way that concept lattices are obtained from binary relations.
Automatic Generation of Language-Independent Features for Cross-Lingual Classification
Authors: Sarai Duek, Shaul Markovitch
Date: 2018-02-12 00:00:00
Tasks: Classification, General Classification, Text Categorization

Many applications require categorization of text documents using predefined
categories. The main approach to performing text categorization is learning
from labeled examples. For many tasks, it may be difficult to find examples in
one language but easy in others. The problem of learning from examples in one
or more languages and classifying (categorizing) in another is called
cross-lingual learning. In this work, we present a novel approach that solves
the general cross-lingual text categorization problem. Our method generates,
for each training document, a set of language-independent features. Using these
features for training yields a language-independent classifier. At the
classification stage, we generate language-independent features for the
unlabeled document, and apply the classifier on the new representation.
  To build the feature generator, we utilize a hierarchical
language-independent ontology, where each concept has a set of support
documents for each language involved. In the preprocessing stage, we use the
support documents to build a set of language-independent feature generators,
one for each language. The collection of these generators is used to map any
document into the language-independent feature space.
  Our methodology works on the most general cross-lingual text categorization
problems, being able to learn from any mix of languages and classify documents
in any other language. We also present a method for exploiting the hierarchical
structure of the ontology to create virtual supporting documents for languages
that do not have them. We tested our method, using Wikipedia as our ontology,
on the most commonly used test collections in cross-lingual text
categorization, and found that it outperforms existing methods.
Fair and Diverse DPP-based Data Summarization
Authors: L. Elisa Celis, Vijay Keswani, Damian Straszak, Amit Deshpande, Tarun Kathuria, Nisheeth K. Vishnoi
Date: 2018-02-12 00:00:00
Proceeding: ICML 2018 7
Tasks: Data Summarization, Diversity, Fairness

Sampling methods that choose a subset of the data proportional to its
diversity in the feature space are popular for data summarization. However,
recent studies have noted the occurrence of bias (under- or over-representation
of a certain gender or race) in such data summarization methods. In this paper
we initiate a study of the problem of outputting a diverse and fair summary of
a given dataset. We work with a well-studied determinantal measure of diversity
and corresponding distributions (DPPs) and present a framework that allows us
to incorporate a general class of fairness constraints into such distributions.
Coming up with efficient algorithms to sample from these constrained
determinantal distributions, however, suffers from a complexity barrier and we
present a fast sampler that is provably good when the input vectors satisfy a
natural property. Our experimental results on a real-world and an image dataset
show that the diversity of the samples produced by adding fairness constraints
is not too far from the unconstrained case, and we also provide a theoretical
explanation of it.
Efficient Bias-Span-Constrained Exploration-Exploitation in Reinforcement Learning
Authors: Ronan Fruit, Matteo Pirotta, Alessandro Lazaric, Ronald Ortner
Date: 2018-02-12 00:00:00
Proceeding: ICML 2018 7
Tasks: Efficient Exploration, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We introduce SCAL, an algorithm designed to perform efficient
exploration-exploitation in any unknown weakly-communicating Markov decision
process (MDP) for which an upper bound $c$ on the span of the optimal bias
function is known. For an MDP with $S$ states, $A$ actions and $\Gamma \leq S$
possible next states, we prove a regret bound of $\widetilde{O}(c\sqrt{\Gamma
SAT})$, which significantly improves over existing algorithms (e.g., UCRL and
PSRL), whose regret scales linearly with the MDP diameter $D$. In fact, the
optimal bias span is finite and often much smaller than $D$ (e.g., $D=\infty$
in non-communicating MDPs). A similar result was originally derived by Bartlett
and Tewari (2009) for REGAL.C, for which no tractable algorithm is available.
In this paper, we relax the optimization problem at the core of REGAL.C, we
carefully analyze its properties, and we provide the first computationally
efficient algorithm to solve it. Finally, we report numerical simulations
supporting our theoretical findings and showing how SCAL significantly
outperforms UCRL in MDPs with large diameter and small span.
Image Inpainting for High-Resolution Textures using CNN Texture Synthesis
Authors: Pascal Laube, Michael Grunwald, Matthias O. Franz, Georg Umlauf
Date: 2017-12-08 00:00:00
Tasks: Image Inpainting, Image Segmentation, Image Super-Resolution, Semantic Segmentation, Super-Resolution, Texture Synthesis, Vocal Bursts Intensity Prediction

Deep neural networks have been successfully applied to problems such as image
segmentation, image super-resolution, coloration and image inpainting. In this
work we propose the use of convolutional neural networks (CNN) for image
inpainting of large regions in high-resolution textures. Due to limited
computational resources processing high-resolution images with neural networks
is still an open problem. Existing methods separate inpainting of global
structure and the transfer of details, which leads to blurry results and loss
of global coherence in the detail transfer step. Based on advances in texture
synthesis using CNNs we propose patch-based image inpainting by a CNN that is
able to optimize for global as well as detail texture statistics. Our method is
capable of filling large inpainting regions, oftentimes exceeding the quality
of comparable methods for high-resolution images. For reference patch look-up
we propose to use the same summary statistics that are used in the inpainting
process.