pwc-archive/papers-with-abstracts huggingface dataset:
Computational Intelligence in Sports: A Systematic Literature Review
Authors: Robson P. Bonidia, Luiz A. L. Rodrigues, Anderson P. Avila-Santos, Danilo S. Sanches, Jacques D. Brancher
Date: 2018-10-30 00:00:00
Tasks: Systematic Literature Review

Recently, data mining studies are being successfully conducted to estimate
several parameters in a variety of domains. Data mining techniques have
attracted the attention of the information industry and society as a whole, due
to a large amount of data and the imminent need to turn it into useful
knowledge. However, the effective use of data in some areas is still under
development, as is the case in sports, which in recent years, has presented a
slight growth; consequently, many sports organizations have begun to see that
there is a wealth of unexplored knowledge in the data extracted by them.
Therefore, this article presents a systematic review of sports data mining.
Regarding years 2010 to 2018, 31 types of research were found in this topic.
Based on these studies, we present the current panorama, themes, the database
used, proposals, algorithms, and research opportunities. Our findings provide a
better understanding of the sports data mining potentials, besides motivating
the scientific community to explore this timely and interesting topic.
AI for the Common Good?! Pitfalls, challenges, and Ethics Pen-Testing
Authors: Bettina Berendt
Date: 2018-10-30 00:00:00
Tasks: Ethics

Recently, many AI researchers and practitioners have embarked on research
visions that involve doing AI for "Good". This is part of a general drive
towards infusing AI research and practice with ethical thinking. One frequent
theme in current ethical guidelines is the requirement that AI be good for all,
or: contribute to the Common Good. But what is the Common Good, and is it
enough to want to be good? Via four lead questions, I will illustrate
challenges and pitfalls when determining, from an AI point of view, what the
Common Good is and how it can be enhanced by AI. The questions are: What is the
problem / What is a problem?, Who defines the problem?, What is the role of
knowledge?, and What are important side effects and dynamics? The illustration
will use an example from the domain of "AI for Social Good", more specifically
"Data Science for Social Good". Even if the importance of these questions may
be known at an abstract level, they do not get asked sufficiently in practice,
as shown by an exploratory study of 99 contributions to recent conferences in
the field. Turning these challenges and pitfalls into a positive
recommendation, as a conclusion I will draw on another characteristic of
computer-science thinking and practice to make these impediments visible and
attenuate them: "attacks" as a method for improving design. This results in the
proposal of ethics pen-testing as a method for helping AI designs to better
contribute to the Common Good.
Learning Cross-Lingual Sentence Representations via a Multi-task Dual-Encoder Model
Authors: Muthuraman Chidambaram, Yinfei Yang, Daniel Cer, Steve Yuan, Yun-Hsuan Sung, Brian Strope, Ray Kurzweil
Date: 2018-10-30 00:00:00
Proceeding: WS 2019 8
Tasks: Few-Shot Learning, Language Modeling, Language Modelling, Sentence

A significant roadblock in multilingual neural language modeling is the lack of labeled non-English data. One potential method for overcoming this issue is learning cross-lingual text representations that can be used to transfer the performance from training on English tasks to non-English tasks, despite little to no task-specific non-English data. In this paper, we explore a natural setup for learning cross-lingual sentence representations: the dual-encoder. We provide a comprehensive evaluation of our cross-lingual representations on a number of monolingual, cross-lingual, and zero-shot/few-shot learning tasks, and also give an analysis of different learned cross-lingual embedding spaces.
General audio tagging with ensembling convolutional neural network and statistical features
Authors: Kele Xu, Boqing Zhu, Qiuqiang Kong, Haibo Mi, Bo Ding, Dezhi Wang, Huaimin Wang
Date: 2018-10-30 00:00:00
Tasks: Audio Tagging, Descriptive, Ensemble Learning, Task 2

Audio tagging aims to infer descriptive labels from audio clips. Audio
tagging is challenging due to the limited size of data and noisy labels. In
this paper, we describe our solution for the DCASE 2018 Task 2 general audio
tagging challenge. The contributions of our solution include: We investigated a
variety of convolutional neural network architectures to solve the audio
tagging task. Statistical features are applied to capture statistical patterns
of audio features to improve the classification performance. Ensemble learning
is applied to ensemble the outputs from the deep classifiers to utilize
complementary information. a sample re-weight strategy is employed for ensemble
training to address the noisy label problem. Our system achieves a mean average
precision (mAP@3) of 0.958, outperforming the baseline system of 0.704. Our
system ranked the 1st and 4th out of 558 submissions in the public and private
leaderboard of DCASE 2018 Task 2 challenge. Our codes are available at
https://github.com/Cocoxili/DCASE2018Task2/.
Cross-Modal Attentional Context Learning for RGB-D Object Detection
Authors: Guanbin Li, Yukang Gan, Hejun Wu, Nong Xiao, Liang Lin
Date: 2018-10-30 00:00:00
Tasks: Autonomous Driving, Object, object-detection, Object Detection

Recognizing objects from simultaneously sensed photometric (RGB) and depth
channels is a fundamental yet practical problem in many machine vision
applications such as robot grasping and autonomous driving. In this paper, we
address this problem by developing a Cross-Modal Attentional Context (CMAC)
learning framework, which enables the full exploitation of the context
information from both RGB and depth data. Compared to existing RGB-D object
detection frameworks, our approach has several appealing properties. First, it
consists of an attention-based global context model for exploiting adaptive
contextual information and incorporating this information into a region-based
CNN (e.g., Fast RCNN) framework to achieve improved object detection
performance. Second, our CMAC framework further contains a fine-grained object
part attention module to harness multiple discriminative object parts inside
each possible object region for superior local feature representation. While
greatly improving the accuracy of RGB-D object detection, the effective
cross-modal information fusion as well as attentional context modeling in our
proposed model provide an interpretable visualization scheme. Experimental
results demonstrate that the proposed method significantly improves upon the
state of the art on all public benchmarks.
DeepTwist: Learning Model Compression via Occasional Weight Distortion
Authors: Dongsoo Lee, Parichay Kapoor, Byeongwook Kim
Date: 2018-10-30 00:00:00
Tasks: model, Model Compression, Quantization

Model compression has been introduced to reduce the required hardware
resources while maintaining the model accuracy. Lots of techniques for model
compression, such as pruning, quantization, and low-rank approximation, have
been suggested along with different inference implementation characteristics.
Adopting model compression is, however, still challenging because the design
complexity of model compression is rapidly increasing due to additional
hyper-parameters and computation overhead in order to achieve a high
compression ratio. In this paper, we propose a simple and efficient model
compression framework called DeepTwist which distorts weights in an occasional
manner without modifying the underlying training algorithms. The ideas of
designing weight distortion functions are intuitive and straightforward given
formats of compressed weights. We show that our proposed framework improves
compression rate significantly for pruning, quantization, and low-rank
approximation techniques while the efforts of additional retraining and/or
hyper-parameter search are highly reduced. Regularization effects of DeepTwist
are also reported.
Informed Democracy: Voting-based Novelty Detection for Action Recognition
Authors: Alina Roitberg, Ziad Al-Halah, Rainer Stiefelhagen
Date: 2018-10-30 00:00:00
Tasks: Action Classification, Action Recognition, Activity Recognition, Novelty Detection, Temporal Action Localization, Zero-Shot Learning

Novelty detection is crucial for real-life applications. While it is common
in activity recognition to assume a closed-set setting, i.e. test samples are
always of training categories, this assumption is impractical in a real-world
scenario. Test samples can be of various categories including those never seen
before during training. Thus, being able to know what we know and what we do
not know is decisive for the model to avoid what can be catastrophic
consequences. We present in this work a novel approach for identifying samples
of activity classes that are not previously seen by the classifier. Our model
employs a voting-based scheme that leverages the estimated uncertainty of the
individual classifiers in their predictions to measure the novelty of a new
input sample. Furthermore, the voting is privileged to a subset of informed
classifiers that can best estimate whether a sample is novel or not when it is
classified to a certain known category. In a thorough evaluation on UCF-101 and
HMDB-51, we show that our model consistently outperforms state-of-the-art in
novelty detection. Additionally, by combining our model with off-the-shelf
zero-shot learning (ZSL) approaches, our model leads to a significant
improvement in action classification accuracy for the generalized ZSL setting.
Band gap prediction for large organic crystal structures with machine learning
Authors: Bart Olsthoorn, R. Matthias Geilhufe, Stanislav S. Borysov, Alexander V. Balatsky
Date: 2018-10-30 00:00:00
Tasks: Band Gap, BIG-bench Machine Learning

Machine-learning models are capable of capturing the structure-property relationship from a dataset of computationally demanding ab initio calculations. Over the past two years, the Organic Materials Database (OMDB) has hosted a growing number of calculated electronic properties of previously synthesized organic crystal structures. The complexity of the organic crystals contained within the OMDB, which have on average 82 atoms per unit cell, makes this database a challenging platform for machine learning applications. In this paper, the focus is on predicting the band gap which represents one of the basic properties of a crystalline materials. With this aim, a consistent dataset of 12 500 crystal structures and their corresponding DFT band gap are released, freely available for download at https://omdb.mathub.io/dataset. An ensemble of two state-of-the-art models reach a mean absolute error (MAE) of 0.388 eV, which corresponds to a percentage error of 13% for an average band gap of 3.05 eV. Finally, the trained models are employed to predict the band gap for 260 092 materials contained within the Crystallography Open Database (COD) and made available online so that the predictions can be obtained for any arbitrary crystal structure uploaded by a user.
Contextual Hourglass Network for Semantic Segmentation of High Resolution Aerial Imagery
Authors: Panfeng Li, Youzuo Lin, Emily Schultz-Fellenz
Date: 2018-10-30 00:00:00
Tasks: Decoder, Segmentation, Semantic Segmentation, Vocal Bursts Intensity Prediction

Semantic segmentation for aerial imagery is a challenging and important problem in remotely sensed imagery analysis. In recent years, with the success of deep learning, various convolutional neural network (CNN) based models have been developed. However, due to the varying sizes of the objects and imbalanced class labels, it can be challenging to obtain accurate pixel-wise semantic segmentation results. To address those challenges, we develop a novel semantic segmentation method and call it Contextual Hourglass Network. In our method, in order to improve the robustness of the prediction, we design a new contextual hourglass module which incorporates attention mechanism on processed low-resolution featuremaps to exploit the contextual semantics. We further exploit the stacked encoder-decoder structure by connecting multiple contextual hourglass modules from end to end. This architecture can effectively extract rich multi-scale features and add more feedback loops for better learning contextual semantics through intermediate supervision. To demonstrate the efficacy of our semantic segmentation method, we test it on Potsdam and Vaihingen datasets. Through the comparisons to other baseline methods, our method yields the best results on overall performance.
Piecewise Strong Convexity of Neural Networks
Authors: Tristan Milne
Date: 2018-10-30 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: image-classification, Image Classification, Learning Theory

We study the loss surface of a feed-forward neural network with ReLU non-linearities, regularized with weight decay. We show that the regularized loss function is piecewise strongly convex on an important open set which contains, under some conditions, all of its global minimizers. This is used to prove that local minima of the regularized loss function in this set are isolated, and that every differentiable critical point in this set is a local minimum, partially addressing an open problem given at the Conference on Learning Theory (COLT) 2015; our result is also applied to linear neural networks to show that with weight decay regularization, there are no non-zero critical points in a norm ball obtaining training error below a given threshold. We also include an experimental section where we validate our theoretical work and show that the regularized loss function is almost always piecewise strongly convex when restricted to stochastic gradient descent trajectories for three standard image classification problems.
Transferable Positive/Negative Speech Emotion Recognition via Class-wise Adversarial Domain Adaptation
Authors: Hao Zhou, Ke Chen
Date: 2018-10-30 00:00:00
Tasks: Domain Adaptation, Emotion Recognition, Speech Emotion Recognition

Speech emotion recognition plays an important role in building more
intelligent and human-like agents. Due to the difficulty of collecting speech
emotional data, an increasingly popular solution is leveraging a related and
rich source corpus to help address the target corpus. However, domain shift
between the corpora poses a serious challenge, making domain shift adaptation
difficult to function even on the recognition of positive/negative emotions. In
this work, we propose class-wise adversarial domain adaptation to address this
challenge by reducing the shift for all classes between different corpora.
Experiments on the well-known corpora EMODB and Aibo demonstrate that our
method is effective even when only a very limited number of target labeled
examples are provided.
Advancing PICO Element Detection in Biomedical Text via Deep Neural Networks
Authors: Di Jin, Peter Szolovits
Date: 2018-10-30 00:00:00
Tasks: feature selection, PICO, Sentence, Unsupervised Pre-training

In evidence-based medicine (EBM), defining a clinical question in terms of the specific patient problem aids the physicians to efficiently identify appropriate resources and search for the best available evidence for medical treatment. In order to formulate a well-defined, focused clinical question, a framework called PICO is widely used, which identifies the sentences in a given medical text that belong to the four components typically reported in clinical trials: Participants/Problem (P), Intervention (I), Comparison (C) and Outcome (O). In this work, we propose a novel deep learning model for recognizing PICO elements in biomedical abstracts. Based on the previous state-of-the-art bidirectional long-short term memory (biLSTM) plus conditional random field (CRF) architecture, we add another layer of biLSTM upon the sentence representation vectors so that the contextual information from surrounding sentences can be gathered to help infer the interpretation of the current one. In addition, we propose two methods to further generalize and improve the model: adversarial training and unsupervised pre-training over large corpora. We tested our proposed approach over two benchmark datasets. One is the PubMed-PICO dataset, where our best results outperform the previous best by 5.5%, 7.9%, and 5.8% for P, I, and O elements in terms of F1 score, respectively. And for the other dataset named NICTA-PIBOSO, the improvements for P/I/O elements are 2.4%, 13.6%, and 1.0% in F1 score, respectively. Overall, our proposed deep learning model can obtain unprecedented PICO element detection accuracy while avoiding the need for any manual feature selection.
Reinforcement Learning and Deep Learning based Lateral Control for Autonomous Driving
Authors: Dong Li, Dongbin Zhao, Qichao Zhang, Yaran Chen
Date: 2018-10-30 00:00:00
Tasks: Autonomous Driving, Deep Learning, Deep Reinforcement Learning, Model Predictive Control, Multi-Task Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

This paper investigates the vision-based autonomous driving with deep
learning and reinforcement learning methods. Different from the end-to-end
learning method, our method breaks the vision-based lateral control system down
into a perception module and a control module. The perception module which is
based on a multi-task learning neural network first takes a driver-view image
as its input and predicts the track features. The control module which is based
on reinforcement learning then makes a control decision based on these
features. In order to improve the data efficiency, we propose visual TORCS
(VTORCS), a deep reinforcement learning environment which is based on the open
racing car simulator (TORCS). By means of the provided functions, one can train
an agent with the input of an image or various physical sensor measurement, or
evaluate the perception algorithm on this simulator. The trained reinforcement
learning controller outperforms the linear quadratic regulator (LQR) controller
and model predictive control (MPC) controller on different tracks. The
experiments demonstrate that the perception module shows promising performance
and the controller is capable of controlling the vehicle drive well along the
track center with visual input.
Recurrent Attention Unit
Authors: Guoqiang Zhong, Guohua Yue, Xiao Ling
Date: 2018-10-30 00:00:00
Tasks: General Classification, Handwriting Recognition, image-classification, Image Classification, Image Description, Language Modeling, Language Modelling, Sentiment Analysis, Sentiment Classification

Recurrent Neural Network (RNN) has been successfully applied in many sequence
learning problems. Such as handwriting recognition, image description, natural
language processing and video motion analysis. After years of development,
researchers have improved the internal structure of the RNN and introduced many
variants. Among others, Gated Recurrent Unit (GRU) is one of the most widely
used RNN model. However, GRU lacks the capability of adaptively paying
attention to certain regions or locations, so that it may cause information
redundancy or loss during leaning. In this paper, we propose a RNN model,
called Recurrent Attention Unit (RAU), which seamlessly integrates the
attention mechanism into the interior of GRU by adding an attention gate. The
attention gate can enhance GRU's ability to remember long-term memory and help
memory cells quickly discard unimportant content. RAU is capable of extracting
information from the sequential data by adaptively selecting a sequence of
regions or locations and pay more attention to the selected regions during
learning. Extensive experiments on image classification, sentiment
classification and language modeling show that RAU consistently outperforms GRU
and other baseline methods.
Long Short-Term Attention
Authors: Guoqiang Zhong, Xin Lin, Kang Chen, Qingyang Li, Kai-Zhu Huang
Date: 2018-10-30 00:00:00

Attention is an important cognition process of humans, which helps humans concentrate on critical information during their perception and learning. However, although many machine learning models can remember information of data, they have no the attention mechanism. For example, the long short-term memory (LSTM) network is able to remember sequential information, but it cannot pay special attention to part of the sequences. In this paper, we present a novel model called long short-term attention (LSTA), which seamlessly integrates the attention mechanism into the inner cell of LSTM. More than processing long short term dependencies, LSTA can focus on important information of the sequences with the attention mechanism. Extensive experiments demonstrate that LSTA outperforms LSTM and related models on the sequence learning tasks.
Gaussian Process Conditional Density Estimation
Authors: Vincent Dutordoir, Hugh Salimbeni, Marc Deisenroth, James Hensman
Date: 2018-10-30 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Density Estimation, Few-Shot Learning, Gaussian Processes, Variational Inference

Conditional Density Estimation (CDE) models deal with estimating conditional
distributions. The conditions imposed on the distribution are the inputs of the
model. CDE is a challenging task as there is a fundamental trade-off between
model complexity, representational capacity and overfitting. In this work, we
propose to extend the model's input with latent variables and use Gaussian
processes (GP) to map this augmented input onto samples from the conditional
distribution. Our Bayesian approach allows for the modeling of small datasets,
but we also provide the machinery for it to be applied to big data using
stochastic variational inference. Our approach can be used to model densities
even in sparse data regions, and allows for sharing learned structure between
conditions. We illustrate the effectiveness and wide-reaching applicability of
our model on a variety of real-world problems, such as spatio-temporal density
estimation of taxi drop-offs, non-Gaussian noise modeling, and few-shot
learning on omniglot images.
Cluster Size Management in Multi-Stage Agglomerative Hierarchical Clustering of Acoustic Speech Segments
Authors: Lerato Lerato, Thomas Niesler
Date: 2018-10-30 00:00:00
Tasks: Clustering, Management

Agglomerative hierarchical clustering (AHC) requires only the similarity
between objects to be known. This is attractive when clustering signals of
varying length, such as speech, which are not readily represented in
fixed-dimensional vector space. However, AHC is characterised by $O(N^2)$ space
and time complexity, making it infeasible for partitioning large datasets. This
has recently been addressed by an approach based on the iterative re-clustering
of independent subsets of the larger dataset. We show that, due to its
iterative nature, this procedure can sometimes lead to unchecked growth of
individual subsets, thereby compromising its effectiveness. We propose the
integration of a simple space management strategy into the iterative process,
and show experimentally that this leads to no loss in performance in terms of
F-measure while guaranteeing that a threshold space complexity is not breached.
Efficient Tree Solver for Hines Matrices on the GPU
Authors: Felix Huber
Date: 2018-10-30 00:00:00
Tasks: GPU

The human brain consists of a large number of interconnected neurons
communicating via exchange of electrical spikes. Simulations play an important
role in better understanding electrical activity in the brain and offers a way
to to compare measured data to simulated data such that experimental data can
be interpreted better. A key component in such simulations is an efficient
solver for the Hines matrices used in computing inter-neuron signal
propagation. In order to achieve high performance simulations, it is crucial to
have an efficient solver algorithm. In this report we explain a new parallel
GPU solver for these matrices which offers fine grained parallelization and
allows for work balancing during the simulation setup.