pwc-archive/papers-with-abstracts huggingface dataset:
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference
Authors: Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, Dmitry Kalenichenko
Date: 2017-12-15 00:00:00
Proceeding: CVPR 2018 6
Tasks: General Classification, Quantization

The rising popularity of intelligent mobile devices and the daunting
computational cost of deep learning-based models call for efficient and
accurate on-device inference schemes. We propose a quantization scheme that
allows inference to be carried out using integer-only arithmetic, which can be
implemented more efficiently than floating point inference on commonly
available integer-only hardware. We also co-design a training procedure to
preserve end-to-end model accuracy post quantization. As a result, the proposed
quantization scheme improves the tradeoff between accuracy and on-device
latency. The improvements are significant even on MobileNets, a model family
known for run-time efficiency, and are demonstrated in ImageNet classification
and COCO detection on popular CPUs.
WACSF - Weighted Atom-Centered Symmetry Functions as Descriptors in Machine Learning Potentials
Authors: Michael Gastegger, Ludwig Schwiedrzik, Marius Bittermann, Florian Berzsenyi, Philipp Marquetand
Date: 2017-12-15 00:00:00
Tasks: BIG-bench Machine Learning

We introduce weighted atom-centered symmetry functions (wACSFs) as
descriptors of a chemical system's geometry for use in the prediction of
chemical properties such as enthalpies or potential energies via machine
learning. The wACSFs are based on conventional atom-centered symmetry functions
(ACSFs) but overcome the undesirable scaling of the latter with increasing
number of different elements in a chemical system. The performance of these two
descriptors is compared using them as inputs in high-dimensional neural network
potentials (HDNNPs), employing the molecular structures and associated
enthalpies of the 133855 molecules containing up to five different elements
reported in the QM9 database as reference data. A substantially smaller number
of wACSFs than ACSFs is needed to obtain a comparable spatial resolution of the
molecular structures. At the same time, this smaller set of wACSFs leads to
significantly better generalization performance in the machine learning
potential than the large set of conventional ACSFs. Furthermore, we show that
the intrinsic parameters of the descriptors can in principle be optimized with
a genetic algorithm in a highly automated manner. For the wACSFs employed here,
we find however that using a simple empirical parametrization scheme is
sufficient in order to obtain HDNNPs with high accuracy.
A Berkeley View of Systems Challenges for AI
Authors: Ion Stoica, Dawn Song, Raluca Ada Popa, David Patterson, Michael W. Mahoney, Randy Katz, Anthony D. Joseph, Michael Jordan, Joseph M. Hellerstein, Joseph E. Gonzalez, Ken Goldberg, Ali Ghodsi, David Culler, Pieter Abbeel
Date: 2017-12-15 00:00:00
Tasks: Machine Translation, speech-recognition, Speech Recognition

With the increasing commoditization of computer vision, speech recognition
and machine translation systems and the widespread deployment of learning-based
back-end technologies such as digital advertising and intelligent
infrastructures, AI (Artificial Intelligence) has moved from research labs to
production. These changes have been made possible by unprecedented levels of
data and computation, by methodological advances in machine learning, by
innovations in systems software and architectures, and by the broad
accessibility of these technologies.
  The next generation of AI systems promises to accelerate these developments
and increasingly impact our lives via frequent interactions and making (often
mission-critical) decisions on our behalf, often in highly personalized
contexts. Realizing this promise, however, raises daunting challenges. In
particular, we need AI systems that make timely and safe decisions in
unpredictable environments, that are robust against sophisticated adversaries,
and that can process ever increasing amounts of data across organizations and
individuals without compromising confidentiality. These challenges will be
exacerbated by the end of the Moore's Law, which will constrain the amount of
data these technologies can store and process. In this paper, we propose
several open research directions in systems, architectures, and security that
can address these challenges and help unlock AI's potential to improve lives
and society.
End-to-end Neural Coreference Resolution
Authors: Kenton Lee, Luheng He, Mike Lewis, Luke Zettlemoyer
Date: 2017-07-21 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Coreference Resolution

We introduce the first end-to-end coreference resolution model and show that
it significantly outperforms all previous work without using a syntactic parser
or hand-engineered mention detector. The key idea is to directly consider all
spans in a document as potential mentions and learn distributions over possible
antecedents for each. The model computes span embeddings that combine
context-dependent boundary representations with a head-finding attention
mechanism. It is trained to maximize the marginal likelihood of gold antecedent
spans from coreference clusters and is factored to enable aggressive pruning of
potential mentions. Experiments demonstrate state-of-the-art performance, with
a gain of 1.5 F1 on the OntoNotes benchmark and by 3.1 F1 using a 5-model
ensemble, despite the fact that this is the first approach to be successfully
trained with no external resources.
Mapping the world population one building at a time
Authors: Tobias G. Tiecke, Xian-Ming Liu, Amy Zhang, Andreas Gros, Nan Li, Gregory Yetman, Talip Kilic, Siobhan Murray, Brian Blankespoor, Espen B. Prydz, Hai-Anh H. Dang
Date: 2017-12-15 00:00:00
Tasks: Disaster Response, Sensitivity

High resolution datasets of population density which accurately map
sparsely-distributed human populations do not exist at a global scale.
Typically, population data is obtained using censuses and statistical modeling.
More recently, methods using remotely-sensed data have emerged, capable of
effectively identifying urbanized areas. Obtaining high accuracy in estimation
of population distribution in rural areas remains a very challenging task due
to the simultaneous requirements of sufficient sensitivity and resolution to
detect very sparse populations through remote sensing as well as reliable
performance at a global scale. Here, we present a computer vision method based
on machine learning to create population maps from satellite imagery at a
global scale, with a spatial sensitivity corresponding to individual buildings
and suitable for global deployment. By combining this settlement data with
census data, we create population maps with ~30 meter resolution for 18
countries. We validate our method, and find that the building identification
has an average precision and recall of 0.95 and 0.91, respectively and that the
population estimates have a standard error of a factor ~2 or less. Based on our
data, we analyze 29 percent of the world population, and show that 99 percent
lives within 36 km of the nearest urban cluster. The resulting high-resolution
population datasets have applications in infrastructure planning, vaccination
campaign planning, disaster response efforts and risk analysis such as high
accuracy flood risk analysis.
Deep Learning for Unsupervised Insider Threat Detection in Structured Cybersecurity Data Streams
Authors: Aaron Tuor, Samuel Kaplan, Brian Hutchinson, Nicole Nichols, Sean Robinson
Date: 2017-10-02 00:00:00
Tasks: Anomaly Detection

Analysis of an organization's computer network activity is a key component of
early detection and mitigation of insider threat, a growing concern for many
organizations. Raw system logs are a prototypical example of streaming data
that can quickly scale beyond the cognitive power of a human analyst. As a
prospective filter for the human analyst, we present an online unsupervised
deep learning approach to detect anomalous network activity from system logs in
real time. Our models decompose anomaly scores into the contributions of
individual user behavior features for increased interpretability to aid
analysts reviewing potential cases of insider threat. Using the CERT Insider
Threat Dataset v6.2 and threat detection recall as our performance metric, our
novel deep and recurrent neural network models outperform Principal Component
Analysis, Support Vector Machine and Isolation Forest based anomaly detection
baselines. For our best model, the events labeled as insider threat activity in
our dataset had an average anomaly score in the 95.53 percentile, demonstrating
our approach's potential to greatly reduce analyst workloads.
Realistic Traffic Generation for Web Robots
Authors: Kyle Brown, Derek Doran
Date: 2017-12-15 00:00:00

Critical to evaluating the capacity, scalability, and availability of web
systems are realistic web traffic generators. Web traffic generation is a
classic research problem, no generator accounts for the characteristics of web
robots or crawlers that are now the dominant source of traffic to a web server.
Administrators are thus unable to test, stress, and evaluate how their systems
perform in the face of ever increasing levels of web robot traffic. To resolve
this problem, this paper introduces a novel approach to generate synthetic web
robot traffic with high fidelity. It generates traffic that accounts for both
the temporal and behavioral qualities of robot traffic by statistical and
Bayesian models that are fitted to the properties of robot traffic seen in web
logs from North America and Europe. We evaluate our traffic generator by
comparing the characteristics of generated traffic to those of the original
data. We look at session arrival rates, inter-arrival times and session
lengths, comparing and contrasting them between generated and real traffic.
Finally, we show that our generated traffic affects cache performance similarly
to actual traffic, using the common LRU and LFU eviction policies.
Deep Burst Denoising
Authors: Clément Godard, Kevin Matzen, Matt Uyttendaele
Date: 2017-12-15 00:00:00
Proceeding: ECCV 2018 9
Tasks: Denoising, Image Denoising, Image Enhancement, Image Super-Resolution, Super-Resolution

Noise is an inherent issue of low-light image capture, one which is
exacerbated on mobile devices due to their narrow apertures and small sensors.
One strategy for mitigating noise in a low-light situation is to increase the
shutter time of the camera, thus allowing each photosite to integrate more
light and decrease noise variance. However, there are two downsides of long
exposures: (a) bright regions can exceed the sensor range, and (b) camera and
scene motion will result in blurred images. Another way of gathering more light
is to capture multiple short (thus noisy) frames in a "burst" and intelligently
integrate the content, thus avoiding the above downsides. In this paper, we use
the burst-capture strategy and implement the intelligent integration via a
recurrent fully convolutional deep neural net (CNN). We build our novel,
multiframe architecture to be a simple addition to any single frame denoising
model, and design to handle an arbitrary number of noisy input frames. We show
that it achieves state of the art denoising results on our burst dataset,
improving on the best published multi-frame techniques, such as VBM4D and
FlexISP. Finally, we explore other applications of image enhancement by
integrating content from multiple frames and demonstrate that our DNN
architecture generalizes well to image super-resolution.
Multi-Attribute Robust Component Analysis for Facial UV Maps
Authors: Stylianos Moschoglou, Evangelos Ververas, Yannis Panagakis, Mihalis Nicolaou, Stefanos Zafeiriou
Date: 2017-12-15 00:00:00
Tasks: 3D Face Alignment, Attribute, Face Alignment, Face Model

Recently, due to the collection of large scale 3D face models, as well as the
advent of deep learning, a significant progress has been made in the field of
3D face alignment "in-the-wild". That is, many methods have been proposed that
establish sparse or dense 3D correspondences between a 2D facial image and a 3D
face model. The utilization of 3D face alignment introduces new challenges and
research directions, especially on the analysis of facial texture images. In
particular, texture does not suffer any more from warping effects (that
occurred when 2D face alignment methods were used). Nevertheless, since facial
images are commonly captured in arbitrary recording conditions, a considerable
amount of missing information and gross outliers is observed (e.g., due to
self-occlusion, or subjects wearing eye-glasses). Given that many annotated
databases have been developed for face analysis tasks, it is evident that
component analysis techniques need to be developed in order to alleviate issues
arising from the aforementioned challenges. In this paper, we propose a novel
component analysis technique that is suitable for facial UV maps containing a
considerable amount of missing information and outliers, while additionally,
incorporates knowledge from various attributes (such as age and identity). We
evaluate the proposed Multi-Attribute Robust Component Analysis (MA-RCA) on
problems such as UV completion and age progression, where the proposed method
outperforms compared techniques. Finally, we demonstrate that MA-RCA method is
powerful enough to provide weak annotations for training deep learning systems
for various applications, such as illumination transfer.
Unsupervised Domain Adaptation for 3D Keypoint Estimation via View Consistency
Authors: Xingyi Zhou, Arjun Karpur, Chuang Gan, Linjie Luo, Qi-Xing Huang
Date: 2017-12-15 00:00:00
Proceeding: ECCV 2018 9
Tasks: Domain Adaptation, Keypoint Estimation, Unsupervised Domain Adaptation

In this paper, we introduce a novel unsupervised domain adaptation technique
for the task of 3D keypoint prediction from a single depth scan or image. Our
key idea is to utilize the fact that predictions from different views of the
same or similar objects should be consistent with each other. Such view
consistency can provide effective regularization for keypoint prediction on
unlabeled instances. In addition, we introduce a geometric alignment term to
regularize predictions in the target domain. The resulting loss function can be
effectively optimized via alternating minimization. We demonstrate the
effectiveness of our approach on real datasets and present experimental results
showing that our approach is superior to state-of-the-art general-purpose
domain adaptation techniques.
Kill Two Birds With One Stone: Boosting Both Object Detection Accuracy and Speed With adaptive Patch-of-Interest Composition
Authors: Shihao Zhang, Weiyao Lin, Ping Lu, Weihua Li, Shuo Deng
Date: 2017-08-12 00:00:00
Tasks: Object, object-detection, Object Detection, Video Understanding

Object detection is an important yet challenging task in video understanding
& analysis, where one major challenge lies in the proper balance between two
contradictive factors: detection accuracy and detection speed. In this paper,
we propose a new adaptive patch-of-interest composition approach for boosting
both the accuracy and speed for object detection. The proposed approach first
extracts patches in a video frame which have the potential to include
objects-of-interest. Then, an adaptive composition process is introduced to
compose the extracted patches into an optimal number of sub-frames for object
detection. With this process, we are able to maintain the resolution of the
original frame during object detection (for guaranteeing the accuracy), while
minimizing the number of inputs in detection (for boosting the speed).
Experimental results on various datasets demonstrate the effectiveness of the
proposed approach.
Bayesian Cluster Enumeration Criterion for Unsupervised Learning
Authors: Freweyni K. Teklehaymanot, Michael Muma, Abdelhak M. Zoubir
Date: 2017-10-22 00:00:00
Tasks: Clustering

We derive a new Bayesian Information Criterion (BIC) by formulating the
problem of estimating the number of clusters in an observed data set as
maximization of the posterior probability of the candidate models. Given that
some mild assumptions are satisfied, we provide a general BIC expression for a
broad class of data distributions. This serves as a starting point when
deriving the BIC for specific distributions. Along this line, we provide a
closed-form BIC expression for multivariate Gaussian distributed variables. We
show that incorporating the data structure of the clustering problem into the
derivation of the BIC results in an expression whose penalty term is different
from that of the original BIC. We propose a two-step cluster enumeration
algorithm. First, a model-based unsupervised learning algorithm partitions the
data according to a given set of candidate models. Subsequently, the number of
clusters is determined as the one associated with the model for which the
proposed BIC is maximal. The performance of the proposed two-step algorithm is
tested using synthetic and real data sets.
Understanding Career Progression in Baseball Through Machine Learning
Authors: Brian Bierig, Jonathan Hollenbeck, Alexander Stroud
Date: 2017-12-15 00:00:00
Tasks: BIG-bench Machine Learning

Professional baseball players are increasingly guaranteed expensive long-term
contracts, with over 70 deals signed in excess of \$90 million, mostly in the
last decade. These are substantial sums compared to a typical franchise
valuation of \$1-2 billion. Hence, the players to whom a team chooses to give
such a contract can have an enormous impact on both competitiveness and profit.
Despite this, most published approaches examining career progression in
baseball are fairly simplistic. We applied four machine learning algorithms to
the problem and soundly improved upon existing approaches, particularly for
batting data.
Hierarchical internal representation of spectral features in deep convolutional networks trained for EEG decoding
Authors: Kay Gregor Hartmann, Robin Tibor Schirrmeister, Tonio Ball
Date: 2017-11-21 00:00:00
Tasks: Brain Computer Interface, EEG, Eeg Decoding, Electroencephalogram (EEG), Sensitivity, Time Series Analysis

Recently, there is increasing interest and research on the interpretability
of machine learning models, for example how they transform and internally
represent EEG signals in Brain-Computer Interface (BCI) applications. This can
help to understand the limits of the model and how it may be improved, in
addition to possibly provide insight about the data itself. Schirrmeister et
al. (2017) have recently reported promising results for EEG decoding with deep
convolutional neural networks (ConvNets) trained in an end-to-end manner and,
with a causal visualization approach, showed that they learn to use spectral
amplitude changes in the input. In this study, we investigate how ConvNets
represent spectral features through the sequence of intermediate stages of the
network. We show higher sensitivity to EEG phase features at earlier stages and
higher sensitivity to EEG amplitude features at later stages. Intriguingly, we
observed a specialization of individual stages of the network to the classical
EEG frequency bands alpha, beta, and high gamma. Furthermore, we find first
evidence that particularly in the last convolutional layer, the network learns
to detect more complex oscillatory patterns beyond spectral phase and
amplitude, reminiscent of the representation of complex visual features in
later layers of ConvNets in computer vision tasks. Our findings thus provide
insights into how ConvNets hierarchically represent spectral EEG features in
their intermediate layers and suggest that ConvNets can exploit and might help
to better understand the compositional structure of EEG time series.
Information Processing by Networks of Quantum Decision Makers
Authors: V. I. Yukalov, E. P. Yukalova, D. Sornette
Date: 2017-12-15 00:00:00

We suggest a model of a multi-agent society of decision makers taking
decisions being based on two criteria, one is the utility of the prospects and
the other is the attractiveness of the considered prospects. The model is the
generalization of quantum decision theory, developed earlier for single
decision makers realizing one-step decisions, in two principal aspects. First,
several decision makers are considered simultaneously, who interact with each
other through information exchange. Second, a multistep procedure is treated,
when the agents exchange information many times. Several decision makers
exchanging information and forming their judgement, using quantum rules, form a
kind of a quantum information network, where collective decisions develop in
time as a result of information exchange. In addition to characterizing
collective decisions that arise in human societies, such networks can describe
dynamical processes occurring in artificial quantum intelligence composed of
several parts or in a cluster of quantum computers. The practical usage of the
theory is illustrated on the dynamic disjunction effect for which three
quantitative predictions are made: (i) the probabilistic behavior of decision
makers at the initial stage of the process is described; (ii) the decrease of
the difference between the initial prospect probabilities and the related
utility factors is proved; (iii) the existence of a common consensus after
multiple exchange of information is predicted. The predicted numerical values
are in very good agreement with empirical data.
Towards Deep Modeling of Music Semantics using EEG Regularizers
Authors: Francisco Raposo, David Martins de Matos, Ricardo Ribeiro, Suhua Tang, Yi Yu
Date: 2017-12-14 00:00:00
Tasks: Cross-Modal Retrieval, EEG, Electroencephalogram (EEG), Retrieval, Transfer Learning

Modeling of music audio semantics has been previously tackled through
learning of mappings from audio data to high-level tags or latent unsupervised
spaces. The resulting semantic spaces are theoretically limited, either because
the chosen high-level tags do not cover all of music semantics or because audio
data itself is not enough to determine music semantics. In this paper, we
propose a generic framework for semantics modeling that focuses on the
perception of the listener, through EEG data, in addition to audio data. We
implement this framework using a novel end-to-end 2-view Neural Network (NN)
architecture and a Deep Canonical Correlation Analysis (DCCA) loss function
that forces the semantic embedding spaces of both views to be maximally
correlated. We also detail how the EEG dataset was collected and use it to
train our proposed model. We evaluate the learned semantic space in a transfer
learning context, by using it as an audio feature extractor in an independent
dataset and proxy task: music audio-lyrics cross-modal retrieval. We show that
our embedding model outperforms Spotify features and performs comparably to a
state-of-the-art embedding model that was trained on 700 times more data. We
further discuss improvements to the model that are likely to improve its
performance.