pwc-archive/papers-with-abstracts huggingface dataset:
Learning without Forgetting
Authors: Zhizhong Li, Derek Hoiem
Date: 2016-06-29 00:00:00
Tasks: Class Incremental Learning, Continual Learning, Disjoint 10-1, Disjoint 15-1, Disjoint 15-5, Domain 1-1, Domain 11-1, Domain 11-5, Incremental Learning, Overlapped 10-1, Overlapped 15-1, Overlapped 15-5

When building a unified vision system or gradually adding new capabilities to
a system, the usual assumption is that training data for all tasks is always
available. However, as the number of tasks grows, storing and retraining on
such data becomes infeasible. A new problem arises where we add new
capabilities to a Convolutional Neural Network (CNN), but the training data for
its existing capabilities are unavailable. We propose our Learning without
Forgetting method, which uses only new task data to train the network while
preserving the original capabilities. Our method performs favorably compared to
commonly used feature extraction and fine-tuning adaption techniques and
performs similarly to multitask learning that uses original task data we assume
unavailable. A more surprising observation is that Learning without Forgetting
may be able to replace fine-tuning with similar old and new task datasets for
improved new task performance.
Entropy Non-increasing Games for the Improvement of Dataflow Programming
Authors: Norbert Bátfai, Renátó Besenczi, Gergő Bogacsovics, Fanny Monori
Date: 2017-02-14 00:00:00

In this article, we introduce a new conception of a family of esport games
called Samu Entropy to try to improve dataflow program graphs like the ones
that are based on Google's TensorFlow. Currently, the Samu Entropy project
specifies only requirements for new esport games to be developed with
particular attention to the investigation of the relationship between esport
and artificial intelligence. It is quite obvious that there is a very close and
natural relationship between esport games and artificial intelligence.
Furthermore, the project Samu Entropy focuses not only on using artificial
intelligence, but on creating AI in a new way. We present a reference game
called Face Battle that implements the Samu Entropy requirements.
A Projection Based Conditional Dependence Measure with Applications to High-dimensional Undirected Graphical Models
Authors: Jianqing Fan, Yang Feng, Lucy Xia
Date: 2015-01-07 00:00:00

Measuring conditional dependence is an important topic in statistics with
broad applications including graphical models. Under a factor model setting, a
new conditional dependence measure based on projection is proposed. The
corresponding conditional independence test is developed with the asymptotic
null distribution unveiled where the number of factors could be
high-dimensional. It is also shown that the new test has control over the
asymptotic significance level and can be calculated efficiently. A generic
method for building dependency graphs without Gaussian assumption using the new
test is elaborated. Numerical results and real data analysis show the
superiority of the new method.
Enhanced Facial Recognition Framework based on Skin Tone and False Alarm Rejection
Authors: Ali Sharifara, Mohd Shafry Mohd Rahim, Farhad Navabifar, Dylan Ebert, Amir Ghaderi, Michalis Papakostas
Date: 2017-02-14 00:00:00
Tasks: Face Detection, Face Recognition, Management

Face detection is one of the challenging tasks in computer vision. Human face
detection plays an essential role in the first stage of face processing
applications such as face recognition, face tracking, image database
management, etc. In these applications, face objects often come from an
inconsequential part of images that contain variations, namely different
illumination, poses, and occlusion. These variations can decrease face
detection rate noticeably. Most existing face detection approaches are not
accurate, as they have not been able to resolve unstructured images due to
large appearance variations and can only detect human faces under one
particular variation. Existing frameworks of face detection need enhancements
to detect human faces under the stated variations to improve detection rate and
reduce detection time. In this study, an enhanced face detection framework is
proposed to improve detection rate based on skin color and provide a validation
process. A preliminary segmentation of the input images based on skin color can
significantly reduce search space and accelerate the process of human face
detection. The primary detection is based on Haar-like features and the
Adaboost algorithm. A validation process is introduced to reject non-face
objects, which might occur during the face detection process. The validation
process is based on two-stage Extended Local Binary Patterns. The experimental
results on the CMU-MIT and Caltech 10000 datasets over a wide range of facial
variations in different colors, positions, scales, and lighting conditions
indicated a successful face detection rate.
A case study on using speech-to-translation alignments for language documentation
Authors: Antonios Anastasopoulos, David Chiang
Date: 2017-02-14 00:00:00
Proceeding: WS 2017 3
Tasks: speech-recognition, Speech Recognition, Translation

For many low-resource or endangered languages, spoken language resources are
more likely to be annotated with translations than with transcriptions. Recent
work exploits such annotations to produce speech-to-translation alignments,
without access to any text transcriptions. We investigate whether providing
such information can aid in producing better (mismatched) crowdsourced
transcriptions, which in turn could be valuable for training speech recognition
systems, and show that they can indeed be beneficial through a small-scale case
study as a proof-of-concept. We also present a simple phonetically aware string
averaging technique that produces transcriptions of higher quality.
Learning to Act by Predicting the Future
Authors: Alexey Dosovitskiy, Vladlen Koltun
Date: 2016-11-06 00:00:00

We present an approach to sensorimotor control in immersive environments. Our
approach utilizes a high-dimensional sensory stream and a lower-dimensional
measurement stream. The cotemporal structure of these streams provides a rich
supervisory signal, which enables training a sensorimotor control model by
interacting with the environment. The model is trained using supervised
learning techniques, but without extraneous supervision. It learns to act based
on raw sensory input from a complex three-dimensional environment. The
presented formulation enables learning without a fixed goal at training time,
and pursuing dynamically changing goals at test time. We conduct extensive
experiments in three-dimensional simulations based on the classical
first-person game Doom. The results demonstrate that the presented approach
outperforms sophisticated prior formulations, particularly on challenging
tasks. The results also show that trained models successfully generalize across
environments and goals. A model trained using the presented approach won the
Full Deathmatch track of the Visual Doom AI Competition, which was held in
previously unseen environments.
Naive Bayes and Text Classification I - Introduction and Theory
Authors: Sebastian Raschka
Date: 2014-10-16 00:00:00
Tasks: Classification, Disease Prediction, Document Classification, General Classification, text-classification, Text Classification

Naive Bayes classifiers, a family of classifiers that are based on the
popular Bayes' probability theorem, are known for creating simple yet well
performing models, especially in the fields of document classification and
disease prediction. In this article, we will look at the main concepts of naive
Bayes classification in the context of document categorization.
On the Relevance of Auditory-Based Gabor Features for Deep Learning in Automatic Speech Recognition
Authors: Angel Mario Castro Martinez, Sri Harish Mallidi, Bernd T. Meyer
Date: 2017-02-14 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Decoder, speech-recognition, Speech Recognition

Previous studies support the idea of merging auditory-based Gabor features
with deep learning architectures to achieve robust automatic speech
recognition, however, the cause behind the gain of such combination is still
unknown. We believe these representations provide the deep learning decoder
with more discriminable cues. Our aim with this paper is to validate this
hypothesis by performing experiments with three different recognition tasks
(Aurora 4, CHiME 2 and CHiME 3) and assess the discriminability of the
information encoded by Gabor filterbank features. Additionally, to identify the
contribution of low, medium and high temporal modulation frequencies subsets of
the Gabor filterbank were used as features (dubbed LTM, MTM and HTM
respectively). With temporal modulation frequencies between 16 and 25 Hz, HTM
consistently outperformed the remaining ones in every condition, highlighting
the robustness of these representations against channel distortions, low
signal-to-noise ratios and acoustically challenging real-life scenarios with
relative improvements from 11 to 56% against a Mel-filterbank-DNN baseline. To
explain the results, a measure of similarity between phoneme classes from DNN
activations is proposed and linked to their acoustic properties. We find this
measure to be consistent with the observed error rates and highlight specific
differences on phoneme level to pinpoint the benefit of the proposed features.
The Cumulative Distribution Transform and Linear Pattern Classification
Authors: Se Rim Park, Soheil Kolouri, Shinjini Kundu, Gustavo Rohde
Date: 2015-07-21 00:00:00
Tasks: Classification, General Classification

Discriminating data classes emanating from sensors is an important problem
with many applications in science and technology. We describe a new transform
for pattern identification that interprets patterns as probability density
functions, and has special properties with regards to classification. The
transform, which we denote as the Cumulative Distribution Transform (CDT) is
invertible, with well defined forward and inverse operations. We show that it
can be useful in `parsing out' variations (confounds) that are `Lagrangian'
(displacement and intensity variations) by converting these to `Eulerian'
(intensity variations) in transform space. This conversion is the basis for our
main result that describes when the CDT can allow for linear classification to
be possible in transform space. We also describe several properties of the
transform and show, with computational experiments that used both real and
simulated data, that the CDT can help render a variety of real world problems
simpler to solve.
The Effect of Color Space Selection on Detectability and Discriminability of Colored Objects
Authors: Amir Rasouli, John K. Tsotsos
Date: 2017-02-14 00:00:00

In this paper, we investigate the effect of color space selection on
detectability and discriminability of colored objects under various conditions.
20 color spaces from the literature are evaluated on a large dataset of
simulated and real images. We measure the suitability of color spaces from two
different perspectives: detectability and discriminability of various color
groups. Through experimental evaluation, we found that there is no single
optimal color space suitable for all color groups. The color spaces have
different levels of sensitivity to different color groups and they are useful
depending on the color of the sought object. Overall, the best results were
achieved in both simulated and real images using color spaces C1C2C3, UVW and
XYZ. In addition, using a simulated environment, we show a practical
application of color space selection in the context of top-down control in
active visual search. The results indicate that on average color space C1C2C3
followed by HSI and XYZ achieve the best time in searching for objects of
various colors. Here, the right choice of color space can improve time of
search on average by 20%. As part of our contribution, we also introduce a
large dataset of simulated 3D objects
Integrating Three Mechanisms of Visual Attention for Active Visual Search
Authors: Amir Rasouli, John K. Tsotsos
Date: 2017-02-14 00:00:00

Algorithms for robotic visual search can benefit from the use of visual
attention methods in order to reduce computational costs. Here, we describe how
three distinct mechanisms of visual attention can be integrated and
productively used to improve search performance. The first is viewpoint
selection as has been proposed earlier using a greedy search over a
probabilistic occupancy grid representation. The second is top-down
object-based attention using a histogram backprojection method, also previously
described. The third is visual saliency. This is novel in the sense that it is
not used as a region-of-interest method for the current image but rather as a
noncombinatorial form of look-ahead in search for future viewpoint selection.
Additionally, the integration of these three attentional schemes within a
single framework is unique and not previously studied. We examine our proposed
method in scenarios where little or no information regarding the environment is
available. Through extensive experiments on a mobile robot, we show that our
method improves visual search performance by reducing the time and number of
actions required.
Regularities and Irregularities in Order Flow Data
Authors: Martin Theissen, Sebastian M. Krause, Thomas Guhr
Date: 2017-02-14 00:00:00

We identify and analyze statistical regularities and irregularities in the
recent order flow of different NASDAQ stocks, focusing on the positions where
orders are placed in the orderbook. This includes limit orders being placed
outside of the spread, inside the spread and (effective) market orders. We find
that limit order placement inside the spread is strongly determined by the
dynamics of the spread size. Most orders, however, arrive outside of the
spread. While for some stocks order placement on or next to the quotes is
dominating, deeper price levels are more important for other stocks. As market
orders are usually adjusted to the quote volume, the impact of market orders
depends on the orderbook structure, which we find to be quite diverse among the
analyzed stocks as a result of the way limit order placement takes place.
Exploring loss function topology with cyclical learning rates
Authors: Leslie N. Smith, Nicholay Topin
Date: 2017-02-14 00:00:00

We present observations and discussion of previously unreported phenomena
discovered while training residual networks. The goal of this work is to better
understand the nature of neural networks through the examination of these new
empirical results. These behaviors were identified through the application of
Cyclical Learning Rates (CLR) and linear network interpolation. Among these
behaviors are counterintuitive increases and decreases in training loss and
instances of rapid training. For example, we demonstrate how CLR can produce
greater testing accuracy than traditional training despite using large learning
rates. Files to replicate these results are available at
https://github.com/lnsmith54/exploring-loss
T-SKIRT: Online Estimation of Student Proficiency in an Adaptive Learning System
Authors: Chaitanya Ekanadham, Yan Karklin
Date: 2017-02-14 00:00:00

We develop T-SKIRT: a temporal, structured-knowledge, IRT-based method for
predicting student responses online. By explicitly accounting for student
learning and employing a structured, multidimensional representation of student
proficiencies, the model outperforms standard IRT-based methods on an online
response prediction task when applied to real responses collected from students
interacting with diverse pools of educational content.
Predicting Auction Price of Vehicle License Plate with Deep Recurrent Neural Network
Authors: Vinci Chow
Date: 2017-01-30 00:00:00
Proceeding: ICLR 2018 1

In Chinese societies, superstition is of paramount importance, and vehicle license plates with desirable numbers can fetch very high prices in auctions. Unlike other valuable items, license plates are not allocated an estimated price before auction. I propose that the task of predicting plate prices can be viewed as a natural language processing (NLP) task, as the value depends on the meaning of each individual character on the plate and its semantics. I construct a deep recurrent neural network (RNN) to predict the prices of vehicle license plates in Hong Kong, based on the characters on a plate. I demonstrate the importance of having a deep network and of retraining. Evaluated on 13 years of historical auction prices, the deep RNN's predictions can explain over 80 percent of price variations, outperforming previous models by a significant margin. I also demonstrate how the model can be extended to become a search engine for plates and to provide estimates of the expected price distribution.
Cell assemblies at multiple time scales with arbitrary lag constellations
Authors: Eleonora Russo, Daniel Durstewitz
Date: 2016-07-04 00:00:00

Hebb's idea of a cell assembly as the fundamental unit of neural information
processing has dominated neuroscience like no other theoretical concept within
the past 60 years. A range of different physiological phenomena, from precisely
synchronized spiking to broadly simultaneous rate increases, has been subsumed
under this term. Yet progress in this area is hampered by the lack of
statistical tools that would enable to extract assemblies with arbitrary
constellations of time lags, and at multiple temporal scales, partly due to the
severe computational burden. Here we present such a unifying methodological and
conceptual framework which detects assembly structure at many different time
scales, levels of precision, and with arbitrary internal organization. Applying
this methodology to multiple single unit recordings from various cortical
areas, we find that there is no universal cortical coding scheme, but that
assembly structure and precision significantly depends on brain area recorded
and ongoing task demands.
FERA 2017 - Addressing Head Pose in the Third Facial Expression Recognition and Analysis Challenge
Authors: Michel F. Valstar, Enrique Sánchez-Lozano, Jeffrey F. Cohn, László A. Jeni, Jeffrey M. Girard, Zheng Zhang, Lijun Yin, Maja Pantic
Date: 2017-02-14 00:00:00
Tasks: Benchmarking, Facial Action Unit Detection, Facial Expression Recognition, Facial Expression Recognition (FER), Gesture Recognition, valid

The field of Automatic Facial Expression Analysis has grown rapidly in recent
years. However, despite progress in new approaches as well as benchmarking
efforts, most evaluations still focus on either posed expressions, near-frontal
recordings, or both. This makes it hard to tell how existing expression
recognition approaches perform under conditions where faces appear in a wide
range of poses (or camera views), displaying ecologically valid expressions.
The main obstacle for assessing this is the availability of suitable data, and
the challenge proposed here addresses this limitation. The FG 2017 Facial
Expression Recognition and Analysis challenge (FERA 2017) extends FERA 2015 to
the estimation of Action Units occurrence and intensity under different camera
views. In this paper we present the third challenge in automatic recognition of
facial expressions, to be held in conjunction with the 12th IEEE conference on
Face and Gesture Recognition, May 2017, in Washington, United States. Two
sub-challenges are defined: the detection of AU occurrence, and the estimation
of AU intensity. In this work we outline the evaluation protocol, the data
used, and the results of a baseline method for both sub-challenges.
Crossmatching variable objects with the Gaia data
Authors: Lorenzo Rimoldini, Krzysztof Nienartowicz, Maria Süveges, Jonathan Charnas, Leanne P. Guy, Grégory Jevardat de Fombelle, Berry Holl, Isabelle Lecoeur-Taïbi, Nami Mowlavi, Diego Ordóñez-Blanco, Laurent Eyer
Date: 2017-02-14 00:00:00
Tasks: Time Series, Time Series Analysis

Tens of millions of new variable objects are expected to be identified in
over a billion time series from the Gaia mission. Crossmatching known variable
sources with those from Gaia is crucial to incorporate current knowledge,
understand how these objects appear in the Gaia data, train supervised
classifiers to recognise known classes, and validate the results of the
Variability Processing and Analysis Coordination Unit (CU7) within the Gaia
Data Analysis and Processing Consortium (DPAC). The method employed by CU7 to
crossmatch variables for the first Gaia data release includes a binary
classifier to take into account positional uncertainties, proper motion,
targeted variability signals, and artefacts present in the early calibration of
the Gaia data. Crossmatching with a classifier makes it possible to automate
all those decisions which are typically made during visual inspection. The
classifier can be trained with objects characterized by a variety of attributes
to ensure similarity in multiple dimensions (astrometry, photometry,
time-series features), with no need for a-priori transformations to compare
different photometric bands, or of predictive models of the motion of objects
to compare positions. Other advantages as well as some disadvantages of the
method are discussed. Implementation steps from the training to the assessment
of the crossmatch classifier and selection of results are described.
Adiabatic Persistent Contrastive Divergence Learning
Authors: Hyeryung Jang, Hyungwon Choi, Yung Yi, Jinwoo Shin
Date: 2016-05-26 00:00:00

This paper studies the problem of parameter learning in probabilistic
graphical models having latent variables, where the standard approach is the
expectation maximization algorithm alternating expectation (E) and maximization
(M) steps. However, both E and M steps are computationally intractable for high
dimensional data, while the substitution of one step to a faster surrogate for
combating against intractability can often cause failure in convergence. We
propose a new learning algorithm which is computationally efficient and
provably ensures convergence to a correct optimum. Its key idea is to run only
a few cycles of Markov Chains (MC) in both E and M steps. Such an idea of
running incomplete MC has been well studied only for M step in the literature,
called Contrastive Divergence (CD) learning. While such known CD-based schemes
find approximated gradients of the log-likelihood via the mean-field approach
in E step, our proposed algorithm does exact ones via MC algorithms in both
steps due to the multi-time-scale stochastic approximation theory. Despite its
theoretical guarantee in convergence, the proposed scheme might suffer from the
slow mixing of MC in E step. To tackle it, we also propose a hybrid approach
applying both mean-field and MC approximation in E step, where the hybrid
approach outperforms the bare mean-field CD scheme in our experiments on
real-world datasets.