pwc-archive/papers-with-abstracts huggingface dataset:
A Learning-Based Visual Saliency Prediction Model for Stereoscopic 3D Video (LBVS-3D)
Authors: Amin Banitalebi-Dehkordi, Mahsa T. Pourazad, Panos Nasiopoulos
Date: 2018-03-13 00:00:00
Tasks: Saliency Prediction, Video Saliency Prediction

Over the past decade, many computational saliency prediction models have been
proposed for 2D images and videos. Considering that the human visual system has
evolved in a natural 3D environment, it is only natural to want to design
visual attention models for 3D content. Existing monocular saliency models are
not able to accurately predict the attentive regions when applied to 3D
image/video content, as they do not incorporate depth information. This paper
explores stereoscopic video saliency prediction by exploiting both low-level
attributes such as brightness, color, texture, orientation, motion, and depth,
as well as high-level cues such as face, person, vehicle, animal, text, and
horizon. Our model starts with a rough segmentation and quantifies several
intuitive observations such as the effects of visual discomfort level, depth
abruptness, motion acceleration, elements of surprise, size and compactness of
the salient regions, and emphasizing only a few salient objects in a scene. A
new fovea-based model of spatial distance between the image regions is adopted
for considering local and global feature calculations. To efficiently fuse the
conspicuity maps generated by our method to one single saliency map that is
highly correlated with the eye-fixation data, a random forest based algorithm
is utilized. The performance of the proposed saliency model is evaluated
against the results of an eye-tracking experiment, which involved 24 subjects
and an in-house database of 61 captured stereoscopic videos. Our stereo video
database as well as the eye-tracking data are publicly available along with
this paper. Experiment results show that the proposed saliency prediction
method achieves competitive performance compared to the state-of-the-art
approaches.
Resource aware design of a deep convolutional-recurrent neural network for speech recognition through audio-visual sensor fusion
Authors: Matthijs Van keirsbilck, Bert Moons, Marian Verhelst
Date: 2018-03-13 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Lip Reading, Phoneme Recognition, Sensor Fusion, speech-recognition, Speech Recognition

Today's Automatic Speech Recognition systems only rely on acoustic signals
and often don't perform well under noisy conditions. Performing multi-modal
speech recognition - processing acoustic speech signals and lip-reading video
simultaneously - significantly enhances the performance of such systems,
especially in noisy environments. This work presents the design of such an
audio-visual system for Automated Speech Recognition, taking memory and
computation requirements into account. First, a Long-Short-Term-Memory neural
network for acoustic speech recognition is designed. Second, Convolutional
Neural Networks are used to model lip-reading features. These are combined with
an LSTM network to model temporal dependencies and perform automatic
lip-reading on video. Finally, acoustic-speech and visual lip-reading networks
are combined to process acoustic and visual features simultaneously. An
attention mechanism ensures performance of the model in noisy environments.
This system is evaluated on the TCD-TIMIT 'lipspeaker' dataset for audio-visual
phoneme recognition with clean audio and with additive white noise at an SNR of
0dB. It achieves 75.70% and 58.55% phoneme accuracy respectively, over 14
percentage points better than the state-of-the-art for all noise levels.
3D Video Quality Assessment
Authors: Amin Banitalebi Dehkordi
Date: 2018-03-13 00:00:00
Tasks: Management, Video Quality Assessment

A key factor in designing 3D systems is to understand how different visual
cues and distortions affect the perceptual quality of 3D video. The ultimate
way to assess video quality is through subjective tests. However, subjective
evaluation is time consuming, expensive, and in most cases not even possible.
An alternative solution is objective quality metrics, which attempt to model
the Human Visual System (HVS) in order to assess the perceptual quality. The
potential of 3D technology to significantly improve the immersiveness of video
content has been hampered by the difficulty of objectively assessing Quality of
Experience (QoE). A no-reference (NR) objective 3D quality metric, which could
help determine capturing parameters and improve playback perceptual quality,
would be welcomed by camera and display manufactures. Network providers would
embrace a full-reference (FR) 3D quality metric, as they could use it to ensure
efficient QoE-based resource management during compression and Quality of
Service (QoS) during transmission.
A Learning-Based Visual Saliency Fusion Model for High Dynamic Range Video (LBVS-HDR)
Authors: Amin Banitalebi-Dehkordi, Yuanyuan Dong, Mahsa T. Pourazad, Panos Nasiopoulos
Date: 2018-03-13 00:00:00
Tasks: Saliency Prediction

Saliency prediction for Standard Dynamic Range (SDR) videos has been well
explored in the last decade. However, limited studies are available on High
Dynamic Range (HDR) Visual Attention Models (VAMs). Considering that the
characteristic of HDR content in terms of dynamic range and color gamut is
quite different than those of SDR content, it is essential to identify the
importance of different saliency attributes of HDR videos for designing a VAM
and understand how to combine these features. To this end we propose a
learning-based visual saliency fusion method for HDR content (LVBS-HDR) to
combine various visual saliency features. In our approach various conspicuity
maps are extracted from HDR data, and then for fusing conspicuity maps, a
Random Forests algorithm is used to train a model based on the collected data
from an eye-tracking experiment. Performance evaluations demonstrate the
superiority of the proposed fusion method against other existing fusion
methods.
Low-Rank Boolean Matrix Approximation by Integer Programming
Authors: Reka Kovacs, Oktay Gunluk, Raphael Hauser
Date: 2018-03-13 00:00:00
Tasks: BIG-bench Machine Learning, Dimensionality Reduction, regression

Low-rank approximations of data matrices are an important dimensionality
reduction tool in machine learning and regression analysis. We consider the
case of categorical variables, where it can be formulated as the problem of
finding low-rank approximations to Boolean matrices. In this paper we give what
is to the best of our knowledge the first integer programming formulation that
relies on only polynomially many variables and constraints, we discuss how to
solve it computationally and report numerical tests on synthetic and real-world
data.
Learning flexible representations of stochastic processes on graphs
Authors: Addison Bohannon, Brian Sadler, Radu Balan
Date: 2017-11-03 00:00:00

Graph convolutional networks adapt the architecture of convolutional neural
networks to learn rich representations of data supported on arbitrary graphs by
replacing the convolution operations of convolutional neural networks with
graph-dependent linear operations. However, these graph-dependent linear
operations are developed for scalar functions supported on undirected graphs.
We propose a class of linear operations for stochastic (time-varying) processes
on directed (or undirected) graphs to be used in graph convolutional networks.
We propose a parameterization of such linear operations using functional
calculus to achieve arbitrarily low learning complexity. The proposed approach
is shown to model richer behaviors and display greater flexibility in learning
representations than product graph methods.
Future Frame Prediction for Anomaly Detection -- A New Baseline
Authors: Wen Liu, Weixin Luo, Dongze Lian, Shenghua Gao
Date: 2017-12-28 00:00:00
Tasks: Anomaly Detection, Optical Flow Estimation, Prediction, Video Anomaly Detection, Video Prediction

Anomaly detection in videos refers to the identification of events that do
not conform to expected behavior. However, almost all existing methods tackle
the problem by minimizing the reconstruction errors of training data, which
cannot guarantee a larger reconstruction error for an abnormal event. In this
paper, we propose to tackle the anomaly detection problem within a video
prediction framework. To the best of our knowledge, this is the first work that
leverages the difference between a predicted future frame and its ground truth
to detect an abnormal event. To predict a future frame with higher quality for
normal events, other than the commonly used appearance (spatial) constraints on
intensity and gradient, we also introduce a motion (temporal) constraint in
video prediction by enforcing the optical flow between predicted frames and
ground truth frames to be consistent, and this is the first work that
introduces a temporal constraint into the video prediction task. Such spatial
and motion constraints facilitate the future frame prediction for normal
events, and consequently facilitate to identify those abnormal events that do
not conform the expectation. Extensive experiments on both a toy dataset and
some publicly available datasets validate the effectiveness of our method in
terms of robustness to the uncertainty in normal events and the sensitivity to
abnormal events.
Streaming Architecture for Large-Scale Quantized Neural Networks on an FPGA-Based Dataflow Platform
Authors: Chaim Baskin, Natan Liss, Evgenii Zheltonozhskii, Alex M. Bronshtein, Avi Mendelson
Date: 2017-07-31 00:00:00
Tasks: General Classification

Deep neural networks (DNNs) are used by different applications that are
executed on a range of computer architectures, from IoT devices to
supercomputers. The footprint of these networks is huge as well as their
computational and communication needs. In order to ease the pressure on
resources, research indicates that in many cases a low precision representation
(1-2 bit per parameter) of weights and other parameters can achieve similar
accuracy while requiring less resources. Using quantized values enables the use
of FPGAs to run NNs, since FPGAs are well fitted to these primitives; e.g.,
FPGAs provide efficient support for bitwise operations and can work with
arbitrary-precision representation of numbers.
  This paper presents a new streaming architecture for running QNNs on FPGAs.
The proposed architecture scales out better than alternatives, allowing us to
take advantage of systems with multiple FPGAs. We also included support for
skip connections, that are used in state-of-the art NNs, and shown that our
architecture allows to add those connections almost for free. All this allowed
us to implement an 18-layer ResNet for 224x224 images classification, achieving
57.5% top-1 accuracy.
  In addition, we implemented a full-sized quantized AlexNet. In contrast to
previous works, we use 2-bit activations instead of 1-bit ones, which improves
AlexNet's top-1 accuracy from 41.8% to 51.03% for the ImageNet classification.
Both AlexNet and ResNet can handle 1000-class real-time classification on an
FPGA.
  Our implementation of ResNet-18 consumes 5x less power and is 4x slower for
ImageNet, when compared to the same NN on the latest Nvidia GPUs. Smaller NNs,
that fit a single FPGA, are running faster then on GPUs on small (32x32)
inputs, while consuming up to 20x less energy and power.
A case for multiple and parallel RRAMs as synaptic model for training SNNs
Authors: Aditya Shukla, Sidharth Prasad, Sandip Lashkare, Udayan Ganguly
Date: 2018-03-13 00:00:00

To enable a dense integration of model synapses in a spiking neural networks
hardware, various nano-scale devices are being considered. Such a device,
besides exhibiting spike-time dependent plasticity (STDP), needs to be highly
scalable, have a large endurance and require low energy for transitioning
between states. In this work, we first introduce and empirically determine two
new specifications for an synapse in SNNs: number of conductance levels per
synapse and maximum learning-rate. To the best of our knowledge, there are no
RRAMs that meet the latter specification. As a solution, we propose the use of
multiple PCMO-RRAMs in parallel within a synapse. While synaptic reading, all
PCMO-RRAMs are simultaneously read and for each synaptic conductance-change
event, the mechanism for conductance STDP is initiated for only one RRAM,
randomly picked from the set. Second, to validate our solution, we
experimentally demonstrate STDP of conductance of a PCMO-RRAM and then show
that due to a large learning-rate, a single PCMO-RRAM fails to model a synapse
in the training of an SNN. As anticipated, network training improves as more
PCMO-RRAMs are added to the synapse. Fourth, we discuss the
circuit-requirements for implementing such a scheme, to conclude that the
requirements are within bounds. Thus, our work presents specifications for
synaptic devices in trainable SNNs, indicates the shortcomings of state-of-art
synaptic contenders, and provides a solution to extrinsically meet the
specifications and discusses the peripheral circuitry that implements the
solution.
Estimating activity cycles with probabilistic methods I. Bayesian Generalised Lomb-Scargle Periodogram with Trend
Authors: N. Olspert, J. Pelt, M. J. Käpylä, J. Lehtinen
Date: 2017-12-21 00:00:00
Tasks: Period Estimation, Time Series Analysis

Period estimation is one of the central topics in astronomical time series
analysis, where data is often unevenly sampled. Especially challenging are
studies of stellar magnetic cycles, as there the periods looked for are of the
order of the same length than the datasets themselves. The datasets often
contain trends, the origin of which is either a real long-term cycle or an
instrumental effect, but these effects cannot be reliably separated, while they
can lead to erroneous period determinations if not properly handled. In this
study we aim at developing a method that can handle the trends properly, and by
performing extensive set of testing, we show that this is the optimal procedure
when contrasted with methods that do not include the trend directly to the
model. The effect of the form of the noise (whether constant or
heteroscedastic) on the results is also investigated. We introduce a Bayesian
Generalised Lomb-Scargle Periodogram with Trend (BGLST), which is a
probabilistic linear regression model using Gaussian priors for the
coefficients and uniform prior for the frequency parameter. We show, using
synthetic data, that when there is no prior information on whether and to what
extent the true model of the data contains a linear trend, the introduced BGLST
method is preferable to the methods which either detrend the data or leave the
data untrended before fitting the periodic model. Whether to use noise with
different than constant variance in the model depends on the density of the
data sampling as well as on the true noise type of the process.
Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning
Authors: Nicolas Papernot, Patrick McDaniel
Date: 2018-03-13 00:00:00
Tasks: Machine Translation, Malware Detection

Deep neural networks (DNNs) enable innovative applications of machine
learning like image recognition, machine translation, or malware detection.
However, deep learning is often criticized for its lack of robustness in
adversarial settings (e.g., vulnerability to adversarial inputs) and general
inability to rationalize its predictions. In this work, we exploit the
structure of deep learning to enable new learning-based inference and decision
strategies that achieve desirable properties such as robustness and
interpretability. We take a first step in this direction and introduce the Deep
k-Nearest Neighbors (DkNN). This hybrid classifier combines the k-nearest
neighbors algorithm with representations of the data learned by each layer of
the DNN: a test input is compared to its neighboring training points according
to the distance that separates them in the representations. We show the labels
of these neighboring points afford confidence estimates for inputs outside the
model's training manifold, including on malicious inputs like adversarial
examples--and therein provides protections against inputs that are outside the
models understanding. This is because the nearest neighbors can be used to
estimate the nonconformity of, i.e., the lack of support for, a prediction in
the training data. The neighbors also constitute human-interpretable
explanations of predictions. We evaluate the DkNN algorithm on several
datasets, and show the confidence estimates accurately identify inputs outside
the model, and that the explanations provided by nearest neighbors are
intuitive and useful in understanding model failures.
Oracle Inequalities for High-dimensional Prediction
Authors: Johannes Lederer, Lu Yu, Irina Gaynanova
Date: 2016-08-01 00:00:00
Tasks: Prediction, Vocal Bursts Intensity Prediction

The abundance of high-dimensional data in the modern sciences has generated
tremendous interest in penalized estimators such as the lasso, scaled lasso,
square-root lasso, elastic net, and many others. In this paper, we establish a
general oracle inequality for prediction in high-dimensional linear regression
with such methods. Since the proof relies only on convexity and continuity
arguments, the result holds irrespective of the design matrix and applies to a
wide range of penalized estimators. Overall, the bound demonstrates that
generic estimators can provide consistent prediction with any design matrix.
From a practical point of view, the bound can help to identify the potential of
specific estimators, and they can help to get a sense of the prediction
accuracy in a given application.
Monitoring Targeted Hate in Online Environments
Authors: Tim Isbister, Magnus Sahlgren, Lisa Kaati, Milan Obaidi, Nazar Akrami
Date: 2018-03-13 00:00:00

Hateful comments, swearwords and sometimes even death threats are becoming a
reality for many people today in online environments. This is especially true
for journalists, politicians, artists, and other public figures. This paper
describes how hate directed towards individuals can be measured in online
environments using a simple dictionary-based approach. We present a case study
on Swedish politicians, and use examples from this study to discuss
shortcomings of the proposed dictionary-based approach. We also outline
possibilities for potential refinements of the proposed approach.
VERSE: Versatile Graph Embeddings from Similarity Measures
Authors: Anton Tsitsulin, Davide Mottin, Panagiotis Karras, Emmanuel Müller
Date: 2018-03-13 00:00:00
Tasks: Link Prediction

Embedding a web-scale information network into a low-dimensional vector space
facilitates tasks such as link prediction, classification, and visualization.
Past research has addressed the problem of extracting such embeddings by
adopting methods from words to graphs, without defining a clearly
comprehensible graph-related objective. Yet, as we show, the objectives used in
past works implicitly utilize similarity measures among graph nodes.
  In this paper, we carry the similarity orientation of previous works to its
logical conclusion; we propose VERtex Similarity Embeddings (VERSE), a simple,
versatile, and memory-efficient method that derives graph embeddings explicitly
calibrated to preserve the distributions of a selected vertex-to-vertex
similarity measure. VERSE learns such embeddings by training a single-layer
neural network. While its default, scalable version does so via sampling
similarity information, we also develop a variant using the full information
per vertex. Our experimental study on standard benchmarks and real-world
datasets demonstrates that VERSE, instantiated with diverse similarity
measures, outperforms state-of-the-art methods in terms of precision and recall
in major data mining tasks and supersedes them in time and space efficiency,
while the scalable sampling-based variant achieves equally good results as the
non-scalable full variant.
NDDR-CNN: Layerwise Feature Fusing in Multi-Task CNNs by Neural Discriminative Dimensionality Reduction
Authors: Yuan Gao, Jiayi Ma, Mingbo Zhao, Wei Liu, Alan L. Yuille
Date: 2018-01-25 00:00:00
Proceeding: CVPR 2019 6
Tasks: Multi-Task Learning, Semantic Segmentation

In this paper, we propose a novel Convolutional Neural Network (CNN)
structure for general-purpose multi-task learning (MTL), which enables
automatic feature fusing at every layer from different tasks. This is in
contrast with the most widely used MTL CNN structures which empirically or
heuristically share features on some specific layers (e.g., share all the
features except the last convolutional layer). The proposed layerwise feature
fusing scheme is formulated by combining existing CNN components in a novel
way, with clear mathematical interpretability as discriminative dimensionality
reduction, which is referred to as Neural Discriminative Dimensionality
Reduction (NDDR). Specifically, we first concatenate features with the same
spatial resolution from different tasks according to their channel dimension.
Then, we show that the discriminative dimensionality reduction can be fulfilled
by 1x1 Convolution, Batch Normalization, and Weight Decay in one CNN. The use
of existing CNN components ensures the end-to-end training and the
extensibility of the proposed NDDR layer to various state-of-the-art CNN
architectures in a "plug-and-play" manner. The detailed ablation analysis shows
that the proposed NDDR layer is easy to train and also robust to different
hyperparameters. Experiments on different task sets with various base network
architectures demonstrate the promising performance and desirable
generalizability of our proposed method. The code of our paper is available at
https://github.com/ethanygao/NDDR-CNN.
Mixed Supervised Object Detection with Robust Objectness Transfer
Authors: Yan Li, Junge Zhang, Kaiqi Huang, Jian-Guo Zhang
Date: 2018-02-27 00:00:00
Tasks: Multiple Instance Learning, Object, object-detection, Object Detection

In this paper, we consider the problem of leveraging existing fully labeled categories to improve the weakly supervised detection (WSD) of new object categories, which we refer to as mixed supervised detection (MSD). Different from previous MSD methods that directly transfer the pre-trained object detectors from existing categories to new categories, we propose a more reasonable and robust objectness transfer approach for MSD. In our framework, we first learn domain-invariant objectness knowledge from the existing fully labeled categories. The knowledge is modeled based on invariant features that are robust to the distribution discrepancy between the existing categories and new categories; therefore the resulting knowledge would generalize well to new categories and could assist detection models to reject distractors (e.g., object parts) in weakly labeled images of new categories. Under the guidance of learned objectness knowledge, we utilize multiple instance learning (MIL) to model the concepts of both objects and distractors and to further improve the ability of rejecting distractors in weakly labeled images. Our robust objectness transfer approach outperforms the existing MSD methods, and achieves state-of-the-art results on the challenging ILSVRC2013 detection dataset and the PASCAL VOC datasets.