pwc-archive/papers-with-abstracts huggingface dataset:
Matrix optimization on universal unitary photonic devices
Authors: Sunil Pai, Ben Bartlett, Olav Solgaard, David A. B. Miller
Date: 2018-08-02 00:00:00

Universal unitary photonic devices can apply arbitrary unitary transformations to a vector of input modes and provide a promising hardware platform for fast and energy-efficient machine learning using light. We simulate the gradient-based optimization of random unitary matrices on universal photonic devices composed of imperfect tunable interferometers. If device components are initialized uniform-randomly, the locally-interacting nature of the mesh components biases the optimization search space towards banded unitary matrices, limiting convergence to random unitary matrices. We detail a procedure for initializing the device by sampling from the distribution of random unitary matrices and show that this greatly improves convergence speed. We also explore mesh architecture improvements such as adding extra tunable beamsplitters or permuting waveguide layers to further improve the training speed and scalability of these devices.
Mixture Matrix Completion
Authors: Daniel L. Pimentel-Alarcón
Date: 2018-08-02 00:00:00
Proceeding: NeurIPS 2018
Tasks: Clustering, Matrix Completion, Recommendation Systems

Completing a data matrix X has become an ubiquitous problem in modern data
science, with applications in recommender systems, computer vision, and
networks inference, to name a few. One typical assumption is that X is
low-rank. A more general model assumes that each column of X corresponds to one
of several low-rank matrices. This paper generalizes these models to what we
call mixture matrix completion (MMC): the case where each entry of X
corresponds to one of several low-rank matrices. MMC is a more accurate model
for recommender systems, and brings more flexibility to other completion and
clustering problems. We make four fundamental contributions about this new
model. First, we show that MMC is theoretically possible (well-posed). Second,
we give its precise information-theoretic identifiability conditions. Third, we
derive the sample complexity of MMC. Finally, we give a practical algorithm for
MMC with performance comparable to the state-of-the-art for simpler related
problems, both on synthetic and real data.
Physics-Based Generative Adversarial Models for Image Restoration and Beyond
Authors: Jinshan Pan, Jiangxin Dong, Yang Liu, Jiawei Zhang, Jimmy Ren, Jinhui Tang, Yu-Wing Tai, Ming-Hsuan Yang
Date: 2018-08-02 00:00:00
Tasks: Deblurring, Image Deblurring, Image Dehazing, Image Restoration, Rain Removal

We present an algorithm to directly solve numerous image restoration problems (e.g., image deblurring, image dehazing, image deraining, etc.). These problems are highly ill-posed, and the common assumptions for existing methods are usually based on heuristic image priors. In this paper, we find that these problems can be solved by generative models with adversarial learning. However, the basic formulation of generative adversarial networks (GANs) does not generate realistic images, and some structures of the estimated images are usually not preserved well. Motivated by an interesting observation that the estimated results should be consistent with the observed inputs under the physics models, we propose a physics model constrained learning algorithm so that it can guide the estimation of the specific task in the conventional GAN framework. The proposed algorithm is trained in an end-to-end fashion and can be applied to a variety of image restoration and related low-level vision problems. Extensive experiments demonstrate that our method performs favorably against the state-of-the-art algorithms.
Classification of Building Information Model (BIM) Structures with Deep Learning
Authors: Francesco Lomio, Ricardo Farinha, Mauri Laasonen, Heikki Huttunen
Date: 2018-08-01 00:00:00
Tasks: BIG-bench Machine Learning, Deep Learning, General Classification

In this work we study an application of machine learning to the construction
industry and we use classical and modern machine learning methods to categorize
images of building designs into three classes: Apartment building, Industrial
building or Other. No real images are used, but only images extracted from
Building Information Model (BIM) software, as these are used by the
construction industry to store building designs. For this task, we compared
four different methods: the first is based on classical machine learning, where
Histogram of Oriented Gradients (HOG) was used for feature extraction and a
Support Vector Machine (SVM) for classification; the other three methods are
based on deep learning, covering common pre-trained networks as well as ones
designed from scratch. To validate the accuracy of the models, a database of
240 images was used. The accuracy achieved is 57% for the HOG + SVM model, and
above 89% for the neural networks.
Saccadic Predictive Vision Model with a Fovea
Authors: Michael Hazoglou, Todd Hylton
Date: 2018-08-01 00:00:00
Tasks: model, Prediction

We propose a model that emulates saccades, the rapid movements of the eye,
called the Error Saccade Model, based on the prediction error of the Predictive
Vision Model (PVM). The Error Saccade Model carries out movements of the
model's field of view to regions with the highest prediction error. Comparisons
of the Error Saccade Model on Predictive Vision Models with and without a fovea
show that a fovea-like structure in the input level of the PVM improves the
Error Saccade Model's ability to pursue detailed objects in its view. We
hypothesize that the improvement is due to poorer resolution in the periphery
causing higher prediction error when an object passes, triggering a saccade to
the next location.
MLCapsule: Guarded Offline Deployment of Machine Learning as a Service
Authors: Lucjan Hanzlik, Yang Zhang, Kathrin Grosse, Ahmed Salem, Max Augustin, Michael Backes, Mario Fritz
Date: 2018-08-01 00:00:00
Tasks: BIG-bench Machine Learning

With the widespread use of machine learning (ML) techniques, ML as a service
has become increasingly popular. In this setting, an ML model resides on a
server and users can query it with their data via an API. However, if the
user's input is sensitive, sending it to the server is undesirable and
sometimes even legally not possible. Equally, the service provider does not
want to share the model by sending it to the client for protecting its
intellectual property and pay-per-query business model.
  In this paper, we propose MLCapsule, a guarded offline deployment of machine
learning as a service. MLCapsule executes the model locally on the user's side
and therefore the data never leaves the client. Meanwhile, MLCapsule offers the
service provider the same level of control and security of its model as the
commonly used server-side execution. In addition, MLCapsule is applicable to
offline applications that require local execution. Beyond protecting against
direct model access, we couple the secure offline deployment with defenses
against advanced attacks on machine learning models such as model stealing,
reverse engineering, and membership inference.
Weather Classification: A new multi-class dataset, data augmentation approach and comprehensive evaluations of Convolutional Neural Networks
Authors: Jose Carlos Villarreal Guerra, Zeba Khanam, Shoaib Ehsan, Rustam Stolkin, Klaus McDonald-Maier
Date: 2018-08-01 00:00:00
Tasks: Data Augmentation, General Classification

Weather conditions often disrupt the proper functioning of transportation
systems. Present systems either deploy an array of sensors or use an in-vehicle
camera to predict weather conditions. These solutions have resulted in
incremental cost and limited scope. To ensure smooth operation of all
transportation services in all-weather conditions, a reliable detection system
is necessary to classify weather in wild. The challenges involved in solving
this problem is that weather conditions are diverse in nature and there is an
absence of discriminate features among various weather conditions. The existing
works to solve this problem have been scene specific and have targeted
classification of two categories of weather. In this paper, we have created a
new open source dataset consisting of images depicting three classes of weather
i.e rain, snow and fog called RFS Dataset. A novel algorithm has also been
proposed which has used super pixel delimiting masks as a form of data
augmentation, leading to reasonable results with respect to ten Convolutional
Neural Network architectures.
Data Augmentation for Robust Keyword Spotting under Playback Interference
Authors: Anirudh Raju, Sankaran Panchapagesan, Xing Liu, Arindam Mandal, Nikko Strom
Date: 2018-08-01 00:00:00
Tasks: Acoustic echo cancellation, Data Augmentation, Keyword Spotting

Accurate on-device keyword spotting (KWS) with low false accept and false
reject rate is crucial to customer experience for far-field voice control of
conversational agents. It is particularly challenging to maintain low false
reject rate in real world conditions where there is (a) ambient noise from
external sources such as TV, household appliances, or other speech that is not
directed at the device (b) imperfect cancellation of the audio playback from
the device, resulting in residual echo, after being processed by the Acoustic
Echo Cancellation (AEC) system. In this paper, we propose a data augmentation
strategy to improve keyword spotting performance under these challenging
conditions. The training set audio is artificially corrupted by mixing in music
and TV/movie audio, at different signal to interference ratios. Our results
show that we get around 30-45% relative reduction in false reject rates, at a
range of false alarm rates, under audio playback from such devices.
Compressible Spectral Mixture Kernels with Sparse Dependency Structures for Gaussian Processes
Authors: Kai Chen, Yijue Dai, Feng Yin, Elena Marchiori, Sergios Theodoridis
Date: 2018-08-01 00:00:00
Tasks: Gaussian Processes, Model Compression

Spectral mixture (SM) kernels comprise a powerful class of generalized kernels for Gaussian processes (GPs) to describe complex patterns. This paper introduces model compression and time- and phase (TP) modulated dependency structures to the original (SM) kernel for improved generalization of GPs. Specifically, by adopting Bienaym\'es identity, we generalize the dependency structure through cross-covariance between the SM components. Then, we propose a novel SM kernel with a dependency structure (SMD) by using cross-convolution between the SM components. Furthermore, we ameliorate the expressiveness of the dependency structure by parameterizing it with time and phase delays. The dependency structure has clear interpretations in terms of spectral density, covariance behavior, and sampling path. To enrich the SMD with effective hyperparameter initialization, compressible SM kernel components, and sparse dependency structures, we introduce a novel structure adaptation (SA) algorithm in the end. A thorough comparative analysis of the SMD on both synthetic and real-life applications corroborates its efficacy.
Direct Sparse Odometry with Rolling Shutter
Authors: David Schubert, Nikolaus Demmel, Vladyslav Usenko, Jörg Stückler, Daniel Cremers
Date: 2018-08-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Visual Odometry

Neglecting the effects of rolling-shutter cameras for visual odometry (VO)
severely degrades accuracy and robustness. In this paper, we propose a novel
direct monocular VO method that incorporates a rolling-shutter model. Our
approach extends direct sparse odometry which performs direct bundle adjustment
of a set of recent keyframe poses and the depths of a sparse set of image
points. We estimate the velocity at each keyframe and impose a
constant-velocity prior for the optimization. In this way, we obtain a near
real-time, accurate direct VO method. Our approach achieves improved results on
challenging rolling-shutter sequences over state-of-the-art global-shutter VO.
Open Category Detection with PAC Guarantees
Authors: Si Liu, Risheek Garrepalli, Thomas G. Dietterich, Alan Fern, Dan Hendrycks
Date: 2018-08-01 00:00:00
Proceeding: ICML 2018 7

Open category detection is the problem of detecting "alien" test instances
that belong to categories or classes that were not present in the training
data. In many applications, reliably detecting such aliens is central to
ensuring the safety and accuracy of test set predictions. Unfortunately, there
are no algorithms that provide theoretical guarantees on their ability to
detect aliens under general assumptions. Further, while there are algorithms
for open category detection, there are few empirical results that directly
report alien detection rates. Thus, there are significant theoretical and
empirical gaps in our understanding of open category detection. In this paper,
we take a step toward addressing this gap by studying a simple, but
practically-relevant variant of open category detection. In our setting, we are
provided with a "clean" training set that contains only the target categories
of interest and an unlabeled "contaminated" training set that contains a
fraction $\alpha$ of alien examples. Under the assumption that we know an upper
bound on $\alpha$, we develop an algorithm with PAC-style guarantees on the
alien detection rate, while aiming to minimize false alarms. Empirical results
on synthetic and standard benchmark datasets demonstrate the regimes in which
the algorithm can be effective and provide a baseline for further advancements.
Inference of Users Demographic Attributes based on Homophily in Communication Networks
Authors: Jorge Brea, Javier Burroni, Carlos Sarraute
Date: 2018-08-01 00:00:00
Tasks: Attribute

Over the past decade, mobile phones have become prevalent in all parts of the
world, across all demographic backgrounds. Mobile phones are used by men and
women across a wide age range in both developed and developing countries.
Consequently, they have become one of the most important mechanisms for social
interaction within a population, making them an increasingly important source
of information to understand human demographics and human behaviour.
  In this work we combine two sources of information: communication logs from a
major mobile operator in a Latin American country, and information on the
demographics of a subset of the users population. This allows us to perform an
observational study of mobile phone usage, differentiated by age groups
categories. This study is interesting in its own right, since it provides
knowledge on the structure and demographics of the mobile phone market in the
studied country.
  We then tackle the problem of inferring the age group for all users in the
network. We present here an exclusively graph-based inference method relying
solely on the topological structure of the mobile network, together with a
topological analysis of the performance of the algorithm. The equations for our
algorithm can be described as a diffusion process with two added properties:
(i) memory of its initial state, and (ii) the information is propagated as a
probability vector for each node attribute (instead of the value of the
attribute itself). Our algorithm can successfully infer different age groups
within the network population given known values for a subset of nodes (seed
nodes). Most interestingly, we show that by carefully analysing the topological
relationships between correctly predicted nodes and the seed nodes, we can
characterize particular subsets of nodes for which our inference method has
significantly higher accuracy.
Cooperative Group Optimization with Ants (CGO-AS): Leverage Optimization with Mixed Individual and Social Learning
Authors: Xiao-Feng Xie, Zun-Jing Wang
Date: 2018-08-01 00:00:00
Tasks: Traveling Salesman Problem

We present CGO-AS, a generalized Ant System (AS) implemented in the framework
of Cooperative Group Optimization (CGO), to show the leveraged optimization
with a mixed individual and social learning. Ant colony is a simple yet
efficient natural system for understanding the effects of primary intelligence
on optimization. However, existing AS algorithms are mostly focusing on their
capability of using social heuristic cues while ignoring their individual
learning. CGO can integrate the advantages of a cooperative group and a
low-level algorithm portfolio design, and the agents of CGO can explore both
individual and social search. In CGO-AS, each ant (agent) is added with an
individual memory, and is implemented with a novel search strategy to use
individual and social cues in a controlled proportion. The presented CGO-AS is
therefore especially useful in exposing the power of the mixed individual and
social learning for improving optimization. The optimization performance is
tested with instances of the Traveling Salesman Problem (TSP). The results
prove that a cooperative ant group using both individual and social learning
obtains a better performance than the systems solely using either individual or
social learning. The best performance is achieved under the condition when
agents use individual memory as their primary information source, and
simultaneously use social memory as their searching guidance. In comparison
with existing AS systems, CGO-AS retains a faster learning speed toward those
higher-quality solutions, especially in the later learning cycles. The leverage
in optimization by CGO-AS is highly possible due to its inherent feature of
adaptively maintaining the population diversity in the individual memory of
agents, and of accelerating the learning process with accumulated knowledge in
the social memory.
Mod-DeepESN: Modular Deep Echo State Network
Authors: Zachariah Carmichael, Humza Syed, Stuart Burtner, Dhireesha Kudithipudi
Date: 2018-08-01 00:00:00
Tasks: Time Series, Time Series Analysis, Time Series Prediction

Neuro-inspired recurrent neural network algorithms, such as echo state
networks, are computationally lightweight and thereby map well onto untethered
devices. The baseline echo state network algorithms are shown to be efficient
in solving small-scale spatio-temporal problems. However, they underperform for
complex tasks that are characterized by multi-scale structures. In this
research, an intrinsic plasticity-infused modular deep echo state network
architecture is proposed to solve complex and multiple timescale temporal
tasks. It outperforms state-of-the-art for time series prediction tasks.
A Learning-Based Framework for Two-Dimensional Vehicle Maneuver Prediction over V2V Networks
Authors: Hossein Nourkhiz Mahjoub, Amin Tahmasbi-Sarvestani, Hadi Kazemi, Yaser P. Fallah
Date: 2018-08-01 00:00:00
Tasks: Parameter Prediction, Prediction, Time Series, Time Series Analysis, Time Series Prediction, Trajectory Prediction

Situational awareness in vehicular networks could be substantially improved
utilizing reliable trajectory prediction methods. More precise situational
awareness, in turn, results in notably better performance of critical safety
applications, such as Forward Collision Warning (FCW), as well as comfort
applications like Cooperative Adaptive Cruise Control (CACC). Therefore,
vehicle trajectory prediction problem needs to be deeply investigated in order
to come up with an end to end framework with enough precision required by the
safety applications' controllers. This problem has been tackled in the
literature using different methods. However, machine learning, which is a
promising and emerging field with remarkable potential for time series
prediction, has not been explored enough for this purpose. In this paper, a
two-layer neural network-based system is developed which predicts the future
values of vehicle parameters, such as velocity, acceleration, and yaw rate, in
the first layer and then predicts the two-dimensional, i.e. longitudinal and
lateral, trajectory points based on the first layer's outputs. The performance
of the proposed framework has been evaluated in realistic cut-in scenarios from
Safety Pilot Model Deployment (SPMD) dataset and the results show a noticeable
improvement in the prediction accuracy in comparison with the kinematics model
which is the dominant employed model by the automotive industry. Both ideal and
nonideal communication circumstances have been investigated for our system
evaluation. For non-ideal case, an estimation step is included in the framework
before the parameter prediction block to handle the drawbacks of packet drops
or sensor failures and reconstruct the time series of vehicle parameters at a
desirable frequency.
Neural Arithmetic Logic Units
Authors: Andrew Trask, Felix Hill, Scott Reed, Jack Rae, Chris Dyer, Phil Blunsom
Date: 2018-08-01 00:00:00
Proceeding: NeurIPS 2018 12

Neural networks can learn to represent and manipulate numerical information,
but they seldom generalize well outside of the range of numerical values
encountered during training. To encourage more systematic numerical
extrapolation, we propose an architecture that represents numerical quantities
as linear activations which are manipulated using primitive arithmetic
operators, controlled by learned gates. We call this module a neural arithmetic
logic unit (NALU), by analogy to the arithmetic logic unit in traditional
processors. Experiments show that NALU-enhanced neural networks can learn to
track time, perform arithmetic over images of numbers, translate numerical
language into real-valued scalars, execute computer code, and count objects in
images. In contrast to conventional architectures, we obtain substantially
better generalization both inside and outside of the range of numerical values
encountered during training, often extrapolating orders of magnitude beyond
trained numerical ranges.
Wavelet Sparse Regularization for Manifold-Valued Data
Authors: Martin Storath, Andreas Weinmann
Date: 2018-08-01 00:00:00

In this paper, we consider the sparse regularization of manifold-valued data
with respect to an interpolatory wavelet/multiscale transform. We propose and
study variational models for this task and provide results on their
well-posedness. We present algorithms for a numerical realization of these
models in the manifold setup. Further, we provide experimental results to show
the potential of the proposed schemes for applications.
SlimNets: An Exploration of Deep Model Compression and Acceleration
Authors: Ini Oguntola, Subby Olubeko, Christopher Sweeney
Date: 2018-08-01 00:00:00
Tasks: Knowledge Distillation, Model Compression

Deep neural networks have achieved increasingly accurate results on a wide
variety of complex tasks. However, much of this improvement is due to the
growing use and availability of computational resources (e.g use of GPUs, more
layers, more parameters, etc). Most state-of-the-art deep networks, despite
performing well, over-parameterize approximate functions and take a significant
amount of time to train. With increased focus on deploying deep neural networks
on resource constrained devices like smart phones, there has been a push to
evaluate why these models are so resource hungry and how they can be made more
efficient. This work evaluates and compares three distinct methods for deep
model compression and acceleration: weight pruning, low rank factorization, and
knowledge distillation. Comparisons on VGG nets trained on CIFAR10 show that
each of the models on their own are effective, but that the true power lies in
combining them. We show that by combining pruning and knowledge distillation
methods we can create a compressed network 85 times smaller than the original,
all while retaining 96% of the original model's accuracy.