pwc-archive/papers-with-abstracts huggingface dataset:
Deep Signal Recovery with One-Bit Quantization
Authors: Shahin Khobahi, Naveed Naimipour, Mojtaba Soltanalian, Yonina C. Eldar
Date: 2018-11-30 00:00:00
Tasks: BIG-bench Machine Learning, Computational Efficiency, Inference Optimization, Quantization

Machine learning, and more specifically deep learning, have shown remarkable
performance in sensing, communications, and inference. In this paper, we
consider the application of the deep unfolding technique in the problem of
signal reconstruction from its one-bit noisy measurements. Namely, we propose a
model-based machine learning method and unfold the iterations of an inference
optimization algorithm into the layers of a deep neural network for one-bit
signal recovery. The resulting network, which we refer to as DeepRec, can
efficiently handle the recovery of high-dimensional signals from acquired
one-bit noisy measurements. The proposed method results in an improvement in
accuracy and computational efficiency with respect to the original framework as
shown through numerical analysis.
Testing Changes in Communities for the Stochastic Block Model
Authors: Aditya Gangrade, Praveen Venkatesh, Bobak Nazer, Venkatesh Saligrama
Date: 2018-11-29 00:00:00
Tasks: Stochastic Block Model, Two-sample testing

We propose and analyze the problems of \textit{community goodness-of-fit and two-sample testing} for stochastic block models (SBM), where changes arise due to modification in community memberships of nodes. Motivated by practical applications, we consider the challenging sparse regime, where expected node degrees are constant, and the inter-community mean degree ($b$) scales proportionally to intra-community mean degree ($a$). Prior work has sharply characterized partial or full community recovery in terms of a "signal-to-noise ratio" ($\mathrm{SNR}$) based on $a$ and $b$. For both problems, we propose computationally-efficient tests that can succeed far beyond the regime where recovery of community membership is even possible. Overall, for large changes, $s \gg \sqrt{n}$, we need only $\mathrm{SNR}= O(1)$ whereas a na\"ive test based on community recovery with $O(s)$ errors requires $\mathrm{SNR}= \Theta(\log n)$. Conversely, in the small change regime, $s \ll \sqrt{n}$, via an information-theoretic lower bound, we show that, surprisingly, no algorithm can do better than the na\"ive algorithm that first estimates the community up to $O(s)$ errors and then detects changes. We validate these phenomena numerically on SBMs and on real-world datasets as well as Markov Random Fields where we only observe node data rather than the existence of links.
Correspondence Analysis of Government Expenditure Patterns
Authors: Hsiang Hsu, Flavio P. Calmon, José Cândido Silveira Santos Filho, Andre P. Calmon, Salman Salamatian
Date: 2018-11-29 00:00:00
Tasks: BIG-bench Machine Learning

We analyze expenditure patterns of discretionary funds by Brazilian congress
members. This analysis is based on a large dataset containing over $7$ million
expenses made publicly available by the Brazilian government. This dataset has,
up to now, remained widely untouched by machine learning methods. Our main
contributions are two-fold: (i) we provide a novel dataset benchmark for
machine learning-based efforts for government transparency to the broader
research community, and (ii) introduce a neural network-based approach for
analyzing and visualizing outlying expense patterns. Our hope is that the
approach presented here can inspire new machine learning methodologies for
government transparency applicable to other developing nations.
Simulated Tempering Langevin Monte Carlo II: An Improved Proof using Soft Markov Chain Decomposition
Authors: Rong Ge, Holden Lee, Andrej Risteski
Date: 2018-11-29 00:00:00

A key task in Bayesian machine learning is sampling from distributions that are only specified up to a partition function (i.e., constant of proportionality). One prevalent example of this is sampling posteriors in parametric distributions, such as latent-variable generative models. However sampling (even very approximately) can be #P-hard. Classical results going back to Bakry and \'Emery (1985) on sampling focus on log-concave distributions, and show a natural Markov chain called Langevin diffusion mixes in polynomial time. However, all log-concave distributions are uni-modal, while in practice it is very common for the distribution of interest to have multiple modes. In this case, Langevin diffusion suffers from torpid mixing. We address this problem by combining Langevin diffusion with simulated tempering. The result is a Markov chain that mixes more rapidly by transitioning between different temperatures of the distribution. We analyze this Markov chain for a mixture of (strongly) log-concave distributions of the same shape. In particular, our technique applies to the canonical multi-modal distribution: a mixture of gaussians (of equal variance). Our algorithm efficiently samples from these distributions given only access to the gradient of the log-pdf. For the analysis, we introduce novel techniques for proving spectral gaps based on decomposing the action of the generator of the diffusion. Previous approaches rely on decomposing the state space as a partition of sets, while our approach can be thought of as decomposing the stationary measure as a mixture of distributions (a "soft partition"). Additional materials for the paper can be found at http://holdenlee.github.io/Simulated%20tempering%20Langevin%20Monte%20Carlo.html. The proof and results have been improved and generalized from the precursor at arXiv:1710.02736.
Learning Groupwise Multivariate Scoring Functions Using Deep Neural Networks
Authors: Qingyao Ai, Xuanhui Wang, Sebastian Bruch, Nadav Golbandi, Michael Bendersky, Marc Najork
Date: 2018-11-11 00:00:00
Tasks: Learning-To-Rank

While in a classification or a regression setting a label or a value is assigned to each individual document, in a ranking setting we determine the relevance ordering of the entire input document list. This difference leads to the notion of relative relevance between documents in ranking. The majority of the existing learning-to-rank algorithms model such relativity at the loss level using pairwise or listwise loss functions. However, they are restricted to univariate scoring functions, i.e., the relevance score of a document is computed based on the document itself, regardless of other documents in the list. To overcome this limitation, we propose a new framework for multivariate scoring functions, in which the relevance score of a document is determined jointly by multiple documents in the list. We refer to this framework as GSFs -- groupwise scoring functions. We learn GSFs with a deep neural network architecture, and demonstrate that several representative learning-to-rank algorithms can be modeled as special cases in our framework. We conduct evaluation using click logs from one of the largest commercial email search engines, as well as a public benchmark dataset. In both cases, GSFs lead to significant performance improvements, especially in the presence of sparse textual features.
TF-Ranking: Scalable TensorFlow Library for Learning-to-Rank
Authors: Rama Kumar Pasumarthi, Sebastian Bruch, Xuanhui Wang, Cheng Li, Michael Bendersky, Marc Najork, Jan Pfeifer, Nadav Golbandi, Rohan Anil, Stephan Wolf
Date: 2018-11-30 00:00:00
Tasks: Document Summarization, Learning-To-Rank, Question Answering, Recommendation Systems

Learning-to-Rank deals with maximizing the utility of a list of examples presented to the user, with items of higher relevance being prioritized. It has several practical applications such as large-scale search, recommender systems, document summarization and question answering. While there is widespread support for classification and regression based learning, support for learning-to-rank in deep learning has been limited. We propose TensorFlow Ranking, the first open source library for solving large-scale ranking problems in a deep learning framework. It is highly configurable and provides easy-to-use APIs to support different scoring mechanisms, loss functions and evaluation metrics in the learning-to-rank setting. Our library is developed on top of TensorFlow and can thus fully leverage the advantages of this platform. For example, it is highly scalable, both in training and in inference, and can be used to learn ranking models over massive amounts of user activity data, which can include heterogeneous dense and sparse features. We empirically demonstrate the effectiveness of our library in learning ranking functions for large-scale search and recommendation applications in Gmail and Google Drive. We also show that ranking models built using our model scale well for distributed training, without significant impact on metrics. The proposed library is available to the open source community, with the hope that it facilitates further academic research and industrial applications in the field of learning-to-rank.
Automatic Relevance Determination in Nonnegative Matrix Factorization with the β-Divergence
Authors: Vincent Y. F. Tan, Cédric Févotte
Date: 2011-11-25 00:00:00
Tasks: Stock Price Prediction

This paper addresses the estimation of the latent dimensionality in
nonnegative matrix factorization (NMF) with the \beta-divergence. The
\beta-divergence is a family of cost functions that includes the squared
Euclidean distance, Kullback-Leibler and Itakura-Saito divergences as special
cases. Learning the model order is important as it is necessary to strike the
right balance between data fidelity and overfitting. We propose a Bayesian
model based on automatic relevance determination in which the columns of the
dictionary matrix and the rows of the activation matrix are tied together
through a common scale parameter in their prior. A family of
majorization-minimization algorithms is proposed for maximum a posteriori (MAP)
estimation. A subset of scale parameters is driven to a small lower bound in
the course of inference, with the effect of pruning the corresponding spurious
components. We demonstrate the efficacy and robustness of our algorithms by
performing extensive experiments on synthetic data, the swimmer dataset, a
music decomposition example and a stock price prediction task.
Multi$^{\mathbf{3}}$Net: Segmenting Flooded Buildings via Fusion of Multiresolution, Multisensor, and Multitemporal Satellite Imagery
Authors: Tim G. J. Rudner, Marc Rußwurm, Jakub Fil, Ramona Pelich, Benjamin Bischke, Veronika Kopackova, Piotr Bilinski
Date: 2018-12-05 00:00:00
Tasks: Decoder, Flooded Building Segmentation, Segmentation

We propose a novel approach for rapid segmentation of flooded buildings by
fusing multiresolution, multisensor, and multitemporal satellite imagery in a
convolutional neural network. Our model significantly expedites the generation
of satellite imagery-based flood maps, crucial for first responders and local
authorities in the early stages of flood events. By incorporating multitemporal
satellite imagery, our model allows for rapid and accurate post-disaster damage
assessment and can be used by governments to better coordinate medium- and
long-term financial assistance programs for affected areas. The network
consists of multiple streams of encoder-decoder architectures that extract
spatiotemporal information from medium-resolution images and spatial
information from high-resolution images before fusing the resulting
representations into a single medium-resolution segmentation map of flooded
buildings. We compare our model to state-of-the-art methods for building
footprint segmentation as well as to alternative fusion approaches for the
segmentation of flooded buildings and find that our model performs best on both
tasks. We also demonstrate that our model produces highly accurate segmentation
maps of flooded buildings using only publicly available medium-resolution data
instead of significantly more detailed but sparsely available very
high-resolution data. We release the first open-source dataset of fully
preprocessed and labeled multiresolution, multispectral, and multitemporal
satellite images of disaster sites along with our source code.
Robust Machine Learning Applied to Astronomical Datasets III: Probabilistic Photometric Redshifts for Galaxies and Quasars in the SDSS and GALEX
Authors: Nicholas M. Ball, Robert J. Brunner, Adam D. Myers, Natalie E. Strand, Stacey L. Alberts, David Tcheng
Date: 2008-04-21 00:00:00

We apply machine learning in the form of a nearest neighbor instance-based
algorithm (NN) to generate full photometric redshift probability density
functions (PDFs) for objects in the Fifth Data Release of the Sloan Digital Sky
Survey (SDSS DR5). We use a conceptually simple but novel application of NN to
generate the PDFs - perturbing the object colors by their measurement error -
and using the resulting instances of nearest neighbor distributions to generate
numerous individual redshifts. When the redshifts are compared to existing SDSS
spectroscopic data, we find that the mean value of each PDF has a dispersion
between the photometric and spectroscopic redshift consistent with other
machine learning techniques, being sigma = 0.0207 +/- 0.0001 for main sample
galaxies to r < 17.77 mag, sigma = 0.0243 +/- 0.0002 for luminous red galaxies
to r < ~19.2 mag, and sigma = 0.343 +/- 0.005 for quasars to i < 20.3 mag. The
PDFs allow the selection of subsets with improved statistics. For quasars, the
improvement is dramatic: for those with a single peak in their probability
distribution, the dispersion is reduced from 0.343 to sigma = 0.117 +/- 0.010,
and the photometric redshift is within 0.3 of the spectroscopic redshift for
99.3 +/- 0.1% of the objects. Thus, for this optical quasar sample, we can
virtually eliminate 'catastrophic' photometric redshift estimates. In addition
to the SDSS sample, we incorporate ultraviolet photometry from the Third Data
Release of the Galaxy Evolution Explorer All-Sky Imaging Survey (GALEX AIS GR3)
to create PDFs for objects seen in both surveys. For quasars, the increased
coverage of the observed frame UV of the SED results in significant improvement
over the full SDSS sample, with sigma = 0.234 +/- 0.010. We demonstrate that
this improvement is genuine. [Abridged]
A Fast Algorithm for Computing High-dimensional Risk Parity Portfolios
Authors: Théophile Griveau-Billion, Jean-Charles Richard, Thierry Roncalli
Date: 2013-11-16 00:00:00
Tasks: Vocal Bursts Intensity Prediction

In this paper we propose a cyclical coordinate descent (CCD) algorithm for
solving high dimensional risk parity problems. We show that this algorithm
converges and is very fast even with large covariance matrices (n > 500).
Comparison with existing algorithms also shows that it is one of the most
efficient algorithms.
Dissecting Person Re-identification from the Viewpoint of Viewpoint
Authors: Xiaoxiao Sun, Liang Zheng
Date: 2018-12-05 00:00:00
Proceeding: CVPR 2019 6
Tasks: Person Re-Identification

Variations in visual factors such as viewpoint, pose, illumination and background, are usually viewed as important challenges in person re-identification (re-ID). In spite of acknowledging these factors to be influential, quantitative studies on how they affect a re-ID system are still lacking. To derive insights in this scientific campaign, this paper makes an early attempt in studying a particular factor, viewpoint. We narrow the viewpoint problem down to the pedestrian rotation angle to obtain focused conclusions. In this regard, this paper makes two contributions to the community. First, we introduce a large-scale synthetic data engine, PersonX. Composed of hand-crafted 3D person models, the salient characteristic of this engine is "controllable". That is, we are able to synthesize pedestrians by setting the visual variables to arbitrary values. Second, on the 3D data engine, we quantitatively analyze the influence of pedestrian rotation angle on re-ID accuracy. Comprehensively, the person rotation angles are precisely customized from 0 to 360, allowing us to investigate its effect on the training, query, and gallery sets. Extensive experiment helps us have a deeper understanding of the fundamental problems in person re-ID. Our research also provides useful insights for dataset building and future practical usage, e.g., a person of a side view makes a better query.
The effects of negative adaptation in Model-Agnostic Meta-Learning
Authors: Tristan Deleu, Yoshua Bengio
Date: 2018-12-05 00:00:00
Tasks: Few-Shot Learning, Meta-Learning, Meta Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

The capacity of meta-learning algorithms to quickly adapt to a variety of
tasks, including ones they did not experience during meta-training, has been a
key factor in the recent success of these methods on few-shot learning
problems. This particular advantage of using meta-learning over standard
supervised or reinforcement learning is only well founded under the assumption
that the adaptation phase does improve the performance of our model on the task
of interest. However, in the classical framework of meta-learning, this
constraint is only mildly enforced, if not at all, and we only see an
improvement on average over a distribution of tasks. In this paper, we show
that the adaptation in an algorithm like MAML can significantly decrease the
performance of an agent in a meta-reinforcement learning setting, even on a
range of meta-training tasks.
Blockchain Enabled Trustless API Marketplace
Authors: Vijay Arya, Sayandeep Sen, Palani Kodeswaran
Date: 2018-12-05 00:00:00
Tasks: image-classification, Image Classification

There has been an unprecedented surge in the number of service providers
offering a wide range of machine learning prediction APIs for tasks such as
image classification, language translation, etc. thereby monetizing the
underlying data and trained models. Typically, a data owner (API provider)
develops a model, often over proprietary data, and leverages the infrastructure
services of a cloud vendor for hosting and serving API requests. Clearly, this
model assumes complete trust between the API Provider and cloud vendor. On the
other hand, a malicious/buggy cloud vendor may copy the APIs and offer an
identical service, under-report model usage metrics, or unfairly discriminate
between different API providers by offering them a nominal share of the
revenue. In this work, we present the design of a blockchain based
decentralized trustless API marketplace that enables all the stakeholders in
the API ecosystem to audit the behavior of the parties without having to trust
a single centralized entity. In particular, our system divides an AI model into
multiple pieces and deploys them among multiple cloud vendors who then
collaboratively execute the APIs. Our design ensures that cloud vendors cannot
collude with each other to steal the combined model, while individual cloud
vendors and clients cannot repudiate their input or model executions.
End-to-end contextual speech recognition using class language models and a token passing decoder
Authors: Zhehuai Chen, Mahaveer Jain, Yongqiang Wang, Michael L. Seltzer, Christian Fuegen
Date: 2018-12-05 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Decoder, speech-recognition, Speech Recognition

End-to-end modeling (E2E) of automatic speech recognition (ASR) blends all
the components of a traditional speech recognition system into a unified model.
Although it simplifies training and decoding pipelines, the unified model is
hard to adapt when mismatch exists between training and test data. In this
work, we focus on contextual speech recognition, which is particularly
challenging for E2E models because it introduces significant mismatch between
training and test data. To improve the performance in the presence of complex
contextual information, we propose to use class-based language models(CLM) that
can populate the classes with contextdependent information in real-time. To
enable this approach to scale to a large number of class members and minimize
search errors, we propose a token passing decoder with efficient token
recombination for E2E systems for the first time. We evaluate the proposed
system on general and contextual ASR, and achieve relative 62% Word Error
Rate(WER) reduction for contextual ASR without hurting performance for general
ASR. We show that the proposed method performs well without modification of the
decoding hyper-parameters across tasks, making it a general solution for E2E
ASR.
HEVC Inter Coding Using Deep Recurrent Neural Networks and Artificial Reference Pictures
Authors: Felix Haub, Thorsten Laude, Jörn Ostermann
Date: 2018-12-05 00:00:00
Tasks: Motion Compensation, Prediction

The efficiency of motion compensated prediction in modern video codecs highly
depends on the available reference pictures. Occlusions and non-linear motion
pose challenges for the motion compensation and often result in high bit rates
for the prediction error. We propose the generation of artificial reference
pictures using deep recurrent neural networks. Conceptually, a reference
picture at the time instance of the currently coded picture is generated from
previously reconstructed conventional reference pictures. Based on these
artificial reference pictures, we propose a complete coding pipeline based on
HEVC. By using the artificial reference pictures for motion compensated
prediction, average BD-rate gains of 1.5% over HEVC are achieved.
An Unpaired Shape Transforming Method for Image Translation and Cross-Domain Retrieval
Authors: Kaili Wang, Liqian Ma, Jose Oramas, Luc van Gool, Tinne Tuytelaars
Date: 2018-12-05 00:00:00
Tasks: Image-to-Image Translation, Retrieval, Translation

We address the problem of unpaired geometric image-to-image translation. Rather than transferring the style of an image as a whole, our goal is to translate the geometry of an object as depicted in different domains while preserving its appearance characteristics. Our model is trained in an unpaired fashion, i.e. without the need of paired images during training. It performs all steps of the shape transfer within a single model and without additional post-processing stages. Extensive experiments on the VITON, CMU-Multi-PIE and our own FashionStyle datasets show the effectiveness of the method. In addition, we show that despite their low-dimensionality, the features learned by our model are useful to the item retrieval task.
SADA: Semantic Adversarial Diagnostic Attacks for Autonomous Applications
Authors: Abdullah Hamdi, Matthias Müller, Bernard Ghanem
Date: 2018-12-05 00:00:00
Tasks: Adversarial Attack, Autonomous Driving, Autonomous Navigation, Diagnostic, object-detection, Object Detection

One major factor impeding more widespread adoption of deep neural networks (DNNs) is their lack of robustness, which is essential for safety-critical applications such as autonomous driving. This has motivated much recent work on adversarial attacks for DNNs, which mostly focus on pixel-level perturbations void of semantic meaning. In contrast, we present a general framework for adversarial attacks on trained agents, which covers semantic perturbations to the environment of the agent performing the task as well as pixel-level attacks. To do this, we re-frame the adversarial attack problem as learning a distribution of parameters that always fools the agent. In the semantic case, our proposed adversary (denoted as BBGAN) is trained to sample parameters that describe the environment with which the black-box agent interacts, such that the agent performs its dedicated task poorly in this environment. We apply BBGAN on three different tasks, primarily targeting aspects of autonomous navigation: object detection, self-driving, and autonomous UAV racing. On these tasks, BBGAN can generate failure cases that consistently fool a trained agent.