pwc-archive/papers-with-abstracts huggingface dataset:
Unsupervised speech representation learning using WaveNet autoencoders
Authors: Jan Chorowski, Ron J. Weiss, Samy Bengio, Aäron van den Oord
Date: 2019-01-25 00:00:00
Tasks: Acoustic Unit Discovery, Decoder, Dimensionality Reduction, Representation Learning, Speech Representation Learning

We consider the task of unsupervised extraction of meaningful latent representations of speech by applying autoencoding neural networks to speech waveforms. The goal is to learn a representation able to capture high level semantic content from the signal, e.g.\ phoneme identities, while being invariant to confounding low level details in the signal such as the underlying pitch contour or background noise. Since the learned representation is tuned to contain only phonetic content, we resort to using a high capacity WaveNet decoder to infer information discarded by the encoder from previous samples. Moreover, the behavior of autoencoder models depends on the kind of constraint that is applied to the latent representation. We compare three variants: a simple dimensionality reduction bottleneck, a Gaussian Variational Autoencoder (VAE), and a discrete Vector Quantized VAE (VQ-VAE). We analyze the quality of learned representations in terms of speaker independence, the ability to predict phonetic content, and the ability to accurately reconstruct individual spectrogram frames. Moreover, for discrete encodings extracted using the VQ-VAE, we measure the ease of mapping them to phonemes. We introduce a regularization scheme that forces the representations to focus on the phonetic content of the utterance and report performance comparable with the top entries in the ZeroSpeech 2017 unsupervised acoustic unit discovery task.
Spurious Vanishing Problem in Approximate Vanishing Ideal
Authors: Hiroshi Kera, Yoshihiko Hasegawa
Date: 2019-01-25 00:00:00
Tasks: General Classification

Approximate vanishing ideal is a concept from computer algebra that studies the algebraic varieties behind perturbed data points. To capture the nonlinear structure of perturbed points, the introduction of approximation to exact vanishing ideals plays a critical role. However, such an approximation also gives rise to a theoretical problem---the spurious vanishing problem---in the basis construction of approximate vanishing ideals; namely, obtained basis polynomials can be approximately vanishing simply because of the small coefficients. In this paper, we propose a first general method that enables various basis construction algorithms to overcome the spurious vanishing problem. In particular, we integrate coefficient normalization with polynomial-based basis constructions, which do not need the proper ordering of monomials to process for basis constructions. We further propose a method that takes advantage of the iterative nature of basis construction so that computationally costly operations for coefficient normalization can be circumvented. Moreover, a coefficient truncation method is proposed for further accelerations. From the experiments, it can be shown that the proposed method overcomes the spurious vanishing problem, resulting in shorter feature vectors while sustaining comparable or even lower classification error.
Estimate Sequences for Stochastic Composite Optimization: Variance Reduction, Acceleration, and Robustness to Noise
Authors: Andrei Kulunchakov, Julien Mairal
Date: 2019-01-25 00:00:00
Tasks: Stochastic Optimization

In this paper, we propose a unified view of gradient-based algorithms for stochastic convex composite optimization by extending the concept of estimate sequence introduced by Nesterov. More precisely, we interpret a large class of stochastic optimization methods as procedures that iteratively minimize a surrogate of the objective, which covers the stochastic gradient descent method and variants of the incremental approaches SAGA, SVRG, and MISO/Finito/SDCA. This point of view has several advantages: (i) we provide a simple generic proof of convergence for all of the aforementioned methods; (ii) we naturally obtain new algorithms with the same guarantees; (iii) we derive generic strategies to make these algorithms robust to stochastic noise, which is useful when data is corrupted by small random perturbations. Finally, we propose a new accelerated stochastic gradient descent algorithm and an accelerated SVRG algorithm with optimal complexity that is robust to stochastic noise.
Multiple Hypothesis Tracking Algorithm for Multi-Target Multi-Camera Tracking with Disjoint Views
Authors: Kwangjin Yoon, Young-min Song, Moongu Jeon
Date: 2019-01-25 00:00:00

In this study, a multiple hypothesis tracking (MHT) algorithm for
multi-target multi-camera tracking (MCT) with disjoint views is proposed. Our
method forms track-hypothesis trees, and each branch of them represents a
multi-camera track of a target that may move within a camera as well as move
across cameras. Furthermore, multi-target tracking within a camera is performed
simultaneously with the tree formation by manipulating a status of each track
hypothesis. Each status represents three different stages of a multi-camera
track: tracking, searching, and end-of-track. The tracking status means targets
are tracked by a single camera tracker. In the searching status, the
disappeared targets are examined if they reappear in other cameras. The
end-of-track status does the target exited the camera network due to its
lengthy invisibility. These three status assists MHT to form the
track-hypothesis trees for multi-camera tracking. Furthermore, they present a
gating technique for eliminating of unlikely observation-to-track association.
In the experiments, they evaluate the proposed method using two datasets,
DukeMTMC and NLPR-MCT, which demonstrates that the proposed method outperforms
the state-of-the-art method in terms of improvement of the accuracy. In
addition, they show that the proposed method can operate in real-time and
online.
Beating Stochastic and Adversarial Semi-bandits Optimally and Simultaneously
Authors: Julian Zimmert, Haipeng Luo, Chen-Yu Wei
Date: 2019-01-25 00:00:00

We develop the first general semi-bandit algorithm that simultaneously achieves $\mathcal{O}(\log T)$ regret for stochastic environments and $\mathcal{O}(\sqrt{T})$ regret for adversarial environments without knowledge of the regime or the number of rounds $T$. The leading problem-dependent constants of our bounds are not only optimal in some worst-case sense studied previously, but also optimal for two concrete instances of semi-bandit problems. Our algorithm and analysis extend the recent work of (Zimmert & Seldin, 2019) for the special case of multi-armed bandit, but importantly requires a novel hybrid regularizer designed specifically for semi-bandit. Experimental results on synthetic data show that our algorithm indeed performs well uniformly over different environments. We finally provide a preliminary extension of our results to the full bandit feedback.
Revisiting the Bethe-Hessian: Improved Community Detection in Sparse Heterogeneous Graphs
Authors: Lorenzo Dall'Amico, Romain Couillet, Nicolas Tremblay
Date: 2019-01-25 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Clustering, Community Detection, Stochastic Block Model

Spectral clustering is one of the most popular, yet still incompletely understood, methods for community detection on graphs. This article studies spectral clustering based on the Bethe-Hessian matrix $H_r = (r^2-1)I_n + D-rA$ for sparse heterogeneous graphs (following the degree-corrected stochastic block model) in a two-class setting. For a specific value $r = \zeta$, clustering is shown to be insensitive to the degree heterogeneity. We then study the behavior of the informative eigenvector of $H_{\zeta}$ and, as a result, predict the clustering accuracy. The article concludes with an overview of the generalization to more than two classes along with extensive simulations on synthetic and real networks corroborating our findings.
Robust estimation of tree structured Gaussian Graphical Model
Authors: Ashish Katiyar, Jessica Hoffmann, Constantine Caramanis
Date: 2019-01-25 00:00:00
Tasks: model

Consider jointly Gaussian random variables whose conditional independence
structure is specified by a graphical model. If we observe realizations of the
variables, we can compute the covariance matrix, and it is well known that the
support of the inverse covariance matrix corresponds to the edges of the
graphical model. Instead, suppose we only have noisy observations. If the noise
at each node is independent, we can compute the sum of the covariance matrix
and an unknown diagonal. The inverse of this sum is (in general) dense. We ask:
can the original independence structure be recovered? We address this question
for tree structured graphical models. We prove that this problem is
unidentifiable, but show that this unidentifiability is limited to a small
class of candidate trees. We further present additional constraints under which
the problem is identifiable. Finally, we provide an O(n^3) algorithm to find
this equivalence class of trees.
Post-training 4-bit quantization of convolution networks for rapid-deployment
Authors: Ron Banner, Yury Nahshan, Elad Hoffer, Daniel Soudry
Date: 2018-10-02 00:00:00
Tasks: Quantization

Convolutional neural networks require significant memory bandwidth and storage for intermediate computations, apart from substantial computing resources. Neural network quantization has significant benefits in reducing the amount of intermediate results, but it often requires the full datasets and time-consuming fine tuning to recover the accuracy lost after quantization. This paper introduces the first practical 4-bit post training quantization approach: it does not involve training the quantized model (fine-tuning), nor it requires the availability of the full dataset. We target the quantization of both activations and weights and suggest three complementary methods for minimizing quantization error at the tensor level, two of whom obtain a closed-form analytical solution. Combining these methods, our approach achieves accuracy that is just a few percents less the state-of-the-art baseline across a wide range of convolutional models. The source code to replicate all experiments is available on GitHub: \url{https://github.com/submission2019/cnn-quantization}.
Virtual Conditional Generative Adversarial Networks
Authors: Haifeng Shi, Guanyu Cai, Yuqin Wang, Shaohua Shang, Lianghua He
Date: 2019-01-25 00:00:00
Tasks: Clustering, Conditional Image Generation, Decoder, Image Generation

When trained on multimodal image datasets, normal Generative Adversarial
Networks (GANs) are usually outperformed by class-conditional GANs and ensemble
GANs, but conditional GANs is restricted to labeled datasets and ensemble GANs
lack efficiency. We propose a novel GAN variant called virtual conditional GAN
(vcGAN) which is not only an ensemble GAN with multiple generative paths while
adding almost zero network parameters, but also a conditional GAN that can be
trained on unlabeled datasets without explicit clustering steps or objectives
other than the adversary loss. Inside the vcGAN's generator, a learnable
``analog-to-digital converter (ADC)" module maps a slice of the inputted
multivariate Gaussian noise to discrete/digital noise (virtual label),
according to which a selector selects the corresponding generative path to
produce the sample. All the generative paths share the same decoder network
while in each path the decoder network is fed with a concatenation of a
different pre-computed amplified one-hot vector and the inputted Gaussian
noise. We conducted a lot of experiments on several balanced/imbalanced image
datasets to demonstrate that vcGAN converges faster and achieves improved
Frech\'et Inception Distance (FID). In addition, we show the training byproduct
that the ADC in vcGAN learned the categorical probability of each mode and that
each generative path generates samples of specific mode, which enables
class-conditional sampling. Codes are available at
\url{https://github.com/annonnymmouss/vcgan}
Distributed Policy Iteration for Scalable Approximation of Cooperative Multi-Agent Policies
Authors: Thomy Phan, Kyrill Schmid, Lenz Belzner, Thomas Gabor, Sebastian Feld, Claudia Linnhoff-Popien
Date: 2019-01-25 00:00:00
Tasks: Decision Making, Multi-agent Reinforcement Learning, Reinforcement Learning

Decision making in multi-agent systems (MAS) is a great challenge due to
enormous state and joint action spaces as well as uncertainty, making
centralized control generally infeasible. Decentralized control offers better
scalability and robustness but requires mechanisms to coordinate on joint tasks
and to avoid conflicts. Common approaches to learn decentralized policies for
cooperative MAS suffer from non-stationarity and lacking credit assignment,
which can lead to unstable and uncoordinated behavior in complex environments.
In this paper, we propose Strong Emergent Policy approximation (STEP), a
scalable approach to learn strong decentralized policies for cooperative MAS
with a distributed variant of policy iteration. For that, we use function
approximation to learn from action recommendations of a decentralized
multi-agent planning algorithm. STEP combines decentralized multi-agent
planning with centralized learning, only requiring a generative model for
distributed black box optimization. We experimentally evaluate STEP in two
challenging and stochastic domains with large state and joint action spaces and
show that STEP is able to learn stronger policies than standard multi-agent
reinforcement learning algorithms, when combining multi-agent open-loop
planning with centralized function approximation. The learned policies can be
reintegrated into the multi-agent planning process to further improve
performance.
Misleading Metadata Detection on YouTube
Authors: Priyank Palod, Ayush Patwari, Sudhanshu Bahety, Saurabh Bagchi, Pawan Goyal
Date: 2019-01-25 00:00:00
Tasks: General Classification

YouTube is the leading social media platform for sharing videos. As a result,
it is plagued with misleading content that includes staged videos presented as
real footages from an incident, videos with misrepresented context and videos
where audio/video content is morphed. We tackle the problem of detecting such
misleading videos as a supervised classification task. We develop UCNet - a
deep network to detect fake videos and perform our experiments on two datasets
- VAVD created by us and publicly available FVC [8]. We achieve a macro
averaged F-score of 0.82 while training and testing on a 70:30 split of FVC,
while the baseline model scores 0.36. We find that the proposed model
generalizes well when trained on one dataset and tested on the other.
SecureBoost: A Lossless Federated Learning Framework
Authors: Kewei Cheng, Tao Fan, Yilun Jin, Yang Liu, Tianjian Chen, Dimitrios Papadopoulos, Qiang Yang
Date: 2019-01-25 00:00:00
Tasks: BIG-bench Machine Learning, Entity Alignment, Federated Learning, Privacy Preserving

The protection of user privacy is an important concern in machine learning, as evidenced by the rolling out of the General Data Protection Regulation (GDPR) in the European Union (EU) in May 2018. The GDPR is designed to give users more control over their personal data, which motivates us to explore machine learning frameworks for data sharing that do not violate user privacy. To meet this goal, in this paper, we propose a novel lossless privacy-preserving tree-boosting system known as SecureBoost in the setting of federated learning. SecureBoost first conducts entity alignment under a privacy-preserving protocol and then constructs boosting trees across multiple parties with a carefully designed encryption strategy. This federated learning system allows the learning process to be jointly conducted over multiple parties with common user samples but different feature sets, which corresponds to a vertically partitioned data set. An advantage of SecureBoost is that it provides the same level of accuracy as the non-privacy-preserving approach while at the same time, reveals no information of each private data provider. We show that the SecureBoost framework is as accurate as other non-federated gradient tree-boosting algorithms that require centralized data and thus it is highly scalable and practical for industrial applications such as credit risk analysis. To this end, we discuss information leakage during the protocol execution and propose ways to provably reduce it.
On Output Activation Functions for Adversarial Losses: A Theoretical Analysis via Variational Divergence Minimization and An Empirical Study on MNIST Classification
Authors: Hao-Wen Dong, Yi-Hsuan Yang
Date: 2019-01-25 00:00:00

Recent years have seen adversarial losses been applied to many fields. Their applications extend beyond the originally proposed generative modeling to conditional generative and discriminative settings. While prior work has proposed various output activation functions and regularization approaches, some open questions still remain unanswered. In this paper, we aim to study the following two research questions: 1) What types of output activation functions form a well-behaved adversarial loss? 2) How different combinations of output activation functions and regularization approaches perform empirically against one another? To answer the first question, we adopt the perspective of variational divergence minimization and consider an adversarial loss well-behaved if it behaves as a divergence-like measure between the data and model distributions. Using a generalized formulation for adversarial losses, we derive the necessary and sufficient conditions of a well-behaved adversarial loss. Our analysis reveals a large class of theoretically valid adversarial losses. For the second question, we propose a simple comparative framework for adversarial losses using discriminative adversarial networks. The proposed framework allows us to efficiently evaluate adversarial losses using a standard evaluation metric such as the classification accuracy. With the proposed framework, we evaluate a comprehensive set of 168 combinations of twelve output activation functions and fourteen regularization approaches on the handwritten digit classification problem to decouple their effects. Our empirical findings suggest that there is no single winning combination of output activation functions and regularization approaches across all settings. Our theoretical and empirical results may together serve as a reference for choosing or designing adversarial losses in future research.
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Authors: Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, Jaewoo Kang
Date: 2019-01-25 00:00:00
Tasks: Drug–drug Interaction Extraction, Few-Shot Learning, Language Modelling, Medical Named Entity Recognition, Medical Relation Extraction, Named Entity Recognition, Named Entity Recognition (NER), Question Answering, Relation Extraction, Representation Learning, Sentence Classification, Zero-Shot Learning

Biomedical text mining is becoming increasingly important as the number of biomedical documents rapidly grows. With the progress in natural language processing (NLP), extracting valuable information from biomedical literature has gained popularity among researchers, and deep learning has boosted the development of effective biomedical text mining models. However, directly applying the advancements in NLP to biomedical text mining often yields unsatisfactory results due to a word distribution shift from general domain corpora to biomedical corpora. In this article, we investigate how the recently introduced pre-trained language model BERT can be adapted for biomedical corpora. We introduce BioBERT (Bidirectional Encoder Representations from Transformers for Biomedical Text Mining), which is a domain-specific language representation model pre-trained on large-scale biomedical corpora. With almost the same architecture across tasks, BioBERT largely outperforms BERT and previous state-of-the-art models in a variety of biomedical text mining tasks when pre-trained on biomedical corpora. While BERT obtains performance comparable to that of previous state-of-the-art models, BioBERT significantly outperforms them on the following three representative biomedical text mining tasks: biomedical named entity recognition (0.62% F1 score improvement), biomedical relation extraction (2.80% F1 score improvement) and biomedical question answering (12.24% MRR improvement). Our analysis results show that pre-training BERT on biomedical corpora helps it to understand complex biomedical texts. We make the pre-trained weights of BioBERT freely available at https://github.com/naver/biobert-pretrained, and the source code for fine-tuning BioBERT available at https://github.com/dmis-lab/biobert.
Ask less - Scale Market Research without Annoying Your Customers
Authors: Venkatesh Umaashankar, Girish Shanmugam S
Date: 2019-01-25 00:00:00
Tasks: Marketing

Market research is generally performed by surveying a representative sample
of customers with questions that includes contexts such as psycho-graphics,
demographics, attitude and product preferences. Survey responses are used to
segment the customers into various groups that are useful for targeted
marketing and communication. Reducing the number of questions asked to the
customer has utility for businesses to scale the market research to a large
number of customers. In this work, we model this task using Bayesian networks.
We demonstrate the effectiveness of our approach using an example market
segmentation of broadband customers.
Model-based Deep Reinforcement Learning for Dynamic Portfolio Optimization
Authors: Pengqian Yu, Joon Sern Lee, Ilya Kulyatin, Zekun Shi, Sakyasingha Dasgupta
Date: 2019-01-25 00:00:00
Tasks: Data Augmentation, Deep Reinforcement Learning, Portfolio Optimization, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Dynamic portfolio optimization is the process of sequentially allocating
wealth to a collection of assets in some consecutive trading periods, based on
investors' return-risk profile. Automating this process with machine learning
remains a challenging problem. Here, we design a deep reinforcement learning
(RL) architecture with an autonomous trading agent such that, investment
decisions and actions are made periodically, based on a global objective, with
autonomy. In particular, without relying on a purely model-free RL agent, we
train our trading agent using a novel RL architecture consisting of an infused
prediction module (IPM), a generative adversarial data augmentation module
(DAM) and a behavior cloning module (BCM). Our model-based approach works with
both on-policy or off-policy RL algorithms. We further design the back-testing
and execution engine which interact with the RL agent in real time. Using
historical {\em real} financial market data, we simulate trading with practical
constraints, and demonstrate that our proposed model is robust, profitable and
risk-sensitive, as compared to baseline trading strategies and model-free RL
agents from prior work.