pwc-archive/papers-with-abstracts huggingface dataset:
Learning Condensed and Aligned Features for Unsupervised Domain Adaptation Using Label Propagation
Authors: Jaeyoon Yoo, Changhwa Park, Yongjun Hong, Sungroh Yoon
Date: 2019-03-12 00:00:00
Tasks: Domain Adaptation, Unsupervised Domain Adaptation

Unsupervised domain adaptation aiming to learn a specific task for one domain
using another domain data has emerged to address the labeling issue in
supervised learning, especially because it is difficult to obtain massive
amounts of labeled data in practice. The existing methods have succeeded by
reducing the difference between the embedded features of both domains, but the
performance is still unsatisfactory compared to the supervised learning scheme.
This is attributable to the embedded features that lay around each other but do
not align perfectly and establish clearly separable clusters. We propose a
novel domain adaptation method based on label propagation and cycle consistency
to let the clusters of the features from the two domains overlap exactly and
become clear for high accuracy. Specifically, we introduce cycle consistency to
enforce the relationship between each cluster and exploit label propagation to
achieve the association between the data from the perspective of the manifold
structure instead of a one-to-one relation. Hence, we successfully formed
aligned and discriminative clusters. We present the empirical results of our
method for various domain adaptation scenarios and visualize the embedded
features to prove that our method is critical for better domain adaptation.
Deep Multi-Agent Reinforcement Learning with Discrete-Continuous Hybrid Action Spaces
Authors: Haotian Fu, Hongyao Tang, Jianye Hao, Zihan Lei, Yingfeng Chen, Changjie Fan
Date: 2019-03-12 00:00:00
Tasks: Deep Reinforcement Learning, Multi-agent Reinforcement Learning, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Deep Reinforcement Learning (DRL) has been applied to address a variety of
cooperative multi-agent problems with either discrete action spaces or
continuous action spaces. However, to the best of our knowledge, no previous
work has ever succeeded in applying DRL to multi-agent problems with
discrete-continuous hybrid (or parameterized) action spaces which is very
common in practice. Our work fills this gap by proposing two novel algorithms:
Deep Multi-Agent Parameterized Q-Networks (Deep MAPQN) and Deep Multi-Agent
Hierarchical Hybrid Q-Networks (Deep MAHHQN). We follow the centralized
training but decentralized execution paradigm: different levels of
communication between different agents are used to facilitate the training
process, while each agent executes its policy independently based on local
observations during execution. Our empirical results on several challenging
tasks (simulated RoboCup Soccer and game Ghost Story) show that both Deep MAPQN
and Deep MAHHQN are effective and significantly outperform existing independent
deep parameterized Q-learning method.
A Capsule-unified Framework of Deep Neural Networks for Graphical Programming
Authors: Yujian Li, Chuanhui Shan
Date: 2019-03-07 00:00:00
Tasks: Deep Learning

Recently, the growth of deep learning has produced a large number of deep
neural networks. How to describe these networks unifiedly is becoming an
important issue. We first formalize neural networks in a mathematical
definition, give their directed graph representations, and prove a generation
theorem about the induced networks of connected directed acyclic graphs. Then,
using the concept of capsule to extend neural networks, we set up a
capsule-unified framework for deep learning, including a mathematical
definition of capsules, an induced model for capsule networks and a universal
backpropagation algorithm for training them. Finally, we discuss potential
applications of the framework to graphical programming with standard graphical
symbols of capsules, neurons, and connections.
Theory III: Dynamics and Generalization in Deep Networks
Authors: Andrzej Banburski, Qianli Liao, Brando Miranda, Lorenzo Rosasco, Fernanda De La Torre, Jack Hidary, Tomaso Poggio
Date: 2019-03-12 00:00:00

The key to generalization is controlling the complexity of the network. However, there is no obvious control of complexity -- such as an explicit regularization term -- in the training of deep networks for classification. We will show that a classical form of norm control -- but kind of hidden -- is present in deep networks trained with gradient descent techniques on exponential-type losses. In particular, gradient descent induces a dynamics of the normalized weights which converge for $t \to \infty$ to an equilibrium which corresponds to a minimum norm (or maximum margin) solution. For sufficiently large but finite $\rho$ -- and thus finite $t$ -- the dynamics converges to one of several margin maximizers, with the margin monotonically increasing towards a limit stationary point of the flow. In the usual case of stochastic gradient descent, most of the stationary points are likely to be convex minima corresponding to a constrained minimizer -- the network with normalized weights-- which corresponds to vanishing regularization. The solution has zero generalization gap, for fixed architecture, asymptotically for $N \to \infty$, where $N$ is the number of training examples. Our approach extends some of the original results of Srebro from linear networks to deep networks and provides a new perspective on the implicit bias of gradient descent. We believe that the elusive complexity control we describe is responsible for the puzzling empirical finding of good predictive performance by deep networks, despite overparametrization.
Generating Compact Geometric Track-Maps for Train Positioning Applications
Authors: Hanno Winter, Stefan Luthardt, Volker Willert, Jürgen Adamy
Date: 2019-03-12 00:00:00

In this paper, we present a method to generate compact geometric track-maps for train-borne localization applications. Therefore, we first give a brief overview on the purpose of track maps in train-positioning applications. It becomes apparent that there are hardly any adequate methods to generate suitable geometric track-maps. This is why we present a novel map generation procedure. It uses an optimization formulation to find the continuous sequence of track geometries that fits the available measurement data best. The optimization is initialized with the results from a localization filter developed in our previous work. The localization filter also provides the required information for shape identification and measurement association. The presented approach will be evaluated on simulated data as well as on real measurements.
Duration-of-Stay Storage Assignment under Uncertainty
Authors: Michael Lingzhi Li, Elliott Wolf, Daniel Wintz
Date: 2019-03-12 00:00:00
Proceeding: ICLR 2020 1

Optimizing storage assignment is a central problem in warehousing. Past literature has shown the superiority of the Duration-of-Stay (DoS) method in assigning pallets, but the methodology requires perfect prior knowledge of DoS for each pallet, which is unknown and uncertain under realistic conditions. The dynamic nature of a warehouse further complicates the validity of synthetic data testing that is often conducted for algorithms. In this paper, in collaboration with a large cold storage company, we release the first publicly available set of warehousing records to facilitate research into this central problem. We introduce a new framework for storage assignment that accounts for uncertainty in warehouses. Then, by utilizing a combination of convolutional and recurrent neural network models, ParallelNet, we show that it is able to predict future shipments well: it achieves up to 29% decrease in MAPE compared to CNN-LSTM on unseen future shipments, and suffers less performance decay over time. The framework is then integrated into a first-of-its-kind Storage Assignment system, which is being piloted in warehouses across the country, with initial results showing up to 19% in labor savings.
Cause Identification of Electromagnetic Transient Events using Spatiotemporal Feature Learning
Authors: Iman Niazazari, Reza Jalilzadeh Hamidi, Hanif Livani, Reza Arghandeh
Date: 2019-03-10 00:00:00

This paper presents a spatiotemporal unsupervised feature learning method for
cause identification of electromagnetic transient events (EMTE) in power grids.
The proposed method is formulated based on the availability of
time-synchronized high-frequency measurement, and using the convolutional
neural network (CNN) as the spatiotemporal feature representation along with
softmax function. Despite the existing threshold-based, or energy-based events
analysis methods, such as support vector machine (SVM), autoencoder, and
tapered multi-layer perception (t-MLP) neural network, the proposed feature
learning is carried out with respect to both time and space. The effectiveness
of the proposed feature learning and the subsequent cause identification is
validated through the EMTP simulation of different events such as line
energization, capacitor bank energization, lightning, fault, and high-impedance
fault in the IEEE 30-bus, and the real-time digital simulation (RTDS) of the
WSCC 9-bus system.
SPMF: A Social Trust and Preference Segmentation-based Matrix Factorization Recommendation Algorithm
Authors: Wei Peng, Baogui Xin
Date: 2019-03-11 00:00:00
Tasks: Marketing

The traditional social recommendation algorithm ignores the following fact:
the preferences of users with trust relationships are not necessarily similar,
and the consideration of user preference similarity should be limited to
specific areas. A social trust and preference segmentation-based matrix
factorization (SPMF) recommendation system is proposed to solve the
above-mentioned problems. Experimental results based on the Ciao and Epinions
datasets show that the accuracy of the SPMF algorithm is significantly higher
than that of some state-of-the-art recommendation algorithms. The proposed SPMF
algorithm is a more accurate and effective recommendation algorithm based on
distinguishing the difference of trust relations and preference domain, which
can support commercial activities such as product marketing.
A Machine Learning Dataset Prepared From the NASA Solar Dynamics Observatory Mission
Authors: Richard Galvez, David F. Fouhey, Meng Jin, Alexandre Szenicer, Andrés Muñoz-Jaramillo, Mark C. M. Cheung, Paul J. Wright, Monica G. Bobra, Yang Liu, James Mason, Rajat Thomas
Date: 2019-03-11 00:00:00
Tasks: BIG-bench Machine Learning

In this paper we present a curated dataset from the NASA Solar Dynamics
Observatory (SDO) mission in a format suitable for machine learning research.
Beginning from level 1 scientific products we have processed various
instrumental corrections, downsampled to manageable spatial and temporal
resolutions, and synchronized observations spatially and temporally. We
illustrate the use of this dataset with two example applications: forecasting
future EVE irradiance from present EVE irradiance and translating HMI
observations into AIA observations. For each application we provide metrics and
baselines for future model comparison. We anticipate this curated dataset will
facilitate machine learning research in heliophysics and the physical sciences
generally, increasing the scientific return of the SDO mission. This work is a
direct result of the 2018 NASA Frontier Development Laboratory Program. Please
see the appendix for access to the dataset.
Wavelet regression and additive models for irregularly spaced data
Authors: Asad Haris, Noah Simon, Ali Shojaie
Date: 2019-03-11 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Additive models, regression

We present a novel approach for nonparametric regression using wavelet basis
functions. Our proposal, $\texttt{waveMesh}$, can be applied to non-equispaced
data with sample size not necessarily a power of 2. We develop an efficient
proximal gradient descent algorithm for computing the estimator and establish
adaptive minimax convergence rates. The main appeal of our approach is that it
naturally extends to additive and sparse additive models for a potentially
large number of covariates. We prove minimax optimal convergence rates under a
weak compatibility condition for sparse additive models. The compatibility
condition holds when we have a small number of covariates. Additionally, we
establish convergence rates for when the condition is not met. We complement
our theoretical results with empirical studies comparing $\texttt{waveMesh}$ to
existing methods.
Provably Correct Learning Algorithms in the Presence of Time-Varying Features Using a Variational Perspective
Authors: Joseph E. Gaudio, Travis E. Gibson, Anuradha M. Annaswamy, Michael A. Bolender
Date: 2019-03-12 00:00:00
Tasks: BIG-bench Machine Learning

Features in machine learning problems are often time-varying and may be related to outputs in an algebraic or dynamical manner. The dynamic nature of these machine learning problems renders current higher order accelerated gradient descent methods unstable or weakens their convergence guarantees. Inspired by methods employed in adaptive control, this paper proposes new algorithms for the case when time-varying features are present, and demonstrates provable performance guarantees. In particular, we develop a unified variational perspective within a continuous time algorithm. This variational perspective includes higher order learning concepts and normalization, both of which stem from adaptive control, and allows stability to be established for dynamical machine learning problems where time-varying features are present. These higher order algorithms are also examined for provably correct learning in adaptive control and identification. Simulations are provided to verify the theoretical results.
Supervised Machine Learning Techniques for Trojan Detection with Ring Oscillator Network
Authors: Kyle Worley, Md Tauhidur Rahman
Date: 2019-03-12 00:00:00
Tasks: BIG-bench Machine Learning, Binary Classification, General Classification

With the globalization of the semiconductor manufacturing process, electronic
devices are powerless against malicious modification of hardware in the supply
chain. The ever-increasing threat of hardware Trojan attacks against integrated
circuits has spurred a need for accurate and efficient detection methods. Ring
oscillator network (RON) is used to detect the Trojan by capturing the
difference in power consumption; the power consumption of a Trojan-free circuit
is different from the Trojan-inserted circuit. However, the process variation
and measurement noise are the major obstacles to detect hardware Trojan with
high accuracy. In this paper, we quantitatively compare four supervised machine
learning algorithms and classifier optimization strategies for maximizing
accuracy and minimizing the false positive rate (FPR). These supervised
learning techniques show an improved false positive rate compared to principal
component analysis (PCA) and convex hull classification by nearly 40% while
maintaining > 90\% binary classification accuracy.
Estimating multi-class dynamic origin-destination demand through a forward-backward algorithm on computational graphs
Authors: Wei Ma, Xidong Pi, Sean Qian
Date: 2019-03-12 00:00:00

Transportation networks are unprecedentedly complex with heterogeneous
vehicular flow. Conventionally, vehicle classes are considered by vehicle
classifications (such as standard passenger cars and trucks). However, vehicle
flow heterogeneity stems from many other aspects in general, e.g.,
ride-sourcing vehicles versus personal vehicles, human driven vehicles versus
connected and automated vehicles. Provided with some observations of vehicular
flow for each class in a large-scale transportation network, how to estimate
the multi-class spatio-temporal vehicular flow, in terms of time-varying
Origin-Destination (OD) demand and path/link flow, remains a big challenge.
This paper presents a solution framework for multi-class dynamic OD demand
estimation (MCDODE) in large-scale networks. The proposed framework is built on
a computational graph with tensor representations of spatio-temporal flow and
all intermediate features involved in the MCDODE formulation. A
forward-backward algorithm is proposed to efficiently solve the MCDODE
formulation on computational graphs. In addition, we propose a novel concept of
tree-based cumulative curves to estimate the gradient of OD demand. A Growing
Tree algorithm is developed to construct tree-based cumulative curves. The
proposed framework is examined on a small network as well as a real-world
large-scale network. The experiment results indicate that the proposed
framework is compelling, satisfactory and computationally plausible.
Deep Learning for Automated Medical Image Analysis
Authors: Wentao Zhu
Date: 2019-03-12 00:00:00
Tasks: Anatomy, Deep Learning, Lung Nodule Detection, Medical Image Analysis

Medical imaging is an essential tool in many areas of medical applications,
used for both diagnosis and treatment. However, reading medical images and
making diagnosis or treatment recommendations require specially trained medical
specialists. The current practice of reading medical images is labor-intensive,
time-consuming, costly, and error-prone. It would be more desirable to have a
computer-aided system that can automatically make diagnosis and treatment
recommendations. Recent advances in deep learning enable us to rethink the ways
of clinician diagnosis based on medical images. In this thesis, we will
introduce 1) mammograms for detecting breast cancers, the most frequently
diagnosed solid cancer for U.S. women, 2) lung CT images for detecting lung
cancers, the most frequently diagnosed malignant cancer, and 3) head and neck
CT images for automated delineation of organs at risk in radiotherapy. First,
we will show how to employ the adversarial concept to generate the hard
examples improving mammogram mass segmentation. Second, we will demonstrate how
to use the weakly labeled data for the mammogram breast cancer diagnosis by
efficiently design deep learning for multi-instance learning. Third, the thesis
will walk through DeepLung system which combines deep 3D ConvNets and GBM for
automated lung nodule detection and classification. Fourth, we will show how to
use weakly labeled data to improve existing lung nodule detection system by
integrating deep learning with a probabilistic graphic model. Lastly, we will
demonstrate the AnatomyNet which is thousands of times faster and more accurate
than previous methods on automated anatomy segmentation.
Progressive Generative Adversarial Binary Networks for Music Generation
Authors: Manan Oza, Himanshu Vaghela, Kriti Srivastava
Date: 2019-03-12 00:00:00
Tasks: Generative Adversarial Network, Music Generation

Recent improvements in generative adversarial network (GAN) training
techniques prove that progressively training a GAN drastically stabilizes the
training and improves the quality of outputs produced. Adding layers after the
previous ones have converged has proven to help in better overall convergence
and stability of the model as well as reducing the training time by a
sufficient amount. Thus we use this training technique to train the model
progressively in the time and pitch domain i.e. starting from a very small time
value and pitch range we gradually expand the matrix sizes until the end result
is a completely trained model giving outputs having tensor sizes [4 (bar) x 96
(time steps) x 84 (pitch values) x 8 (tracks)]. As proven in previously
proposed models deterministic binary neurons also help in improving the
results. Thus we make use of a layer of deterministic binary neurons at the end
of the generator to get binary valued outputs instead of fractional values
existing between 0 and 1.
Elements of Sequential Monte Carlo
Authors: Christian A. Naesseth, Fredrik Lindsten, Thomas B. Schön
Date: 2019-03-12 00:00:00
Tasks: Bayesian Inference, BIG-bench Machine Learning, Variational Inference

A core problem in statistics and probabilistic machine learning is to compute probability distributions and expectations. This is the fundamental problem of Bayesian statistics and machine learning, which frames all inference as expectations with respect to the posterior distribution. The key challenge is to approximate these intractable expectations. In this tutorial, we review sequential Monte Carlo (SMC), a random-sampling-based class of methods for approximate inference. First, we explain the basics of SMC, discuss practical issues, and review theoretical results. We then examine two of the main user design choices: the proposal distributions and the so called intermediate target distributions. We review recent results on how variational inference and amortization can be used to learn efficient proposals and target distributions. Next, we discuss the SMC estimate of the normalizing constant, how this can be used for pseudo-marginal inference and inference evaluation. Throughout the tutorial we illustrate the use of SMC on various models commonly used in machine learning, such as stochastic recurrent neural networks, probabilistic graphical models, and probabilistic programs.
Testing Conditional Independence on Discrete Data using Stochastic Complexity
Authors: Alexander Marx, Jilles Vreeken
Date: 2019-03-12 00:00:00
Tasks: Causal Discovery

Testing for conditional independence is a core aspect of constraint-based
causal discovery. Although commonly used tests are perfect in theory, they
often fail to reject independence in practice, especially when conditioning on
multiple variables.
  We focus on discrete data and propose a new test based on the notion of
algorithmic independence that we instantiate using stochastic complexity.
Amongst others, we show that our proposed test, SCI, is an asymptotically
unbiased as well as $L_2$ consistent estimator for conditional mutual
information (CMI). Further, we show that SCI can be reformulated to find a
sensible threshold for CMI that works well on limited samples. Empirical
evaluation shows that SCI has a lower type II error than commonly used tests.
As a result, we obtain a higher recall when we use SCI in causal discovery
algorithms, without compromising the precision.
Few-Shot and Zero-Shot Learning for Historical Text Normalization
Authors: Marcel Bollmann, Natalia Korchagina, Anders Søgaard
Date: 2019-03-12 00:00:00
Proceeding: WS 2019 11
Tasks: Lemmatization, Multi-Task Learning, Text Normalization, Zero-Shot Learning

Historical text normalization often relies on small training datasets. Recent work has shown that multi-task learning can lead to significant improvements by exploiting synergies with related datasets, but there has been no systematic study of different multi-task learning architectures. This paper evaluates 63~multi-task learning configurations for sequence-to-sequence-based historical text normalization across ten datasets from eight languages, using autoencoding, grapheme-to-phoneme mapping, and lemmatization as auxiliary tasks. We observe consistent, significant improvements across languages when training data for the target task is limited, but minimal or no improvements when training data is abundant. We also show that zero-shot learning outperforms the simple, but relatively strong, identity baseline.