pwc-archive/papers-with-abstracts huggingface dataset:
Deep Learning for IoT Big Data and Streaming Analytics: A Survey
Authors: Mehdi Mohammadi, Ala Al-Fuqaha, Sameh Sorour, Mohsen Guizani
Date: 2017-12-09 00:00:00

In the era of the Internet of Things (IoT), an enormous amount of sensing
devices collect and/or generate various sensory data over time for a wide range
of fields and applications. Based on the nature of the application, these
devices will result in big or fast/real-time data streams. Applying analytics
over such data streams to discover new information, predict future insights,
and make control decisions is a crucial process that makes IoT a worthy
paradigm for businesses and a quality-of-life improving technology. In this
paper, we provide a thorough overview on using a class of advanced machine
learning techniques, namely Deep Learning (DL), to facilitate the analytics and
learning in the IoT domain. We start by articulating IoT data characteristics
and identifying two major treatments for IoT data from a machine learning
perspective, namely IoT big data analytics and IoT streaming data analytics. We
also discuss why DL is a promising approach to achieve the desired analytics in
these types of data and applications. The potential of using emerging DL
techniques for IoT data analytics are then discussed, and its promises and
challenges are introduced. We present a comprehensive background on different
DL architectures and algorithms. We also analyze and summarize major reported
research attempts that leveraged DL in the IoT domain. The smart IoT devices
that have incorporated DL in their intelligence background are also discussed.
DL implementation approaches on the fog and cloud centers in support of IoT
applications are also surveyed. Finally, we shed light on some challenges and
potential directions for future research. At the end of each section, we
highlight the lessons learned based on our experiments and review of the recent
literature.
Neural Adversarial Training for Semi-supervised Japanese Predicate-argument Structure Analysis
Authors: Shuhei Kurita, Daisuke Kawahara, Sadao Kurohashi
Date: 2018-06-04 00:00:00
Proceeding: ACL 2018 7

Japanese predicate-argument structure (PAS) analysis involves zero anaphora
resolution, which is notoriously difficult. To improve the performance of
Japanese PAS analysis, it is straightforward to increase the size of corpora
annotated with PAS. However, since it is prohibitively expensive, it is
promising to take advantage of a large amount of raw corpora. In this paper, we
propose a novel Japanese PAS analysis model based on semi-supervised
adversarial training with a raw corpus. In our experiments, our model
outperforms existing state-of-the-art models for Japanese PAS analysis.
Semantic-Aware Generative Adversarial Nets for Unsupervised Domain Adaptation in Chest X-ray Segmentation
Authors: Cheng Chen, Qi Dou, Hao Chen, Pheng-Ann Heng
Date: 2018-06-02 00:00:00
Tasks: Domain Adaptation, Segmentation, Transfer Learning, Unsupervised Domain Adaptation

In spite of the compelling achievements that deep neural networks (DNNs) have
made in medical image computing, these deep models often suffer from degraded
performance when being applied to new test datasets with domain shift. In this
paper, we present a novel unsupervised domain adaptation approach for
segmentation tasks by designing semantic-aware generative adversarial networks
(GANs). Specifically, we transform the test image into the appearance of source
domain, with the semantic structural information being well preserved, which is
achieved by imposing a nested adversarial learning in semantic label space. In
this way, the segmentation DNN learned from the source domain is able to be
directly generalized to the transformed test image, eliminating the need of
training a new model for every new target dataset. Our domain adaptation
procedure is unsupervised, without using any target domain labels. The
adversarial learning of our network is guided by a GAN loss for mapping data
distributions, a cycle-consistency loss for retaining pixel-level content, and
a semantic-aware loss for enhancing structural information. We validated our
method on two different chest X-ray public datasets for left/right lung
segmentation. Experimental results show that the segmentation performance of
our unsupervised approach is highly competitive with the upper bound of
supervised transfer learning.
Understanding Regularized Spectral Clustering via Graph Conductance
Authors: Yilin Zhang, Karl Rohe
Date: 2018-06-05 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Clustering

This paper uses the relationship between graph conductance and spectral
clustering to study (i) the failures of spectral clustering and (ii) the
benefits of regularization. The explanation is simple. Sparse and stochastic
graphs create a lot of small trees that are connected to the core of the graph
by only one edge. Graph conductance is sensitive to these noisy `dangling
sets'. Spectral clustering inherits this sensitivity. The second part of the
paper starts from a previously proposed form of regularized spectral clustering
and shows that it is related to the graph conductance on a `regularized graph'.
We call the conductance on the regularized graph CoreCut. Based upon previous
arguments that relate graph conductance to spectral clustering (e.g. Cheeger
inequality), minimizing CoreCut relaxes to regularized spectral clustering.
Simple inspection of CoreCut reveals why it is less sensitive to small cuts in
the graph. Together, these results show that unbalanced partitions from
spectral clustering can be understood as overfitting to noise in the periphery
of a sparse and stochastic graph. Regularization fixes this overfitting. In
addition to this statistical benefit, these results also demonstrate how
regularization can improve the computational speed of spectral clustering. We
provide simulations and data examples to illustrate these results.
Graph-based regularization for regression problems with alignment and highly-correlated designs
Authors: Yuan Li, Benjamin Mark, Garvesh Raskutti, Rebecca Willett, Hyebin Song, David Neiman
Date: 2018-03-20 00:00:00
Tasks: Model Selection, regression

Sparse models for high-dimensional linear regression and machine learning have received substantial attention over the past two decades. Model selection, or determining which features or covariates are the best explanatory variables, is critical to the interpretability of a learned model. Much of the current literature assumes that covariates are only mildly correlated. However, in many modern applications covariates are highly correlated and do not exhibit key properties (such as the restricted eigenvalue condition, restricted isometry property, or other related assumptions). This work considers a high-dimensional regression setting in which a graph governs both correlations among the covariates and the similarity among regression coefficients -- meaning there is \emph{alignment} between the covariates and regression coefficients. Using side information about the strength of correlations among features, we form a graph with edge weights corresponding to pairwise covariances. This graph is used to define a graph total variation regularizer that promotes similar weights for correlated features. This work shows how the proposed graph-based regularization yields mean-squared error guarantees for a broad range of covariance graph structures. These guarantees are optimal for many specific covariance graphs, including block and lattice graphs. Our proposed approach outperforms other methods for highly-correlated design in a variety of experiments on synthetic data and real biochemistry data.
Active Feature Acquisition with Supervised Matrix Completion
Authors: Sheng-Jun Huang, Miao Xu, Ming-Kun Xie, Masashi Sugiyama, Gang Niu, Songcan Chen
Date: 2018-02-15 00:00:00
Tasks: Matrix Completion

Feature missing is a serious problem in many applications, which may lead to
low quality of training data and further significantly degrade the learning
performance. While feature acquisition usually involves special devices or
complex process, it is expensive to acquire all feature values for the whole
dataset. On the other hand, features may be correlated with each other, and
some values may be recovered from the others. It is thus important to decide
which features are most informative for recovering the other features as well
as improving the learning performance. In this paper, we try to train an
effective classification model with least acquisition cost by jointly
performing active feature querying and supervised matrix completion. When
completing the feature matrix, a novel target function is proposed to
simultaneously minimize the reconstruction error on observed entries and the
supervised loss on training data. When querying the feature value, the most
uncertain entry is actively selected based on the variance of previous
iterations. In addition, a bi-objective optimization method is presented for
cost-aware active selection when features bear different acquisition costs. The
effectiveness of the proposed approach is well validated by both theoretical
analysis and experimental study.
An Information-Theoretic View for Deep Learning
Authors: Jingwei Zhang, Tongliang Liu, DaCheng Tao
Date: 2018-04-24 00:00:00
Tasks: Deep Learning, speech-recognition, Speech Recognition

Deep learning has transformed computer vision, natural language processing,
and speech recognition\cite{badrinarayanan2017segnet, dong2016image,
ren2017faster, ji20133d}. However, two critical questions remain obscure: (1)
why do deep neural networks generalize better than shallow networks; and (2)
does it always hold that a deeper network leads to better performance?
Specifically, letting $L$ be the number of convolutional and pooling layers in
a deep neural network, and $n$ be the size of the training sample, we derive an
upper bound on the expected generalization error for this network, i.e.,
  \begin{eqnarray*}
  \mathbb{E}[R(W)-R_S(W)] \leq
\exp{\left(-\frac{L}{2}\log{\frac{1}{\eta}}\right)}\sqrt{\frac{2\sigma^2}{n}I(S,W)
}
  \end{eqnarray*} where $\sigma >0$ is a constant depending on the loss
function, $0<\eta<1$ is a constant depending on the information loss for each
convolutional or pooling layer, and $I(S, W)$ is the mutual information between
the training sample $S$ and the output hypothesis $W$. This upper bound shows
that as the number of convolutional and pooling layers $L$ increases in the
network, the expected generalization error will decrease exponentially to zero.
Layers with strict information loss, such as the convolutional layers, reduce
the generalization error for the whole network; this answers the first
question. However, algorithms with zero expected generalization error does not
imply a small test error or $\mathbb{E}[R(W)]$. This is because
$\mathbb{E}[R_S(W)]$ is large when the information for fitting the data is lost
as the number of layers increases. This suggests that the claim `the deeper the
better' is conditioned on a small training error or $\mathbb{E}[R_S(W)]$.
Finally, we show that deep learning satisfies a weak notion of stability and
the sample complexity of deep neural networks will decrease as $L$ increases.
EigenNetworks
Authors: Jonathan Mei, José M. F. Moura
Date: 2018-06-05 00:00:00
Tasks: Descriptive, Time Series, Time Series Analysis

Many applications donot have the benefit of the laws of physics to derive
succinct descriptive models for observed data. In alternative,
interdependencies among $N$ time series $\{ x_{nk}, k>0 \}_{n=1}^{N}$ are
nowadays often captured by a graph or network $G$ that in practice may be very
large. The network itself may change over time as well (i.e., as $G_k$).
Tracking brute force the changes of time varying networks presents major
challenges, including the associated computational problems. Further, a large
set of networks may not lend itself to useful analysis. This paper approximates
the time varying networks $\left\{G_k\right\}$ as weighted linear combinations
of eigennetworks. The eigennetworks are fixed building blocks that are
estimated by first learning the time series of graphs $G_k$ from the data $\{
x_{nk}, k>0 \}_{n=1}^{N}$, followed by a Principal Network Analysis procedure.
The weights of the eigennetwork representation are eigenfeatures and the time
varying networks $\left\{G_k\right\}$ describe a trajectory in eigennetwork
space. These eigentrajectories should be smooth since the networks $G_k$ vary
at a much slower rate than the data $x_{nk}$, except when structural network
shifts occur reflecting potentially an abrupt change in the underlying
application and sources of the data. Algorithms for learning the time series of
graphs $\left\{G_k\right\}$, deriving the eigennetworks, eigenfeatures and
eigentrajectories, and detecting changepoints are presented. Experiments on
simulated data and with two real time series data (a voting record of the US
senate and genetic expression data for the \textit{Drosophila Melanogaster} as
it goes through its life cycle) demonstrate the performance of the learning and
provide interesting interpretations of the eigennetworks.
Embedding Logical Queries on Knowledge Graphs
Authors: William L. Hamilton, Payal Bajaj, Marinka Zitnik, Dan Jurafsky, Jure Leskovec
Date: 2018-06-05 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Complex Query Answering, Knowledge Graphs

Learning low-dimensional embeddings of knowledge graphs is a powerful approach used to predict unobserved or missing edges between entities. However, an open challenge in this area is developing techniques that can go beyond simple edge prediction and handle more complex logical queries, which might involve multiple unobserved edges, entities, and variables. For instance, given an incomplete biological knowledge graph, we might want to predict "em what drugs are likely to target proteins involved with both diseases X and Y?" -- a query that requires reasoning about all possible proteins that {\em might} interact with diseases X and Y. Here we introduce a framework to efficiently make predictions about conjunctive logical queries -- a flexible but tractable subset of first-order logic -- on incomplete knowledge graphs. In our approach, we embed graph nodes in a low-dimensional space and represent logical operators as learned geometric operations (e.g., translation, rotation) in this embedding space. By performing logical operations within a low-dimensional embedding space, our approach achieves a time complexity that is linear in the number of query variables, compared to the exponential complexity required by a naive enumeration-based approach. We demonstrate the utility of this framework in two application studies on real-world datasets with millions of relations: predicting logical relationships in a network of drug-gene-disease interactions and in a graph-based representation of social interactions derived from a popular web forum.
Weighted Unsupervised Learning for 3D Object Detection
Authors: Kamran Kowsari, Manal H. Alassaf
Date: 2016-02-18 00:00:00
Tasks: 3D Object Detection, Clustering, Object, object-detection, Object Detection, Position

This paper introduces a novel weighted unsupervised learning for object
detection using an RGB-D camera. This technique is feasible for detecting the
moving objects in the noisy environments that are captured by an RGB-D camera.
The main contribution of this paper is a real-time algorithm for detecting each
object using weighted clustering as a separate cluster. In a preprocessing
step, the algorithm calculates the pose 3D position X, Y, Z and RGB color of
each data point and then it calculates each data point's normal vector using
the point's neighbor. After preprocessing, our algorithm calculates k-weights
for each data point; each weight indicates membership. Resulting in clustered
objects of the scene.
Analysis of DAWNBench, a Time-to-Accuracy Machine Learning Performance Benchmark
Authors: Cody Coleman, Daniel Kang, Deepak Narayanan, Luigi Nardi, Tian Zhao, Jian Zhang, Peter Bailis, Kunle Olukotun, Chris Re, Matei Zaharia
Date: 2018-06-04 00:00:00
Tasks: Benchmarking, BIG-bench Machine Learning, GPU

Researchers have proposed hardware, software, and algorithmic optimizations to improve the computational performance of deep learning. While some of these optimizations perform the same operations faster (e.g., increasing GPU clock speed), many others modify the semantics of the training procedure (e.g., reduced precision), and can impact the final model's accuracy on unseen data. Due to a lack of standard evaluation criteria that considers these trade-offs, it is difficult to directly compare these optimizations. To address this problem, we recently introduced DAWNBench, a benchmark competition focused on end-to-end training time to achieve near-state-of-the-art accuracy on an unseen dataset---a combined metric called time-to-accuracy (TTA). In this work, we analyze the entries from DAWNBench, which received optimized submissions from multiple industrial groups, to investigate the behavior of TTA as a metric as well as trends in the best-performing entries. We show that TTA has a low coefficient of variation and that models optimized for TTA generalize nearly as well as those trained using standard methods. Additionally, even though DAWNBench entries were able to train ImageNet models in under 3 minutes, we find they still underutilize hardware capabilities such as Tensor Cores. Furthermore, we find that distributed entries can spend more than half of their time on communication. We show similar findings with entries to the MLPERF v0.5 benchmark.
Composite Marginal Likelihood Methods for Random Utility Models
Authors: Zhibing Zhao, Lirong Xia
Date: 2018-06-04 00:00:00
Proceeding: ICML 2018 7
Tasks: Computational Efficiency

We propose a novel and flexible
rank-breaking-then-composite-marginal-likelihood (RBCML) framework for learning
random utility models (RUMs), which include the Plackett-Luce model. We
characterize conditions for the objective function of RBCML to be strictly
log-concave by proving that strict log-concavity is preserved under convolution
and marginalization. We characterize necessary and sufficient conditions for
RBCML to satisfy consistency and asymptotic normality. Experiments on synthetic
data show that RBCML for Gaussian RUMs achieves better statistical efficiency
and computational efficiency than the state-of-the-art algorithm and our RBCML
for the Plackett-Luce model provides flexible tradeoffs between running time
and statistical efficiency.
BindsNET: A machine learning-oriented spiking neural networks library in Python
Authors: Hananel Hazan, Daniel J. Saunders, Hassaan Khan, Darpan T. Sanghavi, Hava T. Siegelmann, Robert Kozma
Date: 2018-06-04 00:00:00
Tasks: BIG-bench Machine Learning, CPU, GPU, Neural Network simulation, OpenAI Gym, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

The development of spiking neural network simulation software is a critical
component enabling the modeling of neural systems and the development of
biologically inspired algorithms. Existing software frameworks support a wide
range of neural functionality, software abstraction levels, and hardware
devices, yet are typically not suitable for rapid prototyping or application to
problems in the domain of machine learning. In this paper, we describe a new
Python package for the simulation of spiking neural networks, specifically
geared towards machine learning and reinforcement learning. Our software,
called BindsNET, enables rapid building and simulation of spiking networks and
features user-friendly, concise syntax. BindsNET is built on top of the PyTorch
deep neural networks library, enabling fast CPU and GPU computation for large
spiking networks. The BindsNET framework can be adjusted to meet the needs of
other existing computing and hardware environments, e.g., TensorFlow. We also
provide an interface into the OpenAI gym library, allowing for training and
evaluation of spiking networks on reinforcement learning problems. We argue
that this package facilitates the use of spiking networks for large-scale
machine learning experimentation, and show some simple examples of how we
envision BindsNET can be used in practice. BindsNET code is available at
https://github.com/Hananel-Hazan/bindsnet
Mitigation of Policy Manipulation Attacks on Deep Q-Networks with Parameter-Space Noise
Authors: Vahid Behzadan, Arslan Munir
Date: 2018-06-04 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Recent developments have established the vulnerability of deep reinforcement
learning to policy manipulation attacks via intentionally perturbed inputs,
known as adversarial examples. In this work, we propose a technique for
mitigation of such attacks based on addition of noise to the parameter space of
deep reinforcement learners during training. We experimentally verify the
effect of parameter-space noise in reducing the transferability of adversarial
examples, and demonstrate the promising performance of this technique in
mitigating the impact of whitebox and blackbox attacks at both test and
training times.
CFCM: Segmentation via Coarse to Fine Context Memory
Authors: Fausto Milletari, Nicola Rieke, Maximilian Baust, Marco Esposito, Nassir Navab
Date: 2018-06-04 00:00:00
Tasks: Decoder, Image Segmentation, Segmentation, Semantic Segmentation

Recent neural-network-based architectures for image segmentation make
extensive usage of feature forwarding mechanisms to integrate information from
multiple scales. Although yielding good results, even deeper architectures and
alternative methods for feature fusion at different resolutions have been
scarcely investigated for medical applications. In this work we propose to
implement segmentation via an encoder-decoder architecture which differs from
any other previously published method since (i) it employs a very deep
architecture based on residual learning and (ii) combines features via a
convolutional Long Short Term Memory (LSTM), instead of concatenation or
summation. The intuition is that the memory mechanism implemented by LSTMs can
better integrate features from different scales through a coarse-to-fine
strategy; hence the name Coarse-to-Fine Context Memory (CFCM). We demonstrate
the remarkable advantages of this approach on two datasets: the Montgomery
county lung segmentation dataset, and the EndoVis 2015 challenge dataset for
surgical instrument segmentation.
FlowNet3D: Learning Scene Flow in 3D Point Clouds
Authors: Xingyu Liu, Charles R. Qi, Leonidas J. Guibas
Date: 2018-06-04 00:00:00
Proceeding: CVPR 2019 6
Tasks: Motion Segmentation

Many applications in robotics and human-computer interaction can benefit from understanding 3D motion of points in a dynamic environment, widely noted as scene flow. While most previous methods focus on stereo and RGB-D images as input, few try to estimate scene flow directly from point clouds. In this work, we propose a novel deep neural network named $FlowNet3D$ that learns scene flow from point clouds in an end-to-end fashion. Our network simultaneously learns deep hierarchical features of point clouds and flow embeddings that represent point motions, supported by two newly proposed learning layers for point sets. We evaluate the network on both challenging synthetic data from FlyingThings3D and real Lidar scans from KITTI. Trained on synthetic data only, our network successfully generalizes to real scans, outperforming various baselines and showing competitive results to the prior art. We also demonstrate two applications of our scene flow output (scan registration and motion segmentation) to show its potential wide use cases.