pwc-archive/papers-with-abstracts huggingface dataset:
Robust Sparse Reduced Rank Regression in High Dimensions
Authors: Kean Ming Tan, Qiang Sun, Daniela Witten
Date: 2018-10-18 00:00:00
Tasks: regression, Vocal Bursts Intensity Prediction

We propose robust sparse reduced rank regression for analyzing large and
complex high-dimensional data with heavy-tailed random noise. The proposed
method is based on a convex relaxation of a rank- and sparsity-constrained
non-convex optimization problem, which is then solved using the alternating
direction method of multipliers algorithm. We establish non-asymptotic
estimation error bounds under both Frobenius and nuclear norms in the
high-dimensional setting. This is a major contribution over existing results in
reduced rank regression, which mainly focus on rank selection and prediction
consistency. Our theoretical results quantify the tradeoff between
heavy-tailedness of the random noise and statistical bias. For random noise
with bounded $(1+\delta)$th moment with $\delta \in (0,1)$, the rate of
convergence is a function of $\delta$, and is slower than the sub-Gaussian-type
deviation bounds; for random noise with bounded second moment, we obtain a rate
of convergence as if sub-Gaussian noise were assumed. Furthermore, the
transition between the two regimes is smooth. We illustrate the performance of
the proposed method via extensive numerical studies and a data application.
Domain Adaptation for Semantic Segmentation via Class-Balanced Self-Training
Authors: Yang Zou, Zhiding Yu, B. V. K. Vijaya Kumar, Jinsong Wang
Date: 2018-10-18 00:00:00
Tasks: Domain Adaptation, Pseudo Label, Semantic Segmentation, Synthetic-to-Real Translation, Unsupervised Domain Adaptation

Recent deep networks achieved state of the art performance on a variety of
semantic segmentation tasks. Despite such progress, these models often face
challenges in real world `wild tasks' where large difference between labeled
training/source data and unseen test/target data exists. In particular, such
difference is often referred to as `domain gap', and could cause significantly
decreased performance which cannot be easily remedied by further increasing the
representation power. Unsupervised domain adaptation (UDA) seeks to overcome
such problem without target domain labels. In this paper, we propose a novel
UDA framework based on an iterative self-training procedure, where the problem
is formulated as latent variable loss minimization, and can be solved by
alternatively generating pseudo labels on target data and re-training the model
with these labels. On top of self-training, we also propose a novel
class-balanced self-training framework to avoid the gradual dominance of large
classes on pseudo-label generation, and introduce spatial priors to refine
generated labels. Comprehensive experiments show that the proposed methods
achieve state of the art semantic segmentation performance under multiple major
UDA settings.
Decoupling Semantic Context and Color Correlation with multi-class cross branch regularization
Authors: Vishal Keshav, Tejpratap G. V. S. L
Date: 2018-10-18 00:00:00
Tasks: Color Constancy

This paper presents a novel design methodology for architecting a
light-weight and faster DNN architecture for vision applications. The
effectiveness of the architecture is demonstrated on Color-Constancy use case
an inherent block in camera and imaging pipelines. Specifically, we present a
multi-branch architecture that disassembles the contextual features and color
properties from an image, and later combines them to predict a global property
(e.g. Global Illumination). We also propose an implicit regularization
technique by designing cross-branch regularization block that enables the
network to retain high generalization accuracy. With a conservative use of best
computational operators, the proposed architecture achieves state-of-the-art
accuracy with 30X lesser model parameters and 70X faster inference time for
color constancy. It is also shown that the proposed architecture is generic and
achieves similar efficiency in other vision applications such as Low-Light
photography.
Policy Gradient in Partially Observable Environments: Approximation and Convergence
Authors: Kamyar Azizzadenesheli, Yisong Yue, Animashree Anandkumar
Date: 2018-10-18 00:00:00
Tasks: Decision Making, Policy Gradient Methods, Reinforcement Learning

Policy gradient is a generic and flexible reinforcement learning approach that generally enjoys simplicity in analysis, implementation, and deployment. In the last few decades, this approach has been extensively advanced for fully observable environments. In this paper, we generalize a variety of these advances to partially observable settings, and similar to the fully observable case, we keep our focus on the class of Markovian policies. We propose a series of technical tools, including a novel notion of advantage function, to develop policy gradient algorithms and study their convergence properties in such environments. Deploying these tools, we generalize a variety of existing theoretical guarantees, such as policy gradient and convergence theorems, to partially observable domains, those which also could be carried to more settings of interest. This study also sheds light on the understanding of policy gradient approaches in real-world applications which tend to be partially observable.
Automatic Brain Tumor Segmentation using Convolutional Neural Networks with Test-Time Augmentation
Authors: Guotai Wang, Wenqi Li, Sebastien Ourselin, Tom Vercauteren
Date: 2018-10-18 00:00:00
Tasks: Brain Tumor Segmentation, Data Augmentation, Segmentation, Small Data Image Classification, Tumor Segmentation

Automatic brain tumor segmentation plays an important role for diagnosis,
surgical planning and treatment assessment of brain tumors. Deep convolutional
neural networks (CNNs) have been widely used for this task. Due to the
relatively small data set for training, data augmentation at training time has
been commonly used for better performance of CNNs. Recent works also
demonstrated the usefulness of using augmentation at test time, in addition to
training time, for achieving more robust predictions. We investigate how
test-time augmentation can improve CNNs' performance for brain tumor
segmentation. We used different underpinning network structures and augmented
the image by 3D rotation, flipping, scaling and adding random noise at both
training and test time. Experiments with BraTS 2018 training and validation set
show that test-time augmentation helps to improve the brain tumor segmentation
accuracy and obtain uncertainty estimation of the segmentation results.
A Self-Organizing Tensor Architecture for Multi-View Clustering
Authors: Lifang He, Chun-Ta Lu, Yong Chen, Jiawei Zhang, Linlin Shen, Philip S. Yu, Fei Wang
Date: 2018-10-18 00:00:00
Tasks: Clustering

In many real-world applications, data are often unlabeled and comprised of
different representations/views which often provide information complementary
to each other. Although several multi-view clustering methods have been
proposed, most of them routinely assume one weight for one view of features,
and thus inter-view correlations are only considered at the view-level. These
approaches, however, fail to explore the explicit correlations between features
across multiple views. In this paper, we introduce a tensor-based approach to
incorporate the higher-order interactions among multiple views as a tensor
structure. Specifically, we propose a multi-linear multi-view clustering (MMC)
method that can efficiently explore the full-order structural information among
all views and reveal the underlying subspace structure embedded within the
tensor. Extensive experiments on real-world datasets demonstrate that our
proposed MMC algorithm clearly outperforms other related state-of-the-art
methods.
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey
Authors: Nguyen Cong Luong, Dinh Thai Hoang, Shimin Gong, Dusit Niyato, Ping Wang, Ying-Chang Liang, Dong In Kim
Date: 2018-10-18 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

This paper presents a comprehensive literature review on applications of deep
reinforcement learning in communications and networking. Modern networks, e.g.,
Internet of Things (IoT) and Unmanned Aerial Vehicle (UAV) networks, become
more decentralized and autonomous. In such networks, network entities need to
make decisions locally to maximize the network performance under uncertainty of
network environment. Reinforcement learning has been efficiently used to enable
the network entities to obtain the optimal policy including, e.g., decisions or
actions, given their states when the state and action spaces are small.
However, in complex and large-scale networks, the state and action spaces are
usually large, and the reinforcement learning may not be able to find the
optimal policy in reasonable time. Therefore, deep reinforcement learning, a
combination of reinforcement learning with deep learning, has been developed to
overcome the shortcomings. In this survey, we first give a tutorial of deep
reinforcement learning from fundamental concepts to advanced models. Then, we
review deep reinforcement learning approaches proposed to address emerging
issues in communications and networking. The issues include dynamic network
access, data rate control, wireless caching, data offloading, network security,
and connectivity preservation which are all important to next generation
networks such as 5G and beyond. Furthermore, we present applications of deep
reinforcement learning for traffic routing, resource sharing, and data
collection. Finally, we highlight important challenges, open issues, and future
research directions of applying deep reinforcement learning.
Distributed $k$-Clustering for Data with Heavy Noise
Authors: Xiangyu Guo, Shi Li
Date: 2018-10-18 00:00:00
Proceeding: NeurIPS 2018
Tasks: Clustering

In this paper, we consider the $k$-center/median/means clustering with
outliers problems (or the $(k, z)$-center/median/means problems) in the
distributed setting. Most previous distributed algorithms have their
communication costs linearly depending on $z$, the number of outliers. Recently
Guha et al. overcame this dependence issue by considering bi-criteria
approximation algorithms that output solutions with $2z$ outliers. For the case
where $z$ is large, the extra $z$ outliers discarded by the algorithms might be
too large, considering that the data gathering process might be costly. In this
paper, we improve the number of outliers to the best possible $(1+\epsilon)z$,
while maintaining the $O(1)$-approximation ratio and independence of
communication cost on $z$. The problems we consider include the $(k, z)$-center
problem, and $(k, z)$-median/means problems in Euclidean metrics.
Implementation of the our algorithm for $(k, z)$-center shows that it
outperforms many previous algorithms, both in terms of the communication cost
and quality of the output solution.
On Statistical Learning of Simplices: Unmixing Problem Revisited
Authors: Amir Najafi, Saeed Ilchi, Amir H. Saberi, Seyed Abolfazl Motahari, Babak H. Khalaj, Hamid R. Rabiee
Date: 2018-10-18 00:00:00

We study the sample complexity of learning a high-dimensional simplex from a set of points uniformly sampled from its interior. Learning of simplices is a long studied problem in computer science and has applications in computational biology and remote sensing, mostly under the name of `spectral unmixing'. We theoretically show that a sufficient sample complexity for reliable learning of a $K$-dimensional simplex up to a total-variation error of $\epsilon$ is $O\left(\frac{K^2}{\epsilon}\log\frac{K}{\epsilon}\right)$, which yields a substantial improvement over existing bounds. Based on our new theoretical framework, we also propose a heuristic approach for the inference of simplices. Experimental results on synthetic and real-world datasets demonstrate a comparable performance for our method on noiseless samples, while we outperform the state-of-the-art in noisy cases.
A Novel Focal Tversky loss function with improved Attention U-Net for lesion segmentation
Authors: Nabila Abraham, Naimul Mefraz Khan
Date: 2018-10-18 00:00:00
Tasks: Image Segmentation, Lesion Segmentation, Medical Image Segmentation, Segmentation, Semantic Segmentation, Skin Lesion Segmentation

We propose a generalized focal loss function based on the Tversky index to
address the issue of data imbalance in medical image segmentation. Compared to
the commonly used Dice loss, our loss function achieves a better trade off
between precision and recall when training on small structures such as lesions.
To evaluate our loss function, we improve the attention U-Net model by
incorporating an image pyramid to preserve contextual features. We experiment
on the BUS 2017 dataset and ISIC 2018 dataset where lesions occupy 4.84% and
21.4% of the images area and improve segmentation accuracy when compared to the
standard U-Net by 25.7% and 3.6%, respectively.
LadderNet: Multi-path networks based on U-Net for medical image segmentation
Authors: Juntang Zhuang
Date: 2018-10-17 00:00:00
Tasks: Decoder, Image Segmentation, Medical Image Segmentation, Retinal Vessel Segmentation, Segmentation, Semantic Segmentation

U-Net has been providing state-of-the-art performance in many medical image segmentation problems. Many modifications have been proposed for U-Net, such as attention U-Net, recurrent residual convolutional U-Net (R2-UNet), and U-Net with residual blocks or blocks with dense connections. However, all these modifications have an encoder-decoder structure with skip connections, and the number of paths for information flow is limited. We propose LadderNet in this paper, which can be viewed as a chain of multiple U-Nets. Instead of only one pair of encoder branch and decoder branch in U-Net, a LadderNet has multiple pairs of encoder-decoder branches, and has skip connections between every pair of adjacent decoder and decoder branches in each level. Inspired by the success of ResNet and R2-UNet, we use modified residual blocks where two convolutional layers in one block share the same weights. A LadderNet has more paths for information flow because of skip connections and residual blocks, and can be viewed as an ensemble of Fully Convolutional Networks (FCN). The equivalence to an ensemble of FCNs improves segmentation accuracy, while the shared weights within each residual block reduce parameter number. Semantic segmentation is essential for retinal disease detection. We tested LadderNet on two benchmark datasets for blood vessel segmentation in retinal images, and achieved superior performance over methods in the literature. The implementation is provided \url{https://github.com/juntang-zhuang/LadderNet}
The Wasserstein transform
Authors: Facundo Mémoli, Zane Smith, Zhengchao Wan
Date: 2018-10-17 00:00:00
Tasks: Denoising

We introduce the Wasserstein transform, a method for enhancing and denoising
datasets defined on general metric spaces. The construction draws inspiration
from Optimal Transportation ideas. We establish precise connections with the
mean shift family of algorithms and establish the stability of both our method
and mean shift under data perturbation.
Multi-Agent Fully Decentralized Value Function Learning with Linear Convergence Rates
Authors: Lucas Cassano, Kun Yuan, Ali H. Sayed
Date: 2018-10-17 00:00:00
Tasks: Efficient Exploration

This work develops a fully decentralized multi-agent algorithm for policy evaluation. The proposed scheme can be applied to two distinct scenarios. In the first scenario, a collection of agents have distinct datasets gathered following different behavior policies (none of which is required to explore the full state space) in different instances of the same environment and they all collaborate to evaluate a common target policy. The network approach allows for efficient exploration of the state space and allows all agents to converge to the optimal solution even in situations where neither agent can converge on its own without cooperation. The second scenario is that of multi-agent games, in which the state is global and rewards are local. In this scenario, agents collaborate to estimate the value function of a target team policy. The proposed algorithm combines off-policy learning, eligibility traces and linear function approximation. The proposed algorithm is of the variance-reduced kind and achieves linear convergence with $O(1)$ memory requirements. The linear convergence of the algorithm is established analytically, and simulations are used to illustrate the effectiveness of the method.
A Periodicity-based Parallel Time Series Prediction Algorithm in Cloud Computing Environments
Authors: Jianguo Chen, Kenli Li, Huigui Rong, Kashif Bilal, Keqin Li, Philip S. Yu
Date: 2018-10-17 00:00:00
Tasks: Cloud Computing, Data Compression, Distributed Computing, Prediction, Time Series, Time Series Analysis, Time Series Prediction

In the era of big data, practical applications in various domains continually
generate large-scale time-series data. Among them, some data show significant
or potential periodicity characteristics, such as meteorological and financial
data. It is critical to efficiently identify the potential periodic patterns
from massive time-series data and provide accurate predictions. In this paper,
a Periodicity-based Parallel Time Series Prediction (PPTSP) algorithm for
large-scale time-series data is proposed and implemented in the Apache Spark
cloud computing environment. To effectively handle the massive historical
datasets, a Time Series Data Compression and Abstraction (TSDCA) algorithm is
presented, which can reduce the data scale as well as accurately extracting the
characteristics. Based on this, we propose a Multi-layer Time Series Periodic
Pattern Recognition (MTSPPR) algorithm using the Fourier Spectrum Analysis
(FSA) method. In addition, a Periodicity-based Time Series Prediction (PTSP)
algorithm is proposed. Data in the subsequent period are predicted based on all
previous period models, in which a time attenuation factor is introduced to
control the impact of different periods on the prediction results. Moreover, to
improve the performance of the proposed algorithms, we propose a parallel
solution on the Apache Spark platform, using the Streaming real-time computing
module. To efficiently process the large-scale time-series datasets in
distributed computing environments, Distributed Streams (DStreams) and
Resilient Distributed Datasets (RDDs) are used to store and calculate these
datasets. Extensive experimental results show that our PPTSP algorithm has
significant advantages compared with other algorithms in terms of prediction
accuracy and performance.
Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity
Authors: Chulhee Yun, Suvrit Sra, Ali Jadbabaie
Date: 2018-10-17 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Memorization

We study finite sample expressivity, i.e., memorization power of ReLU networks. Recent results require $N$ hidden nodes to memorize/interpolate arbitrary $N$ data points. In contrast, by exploiting depth, we show that 3-layer ReLU networks with $\Omega(\sqrt{N})$ hidden nodes can perfectly memorize most datasets with $N$ points. We also prove that width $\Theta(\sqrt{N})$ is necessary and sufficient for memorizing $N$ data points, proving tight bounds on memorization capacity. The sufficiency result can be extended to deeper networks; we show that an $L$-layer network with $W$ parameters in the hidden layers can memorize $N$ data points if $W = \Omega(N)$. Combined with a recent upper bound $O(WL\log W)$ on VC dimension, our construction is nearly tight for any fixed $L$. Subsequently, we analyze memorization capacity of residual networks under a general position assumption; we prove results that substantially reduce the known requirement of $N$ hidden nodes. Finally, we study the dynamics of stochastic gradient descent (SGD), and show that when initialized near a memorizing global minimum of the empirical risk, SGD quickly finds a nearby point with much smaller empirical risk.
Distributed Learning over Unreliable Networks
Authors: Chen Yu, Hanlin Tang, Cedric Renggli, Simon Kassing, Ankit Singla, Dan Alistarh, Ce Zhang, Ji Liu
Date: 2018-10-17 00:00:00
Tasks: BIG-bench Machine Learning

Most of today's distributed machine learning systems assume {\em reliable networks}: whenever two machines exchange information (e.g., gradients or models), the network should guarantee the delivery of the message. At the same time, recent work exhibits the impressive tolerance of machine learning algorithms to errors or noise arising from relaxed communication or synchronization. In this paper, we connect these two trends, and consider the following question: {\em Can we design machine learning systems that are tolerant to network unreliability during training?} With this motivation, we focus on a theoretical problem of independent interest---given a standard distributed parameter server architecture, if every communication between the worker and the server has a non-zero probability $p$ of being dropped, does there exist an algorithm that still converges, and at what speed? The technical contribution of this paper is a novel theoretical analysis proving that distributed learning over unreliable network can achieve comparable convergence rate to centralized or distributed learning over reliable networks. Further, we prove that the influence of the packet drop rate diminishes with the growth of the number of \textcolor{black}{parameter servers}. We map this theoretical result onto a real-world scenario, training deep neural networks over an unreliable network layer, and conduct network simulation to validate the system improvement by allowing the networks to be unreliable.
A Disease Diagnosis and Treatment Recommendation System Based on Big Data Mining and Cloud Computing
Authors: Jianguo Chen, Kenli Li, Huigui Rong, Kashif Bilal, Nan Yang, Keqin Li
Date: 2018-10-17 00:00:00
Tasks: Cloud Computing, Clustering

It is crucial to provide compatible treatment schemes for a disease according
to various symptoms at different stages. However, most classification methods
might be ineffective in accurately classifying a disease that holds the
characteristics of multiple treatment stages, various symptoms, and
multi-pathogenesis. Moreover, there are limited exchanges and cooperative
actions in disease diagnoses and treatments between different departments and
hospitals. Thus, when new diseases occur with atypical symptoms, inexperienced
doctors might have difficulty in identifying them promptly and accurately.
Therefore, to maximize the utilization of the advanced medical technology of
developed hospitals and the rich medical knowledge of experienced doctors, a
Disease Diagnosis and Treatment Recommendation System (DDTRS) is proposed in
this paper. First, to effectively identify disease symptoms more accurately, a
Density-Peaked Clustering Analysis (DPCA) algorithm is introduced for
disease-symptom clustering. In addition, association analyses on
Disease-Diagnosis (D-D) rules and Disease-Treatment (D-T) rules are conducted
by the Apriori algorithm separately. The appropriate diagnosis and treatment
schemes are recommended for patients and inexperienced doctors, even if they
are in a limited therapeutic environment. Moreover, to reach the goals of high
performance and low latency response, we implement a parallel solution for
DDTRS using the Apache Spark cloud platform. Extensive experimental results
demonstrate that the proposed DDTRS realizes disease-symptom clustering
effectively and derives disease treatment recommendations intelligently and
accurately.