pwc-archive/papers-with-abstracts huggingface dataset:
Constructing Unrestricted Adversarial Examples with Generative Models
Authors: Yang Song, Rui Shu, Nate Kushman, Stefano Ermon
Date: 2018-05-21 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Generative Adversarial Network

Adversarial examples are typically constructed by perturbing an existing data
point within a small matrix norm, and current defense methods are focused on
guarding against this type of attack. In this paper, we propose unrestricted
adversarial examples, a new threat model where the attackers are not restricted
to small norm-bounded perturbations. Different from perturbation-based attacks,
we propose to synthesize unrestricted adversarial examples entirely from
scratch using conditional generative models. Specifically, we first train an
Auxiliary Classifier Generative Adversarial Network (AC-GAN) to model the
class-conditional distribution over data samples. Then, conditioned on a
desired class, we search over the AC-GAN latent space to find images that are
likely under the generative model and are misclassified by a target classifier.
We demonstrate through human evaluation that unrestricted adversarial examples
generated this way are legitimate and belong to the desired class. Our
empirical results on the MNIST, SVHN, and CelebA datasets show that
unrestricted adversarial examples can bypass strong adversarial training and
certified defense methods designed for traditional adversarial attacks.
Transforming Cooling Optimization for Green Data Center via Deep Reinforcement Learning
Authors: Yuanlong Li, Yonggang Wen, Kyle Guan, DaCheng Tao
Date: 2017-09-15 00:00:00
Tasks: Deep Reinforcement Learning, Management, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Cooling system plays a critical role in a modern data center (DC). Developing
an optimal control policy for DC cooling system is a challenging task. The
prevailing approaches often rely on approximating system models that are built
upon the knowledge of mechanical cooling, electrical and thermal management,
which is difficult to design and may lead to sub-optimal or unstable
performances. In this paper, we propose utilizing the large amount of
monitoring data in DC to optimize the control policy. To do so, we cast the
cooling control policy design into an energy cost minimization problem with
temperature constraints, and tap it into the emerging deep reinforcement
learning (DRL) framework. Specifically, we propose an end-to-end cooling
control algorithm (CCA) that is based on the actor-critic framework and an
off-policy offline version of the deep deterministic policy gradient (DDPG)
algorithm. In the proposed CCA, an evaluation network is trained to predict an
energy cost counter penalized by the cooling status of the DC room, and a
policy network is trained to predict optimized control settings when gave the
current load and weather information. The proposed algorithm is evaluated on
the EnergyPlus simulation platform and on a real data trace collected from the
National Super Computing Centre (NSCC) of Singapore. Our results show that the
proposed CCA can achieve about 11% cooling cost saving on the simulation
platform compared with a manually configured baseline control algorithm. In the
trace-based study, we propose a de-underestimation validation mechanism as we
cannot directly test the algorithm on a real DC. Even though with DUE the
results are conservative, we can still achieve about 15% cooling energy saving
on the NSCC data trace if we set the inlet temperature threshold at 26.6 degree
Celsius.
A Practical Algorithm for Distributed Clustering and Outlier Detection
Authors: Jiecao Chen, Erfan Sadeqi Azer, Qin Zhang
Date: 2018-05-24 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Clustering, Outlier Detection

We study the classic $k$-means/median clustering, which are fundamental
problems in unsupervised learning, in the setting where data are partitioned
across multiple sites, and where we are allowed to discard a small portion of
the data by labeling them as outliers. We propose a simple approach based on
constructing small summary for the original dataset. The proposed method is
time and communication efficient, has good approximation guarantees, and can
identify the global outliers effectively. To the best of our knowledge, this is
the first practical algorithm with theoretical guarantees for distributed
clustering with outliers. Our experiments on both real and synthetic data have
demonstrated the clear superiority of our algorithm against all the baseline
algorithms in almost all metrics.
Explanation in Artificial Intelligence: Insights from the Social Sciences
Authors: Tim Miller
Date: 2017-06-22 00:00:00
Tasks: Explainable artificial intelligence, Philosophy

There has been a recent resurgence in the area of explainable artificial
intelligence as researchers and practitioners seek to make their algorithms
more understandable. Much of this research is focused on explicitly explaining
decisions or actions to a human observer, and it should not be controversial to
say that looking at how humans explain to each other can serve as a useful
starting point for explanation in artificial intelligence. However, it is fair
to say that most work in explainable artificial intelligence uses only the
researchers' intuition of what constitutes a `good' explanation. There exists
vast and valuable bodies of research in philosophy, psychology, and cognitive
science of how people define, generate, select, evaluate, and present
explanations, which argues that people employ certain cognitive biases and
social expectations towards the explanation process. This paper argues that the
field of explainable artificial intelligence should build on this existing
research, and reviews relevant papers from philosophy, cognitive
psychology/science, and social psychology, which study these topics. It draws
out some important findings, and discusses ways that these can be infused with
work on explainable artificial intelligence.
AutoPruner: An End-to-End Trainable Filter Pruning Method for Efficient Deep Model Inference
Authors: Jian-Hao Luo, Jianxin Wu
Date: 2018-05-23 00:00:00
Tasks: Binarization, channel selection

Channel pruning is an important family of methods to speed up deep model's
inference. Previous filter pruning algorithms regard channel pruning and model
fine-tuning as two independent steps. This paper argues that combining them
into a single end-to-end trainable system will lead to better results. We
propose an efficient channel selection layer, namely AutoPruner, to find less
important filters automatically in a joint training manner. Our AutoPruner
takes previous activation responses as an input and generates a true binary
index code for pruning. Hence, all the filters corresponding to zero index
values can be removed safely after training. We empirically demonstrate that
the gradient information of this channel selection layer is also helpful for
the whole model training. By gradually erasing several weak filters, we can
prevent an excessive drop in model accuracy. Compared with previous
state-of-the-art pruning algorithms (including training from scratch),
AutoPruner achieves significantly better performance. Furthermore, ablation
experiments show that the proposed novel mini-batch pooling and binarization
operations are vital for the success of filter pruning.
Multi-Level Deep Cascade Trees for Conversion Rate Prediction in Recommendation System
Authors: Hong Wen, Jing Zhang, Quan Lin, Keping Yang, Pipei Huang
Date: 2018-05-24 00:00:00
Tasks: Click-Through Rate Prediction, Ensemble Learning

Developing effective and efficient recommendation methods is very challenging
for modern e-commerce platforms. Generally speaking, two essential modules
named "Click-Through Rate Prediction" (\textit{CTR}) and "Conversion Rate
Prediction" (\textit{CVR}) are included, where \textit{CVR} module is a crucial
factor that affects the final purchasing volume directly. However, it is indeed
very challenging due to its sparseness nature. In this paper, we tackle this
problem by proposing multi-Level Deep Cascade Trees (\textit{ldcTree}), which
is a novel decision tree ensemble approach. It leverages deep cascade
structures by stacking Gradient Boosting Decision Trees (\textit{GBDT}) to
effectively learn feature representation. In addition, we propose to utilize
the cross-entropy in each tree of the preceding \textit{GBDT} as the input
feature representation for next level \textit{GBDT}, which has a clear
explanation, i.e., a traversal from root to leaf nodes in the next level
\textit{GBDT} corresponds to the combination of certain traversals in the
preceding \textit{GBDT}. The deep cascade structure and the combination rule
enable the proposed \textit{ldcTree} to have a stronger distributed feature
representation ability. Moreover, inspired by ensemble learning, we propose an
Ensemble \textit{ldcTree} (\textit{E-ldcTree}) to encourage the model's
diversity and enhance the representation ability further. Finally, we propose
an improved Feature learning method based on \textit{EldcTree}
(\textit{F-EldcTree}) for taking adequate use of weak and strong correlation
features identified by pre-trained \textit{GBDT} models. Experimental results
on off-line data set and online deployment demonstrate the effectiveness of the
proposed methods.
Optimal Algorithms for Continuous Non-monotone Submodular and DR-Submodular Maximization
Authors: Rad Niazadeh, Tim Roughgarden, Joshua R. Wang
Date: 2018-05-24 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: BIG-bench Machine Learning

In this paper we study the fundamental problems of maximizing a continuous
non-monotone submodular function over the hypercube, both with and without
coordinate-wise concavity. This family of optimization problems has several
applications in machine learning, economics, and communication systems. Our
main result is the first $\frac{1}{2}$-approximation algorithm for continuous
submodular function maximization; this approximation factor of $\frac{1}{2}$ is
the best possible for algorithms that only query the objective function at
polynomially many points. For the special case of DR-submodular maximization,
i.e. when the submodular functions is also coordinate wise concave along all
coordinates, we provide a different $\frac{1}{2}$-approximation algorithm that
runs in quasilinear time. Both of these results improve upon prior work [Bian
et al, 2017, Soma and Yoshida, 2017].
  Our first algorithm uses novel ideas such as reducing the guaranteed
approximation problem to analyzing a zero-sum game for each coordinate, and
incorporates the geometry of this zero-sum game to fix the value at this
coordinate. Our second algorithm exploits coordinate-wise concavity to identify
a monotone equilibrium condition sufficient for getting the required
approximation guarantee, and hunts for the equilibrium point using binary
search. We further run experiments to verify the performance of our proposed
algorithms in related machine learning applications.
Word Level Font-to-Font Image Translation using Convolutional Recurrent Generative Adversarial Networks
Authors: Ankan Kumar Bhunia, Ayan Kumar Bhunia, Prithaj Banerjee, Aishik Konwer, Abir Bhowmick, Partha Pratim Roy, Umapada Pal
Date: 2018-01-22 00:00:00
Tasks: Translation

Conversion of one font to another font is very useful in real life
applications. In this paper, we propose a Convolutional Recurrent Generative
model to solve the word level font transfer problem. Our network is able to
convert the font style of any printed text images from its current font to the
required font. The network is trained end-to-end for the complete word images.
Thus it eliminates the necessary pre-processing steps, like character
segmentations. We extend our model to conditional setting that helps to learn
one-to-many mapping function. We employ a novel convolutional recurrent model
architecture in the Generator that efficiently deals with the word images of
arbitrary width. It also helps to maintain the consistency of the final images
after concatenating the generated image patches of target font. Besides, the
Generator and the Discriminator network, we employ a Classification network to
classify the generated word images of converted font style to their subsequent
font categories. Most of the earlier works related to image translation are
performed on square images. Our proposed architecture is the first work which
can handle images of varying widths. Word images generally have varying width
depending on the number of characters present. Hence, we test our model on a
synthetically generated font dataset. We compare our method with some of the
state-of-the-art methods for image translation. The superior performance of our
network on the same dataset proves the ability of our model to learn the font
distributions.
Hierarchical Clustering with Structural Constraints
Authors: Vaggos Chatziafratis, Rad Niazadeh, Moses Charikar
Date: 2018-05-24 00:00:00
Proceeding: ICML 2018 7
Tasks: Clustering

Hierarchical clustering is a popular unsupervised data analysis method. For
many real-world applications, we would like to exploit prior information about
the data that imposes constraints on the clustering hierarchy, and is not
captured by the set of features available to the algorithm. This gives rise to
the problem of "hierarchical clustering with structural constraints".
Structural constraints pose major challenges for bottom-up approaches like
average/single linkage and even though they can be naturally incorporated into
top-down divisive algorithms, no formal guarantees exist on the quality of
their output. In this paper, we provide provable approximation guarantees for
two simple top-down algorithms, using a recently introduced optimization
viewpoint of hierarchical clustering with pairwise similarity information
[Dasgupta, 2016]. We show how to find good solutions even in the presence of
conflicting prior information, by formulating a constraint-based regularization
of the objective. We further explore a variation of this objective for
dissimilarity information [Cohen-Addad et al., 2018] and improve upon current
techniques. Finally, we demonstrate our approach on a real dataset for the
taxonomy application.
VisualBackProp for learning using privileged information with CNNs
Authors: Devansh Bisla, Anna Choromanska
Date: 2018-05-24 00:00:00
Tasks: Autonomous Driving, BIG-bench Machine Learning, Common Sense Reasoning, General Classification, Lesion Classification, Skin Lesion Classification

In many machine learning applications, from medical diagnostics to autonomous
driving, the availability of prior knowledge can be used to improve the
predictive performance of learning algorithms and incorporate `physical,'
`domain knowledge,' or `common sense' concepts into training of machine
learning systems as well as verify constraints/properties of the systems. We
explore the learning using privileged information paradigm and show how to
incorporate the privileged information, such as segmentation mask available
along with the classification label of each example, into the training stage of
convolutional neural networks. This is done by augmenting the CNN model with an
architectural component that effectively focuses model's attention on the
desired region of the input image during the training process and that is
transparent to the network's label prediction mechanism at testing. This
component effectively corresponds to the visualization strategy for identifying
the parts of the input, often referred to as visualization mask, that most
contribute to the prediction, yet uses this strategy in reverse to the
classical setting in order to enforce the desired visualization mask instead.
We verify our proposed algorithms through exhaustive experiments on benchmark
ImageNet and PASCAL VOC data sets and achieve improvements in the performance
of $2.4\%$ and $2.7\%$ over standard single-supervision model training.
Finally, we confirm the effectiveness of our approach on skin lesion
classification problem.
Deploy Large-Scale Deep Neural Networks in Resource Constrained IoT Devices with Local Quantization Region
Authors: Yi Yang, Andy Chen, Xiaoming Chen, Jiang Ji, Zhenyang Chen, Yan Dai
Date: 2018-05-24 00:00:00
Tasks: Quantization

Implementing large-scale deep neural networks with high computational
complexity on low-cost IoT devices may inevitably be constrained by limited
computation resource, making the devices hard to respond in real-time. This
disjunction makes the state-of-art deep learning algorithms, i.e. CNN
(Convolutional Neural Networks), incompatible with IoT world. We present a
low-bit (range from 8-bit to 1-bit) scheme with our local quantization region
algorithm. We use models in Caffe model zoo as our example tasks to evaluate
the effect of our low precision data representation scheme. With the available
of local quantization region, we find implementations on top of those schemes
could greatly retain the model accuracy, besides the reduction of computational
complexity. For example, our 8-bit scheme has no drops on top-1 and top-5
accuracy with 2x speedup on Intel Edison IoT platform. Implementations based on
our 4-bit, 2-bit or 1-bit scheme are also applicable to IoT devices with
advances of low computational complexity. For example, the drop on our task is
only 0.7% when using 2-bit scheme, a scheme which could largely save
transistors. Making low-bit scheme usable here opens a new door for further
optimization on commodity IoT controller, i.e. extra speed-up could be achieved
by replacing multiply-accumulate operations with the proposed table look-up
operations. The whole study offers a new approach to relief the challenge of
bring advanced deep learning algorithm to resource constrained low-cost IoT
device.
Taming Convergence for Asynchronous Stochastic Gradient Descent with Unbounded Delay in Non-Convex Learning
Authors: Xin Zhang, Jia Liu, Zhengyuan Zhu
Date: 2018-05-24 00:00:00

Understanding the convergence performance of asynchronous stochastic gradient descent method (Async-SGD) has received increasing attention in recent years due to their foundational role in machine learning. To date, however, most of the existing works are restricted to either bounded gradient delays or convex settings. In this paper, we focus on Async-SGD and its variant Async-SGDI (which uses increasing batch size) for non-convex optimization problems with unbounded gradient delays. We prove $o(1/\sqrt{k})$ convergence rate for Async-SGD and $o(1/k)$ for Async-SGDI. Also, a unifying sufficient condition for Async-SGD's convergence is established, which includes two major gradient delay models in the literature as special cases and yields a new delay model not considered thus far.
Dual Free Adaptive Mini-batch SDCA for Empirical Risk Minimization
Authors: Xi He, Martin Takáč
Date: 2015-10-22 00:00:00

In this paper we develop dual free mini-batch SDCA with adaptive
probabilities for regularized empirical risk minimization. This work is
motivated by recent work of Shai Shalev-Shwartz on dual free SDCA method,
however, we allow a non-uniform selection of "dual" coordinates in SDCA.
Moreover, the probability can change over time, making it more efficient than
fix uniform or non-uniform selection. We also propose an efficient procedure to
generate a random non-uniform mini-batch through iterative process. The work is
concluded with multiple numerical experiments to show the efficiency of
proposed algorithms.
Simple and practical algorithms for $\ell_p$-norm low-rank approximation
Authors: Anastasios Kyrillidis
Date: 2018-05-24 00:00:00

We propose practical algorithms for entrywise $\ell_p$-norm low-rank
approximation, for $p = 1$ or $p = \infty$. The proposed framework, which is
non-convex and gradient-based, is easy to implement and typically attains
better approximations, faster, than state of the art.
  From a theoretical standpoint, we show that the proposed scheme can attain
$(1 + \varepsilon)$-OPT approximations. Our algorithms are not
hyperparameter-free: they achieve the desiderata only assuming algorithm's
hyperparameters are known a priori---or are at least approximable. I.e., our
theory indicates what problem quantities need to be known, in order to get a
good solution within polynomial time, and does not contradict to recent
inapproximabilty results, as in [46].
Resource Aware Person Re-identification across Multiple Resolutions
Authors: Yan Wang, Lequn Wang, Yurong You, Xu Zou, Vincent Chen, Serena Li, Gao Huang, Bharath Hariharan, Kilian Q. Weinberger
Date: 2018-05-22 00:00:00
Proceeding: CVPR 2018 6
Tasks: All, Person Re-Identification

Not all people are equally easy to identify: color statistics might be enough
for some cases while others might require careful reasoning about high- and
low-level details. However, prevailing person re-identification(re-ID) methods
use one-size-fits-all high-level embeddings from deep convolutional networks
for all cases. This might limit their accuracy on difficult examples or makes
them needlessly expensive for the easy ones. To remedy this, we present a new
person re-ID model that combines effective embeddings built on multiple
convolutional network layers, trained with deep-supervision. On traditional
re-ID benchmarks, our method improves substantially over the previous
state-of-the-art results on all five datasets that we evaluate on. We then
propose two new formulations of the person re-ID problem under
resource-constraints, and show how our model can be used to effectively trade
off accuracy and computation in the presence of resource constraints. Code and
pre-trained models are available at https://github.com/mileyan/DARENet.
Complex Relations in a Deep Structured Prediction Model for Fine Image Segmentation
Authors: Cristina Mata, Guy Ben-Yosef, Boris Katz
Date: 2018-05-24 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation, Structured Prediction

Many deep learning architectures for semantic segmentation involve a Fully
Convolutional Neural Network (FCN) followed by a Conditional Random Field (CRF)
to carry out inference over an image. These models typically involve unary
potentials based on local appearance features computed by FCNs, and binary
potentials based on the displacement between pixels. We show that while current
methods succeed in segmenting whole objects, they perform poorly in situations
involving a large number of object parts. We therefore suggest incorporating
into the inference algorithm additional higher-order potentials inspired by the
way humans identify and localize parts. We incorporate two relations that were
shown to be useful to human object identification - containment and attachment
- into the energy term of the CRF and evaluate their performance on the Pascal
VOC Parts dataset. Our experimental results show that the segmentation of fine
parts is positively affected by the addition of these two relations, and that
the segmentation of fine parts can be further influenced by complex structural
features.