pwc-archive/papers-with-abstracts huggingface dataset:
Dynamic Representations Toward Efficient Inference on Deep Neural Networks by Decision Gates
Authors: Mohammad Saeed Shafiee, Mohammad Javad Shafiee, Alexander Wong
Date: 2018-11-05 00:00:00

While deep neural networks extract rich features from the input data, the current trade-off between depth and computational cost makes it difficult to adopt deep neural networks for many industrial applications, especially when computing power is limited. Here, we are inspired by the idea that, while deeper embeddings are needed to discriminate difficult samples (i.e., fine-grained discrimination), a large number of samples can be well discriminated via much shallower embeddings (i.e., coarse-grained discrimination). In this study, we introduce the simple yet effective concept of decision gates (d-gate), modules trained to decide whether a sample needs to be projected into a deeper embedding or if an early prediction can be made at the d-gate, thus enabling the computation of dynamic representations at different depths. The proposed d-gate modules can be integrated with any deep neural network and reduces the average computational cost of the deep neural networks while maintaining modeling accuracy. The proposed d-gate framework is examined via different network architectures and datasets, with experimental results showing that leveraging the proposed d-gate modules led to a ~43% speed-up and 44% FLOPs reduction on ResNet-101 and 55% speed-up and 39% FLOPs reduction on DenseNet-201 trained on the CIFAR10 dataset with only ~2% drop in accuracy. Furthermore, experiments where d-gate modules are integrated into ResNet-101 trained on the ImageNet dataset demonstrate that it is possible to reduce the computational cost of the network by 1.5 GFLOPs without any drop in the modeling accuracy.
Fast Face Image Synthesis with Minimal Training
Authors: Sandipan Banerjee, Walter J. Scheirer, Kevin W. Bowyer, Patrick J. Flynn
Date: 2018-11-05 00:00:00
Tasks: Image Generation

We propose an algorithm to generate realistic face images of both real and
synthetic identities (people who do not exist) with different facial yaw, shape
and resolution.The synthesized images can be used to augment datasets to train
CNNs or as massive distractor sets for biometric verification experiments
without any privacy concerns. Additionally, law enforcement can make use of
this technique to train forensic experts to recognize faces. Our method samples
face components from a pool of multiple face images of real identities to
generate the synthetic texture. Then, a real 3D head model compatible to the
generated texture is used to render it under different facial yaw
transformations. We perform multiple quantitative experiments to assess the
effectiveness of our synthesis procedure in CNN training and its potential use
to generate distractor face images. Additionally, we compare our method with
popular GAN models in terms of visual quality and execution time.
Medical code prediction with multi-view convolution and description-regularized label-dependent attention
Authors: Najmeh Sadoughi, Greg P. Finley, James Fone, Vignesh Murali, Maxim Korenevski, Slava Baryshnikov, Nico Axtmann, Mark Miller, David Suendermann-Oeft
Date: 2018-11-05 00:00:00
Tasks: Medical Code Prediction

A ubiquitous task in processing electronic medical data is the assignment of
standardized codes representing diagnoses and/or procedures to free-text
documents such as medical reports. This is a difficult natural language
processing task that requires parsing long, heterogeneous documents and
selecting a set of appropriate codes from tens of thousands of
possibilities---many of which have very few positive training samples. We
present a deep learning system that advances the state of the art for the
MIMIC-III dataset, achieving a new best micro F1-measure of 55.85\%,
significantly outperforming the previous best result (Mullenbach et al. 2018).
We achieve this through a number of enhancements, including two major novel
contributions: multi-view convolutional channels, which effectively learn to
adjust kernel sizes throughout the input; and attention regularization,
mediated by natural-language code descriptions, which helps overcome sparsity
for thousands of uncommon codes. These and other modifications are selected to
address difficulties inherent to both automated coding specifically and deep
learning generally. Finally, we investigate our accuracy results in detail to
individually measure the impact of these contributions and point the way
towards future algorithmic improvements.
Practical Batch Bayesian Optimization for Less Expensive Functions
Authors: Vu Nguyen, Sunil Gupta, Santu Rana, Cheng Li, Svetha Venkatesh
Date: 2018-11-05 00:00:00
Tasks: Bayesian Optimization, global-optimization

Bayesian optimization (BO) and its batch extensions are successful for
optimizing expensive black-box functions. However, these traditional BO
approaches are not yet ideal for optimizing less expensive functions when the
computational cost of BO can dominate the cost of evaluating the blackbox
function. Examples of these less expensive functions are cheap machine learning
models, inexpensive physical experiment through simulators, and acquisition
function optimization in Bayesian optimization. In this paper, we consider a
batch BO setting for situations where function evaluations are less expensive.
Our model is based on a new exploration strategy using geometric distance that
provides an alternative way for exploration, selecting a point far from the
observed locations. Using that intuition, we propose to use Sobol sequence to
guide exploration that will get rid of running multiple global optimization
steps as used in previous works. Based on the proposed distance exploration, we
present an efficient batch BO approach. We demonstrate that our approach
outperforms other baselines and global optimization methods when the function
evaluations are less expensive.
Intrinsic Universal Measurements of Non-linear Embeddings
Authors: Ke Sun
Date: 2018-11-05 00:00:00
Tasks: BIG-bench Machine Learning

A basic problem in machine learning is to find a mapping $f$ from a low dimensional latent space $\mathcal{Y}$ to a high dimensional observation space $\mathcal{X}$. Modern tools such as deep neural networks are capable to represent general non-linear mappings. A learner can easily find a mapping which perfectly fits all the observations. However, such a mapping is often not considered as good, because it is not simple enough and can overfit. How to define simplicity? We try to make a formal definition on the amount of information imposed by a non-linear mapping $f$. Intuitively, we measure the local discrepancy between the pullback geometry and the intrinsic geometry of the latent space. Our definition is based on information geometry and is independent of the empirical observations, nor specific parameterizations. We prove its basic properties and discuss relationships with related machine learning methods.
Security for Machine Learning-based Systems: Attacks and Challenges during Training and Inference
Authors: Faiq Khalid, Muhammad Abdullah Hanif, Semeen Rehman, Muhammad Shafique
Date: 2018-11-05 00:00:00
Tasks: BIG-bench Machine Learning, Traffic Sign Recognition

The exponential increase in dependencies between the cyber and physical world
leads to an enormous amount of data which must be efficiently processed and
stored. Therefore, computing paradigms are evolving towards machine learning
(ML)-based systems because of their ability to efficiently and accurately
process the enormous amount of data. Although ML-based solutions address the
efficient computing requirements of big data, they introduce (new) security
vulnerabilities into the systems, which cannot be addressed by traditional
monitoring-based security measures. Therefore, this paper first presents a
brief overview of various security threats in machine learning, their
respective threat models and associated research challenges to develop robust
security measures. To illustrate the security vulnerabilities of ML during
training, inferencing and hardware implementation, we demonstrate some key
security threats on ML using LeNet and VGGNet for MNIST and German Traffic Sign
Recognition Benchmarks (GTSRB), respectively. Moreover, based on the security
analysis of ML-training, we also propose an attack that has a very less impact
on the inference accuracy. Towards the end, we highlight the associated
research challenges in developing security measures and provide a brief
overview of the techniques used to mitigate such security threats.
Deep Metric Learning by Online Soft Mining and Class-Aware Attention
Authors: Xinshao Wang, Yang Hua, Elyor Kodirov, Guosheng Hu, Neil M. Robertson
Date: 2018-11-04 00:00:00
Tasks: Metric Learning, Person Re-Identification, Semantic Similarity, Semantic Textual Similarity, Video-Based Person Re-Identification

Deep metric learning aims to learn a deep embedding that can capture the semantic similarity of data points. Given the availability of massive training samples, deep metric learning is known to suffer from slow convergence due to a large fraction of trivial samples. Therefore, most existing methods generally resort to sample mining strategies for selecting nontrivial samples to accelerate convergence and improve performance. In this work, we identify two critical limitations of the sample mining methods, and provide solutions for both of them. First, previous mining methods assign one binary score to each sample, i.e., dropping or keeping it, so they only selects a subset of relevant samples in a mini-batch. Therefore, we propose a novel sample mining method, called Online Soft Mining (OSM), which assigns one continuous score to each sample to make use of all samples in the mini-batch. OSM learns extended manifolds that preserve useful intraclass variances by focusing on more similar positives. Second, the existing methods are easily influenced by outliers as they are generally included in the mined subset. To address this, we introduce Class-Aware Attention (CAA) that assigns little attention to abnormal data samples. Furthermore, by combining OSM and CAA, we propose a novel weighted contrastive loss to learn discriminative embeddings. Extensive experiments on two fine-grained visual categorisation datasets and two video-based person re-identification benchmarks show that our method significantly outperforms the state-of-the-art.
FAdeML: Understanding the Impact of Pre-Processing Noise Filtering on Adversarial Machine Learning
Authors: Faiq Khalid, Muhammmad Abdullah Hanif, Semeen Rehman, Junaid Qadir, Muhammad Shafique
Date: 2018-11-04 00:00:00
Tasks: Adversarial Attack, BIG-bench Machine Learning, Traffic Sign Recognition

Deep neural networks (DNN)-based machine learning (ML) algorithms have
recently emerged as the leading ML paradigm particularly for the task of
classification due to their superior capability of learning efficiently from
large datasets. The discovery of a number of well-known attacks such as dataset
poisoning, adversarial examples, and network manipulation (through the addition
of malicious nodes) has, however, put the spotlight squarely on the lack of
security in DNN-based ML systems. In particular, malicious actors can use these
well-known attacks to cause random/targeted misclassification, or cause a
change in the prediction confidence, by only slightly but systematically
manipulating the environmental parameters, inference data, or the data
acquisition block. Most of the prior adversarial attacks have, however, not
accounted for the pre-processing noise filters commonly integrated with the
ML-inference module. Our contribution in this work is to show that this is a
major omission since these noise filters can render ineffective the majority of
the existing attacks, which rely essentially on introducing adversarial noise.
Apart from this, we also extend the state of the art by proposing a novel
pre-processing noise Filter-aware Adversarial ML attack called FAdeML. To
demonstrate the effectiveness of the proposed methodology, we generate an
adversarial attack image by exploiting the "VGGNet" DNN trained for the "German
Traffic Sign Recognition Benchmarks (GTSRB" dataset, which despite having no
visual noise, can cause a classifier to misclassify even in the presence of
pre-processing noise filters.
SSCNets: Robustifying DNNs using Secure Selective Convolutional Filters
Authors: Hassan Ali, Faiq Khalid, Hammad Tariq, Muhammad Abdullah Hanif, Semeen Rehman, Rehan Ahmed, Muhammad Shafique
Date: 2018-11-04 00:00:00

In this paper, we introduce a novel technique based on the Secure Selective Convolutional (SSC) techniques in the training loop that increases the robustness of a given DNN by allowing it to learn the data distribution based on the important edges in the input image. We validate our technique on Convolutional DNNs against the state-of-the-art attacks from the open-source Cleverhans library using the MNIST, the CIFAR-10, and the CIFAR-100 datasets. Our experimental results show that the attack success rate, as well as the imperceptibility of the adversarial images, can be significantly reduced by adding effective pre-processing functions, i.e., Sobel filtering.
Towards a Zero-One Law for Column Subset Selection
Authors: Zhao Song, David P. Woodruff, Peilin Zhong
Date: 2018-11-04 00:00:00
Proceeding: NeurIPS 2019 12

There are a number of approximation algorithms for NP-hard versions of low rank approximation, such as finding a rank-$k$ matrix $B$ minimizing the sum of absolute values of differences to a given $n$-by-$n$ matrix $A$, $\min_{\textrm{rank-}k~B}\|A-B\|_1$, or more generally finding a rank-$k$ matrix $B$ which minimizes the sum of $p$-th powers of absolute values of differences, $\min_{\textrm{rank-}k~B}\|A-B\|_p^p$. Many of these algorithms are linear time columns subset selection algorithms, returning a subset of $\mathrm{poly}(k \log n)$ columns whose cost is no more than a $\mathrm{poly}(k)$ factor larger than the cost of the best rank-$k$ matrix. The above error measures are special cases of the following general entrywise low rank approximation problem: given an arbitrary function $g:\mathbb{R} \rightarrow \mathbb{R}_{\geq 0}$, find a rank-$k$ matrix $B$ which minimizes $\|A-B\|_g = \sum_{i,j}g(A_{i,j}-B_{i,j})$. A natural question is which functions $g$ admit efficient approximation algorithms? Indeed, this is a central question of recent work studying generalized low rank models. In this work we give approximation algorithms for $\textit{every}$ function $g$ which is approximately monotone and satisfies an approximate triangle inequality, and we show both of these conditions are necessary. Further, our algorithm is efficient if the function $g$ admits an efficient approximate regression algorithm. Our approximation algorithms handle functions which are not even scale-invariant, such as the Huber loss function, which we show have very different structural properties than $\ell_p$-norms, e.g., one can show the lack of scale-invariance causes any column subset selection algorithm to provably require a $\sqrt{\log n}$ factor larger number of columns than $\ell_p$-norms; nevertheless we design the first efficient column subset selection algorithms for such error measures.
Explaining Explanations in AI
Authors: Brent Mittelstadt, Chris Russell, Sandra Wachter
Date: 2018-11-04 00:00:00
Tasks: BIG-bench Machine Learning, Philosophy, Sociology

Recent work on interpretability in machine learning and AI has focused on the
building of simplified models that approximate the true criteria used to make
decisions. These models are a useful pedagogical device for teaching trained
professionals how to predict what decisions will be made by the complex system,
and most importantly how the system might break. However, when considering any
such model it's important to remember Box's maxim that "All models are wrong
but some are useful." We focus on the distinction between these models and
explanations in philosophy and sociology. These models can be understood as a
"do it yourself kit" for explanations, allowing a practitioner to directly
answer "what if questions" or generate contrastive explanations without
external assistance. Although a valuable ability, giving these models as
explanations appears more difficult than necessary, and other forms of
explanation may not have the same trade-offs. We contrast the different schools
of thought on what makes an explanation, and suggest that machine learning
might benefit from viewing the problem more broadly.
QuSecNets: Quantization-based Defense Mechanism for Securing Deep Neural Network against Adversarial Attacks
Authors: Faiq Khalid, Hassan Ali, Hammad Tariq, Muhammad Abdullah Hanif, Semeen Rehman, Rehan Ahmed, Muhammad Shafique
Date: 2018-11-04 00:00:00
Tasks: Quantization

Adversarial examples have emerged as a significant threat to machine learning algorithms, especially to the convolutional neural networks (CNNs). In this paper, we propose two quantization-based defense mechanisms, Constant Quantization (CQ) and Trainable Quantization (TQ), to increase the robustness of CNNs against adversarial examples. CQ quantizes input pixel intensities based on a "fixed" number of quantization levels, while in TQ, the quantization levels are "iteratively learned during the training phase", thereby providing a stronger defense mechanism. We apply the proposed techniques on undefended CNNs against different state-of-the-art adversarial attacks from the open-source \textit{Cleverhans} library. The experimental results demonstrate 50%-96% and 10%-50% increase in the classification accuracy of the perturbed images generated from the MNIST and the CIFAR-10 datasets, respectively, on commonly used CNN (Conv2D(64, 8x8) - Conv2D(128, 6x6) - Conv2D(128, 5x5) - Dense(10) - Softmax()) available in \textit{Cleverhans} library.
False Positive Reduction in Lung Computed Tomography Images using Convolutional Neural Networks
Authors: Gorkem Polat, Ugur Halici, Yesim Serinagaoglu Dogrusoz
Date: 2018-11-04 00:00:00
Tasks: Computed Tomography (CT)

Recent studies have shown that lung cancer screening using annual low-dose
computed tomography (CT) reduces lung cancer mortality by 20% compared to
traditional chest radiography. Therefore, CT lung screening has started to be
used widely all across the world. However, analyzing these images is a serious
burden for radiologists. In this study, we propose a novel and simple framework
that analyzes CT lung screenings using convolutional neural networks (CNNs) and
reduces false positives. Our framework shows that even non-complex
architectures are very powerful to classify 3D nodule data when compared to
traditional methods. We also use different fusions in order to show their power
and effect on the overall score. 3D CNNs are preferred over 2D CNNs because
data are in 3D, and 2D convolutional operations may result in information loss.
Mini-batch is used in order to overcome class-imbalance. Proposed framework has
been validated according to the LUNA16 challenge evaluation and got score of
0.786, which is the average sensitivity values at seven predefined false
positive (FP) points.
Semantic Role Labeling for Knowledge Graph Extraction from Text
Authors: Mehwish Alam, Aldo Gangemi, Valentina Presutti, Diego Reforgiato Recupero
Date: 2018-11-04 00:00:00
Tasks: Dependency Parsing, Semantic Role Labeling

This paper introduces TakeFive, a new semantic role labeling method that
transforms a text into a frame-oriented knowledge graph. It performs dependency
parsing, identifies the words that evoke lexical frames, locates the roles and
fillers for each frame, runs coercion techniques, and formalises the results as
a knowledge graph. This formal representation complies with the frame semantics
used in Framester, a factual-linguistic linked data resource. The obtained
precision, recall and F1 values indicate that TakeFive is competitive with
other existing methods such as SEMAFOR, Pikes, PathLSTM and FRED. We finally
discuss how to combine TakeFive and FRED, obtaining higher values of precision,
recall and F1.
DeepKey: Towards End-to-End Physical Key Replication From a Single Photograph
Authors: Rory Smith, Tilo Burghardt
Date: 2018-11-04 00:00:00
Tasks: Key Detection

This paper describes DeepKey, an end-to-end deep neural architecture capable
of taking a digital RGB image of an 'everyday' scene containing a pin tumbler
key (e.g. lying on a table or carpet) and fully automatically inferring a
printable 3D key model. We report on the key detection performance and describe
how candidates can be transformed into physical prints. We show an example
opening a real-world lock. Our system is described in detail, providing a
breakdown of all components including key detection, pose normalisation,
bitting segmentation and 3D model inference. We provide an in-depth evaluation
and conclude by reflecting on limitations, applications, potential security
risks and societal impact. We contribute the DeepKey Datasets of 5, 300+ images
covering a few test keys with bounding boxes, pose and unaligned mask data.
Texture Synthesis Guided Deep Hashing for Texture Image Retrieval
Authors: Ayan Kumar Bhunia, Perla Sai Raj Kishore, Pranay Mukherjee, Abhirup Das, Partha Pratim Roy
Date: 2018-11-04 00:00:00
Tasks: Data Augmentation, Deep Hashing, Image Retrieval, Retrieval, Texture Image Retrieval, Texture Synthesis

With the large-scale explosion of images and videos over the internet, efficient hashing methods have been developed to facilitate memory and time efficient retrieval of similar images. However, none of the existing works uses hashing to address texture image retrieval mostly because of the lack of sufficiently large texture image databases. Our work addresses this problem by developing a novel deep learning architecture that generates binary hash codes for input texture images. For this, we first pre-train a Texture Synthesis Network (TSN) which takes a texture patch as input and outputs an enlarged view of the texture by injecting newer texture content. Thus it signifies that the TSN encodes the learnt texture specific information in its intermediate layers. In the next stage, a second network gathers the multi-scale feature representations from the TSN's intermediate layers using channel-wise attention, combines them in a progressive manner to a dense continuous representation which is finally converted into a binary hash code with the help of individual and pairwise label information. The new enlarged texture patches also help in data augmentation to alleviate the problem of insufficient texture data and are used to train the second stage of the network. Experiments on three public texture image retrieval datasets indicate the superiority of our texture synthesis guided hashing approach over current state-of-the-art methods.
Logic Attention Based Neighborhood Aggregation for Inductive Knowledge Graph Embedding
Authors: Peifeng Wang, Jialong Han, Chenliang Li, Rong pan
Date: 2018-11-04 00:00:00
Tasks: Graph Embedding, Knowledge Graph Completion, Knowledge Graph Embedding, Knowledge Graphs, World Knowledge

Knowledge graph embedding aims at modeling entities and relations with low-dimensional vectors. Most previous methods require that all entities should be seen during training, which is unpractical for real-world knowledge graphs with new entities emerging on a daily basis. Recent efforts on this issue suggest training a neighborhood aggregator in conjunction with the conventional entity and relation embeddings, which may help embed new entities inductively via their existing neighbors. However, their neighborhood aggregators neglect the unordered and unequal natures of an entity's neighbors. To this end, we summarize the desired properties that may lead to effective neighborhood aggregators. We also introduce a novel aggregator, namely, Logic Attention Network (LAN), which addresses the properties by aggregating neighbors with both rules- and network-based attention weights. By comparing with conventional aggregators on two knowledge graph completion tasks, we experimentally validate LAN's superiority in terms of the desired properties.