pwc-archive/papers-with-abstracts huggingface dataset:
Query Term Weighting based on Query Performance Prediction
Authors: Haggai Roitman
Date: 2019-02-27 00:00:00
Tasks: Prediction, Re-Ranking, Retrieval

This work presents a general query term weighting approach based on query
performance prediction (QPP). To this end, a given term is weighed according to
its predicted effect on query performance. Such an effect is assumed to be
manifested in the responses made by the underlying retrieval method for the
original query and its (simple) variants in the form of a single-term expanded
query. Focusing on search re-ranking as the underlying application, the
effectiveness of the proposed term weighting approach is demonstrated using
several state-of-the-art QPP methods evaluated over TREC corpora.
Cluster Regularized Quantization for Deep Networks Compression
Authors: Yiming Hu, Jianquan Li, Xianlei Long, Shenhua Hu, Jiagang Zhu, Xingang Wang, Qingyi Gu
Date: 2019-02-27 00:00:00
Tasks: Quantization

Deep neural networks (DNNs) have achieved great success in a wide range of
computer vision areas, but the applications to mobile devices is limited due to
their high storage and computational cost. Much efforts have been devoted to
compress DNNs. In this paper, we propose a simple yet effective method for deep
networks compression, named Cluster Regularized Quantization (CRQ), which can
reduce the presentation precision of a full-precision model to ternary values
without significant accuracy drop. In particular, the proposed method aims at
reducing the quantization error by introducing a cluster regularization term,
which is imposed on the full-precision weights to enable them naturally
concentrate around the target values. Through explicitly regularizing the
weights during the re-training stage, the full-precision model can achieve the
smooth transition to the low-bit one. Comprehensive experiments on benchmark
datasets demonstrate the effectiveness of the proposed method.
Counting to Ten with Two Fingers: Compressed Counting with Spiking Neurons
Authors: Yael Hitron, Merav Parter
Date: 2019-02-27 00:00:00
Tasks: Vocal Bursts Valence Prediction

We consider the task of measuring time with probabilistic threshold gates implemented by bio-inspired spiking neurons. In the model of spiking neural networks, network evolves in discrete rounds, where in each round, neurons fire in pulses in response to a sufficiently high membrane potential. This potential is induced by spikes from neighboring neurons that fired in the previous round, which can have either an excitatory or inhibitory effect. We first consider a deterministic implementation of a neural timer and show that $\Theta(\log t)$ (deterministic) threshold gates are both sufficient and necessary. This raised the question of whether randomness can be leveraged to reduce the number of neurons. We answer this question in the affirmative by considering neural timers with spiking neurons where the neuron $y$ is required to fire for $t$ consecutive rounds with probability at least $1-\delta$, and should stop firing after at most $2t$ rounds with probability $1-\delta$ for some input parameter $\delta \in (0,1)$. Our key result is a construction of a neural timer with $O(\log\log 1/\delta)$ spiking neurons. Interestingly, this construction uses only one spiking neuron, while the remaining neurons can be deterministic threshold gates. We complement this construction with a matching lower bound of $\Omega(\min\{\log\log 1/\delta, \log t\})$ neurons. This provides the first separation between deterministic and randomized constructions in the setting of spiking neural networks. Finally, we demonstrate the usefulness of compressed counting networks for synchronizing neural networks.
A Distributionally Robust Optimization Method for Adversarial Multiple Kernel Learning
Authors: Masoud Badiei Khuzani, Hongyi Ren, Md Tauhidul Islam, Lei Xing
Date: 2019-02-27 00:00:00
Tasks: Generalization Bounds, Model Selection, Semantic Segmentation, Small Data Image Classification

We propose a novel data-driven method to learn a mixture of multiple kernels with random features that is certifiabaly robust against adverserial inputs. Specifically, we consider a distributionally robust optimization of the kernel-target alignment with respect to the distribution of training samples over a distributional ball defined by the Kullback-Leibler (KL) divergence. The distributionally robust optimization problem can be recast as a min-max optimization whose objective function includes a log-sum term. We develop a mini-batch biased stochastic primal-dual proximal method to solve the min-max optimization. To debias the minibatch algorithm, we use the Gumbel perturbation technique to estimate the log-sum term. We establish theoretical guarantees for the performance of the proposed multiple kernel learning method. In particular, we prove the consistency, asymptotic normality, stochastic equicontinuity, and the minimax rate of the empirical estimators. In addition, based on the notion of Rademacher and Gaussian complexities, we establish distributionally robust generalization bounds that are tighter than previous known bounds. More specifically, we leverage matrix concentration inequalities to establish distributionally robust generalization bounds. We validate our kernel learning approach for classification with the kernel SVMs on synthetic dataset generated by sampling multvariate Gaussian distributions with differernt variance structures. We also apply our kernel learning approach to the MNIST data-set and evaluate its robustness to perturbation of input images under different adversarial models. More specifically, we examine the robustness of the proposed kernel model selection technique against FGSM, PGM, C\&W, and DDN adversarial perturbations, and compare its performance with alternative state-of-the-art multiple kernel learning paradigms.
Multi-loss-aware Channel Pruning of Deep Networks
Authors: Yiming Hu, Siyang Sun, Jianquan Li, Jiagang Zhu, Xingang Wang, Qingyi Gu
Date: 2019-02-27 00:00:00
Tasks: channel selection, General Classification

Channel pruning, which seeks to reduce the model size by removing redundant
channels, is a popular solution for deep networks compression. Existing channel
pruning methods usually conduct layer-wise channel selection by directly
minimizing the reconstruction error of feature maps between the baseline model
and the pruned one. However, they ignore the feature and semantic distributions
within feature maps and real contribution of channels to the overall
performance. In this paper, we propose a new channel pruning method by
explicitly using both intermediate outputs of the baseline model and the
classification loss of the pruned model to supervise layer-wise channel
selection. Particularly, we introduce an additional loss to encode the
differences in the feature and semantic distributions within feature maps
between the baseline model and the pruned one. By considering the
reconstruction error, the additional loss and the classification loss at the
same time, our approach can significantly improve the performance of the pruned
model. Comprehensive experiments on benchmark datasets demonstrate the
effectiveness of the proposed method.
edge2vec: Representation learning using edge semantics for biomedical knowledge discovery
Authors: Zheng Gao, Gang Fu, Chunping Ouyang, Satoshi Tsutsui, Xiaozhong Liu, Jeremy Yang, Christopher Gessner, Brian Foote, David Wild, Qi Yu, Ying Ding
Date: 2018-09-07 00:00:00
Tasks: Biomedical Information Retrieval, Information Retrieval, Knowledge Graphs, Representation Learning, Retrieval

Representation learning provides new and powerful graph analytical approaches and tools for the highly valued data science challenge of mining knowledge graphs. Since previous graph analytical methods have mostly focused on homogeneous graphs, an important current challenge is extending this methodology for richly heterogeneous graphs and knowledge domains. The biomedical sciences are such a domain, reflecting the complexity of biology, with entities such as genes, proteins, drugs, diseases, and phenotypes, and relationships such as gene co-expression, biochemical regulation, and biomolecular inhibition or activation. Therefore, the semantics of edges and nodes are critical for representation learning and knowledge discovery in real world biomedical problems. In this paper, we propose the edge2vec model, which represents graphs considering edge semantics. An edge-type transition matrix is trained by an Expectation-Maximization approach, and a stochastic gradient descent model is employed to learn node embedding on a heterogeneous graph via the trained transition matrix. edge2vec is validated on three biomedical domain tasks: biomedical entity classification, compound-gene bioactivity prediction, and biomedical information retrieval. Results show that by considering edge-types into node embedding learning in heterogeneous graphs, \textbf{edge2vec}\ significantly outperforms state-of-the-art models on all three tasks. We propose this method for its added value relative to existing graph analytical methodology, and in the real world context of biomedical knowledge discovery applicability.
The Importance of Metric Learning for Robotic Vision: Open Set Recognition and Active Learning
Authors: Benjamin J. Meyer, Tom Drummond
Date: 2019-02-27 00:00:00
Tasks: Active Learning, Metric Learning, Open Set Learning

State-of-the-art deep neural network recognition systems are designed for a
static and closed world. It is usually assumed that the distribution at test
time will be the same as the distribution during training. As a result,
classifiers are forced to categorise observations into one out of a set of
predefined semantic classes. Robotic problems are dynamic and open world; a
robot will likely observe objects that are from outside of the training set
distribution. Classifier outputs in robotic applications can lead to real-world
robotic action and as such, a practical recognition system should not silently
fail by confidently misclassifying novel observations. We show how a deep
metric learning classification system can be applied to such open set
recognition problems, allowing the classifier to label novel observations as
unknown. Further to detecting novel examples, we propose an open set active
learning approach that allows a robot to efficiently query a user about unknown
observations. Our approach enables a robot to improve its understanding of the
true distribution of data in the environment, from a small number of label
queries. Experimental results show that our approach significantly outperforms
comparable methods in both the open set recognition and active learning
problems.
An Editorial Network for Enhanced Document Summarization
Authors: Edward Moroshko, Guy Feigenblat, Haggai Roitman, David Konopnicki
Date: 2019-02-27 00:00:00
Proceeding: WS 2019 11
Tasks: Abstractive Text Summarization, Document Summarization, Sentence

We suggest a new idea of Editorial Network - a mixed extractive-abstractive
summarization approach, which is applied as a post-processing step over a given
sequence of extracted sentences. Our network tries to imitate the decision
process of a human editor during summarization. Within such a process, each
extracted sentence may be either kept untouched, rephrased or completely
rejected. We further suggest an effective way for training the "editor" based
on a novel soft-labeling approach. Using the CNN/DailyMail dataset we
demonstrate the effectiveness of our approach compared to state-of-the-art
extractive-only or abstractive-only baseline methods.
Deeper Connections between Neural Networks and Gaussian Processes Speed-up Active Learning
Authors: Evgenii Tsymbalov, Sergei Makarychev, Alexander Shapeev, Maxim Panov
Date: 2019-02-27 00:00:00
Tasks: Active Learning, Gaussian Processes

Active learning methods for neural networks are usually based on greedy
criteria which ultimately give a single new design point for the evaluation.
Such an approach requires either some heuristics to sample a batch of design
points at one active learning iteration, or retraining the neural network after
adding each data point, which is computationally inefficient. Moreover,
uncertainty estimates for neural networks sometimes are overconfident for the
points lying far from the training sample. In this work we propose to
approximate Bayesian neural networks (BNN) by Gaussian processes, which allows
us to update the uncertainty estimates of predictions efficiently without
retraining the neural network, while avoiding overconfident uncertainty
prediction for out-of-sample points. In a series of experiments on real-world
data including large-scale problems of chemical and physical modeling, we show
superiority of the proposed approach over the state-of-the-art methods.
How Large a Vocabulary Does Text Classification Need? A Variational Approach to Vocabulary Selection
Authors: Wenhu Chen, Yu Su, Yilin Shen, Zhiyu Chen, Xifeng Yan, William Wang
Date: 2019-02-27 00:00:00
Proceeding: NAACL 2019 6
Tasks: General Classification, text-classification, Text Classification

With the rapid development in deep learning, deep neural networks have been
widely adopted in many real-life natural language applications. Under deep
neural networks, a pre-defined vocabulary is required to vectorize text inputs.
The canonical approach to select pre-defined vocabulary is based on the word
frequency, where a threshold is selected to cut off the long tail distribution.
However, we observed that such simple approach could easily lead to under-sized
vocabulary or over-sized vocabulary issues. Therefore, we are interested in
understanding how the end-task classification accuracy is related to the
vocabulary size and what is the minimum required vocabulary size to achieve a
specific performance. In this paper, we provide a more sophisticated
variational vocabulary dropout (VVD) based on variational dropout to perform
vocabulary selection, which can intelligently select the subset of the
vocabulary to achieve the required performance. To evaluate different
algorithms on the newly proposed vocabulary selection problem, we propose two
new metrics: Area Under Accuracy-Vocab Curve and Vocab Size under X\% Accuracy
Drop. Through extensive experiments on various NLP classification tasks, our
variational framework is shown to significantly outperform the frequency-based
and other selection baselines on these metrics.
Distributed Byzantine Tolerant Stochastic Gradient Descent in the Era of Big Data
Authors: Richeng Jin, Xiaofan He, Huaiyu Dai
Date: 2019-02-27 00:00:00
Tasks: BIG-bench Machine Learning

The recent advances in sensor technologies and smart devices enable the
collaborative collection of a sheer volume of data from multiple information
sources. As a promising tool to efficiently extract useful information from
such big data, machine learning has been pushed to the forefront and seen great
success in a wide range of relevant areas such as computer vision, health care,
and financial market analysis. To accommodate the large volume of data, there
is a surge of interest in the design of distributed machine learning, among
which stochastic gradient descent (SGD) is one of the mostly adopted methods.
Nonetheless, distributed machine learning methods may be vulnerable to
Byzantine attack, in which the adversary can deliberately share falsified
information to disrupt the intended machine learning procedures. Therefore, two
asynchronous Byzantine tolerant SGD algorithms are proposed in this work, in
which the honest collaborative workers are assumed to store the model
parameters derived from their own local data and use them as the ground truth.
The proposed algorithms can deal with an arbitrary number of Byzantine
attackers and are provably convergent. Simulation results based on a real-world
dataset are presented to verify the theoretical results and demonstrate the
effectiveness of the proposed algorithms.
Machine learning for subgroup discovery under treatment effect
Authors: Aleksey Buzmakov
Date: 2019-02-27 00:00:00
Tasks: BIG-bench Machine Learning, Marketing, Subgroup Discovery

In many practical tasks it is needed to estimate an effect of treatment on
individual level. For example, in medicine it is essential to determine the
patients that would benefit from a certain medicament. In marketing, knowing
the persons that are likely to buy a new product would reduce the amount of
spam. In this chapter, we review the methods to estimate an individual
treatment effect from a randomized trial, i.e., an experiment when a part of
individuals receives a new treatment, while the others do not. Finally, it is
shown that new efficient methods are needed in this domain.
CN-Probase: A Data-driven Approach for Large-scale Chinese Taxonomy Construction
Authors: Jindong Chen, Ao Wang, Jiangjie Chen, Yanghua Xiao, Zhendong Chu, Jingping Liu, Jiaqing Liang, Wei Wang
Date: 2019-02-27 00:00:00

Taxonomies play an important role in machine intelligence. However, most
well-known taxonomies are in English, and non-English taxonomies, especially
Chinese ones, are still very rare. In this paper, we focus on automatic Chinese
taxonomy construction and propose an effective generation and verification
framework to build a large-scale and high-quality Chinese taxonomy. In the
generation module, we extract isA relations from multiple sources of Chinese
encyclopedia, which ensures the coverage. To further improve the precision of
taxonomy, we apply three heuristic approaches in verification module. As a
result, we construct the largest Chinese taxonomy with high precision about 95%
called CN-Probase. Our taxonomy has been deployed on Aliyun, with over 82
million API calls in six months.
Neural Packet Classification
Authors: Eric Liang, Hang Zhu, Xin Jin, Ion Stoica
Date: 2019-02-27 00:00:00
Tasks: Classification, Deep Reinforcement Learning, General Classification, Navigate, Reinforcement Learning, Reinforcement Learning (RL)

Packet classification is a fundamental problem in computer networking. This
problem exposes a hard tradeoff between the computation and state complexity,
which makes it particularly challenging. To navigate this tradeoff, existing
solutions rely on complex hand-tuned heuristics, which are brittle and hard to
optimize. In this paper, we propose a deep reinforcement learning (RL) approach
to solve the packet classification problem. There are several characteristics
that make this problem a good fit for Deep RL. First, many of the existing
solutions are iteratively building a decision tree by splitting nodes in the
tree. Second, the effects of these actions (e.g., splitting nodes) can only be
evaluated once we are done with building the tree. These two characteristics
are naturally captured by the ability of RL to take actions that have sparse
and delayed rewards. Third, it is computationally efficient to generate data
traces and evaluate decision trees, which alleviate the notoriously high sample
complexity problem of Deep RL algorithms. Our solution, NeuroCuts, uses
succinct representations to encode state and action space, and efficiently
explore candidate decision trees to optimize for a global objective. It
produces compact decision trees optimized for a specific set of rules and a
given performance metric, such as classification time, memory footprint, or a
combination of the two. Evaluation on ClassBench shows that NeuroCuts
outperforms existing hand-crafted algorithms in classification time by 18% at
the median, and reduces both time and memory footprint by up to 3x.
Aurora Guard: Real-Time Face Anti-Spoofing via Light Reflection
Authors: Yao Liu, Ying Tai, Jilin Li, Shouhong Ding, Chengjie Wang, Feiyue Huang, Dongyang Li, Wenshuai Qi, Rongrong Ji
Date: 2019-02-27 00:00:00
Tasks: Face Anti-Spoofing, General Classification

In this paper, we propose a light reflection based face anti-spoofing method
named Aurora Guard (AG), which is fast, simple yet effective that has already
been deployed in real-world systems serving for millions of users.
Specifically, our method first extracts the normal cues via light reflection
analysis, and then uses an end-to-end trainable multi-task Convolutional Neural
Network (CNN) to not only recover subjects' depth maps to assist liveness
classification, but also provide the light CAPTCHA checking mechanism in the
regression branch to further improve the system reliability. Moreover, we
further collect a large-scale dataset containing $12,000$ live and spoofing
samples, which covers abundant imaging qualities and Presentation Attack
Instruments (PAI). Extensive experiments on both public and our datasets
demonstrate the superiority of our proposed method over the state of the arts.
Deep Adversarial Network Alignment
Authors: Tyler Derr, Hamid Karimi, Xiaorui Liu, Jiejun Xu, Jiliang Tang
Date: 2019-02-27 00:00:00
Tasks: Graph Embedding, Network Embedding

Network alignment, in general, seeks to discover the hidden underlying
correspondence between nodes across two (or more) networks when given their
network structure. However, most existing network alignment methods have added
assumptions of additional constraints to guide the alignment, such as having a
set of seed node-node correspondences across the networks or the existence of
side-information. Instead, we seek to develop a general network alignment
algorithm that makes no additional assumptions. Recently, network embedding has
proven effective in many network analysis tasks, but embeddings of different
networks are not aligned. Thus, we present our Deep Adversarial Network
Alignment (DANA) framework that first uses deep adversarial learning to
discover complex mappings for aligning the embedding distributions of the two
networks. Then, using our learned mapping functions, DANA performs an efficient
nearest neighbor node alignment. We perform experiments on real world datasets
to show the effectiveness of our framework for first aligning the graph
embedding distributions and then discovering node alignments that outperform
existing methods.
Deep Reinforcement Learning for Adaptive Caching in Hierarchical Content Delivery Networks
Authors: Alireza Sadeghi, Gang Wang, Georgios B. Giannakis
Date: 2019-02-27 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Caching is envisioned to play a critical role in next-generation content delivery infrastructure, cellular networks, and Internet architectures. By smartly storing the most popular contents at the storage-enabled network entities during off-peak demand instances, caching can benefit both network infrastructure as well as end users, during on-peak periods. In this context, distributing the limited storage capacity across network entities calls for decentralized caching schemes. Many practical caching systems involve a parent caching node connected to multiple leaf nodes to serve user file requests. To model the two-way interactive influence between caching decisions at the parent and leaf nodes, a reinforcement learning framework is put forth. To handle the large continuous state space, a scalable deep reinforcement learning approach is pursued. The novel approach relies on a deep Q-network to learn the Q-function, and thus the optimal caching policy, in an online fashion. Reinforcing the parent node with ability to learn-and-adapt to unknown policies of leaf nodes as well as spatio-temporal dynamic evolution of file requests, results in remarkable caching performance, as corroborated through numerical tests.