pwc-archive/papers-with-abstracts huggingface dataset:
Artificial Neural Networks-Based Machine Learning for Wireless Networks: A Tutorial
Authors: Mingzhe Chen, Ursula Challita, Walid Saad, Changchuan Yin, Mérouane Debbah
Date: 2017-10-09 00:00:00
Tasks: BIG-bench Machine Learning

Next-generation wireless networks must support ultra-reliable, low-latency communication and intelligently manage a massive number of Internet of Things (IoT) devices in real-time, within a highly dynamic environment. This need for stringent communication quality-of-service (QoS) requirements as well as mobile edge and core intelligence can only be realized by integrating fundamental notions of artificial intelligence (AI) and machine learning across the wireless infrastructure and end-user devices. In this context, this paper provides a comprehensive tutorial that introduces the main concepts of machine learning, in general, and artificial neural networks (ANNs), in particular, and their potential applications in wireless communications. For this purpose, we present a comprehensive overview on a number of key types of neural networks that include feed-forward, recurrent, spiking, and deep neural networks. For each type of neural network, we present the basic architecture and training procedure, as well as the associated challenges and opportunities. Then, we provide an in-depth overview on the variety of wireless communication problems that can be addressed using ANNs, ranging from communication using unmanned aerial vehicles to virtual reality and edge caching.For each individual application, we present the main motivation for using ANNs along with the associated challenges while also providing a detailed example for a use case scenario and outlining future works that can be addressed using ANNs. In a nutshell, this article constitutes one of the first holistic tutorials on the development of machine learning techniques tailored to the needs of future wireless networks.
Response to "Counterexample to global convergence of DSOS and SDSOS hierarchies"
Authors: Amir Ali Ahmadi, Anirudha Majumdar
Date: 2017-10-09 00:00:00

In a recent note [8], the author provides a counterexample to the global
convergence of what his work refers to as "the DSOS and SDSOS hierarchies" for
polynomial optimization problems (POPs) and purports that this refutes claims
in our extended abstract [4] and slides in [3]. The goal of this paper is to
clarify that neither [4], nor [3], and certainly not our full paper [5], ever
defined DSOS or SDSOS hierarchies as it is done in [8]. It goes without saying
that no claims about convergence properties of the hierarchies in [8] were ever
made as a consequence. What was stated in [4,3] was completely different: we
stated that there exist hierarchies based on DSOS and SDSOS optimization that
converge. This is indeed true as we discuss in this response. We also emphasize
that we were well aware that some (S)DSOS hierarchies do not converge even if
their natural SOS counterparts do. This is readily implied by an example in our
prior work [5], which makes the counterexample in [8] superfluous. Finally, we
provide concrete counterarguments to claims made in [8] that aim to challenge
the scalability improvements obtained by DSOS and SDSOS optimization as
compared to sum of squares (SOS) optimization.
  [3] A. A. Ahmadi and A. Majumdar. DSOS and SDSOS: More tractable alternatives
to SOS. Slides at the meeting on Geometry and Algebra of Linear Matrix
Inequalities, CIRM, Marseille, 2013. [4] A. A. Ahmadi and A. Majumdar. DSOS and
SDSOS optimization: LP and SOCP-based alternatives to sum of squares
optimization. In proceedings of the 48th annual IEEE Conference on Information
Sciences and Systems, 2014. [5] A. A. Ahmadi and A. Majumdar. DSOS and SDSOS
optimization: more tractable alternatives to sum of squares and semidefinite
optimization. arXiv:1706.02586, 2017. [8] C. Josz. Counterexample to global
convergence of DSOS and SDSOS hierarchies. arXiv:1707.02964, 2017.
On Matching Skulls to Digital Face Images: A Preliminary Approach
Authors: Shruti Nagpal, Maneet Singh, Arushi Jain, Richa Singh, Mayank Vatsa, Afzel Noore
Date: 2017-10-08 00:00:00
Tasks: Face Recognition

Forensic application of automatically matching skull with face images is an
important research area linking biometrics with practical applications in
forensics. It is an opportunity for biometrics and face recognition researchers
to help the law enforcement and forensic experts in giving an identity to
unidentified human skulls. It is an extremely challenging problem which is
further exacerbated due to lack of any publicly available database related to
this problem. This is the first research in this direction with a two-fold
contribution: (i) introducing the first of its kind skull-face image pair
database, IdentifyMe, and (ii) presenting a preliminary approach using the
proposed semi-supervised formulation of transform learning. The experimental
results and comparison with existing algorithms showcase the challenging nature
of the problem. We assert that the availability of the database will inspire
researchers to build sophisticated skull-to-face matching algorithms.
Clickbait detection using word embeddings
Authors: Vijayasaradhi Indurthi, Subba Reddy Oota
Date: 2017-10-08 00:00:00
Tasks: Clickbait Detection, Feature Engineering, regression, Word Embeddings

Clickbait is a pejorative term describing web content that is aimed at
generating online advertising revenue, especially at the expense of quality or
accuracy, relying on sensationalist headlines or eye-catching thumbnail
pictures to attract click-throughs and to encourage forwarding of the material
over online social networks. We use distributed word representations of the
words in the title as features to identify clickbaits in online news media. We
train a machine learning model using linear regression to predict the cickbait
score of a given tweet. Our methods achieve an F1-score of 64.98\% and an MSE
of 0.0791. Compared to other methods, our method is simple, fast to train, does
not require extensive feature engineering and yet moderately effective.
Gender and Ethnicity Classification of Iris Images using Deep Class-Encoder
Authors: Maneet Singh, Shruti Nagpal, Mayank Vatsa, Richa Singh, Afzel Noore, Angshul Majumdar
Date: 2017-10-08 00:00:00
Tasks: Gender Classification, General Classification

Soft biometric modalities have shown their utility in different applications
including reducing the search space significantly. This leads to improved
recognition performance, reduced computation time, and faster processing of
test samples. Some common soft biometric modalities are ethnicity, gender, age,
hair color, iris color, presence of facial hair or moles, and markers. This
research focuses on performing ethnicity and gender classification on iris
images. We present a novel supervised autoencoder based approach, Deep
Class-Encoder, which uses class labels to learn discriminative representation
for the given sample by mapping the learned feature vector to its label. The
proposed model is evaluated on two datasets each for ethnicity and gender
classification. The results obtained using the proposed Deep Class-Encoder
demonstrate its effectiveness in comparison to existing approaches and
state-of-the-art methods.
Reconstruction of Hidden Representation for Robust Feature Extraction
Authors: Zeng Yu, Tianrui Li, Ning Yu, Yi Pan, Hongmei Chen, Bing Liu
Date: 2017-10-08 00:00:00
Tasks: Denoising, Representation Learning

This paper aims to develop a new and robust approach to feature
representation. Motivated by the success of Auto-Encoders, we first theoretical
summarize the general properties of all algorithms that are based on
traditional Auto-Encoders: 1) The reconstruction error of the input can not be
lower than a lower bound, which can be viewed as a guiding principle for
reconstructing the input. Additionally, when the input is corrupted with
noises, the reconstruction error of the corrupted input also can not be lower
than a lower bound. 2) The reconstruction of a hidden representation achieving
its ideal situation is the necessary condition for the reconstruction of the
input to reach the ideal state. 3) Minimizing the Frobenius norm of the
Jacobian matrix of the hidden representation has a deficiency and may result in
a much worse local optimum value. We believe that minimizing the reconstruction
error of the hidden representation is more robust than minimizing the Frobenius
norm of the Jacobian matrix of the hidden representation. Based on the above
analysis, we propose a new model termed Double Denoising Auto-Encoders (DDAEs),
which uses corruption and reconstruction on both the input and the hidden
representation. We demonstrate that the proposed model is highly flexible and
extensible and has a potentially better capability to learn invariant and
robust feature representations. We also show that our model is more robust than
Denoising Auto-Encoders (DAEs) for dealing with noises or inessential features.
Furthermore, we detail how to train DDAEs with two different pre-training
methods by optimizing the objective function in a combined and separate manner,
respectively. Comparative experiments illustrate that the proposed model is
significantly better for representation learning than the state-of-the-art
models.
RUM: network Representation learning throUgh Multi-level structural information preservation
Authors: Yanlei Yu, Zhiwu Lu, Jiajun Liu, Guoping Zhao, Ji-Rong Wen, Kai Zheng
Date: 2017-10-08 00:00:00
Tasks: Representation Learning

We have witnessed the discovery of many techniques for network representation
learning in recent years, ranging from encoding the context in random walks to
embedding the lower order connections, to finding latent space representations
with auto-encoders. However, existing techniques are looking mostly into the
local structures in a network, while higher-level properties such as global
community structures are often neglected. We propose a novel network
representations learning model framework called RUM (network Representation
learning throUgh Multi-level structural information preservation). In RUM, we
incorporate three essential aspects of a node that capture a network's
characteristics in multiple levels: a node's affiliated local triads, its
neighborhood relationships, and its global community affiliations. Therefore
the framework explicitly and comprehensively preserves the structural
information of a network, extending the encoding process both to the local end
of the structural information spectrum and to the global end. The framework is
also flexible enough to take various community discovery algorithms as its
preprocessor. Empirical results show that the representations learned by RUM
have demonstrated substantial performance advantages in real-life tasks.
Micro-Expression Spotting: A Benchmark
Authors: Xiaopeng Hong, Thuong-Khanh Tran, Guoying Zhao
Date: 2017-10-08 00:00:00
Tasks: Micro-Expression Spotting

Micro-expressions are rapid and involuntary facial expressions, which
indicate the suppressed or concealed emotions. Recently, the research on
automatic micro-expression (ME) spotting obtains increasing attention. ME
spotting is a crucial step prior to further ME analysis tasks. The spotting
results can be used as important cues to assist many other human-oriented tasks
and thus have many potential applications. In this paper, by investigating
existing ME spotting methods, we recognize the immediacy of standardizing the
performance evaluation of micro-expression spotting methods. To this end, we
construct a micro-expression spotting benchmark (MESB). Firstly, we set up a
sliding window based multi-scale evaluation framework. Secondly, we introduce a
series of protocols. Thirdly, we also provide baseline results of popular
methods. The MESB facilitates the research on ME spotting with fairer and more
comprehensive evaluation and also enables to leverage the cutting-edge machine
learning tools widely.
Application of a Hybrid Bi-LSTM-CRF model to the task of Russian Named Entity Recognition
Authors: L. T. Anh, M. Y. Arkhipov, M. S. Burtsev
Date: 2017-09-27 00:00:00
Tasks: Named Entity Recognition (NER), Word Embeddings

Named Entity Recognition (NER) is one of the most common tasks of the natural
language processing. The purpose of NER is to find and classify tokens in text
documents into predefined categories called tags, such as person names,
quantity expressions, percentage expressions, names of locations,
organizations, as well as expression of time, currency and others. Although
there is a number of approaches have been proposed for this task in Russian
language, it still has a substantial potential for the better solutions. In
this work, we studied several deep neural network models starting from vanilla
Bi-directional Long Short-Term Memory (Bi-LSTM) then supplementing it with
Conditional Random Fields (CRF) as well as highway networks and finally adding
external word embeddings. All models were evaluated across three datasets:
Gareev's dataset, Person-1000, FactRuEval-2016. We found that extension of
Bi-LSTM model with CRF significantly increased the quality of predictions.
Encoding input tokens with external word embeddings reduced training time and
allowed to achieve state of the art for the Russian NER task.
Object-oriented Neural Programming (OONP) for Document Understanding
Authors: Zhengdong Lu, Xianggen Liu, Haotian Cui, Yukun Yan, Daqi Zheng
Date: 2017-09-26 00:00:00
Proceeding: ACL 2018 7
Tasks: document understanding, Object, Reinforcement Learning, Reinforcement Learning (RL), Semantic Parsing

We propose Object-oriented Neural Programming (OONP), a framework for
semantically parsing documents in specific domains. Basically, OONP reads a
document and parses it into a predesigned object-oriented data structure
(referred to as ontology in this paper) that reflects the domain-specific
semantics of the document. An OONP parser models semantic parsing as a decision
process: a neural net-based Reader sequentially goes through the document, and
during the process it builds and updates an intermediate ontology to summarize
its partial understanding of the text it covers. OONP supports a rich family of
operations (both symbolic and differentiable) for composing the ontology, and a
big variety of forms (both symbolic and differentiable) for representing the
state and the document. An OONP parser can be trained with supervision of
different forms and strength, including supervised learning (SL) ,
reinforcement learning (RL) and hybrid of the two. Our experiments on both
synthetic and real-world document parsing tasks have shown that OONP can learn
to handle fairly complicated ontology with training data of modest sizes.
On Principal Components Regression, Random Projections, and Column Subsampling
Authors: Martin Slawski
Date: 2017-09-23 00:00:00
Tasks: Dimensionality Reduction, regression

Principal Components Regression (PCR) is a traditional tool for dimension
reduction in linear regression that has been both criticized and defended. One
concern about PCR is that obtaining the leading principal components tends to
be computationally demanding for large data sets. While random projections do
not possess the optimality properties of the leading principal subspace, they
are computationally appealing and hence have become increasingly popular in
recent years. In this paper, we present an analysis showing that for random
projections satisfying a Johnson-Lindenstrauss embedding property, the
prediction error in subsequent regression is close to that of PCR, at the
expense of requiring a slightly large number of random projections than
principal components. Column sub-sampling constitutes an even cheaper way of
randomized dimension reduction outside the class of Johnson-Lindenstrauss
transforms. We provide numerical results based on synthetic and real data as
well as basic theory revealing differences and commonalities in terms of
statistical performance.
Smarnet: Teaching Machines to Read and Comprehend Like Human
Authors: Zheqian Chen, Rongqin Yang, Bin Cao, Zhou Zhao, Deng Cai, Xiaofei He
Date: 2017-10-08 00:00:00
Tasks: Question Answering, Reading Comprehension, TriviaQA

Machine Comprehension (MC) is a challenging task in Natural Language
Processing field, which aims to guide the machine to comprehend a passage and
answer the given question. Many existing approaches on MC task are suffering
the inefficiency in some bottlenecks, such as insufficient lexical
understanding, complex question-passage interaction, incorrect answer
extraction and so on. In this paper, we address these problems from the
viewpoint of how humans deal with reading tests in a scientific way.
Specifically, we first propose a novel lexical gating mechanism to dynamically
combine the words and characters representations. We then guide the machines to
read in an interactive way with attention mechanism and memory network. Finally
we add a checking layer to refine the answer for insurance. The extensive
experiments on two popular datasets SQuAD and TriviaQA show that our method
exceeds considerable performance than most state-of-the-art solutions at the
time of submission.
An Expressive Probabilistic Temporal Logic
Authors: Bruno Woltzenlogel Paleo
Date: 2016-03-24 00:00:00

This paper argues that a combined treatment of probabilities, time and
actions is essential for an appropriate logical account of the notion of
probability; and, based on this intuition, describes an expressive
probabilistic temporal logic for reasoning about actions with uncertain
outcomes. The logic is modal and higher-order: modalities annotated by actions
are used to express possibility and necessity of propositions in the next
states resulting from the actions, and a higher-order function is needed to
express the probability operator. The proposed logic is shown to be an adequate
extension of classical mathematical probability theory, and its expressiveness
is illustrated through the formalization of the Monty Hall problem.
Concept Transfer Learning for Adaptive Language Understanding
Authors: Su Zhu, Kai Yu
Date: 2017-06-03 00:00:00
Proceeding: WS 2018 7
Tasks: Domain Adaptation, Transfer Learning

Concept definition is important in language understanding (LU) adaptation since literal definition difference can easily lead to data sparsity even if different data sets are actually semantically correlated. To address this issue, in this paper, a novel concept transfer learning approach is proposed. Here, substructures within literal concept definition are investigated to reveal the relationship between concepts. A hierarchical semantic representation for concepts is proposed, where a semantic slot is represented as a composition of {\em atomic concepts}. Based on this new hierarchical representation, transfer learning approaches are developed for adaptive LU. The approaches are applied to two tasks: value set mismatch and domain adaptation, and evaluated on two LU benchmarks: ATIS and DSTC 2\&3. Thorough empirical studies validate both the efficiency and effectiveness of the proposed method. In particular, we achieve state-of-the-art performance ($F_1$-score 96.08\%) on ATIS by only using lexicon features.
Protein identification with deep learning: from abc to xyz
Authors: Ngoc Hieu Tran, Zachariah Levine, Lei Xin, Baozhen Shan, Ming Li
Date: 2017-10-08 00:00:00
Tasks: Deep Learning, de novo peptide sequencing

Proteins are the main workhorses of biological functions in a cell, a tissue,
or an organism. Identification and quantification of proteins in a given
sample, e.g. a cell type under normal/disease conditions, are fundamental tasks
for the understanding of human health and disease. In this paper, we present
DeepNovo, a deep learning-based tool to address the problem of protein
identification from tandem mass spectrometry data. The idea was first proposed
in the context of de novo peptide sequencing [1] in which convolutional neural
networks and recurrent neural networks were applied to predict the amino acid
sequence of a peptide from its spectrum, a similar task to generating a caption
from an image. We further develop DeepNovo to perform sequence database search,
the main technique for peptide identification that greatly benefits from
numerous existing protein databases. We combine two modules de novo sequencing
and database search into a single deep learning framework for peptide
identification, and integrate de Bruijn graph assembly technique to offer a
complete solution to reconstruct protein sequences from tandem mass
spectrometry data. This paper describes a comprehensive protocol of DeepNovo
for protein identification, including training neural network models, dynamic
programming search, database querying, estimation of false discovery rate, and
de Bruijn graph assembly. Training and testing data, model implementations, and
comprehensive tutorials in form of IPython notebooks are available in our
GitHub repository (https://github.com/nh2tran/DeepNovo).
A GAMP Based Low Complexity Sparse Bayesian Learning Algorithm
Authors: Maher Al-Shoukairi, Philip Schniter, Bhaskar D. Rao
Date: 2017-03-08 00:00:00

In this paper, we present an algorithm for the sparse signal recovery problem
that incorporates damped Gaussian generalized approximate message passing
(GGAMP) into Expectation-Maximization (EM)-based sparse Bayesian learning
(SBL). In particular, GGAMP is used to implement the E-step in SBL in place of
matrix inversion, leveraging the fact that GGAMP is guaranteed to converge with
appropriate damping. The resulting GGAMP-SBL algorithm is much more robust to
arbitrary measurement matrix $\boldsymbol{A}$ than the standard damped GAMP
algorithm while being much lower complexity than the standard SBL algorithm. We
then extend the approach from the single measurement vector (SMV) case to the
temporally correlated multiple measurement vector (MMV) case, leading to the
GGAMP-TSBL algorithm. We verify the robustness and computational advantages of
the proposed algorithms through numerical experiments.
Keynote: Small Neural Nets Are Beautiful: Enabling Embedded Systems with Small Deep-Neural-Network Architectures
Authors: Forrest Iandola, Kurt Keutzer
Date: 2017-10-07 00:00:00
Tasks: speech-recognition, Speech Recognition

Over the last five years Deep Neural Nets have offered more accurate
solutions to many problems in speech recognition, and computer vision, and
these solutions have surpassed a threshold of acceptability for many
applications. As a result, Deep Neural Networks have supplanted other
approaches to solving problems in these areas, and enabled many new
applications. While the design of Deep Neural Nets is still something of an art
form, in our work we have found basic principles of design space exploration
used to develop embedded microprocessor architectures to be highly applicable
to the design of Deep Neural Net architectures. In particular, we have used
these design principles to create a novel Deep Neural Net called SqueezeNet
that requires as little as 480KB of storage for its model parameters. We have
further integrated all these experiences to develop something of a playbook for
creating small Deep Neural Nets for embedded systems.