pwc-archive/papers-with-abstracts huggingface dataset:
Learning linear structural equation models in polynomial time and sample complexity
Authors: Asish Ghoshal, Jean Honorio
Date: 2017-07-15 00:00:00
Tasks: Causal Inference, Computational Efficiency

The problem of learning structural equation models (SEMs) from data is a
fundamental problem in causal inference. We develop a new algorithm --- which
is computationally and statistically efficient and works in the
high-dimensional regime --- for learning linear SEMs from purely observational
data with arbitrary noise distribution. We consider three aspects of the
problem: identifiability, computational efficiency, and statistical efficiency.
We show that when data is generated from a linear SEM over $p$ nodes and
maximum degree $d$, our algorithm recovers the directed acyclic graph (DAG)
structure of the SEM under an identifiability condition that is more general
than those considered in the literature, and without faithfulness assumptions.
In the population setting, our algorithm recovers the DAG structure in
$\mathcal{O}(p(d^2 + \log p))$ operations. In the finite sample setting, if the
estimated precision matrix is sparse, our algorithm has a smoothed complexity
of $\widetilde{\mathcal{O}}(p^3 + pd^7)$, while if the estimated precision
matrix is dense, our algorithm has a smoothed complexity of
$\widetilde{\mathcal{O}}(p^5)$. For sub-Gaussian noise, we show that our
algorithm has a sample complexity of $\mathcal{O}(\frac{d^8}{\varepsilon^2}
\log (\frac{p}{\sqrt{\delta}}))$ to achieve $\varepsilon$ element-wise additive
error with respect to the true autoregression matrix with probability at most
$1 - \delta$, while for noise with bounded $(4m)$-th moment, with $m$ being a
positive integer, our algorithm has a sample complexity of
$\mathcal{O}(\frac{d^8}{\varepsilon^2} (\frac{p^2}{\delta})^{1/m})$.
Rotations and Interpretability of Word Embeddings: the Case of the Russian Language
Authors: Alexey Zobnin
Date: 2017-07-14 00:00:00
Tasks: Word Embeddings

Consider a continuous word embedding model. Usually, the cosines between word
vectors are used as a measure of similarity of words. These cosines do not
change under orthogonal transformations of the embedding space. We demonstrate
that, using some canonical orthogonal transformations from SVD, it is possible
both to increase the meaning of some components and to make the components more
stable under re-learning. We study the interpretability of components for
publicly available models for the Russian language (RusVectores, fastText,
RDT).
A Semantics-Based Measure of Emoji Similarity
Authors: Sanjaya Wijeratne, Lakshika Balasuriya, Amit Sheth, Derek Doran
Date: 2017-07-14 00:00:00
Tasks: Semantic Similarity, Semantic Textual Similarity, Sentiment Analysis

Emoji have grown to become one of the most important forms of communication
on the web. With its widespread use, measuring the similarity of emoji has
become an important problem for contemporary text processing since it lies at
the heart of sentiment analysis, search, and interface design tasks. This paper
presents a comprehensive analysis of the semantic similarity of emoji through
embedding models that are learned over machine-readable emoji meanings in the
EmojiNet knowledge base. Using emoji descriptions, emoji sense labels and emoji
sense definitions, and with different training corpora obtained from Twitter
and Google News, we develop and test multiple embedding models to measure emoji
similarity. To evaluate our work, we create a new dataset called EmoSim508,
which assigns human-annotated semantic similarity scores to a set of 508
carefully selected emoji pairs. After validation with EmoSim508, we present a
real-world use-case of our emoji embedding models using a sentiment analysis
task and show that our models outperform the previous best-performing emoji
embedding model on this task. The EmoSim508 dataset and our emoji embedding
models are publicly released with this paper and can be downloaded from
http://emojinet.knoesis.org/.
EmojiNet: An Open Service and API for Emoji Sense Discovery
Authors: Sanjaya Wijeratne, Lakshika Balasuriya, Amit Sheth, Derek Doran
Date: 2017-07-14 00:00:00

This paper presents the release of EmojiNet, the largest machine-readable
emoji sense inventory that links Unicode emoji representations to their English
meanings extracted from the Web. EmojiNet is a dataset consisting of: (i)
12,904 sense labels over 2,389 emoji, which were extracted from the web and
linked to machine-readable sense definitions seen in BabelNet, (ii) context
words associated with each emoji sense, which are inferred through word
embedding models trained over Google News corpus and a Twitter message corpus
for each emoji sense definition, and (iii) recognizing discrepancies in the
presentation of emoji on different platforms, specification of the most likely
platform-based emoji sense for a selected set of emoji. The dataset is hosted
as an open service with a REST API and is available at
http://emojinet.knoesis.org/. The development of this dataset, evaluation of
its quality, and its applications including emoji sense disambiguation and
emoji sense similarity are discussed.
Predicting multicellular function through multi-layer tissue networks
Authors: Marinka Zitnik, Jure Leskovec
Date: 2017-07-14 00:00:00

Motivation: Understanding functions of proteins in specific human tissues is
essential for insights into disease diagnostics and therapeutics, yet
prediction of tissue-specific cellular function remains a critical challenge
for biomedicine.
  Results: Here we present OhmNet, a hierarchy-aware unsupervised node feature
learning approach for multi-layer networks. We build a multi-layer network,
where each layer represents molecular interactions in a different human tissue.
OhmNet then automatically learns a mapping of proteins, represented as nodes,
to a neural embedding based low-dimensional space of features. OhmNet
encourages sharing of similar features among proteins with similar network
neighborhoods and among proteins activated in similar tissues. The algorithm
generalizes prior work, which generally ignores relationships between tissues,
by modeling tissue organization with a rich multiscale tissue hierarchy. We use
OhmNet to study multicellular function in a multi-layer protein interaction
network of 107 human tissues. In 48 tissues with known tissue-specific cellular
functions, OhmNet provides more accurate predictions of cellular function than
alternative approaches, and also generates more accurate hypotheses about
tissue-specific protein actions. We show that taking into account the tissue
hierarchy leads to improved predictive power. Remarkably, we also demonstrate
that it is possible to leverage the tissue hierarchy in order to effectively
transfer cellular functions to a functionally uncharacterized tissue. Overall,
OhmNet moves from flat networks to multiscale models able to predict a range of
phenotypes spanning cellular subsystems
Simplified Long Short-term Memory Recurrent Neural Networks: part III
Authors: Atra Akandeh, Fathi M. Salem
Date: 2017-07-14 00:00:00

This is part III of three-part work. In parts I and II, we have presented
eight variants for simplified Long Short Term Memory (LSTM) recurrent neural
networks (RNNs). It is noted that fast computation, specially in constrained
computing resources, are an important factor in processing big time-sequence
data. In this part III paper, we present and evaluate two new LSTM model
variants which dramatically reduce the computational load while retaining
comparable performance to the base (standard) LSTM RNNs. In these new variants,
we impose (Hadamard) pointwise state multiplications in the cell-memory network
in addition to the gating signal networks.
Progress Estimation and Phase Detection for Sequential Processes
Authors: Xinyu Li, Yanyi Zhang, Jianyu Zhang, Yueyang Chen, Shuhong Chen, Yue Gu, Moliang Zhou, Richard A. Farneth, Ivan Marsic, Randall S. Burd
Date: 2017-02-28 00:00:00
Tasks: Activity Recognition, Multimodal Deep Learning

Process modeling and understanding are fundamental for advanced
human-computer interfaces and automation systems. Most recent research has
focused on activity recognition, but little has been done on sensor-based
detection of process progress. We introduce a real-time, sensor-based system
for modeling, recognizing and estimating the progress of a work process. We
implemented a multimodal deep learning structure to extract the relevant
spatio-temporal features from multiple sensory inputs and used a novel deep
regression structure for overall completeness estimation. Using process
completeness estimation with a Gaussian mixture model, our system can predict
the phase for sequential processes. The performance speed, calculated using
completeness estimation, allows online estimation of the remaining time. To
train our system, we introduced a novel rectified hyperbolic tangent (rtanh)
activation function and conditional loss. Our system was tested on data
obtained from the medical process (trauma resuscitation) and sports events
(Olympic swimming competition). Our system outperformed the existing
trauma-resuscitation phase detectors with a phase detection accuracy of over
86%, an F1-score of 0.67, a completeness estimation error of under 12.6%, and a
remaining-time estimation error of less than 7.5 minutes. For the Olympic
swimming dataset, our system achieved an accuracy of 88%, an F1-score of 0.58,
a completeness estimation error of 6.3% and a remaining-time estimation error
of 2.9 minutes.
Simplified Long Short-term Memory Recurrent Neural Networks: part II
Authors: Atra Akandeh, Fathi M. Salem
Date: 2017-07-14 00:00:00

This is part II of three-part work. Here, we present a second set of
inter-related five variants of simplified Long Short-term Memory (LSTM)
recurrent neural networks by further reducing adaptive parameters. Two of these
models have been introduced in part I of this work. We evaluate and verify our
model variants on the benchmark MNIST dataset and assert that these models are
comparable to the base LSTM model while use progressively less number of
parameters. Moreover, we observe that in case of using the ReLU activation, the
test accuracy performance of the standard LSTM will drop after a number of
epochs when learning parameter become larger. However all of the new model
variants sustain their performance.
Robust Spatial Filtering with Graph Convolutional Neural Networks
Authors: Felipe Petroski Such, Shagan Sah, Miguel Dominguez, Suhas Pillai, Chao Zhang, Andrew Michael, Nathan Cahill, Raymond Ptucha
Date: 2017-03-02 00:00:00
Tasks: General Classification

Convolutional Neural Networks (CNNs) have recently led to incredible
breakthroughs on a variety of pattern recognition problems. Banks of finite
impulse response filters are learned on a hierarchy of layers, each
contributing more abstract information than the previous layer. The simplicity
and elegance of the convolutional filtering process makes them perfect for
structured problems such as image, video, or voice, where vertices are
homogeneous in the sense of number, location, and strength of neighbors. The
vast majority of classification problems, for example in the pharmaceutical,
homeland security, and financial domains are unstructured. As these problems
are formulated into unstructured graphs, the heterogeneity of these problems,
such as number of vertices, number of connections per vertex, and edge
strength, cannot be tackled with standard convolutional techniques. We propose
a novel neural learning framework that is capable of handling both homogeneous
and heterogeneous data, while retaining the benefits of traditional CNN
successes.
  Recently, researchers have proposed variations of CNNs that can handle graph
data. In an effort to create learnable filter banks of graphs, these methods
either induce constraints on the data or require preprocessing. As opposed to
spectral methods, our framework, which we term Graph-CNNs, defines filters as
polynomials of functions of the graph adjacency matrix. Graph-CNNs can handle
both heterogeneous and homogeneous graph data, including graphs having entirely
different vertex or edge sets. We perform experiments to validate the
applicability of Graph-CNNs to a variety of structured and unstructured
classification problems and demonstrate state-of-the-art results on document
and molecule classification problems.
Semi-supervised Clustering for Short Text via Deep Representation Learning
Authors: Zhiguo Wang, Haitao Mi, Abraham Ittycheriah
Date: 2016-02-22 00:00:00
Proceeding: CONLL 2016 8
Tasks: Clustering, Representation Learning, Short Text Clustering, Text Clustering

In this work, we propose a semi-supervised method for short text clustering,
where we represent texts as distributed vectors with neural networks, and use a
small amount of labeled data to specify our intention for clustering. We design
a novel objective to combine the representation learning process and the
k-means clustering process together, and optimize the objective with both
labeled data and unlabeled data iteratively until convergence through three
steps: (1) assign each short text to its nearest centroid based on its
representation from the current neural networks; (2) re-estimate the cluster
centroids based on cluster assignments from step (1); (3) update neural
networks according to the objective by keeping centroids and cluster
assignments fixed. Experimental results on four datasets show that our method
works significantly better than several other text clustering methods.
Sentence Similarity Learning by Lexical Decomposition and Composition
Authors: Zhiguo Wang, Haitao Mi, Abraham Ittycheriah
Date: 2016-02-23 00:00:00
Proceeding: COLING 2016 12
Tasks: Paraphrase Identification, Question Answering, Sentence, Sentence Similarity

Most conventional sentence similarity methods only focus on similar parts of
two input sentences, and simply ignore the dissimilar parts, which usually give
us some clues and semantic meanings about the sentences. In this work, we
propose a model to take into account both the similarities and dissimilarities
by decomposing and composing lexical semantics over sentences. The model
represents each word as a vector, and calculates a semantic matching vector for
each word based on all words in the other sentence. Then, each word vector is
decomposed into a similar component and a dissimilar component based on the
semantic matching vector. After this, a two-channel CNN model is employed to
capture features by composing the similar and dissimilar components. Finally, a
similarity score is estimated over the composed feature vectors. Experimental
results show that our model gets the state-of-the-art performance on the answer
sentence selection task, and achieves a comparable result on the paraphrase
identification task.
Simplified Long Short-term Memory Recurrent Neural Networks: part I
Authors: Atra Akandeh, Fathi M. Salem
Date: 2017-07-14 00:00:00

We present five variants of the standard Long Short-term Memory (LSTM)
recurrent neural networks by uniformly reducing blocks of adaptive parameters
in the gating mechanisms. For simplicity, we refer to these models as LSTM1,
LSTM2, LSTM3, LSTM4, and LSTM5, respectively. Such parameter-reduced variants
enable speeding up data training computations and would be more suitable for
implementations onto constrained embedded platforms. We comparatively evaluate
and verify our five variant models on the classical MNIST dataset and
demonstrate that these variant models are comparable to a standard
implementation of the LSTM model while using less number of parameters.
Moreover, we observe that in some cases the standard LSTM's accuracy
performance will drop after a number of epochs when using the ReLU
nonlinearity; in contrast, however, LSTM3, LSTM4 and LSTM5 will retain their
performance.
Bilateral Multi-Perspective Matching for Natural Language Sentences
Authors: Zhiguo Wang, Wael Hamza, Radu Florian
Date: 2017-02-13 00:00:00
Tasks: Natural Language Inference, Paraphrase Identification, Sentence

Natural language sentence matching is a fundamental technology for a variety
of tasks. Previous approaches either match sentences from a single direction or
only apply single granular (word-by-word or sentence-by-sentence) matching. In
this work, we propose a bilateral multi-perspective matching (BiMPM) model
under the "matching-aggregation" framework. Given two sentences $P$ and $Q$,
our model first encodes them with a BiLSTM encoder. Next, we match the two
encoded sentences in two directions $P \rightarrow Q$ and $P \leftarrow Q$. In
each matching direction, each time step of one sentence is matched against all
time-steps of the other sentence from multiple perspectives. Then, another
BiLSTM layer is utilized to aggregate the matching results into a fix-length
matching vector. Finally, based on the matching vector, the decision is made
through a fully connected layer. We evaluate our model on three tasks:
paraphrase identification, natural language inference and answer sentence
selection. Experimental results on standard benchmark datasets show that our
model achieves the state-of-the-art performance on all tasks.
On the Complexity of Learning Neural Networks
Authors: Le Song, Santosh Vempala, John Wilmes, Bo Xie
Date: 2017-07-14 00:00:00
Proceeding: NeurIPS 2017 12

The stunning empirical successes of neural networks currently lack rigorous
theoretical explanation. What form would such an explanation take, in the face
of existing complexity-theoretic lower bounds? A first step might be to show
that data generated by neural networks with a single hidden layer, smooth
activation functions and benign input distributions can be learned efficiently.
We demonstrate here a comprehensive lower bound ruling out this possibility:
for a wide class of activation functions (including all currently used), and
inputs drawn from any logconcave distribution, there is a family of
one-hidden-layer functions whose output is a sum gate, that are hard to learn
in a precise sense: any statistical query algorithm (which includes all known
variants of stochastic gradient descent with any loss function) needs an
exponential number of queries even using tolerance inversely proportional to
the input dimensionality. Moreover, this hard family of functions is realizable
with a small (sublinear in dimension) number of activation units in the single
hidden layer. The lower bound is also robust to small perturbations of the true
weights. Systematic experiments illustrate a phase transition in the training
error as predicted by the analysis.
Knowledge will Propel Machine Understanding of Content: Extrapolating from Current Examples
Authors: Amit Sheth, Sujan Perera, Sanjaya Wijeratne, Krishnaprasad Thirunarayan
Date: 2017-07-14 00:00:00

Machine Learning has been a big success story during the AI resurgence. One
particular stand out success relates to learning from a massive amount of data.
In spite of early assertions of the unreasonable effectiveness of data, there
is increasing recognition for utilizing knowledge whenever it is available or
can be created purposefully. In this paper, we discuss the indispensable role
of knowledge for deeper understanding of content where (i) large amounts of
training data are unavailable, (ii) the objects to be recognized are complex,
(e.g., implicit entities and highly subjective content), and (iii) applications
need to use complementary or related data in multiple modalities/media. What
brings us to the cusp of rapid progress is our ability to (a) create relevant
and reliable knowledge and (b) carefully exploit knowledge to enhance ML/NLP
techniques. Using diverse examples, we seek to foretell unprecedented progress
in our ability for deeper understanding and exploitation of multimodal data and
continued incorporation of knowledge in learning techniques.
Deep GrabCut for Object Selection
Authors: Ning Xu, Brian Price, Scott Cohen, Jimei Yang, Thomas Huang
Date: 2017-07-02 00:00:00
Tasks: Decoder, Instance Segmentation, Interactive Segmentation, Object, Segmentation, Semantic Segmentation

Most previous bounding-box-based segmentation methods assume the bounding box
tightly covers the object of interest. However it is common that a rectangle
input could be too large or too small. In this paper, we propose a novel
segmentation approach that uses a rectangle as a soft constraint by
transforming it into an Euclidean distance map. A convolutional encoder-decoder
network is trained end-to-end by concatenating images with these distance maps
as inputs and predicting the object masks as outputs. Our approach gets
accurate segmentation results given sloppy rectangles while being general for
both interactive segmentation and instance segmentation. We show our network
extends to curve-based input without retraining. We further apply our network
to instance-level semantic segmentation and resolve any overlap using a
conditional random field. Experiments on benchmark datasets demonstrate the
effectiveness of the proposed approaches.
Transforming Sensor Data to the Image Domain for Deep Learning - an Application to Footstep Detection
Authors: Monit Shah Singh, Vinaychandran Pondenkandath, Bo Zhou, Paul Lukowicz, Marcus Liwicki
Date: 2017-01-04 00:00:00
Tasks: General Classification, Transfer Learning

Convolutional Neural Networks (CNNs) have become the state-of-the-art in
various computer vision tasks, but they are still premature for most sensor
data, especially in pervasive and wearable computing. A major reason for this
is the limited amount of annotated training data. In this paper, we propose the
idea of leveraging the discriminative power of pre-trained deep CNNs on
2-dimensional sensor data by transforming the sensor modality to the visual
domain. By three proposed strategies, 2D sensor output is converted into
pressure distribution imageries. Then we utilize a pre-trained CNN for transfer
learning on the converted imagery data. We evaluate our method on a gait
dataset of floor surface pressure mapping. We obtain a classification accuracy
of 87.66%, which outperforms the conventional machine learning methods by over
10%.
DocTag2Vec: An Embedding Based Multi-label Learning Approach for Document Tagging
Authors: Sheng Chen, Akshay Soni, Aasish Pappu, Yashar Mehdad
Date: 2017-07-14 00:00:00
Proceeding: WS 2017 8
Tasks: Articles, Multi-Label Learning, TAG

Tagging news articles or blog posts with relevant tags from a collection of
predefined ones is coined as document tagging in this work. Accurate tagging of
articles can benefit several downstream applications such as recommendation and
search. In this work, we propose a novel yet simple approach called DocTag2Vec
to accomplish this task. We substantially extend Word2Vec and Doc2Vec---two
popular models for learning distributed representation of words and documents.
In DocTag2Vec, we simultaneously learn the representation of words, documents,
and tags in a joint vector space during training, and employ the simple
$k$-nearest neighbor search to predict tags for unseen documents. In contrast
to previous multi-label learning methods, DocTag2Vec directly deals with raw
text instead of provided feature vector, and in addition, enjoys advantages
like the learning of tag representation, and the ability of handling newly
created tags. To demonstrate the effectiveness of our approach, we conduct
experiments on several datasets and show promising results against
state-of-the-art methods.