pwc-archive/papers-with-abstracts huggingface dataset:
Item Silk Road: Recommending Items from Information Domains to Social Users
Authors: Xiang Wang, Xiangnan He, Liqiang Nie, Tat-Seng Chua
Date: 2017-06-10 00:00:00
Tasks: Collaborative Ranking, Recommendation Systems

Online platforms can be divided into information-oriented and social-oriented
domains. The former refers to forums or E-commerce sites that emphasize
user-item interactions, like Trip.com and Amazon; whereas the latter refers to
social networking services (SNSs) that have rich user-user connections, such as
Facebook and Twitter. Despite their heterogeneity, these two domains can be
bridged by a few overlapping users, dubbed as bridge users. In this work, we
address the problem of cross-domain social recommendation, i.e., recommending
relevant items of information domains to potential users of social networks. To
our knowledge, this is a new problem that has rarely been studied before.
  Existing cross-domain recommender systems are unsuitable for this task since
they have either focused on homogeneous information domains or assumed that
users are fully overlapped. Towards this end, we present a novel Neural Social
Collaborative Ranking (NSCR) approach, which seamlessly sews up the user-item
interactions in information domains and user-user connections in SNSs. In the
information domain part, the attributes of users and items are leveraged to
strengthen the embedding learning of users and items. In the SNS part, the
embeddings of bridge users are propagated to learn the embeddings of other
non-bridge users. Extensive experiments on two real-world datasets demonstrate
the effectiveness and rationality of our NSCR method.
Critical Hyper-Parameters: No Random, No Cry
Authors: Olivier Bousquet, Sylvain Gelly, Karol Kurach, Olivier Teytaud, Damien Vincent
Date: 2017-06-10 00:00:00
Tasks: Deep Learning, image-classification, Image Classification

The selection of hyper-parameters is critical in Deep Learning. Because of
the long training time of complex models and the availability of compute
resources in the cloud, "one-shot" optimization schemes - where the sets of
hyper-parameters are selected in advance (e.g. on a grid or in a random manner)
and the training is executed in parallel - are commonly used. It is known that
grid search is sub-optimal, especially when only a few critical parameters
matter, and suggest to use random search instead. Yet, random search can be
"unlucky" and produce sets of values that leave some part of the domain
unexplored. Quasi-random methods, such as Low Discrepancy Sequences (LDS) avoid
these issues. We show that such methods have theoretical properties that make
them appealing for performing hyperparameter search, and demonstrate that, when
applied to the selection of hyperparameters of complex Deep Learning models
(such as state-of-the-art LSTM language models and image classification
models), they yield suitable hyperparameters values with much fewer runs than
random search. We propose a particularly simple LDS method which can be used as
a drop-in replacement for grid or random search in any Deep Learning pipeline,
both as a fully one-shot hyperparameter search or as an initializer in
iterative batch optimization.
Online Learning for Neural Machine Translation Post-editing
Authors: Álvaro Peris, Luis Cebrián, Francisco Casacuberta
Date: 2017-06-10 00:00:00
Tasks: Machine Translation, Translation

Neural machine translation has meant a revolution of the field. Nevertheless,
post-editing the outputs of the system is mandatory for tasks requiring high
translation quality. Post-editing offers a unique opportunity for improving
neural machine translation systems, using online learning techniques and
treating the post-edited translations as new, fresh training data. We review
classical learning methods and propose a new optimization algorithm. We
thoroughly compare online learning algorithms in a post-editing scenario.
Results show significant improvements in translation quality and effort
reduction.
Classification of Questions and Learning Outcome Statements (LOS) Into Blooms Taxonomy (BT) By Similarity Measurements Towards Extracting Of Learning Outcome from Learning Material
Authors: Shadi Diab, Badie Sartawi
Date: 2017-06-10 00:00:00
Tasks: General Classification

Blooms Taxonomy (BT) have been used to classify the objectives of learning
outcome by dividing the learning into three different domains; the cognitive
domain, the effective domain and the psychomotor domain. In this paper, we are
introducing a new approach to classify the questions and learning outcome
statements (LOS) into Blooms taxonomy (BT) and to verify BT verb lists, which
are being cited and used by academicians to write questions and (LOS). An
experiment was designed to investigate the semantic relationship between the
action verbs used in both questions and LOS to obtain more accurate
classification of the levels of BT. A sample of 775 different action verbs
collected from different universities allows us to measure an accurate and
clear-cut cognitive level for the action verb. It is worth mentioning that
natural language processing techniques were used to develop our rules as to
induce the questions into chunks in order to extract the action verbs. Our
proposed solution was able to classify the action verb into a precise level of
the cognitive domain. We, on our side, have tested and evaluated our proposed
solution using confusion matrix. The results of evaluation tests yielded 97%
for the macro average of precision and 90% for F1. Thus, the outcome of the
research suggests that it is crucial to analyse and verify the action verbs
cited and used by academicians to write LOS and classify their questions based
on blooms taxonomy in order to obtain a definite and more accurate
classification.
Asynchronous Parallel Stochastic Gradient for Nonconvex Optimization
Authors: Xiangru Lian, Yijun Huang, Yuncheng Li, Ji Liu
Date: 2015-06-27 00:00:00
Proceeding: NeurIPS 2015 12

Asynchronous parallel implementations of stochastic gradient (SG) have been
broadly used in solving deep neural network and received many successes in
practice recently. However, existing theories cannot explain their convergence
and speedup properties, mainly due to the nonconvexity of most deep learning
formulations and the asynchronous parallel mechanism. To fill the gaps in
theory and provide theoretical supports, this paper studies two asynchronous
parallel implementations of SG: one is on the computer network and the other is
on the shared memory system. We establish an ergodic convergence rate
$O(1/\sqrt{K})$ for both algorithms and prove that the linear speedup is
achievable if the number of workers is bounded by $\sqrt{K}$ ($K$ is the total
number of iterations). Our results generalize and improve existing analysis for
convex minimization.
Direct detection of pixel-level myocardial infarction areas via a deep-learning algorithm
Authors: Chenchu Xu, Lei Xu, Zhifan Gao, Shen zhao, Heye Zhang, Yanping Zhang, Xiuquan Du, Shu Zhao, Dhanjoo Ghista, Shuo Li
Date: 2017-06-10 00:00:00
Tasks: Management, Time Series Analysis

Accurate detection of the myocardial infarction (MI) area is crucial for
early diagnosis planning and follow-up management. In this study, we propose an
end-to-end deep-learning algorithm framework (OF-RNN ) to accurately detect the
MI area at the pixel level. Our OF-RNN consists of three different function
layers: the heart localization layers, which can accurately and automatically
crop the region-of-interest (ROI) sequences, including the left ventricle,
using the whole cardiac magnetic resonance image sequences; the motion
statistical layers, which are used to build a time-series architecture to
capture two types of motion features (at the pixel-level) by integrating the
local motion features generated by long short-term memory-recurrent neural
networks and the global motion features generated by deep optical flows from
the whole ROI sequence, which can effectively characterize myocardial
physiologic function; and the fully connected discriminate layers, which use
stacked auto-encoders to further learn these features, and they use a softmax
classifier to build the correspondences from the motion features to the tissue
identities (infarction or not) for each pixel. Through the seamless connection
of each layer, our OF-RNN can obtain the area, position, and shape of the MI
for each patient. Our proposed framework yielded an overall classification
accuracy of 94.35% at the pixel level, from 114 clinical subjects. These
results indicate the potential of our proposed method in aiding standardized MI
assessments.
Recovery Guarantees for One-hidden-layer Neural Networks
Authors: Kai Zhong, Zhao Song, Prateek Jain, Peter L. Bartlett, Inderjit S. Dhillon
Date: 2017-06-10 00:00:00
Proceeding: ICML 2017 8

In this paper, we consider regression problems with one-hidden-layer neural
networks (1NNs). We distill some properties of activation functions that lead
to $\mathit{local~strong~convexity}$ in the neighborhood of the ground-truth
parameters for the 1NN squared-loss objective. Most popular nonlinear
activation functions satisfy the distilled properties, including rectified
linear units (ReLUs), leaky ReLUs, squared ReLUs and sigmoids. For activation
functions that are also smooth, we show $\mathit{local~linear~convergence}$
guarantees of gradient descent under a resampling rule. For homogeneous
activations, we show tensor methods are able to initialize the parameters to
fall into the local strong convexity region. As a result, tensor initialization
followed by gradient descent is guaranteed to recover the ground truth with
sample complexity $ d \cdot \log(1/\epsilon) \cdot \mathrm{poly}(k,\lambda )$
and computational complexity $n\cdot d \cdot \mathrm{poly}(k,\lambda) $ for
smooth homogeneous activations with high probability, where $d$ is the
dimension of the input, $k$ ($k\leq d$) is the number of hidden nodes,
$\lambda$ is a conditioning property of the ground-truth parameter matrix
between the input layer and the hidden layer, $\epsilon$ is the targeted
precision and $n$ is the number of samples. To the best of our knowledge, this
is the first work that provides recovery guarantees for 1NNs with both sample
complexity and computational complexity $\mathit{linear}$ in the input
dimension and $\mathit{logarithmic}$ in the precision.
Bio-Inspired Multi-Layer Spiking Neural Network Extracts Discriminative Features from Speech Signals
Authors: Amirhossein Tavanaei, Anthony Maida
Date: 2017-06-10 00:00:00

Spiking neural networks (SNNs) enable power-efficient implementations due to
their sparse, spike-based coding scheme. This paper develops a bio-inspired SNN
that uses unsupervised learning to extract discriminative features from speech
signals, which can subsequently be used in a classifier. The architecture
consists of a spiking convolutional/pooling layer followed by a fully connected
spiking layer for feature discovery. The convolutional layer of leaky,
integrate-and-fire (LIF) neurons represents primary acoustic features. The
fully connected layer is equipped with a probabilistic spike-timing-dependent
plasticity learning rule. This layer represents the discriminative features
through probabilistic, LIF neurons. To assess the discriminative power of the
learned features, they are used in a hidden Markov model (HMM) for spoken digit
recognition. The experimental results show performance above 96% that compares
favorably with popular statistical feature extraction methods. Our results
provide a novel demonstration of unsupervised feature acquisition in an SNN.
Network Inference via the Time-Varying Graphical Lasso
Authors: David Hallac, Youngsuk Park, Stephen Boyd, Jure Leskovec
Date: 2017-03-06 00:00:00
Tasks: Time Series, Time Series Analysis

Many important problems can be modeled as a system of interconnected
entities, where each entity is recording time-dependent observations or
measurements. In order to spot trends, detect anomalies, and interpret the
temporal dynamics of such data, it is essential to understand the relationships
between the different entities and how these relationships evolve over time. In
this paper, we introduce the time-varying graphical lasso (TVGL), a method of
inferring time-varying networks from raw time series data. We cast the problem
in terms of estimating a sparse time-varying inverse covariance matrix, which
reveals a dynamic network of interdependencies between the entities. Since
dynamic network inference is a computationally expensive task, we derive a
scalable message-passing algorithm based on the Alternating Direction Method of
Multipliers (ADMM) to solve this problem in an efficient way. We also discuss
several extensions, including a streaming algorithm to update the model and
incorporate new observations in real time. Finally, we evaluate our TVGL
algorithm on both real and synthetic datasets, obtaining interpretable results
and outperforming state-of-the-art baselines in terms of both accuracy and
scalability.
Bayesian Approximate Kernel Regression with Variable Selection
Authors: Lorin Crawford, Kris C. Wood, Xiang Zhou, Sayan Mukherjee
Date: 2015-08-05 00:00:00
Tasks: Binary Classification, regression, Variable Selection

Nonlinear kernel regression models are often used in statistics and machine
learning because they are more accurate than linear models. Variable selection
for kernel regression models is a challenge partly because, unlike the linear
regression setting, there is no clear concept of an effect size for regression
coefficients. In this paper, we propose a novel framework that provides an
effect size analog of each explanatory variable for Bayesian kernel regression
models when the kernel is shift-invariant --- for example, the Gaussian kernel.
We use function analytic properties of shift-invariant reproducing kernel
Hilbert spaces (RKHS) to define a linear vector space that: (i) captures
nonlinear structure, and (ii) can be projected onto the original explanatory
variables. The projection onto the original explanatory variables serves as an
analog of effect sizes. The specific function analytic property we use is that
shift-invariant kernel functions can be approximated via random Fourier bases.
Based on the random Fourier expansion we propose a computationally efficient
class of Bayesian approximate kernel regression (BAKR) models for both
nonlinear regression and binary classification for which one can compute an
analog of effect sizes. We illustrate the utility of BAKR by examining two
important problems in statistical genetics: genomic selection (i.e. phenotypic
prediction) and association mapping (i.e. inference of significant variants or
loci). State-of-the-art methods for genomic selection and association mapping
are based on kernel regression and linear models, respectively. BAKR is the
first method that is competitive in both settings.
Reasoning About Liquids via Closed-Loop Simulation
Authors: Connor Schenck, Dieter Fox
Date: 2017-03-05 00:00:00
Tasks: Liquid Simulation

Simulators are powerful tools for reasoning about a robot's interactions with
its environment. However, when simulations diverge from reality, that reasoning
becomes less useful. In this paper, we show how to close the loop between
liquid simulation and real-time perception. We use observations of liquids to
correct errors when tracking the liquid's state in a simulator. Our results
show that closed-loop simulation is an effective way to prevent large
divergence between the simulated and real liquid states. As a direct
consequence of this, our method can enable reasoning about liquids that would
otherwise be infeasible due to large divergences, such as reasoning about
occluded liquid.
Trimming and Improving Skip-thought Vectors
Authors: Shuai Tang, Hailin Jin, Chen Fang, Zhaowen Wang, Virginia R. de Sa
Date: 2017-06-09 00:00:00
Tasks: Decoder, Sentence, text-classification, Text Classification

The skip-thought model has been proven to be effective at learning sentence
representations and capturing sentence semantics. In this paper, we propose a
suite of techniques to trim and improve it. First, we validate a hypothesis
that, given a current sentence, inferring the previous and inferring the next
sentence provide similar supervision power, therefore only one decoder for
predicting the next sentence is preserved in our trimmed skip-thought model.
Second, we present a connection layer between encoder and decoder to help the
model to generalize better on semantic relatedness tasks. Third, we found that
a good word embedding initialization is also essential for learning better
sentence representations. We train our model unsupervised on a large corpus
with contiguous sentences, and then evaluate the trained model on 7 supervised
tasks, which includes semantic relatedness, paraphrase detection, and text
classification benchmarks. We empirically show that, our proposed model is a
faster, lighter-weight and equally powerful alternative to the original
skip-thought model.
Rethinking Skip-thought: A Neighborhood based Approach
Authors: Shuai Tang, Hailin Jin, Chen Fang, Zhaowen Wang, Virginia R. de Sa
Date: 2017-06-09 00:00:00
Proceeding: WS 2017 8
Tasks: General Classification

We study the skip-thought model with neighborhood information as weak
supervision. More specifically, we propose a skip-thought neighbor model to
consider the adjacent sentences as a neighborhood. We train our skip-thought
neighbor model on a large corpus with continuous sentences, and then evaluate
the trained model on 7 tasks, which include semantic relatedness, paraphrase
detection, and classification benchmarks. Both quantitative comparison and
qualitative investigation are conducted. We empirically show that, our
skip-thought neighbor model performs as well as the skip-thought model on
evaluation tasks. In addition, we found that, incorporating an autoencoder path
in our model didn't aid our model to perform better, while it hurts the
performance of the skip-thought model.
A Focal Any-Angle Path-finding Algorithm Based on A* on Visibility Graphs
Authors: Pei Cao, Zhaoyan Fan, Robert X. Gao, Jiong Tang
Date: 2017-06-09 00:00:00

In this research, we investigate the subject of path-finding. A pruned
version of visibility graph based on Candidate Vertices is formulated, followed
by a new visibility check technique. Such combination enables us to quickly
identify the useful vertices and thus find the optimal path more efficiently.
The algorithm proposed is demonstrated on various path-finding cases. The
performance of the new technique on visibility graphs is compared to the
traditional A* on Grids, Theta* and A* on Visibility Graphs in terms of path
length, number of nodes evaluated, as well as computational time. The key
algorithmic contribution is that the new approach combines the merits of
grid-based method and visibility graph-based method and thus yields better
overall performance.
Deep Learning for Isotropic Super-Resolution from Non-Isotropic 3D Electron Microscopy
Authors: Larissa Heinrich, John A. Bogovic, Stephan Saalfeld
Date: 2017-06-09 00:00:00
Tasks: Super-Resolution

The most sophisticated existing methods to generate 3D isotropic
super-resolution (SR) from non-isotropic electron microscopy (EM) are based on
learned dictionaries. Unfortunately, none of the existing methods generate
practically satisfying results. For 2D natural images, recently developed
super-resolution methods that use deep learning have been shown to
significantly outperform the previous state of the art.
  We have adapted one of the most successful architectures (FSRCNN) for 3D
super-resolution, and compared its performance to a 3D U-Net architecture that
has not been used previously to generate super-resolution.
  We trained both architectures on artificially downscaled isotropic ground
truth from focused ion beam milling scanning EM (FIB-SEM) and tested the
performance for various hyperparameter settings.
  Our results indicate that both architectures can successfully generate 3D
isotropic super-resolution from non-isotropic EM, with the U-Net performing
consistently better. We propose several promising directions for practical
application.
Large-scale network motif analysis using compression
Authors: Peter Bloem, Steven de Rooij
Date: 2017-01-08 00:00:00

We introduce a new method for finding network motifs: interesting or informative subgraph patterns in a network. Subgraphs are motifs when their frequency in the data is high compared to the expected frequency under a null model. To compute this expectation, a full or approximate count of the occurrences of a motif is normally repeated on as many as 1000 random graphs sampled from the null model; a prohibitively expensive step. We use ideas from the Minimum Description Length (MDL) literature to define a new measure of motif relevance. With our method, samples from the null model are not required. Instead we compute the probability of the data under the null model and compare this to the probability under a specially designed alternative model. With this new relevance test, we can search for motifs by random sampling, rather than requiring an accurate count of all instances of a motif. This allows motif analysis to scale to networks with billions of links.
Adversarial Deep Structural Networks for Mammographic Mass Segmentation
Authors: Wentao Zhu, Xiang Xiang, Trac. D. Tran, Xiaohui Xie
Date: 2016-12-18 00:00:00
Tasks: Position, Segmentation

Mass segmentation is an important task in mammogram analysis, providing
effective morphological features and regions of interest (ROI) for mass
detection and classification. Inspired by the success of using deep
convolutional features for natural image analysis and conditional random fields
(CRF) for structural learning, we propose an end-to-end network for
mammographic mass segmentation. The network employs a fully convolutional
network (FCN) to model potential function, followed by a CRF to perform
structural learning. Because the mass distribution varies greatly with pixel
position, the FCN is combined with position priori for the task. Due to the
small size of mammogram datasets, we use adversarial training to control
over-fitting. Four models with different convolutional kernels are further
fused to improve the segmentation results. Experimental results on two public
datasets, INbreast and DDSM-BCRP, show that our end-to-end network combined
with adversarial training achieves the-state-of-the-art results.
Diversity-aware Multi-Video Summarization
Authors: Rameswar Panda, Niluthpol Chowdhury Mithun, Amit K. Roy-Chowdhury
Date: 2017-06-09 00:00:00
Tasks: Diversity, Video Summarization

Most video summarization approaches have focused on extracting a summary from
a single video; we propose an unsupervised framework for summarizing a
collection of videos. We observe that each video in the collection may contain
some information that other videos do not have, and thus exploring the
underlying complementarity could be beneficial in creating a diverse
informative summary. We develop a novel diversity-aware sparse optimization
method for multi-video summarization by exploring the complementarity within
the videos. Our approach extracts a multi-video summary which is both
interesting and representative in describing the whole video collection. To
efficiently solve our optimization problem, we develop an alternating
minimization algorithm that minimizes the overall objective function with
respect to one video at a time while fixing the other videos. Moreover, we
introduce a new benchmark dataset, Tour20, that contains 140 videos with
multiple human created summaries, which were acquired in a controlled
experiment. Finally, by extensive experiments on the new Tour20 dataset and
several other multi-view datasets, we show that the proposed approach clearly
outperforms the state-of-the-art methods on the two problems-topic-oriented
video summarization and multi-view video summarization in a camera network.