pwc-archive/papers-with-abstracts huggingface dataset:
Exploiting Sentence and Context Representations in Deep Neural Models for Spoken Language Understanding
Authors: Lina M. Rojas Barahona, Milica Gasic, Nikola Mrkšić, Pei-Hao Su, Stefan Ultes, Tsung-Hsien Wen, Steve Young
Date: 2016-10-13 00:00:00
Proceeding: COLING 2016 12
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Decoder, Representation Learning, Sentence, slot-filling, Slot Filling, speech-recognition, Speech Recognition, Spoken Language Understanding

This paper presents a deep learning architecture for the semantic decoder
component of a Statistical Spoken Dialogue System. In a slot-filling dialogue,
the semantic decoder predicts the dialogue act and a set of slot-value pairs
from a set of n-best hypotheses returned by the Automatic Speech Recognition.
Most current models for spoken language understanding assume (i) word-aligned
semantic annotations as in sequence taggers and (ii) delexicalisation, or a
mapping of input words to domain-specific concepts using heuristics that try to
capture morphological variation but that do not scale to other domains nor to
language variation (e.g., morphology, synonyms, paraphrasing ). In this work
the semantic decoder is trained using unaligned semantic annotations and it
uses distributed semantic representation learning to overcome the limitations
of explicit delexicalisation. The proposed architecture uses a convolutional
neural network for the sentence representation and a long-short term memory
network for the context representation. Results are presented for the publicly
available DSTC2 corpus and an In-car corpus which is similar to DSTC2 but has a
significantly higher word error rate (WER).
Star-galaxy Classification Using Deep Convolutional Neural Networks
Authors: Edward J. Kim, Robert J. Brunner
Date: 2016-08-15 00:00:00
Tasks: BIG-bench Machine Learning, Classification, Feature Engineering, General Classification, Survey

Most existing star-galaxy classifiers use the reduced summary information
from catalogs, requiring careful feature extraction and selection. The latest
advances in machine learning that use deep convolutional neural networks allow
a machine to automatically learn the features directly from data, minimizing
the need for input from human experts. We present a star-galaxy classification
framework that uses deep convolutional neural networks (ConvNets) directly on
the reduced, calibrated pixel values. Using data from the Sloan Digital Sky
Survey (SDSS) and the Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS),
we demonstrate that ConvNets are able to produce accurate and well-calibrated
probabilistic classifications that are competitive with conventional machine
learning techniques. Future advances in deep learning may bring more success
with current and forthcoming photometric surveys, such as the Dark Energy
Survey (DES) and the Large Synoptic Survey Telescope (LSST), because deep
neural networks require very little, manual feature engineering.
Accelerate Monte Carlo Simulations with Restricted Boltzmann Machines
Authors: Li Huang, Lei Wang
Date: 2016-10-10 00:00:00

Despite their exceptional flexibility and popularity, the Monte Carlo methods
often suffer from slow mixing times for challenging statistical physics
problems. We present a general strategy to overcome this difficulty by adopting
ideas and techniques from the machine learning community. We fit the
unnormalized probability of the physical model to a feedforward neural network
and reinterpret the architecture as a restricted Boltzmann machine. Then,
exploiting its feature detection ability, we utilize the restricted Boltzmann
machine for efficient Monte Carlo updates and to speed up the simulation of the
original physical system. We implement these ideas for the Falicov-Kimball
model and demonstrate improved acceptance ratio and autocorrelation time near
the phase transition point.
Automatic View-Point Selection for Inter-Operative Endoscopic Surveillance
Authors: Anant S. Vemuri, Stephane A. Nicolau, Jacques Marescaux, Luc Soler, Nicholas Ayache
Date: 2016-10-13 00:00:00
Tasks: Retrieval

Esophageal adenocarcinoma arises from Barrett's esophagus, which is the most
serious complication of gastroesophageal reflux disease. Strategies for
screening involve periodic surveillance and tissue biopsies. A major challenge
in such regular examinations is to record and track the disease evolution and
re-localization of biopsied sites to provide targeted treatments. In this
paper, we extend our original inter-operative relocalization framework to
provide a constrained image based search for obtaining the best view-point
match to the live view. Within this context we investigate the effect of: the
choice of feature descriptors and color-space; filtering of uninformative
frames and endoscopic modality, for view-point localization. Our experiments
indicate an improvement in the best view-point retrieval rate to [92%,87%] from
[73%,76%] (in our previous approach) for NBI and WL.
Towards end-to-end optimisation of functional image analysis pipelines
Authors: Albert Vilamala, Kristoffer Hougaard Madsen, Lars Kai Hansen
Date: 2016-10-13 00:00:00
Tasks: Brain Decoding

The study of neurocognitive tasks requiring accurate localisation of activity
often rely on functional Magnetic Resonance Imaging, a widely adopted technique
that makes use of a pipeline of data processing modules, each involving a
variety of parameters. These parameters are frequently set according to the
local goal of each specific module, not accounting for the rest of the
pipeline. Given recent success of neural network research in many different
domains, we propose to convert the whole data pipeline into a deep neural
network, where the parameters involved are jointly optimised by the network to
best serve a common global goal. As a proof of concept, we develop a module
able to adaptively apply the most suitable spatial smoothing to every brain
volume for each specific neuroimaging task, and we validate its results in a
standard brain decoding experiment.
Video Fill in the Blank with Merging LSTMs
Authors: Amir Mazaheri, Dong Zhang, Mubarak Shah
Date: 2016-10-13 00:00:00

Given a video and its incomplete textural description with missing words, the
Video-Fill-in-the-Blank (ViFitB) task is to automatically find the missing
word. The contextual information of the sentences are important to infer the
missing words; the visual cues are even more crucial to get a more accurate
inference. In this paper, we presents a new method which intuitively takes
advantage of the structure of the sentences and employs merging LSTMs (to merge
two LSTMs) to tackle the problem with embedded textural and visual cues. In the
experiments, we have demonstrated the superior performance of the proposed
method on the challenging "Movie Fill-in-the-Blank" dataset.
Stroke Sequence-Dependent Deep Convolutional Neural Network for Online Handwritten Chinese Character Recognition
Authors: Baotian Hu, Xin Liu, Xiang-Ping Wu, Qingcai Chen
Date: 2016-10-13 00:00:00

In this paper, we propose a novel model, named Stroke Sequence-dependent Deep
Convolutional Neural Network (SSDCNN), using the stroke sequence information
and eight-directional features for Online Handwritten Chinese Character
Recognition (OLHCCR). On one hand, SSDCNN can learn the representation of
Online Handwritten Chinese Character (OLHCC) by incorporating the natural
sequence information of the strokes. On the other hand, SSDCNN can incorporate
eight-directional features in a natural way. In order to train SSDCNN, we
divide the process of training into two stages: 1) The training data is used to
pre-train the whole architecture until the performance tends to converge. 2)
Fully-connected neural network which is used to combine the stroke
sequence-dependent representation with eight-directional features and softmax
layer are further trained. Experiments were conducted on the OLHCCR competition
tasks of ICDAR 2013. Results show that, SSDCNN can reduce the recognition error
by 50\% (5.13\% vs 2.56\%) compared to the model which only use
eight-directional features. The proposed SSDCNN achieves 97.44\% accuracy which
reduces the recognition error by about 1.9\% compared with the best submitted
system on ICDAR2013 competition. These results indicate that SSDCNN can exploit
the stroke sequence information to learn high-quality representation of OLHCC.
It also shows that the learnt representation and the classical
eight-directional features complement each other within the SSDCNN
architecture.
Generalized Online Transfer Learning for Climate Control in Residential Buildings
Authors: Thomas Grubinger, Georgios Chasparis, Thomas Natschlaeger
Date: 2016-10-13 00:00:00
Tasks: Model Predictive Control, Transfer Learning

This paper presents an online transfer learning framework for improving
temperature predictions in residential buildings. In transfer learning,
prediction models trained under a set of available data from a target domain
(e.g., house with limited data) can be improved through the use of data
generated from similar source domains (e.g., houses with rich data). Given also
the need for prediction models that can be trained online (e.g., as part of a
model-predictive-control implementation), this paper introduces the generalized
online transfer learning algorithm (GOTL). It employs a weighted combination of
the available predictors (i.e., the target and source predictors) and
guarantees convergence to the best weighted predictor. Furthermore, the use of
Transfer Component Analysis (TCA) allows for using more than a single source
domains, since it may facilitate the fit of a single model on more than one
source domains (houses). This allows GOTL to transfer knowledge from more than
one source domains. We further validate our results through experiments in
climate control for residential buildings and show that GOTL may lead to
non-negligible energy savings for given comfort levels.
Voice Conversion from Non-parallel Corpora Using Variational Auto-encoder
Authors: Chin-Cheng Hsu, Hsin-Te Hwang, Yi-Chiao Wu, Yu Tsao, Hsin-Min Wang
Date: 2016-10-13 00:00:00
Tasks: Decoder, Voice Conversion

We propose a flexible framework for spectral conversion (SC) that facilitates
training with unaligned corpora. Many SC frameworks require parallel corpora,
phonetic alignments, or explicit frame-wise correspondence for learning
conversion functions or for synthesizing a target spectrum with the aid of
alignments. However, these requirements gravely limit the scope of practical
applications of SC due to scarcity or even unavailability of parallel corpora.
We propose an SC framework based on variational auto-encoder which enables us
to exploit non-parallel corpora. The framework comprises an encoder that learns
speaker-independent phonetic representations and a decoder that learns to
reconstruct the designated speaker. It removes the requirement of parallel
corpora or phonetic alignments to train a spectral conversion system. We report
objective and subjective evaluations to validate our proposed method and
compare it to SC methods that have access to aligned corpora.
The meaning-frequency law in Zipfian optimization models of communication
Authors: Ramon Ferrer-i-Cancho
Date: 2014-09-25 00:00:00

According to Zipf's meaning-frequency law, words that are more frequent tend
to have more meanings. Here it is shown that a linear dependency between the
frequency of a form and its number of meanings is found in a family of models
of Zipf's law for word frequencies. This is evidence for a weak version of the
meaning-frequency law. Interestingly, that weak law (a) is not an inevitable of
property of the assumptions of the family and (b) is found at least in the
narrow regime where those models exhibit Zipf's law for word frequencies.
Stream Reasoning-Based Control of Caching Strategies in CCN Routers
Authors: Harald Beck, Bruno Bierbaumer, Minh Dao-Tran, Thomas Eiter, Hermann Hellwagner, Konstantin Schekotihin
Date: 2016-10-13 00:00:00

Content-Centric Networking (CCN) research addresses the mismatch between the
modern usage of the Internet and its outdated architecture. Importantly, CCN
routers may locally cache frequently requested content in order to speed up
delivery to end users. Thus, the issue of caching strategies arises, i.e.,
which content shall be stored and when it should be replaced. In this work, we
employ novel techniques towards intelligent administration of CCN routers that
autonomously switch between existing strategies in response to changing content
request patterns. In particular, we present a router architecture for CCN
networks that is controlled by rule-based stream reasoning, following the
recent formal framework LARS which extends Answer Set Programming for streams.
The obtained possibility for flexible router configuration at runtime allows
for faster experimentation and may thus help to advance the further development
of CCN. Moreover, the empirical evaluation of our feasibility study shows that
the resulting caching agent may give significant performance gains.
Bank Card Usage Prediction Exploiting Geolocation Information
Authors: Martin Wistuba, Nghia Duong-Trung, Nicolas Schilling, Lars Schmidt-Thieme
Date: 2016-10-13 00:00:00
Tasks: General Classification, Position, Prediction, regression

We describe the solution of team ISMLL for the ECML-PKDD 2016 Discovery
Challenge on Bank Card Usage for both tasks. Our solution is based on three
pillars. Gradient boosted decision trees as a strong regression and
classification model, an intensive search for good hyperparameter
configurations and strong features that exploit geolocation information. This
approach achieved the best performance on the public leaderboard for the first
task and a decent fourth position for the second task.
Dictionary Update for NMF-based Voice Conversion Using an Encoder-Decoder Network
Authors: Chin-Cheng Hsu, Hsin-Te Hwang, Yi-Chiao Wu, Yu Tsao, Hsin-Min Wang
Date: 2016-10-13 00:00:00
Tasks: Decoder, Speech Enhancement, Speech Synthesis, Voice Conversion

In this paper, we propose a dictionary update method for Nonnegative Matrix
Factorization (NMF) with high dimensional data in a spectral conversion (SC)
task. Voice conversion has been widely studied due to its potential
applications such as personalized speech synthesis and speech enhancement.
Exemplar-based NMF (ENMF) emerges as an effective and probably the simplest
choice among all techniques for SC, as long as a source-target parallel speech
corpus is given. ENMF-based SC systems usually need a large amount of bases
(exemplars) to ensure the quality of the converted speech. However, a small and
effective dictionary is desirable but hard to obtain via dictionary update, in
particular when high-dimensional features such as STRAIGHT spectra are used.
Therefore, we propose a dictionary update framework for NMF by means of an
encoder-decoder reformulation. Regarding NMF as an encoder-decoder network
makes it possible to exploit the whole parallel corpus more effectively and
efficiently when applied to SC. Our experiments demonstrate significant gains
of the proposed system with small dictionaries over conventional ENMF-based
systems with dictionaries of same or much larger size.
Demonstrating Hybrid Learning in a Flexible Neuromorphic Hardware System
Authors: Simon Friedmann, Johannes Schemmel, Andreas Gruebl, Andreas Hartel, Matthias Hock, Karlheinz Meier
Date: 2016-04-18 00:00:00

We present results from a new approach to learning and plasticity in
neuromorphic hardware systems: to enable flexibility in implementable learning
mechanisms while keeping high efficiency associated with neuromorphic
implementations, we combine a general-purpose processor with full-custom analog
elements.
  This processor is operating in parallel with a fully parallel neuromorphic
system consisting of an array of synapses connected to analog, continuous time
neuron circuits. Novel analog correlation sensor circuits process spike events
for each synapse in parallel and in real-time.
  The processor uses this pre-processing to compute new weights possibly using
additional information following its program.
  Therefore, learning rules can be defined in software giving a large degree of
flexibility.
  Synapses realize correlation detection geared towards Spike-Timing Dependent
Plasticity (STDP) as central computational primitive in the analog domain.
  Operating at a speed-up factor of 1000 compared to biological time-scale, we
measure time-constants from tens to hundreds of micro-seconds.
  We analyze variability across multiple chips and demonstrate learning using a
multiplicative STDP rule.
  We conclude, that the presented approach will enable flexible and efficient
learning as a platform for neuroscientific research and technological
applications.
How Transferable are Neural Networks in NLP Applications?
Authors: Lili Mou, Zhao Meng, Rui Yan, Ge Li, Yan Xu, Lu Zhang, Zhi Jin
Date: 2016-03-19 00:00:00
Proceeding: EMNLP 2016 11
Tasks: Transfer Learning

Transfer learning is aimed to make use of valuable knowledge in a source
domain to help model performance in a target domain. It is particularly
important to neural networks, which are very likely to be overfitting. In some
fields like image processing, many studies have shown the effectiveness of
neural network-based transfer learning. For neural NLP, however, existing
studies have only casually applied transfer learning, and conclusions are
inconsistent. In this paper, we conduct systematic case studies and provide an
illuminating picture on the transferability of neural networks in NLP.
Sequence to Backward and Forward Sequences: A Content-Introducing Approach to Generative Short-Text Conversation
Authors: Lili Mou, Yiping Song, Rui Yan, Ge Li, Lu Zhang, Zhi Jin
Date: 2016-07-04 00:00:00
Proceeding: COLING 2016 12
Tasks: Position, Short-Text Conversation

Using neural networks to generate replies in human-computer dialogue systems
is attracting increasing attention over the past few years. However, the
performance is not satisfactory: the neural network tends to generate safe,
universally relevant replies which carry little meaning. In this paper, we
propose a content-introducing approach to neural network-based generative
dialogue systems. We first use pointwise mutual information (PMI) to predict a
noun as a keyword, reflecting the main gist of the reply. We then propose
seq2BF, a "sequence to backward and forward sequences" model, which generates a
reply containing the given keyword. Experimental results show that our approach
significantly outperforms traditional sequence-to-sequence models in terms of
human evaluation and the entropy measure, and that the predicted keyword can
appear at an appropriate position in the reply.
Improved Relation Classification by Deep Recurrent Neural Networks with Data Augmentation
Authors: Yan Xu, Ran Jia, Lili Mou, Ge Li, Yunchuan Chen, Yangyang Lu, Zhi Jin
Date: 2016-01-14 00:00:00
Proceeding: COLING 2016 12
Tasks: Classification, Data Augmentation, General Classification, Relation, Relation Classification

Nowadays, neural networks play an important role in the task of relation
classification. By designing different neural architectures, researchers have
improved the performance to a large extent in comparison with traditional
methods. However, existing neural networks for relation classification are
usually of shallow architectures (e.g., one-layer convolutional neural networks
or recurrent networks). They may fail to explore the potential representation
space in different abstraction levels. In this paper, we propose deep recurrent
neural networks (DRNNs) for relation classification to tackle this challenge.
Further, we propose a data augmentation method by leveraging the directionality
of relations. We evaluated our DRNNs on the SemEval-2010 Task~8, and achieve an
F1-score of 86.1%, outperforming previous state-of-the-art recorded results.
Dialogue Session Segmentation by Embedding-Enhanced TextTiling
Authors: Yiping Song, Lili Mou, Rui Yan, Li Yi, Zinan Zhu, Xiaohua Hu, Ming Zhang
Date: 2016-10-13 00:00:00
Tasks: Word Embeddings

In human-computer conversation systems, the context of a user-issued
utterance is particularly important because it provides useful background
information of the conversation. However, it is unwise to track all previous
utterances in the current session as not all of them are equally important. In
this paper, we address the problem of session segmentation. We propose an
embedding-enhanced TextTiling approach, inspired by the observation that
conversation utterances are highly noisy, and that word embeddings provide a
robust way of capturing semantics. Experimental results show that our approach
achieves better performance than the TextTiling, MMD approaches.
Compressing Neural Language Models by Sparse Word Representations
Authors: Yunchuan Chen, Lili Mou, Yan Xu, Ge Li, Zhi Jin
Date: 2016-10-13 00:00:00
Proceeding: ACL 2016 8
Tasks: Language Modeling, Language Modelling, Word Embeddings

Neural networks are among the state-of-the-art techniques for language
modeling. Existing neural language models typically map discrete words to
distributed, dense vector representations. After information processing of the
preceding context words by hidden layers, an output layer estimates the
probability of the next word. Such approaches are time- and memory-intensive
because of the large numbers of parameters for word embeddings and the output
layer. In this paper, we propose to compress neural language models by sparse
word representations. In the experiments, the number of parameters in our model
increases very slowly with the growth of the vocabulary size, which is almost
imperceptible. Moreover, our approach not only reduces the parameter space to a
large extent, but also improves the performance in terms of the perplexity
measure.
A Neural Network for Coordination Boundary Prediction
Authors: Jessica Ficler, Yoav Goldberg
Date: 2016-10-13 00:00:00
Proceeding: EMNLP 2016 11
Tasks: Prediction

We propose a neural-network based model for coordination boundary prediction.
The network is designed to incorporate two signals: the similarity between
conjuncts and the observation that replacing the whole coordination phrase with
a conjunct tends to produce a coherent sentences. The modeling makes use of
several LSTM networks. The model is trained solely on conjunction annotations
in a Treebank, without using external resources. We show improvements on
predicting coordination boundaries on the PTB compared to two state-of-the-art
parsers; as well as improvement over previous coordination boundary prediction
systems on the Genia corpus.
A Survey of Voice Translation Methodologies - Acoustic Dialect Decoder
Authors: Hans Krupakar, Keerthika Rajvel, Bharathi B, Angel Deborah S, Vallidevi Krishnamurthy
Date: 2016-10-13 00:00:00
Tasks: Decoder, Sentence, Speech Synthesis, Survey, Translation

Speech Translation has always been about giving source text or audio input
and waiting for system to give translated output in desired form. In this
paper, we present the Acoustic Dialect Decoder (ADD) - a voice to voice
ear-piece translation device. We introduce and survey the recent advances made
in the field of Speech Engineering, to employ in the ADD, particularly focusing
on the three major processing steps of Recognition, Translation and Synthesis.
We tackle the problem of machine understanding of natural language by designing
a recognition unit for source audio to text, a translation unit for source
language text to target language text, and a synthesis unit for target language
text to target language speech. Speech from the surroundings will be recorded
by the recognition unit present on the ear-piece and translation will start as
soon as one sentence is successfully read. This way, we hope to give translated
output as and when input is being read. The recognition unit will use Hidden
Markov Models (HMMs) Based Tool-Kit (HTK), hybrid RNN systems with gated memory
cells, and the synthesis unit, HMM based speech synthesis system HTS. This
system will initially be built as an English to Tamil translation device.