pwc-archive/papers-with-abstracts huggingface dataset:
Non-Monotonic Sequential Text Generation
Authors: Sean Welleck, Kianté Brantley, Hal Daumé III, Kyunghyun Cho
Date: 2019-02-05 00:00:00
Proceeding: WS 2019 8
Tasks: Imitation Learning, Position, Text Generation

Standard sequential generation methods assume a pre-specified generation order, such as text generation methods which generate words from left to right. In this work, we propose a framework for training models of text generation that operate in non-monotonic orders; the model directly learns good orders, without any additional annotation. Our framework operates by generating a word at an arbitrary position, and then recursively generating words to its left and then words to its right, yielding a binary tree. Learning is framed as imitation learning, including a coaching method which moves from imitating an oracle to reinforcing the policy's own preferences. Experimental results demonstrate that using the proposed method, it is possible to learn policies which generate text without pre-specifying a generation order, while achieving competitive performance with conventional left-to-right generation.
Gradient Boosting to Boost the Efficiency of Hydraulic Fracturing
Authors: Ivan Makhotin, Dmitry Koroteev, Evgeny Burnaev
Date: 2019-02-05 00:00:00
Tasks: BIG-bench Machine Learning, Prediction

In this paper, we present a data-driven model for forecasting the production increase after hydraulic fracturing (HF). We use data from fracturing jobs performed at one of the Siberian oilfields. The data includes features, characterizing the jobs, and geological information. To predict an oil rate after the fracturing machine learning (ML) technique was applied. We compared the ML-based prediction to a prediction based on the experience of reservoir and production engineers responsible for the HF-job planning. We discuss the potential for further development of ML techniques for predicting changes in oil rate after HF.
Asynchronous Delay-Aware Accelerated Proximal Coordinate Descent for Nonconvex Nonsmooth Problems
Authors: Ehsan Kazemi, Liqiang Wang
Date: 2019-02-05 00:00:00

Nonconvex and nonsmooth problems have recently attracted considerable
attention in machine learning. However, developing efficient methods for the
nonconvex and nonsmooth optimization problems with certain performance
guarantee remains a challenge. Proximal coordinate descent (PCD) has been
widely used for solving optimization problems, but the knowledge of PCD methods
in the nonconvex setting is very limited. On the other hand, the asynchronous
proximal coordinate descent (APCD) recently have received much attention in
order to solve large-scale problems. However, the accelerated variants of APCD
algorithms are rarely studied. In this paper, we extend APCD method to the
accelerated algorithm (AAPCD) for nonsmooth and nonconvex problems that
satisfies the sufficient descent property, by comparing between the function
values at proximal update and a linear extrapolated point using a delay-aware
momentum value. To the best of our knowledge, we are the first to provide
stochastic and deterministic accelerated extension of APCD algorithms for
general nonconvex and nonsmooth problems ensuring that for both bounded delays
and unbounded delays every limit point is a critical point. By leveraging
Kurdyka-Lojasiewicz property, we will show linear and sublinear convergence
rates for the deterministic AAPCD with bounded delays. Numerical results
demonstrate the practical efficiency of our algorithm in speed.
Detection of perfusion ROI as a quality control in perfusion analysis
Authors: Svitlana Alkhimova
Date: 2019-02-04 00:00:00

In perfusion analysis automated approaches for image processing is preferable
due to reduce time-consuming tasks for radiologists. Assessment of perfusion
results quality is important step in development of algorithms for automated
processing. One of them is an assessment of perfusion maps quality based on
detection of perfusion ROI.
Attention in Natural Language Processing
Authors: Andrea Galassi, Marco Lippi, Paolo Torroni
Date: 2019-02-04 00:00:00

Attention is an increasingly popular mechanism used in a wide range of neural architectures. The mechanism itself has been realized in a variety of formats. However, because of the fast-paced advances in this domain, a systematic overview of attention is still missing. In this article, we define a unified model for attention architectures in natural language processing, with a focus on those designed to work with vector representations of the textual data. We propose a taxonomy of attention models according to four dimensions: the representation of the input, the compatibility function, the distribution function, and the multiplicity of the input and/or output. We present the examples of how prior information can be exploited in attention models and discuss ongoing research efforts and open challenges in the area, providing the first extensive categorization of the vast body of literature in this exciting domain.
Optimal Experiment Design in Nonlinear Parameter Estimation with Exact Confidence Regions
Authors: Anwesh Reddy Gottu Mukkula, Radoslav Paulen
Date: 2019-02-03 00:00:00
Tasks: Bilevel Optimization, parameter estimation

A model-based optimal experiment design (OED) of nonlinear systems is studied. OED represents a methodology for optimizing the geometry of the parametric joint-confidence regions (CRs), which are obtained in an a posteriori analysis of the least-squares parameter estimates. The optimal design is achieved by using the available (experimental) degrees of freedom such that more informative measurements are obtained. Unlike the commonly used approaches, which base the OED procedure upon the linearized CRs, we explore a path where we explicitly consider the exact CRs in the OED framework. We use a methodology for a finite parametrization of the exact CRs within the OED problem and we introduce a novel approximation technique of the exact CRs using inner- and outer-approximating ellipsoids as a computationally less demanding alternative. The employed techniques give the OED problem as a finite-dimensional mathematical program of bilevel nature. We use two small-scale illustrative case studies to study various OED criteria and compare the resulting optimal designs with the commonly used linearization-based approach. We also assess the performance of two simple heuristic numerical schemes for bilevel optimization within the studied problems.
Using multi-task learning to improve the performance of acoustic-to-word and conventional hybrid models
Authors: Thai-Son Nguyen, Sebastian Stueker, Alex Waibel
Date: 2019-02-02 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Multi-Task Learning, speech-recognition, Speech Recognition

Acoustic-to-word (A2W) models that allow direct mapping from acoustic signals to word sequences are an appealing approach to end-to-end automatic speech recognition due to their simplicity. However, prior works have shown that modelling A2W typically encounters issues of data sparsity that prevent training such a model directly. So far, pre-training initialization is the only approach proposed to deal with this issue. In this work, we propose to build a shared neural network and optimize A2W and conventional hybrid models in a multi-task manner. Our results show that training an A2W model is much more stable with our multi-task model without pre-training initialization, and results in a significant improvement compared to a baseline model. Experiments also reveal that the performance of a hybrid acoustic model can be further improved when jointly training with a sequence-level optimization criterion such as acoustic-to-word.
Confidential Boosting with Random Linear Classifiers for Outsourced User-generated Data
Authors: Sagar Sharma, Keke Chen
Date: 2018-02-22 00:00:00

User-generated data is crucial to predictive modeling in many applications.
With a web/mobile/wearable interface, a data owner can continuously record data
generated by distributed users and build various predictive models from the
data to improve their operations, services, and revenue. Due to the large size
and evolving nature of users data, data owners may rely on public cloud service
providers (Cloud) for storage and computation scalability. Exposing sensitive
user-generated data and advanced analytic models to Cloud raises privacy
concerns. We present a confidential learning framework, SecureBoost, for data
owners that want to learn predictive models from aggregated user-generated data
but offload the storage and computational burden to Cloud without having to
worry about protecting the sensitive data. SecureBoost allows users to submit
encrypted or randomly masked data to designated Cloud directly. Our framework
utilizes random linear classifiers (RLCs) as the base classifiers in the
boosting framework to dramatically simplify the design of the proposed
confidential boosting protocols, yet still preserve the model quality. A
Cryptographic Service Provider (CSP) is used to assist the Cloud's processing,
reducing the complexity of the protocol constructions. We present two
constructions of SecureBoost: HE+GC and SecSh+GC, using combinations of
homomorphic encryption, garbled circuits, and random masking to achieve both
security and efficiency. For a boosted model, Cloud learns only the RLCs and
the CSP learns only the weights of the RLCs. Finally, the data owner collects
the two parts to get the complete model. We conduct extensive experiments to
understand the quality of the RLC-based boosting and the cost distribution of
the constructions. Our results show that SecureBoost can efficiently learn
high-quality boosting models from protected user-generated data.
Parallel Contextual Bandits in Wireless Handover Optimization
Authors: Igor Colin, Albert Thomas, Moez Draief
Date: 2019-01-21 00:00:00
Tasks: Multi-Armed Bandits, Thompson Sampling

As cellular networks become denser, a scalable and dynamic tuning of wireless
base station parameters can only be achieved through automated optimization.
Although the contextual bandit framework arises as a natural candidate for such
a task, its extension to a parallel setting is not straightforward: one needs
to carefully adapt existing methods to fully leverage the multi-agent structure
of this problem. We propose two approaches: one derived from a deterministic
UCB-like method and the other relying on Thompson sampling. Thanks to its
bayesian nature, the latter is intuited to better preserve the
exploration-exploitation balance in the bandit batch. This is verified on toy
experiments, where Thompson sampling shows robustness to the variability of the
contexts. Finally, we apply both methods on a real base station network dataset
and evidence that Thompson sampling outperforms both manual tuning and
contextual UCB.
Combining Physically-Based Modeling and Deep Learning for Fusing GRACE Satellite Data: Can We Learn from Mismatch?
Authors: Alexander Y. Sun, Bridget R. Scanlon, Zizhan Zhang, David Walling, Soumendra N. Bhanja, Abhijit Mukherjee, Zhi Zhong
Date: 2019-01-31 00:00:00

Global hydrological and land surface models are increasingly used for
tracking terrestrial total water storage (TWS) dynamics, but the utility of
existing models is hampered by conceptual and/or data uncertainties related to
various underrepresented and unrepresented processes, such as groundwater
storage. The gravity recovery and climate experiment (GRACE) satellite mission
provided a valuable independent data source for tracking TWS at regional and
continental scales. Strong interests exist in fusing GRACE data into global
hydrological models to improve their predictive performance. Here we develop
and apply deep convolutional neural network (CNN) models to learn the
spatiotemporal patterns of mismatch between TWS anomalies (TWSA) derived from
GRACE and those simulated by NOAH, a widely used land surface model. Once
trained, our CNN models can be used to correct the NOAH simulated TWSA without
requiring GRACE data, potentially filling the data gap between GRACE and its
follow-on mission, GRACE-FO. Our methodology is demonstrated over India, which
has experienced significant groundwater depletion in recent decades that is
nevertheless not being captured by the NOAH model. Results show that the CNN
models significantly improve the match with GRACE TWSA, achieving a
country-average correlation coefficient of 0.94 and Nash-Sutcliff efficient of
0.87, or 14\% and 52\% improvement respectively over the original NOAH TWSA. At
the local scale, the learned mismatch pattern correlates well with the observed
in situ groundwater storage anomaly data for most parts of India, suggesting
that deep learning models effectively compensate for the missing groundwater
component in NOAH for this study region.
Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks
Authors: Jingjing Wang, Chunxiao Jiang, Haijun Zhang, Yong Ren, Kwang-cheng Chen, Lajos Hanzo
Date: 2019-01-24 00:00:00
Tasks: BIG-bench Machine Learning, Decision Making, parameter estimation, Reinforcement Learning

Future wireless networks have a substantial potential in terms of supporting a broad range of complex compelling applications both in military and civilian fields, where the users are able to enjoy high-rate, low-latency, low-cost and reliable information services. Achieving this ambitious goal requires new radio techniques for adaptive learning and intelligent decision making because of the complex heterogeneous nature of the network structures and wireless services. Machine learning (ML) algorithms have great success in supporting big data analytics, efficient parameter estimation and interactive decision making. Hence, in this article, we review the thirty-year history of ML by elaborating on supervised learning, unsupervised learning, reinforcement learning and deep learning. Furthermore, we investigate their employment in the compelling applications of wireless networks, including heterogeneous networks (HetNets), cognitive radios (CR), Internet of things (IoT), machine to machine networks (M2M), and so on. This article aims for assisting the readers in clarifying the motivation and methodology of the various ML algorithms, so as to invoke them for hitherto unexplored services as well as scenarios of future wireless networks.
Learning Taxonomies of Concepts and not Words using Contextualized Word Representations: A Position Paper
Authors: Lukas Schmelzeisen, Steffen Staab
Date: 2019-01-31 00:00:00
Tasks: Position

Taxonomies are semantic hierarchies of concepts. One limitation of current
taxonomy learning systems is that they define concepts as single words. This
position paper argues that contextualized word representations, which recently
achieved state-of-the-art results on many competitive NLP tasks, are a
promising method to address this limitation. We outline a novel approach for
taxonomy learning that (1) defines concepts as synsets, (2) learns
density-based approximations of contextualized word representations, and (3)
can measure similarity and hypernymy among them.
Adaptive Artificial Intelligent Q&A Platform
Authors: M. R, Akram, C. P, Singhabahu, M. S. M Saad, P, Deleepa, Anupiya, Nugaliyadde, Yashas, Mallawarachchi
Date: 2019-01-19 00:00:00
Tasks: Answer Generation, Form

The paper presents an approach to build a question and answer system that is
capable of processing the information in a large dataset and allows the user to
gain knowledge from this dataset by asking questions in natural language form.
Key content of this research covers four dimensions which are; Corpus
Preprocessing, Question Preprocessing, Deep Neural Network for Answer
Extraction and Answer Generation. The system is capable of understanding the
question, responds to the user's query in natural language form as well. The
goal is to make the user feel as if they were interacting with a person than a
machine.
Squared English Word: A Method of Generating Glyph to Use Super Characters for Sentiment Analysis
Authors: Baohua Sun, Lin Yang, Catherine Chi, Wenhan Zhang, Michael Lin
Date: 2019-01-24 00:00:00
Tasks: General Classification, Sentence, Sentiment Analysis

The Super Characters method addresses sentiment analysis problems by first converting the input text into images and then applying 2D-CNN models to classify the sentiment. It achieves state of the art performance on many benchmark datasets. However, it is not as straightforward to apply in Latin languages as in Asian languages. Because the 2D-CNN model is designed to recognize two-dimensional images, it is better if the inputs are in the form of glyphs. In this paper, we propose SEW (Squared English Word) method generating a squared glyph for each English word by drawing Super Characters images of each English word at the alphabet level, combining the squared glyph together into a whole Super Characters image at the sentence level, and then applying the CNN model to classify the sentiment within the sentence. We applied the SEW method to Wikipedia dataset and obtained a 2.1% accuracy gain compared to the original Super Characters method. For multi-modal data with both structured tabular data and unstructured natural language text, the modified SEW method integrates the data into a single image and classifies sentiment with one unified CNN model.
AD3: Attentive Deep Document Dater
Authors: Swayambhu Nath Ray, Shib Sankar Dasgupta, Partha Talukdar
Date: 2019-01-21 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Document Dating, Event Extraction

Knowledge of the creation date of documents facilitates several tasks such as
summarization, event extraction, temporally focused information extraction etc.
Unfortunately, for most of the documents on the Web, the time-stamp metadata is
either missing or can't be trusted. Thus, predicting creation time from
document content itself is an important task. In this paper, we propose
Attentive Deep Document Dater (AD3), an attention-based neural document dating
system which utilizes both context and temporal information in documents in a
flexible and principled manner. We perform extensive experimentation on
multiple real-world datasets to demonstrate the effectiveness of AD3 over
neural and non-neural baselines.
Diseño de un espacio semántico sobre la base de la Wikipedia. Una propuesta de análisis de la semántica latente para el idioma español
Authors: Dalina Aidee Villa, Igor Barahona, Luis Javier Álvarez
Date: 2019-01-28 00:00:00
Tasks: Dimensionality Reduction

Latent Semantic Analysis (LSA) was initially conceived by the cognitive
psychology at the 90s decade. Since its emergence, the LSA has been used to
model cognitive processes, pointing out academic texts, compare literature
works and analyse political speeches, among other applications. Taking as
starting point multivariate method for dimensionality reduction, this paper
propose a semantic space for Spanish language. Out results include a document
text matrix with dimensions 1.3 x10^6 and 5.9x10^6, which later is decomposed
into singular values. Those singular values are used to semantically words or
text.
A Linear-complexity Multi-biometric Forensic Document Analysis System, by Fusing the Stylome and Signature Modalities
Authors: Sayyed-Ali Hossayni, Yousef Alizadeh-Q, Vahid Tavana, Seyed M. Hosseini Nejad, Mohammad-R Akbarzadeh-T, Esteve Del Acebo, Josep Lluis De la Rosa i Esteva, Enrico Grosso, Massimo Tistarelli, Przemyslaw Kudlacik
Date: 2019-01-26 00:00:00
Tasks: Authorship Attribution

Forensic Document Analysis (FDA) addresses the problem of finding the
authorship of a given document. Identification of the document writer via a
number of its modalities (e.g. handwriting, signature, linguistic writing style
(i.e. stylome), etc.) has been studied in the FDA state-of-the-art. But, no
research is conducted on the fusion of stylome and signature modalities. In
this paper, we propose such a bimodal FDA system (which has vast applications
in judicial, police-related, and historical documents analysis) with a focus on
time-complexity. The proposed bimodal system can be trained and tested with
linear time complexity. For this purpose, we first revisit Multinomial Na\"ive
Bayes (MNB), as the best state-of-the-art linear-complexity authorship
attribution system and, then, prove its superior accuracy to the well-known
linear-complexity classifiers in the state-of-the-art. Then, we propose a fuzzy
version of MNB for being fused with a state-of-the-art well-known
linear-complexity fuzzy signature recognition system. For the evaluation
purposes, we construct a chimeric dataset, composed of signatures and textual
contents of different letters. Despite its linear-complexity, the proposed
multi-biometric system is proven to meaningfully improve its state-of-the-art
unimodal counterparts, regarding the accuracy, F-Score, Detection Error
Trade-off (DET), Cumulative Match Characteristics (CMC), and Match Score
Histograms (MSH) evaluation metrics.
Assessing Partisan Traits of News Text Attributions
Authors: Logan Martel, Edward Newell, Drew Margolin, Derek Ruths
Date: 2019-01-25 00:00:00

On the topic of journalistic integrity, the current state of accurate,
impartial news reporting has garnered much debate in context to the 2016 US
Presidential Election. In pursuit of computational evaluation of news text, the
statements (attributions) ascribed by media outlets to sources provide a common
category of evidence on which to operate. In this paper, we develop an approach
to compare partisan traits of news text attributions and apply it to
characterize differences in statements ascribed to candidate, Hilary Clinton,
and incumbent President, Donald Trump. In doing so, we present a model trained
on over 600 in-house annotated attributions to identify each candidate with
accuracy > 88%. Finally, we discuss insights from its performance for future
research.
StrokeNet: A Neural Painting Environment
Authors: Ningyuan Zheng, Yifan Jiang, Dingjiang Huang
Date: 2019-05-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Reinforcement Learning

We've seen tremendous success of image generating models these years. Generating images through a neural network is usually pixel-based, which is fundamentally different from how humans create artwork using brushes. To imitate human drawing, interactions between the environment and the agent is required to allow trials. However, the environment is usually non-differentiable, leading to slow convergence and massive computation. In this paper we try to address the discrete nature of software environment with an intermediate, differentiable simulation. We present  StrokeNet, a novel model where the agent is trained upon a well-crafted neural approximation of the painting environment. With this approach, our agent was able to learn to write characters such as MNIST digits faster than reinforcement learning approaches in an unsupervised manner. Our primary contribution is the neural simulation of a real-world environment. Furthermore, the agent trained with the emulated environment is able to directly transfer its skills to real-world software.