pwc-archive/papers-with-abstracts huggingface dataset:
Energy Disaggregation via Deep Temporal Dictionary Learning
Authors: Mahdi Khodayar, Jianhui Wang, Zhaoyu Wang
Date: 2018-09-10 00:00:00
Tasks: Dictionary Learning

This paper addresses the energy disaggregation problem, i.e. decomposing the
electricity signal of a whole home to its operating devices. First, we cast the
problem as a dictionary learning (DL) problem where the key electricity
patterns representing consumption behaviors are extracted for each device and
stored in a dictionary matrix. The electricity signal of each device is then
modeled by a linear combination of such patterns with sparse coefficients that
determine the contribution of each device in the total electricity. Although
popular, the classic DL approach is prone to high error in real-world
applications including energy disaggregation, as it merely finds linear
dictionaries. Moreover, this method lacks a recurrent structure; thus, it is
unable to leverage the temporal structure of energy signals. Motivated by such
shortcomings, we propose a novel optimization program where the dictionary and
its sparse coefficients are optimized simultaneously with a deep neural model
extracting powerful nonlinear features from the energy signals. A long
short-term memory auto-encoder (LSTM-AE) is proposed with tunable time
dependent states to capture the temporal behavior of energy signals for each
device. We learn the dictionary in the space of temporal features captured by
the LSTM-AE rather than the original space of the energy signals; hence, in
contrast to the traditional DL, here, a nonlinear dictionary is learned using
powerful temporal features extracted from our deep model. Real experiments on
the publicly available Reference Energy Disaggregation Dataset (REDD) show
significant improvement compared to the state-of-the-art methodologies in terms
of the disaggregation accuracy and F-score metrics.
Development and Evaluation of Recurrent Neural Network based Models for Hourly Traffic Volume and AADT Prediction
Authors: MD Zadid Khan, Sakib Mahmud Khan, Mashrur Chowdhury, Kakan Dey
Date: 2018-08-15 00:00:00
Tasks: Imputation, Missing Values, Prediction, Time Series, Time Series Analysis, Time Series Forecasting

The prediction of high-resolution hourly traffic volumes of a given roadway
is essential for transportation planning. Traditionally, Automatic Traffic
Recorders (ATR) are used to collect this hourly volume data. These large
datasets are time series data characterized by long-term temporal dependencies
and missing values. Regarding the temporal dependencies, all roadways are
characterized by seasonal variations that can be weekly, monthly or yearly,
depending on the cause of the variation. Regarding the missing data in a
time-series sequence, traditional time series forecasting models perform poorly
under the influence of seasonal variations. To address this limitation, robust,
Recurrent Neural Network (RNN) based, multi-step ahead forecasting models are
developed for time-series in this study. The simple RNN, the Gated Recurrent
Unit (GRU) and the Long Short-Term Memory (LSTM) units are used to develop the
model and evaluate its performance. Two approaches are used to address the
missing value issue: masking and imputation, in conjunction with the RNN
models. Six different imputation algorithms are then used to identify the best
model. The analysis indicates that the LSTM model performs better than simple
RNN and GRU models, and imputation performs better than masking to predict
future traffic volume. Based on analysis using 92 ATRs, the LSTM-Median model
is deemed the best model in all scenarios for hourly traffic volume and AADT
prediction, with an average RMSE of 274 and MAPE of 18.91% for hourly traffic
volume prediction and average RMSE of 824 and MAPE of 2.10% for AADT
prediction.
A Correlation Maximization Approach for Cross Domain Co-Embeddings
Authors: Dan Shiebler
Date: 2018-09-10 00:00:00
Tasks: Recommendation Systems

Although modern recommendation systems can exploit the structure in users'
item feedback, most are powerless in the face of new users who provide no
structure for them to exploit. In this paper we introduce ImplicitCE, an
algorithm for recommending items to new users during their sign-up flow.
ImplicitCE works by transforming users' implicit feedback towards auxiliary
domain items into an embedding in the target domain item embedding space.
ImplicitCE learns these embedding spaces and transformation function in an
end-to-end fashion and can co-embed users and items with any differentiable
similarity function. To train ImplicitCE we explore methods for maximizing the
correlations between model predictions and users' affinities and introduce
Sample Correlation Update, a novel and extremely simple training strategy.
Finally, we show that ImplicitCE trained with Sample Correlation Update
outperforms a variety of state of the art algorithms and loss functions on both
a large scale Twitter dataset and the DBLP dataset.
Variance Reduction in Monte Carlo Counterfactual Regret Minimization (VR-MCCFR) for Extensive Form Games using Baselines
Authors: Martin Schmid, Neil Burch, Marc Lanctot, Matej Moravcik, Rudolf Kadlec, Michael Bowling
Date: 2018-09-09 00:00:00
Tasks: counterfactual, Form, Reinforcement Learning

Learning strategies for imperfect information games from samples of
interaction is a challenging problem. A common method for this setting, Monte
Carlo Counterfactual Regret Minimization (MCCFR), can have slow long-term
convergence rates due to high variance. In this paper, we introduce a variance
reduction technique (VR-MCCFR) that applies to any sampling variant of MCCFR.
Using this technique, per-iteration estimated values and updates are
reformulated as a function of sampled values and state-action baselines,
similar to their use in policy gradient reinforcement learning. The new
formulation allows estimates to be bootstrapped from other estimates within the
same episode, propagating the benefits of baselines along the sampled
trajectory; the estimates remain unbiased even when bootstrapping from other
estimates. Finally, we show that given a perfect baseline, the variance of the
value estimates can be reduced to zero. Experimental evaluation shows that
VR-MCCFR brings an order of magnitude speedup, while the empirical variance
decreases by three orders of magnitude. The decreased variance allows for the
first time CFR+ to be used with sampling, increasing the speedup to two orders
of magnitude.
Sentiment analysis for Arabic language: A brief survey of approaches and techniques
Authors: Mo'ath Alrefai, Hossam Faris, Ibrahim Aljarah
Date: 2018-09-08 00:00:00
Tasks: Arabic Sentiment Analysis, Sentiment Analysis

With the emergence of Web 2.0 technology and the expansion of on-line social
networks, current Internet users have the ability to add their reviews, ratings
and opinions on social media and on commercial and news web sites. Sentiment
analysis aims to classify these reviews reviews in an automatic way. In the
literature, there are numerous approaches proposed for automatic sentiment
analysis for different language contexts. Each language has its own properties
that makes the sentiment analysis more challenging. In this regard, this work
presents a comprehensive survey of existing Arabic sentiment analysis studies,
and covers the various approaches and techniques proposed in the literature.
Moreover, we highlight the main difficulties and challenges of Arabic sentiment
analysis, and the proposed techniques in literature to overcome these barriers.
Dual-label Deep LSTM Dereverberation For Speaker Verification
Authors: Hao Zhang, Stephen Zahorian, Xiao Chen, Peter Guzewich, Xiaoyu Liu
Date: 2018-09-08 00:00:00
Tasks: Speaker Verification

In this paper, we present a reverberation removal approach for speaker
verification, utilizing dual-label deep neural networks (DNNs). The networks
perform feature mapping between the spectral features of reverberant and clean
speech. Long short term memory recurrent neural networks (LSTMs) are trained to
map corrupted Mel filterbank (MFB) features to two sets of labels: i) the clean
MFB features, and ii) either estimated pitch tracks or the fast Fourier
transform (FFT) spectrogram of clean speech. The performance of reverberation
removal is evaluated by equal error rates (EERs) of speaker verification
experiments.
Binary Latent Representations for Efficient Ranking: Empirical Assessment
Authors: Maciej Kula
Date: 2017-06-22 00:00:00
Tasks: Recommendation Systems

Large-scale recommender systems often face severe latency and storage
constraints at prediction time. These are particularly acute when the number of
items that could be recommended is large, and calculating predictions for the
full set is computationally intensive. In an attempt to relax these
constraints, we train recommendation models that use binary rather than
real-valued user and item representations, and show that while they are
substantially faster to evaluate, the gains in speed come at a large cost in
accuracy. In our Movielens 1M experiments, we show that reducing the latent
dimensionality of traditional models offers a more attractive accuracy/speed
trade-off than using binary representations.
Pareto Smoothed Importance Sampling
Authors: Aki Vehtari, Daniel Simpson, Andrew Gelman, Yuling Yao, Jonah Gabry
Date: 2015-07-09 00:00:00
Tasks: Diagnostic

Importance weighting is a general way to adjust Monte Carlo integration to account for draws from the wrong distribution, but the resulting estimate can be highly variable when the importance ratios have a heavy right tail. This routinely occurs when there are aspects of the target distribution that are not well captured by the approximating distribution, in which case more stable estimates can be obtained by modifying extreme importance ratios. We present a new method for stabilizing importance weights using a generalized Pareto distribution fit to the upper tail of the distribution of the simulated importance ratios. The method, which empirically performs better than existing methods for stabilizing importance sampling estimates, includes stabilized effective sample size estimates, Monte Carlo error estimates, and convergence diagnostics. The presented Pareto $\hat{k}$ finite sample convergence rate diagnostic is useful for any Monte Carlo estimator.
Beyond Stemming and Lemmatization: Ultra-stemming to Improve Automatic Text Summarization
Authors: Juan-Manuel Torres-Moreno
Date: 2012-09-14 00:00:00
Tasks: Lemmatization, Text Summarization

In Automatic Text Summarization, preprocessing is an important phase to
reduce the space of textual representation. Classically, stemming and
lemmatization have been widely used for normalizing words. However, even using
normalization on large texts, the curse of dimensionality can disturb the
performance of summarizers. This paper describes a new method for normalization
of words to further reduce the space of representation. We propose to reduce
each word to its initial letters, as a form of Ultra-stemming. The results show
that Ultra-stemming not only preserve the content of summaries produced by this
representation, but often the performances of the systems can be dramatically
improved. Summaries on trilingual corpora were evaluated automatically with
Fresa. Results confirm an increase in the performance, regardless of summarizer
system used.
Complete Inference of Causal Relations between Dynamical Systems
Authors: Zsigmond Benkő, Ádám Zlatniczki, Marcell Stippinger, Dániel Fabó, András Sólyom, Loránd Erőss, András Telcs, Zoltán Somogyvári
Date: 2018-08-31 00:00:00
Tasks: EEG, Time Series

From ancient philosophers to modern economists, biologists, and other researchers, there has been a continuous effort to unveil causal relations. The most formidable challenge lies in deducing the nature of the causal relationship: whether it is unidirectional, bidirectional, or merely apparent - implied by an unobserved common cause. While modern technology equips us with tools to collect data from intricate systems such as the planet's ecosystem or the human brain, comprehending their functioning requires the identification and differentiation of causal relationships among the components, all without external interventions. In this context, we introduce a novel method capable of distinguishing and assigning probabilities to the presence of all potential causal relations between two or more time series within dynamical systems. The efficacy of this method is verified using synthetic datasets and applied to EEG (electroencephalographic) data recorded from epileptic patients. Given the universal applicability of our method, it holds promise for diverse scientific fields.
LeMoNADe: Learned Motif and Neuronal Assembly Detection in calcium imaging videos
Authors: Elke Kirschbaum, Manuel Haußmann, Steffen Wolf, Hannah Jakobi, Justus Schneider, Shehabeldin Elzoheiry, Oliver Kann, Daniel Durstewitz, Fred A. Hamprecht
Date: 2018-06-26 00:00:00
Proceeding: ICLR 2019 5

Neuronal assemblies, loosely defined as subsets of neurons with reoccurring
spatio-temporally coordinated activation patterns, or "motifs", are thought to
be building blocks of neural representations and information processing. We
here propose LeMoNADe, a new exploratory data analysis method that facilitates
hunting for motifs in calcium imaging videos, the dominant microscopic
functional imaging modality in neurophysiology. Our nonparametric method
extracts motifs directly from videos, bypassing the difficult intermediate step
of spike extraction. Our technique augments variational autoencoders with a
discrete stochastic node, and we show in detail how a differentiable
reparametrization and relaxation can be used. An evaluation on simulated data,
with available ground truth, reveals excellent quantitative performance. In
real video data acquired from brain slices, with no ground truth available,
LeMoNADe uncovers nontrivial candidate motifs that can help generate hypotheses
for more focused biological investigation.
Distributed Chernoff Test: Optimal decision systems over networks
Authors: Anshuka Rangi, Massimo Franceschetti, Stefano Marano
Date: 2018-09-12 00:00:00
Tasks: Decision Making, Quantization, Two-sample testing

We study "active" decision making over sensor networks where the sensors' sequential probing actions are actively chosen by continuously learning from past observations. We consider two network settings: with and without central coordination. In the first case, the network nodes interact with each other through a central entity, which plays the role of a fusion center. In the second case, the network nodes interact in a fully distributed fashion. In both of these scenarios, we propose sequential and adaptive hypothesis tests extending the classic Chernoff test. We compare the performance of the proposed tests to the optimal sequential test. In the presence of a fusion center, our test achieves the same asymptotic optimality of the Chernoff test, minimizing the risk, expressed by the expected cost required to reach a decision plus the expected cost of making a wrong decision, when the observation cost per unit time tends to zero. The test is also asymptotically optimal in the higher moments of the time required to reach a decision. Additionally, the test is parsimonious in terms of communications, and the expected number of channel uses per network node tends to a small constant. In the distributed setup, our test achieves the same asymptotic optimality of Chernoff's test, up to a multiplicative constant in terms of both risk and the higher moments of the decision time. Additionally, the test is parsimonious in terms of communications in comparison to state-of-the-art schemes proposed in the literature. The analysis of these tests is also extended to account for message quantization and communication over channels with random erasures.
Closed-Book Training to Improve Summarization Encoder Memory
Authors: Yichen Jiang, Mohit Bansal
Date: 2018-09-12 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Abstractive Text Summarization, Decoder, Memorization

A good neural sequence-to-sequence summarization model should have a strong
encoder that can distill and memorize the important information from long input
texts so that the decoder can generate salient summaries based on the encoder's
memory. In this paper, we aim to improve the memorization capabilities of the
encoder of a pointer-generator model by adding an additional 'closed-book'
decoder without attention and pointer mechanisms. Such a decoder forces the
encoder to be more selective in the information encoded in its memory state
because the decoder can't rely on the extra information provided by the
attention and possibly copy modules, and hence improves the entire model. On
the CNN/Daily Mail dataset, our 2-decoder model outperforms the baseline
significantly in terms of ROUGE and METEOR metrics, for both cross-entropy and
reinforced setups (and on human evaluation). Moreover, our model also achieves
higher scores in a test-only DUC-2002 generalizability setup. We further
present a memory ability test, two saliency metrics, as well as several
sanity-check ablations (based on fixed-encoder, gradient-flow cut, and model
capacity) to prove that the encoder of our 2-decoder model does in fact learn
stronger memory representations than the baseline encoder.
Simplicity Creates Inequity: Implications for Fairness, Stereotypes, and Interpretability
Authors: Jon Kleinberg, Sendhil Mullainathan
Date: 2018-09-12 00:00:00
Tasks: Decision Making, Fairness, Prediction

Algorithms are increasingly used to aid, or in some cases supplant, human decision-making, particularly for decisions that hinge on predictions. As a result, two additional features in addition to prediction quality have generated interest: (i) to facilitate human interaction and understanding with these algorithms, we desire prediction functions that are in some fashion simple or interpretable; and (ii) because they influence consequential decisions, we also want them to produce equitable allocations. We develop a formal model to explore the relationship between the demands of simplicity and equity. Although the two concepts appear to be motivated by qualitatively distinct goals, we show a fundamental inconsistency between them. Specifically, we formalize a general framework for producing simple prediction functions, and in this framework we establish two basic results. First, every simple prediction function is strictly improvable: there exists a more complex prediction function that is both strictly more efficient and also strictly more equitable. Put another way, using a simple prediction function both reduces utility for disadvantaged groups and reduces overall welfare relative to other options. Second, we show that simple prediction functions necessarily create incentives to use information about individuals' membership in a disadvantaged group --- incentives that weren't present before simplification, and that work against these individuals. Thus, simplicity transforms disadvantage into bias against the disadvantaged group. Our results are not only about algorithms but about any process that produces simple models, and as such they connect to the psychology of stereotypes and to an earlier economics literature on statistical discrimination.
On the Generalization of Stochastic Gradient Descent with Momentum
Authors: Ali Ramezani-Kebrya, Kimon Antonakopoulos, Volkan Cevher, Ashish Khisti, Ben Liang
Date: 2018-09-12 00:00:00
Tasks: BIG-bench Machine Learning

While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods. In this work, we first show that there exists a convex loss function for which the stability gap for multiple epochs of SGD with standard heavy-ball momentum (SGDM) becomes unbounded. Then, for smooth Lipschitz loss functions, we analyze a modified momentum-based update rule, i.e., SGD with early momentum (SGDEM) under a broad range of step-sizes, and show that it can train machine learning models for multiple epochs with a guarantee for generalization. Finally, for the special case of strongly convex loss functions, we find a range of momentum such that multiple epochs of standard SGDM, as a special form of SGDEM, also generalizes. Extending our results on generalization, we also develop an upper bound on the expected true risk, in terms of the number of training steps, sample size, and momentum. Our experimental evaluations verify the consistency between the numerical results and our theoretical bounds. SGDEM improves the generalization error of SGDM when training ResNet-18 on ImageNet in practical distributed settings.
Game-Based Video-Context Dialogue
Authors: Ramakanth Pasunuru, Mohit Bansal
Date: 2018-09-12 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Retrieval

Current dialogue systems focus more on textual and speech context knowledge
and are usually based on two speakers. Some recent work has investigated static
image-based dialogue. However, several real-world human interactions also
involve dynamic visual context (similar to videos) as well as dialogue
exchanges among multiple speakers. To move closer towards such multimodal
conversational skills and visually-situated applications, we introduce a new
video-context, many-speaker dialogue dataset based on live-broadcast soccer
game videos and chats from Twitch.tv. This challenging testbed allows us to
develop visually-grounded dialogue models that should generate relevant
temporal and spatial event language from the live video, while also being
relevant to the chat history. For strong baselines, we also present several
discriminative and generative models, e.g., based on tridirectional attention
flow (TriDAF). We evaluate these models via retrieval ranking-recall, automatic
phrase-matching metrics, as well as human evaluation studies. We also present
dataset analyses, model ablations, and visualizations to understand the
contribution of different modalities and model components.
Benchmarking and Optimization of Gradient Boosting Decision Tree Algorithms
Authors: Andreea Anghel, Nikolaos Papandreou, Thomas Parnell, Alessandro De Palma, Haralampos Pozidis
Date: 2018-09-12 00:00:00
Tasks: Bayesian Optimization, Benchmarking, GPU

Gradient boosting decision trees (GBDTs) have seen widespread adoption in
academia, industry and competitive data science due to their state-of-the-art
performance in many machine learning tasks. One relative downside to these
models is the large number of hyper-parameters that they expose to the
end-user. To maximize the predictive power of GBDT models, one must either
manually tune the hyper-parameters, or utilize automated techniques such as
those based on Bayesian optimization. Both of these approaches are
time-consuming since they involve repeatably training the model for different
sets of hyper-parameters. A number of software GBDT packages have started to
offer GPU acceleration which can help to alleviate this problem. In this paper,
we consider three such packages: XGBoost, LightGBM and Catboost. Firstly, we
evaluate the performance of the GPU acceleration provided by these packages
using large-scale datasets with varying shapes, sparsities and learning tasks.
Then, we compare the packages in the context of hyper-parameter optimization,
both in terms of how quickly each package converges to a good validation score,
and in terms of generalization performance.