pwc-archive/papers-with-abstracts huggingface dataset:
Marginal Densities, Factor Graph Duality, and High-Temperature Series Expansions
Authors: Mehdi Molkaraie
Date: 2019-01-07 00:00:00
Tasks: Vocal Bursts Intensity Prediction

We prove that the marginal densities of a global probability mass function in a primal normal factor graph and the corresponding marginal densities in the dual normal factor graph are related via local mappings. The mapping depends on the Fourier transform of the local factors of the models. Details of the mapping, including its fixed points, are derived for the Ising model, and then extended to the Potts model. By employing the mapping, we can transform simultaneously all the estimated marginal densities from one domain to the other, which is advantageous if estimating the marginals can be carried out more efficiently in the dual domain. An example of particular significance is the ferromagnetic Ising model in a positive external field, for which there is a rapidly mixing Markov chain (called the subgraphs-world process) to generate configurations in the dual normal factor graph of the model. Our numerical experiments illustrate that the proposed procedure can provide more accurate estimates of marginal densities in various settings.
Reliable and Explainable Machine Learning Methods for Accelerated Material Discovery
Authors: Bhavya Kailkhura, Brian Gallagher, Sookyung Kim, Anna Hiszpanski, T. Yong-Jin Han
Date: 2019-01-05 00:00:00
Tasks: BIG-bench Machine Learning, Transfer Learning

Material scientists are increasingly adopting the use of machine learning
(ML) for making potentially important decisions, such as, discovery,
development, optimization, synthesis and characterization of materials.
However, despite ML's impressive performance in commercial applications,
several unique challenges exist when applying ML in materials science
applications. In such a context, the contributions of this work are twofold.
First, we identify common pitfalls of existing ML techniques when learning from
underrepresented/imbalanced material data. Specifically, we show that with
imbalanced data, standard methods for assessing quality of ML models break down
and lead to misleading conclusions. Furthermore, we found that the model's own
confidence score cannot be trusted and model introspection methods (using
simpler models) do not help as they result in loss of predictive performance
(reliability-explainability trade-off). Second, to overcome these challenges,
we propose a general-purpose explainable and reliable machine-learning
framework. Specifically, we propose a novel pipeline that employs an ensemble
of simpler models to reliably predict material properties. We also propose a
transfer learning technique and show that the performance loss due to models'
simplicity can be overcome by exploiting correlations among different material
properties. A new evaluation metric and a trust score to better quantify the
confidence in the predictions are also proposed. To improve the
interpretability, we add a rationale generator component to our framework which
provides both model-level and decision-level explanations. Finally, we
demonstrate the versatility of our technique on two applications: 1) predicting
properties of crystalline compounds, and 2) identifying novel potentially
stable solar cell materials.
Forecasting residential gas demand: machine learning approaches and seasonal role of temperature forecasts
Authors: Andrea Marziali, Emanuele Fabbiani, Giuseppe De Nicolao
Date: 2019-01-04 00:00:00
Tasks: BIG-bench Machine Learning, Demand Forecasting

Gas demand forecasting is a critical task for energy providers as it impacts on pipe reservation and stock planning. In this paper, the one-day-ahead forecasting of residential gas demand at country level is investigated by implementing and comparing five models: Ridge Regression, Gaussian Process (GP), k-Nearest Neighbour, Artificial Neural Network (ANN), and Torus Model. Italian demand data from 2007 to 2017 are used for training and testing the proposed algorithms. The choice of the relevant covariates and the most significant aspects of the pre-processing and feature extraction steps are discussed in-depth, lending particular attention to the role of one-day-ahead temperature forecasts. Our best model, in terms of Root Mean Squared Error (RMSE), is the ANN, closely followed by the GP. If the Mean Absolute Error (MAE) is taken as an error measure, the GP becomes the best model, although by a narrow margin. A main novel contribution is the development of a model describing the propagation of temperature errors to gas forecasting errors that is successfully validated on experimental data. Being able to predict the quantitative impact of temperature forecasts on gas forecasts could be useful in order to assess potential improvement margins associated with more sophisticated weather forecasts. On the Italian data, it is shown that temperature forecast errors account for some 18% of the mean squared error of gas demand forecasts provided by ANN.
A Solution to Time-Varying Markov Decision Processes
Authors: Lantao Liu, Gaurav S. Sukhatme
Date: 2016-05-03 00:00:00
Tasks: Decision Making

We consider a decision-making problem where the environment varies both in
space and time. Such problems arise naturally when considering e.g., the
navigation of an underwater robot amidst ocean currents or the navigation of an
aerial vehicle in wind. To model such spatiotemporal variation, we extend the
standard Markov Decision Process (MDP) to a new framework called the
Time-Varying Markov Decision Process (TVMDP). The TVMDP has a time-varying
state transition model and transforms the standard MDP that considers only
immediate and static uncertainty descriptions of state transitions, to a
framework that is able to adapt to future time-varying transition dynamics over
some horizon. We show how to solve a TVMDP via a redesign of the MDP value
propagation mechanisms by incorporating the introduced dynamics along the
temporal dimension. We validate our framework in a marine robotics navigation
setting using spatiotemporal ocean data and show that it outperforms prior
efforts.
Lattice Identification and Separation: Theory and Algorithm
Authors: Yuchen He, Sung Ha Kang
Date: 2018-12-19 00:00:00
Tasks: Boundary Detection

Motivated by lattice mixture identification and grain boundary detection, we
present a framework for lattice pattern representation and comparison, and
propose an efficient algorithm for lattice separation. We define new scale and
shape descriptors, which helps to considerably reduce the size of equivalence
classes of lattice bases. These finitely many equivalence relations are fully
characterized by modular group theory. We construct the lattice space
$\mathscr{L}$ based on the equivalent descriptors and define a metric
$d_{\mathscr{L}}$ to accurately quantify the visual similarities and
differences between lattices. Furthermore, we introduce the Lattice
Identification and Separation Algorithm (LISA), which identifies each lattice
patterns from superposed lattices. LISA finds lattice candidates from the high
responses in the image spectrum, then sequentially extracts different layers of
lattice patterns one by one. Analyzing the frequency components, we reveal the
intricate dependency of LISA's performances on particle radius, lattice
density, and relative translations. Various numerical experiments are designed
to show LISA's robustness against a large number of lattice layers, moir\'{e}
patterns and missing particles.
Sentence Rewriting for Semantic Parsing
Authors: Bo Chen, Le Sun, Xianpei Han, Bo An
Date: 2019-01-10 00:00:00
Proceeding: ACL 2016 8
Tasks: Form, Semantic Parsing, Sentence, Sentence ReWriting

A major challenge of semantic parsing is the vocabulary mismatch problem
between natural language and target ontology. In this paper, we propose a
sentence rewriting based semantic parsing method, which can effectively resolve
the mismatch problem by rewriting a sentence into a new form which has the same
structure with its target logical form. Specifically, we propose two
sentence-rewriting methods for two common types of mismatch: a dictionary-based
method for 1-N mismatch and a template-based method for N-1 mismatch. We
evaluate our entence rewriting based semantic parser on the benchmark semantic
parsing dataset -- WEBQUESTIONS. Experimental results show that our system
outperforms the base system with a 3.4% gain in F1, and generates logical forms
more accurately and parses sentences more robustly.
Equalizing Gender Biases in Neural Machine Translation with Word Embeddings Techniques
Authors: Joel Escudé Font, Marta R. Costa-jussà
Date: 2019-01-10 00:00:00
Tasks: Fairness, Machine Translation, Translation, Word Embeddings

Neural machine translation has significantly pushed forward the quality of the field. However, there are remaining big issues with the output translations and one of them is fairness. Neural models are trained on large text corpora which contain biases and stereotypes. As a consequence, models inherit these social biases. Recent methods have shown results in reducing gender bias in other natural language processing tools such as word embeddings. We take advantage of the fact that word embeddings are used in neural machine translation to propose a method to equalize gender biases in neural machine translation using these representations. Specifically, we propose, experiment and analyze the integration of two debiasing techniques over GloVe embeddings in the Transformer translation architecture. We evaluate our proposed system on the WMT English-Spanish benchmark task, showing gains up to one BLEU point. As for the gender bias evaluation, we generate a test set of occupations and we show that our proposed system learns to equalize existing biases from the baseline system.
Emotion Detection using Data Driven Models
Authors: Naveenkumar K S, Vinayakumar R, Soman Kp
Date: 2019-01-10 00:00:00
Tasks: Decision Making

Text is the major method that is used for communication now a days, each and
every day lots of text are created. In this paper the text data is used for the
classification of the emotions. Emotions are the way of expression of the
persons feelings which has an high influence on the decision making tasks.
Datasets are collected which are available publically and combined together
based on the three emotions that are considered here positive, negative and
neutral. In this paper we have proposed the text representation method TFIDF
and keras embedding and then given to the classical machine learning algorithms
of which Logistics Regression gives the highest accuracy of about 75.6%, after
which it is passed to the deep learning algorithm which is the CNN which gives
the state of art accuracy of about 45.25%. For the research purpose the
datasets that has been collected are released.
Self-Monitoring Navigation Agent via Auxiliary Progress Estimation
Authors: Chih-Yao Ma, Jiasen Lu, Zuxuan Wu, Ghassan AlRegib, Zsolt Kira, Richard Socher, Caiming Xiong
Date: 2019-01-10 00:00:00
Proceeding: ICLR 2019 5
Tasks: Natural Language Visual Grounding, Vision and Language Navigation, Vision-Language Navigation, Visual Navigation

The Vision-and-Language Navigation (VLN) task entails an agent following
navigational instruction in photo-realistic unknown environments. This
challenging task demands that the agent be aware of which instruction was
completed, which instruction is needed next, which way to go, and its
navigation progress towards the goal. In this paper, we introduce a
self-monitoring agent with two complementary components: (1) visual-textual
co-grounding module to locate the instruction completed in the past, the
instruction required for the next action, and the next moving direction from
surrounding images and (2) progress monitor to ensure the grounded instruction
correctly reflects the navigation progress. We test our self-monitoring agent
on a standard benchmark and analyze our proposed approach through a series of
ablation studies that elucidate the contributions of the primary components.
Using our proposed method, we set the new state of the art by a significant
margin (8% absolute increase in success rate on the unseen test set). Code is
available at https://github.com/chihyaoma/selfmonitoring-agent .
Reverse-Engineering Satire, or "Paper on Computational Humor Accepted Despite Making Serious Advances"
Authors: Robert West, Eric Horvitz
Date: 2019-01-10 00:00:00
Tasks: Humor Detection, Sentence

Humor is an essential human trait. Efforts to understand humor have called out links between humor and the foundations of cognition, as well as the importance of humor in social engagement. As such, it is a promising and important subject of study, with relevance for artificial intelligence and human-computer interaction. Previous computational work on humor has mostly operated at a coarse level of granularity, e.g., predicting whether an entire sentence, paragraph, document, etc., is humorous. As a step toward deep understanding of humor, we seek fine-grained models of attributes that make a given text humorous. Starting from the observation that satirical news headlines tend to resemble serious news headlines, we build and analyze a corpus of satirical headlines paired with nearly identical but serious headlines. The corpus is constructed via Unfun.me, an online game that incentivizes players to make minimal edits to satirical headlines with the goal of making other players believe the results are serious headlines. The edit operations used to successfully remove humor pinpoint the words and concepts that play a key role in making the original, satirical headline funny. Our analysis reveals that the humor tends to reside toward the end of headlines, and primarily in noun phrases, and that most satirical headlines follow a certain logical pattern, which we term false analogy. Overall, this paper deepens our understanding of the syntactic and semantic structure of satirical news headlines and provides insights for building humor-producing systems.
Revealing interpretable object representations from human behavior
Authors: Charles Y. Zheng, Francisco Pereira, Chris I. Baker, Martin N. Hebart
Date: 2019-01-09 00:00:00
Proceeding: ICLR 2019 5
Tasks: Object

To study how mental object representations are related to behavior, we
estimated sparse, non-negative representations of objects using human
behavioral judgments on images representative of 1,854 object categories. These
representations predicted a latent similarity structure between objects, which
captured most of the explainable variance in human behavioral judgments.
Individual dimensions in the low-dimensional embedding were found to be highly
reproducible and interpretable as conveying degrees of taxonomic membership,
functionality, and perceptual attributes. We further demonstrated the
predictive power of the embeddings for explaining other forms of human
behavior, including categorization, typicality judgments, and feature ratings,
suggesting that the dimensions reflect human conceptual representations of
objects beyond the specific task.
Beyond the EM Algorithm: Constrained Optimization Methods for Latent Class Model
Authors: Hao Chen, Lanshan Han, Alvin Lim
Date: 2019-01-09 00:00:00
Tasks: Marketing

Latent class model (LCM), which is a finite mixture of different categorical distributions, is one of the most widely used models in statistics and machine learning fields. Because of its non-continuous nature and the flexibility in shape, researchers in practice areas such as marketing and social sciences also frequently use LCM to gain insights from their data. One likelihood-based method, the Expectation-Maximization (EM) algorithm, is often used to obtain the model estimators. However, the EM algorithm is well-known for its notoriously slow convergence. In this research, we explore alternative likelihood-based methods that can potential remedy the slow convergence of the EM algorithm. More specifically, we regard likelihood-based approach as a constrained nonlinear optimization problem, and apply quasi-Newton type methods to solve them. We examine two different constrained optimization methods to maximize the log likelihood function. We present simulation study results to show that the proposed methods not only converge in less iterations than the EM algorithm but also produce more accurate model estimators.
Gaussian processes with linear operator inequality constraints
Authors: Christian Agrell
Date: 2019-01-10 00:00:00
Tasks: Gaussian Processes

This paper presents an approach for constrained Gaussian Process (GP) regression where we assume that a set of linear transformations of the process are bounded. It is motivated by machine learning applications for high-consequence engineering systems, where this kind of information is often made available from phenomenological knowledge. We consider a GP $f$ over functions on $\mathcal{X} \subset \mathbb{R}^{n}$ taking values in $\mathbb{R}$, where the process $\mathcal{L}f$ is still Gaussian when $\mathcal{L}$ is a linear operator. Our goal is to model $f$ under the constraint that realizations of $\mathcal{L}f$ are confined to a convex set of functions. In particular, we require that $a \leq \mathcal{L}f \leq b$, given two functions $a$ and $b$ where $a < b$ pointwise. This formulation provides a consistent way of encoding multiple linear constraints, such as shape-constraints based on e.g. boundedness, monotonicity or convexity. We adopt the approach of using a sufficiently dense set of virtual observation locations where the constraint is required to hold, and derive the exact posterior for a conjugate likelihood. The results needed for stable numerical implementation are derived, together with an efficient sampling scheme for estimating the posterior process.
Variable Importance Clouds: A Way to Explore Variable Importance for the Set of Good Models
Authors: Jiayun Dong, Cynthia Rudin
Date: 2019-01-10 00:00:00
Tasks: Causal Inference, image-classification, Image Classification, Marketing

Variable importance is central to scientific studies, including the social sciences and causal inference, healthcare, and other domains. However, current notions of variable importance are often tied to a specific predictive model. This is problematic: what if there were multiple well-performing predictive models, and a specific variable is important to some of them and not to others? In that case, we may not be able to tell from a single well-performing model whether a variable is always important in predicting the outcome. Rather than depending on variable importance for a single predictive model, we would like to explore variable importance for all approximately-equally-accurate predictive models. This work introduces the concept of a variable importance cloud, which maps every variable to its importance for every good predictive model. We show properties of the variable importance cloud and draw connections to other areas of statistics. We introduce variable importance diagrams as a projection of the variable importance cloud into two dimensions for visualization purposes. Experiments with criminal justice, marketing data, and image classification tasks illustrate how variables can change dramatically in importance for approximately-equally-accurate predictive models
A Bayesian Decision Tree Algorithm
Authors: Giuseppe Nuti, Lluís Antoni Jiménez Rugama, Andreea-Ingrid Cross
Date: 2019-01-10 00:00:00
Tasks: General Classification, regression

Bayesian Decision Trees are known for their probabilistic interpretability. However, their construction can sometimes be costly. In this article we present a general Bayesian Decision Tree algorithm applicable to both regression and classification problems. The algorithm does not apply Markov Chain Monte Carlo and does not require a pruning step. While it is possible to construct a weighted probability tree space we find that one particular tree, the greedy-modal tree (GMT), explains most of the information contained in the numerical examples. This approach seems to perform similarly to Random Forests.
Closed-form Expressions for Maximum Mean Discrepancy with Applications to Wasserstein Auto-Encoders
Authors: Raif M. Rustamov
Date: 2019-01-10 00:00:00
Tasks: Form

The Maximum Mean Discrepancy (MMD) has found numerous applications in statistics and machine learning, most recently as a penalty in the Wasserstein Auto-Encoder (WAE). In this paper we compute closed-form expressions for estimating the Gaussian kernel based MMD between a given distribution and the standard multivariate normal distribution. This formula reveals a connection to the Baringhaus-Henze-Epps-Pulley (BHEP) statistic of the Henze-Zirkler test and provides further insights about the MMD. We introduce the standardized version of MMD as a penalty for the WAE training objective, allowing for a better interpretability of MMD values and more compatibility across different hyperparameter settings. Next, we propose using a version of batch normalization at the code layer; this has the benefits of making the kernel width selection easier, reducing the training effort, and preventing outliers in the aggregate code distribution. Our experiments on synthetic and real data show that the analytic formulation improves over the commonly used stochastic approximation of the MMD, and demonstrate that code normalization provides significant benefits when training WAEs.
Deep Learning for Human Affect Recognition: Insights and New Developments
Authors: Philipp V. Rouast, Marc T. P. Adam, Raymond Chiong
Date: 2019-01-09 00:00:00
Tasks: Deep Learning

Automatic human affect recognition is a key step towards more natural
human-computer interaction. Recent trends include recognition in the wild using
a fusion of audiovisual and physiological sensors, a challenging setting for
conventional machine learning algorithms. Since 2010, novel deep learning
algorithms have been applied increasingly in this field. In this paper, we
review the literature on human affect recognition between 2010 and 2017, with a
special focus on approaches using deep neural networks. By classifying a total
of 950 studies according to their usage of shallow or deep architectures, we
are able to show a trend towards deep learning. Reviewing a subset of 233
studies that employ deep neural networks, we comprehensively quantify their
applications in this field. We find that deep learning is used for learning of
(i) spatial feature representations, (ii) temporal feature representations, and
(iii) joint feature representations for multimodal sensor data. Exemplary
state-of-the-art architectures illustrate the progress. Our findings show the
role deep architectures will play in human affect recognition, and can serve as
a reference point for researchers working on related applications.
TraceCaps: A Capsule-based Neural Network for Semantic Segmentation
Authors: Tao Sun, Zhewei Wang, C. D. Smith, Jundong Liu
Date: 2019-01-09 00:00:00
Proceeding: ICLR 2019 5
Tasks: Segmentation, Semantic Segmentation

In this paper, we propose a capsule-based neural network model to solve the semantic segmentation problem. By taking advantage of the extractable part-whole dependencies available in capsule layers, we derive the probabilities of the class labels for individual capsules through a recursive, layer-by-layer procedure. We model this procedure as a traceback pipeline and take it as a central piece to build an end-to-end segmentation network. Under the proposed framework, image-level class labels and object boundaries are jointly sought in an explicit manner, which poses a significant advantage over the state-of-the-art fully convolutional network (FCN) solutions. With the capability to extracted part-whole information, our traceback pipeline can potentially be utilized as the building blocks to design interpretable neural networks. Experiments conducted on modified MNIST and neuroimages demonstrate that our model considerably enhance the segmentation performance compared to the leading FCN variants.