pwc-archive/papers-with-abstracts huggingface dataset:
Entropic Regularization of Markov Decision Processes
Authors: Boris Belousov, Jan Peters
Date: 2019-07-06 00:00:00
Tasks: Reinforcement Learning

An optimal feedback controller for a given Markov decision process (MDP) can in principle be synthesized by value or policy iteration. However, if the system dynamics and the reward function are unknown, a learning agent must discover an optimal controller via direct interaction with the environment. Such interactive data gathering commonly leads to divergence towards dangerous or uninformative regions of the state space unless additional regularization measures are taken. Prior works proposed bounding the information loss measured by the Kullback-Leibler (KL) divergence at every policy improvement step to eliminate instability in the learning dynamics. In this paper, we consider a broader family of $f$-divergences, and more concretely $\alpha$-divergences, which inherit the beneficial property of providing the policy improvement step in closed form at the same time yielding a corresponding dual objective for policy evaluation. Such entropic proximal policy optimization view gives a unified perspective on compatible actor-critic architectures. In particular, common least-squares value function estimation coupled with advantage-weighted maximum likelihood policy improvement is shown to correspond to the Pearson $\chi^2$-divergence penalty. Other actor-critic pairs arise for various choices of the penalty-generating function $f$. On a concrete instantiation of our framework with the $\alpha$-divergence, we carry out asymptotic analysis of the solutions for different values of $\alpha$ and demonstrate the effects of the divergence function choice on common standard reinforcement learning problems.
On a method to construct exponential families by representation theory
Authors: Koichi Tojo, Taro Yoshino
Date: 2019-07-06 00:00:00

Exponential family plays an important role in information geometry. In arXiv:1811.01394, we introduced a method to construct an exponential family $\mathcal{P}=\{p_\theta\}_{\theta\in\Theta}$ on a homogeneous space $G/H$ from a pair $(V,v_0)$. Here $V$ is a representation of $G$ and $v_0$ is an $H$-fixed vector in $V$. Then the following questions naturally arise: (Q1) when is the correspondence $\theta\mapsto p_\theta$ injective? (Q2) when do distinct pairs $(V,v_0)$ and $(V',v_0')$ generate the same family? In this paper, we answer these two questions (Theorems 1 and 2). Moreover, in Section 3, we consider the case $(G,H)=(\mathbb{R}_{>0}, \{1\})$ with a certain representation on $\mathbb{R}^2$. Then we see the family obtained by our method is essentially generalized inverse Gaussian distribution (GIG).
Learning Neural Sequence-to-Sequence Models from Weak Feedback with Bipolar Ramp Loss
Authors: Laura Jehl, Carolin Lawrence, Stefan Riezler
Date: 2019-07-06 00:00:00
Proceeding: TACL 2019 3
Tasks: Machine Translation, Semantic Parsing, Translation

In many machine learning scenarios, supervision by gold labels is not available and consequently neural models cannot be trained directly by maximum likelihood estimation (MLE). In a weak supervision scenario, metric-augmented objectives can be employed to assign feedback to model outputs, which can be used to extract a supervision signal for training. We present several objectives for two separate weakly supervised tasks, machine translation and semantic parsing. We show that objectives should actively discourage negative outputs in addition to promoting a surrogate gold structure. This notion of bipolarity is naturally present in ramp loss objectives, which we adapt to neural models. We show that bipolar ramp loss objectives outperform other non-bipolar ramp loss objectives and minimum risk training (MRT) on both weakly supervised tasks, as well as on a supervised machine translation task. Additionally, we introduce a novel token-level ramp loss objective, which is able to outperform even the best sequence-level ramp loss on both weakly supervised tasks.
Deep Active Inference as Variational Policy Gradients
Authors: Beren Millidge
Date: 2019-07-08 00:00:00
Tasks: Bayesian Inference, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Active Inference is a theory of action arising from neuroscience which casts action and planning as a bayesian inference problem to be solved by minimizing a single quantity - the variational free energy. Active Inference promises a unifying account of action and perception coupled with a biologically plausible process theory. Despite these potential advantages, current implementations of Active Inference can only handle small, discrete policy and state-spaces and typically require the environmental dynamics to be known. In this paper we propose a novel deep Active Inference algorithm which approximates key densities using deep neural networks as flexible function approximators, which enables Active Inference to scale to significantly larger and more complex tasks. We demonstrate our approach on a suite of OpenAIGym benchmark tasks and obtain performance comparable with common reinforcement learning baselines. Moreover, our algorithm shows similarities with maximum entropy reinforcement learning and the policy gradients algorithm, which reveals interesting connections between the Active Inference framework and reinforcement learning.
Void region segmentation in ball grid array using u-net approach and synthetic data
Authors: Vijay Kumar Neeluru, Vikas Ahuja
Date: 2019-07-07 00:00:00

The quality inspection of solder balls by detecting and measuring the void is important to improve the board yield issues in electronic circuits. In general, the inspection is carried out manually, based on 2D or 3D X-ray images. For high quality inspection, it is difficult to detect and measure voids accurately with high repeatability through the manual inspection and the process is time consuming. In need of high quality and fast inspection, various approaches were proposed, but, due to the various challenges like vias, reflections from the plating or vias, inconsistent lighting, noise and void-like artifacts makes these approaches difficult to work in all these challenging conditions. In recent times, deep learning approaches are providing the outstanding accuracy in various computer vision tasks. Considering the need of high quality and fast inspection, in this paper, we applied U-Net to segment the void regions in soldering balls. As it is difficult to get the annotated dataset covering all the variations of void, we proposed an approach to generated the synthetic dataset. The proposed approach is able to segment the voids and can be easily scaled to various electronic products.
Multilingual Universal Sentence Encoder for Semantic Retrieval
Authors: Yinfei Yang, Daniel Cer, Amin Ahmad, Mandy Guo, Jax Law, Noah Constant, Gustavo Hernandez Abrego, Steve Yuan, Chris Tar, Yun-Hsuan Sung, Brian Strope, Ray Kurzweil
Date: 2019-07-09 00:00:00
Proceeding: ACL 2020 6
Tasks: Question Answering, Retrieval, Semantic Retrieval, Sentence, Sentence Embedding, Sentence-Embedding, Transfer Learning, Translation

We introduce two pre-trained retrieval focused multilingual sentence encoding models, respectively based on the Transformer and CNN model architectures. The models embed text from 16 languages into a single semantic space using a multi-task trained dual-encoder that learns tied representations using translation based bridge tasks (Chidambaram al., 2018). The models provide performance that is competitive with the state-of-the-art on: semantic retrieval (SR), translation pair bitext retrieval (BR) and retrieval question answering (ReQA). On English transfer learning tasks, our sentence-level embeddings approach, and in some cases exceed, the performance of monolingual, English only, sentence embedding models. Our models are made available for download on TensorFlow Hub.
An Attention Mechanism for Musical Instrument Recognition
Authors: Siddharth Gururani, Mohit Sharma, Alexander Lerch
Date: 2019-07-09 00:00:00
Tasks: Instrument Recognition

While the automatic recognition of musical instruments has seen significant progress, the task is still considered hard for music featuring multiple instruments as opposed to single instrument recordings. Datasets for polyphonic instrument recognition can be categorized into roughly two categories. Some, such as MedleyDB, have strong per-frame instrument activity annotations but are usually small in size. Other, larger datasets such as OpenMIC only have weak labels, i.e., instrument presence or absence is annotated only for long snippets of a song. We explore an attention mechanism for handling weakly labeled data for multi-label instrument recognition. Attention has been found to perform well for other tasks with weakly labeled data. We compare the proposed attention model to multiple models which include a baseline binary relevance random forest, recurrent neural network, and fully connected neural networks. Our results show that incorporating attention leads to an overall improvement in classification accuracy metrics across all 20 instruments in the OpenMIC dataset. We find that attention enables models to focus on (or `attend to') specific time segments in the audio relevant to each instrument label leading to interpretable results.
A Scheme for Dynamic Risk-Sensitive Sequential Decision Making
Authors: Shuai Ma, Jia Yuan Yu, Ahmet Satir
Date: 2019-07-09 00:00:00
Tasks: Decision Making, Sequential Decision Making

We present a scheme for sequential decision making with a risk-sensitive objective and constraints in a dynamic environment. A neural network is trained as an approximator of the mapping from parameter space to space of risk and policy with risk-sensitive constraints. For a given risk-sensitive problem, in which the objective and constraints are, or can be estimated by, functions of the mean and variance of return, we generate a synthetic dataset as training data. Parameters defining a targeted process might be dynamic, i.e., they might vary over time, so we sample them within specified intervals to deal with these dynamics. We show that: i). Most risk measures can be estimated using return variance; ii). By virtue of the state-augmentation transformation, practical problems modeled by Markov decision processes with stochastic rewards can be solved in a risk-sensitive scenario; and iii). The proposed scheme is validated by a numerical experiment.
Translating neural signals to text using a Brain-Machine Interface
Authors: Janaki Sheth, Ariel Tankus, Michelle Tran, Nader Pouratian, Itzhak Fried, William Speier
Date: 2019-07-09 00:00:00

Brain-Computer Interfaces (BCI) help patients with faltering communication abilities due to neurodegenerative diseases produce text or speech output by direct neural processing. However, practical implementation of such a system has proven difficult due to limitations in speed, accuracy, and generalizability of the existing interfaces. To this end, we aim to create a BCI system that decodes text directly from neural signals. We implement a framework that initially isolates frequency bands in the input signal encapsulating differential information regarding production of various phonemic classes. These bands then form a feature set that feeds into an LSTM which discerns at each time point probability distributions across all phonemes uttered by a subject. Finally, these probabilities are fed into a particle filtering algorithm which incorporates prior knowledge of the English language to output text corresponding to the decoded word. Performance of this model on data obtained from six patients shows encouragingly high levels of accuracy at speeds and bit rates significantly higher than existing BCI communication systems. Further, in producing an output, our network abstains from constraining the reconstructed word to be from a given bag-of-words, unlike previous studies. The success of our proposed approach, offers promise for the employment of a BCI interface by patients in unfettered, naturalistic environments.
Analyzing Phonetic and Graphemic Representations in End-to-End Automatic Speech Recognition
Authors: Yonatan Belinkov, Ahmed Ali, James Glass
Date: 2019-07-09 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Language Modeling, Language Modelling, speech-recognition, Speech Recognition

End-to-end neural network systems for automatic speech recognition (ASR) are trained from acoustic features to text transcriptions. In contrast to modular ASR systems, which contain separately-trained components for acoustic modeling, pronunciation lexicon, and language modeling, the end-to-end paradigm is both conceptually simpler and has the potential benefit of training the entire system on the end task. However, such neural network models are more opaque: it is not clear how to interpret the role of different parts of the network and what information it learns during training. In this paper, we analyze the learned internal representations in an end-to-end ASR model. We evaluate the representation quality in terms of several classification tasks, comparing phonemes and graphemes, as well as different articulatory features. We study two languages (English and Arabic) and three datasets, finding remarkable consistency in how different properties are represented in different layers of the deep neural network.
Sentiment and position-taking analysis of parliamentary debates: A systematic literature review
Authors: Gavin Abercrombie, Riza Batista-Navarro
Date: 2019-07-09 00:00:00
Tasks: Position, Systematic Literature Review

Parliamentary and legislative debate transcripts provide access to information concerning the opinions, positions and policy preferences of elected politicians. They attract attention from researchers from a wide variety of backgrounds, from political and social sciences to computer science. As a result, the problem of automatic sentiment and position-taking analysis has been tackled from different perspectives, using varying approaches and methods, and with relatively little collaboration or cross-pollination of ideas. The existing research is scattered across publications from various fields and venues. In this article we present the results of a systematic literature review of 61 studies, all of which address the automatic analysis of the sentiment and opinions expressed and positions taken by speakers in parliamentary (and other legislative) debates. In this review, we discuss the available research with regard to the aims and objectives of the researchers who work on these problems, the automatic analysis tasks they undertake, and the approaches and methods they use. We conclude by summarizing their findings, discussing the challenges of applying computational analysis to parliamentary debates, and suggesting possible avenues for further research.
On the Semantic Interpretability of Artificial Intelligence Models
Authors: Vivian S. Silva, André Freitas, Siegfried Handschuh
Date: 2019-07-09 00:00:00
Tasks: Decision Making

Artificial Intelligence models are becoming increasingly more powerful and accurate, supporting or even replacing humans' decision making. But with increased power and accuracy also comes higher complexity, making it hard for users to understand how the model works and what the reasons behind its predictions are. Humans must explain and justify their decisions, and so do the AI models supporting them in this process, making semantic interpretability an emerging field of study. In this work, we look at interpretability from a broader point of view, going beyond the machine learning scope and covering different AI fields such as distributional semantics and fuzzy logic, among others. We examine and classify the models according to their nature and also based on how they introduce interpretability features, analyzing how each approach affects the final users and pointing to gaps that still need to be addressed to provide more human-centered interpretability solutions.
Using Temporal and Topological Features for Intrusion Detection in Operational Networks
Authors: Simon D. Duque Anton, Daniel Fraunholz, Hans Dieter Schotten
Date: 2019-07-09 00:00:00
Tasks: Intrusion Detection, Time Series, Time Series Analysis

Until two decades ago, industrial networks were deemed secure due to physical separation from public networks. An abundance of successful attacks proved that assumption wrong. Intrusion detection solutions for industrial application need to meet certain requirements that differ from home- and office-environments, such as working without feedback to the process and compatibility with legacy systems. Industrial systems are commonly used for several decades, updates are often difficult and expensive. Furthermore, most industrial protocols do not have inherent authentication or encryption mechanisms, allowing for easy lateral movement of an intruder once the perimeter is breached. In this work, an algorithm for motif discovery in time series, Matrix Profiles, is used to detect outliers in the timing behaviour of an industrial process. This process was monitored in an experimental environment, containing ground truth labels after attacks were performed. Furthermore, the graph representations of a different industrial data set that has been emulated are used to detect malicious activities. These activities can be derived from anomalous communication patterns, represented as edges in the graph. Finally, an integration concept for both methods is proposed.
Multitask Learning for Blackmarket Tweet Detection
Authors: Udit Arora, William Scott Paka, Tanmoy Chakraborty
Date: 2019-07-09 00:00:00

Online social media platforms have made the world more connected than ever before, thereby making it easier for everyone to spread their content across a wide variety of audiences. Twitter is one such popular platform where people publish tweets to spread their messages to everyone. Twitter allows users to Retweet other users' tweets in order to broadcast it to their network. The more retweets a particular tweet gets, the faster it spreads. This creates incentives for people to obtain artificial growth in the reach of their tweets by using certain blackmarket services to gain inorganic appraisals for their content. In this paper, we attempt to detect such tweets that have been posted on these blackmarket services in order to gain artificially boosted retweets. We use a multitask learning framework to leverage soft parameter sharing between a classification and a regression based task on separate inputs. This allows us to effectively detect tweets that have been posted to these blackmarket services, achieving an F1-score of 0.89 when classifying tweets as blackmarket or genuine.
Event-based attention and tracking on neuromorphic hardware
Authors: Alpha Renner, Matthew Evanusa, Yulia Sandamirskaya
Date: 2019-07-09 00:00:00
Tasks: Object, Object Tracking

We present a fully event-driven vision and processing system for selective attention and tracking, realized on a neuromorphic processor Loihi interfaced to an event-based Dynamic Vision Sensor DAVIS. The attention mechanism is realized as a recurrent spiking neural network that implements attractor-dynamics of dynamic neural fields. We demonstrate capability of the system to create sustained activation that supports object tracking when distractors are present or when the object slows down or stops, reducing the number of generated events.
Procedural Content Generation through Quality Diversity
Authors: Daniele Gravina, Ahmed Khalifa, Antonios Liapis, Julian Togelius, Georgios N. Yannakakis
Date: 2019-07-09 00:00:00
Tasks: Diversity, Evolutionary Algorithms

Quality-diversity (QD) algorithms search for a set of good solutions which cover a space as defined by behavior metrics. This simultaneous focus on quality and diversity with explicit metrics sets QD algorithms apart from standard single- and multi-objective evolutionary algorithms, as well as from diversity preservation approaches such as niching. These properties open up new avenues for artificial intelligence in games, in particular for procedural content generation. Creating multiple systematically varying solutions allows new approaches to creative human-AI interaction as well as adaptivity. In the last few years, a handful of applications of QD to procedural content generation and game playing have been proposed; we discuss these and propose challenges for future work.
Implicit Discourse Relation Identification for Open-domain Dialogues
Authors: Mingyu Derek Ma, Kevin K. Bowden, Jiaqi Wu, Wen Cui, Marilyn Walker
Date: 2019-07-09 00:00:00
Proceeding: ACL 2019 7
Tasks: Implicit Discourse Relation Classification, Implicit Relations, Relation, valid

Discourse relation identification has been an active area of research for many years, and the challenge of identifying implicit relations remains largely an unsolved task, especially in the context of an open-domain dialogue system. Previous work primarily relies on a corpora of formal text which is inherently non-dialogic, i.e., news and journals. This data however is not suitable to handle the nuances of informal dialogue nor is it capable of navigating the plethora of valid topics present in open-domain dialogue. In this paper, we designed a novel discourse relation identification pipeline specifically tuned for open-domain dialogue systems. We firstly propose a method to automatically extract the implicit discourse relation argument pairs and labels from a dataset of dialogic turns, resulting in a novel corpus of discourse relation pairs; the first of its kind to attempt to identify the discourse relations connecting the dialogic turns in open-domain discourse. Moreover, we have taken the first steps to leverage the dialogue features unique to our task to further improve the identification of such relations by performing feature ablation and incorporating dialogue features to enhance the state-of-the-art model.
Systematic quantitative analyses reveal the folk-zoological knowledge embedded in folktales
Authors: Yo Nakawake, Kosuke Sato
Date: 2019-07-09 00:00:00
Tasks: Descriptive

Cultural learning is a unique human capacity essential for a wide range of adaptations. Researchers have argued that folktales have the pedagogical function of transmitting the essential information for the environment. The most important knowledge for foraging and pastoral society is folk-zoological knowledge, such as the predator-prey relationship among wild animals, or between wild and domesticated animals. Here, we analysed the descriptions of the 382 animal folktales using the natural language processing method and descriptive statistics listed in a worldwide tale-type index (Aarne-Thompson-Uther type index). Our analyses suggested that first, the predator-prey relationship frequently appeared in a co-occurrent animal pair within a folktale (e.g., cat and mouse or wolf and pig), and second, the motif of 'deception', describing the antagonistic behaviour among animals, appeared relatively higher in 'wild and domestic animals' and 'wild animals' than other types. Furthermore, the motif of 'deception' appeared more frequently in pairs, corresponding to the predator-prey relationship. These results corresponded with the hypothesis that the combination of animal characters and what happens in stories represented relationships in the real world. The present study demonstrated that the combination of quantitative methods and qualitative data broaden our understanding of the evolutionary aspects of human cultures.