pwc-archive/papers-with-abstracts huggingface dataset:
Learning Nearly Decomposable Value Functions Via Communication Minimization
Authors: Tonghan Wang, Jianhao Wang, Chongyi Zheng, Chongjie Zhang
Date: 2019-10-11 00:00:00
Proceeding: ICLR 2020 1
Tasks: Reinforcement Learning, Starcraft

Reinforcement learning encounters major challenges in multi-agent settings, such as scalability and non-stationarity. Recently, value function factorization learning emerges as a promising way to address these challenges in collaborative multi-agent systems. However, existing methods have been focusing on learning fully decentralized value functions, which are not efficient for tasks requiring communication. To address this limitation, this paper presents a novel framework for learning nearly decomposable Q-functions (NDQ) via communication minimization, with which agents act on their own most of the time but occasionally send messages to other agents in order for effective coordination. This framework hybridizes value function factorization learning and communication learning by introducing two information-theoretic regularizers. These regularizers are maximizing mutual information between agents' action selection and communication messages while minimizing the entropy of messages between agents. We show how to optimize these regularizers in a way that is easily integrated with existing value function factorization methods such as QMIX. Finally, we demonstrate that, on the StarCraft unit micromanagement benchmark, our framework significantly outperforms baseline methods and allows us to cut off more than $80\%$ of communication without sacrificing the performance. The videos of our experiments are available at https://sites.google.com/view/ndq.
Deep Independently Recurrent Neural Network (IndRNN)
Authors: Shuai Li, Wanqing Li, Chris Cook, Yanbo Gao
Date: 2019-10-11 00:00:00
Tasks: Language Modelling, Sequential Image Classification, Skeleton Based Action Recognition

Recurrent neural networks (RNNs) are known to be difficult to train due to the gradient vanishing and exploding problems and thus difficult to learn long-term patterns and construct deep networks. To address these problems, this paper proposes a new type of RNNs with the recurrent connection formulated as Hadamard product, referred to as independently recurrent neural network (IndRNN), where neurons in the same layer are independent of each other and connected across layers. Due to the better behaved gradient backpropagation, IndRNN with regulated recurrent weights effectively addresses the gradient vanishing and exploding problems and thus long-term dependencies can be learned. Moreover, an IndRNN can work with non-saturated activation functions such as ReLU (rectified linear unit) and be still trained robustly. Different deeper IndRNN architectures, including the basic stacked IndRNN, residual IndRNN and densely connected IndRNN, have been investigated, all of which can be much deeper than the existing RNNs. Furthermore, IndRNN reduces the computation at each time step and can be over 10 times faster than the commonly used Long short-term memory (LSTM). Experimental results have shown that the proposed IndRNN is able to process very long sequences and construct very deep networks. Better performance has been achieved on various tasks with IndRNNs compared with the traditional RNN, LSTM and the popular Transformer.
An evolutionary approach to continuously estimate CPR quality parameters from a wrist-worn inertial sensor
Authors: Christian Lins, Björn Friedrich, Andreas Hein, Sebastian Fudickar
Date: 2019-10-11 00:00:00

Cardiopulmonary resuscitation (CPR) is one of the most critical emergency interventions for sudden cardiac arrest. In this paper, a robust sinusoidal model-fitting method based on a Evolution Strategy inspired algorithm for CPR quality parameters -- naming chest compression frequency and depth -- as measured by an inertial measurement unit (IMU) attached to the wrist is presented. The proposed approach will allow bystanders to improve CPR as part of a continuous closed-loop support system once integrated into a smartphone or smartwatch application. By evaluating the model's precision with data recorded by a training mannequin as reference standard, a variance for the compression frequency of $\pm 2.22$ compressions per minute (cpm) has been found for the IMU attached to the wrist. It was found that this previously unconsidered position and thus, the use of smartwatches is a suitable alternative to the typical placement of phones in hand for CPR training.
Accelerometer-Based Gait Segmentation: Simultaneously User and Adversary Identification
Authors: Yujia Ding, Weiqing Gu
Date: 2019-10-11 00:00:00
Tasks: Binary Classification, General Classification, Small Data Image Classification, Time Series, Time Series Analysis

In this paper, we introduce a new gait segmentation method based on accelerometer data and develop a new distance function between two time series, showing novel and effectiveness in simultaneously identifying user and adversary. Comparing with the normally used Neural Network methods, our approaches use geometric features to extract walking cycles more precisely and employ a new similarity metric to conduct user-adversary identification. This new technology for simultaneously identify user and adversary contributes to cybersecurity beyond user-only identification. In particular, the new technology is being applied to cell phone recorded walking data and performs an accuracy of $98.79\%$ for 6 classes classification (user-adversary identification) and $99.06\%$ for binary classification (user only identification). In addition to walking signal, our approach works on walking up, walking down and mixed walking signals. This technology is feasible for both large and small data set, overcoming the current challenges facing to Neural Networks such as tuning large number of hyper-parameters for large data sets and lacking of training data for small data sets. In addition, the new distance function developed here can be applied in any signal analysis.
Regret Bounds for Batched Bandits
Authors: Hossein Esfandiari, Amin Karbasi, Abbas Mehrabian, Vahab Mirrokni
Date: 2019-10-11 00:00:00
Tasks: Multi-Armed Bandits

We present simple and efficient algorithms for the batched stochastic multi-armed bandit and batched stochastic linear bandit problems. We prove bounds for their expected regrets that improve over the best-known regret bounds for any number of batches. In particular, our algorithms in both settings achieve the optimal expected regrets by using only a logarithmic number of batches. We also study the batched adversarial multi-armed bandit problem for the first time and find the optimal regret, up to logarithmic factors, of any algorithm with predetermined batch sizes.
A generalized intelligent quality-based approach for fusing multi-source information
Authors: Fuyuan Xiao
Date: 2019-10-11 00:00:00

In this paper, we propose a generalized intelligent quality-based approach for fusing multi-source information. The goal of the proposed approach intends to fuse the multi-complex-valued distribution information while maintaining a high quality of the fused result by considering the usage of credible information sources.
Software Sustainability: A Systematic Literature Review and Comprehensive Analysis
Authors: Asif Imran, Tevfik Kosar
Date: 2019-10-11 00:00:00
Tasks: Systematic Literature Review

Software Engineering is a constantly evolving subject area that faces new challenges every day as it tries to automate newer business processes. One of the key challenges to the success of a software solution is attaining sustainability. The inability of numerous software to sustain for the desired time-length is caused by limited consideration given towards sustainability during the stages of software development. This review aims to present a detailed and inclusive study covering both the technical and non-technical challenges and approaches of software sustainability. A systematic and comprehensive literature review was conducted based on 107 relevant studies that were selected using the Evidence-Based Software Engineering (EBSE) technique. The study showed that sustainability can be achieved by conducting specific activities at the technical and non-technical levels. The technical level consists of software design, coding, and user experience attributes. The non-technical level consists of documentation, sustainability manifestos, training of software engineers, funding software projects, and leadership skills of project managers to achieve sustainability. This paper groups the existing research efforts based on the above aspects. Next, how those aspects affect open and closed source software is tabulated. Based on the findings of this review, it is seen that both technical and non-technical sustainability aspects are equally important, taking one into contention and ignoring the other will threaten the sustenance of software products.
Dual Neural Network Architecture for Determining Epistemic and Aleatoric Uncertainties
Authors: Augustin Prado, Ravinath Kausik, Lalitha Venkataramanan
Date: 2019-10-10 00:00:00
Tasks: BIG-bench Machine Learning

Deep learning techniques have been shown to be extremely effective for various classification and regression problems, but quantifying the uncertainty of their predictions and separating them into the epistemic and aleatoric fractions is still considered challenging. In oil and gas exploration projects, tools consisting of seismic, sonic, magnetic resonance, resistivity, dielectric and/or nuclear sensors are sent downhole through boreholes to probe the earth's rock and fluid properties. The measurements from these tools are used to build reservoir models that are subsequently used for estimation and optimization of hydrocarbon production. Machine learning algorithms are often used to estimate the rock and fluid properties from the measured downhole data. Quantifying uncertainties of these properties is crucial for rock and fluid evaluation and subsequent reservoir optimization and production decisions. These machine learning algorithms are often trained on a "ground-truth" or core database. During the inference phase which involves application of these algorithms to field data, it is critical that the machine learning algorithm flag data as out of distribution from new geologies that the model was not trained upon. It is also highly important to be sensitive to heteroscedastic aleatoric noise in the feature space arising from the combination of tool and geological conditions. Understanding the source of the uncertainty and reducing them is key to designing intelligent tools and applications such as automated log interpretation answer products for exploration and field development. In this paper we describe a methodology consisting of a system of dual networks comprising of the combination of a Bayesian Neural Network (BNN) and an Artificial Neural Network (ANN) addressing this challenge for geophysical applications.
MUTLA: A Large-Scale Dataset for Multimodal Teaching and Learning Analytics
Authors: Fangli Xu, Lingfei Wu, KP Thai, Carol Hsu, Wei Wang, Richard Tong
Date: 2019-10-05 00:00:00
Tasks: EEG, Electroencephalogram (EEG)

Automatic analysis of teacher and student interactions could be very important to improve the quality of teaching and student engagement. However, despite some recent progress in utilizing multimodal data for teaching and learning analytics, a thorough analysis of a rich multimodal dataset coming for a complex real learning environment has yet to be done. To bridge this gap, we present a large-scale MUlti-modal Teaching and Learning Analytics (MUTLA) dataset. This dataset includes time-synchronized multimodal data records of students (learning logs, videos, EEG brainwaves) as they work in various subjects from Squirrel AI Learning System (SAIL) to solve problems of varying difficulty levels. The dataset resources include user records from the learner records store of SAIL, brainwave data collected by EEG headset devices, and video data captured by web cameras while students worked in the SAIL products. Our hope is that by analyzing real-world student learning activities, facial expressions, and brainwave patterns, researchers can better predict engagement, which can then be used to improve adaptive learning selection and student learning outcomes. An additional goal is to provide a dataset gathered from real-world educational activities versus those from controlled lab environments to benefit the educational learning community.
Developmentally motivated emergence of compositional communication via template transfer
Authors: Tomasz Korbak, Julian Zubek, Łukasz Kuciński, Piotr Miłoś, Joanna Rączaszek-Leonardi
Date: 2019-10-04 00:00:00
Tasks: Zero-shot Generalization

This paper explores a novel approach to achieving emergent compositional communication in multi-agent systems. We propose a training regime implementing template transfer, the idea of carrying over learned biases across contexts. In our method, a sender-receiver pair is first trained with disentangled loss functions and then the receiver is transferred to train a new sender with a standard loss. Unlike other methods (e.g. the obverter algorithm), our approach does not require imposing inductive biases on the architecture of the agents. We experimentally show the emergence of compositional communication using topographical similarity, zero-shot generalization and context independence as evaluation metrics. The presented approach is connected to an important line of work in semiotics and developmental psycholinguistics: it supports a conjecture that compositional communication is scaffolded on simpler communication protocols.
On the Use of Diversity Mechanisms in Dynamic Constrained Continuous Optimization
Authors: Maryam Hasani-Shoreh, Frank Neumann
Date: 2019-10-02 00:00:00
Tasks: Diversity, Evolutionary Algorithms

Population diversity plays a key role in evolutionary algorithms that enables global exploration and avoids premature convergence. This is especially more crucial in dynamic optimization in which diversity can ensure that the population keeps track of the global optimum by adapting to the changing environment. Dynamic constrained optimization problems (DCOPs) have been the target for many researchers in recent years as they comprehend many of the current real-world problems. Regardless of the importance of diversity in dynamic optimization, there is not an extensive study investigating the effects of diversity promotion techniques in DCOPs so far. To address this gap, this paper aims to investigate how the use of different diversity mechanisms may influence the behavior of algorithms in DCOPs. To achieve this goal, we apply and adapt the most common diversity promotion mechanisms for dynamic environments using differential evolution (DE) as our base algorithm. The results show that applying diversity techniques to solve DCOPs in most test cases lead to significant enhancement in the baseline algorithm in terms of modified offline error values.
Review of Learning-based Longitudinal Motion Planning for Autonomous Vehicles: Research Gaps between Self-driving and Traffic Congestion
Authors: Hao Zhou, Jorge Laval, Anye Zhou, Yu Wang, Wenchao Wu, Zhu Qing, Srinivas Peeta
Date: 2019-10-02 00:00:00
Tasks: Autonomous Vehicles, BIG-bench Machine Learning, Imitation Learning, Motion Planning, Reinforcement Learning (RL)

Self-driving technology companies and the research community are accelerating their pace to use machine learning longitudinal motion planning (mMP) for autonomous vehicles (AVs). This paper reviews the current state of the art in mMP, with an exclusive focus on its impact on traffic congestion. We identify the availability of congestion scenarios in current datasets, and summarize the required features for training mMP. For learning methods, we survey the major methods in both imitation learning and non-imitation learning. We also highlight the emerging technologies adopted by some leading AV companies, e.g. Tesla, Waymo, and Comma.ai. We find that: i) the AV industry has been mostly focusing on the long tail problem related to safety and overlooked the impact on traffic congestion, ii) the current public self-driving datasets have not included enough congestion scenarios, and mostly lack the necessary input features/output labels to train mMP, and iii) albeit reinforcement learning (RL) approach can integrate congestion mitigation into the learning goal, the major mMP method adopted by industry is still behavior cloning (BC), whose capability to learn a congestion-mitigating mMP remains to be seen. Based on the review, the study identifies the research gaps in current mMP development. Some suggestions towards congestion mitigation for future mMP studies are proposed: i) enrich data collection to facilitate the congestion learning, ii) incorporate non-imitation learning methods to combine traffic efficiency into a safety-oriented technical route, and iii) integrate domain knowledge from the traditional car following (CF) theory to improve the string stability of mMP.
Monkey Optimization System with Active Membranes: A New Meta-heuristic Optimization System
Authors: Moustafa Zein, Aboul Ella Hassanien, Ammar Adl, Adam Slowik
Date: 2019-09-30 00:00:00

Optimization techniques, used to get the optimal solution in search spaces, have not solved the time-consuming problem. The objective of this study is to tackle the sequential processing problem in Monkey Algorithm and simulating the natural parallel behavior of monkeys. Therefore, a P system with active membranes is constructed by providing a codification for Monkey Algorithm within the context of a cell-like P system, defining accordingly the elements of the model - membrane structure, objects, rules and the behavior of it. The proposed algorithm has modeled the natural behavior of climb process using separate membranes, rather than the original algorithm. Moreover, it introduced the membrane migration process to select the best solution and the time stamp was added as an additional stopping criterion to control the timing of the algorithm. The results indicate a substantial solution for the time consumption problem, significant representation of the natural behavior of monkeys, and considerable chance to reach the best solution in the context of meta-heuristics purpose. In addition, experiments use the commonly used benchmark functions to test the performance of the algorithm as well as the expected time of the proposed P Monkey optimization algorithm and the traditional Monkey Algorithm running on population size. The unit times are calculated based on the complexity of algorithms, where P Monkey takes a time unit to fire rule(s) over a population size n; as soon as, Monkey Algorithm takes a time unit to run a step every mathematical equation over a population size.
What does it mean to solve the problem of discrimination in hiring? Social, technical and legal perspectives from the UK on automated hiring systems
Authors: Javier Sanchez-Monedero, Lina Dencik, Lilian Edwards
Date: 2019-09-28 00:00:00

The ability to get and keep a job is a key aspect of participating in society and sustaining livelihoods. Yet the way decisions are made on who is eligible for jobs, and why, are rapidly changing with the advent and growth in uptake of automated hiring systems (AHSs) powered by data-driven tools. Key concerns about such AHSs include the lack of transparency and potential limitation of access to jobs for specific profiles. In relation to the latter, however, several of these AHSs claim to detect and mitigate discriminatory practices against protected groups and promote diversity and inclusion at work. Yet whilst these tools have a growing user-base around the world, such claims of bias mitigation are rarely scrutinised and evaluated, and when done so, have almost exclusively been from a US socio-legal perspective. In this paper, we introduce a perspective outside the US by critically examining how three prominent automated hiring systems (AHSs) in regular use in the UK, HireVue, Pymetrics and Applied, understand and attempt to mitigate bias and discrimination. Using publicly available documents, we describe how their tools are designed, validated and audited for bias, highlighting assumptions and limitations, before situating these in the socio-legal context of the UK. The UK has a very different legal background to the US in terms not only of hiring and equality law, but also in terms of data protection (DP) law. We argue that this might be important for addressing concerns about transparency and could mean a challenge to building bias mitigation into AHSs definitively capable of meeting EU legal standards. This is significant as these AHSs, especially those developed in the US, may obscure rather than improve systemic discrimination in the workplace.
To Combine or Not To Combine? A Rainbow Deep Reinforcement Learning Agent for Dialog Policies
Authors: Dirk V{\"a}th, Ngoc Thang Vu
Date: 2019-09-01 00:00:00
Proceeding: WS 2019 9
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In this paper, we explore state-of-the-art deep reinforcement learning methods for dialog policy training such as prioritized experience replay, double deep Q-Networks, dueling network architectures and distributional learning. Our main findings show that each individual method improves the rewards and the task success rate but combining these methods in a Rainbow agent, which performs best across tasks and environments, is a non-trivial task. We, therefore, provide insights about the influence of each method on the combination and how to combine them to form a Rainbow agent.
Contextualized Representations for Low-resource Utterance Tagging
Authors: Bhargavi Paranjape, Graham Neubig
Date: 2019-09-01 00:00:00
Proceeding: WS 2019 9
Tasks: Emotion Classification

Utterance-level analysis of the speaker{'}s intentions and emotions is a core task in conversational understanding. Depending on the end objective of the conversational understanding task, different categorical dialog-act or affect labels are expertly designed to cover specific aspects of the speakers{'} intentions or emotions respectively. Accurately annotating with these labels requires a high level of human expertise, and thus applying this process to a large conversation corpus or new domains is prohibitively expensive. The resulting paucity of data limits the use of sophisticated neural models. In this paper, we tackle these limitations by performing unsupervised training of utterance representations from a large corpus of spontaneous dialogue data. Models initialized with these representations achieve competitive performance on utterance-level dialogue-act recognition and emotion classification, especially in low-resource settings encountered when analyzing conversations in new domains.
Capturing Dialogue State Variable Dependencies with an Energy-based Neural Dialogue State Tracker
Authors: Anh Duong Trinh, Robert J. Ross, John D. Kelleher
Date: 2019-09-01 00:00:00
Proceeding: WS 2019 9
Tasks: Dialogue State Tracking, Prediction, Structured Prediction

Dialogue state tracking requires the population and maintenance of a multi-slot frame representation of the dialogue state. Frequently, dialogue state tracking systems assume independence between slot values within a frame. In this paper we argue that treating the prediction of each slot value as an independent prediction task may ignore important associations between the slot values, and, consequently, we argue that treating dialogue state tracking as a structured prediction problem can help to improve dialogue state tracking performance. To support this argument, the research presented in this paper is structured into three stages: (i) analyzing variable dependencies in dialogue data; (ii) applying an energy-based methodology to model dialogue state tracking as a structured prediction task; and (iii) evaluating the impact of inter-slot relationships on model performance. Overall we demonstrate that modelling the associations between target slots with an energy-based formalism improves dialogue state tracking performance in a number of ways.