pwc-archive/papers-with-abstracts huggingface dataset:
Machine Learning based Simulation Optimisation for Trailer Management
Authors: Dylan Rijnen, Jason Rhuggenaath, Paulo R. de O. da Costa, Yingqian Zhang
Date: 2019-07-17 00:00:00
Tasks: BIG-bench Machine Learning, Management

In many situations, simulation models are developed to handle complex real-world business optimisation problems. For example, a discrete-event simulation model is used to simulate the trailer management process in a big Fast-Moving Consumer Goods company. To address the problem of finding suitable inputs to this simulator for optimising fleet configuration, we propose a simulation optimisation approach in this paper. The simulation optimisation model combines a metaheuristic search (genetic algorithm), with an approximation model filter (feed-forward neural network) to optimise the parameter configuration of the simulation model. We introduce an ensure probability that overrules the rejection of potential solutions by the approximation model and we demonstrate its effectiveness. In addition, we evaluate the impact of the parameters of the optimisation model on its effectiveness and show the parameters such as population size, filter threshold, and mutation probability can have a significant impact on the overall optimisation performance. Moreover, we compare the proposed method with a single global approximation model approach and a random-based approach. The results show the effectiveness of our method in terms of computation time and solution quality.
Self-Attentive Hawkes Processes
Authors: Qiang Zhang, Aldo Lipani, Omer Kirnap, Emine Yilmaz
Date: 2019-07-17 00:00:00

Asynchronous events on the continuous time domain, e.g., social media actions and stock transactions, occur frequently in the world. The ability to recognize occurrence patterns of event sequences is crucial to predict which typeof events will happen next and when. A de facto standard mathematical framework to do this is the Hawkes process. In order to enhance expressivity of multivariate Hawkes processes, conventional statistical methods and deep recurrent networks have been employed to modify its intensity function. The former is highly interpretable and requires small size of training data but relies on correct model design while the latter has less dependency on prior knowledge and is more powerful in capturing complicated patterns. We leverage pros and cons of these models and propose a self-attentive Hawkes process(SAHP). The proposed method adapts self-attention to fit the intensity function of Hawkes processes. This design has two benefits:(1) compared with conventional statistical methods, the SAHP is more powerful to identify complicated dependency relationships between temporal events; (2)compared with deep recurrent networks, the self-attention mechanism is able to capture longer historical information, and is more interpretable because the learnt attention weight tensor shows contributions of each historical event. Experiments on four real-world datasets demonstrate the effectiveness of the proposed method.
Output-weighted optimal sampling for Bayesian regression and rare event statistics using few samples
Authors: Themistoklis P. Sapsis
Date: 2019-07-17 00:00:00
Tasks: Active Learning, Experimental Design, regression

For many important problems the quantity of interest is an unknown function of the parameters, which is a random vector with known statistics. Since the dependence of the output on this random vector is unknown, the challenge is to identify its statistics, using the minimum number of function evaluations. This problem can been seen in the context of active learning or optimal experimental design. We employ Bayesian regression to represent the derived model uncertainty due to finite and small number of input-output pairs. In this context we evaluate existing methods for optimal sample selection, such as model error minimization and mutual information maximization. We show that for the case of known output variance, the commonly employed criteria in the literature do not take into account the output values of the existing input-output pairs, while for the case of unknown output variance this dependence can be very weak. We introduce a criterion that takes into account the values of the output for the existing samples and adaptively selects inputs from regions of the parameter space which have important contribution to the output. The new method allows for application to high-dimensional inputs, paving the way for optimal experimental design in high-dimensions.
Low-Shot Classification: A Comparison of Classical and Deep Transfer Machine Learning Approaches
Authors: Peter Usherwood, Steven Smit
Date: 2019-07-17 00:00:00
Tasks: BIG-bench Machine Learning, General Classification, Sentiment Analysis, Sentiment Classification, text-classification, Text Classification, Transfer Learning, valid

Despite the recent success of deep transfer learning approaches in NLP, there is a lack of quantitative studies demonstrating the gains these models offer in low-shot text classification tasks over existing paradigms. Deep transfer learning approaches such as BERT and ULMFiT demonstrate that they can beat state-of-the-art results on larger datasets, however when one has only 100-1000 labelled examples per class, the choice of approach is less clear, with classical machine learning and deep transfer learning representing valid options. This paper compares the current best transfer learning approach with top classical machine learning approaches on a trinary sentiment classification task to assess the best paradigm. We find that BERT, representing the best of deep transfer learning, is the best performing approach, outperforming top classical machine learning algorithms by 9.7% on average when trained with 100 examples per class, narrowing to 1.8% at 1000 labels per class. We also show the robustness of deep transfer learning in moving across domains, where the maximum loss in accuracy is only 0.7% in similar domain tasks and 3.2% cross domain, compared to classical machine learning which loses up to 20.6%.
Almawave-SLU: A new dataset for SLU in Italian
Authors: Valentina Bellomaria, Giuseppe Castellucci, Andrea Favalli, Raniero Romagnoli
Date: 2019-07-17 00:00:00
Tasks: Intent Detection, Question Answering, Spoken Language Understanding

The widespread use of conversational and question answering systems made it necessary to improve the performances of speaker intent detection and understanding of related semantic slots, i.e., Spoken Language Understanding (SLU). Often, these tasks are approached with supervised learning methods, which needs considerable labeled datasets. This paper presents the first Italian dataset for SLU. It is derived through a semi-automatic procedure and is used as a benchmark of various open source and commercial systems.
A Multivariate Extreme Value Theory Approach to Anomaly Clustering and Visualization
Authors: Maël Chiapino, Stéphan Clémençon, Vincent Feuillard, Anne Sabourin
Date: 2019-07-17 00:00:00
Tasks: Clustering, Graph Mining

In a wide variety of situations, anomalies in the behaviour of a complex system, whose health is monitored through the observation of a random vector X = (X1,. .. , X d) valued in R d , correspond to the simultaneous occurrence of extreme values for certain subgroups $\alpha$ $\subset$ {1,. .. , d} of variables Xj. Under the heavy-tail assumption, which is precisely appropriate for modeling these phenomena, statistical methods relying on multivariate extreme value theory have been developed in the past few years for identifying such events/subgroups. This paper exploits this approach much further by means of a novel mixture model that permits to describe the distribution of extremal observations and where the anomaly type $\alpha$ is viewed as a latent variable. One may then take advantage of the model by assigning to any extreme point a posterior probability for each anomaly type $\alpha$, defining implicitly a similarity measure between anomalies. It is explained at length how the latter permits to cluster extreme observations and obtain an informative planar representation of anomalies using standard graph-mining tools. The relevance and usefulness of the clustering and 2-d visual display thus designed is illustrated on simulated datasets and on real observations as well, in the aeronautics application domain.
Stereo Event Lifetime and Disparity Estimation for Dynamic Vision Sensors
Authors: Antea Hadviger, Ivan Marković, Ivan Petrović
Date: 2019-07-17 00:00:00
Tasks: Disparity Estimation, Stereo Matching, Stereo Matching Hand

Event-based cameras are biologically inspired sensors that output asynchronous pixel-wise brightness changes in the scene called events. They have a high dynamic range and temporal resolution of a microsecond, opposed to standard cameras that output frames at fixed frame rates and suffer from motion blur. Forming stereo pairs of such cameras can open novel application possibilities, since for each event depth can be readily estimated; however, to fully exploit asynchronous nature of the sensor and avoid fixed time interval event accumulation, stereo event lifetime estimation should be employed. In this paper, we propose a novel method for event lifetime estimation of stereo event-cameras, allowing generation of sharp gradient images of events that serve as input to disparity estimation methods. Since a single brightness change triggers events in both event-camera sensors, we propose a method for single shot event lifetime and disparity estimation, with association via stereo matching. The proposed method is approximately twice as fast and more accurate than if lifetimes were estimated separately for each sensor and then stereo matched. Results are validated on real-world data through multiple stereo event-camera experiments.
Self Organizing Supply Chains for Micro-Prediction: Present and Future uses of the ROAR Protocol
Authors: Peter Cotton
Date: 2019-07-17 00:00:00

A multi-agent system is trialed as a means of crowd-sourcing inexpensive but high quality streams of predictions. Each agent is a microservice embodying statistical models and endowed with economic self-interest. The ability to fork and modify simple agents is granted to a large number of employees in a firm and empirical lessons are reported. We suggest that one plausible trajectory for this project is the creation of a Prediction Web.
A Link Between the Multiplicative and Additive Functional Asplund's Metrics
Authors: Guillaume Noyel
Date: 2019-07-17 00:00:00

Functional Asplund's metrics were recently introduced to perform pattern matching robust to lighting changes thanks to double-sided probing in the Logarithmic Image Processing (LIP) framework. Two metrics were defined, namely the LIP-multiplicative Asplund's metric which is robust to variations of object thickness (or opacity) and the LIP-additive Asplund's metric which is robust to variations of camera exposure-time (or light intensity). Maps of distances-i.e. maps of these metric values-were also computed between a reference template and an image. Recently, it was proven that the map of LIP-multiplicative As-plund's distances corresponds to mathematical morphology operations. In this paper, the link between both metrics and between their corresponding distance maps will be demonstrated. It will be shown that the map of LIP-additive Asplund's distances of an image can be computed from the map of the LIP-multiplicative Asplund's distance of a transform of this image and vice-versa. Both maps will be related by the LIP isomorphism which will allow to pass from the image space of the LIP-additive distance map to the positive real function space of the LIP-multiplicative distance map. Experiments will illustrate this relation and the robustness of the LIP-additive Asplund's metric to lighting changes.
Differentiable Disentanglement Filter: an Application Agnostic Core Concept Discovery Probe
Authors: Guntis Barzdins, Eduards Sidorovics
Date: 2019-07-17 00:00:00
Tasks: Disentanglement, Visual Grounding, Word Embeddings

It has long been speculated that deep neural networks function by discovering a hierarchical set of domain-specific core concepts or patterns, which are further combined to recognize even more elaborate concepts for the classification or other machine learning tasks. Meanwhile disentangling the actual core concepts engrained in the word embeddings (like word2vec or BERT) or deep convolutional image recognition neural networks (like PG-GAN) is difficult and some success there has been achieved only recently. In this paper we propose a novel neural network nonlinearity named Differentiable Disentanglement Filter (DDF) which can be transparently inserted into any existing neural network layer to automatically disentangle the core concepts used by that layer. The DDF probe is inspired by the obscure properties of the hyper-dimensional computing theory. The DDF proof-of-concept implementation is shown to disentangle concepts within the neural 3D scene representation - a task vital for visual grounding of natural language narratives.
Subspace Inference for Bayesian Deep Learning
Authors: Pavel Izmailov, Wesley J. Maddox, Polina Kirichenko, Timur Garipov, Dmitry Vetrov, Andrew Gordon Wilson
Date: 2019-07-17 00:00:00
Tasks: Bayesian Inference, Deep Learning, image-classification, Image Classification, regression, Variational Inference

Bayesian inference was once a gold standard for learning with neural networks, providing accurate full predictive distributions and well calibrated uncertainty. However, scaling Bayesian inference techniques to deep neural networks is challenging due to the high dimensionality of the parameter space. In this paper, we construct low-dimensional subspaces of parameter space, such as the first principal components of the stochastic gradient descent (SGD) trajectory, which contain diverse sets of high performing models. In these subspaces, we are able to apply elliptical slice sampling and variational inference, which struggle in the full parameter space. We show that Bayesian model averaging over the induced posterior in these subspaces produces accurate predictions and well calibrated predictive uncertainty for both regression and image classification.
Photonic architecture for reinforcement learning
Authors: Fulvio Flamini, Arne Hamann, Sofiène Jerbi, Lea M. Trenkwalder, Hendrik Poulsen Nautrup, Hans J. Briegel
Date: 2019-07-17 00:00:00
Tasks: Active Learning, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

The last decade has seen an unprecedented growth in artificial intelligence and photonic technologies, both of which drive the limits of modern-day computing devices. In line with these recent developments, this work brings together the state of the art of both fields within the framework of reinforcement learning. We present the blueprint for a photonic implementation of an active learning machine incorporating contemporary algorithms such as SARSA, Q-learning, and projective simulation. We numerically investigate its performance within typical reinforcement learning environments, showing that realistic levels of experimental noise can be tolerated or even be beneficial for the learning process. Remarkably, the architecture itself enables mechanisms of abstraction and generalization, two features which are often considered key ingredients for artificial intelligence. The proposed architecture, based on single-photon evolution on a mesh of tunable beamsplitters, is simple, scalable, and a first integration in portable systems appears to be within the reach of near-term technology.
Algorithmic Analysis and Statistical Estimation of SLOPE via Approximate Message Passing
Authors: Zhiqi Bu, Jason Klusowski, Cynthia Rush, Weijie Su
Date: 2019-07-17 00:00:00
Proceeding: NeurIPS 2019 12

SLOPE is a relatively new convex optimization procedure for high-dimensional linear regression via the sorted l1 penalty: the larger the rank of the fitted coefficient, the larger the penalty. This non-separable penalty renders many existing techniques invalid or inconclusive in analyzing the SLOPE solution. In this paper, we develop an asymptotically exact characterization of the SLOPE solution under Gaussian random designs through solving the SLOPE problem using approximate message passing (AMP). This algorithmic approach allows us to approximate the SLOPE solution via the much more amenable AMP iterates. Explicitly, we characterize the asymptotic dynamics of the AMP iterates relying on a recently developed state evolution analysis for non-separable penalties, thereby overcoming the difficulty caused by the sorted l1 penalty. Moreover, we prove that the AMP iterates converge to the SLOPE solution in an asymptotic sense, and numerical simulations show that the convergence is surprisingly fast. Our proof rests on a novel technique that specifically leverages the SLOPE problem. In contrast to prior literature, our work not only yields an asymptotically sharp analysis but also offers an algorithmic, flexible, and constructive approach to understanding the SLOPE problem.
Learning Variable Impedance Control for Contact Sensitive Tasks
Authors: Miroslav Bogdanovic, Majid Khadiv, Ludovic Righetti
Date: 2019-07-17 00:00:00
Tasks: Position, Reinforcement Learning

Reinforcement learning algorithms have shown great success in solving different problems ranging from playing video games to robotics. However, they struggle to solve delicate robotic problems, especially those involving contact interactions. Though in principle a policy directly outputting joint torques should be able to learn to perform these tasks, in practice we see that it has difficulty to robustly solve the problem without any given structure in the action space. In this paper, we investigate how the choice of action space can give robust performance in presence of contact uncertainties. We propose learning a policy giving as output impedance and desired position in joint space and compare the performance of that approach to torque and position control under different contact uncertainties. Furthermore, we propose an additional reward term designed to regularize these variable impedance control policies, giving them interpretability and facilitating their transfer to real systems. We present extensive experiments in simulation of both floating and fixed-base systems in tasks involving contact uncertainties, as well as results for running the learned policies on a real system.
Improving Heart Rate Variability Measurements from Consumer Smartwatches with Machine Learning
Authors: Martin Maritsch, Caterina Bérubé, Mathias Kraus, Vera Lehmann, Thomas Züger, Stefan Feuerriegel, Tobias Kowatsch, Felix Wortmann
Date: 2019-07-17 00:00:00
Tasks: BIG-bench Machine Learning, Heart Rate Variability

The reactions of the human body to physical exercise, psychophysiological stress and heart diseases are reflected in heart rate variability (HRV). Thus, continuous monitoring of HRV can contribute to determining and predicting issues in well-being and mental health. HRV can be measured in everyday life by consumer wearable devices such as smartwatches which are easily accessible and affordable. However, they are arguably accurate due to the stability of the sensor. We hypothesize a systematic error which is related to the wearer movement. Our evidence builds upon explanatory and predictive modeling: we find a statistically significant correlation between error in HRV measurements and the wearer movement. We show that this error can be minimized by bringing into context additional available sensor information, such as accelerometer data. This work demonstrates our research-in-progress on how neural learning can minimize the error of such smartwatch HRV measurements.
Canada Protocol: an ethical checklist for the use of Artificial Intelligence in Suicide Prevention and Mental Health
Authors: Carl-Maria Mörch, Abhishek Gupta, Brian L. Mishara
Date: 2019-07-17 00:00:00
Tasks: Ethics

Introduction: To improve current public health strategies in suicide prevention and mental health, governments, researchers and private companies increasingly use information and communication technologies, and more specifically Artificial Intelligence and Big Data. These technologies are promising but raise ethical challenges rarely covered by current legal systems. It is essential to better identify, and prevent potential ethical risks. Objectives: The Canada Protocol - MHSP is a tool to guide and support professionals, users, and researchers using AI in mental health and suicide prevention. Methods: A checklist was constructed based upon ten international reports on AI and ethics and two guides on mental health and new technologies. 329 recommendations were identified, of which 43 were considered as applicable to Mental Health and AI. The checklist was validated, using a two round Delphi Consultation. Results: 16 experts participated in the first round of the Delphi Consultation and 8 participated in the second round. Of the original 43 items, 38 were retained. They concern five categories: "Description of the Autonomous Intelligent System" (n=8), "Privacy and Transparency" (n=8), "Security" (n=6), "Health-Related Risks" (n=8), "Biases" (n=8). The checklist was considered relevant by most users, and could need versions tailored to each category of target users.
Constrained Concealment Attacks against Reconstruction-based Anomaly Detectors in Industrial Control Systems
Authors: Alessandro Erba, Riccardo Taormina, Stefano Galelli, Marcello Pogliani, Michele Carminati, Stefano Zanero, Nils Ole Tippenhauer
Date: 2019-07-17 00:00:00
Tasks: Anomaly Detection

Recently, reconstruction-based anomaly detection was proposed as an effective technique to detect attacks in dynamic industrial control networks. Unlike classical network anomaly detectors that observe the network traffic, reconstruction-based detectors operate on the measured sensor data, leveraging physical process models learned a priori. In this work, we investigate different approaches to evade prior-work reconstruction-based anomaly detectors by manipulating sensor data so that the attack is concealed. We find that replay attacks (commonly assumed to be very strong) show bad performance (i.e., increasing the number of alarms) if the attacker is constrained to manipulate less than 95% of all features in the system, as hidden correlations between the features are not replicated well. To address this, we propose two novel attacks that manipulate a subset of the sensor readings, leveraging learned physical constraints of the system. Our attacks feature two different attacker models: A white box attacker, which uses an optimization approach with a detection oracle, and a black box attacker, which uses an autoencoder to translate anomalous data into normal data. We evaluate our implementation on two different datasets from the water distribution domain, showing that the detector's Recall drops from 0.68 to 0.12 by manipulating 4 sensors out of 82 in WADI dataset. In addition, we show that our black box attacks are transferable to different detectors: They work against autoencoder-, LSTM-, and CNN-based detectors. Finally, we implement and demonstrate our attacks on a real industrial testbed to demonstrate their feasibility in real-time.