pwc-archive/papers-with-abstracts huggingface dataset:
Order Embeddings from Merged Ontologies using Sketching
Authors: Kenneth L. Clarkson, Sanjana Sahayaraj
Date: 2021-01-06 00:00:00
Tasks: Dimensionality Reduction

We give a simple, low resource method to produce order embeddings from ontologies. Such embeddings map words to vectors so that order relations on the words, such as hypernymy/hyponymy, are represented in a direct way. Our method uses sketching techniques, in particular countsketch, for dimensionality reduction. We also study methods to merge ontologies, in particular those in medical domains, so that order relations are preserved. We give computational results for medical ontologies and for wordnet, showing that our merging techniques are effective and our embedding yields an accurate representation in both generic and specialised domains.
EfficientQA : a RoBERTa Based Phrase-Indexed Question-Answering System
Authors: Sofian Chaybouti, Achraf Saghe, Aymen Shabou
Date: 2021-01-06 00:00:00
Tasks: Extractive Question-Answering, GPU, Natural Language Understanding, Open-Domain Question Answering, Question Answering

State-of-the-art extractive question answering models achieve superhuman performances on the SQuAD benchmark. Yet, they are unreasonably heavy and need expensive GPU computing to answer questions in a reasonable time. Thus, they cannot be used for real-world queries on hundreds of thousands of documents in the open-domain question answering paradigm. In this paper, we explore the possibility to transfer the natural language understanding of language models into dense vectors representing questions and answer candidates, in order to make the task of question-answering compatible with a simple nearest neighbor search task. This new model, that we call EfficientQA, takes advantage from the pair of sequences kind of input of BERT-based models to build meaningful dense representations of candidate answers. These latter are extracted from the context in a question-agnostic fashion. Our model achieves state-of-the-art results in Phrase-Indexed Question Answering (PIQA) beating the previous state-of-art by 1.3 points in exact-match and 1.4 points in f1-score. These results show that dense vectors are able to embed very rich semantic representations of sequences, although these ones were built from language models not originally trained for the use-case. Thus, in order to build more resource efficient NLP systems in the future, training language models that are better adapted to build dense representations of phrases is one of the possibilities.
The Shapley Value of Classifiers in Ensemble Games
Authors: Benedek Rozemberczki, Rik Sarkar
Date: 2021-01-06 00:00:00
Tasks: Classification, Ensemble Pruning, Graph Classification

What is the value of an individual model in an ensemble of binary classifiers? We answer this question by introducing a class of transferable utility cooperative games called \textit{ensemble games}. In machine learning ensembles, pre-trained models cooperate to make classification decisions. To quantify the importance of models in these ensemble games, we define \textit{Troupe} -- an efficient algorithm which allocates payoffs based on approximate Shapley values of the classifiers. We argue that the Shapley value of models in these games is an effective decision metric for choosing a high performing subset of models from the ensemble. Our analytical findings prove that our Shapley value estimation scheme is precise and scalable; its performance increases with size of the dataset and ensemble. Empirical results on real world graph classification tasks demonstrate that our algorithm produces high quality estimates of the Shapley value. We find that Shapley values can be utilized for ensemble pruning, and that adversarial models receive a low valuation. Complex classifiers are frequently found to be responsible for both correct and incorrect classification decisions.
Cauchy-Schwarz Regularized Autoencoder
Authors: Linh Tran, Maja Pantic, Marc Peter Deisenroth
Date: 2021-01-06 00:00:00
Tasks: Clustering, Density Estimation

Recent work in unsupervised learning has focused on efficient inference and learning in latent variables models. Training these models by maximizing the evidence (marginal likelihood) is typically intractable. Thus, a common approximation is to maximize the Evidence Lower BOund (ELBO) instead. Variational autoencoders (VAE) are a powerful and widely-used class of generative models that optimize the ELBO efficiently for large datasets. However, the VAE's default Gaussian choice for the prior imposes a strong constraint on its ability to represent the true posterior, thereby degrading overall performance. A Gaussian mixture model (GMM) would be a richer prior, but cannot be handled efficiently within the VAE framework because of the intractability of the Kullback-Leibler divergence for GMMs. We deviate from the common VAE framework in favor of one with an analytical solution for Gaussian mixture prior. To perform efficient inference for GMM priors, we introduce a new constrained objective based on the Cauchy-Schwarz divergence, which can be computed analytically for GMMs. This new objective allows us to incorporate richer, multi-modal priors into the autoencoding framework. We provide empirical studies on a range of datasets and show that our objective improves upon variational auto-encoding models in density estimation, unsupervised clustering, semi-supervised learning, and face analysis.
Combining Deep Learning and Mathematical Morphology for Historical Map Segmentation
Authors: Yizi Chen, Edwin Carlinet, Joseph Chazalon, Clément Mallet, Bertrand Duménieu, Julien Perret
Date: 2021-01-06 00:00:00
Tasks: Deep Learning, Edge Detection

The digitization of historical maps enables the study of ancient, fragile, unique, and hardly accessible information sources. Main map features can be retrieved and tracked through the time for subsequent thematic analysis. The goal of this work is the vectorization step, i.e., the extraction of vector shapes of the objects of interest from raster images of maps. We are particularly interested in closed shape detection such as buildings, building blocks, gardens, rivers, etc. in order to monitor their temporal evolution. Historical map images present significant pattern recognition challenges. The extraction of closed shapes by using traditional Mathematical Morphology (MM) is highly challenging due to the overlapping of multiple map features and texts. Moreover, state-of-the-art Convolutional Neural Networks (CNN) are perfectly designed for content image filtering but provide no guarantee about closed shape detection. Also, the lack of textural and color information of historical maps makes it hard for CNN to detect shapes that are represented by only their boundaries. Our contribution is a pipeline that combines the strengths of CNN (efficient edge detection and filtering) and MM (guaranteed extraction of closed shapes) in order to achieve such a task. The evaluation of our approach on a public dataset shows its effectiveness for extracting the closed boundaries of objects in historical maps.
Connecting ansatz expressibility to gradient magnitudes and barren plateaus
Authors: Zoë Holmes, Kunal Sharma, M. Cerezo, Patrick J. Coles
Date: 2021-01-06 00:00:00

Parameterized quantum circuits serve as ans\"{a}tze for solving variational problems and provide a flexible paradigm for programming near-term quantum computers. Ideally, such ans\"{a}tze should be highly expressive so that a close approximation of the desired solution can be accessed. On the other hand, the ansatz must also have sufficiently large gradients to allow for training. Here, we derive a fundamental relationship between these two essential properties: expressibility and trainability. This is done by extending the well established barren plateau phenomenon, which holds for ans\"{a}tze that form exact 2-designs, to arbitrary ans\"{a}tze. Specifically, we calculate the variance in the cost gradient in terms of the expressibility of the ansatz, as measured by its distance from being a 2-design. Our resulting bounds indicate that highly expressive ans\"{a}tze exhibit flatter cost landscapes and therefore will be harder to train. Furthermore, we provide numerics illustrating the effect of expressiblity on gradient scalings, and we discuss the implications for designing strategies to avoid barren plateaus.
Smoothed functional-based gradient algorithms for off-policy reinforcement learning: A non-asymptotic viewpoint
Authors: Nithia Vijayan, Prashanth L. A
Date: 2021-01-06 00:00:00
Tasks: Off-policy evaluation, Reinforcement Learning (RL)

We propose two policy gradient algorithms for solving the problem of control in an off-policy reinforcement learning (RL) context. Both algorithms incorporate a smoothed functional (SF) based gradient estimation scheme. The first algorithm is a straightforward combination of importance sampling-based off-policy evaluation with SF-based gradient estimation. The second algorithm, inspired by the stochastic variance-reduced gradient (SVRG) algorithm, incorporates variance reduction in the update iteration. For both algorithms, we derive non-asymptotic bounds that establish convergence to an approximate stationary point. From these results, we infer that the first algorithm converges at a rate that is comparable to the well-known REINFORCE algorithm in an off-policy RL context, while the second algorithm exhibits an improved rate of convergence.
LAEO-Net++: revisiting people Looking At Each Other in videos
Authors: Manuel J. Marin-Jimenez, Vicky Kalogeiton, Pablo Medina-Suarez, Andrew Zisserman
Date: 2021-01-06 00:00:00
Tasks: Mutual Gaze

Capturing the 'mutual gaze' of people is essential for understanding and interpreting the social interactions between them. To this end, this paper addresses the problem of detecting people Looking At Each Other (LAEO) in video sequences. For this purpose, we propose LAEO-Net++, a new deep CNN for determining LAEO in videos. In contrast to previous works, LAEO-Net++ takes spatio-temporal tracks as input and reasons about the whole track. It consists of three branches, one for each character's tracked head and one for their relative position. Moreover, we introduce two new LAEO datasets: UCO-LAEO and AVA-LAEO. A thorough experimental evaluation demonstrates the ability of LAEO-Net++ to successfully determine if two people are LAEO and the temporal window where it happens. Our model achieves state-of-the-art results on the existing TVHID-LAEO video dataset, significantly outperforming previous approaches. Finally, we apply LAEO-Net++ to a social network, where we automatically infer the social relationship between pairs of people based on the frequency and duration that they LAEO, and show that LAEO can be a useful tool for guided search of human interactions in videos. The code is available at https://github.com/AVAuco/laeonetplus.
Hierarchical Multi-timescale Framework For Operation of Dynamic Community Microgrid
Authors: Ashwin Shirsat, Valliappan Muthukaruppan, Rongxing Hu, Ning Lu, Mesut Baran, David Lubkeman, Wenyuan Tang
Date: 2020-11-19 00:00:00
Tasks: Scheduling

Distribution system integrated community microgrids (CMGs) can restore loads during extended outages. The CMG is challenged with limited resource availability, absence of a robust grid-support, and demand-supply uncertainty. To address these challenges, this paper proposes a three-stage hierarchical multi-timescale framework for scheduling and real-time (RT) dispatch of CMGs. The CMG's ability to dynamically expand its boundary to support the neighboring grid sections is also considered. The first stage solves a stochastic day-ahead (DA) scheduling problem to obtain referral plans for optimal resource rationing. The intermediate near real-time scheduling stage updates the DA schedule closer to the dispatch time, followed by the RT dispatch stage. The proposed methodology is validated via numerical simulations on a modified IEEE 123-bus system, which shows superior performance in terms of RT load supplied under different forecast error cases, outage duration scenarios, and against the traditionally used two-stage approach.
RethNet: Object-by-Object Learning for Detecting Facial Skin Problems
Authors: Shohrukh Bekmirzaev, Seoyoung Oh, Sangwook Yoo
Date: 2021-01-06 00:00:00
Tasks: Object, object-detection, Object Detection, Segmentation, Semantic Segmentation

Semantic segmentation is a hot topic in computer vision where the most challenging tasks of object detection and recognition have been handling by the success of semantic segmentation approaches. We propose a concept of object-by-object learning technique to detect 11 types of facial skin lesions using semantic segmentation methods. Detecting individual skin lesion in a dense group is a challenging task, because of ambiguities in the appearance of the visual data. We observe that there exist co-occurrent visual relations between object classes (e.g., wrinkle and age spot, or papule and whitehead, etc.). In fact, rich contextual information significantly helps to handle the issue. Therefore, we propose REthinker blocks that are composed of the locally constructed convLSTM/Conv3D layers and SE module as a one-shot attention mechanism whose responsibility is to increase network's sensitivity in the local and global contextual representation that supports to capture ambiguously appeared objects and co-occurrence interactions between object classes. Experiments show that our proposed model reached MIoU of 79.46% on the test of a prepared dataset, representing a 15.34% improvement over Deeplab v3+ (MIoU of 64.12%).
Attention-based Convolutional Autoencoders for 3D-Variational Data Assimilation
Authors: Julian Mack, Rossella Arcucci, Miguel Molina-Solana, Yi-Ke Guo
Date: 2021-01-06 00:00:00

We propose a new 'Bi-Reduced Space' approach to solving 3D Variational Data Assimilation using Convolutional Autoencoders. We prove that our approach has the same solution as previous methods but has significantly lower computational complexity; in other words, we reduce the computational cost without affecting the data assimilation accuracy. We tested the new method with data from a real-world application: a pollution model of a site in Elephant and Castle, London and found that we could reduce the size of the background covariance matrix representation by O(10^3) and, at the same time, increase our data assimilation accuracy with respect to existing reduced space methods.
Ludii Game Logic Guide
Authors: Éric Piette, Cameron Browne, Dennis J. N. J. Soemers
Date: 2021-01-06 00:00:00

This technical report outlines the fundamental workings of the game logic behind Ludii, a general game system, that can be used to play a wide variety of games. Ludii is a program developed for the ERC-funded Digital Ludeme Project, in which mathematical and computational approaches are used to study how games were played, and spread, throughout history. This report explains how general game states and equipment are represented in Ludii, and how the rule ludemes dictating play are implemented behind the scenes, giving some insight into the core game logic behind the Ludii general game player. This guide is intended to help game designers using the Ludii game description language to understand it more completely and make fuller use of its features when describing their games.
Do We Really Need Deep Learning Models for Time Series Forecasting?
Authors: Shereen Elsayed, Daniela Thyssens, Ahmed Rashed, Hadi Samer Jomaa, Lars Schmidt-Thieme
Date: 2021-01-06 00:00:00
Tasks: Deep Learning, regression, Time Series, Time Series Analysis, Time Series Forecasting

Time series forecasting is a crucial task in machine learning, as it has a wide range of applications including but not limited to forecasting electricity consumption, traffic, and air quality. Traditional forecasting models rely on rolling averages, vector auto-regression and auto-regressive integrated moving averages. On the other hand, deep learning and matrix factorization models have been recently proposed to tackle the same problem with more competitive performance. However, one major drawback of such models is that they tend to be overly complex in comparison to traditional techniques. In this paper, we report the results of prominent deep learning models with respect to a well-known machine learning baseline, a Gradient Boosting Regression Tree (GBRT) model. Similar to the deep neural network (DNN) models, we transform the time series forecasting task into a window-based regression problem. Furthermore, we feature-engineered the input and output structure of the GBRT model, such that, for each training window, the target values are concatenated with external features, and then flattened to form one input instance for a multi-output GBRT model. We conducted a comparative study on nine datasets for eight state-of-the-art deep-learning models that were presented at top-level conferences in the last years. The results demonstrate that the window-based input transformation boosts the performance of a simple GBRT model to levels that outperform all state-of-the-art DNN models evaluated in this paper.
Adversarial Robustness by Design through Analog Computing and Synthetic Gradients
Authors: Alessandro Cappelli, Ruben Ohana, Julien Launay, Laurent Meunier, Iacopo Poli, Florent Krzakala
Date: 2021-01-06 00:00:00
Tasks: Adversarial Robustness, Binarization

We propose a new defense mechanism against adversarial attacks inspired by an optical co-processor, providing robustness without compromising natural accuracy in both white-box and black-box settings. This hardware co-processor performs a nonlinear fixed random transformation, where the parameters are unknown and impossible to retrieve with sufficient precision for large enough dimensions. In the white-box setting, our defense works by obfuscating the parameters of the random projection. Unlike other defenses relying on obfuscated gradients, we find we are unable to build a reliable backward differentiable approximation for obfuscated parameters. Moreover, while our model reaches a good natural accuracy with a hybrid backpropagation - synthetic gradient method, the same approach is suboptimal if employed to generate adversarial examples. We find the combination of a random projection and binarization in the optical system also improves robustness against various types of black-box attacks. Finally, our hybrid training method builds robust features against transfer attacks. We demonstrate our approach on a VGG-like architecture, placing the defense on top of the convolutional features, on CIFAR-10 and CIFAR-100. Code is available at https://github.com/lightonai/adversarial-robustness-by-design.
The Interplay of Demographic Variables and Social Distancing Scores in Deep Prediction of U.S. COVID-19 Cases
Authors: Francesca Tang, Yang Feng, Hamza Chiheb, Jianqing Fan
Date: 2021-01-06 00:00:00
Tasks: Clustering

With the severity of the COVID-19 outbreak, we characterize the nature of the growth trajectories of counties in the United States using a novel combination of spectral clustering and the correlation matrix. As the U.S. and the rest of the world are experiencing a severe second wave of infections, the importance of assigning growth membership to counties and understanding the determinants of the growth are increasingly evident. Subsequently, we select the demographic features that are most statistically significant in distinguishing the communities. Lastly, we effectively predict the future growth of a given county with an LSTM using three social distancing scores. This comprehensive study captures the nature of counties' growth in cases at a very micro-level using growth communities, demographic factors, and social distancing performance to help government agencies utilize known information to make appropriate decisions regarding which potential counties to target resources and funding to.
Liquidity Stress Testing in Asset Management -- Part 1. Modeling the Liability Liquidity Risk
Authors: Thierry Roncalli, Fatma Karray-Meziou, François Pan, Margaux Regnault
Date: 2021-01-06 00:00:00
Tasks: Asset Management, Management

This article is part of a comprehensive research project on liquidity risk in asset management, which can be divided into three dimensions. The first dimension covers liability liquidity risk (or funding liquidity) modeling, the second dimension focuses on asset liquidity risk (or market liquidity) modeling, and the third dimension considers asset-liability liquidity risk management (or asset-liability matching). The purpose of this research is to propose a methodological and practical framework in order to perform liquidity stress testing programs, which comply with regulatory guidelines (ESMA, 2019) and are useful for fund managers. The review of the academic literature and professional research studies shows that there is a lack of standardized and analytical models. The aim of this research project is then to fill the gap with the goal to develop mathematical and statistical approaches, and provide appropriate answers. In this first part that focuses on liability liquidity risk modeling, we propose several statistical models for estimating redemption shocks. The historical approach must be complemented by an analytical approach based on zero-inflated models if we want to understand the true parameters that influence the redemption shocks. Moreover, we must also distinguish aggregate population models and individual-based models if we want to develop behavioral approaches. Once these different statistical models are calibrated, the second big issue is the risk measure to assess normal and stressed redemption shocks. Finally, the last issue is to develop a factor model that can translate stress scenarios on market risk factors into stress scenarios on fund liabilities.
PowerDynamics.jl -- An experimentally validated open-source package for the dynamical analysis of power grids
Authors: Anton Plietzsch, Raphael Kogler, Sabine Auer, Julia Merino, Asier Gil-de-Muro, Jan Liße, Christina Vogel, Frank Hellmann
Date: 2021-01-06 00:00:00

PowerDynamics.jl is a Julia package for time-domain modeling of power grids that is specifically designed for the stability analysis of systems with high shares of renewable energies. It makes use of Julia's state-of-the-art differential equation solvers and is highly performant even for systems with a large number of components. Further, it is compatible with Julia's machine learning libraries and allows for the utilization of these methods for dynamical optimization and parameter fitting. The package comes with a number of predefined models for synchronous machines, transmission lines and inverter systems. However, the strict open-source approach and a macro-based user-interface also allows for an easy implementation of custom-built models which makes it especially interesting for the design and testing of new control strategies for distributed generation units. This paper presents how the modeling concept, implemented component models and fault scenarios have been experimentally tested against measurements in the microgrid lab of TECNALIA.