pwc-archive/papers-with-abstracts huggingface dataset:
Automatically Learning Fallback Strategies with Model-Free Reinforcement Learning in Safety-Critical Driving Scenarios
Authors: Ugo Lecerf, Christelle Yemdji-Tchassi, Sébastien Aubert, Pietro Michiardi
Date: 2022-04-11 00:00:00
Tasks: Autonomous Driving, Autonomous Vehicles, Reinforcement Learning (RL)

When learning to behave in a stochastic environment where safety is critical, such as driving a vehicle in traffic, it is natural for human drivers to plan fallback strategies as a backup to use if ever there is an unexpected change in the environment. Knowing to expect the unexpected, and planning for such outcomes, increases our capability for being robust to unseen scenarios and may help prevent catastrophic failures. Control of Autonomous Vehicles (AVs) has a particular interest in knowing when and how to use fallback strategies in the interest of safety. Due to imperfect information available to an AV about its environment, it is important to have alternate strategies at the ready which might not have been deduced from the original training data distribution. In this paper we present a principled approach for a model-free Reinforcement Learning (RL) agent to capture multiple modes of behaviour in an environment. We introduce an extra pseudo-reward term to the reward model, to encourage exploration to areas of state-space different from areas privileged by the optimal policy. We base this reward term on a distance metric between the trajectories of agents, in order to force policies to focus on different areas of state-space than the initial exploring agent. Throughout the paper, we refer to this particular training paradigm as learning fallback strategies. We apply this method to an autonomous driving scenario, and show that we are able to learn useful policies that would have otherwise been missed out on during training, and unavailable to use when executing the control algorithm.
Task-Synchronized Recurrent Neural Networks
Authors: Mantas Lukoševičius, Arnas Uselis
Date: 2022-04-11 00:00:00

Data are often sampled irregularly in time. Dealing with this using Recurrent Neural Networks (RNNs) traditionally involved ignoring the fact, feeding the time differences as additional inputs, or resampling the data. All these methods have their shortcomings. We propose an elegant straightforward alternative approach where instead the RNN is in effect resampled in time to match the time of the data or the task at hand. We use Echo State Network (ESN) and Gated Recurrent Unit (GRU) as the basis for our solution. Such RNNs can be seen as discretizations of continuous-time dynamical systems, which gives a solid theoretical ground to our approach. Our Task-Synchronized ESN (TSESN) and GRU (TSGRU) models allow for a direct model time setting and require no additional training, parameter tuning, or computation (solving differential equations or interpolating data) compared to their regular counterparts, thus retaining their original efficiency. We confirm empirically that our models can effectively compensate for the time-non-uniformity of the data and demonstrate that they compare favorably to data resampling, classical RNN methods, and alternative RNN models proposed to deal with time irregularities on several real-world nonuniform-time datasets. We open-source the code at https://github.com/oshapio/task-synchronized-RNNs .
Uniform Complexity for Text Generation
Authors: Joseph Marvin Imperial, Harish Tayyar Madabushi
Date: 2022-04-11 00:00:00
Tasks: Machine Translation, Question Answering, Text Generation

Large language models (LLMs) have shown promising results in a wide array of generative NLP tasks, such as summarization and machine translation. In the context of narrative generation, however, existing models still do not capture factors that contribute to producing consistent text. For instance, it is logical that a piece of text or a story should be uniformly readable throughout and that this form of complexity should be controllable. As such, if the complexity of an input text prompt is rated first-grade reading level in the Flesch Reading Ease test, then the generated text continuing the plot should also be within this range of complexity. With this in mind, we introduce Uniform Complexity for Text Generation (UCTG), a new benchmark test which raises the challenge of making generative models observe uniform linguistic properties with respect to prompts. We experiment with over 150+ linguistically and cognitively motivated features for evaluating text complexity in humans and generative models. From our results, we find that models such as GPT-2 struggle to preserve the complexity of input prompts used in its generations, even if finetuned with professionally written texts.
On unsupervised projections and second order signals
Authors: Thomas Lartigue, Sach Mukherjee
Date: 2022-04-11 00:00:00

Linear projections are widely used in the analysis of high-dimensional data. In unsupervised settings where the data harbour latent classes/clusters, the question of whether class discriminatory signals are retained under projection is crucial. In the case of mean differences between classes, this question has been well studied. However, in many contemporary applications, notably in biomedicine, group differences at the level of covariance or graphical model structure are important. Motivated by such applications, in this paper we ask whether linear projections can preserve differences in second order structure between latent groups. We focus on unsupervised projections, which can be computed without knowledge of class labels. We discuss a simple theoretical framework to study the behaviour of such projections which we use to inform an analysis via quasi-exhaustive enumeration. This allows us to consider the performance, over more than a hundred thousand sets of data-generating population parameters, of two popular projections, namely random projections (RP) and Principal Component Analysis (PCA). Across this broad range of regimes, PCA turns out to be more effective at retaining second order signals than RP and is often even competitive with supervised projection. We complement these results with fully empirical experiments showing 0-1 loss using simulated and real data. We study also the effect of projection dimension, drawing attention to a bias-variance trade-off in this respect. Our results show that PCA can indeed be a suitable first-step for unsupervised analysis, including in cases where differential covariance or graphical model structure are of interest.
SoK: Privacy Preserving Machine Learning using Functional Encryption: Opportunities and Challenges
Authors: Prajwal Panzade, Daniel Takabi
Date: 2022-04-11 00:00:00
Tasks: BIG-bench Machine Learning, Cloud Computing, Privacy Preserving

With the advent of functional encryption, new possibilities for computation on encrypted data have arisen. Functional Encryption enables data owners to grant third-party access to perform specified computations without disclosing their inputs. It also provides computation results in plain, unlike Fully Homomorphic Encryption. The ubiquitousness of machine learning has led to the collection of massive private data in the cloud computing environment. This raises potential privacy issues and the need for more private and secure computing solutions. Numerous efforts have been made in privacy-preserving machine learning (PPML) to address security and privacy concerns. There are approaches based on fully homomorphic encryption (FHE), secure multiparty computation (SMC), and, more recently, functional encryption (FE). However, FE-based PPML is still in its infancy and has not yet gotten much attention compared to FHE-based PPML approaches. In this paper, we provide a systematization of PPML works based on FE summarizing state-of-the-art in the literature. We focus on Inner-product-FE and Quadratic-FE-based machine learning models for the PPML applications. We analyze the performance and usability of the available FE libraries and their applications to PPML. We also discuss potential directions for FE-based PPML approaches. To the best of our knowledge, this is the first work to systematize FE-based PPML approaches.
Zero-phase angle asteroid taxonomy classification using unsupervised machine learning algorithms
Authors: M. Colazo, A. Alvarez-Candal, R. Duffard
Date: 2022-04-11 00:00:00
Tasks: BIG-bench Machine Learning, Classification

We are in an era of large catalogs and, thus, statistical analysis tools for large data sets, such as machine learning, play a fundamental role. One example of such a survey is the Sloan Moving Object Catalog (MOC), which lists the astrometric and photometric information of all moving objects captured by the Sloan field of view. One great advantage of this telescope is represented by its set of five filters, allowing for taxonomic analysis of asteroids by studying their colors. However, until now, the color variation produced by the change of phase angle of the object has not been taken into account. In this paper, we address this issue by using absolute magnitudes for classification. We aim to produce a new taxonomic classification of asteroids based on their magnitudes that is unaffected by variations caused by the change in phase angle. We selected 9481 asteroids with absolute magnitudes of Hg, Hi and Hz, computed from the Sloan Moving Objects Catalog using the HG12 system. We calculated the absolute colors with them. To perform the taxonomic classification, we applied a unsupervised machine learning algorithm known as fuzzy C-means. This is a useful soft clustering tool for working with {data sets where the different groups are not completely separated and there are regions of overlap between them. We have chosen to work with the four main taxonomic complexes, C, S, X, and V, as they comprise most of the known spectral characteristics. We classified a total of 6329 asteroids with more than 60% probability of belonging to the assigned taxonomic class, with 162 of these objects having been characterized by an ambiguous classification in the past. By analyzing the sample obtained in the plane Semimajor axis versus inclination, we identified 15 new V-type asteroid candidates outside the Vesta family region.
Fine-grained Noise Control for Multispeaker Speech Synthesis
Authors: Karolos Nikitaras, Georgios Vamvoukakis, Nikolaos Ellinas, Konstantinos Klapsas, Konstantinos Markopoulos, Spyros Raptis, June Sig Sung, Gunu Jho, Aimilios Chalamandaris, Pirros Tsiakoulis
Date: 2022-04-11 00:00:00
Tasks: Expressive Speech Synthesis, Speech Synthesis, text-to-speech, Text to Speech

A text-to-speech (TTS) model typically factorizes speech attributes such as content, speaker and prosody into disentangled representations.Recent works aim to additionally model the acoustic conditions explicitly, in order to disentangle the primary speech factors, i.e. linguistic content, prosody and timbre from any residual factors, such as recording conditions and background noise.This paper proposes unsupervised, interpretable and fine-grained noise and prosody modeling. We incorporate adversarial training, representation bottleneck and utterance-to-frame modeling in order to learn frame-level noise representations. To the same end, we perform fine-grained prosody modeling via a Fully Hierarchical Variational AutoEncoder (FVAE) which additionally results in more expressive speech synthesis.
FederatedScope: A Flexible Federated Learning Platform for Heterogeneity
Authors: Yuexiang Xie, Zhen Wang, Dawei Gao, Daoyuan Chen, Liuyi Yao, Weirui Kuang, Yaliang Li, Bolin Ding, Jingren Zhou
Date: 2022-04-11 00:00:00
Tasks: Federated Learning, Hyperparameter Optimization

Although remarkable progress has been made by existing federated learning (FL) platforms to provide infrastructures for development, these platforms may not well tackle the challenges brought by various types of heterogeneity, including the heterogeneity in participants' local data, resources, behaviors and learning goals. To fill this gap, in this paper, we propose a novel FL platform, named FederatedScope, which employs an event-driven architecture to provide users with great flexibility to independently describe the behaviors of different participants. Such a design makes it easy for users to describe participants with various local training processes, learning goals and backends, and coordinate them into an FL course with synchronous or asynchronous training strategies. Towards an easy-to-use and flexible platform, FederatedScope enables rich types of plug-in operations and components for efficient further development, and we have implemented several important components to better help users with privacy protection, attack simulation and auto-tuning. We have released FederatedScope at https://github.com/alibaba/FederatedScope to promote academic research and industrial deployment of federated learning in a wide range of scenarios.
Team ÚFAL at CMCL 2022 Shared Task: Figuring out the correct recipe for predicting Eye-Tracking features using Pretrained Language Models
Authors: Sunit Bhattacharya, Rishu Kumar, Ondrej Bojar
Date: 2022-04-11 00:00:00
Proceeding: CMCL (ACL) 2022 5
Tasks: Pretrained Multilingual Language Models

Eye-Tracking data is a very useful source of information to study cognition and especially language comprehension in humans. In this paper, we describe our systems for the CMCL 2022 shared task on predicting eye-tracking information. We describe our experiments with pretrained models like BERT and XLM and the different ways in which we used those representations to predict four eye-tracking features. Along with analysing the effect of using two different kinds of pretrained multilingual language models and different ways of pooling the tokenlevel representations, we also explore how contextual information affects the performance of the systems. Finally, we also explore if factors like augmenting linguistic information affect the predictions. Our submissions achieved an average MAE of 5.72 and ranked 5th in the shared task. The average MAE showed further reduction to 5.25 in post task evaluation.
T- Hop: Tensor representation of paths in graph convolutional networks
Authors: Abdulrahman Ibraheem
Date: 2022-04-11 00:00:00
Tasks: Dimensionality Reduction

We describe a method for encoding path information in graphs into a 3-d tensor. We show a connection between the introduced path representation scheme and powered adjacency matrices. To alleviate the heavy computational demands of working with the 3-d tensor, we propose to apply dimensionality reduction on the depth axis of the tensor. We then describe our the reduced 3-d matrix can be parlayed into a plausible graph convolutional layer, by infusing it into an established graph convolutional network framework such as MixHop.
External control of a genetic toggle switch via Reinforcement Learning
Authors: Sara Maria Brancato, Francesco De Lellis, Davide Salzano, Giovanni Russo, Mario di Bernardo
Date: 2022-04-11 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning (RL)

We investigate the problem of using a learning-based strategy to stabilize a synthetic toggle switch via an external control approach. To overcome the data efficiency problem that would render the algorithm unfeasible for practical use in synthetic biology, we adopt a sim-to-real paradigm where the policy is learnt via training on a simplified model of the toggle switch and it is then subsequently exploited to control a more realistic model of the switch parameterized from in-vivo experiments. Our in-silico experiments confirm the viability of the approach suggesting its potential use for in-vivo control implementations.
Non-Convex Optimization with Certificates and Fast Rates Through Kernel Sums of Squares
Authors: Blake Woodworth, Francis Bach, Alessandro Rudi
Date: 2022-04-11 00:00:00

We consider potentially non-convex optimization problems, for which optimal rates of approximation depend on the dimension of the parameter space and the smoothness of the function to be optimized. In this paper, we propose an algorithm that achieves close to optimal a priori computational guarantees, while also providing a posteriori certificates of optimality. Our general formulation builds on infinite-dimensional sums-of-squares and Fourier analysis, and is instantiated on the minimization of multivariate periodic functions.
Projection-free Online Learning with Arbitrary Delays
Authors: Yuanyu Wan, Yibo Wang, Chang Yao, Wei-Wei Tu, Lijun Zhang
Date: 2022-04-11 00:00:00

Projection-free online learning, which eschews the projection operation via less expensive computations such as linear optimization (LO), has received much interest recently due to its efficiency in handling high-dimensional problems with complex constraints. However, previous studies assume that any queried gradient is revealed immediately, which may not hold in practice and limits their applications. To address this limitation, we generalize the online Frank-Wolfe (OFW) algorithm and the online smooth projection-free (OSPF) algorithm, which are state-of-the-art LO-based projection-free online algorithms for non-smooth and smooth functions respectively, into a delayed setting where queried gradients can be delayed by arbitrary rounds. Specifically, the main idea of our generalized OFW is to perform an update similar to the original OFW after receiving any delayed gradient, and play the latest decision for each round. Moreover, the essential change on OSPF is to replace the sum of queried gradients, which is originally utilized in each update, with the sum of available gradients. Despite their simplicities, our novel analysis shows that under a relatively large amount of delay, the generalized OFW and OSPF enjoy the same regret bound as OFW and OSPF in the non-delayed setting, respectively.
Assessment of Massively Multilingual Sentiment Classifiers
Authors: Krzysztof Rajda, Łukasz Augustyniak, Piotr Gramacki, Marcin Gruza, Szymon Woźniak, Tomasz Kajdanowicz
Date: 2022-04-11 00:00:00
Proceeding: WASSA (ACL) 2022 5
Tasks: Sentiment Analysis

Models are increasing in size and complexity in the hunt for SOTA. But what if those 2\% increase in performance does not make a difference in a production use case? Maybe benefits from a smaller, faster model outweigh those slight performance gains. Also, equally good performance across languages in multilingual tasks is more important than SOTA results on a single one. We present the biggest, unified, multilingual collection of sentiment analysis datasets. We use these to assess 11 models and 80 high-quality sentiment datasets (out of 342 raw datasets collected) in 27 languages and included results on the internally annotated datasets. We deeply evaluate multiple setups, including fine-tuning transformer-based models for measuring performance. We compare results in numerous dimensions addressing the imbalance in both languages coverage and dataset sizes. Finally, we present some best practices for working with such a massive collection of datasets and models from a multilingual perspective.
Application of QUBO solver using black-box optimization to structural design for resonance avoidance
Authors: Tadayoshi Matsumori, Masato Taki, Tadashi Kadowaki
Date: 2022-04-11 00:00:00
Tasks: Combinatorial Optimization

Quadratic unconstrained binary optimization (QUBO) solvers can be applied to design an optimal structure to avoid resonance. QUBO algorithms that work on a classical or quantum device have succeeded in some industrial applications. However, their applications are still limited due to the difficulty of transforming from the original optimization problem to QUBO. Recently, black-box optimization (BBO) methods have been proposed to tackle this issue using a machine learning technique and a Bayesian treatment for combinatorial optimization. We employed the BBO methods to design a printed circuit board for resonance avoidance. This design problem is formulated to maximize natural frequency and simultaneously minimize the number of mounting points. The natural frequency, which is the bottleneck for the QUBO formulation, is approximated to a quadratic model in the BBO method. We demonstrated that BBO using a factorization machine shows good performance in both the calculation time and the success probability of finding the optimal solution. Our results can open up QUBO solvers' potential for other applications in structural designs.
Lyapunov-Guided Representation of Recurrent Neural Network Performance
Authors: Ryan Vogt, Yang Zheng, Eli Shlizerman
Date: 2022-04-11 00:00:00
Tasks: Time Series, Time Series Analysis

Recurrent Neural Networks (RNN) are ubiquitous computing systems for sequences and multivariate time series data. While several robust architectures of RNN are known, it is unclear how to relate RNN initialization, architecture, and other hyperparameters with accuracy for a given task. In this work, we propose to treat RNN as dynamical systems and to correlate hyperparameters with accuracy through Lyapunov spectral analysis, a methodology specifically designed for nonlinear dynamical systems. To address the fact that RNN features go beyond the existing Lyapunov spectral analysis, we propose to infer relevant features from the Lyapunov spectrum with an Autoencoder and an embedding of its latent representation (AeLLE). Our studies of various RNN architectures show that AeLLE successfully correlates RNN Lyapunov spectrum with accuracy. Furthermore, the latent representation learned by AeLLE is generalizable to novel inputs from the same task and is formed early in the process of RNN training. The latter property allows for the prediction of the accuracy to which RNN would converge when training is complete. We conclude that representation of RNN through Lyapunov spectrum along with AeLLE provides a novel method for organization and interpretation of variants of RNN architectures.
Learning to Induce Causal Structure
Authors: Nan Rosemary Ke, Silvia Chiappa, Jane Wang, Anirudh Goyal, Jorg Bornschein, Melanie Rey, Theophane Weber, Matthew Botvinic, Michael Mozer, Danilo Jimenez Rezende
Date: 2022-04-11 00:00:00

The fundamental challenge in causal induction is to infer the underlying graph structure given observational and/or interventional data. Most existing causal induction algorithms operate by generating candidate graphs and evaluating them using either score-based methods (including continuous optimization) or independence tests. In our work, we instead treat the inference process as a black box and design a neural network architecture that learns the mapping from both observational and interventional data to graph structures via supervised training on synthetic graphs. The learned model generalizes to new synthetic graphs, is robust to train-test distribution shifts, and achieves state-of-the-art performance on naturalistic graphs for low sample complexity.