OpenRaiser/Intern-Atlas huggingface dataset:
SafeAPT: Safe Simulation-to-Real Robot Learning using Diverse Policies Learned in Simulation
Venue: arXiv cs.RO 2022
Year: 2022

The framework of Simulation-to-real learning, i.e, learning policies in simulation and transferring those policies to the real world is one of the most promising approaches towards data-efficient learning in robotics. However, due to the inevitable reality gap between the simulation and the real world, a policy learned in the simulation may not always generate a safe behaviour on the real robot. As a result, during adaptation of the policy in the real world, the robot may damage itself or cause harm to its surroundings. In this work, we introduce a novel learning algorithm called SafeAPT that leverages a diverse repertoire of policies evolved in the simulation and transfers the most promising safe policy to the real robot through episodic interaction. To achieve this, SafeAPT iteratively learns a probabilistic reward model as well as a safety model using real-world observations combined with simulated experiences as priors. Then, it performs Bayesian optimization on the repertoire with the reward model while maintaining the specified safety constraint using the safety model. SafeAPT allows a robot to adapt to a wide range of goals safely with the same repertoire of policies evolved in the simulation. We compare SafeAPT with several baselines, both in simulated and real robotic experiments and show that SafeAPT finds high-performance policies within a few minutes in the real world while minimizing safety violations during the interactions.
A note on the use of equidistant contours for presenting scientific data
Venue: arXiv cs.HC 2022
Year: 2022

The passionate plea for the use of scientific colour maps misses some aspects in the visual presentation of scientific data. While a linear colour map based on scientific human colour perception is useful for the presentation of some images, like the three examples given of the topography of the earth, an apple and a passport photograph, scientific data are not presented. In this note it will be shown that there is more in scientific oceanographic data as they are presented in forms varying from historic equidistant contours, via a linear black-(grey)-white b&w map, a linear colour map and a nonlinear colour map. From an objective perspective, equidistant contouring is the best means for presenting scientific information in a relatively unbiased way. Nonlinear colour maps may add information to that by highlighting certain aspects also by varying the colour range if needed. Such information is not available from linear colour maps. Finally, the aesthetic aspect of visual data presentation is discussed.
Micro-level Reserving for General Insurance Claims using a Long Short-Term Memory Network
Venue: arXiv cs.LG 2022
Year: 2022

Detailed information about individual claims are completely ignored when insurance claims data are aggregated and structured in development triangles for loss reserving. In the hope of extracting predictive power from the individual claims characteristics, researchers have recently proposed to move away from these macro-level methods in favor of micro-level loss reserving approaches. We introduce a discrete-time individual reserving framework incorporating granular information in a deep learning approach named Long Short-Term Memory (LSTM) neural network. At each time period, the network has two tasks: first, classifying whether there is a payment or a recovery, and second, predicting the corresponding non-zero amount, if any. We illustrate the estimation procedure on a simulated and a real general insurance dataset. We compare our approach with the chain-ladder aggregate method using the predictive outstanding loss estimates and their actual values. Based on a generalized Pareto model for excess payments over a threshold, we adjust the LSTM reserve prediction to account for extreme payments.
Beyond synchronization: Body gestures and gaze direction in duo performance
Venue: arXiv cs.HC 2022
Year: 2022

In this chapter, we focus on two main categories of visual interaction: body gestures and gaze direction. Our focus on body gestures is motivated by research showing that gesture patterns often change during joint action tasks to become more predictable (van der Wel et al., 2016). Moreover, coordination sometimes emerges between musicians at the level of body sway (Chang et al., 2017). Our focus on gaze direction was motivated by the fact that gaze can serve simultaneously as a means of obtaining information about the world and as a means of communicating one's own attention and intent.
Differentially Private Top-k Selection via Canonical Lipschitz Mechanism
Venue: arXiv cs.LG 2022
Year: 2022

Selecting the top-$k$ highest scoring items under differential privacy (DP) is a fundamental task with many applications. This work presents three new results. First, the exponential mechanism, permute-and-flip and report-noisy-max, as well as their oneshot variants, are unified into the Lipschitz mechanism, an additive noise mechanism with a single DP-proof via a mandated Lipschitz property for the noise distribution. Second, this new generalized mechanism is paired with a canonical loss function to obtain the canonical Lipschitz mechanism, which can directly select k-subsets out of $d$ items in $O(dk+d \log d)$ time. The canonical loss function assesses subsets by how many users must change for the subset to become top-$k$. Third, this composition-free approach to subset selection improves utility guarantees by an $Ω(\log k)$ factor compared to one-by-one selection via sequential composition, and our experiments on synthetic and real-world data indicate substantial utility improvements.
Towards Scaling Difference Target Propagation by Learning Backprop Targets
Venue: arXiv cs.NE 2022
Year: 2022

The development of biologically-plausible learning algorithms is important for understanding learning in the brain, but most of them fail to scale-up to real-world tasks, limiting their potential as explanations for learning by real brains. As such, it is important to explore learning algorithms that come with strong theoretical guarantees and can match the performance of backpropagation (BP) on complex tasks. One such algorithm is Difference Target Propagation (DTP), a biologically-plausible learning algorithm whose close relation with Gauss-Newton (GN) optimization has been recently established. However, the conditions under which this connection rigorously holds preclude layer-wise training of the feedback pathway synaptic weights (which is more biologically plausible). Moreover, good alignment between DTP weight updates and loss gradients is only loosely guaranteed and under very specific conditions for the architecture being trained. In this paper, we propose a novel feedback weight training scheme that ensures both that DTP approximates BP and that layer-wise feedback weight training can be restored without sacrificing any theoretical guarantees. Our theory is corroborated by experimental results and we report the best performance ever achieved by DTP on CIFAR-10 and ImageNet 32$\times$32
Don't Change the Algorithm, Change the Data: Exploratory Data for Offline Reinforcement Learning
Venue: arXiv cs.LG 2022
Year: 2022

Recent progress in deep learning has relied on access to large and diverse datasets. Such data-driven progress has been less evident in offline reinforcement learning (RL), because offline RL data is usually collected to optimize specific target tasks limiting the data's diversity. In this work, we propose Exploratory data for Offline RL (ExORL), a data-centric approach to offline RL. ExORL first generates data with unsupervised reward-free exploration, then relabels this data with a downstream reward before training a policy with offline RL. We find that exploratory data allows vanilla off-policy RL algorithms, without any offline-specific modifications, to outperform or match state-of-the-art offline RL algorithms on downstream tasks. Our findings suggest that data generation is as important as algorithmic advances for offline RL and hence requires careful consideration from the community. Code and data can be found at https://github.com/denisyarats/exorl .
MEGA: Model Stealing via Collaborative Generator-Substitute Networks
Venue: arXiv cs.CR 2022
Year: 2022

Deep machine learning models are increasingly deployedin the wild for providing services to users. Adversaries maysteal the knowledge of these valuable models by trainingsubstitute models according to the inference results of thetargeted deployed models. Recent data-free model stealingmethods are shown effective to extract the knowledge of thetarget model without using real query examples, but they as-sume rich inference information, e.g., class probabilities andlogits. However, they are all based on competing generator-substitute networks and hence encounter training instability.In this paper we propose a data-free model stealing frame-work,MEGA, which is based on collaborative generator-substitute networks and only requires the target model toprovide label prediction for synthetic query examples. Thecore of our method is a model stealing optimization con-sisting of two collaborative models (i) the substitute modelwhich imitates the target model through the synthetic queryexamples and their inferred labels and (ii) the generatorwhich synthesizes images such that the confidence of thesubstitute model over each query example is maximized. Wepropose a novel coordinate descent training procedure andanalyze its convergence. We also empirically evaluate thetrained substitute model on three datasets and its applicationon black-box adversarial attacks. Our results show that theaccuracy of our trained substitute model and the adversarialattack success rate over it can be up to 33% and 40% higherthan state-of-the-art data-free black-box attacks.
Finding Directions in GAN's Latent Space for Neural Face Reenactment
Venue: arXiv cs.CV 2022
Year: 2022

This paper is on face/head reenactment where the goal is to transfer the facial pose (3D head orientation and expression) of a target face to a source face. Previous methods focus on learning embedding networks for identity and pose disentanglement which proves to be a rather hard task, degrading the quality of the generated images. We take a different approach, bypassing the training of such networks, by using (fine-tuned) pre-trained GANs which have been shown capable of producing high-quality facial images. Because GANs are characterized by weak controllability, the core of our approach is a method to discover which directions in latent GAN space are responsible for controlling facial pose and expression variations. We present a simple pipeline to learn such directions with the aid of a 3D shape model which, by construction, already captures disentangled directions for facial pose, identity and expression. Moreover, we show that by embedding real images in the GAN latent space, our method can be successfully used for the reenactment of real-world faces. Our method features several favorable properties including using a single source image (one-shot) and enabling cross-person reenactment. Our qualitative and quantitative results show that our approach often produces reenacted faces of significantly higher quality than those produced by state-of-the-art methods for the standard benchmarks of VoxCeleb1 & 2. Source code is available at: https://github.com/StelaBou/stylegan_directions_face_reenactment
The Stories We Tell About Data: Media Types for Data-Driven Storytelling
Venue: arXiv cs.HC 2022
Year: 2022

The emerging practice of data-driven storytelling is framing data using familiar narrative mechanisms such as slideshows, videos, and comics to make even highly complex phenomena understandable. However, current data stories are still not utilizing the full potential of the storytelling domain. One reason for this is that current data-driven storytelling practice does not leverage the full repertoire of media that can be used for storytelling, such as the spoken word, e-learning, and video games. In this paper, we propose a taxonomy focused specifically on media types for the purpose of widening the purview of data-driven storytelling simply by putting more tools into the hands of designers. Using our taxonomy as a generative tool, we also propose three novel storytelling mechanisms, including for live-streaming, gesture-driven oral presentations, and textual reports that dynamically incorporate visual representations.
Reinforcement Learning with Heterogeneous Data: Estimation and Inference
Venue: arXiv cs.LG 2022
Year: 2022

Reinforcement Learning (RL) has the promise of providing data-driven support for decision-making in a wide range of problems in healthcare, education, business, and other domains. Classical RL methods focus on the mean of the total return and, thus, may provide misleading results in the setting of the heterogeneous populations that commonly underlie large-scale datasets. We introduce the K-Heterogeneous Markov Decision Process (K-Hetero MDP) to address sequential decision problems with population heterogeneity. We propose the Auto-Clustered Policy Evaluation (ACPE) for estimating the value of a given policy, and the Auto-Clustered Policy Iteration (ACPI) for estimating the optimal policy in a given policy class. Our auto-clustered algorithms can automatically detect and identify homogeneous sub-populations, while estimating the Q function and the optimal policy for each sub-population. We establish convergence rates and construct confidence intervals for the estimators obtained by the ACPE and ACPI. We present simulations to support our theoretical findings, and we conduct an empirical study on the standard MIMIC-III dataset. The latter analysis shows evidence of value heterogeneity and confirms the advantages of our new method.
Non-adaptive and two-stage coding over the Z-channel
Venue: arXiv cs.IT 2022
Year: 2022

In this paper, we developed new coding strategies for the Z-channel. In particular, we look at the case with two-stage encoding. In this case, the encoder uses noiseless feedback once and adjusts the further encoding strategy based on the previous partial output of the channel. Nevertheless, the developed codes improve the known results with full feedback for small length and 1 error. A tool for the two-stage strategy is the development of a new optimality condition for non-adaptive codes.
Transformer-based Models of Text Normalization for Speech Applications
Venue: arXiv cs.LG 2022
Year: 2022

Text normalization, or the process of transforming text into a consistent, canonical form, is crucial for speech applications such as text-to-speech synthesis (TTS). In TTS, the system must decide whether to verbalize "1995" as "nineteen ninety five" in "born in 1995" or as "one thousand nine hundred ninety five" in "page 1995". We present an experimental comparison of various Transformer-based sequence-to-sequence (seq2seq) models of text normalization for speech and evaluate them on a variety of datasets of written text aligned to its normalized spoken form. These models include variants of the 2-stage RNN-based tagging/seq2seq architecture introduced by Zhang et al. (2019), where we replace the RNN with a Transformer in one or more stages, as well as vanilla Transformers that output string representations of edit sequences. Of our approaches, using Transformers for sentence context encoding within the 2-stage model proved most effective, with the fine-tuned BERT encoder yielding the best performance.
Ground Experiment of Full-Duplex Multi-UAV System Enabled by Directional Antennas
Venue: arXiv cs.IT 2022
Year: 2022

A high performance multi-UAV communication system, which bridges multiple UAVs and ground station, is one of the key enablers to realize a variety of UAV-based systems. To address the issues such as the low spectrum efficiency caused by the co-channel interference, we have proposed a spectrum-efficient full-duplex multi-UA V communication system with low hardware complexity. In this paper, on-ground experiments are conducted to confirm the feasibility and effectiveness of the key feature of the proposed system, i.e., co-channel interference cancellation among UAVs by directional antennas and UAV position control, instead of energy-consuming dedicated self-interference cancellers on UAVs in traditional full-duplex systems. Channel power of interference link between a pair of two UAVs reusing the same channel is measured, and the achievable channel capacity is also measured by a prototype system implemented by software-defined radio devices. The results of different antennas and different antenna heights are also compared. The experimental results agree well with the designs and confirm the feasibility and effectiveness of the proposed system. This ground experiment is a work in progress to provide preliminary results for the multi-UAV-based experiments in the air in the future.
LayoutEnhancer: Generating Good Indoor Layouts from Imperfect Data
Venue: arXiv cs.GR 2022
Year: 2022

We address the problem of indoor layout synthesis, which is a topic of continuing research interest in computer graphics. The newest works made significant progress using data-driven generative methods; however, these approaches rely on suitable datasets. In practice, desirable layout properties may not exist in a dataset, for instance, specific expert knowledge can be missing in the data. We propose a method that combines expert knowledge, for example, knowledge about ergonomics, with a data-driven generator based on the popular Transformer architecture. The knowledge is given as differentiable scalar functions, which can be used both as weights or as additional terms in the loss function. Using this knowledge, the synthesized layouts can be biased to exhibit desirable properties, even if these properties are not present in the dataset. Our approach can also alleviate problems of lack of data and imperfections in the data. Our work aims to improve generative machine learning for modeling and provide novel tools for designers and amateurs for the problem of interior layout creation.
Fully Online Meta-Learning Without Task Boundaries
Venue: arXiv cs.LG 2022
Year: 2022

While deep networks can learn complex functions such as classifiers, detectors, and trackers, many applications require models that continually adapt to changing input distributions, changing tasks, and changing environmental conditions. Indeed, this ability to continuously accrue knowledge and use past experience to learn new tasks quickly in continual settings is one of the key properties of an intelligent system. For complex and high-dimensional problems, simply updating the model continually with standard learning algorithms such as gradient descent may result in slow adaptation. Meta-learning can provide a powerful tool to accelerate adaptation yet is conventionally studied in batch settings. In this paper, we study how meta-learning can be applied to tackle online problems of this nature, simultaneously adapting to changing tasks and input distributions and meta-training the model in order to adapt more quickly in the future. Extending meta-learning into the online setting presents its own challenges, and although several prior methods have studied related problems, they generally require a discrete notion of tasks, with known ground-truth task boundaries. Such methods typically adapt to each task in sequence, resetting the model between tasks, rather than adapting continuously across tasks. In many real-world settings, such discrete boundaries are unavailable, and may not even exist. To address these settings, we propose a Fully Online Meta-Learning (FOML) algorithm, which does not require any ground truth knowledge about the task boundaries and stays fully online without resetting back to pre-trained weights. Our experiments show that FOML was able to learn new tasks faster than the state-of-the-art online learning methods on Rainbow-MNIST, CIFAR100 and CELEBA datasets.
Architecture Matters in Continual Learning
Venue: arXiv cs.LG 2022
Year: 2022

A large body of research in continual learning is devoted to overcoming the catastrophic forgetting of neural networks by designing new algorithms that are robust to the distribution shifts. However, the majority of these works are strictly focused on the "algorithmic" part of continual learning for a "fixed neural network architecture", and the implications of using different architectures are mostly neglected. Even the few existing continual learning methods that modify the model assume a fixed architecture and aim to develop an algorithm that efficiently uses the model throughout the learning experience. However, in this work, we show that the choice of architecture can significantly impact the continual learning performance, and different architectures lead to different trade-offs between the ability to remember previous tasks and learning new ones. Moreover, we study the impact of various architectural decisions, and our findings entail best practices and recommendations that can improve the continual learning performance.
PAGE-PG: A Simple and Loopless Variance-Reduced Policy Gradient Method with Probabilistic Gradient Estimation
Venue: arXiv cs.LG 2022
Year: 2022

Despite their success, policy gradient methods suffer from high variance of the gradient estimate, which can result in unsatisfactory sample complexity. Recently, numerous variance-reduced extensions of policy gradient methods with provably better sample complexity and competitive numerical performance have been proposed. After a compact survey on some of the main variance-reduced REINFORCE-type methods, we propose ProbAbilistic Gradient Estimation for Policy Gradient (PAGE-PG), a novel loopless variance-reduced policy gradient method based on a probabilistic switch between two types of updates. Our method is inspired by the PAGE estimator for supervised learning and leverages importance sampling to obtain an unbiased gradient estimator. We show that PAGE-PG enjoys a $\mathcal{O}\left( ε^{-3} \right)$ average sample complexity to reach an $ε$-stationary solution, which matches the sample complexity of its most competitive counterparts under the same setting. A numerical evaluation confirms the competitive performance of our method on classical control tasks.
On the Interference Cancellation by Reduced Channel Zero Forcing Class of Precodings in Massive MIMO Systems
Venue: arXiv cs.IT 2022
Year: 2022

In this paper, we study the interference cancellation capabilities of receivers and transmitters in multiple-input-multiple-output (MIMO) systems using theoretical calculations and numerical simulations in Quadriga. We study so-called Reduced Channel Zero-Forcing (RCZF) class of precoding as well as Minimum MSE Interference Rejection Combiner (MMSE-IRC) and QR Maximum Likelihood Detection (QR-MLD) receivers. Based on very simple but extremely useful algebraic manipulations, their asymptotical equivalence is proven analytically and demonstrated via simulations. Our theoretical and experimental results confirm that MMSE-IRC and QR-MLD receivers in combination with the RCZF precoding provide complete interference suppression asymptotically.
Exploring layerwise decision making in DNNs
Venue: arXiv cs.LG 2022
Year: 2022

While deep neural networks (DNNs) have become a standard architecture for many machine learning tasks, their internal decision-making process and general interpretability is still poorly understood. Conversely, common decision trees are easily interpretable and theoretically well understood. We show that by encoding the discrete sample activation values of nodes as a binary representation, we are able to extract a decision tree explaining the classification procedure of each layer in a ReLU-activated multilayer perceptron (MLP). We then combine these decision trees with existing feature attribution techniques in order to produce an interpretation of each layer of a model. Finally, we provide an analysis of the generated interpretations, the behaviour of the binary encodings and how these relate to sample groupings created during the training process of the neural network.