Miracle-dz/UniFAR-FTUs huggingface dataset:
Generalizing to New Tasks via One-Shot Compositional Subgoals

—The ability to generalize to previously unseen tasks with little to no supervision is a key challenge in modern machine learning research. It is also a cornerstone of a future “General AI”. Any artiﬁcially intelligent agent deployed in a real world application, must adapt on the ﬂy to unknown environments.Researchers often rely on reinforcement and imitation learning to provide online adaptation to new tasks, through trial and error learning. However, this can be challenging for complex tasks which require many timesteps or large numbers of subtasks to complete. These “long horizon” tasks suffer from sample inefﬁciency and can require extremely long training times before the agent can learn to perform the necessary long-term planning. In this work, we introduce CASE which attempts to ad- dress these issues by training an Imitation Learning agent using adaptive “near future" subgoals. These subgoals are re- calculated at each step using compositional arithmetic in a learned latent representation space. In addition to improving learning efﬁciency for standard long-term tasks, this approach also makes it possible to perform one-shot generalization to previously unseen tasks, given only a single reference trajectory for the task in a different environment. Our experiments show that the proposed approach consistently outperforms the previous state-of-the-art compositional Imitation Learning approach by 30%.
Bridging Sim2Real Gap Using Image Gradients for the Task of End-to-End Autonomous Driving

While simulations are vital for developing and testing robotic agents, the ability to transfer robotics skills learned in simulation to reality is equally challenging. Simulation to realworld transfer, popularly known as Sim2Real transfer, is an indispensable line of research for utilizing knowledge learned from simulated data to derive meaningful inferences from real-world observations and function in actual operational environments. Sim2Real is critical for developing Autonomous Vehicles (AV) and other field-deployed intelligent robotic systems. The NeurIPS 2021 AWS DeepRacer challenge is a part of a series of competitions in the area of AV called The AI Driving Olympics (AI-DO). In this competition, the task is to train a Reinforcement Learning (RL) agent (i.e., an autonomous car) that learns to drive by interacting with a virtual environment (a simulated track). The agent achieves this by taking action based on previous observations and a current state to maximize the expected reward. We test the trained agent’s driving performance when maneuvering an AWS DeepRacer car on a real-world mock track. The driving performance metric is time to lap completion without going off the same in the context of AWS DeepRacer competition. In Fig 1, we show the images of the simulated (left) and actual (right) track as captured by the camera mounted on the vehicle with matching extrinsic parameters.
The Sharp Interface Limit of a Navier-Stokes/Allen-Cahn System with Constant Mobility: Convergence Rates by a Relative Energy Approach

We investigate the sharp interface limit of a diffuse interface system that couples the Allen--Cahn equation with the instationary Navier--Stokes system in a bounded domain in $\mathbb{R}^d$ with $d \in \{2,3\}$. This model is used to describe a propagating front in a viscous incompressible flow with the width of the transition layer being characterized by a small parameter $\varepsilon>0$. We show that the solutions converge to a limit two-phase fluid system with surface tension that couples the mean curvature flow and the Navier--Stokes system. The main assumptions are that the evolution of the limit system is sufficiently regular and that the associated evolving interface does not intersect the boundary of the container. For quantitatively well-prepared initial data, we even establish an optimal convergence rate. This is the first rigorous result of this kind which is valid in all physically relevant ambient dimensions.
Uniform models and short curves for random 3-manifolds

We provide two constructions of hyperbolic metrics on 3-manifolds with Heegaard splittings that satisfy certain topological conditions, which both apply to random Heegaard splitting with asymptotic probability 1. These constructions provide a lot of control on the resulting metric, allowing us to prove various results about the coarse growth rate of geometric invariants, such as diameter and injectivity radius, and about arithmeticity and commensurability in families of random 3-manifolds. For example, we show that the diameter of a random Heegaard splitting grows coarsely linearly in the length of the associated random walk. The constructions only use tools from the deformation theory of Kleinian groups, that is, we do not rely on the solution of the Geometrization Conjecture by Perelman. In particular, we give a proof of Maher’s result that random 3-manifolds are hyperbolic that bypasses Geometrization. 20P05.
A priori estimates and Liouville type results for quasilinear elliptic equations involving gradient terms

In this article we study local and global properties of positive solutions of − ∆ m u = | u | p − 1 u + M |∇ u | q in a domain Ω of R N , with m > 1, p, q > 0 and M ∈ R . Following some ideas used in [6, 7], and by using a direct Bernstein method combined with Keller-Osserman’s estimate, we obtain several a priori estimates as well as Liouville type theorems. Moreover, we prove a local Harnack inequality with the help of Serrin’s classical results.
Learning-Based Orchestration for Dynamic Functional Split and Resource Allocation in vRANs

One of the key benefits of virtualized radio access networks (vRANs) is network management flexibility. However, this versatility raises previously-unseen network management challenges. In this paper, a learning-based zero-touch vRAN orchestration framework (LOFV) is proposed to jointly select the functional splits and allocate the virtualized resources to minimize the long-term management cost. First, testbed measurements of the behaviour between the users’ demand and the virtualized resource utilization are collected using a centralized RAN system. The collected data reveals that there are non-linear and non-monotonic relationships between demand and resource utilization. Then, a comprehensive cost model is proposed that takes resource overprovisioning, declined demand, instantiation and reconfiguration into account. Moreover, the proposed cost model also captures different routing and computing costs for each split. Motivated by our measurement insights and cost model, LOFV is developed using a model-free reinforcement learning paradigm. The proposed solution is constructed from a combination of deep Q-learning and a regression-based neural network that maps the network state and users’ demand into split and resource control decisions. Our numerical evaluations show that LOFV can offer cost savings by up to 69% of the optimal static policy and 45% of the optimal fully dynamic policy.
Beyond General Purpose Machine Translation: The Need for Context-specific Empirical Research to Design for Appropriate User Trust

Machine Translation (MT) has the potential to help people overcome language barriers and is widely used in high-stakes scenarios, such as in hospitals. However, in order to use MT reliably and safely, users need to understand when to trust MT outputs and how to assess the quality of often imperfect translation results. In this paper, we discuss research directions to support users to calibrate trust in MT systems. We share findings from an empirical study in which we conducted semi-structured interviews with 20 clinicians to understand how they communicate with patients across language barriers, and if and how they use MT systems. Based on our findings, we advocate for empirical research on how MT systems are used in practice as an important first step to addressing the challenges in building appropriate trust between users and MT tools.
Enhancing the Transformer Decoder with Transition-based Syntax

Notwithstanding recent advances, syntactic generalization remains a challenge for text decoders. While some studies showed gains from incorporating source-side symbolic syntactic and semantic structure into text generation Transformers, very little work addressed the decoding of such structure. We propose a general approach for tree decoding using a transition-based approach. Examining the challenging test case of incorporating Universal Dependencies syntax into machine translation, we present substantial improvements on test sets that focus on syntactic generalization, while presenting improved or comparable performance on standard MT benchmarks. Further qualitative analysis addresses cases where syntactic generalization in the vanilla Transformer decoder is inadequate and demonstrates the advantages afforded by integrating syntactic information.
RoMFAC: A Robust Mean-Field Actor-Critic Reinforcement Learning against Adversarial Perturbations on States

Multiagent deep reinforcement learning (DRL) makes optimal decisions dependent on system states observed by agents, but any uncertainty on the observations may mislead agents to take wrong actions. The mean-field actor-critic (MFAC) reinforcement learning is well-known in the multiagent field since it can effectively handle a scalability problem. However, it is sensitive to state perturbations that can significantly degrade the team rewards. This work proposes a Robust MFAC (RoMFAC) reinforcement learning that has two innovations: 1) a new objective function of training actors, composed of a policy gradient function that is related to the expected cumulative discount reward on sampled clean states and an action loss function that represents the difference between actions taken on clean and adversarial states and 2) a repetitive regularization of the action loss, ensuring the trained actors to obtain excellent performance. Furthermore, this work proposes a game model named a state-adversarial stochastic game (SASG). Despite the Nash equilibrium of SASG may not exist, adversarial perturbations to states in the RoMFAC are proven to be defensible based on SASG. Experimental results show that RoMFAC is robust against adversarial perturbations while maintaining its competitive performance in environments without perturbations.
SaiNet: Stereo aware inpainting behind objects with generative networks

In this work, we present an end-to-end network for stereo-consistent image inpainting with the objective of inpainting large missing regions behind objects. The proposed model consists of an edge-guided UNet-like network using Partial Convolutions. We enforce multi-view stereo consistency by introducing a disparity loss. More importantly, we develop a training scheme where the model is learned from realistic stereo masks representing object occlusions, instead of the more common random masks. The technique is trained in a supervised way. Our evaluation shows competitive results compared to previous state-of-the-art techniques.
Modelling polymer draft gears

ABSTRACT This paper developed a new and simple approach to model polymer draft gears. Two types of polymer draft gears were modelled and compared with experimental data. Impact characteristics, in-train characteristics and frequency responses of these polymer draft gears were studied and compared with those of a friction draft gear. The impact simulations show that polymer draft gears can withstand higher impact speeds than the friction draft gear. Longitudinal train dynamics simulations show that polymer draft gears have significantly longer deflections than friction draft gears in normal train operations. The maximum draft gear working velocities are lower than 0.2 m/s, which are significantly lower than the impact velocities during shunting operations. Draft gears’ in-train characteristics are similar to their static characteristics but are very different from their impact characteristics; this conclusion has also been reached from frequency response simulations. An analysis of gangway bridge plate failures was also conducted and it was found that they were caused by coupler angling behaviour and long draft gear deflections.
Aspects of two-photon absorption of squeezed light: The continuous-wave limit

We present a theoretical analysis of two-photon absorption of classical and squeezed light valid when one-photon absorption to an intermediate state is either resonant or far-detuned from resonance, and in both the low and high intensity regimes. In this paper we concentrate on continuous-wave excitation, although the approach we develop is more general. We calculate the energy removed from an incident ﬁeld for typical experimental parameters and consider the limiting cases when the photon pairs are narrowband or broadband compared to the molecular linewidths. We ﬁnd an enhancement of the two-photon absorption due to resonant contributions from the large squeezed light bandwidth and due to photon bunching in the low intensity regime. However, in both cases, for the parameters we choose, the one-photon absorption is the dominant process in the region of parameter space where a large enhancement of the two-photon absorption is possible.
Virtual Disk Snapshot Management at Scale

Contrary to the other resources such as CPU, memory and network, for which virtualization is efﬁciently achieved through direct access, disk virtualization is peculiar. In this paper we make four contributions. Our ﬁrst contribution is the characterization of disk utilization in a public large scale cloud infrastructure. It reveals the presence of long snapshot chains, composed sometimes of up to 1000 ﬁles. Our second contribution is to show by experimental measurements that long chains lead to performance and memory footprint scalability issues. Our third contribution is the extension of both the Qcow2 format and its driver in Qemu to address the identiﬁed scalability challenges. Our fourth contribution is the thorough evaluation of our prototype, called S Q EMU , demonstrating that it brings signiﬁcant performance enhancements and memory footprint reduction. For example, it improves the throughput of RocksDB by about 48% compared to vanilla Qemu on a snapshot chain of length 500. The memory overhead on that chain is also reduced by 15x.
Rapid localization of gravitational wave hosts with FIGARO

The copious scientific literature produced after the detection of GW170817 electromagnetic counterpart demonstrated the importance of a prompt and accurate localization of the gravitational wave within the co-moving volume. In this letter, we present figaro, a ready to use and publicly available software that relies on Bayesian non-parametrics. figaro is designed to run in parallel with parameter estimation algorithms to provide updated three-dimensional volume localization information. Differently from any existing algorithms, the analytical nature of the figaro reconstruction allows a ranking of the entries of galaxy catalogues by their probability of being the host of a gravitational wave event, hence providing an additional tool for a prompt electromagnetic follow up of gravitational waves. We illustrate the features of figaro on binary black holes as well as on GW170817. Finally, we demonstrate the robustness of figaro by producing so-called pp-plots and we present a method based on information entropy to assess when, during the parameter estimation run, it is reasonable to begin releasing skymaps.
Random walks and rank one isometries on CAT(0) spaces

Let G be a discrete countable group and μ ∈ Prob(G) a probability measure on G. We define the random walk (Zn)n on G as the sequence Zn := ω1 . ωn, where the ω is are chosen independently according to the law μ. Fixing a point x ∈ X, where X is a given metric space on which G acts by isometries, we want to study the sequence of points (Znx)n. In this paper, we are particularly interested in the asymptotic behaviour of this random walk, and if the space X has the right geometric features, one can hope that (Znx)n is going to converge almost surely in a natural compactification of X. The typical setting in which results have been obtained is when X is of negative curvature, or at least when X has some kind of hyperbolic properties. In the fundamental paper of V. Kaimanovich [Kai00], the convergence of (Znx)n to a point of the visual boundary is proven for groups acting geometrically on proper hyperbolic spaces and several other classes of actions. More recently this result has been extended by J. Maher and G. Tiozzo in [MT18] for groups acting by isometries on non proper hyperbolic spaces. We emphasize on the fact studying random walks on non proper spaces involves different techniques, as in the non proper case the space X is no longer a compactification of X: X might be non compact, and X is no longer open in X. In the sequel, we will only consider proper metric spaces.
Blind Goal-Oriented Massive Access for Future Wireless Networks

Emerging communication networks are envisioned to support massive wireless connectivity of heterogeneous devices with sporadic traffic and diverse requirements in terms of latency, reliability, and bandwidth. Providing multiple access to an increasing number of uncoordinated users and sharing the limited resources become essential in this context. In this work, we revisit the random access (RA) problem and exploit the continuous angular group sparsity feature of wireless channels to propose a novel RA strategy that provides low latency, high reliability, and massive access with limited bandwidth resources in an all-in-one package. To this end, we first design a reconstruction-free goal-oriented optimization problem, which only preserves the angular information required to identify the active devices. To solve this, we propose an alternating direction method of multipliers (ADMM) and derive closed-form expressions for each ADMM step. Then, we design a clustering algorithm that assigns the users in specific groups from which we can identify active stationary devices by their line of sight (LoS) angles. For mobile devices, we propose an alternating minimization algorithm to recover their data and their channel gains simultaneously, which allows us to identify active mobile users. Simulation results show significant performance gains in terms of active user detection and false alarm probabilities as compared to state-of-the-art RA schemes, even with limited number of preambles. Moreover, unlike prior work, the performance of the proposed blind goal-oriented massive access does not depend on the number of devices.
Keynote Talk 3: Verifying Neural Networks Against Backdoor Attacks

Abstract: Neural networks have achieved state-of-the-art performance in solving many problems, including many applications in safety/security-critical systems. Researchers also discovered multiple security issues associated with neural networks. One of them is backdoor attacks, i.e., a neural network may be embedded with a backdoor such that a target output is almost always generated in the presence of a trigger. Existing defense approaches mostly focus on detecting whether a neural network is ‘backdoored’ based on heuristics, e.g., activation patterns. To the best of our knowledge, the only line of work which certifies the absence of backdoor is based on randomized smoothing, which is known to significantly reduce neural network performance. In this work, we propose an approach to verify whether a given neural network is free of backdoor with a certain level of success rate. Our approach integrates statistical sampling as well as abstract interpretation. The experiment results show that our approach effectively verifies the absence of backdoor or generates backdoor triggers.
Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localization

Unsupervised localization and segmentation are long-standing computer vision challenges that involve decom-posing an image into semantically meaningful segments without any labeled data. These tasks are particularly interesting in an unsupervised setting due to the difficulty and cost of obtaining dense image annotations, but existing un-supervised approaches struggle with complex scenes containing multiple objects. Differently from existing methods, which are purely based on deep learning, we take inspiration from traditional spectral segmentation methods by re-framing image decomposition as a graph partitioning problem. Specifically, we examine the eigenvectors of the Laplacian of a feature affinity matrix from self-supervised networks. We find that these eigenvectors already decompose an image into meaningful segments, and can be readily used to localize objects in a scene. Furthermore, by clustering the features associated with these segments across a dataset, we can obtain well-delineated, nameable regions, i.e. semantic segmentations. Experiments on complex datasets (PASCAL VOC, MS-COCO) demonstrate that our simple spectral method outperforms the state-of-the-art in unsupervised localization and segmentation by a significant margin. Furthermore, our method can be readily usedfor a variety of complex image editing tasks, such as background removal and compositing.11Project Page: https://lukemelas.github. io/deep-spectral-segmentation/
Deconstructing NLG Evaluation: Evaluation Practices, Assumptions, and Their Implications

There are many ways to express similar things in text, which makes evaluating natural language generation (NLG) systems difficult. Compounding this difficulty is the need to assess varying quality criteria depending on the deployment setting. While the landscape of NLG evaluation has been well-mapped, practitioners' goals, assumptions, and constraints -- which inform decisions about what, when, and how to evaluate -- are often partially or implicitly stated, or not stated at all. Combining a formative semi-structured interview study of NLG practitioners (N=18) with a survey study of a broader sample of practitioners (N=61), we surface goals, community practices, assumptions, and constraints that shape NLG evaluations, examining their implications and how they embody ethical considerations.
Importance Weighted Structure Learning for Scene Graph Generation

Scene graph generation is a structured prediction task aiming to explicitly model objects and their relationships via constructing a visually-grounded scene graph for an input image. Currently, the message passing neural network based mean field variational Bayesian methodology is the ubiquitous solution for such a task, in which the variational inference objective is often assumed to be the classical evidence lower bound. However, the variational approximation inferred from such loose objective generally underestimates the underlying posterior, which often leads to inferior generation performance. In this paper, we propose a novel importance weighted structure learning method aiming to approximate the underlying log-partition function with a tighter importance weighted lower bound, which is computed from multiple samples drawn from a reparameterizable Gumbel-Softmax sampler. A generic entropic mirror descent algorithm is applied to solve the resulting constrained variational inference task. The proposed method achieves the state-of-the-art performance on various popular scene graph generation benchmarks.