Miracle-dz/UniFAR-FTUs huggingface dataset:
PlanT: Explainable Planning Transformers via Object-Level Representations

Planning an optimal route in a complex environment requires efficient reasoning about the surrounding scene. While human drivers prioritize important objects and ignore details not relevant to the decision, learning-based planners typically extract features from dense, high-dimensional grid representations containing all vehicle and road context information. In this paper, we propose PlanT, a novel approach for planning in the context of self-driving that uses a standard transformer architecture. PlanT is based on imitation learning with a compact object-level input representation. On the Longest6 benchmark for CARLA, PlanT outperforms all prior methods (matching the driving score of the expert) while being 5.3x faster than equivalent pixel-based planning baselines during inference. Combining PlanT with an off-the-shelf perception module provides a sensor-based driving system that is more than 10 points better in terms of driving score than the existing state of the art. Furthermore, we propose an evaluation protocol to quantify the ability of planners to identify relevant objects, providing insights regarding their decision-making. Our results indicate that PlanT can focus on the most relevant object in the scene, even when this object is geometrically distant.
Adversarial Pretraining of Self-Supervised Deep Networks: Past, Present and Future

In this paper, we review adversarial pretraining of self-supervised deep networks including both convolutional neural networks and vision transformers. Unlike the adversarial training with access to labeled examples, adversarial pretraining is complicated as it only has access to unlabeled examples. To incorporate adversaries into pretraining models on either input or feature level, we find that existing approaches are largely categorized into two groups: memory-free instance-wise attacks imposing worst-case perturbations on individual examples, and memory-based adversaries shared across examples over iterations. In particular, we review several representative adversarial pretraining models based on Contrastive Learning (CL) and Masked Image Modeling (MIM), respectively, two popular self-supervised pretraining methods in literature. We also review miscellaneous issues about computing overheads, input-/feature-level adversaries, as well as other adversarial pretraining approaches beyond the above two groups. Finally, we discuss emerging trends and future directions about the relations between adversarial and cooperative pretraining, unifying adversarial CL and MIM pretraining, and the trade-off between accuracy and robustness in adversarial pretraining.
Depth Monocular Estimation with Attention-based Encoder-Decoder Network from Single Image

Depth information is the foundation of perception, essential for autonomous driving, robotics, and other source- constrained applications. Promptly obtaining accurate and effi-cient depth information allows for a rapid response in dynamic environments. Sensor-based methods using LIDAR and RADAR obtain high precision at the cost of high power consumption, price, and volume. While due to advances in deep learning, vision-based approaches have recently received much attention and can overcome these drawbacks. In this work, we explore an extreme scenario in vision-based settings: estimate a depth map from one monocular image severely plagued by grid artifacts and blurry edges. To address this scenario, We first design a convolutional attention mechanism block (CAMB) which consists of channel attention and spatial attention sequentially and insert these CAMBs into skip connections. As a result, our novel approach can find the focus of current image with minimal overhead and avoid losses of depth features. Next, by combining the depth value, the gradients of X axis, Y axis and diagonal directions, and the structural similarity index measure (SSIM), we propose our novel loss function. Moreover, we utilize pixel blocks to accelerate the computation of the loss function. Finally, we show, through comprehensive experiments on two large- scale image datasets, i.e. KITTI and NYU - V2, that our method outperforms several representative baselines.
Learning Explicit Object-Centric Representations with Vision Transformers

With the recent successful adaptation of transformers to the vision domain, particularly when trained in a self-supervised fashion, it has been shown that vision transformers can learn impressive object-reasoning-like behaviour and features expressive for the task of object segmentation in images. In this paper, we build on the self-supervision task of masked autoencoding and explore its effectiveness for explicitly learning object-centric representations with transformers. To this end, we design an object-centric autoencoder using transformers only and train it end-to-end to reconstruct full images from unmasked patches. We show that the model efficiently learns to decompose simple scenes as measured by segmentation metrics on several multi-object benchmarks.
Shortest Edit Path Crossover: A Theory-driven Solution to the Permutation Problem in Evolutionary Neural Architecture Search

Population-based search has recently emerged as a possible alternative to Reinforcement Learning (RL) for black-box neural architecture search (NAS). It performs well in practice even though it is not theoretically well understood. In particular, whereas traditional population-based search methods such as evolutionary algorithms (EAs) draw much power from crossover operations, it is difficult to take advantage of them in NAS. The main obstacle is believed to be the permutation problem: The mapping between genotype and phenotype in traditional graph representations is many-to-one, leading to a disruptive effect of standard crossover. This paper presents the first theoretical analysis of the behaviors of mutation, crossover and RL in black-box NAS, and proposes a new crossover operator based on the shortest edit path (SEP) in graph space. The SEP crossover is shown theoretically to overcome the permutation problem, and as a result, have a better expected improvement compared to mutation, standard crossover and RL. Further, it empirically outperform these other methods on state-of-the-art NAS benchmarks. The SEP crossover therefore allows taking full advantage of population-based search in NAS, and the underlying theory can serve as a foundation for deeper understanding of black-box NAS methods in general.
A pixel space method for testing dipole modulation in the CMB polarization

We introduce a pixel space method to detect dipole modulation or hemispherical power asymmetry in the cosmic microwave background (CMB) polarization. The method relies on the use of squared total polarized flux whose ensemble average picks up a dipole due to the dipole modulation in the CMB polarization. The method is useful since it can be applied easily to partial sky. We define several statistics to characterize the amplitude of the detected signal. By simulations we show that the method can be used to reliably extract the signal at 2.7$\sigma$ level or higher in future CORE-like missions, assuming that the signal is present in the CMB polarization at the level detected by the Planck mission in the CMB temperature. An application of the method to the 2018 Planck data does not detect a significant effect, when taking into account the presence of correlated detector noise and residual systematics in data. Using the FFP10 we find the presence of a very strong bias which might be masking any real effect.
GLIF: A Unified Gated Leaky Integrate-and-Fire Neuron for Spiking Neural Networks

Spiking Neural Networks (SNNs) have been studied over decades to incorporate their biological plausibility and leverage their promising energy efficiency. Throughout existing SNNs, the leaky integrate-and-fire (LIF) model is commonly adopted to formulate the spiking neuron and evolves into numerous variants with different biological features. However, most LIF-based neurons support only single biological feature in different neuronal behaviors, limiting their expressiveness and neuronal dynamic diversity. In this paper, we propose GLIF, a unified spiking neuron, to fuse different bio-features in different neuronal behaviors, enlarging the representation space of spiking neurons. In GLIF, gating factors, which are exploited to determine the proportion of the fused bio-features, are learnable during training. Combining all learnable membrane-related parameters, our method can make spiking neurons different and constantly changing, thus increasing the heterogeneity and adaptivity of spiking neurons. Extensive experiments on a variety of datasets demonstrate that our method obtains superior performance compared with other SNNs by simply changing their neuronal formulations to GLIF. In particular, we train a spiking ResNet-19 with GLIF and achieve $77.35\%$ top-1 accuracy with six time steps on CIFAR-100, which has advanced the state-of-the-art. Codes are available at \url{https://github.com/Ikarosy/Gated-LIF}.
The Virality of Hate Speech on Social Media

Online hate speech is responsible for violent attacks such as, e.g., the Pittsburgh synagogue shooting in 2018, thereby posing a significant threat to vulnerable groups and society in general. However, little is known about what makes hate speech on social media go viral. In this paper, we collect N = 25,219 cascades with 65,946 retweets from X (formerly known as Twitter) and classify them as hateful vs. normal. Using a generalized linear regression, we then estimate differences in the spread of hateful vs. normal content based on author and content variables. We thereby identify important determinants that explain differences in the spreading of hateful vs. normal content. For example, hateful content authored by verified users is disproportionally more likely to go viral than hateful content from non-verified ones: hateful content from a verified user (as opposed to normal content) has a 3.5 times larger cascade size, a 3.2 times longer cascade lifetime, and a 1.2 times larger structural virality. Altogether, we offer novel insights into the virality of hate speech on social media.
Mangrove: Learning Galaxy Properties from Merger Trees

Efficiently mapping baryonic properties onto dark matter is a major challenge in astrophysics. Although semianalytic models (SAMs) and hydrodynamical simulations have made impressive advances in reproducing galaxy observables across cosmologically significant volumes, these methods still require significant computation times, representing a barrier to many applications. Graph neural networks have recently proven to be the natural choice for learning physical relations. Among the most inherently graph-like structures found in astrophysics are the dark matter merger trees that encode the evolution of dark matter halos. In this paper, we introduce a new, graph-based emulator framework, Mangrove, and show that it emulates the galactic stellar mass, cold gas mass and metallicity, instantaneous and time-averaged star formation rate, and black hole mass—as predicted by an SAM—with an rms error up to 2 times lower than other methods across a (75 Mpc/h)3 simulation box in 40 s, 4 orders of magnitude faster than the SAM. We show that Mangrove allows for quantification of the dependence of galaxy properties on merger history. We compare our results to the current state of the art in the field and show significant improvements for all target properties. Mangrove is publicly available: https://github.com/astrockragh/Mangrove.
Holography as a resource for non-local quantum computation

If two parties share sufficient entanglement, they are able to implement any channel on a shared bipartite state via non-local quantum computation – a protocol consisting of local operations and a single simultaneous round of quantum communication. Such a protocol can occur in the AdS/CFT correspondence, with the two parties represented by regions of the CFT, and the holographic state serving as a resource to provide the necessary entanglement. This boundary non-local computation is dual to the local implementation of a channel in the bulk AdS theory. Previous work on this phenomenon was obstructed by the divergent entanglement between adjacent CFT regions, and tried to circumvent this issue by assuming that certain regions are irrelevant. However, the absence of these regions introduces violent phenomena that prevent the CFT from implementing the protocol. Instead, we resolve the issue of divergent entanglement by using a finite-memory quantum simulation of the CFT. We show that any finite-memory quantum system on a circular lattice yields a protocol for non-local quantum computation. In the case of a quantum simulation of a holographic CFT, we carefully show that this protocol implements the channel performed by the local bulk dynamics. Under plausible physical assumptions about quantum computation in the bulk, our results imply that non-local quantum computation can be performed for any polynomially complex unitary with a polynomial amount of entanglement. Finally, we provide a concrete example of a holographic code whose bulk dynamics correspond to a Clifford gate, and use our results to show that this corresponds to a non-local quantum computation protocol for this gate. ar X iv :2 21 0. 13 50 0v 1 [ qu an tph ] 2 4 O ct 2 02 2
A Task Allocation Framework for Human Multi-Robot Collaborative Settings

The requirements of modern production systems together with more advanced robotic technologies have fostered the integration of teams comprising humans and autonomous robots. While this integration has the potential to provide various benefits, it also raises questions about how to effectively manage these teams, taking into account the different characteristics of the agents involved. This paper presents a framework for task allocation in a human multi-robot collaborative scenario. The proposed solution combines an optimal offline allocation with an online reallocation strategy which accounts for inaccuracies of the offline plan and/or unforeseen events, human subjective preferences and cost of task switching. Experiments with two manipulators cooperating with a human operator in a box filling task are presented.
Context-Situated Pun Generation

Previous work on pun generation commonly begins with a given pun word (a pair of homophones for heterographic pun generation and a polyseme for homographic pun generation) and seeks to generate an appropriate pun. While this may enable efficient pun generation, we believe that a pun is most entertaining if it fits appropriately within a given context, e.g., a given situation or dialogue. In this work, we propose a new task, context-situated pun generation, where a specific context represented by a set of keywords is provided, and the task is to first identify suitable pun words that are appropriate for the context, then generate puns based on the context keywords and the identified pun words. We collect a new dataset, CUP (Context-sitUated Pun), containing 4.5k tuples of context words and pun pairs. Based on the new data and setup, we propose a pipeline system for context-situated pun generation, including a pun word retrieval module that identifies suitable pun words for a given context, and a pun generation module that generates puns from context keywords and pun words. Human evaluation shows that 69% of our top retrieved pun words can be used to generate context-situated puns, and our generation module yields successful puns 31% of the time given a plausible tuple of context words and pun pair, almost tripling the yield of a state-of-the-art pun generation model. With an end-to-end evaluation, our pipeline system with the top-1 retrieved pun pair for a given context can generate successful puns 40% of the time, better than all other modeling variations but 32% lower than the human success rate. This highlights the difficulty of the task, and encourages more research in this direction.
Nanostructural polymorphism in the low-birefringence chiral phase of an achiral bent-shaped dimer

Polymorphism, the phenomenon that a species can exist in many discrete forms, is common in nature, such as hair colors in an animal species, flower colors in a tree species, and blood types in humans, etc. In materials science, it refers to a solid that can exist in multiple forms with different crystalline structures. In the liquid crystals field, however, polymorphism is hard to find because a discontinuous structural variation is basically impossible because of their fluid or partially fluid nature. Herein we show that the B4 and DC phases that for many years have been classified as distinctive phases are connected, in terms of their nano-architectures, based on the study of a single compound, a flexible bent-shaped dimer. The surrounding solvent is the key to assisting the dimeric molecules in morphing and adopting different supramolecular structures at the mesoscale. Furthermore, we accidentally find a novel nanotube-like
Satyendra Nath Bose: Quantum statistics to Bose-Einstein condensation

: Satyendra Nath (S.N.) Bose is one of the great Indian scientists. His remarkable work on the black body radiation or derivati on of Planck’s law led to quantum statistics, in particular, the statistics of photon. Albert Einstein applied Bose’s idea to a g as made of atoms and predicted a new state of matter now called Bose-Einstein condensate. It took 70 years to observe the predicted condensation phenomenon in the laboratory. With a brief introduction to the formative period of Professor Bose, this research survey begins with the founding works on quantum statistics and, subsequently, provides a brief account of the series of events terminating in the experimental realization of Bose-Einstein condensation. We also provide two simple examples to visualize the role of synthetic spin-orbit coupling in a quasi-one-dimensional condensate with attractive atom-atom interaction.
Boosting the Cycle Counting Power of Graph Neural Networks with I2-GNNs

Message Passing Neural Networks (MPNNs) are a widely used class of Graph Neural Networks (GNNs). The limited representational power of MPNNs inspires the study of provably powerful GNN architectures. However, knowing one model is more powerful than another gives little insight about what functions they can or cannot express. It is still unclear whether these models are able to approximate specific functions such as counting certain graph substructures, which is essential for applications in biology, chemistry and social network analysis. Motivated by this, we propose to study the counting power of Subgraph MPNNs, a recent and popular class of powerful GNN models that extract rooted subgraphs for each node, assign the root node a unique identifier and encode the root node's representation within its rooted subgraph. Specifically, we prove that Subgraph MPNNs fail to count more-than-4-cycles at node level, implying that node representations cannot correctly encode the surrounding substructures like ring systems with more than four atoms. To overcome this limitation, we propose I$^2$-GNNs to extend Subgraph MPNNs by assigning different identifiers for the root node and its neighbors in each subgraph. I$^2$-GNNs' discriminative power is shown to be strictly stronger than Subgraph MPNNs and partially stronger than the 3-WL test. More importantly, I$^2$-GNNs are proven capable of counting all 3, 4, 5 and 6-cycles, covering common substructures like benzene rings in organic chemistry, while still keeping linear complexity. To the best of our knowledge, it is the first linear-time GNN model that can count 6-cycles with theoretical guarantees. We validate its counting power in cycle counting tasks and demonstrate its competitive performance in molecular prediction benchmarks.
Evaluating Parameter Efficient Learning for Generation

Parameter efficient learning methods (PERMs)have recently gained significant attention asthey provide an efficient way for pre-trainedlanguage models (PLMs) to adapt to a downstream task. However, these conclusions aremostly drawn from in-domain evaluations overthe full training set. In this paper, we presentcomparisons between PERMs and finetuningfrom three new perspectives: (1) the effect ofsample and model size to in-domain evaluations, (2) generalization to unseen domains andnew datasets, and (3) the faithfulness of generations. Our results show that for in-domainsettings (a) there is a cross point of samplesize for which PERMs will perform better thanfinetuning when training with fewer samples,and (b) larger PLMs have larger cross points.For cross-domain and cross-dataset cases, weshow that (a) Adapter (Houlsby et al., 2019)performs the best amongst all the PERMs studied here, and (b) it outperforms finetuning ifthe task dataset is below a certain size. Wealso compare the faithfulness of generationsand show that PERMs can achieve better faithfulness score than finetuning, especially forsmall training set, by as much as 6%. Finally,we apply Adapter to MT-NLG 530b (Smithet al., 2022) and achieve new state-of-the-artresults on Xsum (Narayan et al., 2018) for allROUGE scores (ROUGE-1 49.17, ROUGE-227.20, ROUGE-L 40.98).
Temporally Disentangled Representation Learning

Recently in the field of unsupervised representation learning, strong identifiability results for disentanglement of causally-related latent variables have been established by exploiting certain side information, such as class labels, in addition to independence. However, most existing work is constrained by functional form assumptions such as independent sources or further with linear transitions, and distribution assumptions such as stationary, exponential family distribution. It is unknown whether the underlying latent variables and their causal relations are identifiable if they have arbitrary, nonparametric causal influences in between. In this work, we establish the identifiability theories of nonparametric latent causal processes from their nonlinear mixtures under fixed temporal causal influences and analyze how distribution changes can further benefit the disentanglement. We propose \textbf{\texttt{TDRL}}, a principled framework to recover time-delayed latent causal variables and identify their relations from measured sequential data under stationary environments and under different distribution shifts. Specifically, the framework can factorize unknown distribution shifts into transition distribution changes under fixed and time-varying latent causal relations, and under observation changes in observation. Through experiments, we show that time-delayed latent causal influences are reliably identified and that our approach considerably outperforms existing baselines that do not correctly exploit this modular representation of changes. Our code is available at: \url{https://github.com/weirayao/tdrl}.
Random unitaries, Robustness, and Complexity of Entanglement

It is widely accepted that the dynamic of entanglement in presence of a generic circuit can be predicted by the knowledge of the statistical properties of the entanglement spectrum. We tested this assumption by applying a Metropolis-like entanglement cooling algorithm generated by different sets of local gates, on states sharing the same statistic. We employ the ground states of a unique model, namely the one-dimensional Ising chain with a transverse field, but belonging to different macroscopic phases such as the paramagnetic, the magnetically ordered, and the topological frustrated ones. Quite surprisingly, we observe that the entanglement dynamics are strongly dependent not just on the different sets of gates but also on the phase, indicating that different phases can possess different types of entanglement (which we characterize as purely local, GHZ-like, and W-state-like) with different degree of resilience against the cooling process. Our work highlights the fact that the knowledge of the entanglement spectrum alone is not sufficient to determine its dynamics, thereby demonstrating its incompleteness as a characterization tool. Moreover, it shows a subtle interplay between locality and non-local constraints.