Miracle-dz/UniFAR-FTUs huggingface dataset:
UFO: Uncertainty-aware LiDAR-image Fusion for Off-road Semantic Terrain Map Estimation

Autonomous off-road navigation requires an accurate semantic understanding of the environment, often converted into a bird's-eye view (BEV) representation for various downstream tasks. While learning-based methods have shown success in generating local semantic terrain maps directly from sensor data, their efficacy in off-road environments is hindered by challenges in accurately representing uncertain terrain features. This paper presents a learning-based fusion method for generating dense terrain classification maps in BEV. By performing LiDAR-image fusion at multiple scales, our approach enhances the accuracy of semantic maps generated from an RGB image and a single-sweep LiDAR scan. Utilizing uncertainty-aware pseudo-labels further enhances the network's ability to learn reliably in off-road environments without requiring precise 3D annotations. By conducting thorough experiments using off-road driving datasets, we demonstrate that our method can improve accuracy in off-road terrains, validating its efficacy in facilitating reliable and safe autonomous navigation in challenging off-road settings.
Error bounds for particle gradient descent, and extensions of the log-Sobolev and Talagrand inequalities

We prove non-asymptotic error bounds for particle gradient descent (PGD)~(Kuntz et al., 2023), a recently introduced algorithm for maximum likelihood estimation of large latent variable models obtained by discretizing a gradient flow of the free energy. We begin by showing that, for models satisfying a condition generalizing both the log-Sobolev and the Polyak--{\L}ojasiewicz inequalities (LSI and P{\L}I, respectively), the flow converges exponentially fast to the set of minimizers of the free energy. We achieve this by extending a result well-known in the optimal transport literature (that the LSI implies the Talagrand inequality) and its counterpart in the optimization literature (that the P{\L}I implies the so-called quadratic growth condition), and applying it to our new setting. We also generalize the Bakry--\'Emery Theorem and show that the LSI/P{\L}I generalization holds for models with strongly concave log-likelihoods. For such models, we further control PGD's discretization error, obtaining non-asymptotic error bounds. While we are motivated by the study of PGD, we believe that the inequalities and results we extend may be of independent interest.
Fast Ergodic Search with Kernel Functions

Ergodic search enables optimal exploration of an information distribution while guaranteeing the asymptotic coverage of the search space. However, current methods typically have exponential computation complexity in the search space dimension and are restricted to Euclidean space. We introduce a computationally efficient ergodic search method. Our contributions are two-fold. First, we develop a kernel-based ergodic metric and generalize it from Euclidean space to Lie groups. We formally prove the proposed metric is consistent with the standard ergodic metric while guaranteeing linear complexity in the search space dimension. Secondly, we derive the first-order optimality condition of the kernel ergodic metric for nonlinear systems, which enables efficient trajectory optimization. Comprehensive numerical benchmarks show that the proposed method is at least two orders of magnitude faster than the state-of-the-art algorithm. Finally, we demonstrate the proposed algorithm with a peg-in-hole insertion task. We formulate the problem as a coverage task in the space of SE(3) and use a 30-second-long human demonstration as the prior distribution for ergodic coverage. Ergodicity guarantees the asymptotic solution of the peg-in-hole problem so long as the solution resides within the prior information distribution, which is seen in the 100\% success rate.
Strain-induced valley transport in CrBr$_3$/WSe$_2$/CrBr$_3$ van der Waals heterostructure

Two dimensional magnetic materials are at the forefront of the next generation of spintronic devices. The possibility to interface them with other van der Waals materials such as transition metal dichalcogenides has opened new possibilities for the observation of new and exiting physical phenomena. Here, we present a proof-of-concept valleytronic device based on CrBr$_3$-encapsulated WSe$_2$ showing an unprecedented valley splitting of $\sim 100$ meV under compressive strain of the WSe$_2$, able to be tuned by the relative magnetization of the encapsulating layers. Multiscale transport simulations performed on this device show a spin-valley current with a polarization higher than 80$\%$ than is maintained in a range of $\sim$ 0.3 V gate voltage in a field-effect transistor configuration. The impact of the stacking configuration on the valley splitting is also evaluated.
Logic Rules as Explanations for Legal Case Retrieval

In this paper, we address the issue of using logic rules to explain the results from legal case retrieval. The task is critical to legal case retrieval because the users (e.g., lawyers or judges) are highly specialized and require the system to provide logical, faithful, and interpretable explanations before making legal decisions. Recently, research efforts have been made to learn explainable legal case retrieval models. However, these methods usually select rationales (key sentences) from the legal cases as explanations, failing to provide faithful and logically correct explanations. In this paper, we propose Neural-Symbolic enhanced Legal Case Retrieval (NS-LCR), a framework that explicitly conducts reasoning on the matching of legal cases through learning case-level and law-level logic rules. The learned rules are then integrated into the retrieval process in a neuro-symbolic manner. Benefiting from the logic and interpretable nature of the logic rules, NS-LCR is equipped with built-in faithful explainability. We also show that NS-LCR is a model-agnostic framework that can be plugged in for multiple legal retrieval models. To showcase NS-LCR's superiority, we enhance existing benchmarks by adding manually annotated logic rules and introducing a novel explainability metric using Large Language Models (LLMs). Our comprehensive experiments reveal NS-LCR's effectiveness for ranking, alongside its proficiency in delivering reliable explanations for legal case retrieval.
A Survey on Evaluation of Out-of-Distribution Generalization

Machine learning models, while progressively advanced, rely heavily on the IID assumption, which is often unfulfilled in practice due to inevitable distribution shifts. This renders them susceptible and untrustworthy for deployment in risk-sensitive applications. Such a significant problem has consequently spawned various branches of works dedicated to developing algorithms capable of Out-of-Distribution (OOD) generalization. Despite these efforts, much less attention has been paid to the evaluation of OOD generalization, which is also a complex and fundamental problem. Its goal is not only to assess whether a model's OOD generalization capability is strong or not, but also to evaluate where a model generalizes well or poorly. This entails characterizing the types of distribution shifts that a model can effectively address, and identifying the safe and risky input regions given a model. This paper serves as the first effort to conduct a comprehensive review of OOD evaluation. We categorize existing research into three paradigms: OOD performance testing, OOD performance prediction, and OOD intrinsic property characterization, according to the availability of test data. Additionally, we briefly discuss OOD evaluation in the context of pretrained models. In closing, we propose several promising directions for future research in OOD evaluation.
Controllable Prompt Tuning For Balancing Group Distributional Robustness

Models trained on data composed of different groups or domains can suffer from severe performance degradation under distribution shifts. While recent methods have largely focused on optimizing the worst-group objective, this often comes at the expense of good performance on other groups. To address this problem, we introduce an optimization scheme to achieve good performance across groups and find a good solution for all without severely sacrificing performance on any of them. However, directly applying such optimization involves updating the parameters of the entire network, making it both computationally expensive and challenging. Thus, we introduce Controllable Prompt Tuning (CPT), which couples our approach with prompt-tuning techniques. On spurious correlation benchmarks, our procedures achieve state-of-the-art results across both transformer and non-transformer architectures, as well as unimodal and multimodal data, while requiring only 0.4% tunable parameters.
Effect of Feathers on Drag in Plunge Diving Birds

This study explores the impact of feathers on the hydrodynamic drag experienced by diving birds, which is critical to their foraging efficiency and survival. Employing a novel experimental approach, we analyzed the kinematics of both feathered and non-feathered projectiles during their transition from air to water using high-speed imaging and an onboard accelerometer. The drag coefficients were determined through two methods: a direct calculation from the acceleration data and a theoretical approach fitted to the observed velocity profiles. Our results indicate that feathers significantly increase the drag force during water entry, with feathered projectiles exhibiting approximately double the drag coefficient of their smooth counterparts. These findings provide new insights into the role of avian feather morphology in diving mechanics and have potential implications for the design of bio-inspired aquatic vehicles in engineering. The study also discusses the biological implications of increased drag due to feathers and suggests that factors such as body shape might play a more critical role in the diving capabilities of birds than previously understood.
Modeling Multimodal Social Interactions: New Challenges and Baselines with Densely Aligned Representations

Understanding social interactions involving both verbal and non-verbal cues is essential to effectively interpret social situations. However, most prior works on multimodal social cues focus predominantly on single-person behaviors or rely on holistic visual representations that are not densely aligned to utterances in multi-party environments. They are limited in modeling the intricate dynamics of multi-party interactions. In this paper, we introduce three new challenging tasks to model the fine-grained dynamics between multiple people: speaking target identification, pronoun coreference resolution, and mentioned player prediction. We contribute extensive data annotations to curate these new challenges in social deduction game settings. Furthermore, we propose a novel multimodal baseline that leverages densely aligned language-visual representations by synchronizing visual features with their corresponding utterances. This facilitates concurrently capturing verbal and non-verbal cues pertinent to social reasoning. Experiments demonstrate the effectiveness of the proposed approach with densely aligned multimodal representations in modeling social interactions. We will release our benchmarks and source code to facilitate further research.
FENICE: Factuality Evaluation of summarization based on Natural language Inference and Claim Extraction

Recent advancements in text summarization, particularly with the advent of Large Language Models (LLMs), have shown remarkable performance. However, a notable challenge persists as a substantial number of automatically-generated summaries exhibit factual inconsistencies, such as hallucinations. In response to this issue, various approaches for the evaluation of consistency for summarization have emerged. Yet, these newly-introduced metrics face several limitations, including lack of interpretability, focus on short document summaries (e.g., news articles), and computational impracticality, especially for LLM-based metrics. To address these shortcomings, we propose Factuality Evaluation of summarization based on Natural language Inference and Claim Extraction (FENICE), a more interpretable and efficient factuality-oriented metric. FENICE leverages an NLI-based alignment between information in the source document and a set of atomic facts, referred to as claims, extracted from the summary. Our metric sets a new state of the art on AGGREFACT, the de-facto benchmark for factuality evaluation. Moreover, we extend our evaluation to a more challenging setting by conducting a human annotation process of long-form summarization.
Galled Tree-Child Networks

We propose the class of galled tree-child networks which is obtained as intersection of the classes of galled networks and tree-child networks. For the latter two classes, (asymptotic) counting results and stochastic results have been proved with very different methods. We show that a counting result for the class of galled tree-child networks follows with similar tools as used for galled networks, however, the result has a similar pattern as the one for tree-child networks. In addition, we also consider the (suitably scaled) numbers of reticulation nodes of random galled tree-child networks and show that they are asymptotically normal distributed. This is in contrast to the limit laws of the corresponding quantities for galled networks and tree-child networks which have been both shown to be discrete.
Preventing Reward Hacking with Occupancy Measure Regularization

Reward hacking occurs when an agent performs very well with respect to a"proxy"reward function (which may be hand-specified or learned), but poorly with respect to the unknown true reward. Since ensuring good alignment between the proxy and true reward is extremely difficult, one approach to prevent reward hacking is optimizing the proxy conservatively. Prior work has particularly focused on enforcing the learned policy to behave similarly to a"safe"policy by penalizing the KL divergence between their action distributions (AD). However, AD regularization doesn't always work well since a small change in action distribution at a single state can lead to potentially calamitous outcomes, while large changes might not be indicative of any dangerous activity. Our insight is that when reward hacking, the agent visits drastically different states from those reached by the safe policy, causing large deviations in state occupancy measure (OM). Thus, we propose regularizing based on the OM divergence between policies instead of AD divergence to prevent reward hacking. We theoretically establish that OM regularization can more effectively avoid large drops in true reward. Then, we empirically demonstrate in a variety of realistic environments that OM divergence is superior to AD divergence for preventing reward hacking by regularizing towards a safe policy. Furthermore, we show that occupancy measure divergence can also regularize learned policies away from reward hacking behavior. Our code and data are available at https://github.com/cassidylaidlaw/orpo
Fantastic Semantics and Where to Find Them: Investigating Which Layers of Generative LLMs Reflect Lexical Semantics

Large language models have achieved remarkable success in general language understanding tasks. However, as a family of generative methods with the objective of next token prediction, the semantic evolution with the depth of these models are not fully explored, unlike their predecessors, such as BERT-like architectures. In this paper, we specifically investigate the bottom-up evolution of lexical semantics for a popular LLM, namely Llama2, by probing its hidden states at the end of each layer using a contextualized word identification task. Our experiments show that the representations in lower layers encode lexical semantics, while the higher layers, with weaker semantic induction, are responsible for prediction. This is in contrast to models with discriminative objectives, such as mask language modeling, where the higher layers obtain better lexical semantics. The conclusion is further supported by the monotonic increase in performance via the hidden states for the last meaningless symbols, such as punctuation, in the prompting strategy.
Statistical Query Lower Bounds for Learning Truncated Gaussians

We study the problem of estimating the mean of an identity covariance Gaussian in the truncated setting, in the regime when the truncation set comes from a low-complexity family $\mathcal{C}$ of sets. Specifically, for a fixed but unknown truncation set $S \subseteq \mathbb{R}^d$, we are given access to samples from the distribution $\mathcal{N}(\boldsymbol{ \mu}, \mathbf{ I})$ truncated to the set $S$. The goal is to estimate $\boldsymbol\mu$ within accuracy $\epsilon>0$ in $\ell_2$-norm. Our main result is a Statistical Query (SQ) lower bound suggesting a super-polynomial information-computation gap for this task. In more detail, we show that the complexity of any SQ algorithm for this problem is $d^{\mathrm{poly}(1/\epsilon)}$, even when the class $\mathcal{C}$ is simple so that $\mathrm{poly}(d/\epsilon)$ samples information-theoretically suffice. Concretely, our SQ lower bound applies when $\mathcal{C}$ is a union of a bounded number of rectangles whose VC dimension and Gaussian surface are small. As a corollary of our construction, it also follows that the complexity of the previously known algorithm for this task is qualitatively best possible.
Boundary Behavior of Compact Manifolds With Scalar Curvature Lower Bounds and Static Quasi-Local Mass of Tori

A classic result of Shi and Tam states that a 2-sphere of positive Gauss and mean curvature bounding a compact 3-manifold with nonnegative scalar curvature, must have total mean curvature not greater than that of the isometric embedding into Euclidean 3-space, with equality only for domains in this reference manifold. We generalize this result to 2-tori of Guass curvature greater than $-1$, which bound a compact 3-manifold having scalar curvature not less than $-6$ and at least one other boundary component satisfying a 'trapping condition'. The conclusion is that the total weighted mean curvature is not greater than that of an isometric embedding into the Kottler manifold, with equality only for domains in this space. Examples are given to show that the assumption of a secondary boundary component cannot be removed. The result gives a positive mass theorem for the static Brown-York mass of tori, in analogy to the Shi-Tam positivity of the standard Brown-York mass, and represents the first such quasi-local mass positivity result for non-spherical surfaces. Furthermore, we prove a Penrose-type inequality in this setting.
HeAR -- Health Acoustic Representations

Health acoustic sounds such as coughs and breaths are known to contain useful health signals with significant potential for monitoring health and disease, yet are underexplored in the medical machine learning community. The existing deep learning systems for health acoustics are often narrowly trained and evaluated on a single task, which is limited by data and may hinder generalization to other tasks. To mitigate these gaps, we develop HeAR, a scalable self-supervised learning-based deep learning system using masked autoencoders trained on a large dataset of 313 million two-second long audio clips. Through linear probes, we establish HeAR as a state-of-the-art health audio embedding model on a benchmark of 33 health acoustic tasks across 6 datasets. By introducing this work, we hope to enable and accelerate further health acoustics research.
A Simple and Accurate Method for Computing Optimized Effective Potentials for Exact Exchange Energy

The inverse Kohn-Sham density-functional theory (inv-KS) for the electron density of the Hartree-Fock (HF) wave function was revisited within the context of the optimized effective potential (HF- OEP). First, it is proved that the exchange potential created by the inv-KS is equivalent to the potential obtained by the HF-OEP when the HF-OEP realizes the HF energy of the system under consideration. Next the real-space grid (RSG) implementations of the inv-KS and the HF-OEP are addressed. The total HF energies EHF for the wave functions on the effective potentials optimized by the inv-KS are computed for a set of small molecules. It is found that the mean absolute deviation (MAD) of EHF from the HF energy is clearly smaller than the MAD of EHF, demonstrating that the inv-KS is advantageous in constructing the detailed structure of the exchange potential vx as compared with the HF-OEP. The inv-KS method is also applied to an ortho-benzyne radical known as a strongly correlated polyatomic molecule. It is revealed that the spin populations on the atomic sites computed by the UHF calculation can be faithfully reproduced by the wave functions on the inv-KS potential.
Language Guided Exploration for RL Agents in Text Environments

Real-world sequential decision making is characterized by sparse rewards and large decision spaces, posing significant difficulty for experiential learning systems like $\textit{tabula rasa}$ reinforcement learning (RL) agents. Large Language Models (LLMs), with a wealth of world knowledge, can help RL agents learn quickly and adapt to distribution shifts. In this work, we introduce Language Guided Exploration (LGE) framework, which uses a pre-trained language model (called GUIDE ) to provide decision-level guidance to an RL agent (called EXPLORER). We observe that on ScienceWorld (Wang et al.,2022), a challenging text environment, LGE outperforms vanilla RL agents significantly and also outperforms other sophisticated methods like Behaviour Cloning and Text Decision Transformer.
Status of Birkhoff's theorem in polymerized semiclassical regime of Loop Quantum Gravity

The collapse of a spherically symmetric ball of dust has been intensively studied in Loop Quantum Gravity (LQG). From a quantum theory, it is possible to recover a semiclassical regime through a polymerization procedure. In this setting, general solutions to the polymerized Einstein field equations (PEFE) will be discussed both for the interior and the exterior of the dust cloud. Exterior solutions are particularly interesting since they may lead to a semiclassical version of the Birkhoff's theorem. It is seen that if time independence of the vacuum is imposed, there exists a unique class of solutions depending on two parameters. Nevertheless, the possibility of more intricate time dependent solutions is not ruled out completely. Ultimately, these results will be compared to a model of spherical collapse obtained independently from the Einstein equations.