Miracle-dz/UniFAR-FTUs huggingface dataset:
Multimodal and Force-Matched Imitation Learning with a See-Through Visuotactile Sensor

Kinesthetic Teaching is a popular approach to collecting expert robotic demonstrations of contact-rich tasks for imitation learning (IL), but it typically only measures motion, ignoring the force placed on the environment by the robot. Furthermore, contact-rich tasks require accurate sensing of both reaching and touching, which can be difficult to provide with conventional sensing modalities. We address these challenges with a See-Through-your-Skin (STS) visuotactile sensor, using the sensor both (i) as a measurement tool to improve kinesthetic teaching, and (ii) as a policy input in contact-rich door manipulation tasks. An STS sensor can be switched between visual and tactile modes by leveraging a semi-transparent surface and controllable lighting, allowing for both pre-contact visual sensing and during-contact tactile sensing with a single sensor. First, we propose tactile force matching, a methodology that enables a robot to match forces read during kinesthetic teaching using tactile signals. Second, we develop a policy that controls STS mode switching, allowing a policy to learn the appropriate moment to switch an STS from its visual to its tactile mode. Finally, we study multiple observation configurations to compare and contrast the value of visual and tactile data from an STS with visual data from a wrist-mounted eye-in-hand camera. With over 3,000 test episodes from real-world manipulation experiments, we find that the inclusion of force matching raises average policy success rates by 62.5%, STS mode switching by 30.3%, and STS data as a policy input by 42.5%. Our results highlight the utility of see-through tactile sensing for IL, both for data collection to allow force matching, and for policy execution to allow accurate task feedback.
Non-Stokesian dynamics of magnetic helical nanoswimmers under confinement

Electromagnetically propelled helical nanoswimmers offer great potential for nanorobotic applications. Here, the effect of confinement on their propulsion is characterized using lattice-Boltzmann simulations. Two principal mechanisms give rise to their forward motion under confinement: 1) pure swimming, and 2) the thrust created by the differential pressure due to confinement. Under strong confinement, they face greater rotational drag, but display a faster propulsion for fixed driving frequency in agreement with experimental findings. This is due to the increased differential pressure created by the boundary walls when they are sufficiently close to each other and the particle. Two new analytical relations are presented: 1) for predicting the swimming speed of an unconfined particle as a function of its angular speed and geometrical properties, and 2) an empirical expression to accurately predict the propulsion speed of a confined swimmer as a function of the degree of confinement and its unconfined swimming speed. At low driving frequencies and degrees of confinement, the systems retain the expected linear behavior consistent with the predictions of the Stokes equation. However, as the driving frequency and/or the degree of confinement increase, their impact on propulsion leads to increasing deviations from the Stokesian regime and emergence of nonlinear behavior.
IndoToD: A Multi-Domain Indonesian Benchmark For End-to-End Task-Oriented Dialogue Systems

Task-oriented dialogue (ToD) systems have been mostly created for high-resource languages, such as English and Chinese. However, there is a need to develop ToD systems for other regional or local languages to broaden their ability to comprehend the dialogue contexts in various languages. This paper introduces IndoToD, an end-to-end multi domain ToD benchmark in Indonesian. We extend two English ToD datasets to Indonesian, comprising four different domains by delexicalization to efficiently reduce the size of annotations. To ensure a high-quality data collection, we hire native speakers to manually translate the dialogues. Along with the original English datasets, these new Indonesian datasets serve as an effective benchmark for evaluating Indonesian and English ToD systems as well as exploring the potential benefits of cross-lingual and bilingual transfer learning approaches.
RoboVQA: Multimodal Long-Horizon Reasoning for Robotics

We present a scalable, bottom-up and intrinsically diverse data collection scheme that can be used for high-level reasoning with long and medium horizons and that has 2.2x higher throughput compared to traditional narrow top-down step-by-step collection. We collect realistic data by performing any user requests within the entirety of 3 office buildings and using multiple robot and human embodiments. With this data, we show that models trained on all embodiments perform better than ones trained on the robot data only, even when evaluated solely on robot episodes. We find that for a fixed collection budget it is beneficial to take advantage of cheaper human collection along with robot collection. We release a large and highly diverse (29,520 unique instructions) dataset dubbed RoboVQA containing 829,502 (video, text) pairs for robotics-focused visual question answering. We also demonstrate how evaluating real robot experiments with an intervention mechanism enables performing tasks to completion, making it deployable with human oversight even if imperfect while also providing a single performance metric. We demonstrate a single video-conditioned model named RoboVQA-VideoCoCa trained on our dataset that is capable of performing a variety of grounded high-level reasoning tasks in broad realistic settings with a cognitive intervention rate 46% lower than the zero-shot state of the art visual language model (VLM) baseline and is able to guide real robots through long-horizon tasks. The performance gap with zero-shot state-of-the-art models indicates that a lot of grounded data remains to be collected for real-world deployment, emphasizing the critical need for scalable data collection approaches. Finally, we show that video VLMs significantly outperform single-image VLMs with an average error rate reduction of 19% across all VQA tasks. Data and videos available at https://robovqa.github. io
Ultra-Fast Generation of Air Shower Images for Imaging Air Cherenkov Telescopes using Generative Adversarial Networks

For the analysis of data taken by Imaging Air Cherenkov Telescopes (IACTs), a large number of air shower simulations are needed to derive the instrument response. The simulations are very complex, involving computational and memory-intensive calculations, and are usually performed repeatedly for different observation intervals to take into account the varying optical sensitivity of the instrument. The use of generative models based on deep neural networks offers the prospect for memory-efficient storing of huge simulation libraries and cost-effective generation of a large number of simulations in an extremely short time. In this work, we use Wasserstein Generative Adversarial Networks to generate photon showers for an IACT equipped with the FlashCam design, which has more than $1{,}500$ pixels. Using simulations of the H.E. S.S. experiment, we demonstrate the successful generation of high-quality IACT images. The analysis includes a comprehensive study of the generated image quality based on low-level observables and the well-known Hillas parameters that describe the shower shape. We demonstrate for the first time that the generated images have high fidelity with respect to low-level observables, the Hillas parameters, their physical properties, as well as their correlations. The found increase in generation speed in the order of $10^5$ yields promising prospects for fast and memory-efficient simulations of air showers for IACTs.
Offline Imitation from Observation via Primal Wasserstein State Occupancy Matching

In real-world scenarios, arbitrary interactions with the environment can often be costly, and actions of expert demonstrations are not always available. To reduce the need for both, Offline Learning from Observations (LfO) is extensively studied, where the agent learns to solve a task with only expert states and \textit{task-agnostic} non-expert state-action pairs. The state-of-the-art DIstribution Correction Estimation (DICE) methods minimize the state occupancy divergence between the learner and expert policies. However, they are limited to either $f$-divergences (KL and $\chi^2$) or Wasserstein distance with Rubinstein duality, the latter of which constrains the underlying distance metric crucial to the performance of Wasserstein-based solutions. To address this problem, we propose Primal Wasserstein DICE (PW-DICE), which minimizes the primal Wasserstein distance between the expert and learner state occupancies with a pessimistic regularizer and leverages a contrastively learned distance as the underlying metric for the Wasserstein distance. Theoretically, we prove that our framework is a generalization of the state-of-the-art, SMODICE, and unifies $f$-divergence and Wasserstein minimization. Empirically, we find that PW-DICE improves upon several state-of-the-art methods on multiple testbeds.
The Megalithic Landscape of Central Sulawesi, Indonesia: Combnng Archaeologcal and Palynologcal Investgatons

Central Sulawesi is characterized as both a hot spot of biodiversity as well as a megalithic landscape with a clustered distribution of megalithic sites. One of the main goals of this preliminary evaluation is to untangle the development of societies with monumental constructions in relation to environmental change and diversity. Initial results presented in this paper highlight the research potential of a combined investigation of ecological data and archaeological surveys. The conclusions are based both on pollen data as well as data regarding the spatial distribution and architecture of megalithic " kalambas" (jars), combined with the interpretation of the few available excavation reports. The palynological investigations concentrate on a comparison of on-site and off-site cores, to identify both the general vegetation dynamics as well as local anthropogenic influences that correspond with the environment. The archaeological considerations are based on spatial analyses and typological comparison of the architectural features of 15 sites (including 93 megaliths) in the Besoa valley and particularly at the Pokekea site. As a result of this work, we assume the existence of one central prehistoric domestic site with ritual activities and an associated spread of smaller hamlets, reflected in the distribution of megaliths for each basin. However, the chronological framework of the development is still uncertain. Anyhow, we offer a terminus ante quem for the erection of kalambas of around AD 830. Hypothetically, an initial opening of the landscape around 50 BC, as indicated by pollen analyses, may be linked to a possible early construction phase of the monuments. However, the combination of both environmental and preliminary archaeological analyses results in the first interpretations of ecological and monumental developments in the central Sulawesian region.
A New Approach to the CO2 Airborne Fraction: Enhancing Statistical Precision and Tackling Zero Emissions

The global fraction of anthropogenically emitted CO2 that stays in the atmosphere, the CO2 airborne fraction, has been fluctuating around a constant value over the recent period of 1959-2021. The consensus estimate of the airborne fraction is around $44\%$, indicating that approximately $44\%$ of emitted CO2 stays in the atmosphere and the remaining $56\%$ is absorbed by the oceanic and terrestrial biospheres. In this study, we show that the conventional estimator of the airborne fraction suffers from a number of statistical deficiencies, such as non-existence of moments and a non-Gaussian limiting distribution. We formulate a new estimator of the airborne fraction that does not suffer from these deficiencies. We show that the new estimator is superconsistent, has a Gaussian limiting distribution, and is able to reduce estimation uncertainty substantially. Our empirical analysis leads to an estimate of the airborne fraction over 1959-2021 of $47.6\%$ ($\pm 1.1\%$; $1 \sigma$). Hence, our empirical results imply a higher, and better constrained, airborne fraction than the current consensus. Our theoretical results indicate that a conventional analysis of the airborne fraction may be problematic, especially in future low emissions scenarios such as those implied by"net zero"international agreements. Our newly proposed estimator avoids these issues and thus provides a promising approach to future studies of the airborne fraction, especially in low emission scenarios.
LLM4Drive: A Survey of Large Language Models for Autonomous Driving

Autonomous driving technology, a catalyst for revolutionizing transportation and urban mobility, has the tend to transition from rule-based systems to data-driven strategies. Traditional module-based systems are constrained by cumulative errors among cascaded modules and inflexible pre-set rules. In contrast, end-to-end autonomous driving systems have the potential to avoid error accumulation due to their fully data-driven training process, although they often lack transparency due to their"black box"nature, complicating the validation and traceability of decisions. Recently, large language models (LLMs) have demonstrated abilities including understanding context, logical reasoning, and generating answers. A natural thought is to utilize these abilities to empower autonomous driving. By combining LLM with foundation vision models, it could open the door to open-world understanding, reasoning, and few-shot learning, which current autonomous driving systems are lacking. In this paper, we systematically review a research line about \textit{Large Language Models for Autonomous Driving (LLM4AD)}. This study evaluates the current state of technological advancements, distinctly outlining the principal challenges and prospective directions for the field. For the convenience of researchers in academia and industry, we provide real-time updates on the latest advances in the field as well as relevant open-source resources via the designated link: https://github.com/Thinklab-SJTU/Awesome-LLM4AD.
Winding number criterion for the origin to belong to the numerical range of a matrix on a loop of matrices

Let $A:[0,1]\to GL(n,\mathbb{C})$ be continuous with $A(0)=A(1)$, thus the winding number of $\det A$ is well-defined. If the winding number is not divisible by $n$, then the origin belongs to the numerical range of $A(\phi)$ for some $\phi \in [0,1]$.
Weighted weak-type inequalities for maximal operators and singular integrals

We prove quantitative, one-weight, weak-type estimates for maximal operators, singular integrals, fractional maximal operators and fractional integral operators. We consider a kind of weak-type inequality that was first studied by Muckenhoupt and Wheeden and later by Cruz-Uribe, Martell and Perez. We obtain quantitative estimates for these operators in both the scalar and matrix weighted setting using sparse domination techniques. Our results extend those obtained by Cruz-Uribe, Isralowitz, Moen, Pott, and Rivera-R\'ios for singular integrals and maximal operators when $p=1$.
Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusion

Learning world models can teach an agent how the world works in an unsupervised manner. Even though it can be viewed as a special case of sequence modeling, progress for scaling world models on robotic applications such as autonomous driving has been somewhat less rapid than scaling language models with Generative Pre-trained Transformers (GPT). We identify two reasons as major bottlenecks: dealing with complex and unstructured observation space, and having a scalable generative model. Consequently, we propose a novel world modeling approach that first tokenizes sensor observations with VQVAE, then predicts the future via discrete diffusion. To efficiently decode and denoise tokens in parallel, we recast Masked Generative Image Transformer into the discrete diffusion framework with a few simple changes, resulting in notable improvement. When applied to learning world models on point cloud observations, our model reduces prior SOTA Chamfer distance by more than 65% for 1s prediction, and more than 50% for 3s prediction, across NuScenes, KITTI Odometry, and Argoverse2 datasets. Our results demonstrate that discrete diffusion on tokenized agent experience can unlock the power of GPT-like unsupervised learning for robotic agents.
Combating Bilateral Edge Noise for Robust Link Prediction

Although link prediction on graphs has achieved great success with the development of graph neural networks (GNNs), the potential robustness under the edge noise is still less investigated. To close this gap, we first conduct an empirical study to disclose that the edge noise bilaterally perturbs both input topology and target label, yielding severe performance degradation and representation collapse. To address this dilemma, we propose an information-theory-guided principle, Robust Graph Information Bottleneck (RGIB), to extract reliable supervision signals and avoid representation collapse. Different from the basic information bottleneck, RGIB further decouples and balances the mutual dependence among graph topology, target labels, and representation, building new learning objectives for robust representation against the bilateral noise. Two instantiations, RGIB-SSL and RGIB-REP, are explored to leverage the merits of different methodologies, i.e., self-supervised learning and data reparameterization, for implicit and explicit data denoising, respectively. Extensive experiments on six datasets and three GNNs with diverse noisy scenarios verify the effectiveness of our RGIB instantiations. The code is publicly available at: https://github.com/tmlr-group/RGIB.
Sharp Noisy Binary Search with Monotonic Probabilities

We revisit the noisy binary search model of Karp and Kleinberg, in which we have $n$ coins with unknown probabilities $p_i$ that we can flip. The coins are sorted by increasing $p_i$, and we would like to find where the probability crosses (to within $\varepsilon$) of a target value $\tau$. This generalized the fixed-noise model of Burnashev and Zigangirov , in which $p_i = \frac{1}{2} \pm \varepsilon$, to a setting where coins near the target may be indistinguishable from it. Karp and Kleinberg showed that $\Theta(\frac{1}{\varepsilon^2} \log n)$ samples are necessary and sufficient for this task. We produce a practical algorithm by solving two theoretical challenges: high-probability behavior and sharp constants. We give an algorithm that succeeds with probability $1-\delta$ from \[ \frac{1}{C_{\tau, \varepsilon}} \cdot \left(\lg n + O(\log^{2/3} n \log^{1/3} \frac{1}{\delta} + \log \frac{1}{\delta})\right) \] samples, where $C_{\tau, \varepsilon}$ is the optimal such constant achievable. For $\delta>n^{-o(1)}$ this is within $1 + o(1)$ of optimal, and for $\delta \ll 1$ it is the first bound within constant factors of optimal.
A note on gluing cosilting objects

Based on the recent work of M. Saorin and A. Zvonoreva of gluing (co)silting objects and L.A. Hugel, R. Laking, J. Stovicek and J. Vitora of mutating (co)silting objects, we first study further on gluing pure-injective cosilting objects in algebraic compactly generated triangulated categories and gluing cosilting complexes in the unbounded derived categories of rings. Then we discuss the compatibility of the gluing method and the mutation operation of cosilting objects.
Scalable Counterfactual Distribution Estimation in Multivariate Causal Models

We consider the problem of estimating the counterfactual joint distribution of multiple quantities of interests (e.g., outcomes) in a multivariate causal model extended from the classical difference-in-difference design. Existing methods for this task either ignore the correlation structures among dimensions of the multivariate outcome by considering univariate causal models on each dimension separately and hence produce incorrect counterfactual distributions, or poorly scale even for moderate-size datasets when directly dealing with such multivariate causal model. We propose a method that alleviates both issues simultaneously by leveraging a robust latent one-dimensional subspace of the original high-dimension space and exploiting the efficient estimation from the univariate causal model on such space. Since the construction of the one-dimensional subspace uses information from all the dimensions, our method can capture the correlation structures and produce good estimates of the counterfactual distribution. We demonstrate the advantages of our approach over existing methods on both synthetic and real-world data.
Diffusion Models for Reinforcement Learning: A Survey

Diffusion models surpass previous generative models in sample quality and training stability. Recent works have shown the advantages of diffusion models in improving reinforcement learning (RL) solutions. This survey aims to provide an overview of this emerging field and hopes to inspire new avenues of research. First, we examine several challenges encountered by RL algorithms. Then, we present a taxonomy of existing methods based on the roles of diffusion models in RL and explore how the preceding challenges are addressed. We further outline successful applications of diffusion models in various RL-related tasks. Finally, we conclude the survey and offer insights into future research directions. We are actively maintaining a GitHub repository for papers and other related resources in utilizing diffusion models in RL: https://github.com/apexrl/Diff4RLSurvey.
The future of global water stress: An integrated assessment

We assess the ability of global water systems, resolved at 282 assessment subregions (ASRs), to the meet water requirements under integrated projections of socioeconomic growth and climate change. We employ a water resource system (WRS) component embedded within the Massachusetts Institute of Technology Integrated Global System Model (IGSM) framework in a suite of simulations that consider a range of climate policies and regional hydroclimate changes out to 2050. For many developing nations, water demand increases due to population growth and economic activity have a much stronger effect on water stress than climate change. By 2050, economic growth and population change alone can lead to an additional 1.8 billion people living under at least moderate water stress, with 80% of these located in developing countries. Uncertain regional climate change can play a secondary role to either exacerbate or dampen the increase in water stress. The strongest climate impacts on water stress are observed in Africa, but strong impacts also occur over Europe, Southeast Asia, and North America. The combined effects of socioeconomic growth and uncertain climate change lead to a 1.0–1.3 billion increase of the world's 2050 projected population living with overly exploited water conditions—where total potential water requirements will consistently exceed surface water supply. This would imply that adaptive measures would be taken to meet these surface water shortfalls and include: water‐use efficiency, reduced and/or redirected consumption, recurrent periods of water emergencies or curtailments, groundwater depletion, additional interbasin transfers, and overdraw from flow intended to maintain environmental requirements.
A CLASSIFICATION OF LOCALLY CHERN HOMOGENOUS HERMITIAN MANIFOLDS

We apply the algebraic consideration of holonomy systems to study those Hermitian manifolds whose Chern connection is Ambrose-Singer and prove structure theorems for such manifolds. The main result asserts that the universal cover of such a Hermitian manifold must be the product of a complex Lie group and Hermitian symmetric spaces.