Miracle-dz/UniFAR-FTUs huggingface dataset:
A pose and shear-based tactile robotic system for object tracking, surface following and object pushing

Tactile perception is a crucial sensing modality in robotics, particularly in scenarios that require precise manipulation and safe interaction with other objects. Previous research in this area has focused extensively on tactile perception of contact poses as this is an important capability needed for tasks such as traversing an object's surface or edge, manipulating an object, or pushing an object along a predetermined path. Another important capability needed for tasks such as object tracking and manipulation is estimation of post-contact shear but this has received much less attention. Indeed, post-contact shear has often been considered a"nuisance variable"and is removed if possible because it can have an adverse effect on other types of tactile perception such as contact pose estimation. This paper proposes a tactile robotic system that can simultaneously estimate both the contact pose and post-contact shear, and use this information to control its interaction with other objects. Moreover, our new system is capable of interacting with other objects in a smooth and continuous manner, unlike the stepwise, position-controlled systems we have used in the past. We demonstrate the capabilities of our new system using several different controller configurations, on tasks including object tracking, surface following, single-arm object pushing, and dual-arm object pushing.
AssistGPT: A General Multi-modal Assistant that can Plan, Execute, Inspect, and Learn

Recent research on Large Language Models (LLMs) has led to remarkable advancements in general NLP AI assistants. Some studies have further explored the use of LLMs for planning and invoking models or APIs to address more general multi-modal user queries. Despite this progress, complex visual-based tasks still remain challenging due to the diverse nature of visual tasks. This diversity is reflected in two aspects: 1) Reasoning paths. For many real-life applications, it is hard to accurately decompose a query simply by examining the query itself. Planning based on the specific visual content and the results of each step is usually required. 2) Flexible inputs and intermediate results. Input forms could be flexible for in-the-wild cases, and involves not only a single image or video but a mixture of videos and images, e.g., a user-view image with some reference videos. Besides, a complex reasoning process will also generate diverse multimodal intermediate results, e.g., video narrations, segmented video clips, etc. To address such general cases, we propose a multi-modal AI assistant, AssistGPT, with an interleaved code and language reasoning approach called Plan, Execute, Inspect, and Learn (PEIL) to integrate LLMs with various tools. Specifically, the Planner is capable of using natural language to plan which tool in Executor should do next based on the current reasoning progress. Inspector is an efficient memory manager to assist the Planner to feed proper visual information into a specific tool. Finally, since the entire reasoning process is complex and flexible, a Learner is designed to enable the model to autonomously explore and discover the optimal solution. We conducted experiments on A-OKVQA and NExT-QA benchmarks, achieving state-of-the-art results. Moreover, showcases demonstrate the ability of our system to handle questions far more complex than those found in the benchmarks.
Robust universal relations in neutron star asteroseismology

The non-radial oscillations of the neutron stars (NSs) have been suggested as a useful tool to probe the composition of neutron star matter (NSM). With this scope in mind, we consider a large number of equations of states (EOSs) that are consistent with nuclear matter properties and pure neutron matter EOS based on a chiral effective field theory (chEFT) calculation for the low densities and perturbative QCD EOS at very high densities. This ensemble of EOSs is also consistent with astronomical observations, gravitational waves in GW170817, mass and radius measurements from Neutron star Interior Composition ExploreR (NICER). We analyze the robustness of known universal relations (URs) among the quadrupolar $f$ mode frequencies, masses and radii with such a large number of EOSs and we find a new UR that results from a strong correlation between the $f$ mode frequencies and the radii of NSs. Such a correlation is very useful in accurately determining the radius from a measurement of $f$ mode frequencies in the near future. We also show that the quadrupolar $f$ mode frequencies of NS of masses 2.0 M$_\odot$ and above lie in the range $\sim$ 2-3 kHz in this ensemble of physically realistic EOSs. A NS of mass 2M$_{\odot}$ with a low $f$ mode frequency may indicate the existence of non-nucleonic degrees of freedom.
Combinatorial generation via permutation languages. VI. Binary trees

In this paper we propose a notion of pattern avoidance in binary trees that generalizes the avoidance of contiguous tree patterns studied by Rowland and non-contiguous tree patterns studied by Dairyko, Pudwell, Tyner, and Wynn. Specifically, we propose algorithms for generating different classes of binary trees that are characterized by avoiding one or more of these generalized patterns. This is achieved by applying the recent Hartung-Hoang-M\"utze-Williams generation framework, by encoding binary trees via permutations. In particular, we establish a one-to-one correspondence between tree patterns and certain mesh permutation patterns. We also conduct a systematic investigation of all tree patterns on at most 5 vertices, and we establish bijections between pattern-avoiding binary trees and other combinatorial objects, in particular pattern-avoiding lattice paths and set partitions.
Dynamical detection of a companion driving a spiral arm in a protoplanetary disk

Radio and near-infrared observations have observed dozens of protoplanetary disks that host spiral arm features. Numerical simulations have shown that companions may excite spiral density waves in protoplanetary disks via companion–disk interaction. However, the lack of direct observational evidence for spiral-driving companions poses challenges to current theories of companion–disk interaction. Here we report multi-epoch observations of the binary system HD 100453 with the Spectro-Polarimetric High-contrast Exoplanet REsearch (SPHERE) facility at the Very Large Telescope. By recovering the spiral features via robustly removing starlight contamination, we measure spiral motion across 4 yr to perform dynamical motion analyses. The spiral pattern motion is consistent with the orbital motion of the eccentric companion. With this first observational evidence of a companion driving a spiral arm among protoplanetary disks, we directly and dynamically confirm the long-standing theory on the origin of spiral features in protoplanetary disks. With the pattern motion of companion-driven spirals being independent of companion mass, here we establish a feasible way of searching for hidden spiral-arm-driving planets that are beyond the detection of existing ground-based high-contrast imagers.
Spatially resolved mock observations of stellar kinematics: full radiative transfer treatment of simulated galaxies

We present a framework to build realistic mock spectroscopic observations for state-of-the-art hydrodynamical simulations, using high spectral resolution stellar population models and full radiative transfer treatment with skirt. As a first application, we generate stellar continuum mock observations for the Auriga cosmological zoom simulations emulating integral-field observations from the Sydney–AOO Multi-object Integral Field Spectrograph (SAMI) Galaxy Survey. We perform spectral fitting on our synthetic cubes and compute the resulting rotation velocity (Vrot) and velocity dispersion within 1Re (σe) for a subset of the Auriga sample. We find that the kinematics produced by Auriga are in good agreement with the observations from the SAMI galaxy survey after taking into account the effects of dust and the systematics produced by the observation limitations. We also explore the effects of seeing convolution, inclination, and attenuation on the line-of-sight velocity distribution. For highly inclined galaxies, these effects can lead to an artificial decrease in the measured V/σ by nearly a factor of two (after inclination correction). We also demonstrate the utility of our method for high-redshift galaxies by emulating spatially resolved continuum spectra from the Large Early Galaxy Census (LEGA-C) survey and, looking forward, the European Extremely Large Telescope (E-ELT) High Angular Resolution Monolithic Optical and Near-infrared Integral field spectrograph (HARMONI). Our framework represents a crucial link between the ground truth for stellar populations and kinematics in simulations and the observed stellar continuum observations at low and high redshift.
A Gromov-Wasserstein Geometric View of Spectrum-Preserving Graph Coarsening

Graph coarsening is a technique for solving large-scale graph problems by working on a smaller version of the original graph, and possibly interpolating the results back to the original graph. It has a long history in scientific computing and has recently gained popularity in machine learning, particularly in methods that preserve the graph spectrum. This work studies graph coarsening from a different perspective, developing a theory for preserving graph distances and proposing a method to achieve this. The geometric approach is useful when working with a collection of graphs, such as in graph classification and regression. In this study, we consider a graph as an element on a metric space equipped with the Gromov--Wasserstein (GW) distance, and bound the difference between the distance of two graphs and their coarsened versions. Minimizing this difference can be done using the popular weighted kernel $K$-means method, which improves existing spectrum-preserving methods with the proper choice of the kernel. The study includes a set of experiments to support the theory and method, including approximating the GW distance, preserving the graph spectrum, classifying graphs using spectral information, and performing regression using graph convolutional networks. Code is available at https://github.com/ychen-stat-ml/GW-Graph-Coarsening .
Hybrids of Constraint-based and Noise-based Algorithms for Causal Discovery from Time Series

Constraint-based and noise-based methods have been proposed to discover summary causal graphs from observational time series under strong assumptions which can be violated or impossible to verify in real applications. Recently, a hybrid method (Assaad et al, 2021) that combines these two approaches, proved to be robust to assumption violation. However, this method assumes that the summary causal graph is acyclic, but cycles are common in many applications. For example, in ecological communities, there may be cyclic relationships between predator and prey populations, creating feedback loops. Therefore, this paper presents two new frameworks for hybrids of constraint-based and noise-based methods that can discover summary causal graphs that may or may not contain cycles. For each framework, we provide two hybrid algorithms which are experimentally tested on simulated data, realistic ecological data, and real data from various applications. Experiments show that our hybrid approaches are robust and yield good results over most datasets.
Ball Trajectory Inference from Multi-Agent Sports Contexts Using Set Transformer and Hierarchical Bi-LSTM

As artificial intelligence spreads out to numerous fields, the application of AI to sports analytics is also in the spotlight. However, one of the major challenges is the difficulty of automated acquisition of continuous movement data during sports matches. In particular, it is a conundrum to reliably track a tiny ball on a wide soccer pitch with obstacles such as occlusion and imitations. Tackling the problem, this paper proposes an inference framework of ball trajectory from player trajectories as a cost-efficient alternative to ball tracking. We combine Set Transformers to get permutation-invariant and equivariant representations of the multi-agent contexts with a hierarchical architecture that intermediately predicts the player ball possession to support the final trajectory inference. Also, we introduce the reality loss term and postprocessing to secure the estimated trajectories to be physically realistic. The experimental results show that our model provides natural and accurate trajectories as well as admissible player ball possession at the same time. Lastly, we suggest several practical applications of our framework including missing trajectory imputation, semi-automated pass annotation, automated zoom-in for match broadcasting, and calculating possession-wise running performance metrics.
Multi-Object Manipulation via Object-Centric Neural Scattering Functions

Learned visual dynamics models have proven effective for robotic manipulation tasks. Yet, it remains unclear how best to represent scenes involving multi-object interactions. Current methods decompose a scene into discrete objects, but they struggle with precise modeling and manipulation amid challenging lighting conditions as they only encode appearance tied with specific illuminations. In this work, we propose using object-centric neural scattering functions (OSFs) as object representations in a model-predictive control framework. OSFs model per-object light transport, enabling compositional scene re-rendering under object rearrangement and varying lighting conditions. By combining this approach with inverse parameter estimation and graph-based neural dynamics models, we demonstrate improved model-predictive control performance and generalization in compositional multi-object environments, even in previously unseen scenarios and harsh lighting conditions.
Inference in IV models with clustered dependence, many instruments and weak identification

Data clustering reduces the effective sample size from the number of observations towards the number of clusters. For instrumental variable models I show that this reduced effective sample size makes the instruments more likely to be weak, in the sense that they contain little information about the endogenous regressor, and many, in the sense that their number is large compared to the sample size. Clustered data therefore increases the need for many and weak instrument robust tests. However, none of the previously developed many and weak instrument robust tests can be applied to this type of data as they all require independent observations. I therefore adapt two types of such tests to clustered data. First, I derive cluster jackknife Anderson-Rubin and score tests by removing clusters rather than individual observations from the statistics. Second, I propose a cluster many instrument Anderson-Rubin test which improves on the first type of tests by using a more optimal, but more complex, weighting matrix. I show that if the clusters satisfy an invariance assumption the higher complexity poses no problems. By revisiting a study on the effect of queenly reign on war I show the empirical relevance of the new tests.
Kalman Filter for Online Classification of Non-Stationary Data

In Online Continual Learning (OCL) a learning system receives a stream of data and sequentially performs prediction and training steps. Important challenges in OCL are concerned with automatic adaptation to the particular non-stationary structure of the data, and with quantification of predictive uncertainty. Motivated by these challenges we introduce a probabilistic Bayesian online learning model by using a (possibly pretrained) neural representation and a state space model over the linear predictor weights. Non-stationarity over the linear predictor weights is modelled using a parameter drift transition density, parametrized by a coefficient that quantifies forgetting. Inference in the model is implemented with efficient Kalman filter recursions which track the posterior distribution over the linear weights, while online SGD updates over the transition dynamics coefficient allows to adapt to the non-stationarity seen in data. While the framework is developed assuming a linear Gaussian model, we also extend it to deal with classification problems and for fine-tuning the deep learning representation. In a set of experiments in multi-class classification using data sets such as CIFAR-100 and CLOC we demonstrate the predictive ability of the model and its flexibility to capture non-stationarity.
Language models are not naysayers: an analysis of language models on negation benchmarks

Negation has been shown to be a major bottleneck for masked language models, such as BERT. However, whether this finding still holds for larger-sized auto-regressive language models (“LLMs”) has not been studied comprehensively. With the ever-increasing volume of research and applications of LLMs, we take a step back to evaluate the ability of current-generation LLMs to handle negation, a fundamental linguistic phenomenon that is central to language understanding. We evaluate different LLMs - including the open-source GPT-neo, GPT-3, and InstructGPT - against a wide range of negation benchmarks. Through systematic experimentation with varying model sizes and prompts, we show that LLMs have several limitations including insensitivity to the presence of negation, an inability to capture the lexical semantics of negation, and a failure to reason under negation.
Anisotropic Ionizing Illumination from an M-type Pre-main-sequence Star, DM Tau

The powerful, high-energy magnetic activities of young stars play important roles in the magnetohydrodynamics in the innermost parts of the protoplanetary disks. In addition, the associated UV and X-ray emission dictates the photochemistry; moreover, the corona activities can affect the atmosphere of a newborn extrasolar planet. How the UV and X-ray photons are generated and how they illuminate the disks are not well understood. Here we report the analyses of the optical and infrared (OIR) photometric monitoring observations and the high angular resolution centimeter-band images of the low-mass (M1-type) pre-main-sequence star DM Tau. We found that the OIR photometric light curves present periodic variations, which suggests that the host young star is rotating in the same direction as the natal disk and is hosting at least one giant cold spot. In addition, we resolved that the ionized gas in the DM Tau disk is localized and its spatial distribution is varying with time. All the present observations can be coherently interpreted, if the giant cold spot is the dominant anisotropic UV and/or X-ray source that illuminates the ambient cone-like region. These results indicate that a detailed theoretical model of the high-energy protostellar emission is essential in understanding the space weather around extrasolar planets and the origin of life.
Knowledge Distillation of Large Language Models

Knowledge Distillation (KD) is a promising technique for reducing the high computational demand of large language models (LLMs). However, previous KD methods are primarily applied to white-box classification models or training small models to imitate black-box model APIs like ChatGPT. How to effectively distill the knowledge of white-box LLMs into small models is still under-explored, which becomes more important with the prosperity of open-source LLMs. In this work, we propose a KD approach that distills LLMs into smaller language models. We first replace the forward Kullback-Leibler divergence (KLD) objective in the standard KD approaches with reverse KLD, which is more suitable for KD on generative language models, to prevent the student model from overestimating the low-probability regions of the teacher distribution. Then, we derive an effective optimization approach to learn this objective. The student models are named MiniLLM. Extensive experiments in the instruction-following setting show that MiniLLM generates more precise responses with higher overall quality, lower exposure bias, better calibration, and higher long-text generation performance than the baselines. Our method is scalable for different model families with 120M to 13B parameters. Our code, data, and model checkpoints can be found in https://github.com/microsoft/LMOps/tree/main/minillm.
Quasiperiodic Peak Energy Oscillations in X-Ray Bursts from SGR J1935+2154

Magnetars are young neutron stars powered by the strongest magnetic fields in the Universe (1013–15 G). Their transient X-ray emission usually manifests as short (a few hundred milliseconds), bright, energetic (∼1040–41 erg) X-ray bursts. Since its discovery in 2014, SGR J1935+2154 has become one of the most prolific magnetars, exhibiting very active bursting episodes and other fascinating events, such as pulse timing antiglitches and fast radio bursts. Here we present evidence for possible 42 Hz (24 ms) quasiperiodic oscillations in the ν F ν spectrum peak energy (E p ) identified in a unique burst detected with the Fermi Gamma-ray Burst Monitor in 2022 January. While quasiperiodic oscillations have been previously reported in the intensity of magnetar burst light curves, quasiperiodic oscillations in E p have not. We also find an additional event from the same outburst that appears to exhibit a similar character in E p , albeit of lower statistical quality. For these two exceptional transients, such E p oscillations can be explained by magnetospheric density and pressure perturbations. For burst-emitting plasma consisting purely of e + e − pairs, these acoustic modes propagate along a highly magnetized flux tube of length up to around L ∼ 130 neutron star radii, with L being lower if ions are present in the emission zone. Detailed time-resolved analyses of other magnetar bursts are encouraged to evaluate the rarity of these events and their underlying mechanisms.
Possibility of Spontaneous Symmetry Breaking in the Nambu-Jona-Lasinio model with torsion

We discuss the Nambu-Jona-Lasinio (NJL) model in curved space-time with torsion in the leading order of the $1/N$ expansion. The effective potential of the torsion sector is calculated using the new technique based on the nonlocal part of anomaly-induced action which was recently found to produce the effective potential in the low-energy limit. The spontaneous symmetry breaking caused by the change of the torsion term is found, confirming the statements known from the existing literature. Furthermore, the gap equation is calculated as a function of curvature and torsion. Finally, the behavior of the effective four fermions coupling constant as a function of the torsion is discussed.
The Origin of Universality in the Inner Edges of Planetary Systems

The characteristic orbital period of the innermost objects within the galactic census of planetary and satellite systems appears to be nearly universal, with P on the order of a few days. This paper presents a theoretical framework that provides a simple explanation for this phenomenon. By considering the interplay between disk accretion, magnetic field generation by convective dynamos, and Kelvin–Helmholtz contraction, we derive an expression for the magnetospheric truncation radius in astrophysical disks and find that the corresponding orbital frequency is independent of the mass of the host body. Our analysis demonstrates that this characteristic frequency corresponds to a period of P ∼ 3 days although intrinsic variations in system parameters are expected to introduce a factor of a ∼2–3 spread in this result. Standard theory of orbital migration further suggests that planets should stabilize at an orbital period that exceeds disk truncation by a small margin. Cumulatively, our findings predict that the periods of close-in bodies should span P ∼ 2–12 days—a range that is consistent with observations.