Miracle-dz/UniFAR-FTUs huggingface dataset:
S2abEL: A Dataset for Entity Linking from Scientific Tables

Entity linking (EL) is the task of linking a textual mention to its corresponding entry in a knowledge base, and is critical for many knowledge-intensive NLP applications. When applied to tables in scientific papers, EL is a step toward large-scale scientific knowledge bases that could enable advanced scientific question answering and analytics. We present the first dataset for EL in scientific tables. EL for scientific tables is especially challenging because scientific knowledge bases can be very incomplete, and disambiguating table mentions typically requires understanding the papers's tet in addition to the table. Our dataset, S2abEL, focuses on EL in machine learning results tables and includes hand-labeled cell types, attributed sources, and entity links from the PaperswithCode taxonomy for 8,429 cells from 732 tables. We introduce a neural baseline method designed for EL on scientific tables containing many out-of-knowledge-base mentions, and show that it significantly outperforms a state-of-the-art generic table EL method. The best baselines fall below human performance, and our analysis highlights avenues for improvement.
The Ideal Continual Learner: An Agent That Never Forgets

The goal of continual learning is to find a model that solves multiple learning tasks which are presented sequentially to the learner. A key challenge in this setting is that the learner may forget how to solve a previous task when learning a new task, a phenomenon known as catastrophic forgetting. To address this challenge, many practical methods have been proposed, including memory-based, regularization-based, and expansion-based methods. However, a rigorous theoretical understanding of these methods remains elusive. This paper aims to bridge this gap between theory and practice by proposing a new continual learning framework called Ideal Continual Learner (ICL), which is guaranteed to avoid catastrophic forgetting by construction. We show that ICL unifies multiple well-established continual learning methods and gives new theoretical insights into the strengths and weaknesses of these methods. We also derive generalization bounds for ICL which allow us to theoretically quantify how rehearsal affects generalization. Finally, we connect ICL to several classic subjects and research topics of modern interest, which allows us to make historical remarks and inspire future directions.
Intersection Theory on Weighted Blowups of F-theory Vacua

Generalizing the results of 1211.6077 and 1703.00905, we prove a formula for the pushforward of an arbitrary analytic function of the exceptional divisor class of a weighted blowup of an algebraic variety centered at a smooth complete intersection with normal crossing. We check this formula extensively by computing the generating function of intersection numbers of a weighted blowup of the generic SU(5) Tate model over arbitrary smooth base, and comparing the answer to known results. Motivated by applications to four-dimensional F-theory flux compactifications, we use our formula to compute the intersection pairing on the vertical part of the middle cohomology of elliptic Calabi-Yau 4-folds resolving the generic F$_4$ and Sp(6) Tate models with non-minimal singularities. These resolutions lead to non-flat fibrations in which certain fibers contain 3-fold (divisor) components, whose physical interpretation in M/F-theory remains to be fully explored.
Extremum Seeking Nonlinear Regulator with Concurrent Uncertainties in Exosystems and Control Directions

This paper proposes a non-adaptive control solution framework to the practical output regulation problem (PORP) for a class of nonlinear systems with uncertain parameters, unknown control directions and uncertain exosystem dynamics. The concurrence of the unknown control directions and uncertainties in both the system dynamics and the exosystem pose a significant challenge to solving this problem. Moreover, in light of the nonlinear internal model approach, we first convert the robust PORP into a robust non-adaptive stabilization problem for the augmented system with integral Input-to-State Stable (iISS) inverse dynamics. By employing an extremum-seeking control approach, the construction of our solution method avoids the use of Nussbaum-type gain techniques to address the robust PORP subject to time-varying control directions. The stability of the non-adaptive output regulation design is proven via a Lie bracket averaging technique where uniform ultimate boundedness of the closed-loop signals is guaranteed. As a result, the estimation and tracking errors converge to zero exponentially, provided that the frequency of the dither signal goes to infinity. Finally, a simulation example with unknown coefficients is provided to exemplify the validity of the proposed control solution frameworks.
Cold dark matter particle mass and properties and axion-like dark radiation in $\Lambda$CDM cosmology

A theory is presented for the mass, size, lifetime, and other properties of cold dark matter particles within the $\Lambda$CDM cosmology. Using Illustris simulations, we demonstrate the mass and energy cascade in self-gravitating collisionless dark matter that facilitates the hierarchical structure formation of dark matter haloes. A scale-independent rate of energy cascade $\varepsilon_u \approx 10^{-7}m^2/s^3$ can be identified. Energy cascade leads to universal scaling laws on relevant scales $r$, i.e. a two-thirds law for kinetic energy ($v_r^2\propto \varepsilon_u^{2/3}r^{2/3}$) and a four-thirds law for halo density ($\rho_r\propto\varepsilon_u^{2/3}G^{-1}r^{-4/3}$), where $G$ is the gravitational constant. Both scaling laws can be confirmed by simulations and galaxy rotation curves. For cold and collisionless dark matter interacting via gravity only and because of the scale independence of $\varepsilon_u$, these scaling laws can be extended down to the smallest scale where quantum effect is important. Combined with the uncertainty principle and virial theorem on that scale, we estimate a mass $m_X=(\varepsilon_u\hbar^5G^{-4})^{1/9}=10^{12}$GeV, size $l_X=(\varepsilon_u^{-1}\hbar G)^{1/3}=10^{-13}$m, and lifetime $\tau_X=c^2/\varepsilon_u=10^{16}$years for cold dark matter particles. Here $\hbar$ is Planck constant, and $c$ is the speed of light. The energy scale $E_X=(\varepsilon_u^5\hbar^7G^{-2})^{1/9}=10^{-9}$eV strongly suggests a dark radiation to provide a viable mechanism for energy dissipation. The axion-like dark radiation should be produced at an early time $t_X=(\varepsilon_u^{-5}\hbar^2G^2)^{1/9}=10^{-6}$s (quark epoch) with a mass of $10^{-9}$eV, a GUT scale decay constant $10^{16}$GeV, an axion-photon coupling constant $10^{-18}$GeV$^{-1}$, and energy density 1$\%$ of the photon energy in CMB. Potential extension to self-interacting dark matter is also presented.
Embeddability of graphs and Weihrauch degrees

We study the complexity of the following related computational tasks concerning a fixed countable graph G: 1. Does a countable graph H provided as input have a(n induced) subgraph isomorphic to G? 2. Given a countable graph H that has a(n induced) subgraph isomorphic to G, find such a subgraph. The framework for our investigations is given by effective Wadge reducibility and by Weihrauch reducibility. Our work follows on"Reverse mathematics and Weihrauch analysis motivated by finite complexity theory"(Computability, 2021) by BeMent, Hirst and Wallace, and we answer several of their open questions.
Hierarchical Dialogue Understanding with Special Tokens and Turn-level Attention

Compared with standard text, understanding dialogue is more challenging for machines as the dynamic and unexpected semantic changes in each turn. To model such inconsistent semantics, we propose a simple but effective Hierarchical Dialogue Understanding model, HiDialog. Specifically, we first insert multiple special tokens into a dialogue and propose the turn-level attention to learn turn embeddings hierarchically. Then, a heterogeneous graph module is leveraged to polish the learned embeddings. We evaluate our model on various dialogue understanding tasks including dialogue relation extraction, dialogue emotion recognition, and dialogue act classification. Results show that our simple approach achieves state-of-the-art performance on all three tasks above. All our source code is publicly available at https://github.com/ShawX825/HiDialog.
Time series clustering based on prediction accuracy of global forecasting models

In this paper, a novel method to perform model-based clustering of time series is proposed. The procedure relies on two iterative steps: (i) K global forecasting models are fitted via pooling by considering the series pertaining to each cluster and (ii) each series is assigned to the group associated with the model producing the best forecasts according to a particular criterion. Unlike most techniques proposed in the literature, the method considers the predictive accuracy as the main element for constructing the clustering partition, which contains groups jointly minimizing the overall forecasting error. Thus, the approach leads to a new clustering paradigm where the quality of the clustering solution is measured in terms of its predictive capability. In addition, the procedure gives rise to an effective mechanism for selecting the number of clusters in a time series database and can be used in combination with any class of regression model. An extensive simulation study shows that our method outperforms several alternative techniques concerning both clustering effectiveness and predictive accuracy. The approach is also applied to perform clustering in several datasets used as standard benchmarks in the time series literature, obtaining great results.
Spin-orientation-resolved attosecond chronoscopy in strong field ionization

Attosecond chronoscopy represents a major breakthrough in the study of ultrafast phenomena and has the potential to revolutionize our understanding of the fundamental physics of matter. We theoretically investigate the spin-orientation-resolved attosecond chronoscopy for the first time by the circular RABBIT technique for Kr atoms. Due to the spin-orbit interaction and the sensitivity of ionization in circularly polarized fields to the sense of electron rotation in the initial state, the spin-resolved ionization rates of photoelectrons emitted from $^2P_{1/2}$ and $^2P_{3/2}$ channels can be expressed via $m$-resolved ionization rates of initial state, where $m$ is the orbital magnetic quantum number. We demonstrate that the yields difference between spin-up and spin-down photoelectrons from each channel are closely associated with the different behaviors of corresponding Wigner time delay. We find that the Wigner time delay between spin-up and spin-down photoelectrons in the polarization plane can reach several tens of attoseconds in the co-rotating geometry, but a few attoseconds in the counter-rotating geometry. Our approach opens up a new avenue for probing the spin-dependent behavior of Wigner time delay, and lays the foundation for spin-orientation-resolved attosecond chronoscopy, which can be verified by the current experimental techniques.
Indexability of Finite State Restless Multi-Armed Bandit and Rollout Policy

We consider finite state restless multi-armed bandit problem. The decision maker can act on M bandits out of N bandits in each time step. The play of arm (active arm) yields state dependent rewards based on action and when the arm is not played, it also provides rewards based on the state and action. The objective of the decision maker is to maximize the infinite horizon discounted reward. The classical approach to restless bandits is Whittle index policy. In such policy, the M arms with highest indices are played at each time step. Here, one decouples the restless bandits problem by analyzing relaxed constrained restless bandits problem. Then by Lagrangian relaxation problem, one decouples restless bandits problem into N single-armed restless bandit problems. We analyze the single-armed restless bandit. In order to study the Whittle index policy, we show structural results on the single armed bandit model. We define indexability and show indexability in special cases. We propose an alternative approach to verify the indexable criteria for a single armed bandit model using value iteration algorithm. We demonstrate the performance of our algorithm with different examples. We provide insight on condition of indexability of restless bandits using different structural assumptions on transition probability and reward matrices. We also study online rollout policy and discuss the computation complexity of algorithm and compare that with complexity of index computation. Numerical examples illustrate that index policy and rollout policy performs better than myopic policy.
Industry Classification Using a Novel Financial Time-Series Case Representation

The financial domain has proven to be a fertile source of challenging machine learning problems across a variety of tasks including prediction, clustering, and classification. Researchers can access an abundance of time-series data and even modest performance improvements can be translated into significant additional value. In this work, we consider the use of case-based reasoning for an important task in this domain, by using historical stock returns time-series data for industry sector classification. We discuss why time-series data can present some significant representational challenges for conventional case-based reasoning approaches, and in response, we propose a novel representation based on stock returns embeddings, which can be readily calculated from raw stock returns data. We argue that this representation is well suited to case-based reasoning and evaluate our approach using a large-scale public dataset for the industry sector classification task, demonstrating substantial performance improvements over several baselines using more conventional representations.
Investigating Vision Foundational Models for Tactile Representation Learning

Tactile representation learning (TRL) equips robots with the ability to leverage touch information, boosting performance in tasks such as environment perception and object manipulation. However, the heterogeneity of tactile sensors results in many sensor- and task-specific learning approaches. This limits the efficacy of existing tactile datasets, and the subsequent generalisability of any learning outcome. In this work, we investigate the applicability of vision foundational models to sensor-agnostic TRL, via a simple yet effective transformation technique to feed the heterogeneous sensor readouts into the model. Our approach recasts TRL as a computer vision (CV) problem, which permits the application of various CV techniques for tackling TRL-specific challenges. We evaluate our approach on multiple benchmark tasks, using datasets collected from four different tactile sensors. Empirically, we demonstrate significant improvements in task performance, model robustness, as well as cross-sensor and cross-task knowledge transferability with limited data requirements.
SRPT vs Smith Predictor for Vehicle Teleoperation

Vehicle teleoperation has potential applications in fallback solutions for autonomous vehicles, remote delivery services, and hazardous operations. However, network delays and limited situational awareness can compromise teleoperation performance and increase the cognitive workload of human operators. To address these issues, we previously introduced the novel successive reference pose tracking (SRPT) approach, which transmits successive reference poses to the vehicle instead of steering commands. This paper compares the stability and performance of SRPT with Smith predictor-based approaches for direct vehicle teleoperation in challenging scenarios. The Smith predictor approach is further categorized, one with Lookahead driver and second with Stanley driver. Simulations are conducted in a Simulink environment, considering variable network delays and different vehicle speeds, and include maneuvers such as tight corners, slalom, low-adhesion roads, and strong crosswinds. The results show that the SRPT approach significantly improves stability and reference tracking performance, with negligible effect of network delays on path tracking. Our findings demonstrate the effectiveness of SRPT in eliminating the detrimental effect of network delays in vehicle teleoperation.
ISAAC Newton: Input-based Approximate Curvature for Newton's Method

We present ISAAC (Input-baSed ApproximAte Curvature), a novel method that conditions the gradient using selected second-order information and has an asymptotically vanishing computational overhead, assuming a batch size smaller than the number of neurons. We show that it is possible to compute a good conditioner based on only the input to a respective layer without a substantial computational overhead. The proposed method allows effective training even in small-batch stochastic regimes, which makes it competitive to first-order as well as second-order methods.
TPMIL: Trainable Prototype Enhanced Multiple Instance Learning for Whole Slide Image Classification

Digital pathology based on whole slide images (WSIs) plays a key role in cancer diagnosis and clinical practice. Due to the high resolution of the WSI and the unavailability of patch-level annotations, WSI classification is usually formulated as a weakly supervised problem, which relies on multiple instance learning (MIL) based on patches of a WSI. In this paper, we aim to learn an optimal patch-level feature space by integrating prototype learning with MIL. To this end, we develop a Trainable Prototype enhanced deep MIL (TPMIL) framework for weakly supervised WSI classification. In contrast to the conventional methods which rely on a certain number of selected patches for feature space refinement, we softly cluster all the instances by allocating them to their corresponding prototypes. Additionally, our method is able to reveal the correlations between different tumor subtypes through distances between corresponding trained prototypes. More importantly, TPMIL also enables to provide a more accurate interpretability based on the distance of the instances from the trained prototypes which serves as an alternative to the conventional attention score-based interpretability. We test our method on two WSI datasets and it achieves a new SOTA. GitHub repository: https://github.com/LitaoYang-Jet/TPMIL
StyleGenes: Discrete and Efficient Latent Distributions for GANs

We propose a discrete latent distribution for Generative Adversarial Networks (GANs). Instead of drawing latent vectors from a continuous prior, we sample from a finite set of learnable latents. However, a direct parametrization of such a distribution leads to an intractable linear increase in memory in order to ensure sufficient sample diversity. We address this key issue by taking inspiration from the encoding of information in biological organisms. Instead of learning a separate latent vector for each sample, we split the latent space into a set of genes. For each gene, we train a small bank of gene variants. Thus, by independently sampling a variant for each gene and combining them into the final latent vector, our approach can represent a vast number of unique latent samples from a compact set of learnable parameters. Interestingly, our gene-inspired latent encoding allows for new and intuitive approaches to latent-space exploration, enabling conditional sampling from our unconditionally trained model. Moreover, our approach preserves state-of-the-art photo-realism while achieving better disentanglement than the widely-used StyleMapping network.
Representing Additive Gaussian Processes by Sparse Matrices

Among generalized additive models, additive Mat\'ern Gaussian Processes (GPs) are one of the most popular for scalable high-dimensional problems. Thanks to their additive structure and stochastic differential equation representation, back-fitting-based algorithms can reduce the time complexity of computing the posterior mean from $O(n^3)$ to $O(n\log n)$ time where $n$ is the data size. However, generalizing these algorithms to efficiently compute the posterior variance and maximum log-likelihood remains an open problem. In this study, we demonstrate that for Additive Mat\'ern GPs, not only the posterior mean, but also the posterior variance, log-likelihood, and gradient of these three functions can be represented by formulas involving only sparse matrices and sparse vectors. We show how to use these sparse formulas to generalize back-fitting-based algorithms to efficiently compute the posterior mean, posterior variance, log-likelihood, and gradient of these three functions for additive GPs, all in $O(n \log n)$ time. We apply our algorithms to Bayesian optimization and propose efficient algorithms for posterior updates, hyperparameters learning, and computations of the acquisition function and its gradient in Bayesian optimization. Given the posterior, our algorithms significantly reduce the time complexity of computing the acquisition function and its gradient from $O(n^2)$ to $O(\log n)$ for general learning rate, and even to $O(1)$ for small learning rate.
A wall-time minimizing parallelization strategy for approximate Bayesian computation

Approximate Bayesian Computation (ABC) is a widely applicable and popular approach to estimating unknown parameters of mechanistic models. As ABC analyses are computationally expensive, parallelization on high-performance infrastructure is often necessary. However, the existing parallelization strategies leave computing resources unused at times and thus do not optimally leverage them yet. We present look-ahead scheduling, a wall-time minimizing parallelization strategy for ABC Sequential Monte Carlo algorithms, which avoids idle times of computing units by preemptive sampling of subsequent generations. This allows to utilize all available resources. The strategy can be integrated with e.g. adaptive distance function and summary statistic selection schemes, which is essential in practice. Our key contribution is the theoretical assessment of the strategy of preemptive sampling and the proof of unbiasedness. Complementary, we provide an implementation and evaluate the strategy on different problems and numbers of parallel cores, showing speed-ups of typically 10-20% and up to 50% compared to the best established approach, with some variability. Thus, the proposed strategy allows to improve the cost and run-time efficiency of ABC methods on high-performance infrastructure.
High Tide or Riptide on the Cosmic Shoreline? A Water-rich Atmosphere or Stellar Contamination for the Warm Super-Earth GJ 486b from JWST Observations

Planets orbiting M-dwarf stars are prime targets in the search for rocky exoplanet atmospheres. The small size of M dwarfs renders their planets exceptional targets for transmission spectroscopy, facilitating atmospheric characterization. However, it remains unknown whether their host stars’ highly variable extreme-UV radiation environments allow atmospheres to persist. With JWST, we have begun to determine whether or not the most favorable rocky worlds orbiting M dwarfs have detectable atmospheres. Here, we present a 2.8–5.2 μm JWST NIRSpec/G395H transmission spectrum of the warm (700 K, 40.3× Earth’s insolation) super-Earth GJ 486b (1.3 R ⊕ and 3.0 M ⊕). The measured spectrum from our two transits of GJ 486b deviates from a flat line at 2. 2σ − 3. 3σ, based on three independent reductions. Through a combination of forward and retrieval models, we determine that GJ 486b either has a water-rich atmosphere (with the most stringent constraint on the retrieved water abundance of H2O > 10% to 2σ) or the transmission spectrum is contaminated by water present in cool unocculted starspots. We also find that the measured stellar spectrum is best fit by a stellar model with cool starspots and hot faculae. While both retrieval scenarios provide equal quality fits ( χν2=1.0 ) to our NIRSpec/G395H observations, shorter wavelength observations can break this degeneracy and reveal if GJ 486b sustains a water-rich atmosphere.