Miracle-dz/UniFAR-FTUs huggingface dataset:
Knowledge Graph Self-Supervised Rationalization for Recommendation

In this paper, we introduce a new self-supervised rationalization method, called KGRec, for knowledge-aware recommender systems. To effectively identify informative knowledge connections, we propose an attentive knowledge rationalization mechanism that generates rational scores for knowledge triplets. With these scores, KGRec integrates generative and contrastive self-supervised tasks for recommendation through rational masking. To highlight rationales in the knowledge graph, we design a novel generative task in the form of masking-reconstructing. By masking important knowledge with high rational scores, KGRec is trained to rebuild and highlight useful knowledge connections that serve as rationales. To further rationalize the effect of collaborative interactions on knowledge graph learning, we introduce a contrastive learning task that aligns signals from knowledge and user-item interaction views. To ensure noise-resistant contrasting, potential noisy edges in both graphs judged by the rational scores are masked. Extensive experi-ments on three real-world datasets demonstrate that KGRec outperforms state-of-the-art methods. We also provide the implementation codes for our approach at https://github.com/HKUDS/KGRec.
A Remarkably Accurate Predictor of Sunspot Cycle Amplitude

The area ratios of sunspots to white light faculae in the first two years of sunspot cycles 12-21 correlate remarkably well with the peak amplitudes of those cycles between 1878-1980 (Brown and Evans, 1980). This finding could not be used to predict subsequent cycle amplitudes because the Royal Greenwich Observatory program of facular area measurements was discontinued in 1976. We use continuum images from the Michelson Doppler Imager (MDI) and the Heliospheric and Magnetic Imager (HMI) to show that the close relation holds also for cycle 24, and we predict an amplitude of approximately 185 for the current cycle 25.
The Role of Subgroup Separability in Group-Fair Medical Image Classification

We investigate performance disparities in deep classifiers. We find that the ability of classifiers to separate individuals into subgroups varies substantially across medical imaging modalities and protected characteristics; crucially, we show that this property is predictive of algorithmic bias. Through theoretical analysis and extensive empirical evaluation, we find a relationship between subgroup separability, subgroup disparities, and performance degradation when models are trained on data with systematic bias such as underdiagnosis. Our findings shed new light on the question of how models become biased, providing important insights for the development of fair medical imaging AI.
Secure-by-Construction Synthesis for Control Systems

In this paper, we present the synthesis of secure-by-construction controllers that address safety and security properties simultaneously in cyber-physical systems. Our focus is on studying a specific security property called opacity, which characterizes the system's ability to maintain plausible deniability of its secret behavior in the presence of an intruder. These controllers are synthesized based on a concept of so-called (augmented) control barrier functions, which we introduce and discuss in detail. We propose conditions that facilitate the construction of the desired (augmented) control barrier functions and their corresponding secure-by-construction controllers. To compute these functions, we propose an iterative scheme that leverages iterative sum-of-square programming techniques. This approach enables efficient computation of these functions, particularly for polynomial systems. Moreover, we demonstrate the flexibility of our approach by incorporating user-defined cost functions into the construction of secure-by-construction controllers. Finally, we validate the effectiveness of our results through two case studies, illustrating the practical applicability and benefits of our proposed approach.
Principal subbundles for dimension reduction

In this paper we demonstrate how sub-Riemannian geometry can be used for manifold learning and surface reconstruction by combining local linear approximations of a point cloud to obtain lower dimensional bundles. Local approximations obtained by local PCAs are collected into a rank $k$ tangent subbundle on $\mathbb{R}^d$, $k<d$, which we call a principal subbundle. This determines a sub-Riemannian metric on $\mathbb{R}^d$. We show that sub-Riemannian geodesics with respect to this metric can successfully be applied to a number of important problems, such as: explicit construction of an approximating submanifold $M$, construction of a representation of the point-cloud in $\mathbb{R}^k$, and computation of distances between observations, taking the learned geometry into account. The reconstruction is guaranteed to equal the true submanifold in the limit case where tangent spaces are estimated exactly. Via simulations, we show that the framework is robust when applied to noisy data. Furthermore, the framework generalizes to observations on an a priori known Riemannian manifold.
Angle-selective all-dielectric Huygens’ metasurfaces

We experimentally and numerically study the angularly resolved transmission properties of dielectric metasurfaces consisting of silicon nanodisks which support electric and magnetic dipolar Mie-type resonances in the near-infrared spectral range. First, we concentrate on Huygens’ metasurfaces which are characterised by a spectral overlap of the fundamental electric and magnetic dipole resonances of the silicon nanodisks at normal incidence. Huygens’ metasurfaces exhibit a high transmitted intensity over the spectral width of the resonances due to impedance matching, while the transmitted phase shows a variation of 2π as the wavelength is swept across the width of the resonances. We observe that the transmittance of the Huygens’ metasurfaces depends on the incidence angle and is sensitive to polarisation for non-normal incidence. As the incidence angle is increased starting from normal incidence, the two dipole resonances are shifted out of the spectral overlap and the resonant features appear as pronounced transmittance minima. Next, we consider a metasurface with an increased nanodisk radius as compared to the Huygens’ metasurface, which supports spectrally separate electric and magnetic dipole resonances at normal incidence. We show that for TM polarisation, we can shift the resonances of this metasurface into spectral overlap and regain the high resonant transmittance characteristic of Huygens’ metasurfaces at a particular incidence angle. Furthermore, both metasurfaces are demonstrated to reject all TM polarised light incident under angles other than the design overlap angle at their respective operation frequency. Our experimental observations are in good qualitative agreement with numerical calculations.
Metal-enriched Pair-instability Supernovae: Effects of Rotation

In this paper, we revisit metal-enriched rotating pair-instability supernovae (PISNe) models for metallicities consistent with the Small Magellanic Cloud, the Large Magellanic Cloud (LMC), and 0.1Z ⊙. By calculating multiple models, we intend to clarify mass ranges and the ejected 56Ni masses from the PISNe, and mass-loss histories for progenitors. We find that the choice of the Wolf–Rayet (WR) mass-loss rates is important, and we adopt the recently proposed rate of Sander & Vink, which covers the mass ranges for PISNe progenitors. We show that slow rotation lowers the PISN range because the core mass increases by rotational mixing. On the other hand, when we assume a typical rotation speed for observed OB stars, the mass-loss increase becomes more significant, and the final stellar masses are lower than in nonrotating models. As a result, the typical mass range for bright supernovae (SNe), with a 56Ni mass higher than 10M ⊙ for these fast-rotating models is more than 400 and 350M ⊙ for LMC and 0.1Z ⊙ metallicities, respectively. It is interesting that unlike in previous works, we find oxygen-rich progenitors for most cases. This O-rich progenitor may be consistent with the recently identified PISN candidate SN2018ibb. He-rich progenitors are seen only for relatively dim and metal-poor (Z ≲ 0.1Z ⊙) PISNe. We also discuss the black hole mass gap for metal-enriched PISNe, and we show that the upper bound for the gap is lower than in the Population III case.
Focused Transformer: Contrastive Training for Context Scaling

Large language models have an exceptional capability to incorporate new information in a contextual manner. However, the full potential of such an approach is often restrained due to a limitation in the effective context length. One solution to this issue is to endow an attention layer with access to an external memory, which comprises of (key, value) pairs. Yet, as the number of documents increases, the proportion of relevant keys to irrelevant ones decreases, leading the model to focus more on the irrelevant keys. We identify a significant challenge, dubbed the distraction issue, where keys linked to different semantic values might overlap, making them hard to distinguish. To tackle this problem, we introduce the Focused Transformer (FoT), a technique that employs a training process inspired by contrastive learning. This novel approach enhances the structure of the (key, value) space, enabling an extension of the context length. Our method allows for fine-tuning pre-existing, large-scale models to lengthen their effective context. This is demonstrated by our fine-tuning of $3B$ and $7B$ OpenLLaMA checkpoints. The resulting models, which we name LongLLaMA, exhibit advancements in tasks requiring a long context. We further illustrate that our LongLLaMA models adeptly manage a $256 k$ context length for passkey retrieval.
Math Agents: Computational Infrastructure, Mathematical Embedding, and Genomics

The advancement in generative AI could be boosted with more accessible mathematics. Beyond human-AI chat, large language models (LLMs) are emerging in programming, algorithm discovery, and theorem proving, yet their genomics application is limited. This project introduces Math Agents and mathematical embedding as fresh entries to the"Moore's Law of Mathematics", using a GPT-based workflow to convert equations from literature into LaTeX and Python formats. While many digital equation representations exist, there's a lack of automated large-scale evaluation tools. LLMs are pivotal as linguistic user interfaces, providing natural language access for human-AI chat and formal languages for large-scale AI-assisted computational infrastructure. Given the infinite formal possibility spaces, Math Agents, which interact with math, could potentially shift us from"big data"to"big math". Math, unlike the more flexible natural language, has properties subject to proof, enabling its use beyond traditional applications like high-validation math-certified icons for AI alignment aims. This project aims to use Math Agents and mathematical embeddings to address the ageing issue in information systems biology by applying multiscalar physics mathematics to disease models and genomic data. Generative AI with episodic memory could help analyse causal relations in longitudinal health records, using SIR Precision Health models. Genomic data is suggested for addressing the unsolved Alzheimer's disease problem.
Retrieval from Mixed Sampling Frequency: Generic Identifiability in the Unit Root VAR

The"REtrieval from MIxed Sampling"(REMIS) approach based on blocking developed in Anderson et al. (2016a) is concerned with retrieving an underlying high frequency model from mixed frequency observations. In this paper we investigate parameter-identifiability in the Johansen (1995) vector error correction model for mixed frequency data. We prove that from the second moments of the blocked process after taking differences at lag N (N is the slow sampling rate), the parameters of the high frequency system are generically identified. We treat the stock and the flow case as well as deterministic terms.
Oriented Object Detection in Optical Remote Sensing Images using Deep Learning: A Survey

Oriented object detection is one of the most fundamental and challenging tasks in remote sensing, aiming at locating the oriented objects of numerous predefined object categories. Recently, deep learning based methods have achieved remarkable performance in detecting oriented objects in optical remote sensing imagery. However, a thorough review of the literature in remote sensing has not yet emerged. Therefore, we give a comprehensive survey of recent advances and cover many aspects of oriented object detection, including problem definition, commonly used datasets, evaluation protocols, detection frameworks, oriented object representations, and feature representations. Besides, the state-of-the-art methods are analyzed and discussed. We finally discuss future research directions to put forward some useful research guidance. We believe that this survey shall be valuable to researchers across academia and industry
Surface Brightness Fluctuations

The Surface Brightness Fluctuation (SBF) method is a powerful tool for determining distances to early-type galaxies. The method measures the intrinsic variance in a galaxy's surface brightness distribution to determine its distance with an accuracy of about 5%. Here, we discuss the mathematical formalism behind the SBF technique, its calibration, and the practicalities of how measurements are performed. We review the various sources of uncertainties that affect the method and discuss how they can be minimized or controlled through careful observations and data analysis. The SBF technique has already been successfully applied to a large number of galaxies and used for deriving accurate constraints on the Hubble-Lema\^itre constant $H_0$. An approved JWST program will greatly reduce the systematic uncertainties by establishing a firm zero-point calibration using tip of the red giant branch (TRGB) distances. We summarize the existing results and discuss the excellent potential of the SBF method for improving the current constraints on $H_0$.
Contrast Is All You Need

In this study, we analyze data-scarce classification scenarios, where available labeled legal data is small and imbalanced, potentially hurting the quality of the results. We focused on two finetuning objectives; SetFit (Sentence Transformer Finetuning), a contrastive learning setup, and a vanilla finetuning setup on a legal provision classification task. Additionally, we compare the features that are extracted with LIME (Local Interpretable Model-agnostic Explanations) to see which particular features contributed to the model's classification decisions. The results show that a contrastive setup with SetFit performed better than vanilla finetuning while using a fraction of the training samples. LIME results show that the contrastive learning approach helps boost both positive and negative features which are legally informative and contribute to the classification results. Thus a model finetuned with a contrastive objective seems to base its decisions more confidently on legally informative features.
Learning Symbolic Rules over Abstract Meaning Representations for Textual Reinforcement Learning

Text-based reinforcement learning agents have predominantly been neural network-based models with embeddings-based representation, learning uninterpretable policies that often do not generalize well to unseen games. On the other hand, neuro-symbolic methods, specifically those that leverage an intermediate formal representation, are gaining significant attention in language understanding tasks. This is because of their advantages ranging from inherent interpretability, the lesser requirement of training data, and being generalizable in scenarios with unseen data. Therefore, in this paper, we propose a modular, NEuro-Symbolic Textual Agent (NESTA) that combines a generic semantic parser with a rule induction system to learn abstract interpretable rules as policies. Our experiments on established text-based game benchmarks show that the proposed NESTA method outperforms deep reinforcement learning-based techniques by achieving better generalization to unseen test games and learning from fewer training interactions.
FLuID: Mitigating Stragglers in Federated Learning using Invariant Dropout

Federated Learning (FL) allows machine learning models to train locally on individual mobile devices, synchronizing model updates via a shared server. This approach safeguards user privacy; however, it also generates a heterogeneous training environment due to the varying performance capabilities across devices. As a result, straggler devices with lower performance often dictate the overall training time in FL. In this work, we aim to alleviate this performance bottleneck due to stragglers by dynamically balancing the training load across the system. We introduce Invariant Dropout, a method that extracts a sub-model based on the weight update threshold, thereby minimizing potential impacts on accuracy. Building on this dropout technique, we develop an adaptive training framework, Federated Learning using Invariant Dropout (FLuID). FLuID offers a lightweight sub-model extraction to regulate computational intensity, thereby reducing the load on straggler devices without affecting model quality. Our method leverages neuron updates from non-straggler devices to construct a tailored sub-model for each straggler based on client performance profiling. Furthermore, FLuID can dynamically adapt to changes in stragglers as runtime conditions shift. We evaluate FLuID using five real-world mobile clients. The evaluations show that Invariant Dropout maintains baseline model efficiency while alleviating the performance bottleneck of stragglers through a dynamic, runtime approach.
BrickPal: Augmented Reality-based Assembly Instructions for Brick Models

The assembly instruction is a mandatory component of Lego-like brick sets.The conventional production of assembly instructions requires a considerable amount of manual fine-tuning, which is intractable for casual users and customized brick sets.Moreover, the traditional paper-based instructions lack expressiveness and interactivity.To tackle the two problems above, we present BrickPal, an augmented reality-based system, which visualizes assembly instructions in an augmented reality head-mounted display. It utilizes Natural Language Processing (NLP) techniques to generate plausible assembly sequences, and provide real-time guidance in the AR headset.Our user study demonstrates BrickPal's effectiveness at assisting users in brick assembly compared to traditional assembly methods. Additionally, the NLP algorithm-generated assembly sequences achieve the same usability with manually adapted sequences.
Joint evolution of a Lorentz-covariant massless scalar field and its point-charge source in one space dimension

In this paper we prove that the static solution of the Cauchy problem for a massless real scalar field that is sourced by a point charge in $1+1$ dimensions is asymptotically stable under perturbation by compactly-supported incoming radiation. This behavior is due to the process of back-reaction. Taking the approach of Kiessling, we rigorously derive the expression for the force on the particle from the principle of total energy-momentum conservation. We provide a simple, closed form for the self-force resulting from back-reaction, and show that it is restorative, i.e. proportional to negative velocity, and causes the charge to return to rest after the radiation passes through. We establish these results by studying the joint evolution problem for the particle-scalar field system, and proving its global well-posedness and the claimed asymptotic behavior.
ContainerGym: A Real-World Reinforcement Learning Benchmark for Resource Allocation

We present ContainerGym, a benchmark for reinforcement learning inspired by a real-world industrial resource allocation task. The proposed benchmark encodes a range of challenges commonly encountered in real-world sequential decision making problems, such as uncertainty. It can be configured to instantiate problems of varying degrees of difficulty, e.g., in terms of variable dimensionality. Our benchmark differs from other reinforcement learning benchmarks, including the ones aiming to encode real-world difficulties, in that it is directly derived from a real-world industrial problem, which underwent minimal simplification and streamlining. It is sufficiently versatile to evaluate reinforcement learning algorithms on any real-world problem that fits our resource allocation framework. We provide results of standard baseline methods. Going beyond the usual training reward curves, our results and the statistical tools used to interpret them allow to highlight interesting limitations of well-known deep reinforcement learning algorithms, namely PPO, TRPO and DQN.
General model discovery using statistical evaluation maps

On any given dataset, a very wide variety of statistical models may be applicable, based on experience and tradition, robustness and sensitivity requirements, algorithmic, computational or philosophical considerations, risk and eventual usage. We propose a technique to compare such models in a very general framework. We establish that under general conditions, statistical models that adequately explain properties of the data can be well separated from those that do not. Our resampling-based approach achieves concurrent ranking of models and consistent approximation of sampling distribution of parameter estimators under any model, thus enabling inference within each model. Consequently, our proposal is one of simultaneous model discovery and inference. For traditional covariate selection problems where there are $p$ covariates, our proposal results in a fast and parallel algorithm that fits only a single model and evaluates $p +1$ models, as opposed to the traditional requirement of fitting and evaluating $2^{p}$ models. We illustrate in simulation experiments that our proposed method typically performs better than or competitively with currently used methods for model selection. We use our procedure to elicit climatic drivers of Indian monsoon precipitation.
A design theory for transparency of information privacy practices

Public policy initiatives demand transparency of information privacy practices (TIPP) since the 1970s, at least, when a first version of the fair information practice principles (FIPP) was published. Decades later, information system (IS) providers are still struggling to meet this demand. Apparently, there is a tension between socially desirable guardrails for privacy practices (e.g., allowing for anonymity) and economic value creation (e.g., exploiting personalization surplus). To inform IS designs that can change such avaricious states to the benefit of consumers, we present a theoretical foundation (TIPP theory) explaining and prescribing what needs to be built to establish TIPP in the real world. On the one hand, TIPP theory explains why IS providers, who have privacy practices to hide, should not go beyond IS designs that meet legal demands. More importantly, TIPP theory explains and prescribes what IS providers need to build to actually establish TIPP and be able to differentiate from competitors by doing so. TIPP theory will be useful for all involved parties: IS providers are enabled to avoid privacy notice fallacies, public policy makers can give more details in their related guidance and laws published, and consumers can, ultimately, interact with IS that actually establish TIPP.