Miracle-dz/UniFAR-FTUs huggingface dataset:
Descriptive Kernel Convolution Network with Improved Random Walk Kernel

Graph kernels used to be the dominant approach to feature engineering for structured data, which are superseded by modern GNNs as the former lacks learnability. Recently, a suite of Kernel Convolution Networks (KCNs) successfully revitalized graph kernels by introducing learnability, which convolves input with learnable hidden graphs using a certain graph kernel. The random walk kernel (RWK) has been used as the default kernel in many KCNs, gaining increasing attention. In this paper, we first revisit the RWK and its current usage in KCNs, revealing several shortcomings of the existing designs, and propose an improved graph kernel RWK+, by introducing color-matching random walks and deriving its efficient computation. We then propose RWK+CN, a KCN that uses RWK+ as the core kernel to learn descriptive graph features with an unsupervised objective, which can not be achieved by GNNs. Further, by unrolling RWK+, we discover its connection with a regular GCN layer, and propose a novel GNN layer RWK+Conv. In the first part of experiments, we demonstrate the descriptive learning ability of RWK+CN with the improved random walk kernel RWK+ on unsupervised pattern mining tasks; in the second part, we show the effectiveness of RWK+ for a variety of KCN architectures and supervised graph learning tasks, and demonstrate the expressiveness of RWK+Conv layer, especially on the graph-level tasks. RWK+ and RWK+Conv adapt to various real-world applications, including web applications such as bot detection in a web-scale Twitter social network, and community classification in Reddit social interaction networks.
Mixed finite elements for the Gross-Pitaevskii eigenvalue problem: a priori error analysis and guaranteed lower energy bound

We establish an a priori error analysis for the lowest-order Raviart-Thomas finite element discretisation of the nonlinear Gross-Pitaevskii eigenvalue problem. Optimal convergence rates are obtained for the primal and dual variables as well as for the eigenvalue and energy approximations. In contrast to conformal approaches, which naturally imply upper energy bounds, the proposed mixed discretisation provides a guaranteed and asymptotically exact lower bound for the ground state energy. The theoretical results are illustrated by a series of numerical experiments.
Morphometry on the sphere: Cartesian and irreducible Minkowski tensors explained and implemented

Minkowski tensors are comprehensive shape descriptors that robustly capture n-point information in complex random geometries and that have already been extensively applied in the Euclidean plane. Here, we devise a novel framework for Minkowski tensors on the sphere. We first advance the theory by introducing irreducible Minkowski tensors, which avoid the redundancies of previous representations. We, moreover, generalize Minkowski sky maps to the sphere, i.e., a concept of local anisotropy, which easily adjusts to masked data. We demonstrate the power of our new procedure by applying it to simulations and real data of the Cosmic Microwave Background, finding an anomalous region close to the well-known Cold Spot. The accompanying open-source software, litchi, used to generate these maps from data in the HEALPix-format is made publicly available to facilitate broader integration of Minkowski maps in other fields, such as fluid demixing, porous structures, or geosciences more generally.
Exploring Group and Symmetry Principles in Large Language Models

Large Language Models (LLMs) have demonstrated impressive performance across a wide range of applications; however, assessing their reasoning capabilities remains a significant challenge. In this paper, we introduce a framework grounded in group and symmetry principles, which have played a crucial role in fields such as physics and mathematics, and offer another way to evaluate their capabilities. While the proposed framework is general, to showcase the benefits of employing these properties, we focus on arithmetic reasoning and investigate the performance of these models on four group properties: closure, identity, inverse, and associativity. Our findings reveal that LLMs studied in this work struggle to preserve group properties across different test regimes. In the closure test, we observe biases towards specific outputs and an abrupt degradation in their performance from 100% to 0% after a specific sequence length. They also perform poorly in the identity test, which represents adding irrelevant information in the context, and show sensitivity when subjected to inverse test, which examines the robustness of the model with respect to negation. In addition, we demonstrate that breaking down problems into smaller steps helps LLMs in the associativity test that we have conducted. To support these tests we have developed a synthetic dataset which will be released.
Premier-TACO is a Few-Shot Policy Learner: Pretraining Multitask Representation via Temporal Action-Driven Contrastive Loss

We present Premier-TACO, a multitask feature representation learning approach designed to improve few-shot policy learning efficiency in sequential decision-making tasks. Premier-TACO leverages a subset of multitask offline datasets for pretraining a general feature representation, which captures critical environmental dynamics and is fine-tuned using minimal expert demonstrations. It advances the temporal action contrastive learning (TACO) objective, known for state-of-the-art results in visual control tasks, by incorporating a novel negative example sampling strategy. This strategy is crucial in significantly boosting TACO's computational efficiency, making large-scale multitask offline pretraining feasible. Our extensive empirical evaluation in a diverse set of continuous control benchmarks including Deepmind Control Suite, MetaWorld, and LIBERO demonstrate Premier-TACO's effectiveness in pretraining visual representations, significantly enhancing few-shot imitation learning of novel tasks. Our code, pretraining data, as well as pretrained model checkpoints will be released at https://github.com/PremierTACO/premier-taco. Our project webpage is at https://premiertaco.github. io.
A self-supervised framework for learning whole slide representations

Whole slide imaging is fundamental to biomedical microscopy and computational pathology. However, whole slide images (WSIs) present a complex computer vision challenge due to their gigapixel size, diverse histopathologic features, spatial heterogeneity, and limited/absent data annotations. These challenges highlight that supervised training alone can result in suboptimal whole slide representations. Self-supervised representation learning can achieve high-quality WSI visual feature learning for downstream diagnostic tasks, such as cancer diagnosis or molecular genetic prediction. Here, we present a general self-supervised whole slide learning (S3L) framework for gigapixel-scale self-supervision of WSIs. S3L combines data transformation strategies from transformer-based vision and language modeling into a single unified framework to generate paired views for self-supervision. S3L leverages the inherent regional heterogeneity, histologic feature variability, and information redundancy within WSIs to learn high-quality whole-slide representations. We benchmark S3L visual representations on two diagnostic tasks for two biomedical microscopy modalities. S3L significantly outperforms WSI baselines for cancer diagnosis and genetic mutation prediction. Additionally, S3L achieves good performance using both in-domain and out-of-distribution patch encoders, demonstrating good flexibility and generalizability.
Diagram calculus for a type affine $C$ Temperley--Lieb algebra, II

In this paper, we present an infinite dimensional associative diagram algebra that satisfies the relations of the generalized Temperley--Lieb algebra having a basis indexed by the fully commutative elements (in the sense of Stembridge) of the Coxeter group of type affine $C$. Moreover, we provide an explicit description of a basis for the diagram algebra. In the sequel to this paper, we show that this diagrammatic representation is faithful. The results of this paper and its sequel will be used to construct a Jones-type trace on the Hecke algebra of type affine $C$, allowing us to non-recursively compute leading coefficients of certain Kazhdan--Lusztig polynomials.
Small-angle X-ray resonant magnetic scattering at the Co M$_{2,3}$ and L$_3$ edges observed with photoemission electron microscopy

X-ray magnetic circular dichroism is an efficient contrast mechanism allowing for a direct sensitivity to magnetization. Combined with an imaging technique such as photoemission electron microscopy, it has been successfully applied to high-resolution investigations of ferromagnetic thin films but also of three-dimensional systems thanks to the transmission-type contrast in their shadow. Our focus in this work is the wave-optics scattering pattern that can be observed near such a shadow's rim. Taking advantage of non-uniform magnetic states present in near-micron-size Co$_{1-x}$Gd$_x$ beads, we first show how X-ray resonant magnetic scattering affects the Fresnel diffraction at the Co L$_3$ edge. In order to confirm this observation, we then turn to the Co M$_{2,3}$ edges. There, we measure magnetic scattering patterns with a significantly increased spatial extent (due to the larger wavelength), despite the signal's weakness. The patterns' origin is supported by a comparison between our experimental data and a simple analytical model, then numerical simulations.
Veni, Vidi, Vici: Solving the Myriad of Challenges before Knowledge Graph Learning

Knowledge Graphs (KGs) have become increasingly common for representing large-scale linked data. However, their immense size has required graph learning systems to assist humans in analysis, interpretation, and pattern detection. While there have been promising results for researcher- and clinician-empowerment through a variety of KG learning systems, we identify four key deficiencies in state-of-the-art graph learning that simultaneously limit KG learning performance and diminish the ability of humans to interface optimally with these learning systems. These deficiencies are: 1) lack of expert knowledge integration, 2) instability to node degree extremity in the KG, 3) lack of consideration for uncertainty and relevance while learning, and 4) lack of explainability. Furthermore, we characterise state-of-the-art attempts to solve each of these problems and note that each attempt has largely been isolated from attempts to solve the other problems. Through a formalisation of these problems and a review of the literature that addresses them, we adopt the position that not only are deficiencies in these four key areas holding back human-KG empowerment, but that the divide-and-conquer approach to solving these problems as individual units rather than a whole is a significant barrier to the interface between humans and KG learning systems. We propose that it is only through integrated, holistic solutions to the limitations of KG learning systems that human and KG learning co-empowerment will be efficiently affected. We finally present our ”Veni, Vidi, Vici” framework that sets a roadmap for effectively and efficiently shifting to a holistic co-empowerment model in both the KG learning and the broader machine learning domain.
Phase-driven Domain Generalizable Learning for Nonstationary Time Series

Monitoring and recognizing patterns in continuous sensing data is crucial for many practical applications. These real-world time-series data are often nonstationary, characterized by varying statistical and spectral properties over time. This poses a significant challenge in developing learning models that can effectively generalize across different distributions. In this work, based on our observation that nonstationary statistics are intrinsically linked to the phase information, we propose a time-series learning framework, PhASER. It consists of three novel elements: 1) phase augmentation that diversifies non-stationarity while preserving discriminatory semantics, 2) separate feature encoding by viewing time-varying magnitude and phase as independent modalities, and 3) feature broadcasting by incorporating phase with a novel residual connection for inherent regularization to enhance distribution invariant learning. Upon extensive evaluation on 5 datasets from human activity recognition, sleep-stage classification, and gesture recognition against 10 state-of-the-art baseline methods, we demonstrate that PhASER consistently outperforms the best baselines by an average of 5% and up to 13% in some cases. Moreover, PhASER's principles can be applied broadly to boost the generalization ability of existing time series classification models.
TimEHR: Image-based Time Series Generation for Electronic Health Records

Time series in Electronic Health Records (EHRs) present unique challenges for generative models, such as irregular sampling, missing values, and high dimensionality. In this paper, we propose a novel generative adversarial network (GAN) model, TimEHR, to generate time series data from EHRs. In particular, TimEHR treats time series as images and is based on two conditional GANs. The first GAN generates missingness patterns, and the second GAN generates time series values based on the missingness pattern. Experimental results on three real-world EHR datasets show that TimEHR outperforms state-of-the-art methods in terms of fidelity, utility, and privacy metrics.
Distinguishing the Observational Signatures of Hot Spots Orbiting Reissner-Nordstr\"om Spacetime

This paper delves into observable signatures of hot spots orbiting Reissner-Nordstr\"om (RN) black holes and naked singularities. In a RN black hole case, we find two discernible lensing image tracks in time integrated images capturing a complete orbit of hot spots, and a image shadow within the critical curve where photons with a small impact parameter fall into the event horizon. Conversely, in RN singularities, additional image tracks can be found inner the critical curve, originating from photons reflected by the infinitely high effective potential well. Moreover, we found incomplete and converge tracks from the time integrated images of hot spot orbiting RN singularities lacking of a photon sphere. The presence of these additional image tracks exerts a significant influence on temporal magnitudes at their local maxima, allowing us to differentiate between RN black holes and RN naked singularities.
Asking the Right Question at the Right Time: Human and Model Uncertainty Guidance to Ask Clarification Questions

Clarification questions are an essential dialogue tool to signal misunderstanding, ambiguities, and under-specification in language use. While humans are able to resolve uncertainty by asking questions since childhood, modern dialogue systems struggle to generate effective questions. To make progress in this direction, in this work we take a collaborative dialogue task as a testbed and study how model uncertainty relates to human uncertainty—an as yet under-explored problem. We show that model uncertainty does not mirror human clarification-seeking behavior, which suggests that using human clarification questions as supervision for deciding when to ask may not be the most effective way to resolve model uncertainty. To address this issue, we propose an approach to generating clarification questions based on model uncertainty estimation, compare it to several alternatives, and show that it leads to significant improvements in terms of task success. Our findings highlight the importance of equipping dialogue systems with the ability to assess their own uncertainty and exploit in interaction.
DeAL: Decoding-time Alignment for Large Language Models

Large Language Models (LLMs) are nowadays expected to generate content aligned with human preferences. Current work focuses on alignment at model training time, through techniques such as Reinforcement Learning with Human Feedback (RLHF). However, it is unclear if such methods are an effective choice to teach alignment objectives to the model. First, the inability to incorporate multiple, custom rewards and reliance on a model developer's view of universal and static principles are key limitations. Second, the residual gaps in model training and the reliability of such approaches are also questionable (e.g. susceptibility to jail-breaking even after safety training). To address these, we propose DeAL, a framework that allows the user to customize reward functions and enables Decoding-time Alignment of LLMs (DeAL). At its core, we view decoding as a heuristic-guided search process and facilitate the use of a wide variety of alignment objectives. Our experiments with programmatic constraints such as keyword and length constraints (studied widely in the pre-LLM era) and abstract objectives such as harmlessness and helpfulness (proposed in the post-LLM era) show that we can DeAL with fine-grained trade-offs, improve adherence to alignment objectives, and address residual gaps in LLMs. Lastly, while DeAL can be effectively paired with RLHF and prompting techniques, its generality makes decoding slower, an optimization we leave for future work.
Multiple surface wetting events in the greater Meridiani Planum region, Mars: Evidence from valley networks within ancient cratered highlands

Morphological characterization of valley networks in three exposures of ancient cratered highlands (Nhc1) in the greater Meridiani Planum region yields insight into the Martian aqueous history. From our mapping, key regional differences are apparent in fine‐scale valley network attributes including morphologic type, planimetric form, density, and links to candidate paleolakes. This information, combined with crater retention age (inferred exposure age), provides new details on the relative timing and nature of aqueous processes in the region. Newly identified pitted‐type valley networks have morphological similarity to terrestrial pitted landforms in an evaporite setting. We interpret the pitted valley networks to reflect late‐stage groundwater processes concentrated along the former fluvial conduits. Evidence from this study indicates that localized reactivation of valley networks occurred during or after exhumation of eastern Nhc1 unit.
Physical, chemical and morphological evolution of incipient soot obtained from molecular dynamics simulation of acetylene pyrolysis

Incipient soot particles obtained from a series of reactive molecular dynamics simulations were studied to understand the evolution of physical, chemical, and morphological properties of incipient soot. Reactive molecular dynamics simulations of acetylene pyrolysis were performed using ReaxFF potential at 1350, 1500, 1650, and 1800 K. A total of 3324 incipient soot particles were extracted from the simulations at various stages of development. Features such as the number of carbon and hydrogen atoms, number of ring structures, mass, C/H ratio, radius of gyration, surface area, volume, atomic fractal dimension, and density were calculated for each particle. The calculated values of density and C/H ratio matched well with experimental values reported in the literature. Based on the calculated features, the particles were classified in two types: type 1 and type 2 particles. It was found that type 1 particles show significant morphological evolution while type 2 particles undergo chemical restructuring without any significant morphological change. The particle volume was found to be well-correlated with the number of carbon atoms in both type 1 and type 2 particle, whereas surface area was found to be correlated with the number of carbon atoms only for type 1 particles. A correlation matrix comparing the level of correlation between any two features for both type 1 and type 2 particle was created. Finally, based on the calculated statistics, a set of correlations among various physical and morphological parameters of incipient soot was proposed.
BarlowTwins-CXR : Enhancing Chest X-Ray abnormality localization in heterogeneous data with cross-domain self-supervised learning

Background: Chest X-ray imaging-based abnormality localization, essential in diagnosing various diseases, faces significant clinical challenges due to complex interpretations and the growing workload of radiologists. While recent advances in deep learning offer promising solutions, there is still a critical issue of domain inconsistency in cross-domain transfer learning, which hampers the efficiency and accuracy of diagnostic processes. This study aims to address the domain inconsistency problem and improve autonomic abnormality localization performance of heterogeneous chest X-ray image analysis, by developing a self-supervised learning strategy called"BarlwoTwins-CXR". Methods: We utilized two publicly available datasets: the NIH Chest X-ray Dataset and the VinDr-CXR. The BarlowTwins-CXR approach was conducted in a two-stage training process. Initially, self-supervised pre-training was performed using an adjusted Barlow Twins algorithm on the NIH dataset with a Resnet50 backbone pre-trained on ImageNet. This was followed by supervised fine-tuning on the VinDr-CXR dataset using Faster R-CNN with Feature Pyramid Network (FPN). Results: Our experiments showed a significant improvement in model performance with BarlowTwins-CXR. The approach achieved a 3% increase in mAP50 accuracy compared to traditional ImageNet pre-trained models. In addition, the Ablation CAM method revealed enhanced precision in localizing chest abnormalities. Conclusion: BarlowTwins-CXR significantly enhances the efficiency and accuracy of chest X-ray image-based abnormality localization, outperforming traditional transfer learning methods and effectively overcoming domain inconsistency in cross-domain scenarios. Our experiment results demonstrate the potential of using self-supervised learning to improve the generalizability of models in medical settings with limited amounts of heterogeneous data.
Quadratic lifespan for the sublinear $\alpha$-SQG sharp front problem

In this paper we consider the generalized surface quasi-geostrophic $\alpha$-SQG equations, in the"sublinear regime"$\alpha \in (0, 1)$ and we study the stability of vortex patches close to vortex discs. We shall prove that for regular, Sobolev initial vortex patches $\varepsilon$-close to a vortex disc, the solutions stay $\varepsilon$-close to a vortex disc for a time interval of order $O(\varepsilon^{- 2})$. The proof is based on a paradifferential Birkhoff normal form reduction, implemented in the case where the dispersion relation is sublinear.