Miracle-dz/UniFAR-FTUs huggingface dataset:
ChemicalX: A Deep Learning Library for Drug Pair Scoring

In this paper, we introduce ChemicalX, a PyTorch-based deep learning library designed for providing a range of state of the art models to solve the drug pair scoring task. The primary objective of the library is to make deep drug pair scoring models accessible to machine learning researchers and practitioners in a streamlined framework. The design of ChemicalX reuses existing high level model training utilities, geometric deep learning, and deep chemistry layers from the PyTorch ecosystem. Our system provides neural network layers, custom pair scoring architectures, data loaders, and batch iterators for end users. We showcase these features with example code snippets and case studies to highlight the characteristics of ChemicalX. A range of experiments on real world drug-drug interaction, polypharmacy side effect, and combination synergy prediction tasks demonstrate that the models available in ChemicalX are effective at solving the pair scoring task. Finally, we show that ChemicalX could be used to train and score machine learning models on large drug pair datasets with hundreds of thousands of compounds on commodity hardware.
Robust and fair work allocation

In today's digital world, interaction with online platforms is ubiquitous, and thus content moderation is important for protecting users from content that do not comply with pre-established community guidelines. Having a robust content moderation system throughout every stage of planning is particularly important. We study the short-term planning problem of allocating human content reviewers to different harmful content categories. We use tools from fair division and study the application of competitive equilibrium and leximin allocation rules. Furthermore, we incorporate, to the traditional Fisher market setup, novel aspects that are of practical importance. The first aspect is the forecasted workload of different content categories. We show how a formulation that is inspired by the celebrated Eisenberg-Gale program allows us to find an allocation that not only satisfies the forecasted workload, but also fairly allocates the remaining reviewing hours among all content categories. The resulting allocation is also robust as the additional allocation provides a guardrail in cases where the actual workload deviates from the predicted workload. The second practical consideration is time dependent allocation that is motivated by the fact that partners need scheduling guidance for the reviewers across days to achieve efficiency. To address the time component, we introduce new extensions of the various fair allocation approaches for the single-time period setting, and we show that many properties extend in essence, albeit with some modifications. Related to the time component, we additionally investigate how to satisfy markets' desire for smooth allocation (e.g., partners for content reviewers prefer an allocation that does not vary much from time to time, to minimize staffing switch). We demonstrate the performance of our proposed approaches through real-world data obtained from Meta.
Random Forests Weighted Local Fréchet Regression with Theoretical Guarantee

Statistical analysis is increasingly confronted with complex data from general metric spaces, such as symmetric positive deﬁnite matrix-valued data and probability distribution functions. [47] and [17] establish a general paradigm of Fréchet regression with complex metric space valued responses and Euclidean predictors. However, their proposed local Fréchet regression approach involves nonparametric kernel smoothing and suffers from the curse of dimensionality. To address this issue, we in this paper propose a novel random forests weighted local Fréchet regression paradigm. The main mechanism of our approach relies on the adaptive kernels generated by random forests. Our ﬁrst method utilizes these weights as the local average to solve the Fréchet mean, while the second method performs local linear Fréchet regression, making both methods locally adaptive. Our proposals signiﬁcantly improve existing Fréchet regression methods. Based on the theory of inﬁ-nite order U-processes and inﬁnite order M m n -estimator, we establish the consistency, rate of convergence, and asymptotic normality for our proposed random forests weighted Fréchet regression estimator, which covers the current large sample theory of random forests with Euclidean responses as a special case. Numerical studies show the superiority of our proposed two methods for Fréchet regression with several commonly encountered types of responses such as probability distribution functions, symmetric positive deﬁnite matrices, and sphere data. The practical merits of our proposals are also demonstrated through the application to the human mortality distribution data.
Measuring disentangled generative spatio-temporal representation

Disentangled representation learning oﬀers useful properties such as dimension reduction and interpretability, which are essential to modern deep learning approaches. Although deep learning techniques have been widely applied to spatio-temporal data mining, there has been little attention to further disentangle the latent features and understanding their contribution to the model performance, particularly their mutual information and correlation across features. In this study, we adopt two state-of-the-art disentangled representation learning methods and apply them to three large-scale public spatio-temporal datasets. To evaluate their performance, we propose an internal evaluation metric focusing on the degree of correlations among latent variables of the learned representations and the prediction performance of the downstream tasks. Empirical results show that our modiﬁed method can learn disentangled representations that achieve the same level of performance as existing state-of-the-art ST deep learning methods in a spatio-temporal sequence forecasting problem. Addition-ally, we ﬁnd that our methods can be used to discover real-world spatial-temporal semantics to describe the variables in the learned representation.
Characterizations of Adjoint Sobolev Embedding Operators for Inverse Problems

We consider the Sobolev embedding operator Es : H s(Ω) → L2(Ω) and its role in the solution of inverse problems. In particular, we collect various properties and representations of its adjoint operator E∗ s , which is a common component in both iterative and variational regularization methods. These include e.g. variational representations and connections to boundary value problems, Fourier and wavelet representations, as well as connections to spatial filters. While many of these results are already known to researchers from different fields, an overview or reference work is still missing. Hence, in this paper we aim to fill this gap, providing a collection of representations of E∗ s which can serve both as a reference as well as a useful guide for its efficient numerical implementation in practice.
EXCESS workshop: Descriptions of rising low-energy spectra

Many low-threshold experiments observe sharply rising event rates of yet unknown origins below a few hundred eV, and larger than expected from known backgrounds. Due to the significant impact of this excess on the dark matter or neutrino sensitivity of these experiments, a collective effort has been started to share the knowledge about the individual observations. For this, the EXCESS Workshop was initiated. In its first iteration in June 2021, ten rare event search collaborations contributed to this initiative via talks and discussions. The contributing collaborations were CONNIE, CRESST, DAMIC, EDELWEISS, MINER, NEWS-G, NUCLEUS, RICOCHET, SENSEI and SuperCDMS. They presented data about their observed energy spectra and known backgrounds together with details about the respective measurements. In this paper, we summarize the presented information and give a comprehensive overview of the similarities and differences between the distinct measurements. The provided data is furthermore publicly available on the workshop's data repository together with a plotting tool for visualization.
Unsupervised Time-Series Representation Learning with Iterative Bilinear Temporal-Spectral Fusion

Unsupervised/self-supervised time series representation learning is a challenging problem because of its complex dynamics and sparse annotations. Existing works mainly adopt the framework of contrastive learning with the time-based augmentation techniques to sample positives and negatives for contrastive training. Nevertheless, they mostly use segment-level augmentation derived from time slicing, which may bring about sampling bias and incorrect optimization with false negatives due to the loss of global context. Besides, they all pay no attention to incorporate the spectral information in feature representation. In this paper, we propose a unified framework, namely Bilinear Temporal-Spectral Fusion (BTSF). Specifically, we firstly utilize the instance-level augmentation with a simple dropout on the entire time series for maximally capturing long-term dependencies. We devise a novel iterative bilinear temporal-spectral fusion to explicitly encode the affinities of abundant time-frequency pairs, and iteratively refines representations in a fusion-and-squeeze manner with Spectrum-to-Time (S2T) and Time-to-Spectrum (T2S) Aggregation modules. We firstly conducts downstream evaluations on three major tasks for time series including classification, forecasting and anomaly detection. Experimental results shows that our BTSF consistently significantly outperforms the state-of-the-art methods.
Spectral dimensions of Krein-Feller operators in higher dimensions

We study the spectral dimensions of Krein-Feller operators for arbitrary for arbitrary finite Borel measures $\nu$ on the $d$-dimensional unit cube ($d\geq2$) via a form approach. We make use of the spectral partition function of $\nu$ as introduced in [Kesseb\"ohmer and Niemann, Exact asymptotic order for adaptive approximations. 2023, arXiv:2312.16644] and, assuming that the lower $\infty$-dimension of $\nu$ exceeds $d-2$, we identify the upper Neumann spectral dimension as the unique zero of the spectral partition function, thus revealing the intrinsic connection of these spectral and fractal-geometric quantities. We show that if the lower $\infty$-dimension of $\nu$ is strictly less than $d-2$, the form approach breaks down. Examples are given for the critical case, that is the lower $\infty$-dimension of $\nu$ equals $d-2$. We provide additional regularity assumptions on the spectral partition function, guaranteeing that the Neumann spectral dimension exists and coincides with the Dirichlet spectral dimension. Several prominent examples previously treated in the literature are provided, namely absolutely continuous measures and more generally Ahlfors-David regular measures, and examples not previously treated in the literature, namely self-conformal measures with or without overlaps, for which we show that both the Dirichlet and Neumann spectral dimensions exist and how they can be obtained from the $L^{q}$-spectrum of the measures. We demonstrate how our approach can be used to obtain upper and lower asymptotic spectral bounds for the case of Ahlfors-David regular measures. Moreover, we provide sharp bounds for the upper Neumann spectral dimension in terms of the upper Minkowski dimension of the support of $\nu$ and its lower $\infty$-dimension. Finally, we give an example for which the spectral dimension does not exist.
Lebesgue Induction and Tonelli's Theorem in Coq

Lebesgue integration is a well-known mathematical tool, used for instance in probability theory, real analysis, and numerical mathematics. Thus its formalization in a proof assistant is to be designed to fit different goals and projects. Once Lebesgue integral is formally defined and the first lemmas are proved, the question of the convenience of the formalization naturally arises. To check it, a useful extension is the Tonelli theorem, stating that the (double) integral of a nonnegative measurable function of two variables can be computed by iterated integrals, and allowing to switch the order of integration. Therefore, we need to define and prove results on product spaces, hoping that they can easily derive from the existing ones on a single space. This article describes the formal definition and proof in Coq of product $\sigma$-algebras, product measures and their uniqueness, the construction of iterated integrals, up to the Tonelli theorem. We also advertise the \emph{Lebesgue induction principle} provided by an inductive type for {\nonnegative} measurable functions.
Generalization Bounds via Convex Analysis

Since the celebrated works of Russo and Zou (2016, 2019) and Xu and Raginsky (2017), it has been well known that the generalization error of supervised learning algorithms can be bounded in terms of the mutual information between their input and the output, given that the loss of any ﬁxed hypothesis has a subgaussian tail. In this work, we generalize this result beyond the standard choice of Shannon’s mutual information to measure the dependence between the input and the output. Our main result shows that it is indeed possible to replace the mutual information by any strongly convex function of the joint input-output distribution, with the subgaussianity condition on the losses replaced by a bound on an appropriately chosen norm capturing the geometry of the dependence measure. This allows us to derive a range of generalization bounds that are either entirely new or strengthen previously known ones. Examples include bounds stated in terms of p -norm divergences and the Wasserstein-2 distance, which are respectively applicable for heavy-tailed loss distributions and highly smooth loss functions. Our analysis is entirely based on elementary tools from convex analysis by tracking the growth of a potential function associated with the dependence measure and the loss function.
Deconstructing the Inductive Biases of Hamiltonian Neural Networks

Physics-inspired neural networks (NNs), such as Hamiltonian or Lagrangian NNs, dramatically outperform other learned dynamics models by leveraging strong inductive biases. These models, however, are challenging to apply to many real world systems, such as those that don't conserve energy or contain contacts, a common setting for robotics and reinforcement learning. In this paper, we examine the inductive biases that make physics-inspired models successful in practice. We show that, contrary to conventional wisdom, the improved generalization of HNNs is the result of modeling acceleration directly and avoiding artificial complexity from the coordinate system, rather than symplectic structure or energy conservation. We show that by relaxing the inductive biases of these models, we can match or exceed performance on energy-conserving systems while dramatically improving performance on practical, non-conservative systems. We extend this approach to constructing transition models for common Mujoco environments, showing that our model can appropriately balance inductive biases with the flexibility required for model-based control.
Distilling Hypernymy Relations from Language Models: On the Effectiveness of Zero-Shot Taxonomy Induction

In this paper, we analyze zero-shot taxonomy learning methods which are based on distilling knowledge from language models via prompting and sentence scoring. We show that, despite their simplicity, these methods outperform some supervised strategies and are competitive with the current state-of-the-art under adequate conditions. We also show that statistical and linguistic properties of prompts dictate downstream performance.
REvolveR: Continuous Evolutionary Models for Robot-to-robot Policy Transfer

A popular paradigm in robotic learning is to train a policy from scratch for every new robot. This is not only inefficient but also often impractical for complex robots. In this work, we consider the problem of transferring a policy across two different robots with significantly different parameters such as kinematics and morphology. Existing approaches that train a new policy by matching the action or state transition distribution, including imitation learning methods, fail due to optimal action and/or state distribution being mismatched in different robots. In this paper, we propose a novel method named $REvolveR$ of using continuous evolutionary models for robotic policy transfer implemented in a physics simulator. We interpolate between the source robot and the target robot by finding a continuous evolutionary change of robot parameters. An expert policy on the source robot is transferred through training on a sequence of intermediate robots that gradually evolve into the target robot. Experiments on a physics simulator show that the proposed continuous evolutionary model can effectively transfer the policy across robots and achieve superior sample efficiency on new robots. The proposed method is especially advantageous in sparse reward settings where exploration can be significantly reduced. Code is released at https://github.com/xingyul/revolver.
Networks and Identity Drive Geographic Properties of the Diffusion of Linguistic Innovation

Adoption of cultural innovation (e.g., music, beliefs, language) is often geographically correlated, with adopters largely residing within the boundaries of relatively few well-studied, socially significant areas. These cultural regions are often hypothesized to be the result of either (i) identity performance driving the adoption of cultural innovation, or (ii) homophily in the networks underlying diffusion. In this study, we show that demographic identity and network topology are both required to model the diffusion of innovation, as they play complementary roles in producing its spatial properties. We develop an agent-based model of cultural adoption, and validate geographic patterns of transmission in our model against a novel dataset of innovative words that we identify from a 10% sample of Twitter. Using our model, we are able to directly compare a combined network + identity model of diffusion to simulated network-only and identity-only counterfactuals—allowing us to test the separate and combined roles of network and identity. While social scientists often treat either network or identity as the core social structure in modeling culture change, we show that key geographic properties of diffusion actually depend on both factors as each one influences different mechanisms of diffusion. Specifically, the network principally drives spread among urban counties via weak-tie diffusion, while identity plays a disproportionate role in transmission among rural counties via strong-tie diffusion. Diffusion between urban and rural areas, a key component in innovation diffusing nationally, requires both network and identity. Our work suggests that models must integrate both factors in order to understand and reproduce the adoption of innovation.
An introduction to symmetric edge graphs

Given a graph X, we define the symmetric edge graph of X, denoted as γ(X) and study various properties of γ(X) with respect to X. If X ′′ denotes the Kronecker double cover of X, we characterize all graphs which satisfy γ(X) = L(X ′′), where L(Y ) is the line graph of Y.
Predicting Human Similarity Judgments Using Large Language Models

Similarity judgments provide a well-established method for accessing mental representations, with applications in psychology, neuroscience and machine learning. However, collecting similarity judgments can be prohibitively expensive for naturalistic datasets as the number of comparisons grows quadratically in the number of stimuli. One way to tackle this problem is to construct approximation procedures that rely on more accessible proxies for predicting similarity. Here we leverage recent advances in language models and online recruitment, proposing an efficient domain-general procedure for predicting human similarity judgments based on text descriptions. Intuitively, similar stimuli are likely to evoke similar descriptions, allowing us to use description similarity to predict pairwise similarity judgments. Crucially, the number of descriptions required grows only linearly with the number of stimuli, drastically reducing the amount of data required. We test this procedure on six datasets of naturalistic images and show that our models outperform previous approaches based on visual information.
Spectral Propagation Graph Network for Few-shot Time Series Classification

Few-shot Time Series Classification (few-shot TSC) is a challenging problem in time series analysis. It is more difficult to classify when time series of the same class are not completely consistent in spectral domain or time series of different classes are partly consistent in spectral domain. To address this problem, we propose a novel method named Spectral Propagation Graph Network (SPGN) to explicitly model and propagate the spectrum-wise relations between different time series with graph network. To the best of our knowledge, SPGN is the first to utilize spectral comparisons in different intervals and involve spectral propagation across all time series with graph networks for few-shot TSC. SPGN first uses bandpass filter to expand time series in spectral domain for calculating spectrum-wise relations between time series. Equipped with graph networks, SPGN then integrates spectral relations with label information to make spectral propagation. The further study conveys the bi-directional effect between spectral relations acquisition and spectral propagation. We conduct extensive experiments on few-shot TSC benchmarks. SPGN outperforms state-of-the-art results by a large margin in $4\% \sim 13\%$. Moreover, SPGN surpasses them by around $12\%$ and $9\%$ under cross-domain and cross-way settings respectively.
Learning Latent Causal Dynamics

One critical challenge of time-series modeling is how to learn and quickly correct the model under unknown distribution shifts. In this work, we propose a principled framework, called LiLY, to first recover time-delayed latent causal variables and identify their relations from measured temporal data under different distribution shifts. The correction step is then formulated as learning the low-dimensional change factors with a few samples from the new environment, leveraging the identified causal structure. Specifically, the framework factorizes unknown distribution shifts into transition distribution changes caused by fixed dynamics and time-varying latent causal relations, and by global changes in observation. We establish the identifiability theories of nonparametric latent causal dynamics from their nonlinear mixtures under fixed dynamics and under changes. Through experiments, we show that time-delayed latent causal influences are reliably identified from observed variables under different distribution changes. By exploiting this modular representation of changes, we can efficiently learn to correct the model under unknown distribution shifts with only a few samples.