Miracle-dz/UniFAR-FTUs huggingface dataset:
D-CIPHER: Discovery of Closed-form Partial Differential Equations

Closed-form differential equations, including partial differential equations and higher-order ordinary differential equations, are one of the most important tools used by scientists to model and better understand natural phenomena. Discovering these equations directly from data is challenging because it requires modeling relationships between various derivatives that are not observed in the data (equation-data mismatch) and it involves searching across a huge space of possible equations. Current approaches make strong assumptions about the form of the equation and thus fail to discover many well-known systems. Moreover, many of them resolve the equation-data mismatch by estimating the derivatives, which makes them inadequate for noisy and infrequently sampled systems. To this end, we propose D-CIPHER, which is robust to measurement artifacts and can uncover a new and very general class of differential equations. We further design a novel optimization procedure, CoLLie, to help D-CIPHER search through this class efficiently. Finally, we demonstrate empirically that it can discover many well-known equations that are beyond the capabilities of current methods.
Zero-shot Transfer Learning within a Heterogeneous Graph via Knowledge Transfer Networks

Data continuously emitted from industrial ecosystems such as social or e-commerce platforms are commonly represented as heterogeneous graphs (HG) composed of multiple node/edge types. State-of-the-art graph learning methods for HGs known as heterogeneous graph neural networks (HGNNs) are applied to learn deep context-informed node representations. However, many HG datasets from industrial applications suffer from label imbalance between node types. As there is no direct way to learn using labels rooted at different node types, HGNNs have been applied to only a few node types with abundant labels. We propose a zero-shot transfer learning module for HGNNs called a Knowledge Transfer Network (KTN) that transfers knowledge from label-abundant node types to zero-labeled node types through rich relational information given in the HG. KTN is derived from the theoretical relationship, which we introduce in this work, between distinct feature extractors for each node type given in an HGNN model. KTN improves performance of 6 different types of HGNN models by up to 960% for inference on zero-labeled node types and outperforms state-of-the-art transfer learning baselines by up to 73% across 18 different transfer learning tasks on HGs.
On the Paradox of Certified Training

Certified defenses based on convex relaxations are an established technique for training provably robust models. The key component is the choice of relaxation, varying from simple intervals to tight polyhedra. Counterintuitively, loose interval-based training often leads to higher certified robustness than what can be achieved with tighter relaxations, which is a well-known but poorly understood paradox. While recent works introduced various improvements aiming to circumvent this issue in practice, the fundamental problem of training models with high certified robustness remains unsolved. In this work, we investigate the underlying reasons behind the paradox and identify two key properties of relaxations, beyond tightness, that impact certified training dynamics: continuity and sensitivity. Our extensive experimental evaluation with a number of popular convex relaxations provides strong evidence that these factors can explain the drop in certified robustness observed for tighter relaxations. We also systematically explore modifications of existing relaxations and discover that improving unfavorable properties is challenging, as such attempts often harm other properties, revealing a complex tradeoff. Our findings represent an important first step towards understanding the intricate optimization challenges involved in certified training.
When Do Flat Minima Optimizers Work?

Recently, flat-minima optimizers, which seek to find parameters in low-loss neighborhoods, have been shown to improve a neural network's generalization performance over stochastic and adaptive gradient-based optimizers. Two methods have received significant attention due to their scalability: 1. Stochastic Weight Averaging (SWA), and 2. Sharpness-Aware Minimization (SAM). However, there has been limited investigation into their properties and no systematic benchmarking of them across different domains. We fill this gap here by comparing the loss surfaces of the models trained with each method and through broad benchmarking across computer vision, natural language processing, and graph representation learning tasks. We discover several surprising findings from these results, which we hope will help researchers further improve deep learning optimizers, and practitioners identify the right optimizer for their problem.
Contextual Combinatorial Bandits with Changing Action Sets via Gaussian Processes

We consider a contextual bandit problem with a combinatorial action set and time-varying base arm availability. At the beginning of each round, the agent observes the set of available base arms and their contexts and then selects an action that is a feasible subset of the set of available base arms to maximize its cumulative reward in the long run. We assume that the mean outcomes of base arms are samples from a Gaussian Process (GP) indexed by the context set X , and the expected reward is Lipschitz continuous in expected base arm outcomes. For this setup, we propose an algorithm called Optimistic Combinatorial Learning and Optimization with Kernel Upper Conﬁdence Bounds (O’CLOK-UCB) and prove that it incurs ˜ O ( (cid:112) λ ∗ ( K ) KTγ T ) regret with high probability, where γ T is the maximum information gain associated with the set of base arm contexts that appeared in the ﬁrst T rounds, K is the maximum cardinality of any feasible action over all rounds and λ ∗ ( K ) is the maximum eigenvalue of all covariance matrices of selected actions up to time T , which is a function of K . To dramati-cally speed up the algorithm, we also propose a variant of O’CLOK-UCB that uses sparse GPs. Finally, we experimentally show that both algorithms exploit inter-base arm outcome correlation and vastly outperform the previous state-of-the-art UCB-based algorithms in realistic setups.
Deep Transformers Thirst for Comprehensive-Frequency Data

Current researches indicate that inductive bias (IB) can improve Vision Transformer (ViT) performance. However, they introduce a pyramid structure concurrently to counteract the incremental FLOPs and parameters caused by introducing IB. This structure destroys the unification of computer vision and natural language processing (NLP) and complicates the model. We study an NLP model called LSRA, which introduces IB with a pyramid-free structure. We analyze why it outperforms ViT, discovering that introducing IB increases the share of high-frequency data in each layer, giving"attention"to more information. As a result, the heads notice more diverse information, showing better performance. To further explore the potential of transformers, we propose EIT, which Efficiently introduces IB to ViT with a novel decreasing convolutional structure under a pyramid-free structure. EIT achieves competitive performance with the state-of-the-art (SOTA) methods on ImageNet-1K and achieves SOTA performance over the same scale models which have the pyramid-free structure.
The Eigenlearning Framework: A Conservation Law Perspective on Kernel Regression and Wide Neural Networks

We derive simple closed-form estimates for the test risk and other generalization metrics of kernel ridge regression (KRR). Relative to prior work, our derivations are greatly simplified and our final expressions are more readily interpreted. These improvements are enabled by our identification of a sharp conservation law which limits the ability of KRR to learn any orthonormal basis of functions. Test risk and other objects of interest are expressed transparently in terms of our conserved quantity evaluated in the kernel eigenbasis. We use our improved framework to: i) provide a theoretical explanation for the"deep bootstrap"of Nakkiran et al (2020), ii) generalize a previous result regarding the hardness of the classic parity problem, iii) fashion a theoretical tool for the study of adversarial robustness, and iv) draw a tight analogy between KRR and a well-studied system in statistical physics.
Sur le théorème de Schneider-Lang

Le theoreme de Schneider-Lang est un critere classique de transcendance pour des nombres complexes. Il dit que des fonctions meromorphes d'ordre fini, verifiant une equation differentielle polynomiale a coefficients dans un corps de nombres et algebriquement independantes ne peuvent prendre simultanement des valeurs dans ce corps de nombres qu'en un nombre fini de points. Dans cette these, nous demontrons des generalisations geometriques de ce critere, valables sur le corps des nombres complexes ou sur un corps p-adique. Ces resultats s'appuient sur des lemmes de Schwarz adaptes, que nous avons etablis. En dimension 1, nous demontrons un theoreme concernant des sous-schemas formels admettant une uniformisation par une courbe algebrique affine. En dimension superieure, notre theoreme s'applique a des sous-schemas formels admettant une uniformisation par un produit d'ouverts de la droite affine, sous l'hypothese supplementaire que l'ensemble des points etudies est un produit cartesien. Les demonstrations de ces resultats reposent sur la methode des pentes developpee par J.-B. Bost et utilisent le langage de la geometrie d'Arakelov.
Microscopic theory of polariton lasing via vibronically assisted scattering

Polariton lasing has recently been observed in strongly coupled crystalline anthracene microcavities. A simple model is developed describing the onset of the nonlinear threshold based on a master equation including the relevant relaxation processes and employing realistic material parameters. The mechanism governing the buildup of the polariton population, namely, bosonic stimulated scattering from the exciton reservoir via a vibronically assisted process, is characterized and its efficiency calculated on the basis of a microscopic theory. The role of polariton-polariton bimolecular quenching is identified and temperature-dependent effects are discussed.
Four loop renormalization of the Gross-Neveu model

We renormalize the $SU(N)$ Gross-Neveu model in the modified minimal subtraction scheme at four loops and determine the $\ensuremath{\beta}$ function at this order. The theory ceases to be multiplicatively renormalizable when dimensionally regularized due to the generation of evanescent 4-Fermi operators. The first of these appears at three loops and we correctly take their effect into account in deriving the renormalization group functions. We use the results to provide estimates of critical exponents relevant to phase transitions in graphene.
THE SPIN OF THE BLACK HOLE 4U 1543-47

We present a new analysis of Rossi X-Ray Timing Explorer observations of the 2002 outburst of the transient X-ray nova 4U 1543-47. We focus on observations in the high/soft state, and attempt to measure the “spin” of the black hole by simultaneously fitting the thermal disk continuum and by modeling the broadened iron k-shell emission lines and additional blurred reflection features. Previous works have found that use of these methods individually returns contradictory values for the dimensionless spin parameter a* = cJ/GM2. We find that when used in conjunction with each other, a moderate spin is obtained () that is actually consistent with both other values within errors. We discuss limitations of our analysis, systematic uncertainties, and implications of this measurement, and compare our result to those previously claimed for 4U 1543-47.
Plasma production with rotating ion cyclotron waves excited by Nagoya Type-III antennas in RFC-XX

A method of plasma production by ion cyclotron wave heating has been developed in RFC-XX. Nagoya Type-III antennas were used for wave excitation, and gas was supplied through a gas box. The effect of the rotating field excitation on plasma production and heating was investigated. In the m = − 1 rotational mode (rotation in the direction of ion cyclotron motion), the plasma density profile is flat within the gas box bore with a line integrated density nℓ = 3 × 1013cm−2, and the ion temperature is Ti ≈ 150 eV. For the m = +1 mode, a high density plasma was obtained with a different profile having a peak at the centre. In this mode, the line integrated density is nℓ = 3 × 1014cm−2, with the peak density n(0) = 7 × 1013cm−3.
The impact of turbulence and magnetic field orientation on star-forming filaments

We present simulations of collapsing filaments studying the impact of turbulence and magnetic field morphologies on their evolution and star formation properties. We vary the mass per unit length of the filaments as well as the orientation of the magnetic field with respect to the major axis. We find that the filaments, which have no or a perpendicular magnetic field, typically reveal a smaller width than the universal width of 0.1 pc proposed by e.g. Arzoumanian et al. 2011. We show that this also holds in the presence of supersonic turbulence and that accretion driven turbulence is too weak to stabilize the filaments along their radial direction. On the other hand, we find that a magnetic field that is parallel to the major axis can stabilize the filament against radial collapse resulting in widths of 0.1 pc. Furthermore, depending on the filament mass and magnetic field configuration, gravitational collapse and fragmentation in filaments occurs either in an edge-on way, uniformly distributed across the entire length, or in a mixed way. In the presence of initially moderate density perturbations, a centralized collapse towards a common gravitational centre occurs. Our simulations can thus reproduce different modes of fragmentation observed recently in star forming filaments. Moreover, we find that turbulent motions influence the distance between individual fragments along the filament, which does not always match the results of a Jeans analysis.
Topological insulators on the Lieb and perovskite lattices

Electrons hopping on the sites of a two-dimensional Lieb lattice and three-dimensional edge-centered cubic (perovskite) lattice are shown to form topologically nontrivial insulating phases when spin-orbit coupling is introduced. These simple models on lattices with cubic symmetry show a Dirac-type structure in the excitation spectrum but with the unusual feature that there is a dispersionless band through the center of the spectrum and only a single Dirac cone per Brillouin zone.
THE MASS–Lx RELATION FOR MODERATE LUMINOSITY X-RAY CLUSTERS

We present measurements of the masses of a sample of 25 moderate X-ray luminosity clusters of galaxies from the 160 square degree ROSAT survey. The masses were obtained from a weak-lensing analysis of deep F814W images obtained using the Advanced Camera for Surveys. We present an accurate empirical correction for the effect of charge transfer (in)efficiency on the shapes of faint galaxies. A significant lensing signal is detected around most of the clusters. The lensing mass correlates tightly with the cluster richness. We measured the intrinsic scatter in the scaling relation between M2500 and LX to be . The best-fit power-law slope and normalization are found to be α = 0.68 ± 0.07 and MX = (1.2 ± 0.12) × h−1701014 M☉ (for LX = 2 × 1044h−270 erg s−1). These results agree well with a number of recent studies, but the normalization is lower compared to the study of Rykoff et al. One explanation for this difference may be the fact that (sub)structures projected along the line of sight boost both the galaxy counts and the lensing mass. Such superpositions lead to an increased mass at a given LX when clusters are binned by richness.
Use of High-Plex Data Reveals Novel Insights into the Tumour Microenvironment of Clear Cell Renal Cell Carcinoma

Although Immune Checkpoint Inhibitors (ICIs) have significantly improved Clear Cell Renal Cell Carcinoma (ccRCC) prognosis, about one third of patients experience recurrence. Current prognostic algorithms like the Leibovich Score (LS) rely on morphological features manually assessed by pathologists, and are therefore subject to bias. Moreover, these tools do not consider the heterogeneous molecular milieu present in the Tumour Microenvironment (TME), which may have prognostic value. We systematically developed a semi-automated method to investigate 62 markers and their combinations in 150 primary ccRCCs using multiplex Immunofluorescence (mIF), NanoString GeoMx (R) Digital Spatial Profiling (DSP) and Artificial Intelligence (AI)-assisted image analysis in order to find novel prognostic signatures and investigate their spatial relationship. We found that coexpression of Cancer Stem Cell (CSC) and Epithelial-to-Mesenchymal Transition (EMT) markers such as OCT4 and ZEB1 are indicative of poor outcome. OCT4 and the immune markers CD8, CD34 and CD163 significantly stratified patients at intermediate LS. Furthermore, augmenting the LS with OCT4 and CD34 improved patient stratification by outcome. Our results support the hypothesis that combining molecular markers has prognostic value and can be integrated with morphological features to improve risk stratification and personalised therapy. To conclude, GeoMx (R) DSP and AI image analysis are complementary tools providing high multiplexing capability required to investigate the TME of ccRCC, while reducing observer bias.
Menthol augmented niosomes for enhanced intestinal absorption of lopinavir

Abstract Lopinavir is effective in treatment of HIV infection but experiences low oral bioavailability due to poor solubility, pre-systemic metabolism, and P-gp intestinal efflux. Co-processing with menthol enhanced its dissolution and intestinal permeability. Niosomes comprising Span 60, cholesterol, and poloxamer 407 were formulated in absence and presence of menthol. These were evaluated for size, morphology, entrapment efficiency (EE%), lopinavir release, and intestinal absorption. The later employed in situ rabbit intestinal absorption model. Niosomes were spherical with vesicle size of 140.2 ± 23 and 148.2 ± 27 nm for standard and menthol containing niosomes, respectively. The EE% values were 94.4% and 96.3% for both formulations, respectively. Niosomes underwent slow release during the time course of absorption with menthol hastening lopinavir release, but the release did not exceed 9%. Niosmoal encapsulation enhanced lopinavir intestinal absorption compared with drug solution. This was reflected from the fraction absorbed from duodenum, which was 24.15%, 73.09%, and 83.23% for solution, standard niosomes and menthol containing vesicles, respectively. These values were 34.32%, 80.8%, and 86.56% for the same formulations in case of jejuno-ileum. Lopinavir absorption from niosomes did not depend on release supporting intact vesicle absorption. The study introduced menthol containing niosomes as carriers for enhanced lopinavir intestinal absorption. Graphical Abstract
Entanglement sharing through noisy qubit channels: One-shot optimal singlet fraction

Maximally entangled states--a resource for quantum information processing--can only be shared through noiseless quantum channels, whereas in practice channels are noisy. Here we ask: Given a noisy quantum channel, what is the maximum attainable purity (measured by singlet fraction) of shared entanglement for single channel use and local trace preserving operations? We find an exact formula of the maximum singlet fraction attainable for a qubit channel and give an explicit protocol to achieve the optimal value. The protocol distinguishes between unital and nonunital channels and requires no local post-processing. In particular, the optimal singlet fraction is achieved by transmitting part of an appropriate pure entangled state, which is maximally entangled if and only if the channel is unital. A linear function of the optimal singlet fraction is also shown to be an upper bound on the distillable entanglement of the mixed state dual to the channel.
Impact of prenatal cannabis exposure on functional connectivity of the salience network in children

Cannabis use among pregnant people has increased over the past decade. This is of concern as prenatal cannabis exposure (PCE) is associated with cognitive, motor, and social deficits among offspring. Here, we examined resting‐state functional connectivity (rsFC) of the salience network (SN)—a core neurocognitive network that integrates emotional and sensory information—in children with (vs. without) PCE. Using neuroimaging and developmental history data collected from 10,719 children (M ± SD = 9.92 ± 0.62 years; 47.9% female) from the Adolescent Brain Cognitive Development study, we assessed the impact of parent‐reported PCE (before or after knowledge of pregnancy) on rsFC within and between the SN and five other core neurocognitive networks. We also evaluated whether SN rsFC mediated the association between PCE and child psychopathology. Results showed that PCE before (but not after) knowledge of pregnancy was associated with lower SN‐ventral attention network (VAN) rsFC. Furthermore, psychotic‐like experiences mediated the association between PCE and SN‐VAN rsFC, and reversal of the model was also significant, such that SN‐VAN rsFC mediated the association between PCE and psychotic‐like symptoms. However, these mediation effects were no longer significant after the inclusion of covariates. Taken together, these findings suggest that developmental alterations in SN‐VAN interactions may explain the previously reported association between PCE and elevated risk of child psychopathology.
Upper cortical layer–driven network impairment in schizophrenia

Schizophrenia is one of the most widespread and complex mental disorders. To characterize the impact of schizophrenia, we performed single-nucleus RNA sequencing (snRNA-seq) of >220,000 neurons from the dorsolateral prefrontal cortex of patients with schizophrenia and matched controls. In addition, >115,000 neurons were analyzed topographically by immunohistochemistry. Compositional analysis of snRNA-seq data revealed a reduction in abundance of GABAergic neurons and a concomitant increase in principal neurons, most pronounced for upper cortical layer subtypes, which was substantiated by histological analysis. Many neuronal subtypes showed extensive transcriptomic changes, the most marked in upper-layer GABAergic neurons, including down-regulation in energy metabolism and up-regulation in neurotransmission. Transcription factor network analysis demonstrated a developmental origin of transcriptomic changes. Last, Visium spatial transcriptomics further corroborated upper-layer neuron vulnerability in schizophrenia. Overall, our results point toward general network impairment within upper cortical layers as a core substrate associated with schizophrenia symptomatology.