Miracle-dz/UniFAR-FTUs huggingface dataset:
Electron-phonon relaxation in periodic granular films

We study the electron-phonon relaxation in the model of a granular metal film, where the grains are formed by regularly arranged potential barriers of arbitrary transparency. The relaxation rate of Debye acoustic phonons is calculated taking into account two mechanisms of electron-phonon scattering: the standard Frohlich interaction of the lattice deformation with the electron density and the interaction mediated by the displacement of grain boundaries dragged by the lattice vibration. At lowest temperatures, the electron-phonon cooling power follows the power-law temperature dependence typical for clean systems, but with the prefactor growing as the transparency of the grain boundaries decreases.
Exploring Durham University Physics exams with Large Language Models

The emergence of advanced Natural Language Processing (NLP) models like ChatGPT has raised concerns among universities regarding AI-driven exam completion. This paper provides a comprehensive evaluation of the proficiency of GPT-4 and GPT-3.5 in answering a set of 42 exam papers derived from 10 distinct physics courses, administered at Durham University over the span of 2018 to 2022, totalling 593 questions and 2504 available marks. These exams, spanning both undergraduate and postgraduate levels, include traditional pre-COVID and adaptive COVID-era formats. Questions from the years 2018-2020 were designed for pre-COVID in person adjudicated examinations whereas the 2021-2022 exams were set for varying COVID-adapted conditions including open-book conditions. To ensure a fair evaluation of AI performances, the exams completed by AI were assessed by the original exam markers. However, due to staffing constraints, only the aforementioned 593 out of the total 1280 questions were marked. GPT-4 and GPT-3.5 scored an average of 49.4\% and 38.6\%, respectively, suggesting only the weaker students would potential improve their marks if using AI. For exams from the pre-COVID era, the average scores for GPT-4 and GPT-3.5 were 50.8\% and 41.6\%, respectively. However, post-COVID, these dropped to 47.5\% and 33.6\%. Thus contrary to expectations, the change to less fact-based questions in the COVID era did not significantly impact AI performance for the state-of-the-art models such as GPT-4. These findings suggest that while current AI models struggle with university-level Physics questions, an improving trend is observable. The code used for automated AI completion is made publicly available for further research.
CamemBERT-bio: a Tasty French Language Model Better for your Health

Clinical data in hospitals are increasingly accessible for research through clinical data warehouses, however these documents are unstructured. It is therefore necessary to extract information from medical reports to conduct clinical studies. Transfer learning with BERT-like models such as CamemBERT has allowed major advances, especially for named entity recognition. However, these models are trained for plain language and are less efficient on biomedical data. This is why we propose a new French public biomedical dataset on which we have continued the pre-training of CamemBERT. Thus, we introduce a first version of CamemBERT-bio, a specialized public model for the French biomedical domain that shows 2.54 points of F1 score improvement on average on different biomedical named entity recognition tasks. Our findings demonstrate the success of continual pre-training from a French model and contrast with recent proposals on the same domain and language. One of our key contributions highlights the importance of using a standard evaluation protocol that enables a clear view of the current state-of-the-art for French biomedical models.
Unveiling the orbital-selective electronic band reconstruction through the structural phase transition in 
TaTe2

Tantalum ditelluride TaTe$_2$ belongs to the family of layered transition metal dichalcogenides but exhibits a unique structural phase transition at around 170 K that accompanies the rearrangement of the Ta atomic network from a"ribbon chain"to a"butterfly-like"pattern. While multiple mechanisms including Fermi surface nesting and chemical bonding instabilities have been intensively discussed, the origin of this transition remains elusive. Here we investigate the electronic structure of single-crystalline TaTe$_2$ with a particular focus on its modifications through the phase transition, by employing core-level and angle-resolved photoemission spectroscopy combined with first-principles calculations. Temperature-dependent core-level spectroscopy demonstrates a splitting of the Ta $4f$ core-level spectra through the phase transition indicative of the Ta-dominated electronic state reconstruction. Low-energy electronic state measurements further reveal an unusual kink-like band reconstruction occurring at the Brillouin zone boundary, which cannot be explained by Fermi surface nesting or band folding effects. On the basis of the orbital-projected band calculations, this band reconstruction is mainly attributed to the modifications of specific Ta $5d$ states, namely the $d_{XY}$ orbitals (the ones elongating along the ribbon chains) at the center Ta sites of the ribbon chains. The present results highlight the strong orbital-dependent electronic state reconstruction through the phase transition in this system and provide fundamental insights towards understanding complex electron-lattice-bond coupled phenomena.
On the Periods of Twisted Moments of the Kloosterman Connection

This paper aims to study the Betti homology and de Rham cohomology of twisted symmetric powers of the Kloosterman connection of rank two on the torus. We compute the period pairing and, with respect to certain bases, interpret these associated period numbers in terms of the Bessel moments. Via the rational structures on Betti homology and de Rham cohomology, we prove the $\mathbb{Q}$-linear and quadratic relations among these Bessel moments.
Anomaly Detection in Networks via Score-Based Generative Models

Node outlier detection in attributed graphs is a challenging problem for which there is no method that would work well across different datasets. Motivated by the state-of-the-art results of score-based models in graph generative modeling, we propose to incorporate them into the aforementioned problem. Our method achieves competitive results on small-scale graphs. We provide an empirical analysis of the Dirichlet energy, and show that generative models might struggle to accurately reconstruct it.
The effects of wing inertial forces and mean stroke angle on the pitch stability of hovering insects

This paper discusses the wing inertial effects on stability of pitch motion of hovering insects. The paper also presents a dynamic model appropriate for using averaging techniques and discusses the pitch stability results derived from the model. The model is used to predict the body angle of five insect species during hover, which are in good agreement with the available experimental results from different literature. The results suggest that the wing inertia forces have a considerable effect on pitch dynamics of insect flight and should not be ignored in dynamic analysis of hovering insects. The results also suggest that, though the pitch stability of hovering insects is open-loop stable, it may not be vibrationally stabilized. Instead, the pitch stability is a balance of the moment of insect's weight and the aerodynamic moment due to flapping kinematics with a nonzero mean stroke angle. Experiments with a flapping wing device confirm this results. To clearly explain the used model and clarify the difference between vibrational and non-vibrational stabilization, first this paper discusses the vibrational control of a three-degree-of-freedom force-input pendulum with its pivot moving in a vertical plane.
Maximum State Entropy Exploration using Predecessor and Successor Representations

Animals have a developed ability to explore that aids them in important tasks such as locating food, exploring for shelter, and finding misplaced items. These exploration skills necessarily track where they have been so that they can plan for finding items with relative efficiency. Contemporary exploration algorithms often learn a less efficient exploration strategy because they either condition only on the current state or simply rely on making random open-loop exploratory moves. In this work, we propose $\eta\psi$-Learning, a method to learn efficient exploratory policies by conditioning on past episodic experience to make the next exploratory move. Specifically, $\eta\psi$-Learning learns an exploration policy that maximizes the entropy of the state visitation distribution of a single trajectory. Furthermore, we demonstrate how variants of the predecessor representation and successor representations can be combined to predict the state visitation entropy. Our experiments demonstrate the efficacy of $\eta\psi$-Learning to strategically explore the environment and maximize the state coverage with limited samples.
Scaling symmetries and canonoid transformations in Hamiltonian systems

We investigate various types of symmetries and their mutual relationships in Hamiltonian systems defined on manifolds with different geometric structures: symplectic, cosymplectic, contact and cocontact. In each case we pay special attention to non-standard (non-canonical) symmetries, in particular scaling symmetries and canonoid transformations, as they provide new interesting tools for the qualitative study of these systems. Our main results are the characterizations of these non-standard symmetries and the analysis of their relation with conserved (or dissipated) quantities.
Homological Neural Networks: A Sparse Architecture for Multivariate Complexity

The rapid progress of Artificial Intelligence research came with the development of increasingly complex deep learning models, leading to growing challenges in terms of computational complexity, energy efficiency and interpretability. In this study, we apply advanced network-based information filtering techniques to design a novel deep neural network unit characterized by a sparse higher-order graphical architecture built over the homological structure of underlying data. We demonstrate its effectiveness in two application domains which are traditionally challenging for deep learning: tabular data and time series regression problems. Results demonstrate the advantages of this novel design which can tie or overcome the results of state-of-the-art machine learning and deep learning models using only a fraction of parameters.
Phonon dynamic behaviors induced by amorphous interlayer at heterointerfaces

Interface impedes heat flow in heterostructures and the interfacial thermal resistance (ITR) has become a critical issue for thermal dissipation in electronic devices. To explore the mechanism leading to the ITR, in this work, the dynamic behaviors of phonons passing through the GaN/AlN interface with an amorphous interlayer is investigated by using phonon wave packet simulation. It is found the amorphous interlayer significantly impedes phonon transport across the interface, and leads to remarkable phonon mode conversions, such as LA$\rightarrow$TA, TA$\rightarrow$LA, and LA$\rightarrow$TO conversion. However, due to mode conversion and inelastic scattering, we found a portion of high-frequency TA phonons, which are higher than the cut-off frequency and cannot transmit across the ideal sharp interface, can partially transmit across the amorphous interlayer, which introduces additional thermal transport channels through the interface and has positive effect on interfacial thermal conductance. According to phonon transmission coefficient, it is found the ITR increases with increasing of amorphous interlayer thickness L. The phonon transmission coefficient exhibits an obvious oscillation behavior, which is attributed to the multiple phonon scattering in the amorphous interlayer, and the oscillation period is further revealed to be consistent with the theoretical prediction by the two-beam interference equation. In addition, obvious phonon frequency shifts and phonon energy localization phenomena were observed in the amorphous interlayer. Finally, to improve phonon transmission, the interface morphology was further optimized via the annealing reconstruction technique, which results in re-crystallization of the amorphous interlayer and the decrease of ITR by ~21% as L=2 nm.
Quantifying the Topology of Magnetic Skyrmions in three Dimensions

Magnetic skyrmions have so far been treated as two-dimensional spin structures characterized by a topological winding number describing the rotation of spins across the skyrmion. However, in real systems with a finite thickness of the material being larger than the magnetic exchange length, the skyrmion spin texture extends into the third dimension and cannot be assumed as homogeneous. Using soft x-ray laminography we reconstruct with about 20nm spatial (voxel) resolution the full three-dimensional spin texture of a skyrmion in an 800 nm diameter and 95 nm thin disk patterned into a trilayer [Ir/Co/Pt] thin film structure. A quantitative analysis finds that the evolution of the radial profile of the topological skyrmion number and the chirality is non-uniform across the thickness of the disk. Estimates of local micromagnetic energy densities suggest that the changes in topological profile are related to non-uniform competing energetic interactions. Theoretical calculations and micromagnetic simulations are consistent with the experimental findings. Our results provide the foundation for nanoscale magnetic metrology for future tailored spintronics devices using topology as a design parameter, and have the potential to reverse-engineer a spin Hamiltonian from macroscopic data, tying theory more closely to experiment.
Max-Margin Token Selection in Attention Mechanism

Attention mechanism is a central component of the transformer architecture which led to the phenomenal success of large language models. However, the theoretical principles underlying the attention mechanism are poorly understood, especially its nonconvex optimization dynamics. In this work, we explore the seminal softmax-attention model $f(\boldsymbol{X})=\langle \boldsymbol{Xv}, \texttt{softmax}(\boldsymbol{XWp})\rangle$, where $\boldsymbol{X}$ is the token sequence and $(\boldsymbol{v},\boldsymbol{W},\boldsymbol{p})$ are trainable parameters. We prove that running gradient descent on $\boldsymbol{p}$, or equivalently $\boldsymbol{W}$, converges in direction to a max-margin solution that separates $\textit{locally-optimal}$ tokens from non-optimal ones. This clearly formalizes attention as an optimal token selection mechanism. Remarkably, our results are applicable to general data and precisely characterize $\textit{optimality}$ of tokens in terms of the value embeddings $\boldsymbol{Xv}$ and problem geometry. We also provide a broader regularization path analysis that establishes the margin maximizing nature of attention even for nonlinear prediction heads. When optimizing $\boldsymbol{v}$ and $\boldsymbol{p}$ simultaneously with logistic loss, we identify conditions under which the regularization paths directionally converge to their respective hard-margin SVM solutions where $\boldsymbol{v}$ separates the input features based on their labels. Interestingly, the SVM formulation of $\boldsymbol{p}$ is influenced by the support vector geometry of $\boldsymbol{v}$. Finally, we verify our theoretical findings via numerical experiments and provide insights.
Wasserstein Generative Regression

In this paper, we propose a new and unified approach for nonparametric regression and conditional distribution learning. Our approach simultaneously estimates a regression function and a conditional generator using a generative learning framework, where a conditional generator is a function that can generate samples from a conditional distribution. The main idea is to estimate a conditional generator that satisfies the constraint that it produces a good regression function estimator. We use deep neural networks to model the conditional generator. Our approach can handle problems with multivariate outcomes and covariates, and can be used to construct prediction intervals. We provide theoretical guarantees by deriving non-asymptotic error bounds and the distributional consistency of our approach under suitable assumptions. We also perform numerical experiments with simulated and real data to demonstrate the effectiveness and superiority of our approach over some existing approaches in various scenarios.
Modeling the Multiwavelength Evolution of the V960 Mon System

We study the evolution of the FU Ori object V960 Mon since its outburst, using available multiwavelength photometric time series over 8 yr, complemented by several epochs of moderate-dispersion spectrophotometry. We find that the source fading can be well-described by a decrease in the temperature of the inner disk, which results from a combination of decreasing accretion rate and increasing inner disk radius. We model the system with a disk atmosphere model that produces the observed variations in multiband photometry (this paper) and high-resolution spectral lines (a companion paper).
Supervised Pretraining Can Learn In-Context Reinforcement Learning

Large transformer models trained on diverse datasets have shown a remarkable ability to learn in-context, achieving high few-shot performance on tasks they were not explicitly trained to solve. In this paper, we study the in-context learning capabilities of transformers in decision-making problems, i.e., reinforcement learning (RL) for bandits and Markov decision processes. To do so, we introduce and study Decision-Pretrained Transformer (DPT), a supervised pretraining method where the transformer predicts an optimal action given a query state and an in-context dataset of interactions, across a diverse set of tasks. This procedure, while simple, produces a model with several surprising capabilities. We find that the pretrained transformer can be used to solve a range of RL problems in-context, exhibiting both exploration online and conservatism offline, despite not being explicitly trained to do so. The model also generalizes beyond the pretraining distribution to new tasks and automatically adapts its decision-making strategies to unknown structure. Theoretically, we show DPT can be viewed as an efficient implementation of Bayesian posterior sampling, a provably sample-efficient RL algorithm. We further leverage this connection to provide guarantees on the regret of the in-context algorithm yielded by DPT, and prove that it can learn faster than algorithms used to generate the pretraining data. These results suggest a promising yet simple path towards instilling strong in-context decision-making abilities in transformers.
Astrophysical Parameters of the Open Cluster Berkeley 6

In this study, the structural and basic astrophysical parameters of poorly studied open cluster Berkeley 6 is calculated. Analyses of the cluster are carried out using the third photometric, spectroscopic and astrometric data release of Gaia (Gaia DR3). The membership probabilities of stars located through the cluster region are calculated by considering their astrometric data. Thus, we identified 119 physical members for Berkeley 6. The colour excess, distance and age of the cluster are determined simultaneously on the colour-magnitude diagram. We fitted solar metallicity PARSEC isochrones to the colour-magnitude diagram by considering the most probable member stars and obtained E(GBP-GRP) colour excess as 0.918±0.145 mag. Distance and age of the cluster are determined as d=2625±337 pc and t=350±50 Myr, respectively.
Parameterized Complexity of Chordal Conversion for Sparse Semidefinite Programs with Small Treewidth

If a sparse semidefinite program (SDP), specified over $n\times n$ matrices and subject to $m$ linear constraints, has an aggregate sparsity graph $G$ with small treewidth, then chordal conversion will frequently allow an interior-point method to solve the SDP in just $O(m+n)$ time per-iteration. This is a significant reduction over the minimum $\Omega(n^{3})$ time per-iteration for a direct solution, but a definitive theoretical explanation was previously unknown. Contrary to popular belief, the speedup is not guaranteed by a small treewidth in $G$, as a diagonal SDP would have treewidth zero but can still necessitate up to $\Omega(n^{3})$ time per-iteration. Instead, we construct an extended aggregate sparsity graph $\overline{G}\supseteq G$ by forcing each constraint matrix $A_{i}$ to be its own clique in $G$. We prove that a small treewidth in $\overline{G}$ does indeed guarantee that chordal conversion will solve the SDP in $O(m+n)$ time per-iteration, to $\epsilon$-accuracy in at most $O(\sqrt{m+n}\log(1/\epsilon))$ iterations. For classical SDPs like the MAX-$k$-CUT relaxation and the Lovasz Theta problem, the two sparsity graphs coincide $G=\overline{G}$, so our result provide a complete characterization for the complexity of chordal conversion, showing that a small treewidth is both necessary and sufficient for $O(m+n)$ time per-iteration. Real-world SDPs like the AC optimal power flow relaxation have different graphs $G\subseteq\overline{G}$ with similar small treewidths; while chordal conversion is already widely used on a heuristic basis, in this paper we provide the first rigorous guarantee that it solves such SDPs in $O(m+n)$ time per-iteration. [Supporting code at https://github.com/ryz-codes/chordalConv/]
On $p$-adic adjoint $L$-functions for Bianchi cuspforms: the $p$-split case

We construct a Hecke-equivariant pairing on the overconvergent cohomology of Bianchi threefolds. Applying the strategy of Kim and Bella\"iche, we use this pairing to construct $p$-adic adjoint $L$-functions for Bianchi cuspforms and show that it detects the ramification locus of the cuspidal Bianchi eigenvariety over the weight space. Combining results of Barrera Salazar--Williams, we show a non-vanishing result of this $p$-adic adjoint $L$-function at certain points. Finally, we obtain a formula relating this pairing with the adjoint $L$-values of the corresponding cuspidal Bianchi eigenforms.
The Underlying Scaling Laws and Universal Statistical Structure of Complex Datasets

We study universal traits which emerge both in real-world complex datasets, as well as in artificially generated ones. Our approach is to analogize data to a physical system and employ tools from statistical physics and Random Matrix Theory (RMT) to reveal their underlying structure. We focus on the feature-feature covariance matrix, analyzing both its local and global eigenvalue statistics. Our main observations are: (i) The power-law scalings that the bulk of its eigenvalues exhibit are vastly different for uncorrelated normally distributed data compared to real-world data, (ii) this scaling behavior can be completely modeled by generating gaussian data with long range correlations, (iii) both generated and real-world datasets lie in the same universality class from the RMT perspective, as chaotic rather than integrable systems, (iv) the expected RMT statistical behavior already manifests for empirical covariance matrices at dataset sizes significantly smaller than those conventionally used for real-world training, and can be related to the number of samples required to approximate the population power-law scaling behavior, (v) the Shannon entropy is correlated with local RMT structure and eigenvalues scaling, and substantially smaller in strongly correlated datasets compared to uncorrelated synthetic data, and requires fewer samples to reach the distribution entropy. These findings show that with sufficient sample size, the Gram matrix of natural image datasets can be well approximated by a Wishart random matrix with a simple covariance structure, opening the door to rigorous studies of neural network dynamics and generalization which rely on the data Gram matrix.