Miracle-dz/UniFAR-FTUs huggingface dataset:
Visually-Aware Audio Captioning With Adaptive Audio-Visual Attention

Audio captioning aims to generate text descriptions of audio clips. In the real world, many objects produce similar sounds. How to accurately recognize ambiguous sounds is a major challenge for audio captioning. In this work, inspired by inherent human multimodal perception, we propose visually-aware audio captioning, which makes use of visual information to help the description of ambiguous sounding objects. Specifically, we introduce an off-the-shelf visual encoder to extract video features and incorporate the visual features into an audio captioning system. Furthermore, to better exploit complementary audio-visual contexts, we propose an audio-visual attention mechanism that adaptively integrates audio and visual context and removes the redundant information in the latent space. Experimental results on AudioCaps, the largest audio captioning dataset, show that our proposed method achieves state-of-the-art results on machine translation metrics.
A Solvable Model of Neural Scaling Laws

Large language models with a huge number of parameters, when trained on near internet-sized number of tokens, have been empirically shown to obey neural scaling laws: specifically, their performance behaves predictably as a power law in either parameters or dataset size until bottlenecked by the other resource. To understand this better, we first identify the necessary properties allowing such scaling laws to arise and then propose a statistical model – a joint generative data model and random feature model – that captures this neural scaling phenomenology. By solving this model in the dual limit of large training set size and large number of parameters, we gain insight into (i) the statistical structure of datasets and tasks that lead to scaling laws, (ii) the way nonlinear feature maps, such as those provided by neural networks, enable scaling laws when trained on these datasets, (iii) the optimality of the equiparameterization scaling of training sets and parameters, and (iv) whether such scaling laws can break down and how they behave when they do. Key findings are the manner in which the power laws that occur in the statistics of natural datasets are extended by nonlinear random feature maps and then translated into power-law scalings of the test loss and how the finite extent of the data’s spectral power law causes the model’s performance to plateau. Equal contribution. ar X iv :2 21 0. 16 85 9v 1 [ cs . L G ] 3 0 O ct 2 02 2
Learning to Compare Nodes in Branch and Bound with Graph Neural Networks

Branch-and-bound approaches in integer programming require ordering portions of the space to explore next, a problem known as node comparison. We propose a new siamese graph neural network model to tackle this problem, where the nodes are represented as bipartite graphs with attributes. Similar to prior work, we train our model to imitate a diving oracle that plunges towards the optimal solution. We evaluate our method by solving the instances in a plain framework where the nodes are explored according to their rank. On three NP-hard benchmarks chosen to be particularly primal-difficult, our approach leads to faster solving and smaller branch- and-bound trees than the default ranking function of the open-source solver SCIP, as well as competing machine learning methods. Moreover, these results generalize to instances larger than used for training. Code for reproducing the experiments can be found at https://github.com/ds4dm/learn2comparenodes.
Impurity states and cooperative magnetic order in Fe-based superconductors

We study impurity bound states and impurity-induced order in the superconducting state of LiFeAs within a realistic five-band model based on the band structure and impurity potentials obtained from density functional theory (DFT). In agreement with recent experiments, we find that Co impurities are too weak produce sub-gap bound states, whereas stronger impurities like Cu do. We also obtain the bound state spectrum for magnetic impurities, such as Mn, and show how spin-resolved tunnelling may determine the nature of the various defect sites in iron pnictides, a prerequisite for using impurity bound states as a probe of the ground state pairing symmetry. Lastly we show how impurities pin both orbital and magnetic order, providing an explanation for a growing set of experimental evidence for unusual magnetic phases in doped iron pnictides.
Spontaneous emergence of groups and signaling diversity in dynamic networks

We study the coevolution of network structure and signaling behavior. We model agents who can preferentially associate with others in a dynamic network while they also learn to play a simple sender-receiver game. We have four major findings. First, signaling interactions in dynamic networks are sufficient to cause the endogenous formation of distinct signaling groups, even in an initially homogeneous population. Second, dynamic networks allow the emergence of novel hybrid signaling groups that do not converge on a single common signaling system but are instead composed of different yet complementary signaling strategies. We show that the presence of these hybrid groups promotes stable diversity in signaling among other groups in the population. Third, we find important distinctions in information processing capacity of different groups: hybrid groups diffuse information more quickly initially but at the cost of taking longer to reach all group members. Fourth, our findings pertain to all common interest signaling games, are robust across many parameters, and mitigate known problems of inefficient communication.
Modeling disks and magnetic outflows around a forming massive star: I. Investigating the two layer-structure of the accretion disk

Like their lower mass siblings, massive protostars can be expected to: a) be surrounded by circumstellar disks and b) launch magnetically-driven jets and outflows. The disk formation and global evolution is thereby controlled by advection of angular momentum from large scales, the efficiency of magnetic braking and the resistivity of the medium, and the internal thermal and magnetic pressures of the disk. We perform a series of 30 simulations of a massive star forming from the gravitational collapse of a molecular cloud threaded by an initially-uniform magnetic field, starting from different values for the mass of the cloud, its initial density and rotation profiles, its rotational energy content, the magnetic field strength, and the resistivity of the material. The gas and dust is modeled with the methods of resistive magnetohydrodynamics, also considering radiation transport of thermal emission and self-gravity. After the initial infall phase dominated by the gravitational collapse, an accretion disk is formed, shortly followed by the launching of magnetically-driven outflows. Two layers can be distinguished in the accretion disk: a thin layer, vertically supported by thermal pressure, and a thick layer, vertically supported by magnetic pressure. We observe the effects of magnetic braking in the inner ~50 au of the disk at late times in our fiducial case. The parameter study reveals that the size of the disk is mostly determined by the density and rotation profiles of the initial mass reservoir and not by the magnetic field strength. Magnetic pressure can slightly increase the size of the accretion disk, while magnetic braking is more relevant in the innermost parts of the disk as opposed to the outer disk. From the parameter study, we infer that multiple initial conditions for the onset of gravitational collapse are able to produce a given disk size and protostellar mass.
The First Interferometric Measurements of NH2D/NH3 Ratio in Hot Corinos

The chemical evolution of nitrogen during star and planet formation is still not fully understood. Ammonia (NH3) is a key specie in the understanding of the molecular evolution in star-forming clouds and nitrogen isotope fractionation. In this paper, we present high-spatial-resolution observations of multiple emission lines of NH3 toward the protobinary system NGC1333 IRAS4A with the Karl G. Jansky Very Large Array. We spatially resolved the binary (hereafter, 4A1 and 4A2) and detected compact emission of NH3 transitions with high excitation energies (≳100 K) from the vicinity of the protostars, indicating the NH3 ice has sublimated at the inner hot region. The NH3 column density is estimated to be ∼1017–1018 cm−2. We also detected two NH2D transitions, allowing us to constrain the deuterium fractionation of ammonia. The NH2D/NH3 ratios are as high as ∼0.3–1 in both 4A1 and 4A2. From comparisons with the astrochemical models in the literature, the high NH2D/NH3 ratios suggest that the formation of NH3 ices mainly started in the prestellar phase after the formation of bulk water ice finished, and that the primary nitrogen reservoir in the star-forming cloud could be atomic nitrogen (or N atoms) rather than nitrogen-bearing species such as N2 and NH3. The implications on the physical properties of IRAS4A’s cores are discussed as well.
Comparison between 1D radial and 0D global models for low-pressure oxygen DC glow discharges

This work focuses on the comparison between a zero-dimensional (0D) global model (LoKI) and a one-dimensional (1D) radial fluid model for the positive column of oxygen DC glow discharges in a tube of 1 cm inner radius at pressures between 0.5 Torr and 10 Torr. The data used in the two models are the same, so that the difference between the models is reduced to dimensionality. A good agreement is found between the two models on the main discharge parameters (gas temperature, electron density, reduced electric field and dissociation fraction), with relative differences below 5%. The agreement on other species average number densities, charged and neutral, is slightly worse, with relative differences increasing with pressure from 11% at 0.5 Torr to 57% at 10 Torr. The success of the 0D global model in describing these plasmas through volume averaged quantities decreases with pressure, due to pressure-driven narrowing of radial profiles. Hence, in the studied conditions, we recommend the use of volume-averaged models only in the pressure range up to 10 Torr.
A law of data separation in deep learning

Significance The practice of deep learning has long been shrouded in mystery, leading many to believe that the inner workings of these black-box models are chaotic during training. In this paper, we challenge this belief by presenting a simple and approximate law that deep neural networks follow when processing data in the intermediate layers. This empirical law is observed in a class of modern network architectures for vision tasks, and its emergence is shown to bring important benefits for the trained models. The significance of this law is that it allows for a perspective that provides useful insights into the practice of deep learning.
Temperature Dependent Bending Rigidity of Graphene

The mechanical properties of two-dimensional (2D) materials are important for a wide range of applications including composite materials, van der Waals materials and ﬂexible electronics. Several aspects are highly debated in the literature: For example, the theoretically predicted bending rigidity, κ ( T ) , of the most important 2D material, graphene, varies from 0.8 eV to 10 eV for predictions at 0 K and there are predictions that it could increase or decrease with temperature depending on the applied theory. Here we present the ﬁrst systematic experimental study of the temperature dependence κ ( T ) of graphene. Extracting values of κ from the ZA-phonon dispersion relation measured with helium atom scattering, we ﬁnd the value of κ ( T ) to increase with sample temperature. A linear ﬁt in the temperature range (120 K-480 K) gives a bending rigidity of κ ( T ) = (1 . 1 ± 0 . 1)[ eV ] + (0 . 0013 ± 0 . 0001)[ eV/K ] × T .
Nonlinear Causal Discovery via Kernel Anchor Regression

Learning causal relationships is a fundamental problem in science. Anchor regression has been developed to address this problem for a large class of causal graphical models, though the relationships between the variables are assumed to be linear. In this work, we tackle the nonlinear setting by proposing kernel anchor regression (KAR). Beyond the natural formula-tion using a classic two-stage least square estimator, we also study an improved variant that involves nonparametric regression in three separate stages. We provide convergence results for the proposed KAR estimators and the identiﬁability conditions for KAR to learn the nonlinear structural equation models (SEM). Experimental results demonstrate the superior performances of the proposed KAR estimators over existing baselines.
Representation Learning for General-sum Low-rank Markov Games

We study multi-agent general-sum Markov games with nonlinear function approximation. We focus on low-rank Markov games whose transition matrix admits a hidden low-rank structure on top of an unknown non-linear representation. The goal is to design an algorithm that (1) finds an $\varepsilon$-equilibrium policy sample efficiently without prior knowledge of the environment or the representation, and (2) permits a deep-learning friendly implementation. We leverage representation learning and present a model-based and a model-free approach to construct an effective representation from the collected data. For both approaches, the algorithm achieves a sample complexity of poly$(H,d,A,1/\varepsilon)$, where $H$ is the game horizon, $d$ is the dimension of the feature vector, $A$ is the size of the joint action space and $\varepsilon$ is the optimality gap. When the number of players is large, the above sample complexity can scale exponentially with the number of players in the worst case. To address this challenge, we consider Markov games with a factorized transition structure and present an algorithm that escapes such exponential scaling. To our best knowledge, this is the first sample-efficient algorithm for multi-agent general-sum Markov games that incorporates (non-linear) function approximation. We accompany our theoretical result with a neural network-based implementation of our algorithm and evaluate it against the widely used deep RL baseline, DQN with fictitious play.
The Orbits and Dynamical Masses of the Castor System

Castor is a system of six stars in which the two brighter objects, Castor A and B, revolve around each other every ∼450 yr and are both short-period spectroscopic binaries. They are attended by the more distant Castor C, which is also a binary. Here we report interferometric observations with the Center for High Angular Resolution Astronomy (CHARA) array that spatially resolve the companions in Castor A and B for the first time. We complement these observations with new radial velocity measurements of A and B spanning 30 yr, with the Hipparcos intermediate data, and with existing astrometric observations of the visual AB pair obtained over the past three centuries. We perform a joint orbital solution to solve simultaneously for the three-dimensional orbits of Castor A and B as well as the AB orbit. We find that they are far from being coplanar: the orbit of A is nearly at right angles (92°) relative to the wide orbit, and that of B is inclined about 59° compared to AB. We determine the dynamical masses of the four stars in Castor A and B to a precision better than 1%. We also determine the radii of the primary stars of both subsystems from their angular diameters measured with the CHARA array, and use them together with stellar evolution models to infer an age for the system of 290 Myr. The new knowledge of the orbits enables us to measure the slow motion of Castor C as well, which may assist future studies of the dynamical evolution of this remarkable sextuple system.
Learning to Optimize Permutation Flow Shop Scheduling via Graph-based Imitation Learning

The permutation flow shop scheduling (PFSS), aiming at finding the optimal permutation of jobs, is widely used in manufacturing systems. When solving large-scale PFSS problems, traditional optimization algorithms such as heuristics could hardly meet the demands of both solution accuracy and computational efficiency, thus learning-based methods have recently garnered more attention. Some work attempts to solve the problems by reinforcement learning methods, which suffer from slow convergence issues during training and are still not accurate enough regarding the solutions. To that end, we propose to train the model via expert-driven imitation learning, which accelerates convergence more stably and accurately. Moreover, in order to extract better feature representations of input jobs, we incorporate the graph structure as the encoder. The extensive experiments reveal that our proposed model obtains significant promotion and presents excellent generalizability in large-scale problems with up to 1000 jobs. Compared to the state-of-the-art reinforcement learning method, our model's network parameters are reduced to only 37% of theirs, and the solution gap of our model towards the expert solutions decreases from 6.8% to 1.3% on average. The code is available at: https://github.com/longkangli/PFSS-IL.
A Chemical Modeling Roadmap Linking Protoplanetary Disks and Exoplanet Atmospheres

Exoplanet atmospheres and protoplanetary disk chemistry are both active fields of research. Chronologically, they represent opposite ends of the process of planet formation: planets form in protoplanetary disks, and the composition of planets and their atmospheres should reflect the composition of the disk material they form from. The last decade of discoveries and insights by the ALMA observatory has provided a quantum leap in our understanding of the structures and chemical compositions of protoplanetary disks. Likewise, both ground and space-based facilities have provided peeks into the compositions of exoplanet atmospheres. However, the near future, with, first and foremost, the James Webb Space Telescope, and later also ESO’s Extremely Large Telescope and ESA’s Ariel mission are expected to provided constraints on exoplanet atmospheric composition with unprecedented precision, and also to shed new light on gas and ice compositions of protoplanetary disks. It is therefore timely to evaluate the current state of modelling efforts attempting to link protoplanetary disk chemistry and planet formation to exoplanet atmospheric compositions. The author has a background in astrochemistry related to planet formation. As such, this review discusses the differences between the simpler chemical approach of “iceline chemistry” and the more chemically rigorous approach of utilising chemical kinetics. It is outlined which chemical effects may be at play in a planet-forming disk midplane, which effects are relevant under different conditions, and which tools are available for modelling chemical kinetics in a disk midplane. The review goes on to discuss some important efforts in the planet formation modelling community to treat chemical evolution, and, vice versa, efforts in the chemical modelling community to implement more physical effects related to planet formation into the chemical modelling. The aim of this review is both to outline some concepts related to planet formation chemistry, but also to encourage, not just collaboration between the planet formation modelling community and the astrochemical community, but also assistance and guidance from one community to the other. Guidance, regarding which effects, out of many, might be more relevant than others under certain planet formation conditions, and regarding why certain included effects lead to certain important modelling outcomes. As the research fields of exoplanet atmospheres and protoplanetary disks near new frontiers in observational insights
Fano 4-folds with nef tangent bundle in positive characteristic

In characteristic $0$, the Campana-Peternell conjecture claims that the only smooth Fano variety with nef tangent bundle should be homogeneous. In this paper, we study the positive characteristic version of the Campana-Peternell conjecture. In particular, we give an affirmative answer for Fano $4$-folds with nef tangent bundle and Picard number greater than one.
The algebraic origin of the Doppler factor in the Liénard–Wiechert potentials

After reviewing the algebraic derivation of the Doppler factor in the Liénard–Wiechert potentials of an electrically charged point particle, we conclude that the Dirac delta function used in electrodynamics must be the one obeying the weak definition, non-zero in an infinitesimal neighborhood, and not the one obeying the strong definition, non-zero in a point. This conclusion emerges from our analysis of (a) the derivation of an important Dirac delta function identity, which generates the Doppler factor, (b) the linear superposition principle implicitly used by the Green function method, and (c) the two equivalent formulations of the Schwarzschild–Tetrode–Fokker action. As a consequence, in full agreement with our previous discussion of the geometrical origin of the Doppler factor, we conclude that the electromagnetic interaction takes place not between points in Minkowski space, but between corresponding infinitesimal segments along the worldlines of the particles.
Flexible Machine Learning Estimation of Conditional Average Treatment Effects: A Blessing and a Curse

Causal inference from observational data requires untestable identification assumptions. If these assumptions apply, machine learning methods can be used to study complex forms of causal effect heterogeneity. Recently, several machine learning methods were developed to estimate the conditional average treatment effect (ATE). If the features at hand cannot explain all heterogeneity, the individual treatment effects can seriously deviate from the conditional ATE. In this work, we demonstrate how the distributions of the individual treatment effect and the conditional ATE can differ when a causal random forest is applied. We extend the causal random forest to estimate the difference in conditional variance between treated and controls. If the distribution of the individual treatment effect equals that of the conditional ATE, this estimated difference in variance should be small. If they differ, an additional causal assumption is necessary to quantify the heterogeneity not captured by the distribution of the conditional ATE. The conditional variance of the individual treatment effect can be identified when the individual effect is independent of the outcome under no treatment given the measured features. Then, in the cases where the individual treatment effect and conditional ATE distributions differ, the extended causal random forest can appropriately estimate the variance of the individual treatment effect distribution, whereas the causal random forest fails to do so.
Exemplar Guided Deep Neural Network for Spatial Transcriptomics Analysis of Gene Expression Prediction

Spatial transcriptomics (ST) is essential for understanding diseases and developing novel treatments. It measures gene expression of each fine-grained area (i.e., different windows) in the tissue slide with low throughput. This paper proposes an Exemplar Guided Network (EGN) to accurately and efficiently predict gene expression directly from each window of a tissue slide image. We apply exemplar learning to dynamically boost gene expression prediction from nearest/similar exemplars of a given tissue slide image window. Our EGN framework composes of three main components: 1) an extractor to structure a representation space for unsupervised exemplar retrievals; 2) a vision transformer (ViT) backbone to progressively extract representations of the input window; and 3) an Exemplar Bridging (EB) block to adaptively revise the intermediate ViT representations by using the nearest exemplars. Finally, we complete the gene expression prediction task with a simple attention-based prediction block. Experiments on standard benchmark datasets indicate the superiority of our approach when comparing with the past state-of-the-art (SOTA) methods.