Miracle-dz/UniFAR-FTUs huggingface dataset:
Bayesian inference is facilitated by modular neural networks with different time scales

Various animals, including humans, have been suggested to perform Bayesian inferences to handle noisy, time-varying external information. In performing Bayesian inference by the brain, the prior distribution must be acquired and represented by sampling noisy external inputs. However, the mechanism by which neural activities represent such distributions has not yet been elucidated. Our findings reveal that networks with modular structures, composed of fast and slow modules, are adept at representing this prior distribution, enabling more accurate Bayesian inferences. Specifically, the modular network that consists of a main module connected with input and output layers and a sub-module with slower neural activity connected only with the main module outperformed networks with uniform time scales. Prior information was represented specifically by the slow sub-module, which could integrate observed signals over an appropriate period and represent input means and variances. Accordingly, the neural network could effectively predict the time-varying inputs. Furthermore, by training the time scales of neurons starting from networks with uniform time scales and without modular structure, the above slow-fast modular network structure and the division of roles in which prior knowledge is selectively represented in the slow sub-modules spontaneously emerged. These results explain how the prior distribution for Bayesian inference is represented in the brain, provide insight into the relevance of modular structure with time scale hierarchy to information processing, and elucidate the significance of brain areas with slower time scales.
Equivariant cohomology and depth

Let n ≥ 1 be an integer, let V = ( Z / 2 Z ) n and let X be a V -CW-complex. If X is a ﬁnite CW -complexe, the equivariant modulo 2 cohomology of the V -CW-complexe X , denoted by H ∗ V ( X, F 2 ) , is a ﬁnite type module over the modulo 2 cohomology of the group V , denoted by H ∗ ( V, F 2 ) . Let dth H ∗ V H ∗ V ( X, F 2 ) be the depth of the ﬁnite type H ∗ ( V, F 2 ) -module H ∗ V ( X, F 2 ) relatively to the augmentation ideal, g Let W be a subgroup of V , the ﬁnite V -CW-complex X is a ﬁnite W -CW-complex, we denote by dth H ∗ W H ∗ W ( X, F 2 ) the depth of the ﬁnite type H ∗ ( W, F 2 ) -module H ∗ W ( X, F 2 ) relatively to the augmentation ideal , The aim of this paper is to study a question raised in 2005 in [BouHZ] concerning the comparaison of dth H ∗ V H ∗ V ( X, F 2 ) and dth H ∗ W H ∗ W ( X, F 2 ) . We prove the following result:
Hawking Mass Monotonicity for Initial Data Sets

We introduce new systems of PDE on initial data sets $(M,g,k)$ whose solutions model double-null foliations. This allows us to generalize Geroch's monotonicity formula for the Hawking mass under inverse mean curvature flow to initial data sets satisfying the dominant energy condition. We study the existence theory of these systems and give geometric applications.
SLAM: Semantic Learning based Activation Map for Weakly Supervised Semantic Segmentation

Recent mainstream weakly-supervised semantic segmentation (WSSS) approaches based on image-level annotations mainly relies on binary image-level classiﬁcation with limited representation capacity. In this paper, we propose a novel semantic learning based framework for WSSS, named SLAM (Semantic Learning based Activation Map). We ﬁrstly design a semantic encoder to learn semantics of each object category and extract category-speciﬁc semantic embeddings from an input image. The semantic embeddings of foreground and background are then integrated to a segmentation network to learn the activation map. Four loss functions, i.e, category-foreground, category-background, activation regularization, and consistency loss are proposed to ensure the correctness, completeness, compactness and consistency of the activation map. Experimental results show that our semantic learning based SLAM achieves much better performance than binary image-level classiﬁ-cation based approaches, i.e., around 3% mIoU higher than OC-CSE [13], CPN [37] on PASCAL VOC dataset. Our SLAM also surpasses AMN [16] trained with strong per-pixel constraint and CLIMS [33] utilizing extra multi-modal knowledge. Code will be made available.
DiscoSense: Commonsense Reasoning with Discourse Connectives

We present DiscoSense, a benchmark for commonsense reasoning via understanding a wide variety of discourse connectives. We generate compelling distractors in DiscoSense using Conditional Adversarial Filtering, an extension of Adversarial Filtering that employs conditional generation. We show that state-of-the-art pre-trained language models struggle to perform well on DiscoSense, which makes this dataset ideal for evaluating next-generation commonsense reasoning systems.
Effective quantum extended spacetime of polymer Schwarzschild black hole

The physical interpretation and eventual fate of gravitational singularities in a theory surpassing classical general relativity are puzzling questions that have generated a great deal of interest among various quantum gravity approaches. In the context of loop quantum gravity (LQG), one of the major candidates for a non-perturbative background-independent quantisation of general relativity, considerable effort has been devoted to construct effective models in which these questions can be studied. In these models, classical singularities are replaced by a ‘bounce’ induced by quantum geometry corrections. Undesirable features may arise however depending on the details of the model. In this paper, we focus on Schwarzschild black holes and propose a new effective quantum theory based on polymerisation of new canonical phase space variables inspired by those successful in loop quantum cosmology. The quantum corrected spacetime resulting from the solutions of the effective dynamics is characterised by infinitely many pairs of trapped and anti-trapped regions connected via a space-like transition surface replacing the central singularity. Quantum effects become relevant at a unique mass independent curvature scale, while they become negligible in the low curvature region near the horizon. The effective quantum metric describes also the exterior regions and asymptotically classical Schwarzschild geometry is recovered. We however find that physically acceptable solutions require us to select a certain subset of initial conditions, corresponding to a specific mass (de-)amplification after the bounce. We also sketch the corresponding quantum theory and explicitly compute the kernel of the Hamiltonian constraint operator.
Auto-Encoder Neural Network Incorporating X-Ray Fluorescence Fundamental Parameters with Machine Learning

We consider energy-dispersive X-ray Fluorescence (EDXRF) applications where the fundamental parameters method is impractical such as when instrument parameters are unavailable. For example, on a mining shovel or conveyor belt, rocks are constantly moving (leading to varying angles of incidence and distances) and there may be other factors not accounted for (like dust). Neural networks do not require instrument and fundamental parameters but training neural networks requires XRF spectra labelled with elemental composition, which is often limited because of its expense. We develop a neural network model that learns from limited labelled data and also benefits from domain knowledge by learning to invert a forward model. The forward model uses transition energies and probabilities of all elements and parameterized distributions to approximate other fundamental and instrument parameters. We evaluate the model and baseline models on a rock dataset from a lithium mineral exploration project. Our model works particularly well for some low-Z elements (Li, Mg, Al, and K) as well as some high-Z elements (Sn and Pb) despite these elements being outside the suitable range for common spectrometers to directly measure, likely owing to the ability of neural networks to learn correlations and non-linear relationships.
Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs

Social intelligence and Theory of Mind (TOM), i.e., the ability to reason about the different mental states, intents, and reactions of all people involved, allows humans to effectively navigate and understand everyday social interactions. As NLP systems are used in increasingly complex social situations, their ability to grasp social dynamics becomes crucial.In this work, we examine the open question of social intelligence and Theory of Mind in modern NLP systems from an empirical and theorybased perspective. We show that one of today’s largest language models (GPT-3; Brown et al., 2020) lacks this kind of social intelligence out-of-the box, using two tasks: SocialIQa (Sap et al., 2019), which measure models’ ability to understand intents and reactions of participants of social interactions, and ToMi (Le, Boureau, and Nickel, 2019), which measures whether models can infer mental states and realities of participants of situations.Our results show that models struggle substantially at these Theory of Mind tasks, with well-below-human accuracies of 55% and 60% on SocialIQa and ToMi, respectively. To conclude, we draw on theories from pragmatics to contextualize this shortcoming of large language models, by examining the limitations stemming from their data, neural architecture, and training paradigms. Challenging the prevalent narrative that only scale is needed, we posit that person-centric NLP approaches might be more effective towards neural Theory of Mind.
Recovering the Physical Properties of Molecular Gas in Galaxies from CO SLED Modeling

Modeling of the spectral line energy distribution (SLED) of the CO molecule can reveal the physical conditions (temperature and density) of molecular gas in Galactic clouds and other galaxies. Recently, the Herschel Space Observatory and ALMA have offered, for the first time, a comprehensive view of the rotational J = 4−3 through J = 13−12 lines, which arise from a complex, diverse range of physical conditions that must be simplified to one, two, or three components when modeled. Here we investigate the recoverability of physical conditions from SLEDs produced by galaxy evolution simulations containing a large dynamical range in physical properties. These simulated SLEDs were generally fit well by one component of gas whose properties largely resemble or slightly underestimate the luminosity-weighted properties of the simulations when clumping due to nonthermal velocity dispersion is taken into account. If only modeling the first three rotational lines, the median values of the marginalized parameter distributions better represent the luminosity-weighted properties of the simulations, but the uncertainties in the fitted parameters are nearly an order of magnitude, compared to approximately 0.2 dex in the “best-case” scenario of a fully sampled SLED through J = 10−9. This study demonstrates that while common CO SLED modeling techniques cannot reveal the underlying complexities of the molecular gas, they can distinguish bulk luminosity-weighted properties that vary with star formation surface densities and galaxy evolution, if a sufficient number of lines are detected and modeled.
Tools for Extracting Spatio-Temporal Patterns in Meteorological Image Sequences: From Feature Engineering to Attention-Based Neural Networks

Atmospheric processes involve both space and time. This is why human analysis of atmospheric imagery can often extract more information from animated loops of image sequences than from individual images. Automat-ing such an analysis requires the ability to identify spatio-temporal patterns in image sequences which is a very challenging task, because of the endless possibilities of patterns in both space and time. In this paper we review different concepts and techniques that are useful to extract spatio-temporal context speciﬁcally for meteorological applications. In this survey we ﬁrst motivate the need for these approaches in meteorology using two applications, solar forecasting and detecting convection from satellite imagery. Then we provide an overview of many different concepts and techniques that are helpful for the interpretation of meteorological image sequences, such as (1) feature engineering methods to strengthen the desired signal in the input, using meteorological knowledge, classic image processing, harmonic analysis and topological data analysis; (2) explain how different convolution ﬁlters (2D/3D/LSTM-convolution) can be utilized strategically in convolutional neural network architectures to ﬁnd patterns in both space and time; (3) discuss the powerful new concept of “attention” in neural networks and the powerful abilities it brings to the interpretation of image sequences; (4) brieﬂy survey strategies from unsupervised, self-supervised and transfer learning to reduce the need for large labeled datasets. We hope that presenting
Unveiling a cluster of protostellar disks around the massive protostar GGD 27 MM1

Context. Most stars form in clusters and thus it is important to characterize the protostellar disk population in dense environments to assess whether the environment plays a role in the subsequent evolution. Specifically, it is critical to evaluate whether planet formation is altered with respect to more isolated stars formed in dark clouds. Aims. We seek to investigate the properties of the protostellar disks in the GGD 27 cluster and compare these with those obtained from disks formed in nearby regions. Methods. We used ALMA to observe the star-forming region GGD 27 at 1.14 mm with an unprecedented angular resolution, 40 mas (∼56 au), and sensitivity (∼0.002 M⊙). Results. We detected a cluster of 25 continuum sources, most of which likely trace disks around Class 0/I protostars. Excluding the two most massive objects, disks masses are in the range 0.003–0.05 M⊙. The analysis of the cluster properties indicates that GGD 27 displays moderate subclustering. This result, combined with the dynamical timescale of the radio jet (∼104 years), suggests the youthfulness of the cluster. The lack of disk mass segregation signatures may support this as well. We found a clear paucity of disks with Rdisk > 100 au. The median value of the radius is 34 au; this value is smaller than the median of 92 au for Taurus but comparable to the value found in Ophiuchus and in the Orion Nebula Cluster. In GGD 27 there is no evidence of a distance-dependent disk mass distribution (i.e., disk mass depletion due to external photoevaporation), most likely due to the cluster youth. There is a clear deficit of disks for distances <0.02 pc. Stars can only form larger and more massive disksfor distances >0.04 pc. This suggests that dynamical interactions far from the cluster center are weaker, although the small disks found could be the result of disk truncation. This work demonstrates the potential to characterize disks from low-mass young stellar objects in distant and massive (still deeply embedded) clustered environments.
Normalized solutions to Kirchhoff type equations with a critical growth nonlinearity

In this paper, we are concerned with normalized solutions of the Kirchhoﬀ type equation . When N = 2 and f has exponential critical growth at inﬁnity, normalized mountain pass type solutions are obtained via the variational methods. When N ≥ 4, M ( t ) = a + bt with a , b > 0 and f has Sobolev critical growth at inﬁnity, we investigate the existence of normalized ground state solutions and normalized mountain pass type solutions. Moreover, the non-existence of normalized solutions is also considered.
A Regularized Conditional GAN for Posterior Sampling in Inverse Problems

In inverse problems, one seeks to reconstruct an image from incomplete and/or degraded measurements. Such problems arise in magnetic resonance imaging (MRI), computed tomography, deblurring, superresolution, inpainting, and other applications. It is often the case that many image hypotheses are consistent with both the measurements and prior information, and so the goal is not to recover a single “best” hypothesis but rather to explore the space of probable hypotheses, i.e., to sample from the posterior distribution. In this work, we propose a regularized conditional Wasserstein GAN that can generate dozens of high-quality posterior samples per second. Using quantitative evaluation metrics like conditional Fréchet inception distance, we demonstrate that our method produces state-of-the-art posterior samples in both multicoil MRI and inpainting applications.
Parallel and perpendicular alignments of velocity gradient and magnetic field observed in the molecular clouds L1478 and L1482

Star formation is a complex process that typically occurs in dense regions of molecular clouds mainly regulated by magnetic fields, magnetohydrodynamic (MHD) turbulence, and self-gravity. However, it remains a challenging endeavor to trace the magnetic field and determine regions of gravitational collapse where the star is forming. Based on the anisotropic properties of MHD turbulence, a new technique termed Velocity Gradient Technique (VGT) has been proposed to address these challenges. In this study, we apply the VGT to two regions of the giant California Molecular Cloud (CMC), namely, L1478 and L1482, and analyze the difference in their physical properties. We use the 12CO (J = 2 - 1), 13CO (J = 2 - 1), and C18O (J = 2 - 1) emission lines observed with the Heinrich Hertz Submillimeter Telescope. We compare VGT results calculated in the resolutions of 3.3′ and 10′ to Planck polarization at 353 GHz and 10′ to determine areas of MHD turbulence dominance and self-gravity dominance. We show that the resolution difference can introduce misalignment between the two measurements. We find the VGT-measured magnetic fields globally agree with that from Planck in L1478 suggesting self-gravity’s effect is insignificant. The best agreement appears in VGT-12CO. As for L1482, the VGT measurements are statistically perpendicular to the Planck polarization indicating the dominance of self-gravity. This perpendicular alignment is more significant in VGT-13CO and VGT-C18O.
MARS: Meta-Learning as Score Matching in the Function Space

Meta-learning aims to extract useful inductive biases from a set of related datasets. In Bayesian meta-learning, this is typically achieved by constructing a prior distribution over neural network parameters. However, specifying families of computationally viable prior distributions over the high-dimensional neural network parameters is difficult. As a result, existing approaches resort to meta-learning restrictive diagonal Gaussian priors, severely limiting their expressiveness and performance. To circumvent these issues, we approach meta-learning through the lens of functional Bayesian neural network inference, which views the prior as a stochastic process and performs inference in the function space. Specifically, we view the meta-training tasks as samples from the data-generating process and formalize meta-learning as empirically estimating the law of this stochastic process. Our approach can seamlessly acquire and represent complex prior knowledge by meta-learning the score function of the data-generating process marginals instead of parameter space priors. In a comprehensive benchmark, we demonstrate that our method achieves state-of-the-art performance in terms of predictive accuracy and substantial improvements in the quality of uncertainty estimates.
Designing Universal Causal Deep Learning Models: The Case of Infinite-Dimensional Dynamical Systems from Stochastic Analysis

Causal operators (CO), such as various solution operators to stochastic differential equations, play a central role in contemporary stochastic analysis; however, there is still no canonical framework for designing Deep Learning (DL) models capable of approximating COs. This paper proposes a"geometry-aware'"solution to this open problem by introducing a DL model-design framework that takes suitable infinite-dimensional linear metric spaces as inputs and returns a universal sequential DL model adapted to these linear geometries. We call these models Causal Neural Operators (CNOs). Our main result states that the models produced by our framework can uniformly approximate on compact sets and across arbitrarily finite-time horizons H\"older or smooth trace class operators, which causally map sequences between given linear metric spaces. Our analysis uncovers new quantitative relationships on the latent state-space dimension of CNOs which even have new implications for (classical) finite-dimensional Recurrent Neural Networks (RNNs). We find that a linear increase of the CNO's (or RNN's) latent parameter space's dimension and of its width, and a logarithmic increase of its depth imply an exponential increase in the number of time steps for which its approximation remains valid. A direct consequence of our analysis shows that RNNs can approximate causal functions using exponentially fewer parameters than ReLU networks.
Star formation of far-IR AGN and non-AGN galaxies in the green valley: possible implication of AGN positive feedback

In this paper we present the star formation properties of $\rm{I_{subaru}\leq}$ 23 X-ray detected AGN and non-AGN galaxies in the green valley with far-IR (FIR) emission, using data from the COSMOS field. We measured star formation rates (SFR) using FIR Herschel/PACS data and we observed the location of AGN and non-AGN galaxies on the main-sequence of star formation. We went a step further in analysing the importance of AGN in quenching star formation in the green valley, the region proposed to be the transitional phase in galaxy evolution where galaxies are moving from later- to earlier-types. We found that most of our green valley X-ray detected AGN with far-IR emission have SFRs higher than the ones of inactive galaxies at fixed stellar mass ranges, the result that is different when considering optical data. These FIR AGN have still very active star formation, being located either on or above the main-sequence of star formation (in total 82\% of our sample). Therefore, they do not show signs of star formation quenching, but rather its enhancement. Our results may suggest that for X-ray detected AGN with FIR emission if there is an influence of AGN feedback on the star formation in the green valley the scenario of AGN positive feedback seem to take place, rather than the negative one.
Symmetric (Optimistic) Natural Policy Gradient for Multi-agent Learning with Parameter Convergence

Multi-agent interactions are increasingly important in the context of reinforcement learning, and the theoretical foundations of policy gradient methods have attracted surging research interest. We investigate the global convergence of natural policy gradient (NPG) algorithms in multi-agent learning. We first show that vanilla NPG may not have parameter convergence, i.e., the convergence of the vector that parameterizes the policy, even when the costs are regularized (which enabled strong convergence guarantees in the policy space in the literature). This non-convergence of parameters leads to stability issues in learning, which becomes especially relevant in the function approximation setting, where we can only operate on low-dimensional parameters, instead of the high-dimensional policy. We then propose variants of the NPG algorithm, for several standard multi-agent learning scenarios: two-player zero-sum matrix and Markov games, and multi-player monotone games, with global last-iterate parameter convergence guarantees. We also generalize the results to certain function approximation settings. Note that in our algorithms, the agents take symmetric roles. Our results might also be of independent interest for solving nonconvex-nonconcave minimax optimization problems with certain structures. Simulations are also provided to corroborate our theoretical findings.