Miracle-dz/UniFAR-FTUs huggingface dataset:
Predictive Coding beyond Gaussian Distributions

A large amount of recent research has the far-reaching goal of finding training methods for deep neural networks that can serve as alternatives to backpropagation (BP). A prominent example is predictive coding (PC), which is a neuroscience-inspired method that performs inference on hierarchical Gaussian generative models. These methods, however, fail to keep up with modern neural networks, as they are unable to replicate the dynamics of complex layers and activation functions. In this work, we solve this problem by generalizing PC to arbitrary probability distributions, enabling the training of architectures, such as transformers, that are hard to approximate with only Gaussian assumptions. We perform three experimental analyses. First, we study the gap between our method and the standard formulation of PC on multiple toy examples. Second, we test the reconstruction quality on variational autoencoders, where our method reaches the same reconstruction quality as BP. Third, we show that our method allows us to train transformer networks and achieve a performance comparable with BP on conditional language models. More broadly, this method allows neuroscience-inspired learning to be applied to multiple domains, since the internal distributions can be flexibly adapted to the data, tasks, and architectures used.
A Survey of Deep Face Restoration: Denoise, Super-Resolution, Deblur, Artifact Removal

Face Restoration (FR) aims to restore High-Quality (HQ) faces from Low-Quality (LQ) input images, which is a domain-specific image restoration problem in the low-level computer vision area. The early face restoration methods mainly use statistic priors and degradation models, which are difficult to meet the requirements of real-world applications in practice. In recent years, face restoration has witnessed great progress after stepping into the deep learning era. However, there are few works to study deep learning-based face restoration methods systematically. Thus, this paper comprehensively surveys recent advances in deep learning techniques for face restoration. Specifically, we first summarize different problem formulations and analyze the characteristic of the face image. Second, we discuss the challenges of face restoration. Concerning these challenges, we present a comprehensive review of existing FR methods, including prior based methods and deep learning-based methods. Then, we explore developed techniques in the task of FR covering network architectures, loss functions, and benchmark datasets. We also conduct a systematic benchmark evaluation on representative methods. Finally, we discuss future directions, including network designs, metrics, benchmark datasets, applications,etc. We also provide an open-source repository for all the discussed methods, which is available at https://github.com/TaoWangzj/Awesome-Face-Restoration.
Deep Factorization Model for Robust Recommendation

Recently, malevolent user hacking has become a huge problem for real-world companies. In order to learn predictive models for recommender systems, factorization techniques have been developed to deal with user-item ratings. In this paper, we suggest a broad architecture of a factorization model with adversarial training to get over these issues. The effectiveness of our systems is demonstrated by experimental findings on real-world datasets.
The G\"opel variety

In this paper we will prove that the six-dimensional Gopel variety in $P^{134}$ is generated by 120 linear, 35 cubic and 35 quartic relations. This result was already obtained in [RS] , but the authors used a statement in [Co] saying that the Gopel variety set theoretically is generated by the linear and cubic relations alone. Unfortunately this statement is false. There are 120 extra points. Nevertheless the results stated in [RS] are correct. There are required several changes that we will illustrate in some detail
Multilayer Perceptron Network Discriminates Larval Zebrafish Genotype using Behaviour

Zebrafish are a common model organism used to identify new disease therapeutics. High-throughput drug screens can be performed on larval zebrafish in multi-well plates by observing changes in behaviour following a treatment. Analysis of this behaviour can be difficult, however, due to the high dimensionality of the data obtained. Statistical analysis of individual statistics (such as the distance travelled) is generally not powerful enough to detect meaningful differences between treatment groups. Here, we propose a method for classifying zebrafish models of Parkinson's disease by genotype at 5 days old. Using a set of 2D behavioural features, we train a multi-layer perceptron neural network. We further show that the use of integrated gradients can give insight into the impact of each behaviour feature on genotype classifications by the model. In this way, we provide a novel pipeline for classifying zebrafish larvae, beginning with feature preparation and ending with an impact analysis of said features.
A Reliable Multipath Routing Protocol Based on Link Stability

Wireless NanoSensor Network (WNSN) is a new type of sensor network with broad application prospects. In view of the limited energy of nanonodes and unstable links in WNSNs, we propose a reliable multi-path routing based on link stability (RMRLS). RMRLS selects the optimal path which perfects best in the link stability evaluation model, and then selects an alternative route by the routing similarity judgment model. RMRLS uses tew paths to cope with changes in the network topology. The simulation shows that the RMRLS protocol has advantages in data packet transmission success rate and average throughput, which can improve the stability and reliability of the network.
Baryon history and cosmic star formation in non-Gaussian cosmological models: numerical simulations

We present the first numerical, N-body, hydrodynamical, chemical simulations of cosmic structure formation in the framework of non-Gaussian models. We study the impact of primordial non-Gaussianities on early chemistry (e−, H, H+, H−, He, He+, He++, H2, H+2, D, D+, HD, HeH+), molecular and atomic gas cooling, star formation, metal (C, O, Si, Fe, Mg, S) enrichment, Population III (popIII) and Population II–I (popII) transition and on the evolution of ‘visible’ objects. We find that non-Gaussianities can have some consequences on baryonic structure formation at very early epochs, but the subsequent evolution at later times washes out any difference among the various models. When assuming reasonable values for primordial non-Gaussian perturbations, it turns out that they are responsible for (i) altering early molecular fractions in the cold, dense gas phase by ∼10 per cent; (ii) inducing small temperature fluctuations of ≲10 per cent during the cosmic evolution of primordial objects; (iii) influencing the onset of the first star formation events, at z > 15, and of the popIII/popII transition by up to some 107 yr; (iv) determining variations of ≲10 per cent in the gas cloud and stellar mass distributions after the formation of the first structures; and (v) causing only mild variations in the chemical history of the Universe. We stress, though, that purely non-Gaussian effects might be difficult to address, since they are strictly intertwined with additional physical phenomena (e.g. primordial gas bulk flows, unknown primordial popIII stellar mass function etc.) that have a similar or stronger impact on the behaviour of the baryons.
Unleashing the Power of Graph Data Augmentation on Covariate Distribution Shift

The issue of distribution shifts is emerging as a critical concern in graph representation learning. From the perspective of invariant learning and stable learning, a recently well-established paradigm for out-of-distribution generalization, stable features of the graph are assumed to causally determine labels, while environmental features tend to be unstable and can lead to the two primary types of distribution shifts. The correlation shift is often caused by the spurious correlation between environmental features and labels that differs between the training and test data; the covariate shift often stems from the presence of new environmental features in test data. However, most strategies, such as invariant learning or graph augmentation, typically struggle with limited training environments or perturbed stable features, thus exposing limitations in handling the problem of covariate shift. To address this challenge, we propose a simple-yet-effective data augmentation strategy, Adversarial Invariant Augmentation (AIA), to handle the covariate shift on graphs. Specifically, given the training data, AIA aims to extrapolate and generate new environments, while concurrently preserving the original stable features during the augmentation process. Such a design equips the graph classification model with an enhanced capability to identify stable features in new environments, thereby effectively tackling the covariate shift in data. Extensive experiments with in-depth empirical analysis demonstrate the superiority of our approach. The implementation codes are publicly available at https://github.com/yongduosui/AIA.
Contrastive introspection (ConSpec) to rapidly identify invariant prototypes for success in RL

Reinforcement learning (RL) algorithms have achieved notable success in recent years, but still struggle with fundamental issues in long-term credit assignment. It remains diﬃcult to learn in situations where success is contingent upon multiple critical steps that are distant in time from each other and from a sparse reward; as is often the case in real life. Moreover, how RL algorithms assign credit in these diﬃcult situations is typically not coded in a way that can rapidly generalize to new situations. Here, we present an approach using oﬄine contrastive learning, which we call contrastive introspection (ConSpec), that can be added to any existing RL algorithm and addresses both issues. In ConSpec, a contrastive loss is used during oﬄine replay to identify invariances among successful episodes. This takes advantage of the fact that it is easier to retrospectively identify the small set of steps that success is contingent upon than it is to prospectively predict reward at every step taken in the environment. ConSpec stores this knowledge in a collection of prototypes summarizing the intermediate states required for success. During training, arrival at any state that matches these prototypes generates an intrinsic reward that is added to any external rewards. As well, the reward shaping provided by ConSpec can be made to preserve the optimal policy of the underlying RL agent. The prototypes in ConSpec provide two key beneﬁts for credit assignment: (1) They enable rapid identiﬁcation of all the critical states. (2) They do so in a readily interpretable manner, enabling out of distribution generalization when sensory features are altered. In summary, ConSpec is a modular system that can be added to any existing RL algorithm to improve its long-term credit assignment.
On  Vacuum Free Boundary Problem of the  Spherically Symmetric  Euler  Equations with  Damping and Solid Core

In this paper, the global existence of smooth solution and the long-time asymptotic stability of the equilibrium to vacuum free boundary problem of the spherically symmetric Euler equations with damping and solid core have been obtained for arbitrary finite positive gas constant $A$ in the state equation $p=A \rho^\gamma$ with $p$ being the pressure and $\rho$ the density, provided that $\gamma>4/3,$ initial perturbation is small and the radius of the equilibrium $R$ is suitably larger than the radius of the solid core $r_0$. Moreover, we obtain the pointwise convergence from the smooth solution to the equilibrium in a surprisingly exponential time-decay rate. The proof is mainly based on weighted energy method in Lagrangian coordinate.
Ubiquity of bound states for the strongly coupled polaron

We study the spectrum of the Fr\"ohlich Hamiltonian for the polaron at fixed total momentum. We prove the existence of excited eigenvalues between the ground state energy and the essential spectrum at strong coupling. In fact, our main result shows that the number of excited energy bands diverges in the strong coupling limit. To prove this we derive upper bounds for the min-max values of the corresponding fiber Hamiltonians and compare them with the bottom of the essential spectrum, a lower bound on which was recently obtained in [1]. The upper bounds are given in terms of the ground state energy band shifted by momentum-independent excitation energies determined by an effective Hamiltonian of Bogoliubov-type.
On a family of surfaces of general type attached to abelian fourfolds and the Weyl group W(E_6)

We study a family of surfaces of general type that arises from the intersections of two translates of the theta divisor on a principally polarized complex abelian fourfold. In particular we determine the Néron-Severi lattices of these surfaces and show that for a general abelian fourfold the monodromy group of the associated variation of Hodge structures is a subgroup of index at most two in the Weyl group W (E6), whereas for a Jacobian variety it degenerates to a subgroup of the symmetric group S6.
PASTA: Table-Operations Aware Fact Verification via Sentence-Table Cloze Pre-training

Fact verification has attracted a lot of attention recently, e.g., in journalism, marketing, and policymaking, as misinformation and dis- information can sway one’s opinion and affect one’s actions. While fact-checking is a hard task in general, in many cases, false statements can be easily debunked based on analytics over tables with reliable information. Hence, table- based fact verification has recently emerged as an important and growing research area. Yet, progress has been limited due to the lack of datasets that can be used to pre-train language models (LMs) to be aware of common table operations, such as aggregating a column or comparing tuples. To bridge this gap, this paper introduces PASTA for table-based fact verification via pre-training with synthesized sentence–table cloze questions. In particular, we design six types of common sentence–table cloze tasks, including Filter, Aggregation, Superlative, Comparative, Ordinal, and Unique, based on which we synthesize a large corpus consisting of 1.2 million sentence–table pairs from WikiTables. PASTA uses a recent pre-trained LM, DeBERTaV3, and further pre- trains it on our corpus. Our experimental results show that PASTA achieves new state-of-the-art (SOTA) performance on two table-based fact verification datasets TabFact and SEM-TAB- FACTS. In particular, on the complex set of TabFact, which contains multiple operations, PASTA largely outperforms previous SOTA by 4.7% (85.6% vs. 80.9%), and the gap between PASTA and human performance on the small test set is narrowed to just 1.5% (90.6% vs. 92.1%).
Implicit Graphon Neural Representation

Graphons are general and powerful models for generating graphs of varying size. In this paper, we propose to directly model graphons using neural networks, obtaining Implicit Graphon Neural Representation (IGNR). Existing work in modeling and reconstructing graphons often approximates a target graphon by a fixed resolution piece-wise constant representation. Our IGNR has the benefit that it can represent graphons up to arbitrary resolutions, and enables natural and efficient generation of arbitrary sized graphs with desired structure once the model is learned. Furthermore, we allow the input graph data to be unaligned and have different sizes by leveraging the Gromov-Wasserstein distance. We first demonstrate the effectiveness of our model by showing its superior performance on a graphon learning task. We then propose an extension of IGNR that can be incorporated into an auto-encoder framework, and demonstrate its good performance under a more general setting of graphon learning. We also show that our model is suitable for graph representation learning and graph generation.
Conjectures on counting associative 3-folds
 in 𝐺₂-manifolds

There is a strong analogy between compact, torsion-free $G_2$-manifolds $(X,\varphi,*\varphi)$ and Calabi-Yau 3-folds $(Y,J,g,\omega)$. We can also generalize $(X,\varphi,*\varphi)$ to 'tamed almost $G_2$-manifolds' $(X,\varphi,\psi)$, where we compare $\varphi$ with $\omega$ and $\psi$ with $J$. Associative 3-folds in $X$, a special kind of minimal submanifold, are analogous to $J$-holomorphic curves in $Y$. Several areas of Symplectic Geometry -- Gromov-Witten theory, Quantum Cohomology, Lagrangian Floer cohomology, Fukaya categories -- are built using 'counts' of moduli spaces of $J$-holomorphic curves in $Y$, but give an answer depending only on the symplectic manifold $(Y,\omega)$, not on the (almost) complex structure $J$. We investigate whether it may be possible to define interesting invariants of tamed almost $G_2$-manifolds $(X,\varphi,\psi)$ by 'counting' compact associative 3-folds $N\subset X$, such that the invariants depend only on $\varphi$, and are independent of the 4-form $\psi$ used to define associative 3-folds. We conjecture that one can define a superpotential $\Phi_\psi:{\mathcal U}\to\Lambda_{>0}$ 'counting' associative $\mathbb Q$-homology 3-spheres $N\subset X$ which is deformation-invariant in $\psi$ for $\varphi$ fixed, up to certain reparametrizations $\Upsilon:{\mathcal U}\to{\mathcal U}$ of the base ${\mathcal U}=$Hom$(H_3(X;{\mathbb Z}),1+\Lambda_{>0})$, where $\Lambda_{>0}$ is a Novikov ring. Using this we define a notion of '$G_2$ quantum cohomology'. These ideas may be relevant to String Theory or M-Theory on $G_2$-manifolds. We also discuss Donaldson and Segal's proposal in arXiv:0902.3239, section 6.2, to define invariants 'counting' $G_2$-instantons on tamed almost $G_2$-manifolds $(X,\varphi,\psi)$, with 'compensation terms' counting weighted pairs of a $G_2$-instanton and an associative 3-fold, and suggest some modifications to it.
Cold quarks in medium: an equation of state

We derive a compact, semialgebraic expression for the cold-quark-matter equation of state (EoS) in a covariant model that exhibits coincident deconfinement and chiral symmetry restoring transitions in medium. In doing so we obtain algebraic expressions for the number- and scalar-density distributions in both the confining Nambu and the deconfined Wigner phases and the vacuum-pressure difference between these phases, which defines a bag constant. Our qualitative study illustrates that a confining interaction can materially alter distribution functions from those of a Fermi gas and impact significantly on a system's thermodynamic properties, possibilities that are apparent in the EoS.
Noisy Channel for Automatic Text Simplification

In this paper we present a simple re-ranking method for Automatic Sentence Simplification based on the noisy channel scheme. Instead of directly computing the best simplification given a complex text, the re-ranking method also considers the probability of the simple sentence to produce the complex counterpart, as well as the probability of the simple text itself, according to a language model. Our experiments show that combining these scores outperform the original system in three different English datasets, yielding the best known result in one of them. Adopting the noisy channel scheme opens new ways to infuse additional information into ATS systems, and thus to control important aspects of them, a known limitation of end-to-end neural seq2seq generative models.
A Spectral Analysis of Graph Neural Networks on Dense and Sparse Graphs

In this work we propose a random graph model that can produce graphs at different levels of sparsity. We analyze how sparsity affects the graph spectra, and thus the performance of graph neural networks (GNNs) in node classification on dense and sparse graphs. We compare GNNs with spectral methods known to provide consistent estimators for community detection on dense graphs, a closely related task. We show that GNNs can outperform spectral methods on sparse graphs, and illustrate these results with numerical examples on both synthetic and real graphs.
Non-Markovian dynamics of a single-mode cavity strongly coupled to an inhomogeneously broadened spin ensemble

We study the dynamics of a spin ensemble strongly coupled to a single-mode resonator driven by external pulses. When the mean frequency of the spin ensemble is in resonance with the cavity mode, damped Rabi oscillations are found between the spin ensemble and the cavity mode which we describe very accurately, including the dephasing effect of the inhomogeneous spin broadening. We demonstrate that a precise knowledge of this broadening is crucial both for a qualitative and a quantitative understanding of the temporal spin-cavity dynamics. On this basis we show that coherent oscillations between the spin ensemble and the cavity can be enhanced by a few orders of magnitude, when driving the system with pulses that match special resonance conditions. Our theoretical approach is tested successfully with an experiment based on an ensemble of negatively charged nitrogen-vacancy (NV) centers in diamond strongly coupled to a superconducting coplanar single-mode waveguide resonator.
Exact reconstruction formulas for a Radon transform over cones

Inversion of Radon transforms is the mathematical foundation of many modern tomographic imaging modalities. In this paper, we study a conical Radon transform, which is important for computed tomography taking Compton scattering into account. This transform integrates a function in Rd? > over all conical surfaces having vertices on a hyperplane and symmetry axis orthogonal to this plane. As the main result, we derive exact reconstruction formulas of the filtered back-projection type for inverting this transform. Further, a Fourier slice identity is established.
Unfitted mixed finite element methods for elliptic interface problems

In this article, new unfitted mixed finite elements are presented for elliptic interface problems with jump coefficients. Our model is based on a fictitious domain formulation with distributed Lagrange multiplier. The relevance of our investigations is better seen when applied to the framework of fluid‐structure interaction problems. Two finite element schemes with piecewise constant Lagrange multiplier are proposed and their stability is proved theoretically. Numerical results compare the performance of those elements, confirming the theoretical proofs and verifying that the schemes converge with optimal rates.
Blow-up results for semilinear wave equations in the super-conformal case

We consider the semilinear wave equation in higher dimensions with power nonlinearity in the super-conformal range, and its perturbations with lower order terms, including the Klein-Gordon equation. We improve the upper bounds on blow-up solutions previously obtained by Killip, Stovall and Vi\c{s}an [6]. Our proof uses the similarity variables' setting. We consider the equation in that setting as a perturbation of the conformal case, and we handle the extra terms thanks to the ideas we already developed in [5] for perturbations of the pure power case with lower order terms.
Analytical Scattering Solution of Huygens' Refracting Metasurfaces Reveals Unknown Functionalities

Scattering of refracting Huygens’ metasurfaces is revisited. A new analytical closed-form solution is obtained for the two-dimensional problem under transverse electric (TE) plane-wave incidence – a solution which gives the scattering for any (in-plane) angle of incidence and frequency. In the process of obtaining the solution, interesting mathematical concepts are considered – such as solving and enforcing uniqueness upon an inﬁnite system of equations. The obtained solution uncovers two previously unknown functionalities of refracting Huygens’ metasurfaces – namely reﬂectionless refraction and oblique absorption.