Miracle-dz/UniFAR-FTUs huggingface dataset:
Mapping hydrodynamics for the facilitated exclusion and zero-range processes

We derive the hydrodynamic limit for two degenerate lattice gases, the \emph{facilitated exclusion process} (FEP) and the \emph{facilitated zero-range process} (FZRP), both in the symmetric and the asymmetric case. For both processes, the hydrodynamic limit in the symmetric case takes the form of a diffusive Stefan problem, whereas the asymmetric case is characterized by a hyperbolic Stefan problem. Although the FZRP is attractive, a property that we extensively use to derive its hydrodynamic limits in both cases, the FEP is not. To derive the hydrodynamic limit for the latter, we exploit that of the zero-range process, together with a classical mapping between exclusion and zero-range processes, both at the microscopic and macroscopic level. Due to the degeneracy of both processes, the asymmetric case is a new result, but our work also provides a simpler proof than the one that was previously proposed for the FEP in the symmetric case in \cite{blondel2021stefan}.
Optimal learning rate schedules in high-dimensional non-convex optimization problems

Learning rate schedules are ubiquitously used to speed up and improve optimisation. Many different policies have been introduced on an empirical basis, and theoretical analyses have been developed for convex settings. However, in many realistic problems the loss-landscape is high-dimensional and non convex -- a case for which results are scarce. In this paper we present a first analytical study of the role of learning rate scheduling in this setting, focusing on Langevin optimization with a learning rate decaying as $\eta(t)=t^{-\beta}$. We begin by considering models where the loss is a Gaussian random function on the $N$-dimensional sphere ($N\rightarrow \infty$), featuring an extensive number of critical points. We find that to speed up optimization without getting stuck in saddles, one must choose a decay rate $\beta<1$, contrary to convex setups where $\beta=1$ is generally optimal. We then add to the problem a signal to be recovered. In this setting, the dynamics decompose into two phases: an \emph{exploration} phase where the dynamics navigates through rough parts of the landscape, followed by a \emph{convergence} phase where the signal is detected and the dynamics enter a convex basin. In this case, it is optimal to keep a large learning rate during the exploration phase to escape the non-convex region as quickly as possible, then use the convex criterion $\beta=1$ to converge rapidly to the solution. Finally, we demonstrate that our conclusions hold in a common regression task involving neural networks.
Reproducibility in Optimization: Theoretical Framework and Limits

We initiate a formal study of reproducibility in optimization. We define a quantitative measure of reproducibility of optimization procedures in the face of noisy or error-prone operations such as inexact or stochastic gradient computations or inexact initialization. We then analyze several convex optimization settings of interest such as smooth, non-smooth, and strongly-convex objective functions and establish tight bounds on the limits of reproducibility in each setting. Our analysis reveals a fundamental trade-off between computation and reproducibility: more computation is necessary (and sufficient) for better reproducibility.
Missing Data Imputation and Acquisition with Deep Hierarchical Models and Hamiltonian Monte Carlo

Variational Autoencoders (VAEs) have recently been highly successful at imputing and acquiring heterogeneous missing data. However, within this specific application domain, existing VAE methods are restricted by using only one layer of latent variables and strictly Gaussian posterior approximations. To address these limitations, we present HH-VAEM, a Hierarchical VAE model for mixed-type incomplete data that uses Hamiltonian Monte Carlo with automatic hyper-parameter tuning for improved approximate inference. Our experiments show that HH-VAEM outperforms existing baselines in the tasks of missing data imputation and supervised learning with missing features. Finally, we also present a sampling-based approach for efficiently computing the information gain when missing features are to be acquired with HH-VAEM. Our experiments show that this sampling-based approach is superior to alternatives based on Gaussian approximations.
Universal Hopfield Networks: A General Framework for Single-Shot Associative Memory Models

A large number of neural network models of associative memory have been proposed in the literature. These include the classical Hopfield networks (HNs), sparse distributed memories (SDMs), and more recently the modern continuous Hopfield networks (MCHNs), which possess close links with self-attention in machine learning. In this paper, we propose a general framework for understanding the operation of such memory networks as a sequence of three operations: similarity, separation, and projection. We derive all these memory models as instances of our general framework with differing similarity and separation functions. We extend the mathematical framework of Krotov & Hopfield (2020) to express general associative memory models using neural network dynamics with local computation, and derive a general energy function that is a Lyapunov function of the dynamics. Finally, using our framework, we empirically investigate the capacity of using different similarity functions for these associative memory models, beyond the dot product similarity measure, and demonstrate empirically that Euclidean or Manhattan distance similarity metrics perform substantially better in practice on many tasks, enabling a more robust retrieval and higher memory capacity than existing models.
Hopf bifurcations in the full SKT model and where to find them

In this paper, we consider the Shigesada–Kawasaki–Teramoto (SKT) model, which presents cross-diffusion terms describing competition pressure effects. Even though the reaction part does not present the activator–inhibitor structure, cross-diffusion can destabilise the homogeneous equilibrium. However, in the full cross-diffusion system and weak competition regime, the cross-diffusion terms have an opposite effect and the bifurcation structure of the system modifies increasing the interspecific competition pressure. The major changes in the bifurcation structure, the type of pitchfork bifurcations on the homogeneous branch, as well as the presence of Hopf bifurcation points are here investigated. Through weakly nonlinear analysis, we can predict the type of pitchfork bifurcation. Increasing the additional cross-diffusion coefficients, the first two pitchfork bifurcation points from super-critical become sub-critical, leading to the appearance of a multi-stability region. The interspecific competition pressure also influences the possible appearance of stable time-period spatial patterns appearing through a Hopf bifurcation point.
Exceptional surgeries on hyperbolic fibered knots

Let K ⊂ S be a hyperbolic fibered knot such that S p/q(K), the p q – surgery on K, is non-hyperbolic. We prove that if the monodromy of K is right-veering, then 0 ≤ p q ≤ 4g(K). The upper bound 4g(K) cannot be attained if S p/q(K) is a small Seifert fibered L-space. If the monodromy of K is neither right-veering nor left-veering, then |q| ≤ 3. As a corollary, for any given positive torus knot T , if p/q ≥ 4g(T )+ 4, then p/q is a characterizing slope. This improves earlier bounds of Ni– Zhang and McCoy. We also prove that some finite/cyclic slopes are characterizing. More precisely, 14 is characterizing for T4,3, 17 is characterizing for T5,3, and 4n + 1 is characterizing for T2n+1,2 except when n = 5. By a recent theorem of Tange, this shows that T2n+1,2 is the only knot in S 3 admitting a lens space surgery while the Alexander polynomial has the form t − t + t + lower order terms. In the appendix, we prove that if the rank of the second term of the knot Floer homology of a fibered knot is 1, then the monodromy is either right-veering or left-veering.
Rethinking Adam: A Twofold Exponential Moving Average Approach

Adaptive gradient methods, e.g. \textsc{Adam}, have achieved tremendous success in machine learning. Scaling the learning rate element-wisely by a certain form of second moment estimate of gradients, such methods are able to attain rapid training of modern deep neural networks. Nevertheless, they are observed to suffer from compromised generalization ability compared with stochastic gradient descent (\textsc{SGD}) and tend to be trapped in local minima at an early stage during training. Intriguingly, we discover that substituting the gradient in the second raw moment estimate term with its momentumized version in \textsc{Adam} can resolve the issue. The intuition is that gradient with momentum contains more accurate directional information and therefore its second moment estimation is a more favorable option for learning rate scaling than that of the raw gradient. Thereby we propose \textsc{AdaMomentum} as a new optimizer reaching the goal of training fast while generalizing much better. We further develop a theory to back up the improvement in generalization and provide convergence guarantees under both convex and nonconvex settings. Extensive experiments on a wide range of tasks and models demonstrate that \textsc{AdaMomentum} exhibits state-of-the-art performance and superior training stability consistently.
Neutrino astronomy as a probe of physics beyond the Standard Model: Decay of sub-MeV 
B−L
 gauge boson dark matter

The $U(1)_{B\textrm{--}L}$ symmetry, the essential component in the seesaw mechanism and leptogenesis, is naturally equipped with a massive gauge boson. If this gauge boson is the dark matter, the scenario consistent with the seesaw mechanism predicts the gauge coupling to be of the order of $\mathcal{O}(10^{-19})$ for masses $\lesssim1$ MeV, dominantly decaying into active neutrinos. We stress and explore the important role of astrophysical neutrinos of energies from $\mathcal{O}(1)$ keV to $\sim1$ MeV in testing the well-motivated $B$-$L$ symmetry extension to the Standard Model, which has been missed in the literature to date. Compared to other dark matter models, the neutrino flux in the sub-MeV energy range is a unique prediction in our setup and, once detected, would serve as a smoking gun for the existence of this $B$-$L$ gauge boson and its role as the dark matter particle, opening new windows to tackle cosmological and astrophysical conundra.
Towards Compositional Adversarial Robustness: Generalizing Adversarial Training to Composite Semantic Perturbations

Model robustness against adversarial examples of single perturbation type such as the $\ell_{p}$ -norm has been widely studied, yet its generalization to more realistic scenarios involving multiple semantic perturbations and their composition remains largely unexplored. In this paper, we first propose a novel method for generating composite adversarial examples. Our method can find the optimal attack composition by utilizing component-wise projected gradient descent and automatic attack-order scheduling. We then propose generalized adversarial training (GAT) to extend model robustness from $\ell_{p}$ -ball to composite semantic perturbations, such as the combination of Hue, Saturation, Brightness, Contrast, and Rotation. Results obtained using ImageNet and CIFAR-10 datasets indicate that GAT can be robust not only to all the tested types of a single attack, but also to any combination of such attacks. GAT also outperforms baseline $\ell_{\infty}$ -norm bounded adversarial training approaches by a significant margin.
A Neural Phillips Curve and a Deep Output Gap

Many problems plague the estimation of Phillips curves. Among them is the hurdle that the two key components, inflation expectations and the output gap, are both unobserved. Traditional remedies include creating reasonable proxies for the notable absentees or extracting them via some form of assumptions-heavy filtering procedure. I propose an alternative route: a Hemisphere Neural Network (HNN) whose peculiar architecture yields a final layer where components can be interpreted as latent states within a Neural Phillips Curve. There are benefits. First, HNN conducts the supervised estimation of nonlinearities that arise when translating a high-dimensional set of observed regressors into latent states. Second, computations are fast. Third, forecasts are economically interpretable. Fourth, inflation volatility can also be predicted by merely adding a hemisphere to the model. Among other findings, the contribution of real activity to inflation appears severely underestimated in traditional econometric specifications. Also, HNN captures out-of-sample the 2021 upswing in inflation and attributes it first to an abrupt and sizable disanchoring of the expectations component, followed by a wildly positive gap starting from late 2020. HNN’s gap unique path comes from dispensing with unemployment and GDP in favor of an amalgam of nonlinearly processed alternative tightness indicators – some of which are skyrocketing as of early 2022. *Departement des Sciences Économiques, goulet_coulombe. philippe@uqam.ca. First and foremost, I am grateful to Hugo Couture for outstanding research assistance. For helpful discussions and/or comments on an earlier draft, I would like to thank Frank Diebold, Mikael Frenette, Alain Guay, Maximilian Goebel, Guillaume Poulin-Bellisle, Josefine Quast, Dalibor Stevanovic, David Wigglesworth, Boyuan Zhang, as well as participants at the UQAM weekly seminar, at at the CIRANO Montreal Macro Workshop, the SKEMA/CERGY Inflation Forecasting Workshop 2021, at the Bank of Canada Macro Brownbag Seminar, at the CFE 2021. ar X iv :2 20 2. 04 14 6v 1 [ ec on . E M ] 8 F eb 2 02 2
Semiparametric Bayesian Estimation of Dynamic Discrete Choice Models

We propose a tractable semiparametric estimation method for structural dynamic discrete choice models. The distribution of additive utility shocks in the proposed framework is modeled by location-scale mixtures of extreme value distributions with varying numbers of mixture components. Our approach exploits the analytical tractability of extreme value distributions in the multinomial choice settings and the flexibility of the location-scale mixtures. We implement the Bayesian approach to inference using Hamiltonian Monte Carlo and an approximately optimal reversible jump algorithm. In our simulation experiments, we show that the standard dynamic logit model can deliver misleading results, especially about counterfactuals, when the shocks are not extreme value distributed. Our semiparametric approach delivers reliable inference in these settings. We develop theoretical results on approximations by location-scale mixtures in an appropriate distance and posterior concentration of the set identified utility parameters and the distribution of shocks in the model.
Quantum limits to resolution and discrimination of spontaneous emission lifetimes

In this work we investigate the quantum information theoretical limits to several tasks related to lifetime estimation and discrimination of a two-level spontaneous optical emitter. We focus in particular on the model problem of resolving two mutually incoherent exponential decays with highly overlapping temporal probability profiles. Mirroring recent work on quantum-inspired super-resolution of point emitters, we find that direct lifetime measurement suffers from an analogue of"Rayleigh's Curse"when the time constants of the two decay channels approach one another. We propose alternative measurement schemes that circumvent this limit, and also demonstrate superiority to direct measurement for a related binary hypothesis test. Our findings add to a growing list of examples in which a quantum analysis uncovers significant information gains for certain tasks in opto-molecular metrology that do not rely on multiphoton interference, but evidently do benefit from a more thorough exploitation of the coherence properties of single photons.
On the Implicit Bias of Gradient Descent for Temporal Extrapolation

When using recurrent neural networks (RNNs) it is common practice to apply trained models to sequences longer than those seen in training. This"extrapolating"usage deviates from the traditional statistical learning setup where guarantees are provided under the assumption that train and test distributions are identical. Here we set out to understand when RNNs can extrapolate, focusing on a simple case where the data generating distribution is memoryless. We first show that even with infinite training data, there exist RNN models that interpolate perfectly (i.e., they fit the training data) yet extrapolate poorly to longer sequences. We then show that if gradient descent is used for training, learning will converge to perfect extrapolation under certain assumptions on initialization. Our results complement recent studies on the implicit bias of gradient descent, showing that it plays a key role in extrapolation when learning temporal prediction models.
MaskGIT: Masked Generative Image Transformer

Generative transformers have experienced rapid popularity growth in the computer vision community in synthesizing high-fidelity and high-resolution images. The best generative transformer models so far, however, still treat an image naively as a sequence of tokens, and decode an image sequentially following the raster scan ordering (i.e. line-by-line). We find this strategy neither optimal nor efficient. This paper proposes a novel image synthesis paradigm using a bidirectional transformer decoder, which we term MaskGIT. During training, MaskGIT learns to predict randomly masked tokens by attending to tokens in all directions. At inference time, the model begins with generating all tokens of an image simultaneously, and then refines the image iteratively conditioned on the previous generation. Our experiments demonstrate that MaskGIT significantly outperforms the state-of-the-art transformer model on the ImageNet dataset, and accelerates autoregressive decoding by up to 48x. Besides, we illustrate that MaskGIT can be easily extended to various image editing tasks, such as inpainting, extrapolation, and image manipulation. Project page: masked-generative-image-transformer.github. io.
Kuznetsov’s Fano threefold conjecture via K3 categories and enhanced group actions

Abstract We settle the last open case of Kuznetsov’s conjecture on the derived categories of Fano threefolds. Contrary to the original conjecture, we prove the Kuznetsov components of quartic double solids and Gushel–Mukai threefolds are never equivalent, as recently shown independently by Zhang. On the other hand, we prove the modified conjecture asserting their deformation equivalence. Our proof of nonequivalence combines a categorical Enriques-K3 correspondence with the Hodge theory of categories. Along the way, we obtain a categorical description of the periods of Gushel–Mukai varieties, which we use to resolve a conjecture of Kuznetsov and the second author on the birational categorical Torelli problem, as well as to give a simple proof of a theorem of Debarre and Kuznetsov on the fibers of the period map. Our proof of deformation equivalence relies on results of independent interest about obstructions to enhancing group actions on categories.
Detecting and Localizing Copy-Move and Image-Splicing Forgery

In the world of fake news and deepfakes, there have been an alarmingly large number of cases of images being tampered with and published in newspapers, used in court, and posted on social media for defamation purposes. Detecting these tampered images is an important task and one we try to tackle. In this paper, we focus on the methods to detect if an image has been tampered with using both Deep Learning and Image transformation methods and comparing the performances and robustness of each method. We then attempt to identify the tampered area of the image and predict the corresponding mask. Based on the results, suggestions and approaches are provided to achieve a more robust framework to detect and identify the forgeries.
Quenching of Massive Disk Galaxies in the IllustrisTNG Simulation

A rare population of massive disk galaxies have been found to invade the red sequence dominated by early-type galaxies. The formation and origins of these red/quenched massive disk galaxies have recently gained great interest. The quenching mechanisms that are usually proposed, such as bar quenching and environment quenching, do not seem to be suitable for those bulgeless quenched disks in a low-density environment. In this paper, we use the TNG300 simulation to investigate the formation of massive quenched central disk galaxies. It is found that these galaxies contain less gas than their star-forming counterparts and harbor giant super massive black holes (SMBHs; above 108 M ⊙). By tracing their formation history, we found that quenched disk galaxies formed early and preserved disk morphology for cosmological timescales. They have experienced less than one major merger on average, and mini-mergers (mass ratio <1/10) have mainly contributed to the growth of their SMBHs. In the IllustrisTNG simulation, the black hole feedback mode switches from thermal to kinetic feedback when the black hole mass is more massive than ∼108 M ⊙, which is more efficient to eject gas outside of the galaxy and to suppress further cooling of the hot gaseous halo. We conclude that the dominant quenching mechanism in massive red/quenched disk galaxies is kinetic active galactic nuclei feedback.
Sparse superposition codes under VAMP decoding with generic rotational invariant coding matrices

Sparse superposition codes were originally proposed as a capacity-achieving communication scheme over the gaussian channel, whose coding matrices were made of i.i. d. gaussian entries [1]. We extend this coding scheme to more generic ensembles of rotational invariant coding matrices with arbitrary spectrum, which include the gaussian ensemble as a special case. We further introduce and analyse a decoder based on vector approximate message-passing (VAMP) [2]. Our main findings, based on both a standard replica symmetric potential theory and state evolution analysis, are the superiority of certain structured ensembles of coding matrices (such as partial row-orthogonal) when compared to i.i. d. matrices, as well as a spectrum-independent upper bound on VAMP’s threshold. Most importantly, we derive a simple “spectral criterion” for the scheme to be at the same time capacity-achieving while having the best possible algorithmic threshold, in the “large section size” asymptotic limit. Our results therefore provide practical design principles for the coding matrices in this promising communication scheme.
Exploring dimension learning via a penalized probabilistic principal component analysis

Establishing a low-dimensional representation of the data leads to efficient data learning strategies. In many cases, the reduced dimension needs to be explicitly stated and estimated from the data. We explore the estimation of dimension in finite samples as a constrained optimization problem, where the estimated dimension is a maximizer of a penalized profile likelihood criterion within the framework of a probabilistic principal components analysis. Unlike other penalized maximization problems that require an ‘optimal’ penalty tuning parameter, we propose a data-averaging procedure whereby the estimated dimension emerges as the most favourable choice over a range of plausible penalty parameters. The proposed heuristic is compared to a large number of alternative criteria in simulations and an application to gene expression data. Extensive simulation studies reveal that none of the methods uniformly dominate the other and highlight the importance of subject-specific knowledge in choosing statistical methods for dimension learning. Our application results also suggest that gene expression data have a higher intrinsic dimension than previously thought. Overall, our proposed heuristic strikes a good balance and is the method of choice when model assumptions deviated moderately.