# The high-d landscapes paradigm: spin-glasses, and beyond

Valentina Ros, Yan V. Fyodorov

► To cite this version:

Valentina Ros, Yan V. Fyodorov. The high-d landscapes paradigm: spin-glasses, and beyond. Patrick Charbonneau; Enzo Marinari; Giorgio Parisi; Federico Ricci-Tersenghi; Gabriele Sicuro; Francesco Zamponi; Marc Mézard. Spin Glass Theory and Far Beyond: Replica Symmetry Breaking after 40 Years, World Scientific Publishing Company, 24 p., 2022, 10.48550/arXiv.2209.07975 . hal-03807198

HAL Id: hal-03807198

https://hal.science/hal-03807198

Submitted on 9 Oct 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Chapter 6

## The high-$d$ landscapes paradigm: spin-glasses, and beyond

Valentina Ros† and Yan V. Fyodorov‡

*Université Paris-Saclay, CNRS, LPTMS, 91405, Orsay, France

†Department of Mathematics, King's College London, United Kingdom

In this Chapter we review recent developments on the characterization of random landscapes in high-dimension. We focus in particular on the problem of characterizing the landscape topology and geometry, discussing techniques to count and classify its stationary points and stressing connections with the statistical physics of disordered systems and with random matrix theory.

<footnote>†valentina.ros@universite-paris-saclay.fr</footnote>

<footnote>‡yan.fyodorov@kcl.ac.uk</footnote>
The idea of corrugated landscapes was ubiquitous in many sciences, however it was not easy to put all these things together and to produce a theory for these kind of phenomena [1]. As G. Parisi recalls in his Nobel lecture, the study of glasses has been instrumental in providing such a theory, allowing to put into a quantitative framework what was up to then regarded mostly as a ‘useful metaphor’ [2]. In an attempt to characterize metastability and slow dynamics in glasses, several tools to count and classify the stationary points (local minima, maxima and saddles) of complicated, very non-convex (free)-energy landscapes have been conceived and developed. Given that corrugated landscapes are ubiquitous, these techniques are expected to play a relevant role in other contexts involving rugged landscapes to optimize, being them fitness landscapes in biology [3, 4], utility functions in economics [5], cost functions in constraint satisfaction or inference problems [6–8], loss landscapes in supervised learning [9], and obviously energy landscapes in condensed and soft matter physics [10], but also string theory and cosmology [11, 12]. Most of these settings are naturally high-dimensional: for instance, the space of genotypes over which fitness landscapes are defined is combinatorially large; in the same vein, training huge artificial neural networks like those used in current deep learning applications requires to find good minima of loss landscapes depending on an extremely large number of parameters. Therefore, techniques developed in the context of the mean-field study of glasses [13], where this high-dimensional limit is built in, are potentially useful to tackle also newly-emergent problems in biology, computer science and so on. In addition, the complexity of the landscapes and the proliferating number of local attractors of optimization algorithms (local minima or, in the language of glasses, metastable states) makes it reasonable to adopt a statistical framework. In fact, in several fields it is costumary to model these landscapes by means of random functions, and to seek for a statistical description of their properties. The problem is also of great mathematical interest, lying at the intersection between statistics, probability and differential geometry [14]. In this Chapter, we aim at briefly summarising the efforts made to substantiate this landscape paradigm, with a particular focus on more recent developments and applications. We will not discuss dynamics, for which we refer the reader to the other Chapters of this book.

## 6.1. Modelling complex landscapes: high-d random fields

Complex high-$d$ landscapes are usually modelled as random scalar fields $V(s)$ defined on configuration spaces $C_N$ of high dimensionality $N \gg 1$, $s = (s_1, \cdots, s_N) \in C_N$. The variables $s_i$ (representing the state of a spin, particle, neuronal connection, gene, agent and so on) are either discrete, e.g. $s_i = \pm 1$, or continuous, e.g. $s_i \in \mathbb{R}$. In the following we denote with $\|s - s'\|$ the Hamming or Euclidean distance between two configurations, and we refer to the normalized scalar product $s \cdot s'/N$ as their *overlap*. The landscape is random in the sense that the field $V(s)$ at each configuration $s$ depends explicitly on the *disorder*, i.e. on certain random variables. We denote with $\mathbb{E}[\cdot]$ the average over this randomness. The interest lies in characterizing the structure of the landscape statistically, and in particular in capturing *typical* properties which occur
with probability converging to one as $N \to \infty$, as well as *atypical* properties, that in the large-$N$ limit are captured by large-deviation theory. To do so in a quantitative way, it is necessary to make some assumptions on the fluctuations of $\mathcal{V}(s)$.

We focus mostly on the simplest and most ubiquitous case, that of isotropic Gaussian random fields: $\mathcal{V}(s)$ is thus assumed to have Gaussian fluctuations, with correlations between the field at two different configurations $s$ and $s'$ that depend only on the distance between them in $C_N$,

$$ \mathbb{E}[\mathcal{V}(s)\mathcal{V}(s')] - \mathbb{E}[\mathcal{V}(s)]\mathbb{E}[\mathcal{V}(s')] = NF_2\left(\frac{\|s - s'\|^2}{2N}\right). \quad (6.1) $$

Notable members of this class of random fields are obtained with the following parametrization:

$$ \mathcal{V}_{\mathbf{J}}(\mathbf{s}) = \sum_{p=2}^{\infty} \frac{\alpha_p}{N^{\frac{p-1}{2}}} \sum_{i_1, i_2, \dots, i_p} J_{i_1 i_2 \dots i_p} s_{i_1} \cdots s_{i_p}, \quad (6.2) $$

where for any $p$ the parameters $J_{i_1 i_2 \dots i_p}$ are independent, centered Gaussian variables with unit variance, and the configuration space $C_N$ is either a discrete hypercube $\{\pm 1\}^{\otimes N}$, or the surface $S_N$ of the $N$-dimensional hypersphere $S_N = \{\mathbf{s} : \sum_{i=1}^N s_i^2 = N\}$. In the latter case, the constraint $\sum_p 2^p \alpha_p < \infty$ guarantees that the field is smooth and Morse almost surely, meaning that all its stationary points are non-degenerate [15]. It is straightforward to see that the covariance of (6.2) is isotropic:

$$ \mathbb{E}[\mathcal{V}_{\mathbf{J}}(\mathbf{s})\mathcal{V}_{\mathbf{J}}(\mathbf{s}')] = N \sum_{p=2}^{\infty} \alpha_p^2 \left(\frac{\mathbf{s} \cdot \mathbf{s}'}{N}\right)^p \quad (6.3) $$

and thus (6.1) holds with $F_2(x) = \sum_p \alpha_p^2 (1-x)^p$. In the physics literature, random functions of the form (6.2) go under the name of (spherical) $p$-spin Hamiltonians: the spherical ones (when $C_N = S_N$) have been introduced in [16, 17] as a generalization to continuous variables of the standard spin-glass models defined for binary variables $s_i = \pm 1$, such as the Sherrington-Kirkpatrick model [18]. When $\alpha_{p'} = \delta_{p,p'}$ for a fixed $p$, the model is referred to as pure (as opposed to mixed) in physics, or as random Gaussian monomial in mathematics. Notice also that given that $\mathbf{s} \cdot \mathbf{s}'/N \le 1$, when $p \to \infty$ the covariance (6.3) vanishes, and the landscape reduces to that of the ‘simplest spin-glass’, the random energy model [19], which is uncorrelated at each point in configuration space.

Landscapes of the form (6.2) are centered; in many cases of interest, however, it makes sense to tilt the isotropic Gaussian field $\mathcal{V}(s)$ with non-random functions of $s$, for instance:

$$ \mathcal{F}(s; r) = \mathcal{V}(s) - rNf\left(\frac{s \cdot s_*}{N}\right), \quad (6.4) $$

where $s_*$ is some special configuration in $C_N$, and $f$ some (usually convex) function. These types of models emerge very naturally in inference settings [7, 20], where the special configuration $s_*$ is a signal embedded in the noise represented by the fluctuating part of the field. They appear also in the theoretical biology literature [21], the special configuration $s_*$ representing some preferred genotypic configuration, or the native
conformation in the landscape associated to proteins [22]. When $f$ is linear, the tilting function simply represents a magnetic field aligned with the direction $\mathbf{s}_*$. Deterministic terms of this form break the statistical isotropy, but ‘weakly’: the statistics depends on $\mathbf{s}$ only through its overlap (or the distance) to the special configuration $\mathbf{s}_*$. Tuning $r$, which measures the strength of deterministic contribution with respect to the fluctuat- ing ones, can generate transitions in the structure of the landscape (see below). In a somehow similar vein, one may consider soft variables $s_i \in [a, b] \subseteq \mathbb{R}$ and replace the spherical constraint with a confining potential [23]:

$$ \mathcal{F}(\mathbf{s}; \mu) = \mathcal{V}(\mathbf{s}) + \frac{\mu}{2} \mathbf{s} \cdot \mathbf{s}, \qquad (6.5) $$

with $\mu$ some positive mass parameter. We remark that the case of anisotropic mass $\mu \to \mu_i$ with $\mu_i$ following a certain distribution has also attracted attention recently [24–27]. Functions like (6.5) can also be thought of as the simplest incarnation of a wider class of models involving random functionals rather than functions, such as:

$$ \mathcal{F}[\mathbf{s}(\mathbf{x}); \kappa, \mu] = \int d\mathbf{x} \left( \mathcal{V}[\mathbf{s}(\mathbf{x}), \mathbf{x}] + \frac{\kappa}{2} (\nabla \mathbf{s}(\mathbf{x}))^2 + \frac{\mu}{2} \mathbf{s}(\mathbf{x})^2 \right) \qquad (6.6) $$

where now $\mathbf{s}(\mathbf{x})$ is an $N$-dimensional vector field depending on some internal $d$-dimensional state $\mathbf{x}$, the term proportional to $\kappa$ is an elastic term, $\mathcal{V}[\mathbf{s}(\mathbf{x})]$ is again a centered Gaussian field and isotropy now takes the form:

$$ \mathbb{E} [ \mathcal{V}[\mathbf{s}(\mathbf{x}), \mathbf{x}] \mathcal{V}[\mathbf{s}'(\mathbf{y}), \mathbf{y}] ] = N \delta(\mathbf{x} - \mathbf{y}) F_2 \left( \frac{\|\mathbf{s}(\mathbf{x}) - \mathbf{s}'(\mathbf{x})\|^2}{2N} \right). \qquad (6.7) $$

Of course, for $d \to 0$ this reduces to (6.5). The landscape (6.6), often referred to as the random elastic manifold energy landscape, actually appears in a broad variety of optimization problems (see for instance [28]), in which a random potential favouring configurations supported in the spots where $\mathcal{V}[\mathbf{s}(\mathbf{x}), \mathbf{x}]$ is lower, competes with elastic terms which instead promote smoother and flatter configurations.

It is worth to mention another class of Gaussian random functions with a different parametrization with respect to that of (6.2):

$$ \mathcal{V}_{\mathbf{J}}(\mathbf{s}; \alpha) = \frac{1}{\alpha N} \sum_{\mu=1}^{\alpha N} \phi(\mathbf{J}^{\mu} \cdot \mathbf{s}), \qquad (6.8) $$

where $\phi$ is a (non-linear) function, $\alpha \in (0, 1]$ and the randomness is encoded in the vectors $\mathbf{J}^{\mu} \in \mathbb{R}^N$, which in most applications are assumed to be independent Gaussian vectors. These type of functions have been extensively studied in statistical physics and in the statistical theory of learning [29–32]. An analog version of (6.4) now reads:

$$ \mathcal{V}(\mathbf{s}; \alpha) = \frac{1}{2\alpha N} \sum_{\mu=1}^{\alpha N} [\phi(\mathbf{J}^{\mu} \cdot \mathbf{s}) - \phi(\mathbf{J}^{\mu} \cdot \mathbf{s}^*)]^2, \qquad (6.9) $$

which can be seen as a toy model to study problems of generalization in machine learning in an inference-like setup [33].

In the following, we denote the landscape generically with $\mathcal{F}(\mathbf{s})$ and we mostly focus on continuous spaces $C_N$, for which methods relying on continuous differential calculus (see Sec. 6.3.3) apply. Let us stress however that the landscapes (6.3) on discrete spaces
are paradigmatic mean-field (fully-connected) models of energy functions, to which one can associate an *averaged* free-energy function, the so called Thouless-Anderson-Palmer or *TAP free-energy* [34], which depends on continuous variables — the local magnetization. Techniques similar to those discussed in Sec. 6.3.3 can then be (and have been very extensively) applied to study properties of these functions, too [35–38].

## 6.2. The questions: optimization, topology and geometry

The *landscape paradigm* is motivated by the problem of understanding the time evolution of complex systems, whenever such evolution can be thought of as some effective optimization process: either the minimization of an energy or cost, or the maximization of a fitness or utility. Typical optimization algorithms update the system configuration using the local gradient of the function to optimize $\mathcal{F}(s)$, sometimes combined with stochastic terms $\epsilon(t)$ which are unbiased with respect to the landscape,

$$ \frac{d s(t)}{d t} = -\nabla \mathcal{F}(s) + \epsilon(t). \qquad (6.10) $$

Gradient descent, Langevin dynamics and (in some broader sense) stochastic gradient descent algorithms used in current machine learning applications [39–41] are of this form. In physics, $\epsilon(t)$ is interpreted as an effective term resulting from interactions with a thermal bath, and it is usually chosen as a centered Gaussian process with equal-time variance proportional to the temperature. The information of interest to understand these algorithms is frequently associated not only with the property of the global attractor of (6.10), the global minimum, but also with the number and position of other stationary points $s_{st}$ (minima, maxima and saddle points) on the landscape surface, defined by $\nabla \mathcal{F}(s_{st}) = 0$. For clarity, we classify the landscapes properties related to their distribution into *topological* and *geometrical* properties.

### 6.2.1. Landscape’s topology

Loosely speaking, the landscape topology has to do with the total number of stationary points. The word topology in this context is motivated by Morse theory, which allows to relate topological properties of a manifold to the property of the stationary points of differentiable functions defined on it. Some questions of interest in this context are:

*The complexity of level sets.* Random landscapes in general display plenty of local minima connected by saddles. To interpret the system’s dynamical evolution it is crucial to know the number $N_N(f; k)$ of stationary points belonging to specific level sets of the landscape, i.e. such that $\lim_{N \to \infty} N^{-1} \mathcal{F}(s_{st}) = f \in \mathbb{R}$, and having a certain stability index $k \in \{0, \dots, N\}$ counting the number of independent directions in $C_N$ along which the landscape has a negative curvature ($k = 0$ for minima, $k = N$ for maxima, and all other values correspond to saddles). In the limit $N \gg 1$, the number $N_N(f, k)$ typically diverges exponentially, defining an entropy $\Sigma$ called *the landscape complexity*:

$$ \Sigma(f, k) \equiv \lim_{N \to \infty} \frac{\log N_N(f, k)}{N}. \qquad (6.11) $$
The complexity (6.11) gives information, for instance, on how deep the system can descend in the landscape without encountering local minima, and thus at which level sets one expects metastability to matter and to slow-down the optimization dynamics.

*Topology trivialization transitions.* Landscapes depending on external parameters, such as $r$ in (6.4) or $\mu$ in (6.5), can undergo transitions when tuning the parameters, between regimes where $\Sigma > 0$ and regimes where the complexity vanishes and the number of stationary points is thus sub-exponential. These transitions are known in the literature as topology trivialization transitions [42], and have a natural interpretation in terms of optimization, which is expected to be a *hard task* (hindered by metastability) when $\Sigma > 0$, and to become *easy* when $\Sigma = 0$.

### 6.2.2. Landscape's geometry

We call geometrical properties the landscape's features which have to do with how stationary points are distributed in the space $C_N$. The word geometrical is motivated by the fact that these features involve notions of distance and position in $C_N$. Questions of interest in this context are:

*Correlations between stationary points.* Stationary points of Morse functions are isolated, and one may be interested in understanding what is the typical distance (or overlap) between them or, given some reference configuration $s_*$, what is the number of minima, saddles and maxima that are at fixed overlap from it. We use for the reference configuration the same notation as for the special configuration in the models (6.4) and (6.9) since in these cases it would be a natural choice, but more generally $s_*$ may be a stationary point (e.g. the global minimum of the landscape). Questions of this type can be addressed by computing a geometrically-constrained complexity $\Sigma(f, k; q)$ as a function of an additional parameter, the overlap:

$$q = \lim_{N \to \infty} \frac{s_{st} \cdot s_{*}}{N}. \qquad (6.12)$$

Having this refined geometrical knowledge is relevant to understand ‘how badly’ metastability affects the underlying optimization problem, i.e. whether the local minima trapping the system under the dynamics (6.10) are similar (magnetized towards) the global minimum, or the signal or any other seeked configuration $s_*$ (i.e. $q > 0$), or whether they are far away and uncorrelated to it (i.e. $q = 0$). In the first case, in the inference setting the term *partial recovery* is used, to signify that minimization algorithms are likely to converge to local attractors that are at least partially informative on the signal configuration $s_*$, being at non-zero overlap with it.

*Distribution of barriers.* Ruggedness implies that moving from one local minimum to another requires to climb up in the landscape: local minima are separated by landscape's barriers, which correspond to saddles that are nearby the minima in $C_N$, but located at higher level sets. When the landscape is highly non-convex, the number of (low-index) saddles surrounding a given local minimum may be exponentially large, and
thus associated to a complexity providing information on the distribution of barriers. Determining the statistics of barriers is a non-trivial task, as it requires to control both geometrical properties (one wants to target only stationary points that are in the vicinity of a certain local minimum), but also to have a refined control of their stability; in fact, one is looking for saddles, but not arbitrary ones: such saddles must be connected to the reference minimum in $C_N$, meaning that following the unstable direction of the saddle one should move in the direction of the minimum. This non-trivial task is very relevant for understanding stochastic dynamics in regimes in which (weak) noise terms (such as $\epsilon(t)$ in (6.10)) contrast the gradient and allow the system to climb up and escape from trapping local minima via activated jumps. The resulting dynamics (which corresponds to a high-dimensional version of the well-known Kramer's escape problem) is slow, dominated by rare events (jumps between different minima) in which the configuration of the system changes substantially, and it is very challenging to describe quantitatively in the high-dimensional setting.

### 6.2.3. Dealing with the randomness: average vs typical

The above properties can be understood for fixed realizations of $\mathcal{F}(s)$, but the main interest lies in characterizing it statistically. Quantities like $N_N$ are random variables, and one needs to give a statistical meaning to the complexity. The first non-trivial quantity of interest is the first moment of $N_N$, the asymptotics of which defines the so called annealed complexity:

$$ \Sigma_{\text{ann}} \equiv \lim_{N \to \infty} \frac{\log \mathbb{E}[N_N]}{N}. \qquad (6.13) $$

As it often happens in disordered systems, however, the average of quantities fluctuating exponentially in $N \gg 1$ like $N_N$ is in general much larger than the typical value, and the two remain different when $N \to \infty$: in other words, $N_N$ is not self-averaging [43]. One would like then to describe typical realizations of $\mathcal{F}(s)$, and thus to compute the asymptotics of the typical value of $N_N$ given by the quenched complexity:

$$ \Sigma_{\text{que}} \equiv \lim_{N \to \infty} \mathbb{E} \left[ \frac{\log N_N}{N} \right]. \qquad (6.14) $$

Thus, one needs to compute averages of logarithms. The same problem obviously appears when dealing with equilibrium properties: one is in general not interested in getting the average of the partition function

$$ Z_{\beta, \mathbf{J}} = \int_{C_N} d\mathbf{s} \ e^{-\beta \mathcal{E}_{\mathbf{J}}(\mathbf{s})}, \qquad (6.15) $$

but rather of the free-energy, which is self-averaging:

$$ F(\beta) = - \lim_{N \to \infty} \mathbb{E} \left[ \frac{1}{N\beta} \log Z_{\beta, \mathbf{J}} \right]. \qquad (6.16) $$

Replicas have been introduced [44, 45] precisely as a trick to extract the expectation value of the logarithm from higher moments of the random variable, via the identity:

$$ \log N_N = \lim_{n \to 0} \frac{[N_N]^n - 1}{n}. \qquad (6.17) $$
The quenched complexities discussed below are computed making use of this trick; this requires to make some hypothesis on the structure (symmetric or not) of the correlations (overlaps) between different stationary points, similarly to what has to be done in the equilibrium setting [13]; in some fortunate cases, this structure takes a simple form and quenched and annealed complexities match (see Sec. 6.4 for an example) and can be computed in a mathematically rigorous way. In general though, the use of the replica trick forces one to give up the benefit of mathematical rigour, disclosing in exchange a universe of fascinating properties of the underlying complex systems, as testified by the other Chapters of this book.

## 6.3. Techniques for non-convex landscapes

We now give a brief summary of the techniques that have been developed to gain information on stationary points of random landscapes. To facilitate the reading, we begin with a few comments to clarify our terminology.

### 6.3.1. (Free)-energies, equilibrium, metastability

Optimizing a complex, high-dimensional landscape means seeking for its global minimum. Of course, when the landscape is very non-convex local algorithms of the form (6.10) fail in locating such configuration(s), as they get trapped into local minima located at higher level-sets of the landscape. In the literature on spin-glasses, the random functions (6.2) are effective energy functions and the optimal configuration(s), the system's ground state(s), correspond to the equilibrium state at infinite inverse temperature $\beta \to \infty$, i.e. the configuration on which the Boltzmann measure

$$ \nu_{\beta, \mathbf{J}}(s) \equiv \frac{e^{-\beta \mathcal{E}_{\mathbf{J}}(s)}}{\mathcal{Z}_{\beta, \mathbf{J}}} ds, \quad \mathcal{Z}_{\beta, \mathbf{J}} = \int_{C_N} ds \, e^{-\beta \mathcal{E}_{\mathbf{J}}(s)} \quad (6.18) $$

concentrates in that limit. In glassy systems, the above statements generalize to a full range of $\beta$: stochastic Langevin dynamics initialized randomly is unable to converge to the equilibrium states identified by the Boltzmann measure at the temperature corresponding to the noise strength, as it gets attracted by metastable states having an intensive free-energy $f$ higher than the equilibrium one (6.16). These metastable states appear to be exponentially numerous, and thus associated to a configurational complexity $\Sigma(f, \beta)$, which reduces to the complexity of local minima of the energy landscape when $\beta \to \infty$. Moreover, the Boltzmann measure itself appears to fracture into an exponential multiplicity $\Sigma(f_{\text{eq}}, \beta)$ of equilibrium states, whose internal free energy $f_{\text{eq}}$ is related to (6.16) by $F(\beta) = f_{\text{eq}} - \beta^{-1}\Sigma(f_{\text{eq}}, \beta)$. In mean-field models, these equilibrium and metastable states can be identified with stable stationary points of the TAP free-energy functionals (see Sec. 6.1) in a statistical sense, once averaging over the couplings $\mathbf{J}$ is performed.

### 6.3.2. Probing metastability within equilibrium formalisms

Characterizing metastable states is a purely dynamical problem [46]. However, in the literature on mean-field glasses techniques have been developed to extract information
on metastability via (suitably modified) thermodynamic calculations involving real repli-cas [47] or copies [48] or clones [49] of the system. The basic idea is that the structure of complex free-energy landscape can be probed considering copies of the system which evolve in the same landscape being weakly coupled to each others. We briefly review these approaches in the following, without entering into any technicality.

**Legendre transforms, or the Monasson method.** It is shown in [48] that $\Sigma(f, \beta)$ is related via a Legendre transform to the free-energy of several (say $m$) copies of the system interacting via an infinitesimal coupling term. The coupling is assumed to be strong enough to force the copies to explore the same metastable state, but weak enough so that they can be considered as independent within the state. Under this assumption, a standard thermodynamic calculation shows that the intensive free-energy $F(m, \beta)$ of the coupled system is given by:

$$F(m, \beta) = \inf_f \left[ fm - \frac{1}{\beta} \Sigma(f, \beta) \right]. \qquad (6.19)$$

Introducing

$$f_*(m, \beta) = \arg\inf_f \left[ fm - \frac{1}{\beta} \Sigma(f, \beta) \right], \qquad (6.20)$$

we see how the simple equality (6.19) allows to get $\Sigma(f, \beta)$ parametrically in the conjugate variable $m$, if the free energy $F(m, \beta)$ is known: it suffices to fix $m$, determine the corresponding $f_*(m, \beta) = \partial_m F(m, \beta) \to f$, and get the entropy associated to it as $\Sigma(f, \beta) = \beta fm - \beta F(m, \beta)$ evaluated at the corresponding $m$. The complexity curve is thus reconstructed by tuning $m$; for $m \to 1$, one gets the complexity of the equilibrium states and their internal free energy. This method boils down to computing the free-energy of a (slightly complicated) disordered system, which can be done exploiting the machinery (namely, replica theory) developed in the study of the thermodynamics of glasses. As such, it has been used extensively in the physics literature to explore metastability, for instance in more realistic models of glasses involving particles in high-dimension [50, 51], and even beyond the field of glasses [52].

**Large deviations, or the Franz-Parisì potential.** The Franz-Parisì potential $V_{FP}(q)$ is a large deviation function which measures the free-energy cost to keep one copy $\mathbf{s}$ of the system at fixed overlap $q$ from another copy $\mathbf{s}_0$, where both of them are weighted with a Bolzmann measure (6.18) with two (in general different) inverse temperatures. Explicitly,

$$V_{FP}(q) = - \lim_{N \to \infty} \frac{1}{N \beta} \mathbb{E} \left[ \frac{1}{\mathcal{Z}_{\beta_0, \mathbf{J}}} \int_{C_N} \nu_{\beta_0, \mathbf{J}}(\mathbf{s}_0) (\log \mathcal{Z}_{\beta, \mathbf{J}}(\mathbf{s}_0; q) + F(\beta)) \right], \qquad (6.21)$$

where $F(\beta)$ is as in (6.16) and

$$\mathcal{Z}_{\beta, \mathbf{J}}(\mathbf{s}_0; q) = \int_{C_N} \nu_{\beta, \mathbf{J}}(\mathbf{s}) \delta\left(q - \frac{\mathbf{s} \cdot \mathbf{s}_0}{N}\right). \qquad (6.22)$$

As shown in detail in [53, 54] and subsequent literature, in presence of metastability the potential $V_{FP}(q)$ is non-monotonic. When $\beta = \beta_0$ and one local minimum is present
as some value of $q = q_{\text{EA}}$, one has $\beta V_{\text{FP}}(q_{\text{EA}}) = \Sigma(f_{\text{eq}}, \beta)$, i.e. the potential gives the complexity of the equilibrium states. Indeed, $e^{-N\beta V_{\text{FP}}(q_{\text{EA}})}$ measures the (exponentially small) probability two find two copies of the system in the same equilibrium state, which can be thought of as being proportional to the inverse of the number of such states, $e^{-N\Sigma(f_{\text{eq}}, \beta)}$. The Franz-Parisi potential or variations of it have become a standard tool in physics [55–59], mathematics [60, 61] and computer science [62], not only to compute complexities but also to study the relationship between equilibrium states at different temperatures via the so called state following procedure. Recently, its large deviations have been studied as well [63].

These methods are very insightful but they are tailored to track stable states or local minima; in general, they need to be adapted to count states or minima that are only marginally stable, and it seems very hard to generalize them to count unstable stationary points and thus to get information on saddles and landscape’s barriers. For these aims, direct counting techniques (see the following section) seem to be more suitable.

### 6.3.3. Direct counting methods: the Kac–Rice formalism

The landscape problem, as we have formulated it so far, is a bare counting problem: one wants to determine the number of distinct configurations $s_{\text{st}}$ that satisfy $\nabla \mathcal{F}(s_{\text{st}}) = 0$. For $s \in \mathcal{C}_N = \mathbb{R}^N$ this amount to finding all solutions of the simultaneous conditions

$$f_i(s) = \frac{\partial}{\partial s_i} \mathcal{F}(s) = 0 \quad i = 1, \dots, N. \qquad (6.23)$$

For general manifolds $\mathcal{C}_N$, the functions $f_i(s)$ may take a more general form: on the sphere $\mathcal{C}_N = \mathcal{S}_N$, the gradient $\nabla \mathcal{F}(s)$ lies in the tangent plane to the sphere at $s$, and thus the $f_i(s)$ are the projections of $\partial_{s_i} \mathcal{F}(s)$ on the tangent plane (equivalently, the spherical constraint can be encoded in $f_i$ with a Lagrange multiplier). The Kac formula [64] for the number $\mathcal{N}_N$ of isolated solutions of (6.23) reads:

$$\mathcal{N}_N = \int_{\mathcal{C}_N} d\mathbf{s} \, \delta(f_1) \cdots \delta(f_N) \left| \det \left( \frac{\partial f_i}{\partial s_j} \right) \right|, \qquad (6.24)$$

where $\delta(f)$ stands for the Dirac delta-function, and appropriate smoothness of the functions $f_i(s)$ is assumed. One can recognize in the determinant in (6.24) a Jacobian of a change of variables in the delta-function: for $f_i(s) = \partial_{s_i} \mathcal{F}(s)$, the matrix is nothing but the Hessian matrix, whose eigenvalues give the curvature of the landscape in the vicinity of the stationary point, and thus control its linear stability. For $f_i$ containing random Gaussian terms, Rice [65, 66] (see also [67]) derived an expression for the average of $\mathcal{N}_N$, which we write here as:

$$\mathbb{E}[\mathcal{N}_N] = \int_{\mathcal{C}_N} d\mathbf{s} \, P_{\mathbf{f}(\mathbf{s})}(\mathbf{0}) \mathbb{E}\left[\left|\det\left(\frac{\partial f_i}{\partial s_j}\right)\right| \mid f_i(\mathbf{s}) = 0 \ \forall i\right], \qquad (6.25)$$

where $P_{\mathbf{f}(\mathbf{s})}(\mathbf{0})$ is the joint density of the variables $f_i(s)$ evaluated at 0, and $\mathbb{E}[\cdot|\cdot]$ a conditional expectation value. For $f_i(s) = \partial_{s_i} \mathcal{F}(s)$, the mean number of stationary points can thus simply be written as $\mathbb{E}[\mathcal{N}_N] = \int_{\mathcal{C}_N} d\mathbf{s} \, \rho_{\text{st}}(\mathbf{s})$, with $\rho_{\text{st}}(\mathbf{s})$ being the
corresponding mean density

$$ \rho_{\text{st}}(\mathbf{s}) = \mathbb{E} \left[ \left| \det \left( \partial_{s_i s_j} \mathcal{F}(\mathbf{s}) \right) \right| \prod_{i=1}^{N} \delta(\partial_{s_i} \mathcal{F}(\mathbf{s})) \right]. \quad (6.26) $$

The equation (6.25) is the simplest instance of a *Kac-Rice (KR) formula*. A nice review of the history of this formula can be found in [68] and in references therein; its applications to the high-d setting are discussed [69], see also [15, 70] for a mathematical exposition. The formula can be generalized to count stationary points satisfying specific constrained (belonging to a certain level set or having a certain index) by inserting in the expectation value (6.26) characteristic functions enforcing these conditions.

It is clear from (6.13) that to get the *annealed complexity* of the landscape or of any of its level sets, it suffices to get the large-N asymptotics of (6.24). The *quenched complexity* (6.14), however, requires to go beyond the computation of the first moment. To this aim, one has to introduce Kac-Rice formulas for the higher moments of $\mathcal{N}_N$ [71] or *replicated Kac-Rice* in the language of [72], which in the notation used above can be written as:

$$ \mathbb{E}[\mathcal{N}_N^n] = \int_{C_N} \prod_{a=1}^n d\mathbf{s}^a P_{f}(\mathbf{s}^a)(0) \mathbb{E}\left[\left|\prod_{a=1}^n \det\left(\frac{\partial f_i(\mathbf{s}^a)}{\partial s_j^a}\right)\right| \mid f_i(\mathbf{s}^a) = 0 \ \forall i, a\right], \quad (6.27) $$

where now the conditioning to $f_i(\mathbf{s}^a) = 0$ has to be imposed for all $a = 1, \dots, n$, and $P_{f}(\mathbf{s}^a)(0)$ is the joint probability density associated to this event. To determine the asymptotics of (6.25) or (6.27) one has to deal with the conditional expectation of the determinant. In much of the spin-glass literature this term is dealt with approximately; it is only in more recent years that it was realized that random matrix theory allows to treat it exactly, and thus to get a more thorough comprehension of the stability properties of the stationary points one is counting.

**Dealing with determinants, approximately: integral representations.** The absolute value in (6.25) is not an innocent detail: as it follows from Morse theory [15], omitting it would give a topological invariant (the Euler characteristics of $C_N$) rather than $\mathcal{N}_N$. Despite being well aware of this fact [73], early works in the physical literature succeeded nevertheless to count minima of energy or TAP free-energy landscapes by largely omitting the modulus, and by introducing integral representations of the determinant (with its own sign) in terms of commuting [35] or anti-commuting (Grassmann) variables [74, 75]. This was justified by the expectation that in the large-N limit the bottom of the landscape is dominated by minima all having strictly positive determinant; thus, when restricting the counting problem to level sets corresponding to low values of the (free)-energy, the absolute value can be safely dropped. This is in fact the case for models such as the spherical pure p-spin model: below a certain threshold value of the energy the number of stable local minima is exponentially (in N) larger than the number of saddles, and the calculation of the total complexity (with no constraint on the index) of stationary points of low energy performed in this way [17] reproduces correctly the complexity of local minima [76]. However, the situation becomes complicated when the dominant stationary points are only marginally stable (i.e. such that
some eigenvalue of the Hessian vanishes and the states are very fragile under external perturbations), as it is the case for the models (6.2) defined on discrete spaces $s_i = \pm 1$, including the Sherrington-Kirkpatrick model [77-79]. The calculation of the TAP complexity in these cases proved to be particularly challenging and sometimes not easily interpretable: an illustration of this is the extensive debate one finds in the physical literature [35, 80-86] on the connection between marginality and the breaking of the BRST supersymmetry [87] — a symmetry between the commuting and the anticommuting variables introduced with the integral representation of the determinant.

**Dealing with determinants, exactly: random matrix theory.** The paper [42] was seemingly the first one in which it was observed that the application of the Kac-Rice formula to a certain class of random Gaussian landscapes (in that work, model (6.5)) can be conveniently cast into a Random Matrix Theory (RMT) problem. The random matrices in question are obviously the Hessian matrices: the approach of [42] consists in determining their distribution directly, instead of giving integral representations of their determinant. The condition (6.1) is then crucial, as it translates into properties of invariance of the corresponding random matrix ensembles, allowing one to perform explicit calculations. Indeed, one easily sees that (6.1) implies for a centered Gaussian field $V(s)$:

$$ 
\begin{aligned}
\mathbb{E}[\partial_{s_l s_i}^2 V(s) \partial_{s'_j s'_k}^2 V(s')] &= N \partial_{s_l s_i}^2 \partial_{s'_j s'_k}^2 F_2 \left( \frac{\|s-s'\|^2}{2N} \right) \\
&= \frac{1}{N} F_2'' \left( \frac{\|s-s'\|^2}{2N} \right) [\delta_{ki}\delta_{jl} + \delta_{ji}\delta_{kl} + \delta_{jk}\delta_{li}] \\
&\quad + \frac{1}{N^2} F_2''' \left( \frac{\|s-s'\|^2}{2N} \right) g_1(s,s') + \frac{1}{N^3} F_2'''' \left( \frac{\|s-s'\|^2}{2N} \right) g_2(s,s')
\end{aligned}
\quad (6.28) $$

where

$$ g_1(s, s') = \delta_{kl}(s_j - s'_j)(s_i - s'_i) + \delta_{jl}(s_k - s'_k)(s_i - s'_i) + \delta_{il}(s_j - s'_j)(s_k - s'_k) \\ + \delta_{kj}(s_l - s'_l)(s_i - s'_i) \quad (6.29) $$

$$ g_2(s, s') = (s_k - s'_k)(s_i - s'_i)(s_j - s'_j)(s_l - s'_l). $$

Therefore, the correlations between entries of the Hessian matrix, obtained setting $s = s'$, simply read:

$$ \mathbb{E} \left[ \partial_{s_l s_i}^2 V(s) \partial_{s'_j s'_k}^2 V(s) \right] = \frac{1}{N} F_2''(0) [\delta_{ki}\delta_{jl} + \delta_{ji}\delta_{kl} + \delta_{jk}\delta_{li}], \quad (6.30) $$

implying GOE-like statistics. Similarly, one can deduce that the matrix elements of the Hessian and the first derivatives $\partial_{s_i} V(s)$ at the same point in $C_N$ are independent, thus the conditioning in the expectation value (6.25) is immaterial, rendering the calculation of the first moment particularly simple. This mapping to RMT has been extended and developed in [88-91], culminating in the works [14, 92] that considerably advanced RMT-based techniques for counting stationary points with a fixed index and conditioned on the value of the landscape (for (6.2) on the sphere $S_N$).

Computing the higher moments (6.27) is more challenging, as correlations enter into play. Indeed, one has to determine the joint expectation of the product of determinants
at different configurations, which as it follows from (6.28) are correlated to each oth-
ers; moreover, the Hessian at one configuration **s** is correlated to the derivatives of the
random field at another configuration **s'**, and therefore the conditioning in (6.27) is no
longer immaterial. However, such calculation can be carried out explicitly for isotropic
Gaussian fields [72, 93]; crucial ingredients are the facts that (i) the conditioning mod-
ifies the invariant Hessian statistics (6.30) by means of *finite-rank perturbations*, the
effect of which on the Hessian spectrum can be studied explicitly with RMT techniques,
(ii) the correlations between the Hessian matrices at different configurations is negligible
when computing the joint expectation value of the determinants to leading order in *N*
(see also [94] for a rigorous proof of this statement for *n* = 2).

The RMT approach is thus fully controllable and it allows one to determine an-
nealed landscape’s complexities within a mathematically rigorous framework. It can be
extended to quenched complexities embedding the replica trick in the formalism. In
particular, the RMT setup is essential to characterize the Hessian statistics well beyond
the leading order contribution of the determinant, allowing one to study the emergence
of *isolated eigenvalues* in the spectrum and thus to control in detail the stability of the
stationary points one is counting; this is fundamental to address the questions related
to the landscape geometry and the distribution of barriers, as we hint at in the following
Section.

6.4. Recent results on random landscapes: a short summary

In this Section we give an overview of recent results, organizing the presentation around
the questions introduced in Sec. 6.2 and focusing mostly on recent developments involv-
ing KR formulas and RMT.

**On the complexity of level sets.** The spherical pure *p*-spin model (6.2) with $\alpha_{p'} = \delta_{p,p'}$, $p \ge 3$ and $C_N = S_N$ has quickly become one of the most paradigmatic models in the theoretical literature on glasses [95], as well as the favourite playground to test and compare the approaches recalled in Sec. 6.3, which all give consistent results for this model. The expression for the annealed complexity $\Sigma(f)$ of the stationary points at intensive energy $f$ has been proven rigorously in [14, 92] using KR. It was known from replica calculations [17] that in the pure model quenched and annealed complexities coincide, a statement made rigorous in [94] by means of the second moment method. The works [14, 92] go beyond the calculation of the total complexity by determining the explicit expression of the annealed complexity $\Sigma(f, k)$ of stationary points at fixed index $k$. The calculation is elegantly framed in the RMT setting: the complexity of saddles of index $k$ is obtained from the large deviation function of the smallest eigenvalues of a GOE matrix [96]. The resulting picture is that of an energy landscape that is hierarchically organized at it bottom (below the *threshold energy* mentioned in Sec. 6.3.3): at fixed $f$ the complexity curves are ordered in the index $k$, the highest one being that of minima ($k=0$), followed by that of saddles of index 1, 2, 3, .... The curves never cross, meaning that at low energy densities saddles are exponentially numerous, but exponentially less
numerous than minima (the suppression in their number is precisely due to the large
deviation cost mentioned above). The equivalence between quenched and annealed
complexity extends to the fixed index case [97]. It should be mentioned that for this
model, the curves $\Sigma(f, k)$ were already obtained in [76] within the approximations of
Sec. 6.3.3, which are effective here in part thanks to the lacking of crossings between the
$\Sigma(f, k)$ for different $k$ (and thus different Hessians sign). Another fortunate property of
the pure model is the fact that no *temperature chaos* is present, implying that results
on the complexity of the energy landscape can be extended *adiabatically* to the free-
energy landscape at low temperature. This is due to the fact that the energy function
is homogeneous (a polynomial of fixed degree), and temperature enters as a global
rescaling factor [98]. A lot of progress on this topic has been made recently on the
mathematical side [99–103].

These simple features of the pure model are quite fragile: already adding a deter-
ministic field breaks the identity between quenched and annealed complexity [104], and
the two remain in general different for mixed models. In the latter case the landscape
structure becomes more intricate (see [105, 106] for the calculation of the annealed
complexity of a *p* + *k* model with the techniques of Sec. 6.3.3). The interest in these
models has reignited recently [107] partly due to the discovery of new forms of ergod-
icity breaking in the Langevin dynamics associated to these systems [58], and partly
due to their emergence in problems of inference [108], or non-linear optics [109]. Recent
mathematical work on the free-energy of mixed models [110] also elucidate the mech-
anisms behind the Parisi replica symmetry breaking, complementing approaches based
on a direct analysis of Parisi measures [111]. Intriguing features of the complexity of
stationary points of the pure model defined on complex configuration spaces $C_N \subseteq \mathbb{C}^N$
are discussed in [112, 113].

**On topology trivialization transitions.** Topology trivialization transitions have been identified and discussed in the recent works [42, 69, 89, 90, 114]: they occur when a control parameter, like the signal-to-noise ratio *r* in (6.4) or the curvature *μ* in (6.5), exceed a critical value set by the variance of the local Hessian, $F_2''(0)$ in (6.30). Such critical values are exactly those where zero-temperature replica symmetry breaking mechanisms cease to be operative [89], reflecting the change in the nature of the landscape from supporting exponentially many stationary points to only a few. The transition is further accompanied by the change in the Hessian spectrum at the global minimum of the associated landscape [115]. In the trivial phase $\Sigma_{\text{que}} = \Sigma_{\text{ann}} = 0$, and in some cases more precise asymptotics of $N_N$ can be determined [116, 117]. A particularly interesting example is provided by the model (6.6) in presence of a force term, where the topology trivialization transition corresponds to the so called *depinning* dynamical transition [118–120]. Annealed complexities in that case have been rigorously and elegantly computed in [25] using advanced RMT insights into the properties of expectations of random determinants obtained by the same authors in an accompanying paper [121]. For further discussion of topology trivialization (or lack of it) for different types of random landscapes see [8, 122].
**On the correlations between stationary points.** The models (6.4) are prototypical examples of landscapes where deterministic, convex contributions compete with fluctuating terms, giving rise to a variety of transitions in both the topology and geometry of the landscape. The geometry of a variation of the $p+k$ model obtained choosing $V(s)$ to be a pure $p$-spin model and $f(x) = x^k/k$ in has been studied in [93] (see [123] for the study of the equilibrium properties) by means of the computation of a quenched constrained complexity $\Sigma(f, k; q)$. This study has unveiled the occurrence of three different transitions as a function of $r$: (i) a transition related to recovery, occurring when the deepest minimum of the landscape becomes correlated with $s_*$ and thus informative on its position in $C_N$, implying that partial information on $s_*$ could be recovered by an effective landscape minimization; (ii) a geometrical transition occurring when the landscape in the vicinity of the global minimum ceases to be rugged, implying that optimization algorithms initialized with a bias towards the direction of $s_*$ are able to converge to the global minimum; (iii) a topology trivialization transition occurring when the whole landscape becomes convex, even far away from $s_*$. In this model, the trivialization of the landscape is induced by the low rank perturbations to the Hessian matrices at the stationary points, which generate an isolated eigenvalue that becomes negative, signalling that the local minima develop an instability (a direction of negative curvature) towards $s_*$. In a certain regime of parameters, the landscape is dominated by marginally stable states with a single zero mode of the Hessian, similarly to what happens in other models [38, 105]: in [93] the functional dependence of this special eigenvalue on $f, r$ and on the overlap $q$ is obtained within a quenched formalism combined with RMT. Based on this analysis, a classification of tilted random landscapes emerges: for $k=1$ ($p$-spin in a field) the landscape is very sensitive to the presence of $s_*$ and the exponential majority of local minima are informative—the quenched complexity of this model was computed in [104]. For $k=2$, a topology trivialization transition occurs for values of $r \sim O(N^0)$ thanks to the weak instability given by the isolated eigenvalue, while for $k=3$ the transition occurs for $r \sim O(N^\alpha)$ with $\alpha > 0$. This classification emerges with a broader generality when studying the performance of optimization dynamics [39]. We stress that the case $k \ge 3$ is particularly interesting in the inference setting, as it includes the so called spiked-tensor problem (when $p=k$) that has attracted a lot of attention in recent literature as a prototypical problem with a statistical-to-algorithmic gap [20, 124–127].

**On the distribution of barriers.** The recent advances on the KR formalism allow one to revisit the question on the distribution of barriers [128, 129], identified here with the difference between the values of $F(s)$ at one minimum and that at a nearby, connected saddle. To extract this information, it is necessary to study the landscape locally, i.e. to determine the distribution of stationarity points in the vicinity of any arbitrary local minimum chosen as a reference point, and to study their stability. In high-$N$, this is in essence a large deviation calculation. Early attempts to address this problem for the spherical, pure $p$-spin model have been made in [130] by means of a variation of the Franz-Parisie potential, and in [131] computing a constrained complexity in the annealed framework. The quenched calculation has been done in [72], and the analysis
of the Hessian statistics has revealed a crossing between a population of dominating rank-1 saddles sufficiently close to the reference minimum, and a population of minima dominating far-away in configuration space. It is also shown that the local maximum of the Franz-Parisi potential, which gives an upper bound to the dynamical barrier [98], actually likely corresponds to a local minimum in the landscape rather than a saddle. The complexity of low-rank connected saddles in the region dominated by minima has been determined in [132] via a mapping to a large deviation problem for the smallest eigenvalue and eigenvector of a random matrix perturbed by additive and multiplicative finite-rank perturbations, generalizing the RMT results in [133]. Combining this refined knowledge on the landscape structure with the direct study of activated dynamics for the pure *p*-spin model is an ongoing research direction [134–137]. Recent progress has been made also in the study of free-energy barriers of the Sherrington-Kirkpatrick model with Ising variables [138], that (at variance with the *p*-spin case) are not scaling linearly in *N*. This has been made possible by a careful study of the scaling of the coefficients of the expansion of the TAP free energy around a minimum.

**6.5. Open challenges: landscape paradigm, and beyond**

High-*d* landscapes have been first introduced long ago as a visual aid for the dynamical evolution of complicated systems. It is within the field of glasses that this turned into an established theory, with the development of tools to capture and characterize statistically the complexity arising in the high-*d* limit. Recently these tools are gaining an ever-increasing importance, boosted by the impressive growth of available data on complex systems that are inherently high-*d* (organisms, neuronal systems, ecosystems, deep networks) and whose evolution can be interpreted as a landscape optimization. We can thus talk about a *high-* *d* *landscape paradigm*; nonetheless, substantial work still has to be done to advance the landscape program.

First, even though one might expect that isotropic Gaussian fields are justified by some sort of central limit theory arising in high-*d*, it would be desirable to export the tools described in this Chapters to settings in which standard spin-glass functions are not the suitable landscape models. Problems of supervised learning are a playground for this, and indeed in recent years there has been a continuously evolving effort to study different models mimicking loss landscapes emerging in realistic machine learning applications [139, 140]. Similarly, problems of reconstruction of encrypted signals give rise to optimization landscapes that are non-Gaussian, given for example by a sum of squared Gaussian terms [141] (see also [142] for applications of similar models to confluent tissues): understanding the landscape structure in this case is certainly an open direction worth to explore. Within the Kac-Rice formalism, giving up Gaussianity implies that the conditioning of the statistics of the Hessian becomes a more challenging problem: an example of this is discussed in [143], where landscapes of the form (6.8), (6.9) are considered and the complexity problem (within the annealed approximation) is rephrased in terms of a variational problem for the spectral density of the Hessian. The recognition of random-matrix theory as a crucial ingredient for complexity calculations is promising in this respect, as one might hope to derive general (universal) results
without needing to restrict form the start to standard invariant ensembles [144, 145].

Methodologically, the replicated Kac–Rice formalism for the quenched complexity has been used so far only within simple replica symmetry-breaking schemes, and it would be interesting to use it e.g. in contexts in which full replica-symmetry-breaking occurs [146]. In particular, studying the finite-rank perturbations to the Hessian statistics induced by this structure is an interesting problem, which might clarify some of the questions related to the stability of minima in models where marginality is relevant. From the mathematical perspective, it remains a huge open problem how to derive quenched complexities without needing to invoke the replica trick.

It is also worth mentioning that Kac–Rice methods can be used to address proper-
ties of complex high-dimensional systems beyond the landscape paradigm, by counting
statistics of equilibrium points of *non-gradient* autonomous dynamics. The motivations
for such studies come from applications in fields ranging from neural networks to ecol-
ogy, economics, and nonlinear wave scattering. Progress in this direction has been made
in [147–153] for the calculation on the annealed complexity of the equilibria of the asso-
ciated systems of equations, and in [154] for the quenched complexity. Understanding
how the properties of a conservative system change under non-gradient perturbations is
certainly a question with a huge theoretical interest and a broad range of applications.

Finally, we find it appropriate to conclude this Chapter by recalling that getting
a refined information on the landscape topology and geometry can hopefully shade
light and guide us into the comprehension of the dynamical evolution of the complex
systems associated to it: establishing quantitatively this connection between landscape
and dynamics is the underlying goal of the landscape program, and thus the most
relevant perspective.

Acknowledgments

The authors thank their collaborators, in particular Gérard Ben Arous for the many enlightening discussions on the topic of this chapter. VR acknowledges funding by the “Investissements d’Avenir” LabEx PALM (ANR-10-LABX-0039-PALM). YVF acknowledges the support by the EPSRC Grant EP/V002473/1 “Random Hessians and Jacobians: theory and applications”.
References

[1] G. Parisi, Nobel prize lecture, *NobelPrize.org. Nobel Prize Outreach AB 2022*. Wed. 13 Jul 2022. (2022).

[2] S. Wright. The roles of mutation, inbreeding, crossbreeding and selection in evolution, p 356–366. In *Proceedings of the Sixth Annual Congress of Genetics*, vol. 1, (1932).

[3] P. G. Wolynes, Landscapes, funnels, glasses, and folding: From metaphor to software, *Proc. Am. Philos. Soc.* **145**(4), 555–563, (2001).

[4] R. H. Austin. Free energies, landscapes, and fitness in evolution dynamics. In *Quantitative Biology: From Molecular to Cellular Systems*, pp. 1–21. CRC Press, (2012).

[5] P. Krugman, Complex landscapes in economic geography, *Am. Econ. Rev.* **84**(2), 412–416, (1994).

[6] A. Montanari, F. Ricci-Tersenghi, and G. Semerjian, Clusters of solutions and replica symmetry breaking in random *k*-satisfiability, *J. Stat. Mech.: Theory Exp.* **2008**(04), P04004, (2008).

[7] M. Mezard and A. Montanari, *Information, physics, and computation*. (Oxford University Press, 2009).

[8] Y. V. Fyodorov and R. Tublin, Optimization landscape in the simplest constrained random least-square problem, *J. Phys. A.* **55**(24), 244008, (2022).

[9] A. Choromanska, M. Henaff, M. Mathieu, G. B. Arous, and Y. LeCun. The loss surfaces of multilayer networks. In *Artificial intelligence and statistics*, pp. 192–204. PMLR, (2015).

[10] M. Goldstein, Viscous liquids and the glass transition: a potential energy barrier picture, *J. Chem. Phys.* **51**(9), 3728–3739, (1969).

[11] A. Masoumi, A. Vilenkin, and M. Yamada, Inflation in multi-field random gaussian landscapes, *J. Cosmol. Astropart. Phys.* **2017**(12), 035, (2017).

[12] L. L. Feng, S. Hotchkiss, and R. Easther, The distribution of vacua in random landscape potentials, *J. Cosmol. Astropart. Phys.* **2021**(01), 029, (2021).

[13] M. Mézard, G. Parisi, and M. A. Virasoro, *Spin glass theory and beyond: An Introduction to the Replica Method and Its Applications*. vol. 9, (World Scientific Publishing Company, 1987).

[14] A. Auffinger, G. B. Arous, and J. Černý, Random matrices and complexity of spin glasses, *Commun. Pure Appl. Math.* **66**(2), 165–201, (2013).

[15] R. J. Adler and J. E. Taylor, *Random fields and geometry*. vol. 80, (Springer, 2007).

[16] D. J. Gross and M. Mézard, The simplest spin glass, *Nucl. Phys. B.* **240**(4), 431–452, (1984).

[17] A. Crisanti and H.-J. Sommers, Thouless-Anderson-Palmer approach to the spherical p-spin spin glass model, *J. Physique I. 5*(7), 805–813, (1995).

[18] D. Sherrington and S. Kirkpatrick, Solvable model of a spin-glass, *Phys. Rev. Lett.* **35** (26), 1792, (1975).

[19] B. Derrida, Random-energy model: Limit of a family of disordered models, *Phys. Rev. Lett.* **45**(2), 79, (1980).

[20] E. Richard and A. Montanari, A statistical model for tensor pca, *NeurIPS.* **27**, (2014).

[21] J. Neidhart, I. G. Szendro, and J. Krug, Adaptation in tunably rugged fitness landscapes: the rough mount fuji model, *Genetics.* **198**(2), 699–721, (2014).

[22] J. N. Onuchic, Z. Luthey-Schulten, and P. G. Wolynes, Theory of protein folding: the energy landscape perspective, *Annu. Rev. Phys. Chem.* **48**(1), 545–600, (1997).

[23] Y. V. Fyodorov and H.-J. Sommers, Classical particle in a box with random potential: exploiting rotational symmetry of replicated hamiltonian, *Nucl. Phys. B.* **764**(3), 128–167, (2007).

[24] E. Bouchbinder, E. Lerner, C. Rainone, P. Urbani, and F. Zamponi, Low-frequency vibrational spectrum of mean-field disordered systems, *Phys. Rev. B.* **103**(17), 174202,
(2021).

[25] G. B. Arous, P. Bourgade, and B. McKenna, Landscape complexity beyond invariance and the elastic manifold, arXiv:2105.05051, (2021).

[26] P. Mergny and S. N. Majumdar, Stability of large complex systems with heterogeneous relaxation dynamics, *J. Stat. Mech.: Theory Exp.* **2021**(12), 123301, (2021).

[27] I. Gershenzon, O. Raz, E. Subag, O. Zeitouni, et al., On-site potential creates complexity in systems with disordered coupling, arXiv preprint arXiv:2206.10554, (2022).

[28] P. Urbani, Disordered high-dimensional optimal control, *J. Phys. A.* **54**(32), 324001, (2021).

[29] R. Dietrich, M. Opper, and H. Sompolinsky, Statistical mechanics of support vector networks, *Phys. Rev. Lett.* **82**(14), 2975, (1999).

[30] A. Engel and C. Van den Broeck, *Statistical mechanics of learning*. (Cambridge University Press, 2001).

[31] E. Gardner, The space of interactions in neural network models, *J. Phys. A.* **21**(1), 257, (1988).

[32] S. Franz and G. Parisi, The simplest model of jamming, *J. Phys. A.* **49**(14), 145001, (2016).

[33] L. Zdeborová and F. Krzakala, Statistical physics of inference: Thresholds and algorithms, *Adv. Phys.* **65**(5), 453–552, (2016).

[34] D. J. Thouless, P. W. Anderson, and R. G. Palmer, Solution of ‘solvable model of a spin glass’, *Philos. Mag.* **35**(3), 593–601, (1977).

[35] A. J. Bray and M. A. Moore, Metastable states in spin glasses, *J. Phys. C.* **13**(19), L469, (1980).

[36] A. Bray and M. Moore, Metastable states in the solvable spin glass model, *J. Phys. A.* **14**(9), L377, (1981).

[37] A. Cavagna, I. Giardina, and G. Parisi, Numerical study of metastable states in ising spin glasses, *Phys. Rev. Lett.* **92**(12), 120603, (2004).

[38] T. Aspelmeier, A. Bray, and M. Moore, Complexity of ising spin glasses, *Phys. Rev. Lett.* **92**(8), 087203, (2004).

[39] G. B. Arous, R. Gheissari, and A. Jagannath, Online stochastic gradient descent on non-convex losses from high-dimensional inference., *J. Mach. Learn. Res.* **22**, 106–1, (2021).

[40] G. B. Arous, R. Gheissari, and A. Jagannath, High-dimensional limit theorems for sgd: Effective dynamics and critical scaling, arXiv preprint arXiv:2206.04030, (2022).

[41] F. Mignacco and P. Urbani, The effective noise of stochastic gradient descent, arXiv preprint arXiv:2112.10852, (2021).

[42] Y. V. Fyodorov, Complexity of random energy landscapes, glass transition, and absolute value of the spectral determinant of random matrices, *Phys. Rev. Lett.* **92**(24), 240601, (2004).

[43] B. Derrida, From random walks to spin glasses, *Physica D.* **107**(2-4), 186–198, (1997).

[44] M. Kac. On certain toeplitz-like matrices and their relation to the problem of lattice vibrations. In *Arkiv for Der Fysiske Seminar i Trondheim*, vol. 11, (1968).

[45] S. F. Edwards and P. W. Anderson, Theory of spin glasses, *J. Phys. F.* **5**(5), 965, (1975).

[46] J. Kurchan, Six out of equilibrium lectures, arXiv preprint arXiv:0901.1271, (2009).

[47] S. Franz, G. Parisi, and M. A. Virasoro, The replica method on and off equilibrium, *J. Physique I.* **2**(10), 1869–1880, (1992).

[48] R. Monasson, Structural glass transition and the entropy of the metastable states, *Phys. Rev. Lett.* **75**(15), 2847, (1995).

[49] M. Mézard, How to compute the thermodynamics of a glass using a cloned liquid, *Physica A.* **265**(3-4), 352–369, (1999).

[50] P. Charbonneau, J. Kurchan, G. Parisi, P. Urbani, and F. Zamponi, Glass and jamming transitions: From exact results to finite-dimensional descriptions, *Annu. Rev. Condens.*
*Matter Phys.* **8**, 1–26, (2017).

[51] G. Parisi, P. Urbani, and F. Zamponi, *Theory of simple glasses: exact solutions in infinite dimensions*. (Cambridge University Press, 2020).

[52] E. De Giuli and A. Zee, Glassy gravity, *EPL* **133**(2), 20008, (2021).

[53] S. Franz and G. Parisi, Recipes for metastable states in spin glasses, *J. Physique I.* **5** (11), 1401–1415, (1995).

[54] S. Franz and G. Parisi, Effective potential in glassy systems: theory and simulations, *Physica A.* **261**(3-4), 317–339, (1998).

[55] A. Barrat, S. Franz, and G. Parisi, Temperature evolution and bifurcations of metastable states in mean-field spin glasses, with connections with structural glasses, *J. Phys. A.* **30**(16), 5593, (1997).

[56] F. Krzakala and L. Zdeborová, Following gibbs states adiabatically—the energy landscape of mean-field glassy systems, *EPL* **90**(6), 66002, (2010).

[57] C. Rainone, P. Urbani, H. Yoshino, and F. Zamponi, Following the evolution of hard sphere glasses in infinite dimensions under external perturbations: Compression and shear strain, *Phys. Rev. Lett.* **114**(1), 015701, (2015).

[58] G. Folena, S. Franz, and F. Ricci-Tersenghi, Rethinking mean-field glassy dynamics and its relation with the energy landscape: The surprising case of the spherical mixed p-spin model, *Phys. Rev. X.* **10**(3), 031045, (2020).

[59] F. Ricci-Tersenghi and G. Semerjian, On the cavity method for decimated random constraint satisfaction problems and the analysis of belief propagation guided decimation algorithms, *J. Stat. Mech.: Theory Exp.* **2009**(09), P09001, (2009).

[60] D. Panchenko, Free energy in the mixed p-spin models with vector spins, *Ann. Probab.* **46**(2), 865–896, (2018).

[61] G. Ben Arous and A. Jagannath, Spectral gap estimates in mean field spin glasses, *Commun. Math. Phys.* **361**(1), 1–52, (2018).

[62] A. S. Bandeira, A. E. Alaoui, S. B. Hopkins, T. Schramm, A. S. Wein, and I. Zadik, The franz-parisi criterion and computational trade-offs in high dimensional statistics, *arXiv preprint arXiv:2205.09727*. (2022).

[63] S. Franz and J. Rocchi, Large deviations of glassy effective potentials, *J. Phys. A.* **53** (48), 485002, (2020).

[64] M. Kac, On the average number of real roots of a random algebraic equation, *Bull. Am. Math. Soc.* **49**(4), 314–320, (1943).

[65] S. Rice, Mathematical analysis of random noise, *Bell Syst. tech. j.* **23**, 1–114, (1943).

[66] S. O. Rice, Mathematical analysis of random noise, *Bell Syst. tech. j.* **24**(1), 46–156, (1945).

[67] K. Ito, The expected number of zeros of continuous stationary gaussian processes, *J. Math. Kyoto Univ.* **3**(2), 207–216, (1963).

[68] C. Berzin, A. Latour, and J. León, Kac-rice formula: A contemporary overview of the main results and applications, *arXiv:2205.08742*. (2022).

[69] Y. V. Fyodorov, High-dimensional random fields and random matrix theory, *Markov Proc. Rel. Fields.* **21**(3), 483–518, (2015).

[70] J.-M. Azaïs and M. Wschebor, *Level sets and extrema of random processes and fields*. (John Wiley & Sons, 2009).

[71] H. Cramér and M. R. Leadbetter, *Stationary and related stochastic processes: Sample function properties and their applications*. (Courier Corporation, 2013).

[72] V. Ros, G. Biroli, and C. Cammarota, Complexity of energy barriers in mean-field glassy systems, *EPL* **126**(2), 20003, (2019).

[73] J. Kurchan, Replica trick to calculate means of absolute values: applications to stochastic equations, *J. Phys. A.* **24**(21), 4969, (1991).

[74] K. Efetov, *Supersymmetry in disorder and chaos*. (Cambridge university press, 1999).

[75] G. Parisi and N. Sourlas, Supersymmetric field theories and stochastic differential equations, *J. Phys. A.* **24**(1), 1–20, (1981).
tions, Nucl. Phys. B. 206(2), 321-332, (1982).

[76] A. Cavagna, I. Giardina, and G. Parisi, Stationary points of the thouless-anderson-palmer free energy, Phys. Rev. B. 57(18), 11251, (1998).

[77] A. J. Bray and M. A. Moore, Evidence for massless modes in the 'solvable model' of a spin glass, Journal of Physics C: Solid State Physics. 12(11), L441, (1979).

[78] G. Parisi and M. Potters, On the number of metastable states in spin glasses, EPL. 32 (1), 13, (1995).

[79] M. Müller, L. Leuzzi, and A. Crisanti, Marginal states in mean-field glasses, Phys. Rev. B. 74(13), 134431, (2006).

[80] A. Cavagna, I. Giardina, G. Parisi, and M. Mézard, On the formal equivalence of the tap and thermodynamic methods in the sk model, J. Phys. A. 36(5), 1175, (2003).

[81] A. Annibale, A. Cavagna, I. Giardina, and G. Parisi, Supersymmetric complexity in the sherrington-kirkpatrick model, Phys. Rev. E. 68(6), 061103, (2003).

[82] A. Crisanti, L. Leuzzi, G. Parisi, and T. Rizzo, Complexity in the Sherrington-Kirkpatrick model in the annealed approximation, Phys. Rev. B. 68(17), 174401, (2003).

[83] A. Crisanti, L. Leuzzi, G. Parisi, and T. Rizzo, Quenched computation of the dependence of complexity on the free energy in the sherrington-kirkpatrick model, Phys. Rev. B. 70 (6), 064423, (2004).

[84] T. Rizzo, Tap complexity, the cavity method and supersymmetry, J. Phys. A. 38(15), 3287, (2005).

[85] A. Crisanti, L. Leuzzi, and T. Rizzo, Complexity in mean-field spin-glass models: Ising p-spin, Phys. Rev. B. 71(9), 094202, (2005).

[86] G. Parisi, Computing the number of metastable states in infinite-range models, arXiv:cond-mat/0602349. (2006).

[87] C. Becchi, A. Rouet, and R. Stora, Renormalization of the abelian higgs-kibble model, Commun. Math. Phys. 42(2), 127-162, (1975).

[88] A. J. Bray and D. S. Dean, Statistics of critical points of gaussian fields on large-dimensional spaces, Phys. Rev. Lett. 98(15), 150201, (2007).

[89] Y. V. Fyodorov and I. Williams, Replica symmetry breaking condition exposed by random matrix calculation of landscape complexity, J. Stat. Phys. 129(5), 1081-1116, (2007).

[90] Y. V. Fyodorov and C. Nadal, Critical behavior of the number of minima of a random landscape at the glass transition point and the tracy-widom distribution, Phys. Rev. Lett. 109(16), 167203, (2012).

[91] J. Grela and B. A. Khoruzhenko, Glass-like transition described by toppling of stability hierarchy, J. Phys. A. 55(15), 154001, (2022).

[92] A. Auffinger and G. B. Arous, Complexity of random smooth functions on the high-dimensional sphere, Ann. Probab. 41(6), 4214-4247, (2013).

[93] V. Ros, G. B. Arous, G. Biroli, and C. Cammarota, Complex energy landscapes in spiked-tensor and simple glassy models: Ruggedness, arrangements of local minima, and phase transitions, Phys. Rev. X. 9(1), 011003, (2019).

[94] E. Subag, The complexity of spherical p-spin models—a second moment approach, Ann. Probab. 45(5), 3385-3450, (2017).

[95] T. R. Kirkpatrick and D. Thirumalai, p-spin-interaction spin-glass models: Connections with the structural glass problem, Phys. Rev. B. 36(10), 5388, (1987).

[96] G. B. Arous and A. Guionnet, Large deviations for wigner's law and voiculescu's non-commutative entropy, Probab. Theory Relat. Fields. 108(4), 517-542, (1997).

[97] A. Auffinger and J. Gold, The number of saddles of the spherical p-spin model, arXiv:2007.09269. (2020).

[98] J. Kurchan, G. Parisi, and M. A. Virasoro, Barriers and metastable states as saddle points in the replica approach, J. Physique I. 3(8), 1819-1838, (1993).

[99] G. B. Arous, E. Subag, and O. Zeitouni, Geometry and temperature chaos in mixed
spherical spin glasses at low temperature: the perturbative regime, *Commun. Pure Appl. Math.* **73**(8), 1732–1828, (2020).

[100] E. Subag, The geometry of the Gibbs measure of pure spherical spin glasses, *Invent. Math.* **210**(1), 135–209, (2017).

[101] E. Subag, The free energy of spherical pure p-spin models–computation from the TAP approach, *arXiv:2101.04352*. (2021).

[102] E. Subag and O. Zeitouni, The extremal process of critical points of the pure p-spin spherical spin glass model, *Probab. Theory Relat. Fields.* **168**(3), 773–820, (2017).

[103] E. Subag and O. Zeitouni, Concentration of the complexity of spherical pure p-spin models at arbitrary energies, *J. Math. Phys.* **62**(12), 123301, (2021).

[104] A. Cavagna, J. P. Garrahan, and I. Giardina, Quenched complexity of the mean-field p-spin spherical model with external magnetic field, *J. Phys. A.* **32**(5), 711, (1999).

[105] A. Annibale, G. Gualdi, and A. Cavagna, Coexistence of supersymmetric and supersymmetry-breaking states in spherical spin-glasses, *J. Phys. A.* **37**(47), 11311, (2004).

[106] A. Crisanti and L. Leuzzi, Spherical 2+ p spin-glass model: An exactly solvable model for glass to spin-glass transition, *Phys. Rev. Lett.* **93**(21), 217203, (2004).

[107] D. Barbier and L. F. Cugliandolo, A constrained tap approach for disordered spin models: application to the mixed spherical case, *J. Stat. Mech.: Theory Exp.* **2020**(6), 063207, (2020).

[108] S. S. Mannelli, F. Krzakala, P. Urbani, and L. Zdeborova. Passed & spurious: Descent algorithms and local minima in spiked matrix-tensor models. In *ICML*, pp. 4333–4342. PMLR, (2019).

[109] F. Antenucci, A. Crisanti, and L. Leuzzi, Complex spherical 2+ 4 spin glass: A model for nonlinear optics in random media, *Phys. Rev. A.* **91**(5), 053816, (2015).

[110] E. Subag, Free energy landscapes in spherical spin glasses, *arXiv preprint arXiv:1804.10576*. (2018).

[111] A. Auffinger and W.-K. Chen, On the energy landscape of spherical spin glasses, *Adv. Math.* **330**, 553–588, (2018).

[112] J. Kent-Dobias and J. Kurchan, Complex complex landscapes, *Phys. Rev. Research.* **3** (2), 023064, (2021).

[113] J. Kent-Dobias and J. Kurchan, Analytic continuation over complex landscapes, *arXiv:2204.06072*. (2022).

[114] Y. V. Fyodorov and P. Le Doussal, Topology trivialization and large deviations for the minimum in the simplest random optimization, *J. Stat. Phys.* **154**(1), 466–490, (2014).

[115] Y. V. Fyodorov and P. Le Doussal, Hessian spectrum at the global minimum of high-dimensional random landscapes, *J. Phys. A.* **51**(47), 474002, (2018).

[116] D. Belius, J. Černý, S. Nakajima, and M. A. Schmidt, Triviality of the geometry of mixed p-spin spherical hamiltonians with external field, *J. Stat. Phys.* **186**(1), 1–34, (2022).

[117] A. Auffinger, G. Ben Arous, and Z. Li, Sharp complexity asymptotics and topological trivialization for the (p, k) spiked tensor model, *J. Math. Phys.* **63**(4), 043303, (2022).

[118] Y. V. Fyodorov, P. Le Doussal, A. Rosso, and C. Texier, Exponential number of equilibria and depinning threshold for a directed polymer in a random potential, *Ann. Phys. (N.Y.)*. **397**, 1–64, (2018).

[119] Y. V. Fyodorov and P. Le Doussal, Manifolds pinned by a high-dimensional random landscape: Hessian at the global energy minimum, *J. Stat. Phys.* **179**(1), 176–215, (2020).

[120] Y. V. Fyodorov and P. Le Doussal, Manifolds in a high-dimensional random landscape: Complexity of stationary points and depinning, *Phys. Rev. E.* **101**(2), 020101, (2020).

[121] G. B. Arous, P. Bourgade, and B. McKenna, Exponential growth of random determinants beyond invariance, *arXiv:2105.05000*. (2021).

[122] B. Lacroix-A-Chez-Toine, S. B. Fedeli, and Y. V. Fyodorov, Superposition of random plane waves in high spatial dimensions: Random matrix approach to landscape complex-
ity, arXiv:2202.03815. (2022).

[123] P. Gillin and D. Sherrington, $p > 2$ spin glasses with first-order ferromagnetic transitions, *J. Phys. A.* **33**(16), 3081, (2000).

[124] A. Perry, A. S. Wein, and A. S. Bandeira. Statistical limits of spiked tensor models. In *Ann. I. H. Poincaré Pr.*, vol. 56, pp. 230–264. Institut Henri Poincaré, (2020).

[125] G. B. Arous, S. Mei, A. Montanari, and M. Nica, The landscape of the spiked tensor model, *Commun. Pure Appl. Math.* **72**(11), 2282–2330, (2019).

[126] G. B. Arous, R. Gheissari, and A. Jagannath, Algorithmic thresholds for tensor pca, *Ann. Probab.* **48**(4), 2052–2087, (2020).

[127] J. H. de Morais Goulart, R. Couillet, and P. Comon, A random matrix perspective on random tensors, *stat. 1050*, 15, (2022).

[128] G. Rodgers and M. Moore, Distribution of barrier heights in infinite-range spin glass models, *J. Phys. A.* **22**(8), 1085, (1989).

[129] A. Lopatin and L. Ioffe, Barriers in the p-spin interacting spin-glass model: The dynamical approach, *Phys. Rev. Lett.* **84**(18), 4208, (2000).

[130] A. Cavagna, I. Giardina, and G. Parisi, Structure of metastable states in spin glasses by means of a three replica potential, *J. Phys. A.* **30**(13), 4449, (1997).

[131] A. Cavagna, I. Giardina, and G. Parisi, An investigation of the hidden structure of states in a mean-field spin-glass model, *J. Phys. A.* **30**(20), 7021, (1997).

[132] V. Ros, Distribution of rare saddles in the p-spin energy landscape, *J. Phys. A.* **53**(12), 125002, (2020).

[133] G. Biroli and A. Guionnet, Large deviations for the largest eigenvalues and eigenvectors of spiked gaussian random matrices, *Electron. Commun. Probab.* **25**, 1–13, (2020).

[134] V. Ros, G. Biroli, and C. Cammarota, Dynamical instantons and activated processes in mean-field glass models, *SciPost Physics.* **10**(1), 002, (2021).

[135] T. Rizzo, Path integral approach unveils role of complex energy landscape for activated dynamics of glassy systems, *Phys. Rev. B.* **104**(9), 094203, (2021).

[136] D. A. Stariolo and L. F. Cugliandolo, Barriers, trapping times, and overlaps between local minima in the dynamics of the disordered ising p-spin model, *Phys. Rev. E.* **102**(2), 022126, (2020).

[137] M. R. Carbone and M. Baity-Jesi, Competition between barrier-and entropy-driven activation in glasses, *arXiv:2201.01208*. (2022).

[138] T. Aspelmeier and M. Moore, Free-energy barriers in the sherrington-kirkpatrick model, *Phys. Rev. E.* **105**(3), 034138, (2022).

[139] N. P. Baskerville, J. P. Keating, F. Mezzadri, and J. Najnudel, A spin glass model for the loss surfaces of generative adversarial networks, *J. Stat. Phys.* **186**(2), 1–45, (2022).

[140] N. P. Baskerville, J. P. Keating, F. Mezzadri, and J. Najnudel, The loss surfaces of neural networks with general activation functions, *J. Stat. Mech.: Theory Exp.* **2021**(6), 064001, (2021).

[141] Y. V. Fyodorov, A spin glass model for reconstructing nonlinearly encrypted signals corrupted by noise, *J. Stat. Phys.* **175**(5), 789–818, (2019).

[142] P. Urbani, A continuous constraint satisfaction problem for the rigidity transition in confluent tissues, *arXiv preprint arXiv:2208.11730*. (2022).

[143] A. Maillard, G. B. Arous, and G. Biroli. Landscape complexity for the empirical risk of generalized linear models. In *MSML*, pp. 287–327. PMLR, (2020).

[144] J. Ipsen and P. Forrester, Kac–rice fixed point analysis for single-and multi-layered complex systems, *J. Phys. A.* **51**(47), 474003, (2018).

[145] N. P. Baskerville, J. P. Keating, F. Mezzadri, J. Najnudel, and D. Granziol, Universal characteristics of deep neural network loss surfaces from random matrix theory, *arXiv:2205.08601*. (2022).

[146] J. Kent-Dobias and J. Kurchan, How to count in hierarchical landscapes: a 'full' solution to mean-field complexity, *arXiv:2207.06161*. (2022).
[147] G. Wainrib and J. Touboul, Topological and dynamical complexity of random neural networks, *Phys. Rev. Lett.* **110**(11), 118101, (2013).

[148] Y. V. Fyodorov and B. A. Khoruzhenko, Nonlinear analogue of the may-wigner instability transition, *Proc. Natl. Acad. Sci. USA.* **113**(25), 6827-6832, (2016).

[149] Y. V. Fyodorov, Topology trivialization transition in random non-gradient autonomous odes on a sphere, *J. Stat. Mech.: Theory Exp.* **2016**(12), 124003, (2016).

[150] J. R. Ipsen, May-wigner transition in large random dynamical systems, *J. Stat. Mech.: Theory Exp.* **2017**(9), 093209, (2017).

[151] G. Ben Arous, Y. V. Fyodorov, and B. A. Khoruzhenko, Counting equilibria of large complex systems by instability index, *Proc. Natl. Acad. Sci. USA.* **118**(34), e2023719118, (2021).

[152] S. B. Fedeli, Y. V. Fyodorov, and J. Ipsen, Nonlinearity-generated resilience in large complex systems, *Phys. Rev. E.* **103**(2), 022201, (2021).

[153] B. Lacroix-A-Chez-Toine and Y. V. Fyodorov, Counting equilibria in a random non-gradient dynamics with heterogeneous relaxation rates, *J. Phys. A.* **55**(14), 144001, (2022).

[154] V. Ros, G. Biroli, G. Bunin, F. Roy, and A. Turner, Generalized Lotka-Volterra equations with random, non-reciprocal interactions: the typical number of equilibria, *to appear*. (2022).