# Thermodynamic Formalism in Neuronal Dynamics and Spike Train Statistics
## 2 Mathematical Setting of Thermodynamic Formalism
### Correlations
#### 2.3.4 Large Deviations  
The central limit theorem describes small fluctuations in the limit when \(n\) goes to infinity. Rare events that are exponentially small are the object of study of the large deviations theory.  
An empirical average \(A_{n}(f)\) satisfies a large deviation principle (LDP) with rate function \(I_{f}\), if the following limit exists:  
\[I_{f}(s):=-\lim_{n\to\infty}\frac{1}{n}\log\mathbb{P}\left(\left\{A_{n}(f)>s \right\}\right), \tag{11}\]  
for \(s\in\mathbb{R}\). The above condition for large \(n\) implies that \(\mathbb{P}\left(\left\{A_{n}(f)>s\right\}\right)\approx e^{-nI_{f}(s)}\). In particular, if \(s>\int f\mathrm{d}\mu\), then \(\mathbb{P}\left(\left\{A_{n}(f)>s\right\}\right)\) should tend to zero as \(n\) increases. The rate function tells us precisely how fast this probability goes to zero. Computing the rate function from Equation (11), may be a laborious task. The Gartner-Ellis theorem provides a way to compute \(I_{f}\) more easily [63]. To this end, let us introduce the _scaled cumulant generating function_ (SCGF) associated with the observable \(f\), by  
\[\Gamma_{f}(k)=:\lim_{n\to\infty}\frac{1}{n}\log\int e^{nkA_{n}(f)}\mathrm{d}\mu \quad k\in\mathbb{R},\]  
whenever the limit exists. The name comes from the fact that the \(n\)-th cumulant of \(f\) can be obtained by successive differentiation operations over \(\Gamma_{f}(k)\) with respect to \(k\), and then evaluating the result at \(k=0\). If \(\Gamma_{f}\) is differentiable, then the Gartner-Ellis theorem ensures that the average \(A_{n}(f)\) satisfies a LDP with rate function given by the Legendre transform of \(\Gamma_{f}\), that is  
\[I_{f}(s)=\max_{k\in\mathbb{R}}\{ks-\Gamma_{f}(k)\}.\]  
Therefore, one can study the large deviations of empirical averages \(A_{n}(f)\) by first computing their SCGF, characterise its differentiability, and then find the Legendre transform. We compute this function in the context of spike train statistics later in Section 2.6.  
If \(\Gamma_{f}(k)\) is differentiable then \(I_{f}(s)\) is convex [87], thus has a unique global minimum \(s^{*}\) such that \(I_{f}(s^{*})=0\), then \(I_{f}^{\prime}(s^{*})=0\). Assume that \(I_{f}(s)\) admits a Taylor expansion around \(s^{*}\), then for \(s\) close to \(s^{*}\),  
\[I_{f}(s)=I_{f}\left(s^{*}\right)+I_{f}^{\prime}\left(s^{*}\right)\left(s-s^{*} \right)+\frac{I_{f}^{\prime\prime}\left(s^{*}\right)\left(s-s^{*}\right)^{2}} {2}+O\left(s-s^{*}\right)^{3}.\]  
Because \(I_{f}\left(s^{*}\right)=0\) and \(I_{f}^{\prime}\left(s^{*}\right)=0\), for large values of \(n\) we obtain from (11)  
\[\mathbb{P}\left(\left\{A_{n}(f)>s\right\}\right) \approx e^{-nI_{f}(s)}\] \[\approx e^{-n\left(\frac{I_{f}^{\prime\prime}\left(s^{*}\right) \left(s-s^{*}\right)^{2}}{2}\right)}\]  
Therefore, the small deviations of \(A_{t}(f)\) around \(s^{*}\) are Gaussian with variance \(1/nI_{f}^{\prime\prime}(s^{*})\). In this way, the LDP can be regarded as an extension of the CLT.  
The large deviation principle plays an important role in statistical mechanics, in particular in spin glass dynamics [59]. A large deviation principle can be used in order to relate entropy and free energy (here pressure) through a Legendre transform and to explain why variational principles arise in statistical mechanics [63; 88]. As mentioned in the introduction, large deviations is the common theoretical principle linking dynamic mean field theory, maximum entropy principle, maximum likelihood, and Thermodynamic Formalism, although this link has not been studied in detail, to our best knowledge.