SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\dot {z} (t) = \frac {\hat {q} (0) e ^ {- \hat {z} (t)}}{\hat {p} (0) + \hat {q} (0) e ^ {- \hat {z} (t)}} - \frac {e ^ {- \hat {z} _ {*}}}{1 + e ^ {- \hat {z} _ {*}}}. \tag {73}
$$
This system has a unique stable point equal to
$$
\tilde {z} _ {*} := \hat {z} _ {*} + \log (\hat {q} (0) / \hat {p} (0)). \tag {74}
$$
Remark. If the algorithm had been started at  $z(0) \neq 0$ , one could check that the stable point would become  $\hat{z}_{*} + \log (\hat{q}(0) / \hat{p}(0)) + z(0)$ .
Freezing the weights leads to mode collapse
If the walkers still evolve under the Langevin dynamics in (67) but the weights are frozen at  $e^{A_t^i} = e^0 = 1$  ('unweighted' algorithm), then (66) becomes
$$
\dot {z} (t) = \frac {\sum_ {i = 1} ^ {N} \partial_ {z} U _ {z (t)} \left(X _ {t} ^ {i}\right)}{N} - \frac {\sum_ {i = 1} ^ {n} \partial_ {z} U _ {z (t)} \left(x _ {*} ^ {i}\right)}{n}. \tag {75}
$$
Keeping the same notation as in the last subsection, the second term is still approximated by  $\hat{q}_{*}$ , but this time the first term is instead approximated by
$$
\frac {\sum_ {i : X _ {t} ^ {i} \in I _ {b}} 1}{n} = \hat {q} (t) \approx \hat {q} (0).
$$
Consequently, (75) is a perturbation of the system
$$
\dot {z} (t) = \hat {q} (0) - \frac {e ^ {- \hat {z} _ {*}}}{1 + e ^ {- \hat {z} _ {*}}} =: \hat {\gamma}, \tag {76}
$$
which no longer depends on  $t$  and thus leads to  $z(t) = \hat{\gamma} t$  and to mode collapse.
The PCD algorithm leads to no-learning
In the preceding paragraph, at initialization, the walkers  $X_0^i$  are distributed according to the initial model  $\rho_{z(0)}$ . In the PCD algorithm 4, the walkers are instead initialized directly at the training data  $\{x_{*}^{i}\}_{i = 1}^{n}$ . However, in this case, the analysis of the preceding paragraph remains essentially the same, with a single difference: the initial proportion of walkers that are close to  $a$ , noted  $\hat{q}(0)$ , is now exactly  $\hat{q}_{*}$ . Thus, (66) becomes a perturbation of
$$
\dot {z} (t) = \hat {q} _ {*} - \hat {q} _ {*} = 0. \tag {77}
$$
The parameters remain constantly equal to its initial value  $z(0)$ , i.e. there is no learning.
The CD algorithm leads to no-learning
The Continuous-time Contrastive-Divergence algorithm is minimizing the Stein score and is equivalent to Score-Matching as mentioned above: the direction of the gradient of the log-likelihood is that of the gradient of the Stein Score, leading to no-learning. With the estimation given by the training samples, the analysis is exactly the same as above:
$$
\begin{array}{l} \dot {z} (t) = \frac {1}{n} \sum_ {i = 1} ^ {n} \partial_ {z} | \nabla U _ {z} (x _ {*} ^ {i}) - \nabla U _ {z _ {*}} (x _ {*} ^ {i}) | ^ {2} \\ = \frac {1}{n} \sum_ {i = 1} ^ {n} 2 \partial_ {z} \nabla U _ {z} \left(x _ {*} ^ {i}\right) \times \left(\nabla U _ {z} \left(x _ {*} ^ {i}\right) - \nabla U _ {z _ {*}} \left(x _ {*} ^ {i}\right)\right). \\ \end{array}
$$
If  $x_{*}^{i} \in I_{a}$ , then as explained in the proof of Lemma 62,  $\nabla U_{s}(x_{*}^{i}) \approx (x_{*}^{i} - a)$  for every  $s$ , hence  $\partial_z \nabla U_z(x_{*}^i) \times (\nabla U_z(x_{*}^i) - \nabla U_{z_*}(x_{*}^i)) \approx 0$ . The same holds for  $x_{*}^{i} \in I_{b}$ , leading to (66) being a perturbation of the system
$$
\dot {z} (t) = 0. \tag {78}
$$
C.4 On mode-collapse oscillations
Most of the approximations performed earlier rely on (A2), that is, the learned parameter  $z(t)$  remains in a compact set.
However, in the unweighted algorithm, this is no longer the case at large time scales, since  $z(t)$  diverges from (76); in particular, the approximations from Lemma 1 become meaningless. In fact, Proposition 3 is no longer relevant. The core of Proposition 3 rests upon the fact that if a walker  $X_{t}^{i}$  is close to  $a$ , then its dynamics (67) is close to an Ornstein-Uhlenbeck process since  $\nabla U_{z(t)}(X_t^i)\approx (X_t^i -a)$ . This fails when  $z(t)$  has a large absolute value. Let us suppose for instance that  $z(t)$  is very small (negative), so that  $e^{-z(t)}$  is very large,  $|z(t)|\gg |x - b|^2$ . In (63), the first term of the numerator is dominated by the second term. Overall, we get
$$
\nabla U _ {z (t)} \left(X _ {t} ^ {i}\right) \approx \left(X _ {t} ^ {i} - b\right),
$$
and this is valid for all  $X_{t}^{i}$ . Consequently, all the walkers now undergo an Ornstein-Uhlenbeck process centered at  $b$  and in particular, the walkers that are close to  $a$  are exponentially fast transferred to the region close to  $b$ . At this point, the first term in (75) becomes close to 1, leading to the approximated system  $\dot{z}(t) = 1 - \hat{q}_{*} \colon z(t)$  oscillates back to  $+\infty$ , until the same phenomenon happens again and all the walkers transfer to the region close to  $a$ .
This leads to an oscillating behavior that can be observed on longer time scales (see Figure 8 for example). We do not think that this phenomenon is relevant to real-world situations since most learning algorithms are trained for a limited time period.