## 2 Materials and Methods
### Tempered EM  
#### 2.3.1 Context and Motivation  
In this section, we consider another particular case of Deterministic Approximate EM: the Tempered EM (or \"tmp-EM\"), first introduced in [31]. We first prove that under mild conditions, tmp-EM verifies the hypotheses of Theorem 1, hence benefits from the state of the art EM convergence guarantees. In particular, we prove that the choice of the temperature profile is almost completely free. This justifies the use of a wider array of temperature profiles than the ones specified in [31,32]. Then, we demonstrate the interest of tmp-EM with non-monotonous profiles, on Mixture estimation problems with hard to escape initialisations. _Tempering_ or _annealing_ is a common technique in the world of non-convex optimisation. With a non-convex objective function \(g\), naively following the gradients usually leads to undesirable local extrema close to the initial point. A common remedy is to elevate \(g\) to the power \(\frac{1}{T_{n}}\), with \(\{T_{n}\}_{n\in\mathbb{N}}\) a sequence of temperatures tending towards \(1\) as the number \(n\) of steps of the procedure increases. When \(T_{n}>1\), the shape of the new objective function \(g^{\frac{1}{4n}}\) is flattened, making the gradients less strong, and the potential wells less attractive, but without changing the hierarchy of the values. This allows the optimisation procedure to explore more before converging. This concept is put in application in many state of the art procedures. The most iconic probably being the Simulated Annealing, introduced and developed in [28,34,35], where in particular \(T_{n}\longrightarrow 0\) instead of \(1\). It is one of the few optimisation technique proven to find global optimum of non-convex functions. The Parallel Tempering (or Annealing MCMC) developed in [29,30,36] also makes use of these ideas to improve the MCMC simulation of a target probability distribution. Since non-convex objectives are a common occurrence in the EM literature, [31] introduced the _Deterministic Annealed EM_, where, in the E step, the conditional probability is replaced by a tempered approximated distribution: \(\bar{p}_{n,\theta}(z)\propto p_{\theta}^{\frac{1}{4n}}(z)\propto h(z;\theta) ^{\frac{1}{4n}}\) (renormalised such that \(\int_{z}\bar{p}_{n,\theta}(z)dz=1\)). For the temperature profile they consider specific sequences \(\{T_{n}\}_{n\in\mathbb{N}}\) decreasingly converging to \(1\). Another specific, non-monotonous, temperature scheme was later proposed by [PERSON] and [PERSON] [32]. In both cases, theoretical convergence guarantees are lacking. Later, in [10], tempering has been applied to the SAEM, with convergence guarantees provided for any temperature scheme \(T_{n}\longrightarrow 1\). With Theorem 3, we show that the Deterministic Annealing EM, or tmp-EM, is a specific case of the Deterministic Approximate EM of Section 2.1, hence can benefit from the same convergence properties. In particular, we show that any temperature scheme \(\{T_{n}\}_{n}\in(\mathbb{R}^{*})^{\mathbb{N}},\ T_{n}\underset{n\to\infty}{ \longrightarrow}1\) guarantees the state of the art convergence. **Remark 3**. : _Elevating \(p_{\theta}(z)\) to the power \(\frac{1}{T_{n}}\), as is done here and in [31,32], is not equivalent to elevating to the power \(\frac{1}{T_{n}}\) the objective function \(g(\theta)\), which would be expected for a typical annealed or tempered optimisation procedure. It is not equivalent either to elevating to the power \(\frac{1}{T_{n}}\) the proxy function \(\mathbb{E}_{z\sim p_{\theta_{n}}(z)}[h(z;\theta)]\) that is optimised in the M step. Instead, the weights \(p_{\theta_{n}}(z)\) (or equivalently, the terms \(h(z;\theta_{n})\)) used in the calculation of \(\mathbb{E}_{z\sim p_{\theta_{n}}(z)}[h(z;\theta)]\) are the tempered terms. This still results in the desired behaviour and is only a more \"structured\" tempering. Indeed, with this tempering, it is the estimated distribution of the latent variable \(z\) that are made less unequivocal, with weaker modes, at each step.