## 2 Materials and Methods  
### Deterministic Approximate EM Algorithm and Its Convergence for the Exponential Family  
#### 2.1.1 Context and Motivation  
In this section, we propose a new class of deterministic EM algorithms with approximated E step. This class of algorithms is very general. In particular, it includes methods with deterministic approximations of intractable normalisation constant. It also includes optimisation-oriented approximations, such as the annealing approximation used to escape sub-optimal local extrema of the objective. In addition, combinations of these two types of approximations are also covered. We prove that members of this class benefit from the same convergence guarantees that can be found in the state of the art references [2,4,5] for the classical EM algorithm, and under similar model assumptions. The only condition on the approximated distribution being that it converges towards the real conditional probability distribution with a \(l_{2}\) regularity. Like the authors of [5,7,10], we work with probability density functions belonging to the exponential family. The specific properties of which are provident in the hypothesis \(M1\) of Theorem 1. EM algorithms are most often considered in the context of the _missing data problem_, see [5]. The formulation of the problem is the following: we observe a random variable \(x\), described by a parametric probability density function (pdf) noted \(g(\theta)\) with parameter \(\theta\in\Theta\subset\mathbb{R}^{l},\ l\in\mathbb{N}^{*}\). We assume that there exists a hidden variable \(z\) informing the behaviour of the observed variable \(x\) such that \(g(\theta)\) can be expressed as the integral of the complete likelihood \(h(z;\theta)\): \(g(\theta)=\int_{z}h(z;\theta)\mu(dz)\), with \(\mu\) the reference measure. We denote \(p_{\theta}(z):=h(z;\theta)/g(\theta)\) the conditional density function of \(z\). For a given sample \(x\), the goal is to maximise the likelihood \(g(\theta)\) with respect to \(\theta\). In all these notations, and throughout the rest of the paper, we omit \(x\) as a variable since it is considered fixed once and for all, and everything is done conditionally to \(x\). In particular, in many applications, the observed data \(x\) is made of \(N\in\mathbb{N}^{*}\) samples: \(x:=(x^{(1)},x^{(N)})\). In such cases, the sample size \(N<+\infty\) is supposed finite and fixed once and for all. We are never in the asymptotic regime \(N\longrightarrow+\infty\). The foundation of the EM algorithm is that while \(\ln g(\theta)\) is hard to maximise in \(\theta\), the functions \(\theta\mapsto\ln h(z;\theta)\) and even \(\theta\mapsto\mathbb{E}_{z}[\ln h(z;\theta)]\) are easier to work with because of the information added by the latent variable \(z\) (or its distribution). In practice however, the actual value of \(z\) is unknown and its distribution \(p_{\theta}(z)\) dependent on \(\theta\). Hence, the EM was introduced by [PERSON] [1] as the two-stages procedure starting from an initial point \(\theta_{0}\) and iterated over the number of steps \(n\):  
(E) With the current parameter \(\theta_{n}\), calculate the conditional probability \(p_{\theta_{n}}(z)\);  
(M) To get \(\theta_{n+1}\), maximise in \(\theta\in\Theta\) the function \(\theta\mapsto\mathbb{E}_{z\sim p_{\theta_{n}}(z)}[\ln h(z;\theta)]\);  
Which can be summarised as:  
\[\theta_{n+1}:=T(\theta_{n}):=\operatorname*{argmax}_{\theta\in\Theta}\mathbb{ E}_{z\sim p_{\theta_{n}}(z)}[\ln h(z;\theta)]\,, \tag{1}\]  
where we call \(T\) the point to point map in \(\Theta\) corresponding to one EM step. We will not redo the basic theory of the exact EM here, but this procedure noticeably increase \(g(\theta_{n})\) at each new step \(n\). However, as discussed in the introduction, in many cases, one may prefer to or need to use an approximation of \(p_{\theta_{n}}(z)\) instead of the exact analytical value.