We will next show how the re-estimation step can be modified so that the convergence rate will be independent of which yields a significant speedup if is small. Consider the case that we estimate the sources one at a time and that the sources are assumed to be whitened and the mixing matrix A orthonormal. Denote one of the source signals in the optimal solution as . By optimality we mean that the standard EM-algorithm will eventually converge to with . When we have not yet found the optimal vector , we have where .
G is mostly due to the other sources and to a small extent the Gaussian noise in the data.
We can think that the E-step filters away the noise by making use of the knowledge about the prior distribution of s . G is due to other sources and a slow convergence results. From this point of view, it is obvious that we can speed up the convergence if we can filter away also that part of s G which is due to other sources. When we are far from the optimal solution, it is natural to assume that and . Since a and s 0 are linearly related, we get If we can compute a G , we can adjust the vector a to take into account the apparent noise due to other sources.