  In many contexts Gaussian Mixtures (GM) are used to approximate probability
distributions, possibly time-varying. In some applications the number of GM
components exponentially increases over time, and reduction procedures are
required to keep them reasonably limited. The GM reduction (GMR) problem can be
formulated by choosing different measures of the dissimilarity of GMs before
and after reduction, like the Kullback-Leibler Divergence (KLD) and the
Integral Squared Error (ISE). Since in no case the solution is obtained in
closed form, many approximate GMR algorithms have been proposed in the past
three decades, although none of them provides optimality guarantees. In this
work we discuss the importance of the choice of the dissimilarity measure and
the issue of consistency of all steps of a reduction algorithm with the chosen
measure. Indeed, most of the existing GMR algorithms are composed by several
steps which are not consistent with a unique measure, and for this reason may
produce reduced GMs far from optimality. In particular, the use of the KLD, of
the ISE and normalized ISE is discussed and compared in this perspective.
