  We consider a bivariate Markov chain $Z=\{Z_k\}_{k \geq 1}=\{(X_k,Y_k)\}_{k
\geq 1}$ taking values on product space ${\cal Z}={\cal X} \times{ \cal Y}$,
where ${\cal X}$ is possibly uncountable space and ${\cal Y}=\{1,\ldots, |{\cal
Y}|\}$ is a finite state-space. The purpose of the paper is to find sufficient
conditions that guarantee the exponential convergence of smoothing, filtering
and predictive probabilities: $$\sup_{n\geq t}\|P(Y_{t:\infty}\in
\cdot|X_{l:n})-P(Y_{t:\infty}\in \cdot|X_{s:n}) \|_{\rm TV} \leq K_s
\alpha^{t}, \quad \mbox{a.s.}$$ Here $t\geq s\geq l\geq 1$, $K_s$ is
$\sigma(X_{s:\infty})$-measurable finite random variable and $\alpha\in (0,1)$
is fixed. In the second part of the paper, we establish two-sided versions of
the above-mentioned convergence. We show that the desired convergences hold
under fairly general conditions. A special case of above-mentioned very general
model is popular hidden Markov model (HMM). We prove that in HMM-case, our
assumptions are more general than all similar mixing-type of conditions
encountered in practice, yet relatively easy to verify.
