  Decentralized Partially Observable Markov Decision Process (DEC-POMDP) models
sequential decision making problems by a team of agents. Since the planning of
DEC-POMDP can be interpreted as the maximum likelihood estimation for the
latent variable model, DEC-POMDP can be solved by EM algorithm. However, in EM
for DEC-POMDP, the forward-backward algorithm needs to be calculated up to the
infinite horizon, which impairs the computational efficiency. In this paper, we
propose Bellman EM algorithm (BEM) and Modified Bellman EM algorithm (MBEM) by
introducing the forward and backward Bellman equations into EM. BEM can be more
efficient than EM because BEM calculates the forward and backward Bellman
equations instead of the forward-backward algorithm up to the infinite horizon.
However, BEM cannot always be more efficient than EM when the size of problems
is large because BEM calculates an inverse matrix. We circumvent this
shortcoming in MBEM by calculating the forward and backward Bellman equations
without the inverse matrix. Our numerical experiments demonstrate that the
convergence of MBEM is faster than that of EM.
