  The recent mean field game (MFG) formalism facilitates otherwise intractable
computation of approximate Nash equilibria in many-agent settings. In this
paper, we consider discrete-time finite MFGs subject to finite-horizon
objectives. We show that all discrete-time finite MFGs with non-constant fixed
point operators fail to be contractive as typically assumed in existing MFG
literature, barring convergence via fixed point iteration. Instead, we
incorporate entropy-regularization and Boltzmann policies into the fixed point
iteration. As a result, we obtain provable convergence to approximate fixed
points where existing methods fail, and reach the original goal of approximate
Nash equilibria. All proposed methods are evaluated with respect to their
exploitability, on both instructive examples with tractable exact solutions and
high-dimensional problems where exact methods become intractable. In
high-dimensional scenarios, we apply established deep reinforcement learning
methods and empirically combine fictitious play with our approximations.
