  We develop an exactly solvable framework of Markov decision process with a
finite horizon, and continuous state and action spaces. We first review the
exact solution of conventional linear quadratic regulation with a linear
transition and a Gaussian noise, whose optimal policy does not depend on the
Gaussian noise, which is an undesired feature in the presence of significant
noises. It motivates us to investigate exact solutions which depend on noise.
To do so, we generalize the reward accumulation to be a general binary
commutative and associative operation. By a new multiplicative accumulation, we
obtain an exact solution of optimization assuming linear transitions with a
Gaussian noise and the optimal policy is noise dependent in contrast to the
additive accumulation. Furthermore, we also show that the multiplicative scheme
is a general framework that covers the additive one with an arbitrary
precision, which is a model-independent principle.
