Initial Discrete State Markov Decision Process implementations