debugged version - full working

before trying to change the underlying structure to record the value
from state-action pairs instead of just the optimal for each state