Still not converging. Redid reward system to be +1 per move,
and also have a variable that is the base expected reward
so it can be positive and others can be 0.

I don't know exactly why it's not converging.... i don't know if
in theory, if it has a small enough a it would converge eventually
after a long time, or if my algo is violating something.

Plan:
Do updates on an episode by episode basis; have a very distinct
choosing via policy, and policy updating stage. And instead of
updating on each move, update all at the end of the episode.
Idk why the current method isn't working, but I think doing things
this way might help.

Also verify taht it works well with hand-tuned parameters, which it looks
like it does.