UCB needs its parameters tuned but is otherwise working. We should probably just scale the reward by 1/10000 or something