We update by transitioning to a new state. Return a number saying how well this agent is doing. Value iteration, value determination, and policy iteration. MDP agents pass in an mdp and expect a policy in return. A state is a sink if there are no actions that can lead to another state.
Sinks can arise by accident during reinforcement learning of an environment model.
Because they cause infinite loops, they must be detected. Given an initial policy P and initial utilities U, calculate the optimal policy. Do this by value determination alternating with policy update. Given a fixed policy and a model, calculate the value of each state. This version does it by an iterative process similar to value iteration.