Before revision:  To overcome this, we develop an algorithm that uses upper confidence bounds on the human policy to find a sequence of switching policies whose total regret with respect to the optimal switching policy is sublinear . Simulation experiments on two important tasks in autonomous driving---lane keeping and obstacle avoidance---demonstrate the effectiveness of the proposed algorithms and illustrate our theoretical findings .
After revision:  To overcome this, we develop an algorithm that uses upper confidence bounds on the agents' policies and the environment's transition probabilities to find a sequence of switching policies whose total regret with respect to the optimal switching policy is sublinear . Simulation experiments on two important tasks in autonomous driving---lane keeping and obstacle avoidance---demonstrate the effectiveness of the proposed algorithms and illustrate our theoretical findings .