Before revision:  We propose a dynamic treatment  policy which attains the minimax optimal regret relative to the unknown best treatment in this dynamic setting. We allow for the data to arrive in batches as, say, unemployment programs only start once a month or blood samples are only send to the laboratory for investigation in batches. Furthermore, we show that the minimax optimality does not come at the price of overly aggressive experimentation as we provide upper bounds on the expected number of times any suboptimal treatment is assigned.
After revision:  We propose a dynamic treatment policy which attains the minimax optimal regret is proposed. We also demonstrate that the expected number of times any suboptimal treatment is assigned.