Before revision:  We discuss the asymptotic properties of the estimators and small sample regret guarantees of the estimated policy .
After revision:  We derive the estimators' asymptotic properties, and small sample regret guarantees of the estimated policy .