Before revision:  Motivated by the econometrics and statistics literatures on causal effects estimation, we study a new consideration to the exploration vs. exploitation framework , which is that the way exploration is conducted in the present may contribute to the bias and variance in the potential outcome model estimation in subsequent stages of learning.
After revision:  Motivated by the econometrics and statistics literatures on causal effects estimation, we study a new consideration to the exploration vs. exploitation framework that does not arise in non-contextual bandits: the way exploration is conducted in the present may contribute to the bias and variance in the potential outcome model estimation in subsequent stages of learning.