Before revision:  We study a consideration for the exploration vs. exploitation framework that does not arise in non-contextual bandits : the way exploration  is conducted in the present may affect the bias and variance in the potential outcome model estimation in subsequent stages of learning.
After revision:  We study a consideration for the exploration vs. exploitation framework that does not arise in non-contextual bandits : the way exploration is conducted in the present affects the bias and variance in the potential outcome model estimation in subsequent stages of learning.