Before revision:  We study a consideration for the exploration vs. exploitation framework that does not arise in non-contextual bandits : the way exploration  is conducted in the present may affect the bias and variance in the potential outcome model estimation in subsequent stages of learning.
After revision:  We study a consideration for the exploration vs. exploitation framework that does not arise in multi-armed bandits but is crucial in contextual bandits; the way exploration is conducted in the present may affect the bias and variance in the potential outcome model estimation in subsequent stages of learning.