  We study reinforcement learning (RL) with linear function approximation under
the adaptivity constraint. We consider two popular limited adaptivity models:
batch learning model and rare policy switch model, and propose two efficient
online RL algorithms for linear Markov decision processes. In specific, for the
batch learning model, our proposed LSVI-UCB-Batch algorithm achieves an $\tilde
O(\sqrt{d^3H^3T} + dHT/B)$ regret, where $d$ is the dimension of the feature
mapping, $H$ is the episode length, $T$ is the number of interactions and $B$
is the number of batches. Our result suggests that it suffices to use only
$\sqrt{T/dH}$ batches to obtain $\tilde O(\sqrt{d^3H^3T})$ regret. For the rare
policy switch model, our proposed LSVI-UCB-RareSwitch algorithm enjoys an
$\tilde O(\sqrt{d^3H^3T[1+T/(dH)]^{dH/B}})$ regret, which implies that $dH\log
T$ policy switches suffice to obtain the $\tilde O(\sqrt{d^3H^3T})$ regret. Our
algorithms achieve the same regret as the LSVI-UCB algorithm (Jin et al.,
2019), yet with a substantially smaller amount of adaptivity.
