Before revision:  Both algorithms are based on an optimistic least-squares version of value iteration analogous to the finite-horizon backward induction approach from Jin et al. 2020.
After revision:  Both algorithms are special cases of a more general one, which has \sqrt{K the finite-horizon backward induction approach from Jin et al. 2020.