Move some iterator logic into IteratorView