Hold mmap_sem during page walk