Clear VFS reference to zfsvfs before we start to release it