now selects by interation and queries twice to avoid MAX_RETIES 404 errors