adapt crawler to latest changes