The crawler can now be run on its own