add the configurable crawler