Added support to monitor the crawler