removing some debug code from crawler