Made minor changes to crawler