Ideally we should just reuse the client. But the crawler does need a different site-config (may be change the contract). We can atleast be good citizens and shut it down, it may reduce number of CLOSE_WAITs probably