Mightn't it be better to put this code into the Crawler so that all tests can benefit from it? It's not specific to this test, is it?  Also, for some reason I cannot get the test to complete, even with your change in place. It just seems to run forever.