added code for better processing of the vbulleting index page crawl