made it more mem efficient. each url is loaded in browser real time, instead of waitign for crawl to finish.