lastest crawler

modified a lot of things. its working perfectly. I just need to focus
on making the url search better.