Spider would crawl all the bbc news from bbc.co.uk