crawler updated, database needs updateing to make crawling manageable