update web crawler with recent revises