crawlie now checks the hashes of the content to check if it needs updating