crawler sleep and db error handling