catch any exceptions from each crawler