Adds failover logic (and test) to ingestion workers

Ingestion workers now publish ExceptionEvents when corruption occurs,
the stream publishers, or a managing entity could listen for these
events and attempt to either clean the coverages or stop ingestion until
further investigation is complete.  In any case this is a step in
bullet-proofing the system.

Part of [OOIION-660](https://jira.oceanobservatories.org/tasks/browse/OOIION-660)