implement rest of message deduplication and exception handling

this still needs configuration support, cleaning up and tests, but works for now.