bitly/nsq — pull request #422 by mreiferson (2014-07-27 15:01:14+00:00)
Title: nsqd: "unhealthy mode" when diskqueue writes fail

Hailo reported an issue where `nsqd` could not write to a failed ephemeral disk on EC2 but the node continued to accept `PUB`.

In this scenario, `nsqd` should enter an "unhealthy mode" where `PUB`s are rejected until the issue can be investigated.

This feedback mechanism to publishers is crucial for them to be able to attempt delivery to another (healthy) node in a cluster or react to this condition in some other way.

Initially, I don't think it's necessary for it to attempt to "heal" itself. I imagine that in most cases you would want to restart the node anyway, which would clear this state.

cc @boyand