CiscoSystems/heat — pull request #3 by zhencisco (2014-11-25 23:44:11+00:00)
Title: Gracefully handle consumer cancel notifications

Back ported from oslo.messaging in Icehouse.
With mirrored queues and clustered rabbit nodes a queue is still
mastered by a single rabbit node. When the rabbit node dies an
election occurs amongst the remaining nodes and a new master is
elected. When a slave is promoted to master it will close all the
open channels to its consumers but it will not close the
connections. This is reported to consumers as a consumer cancel
notification (CCN). Consumers need to re-subscribe to these queues
when they recieve a CCN.

kombu 2.1.4+ reports CCNs as channel errors. This patch updates
the ensure function to be more inline with the upstream kombu
functionality. We now monitor for channel errors as well as
connection errors and initiate a reconnect if we detect an error.

Implements: rally-user-story US6888
Upstream-Review: https://review.openstack.org/#/c/77276/
Not-in-upstream: true