Repository: celery/celery

Issue:
RabbitMQ task_queue_ha_policy has no effect
<!--
Please fill this template entirely and do not erase parts of it.
We reserve the right to close without a response
bug reports which are incomplete.
-->
# Checklist
<!--
To check an item on the list replace [ ] with [x].
-->

- [x] I have checked the [issues list](https://github.com/celery/celery/issues?utf8=%E2%9C%93&q=is%3Aissue+label%3A%22Category%3A+Documentation%22+)
  for similar or identical bug reports.
- [x] I have checked the [pull requests list](https://github.com/celery/celery/pulls?q=is%3Apr+label%3A%22Category%3A+Documentation%22)
  for existing proposed fixes.
- [x] I have checked the [commit log](https://github.com/celery/celery/commits/master)
  to find out if the bug was already fixed in the master branch.
- [x] I have included all related issues and possible duplicate issues in this issue
      (If there are none, check this box anyway).

## Related Issues and Possible Duplicates
<!--
Please make sure to search and mention any related issues
or possible duplicates to this issue as requested by the checklist above.

This may or may not include issues in other repositories that the Celery project
maintains or other repositories that are dependencies of Celery.

If you don't know how to mention issues, please refer to Github's documentation
on the subject: https://help.github.com/en/articles/autolinked-references-and-urls#issues-and-pull-requests
-->

#### Related Issues

- None

#### Possible Duplicates

- None

# Description

The RabbitMQ setting [task_queue_ha_policy](https://docs.celeryproject.org/en/stable/userguide/configuration.html#task-queue-ha-policy) no longer has any affect, as [described in the release notes for version 3](https://www.rabbitmq.com/blog/2012/11/19/breaking-things-with-rabbitmq-3-0/).

This was [reported to the mailing list here](https://groups.google.com/u/1/g/celery-users/c/hiZPiz2JWo8/m/2_q_Q5sM0BIJ) (in 2013!) and the answer then was that it's deprecated and will be removed.

# Suggestions

Remove the configuration parameter and maybe reference the blog post and/or docs on how to set a policy.

RabbitMQ task_queue_ha_policy has no effect
<!--
Please fill this template entirely and do not erase parts of it.
We reserve the right to close without a response
bug reports which are incomplete.
-->
# Checklist
<!--
To check an item on the list replace [ ] with [x].
-->

- [x] I have checked the [issues list](https://github.com/celery/celery/issues?utf8=%E2%9C%93&q=is%3Aissue+label%3A%22Category%3A+Documentation%22+)
  for similar or identical bug reports.
- [x] I have checked the [pull requests list](https://github.com/celery/celery/pulls?q=is%3Apr+label%3A%22Category%3A+Documentation%22)
  for existing proposed fixes.
- [x] I have checked the [commit log](https://github.com/celery/celery/commits/master)
  to find out if the bug was already fixed in the master branch.
- [x] I have included all related issues and possible duplicate issues in this issue
      (If there are none, check this box anyway).

## Related Issues and Possible Duplicates
<!--
Please make sure to search and mention any related issues
or possible duplicates to this issue as requested by the checklist above.

This may or may not include issues in other repositories that the Celery project
maintains or other repositories that are dependencies of Celery.

If you don't know how to mention issues, please refer to Github's documentation
on the subject: https://help.github.com/en/articles/autolinked-references-and-urls#issues-and-pull-requests
-->

#### Related Issues

- None

#### Possible Duplicates

- None

# Description

The RabbitMQ setting [task_queue_ha_policy](https://docs.celeryproject.org/en/stable/userguide/configuration.html#task-queue-ha-policy) no longer has any affect, as [described in the release notes for version 3](https://www.rabbitmq.com/blog/2012/11/19/breaking-things-with-rabbitmq-3-0/).

This was [reported to the mailing list here](https://groups.google.com/u/1/g/celery-users/c/hiZPiz2JWo8/m/2_q_Q5sM0BIJ) (in 2013!) and the answer then was that it's deprecated and will be removed.

# Suggestions

Remove the configuration parameter and maybe reference the blog post and/or docs on how to set a policy.



Discussion:
contributions are wellcome
@auvipy @safwanrahman The fix that has been merged here does not address the problem.

The [RabbitMQ release notes](https://www.rabbitmq.com/blog/2012/11/19/breaking-things-with-rabbitmq-3-0/) describe how **it is no longer possible** to configure HA at queue declaration time. You have to use a policy, which is set on the broker and cannot be controlled by the client.




contributions are wellcome
@auvipy @safwanrahman The fix that has been merged here does not address the problem.

The [RabbitMQ release notes](https://www.rabbitmq.com/blog/2012/11/19/breaking-things-with-rabbitmq-3-0/) describe how **it is no longer possible** to configure HA at queue declaration time. You have to use a policy, which is set on the broker and cannot be controlled by the client.





Resolving patch:
diff --git a/celery/app/amqp.py b/celery/app/amqp.py
--- a/celery/app/amqp.py
+++ b/celery/app/amqp.py
@@ -46,7 +46,6 @@ class Queues(dict):
         create_missing (bool): By default any unknown queues will be
             added automatically, but if this flag is disabled the occurrence
             of unknown queues in `wanted` will raise :exc:`KeyError`.
-        ha_policy (Sequence, str): Default HA policy for queues with none set.
         max_priority (int): Default x-max-priority for queues with none set.
     """
 
@@ -55,14 +54,13 @@ class Queues(dict):
     _consume_from = None
 
     def __init__(self, queues=None, default_exchange=None,
-                 create_missing=True, ha_policy=None, autoexchange=None,
+                 create_missing=True, autoexchange=None,
                  max_priority=None, default_routing_key=None):
         dict.__init__(self)
         self.aliases = WeakValueDictionary()
         self.default_exchange = default_exchange
         self.default_routing_key = default_routing_key
         self.create_missing = create_missing
-        self.ha_policy = ha_policy
         self.autoexchange = Exchange if autoexchange is None else autoexchange
         self.max_priority = max_priority
         if queues is not None and not isinstance(queues, Mapping):
@@ -122,10 +120,6 @@ def _add(self, queue):
             queue.exchange = self.default_exchange
         if not queue.routing_key:
             queue.routing_key = self.default_routing_key
-        if self.ha_policy:
-            if queue.queue_arguments is None:
-                queue.queue_arguments = {}
-            self._set_ha_policy(queue.queue_arguments)
         if self.max_priority is not None:
             if queue.queue_arguments is None:
                 queue.queue_arguments = {}
@@ -133,13 +127,6 @@ def _add(self, queue):
         self[queue.name] = queue
         return queue
 
-    def _set_ha_policy(self, args):
-        policy = self.ha_policy
-        if isinstance(policy, (list, tuple)):
-            return args.update({'ha-mode': 'nodes',
-                                'ha-params': list(policy)})
-        args['ha-mode'] = policy
-
     def _set_max_priority(self, args):
         if 'x-max-priority' not in args and self.max_priority is not None:
             return args.update({'x-max-priority': self.max_priority})
@@ -251,7 +238,7 @@ def create_task_message(self):
     def send_task_message(self):
         return self._create_task_sender()
 
-    def Queues(self, queues, create_missing=None, ha_policy=None,
+    def Queues(self, queues, create_missing=None,
                autoexchange=None, max_priority=None):
         # Create new :class:`Queues` instance, using queue defaults
         # from the current configuration.
@@ -259,8 +246,6 @@ def Queues(self, queues, create_missing=None, ha_policy=None,
         default_routing_key = conf.task_default_routing_key
         if create_missing is None:
             create_missing = conf.task_create_missing_queues
-        if ha_policy is None:
-            ha_policy = conf.task_queue_ha_policy
         if max_priority is None:
             max_priority = conf.task_queue_max_priority
         if not queues and conf.task_default_queue:
@@ -271,7 +256,7 @@ def Queues(self, queues, create_missing=None, ha_policy=None,
                         else autoexchange)
         return self.queues_cls(
             queues, self.default_exchange, create_missing,
-            ha_policy, autoexchange, max_priority, default_routing_key,
+            autoexchange, max_priority, default_routing_key,
         )
 
     def Router(self, queues=None, create_missing=None):
diff --git a/celery/app/defaults.py b/celery/app/defaults.py
--- a/celery/app/defaults.py
+++ b/celery/app/defaults.py
@@ -267,7 +267,6 @@ def __repr__(self):
             type='dict', old={'celery_task_publish_retry_policy'},
         ),
         queues=Option(type='dict'),
-        queue_ha_policy=Option(None, type='string'),
         queue_max_priority=Option(None, type='int'),
         reject_on_worker_lost=Option(type='bool'),
         remote_tracebacks=Option(False, type='bool'),


Test patch:
diff --git a/t/unit/app/test_amqp.py b/t/unit/app/test_amqp.py
--- a/t/unit/app/test_amqp.py
+++ b/t/unit/app/test_amqp.py
@@ -89,23 +89,6 @@ def test_setitem_adds_default_exchange(self):
         q['foo'] = queue
         assert q['foo'].exchange == q.default_exchange
 
-    @pytest.mark.parametrize('ha_policy,qname,q,qargs,expected', [
-        (None, 'xyz', 'xyz', None, None),
-        (None, 'xyz', 'xyz', {'x-foo': 'bar'}, {'x-foo': 'bar'}),
-        ('all', 'foo', Queue('foo'), None, {'ha-mode': 'all'}),
-        ('all', 'xyx2',
-         Queue('xyx2', queue_arguments={'x-foo': 'bar'}),
-         None,
-         {'ha-mode': 'all', 'x-foo': 'bar'}),
-        (['A', 'B', 'C'], 'foo', Queue('foo'), None, {
-            'ha-mode': 'nodes',
-            'ha-params': ['A', 'B', 'C']}),
-    ])
-    def test_with_ha_policy(self, ha_policy, qname, q, qargs, expected):
-        queues = Queues(ha_policy=ha_policy, create_missing=False)
-        queues.add(q, queue_arguments=qargs)
-        assert queues[qname].queue_arguments == expected
-
     def test_select_add(self):
         q = Queues()
         q.select(['foo', 'bar'])
@@ -118,11 +101,6 @@ def test_deselect(self):
         q.deselect('bar')
         assert sorted(q._consume_from.keys()) == ['foo']
 
-    def test_with_ha_policy_compat(self):
-        q = Queues(ha_policy='all')
-        q.add('bar')
-        assert q['bar'].queue_arguments == {'ha-mode': 'all'}
-
     def test_add_default_exchange(self):
         ex = Exchange('fff', 'fanout')
         q = Queues(default_exchange=ex)
@@ -143,12 +121,6 @@ def test_alias(self):
         ({'max_priority': 10},
          'moo', Queue('moo', queue_arguments=None),
          {'x-max-priority': 10}),
-        ({'ha_policy': 'all', 'max_priority': 5},
-         'bar', 'bar',
-         {'ha-mode': 'all', 'x-max-priority': 5}),
-        ({'ha_policy': 'all', 'max_priority': 5},
-         'xyx2', Queue('xyx2', queue_arguments={'x-max-priority': 2}),
-         {'ha-mode': 'all', 'x-max-priority': 2}),
         ({'max_priority': None},
          'foo2', 'foo2',
          None),
@@ -255,10 +227,6 @@ def test_countdown_negative(self):
         with pytest.raises(ValueError):
             self.app.amqp.as_task_v2(uuid(), 'foo', countdown=-1232132323123)
 
-    def test_Queues__with_ha_policy(self):
-        x = self.app.amqp.Queues({}, ha_policy='all')
-        assert x.ha_policy == 'all'
-
     def test_Queues__with_max_priority(self):
         x = self.app.amqp.Queues({}, max_priority=23)
         assert x.max_priority == 23