Repository: zalando/patroni

Issue:
python3.7: "async" is a keyword now
Installing patroni with python3.7 fails: (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=904369)
```
patroni (1.4.4-2) wird eingerichtet ...
  File "/usr/lib/python3/dist-packages/patroni/ha.py", line 610
    'offline':          dict(stop='fast', checkpoint=False, release=False, offline=True, async=False),
                                                                                             ^
SyntaxError: invalid syntax
```
The problem exists also in python3-kubernetes where the parameter was renamed from "async" to "async_req" in 7.0.0a1: https://github.com/kubernetes-client/python/blob/master/CHANGELOG.md

A new release with a fix would be appreciated so we can update the Debian package. Thanks!


Discussion:
Thanks for reporting! PRs are also welcome :smile: 
I think we should start by testing Python 3.7 on Travis CI, i.e. adding "3.7" to https://github.com/zalando/patroni/blob/master/.travis.yml

For reference the Python 3.7 change log which mentions `async` as keyword: https://docs.python.org/3/whatsnew/3.7.html
Running the unit tests with Python 3.7 show the problem:

```
$ python3.7 -m pytest
...
===================================================================================================== ERRORS ======================================================================================================
_______________________________________________________________________________________ ERROR collecting tests/test_api.py ________________________________________________________________________________________
../../.local/share/virtualenvs/patroni-l8N-lQXh/lib/python3.7/site-packages/_pytest/python.py:469: in _importtestmodule
    mod = self.fspath.pyimport(ensuresyspath=importmode)
../../.local/share/virtualenvs/patroni-l8N-lQXh/lib/python3.7/site-packages/py/_path/local.py:668: in pyimport
    __import__(modname)
<frozen importlib._bootstrap>:983: in _find_and_load
    ???
<frozen importlib._bootstrap>:967: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:668: in _load_unlocked
    ???
<frozen importlib._bootstrap>:638: in _load_backward_compatible
    ???
../../.local/share/virtualenvs/patroni-l8N-lQXh/lib/python3.7/site-packages/_pytest/assertion/rewrite.py:226: in load_module
    py.builtin.exec_(co, mod.__dict__)
tests/test_api.py:9: in <module>
    from patroni.ha import _MemberStatus
E     File "/home/hjacobs/workspace/patroni/patroni/ha.py", line 600
E       'offline':          dict(stop='fast', checkpoint=False, release=False, offline=True, async=False),
E                                                                                                ^
E   SyntaxError: invalid syntax
------------------------------------------------------------------------------------------------- Captured stderr -------------------------------------------------------------------------------------------------
/home/hjacobs/.local/share/virtualenvs/patroni-l8N-lQXh/lib/python3.7/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
  """)
_______________________________________________________________________________________ ERROR collecting tests/test_ctl.py ________________________________________________________________________________________
../../.local/share/virtualenvs/patroni-l8N-lQXh/lib/python3.7/site-packages/_pytest/python.py:469: in _importtestmodule
    mod = self.fspath.pyimport(ensuresyspath=importmode)
../../.local/share/virtualenvs/patroni-l8N-lQXh/lib/python3.7/site-packages/py/_path/local.py:668: in pyimport
    __import__(modname)
<frozen importlib._bootstrap>:983: in _find_and_load
    ???
<frozen importlib._bootstrap>:967: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:668: in _load_unlocked
    ???
<frozen importlib._bootstrap>:638: in _load_backward_compatible
    ???
../../.local/share/virtualenvs/patroni-l8N-lQXh/lib/python3.7/site-packages/_pytest/assertion/rewrite.py:226: in load_module
    py.builtin.exec_(co, mod.__dict__)
tests/test_ctl.py:17: in <module>
    from test_ha import get_cluster_initialized_without_leader, get_cluster_initialized_with_leader, \
<frozen importlib._bootstrap>:983: in _find_and_load
    ???
<frozen importlib._bootstrap>:967: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:668: in _load_unlocked
    ???
<frozen importlib._bootstrap>:638: in _load_backward_compatible
    ???
../../.local/share/virtualenvs/patroni-l8N-lQXh/lib/python3.7/site-packages/_pytest/assertion/rewrite.py:226: in load_module
    py.builtin.exec_(co, mod.__dict__)
tests/test_ha.py:11: in <module>
    from patroni.ha import Ha, _MemberStatus
E     File "/home/hjacobs/workspace/patroni/patroni/ha.py", line 600
E       'offline':          dict(stop='fast', checkpoint=False, release=False, offline=True, async=False),
E                                                                                                ^
E   SyntaxError: invalid syntax
________________________________________________________________________________________ ERROR collecting tests/test_ha.py ________________________________________________________________________________________
../../.local/share/virtualenvs/patroni-l8N-lQXh/lib/python3.7/site-packages/_pytest/python.py:469: in _importtestmodule
    mod = self.fspath.pyimport(ensuresyspath=importmode)
../../.local/share/virtualenvs/patroni-l8N-lQXh/lib/python3.7/site-packages/py/_path/local.py:668: in pyimport
    __import__(modname)
<frozen importlib._bootstrap>:983: in _find_and_load
    ???
<frozen importlib._bootstrap>:967: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:668: in _load_unlocked
    ???
<frozen importlib._bootstrap>:638: in _load_backward_compatible
    ???
../../.local/share/virtualenvs/patroni-l8N-lQXh/lib/python3.7/site-packages/_pytest/assertion/rewrite.py:226: in load_module
    py.builtin.exec_(co, mod.__dict__)
tests/test_ha.py:11: in <module>
    from patroni.ha import Ha, _MemberStatus
E     File "/home/hjacobs/workspace/patroni/patroni/ha.py", line 600
E       'offline':          dict(stop='fast', checkpoint=False, release=False, offline=True, async=False),
E                                                                                                ^
E   SyntaxError: invalid syntax
____________________________________________________________________________________ ERROR collecting tests/test_kubernetes.py ____________________________________________________________________________________
../../.local/share/virtualenvs/patroni-l8N-lQXh/lib/python3.7/site-packages/_pytest/python.py:469: in _importtestmodule
    mod = self.fspath.pyimport(ensuresyspath=importmode)
../../.local/share/virtualenvs/patroni-l8N-lQXh/lib/python3.7/site-packages/py/_path/local.py:668: in pyimport
    __import__(modname)
<frozen importlib._bootstrap>:983: in _find_and_load
    ???
<frozen importlib._bootstrap>:967: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:668: in _load_unlocked
    ???
<frozen importlib._bootstrap>:638: in _load_backward_compatible
    ???
../../.local/share/virtualenvs/patroni-l8N-lQXh/lib/python3.7/site-packages/_pytest/assertion/rewrite.py:226: in load_module
    py.builtin.exec_(co, mod.__dict__)
tests/test_kubernetes.py:4: in <module>
    from patroni.dcs.kubernetes import Kubernetes, KubernetesError, k8s_client, k8s_watch, RetryFailedError
patroni/dcs/kubernetes.py:10: in <module>
    from kubernetes import client as k8s_client, config as k8s_config, watch as k8s_watch
../../.local/share/virtualenvs/patroni-l8N-lQXh/lib/python3.7/site-packages/kubernetes/__init__.py:19: in <module>
    import kubernetes.client
../../.local/share/virtualenvs/patroni-l8N-lQXh/lib/python3.7/site-packages/kubernetes/client/__init__.py:540: in <module>
    from .apis.admissionregistration_api import AdmissionregistrationApi
../../.local/share/virtualenvs/patroni-l8N-lQXh/lib/python3.7/site-packages/kubernetes/client/apis/__init__.py:4: in <module>
    from .admissionregistration_api import AdmissionregistrationApi
E     File "/home/hjacobs/.local/share/virtualenvs/patroni-l8N-lQXh/lib/python3.7/site-packages/kubernetes/client/apis/admissionregistration_api.py", line 120
E       async=params.get('async'),
E           ^
E   SyntaxError: invalid syntax


Resolving patch:
diff --git a/patroni/ha.py b/patroni/ha.py
--- a/patroni/ha.py
+++ b/patroni/ha.py
@@ -597,10 +597,10 @@ def demote(self, mode):
                 PostgreSQL as quickly as possible without regard for data durability. May only be called synchronously.
         """
         mode_control = {
-            'offline':          dict(stop='fast', checkpoint=False, release=False, offline=True, async=False),
-            'graceful':         dict(stop='fast', checkpoint=True, release=True, offline=False, async=False),
-            'immediate':        dict(stop='immediate', checkpoint=False, release=True, offline=False, async=True),
-            'immediate-nolock': dict(stop='immediate', checkpoint=False, release=False, offline=False, async=True),
+            'offline':          dict(stop='fast', checkpoint=False, release=False, offline=True, async_req=False),
+            'graceful':         dict(stop='fast', checkpoint=True, release=True, offline=False, async_req=False),
+            'immediate':        dict(stop='immediate', checkpoint=False, release=True, offline=False, async_req=True),
+            'immediate-nolock': dict(stop='immediate', checkpoint=False, release=False, offline=False, async_req=True),
         }[mode]
 
         self.state_handler.trigger_check_diverged_lsn()
@@ -620,7 +620,7 @@ def demote(self, mode):
         # FIXME: with mode offline called from DCS exception handler and handle_long_action_in_progress
         # there could be an async action already running, calling follow from here will lead
         # to racy state handler state updates.
-        if mode_control['async']:
+        if mode_control['async_req']:
             self._async_executor.schedule('starting after demotion')
             self._async_executor.run_async(self.state_handler.follow, (node_to_follow,))
         else:


Test patch: