Repository: saltstack/salt (Python)

Issue:
PR 48222 broke the GlusterFS module for versions not 4.0 and above
### Description of Issue/Question
PR #48222 breaks this module, if you're running a version of GlusterFS not 4.0 and above.

### Setup
I upgraded from 2017.7.0 to 2018.3.3 (looks like this change was also backported to 2017.7 branch and later releases).

Run a version of GlusterFS not 4.0+.  Specifically, I'm running a 3.x version (test environment has 3.7.9).  The `gluster --version` command returns `glusterfs 3.7.9 built on Mar 20 2016 03:19:49` with that version and so with the change from `[1]` to `[-1]` in the line.split is what causes the issue, as the module now returns `03:19:49` and therefore cannot be processed correctly.

I have no idea how a 4.0+ return looks like, but the PR broke this module for older versions of GlusteFS.

### Steps to Reproduce Issue
With a GlusterFS version of 3.x, run just about any of the module commands, as they're passed through _gluster_xml method that calls _get_version().  You should get a stack trace back similar to mine:

```
An exception occurred in this state: Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/salt/state.py", line 1913, in call
    **cdata['kwargs'])
  File "/usr/lib/python2.7/site-packages/salt/loader.py", line 1898, in wrapper
    return f(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/salt/states/glusterfs.py", line 84, in peered
    peers = __salt__['glusterfs.peer_status']()
  File "/usr/lib/python2.7/site-packages/salt/modules/glusterfs.py", line 162, in peer_status
    root = _gluster_xml('peer status')
  File "/usr/lib/python2.7/site-packages/salt/modules/glusterfs.py", line 74, in _gluster_xml
    if _get_version() < (3, 6,):
  File "/usr/lib/python2.7/site-packages/salt/modules/glusterfs.py", line 40, in _get_version
    version = [int(i) for i in version]
ValueError: invalid literal for int() with base 10: '03:19:49'
```

### Versions Report
```
salt --versions-report
Salt Version:
           Salt: 2018.3.3
 
Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: Not Installed
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.7.2
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.5 (default, Jul 13 2018, 13:06:57)
   python-gnupg: Not Installed
         PyYAML: 3.12
          PyZMQ: 15.3.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4
 
System Versions:
           dist: centos 7.5.1804 Core
         locale: UTF-8
        machine: x86_64
        release: 3.10.0-862.14.4.el7.x86_64
         system: Linux
        version: CentOS Linux 7.5.1804 Core
```



Discussion:
https://github.com/saltstack/salt/commit/d5d01f66988e79ff4d5e077af8b8c77fcb2dd071
@Martin819 can you take a look at this, looks like you made the offending change.

Thanks,
Daniel
Maybe pulling out the version from the line with a regex might be a way that works for both 4.0+ versions and versions older than 4.0?
Can someone could provide the full output of the `gluster --version` command for a 4.0+ version of Glusterfs?

I think it would be good to mock tests for the method, if possible, to validate the method can correctly return the version from the differing outputs.  Here is the full output of that command from my machine:

```
glusterfs 3.7.9 built on Mar 20 2016 03:19:49
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU General Public License.
```
```
glusterfs 6.1
Repository revision: git://git.gluster.org/glusterfs.git
Copyright (c) 2006-2016 Red Hat, Inc. <https://www.gluster.org/>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
It is licensed to you under your choice of the GNU Lesser
General Public License, version 3 or any later version (LGPLv3
or later), or the GNU General Public License, version 2 (GPLv2),
in all cases as published by the Free Software Foundation.
```



Manpage of debian stretch of glusterfs 3.8.8:
https://manpages.debian.org/stretch/glusterfs-server/gluster.8.en.html

```
To run the program and display gluster prompt:

gluster [--remote-host=<gluster_node>] [--mode=script] [--xml]

(or)

To specify a command directly:

gluster [commands] [options] [--remote-host=<gluster_node>] [--mode=script] [--xml]
```
As the manpage suggests commands should be passed directly via the `gluster ` command instead of passing the commands via stdin as the function `_gluster_xml ` https://github.com/saltstack/salt/blob/develop/salt/modules/glusterfs.py#L69 does which runs through the **interactive glusterfs prompt** which also generates the welcome banner as described in the issue #52318.

The python bindings by gluster invoke the commands as arguments to `gluster` https://github.com/gluster/glustercli-python/blob/master/gluster/cli/utils.py#L34

Why must the commands be passed through stdin? The only use is here https://github.com/saltstack/salt/blob/develop/salt/modules/glusterfs.py#L76 with the comment:
```
# We will pass the command string as stdin to allow for much longer
# command strings. This is especially useful for creating large volumes
# where the list of bricks exceeds 128 characters.
```
How is there a limit of 128 chars? Is this specific to cmd.run?
Hadn't realized someone had responded to this issue... don't know where the github notification went.  Looks like from the output provided by dgengtek, the version is still the second item in the output and therefore there was no need to change the position check from 1 to -1.  Assuming the version is always the second item on the line that starts with `glusterfs`, reverting the -1 back to 1 will resolve the version check issue.
ZD-6204.



Patch:
diff --git a/changelog/50707.fixed b/changelog/50707.fixed
new file mode 100644
index 000000000000..afc0e663e94f
--- /dev/null
+++ b/changelog/50707.fixed
@@ -0,0 +1,1 @@
+Fixed glusterfs version parsing for pre-4.0
diff --git a/salt/modules/glusterfs.py b/salt/modules/glusterfs.py
index d12f7ee83e62..90241d5d422a 100644
--- a/salt/modules/glusterfs.py
+++ b/salt/modules/glusterfs.py
@@ -1,22 +1,16 @@
-# -*- coding: utf-8 -*-
 """
 Manage a glusterfs pool
 """
-from __future__ import absolute_import, print_function, unicode_literals
 
-# Import python libs
 import logging
+import re
 import sys
 import xml.etree.ElementTree as ET
 
-# Import salt libs
 import salt.utils.cloud
 import salt.utils.path
 from salt.exceptions import CommandExecutionError, SaltInvocationError
 
-# Import 3rd-party libs
-from salt.ext import six
-
 log = logging.getLogger(__name__)
 
 
@@ -35,8 +29,9 @@ def _get_version():
     cmd = "gluster --version"
     result = __salt__["cmd.run"](cmd).splitlines()
     for line in result:
-        if line.startswith("glusterfs"):
-            version = line.split()[-1].split(".")
+        m = re.match(r"glusterfs ((?:\d+\.)+\d+)", line)
+        if m:
+            version = m.group(1).split(".")
             version = [int(i) for i in version]
     return tuple(version)
 
@@ -75,11 +70,11 @@ def _gluster_xml(cmd):
     # where the list of bricks exceeds 128 characters.
     if _get_version() < (3, 6,):
         result = __salt__["cmd.run"](
-            'script -q -c "gluster --xml --mode=script"', stdin="{0}\n\004".format(cmd)
+            'script -q -c "gluster --xml --mode=script"', stdin="{}\n\004".format(cmd)
         )
     else:
         result = __salt__["cmd.run"](
-            "gluster --xml --mode=script", stdin="{0}\n".format(cmd)
+            "gluster --xml --mode=script", stdin="{}\n".format(cmd)
         )
 
     try:
@@ -208,9 +203,9 @@ def peer(name):
 
     """
     if salt.utils.cloud.check_name(name, "a-zA-Z0-9._-"):
-        raise SaltInvocationError('Invalid characters in peer name "{0}"'.format(name))
+        raise SaltInvocationError('Invalid characters in peer name "{}"'.format(name))
 
-    cmd = "peer probe {0}".format(name)
+    cmd = "peer probe {}".format(name)
     return _gluster(cmd)
 
 
@@ -276,7 +271,7 @@ def create_volume(
         "gluster2:/export/vol2/brick"]' replica=2 start=True
     """
     # If single brick given as a string, accept it
-    if isinstance(bricks, six.string_types):
+    if isinstance(bricks, str):
         bricks = [bricks]
 
     # Error for block devices with multiple bricks
@@ -291,11 +286,11 @@ def create_volume(
             peer_name, path = brick.split(":")
             if not path.startswith("/"):
                 raise SaltInvocationError(
-                    "Brick paths must start with / in {0}".format(brick)
+                    "Brick paths must start with / in {}".format(brick)
                 )
         except ValueError:
             raise SaltInvocationError(
-                "Brick syntax is <peer>:<path> got {0}".format(brick)
+                "Brick syntax is <peer>:<path> got {}".format(brick)
             )
 
     # Validate arbiter config
@@ -305,17 +300,17 @@ def create_volume(
         )
 
     # Format creation call
-    cmd = "volume create {0} ".format(name)
+    cmd = "volume create {} ".format(name)
     if stripe:
-        cmd += "stripe {0} ".format(stripe)
+        cmd += "stripe {} ".format(stripe)
     if replica:
-        cmd += "replica {0} ".format(replica)
+        cmd += "replica {} ".format(replica)
     if arbiter:
         cmd += "arbiter 1 "
     if device_vg:
         cmd += "device vg "
     if transport != "tcp":
-        cmd += "transport {0} ".format(transport)
+        cmd += "transport {} ".format(transport)
     cmd += " ".join(bricks)
     if force:
         cmd += " force"
@@ -360,7 +355,7 @@ def status(name):
         salt '*' glusterfs.status myvolume
     """
     # Get volume status
-    root = _gluster_xml("volume status {0}".format(name))
+    root = _gluster_xml("volume status {}".format(name))
     if not _gluster_ok(root):
         # Most probably non-existing volume, the error output is logged
         # This return value is easy to test and intuitive
@@ -386,7 +381,7 @@ def etree_legacy_wrap(t):
         hostname = node.find("hostname").text
         if hostname not in ("NFS Server", "Self-heal Daemon"):
             path = node.find("path").text
-            ret["bricks"]["{0}:{1}".format(hostname, path)] = etree_legacy_wrap(node)
+            ret["bricks"]["{}:{}".format(hostname, path)] = etree_legacy_wrap(node)
         elif hostname == "NFS Server":
             peerid = node.find("peerid").text
             true_hostname = hostref[peerid]
@@ -429,7 +424,7 @@ def info(name=None):
 
         bricks = {}
         for i, brick in enumerate(_iter(volume, "brick"), start=1):
-            brickkey = "brick{0}".format(i)
+            brickkey = "brick{}".format(i)
             bricks[brickkey] = {"path": brick.text}
             for child in brick:
                 if not child.tag == "name":
@@ -463,9 +458,9 @@ def start_volume(name, force=False):
 
         salt '*' glusterfs.start mycluster
     """
-    cmd = "volume start {0}".format(name)
+    cmd = "volume start {}".format(name)
     if force:
-        cmd = "{0} force".format(cmd)
+        cmd = "{} force".format(cmd)
 
     volinfo = info(name)
     if name not in volinfo:
@@ -505,7 +500,7 @@ def stop_volume(name, force=False):
         log.warning("Attempt to stop already stopped volume %s", name)
         return True
 
-    cmd = "volume stop {0}".format(name)
+    cmd = "volume stop {}".format(name)
     if force:
         cmd += " force"
 
@@ -545,7 +540,7 @@ def delete_volume(target, stop=True):
         if not stop_volume(target, force=True):
             return False
 
-    cmd = "volume delete {0}".format(target)
+    cmd = "volume delete {}".format(target)
     return _gluster(cmd)
 
 
@@ -573,9 +568,9 @@ def add_volume_bricks(name, bricks):
 
     new_bricks = []
 
-    cmd = "volume add-brick {0}".format(name)
+    cmd = "volume add-brick {}".format(name)
 
-    if isinstance(bricks, six.string_types):
+    if isinstance(bricks, str):
         bricks = [bricks]
 
     volume_bricks = [x["path"] for x in volinfo[name]["bricks"].values()]
@@ -590,7 +585,7 @@ def add_volume_bricks(name, bricks):
 
     if new_bricks:
         for brick in new_bricks:
-            cmd += " {0}".format(brick)
+            cmd += " {}".format(brick)
         return _gluster(cmd)
     return True
 
@@ -609,7 +604,7 @@ def enable_quota_volume(name):
         salt '*' glusterfs.enable_quota_volume <volume>
     """
 
-    cmd = "volume quota {0} enable".format(name)
+    cmd = "volume quota {} enable".format(name)
     if not _gluster(cmd):
         return False
     return True
@@ -629,7 +624,7 @@ def disable_quota_volume(name):
         salt '*' glusterfs.disable_quota_volume <volume>
     """
 
-    cmd = "volume quota {0} disable".format(name)
+    cmd = "volume quota {} disable".format(name)
     if not _gluster(cmd):
         return False
     return True
@@ -657,11 +652,11 @@ def set_quota_volume(name, path, size, enable_quota=False):
 
         salt '*' glusterfs.set_quota_volume <volume> <path> <size> enable_quota=True
     """
-    cmd = "volume quota {0}".format(name)
+    cmd = "volume quota {}".format(name)
     if path:
-        cmd += " limit-usage {0}".format(path)
+        cmd += " limit-usage {}".format(path)
     if size:
-        cmd += " {0}".format(size)
+        cmd += " {}".format(size)
 
     if enable_quota:
         if not enable_quota_volume(name):
@@ -687,9 +682,9 @@ def unset_quota_volume(name, path):
 
         salt '*' glusterfs.unset_quota_volume <volume> <path>
     """
-    cmd = "volume quota {0}".format(name)
+    cmd = "volume quota {}".format(name)
     if path:
-        cmd += " remove {0}".format(path)
+        cmd += " remove {}".format(path)
 
     if not _gluster(cmd):
         return False
@@ -709,7 +704,7 @@ def list_quota_volume(name):
 
         salt '*' glusterfs.list_quota_volume <volume>
     """
-    cmd = "volume quota {0}".format(name)
+    cmd = "volume quota {}".format(name)
     cmd += " list"
 
     root = _gluster_xml(cmd)
@@ -740,7 +735,7 @@ def get_op_version(name):
         salt '*' glusterfs.get_op_version <volume>
     """
 
-    cmd = "volume get {0} cluster.op-version".format(name)
+    cmd = "volume get {} cluster.op-version".format(name)
     root = _gluster_xml(cmd)
 
     if not _gluster_ok(root):
@@ -774,7 +769,7 @@ def get_max_op_version():
     if _get_version() < (3, 10,):
         return (
             False,
-            "Glusterfs version must be 3.10+.  Your version is {0}.".format(
+            "Glusterfs version must be 3.10+.  Your version is {}.".format(
                 str(".".join(str(i) for i in _get_version()))
             ),
         )
@@ -814,7 +809,7 @@ def set_op_version(version):
         salt '*' glusterfs.set_op_version <volume>
     """
 
-    cmd = "volume set all cluster.op-version {0}".format(version)
+    cmd = "volume set all cluster.op-version {}".format(version)
     root = _gluster_xml(cmd)
 
     if not _gluster_ok(root):


Test patch:
diff --git a/tests/unit/modules/test_glusterfs.py b/tests/unit/modules/test_glusterfs.py
index 1b97749b1d12..7d5206f7c7b3 100644
--- a/tests/unit/modules/test_glusterfs.py
+++ b/tests/unit/modules/test_glusterfs.py
@@ -1,32 +1,26 @@
-# -*- coding: utf-8 -*-
 """
     :codeauthor: Jayesh Kariya <jayeshk@saltstack.com>
     :codeauthor: Joe Julian <me@joejulian.name>
 """
 
-# Import Python libs
-from __future__ import absolute_import, print_function, unicode_literals
 
-# Import Salt Libs
 import salt.modules.glusterfs as glusterfs
 from salt.exceptions import SaltInvocationError
-
-# Import Salt Testing Libs
 from tests.support.mixins import LoaderModuleMockMixin
 from tests.support.mock import MagicMock, patch
 from tests.support.unit import TestCase
 
 
-class GlusterResults(object):
+class GlusterResults:
     """ This class holds the xml results from gluster cli transactions """
 
-    class v34(object):
+    class v34:
         """ This is for version 3.4 results """
 
-        class list_peers(object):
+        class list_peers:
             """ results from "peer status" """
 
-        class peer_probe(object):
+        class peer_probe:
             fail_cant_connect = fail_bad_hostname = "\n".join(
                 [
                     '<?xml version="1.0" encoding="UTF-8" standalone="yes"?>',
@@ -123,8 +117,8 @@ class peer_probe(object):
             ]
             success_first_ip_from_second_first_time = success_reverse_already_peer["ip"]
 
-    class v37(object):
-        class peer_probe(object):
+    class v37:
+        class peer_probe:
             fail_cant_connect = fail_bad_hostname = "\n".join(
                 [
                     '<?xml version="1.0" encoding="UTF-8" standalone="yes"?>',
@@ -223,6 +217,26 @@ class peer_probe(object):
             ]
 
 
+#  gluster --version output collected in the wild.
+version_output_362 = """
+glusterfs 3.6.2 built on Jan 22 2015 12:59:57
+Repository revision: git://git.gluster.com/glusterfs.git
+Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
+GlusterFS comes with ABSOLUTELY NO WARRANTY.
+You may redistribute copies of GlusterFS under the terms of the GNU General Public License
+   """
+
+version_output_61 = """
+glusterfs 6.1
+Repository revision: git://git.gluster.org/glusterfs.git
+Copyright (c) 2006-2016 Red Hat, Inc. <https://www.gluster.org/>
+GlusterFS comes with ABSOLUTELY NO WARRANTY.
+It is licensed to you under your choice of the GNU Lesser
+General Public License, version 3 or any later version (LGPLv3
+or later), or the GNU General Public License, version 2 (GPLv2),
+in all cases as published by the Free Software Foundation.
+"""
+
 xml_peer_present = """
 <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
 <cliOutput>
@@ -488,6 +502,33 @@ def setup_loader_modules(self):
 
     # 'peer_status' function tests: 1
 
+    def test__get_version(self):
+        """
+        Test parsing of gluster --version.
+        """
+        mock_version = MagicMock(return_value="foo")
+        with patch.dict(glusterfs.__salt__, {"cmd.run": mock_version}):
+            self.assertEqual(glusterfs._get_version(), (3, 6), msg="default behaviour")
+
+        mock_version = MagicMock(return_value=version_output_362)
+        with patch.dict(glusterfs.__salt__, {"cmd.run": mock_version}):
+            self.assertEqual(glusterfs._get_version(), (3, 6, 2))
+
+        mock_version = MagicMock(return_value=version_output_61)
+        with patch.dict(glusterfs.__salt__, {"cmd.run": mock_version}):
+            self.assertEqual(glusterfs._get_version(), (6, 1))
+
+        more_versions = {
+            "6.0": (6, 0),
+            "4.1.10": (4, 1, 10),
+            "5.13": (5, 13),
+            "10.0": (10, 0),
+        }
+        for v in more_versions:
+            mock_version = MagicMock(return_value="glusterfs {}".format(v))
+            with patch.dict(glusterfs.__salt__, {"cmd.run": mock_version}):
+                self.assertEqual(glusterfs._get_version(), more_versions[v])
+
     def test_peer_status(self):
         """
         Test gluster peer status