Repository: ansible/ansible
Pull request: Fixes #45983: Add gluster_heal_facts module for Gluster storage.
This module will add support to extract self-heal status of a cluster
and return the number of files that are still in progress for the
healing process to complete.

Signed-off-by: Devyani Kota <dkota@redhat.com>

##### SUMMARY
<!--- Describe the change below, including rationale and design decisions -->
Fixes #45983 : Adds gluster_heal_facts module
<!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue -->

##### ISSUE TYPE
- New Module Pull Request

##### COMPONENT NAME
<!--- Write the short name of the module, plugin, task or feature below -->
storage/glusterfs/gluster_heal_facts

##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes -->
```
ansible 2.4.1.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/devyani/work/gdeploy/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /bin/ansible
  python version = 2.7.13 (default, Dec  1 2017, 09:21:53) [GCC 6.4.1 20170727 (Red Hat 6.4.1-1)]

```

##### ADDITIONAL INFORMATION
In a cluster, when a brick goes offline and comes back online, self-heal needs to be
triggered to bring all the replicas in sync.
This module will help list the files that still need to be healed, and the ones that have healed.
gluster volume heal test-volume info
```
Brick server1:/glusterfs/test-volume_0
Number of entries: 0
 
Brick server2:/glusterfs/test-volume_1
Number of entries: 101 
/95.txt
/32.txt
/66.txt
/35.txt
```


Problem statement:
storage: Add gluster_heal_facts module to extract cluster heal status.
##### SUMMARY
This module will add support to extract self-heal status of a cluster and
return the number of files that are still in progress for the healing process
to complete.

##### ISSUE TYPE
- Feature Idea

##### COMPONENT NAME
Gather heal status of a cluster.
storage/glusterfs

##### ADDITIONAL INFORMATION
In a cluster, when a brick goes offline and comes back online, self-heal needs to be
triggered to bring all the replicas in sync.
This module will help list the files that still need to be healed, and the ones that have healed.

<!--- Paste example playbooks or commands between quotes below -->
```yaml
---
- name: Extract self-heal status
  gluster_heal_facts:
           vol_name="{{ volname }}"
           heal_type="{{ heal_type }}"
```

Hints from issue discussion:
Files identified in the description:
* [lib/ansible/modules/storage/glusterfs/__init__.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/storage/glusterfs/__init__.py)
* [lib/ansible/parsing/utils/yaml.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/parsing/utils/yaml.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->

Commit message: Updates #45983: Add support for rebalance_status for Gluster storage.

Added support to extract rebalance status of a cluster and return the
number of files that are still in progress for the rebalance process
to complete.

Signed-off-by: Devyani Kota <dkota@redhat.com>

Patch to review:
diff --git a/lib/ansible/modules/storage/glusterfs/gluster_heal_facts.py b/lib/ansible/modules/storage/glusterfs/gluster_heal_facts.py
new file mode 100644
index 00000000000000..9dec3dc067bd14
--- /dev/null
+++ b/lib/ansible/modules/storage/glusterfs/gluster_heal_facts.py
@@ -0,0 +1,183 @@
+#!/usr/bin/python
+# -*- coding: utf-8 -*-
+#
+# Copyright (c) 2016 Red Hat, Inc.
+# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
+
+from __future__ import absolute_import, division, print_function
+__metaclass__ = type
+
+ANSIBLE_METADATA = {'metadata_version': '1.1',
+                    'status': ['preview'],
+                    'supported_by': 'community'}
+
+
+DOCUMENTATION = '''
+---
+module: gluster_heal_facts
+short_description: Gather facts about self-heal or rebalance status
+author: "Devyani Kota (@devyanikota)"
+version_added: "2.8"
+description:
+  - Gather facts about either self-heal or rebalance status.
+options:
+  name:
+    description:
+      - The volume name.
+    required: true
+    aliases: ['volume']
+  status_filter:
+    default: "self-heal"
+    choices: ["self-heal", "rebalance"]
+    description:
+      - Determines which facts are to be returned. If the status_filter is
+        self-heal, status of self-heal, along with the number of files still
+        in process are returned. If status_filter is rebalance,
+        rebalance status is returned.
+requirements:
+  - GlusterFS > 3.2
+'''
+
+EXAMPLES = '''
+# Gather facts about all gluster hosts in the cluster and return self-heal
+# status along with the number of files still in process:
+- gluster_heal_facts:
+    name: test_volume
+    status_filter: self-heal
+- debug:
+    var: gluster_hosts
+'''
+
+RETURN = '''
+name:
+    description: GlusterFS volume name
+    returned: always
+    type: str
+status_filter:
+    description: Whether self-heal or rebalance status is to be returned
+    returned: always
+    type: str
+heal_info:
+    description: List of files that still need healing process
+    returned: On success
+    type: list
+rebalance_status:
+    description: Status of rebalance operation
+    returned: On success
+    type: list
+'''
+
+import traceback
+
+from ansible.module_utils.basic import AnsibleModule
+from ansible.module_utils._text import to_native
+from distutils.version import LooseVersion
+
+glusterbin = ''
+
+
+def run_gluster(gargs, **kwargs):
+    global glusterbin
+    global module
+    args = [glusterbin, '--mode=script']
+    args.extend(gargs)
+    try:
+        rc, out, err = module.run_command(args, **kwargs)
+        if rc != 0:
+            module.fail_json(msg='error running gluster (%s) command (rc=%d): %s' %
+                             (' '.join(args), rc, out or err), exception=traceback.format_exc())
+    except Exception as e:
+        module.fail_json(msg='error running gluster (%s) command: %s' % (' '.join(args),
+                                                                         to_native(e)), exception=traceback.format_exc())
+    return out
+
+
+def get_self_heal_status(name):
+    out = run_gluster(['volume', 'heal', name, 'info'])
+    raw_out = out.split("\n")
+    heal_info = []
+    # return files that still need healing.
+    for line in raw_out:
+        if 'Brick' in line:
+            br_dict = {}
+            br_dict['brick'] = line.strip().strip("Brick")
+        elif 'Status' in line:
+            br_dict['status'] = line.split(":")[1].strip()
+        elif 'Number' in line:
+            br_dict['no_of_entries'] = line.split(":")[1].strip()
+            if int(br_dict['no_of_entries']) != 0:
+                heal_info.append(br_dict)
+            else:
+                continue
+    return heal_info
+
+
+def get_rebalance_status(name):
+    out = run_gluster(['volume', 'rebalance', name, 'status'])
+    raw_out = out.split("\n")
+    rebalance_status = []
+    # return the files that are either still 'in progress' state or 'completed'.
+    for line in raw_out:
+        node_dict = {}
+        line = " ".join(line.split())
+        line_vals = line.split(" ")
+        node_dict['node'] = line_vals[0]
+        node_dict['rebalanced_files'] = line_vals[1]
+        node_dict['failures'] = line_vals[4]
+        if 'in progress' in line:
+            node_dict['status'] = line_vals[5] + line_vals[6]
+            rebalance_status.append(node_dict)
+        elif 'completed' in line:
+            node_dict['status'] = line_vals[5]
+            rebalance_status.append(node_dict)
+    return rebalance_status
+
+
+def is_invalid_gluster_version(module, required_version):
+    cmd = module.get_bin_path('gluster', True) + ' --version'
+    result = module.run_command(cmd)
+    ver_line = result[1].split('\n')[0]
+    version = ver_line.split(' ')[1]
+    # If the installed version is less than 3.2, it is an invalid version
+    # return True
+    return LooseVersion(version) < LooseVersion(required_version)
+
+
+def main():
+    global module
+    global glusterbin
+    module = AnsibleModule(
+        argument_spec=dict(
+            name=dict(type='str', required=True, aliases=['volume']),
+            status_filter=dict(type='str', default='self-heal', choices=['self-heal', 'rebalance']),
+        ),
+    )
+
+    glusterbin = module.get_bin_path('gluster', True)
+    required_version = "3.2"
+    status_filter = module.params['status_filter']
+    volume_name = module.params['name']
+    heal_info = ''
+    rebalance_status = ''
+
+    # Verify if required GlusterFS version is installed
+    if is_invalid_gluster_version(module, required_version):
+        module.fail_json(msg="GlusterFS version > %s is required" %
+                         required_version)
+
+    try:
+        if status_filter == "self-heal":
+            heal_info = get_self_heal_status(volume_name)
+        elif status_filter == "rebalance":
+            rebalance_status = get_rebalance_status(volume_name)
+    except Exception:
+        module.fail_json(msg="Invalid heal status option.")
+
+    facts = {}
+    facts['glusterfs'] = {'volume': volume_name, 'status_filter': status_filter, 'heal_info': heal_info, 'rebalance': rebalance_status}
+
+    module.exit_json(ansible_facts=facts)
+
+
+if __name__ == '__main__':
+    main()


Code review comments:
diff_hunk: @@ -0,0 +1,183 @@
+#!/usr/bin/python
+# -*- coding: utf-8 -*-
+#
+# Copyright (c) 2016 Red Hat, Inc.
+# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
+
+from __future__ import absolute_import, division, print_function
+__metaclass__ = type
+
+ANSIBLE_METADATA = {'metadata_version': '1.1',
+                    'status': ['preview'],
+                    'supported_by': 'community'}
+
+
+DOCUMENTATION = '''
+---
+module: gluster_heal_facts
+short_description: Gather facts about self-heal or rebalance status
+author: "Devyani Kota (@devyanikota)"
+version_added: "2.8"
+description:
+  - Gather facts about either self-heal or rebalance status.
+options:
+  name:
+    description:
+      - The volume name.
+    required: true
+    aliases: ['volume']
+  status_filter:
+    default: "self-heal"
+    choices: ["self-heal", "rebalance"]
+    description:
+      - Determines which facts are to be returned. If the status_filter is
+        self-heal, status of self-heal, along with the number of files still
+        in process are returned. If status_filter is rebalance,
+        rebalance status is returned.
+requirements:
+  - GlusterFS > 3.2
+'''
+
+EXAMPLES = '''
+# Gather facts about all gluster hosts in the cluster and return self-heal
+# status along with the number of files still in process:
+- gluster_heal_facts:
+    name: test_volume
+    status_filter: self-heal
+- debug:
+    var: gluster_hosts
+'''
+
+RETURN = '''
+name:
+    description: GlusterFS volume name
+    returned: always
+    type: str
+status_filter:
+    description: Whether self-heal or rebalance status is to be returned
+    returned: always
+    type: str
+heal_info:
+    description: List of files that still need healing process
+    returned: On success
+    type: list
+rebalance_status:
+    description: Status of rebalance operation
+    returned: On success
+    type: list
+'''
+
+import traceback
+
+from ansible.module_utils.basic import AnsibleModule
+from ansible.module_utils._text import to_native
+from distutils.version import LooseVersion
+
+glusterbin = ''
+
+
+def run_gluster(gargs, **kwargs):
+    global glusterbin
+    global module
+    args = [glusterbin, '--mode=script']
+    args.extend(gargs)
+    try:
+        rc, out, err = module.run_command(args, **kwargs)
+        if rc != 0:
+            module.fail_json(msg='error running gluster (%s) command (rc=%d): %s' %
+                             (' '.join(args), rc, out or err), exception=traceback.format_exc())
+    except Exception as e:
+        module.fail_json(msg='error running gluster (%s) command: %s' % (' '.join(args),
+                                                                         to_native(e)), exception=traceback.format_exc())
+    return out
+
+
+def get_self_heal_status(name):
+    out = run_gluster(['volume', 'heal', name, 'info'])
path: lib/ansible/modules/storage/glusterfs/gluster_heal_facts.py
text: @user1:
```suggestion
    out = run_gluster(['volume', 'heal', name, 'info'], environ_update=dict(LANG='C', LC_ALL='C', LC_MESSAGES='C'))
```

diff_hunk: @@ -0,0 +1,183 @@
+#!/usr/bin/python
+# -*- coding: utf-8 -*-
+#
+# Copyright (c) 2016 Red Hat, Inc.
path: lib/ansible/modules/storage/glusterfs/gluster_heal_facts.py
text: @user1:
```suggestion
# Copyright: (c) 2016, Red Hat, Inc.
```

diff_hunk: @@ -0,0 +1,183 @@
+#!/usr/bin/python
+# -*- coding: utf-8 -*-
+#
+# Copyright (c) 2016 Red Hat, Inc.
+# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
+
+from __future__ import absolute_import, division, print_function
+__metaclass__ = type
+
+ANSIBLE_METADATA = {'metadata_version': '1.1',
+                    'status': ['preview'],
+                    'supported_by': 'community'}
+
+
+DOCUMENTATION = '''
+---
+module: gluster_heal_facts
+short_description: Gather facts about self-heal or rebalance status
+author: "Devyani Kota (@devyanikota)"
+version_added: "2.8"
+description:
+  - Gather facts about either self-heal or rebalance status.
+options:
+  name:
+    description:
+      - The volume name.
+    required: true
+    aliases: ['volume']
+  status_filter:
+    default: "self-heal"
+    choices: ["self-heal", "rebalance"]
+    description:
+      - Determines which facts are to be returned. If the status_filter is
+        self-heal, status of self-heal, along with the number of files still
+        in process are returned. If status_filter is rebalance,
+        rebalance status is returned.
+requirements:
+  - GlusterFS > 3.2
+'''
+
+EXAMPLES = '''
+# Gather facts about all gluster hosts in the cluster and return self-heal
path: lib/ansible/modules/storage/glusterfs/gluster_heal_facts.py
text: @user1:
```suggestion
- name: Gather self-heal facts about all gluster hosts in the cluster
  gluster_heal_facts:
    name: test_volume
    status_filter: self-heal
  register: self_heal_status
- debug:
    var: self_heal_status
```

diff_hunk: @@ -0,0 +1,183 @@
+#!/usr/bin/python
+# -*- coding: utf-8 -*-
+#
+# Copyright (c) 2016 Red Hat, Inc.
+# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
+
+from __future__ import absolute_import, division, print_function
+__metaclass__ = type
+
+ANSIBLE_METADATA = {'metadata_version': '1.1',
+                    'status': ['preview'],
+                    'supported_by': 'community'}
+
+
+DOCUMENTATION = '''
+---
+module: gluster_heal_facts
+short_description: Gather facts about self-heal or rebalance status
+author: "Devyani Kota (@devyanikota)"
+version_added: "2.8"
+description:
+  - Gather facts about either self-heal or rebalance status.
+options:
+  name:
+    description:
+      - The volume name.
+    required: true
+    aliases: ['volume']
+  status_filter:
+    default: "self-heal"
+    choices: ["self-heal", "rebalance"]
+    description:
+      - Determines which facts are to be returned. If the status_filter is
path: lib/ansible/modules/storage/glusterfs/gluster_heal_facts.py
text: @user1:
```suggestion
      - Determines which facts are to be returned.
      - If the C(status_filter) is C(self-heal), status of self-heal, along with the number of files still in process are returned.
      - If the C(status_filter) is C(rebalance), rebalance status is returned.
```