Repository: ansible/ansible

Issue:
Ansible Facts report wrong PVS free size
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and master branch are affected too.
Checked
-->

##### ISSUE TYPE
<!--- Pick one below and delete the rest: -->
 - Bug Report

##### COMPONENT NAME
<!--- Name of the module/plugin/task/feature -->
<!--- Please do not include extra details here, e.g. "vyos_command" not "the network module vyos_command"

setup

##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes below -->
```
ansible 2.4.2.0
  config file = /home/ucm/AWX/common/ansible.cfg
  configured module search path = [u'/home/ucm/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /home/ucm/virtual/lib/python2.7/site-packages/ansible
  executable location = /home/ucm/virtual/bin/ansible
  python version = 2.7.5 (default, May  3 2017, 07:55:04) [GCC 4.8.5 20150623 (Red Hat 4.8.5-14)]

```

##### CONFIGURATION
<!---
If using Ansible 2.4 or above, paste the results of "ansible-config dump --only-changed"

Otherwise, mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).

-->
```
ANSIBLE_NOCOWS(/home/ucm/AWX/common/ansible.cfg) = True
ANSIBLE_SSH_ARGS(/home/ucm/AWX/common/ansible.cfg) = -o StrictHostKeyChecking=no -o ControlMaster=auto -o ControlPersist=60s
CACHE_PLUGIN(/home/ucm/AWX/common/ansible.cfg) = jsonfile
CACHE_PLUGIN_CONNECTION(/home/ucm/AWX/common/ansible.cfg) = /home/ucm/AWX/facts_cache
CACHE_PLUGIN_TIMEOUT(/home/ucm/AWX/common/ansible.cfg) = 3600
DEFAULT_BECOME(/home/ucm/AWX/common/ansible.cfg) = True
DEFAULT_BECOME_METHOD(/home/ucm/AWX/common/ansible.cfg) = sudo
DEFAULT_BECOME_USER(/home/ucm/AWX/common/ansible.cfg) = root
DEFAULT_FORKS(/home/ucm/AWX/common/ansible.cfg) = 5
DEFAULT_GATHERING(/home/ucm/AWX/common/ansible.cfg) = smart
DEFAULT_HOST_LIST(/home/ucm/AWX/common/ansible.cfg) = [u'/home/ucm/AWX/inventory']
DEFAULT_LOG_PATH(/home/ucm/AWX/common/ansible.cfg) = /tmp/ansible/ansible.log
DEFAULT_PRIVATE_KEY_FILE(/home/ucm/AWX/common/ansible.cfg) = /home/ucm/.ssh/id_rsa
DEFAULT_REMOTE_USER(/home/ucm/AWX/common/ansible.cfg) = ucm
DEFAULT_ROLES_PATH(/home/ucm/AWX/common/ansible.cfg) = [u'/home/ucm/AWX/common/roles']
DEFAULT_SCP_IF_SSH(/home/ucm/AWX/common/ansible.cfg) = True
DEFAULT_STDOUT_CALLBACK(/home/ucm/AWX/common/ansible.cfg) = skippy
DEFAULT_SUDO_USER(/home/ucm/AWX/common/ansible.cfg) = root
DEFAULT_VAULT_PASSWORD_FILE(/home/ucm/AWX/common/ansible.cfg) = /home/ucm/.vault_pass.txt
DEPRECATION_WARNINGS(/home/ucm/AWX/common/ansible.cfg) = False
RETRY_FILES_SAVE_PATH(/home/ucm/AWX/common/ansible.cfg) = /tmp/ansible
```

##### OS / ENVIRONMENT
<!---
Mention the OS you are running Ansible from, and the OS you are
managing, or say "N/A" for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.
-->
RedHat Enterprise Linux 7
##### SUMMARY
<!--- Explain the problem briefly -->
While gathering info about available 'disk' space on lvm's i noticed a lot of them reporting 51(gb) free while this couldnt be.
##### STEPS TO REPRODUCE
Reproducable always (from RH6 to latest RH7) all machines report the same basically.
<!---
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used.
-->

<!--- Paste example playbooks or commands between quotes below -->
ansible -m setup all --limit=XXX -a 'filter=ansible_lvm'
```yaml
 {
    "ansible_facts": {
        "ansible_lvm": {
            "lvs": {
                "lvdata": {
                    "size_g": "45",
                    "vg": "XX849vg01"
                },
                "lvdocker": {
                    "size_g": "9",
                    "vg": "rootvg"
                },
                "lvhome": {
                    "size_g": "2",
                    "vg": "rootvg"
                },
                "lvroot": {
                    "size_g": "4",
                    "vg": "rootvg"
                },
                "lvswap": {
                    "size_g": "8",
                    "vg": "swapvg"
                },
                "lvtmp": {
                    "size_g": "2",
                    "vg": "rootvg"
                },
                "lvvar": {
                    "size_g": "2",
                    "vg": "rootvg"
                }
            },
            "pvs": {
                "/dev/sda2": {
                    "free_g": "51",
                    "size_g": "19",
                    "vg": "rootvg"
                },
                "/dev/sdb1": {
                    "free_g": "00",
                    "size_g": "8",
                    "vg": "swapvg"
                },
                "/dev/sdc1": {
                    "free_g": "00",
                    "size_g": "100",
                    "vg": "XX849vg01"
                }
            },
            "vgs": {
                "XX849vg01": {
                    "free_g": "00",
                    "num_lvs": "1",
                    "num_pvs": "1",
                    "size_g": "55"
                },
                "rootvg": {
                    "free_g": "0",
                    "num_lvs": "5",
                    "num_pvs": "1",
                    "size_g": "51"
                },
                "swapvg": {
                    "free_g": "0",
                    "num_lvs": "1",
                    "num_pvs": "1",
                    "size_g": "00"
                }
            }
        }
    },
    "changed": false
}


```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
at ansible_lvm['pvs']['/dev/sda2']['free_g'] it reports 51. It should be

```
]$ sudo pvs
  PV         VG        Fmt  Attr PSize    PFree
  /dev/sda2  rootvg    lvm2 a--   <19,51g <1,51g
  /dev/sdb1  swapvg    lvm2 a--    <8,00g      0
  /dev/sdc1  XX849vg01 lvm2 a--  <100,00g <55,00g
```
at ansible_lvm['vgs']['XX849vg01']['free_g'] it reports 0 It should be
```
 sudo vgs
  VG        #PV #LV #SN Attr   VSize    VFree
  XX849vg01   1   1   0 wz--n- <100,00g <55,00g
  rootvg      1   5   0 wz--n-  <19,51g      0
  swapvg      1   1   0 wz--n-   <8,00g      0
```

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->

<!--- Paste verbatim command output between quotes below -->

As mentioned above, it reports whatever it finds behind the ','. 55,00 (vg) becomes 00. The 1.51g becomes 51g in case of the pv.
```

```



Discussion:
This is caused by the fact that the decimal separator of configured locale is a comma `,`. What is the output of the `locale` command on the target host? I am able to reproduce this by setting the locale to `nl_NL.UTF-8` (Netherlands uses comma as the separator)

```
LANG=nl_NL.UTF-8 pvs --noheadings --nosuffix --units g --separator , /dev/sdb1
  /dev/sda2,centos,lvm2,a--,7,00,1                                                                     
LANG=en_US.UTF-8 pvs --noheadings --nosuffix --units g --separator , /dev/sdb1     
  /dev/sda2,centos,lvm2,a--,7.00,1
```
Module `linux.py` has `,` hardcoded as the separator
```
lvm_util_options = '--noheadings --nosuffix --units g --separator ,'
```
I'd like to fix this issue, any feedback from someone who is more involved in the project will be very useful. One possible solution is to use a non-ambiguous separator (`|`, `+` or similar) in `lvm_util_options`, or not specify a separator at all. `pvs` output is separated by a space by default, so the following code should work with `nl_NL.UTF-8` or any other locale that uses a comma as the decimal separator

```python
# ... SNIP ...
lvm_util_options = '--noheadings --nosuffix --units g'
rc, pv_lines, err = self.module.run_command('%s %s' % (pvs_path, lvm_util_options))
    for pv_line in pv_lines.splitlines():
        items = pv_line.strip().split()
# ... SNIP ...
```
output
```json
"pvs": {
    "/dev/sdb1": {
    "free_g": "1,00",
    "size_g": "2,00",
    "vg": "ivicavg"
    },
    "/dev/sdc1": {
    "free_g": "3,00",
    "size_g": "3,00",
    "vg": "ivicavg"
    }
}
```
As requested
```
 ~]# locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC=nl_NL.UTF-8
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY=nl_NL.UTF-8
LC_MESSAGES="en_US.UTF-8"
LC_PAPER=nl_NL.UTF-8
LC_NAME=nl_NL.UTF-8
LC_ADDRESS=nl_NL.UTF-8
LC_TELEPHONE=nl_NL.UTF-8
LC_MEASUREMENT=nl_NL.UTF-8
LC_IDENTIFICATION=nl_NL.UTF-8
LC_ALL=
```
You are indeed correct, thanks you for the research. I'll try to see if i can 'force' the LC_NUMERIC as a work around.
```
[root@XX483 ~]# export LC_NUMERIC=en_US.UTF-8
[root@XX483 ~]# pvs
  PV         VG        Fmt  Attr PSize   PFree
  /dev/sda2  rootvg    lvm2 a--  <19.00g      0
  /dev/sdb1  swapvg    lvm2 a--   <8.00g      0
  /dev/sdc1  XX483vg01 lvm2 a--  <70.00g <25.00g
```

adding "export LC_NUMERIC=en_US.UTF-8" in ansible user's bashrc works
```
          "pvs": {
                "/dev/sda2": {
                    "free_g": "0",
                    "size_g": "19.00",
                    "vg": "rootvg"
                },
                "/dev/sdb1": {
                    "free_g": "0",
                    "size_g": "8.00",
                    "vg": "swapvg"
                },
                "/dev/sdc1": {
                    "free_g": "25.00",
                    "size_g": "70.00",
                    "vg": "XX483vg01"
                }
            },
```
@Riccardo1976: could you reopen this issue? I don't think it should be considered fixed, just because there is a workaround. Ansible should work correctly for users that have a non-english locale.
As requested by @schweikert , issue reopened. 
I'm happy using my work around. Found that this happened during installation by selecting Country/City it seems that it then also includes all locale(s).

I also agree with the re-open request to have it corrected in the Ansible code.
@ivica-k Would you be interested in submitting a PR for this ? This can be solved by adding following line in `populate` function in `hardware/linux.py` just before starting fact gathering - 

```
self.module.run_command_environ_update = {'LANG': 'C', 'LC_ALL': 'C', 'LC_NUMERIC': 'C'}
```

This line will modify `run_command` environment variables and screen scrapping will be easy. 

Let me know if you need any help.

Thanks,

Resolving patch:
diff --git a/lib/ansible/module_utils/facts/hardware/linux.py b/lib/ansible/module_utils/facts/hardware/linux.py
--- a/lib/ansible/module_utils/facts/hardware/linux.py
+++ b/lib/ansible/module_utils/facts/hardware/linux.py
@@ -31,7 +31,7 @@
 from ansible.module_utils.facts.hardware.base import Hardware, HardwareCollector
 from ansible.module_utils.facts.utils import get_file_content, get_file_lines, get_mount_size
 
-# import this as a module to ensure we get the same module isntance
+# import this as a module to ensure we get the same module instance
 from ansible.module_utils.facts import timeout
 
 
@@ -78,6 +78,7 @@ class LinuxHardware(Hardware):
 
     def populate(self, collected_facts=None):
         hardware_facts = {}
+        self.module.run_command_environ_update = {'LANG': 'C', 'LC_ALL': 'C', 'LC_NUMERIC': 'C'}
 
         cpu_facts = self.get_cpu_facts(collected_facts=collected_facts)
         memory_facts = self.get_memory_facts()


Test patch: