Repository: saltstack/salt
Pull request: Fix Python 2 syntax used in status commands, notably status.netdev
### What does this PR do?
Fixes Python 2 syntax that was used and had not been converted to Python 3.
In Python 2 a dictionary.keys() function returns a list which could be joined with another list using '+'.
In Python 3 a distionary.keys() function returns a view object which cannot be added to another view object with a '+', hence using itertools.chain functionality to iterate over two view objects.

### What issues does this PR fix or reference?
Fixes: https://github.com/saltstack/salt/issues/60909

### Previous Behavior
Would cause TypeError due to unsupported behavior
```
  File "/opt/saltstack/salt/run/salt/modules/status.py", line 1413, in aix_netdev
    __grains__["ip4_interfaces"].keys() + __grains__["ip6_interfaces"].keys()
TypeError: unsupported operand type(s) for +: 'dict_keys' and 'dict_keys'
```

### New Behavior
Using itertools.chain functionality to iterate over two view objects returned from the dictionary keys() function

### Merge requirements satisfied?
**[NOTICE] Bug fixes or features added to Salt require tests.**
<!-- Please review the [test documentation](https://docs.saltproject.io/en/master/topics/tutorials/writing_tests.html) for details on how to implement tests into Salt's test suite. -->
- [ ] Docs
- [X] Changelog - https://docs.saltproject.io/en/master/topics/development/changelog.html
- [X] Tests written/updated

### Commits signed with GPG?
Yes

Please review [Salt's Contributing Guide](https://docs.saltproject.io/en/master/topics/development/contributing.html) for best practices.

See GitHub's [page on GPG signing](https://help.github.com/articles/signing-commits-using-gpg/) for more information about signing commits with GPG.


Problem statement:
[BUG] status.netdev is no longer working on AIX
**Description**
The execution module command is no longer working in Python 3 versions of AIX salt-minion, getting dict handling errors



**Setup**
Run the command
```
salt <minion> status.netdev
```

Please be as specific as possible and give set-up details.

- [ ] on-prem machine
- [ ] VM (Virtualbox, KVM, etc. please specify)
- [ ] VM running on a cloud service, please be explicit and add details
- [ ] container (Kubernetes, Docker, containerd, etc. please specify)
- [ ] or a combination, please be explicit
- [ ] jails if it is FreeBSD

  
**Steps to Reproduce the behavior**
```
root@p1243-pvm2:/home/cecuser/test_3003.3/salt_3003.3$ salt-call --local status.netdev 

Passed invalid arguments: unsupported operand type(s) for +: 'dict_keys' and 'dict_keys'.

Usage:

    .. versionchanged:: 2016.3.2
        Return the network device stats for this minion

    .. versionchanged:: 2016.11.4
        Added support for AIX

    CLI Example:

    .. code-block:: bash

        salt '*' status.netdev
```
    

**Expected behavior**
Should return information on netdev

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Versions Report**
<details><summary>salt --versions-report</summary>
```
root@p1243-pvm2:/home/cecuser/test_3003.3/salt_3003.3$ salt-call --local test.versions
local:
    Salt Version:
              Salt: 3003.3
     
    Dependency Versions:
              cffi: 1.14.4
          cherrypy: 17.4.1
          dateutil: 2.8.1
         docker-py: Not Installed
             gitdb: Not Installed
         gitpython: Not Installed
            Jinja2: 2.11.3
           libgit2: Not Installed
          M2Crypto: Not Installed
              Mako: Not Installed
           msgpack: 1.0.2
      msgpack-pure: Not Installed
      mysql-python: Not Installed
         pycparser: 2.17
          pycrypto: 3.10.1
      pycryptodome: 3.9.8
            pygit2: Not Installed
            Python: 3.7.10 (default, Jun  1 2021, 05:23:20)
      python-gnupg: 0.4.4
            PyYAML: 5.4.1
             PyZMQ: 18.1.1
             smmap: Not Installed
           timelib: 0.2.4
           Tornado: 4.5.3
               ZMQ: 4.3.3
     
    System Versions:
              dist: aix 1 
            locale: ISO8859-1
           machine: 00F9C4904C00
           release: 1
            system: AIX
           version: AIX 1 
```
</details>

**Additional context**
This worked in 2016.11.x versions.
root@p1243-pvm2:/home/cecuser/test_2016.11/salt_2016.11.3$ salt-call --local status.netdev
[WARNING ] The tools 'dmidecode' and 'lspci' failed to execute because they do not exist on the system of the user running this instance or the user does not have the necessary permissions to execute them. Grains output might not be accurate.
local:
    This method is unsupported on the current operating system!

Need to find 2016.11.4 to test with

Hints from issue discussion:
Here is working with 2017.7.8
```root@p1243-pvm2:/home/cecuser/test_2017.7/salt_2017.7.8# salt-call --local status.netdev
local:
    ----------
    en0:
        |_
          ----------
          ipv4:
              ----------
              Address:
                  129.40.60.162
              Coll:
                  0
              Ierrs:
                  0
              Ipkts:
                  2171444
              Mtu:
                  1500
              Network:
                  129.40.60.1
              Oerrs:
                  0
              Opkts:
                  1640313
    en1:
        |_
          ----------
          ipv4:
              ----------
              Address:
                  172.24.60.162
              Coll:
                  0
              Ierrs:
                  0
              Ipkts:
                  13775
              Mtu:
                  1500
              Network:
                  172.24.60.1
              Oerrs:
                  0
              Opkts:
                  5227
    lo0:
        |_
          ----------
          ipv4:
              ----------
              Address:
                  127.0.0.1
              Coll:
                  0
              Ierrs:
                  0
              Ipkts:
                  473364
              Mtu:
                  16896
              Network:
                  127
              Oerrs:
                  0
              Opkts:
                  473364
        |_
          ----------
          ipv6:
              ----------
              Address:
                  473364
              Ierrs:
                  473364
              Ipkts:
                  0
              Mtu:
                  16896
              Network:
                  ::1%1
              Oerrs:
                  0
              Opkts:
                  0
root@p1243-pvm2:/home/cecuser/test_2017.7/salt_2017.7.8# 
```
Note the old Python 2 based Salt 3000.4 status.netdev fails due to IPv6 handling failing, and the new Python 3 based Salt 3004 fails similarly
```
[DEBUG   ] LazyLoaded direct_call.execute

Passed invalid arguments: unsupported operand type(s) for +: 'dict_keys' and 'dict_keys'.

Usage:
Traceback (most recent call last):
  File "salt/cli/caller.py", line 219, in call
    self.opts, data, func, args, kwargs
  File "salt/loader/lazy.py", line 149, in __call__
    return self.loader.run(run_func, *args, **kwargs)
  File "salt/loader/lazy.py", line 1201, in run
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
  File "salt/loader/lazy.py", line 1216, in _run_as
    return _func_or_method(*args, **kwargs)
  File "/opt/saltstack/salt/run/salt/executors/direct_call.py", line 10, in execute
    return func(*args, **kwargs)
  File "salt/loader/lazy.py", line 149, in __call__
    return self.loader.run(run_func, *args, **kwargs)
  File "salt/loader/lazy.py", line 1201, in run
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
  File "salt/loader/lazy.py", line 1216, in _run_as
    return _func_or_method(*args, **kwargs)
  File "/opt/saltstack/salt/run/salt/modules/status.py", line 1492, in netdev
    return get_version.get(__grains__["kernel"], lambda: errmsg)()
  File "/opt/saltstack/salt/run/salt/modules/status.py", line 1413, in aix_netdev
    __grains__["ip4_interfaces"].keys() + __grains__["ip6_interfaces"].keys()
TypeError: unsupported operand type(s) for +: 'dict_keys' and 'dict_keys'
```
Problem is due to using Python 2 construct in Python 3, see https://github.com/saltstack/salt/blob/9b1b791d212a6810c430dd15c63fbce3a4f7e1d6/salt/modules/status.py#L1413-L1415

use of keys in Python 3 returns a view object containing a list of keys and not a list of keys as in python 2.
Cannot add two view objects together with '**+**' as you could with a list in Python 2.
Also have the same problem in function sunos_netdev

Need to ensure pre-commit catches this usage in future

Commit message: Added skip test if Python 3.5 or less (Debian 9), since OrderedDict is not default

Patch to review:
diff --git a/changelog/60909.fixed b/changelog/60909.fixed
new file mode 100644
index 000000000000..c3e86b78560c
--- /dev/null
+++ b/changelog/60909.fixed
@@ -0,0 +1 @@
+Fixed Python 2 syntax for Python 3, allow for view objects returned by dictionary keys() function
diff --git a/salt/modules/status.py b/salt/modules/status.py
index 729acbe0ecee..7ca468b966d8 100644
--- a/salt/modules/status.py
+++ b/salt/modules/status.py
@@ -7,6 +7,7 @@
 import copy
 import datetime
 import fnmatch
+import itertools
 import logging
 import os
 import re
@@ -1369,7 +1370,9 @@ def sunos_netdev():
         """
         ret = {}
         ##NOTE: we cannot use hwaddr_interfaces here, so we grab both ip4 and ip6
-        for dev in __grains__["ip4_interfaces"].keys() + __grains__["ip6_interfaces"]:
+        for dev in itertools.chain(
+            __grains__["ip4_interfaces"].keys(), __grains__["ip6_interfaces"].keys()
+        ):
             # fetch device info
             netstat_ipv4 = __salt__["cmd.run"](
                 "netstat -i -I {dev} -n -f inet".format(dev=dev)
@@ -1410,15 +1413,15 @@ def aix_netdev():
         ret = {}
         fields = []
         procn = None
-        for dev in (
-            __grains__["ip4_interfaces"].keys() + __grains__["ip6_interfaces"].keys()
+        for dev in itertools.chain(
+            __grains__["ip4_interfaces"].keys(), __grains__["ip6_interfaces"].keys()
         ):
             # fetch device info
-            # root@la68pp002_pub:/opt/salt/lib/python2.7/site-packages/salt/modules# netstat -i -n -I en0 -f inet6
+            # root@la68pp002_pub:# netstat -i -n -I en0 -f inet
             # Name  Mtu   Network     Address            Ipkts Ierrs    Opkts Oerrs  Coll
             # en0   1500  link#3      e2.eb.32.42.84.c 10029668     0   446490     0     0
             # en0   1500  172.29.128  172.29.149.95    10029668     0   446490     0     0
-            # root@la68pp002_pub:/opt/salt/lib/python2.7/site-packages/salt/modules# netstat -i -n -I en0 -f inet6
+            # root@la68pp002_pub:# netstat -i -n -I en0 -f inet6
             # Name  Mtu   Network     Address            Ipkts Ierrs    Opkts Oerrs  Coll
             # en0   1500  link#3      e2.eb.32.42.84.c 10029731     0   446499     0     0
 
diff --git a/tests/pytests/unit/modules/test_aix_status.py b/tests/pytests/unit/modules/test_aix_status.py
new file mode 100644
index 000000000000..2fc144b35325
--- /dev/null
+++ b/tests/pytests/unit/modules/test_aix_status.py
@@ -0,0 +1,175 @@
+import logging
+import sys
+
+import salt.modules.status as status
+from tests.support.mock import MagicMock, patch
+
+try:
+    import pytest
+except ImportError:
+    pytest = None
+
+log = logging.getLogger(__name__)
+
+
+@pytest.fixture
+def configure_loader_modules():
+
+    return {
+        status: {
+            "__grains__": {
+                "ip4_interfaces": {
+                    "en0": ["129.40.94.58"],
+                    "en1": ["172.24.94.58"],
+                    "lo0": ["127.0.0.1"],
+                },
+                "ip6_interfaces": {"en0": [], "en1": [], "lo0": ["1"]},
+                "kernel": "AIX",
+                "osarch": "PowerPC_POWER8",
+                "os": "AIX",
+                "os_family": "AIX",
+                "osmajorrelease": 7,
+            },
+        },
+    }
+
+
+@pytest.mark.skipif(
+    sys.version_info[0] == 3 and sys.version_info[1] <= 5,
+    reason="run on Python 3.6 or greater where OrderedDict is default",
+)
+def test_netdev():
+    """
+    Test status.netdev for AIX
+
+    :return:
+    """
+    # Output from netstat -i -n -I <en0|en1|lo0> -f inet
+    netstat_inet4_en0 = """Name   Mtu   Network     Address                 Ipkts     Ierrs        Opkts     Oerrs  Coll
+en0    1500  link#2      fa.41.f5.e9.bd.20  1523125     0   759364     0     0
+en0    1500  129.40.94.5 129.40.94.58      1523125     0   759364     0     0
+"""
+
+    netstat_inet4_en1 = """Name   Mtu   Network     Address                 Ipkts     Ierrs        Opkts     Oerrs  Coll
+en1    1500  link#3      fa.41.f5.e9.bd.21     1089     0      402     0     0
+en1    1500  172.24.94.5 172.24.94.58         1089     0      402     0     0
+"""
+
+    netstat_inet4_lo0 = """Name   Mtu   Network     Address                 Ipkts     Ierrs        Opkts     Oerrs  Coll
+lo0    16896 link#1                          25568     0    25568     0     0
+lo0    16896 127         127.0.0.1           25568     0    25568     0     0
+"""
+
+    # Output from netstat -i -n -I <en0|en1|lo0> -f inet6
+    netstat_inet6_en0 = """Name   Mtu   Network     Address                 Ipkts     Ierrs        Opkts     Oerrs  Coll
+en0    1500  link#2      fa.41.f5.e9.bd.20  1523160     0   759397     0     0
+"""
+
+    netstat_inet6_en1 = """Name   Mtu   Network     Address                 Ipkts     Ierrs        Opkts     Oerrs  Coll
+en1    1500  link#3      fa.41.f5.e9.bd.21     1089     0      402     0     0
+"""
+
+    netstat_inet6_lo0 = """Name   Mtu   Network     Address                 Ipkts     Ierrs        Opkts     Oerrs  Coll
+lo0    16896 link#1                          25611     0    25611     0     0
+lo0    16896 ::1%1                           25611     0    25611     0     0
+"""
+
+    # allow en0, en1 and lo0 for ipv4 and ipv6
+    netstats_out = MagicMock(
+        side_effect=[
+            netstat_inet4_en0,
+            netstat_inet6_en0,
+            netstat_inet4_en1,
+            netstat_inet6_en1,
+            netstat_inet4_lo0,
+            netstat_inet6_lo0,
+            netstat_inet4_en0,
+            netstat_inet6_en0,
+            netstat_inet4_en1,
+            netstat_inet6_en1,
+            netstat_inet4_lo0,
+            netstat_inet6_lo0,
+        ]
+    )
+
+    with patch.dict(
+        status.__grains__,
+        {
+            "osarch": "PowerPC_POWER8",
+            "ip4_interfaces": {
+                "en0": ["129.40.94.58"],
+                "en1": ["172.24.94.58"],
+                "lo0": ["127.0.0.1"],
+            },
+            "ip6_interfaces": {
+                "en0": [],
+                "en1": [],
+                "lo0": ["::1"],
+            },
+            "kernel": "AIX",
+        },
+    ), patch.dict(status.__salt__, {"cmd.run": netstats_out}):
+        netdev_out = status.netdev()
+        assert netstats_out.call_count == 12
+        netstats_out.assert_any_call("netstat -i -n -I en0 -f inet")
+        netstats_out.assert_any_call("netstat -i -n -I en1 -f inet")
+        netstats_out.assert_any_call("netstat -i -n -I lo0 -f inet")
+        netstats_out.assert_any_call("netstat -i -n -I en0 -f inet6")
+        netstats_out.assert_any_call("netstat -i -n -I en1 -f inet6")
+        netstats_out.assert_any_call("netstat -i -n -I lo0 -f inet6")
+        expected = {
+            "en0": [
+                {
+                    "ipv4": {
+                        "Mtu": "1500",
+                        "Network": "129.40.94.5",
+                        "Address": "129.40.94.58",
+                        "Ipkts": "1523125",
+                        "Ierrs": "0",
+                        "Opkts": "759364",
+                        "Oerrs": "0",
+                        "Coll": "0",
+                    }
+                }
+            ],
+            "en1": [
+                {
+                    "ipv4": {
+                        "Mtu": "1500",
+                        "Network": "172.24.94.5",
+                        "Address": "172.24.94.58",
+                        "Ipkts": "1089",
+                        "Ierrs": "0",
+                        "Opkts": "402",
+                        "Oerrs": "0",
+                        "Coll": "0",
+                    }
+                }
+            ],
+            "lo0": [
+                {
+                    "ipv4": {
+                        "Mtu": "16896",
+                        "Network": "127",
+                        "Address": "127.0.0.1",
+                        "Ipkts": "25568",
+                        "Ierrs": "0",
+                        "Opkts": "25568",
+                        "Oerrs": "0",
+                        "Coll": "0",
+                    }
+                },
+                {
+                    "ipv6": {
+                        "Mtu": "16896",
+                        "Network": "::1%1",
+                        "Address": "25611",
+                        "Ipkts": "0",
+                        "Ierrs": "25611",
+                        "Opkts": "0",
+                        "Oerrs": "0",
+                    }
+                },
+            ],
+        }
+        assert netdev_out == expected


Code review comments:
diff_hunk: @@ -0,0 +1,175 @@
+import logging
+import sys
+
+import salt.modules.status as status
+from tests.support.mock import MagicMock, patch
+
+try:
path: tests/pytests/unit/modules/test_aix_status.py
text: @user1:
Why would pytest not be available here?  And what happens when it's not?

@author:
Removed try/except around import pytest