Repository: RedHatInsights/insights-core

Issue:
Issue in insights_commands/ethtool_-k_* and other ethtool spec
It seems there is a bug in spec creation for `insights_commands/ethtool_-k_*` and other related specs related to **ethtool** command when a **VLAN connection** configured on the host. The spec which is currently created is `ethtool_-i_bond0.123_bond0` which should be actually `ethtool_-i_bond0.123` and due to which we are getting `Cannot get device feature names: No such device` when we open `insights_commands/ethtool_-k_ethtool_-k_bond0.123_bond0` logically this device does not exist on this host. It looks that the interface name is referred from `/sys/class/net` and then referred to ethtool command, but for VLAN interface seems it fails to refer the correct interface name. 

Below is an example for reference.

```
[root@rhel7-box vagrant]# ls insights-rhel7-box-20190403111343/sys/class/net/
bond0/     bond0.123/ enp0s3/    enp0s8/    enp0s9/    lo/
``` 

```
[root@rhel7-box vagrant]# ls insights-rhel7-box-20190403111343/insights_commands/ethtool_-k_
ethtool_-k_bond0            ethtool_-k_bond0.123_bond0  ethtool_-k_enp0s3           ethtool_-k_enp0s8           ethtool_-k_enp0s9
```


Discussion:
@gravitypriest, I think this is a client issue, let us know if that's not correct.
It sort of is, I remember seeing emails about this issue before.

The pre-command we use for getting the devices for the ethtool commands is `/sbin/ip -o link | awk -F ': ' '/.*link\\/ether/ {print $2}`, but it looks like it's not providing the expected device name in this case. Perhaps we should consider altering the command in uploader.json.
@gravitypriest I'm not sure if this is related, but for a while we've been seeing exceptions like

```
insights.parsers.ParseException: ('ethtool: bad command line argument for ethtool', ['ethtool: bad command line argument(s)', 'For more information run ethtool -h'])
```
and
```
insights.parsers.ParseException: bad line: For more information run ethtool -h
```

in about ~90% of archives.
@csams any logged examples of invalid commands getting made?

Resolving patch:
diff --git a/insights/specs/datasources/ethernet.py b/insights/specs/datasources/ethernet.py
new file mode 100644
--- /dev/null
+++ b/insights/specs/datasources/ethernet.py
@@ -0,0 +1,57 @@
+"""
+Custom datasource for gathering a list of the ethernet interface names.
+"""
+from insights.core.context import HostContext
+from insights.core.dr import SkipComponent
+from insights.core.plugins import datasource
+from insights.core.spec_factory import simple_command
+from insights.specs import Specs
+
+
+class LocalSpecs(Specs):
+    """ Local specs used only by ethernet_interfaces datasource. """
+    ip_link = simple_command("/sbin/ip -o link")
+
+
+@datasource(LocalSpecs.ip_link, HostContext)
+def interfaces(broker):
+    """
+    This datasource provides a list of the ethernet interfaces available.
+
+    Typical content of the spec is::
+
+        1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000\\    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
+        2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000\\    link/ether 52:54:00:13:14:b5 brd ff:ff:ff:ff:ff:ff
+        3: enp8s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000\\    link/ether 52:54:00:e5:11:d4 brd ff:ff:ff:ff:ff:ff
+        4: enp1s0.2@enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000\\    link/ether 52:54:00:13:14:b5 brd ff:ff:ff:ff:ff:ff
+        5: ib0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 4092 qdisc mq state DOWN group default qlen 256\\    link/infiniband 00:01:02:03:fd:90:0:00:00:00:00:00:ef:0d:8b:02:01:d9:82:fd
+
+    Note:
+        This datasource may be executed using the following command:
+
+        ``insights cat --no-header ethernet_interfaces``
+
+    Sample data returned::
+
+        ['enp1s0', 'enp8s0', 'enp1s0.2']
+
+    Returns:
+        list: List of the ethernet interfaces available.
+
+    Raises:
+        SkipComponent: When there is not any content.
+    """
+    content = broker[LocalSpecs.ip_link].content
+    if content:
+        ifaces = []
+        for x in content:
+            # Only process lines that have link/ether, this should exclude non ethernet devices.
+            if "link/ether" in x:
+                # Split first on : the interface name should be the second entry.
+                # Then split again on @ since vlans append @ and the parent interface name to the end.
+                ifaces.append(x.split(':')[1].split('@')[0].strip())
+
+        if ifaces:
+            return sorted(ifaces)
+
+    raise SkipComponent
diff --git a/insights/specs/default.py b/insights/specs/default.py
--- a/insights/specs/default.py
+++ b/insights/specs/default.py
@@ -38,7 +38,7 @@
 from insights.parsers.mount import Mount
 from insights.specs import Specs
 from insights.specs.datasources import (
-    cloud_init, candlepin_broker, get_running_commands, ipcs, package_provides, ps as ps_datasource)
+    cloud_init, candlepin_broker, ethernet, get_running_commands, ipcs, package_provides, ps as ps_datasource)
 
 
 logger = logging.getLogger(__name__)
@@ -244,13 +244,13 @@ def du_dirs_list(broker):
                                        "/usr/lib/udev/rules.d/40-redhat.rules", "/usr/local/lib/udev/rules.d/40-redhat.rules"])
     etcd_conf = simple_file("/etc/etcd/etcd.conf")
     ethernet_interfaces = listdir("/sys/class/net", context=HostContext)
-    ethtool = foreach_execute(ethernet_interfaces, "/sbin/ethtool %s")
-    ethtool_S = foreach_execute(ethernet_interfaces, "/sbin/ethtool -S %s")
-    ethtool_T = foreach_execute(ethernet_interfaces, "/sbin/ethtool -T %s")
-    ethtool_c = foreach_execute(ethernet_interfaces, "/sbin/ethtool -c %s")
-    ethtool_g = foreach_execute(ethernet_interfaces, "/sbin/ethtool -g %s")
-    ethtool_i = foreach_execute(ethernet_interfaces, "/sbin/ethtool -i %s")
-    ethtool_k = foreach_execute(ethernet_interfaces, "/sbin/ethtool -k %s")
+    ethtool = foreach_execute(ethernet.interfaces, "/sbin/ethtool %s")
+    ethtool_S = foreach_execute(ethernet.interfaces, "/sbin/ethtool -S %s")
+    ethtool_T = foreach_execute(ethernet.interfaces, "/sbin/ethtool -T %s")
+    ethtool_c = foreach_execute(ethernet.interfaces, "/sbin/ethtool -c %s")
+    ethtool_g = foreach_execute(ethernet.interfaces, "/sbin/ethtool -g %s")
+    ethtool_i = foreach_execute(ethernet.interfaces, "/sbin/ethtool -i %s")
+    ethtool_k = foreach_execute(ethernet.interfaces, "/sbin/ethtool -k %s")
     facter = simple_command("/usr/bin/facter")
     fc_match = simple_command("/bin/fc-match -sv 'sans:regular:roman' family fontformat")
     fcoeadm_i = simple_command("/usr/sbin/fcoeadm -i")


Test patch:
diff --git a/insights/tests/datasources/test_ethernet.py b/insights/tests/datasources/test_ethernet.py
new file mode 100644
--- /dev/null
+++ b/insights/tests/datasources/test_ethernet.py
@@ -0,0 +1,37 @@
+import pytest
+
+from insights.core.dr import SkipComponent
+from insights.specs.datasources.ethernet import interfaces, LocalSpecs
+from mock.mock import Mock
+
+RELATIVE_PATH = "insights_commands/ethernet_interfaces"
+
+IP_LINK = """
+1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000\\    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
+2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000\\    link/ether 52:54:00:13:14:b5 brd ff:ff:ff:ff:ff:ff
+3: enp8s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000\\    link/ether 52:54:00:e5:11:d4 brd ff:ff:ff:ff:ff:ff
+4: enp1s0.2@enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000\\    link/ether 52:54:00:13:14:b5 brd ff:ff:ff:ff:ff:ff
+5: ib0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 4092 qdisc mq state DOWN group default qlen 256\\    link/infiniband 00:01:02:03:fd:90:0:00:00:00:00:00:ef:0d:8b:02:01:d9:82:fd
+"""
+
+IP_LINK_BAD = ""
+
+EXPECTED = ['enp1s0', 'enp8s0', 'enp1s0.2']
+
+
+def test_ethernet_interfaces():
+    ip_link_command = Mock()
+    ip_link_command.content = IP_LINK.splitlines()
+    broker = {LocalSpecs.ip_link: ip_link_command}
+    result = interfaces(broker)
+    assert result is not None
+    assert isinstance(result, list)
+    assert result == sorted(EXPECTED)
+
+
+def test_ethernet_interfaces_bad():
+    ip_link_command = Mock()
+    ip_link_command.content = IP_LINK_BAD.splitlines()
+    broker = {LocalSpecs.ip_link: ip_link_command}
+    with pytest.raises(SkipComponent):
+        interfaces(broker)