internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: RedHatInsights/insights-core

Issue:
Imprprovement for bond parser
Currently, the bond parser is not supporting to check if the primary slave is configured with "primary_reselect=failure".

Parser PR: https://github.com/RedHatInsights/insights-core/pull/2477



Discussion:

Resolving patch:
diff --git a/insights/parsers/bond.py b/insights/parsers/bond.py
--- a/insights/parsers/bond.py
+++ b/insights/parsers/bond.py
@@ -137,7 +137,7 @@ def parse_content(self, content):
             elif line.strip().startswith("ARP IP target/s (n.n.n.n form):"):
                 self._arp_ip_target = line.strip().split(':', 1)[1].strip()
             elif line.strip().startswith("Primary Slave"):
-                self._primary_slave = line.split(":", 1)[1].split()[0]
+                self._primary_slave = line.split(":", 1)[1].strip()
 
     @property
     def bond_mode(self):


Test patch:
diff --git a/insights/parsers/tests/test_bond.py b/insights/parsers/tests/test_bond.py
--- a/insights/parsers/tests/test_bond.py
+++ b/insights/parsers/tests/test_bond.py
@@ -223,7 +223,7 @@ def test_bond_class():
     assert bond_obj_4.file_name == 'bond0'
     assert bond_obj_4.arp_polling_interval == "1000"
     assert bond_obj_4.arp_ip_target == "10.152.1.1"
-    assert bond_obj_4.primary_slave == 'em3'
+    assert bond_obj_4.primary_slave == 'em3 (primary_reselect failure)'
 
     with pytest.raises(ParseException) as exc:
         bond_obj = Bond(context_wrap(BONDINFO_UNKNOWN_BOND_MODE, CONTEXT_PATH))
Repository: RedHatInsights/insights-core

Issue:
InsightsEvaluator ignores its stream argument during construction
The [InsightsEvaluator](https://github.com/RedHatInsights/insights-core/blob/master/insights/core/evaluators.py#L121) ignores its stream argument and passes `sys.stdout` to its superclass constructor.


Discussion:

Resolving patch:
diff --git a/insights/core/evaluators.py b/insights/core/evaluators.py
--- a/insights/core/evaluators.py
+++ b/insights/core/evaluators.py
@@ -118,7 +118,7 @@ def handle_result(self, plugin, r):
 
 class InsightsEvaluator(SingleEvaluator):
     def __init__(self, broker=None, system_id=None, stream=sys.stdout, incremental=False):
-        super(InsightsEvaluator, self).__init__(broker, stream=sys.stdout, incremental=incremental)
+        super(InsightsEvaluator, self).__init__(broker, stream=stream, incremental=incremental)
         self.system_id = system_id
         self.branch_info = {}
         self.product = "rhel"


Test patch:
Repository: RedHatInsights/insights-core

Issue:
Remove defunct entry_points
These scripts no longer exist. We should remove the entry_points.

* [insights.tools.generate_api_config](https://github.com/RedHatInsights/insights-core/blob/master/setup.py#L23)
* [insights.tools.perf](https://github.com/RedHatInsights/insights-core/blob/master/setup.py#L24)


Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -20,8 +20,6 @@
         'insights-inspect = insights.tools.insights_inspect:main',
         'insights-info = insights.tools.query:main',
         'insights-ocpshell= insights.ocpshell:main',
-        'gen_api = insights.tools.generate_api_config:main',
-        'insights-perf = insights.tools.perf:main',
         'client = insights.client:run',
         'mangle = insights.util.mangle:main'
     ]


Test patch:
Repository: RedHatInsights/insights-core

Issue:
httpd_conf combiner doesn't handle identifier with digits in their names
The [Name definition](https://github.com/RedHatInsights/insights-core/blob/master/insights/combiners/httpd_conf.py#L278) in the http conf parser doesn't allow digits. That causes it to fail when parsing things like `H2Push`.

Change to something like:

```python
First = InSet(string.ascii_letters + "_/")
Rest = String(string.ascii_letters + "_/" + string.digits)
Name = (First + Rest).map("".join)
```


Discussion:

Resolving patch:
diff --git a/insights/combiners/httpd_conf.py b/insights/combiners/httpd_conf.py
--- a/insights/combiners/httpd_conf.py
+++ b/insights/combiners/httpd_conf.py
@@ -275,7 +275,10 @@ def __init__(self, ctx):
         Complex = Forward()
         Comment = (WS >> OneLineComment("#")).map(lambda x: None)
 
-        Name = String(string.ascii_letters + "_/")
+        First = InSet(string.ascii_letters + "_/")
+        Rest = String(string.ascii_letters + "_/" + string.digits)
+        Name = (First + Rest).map("".join)
+
         Num = Number & (WSChar | LineEnd)
 
         StartName = WS >> PosMarker(StartTagName(Letters)) << WS


Test patch:
diff --git a/insights/combiners/tests/test_httpd_conf_tree.py b/insights/combiners/tests/test_httpd_conf_tree.py
--- a/insights/combiners/tests/test_httpd_conf_tree.py
+++ b/insights/combiners/tests/test_httpd_conf_tree.py
@@ -5,6 +5,10 @@
 from insights.parsers import SkipException
 import pytest
 
+HTTPD_CONF_MIXED_NAME = '''
+H2Push on
+'''
+
 
 HTTPD_CONF_MIXED = '''
 JustFotTest_NoSec "/var/www/cgi"
@@ -785,3 +789,9 @@ def test_recursive_includes():
         httpd1 = _HttpdConf(context_wrap(MULTIPLE_INCLUDES, path='/etc/httpd/conf/httpd.conf'))
         httpd2 = _HttpdConf(context_wrap(MULTIPLE_INCLUDES, path='/etc/httpd/conf.d/05-foreman.d/hello.conf'))
         HttpdConfTree([httpd1, httpd2])
+
+
+def test_mixed_name():
+    httpd1 = _HttpdConf(context_wrap(HTTPD_CONF_MIXED_NAME, path='/etc/httpd/conf/httpd.conf'))
+    result = HttpdConfTree([httpd1])
+    assert len(result.doc["H2Push"]) == 1
Repository: RedHatInsights/insights-core

Issue:
CommandParser should catch "...Not a directory"
The spam assassin spec when run by the client sometimes returns "...Not a directory," which the [CommandParser](https://github.com/RedHatInsights/insights-core/blob/master/insights/core/__init__.py#L528) doesn't catch.


Discussion:

Resolving patch:
diff --git a/insights/core/__init__.py b/insights/core/__init__.py
--- a/insights/core/__init__.py
+++ b/insights/core/__init__.py
@@ -527,6 +527,7 @@ class CommandParser(Parser):
 
     __bad_single_lines = [
             "no such file or directory",
+            "not a directory",
             "command not found",
             "no module named",
             "no files found for",


Test patch:
diff --git a/insights/tests/test_commandparser.py b/insights/tests/test_commandparser.py
--- a/insights/tests/test_commandparser.py
+++ b/insights/tests/test_commandparser.py
@@ -6,6 +6,7 @@
 CMF = "blah: Command not found"
 NO_FILES_FOUND = "No files found for docker.service"
 NO_SUCH_FILE = "/usr/bin/blah: No such file or directory"
+NOT_A_DIRECTORY = "/etc/mail/spamassassin/channel.d: Not a directory"
 MULTI_LINE = """
 blah: Command not found
 /usr/bin/blah: No such file or directory
@@ -41,6 +42,12 @@ def test_no_such_file_or_directory():
     assert "No such file or directory" in str(e.value)
 
 
+def test_not_a_directory():
+    with pytest.raises(ContentException) as e:
+        MockParser(context_wrap(NOT_A_DIRECTORY))
+    assert "Not a directory" in str(e.value)
+
+
 def test_multi_line():
     assert MULTI_LINE.split('\n') == MockParser(context_wrap(MULTI_LINE)).data
Repository: RedHatInsights/insights-core

Issue:
.line is None in all Parsr usages except for _HttpdConf and _NginxConf

Details: 

* https://github.com/RedHatInsights/insights-core/pull/2767#issuecomment-708893132 (describes more details)
* https://github.com/RedHatInsights/insights-core/pull/2797/files#diff-8bd49d3d40861a5372432dbc7ef98f917eb611e69ac8a868da5800ed2d14cd29R180 (TODO in test)


Discussion:

Resolving patch:
diff --git a/insights/parsr/query/__init__.py b/insights/parsr/query/__init__.py
--- a/insights/parsr/query/__init__.py
+++ b/insights/parsr/query/__init__.py
@@ -292,6 +292,21 @@ def get_crumbs(self):
         """
         return sorted(set(c.get_crumbs() for c in self.children))
 
+    @property
+    def line(self):
+        """
+        Returns the line of the child if only one child exists. This helps
+        queries behave more like dictionaries when you know only one result
+        should exist.
+        """
+        if len(self.children) == 0:
+            return None
+
+        if len(self.children) == 1:
+            return self.children[0].line
+
+        raise Exception("More than one value to return.")
+
     @property
     def string_value(self):
         """
@@ -299,8 +314,12 @@ def string_value(self):
         helps queries behave more like dictionaries when you know only one
         result should exist.
         """
+        if len(self.children) == 0:
+            return None
+
         if len(self.children) == 1:
             return self.children[0].string_value
+
         raise Exception("More than one value to return.")
 
     @property


Test patch:
Repository: RedHatInsights/insights-core

Issue:
first_file in insights_archive isn't bound to the right context
The first_file helper in [insights_archive.py](https://github.com/RedHatInsights/insights-core/blob/master/insights/specs/insights_archive.py#L7) isn't bound to the `HostArchiveContext`, so it will try to fire for any context that has a filesystem root.


Discussion:

Resolving patch:
diff --git a/insights/specs/insights_archive.py b/insights/specs/insights_archive.py
--- a/insights/specs/insights_archive.py
+++ b/insights/specs/insights_archive.py
@@ -5,6 +5,7 @@
 
 simple_file = partial(simple_file, context=HostArchiveContext)
 glob_file = partial(glob_file, context=HostArchiveContext)
+first_file = partial(first_file, context=HostArchiveContext)
 
 
 class InsightsArchiveSpecs(Specs):


Test patch:
Repository: RedHatInsights/insights-core

Issue:
Parsing exception during collection
My recent [PR](https://github.com/RedHatInsights/insights-core/pull/2883) uses `six.ensure_str`, which doesn't exist in the version of `six` depended on by insights-client. This causes what appears to be an unrelated parsing exception during collection.


Discussion:

Resolving patch:
diff --git a/insights/parsr/query/__init__.py b/insights/parsr/query/__init__.py
--- a/insights/parsr/query/__init__.py
+++ b/insights/parsr/query/__init__.py
@@ -35,6 +35,18 @@
     pass
 
 
+def _make_str(s):
+    """
+    Inspired by six.ensure_str in six version 1.15. See the six module for
+    copyright notice.
+    """
+    if six.PY2 and isinstance(s, six.text_type):
+        return s.encode("utf-8", "strict")
+    if six.PY3 and isinstance(s, six.binary_type):
+        return s.decode("utf-8", "strict")
+    return s
+
+
 class Entry(object):
     """
     Entry is the base class for the data model, which is a tree of Entry
@@ -43,7 +55,13 @@ class Entry(object):
     __slots__ = ("_name", "attrs", "children", "parent", "lineno", "src")
 
     def __init__(self, name=None, attrs=None, children=None, lineno=None, src=None):
-        self._name = intern(six.ensure_str(name)) if name is not None else name
+        if type(name) is str:
+            self._name = intern(name)
+        elif isinstance(name, (six.text_type, six.binary_type)):
+            self._name = intern(_make_str(name))
+        else:
+            self._name = name
+
         self.attrs = attrs if isinstance(attrs, (list, tuple)) else tuple()
         self.children = children if isinstance(children, (list, tuple)) else []
         self.parent = None


Test patch:
Repository: RedHatInsights/insights-core

Issue:
set_enabled should accept components in addition to strings.
The pydoc for [set_enabled](https://github.com/RedHatInsights/insights-core/blob/master/insights/core/dr.py#L89) says it accepts the fqdn of an object or the object itself, but it raises an exception (logged at DEBUG) if you pass the object.


Discussion:

Resolving patch:
diff --git a/insights/core/dr.py b/insights/core/dr.py
--- a/insights/core/dr.py
+++ b/insights/core/dr.py
@@ -89,10 +89,11 @@ def add(a, b):
 def set_enabled(component, enabled=True):
     """
     Enable a component for evaluation. If set to False, the component is
-    skipped, and all components that require it will not execute. If component
-    is a fully qualified name string of a callable object instead of the
-    callable object itself, the component's module is loaded as a side effect
-    of calling this function.
+    skipped, and all components that require it will not execute.
+
+    If component is a fully qualified name string of a callable object
+    instead of the callable object itself, the component's module is loaded
+    as a side effect of calling this function.
 
     Args:
         component (str or callable): fully qualified name of the component or
@@ -102,7 +103,11 @@ def set_enabled(component, enabled=True):
     Returns:
         None
     """
-    ENABLED[get_component(component) or component] = enabled
+    if isinstance(component, six.string_types):
+        component = get_component(component)
+
+    if component:
+        ENABLED[component] = enabled
 
 
 def is_enabled(component):


Test patch:
diff --git a/insights/tests/test_dr_enabled.py b/insights/tests/test_dr_enabled.py
--- a/insights/tests/test_dr_enabled.py
+++ b/insights/tests/test_dr_enabled.py
@@ -1,5 +1,5 @@
 from insights import combiner, dr
-
+from insights.core.context import HostContext
 from insights.parsers.uname import Uname
 
 
@@ -21,9 +21,13 @@ def test_enabled_object():
     assert dr.is_enabled(Uname)
 
 
+def test_set_enabled_object():
+    dr.set_enabled(HostContext)
+    assert dr.is_enabled(HostContext)
+
+
 def test_disabled_string():
     dr.set_enabled("insights.core.context.HostContext", False)
-    from insights.core.context import HostContext
     assert not dr.is_enabled(HostContext)
Repository: RedHatInsights/insights-core

Issue:
Remove verbose ImportError logs
The [ImportError logs generated during dynamic import](https://github.com/RedHatInsights/insights-core/blob/master/insights/core/dr.py#L160) are verbose and unhelpful. Let's remove them.

Related to https://bugzilla.redhat.com/show_bug.cgi?id=1920989.


Discussion:

Resolving patch:
diff --git a/insights/core/dr.py b/insights/core/dr.py
--- a/insights/core/dr.py
+++ b/insights/core/dr.py
@@ -156,9 +156,7 @@ def _import_component(name):
     for f in (_get_from_module, _get_from_class):
         try:
             return f(name)
-        except Exception as e:
-            log.debug("Couldn't load %s" % name)
-            log.debug(e, exc_info=True)
+        except:
             pass
 
 


Test patch:
Repository: RedHatInsights/insights-core

Issue:
Remove CalledProcessError logs generated while creating a core3 archive
Failing commands [generate a CalledProcessError in the collection log during serialization](https://github.com/RedHatInsights/insights-core/blob/master/insights/core/serde.py#L194). Those errors are already captured in the archive metadata, and they're causing confusion for people inspecting the log for other kinds of errors. Let's not log them.

Related to https://bugzilla.redhat.com/show_bug.cgi?id=1920989.


Discussion:

Resolving patch:
diff --git a/insights/core/serde.py b/insights/core/serde.py
--- a/insights/core/serde.py
+++ b/insights/core/serde.py
@@ -191,7 +191,6 @@ def dehydrate(self, comp, broker):
                 doc["results"] = marshal(value, root=self.data, pool=self.pool)
             except Exception:
                 errors.append(traceback.format_exc())
-                log.debug(traceback.format_exc())
                 doc["results"] = None
             finally:
                 doc["ser_time"] = time.time() - start


Test patch:
Repository: RedHatInsights/insights-core

Issue:
Sap HDB version parser cannot handle multiple inputs
The current spec sap_hdb_version is multi_output, but the implementation in the JSON concatenates all output in one file.  The HDBVersion parser expects one set of command output per file so it will not work correctly if there is more than one SAP SID.  The parser does work correctly for core collection which does the right thing regarding command output.


Discussion:
@xiangce @SteveHNH FYI.
@bfahr, Since the "HDB version" command works for its own SAP HANA instance only, it's incorrect to let the HDBVersion parser parse all the output if there are multiple SAP HANA instances (it's rare, but I'm not sure if it's possible) running.

In other words, I don't think it's correct theoretically to use the "insights-cat" to collect this spec, since it impossible for the rule to check which HDBVersion stands for which SAP HANA, which is done via checking the "file_path"

HTH,

Resolving patch:
diff --git a/insights/specs/insights_archive.py b/insights/specs/insights_archive.py
--- a/insights/specs/insights_archive.py
+++ b/insights/specs/insights_archive.py
@@ -201,7 +201,6 @@ class InsightsArchiveSpecs(Specs):
     rhev_data_center = simple_file("insights_commands/python_-m_insights.tools.cat_--no-header_rhev_data_center")
     rndc_status = simple_file("insights_commands/rndc_status")
     rpm_V_packages = first_file(["insights_commands/rpm_-V_coreutils_procps_procps-ng_shadow-utils_passwd_sudo_chrony", "insights_commands/rpm_-V_coreutils_procps_procps-ng_shadow-utils_passwd_sudo"])
-    sap_hdb_version = simple_file("insights_commands/python_-m_insights.tools.cat_--no-header_sap_hdb_version")
     saphostctl_getcimobject_sapinstance = simple_file("insights_commands/usr.sap.hostctrl.exe.saphostctrl_-function_GetCIMObject_-enuminstances_SAPInstance")
     satellite_content_hosts_count = first_file([
         "insights_commands/sudo_-iu_postgres_.usr.bin.psql_-d_foreman_-c_select_count_from_hosts",


Test patch:
Repository: RedHatInsights/insights-core

Issue:
Disallow empty filters
Currently rules can add whatever filters they like. We should guard against simple placeholders like empty strings that will allow everything.


Discussion:

Resolving patch:
diff --git a/insights/core/filters.py b/insights/core/filters.py
--- a/insights/core/filters.py
+++ b/insights/core/filters.py
@@ -69,14 +69,21 @@ def add_filter(component, patterns):
     def inner(component, patterns):
         if component in _CACHE:
             del _CACHE[component]
+
+        types = six.string_types + (list, set)
+        if not isinstance(patterns, types):
+            raise TypeError("Filter patterns must be of type string, list, or set.")
+
         if isinstance(patterns, six.string_types):
-            FILTERS[component].add(patterns)
+            patterns = set([patterns])
         elif isinstance(patterns, list):
-            FILTERS[component] |= set(patterns)
-        elif isinstance(patterns, set):
-            FILTERS[component] |= patterns
-        else:
-            raise TypeError("patterns must be string, list, or set.")
+            patterns = set(patterns)
+
+        for pat in patterns:
+            if not pat:
+                raise Exception("Filter patterns must not be empy.")
+
+        FILTERS[component] |= patterns
 
     if not plugins.is_datasource(component):
         for dep in dr.run_order(dr.get_dependency_graph(component)):


Test patch:
diff --git a/insights/tests/test_filters.py b/insights/tests/test_filters.py
--- a/insights/tests/test_filters.py
+++ b/insights/tests/test_filters.py
@@ -108,3 +108,8 @@ def test_add_filter_exception_not_filterable():
 def test_add_filter_exception_raw():
     with pytest.raises(Exception):
         filters.add_filter(Specs.metadata_json, "[]")
+
+
+def test_add_filter_exception_empty():
+    with pytest.raises(Exception):
+        filters.add_filter(Specs.ps_aux, "")
Repository: RedHatInsights/insights-core

Issue:
Corosync-cmapctl datasource should handle errors gracefully
The [datasource corosync_cmapctl_cmd_list datasource](https://github.com/RedHatInsights/insights-core/blob/6fd58aeab11232a62ea97ba576220abca808c660/insights/specs/default.py#L300) is causing the `corosync_cmapctl` spec to throw exceptions when the `corosync-cmapctl` command does not exist.  Also the commands are not consistent in the list created by the datasource.  


Discussion:

Resolving patch:
diff --git a/insights/specs/default.py b/insights/specs/default.py
--- a/insights/specs/default.py
+++ b/insights/specs/default.py
@@ -9,6 +9,7 @@
 """
 
 import logging
+import os
 import re
 import json
 
@@ -304,13 +305,22 @@ def corosync_cmapctl_cmd_list(broker):
         Returns:
             list: A list of related corosync-cmapctl commands based the RHEL version.
         """
-        if broker.get(IsRhel7):
-            return ["/usr/sbin/corosync-cmapctl", 'corosync-cmapctl -d runtime.schedmiss.timestamp', 'corosync-cmapctl -d runtime.schedmiss.delay']
-        if broker.get(IsRhel8):
-            return ["/usr/sbin/corosync-cmapctl", '/usr/sbin/corosync-cmapctl -m stats', '/usr/sbin/corosync-cmapctl -C schedmiss']
+        corosync_cmd = '/usr/sbin/corosync-cmapctl'
+        if os.path.exists(corosync_cmd):
+            if broker.get(IsRhel7):
+                return [
+                    corosync_cmd,
+                    ' '.join([corosync_cmd, '-d runtime.schedmiss.timestamp']),
+                    ' '.join([corosync_cmd, '-d runtime.schedmiss.delay'])]
+            if broker.get(IsRhel8):
+                return [
+                    corosync_cmd,
+                    ' '.join([corosync_cmd, '-m stats']),
+                    ' '.join([corosync_cmd, '-C schedmiss'])]
+
         raise SkipComponent()
-    corosync_cmapctl = foreach_execute(corosync_cmapctl_cmd_list, "%s")
 
+    corosync_cmapctl = foreach_execute(corosync_cmapctl_cmd_list, "%s")
     corosync_conf = simple_file("/etc/corosync/corosync.conf")
     cpu_cores = glob_file("sys/devices/system/cpu/cpu[0-9]*/online")
     cpu_siblings = glob_file("sys/devices/system/cpu/cpu[0-9]*/topology/thread_siblings_list")


Test patch:
Repository: RedHatInsights/insights-core

Issue:
When deserializing core collection archives don't throw generic exceptions
There's a condition where during core collection a file may be collected but it is not saved in the archive because it is empty or not text or json.  In a core-collection archive there may be metadata that indicates a file that was removed should be present in the archive. When insights-core tries to deserialize files from this type of archive a generic exception will be thrown which is not representative of the condition.

Here's an example of the log messages:

```
WARNING:insights.core.serde:Unrecognized type: insights.core.spec_factory.TextFileProvider
WARNING:insights.core.serde:Unrecognized type: insights.core.spec_factory.CommandOutputProvider
```

The `deserialize` and `hydrate` functions should distinguish between a `ContentException`, which is this case, or some other type of exception and only log `ContentException` messages at DEBUG level and not WARNING level.


Discussion:

Resolving patch:
diff --git a/insights/core/serde.py b/insights/core/serde.py
--- a/insights/core/serde.py
+++ b/insights/core/serde.py
@@ -87,11 +87,11 @@ def serialize(obj, root=None):
 
 
 def deserialize(data, root=None):
-    try:
-        (_type, from_dict) = DESERIALIZERS.get(data["type"])
-        return from_dict(_type, data["object"], root=root)
-    except Exception:
+    type_data = DESERIALIZERS.get(data["type"])
+    if type_data is None:
         raise Exception("Unrecognized type: %s" % data["type"])
+    (_type, from_dict) = type_data
+    return from_dict(_type, data["object"], root=root)
 
 
 def marshal(v, root=None, pool=None):
@@ -146,6 +146,8 @@ def hydrate(self, broker=None):
         Loads a Broker from a previously saved one. A Broker is created if one
         isn't provided.
         """
+        from insights.core.spec_factory import ContentException
+
         broker = broker or dr.Broker()
         for path in glob(os.path.join(self.meta_data, "*")):
             try:
@@ -156,6 +158,8 @@ def hydrate(self, broker=None):
                     if results:
                         broker[comp] = results
                         broker.exec_times[comp] = exec_time + ser_time
+            except ContentException as ex:
+                log.debug(ex)
             except Exception as ex:
                 log.warning(ex)
         return broker


Test patch: