internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: RedHatInsights/insights-core

Issue:
ifcfg parser to support for MASTER and TEAM_MASTER keys slave type
We need to update the ifcfg parser to support TEAMING and BONDING slave type in the configuration file, so, that we can use MASTER' and 'TEAM_MASTER' keys in raw format.

For ex-  `obj['MASTER']="'bond0'"` or `obj['TEAM_MASTER']="'team0'"`


Discussion:

Resolving patch:
diff --git a/insights/parsers/ifcfg.py b/insights/parsers/ifcfg.py
--- a/insights/parsers/ifcfg.py
+++ b/insights/parsers/ifcfg.py
@@ -95,7 +95,7 @@ def parse_content(self, content):
 
             # In some cases we want to know what the actual value-side
             # of the key is before dequoting and stripping.
-            if key in ["DEVICE", "MASTER", "BONDING_OPTS"]:
+            if key in ["DEVICE", "MASTER", "TEAM_MASTER", "BONDING_OPTS"]:
                 self.data["raw_{0}_value".format(key.split('_')[0].lower())] = value
             if key != "DEVICE":
                 value = value.strip().strip(QUOTES)


Test patch:
diff --git a/insights/parsers/tests/test_ifcfg.py b/insights/parsers/tests/test_ifcfg.py
--- a/insights/parsers/tests/test_ifcfg.py
+++ b/insights/parsers/tests/test_ifcfg.py
@@ -111,7 +111,7 @@
 """.strip()
 
 IFCFG_TEST_RAW_MASTER_VALUE = """
-DEVICE="eth0"
+DEVICE="eth2"
 IPADDR=10.11.96.172
 NETMASK=255.255.252.0
 BOOTPROTO=none
@@ -121,6 +121,15 @@
 MASTER="bond0"
 """.strip()
 
+IFCFG_TEST_RAW_TEAM_MASTER_VALUE = """
+TYPE=Ethernet
+DEVICE="eth1"
+BOOTPROTO=none
+ONBOOT=yes
+TEAM_MASTER="team0"
+MTU=9000
+""".strip()
+
 IFCFG_TEST_RAW_BONDING_VALUE = """
 DEVICE="bond0"
 IPADDR=10.11.96.172
@@ -145,7 +154,9 @@
 
 IFCFG_PATH_NAMED_BOND_MODE = "etc/sysconfig/network-scripts/ifcfg-en0"
 
-IFCFG_PATH_NAMED_BOND_MASTER = "etc/sysconfig/network-scripts/ifcfg-eth0"
+IFCFG_PATH_ETH1 = "etc/sysconfig/network-scripts/ifcfg-eth1"
+
+IFCFG_PATH_ETH2 = "etc/sysconfig/network-scripts/ifcfg-eth2"
 
 IFCFG_TEST_BADLY_NAMED_BOND_MODE = """
 DEVICE=bond0
@@ -326,11 +337,11 @@ def test_ifcfg_blankline():
 
 def test_ifcfg_raw_bonding_master_value():
     context = context_wrap(IFCFG_TEST_RAW_MASTER_VALUE)
-    context.path = IFCFG_PATH_NAMED_BOND_MASTER
+    context.path = IFCFG_PATH_ETH2
 
     r = IfCFG(context)
 
-    assert r["raw_device_value"] == '"eth0"'
+    assert r["raw_device_value"] == '"eth2"'
     assert r["raw_master_value"] == '"bond0"'
 
 
@@ -342,3 +353,13 @@ def test_ifcfg_raw_bonding_value():
 
     assert r["raw_device_value"] == '"bond0"'
     assert r["raw_bonding_value"] == '"mode=balance-xor primary=eth1 arp_interval=1000 arp_ip_target=+10.11.96.1 downdelay =0"'
+
+
+def test_ifcfg_raw_team_master_value():
+    context = context_wrap(IFCFG_TEST_RAW_TEAM_MASTER_VALUE)
+    context.path = IFCFG_PATH_ETH1
+
+    r = IfCFG(context)
+
+    assert r["raw_device_value"] == '"eth1"'
+    assert r["raw_team_value"] == '"team0"'
Repository: RedHatInsights/insights-core

Issue:
RedhatRelease parser failed to parse minor release version in some scenarios
In few cases where redhat_release content is something similar to below, RedhatRelease parser fails to get the minor version extracted from it

Run:
```
>>> from insights.parsers.redhat_release import RedhatRelease
>>> from insights.tests import context_wrap
>>> RedhatRelease(context_wrap("Red Hat Enterprise Linux release 7.5-0.14")).major
7
>>> RedhatRelease(context_wrap("Red Hat Enterprise Linux release 7.5-0.14")).minor
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/insights/insights-core/insights/parsers/redhat_release.py", line 59, in minor
    return int(s[1])
ValueError: invalid literal for int() with base 10: '5-0'
>>> RedhatRelease(context_wrap("Red Hat Enterprise Linux release 7.5-0.14")).version
'7.5-0.14'
```


Discussion:

Resolving patch:
diff --git a/insights/parsers/redhat_release.py b/insights/parsers/redhat_release.py
--- a/insights/parsers/redhat_release.py
+++ b/insights/parsers/redhat_release.py
@@ -54,7 +54,7 @@ def major(self):
     @property
     def minor(self):
         """int: the minor version of this OS."""
-        s = self.parsed["version"].split(".")
+        s = self.parsed["version"].split("-", 1)[0].split(".")
         if len(s) > 1:
             return int(s[1])
 


Test patch:
diff --git a/insights/parsers/tests/test_redhat_release.py b/insights/parsers/tests/test_redhat_release.py
--- a/insights/parsers/tests/test_redhat_release.py
+++ b/insights/parsers/tests/test_redhat_release.py
@@ -10,6 +10,10 @@
 Red Hat Enterprise Linux Server release 7.2 (Maipo)
 """.strip()
 
+REDHAT_RELEASE3 = """
+Red Hat Enterprise Linux release 7.5-0.14
+""".strip()
+
 RHVH_RHV40 = """
 Red Hat Enterprise Linux release 7.3
 """.strip()
@@ -43,6 +47,16 @@ def test_rhe7():
     assert release.product == "Red Hat Enterprise Linux Server"
 
 
+def test_rhe75_0_14():
+    release = RedhatRelease(context_wrap(REDHAT_RELEASE3))
+    assert release.raw == REDHAT_RELEASE3
+    assert release.major == 7
+    assert release.minor == 5
+    assert release.version == "7.5-0.14"
+    assert release.is_rhel
+    assert release.product == "Red Hat Enterprise Linux"
+
+
 def test_rhevh35():
     release = RedhatRelease(context_wrap(RHEVH_RHEV35))
     assert release.raw == RHEVH_RHEV35
Repository: RedHatInsights/insights-core

Issue:
Exceptions on the command line with invalid arguments in the archive position
When trying to run `python -m insights foo` or `python -m insights existent_but_invalid.py`, an exception is raised all the way to the top, and if the file exists, a partial execution against the current machine happens. We should catch the exception, never partially execute, and provide a simple message for a better CLI experience.


Discussion:

Resolving patch:
diff --git a/insights/__init__.py b/insights/__init__.py
--- a/insights/__init__.py
+++ b/insights/__init__.py
@@ -28,7 +28,7 @@
 from .core import YAMLParser, JSONParser, XMLParser, CommandParser  # noqa: F401
 from .core import AttributeDict  # noqa: F401
 from .core import Syslog  # noqa: F401
-from .core.archives import COMPRESSION_TYPES, extract  # noqa: F401
+from .core.archives import COMPRESSION_TYPES, extract, InvalidArchive, InvalidContentType  # noqa: F401
 from .core import dr  # noqa: F401
 from .core.context import ClusterArchiveContext, HostContext, HostArchiveContext, SerializedArchiveContext  # noqa: F401
 from .core.dr import SkipComponent  # noqa: F401
@@ -260,17 +260,25 @@ class Args(object):
 
     broker = dr.Broker()
 
-    if formatter:
-        formatter.preprocess(broker)
-        broker = _run(broker, graph, root, context=context, inventory=inventory)
-        formatter.postprocess(broker)
-    elif print_component:
-        broker = _run(broker, graph, root, context=context, inventory=inventory)
-        broker.print_component(print_component)
-    else:
-        broker = _run(broker, graph, root, context=context, inventory=inventory)
-
-    return broker
+    try:
+        if formatter:
+            formatter.preprocess(broker)
+            broker = _run(broker, graph, root, context=context, inventory=inventory)
+            formatter.postprocess(broker)
+        elif print_component:
+            broker = _run(broker, graph, root, context=context, inventory=inventory)
+            broker.print_component(print_component)
+        else:
+            broker = _run(broker, graph, root, context=context, inventory=inventory)
+
+        return broker
+    except (InvalidContentType, InvalidArchive):
+        if args and args.archive:
+            path = args.archive
+            msg = "Invalid directory or archive. Did you mean to pass -p {p}?"
+            log.error(msg.format(p=path))
+        else:
+            raise
 
 
 def main():


Test patch:
Repository: RedHatInsights/insights-core

Issue:
streaming bug with python3.7
```
insights-cat ceph_insights
CommandOutputProvider(""echo 'hello'"")
hello
Traceback (most recent call last):
  File "/home/example/src/insights-core/insights/core/spec_factory.py", line 95, in stream
    st.send(None)
StopIteration

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/example/src/insights-core/env/bin/insights-cat", line 11, in <module>
    load_entry_point('insights-core', 'console_scripts', 'insights-cat')()
  File "/home/example/src/insights-core/insights/tools/cat.py", line 162, in main
    run(spec, archive=args.archive, quiet=args.quiet)
  File "/home/example/src/insights-core/insights/tools/cat.py", line 147, in run
    dump_spec(value, quiet=quiet)
  File "/home/example/src/insights-core/insights/tools/cat.py", line 115, in dump_spec
    for d in v.stream():
RuntimeError: generator raised StopIteration
```


Discussion:

Resolving patch:
diff --git a/insights/core/spec_factory.py b/insights/core/spec_factory.py
--- a/insights/core/spec_factory.py
+++ b/insights/core/spec_factory.py
@@ -92,7 +92,6 @@ def stream(self):
         st = self._stream()
         for l in next(st):
             yield l.rstrip("\n")
-        st.send(None)
 
     def _stream(self):
         raise NotImplemented()
@@ -220,15 +219,14 @@ def _stream(self):
         try:
             if self._content:
                 yield self._content
-                raise StopIteration
-
-            args = self.create_args()
-            if args:
-                with streams.connect(*args, env=SAFE_ENV) as s:
-                    yield s
             else:
-                with open(self.path, "rU") as f:  # universal newlines
-                    yield f
+                args = self.create_args()
+                if args:
+                    with streams.connect(*args, env=SAFE_ENV) as s:
+                        yield s
+                else:
+                    with open(self.path, "rU") as f:  # universal newlines
+                        yield f
         except StopIteration:
             raise
         except Exception as ex:
@@ -330,11 +328,10 @@ def _stream(self):
         try:
             if self._content:
                 yield self._content
-                raise StopIteration
-
-            args = self.create_args()
-            with self.ctx.connect(*args, env=self.create_env(), timeout=self.timeout) as s:
-                yield s
+            else:
+                args = self.create_args()
+                with self.ctx.connect(*args, env=self.create_env(), timeout=self.timeout) as s:
+                    yield s
         except StopIteration:
             raise
         except Exception as ex:


Test patch:
Repository: RedHatInsights/insights-core

Issue:
LogFileOutput.token_scan does not support `list`
The `keep_scan` already supports `list` as the target.
```python
MSG_FILTERED = """
May 18 15:24:28 lxc-rhel68-sat56 yum[11597]: Installed: lynx-2.8.6-27.el6.x86_64
May 18 15:36:19 lxc-rhel68-sat56 yum[11954]: Updated: sos-3.2-40.el6.noarch
May 18 15:36:19 lxc-rhel68-sat56 yum[11954]: Updated: bash-3.2-40.el6.noarch
""".strip()
```

```python
Messages.keep_scan('all_yum_updated_lines', ['yum', 'Updated'])
obj = Messages(context_wrap(MSG_FILTERED))

print(len(obj.all_yum_updated_lines))
>>> 2
```

But `token_scan` reports errors:
```python
Messages.token_scan('has_yum_updated_lines', ['yum', 'Updated'])
obj = Messages(context_wrap(MSG_FILTERED))

print(obj.has_yum_updated_lines)    <- True or False is expected here

...
/usr/local/lib/python3.7/site-packages/insights/core/__init__.py in <genexpr>(.0)
    889         Returns true if any line contains the given text string.
    890         """
--> 891         return any(s in l for l in self.lines)
    892 
    893     def _parse_line(self, line):

TypeError: 'in <string>' requires string as left operand, not list
```

To keep the two `_scan` functions work in the same way, need to ensure that `token_scan` also supports `list` as the scan target.


Discussion:

Resolving patch:
diff --git a/insights/core/__init__.py b/insights/core/__init__.py
--- a/insights/core/__init__.py
+++ b/insights/core/__init__.py
@@ -888,7 +888,8 @@ def __contains__(self, s):
         """
         Returns true if any line contains the given text string.
         """
-        return any(s in l for l in self.lines)
+        search_by_expression = self._valid_search(s)
+        return any(search_by_expression(l) for l in self.lines)
 
     def _parse_line(self, line):
         """


Test patch:
diff --git a/insights/tests/test_logfileoutput.py b/insights/tests/test_logfileoutput.py
--- a/insights/tests/test_logfileoutput.py
+++ b/insights/tests/test_logfileoutput.py
@@ -110,6 +110,28 @@ def count_lost_messages(self):
     assert log.lost_messages == 'lost 451 messages in 3 lines'
 
 
+def test_messages_scanners_list():
+    # Messages that are present can be kept
+    FakeMessagesClass.keep_scan('puppet_master_manifest_logs', ['puppet-master', 'manifest'])
+    FakeMessagesClass.keep_scan('puppet_master_first', ['puppet-master', 'first'])
+    # Token scan of something that's absent
+    FakeMessagesClass.token_scan('error_missing', ['ERROR', 'Missing'])
+    FakeMessagesClass.token_scan('error_info', ['ERROR', 'info'])
+
+    ctx = context_wrap(MESSAGES, path='/var/log/messages')
+    log = FakeMessagesClass(ctx)
+
+    assert hasattr(log, 'puppet_master_manifest_logs')
+    assert len(log.puppet_master_manifest_logs) == 1
+    assert hasattr(log, 'error_missing')
+    assert log.error_missing
+
+    assert hasattr(log, 'puppet_master_first')
+    assert len(log.puppet_master_first) == 0
+    assert hasattr(log, 'error_info')
+    assert log.error_info is False
+
+
 def test_messages_get_after():
     ctx = context_wrap(MESSAGES, path='/var/log/messages')
     log = FakeMessagesClass(ctx)
Repository: RedHatInsights/insights-core

Issue:
parse_fixed_table header prefix bug
The [parse_fix_table](https://github.com/RedHatInsights/insights-core/blob/master/insights/parsers/__init__.py#L341) function calls `.index` on the header line with each of the column names to find the starting index to use when parsing rows. This fails when any header is a prefix of any other that comes before it:
```
NAMESPACE    NAME    LABELS
default      foo     bar=1
```
Both `NAMESPACE` and `NAME` will have column indices of 0.


Discussion:

Resolving patch:
diff --git a/insights/parsers/__init__.py b/insights/parsers/__init__.py
--- a/insights/parsers/__init__.py
+++ b/insights/parsers/__init__.py
@@ -325,6 +325,13 @@ def parse_fixed_table(table_lines,
         [{'Column1': 'data1', 'Column2': 'data 2', 'Column3': 'data   3'},
          {'Column1': 'data4', 'Column2': 'data5', 'Column3': 'data6'}]
     """
+    def calc_column_indices(line, headers):
+        idx = []
+        for h in headers:
+            i = idx[-1] + 1 if idx else 0
+            idx.append(line.index(h, i))
+        return idx
+
     first_line = calc_offset(table_lines, heading_ignore)
     try:
         last_line = len(table_lines) - calc_offset(reversed(table_lines),
@@ -338,7 +345,9 @@ def parse_fixed_table(table_lines,
         for old_val, new_val in header_substitute:
             header = header.replace(old_val, new_val)
     col_headers = header.strip().split()
-    col_index = [header.index(c) for c in col_headers]
+    col_index = calc_column_indices(header, col_headers)
+    print(col_headers)
+    print(col_index)
 
     table_data = []
     for line in table_lines[first_line + 1:last_line]:


Test patch:
diff --git a/insights/parsers/tests/test_parsers_module.py b/insights/parsers/tests/test_parsers_module.py
--- a/insights/parsers/tests/test_parsers_module.py
+++ b/insights/parsers/tests/test_parsers_module.py
@@ -225,6 +225,12 @@ def test_calc_offset():
 """.strip()
 
 
+FIXED_CONTENT_DUP_HEADER_PREFIXES = """
+NAMESPACE    NAME    LABELS
+default      foo     app=superawesome
+""".strip()
+
+
 def test_parse_fixed_table():
     data = parse_fixed_table(FIXED_CONTENT_1.splitlines())
     assert len(data) == 3
@@ -280,6 +286,9 @@ def test_parse_fixed_table():
     assert data[4] == {'Column1': 'fooTrailing', 'Column_2': 'non-data li', 'Column_3': 'ne'}
     assert data[5] == {'Column1': 'foo Another', 'Column_2': 'trailing no', 'Column_3': 'n-data line'}
 
+    data = parse_fixed_table(FIXED_CONTENT_DUP_HEADER_PREFIXES.splitlines())
+    assert data[0] == {'NAMESPACE': 'default', 'NAME': 'foo', 'LABELS': 'app=superawesome'}
+
 
 def test_optlist_standard():
     d = optlist_to_dict('key1,key2=val2,key1=val1,key3')
Repository: RedHatInsights/insights-core

Issue:
Insights Archive spec is Incorrect
In #1817 the spec for the `aws_instance_type` insights archive should be the mangled version of the `python -m` command as specified in [insights core assets PR 132](https://github.com/RedHatInsights/insights-core-assets/pull/132) instead of the `curl` command.


Discussion:

Resolving patch:
diff --git a/insights/specs/insights_archive.py b/insights/specs/insights_archive.py
--- a/insights/specs/insights_archive.py
+++ b/insights/specs/insights_archive.py
@@ -11,7 +11,7 @@ class InsightsArchiveSpecs(Specs):
 
     all_installed_rpms = glob_file("insights_commands/rpm_-qa*")
     auditctl_status = simple_file("insights_commands/auditctl_-s")
-    aws_instance_type = simple_file("insights_commands/curl_http_..169.254.169.254.latest.meta-data.instance-type_--connect-timeout_5")
+    aws_instance_type = simple_file("insights_commands/python_-m_insights.tools.cat_--no-header_aws_instance_type")
     bios_uuid = simple_file("insights_commands/dmidecode_-s_system-uuid")
     blkid = simple_file("insights_commands/blkid_-c_.dev.null")
     brctl_show = simple_file("insights_commands/brctl_show")


Test patch:
Repository: RedHatInsights/insights-core

Issue:
Add "display_name" spec to specs
The client drops a file into the root of the archive called "display_name" that is needed for canonical fact info by other systems. We need a `RegistryPoint` in `insights.specs` and an implementation in `insights.specs.insights_archive`.


Discussion:

Resolving patch:
diff --git a/insights/specs/__init__.py b/insights/specs/__init__.py
--- a/insights/specs/__init__.py
+++ b/insights/specs/__init__.py
@@ -103,6 +103,7 @@ class Specs(SpecSet):
     dirsrv_access = RegistryPoint(multi_output=True, filterable=True)
     dirsrv_errors = RegistryPoint(multi_output=True, filterable=True)
     display_java = RegistryPoint()
+    display_name = RegistryPoint()
     dmesg = RegistryPoint(filterable=True)
     dmidecode = RegistryPoint()
     dmsetup_info = RegistryPoint()
diff --git a/insights/specs/insights_archive.py b/insights/specs/insights_archive.py
--- a/insights/specs/insights_archive.py
+++ b/insights/specs/insights_archive.py
@@ -39,6 +39,7 @@ class InsightsArchiveSpecs(Specs):
     dig_edns = simple_file("insights_commands/dig_edns_0_._SOA")
     dig_noedns = simple_file("insights_commands/dig_noedns_._SOA")
     display_java = simple_file("insights_commands/alternatives_--display_java")
+    display_name = simple_file("display_name")
     dmesg = simple_file("insights_commands/dmesg")
     dmidecode = simple_file("insights_commands/dmidecode")
     dmsetup_info = simple_file("insights_commands/dmsetup_info_-C")


Test patch:
Repository: RedHatInsights/insights-core

Issue:
Directories of exploded archives not recognized as cluster
Directories of exploded archives aren't recognized as a cluster but are erroneously identified as whatever context matches the first marker file we encounter after recursively enumerating every file in all subdirectories.


Discussion:

Resolving patch:
diff --git a/insights/core/cluster.py b/insights/core/cluster.py
--- a/insights/core/cluster.py
+++ b/insights/core/cluster.py
@@ -1,8 +1,10 @@
 #!/usr/bin/env python
 import itertools
-import pandas as pd
+import os
 from collections import defaultdict
 
+import pandas as pd
+
 from ansible.parsing.dataloader import DataLoader
 from ansible.inventory.manager import InventoryManager
 
@@ -46,8 +48,14 @@ def attach_machine_id(result, mid):
 
 def process_archives(archives):
     for archive in archives:
-        with extract(archive) as ex:
-            ctx = create_context(ex.tmp_dir)
+        if os.path.isfile(archive):
+            with extract(archive) as ex:
+                ctx = create_context(ex.tmp_dir)
+                broker = dr.Broker()
+                broker[ctx.__class__] = ctx
+                yield dr.run(broker=broker)
+        else:
+            ctx = create_context(archive)
             broker = dr.Broker()
             broker[ctx.__class__] = ctx
             yield dr.run(broker=broker)
diff --git a/insights/core/hydration.py b/insights/core/hydration.py
--- a/insights/core/hydration.py
+++ b/insights/core/hydration.py
@@ -37,13 +37,15 @@ def identify(files):
     if not common_path:
         raise archives.InvalidArchive("Unable to determine common path")
 
-    if any(f.endswith(archives.COMPRESSION_TYPES) for f in os.listdir(common_path)):
-        return common_path, ClusterArchiveContext
-
     return common_path, HostArchiveContext
 
 
 def create_context(path, context=None):
+    top = os.listdir(path)
+    arc = [os.path.join(path, f) for f in top if f.endswith(archives.COMPRESSION_TYPES)]
+    if arc:
+        return ClusterArchiveContext(path, all_files=arc)
+
     all_files = get_all_files(path)
     if not all_files:
         raise archives.InvalidArchive("No files in archive")


Test patch:
Repository: RedHatInsights/insights-core

Issue:
Incorrect insights.components.rhel_version module doc
The [example](https://github.com/RedHatInsights/insights-core/blob/master/insights/components/rhel_version.py#L17) in the insights.components.rhel_version pydoc doesn't work. The objects don't have a `rhel_version` attribute.


Discussion:

Resolving patch:
diff --git a/insights/components/rhel_version.py b/insights/components/rhel_version.py
--- a/insights/components/rhel_version.py
+++ b/insights/components/rhel_version.py
@@ -2,24 +2,14 @@
 IsRhel6, IsRhel7 and IsRhel8
 ===============================
 
-The ``IsRhel*`` components each use the ``RedhatRelease`` combiner to
-retrieve the RHEL version information.
-Each component checks if the release version matches the version it represents,
-if the version does not match what is expected the class raises ``SkipComponent``
-so that the dependent component will not fire.
-Can be added as a dependency of a parser so that the parser only fires if the
-``IsRhel*`` dependency is met.
-
-An example from the following ``/etc/redhat_release`` file output::
-
-    Red Hat Enterprise Linux release 8.0 (Ootpa)
-
-Example:
-
-    >>> type(IsRhel8)
-    <class 'insights.components.rhel_version.Is_Rhel8'>
-    >>> is_rhel8.rhel_version
-    '8.0'
+An ``IsRhel*`` component is valid if the
+:py:class:`insights.combiners.redhat_release.RedHatRelease` combiner indicates
+the major RHEL version represented by the component. Otherwise, it raises a
+:py:class:`insights.core.dr.SkipComponent` to prevent dependent components from
+executing.
+
+In particular, an ``IsRhel*`` component can be added as a dependency of a
+parser to limit it to a given version.
 """
 
 from insights.core.plugins import component


Test patch: