Repository: pywbem/pywbemtools (language: python)

Issue:
Test failures with "instance shrub" command
From https://github.com/pywbem/pywbemtools/actions/runs/3214416794/jobs/5254794418:

```
E       Traceback (most recent call last):
E         File "/opt/hostedtoolcache/Python/3.10.7/x64/bin/pywbemcli", line 33, in <module>
E           sys.exit(load_entry_point('pywbemtools', 'console_scripts', 'pywbemcli')())
E         File "/home/runner/work/pywbemtools/pywbemtools/pywbemtools/_click_extensions.py", line 128, in __call__
E           return self.main(*args, **kwargs)
E         File "/opt/hostedtoolcache/Python/3.10.7/x64/lib/python3.10/site-packages/click/core.py", line 1055, in main
E           rv = self.invoke(ctx)
E         File "/opt/hostedtoolcache/Python/3.10.7/x64/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
E           return _process_result(sub_ctx.command.invoke(sub_ctx))
E         File "/opt/hostedtoolcache/Python/3.10.7/x64/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
E           return _process_result(sub_ctx.command.invoke(sub_ctx))
E         File "/opt/hostedtoolcache/Python/3.10.7/x64/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
E           return ctx.invoke(self.callback, **ctx.params)
E         File "/opt/hostedtoolcache/Python/3.10.7/x64/lib/python3.10/site-packages/click/core.py", line 760, in invoke
E           return __callback(*args, **kwargs)
E         File "/opt/hostedtoolcache/Python/3.10.7/x64/lib/python3.10/site-packages/click/decorators.py", line 38, in new_func
E           return f(get_current_context().obj, *args, **kwargs)
E         File "/home/runner/work/pywbemtools/pywbemtools/pywbemtools/pywbemcli/_cmd_instance.py", line 703, in instance_shrub
E           context.execute_cmd(lambda: cmd_instance_shrub(context, instancename, options))  # noqa: E501
E         File "/home/runner/work/pywbemtools/pywbemtools/pywbemtools/pywbemcli/_context_obj.py", line 319, in execute_cmd
E           cmd()  # The pywbemcli command function call.
E         File "/home/runner/work/pywbemtools/pywbemtools/pywbemtools/pywbemcli/_cmd_instance.py", line 703, in <lambda>
E           context.execute_cmd(lambda: cmd_instance_shrub(context, instancename, options))  # noqa: E501
E         File "/home/runner/work/pywbemtools/pywbemtools/pywbemtools/pywbemcli/_cmd_instance.py", line 1585, in cmd_instance_shrub
E           shrub.display_shrub(context.output_format, options['summary'])
E         File "/home/runner/work/pywbemtools/pywbemtools/pywbemtools/pywbemcli/_association_shrub.py", line 411, in display_shrub
E           click.echo(self.build_shrub_table(output_format, summary))
E         File "/home/runner/work/pywbemtools/pywbemtools/pywbemtools/pywbemcli/_association_shrub.py", line 552, in build_shrub_table
E           return format_table(rows, headers, title, table_format=output_format)
E         File "/home/runner/work/pywbemtools/pywbemtools/pywbemtools/_output_formatting.py", line 431, in format_table
E           result = tabulate.tabulate(rows, headers, tablefmt=table_format)
E         File "/opt/hostedtoolcache/Python/3.10.7/x64/lib/python3.10/site-packages/tabulate/__init__.py", line 2048, in tabulate
E           list_of_lists, headers = _normalize_tabular_data(
E         File "/opt/hostedtoolcache/Python/3.10.7/x64/lib/python3.10/site-packages/tabulate/__init__.py", line 1471, in _normalize_tabular_data
E           rows = list(map(lambda r: r if _is_separating_line(r) else list(r), rows))
E         File "/opt/hostedtoolcache/Python/3.10.7/x64/lib/python3.10/site-packages/tabulate/__init__.py", line 1471, in <lambda>
E           rows = list(map(lambda r: r if _is_separating_line(r) else list(r), rows))
E         File "/opt/hostedtoolcache/Python/3.10.7/x64/lib/python3.10/site-packages/tabulate/__init__.py", line 108, in _is_separating_line
E           or (len(row) >= 2 and row[1] == SEPARATING_LINE)
E         File "/opt/hostedtoolcache/Python/3.10.7/x64/lib/python3.10/site-packages/pywbem/_cim_obj.py", line 3500, in __eq__
E           raise TypeError(
E       TypeError: other must be CIMClassName, but is: <class 'str'>
```

Occurrences (all with the same symptom, above):
```
=========================== short test summary info ============================
FAILED tests/unit/pywbemcli/test_instance_cmds.py::TestSubcmd::test_execute_pywbemcli[Verify instance command shrub, simple table request w ns-inputs273-exp_response273-simple_assoc_mock_model.mof-True]
FAILED tests/unit/pywbemcli/test_instance_cmds.py::TestSubcmd::test_execute_pywbemcli[Verify instance command shrub, simple table request w ns, fullpath-inputs274-exp_response274-simple_assoc_mock_model.mof-True]
FAILED tests/unit/pywbemcli/test_instance_cmds.py::TestSubcmd::test_execute_pywbemcli[Verify instance command shrub, simple table; ns in request-inputs275-exp_response275-simple_assoc_mock_model.mof-True]
FAILED tests/unit/pywbemcli/test_instance_cmds.py::TestSubcmd::test_execute_pywbemcli[Verify instance command shrub, simple table; ns/host in request-inputs276-exp_response276-simple_assoc_mock_model.mof-True]
FAILED tests/unit/pywbemcli/test_instance_cmds.py::TestSubcmd::test_execute_pywbemcli[Verify instance command shrub, complex table-inputs279-exp_response279-complex_assoc_model.mof-True]
FAILED tests/unit/pywbemcli/test_instance_cmds.py::TestSubcmd::test_execute_pywbemcli[Verify instance command shrub, complex table, fullpath-inputs280-exp_response280-complex_assoc_model.mof-True]
FAILED tests/unit/pywbemcli/test_instance_cmds.py::TestSubcmd::test_execute_pywbemcli[Verify instance command shrub, complex table, summary-inputs281-exp_response281-complex_assoc_model.mof-True]
FAILED tests/unit/pywbemcli/test_instance_cmds.py::TestSubcmd::test_execute_pywbemcli[Verify instance command shrub, complex table, summary, fullpath-inputs282-exp_response282-complex_assoc_model.mof-True]
FAILED tests/unit/pywbemcli/test_instance_cmds.py::TestSubcmd::test_execute_pywbemcli[Verify instance command shrub, with reduced terminal width to trigger path folding-inputs284-exp_response284-simple_assoc_mock_model.mof-True]
FAILED tests/unit/pywbemcli/test_instance_cmds.py::TestSubcmd::test_execute_pywbemcli[Verify instance command shrub, with reduced terminal width to trigger path folding and --fullpath-inputs285-exp_response285-simple_assoc_mock_model.mof-True]
==== 10 failed, 1551 passed, 20 skipped, 370 warnings in 1019.03s (0:16:59) ====
```
testmock/wbemserver_mock_class.py issue with getting instance
In once case in this test code, where the path for an instance get includes the host name, the request fails with pywbem 1.5.0-dev but passes with pywbem 1.4.1.

The error returned is:

Mock script tests/unit/pywbemcli/testmock/wbemserver_mock_script.py failed: 4 (CIM_ERR_INVALID_PARAMETER): Reference property 'ManagedElement' association end point CIMInstanceName(classname='MCK_StorageComputerSystem', keybindings=NocaseDict({'Name': '10.1.2.3', 'CreationClassName': 'MCK_StorageComputerSystem'}), namespace='root/cimv2', host='FakedUrl:5988') does not exist 
Aborted!

Removing the set host corrects the issue for both pywbem 1.4.1 and pywbem 1.5.0

                central_inst_path = CIMInstanceName(
                    item[1][0],
                    keybindings=item[1][1],
                    host=conn.host,   ## <<<<<<<<<<<<<<<<<Here lay dragons
                    namespace=ns)

Removing the line host=conn.host fixes the issue but this means we have a weakness in the mock instance get and possibly create to be sure use of host in the repository is consistent.



testmock/wbemserver_mock_class.py issue with getting instance
In once case in this test code, where the path for an instance get includes the host name, the request fails with pywbem 1.5.0-dev but passes with pywbem 1.4.1.

The error returned is:

Mock script tests/unit/pywbemcli/testmock/wbemserver_mock_script.py failed: 4 (CIM_ERR_INVALID_PARAMETER): Reference property 'ManagedElement' association end point CIMInstanceName(classname='MCK_StorageComputerSystem', keybindings=NocaseDict({'Name': '10.1.2.3', 'CreationClassName': 'MCK_StorageComputerSystem'}), namespace='root/cimv2', host='FakedUrl:5988') does not exist 
Aborted!

Removing the set host corrects the issue for both pywbem 1.4.1 and pywbem 1.5.0

                central_inst_path = CIMInstanceName(
                    item[1][0],
                    keybindings=item[1][1],
                    host=conn.host,   ## <<<<<<<<<<<<<<<<<Here lay dragons
                    namespace=ns)

Removing the line host=conn.host fixes the issue but this means we have a weakness in the mock instance get and possibly create to be sure use of host in the repository is consistent.





Discussion / hints:

The issue is actually in pywbem _instance_write_provider about line 842 

    return inst_store.object_exists(path)

where the instance path is:

CIMInstanceName(classname='MCK_StorageComputerSystem', keybindings=NocaseDict({'Name': '10.1.2.3', 'CreationClassName': 'MCK_StorageComputerSystem'}), namespace='root/cimv2', host='FakedUrl:5988'), one of the
reference properties

This is a call to the mock repository which by definition is sets the key  for all instances with host = None.   In this case,
the user put the host name into the path.  In previous pywbem versions we did not check for valid path on associations.

Since the mocker was designed to be a local mechanism and we did not consider cross-host associations, the most logical
solution is to just test for the existence of host and remove it.  The only possibility beyond that would be to check the host name against the conn url and only remove it if they match.  Otherwise, fail as we do now.  However, this means adding a new parameter to the set of classes that represent the providers (BaseProvider, InstanceWriteProvider, NamespaceProvider, etc. to be able to at least pass on the host name element so that the InstanceWriteProvider.CreateInstance could test the host name of any references to see if they match the mocker host name.

The alternative is to simply set the host name to None  before testing for the existence of references and Issue a warning for any CreateInstance association reference property that includes a host name.


The issue is actually in pywbem _instance_write_provider about line 842 

    return inst_store.object_exists(path)

where the instance path is:

CIMInstanceName(classname='MCK_StorageComputerSystem', keybindings=NocaseDict({'Name': '10.1.2.3', 'CreationClassName': 'MCK_StorageComputerSystem'}), namespace='root/cimv2', host='FakedUrl:5988'), one of the
reference properties

This is a call to the mock repository which by definition is sets the key  for all instances with host = None.   In this case,
the user put the host name into the path.  In previous pywbem versions we did not check for valid path on associations.

Since the mocker was designed to be a local mechanism and we did not consider cross-host associations, the most logical
solution is to just test for the existence of host and remove it.  The only possibility beyond that would be to check the host name against the conn url and only remove it if they match.  Otherwise, fail as we do now.  However, this means adding a new parameter to the set of classes that represent the providers (BaseProvider, InstanceWriteProvider, NamespaceProvider, etc. to be able to at least pass on the host name element so that the InstanceWriteProvider.CreateInstance could test the host name of any references to see if they match the mocker host name.

The alternative is to simply set the host name to None  before testing for the existence of references and Issue a warning for any CreateInstance association reference property that includes a host name.



Fix patch:
diff --git a/docs/changes.rst b/docs/changes.rst
index 39786b09..fcced93a 100644
--- a/docs/changes.rst
+++ b/docs/changes.rst
@@ -23,6 +23,13 @@ Released: not yet
 
 * Fixed new formatting issues raised by flake8 5.0.
 
+* Fixed issue where package tabulate 0.9.0 causes failure of table build
+  by restricting tabulate version to < 0.9.0 for this version of pywbemcli
+  (see issue #1205)
+
+* Fixed issue that causes failure of tests with pwbem 1.5.0 because of
+  CIMInstanceName host attribute.  (see issue #1203)
+
 **Enhancements:**
 
 **Cleanup:**
diff --git a/requirements.txt b/requirements.txt
index 6ee00296..b59d1151 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -34,8 +34,9 @@ click-spinner>=0.1.8
 click-repl>=0.1.6
 asciitree>=0.3.3
 # tabulate 0.8.8 fixes ImportError on Python 3.10.
-tabulate>=0.8.2; python_version <= '3.9'
-tabulate>=0.8.8; python_version >= '3.10'
+# tabulate 0.9.0 causes failure, issue #1205
+tabulate>=0.8.2,<0.9.0; python_version <= '3.9'
+tabulate>=0.8.8,<0.9.0; python_version >= '3.10'
 toposort>=1.6
 # psutil does not install on Python 3.4 on Windows due to missing MS C++ 10.0 on GitHub Actions.
 # psutil 5.8.0 fixes an install error on Python 3.10


Test patch:
diff --git a/tests/unit/pywbemcli/testmock/wbemserver_mock_class.py b/tests/unit/pywbemcli/testmock/wbemserver_mock_class.py
index 403e5135..0b42cd6a 100644
--- a/tests/unit/pywbemcli/testmock/wbemserver_mock_class.py
+++ b/tests/unit/pywbemcli/testmock/wbemserver_mock_class.py
@@ -324,7 +324,6 @@ def __init__(self, conn, server, interop_ns=None, server_mock_data=None,
                 central_inst_path = CIMInstanceName(
                     item[1][0],
                     keybindings=item[1][1],
-                    host=conn.host,
                     namespace=ns)
                 prof_insts = self.wbem_server.get_selected_profiles(
                     registered_org=profile_name[0],