Repository: AgnostiqHQ/covalent (python)

Issue:
`get_result` not working in self hosted scenario
### Environment

- **Covalent version**: 0.209.1rc0
- **Python version**: 3.8.15
- **Operating system**: Ubuntu 20.04.5 LTS


### What is happening?

When hosting covalent server on a remote machine, the `get_result` function throws an  `AttributeError` exception when trying to retrieve even a successfully completed workflow's result. Submission does not seem to be a problem, and when checking the result on the UI, it seems that the workflow was also completed successfully.

### How can we reproduce the issue?

Can be reproduced by submitting the workflow on a remote covalent server, and calling the `get_result` function to retrieve the result.

The following is the traceback:

```python
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[2], line 1
----> 1 result = ct.get_result(dispatch_id)
      2 print(result)

File ~/dev/covalent/covalent/_results_manager/results_manager.py:59, in get_result(dispatch_id, wait)
     54 try:
     55     result = _get_result_from_dispatcher(
     56         dispatch_id,
     57         wait,
     58     )
---> 59     result_object = pickle.loads(codecs.decode(result["result"].encode(), "base64"))
     61 except MissingLatticeRecordError as ex:
     62     app_log.warning(
     63         f"Dispatch ID {dispatch_id} was not found in the database. Incorrect dispatch id."
     64     )

File ~/anaconda3/envs/cova/lib/python3.8/site-packages/networkx/classes/reportviews.py:983, in OutEdgeView.__setstate__(self, state)
    981 def __setstate__(self, state):
    982     self._graph = G = state["_graph"]
--> 983     self._adjdict = G._succ if hasattr(G, "succ") else G._adj
    984     self._nodes_nbrs = self._adjdict.items

AttributeError: 'MultiDiGraph' object has no attribute '_adj'
```

### What should happen?

`get_result` should return the result object instead of throwing an exception. An addition to the scope of this issue is to ease the access to get result from a remote server; similar to how a user can pass a url to `ct.dispatch`, they should be able to pass a url to `ct.get_result` as well.

### Any suggestions?

It seems that this issue comes from `networkx` dependency not matching between the client and the server. Even though both were installed using the same method. We should probably look into pinning its version instead.

Pull request:
Improving `get_result` for remote servers
<!--
⚠️ Your pull request title should be short detailed and understandable for all.
⚠️ Increment the version number in the /VERSION file. If this is a bugfix, increment the patch version (the rightmost number), for example 0.18.2 becomes 0.18.3. If it's a new feature, increment the minor version (the middle number), for example 0.18.2 becomes 0.19.0.
⚠️ Add a note to /CHANGELOG.md with your version number and the date, summarizing the changes.
⚠️ If your pull request fixes an open issue, please link to the issue.
⚠️ Rebase the latest changes from the develop branch. Assuming origin points to https://github.com/AgnostiqHQ/covalent, you should run git rebase origin/develop.
-->

- [x] I have added the tests to cover my changes.
- [x] I have updated the documentation and CHANGELOG accordingly.
- [x] I have read the CONTRIBUTING document.

Resolves #1457 

Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index ae0ee802..f5485551 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,6 +7,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [UNRELEASED]
 
+### Fixed
+
+- The range of `networkx` versions in requirements.txt weren't compatible with each other, thus it is pinned to `2.8.6` now
+
+### Added
+
+- Added `dispatcher_addr` argument to `ct.get_result` similar to `ct.dispatch` so that it doesn't always fallback to using the default configured address
+
+### Tests
+
+- Updated `_get_result_from_dispatcher` test to verify whether using a link directly works or not
+
 ### Docs
 - Revised UI reference. Added Settings page documentation.
 
@@ -17,12 +29,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Casey Jao <casey@agnostiq.ai>
 
 
-
 ### Fixed
 
 - Fixed naming of collection nodes (was breaking postprocessing)
 - Restored compatibility with stable release of AWS executors
 
+
 ## [0.212.0-rc.0] - 2023-01-13
 
 ### Authors
diff --git a/covalent/_dispatcher_plugins/local.py b/covalent/_dispatcher_plugins/local.py
index 91bff06d..afe2c33c 100644
--- a/covalent/_dispatcher_plugins/local.py
+++ b/covalent/_dispatcher_plugins/local.py
@@ -41,9 +41,7 @@ class LocalDispatcher(BaseDispatcher):
     @staticmethod
     def dispatch(
         orig_lattice: Lattice,
-        dispatcher_addr: str = get_config("dispatcher.address")
-        + ":"
-        + str(get_config("dispatcher.port")),
+        dispatcher_addr: str = None,
     ) -> Callable:
         """
         Wrapping the dispatching functionality to allow input passing
@@ -54,12 +52,17 @@ class LocalDispatcher(BaseDispatcher):
 
         Args:
             orig_lattice: The lattice/workflow to send to the dispatcher server.
-            dispatcher_addr: The address of the dispatcher server.
+            dispatcher_addr: The address of the dispatcher server.  If None then then defaults to the address set in Covalent's config.
 
         Returns:
             Wrapper function which takes the inputs of the workflow as arguments
         """
 
+        if dispatcher_addr is None:
+            dispatcher_addr = (
+                get_config("dispatcher.address") + ":" + str(get_config("dispatcher.port"))
+            )
+
         @wraps(orig_lattice)
         def wrapper(*args, **kwargs) -> str:
             """
@@ -92,9 +95,7 @@ class LocalDispatcher(BaseDispatcher):
     @staticmethod
     def dispatch_sync(
         lattice: Lattice,
-        dispatcher_addr: str = get_config("dispatcher.address")
-        + ":"
-        + str(get_config("dispatcher.port")),
+        dispatcher_addr: str = None,
     ) -> Callable:
         """
         Wrapping the synchronous dispatching functionality to allow input
@@ -105,12 +106,17 @@ class LocalDispatcher(BaseDispatcher):
 
         Args:
             orig_lattice: The lattice/workflow to send to the dispatcher server.
-            dispatcher_addr: The address of the dispatcher server.
+            dispatcher_addr: The address of the dispatcher server. If None then then defaults to the address set in Covalent's config.
 
         Returns:
             Wrapper function which takes the inputs of the workflow as arguments
         """
 
+        if dispatcher_addr is None:
+            dispatcher_addr = (
+                get_config("dispatcher.address") + ":" + str(get_config("dispatcher.port"))
+            )
+
         @wraps(lattice)
         def wrapper(*args, **kwargs) -> Result:
             """
diff --git a/covalent/_results_manager/results_manager.py b/covalent/_results_manager/results_manager.py
index 454e215a..589b0f20 100644
--- a/covalent/_results_manager/results_manager.py
+++ b/covalent/_results_manager/results_manager.py
@@ -20,6 +20,7 @@
 
 
 import codecs
+import contextlib
 import os
 from typing import Dict, List, Optional, Union
 
@@ -38,13 +39,14 @@ app_log = logger.app_log
 log_stack_info = logger.log_stack_info
 
 
-def get_result(dispatch_id: str, wait: bool = False) -> Result:
+def get_result(dispatch_id: str, wait: bool = False, dispatcher_addr: str = None) -> Result:
     """
     Get the results of a dispatch from a file.
 
     Args:
         dispatch_id: The dispatch id of the result.
         wait: Controls how long the method waits for the server to return a result. If False, the method will not wait and will return the current status of the workflow. If True, the method will wait for the result to finish and keep retrying for sys.maxsize.
+        dispatcher_addr: Dispatcher server address, if None then defaults to the address set in Covalent's config.
 
     Returns:
         The result from the file.
@@ -55,6 +57,7 @@ def get_result(dispatch_id: str, wait: bool = False) -> Result:
         result = _get_result_from_dispatcher(
             dispatch_id,
             wait,
+            dispatcher_addr,
         )
         result_object = pickle.loads(codecs.decode(result["result"].encode(), "base64"))
 
@@ -71,7 +74,7 @@ def get_result(dispatch_id: str, wait: bool = False) -> Result:
 def _get_result_from_dispatcher(
     dispatch_id: str,
     wait: bool = False,
-    dispatcher: str = get_config("dispatcher.address") + ":" + str(get_config("dispatcher.port")),
+    dispatcher_addr: str = None,
     status_only: bool = False,
 ) -> Dict:
 
@@ -82,7 +85,7 @@ def _get_result_from_dispatcher(
         dispatch_id: The dispatch id of the result.
         wait: Controls how long the method waits for the server to return a result. If False, the method will not wait and will return the current status of the workflow. If True, the method will wait for the result to finish and keep retrying for sys.maxsize.
         status_only: If true, only returns result status, not the full result object, default is False.
-        dispatcher: Dispatcher server address, defaults to the address set in covalent.config.
+        dispatcher_addr: Dispatcher server address, if None then defaults to the address set in Covalent's config.
 
     Returns:
         The result object from the server.
@@ -91,12 +94,17 @@ def _get_result_from_dispatcher(
         MissingLatticeRecordError: If the result is not found.
     """
 
+    if dispatcher_addr is None:
+        dispatcher_addr = (
+            get_config("dispatcher.address") + ":" + str(get_config("dispatcher.port"))
+        )
+
     retries = int(EXTREME) if wait else 5
 
     adapter = HTTPAdapter(max_retries=Retry(total=retries, backoff_factor=1))
     http = requests.Session()
     http.mount("http://", adapter)
-    url = "http://" + dispatcher + "/api/result/" + dispatch_id
+    url = f"http://{dispatcher_addr}/api/result/{dispatch_id}"
     response = http.get(
         url,
         params={"wait": bool(int(wait)), "status_only": status_only},
@@ -104,13 +112,12 @@ def _get_result_from_dispatcher(
     if response.status_code == 404:
         raise MissingLatticeRecordError
     response.raise_for_status()
-    result = response.json()
-    return result
+    return response.json()
 
 
 def _delete_result(
     dispatch_id: str,
-    results_dir: str = os.environ.get("COVALENT_DATA_DIR") or get_config("dispatcher.results_dir"),
+    results_dir: str = None,
     remove_parent_directory: bool = False,
 ) -> None:
     """
@@ -128,6 +135,9 @@ def _delete_result(
         FileNotFoundError: If the result file is not found.
     """
 
+    if results_dir is None:
+        results_dir = os.environ.get("COVALENT_DATA_DIR") or get_config("dispatcher.results_dir")
+
     import shutil
 
     result_folder_path = os.path.join(results_dir, f"{dispatch_id}")
@@ -135,10 +145,8 @@ def _delete_result(
     if os.path.exists(result_folder_path):
         shutil.rmtree(result_folder_path, ignore_errors=True)
 
-    try:
+    with contextlib.suppress(OSError):
         os.rmdir(results_dir)
-    except OSError:
-        pass
 
     if remove_parent_directory:
         shutil.rmtree(results_dir, ignore_errors=True)
diff --git a/requirements-client.txt b/requirements-client.txt
index 6bc3b61f..a55d0897 100644
--- a/requirements-client.txt
+++ b/requirements-client.txt
@@ -2,7 +2,7 @@ aiofiles==0.8.0
 aiohttp==3.8.1
 cloudpickle==2.0.0
 furl==2.1.3
-networkx==2.5
+networkx==2.8.6
 requests>=2.24.0
 simplejson==3.17.6
 toml==0.10.2
diff --git a/requirements.txt b/requirements.txt
index 680e7027..13405a91 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -6,7 +6,7 @@ cloudpickle>=2.0.0,<=2.2.0
 dask[distributed]>=2022.6.0,<=2022.9.0
 fastapi>=0.79.0,<=0.83.0
 furl==2.1.3
-networkx>=2.5,<=2.8.6
+networkx==2.8.6
 psutil>=5.9.0,<=5.9.2
 pydantic>=1.10.1,<=1.10.2
 python-socketio==5.7.1


Test patch:
diff --git a/tests/covalent_tests/results_manager_tests/results_manager_test.py b/tests/covalent_tests/results_manager_tests/results_manager_test.py
index 11b3431e..e94c5469 100644
--- a/tests/covalent_tests/results_manager_tests/results_manager_test.py
+++ b/tests/covalent_tests/results_manager_tests/results_manager_test.py
@@ -23,6 +23,8 @@
 from http.client import HTTPMessage
 from unittest.mock import ANY, Mock, call
 
+import pytest
+
 from covalent._results_manager import wait
 from covalent._results_manager.results_manager import _get_result_from_dispatcher
 from covalent._shared_files.config import get_config
@@ -30,7 +32,14 @@ from covalent._shared_files.config import get_config
 DISPATCH_ID = "91c3ee18-5f2d-44ee-ac2a-39b79cf56646"
 
 
-def test_get_result_from_dispatcher(mocker):
+@pytest.mark.parametrize(
+    "dispatcher_addr",
+    [
+        get_config("dispatcher.address") + ":" + str(get_config("dispatcher.port")),
+        "localhost:48008",
+    ],
+)
+def test_get_result_from_dispatcher(mocker, dispatcher_addr):
     retries = 10
     getconn_mock = mocker.patch("urllib3.connectionpool.HTTPConnectionPool._get_conn")
     mocker.patch("requests.Response.json", return_value=True)
@@ -41,9 +50,8 @@ def test_get_result_from_dispatcher(mocker):
     mock_response.append(Mock(status=200, msg=HTTPMessage()))
     getconn_mock.return_value.getresponse.side_effect = mock_response
     dispatch_id = "9d1b308b-4763-4990-ae7f-6a6e36d35893"
-    dispatcher = get_config("dispatcher.address") + ":" + str(get_config("dispatcher.port"))
     _get_result_from_dispatcher(
-        dispatch_id, wait=wait.LONG, dispatcher=dispatcher, status_only=False
+        dispatch_id, wait=wait.LONG, dispatcher_addr=dispatcher_addr, status_only=False
     )
     assert (
         getconn_mock.return_value.request.mock_calls