Repository: dwavesystems/dwave-cloud-client
Pull request #643 (2024-07-27 20:38:15)

Speed up imports
Top-level namespace import is 20x faster; api and config imports around 30% faster; `dwave.cloud.Client` as before.

codecov[bot]: ## [Codecov](https://app.codecov.io/gh/dwavesystems/dwave-cloud-client/pull/643?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dwavesystems) Report
Attention: Patch coverage is `82.05128%` with `7 lines` in your changes missing coverage. Please review.
> Project coverage is 88.51%. Comparing base [(`7ea8531`)](https://app.codecov.io/gh/dwavesystems/dwave-cloud-client/commit/7ea85313deedc6d8cb9c66dfc35c09ce957e3be1?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dwavesystems) to head [(`9704ee6`)](https://app.codecov.io/gh/dwavesystems/dwave-cloud-client/commit/9704ee6b1333e5bf1fba2d2a09ecc911a1ecd525?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dwavesystems).

| [Files](https://app.codecov.io/gh/dwavesystems/dwave-cloud-client/pull/643?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dwavesystems) | Patch % | Lines |
|---|---|---|
| [dwave/cloud/\_\_init\_\_.py](https://app.codecov.io/gh/dwavesystems/dwave-cloud-client/pull/643?src=pr&el=tree&filepath=dwave%2Fcloud%2F__init__.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dwavesystems#diff-ZHdhdmUvY2xvdWQvX19pbml0X18ucHk=) | 81.81% | [4 Missing :warning: ](https://app.codecov.io/gh/dwavesystems/dwave-cloud-client/pull/643?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dwavesystems) |
| [dwave/cloud/api/client.py](https://app.codecov.io/gh/dwavesystems/dwave-cloud-client/pull/643?src=pr&el=tree&filepath=dwave%2Fcloud%2Fapi%2Fclient.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dwavesystems#diff-ZHdhdmUvY2xvdWQvYXBpL2NsaWVudC5weQ==) | 85.71% | [1 Missing :warning: ](https://app.codecov.io/gh/dwavesystems/dwave-cloud-client/pull/643?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dwavesystems) |
| [dwave/cloud/api/resources.py](https://app.codecov.io/gh/dwavesystems/dwave-cloud-client/pull/643?src=pr&el=tree&filepath=dwave%2Fcloud%2Fapi%2Fresources.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dwavesystems#diff-ZHdhdmUvY2xvdWQvYXBpL3Jlc291cmNlcy5weQ==) | 75.00% | [1 Missing :warning: ](https://app.codecov.io/gh/dwavesystems/dwave-cloud-client/pull/643?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dwavesystems) |
| [dwave/cloud/auth/flows.py](https://app.codecov.io/gh/dwavesystems/dwave-cloud-client/pull/643?src=pr&el=tree&filepath=dwave%2Fcloud%2Fauth%2Fflows.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dwavesystems#diff-ZHdhdmUvY2xvdWQvYXV0aC9mbG93cy5weQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/dwavesystems/dwave-cloud-client/pull/643?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dwavesystems) |

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##           master     #643      +/-   ##
==========================================
- Coverage   88.61%   88.51%   -0.11%     
==========================================
  Files          42       42              
  Lines        4586     4606      +20     
==========================================
+ Hits         4064     4077      +13     
- Misses        522      529       +7     
```

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/dwavesystems/dwave-cloud-client/pull/643?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dwavesystems).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dwavesystems).


Patch:
diff --git a/.circleci/config.yml b/.circleci/config.yml
index 041fef4..407aa02 100644
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -265,7 +265,7 @@ jobs:
             . env/bin/activate
             pip install -r benchmarks/requirements.txt
             asv machine --yes
-            asv run -q
+            asv run -q HEAD^!
 
   build-dist:
     docker:
diff --git a/dwave/cloud/__init__.py b/dwave/cloud/__init__.py
index 6e95bcf..c2fa854 100644
--- a/dwave/cloud/__init__.py
+++ b/dwave/cloud/__init__.py
@@ -15,10 +15,9 @@
 import sys
 import logging
 import importlib
+import types
+import warnings
 
-from dwave.cloud.client import Client
-from dwave.cloud.solver import Solver
-from dwave.cloud.computation import Future
 from dwave.cloud.utils.logging import add_loglevel, configure_logging_from_env
 
 __all__ = ['Client', 'Solver', 'Future']
@@ -35,11 +34,45 @@ add_loglevel('TRACE', 5)
 configure_logging_from_env(logger)
 
 
-# alias dwave.cloud.client.{qpu,sw,hybrid} as dwave.cloud.*
+# lazily import Client/Solver/Future -- only when actually asked for
+def __getattr__(cls):
+    modname = {'Client': 'client', 'Solver': 'solver', 'Future': 'computation'}
+    if cls in modname:
+        mod = importlib.import_module(f'dwave.cloud.{modname[cls]}')
+        return getattr(mod, cls)
+
+    raise AttributeError(f"module '{__name__}' has no attribute '{cls}'")
+
+def __dir__():
+    # dev note: make sure __all__ is limited to symbols we import lazily
+    yield from __all__
+    yield from globals()
+
+
+# lazily alias dwave.cloud.client.{qpu,sw,hybrid} as dwave.cloud.*
+# (i.e. don't preemptively import Client from submodules)
 def _alias_old_client_submodules():
+    class _submod(types.ModuleType):
+        def __getattr__(self, name):
+            modname = self.__name__
+            if name == 'Client':
+                warnings.warn(
+                    f"Importing 'Client' from 'dwave.cloud.{modname}' has been "
+                    f"deprecated and will be removed in dwave-cloud-client 0.14. "
+                    f"Import 'Client' directly from 'dwave.cloud.client.{modname}'.",
+                    DeprecationWarning, stacklevel=2)
+
+                mod = importlib.import_module(f'dwave.cloud.client.{modname}')
+                return getattr(mod, 'Client')
+
+            raise AttributeError(f"module '{self.__name__}' has no attribute '{name}'")
+
+        def __dir__(self):
+            self.__getattr__('Client')      # force submodule import
+            yield from dir(sys.modules[f'dwave.cloud.client.{self.__name__}'])
+
     for name in ('qpu', 'sw', 'hybrid'):
-        # note: create both module and local attribute
-        globals()[name] = sys.modules['dwave.cloud.{}'.format(name)] = \
-            importlib.import_module('dwave.cloud.client.{}'.format(name))
+        globals()[name] = sys.modules[f'dwave.cloud.{name}'] = _submod(name)
 
+# TODO: remove in 0.14.0
 _alias_old_client_submodules()
diff --git a/dwave/cloud/api/client.py b/dwave/cloud/api/client.py
index 0eed092..d0d58d7 100644
--- a/dwave/cloud/api/client.py
+++ b/dwave/cloud/api/client.py
@@ -16,19 +16,21 @@ from __future__ import annotations
 
 import logging
 from collections import deque, namedtuple
-from typing import Optional, Union
+from typing import Optional, Union, TYPE_CHECKING
 
 import requests
 import urllib3
 from packaging.specifiers import SpecifierSet
 from werkzeug.http import parse_options_header, dump_options_header
 
+import dwave.cloud.config   # don't use `from` to break circular import (config <> api.constants)
 from dwave.cloud.api import constants, exceptions
-from dwave.cloud.config import load_config, validate_config_v1
-from dwave.cloud.config.models import ClientConfig
 from dwave.cloud.utils.exception import is_caused_by
 from dwave.cloud.utils.http import PretimedHTTPAdapter, BaseUrlSession, default_user_agent
 
+if TYPE_CHECKING:
+    from dwave.cloud.config.models import ClientConfig
+
 __all__ = ['DWaveAPIClient', 'SolverAPIClient', 'MetadataAPIClient', 'LeapAPIClient']
 
 logger = logging.getLogger(__name__)
@@ -326,8 +328,8 @@ class DWaveAPIClient:
         logger.trace(f"{cls.__name__}.from_config_file("
                      f"config_file={config_file!r}, profile={profile!r}, **{kwargs!r})")
 
-        options = load_config(config_file=config_file, profile=profile, **kwargs)
-        config = validate_config_v1(options)
+        options = dwave.cloud.config.load_config(config_file=config_file, profile=profile, **kwargs)
+        config = dwave.cloud.config.validate_config_v1(options)
         return cls.from_config_model(config)
 
     @classmethod
@@ -346,7 +348,7 @@ class DWaveAPIClient:
         """
         logger.trace(f"{cls.__name__}.from_config(config={config!r}, **{kwargs!r}")
 
-        if isinstance(config, ClientConfig):
+        if isinstance(config, dwave.cloud.config.ClientConfig):
             return cls.from_config_model(config, **kwargs)
 
         kwargs['config_file'] = config
diff --git a/dwave/cloud/api/resources.py b/dwave/cloud/api/resources.py
index 6ef9fca..99fa62a 100644
--- a/dwave/cloud/api/resources.py
+++ b/dwave/cloud/api/resources.py
@@ -12,9 +12,11 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+from __future__ import annotations
+
 import io
 import json
-from typing import List, Union, Optional, Callable, get_type_hints
+from typing import List, Union, Optional, Callable, get_type_hints, TYPE_CHECKING
 from functools import wraps
 
 from pydantic import TypeAdapter
@@ -22,9 +24,11 @@ from pydantic import TypeAdapter
 from dwave.cloud.api.client import (
     DWaveAPIClient, SolverAPIClient, MetadataAPIClient, LeapAPIClient)
 from dwave.cloud.api import constants, models
-from dwave.cloud.config.models import ClientConfig
 from dwave.cloud.utils.coders import NumpyEncoder
 
+if TYPE_CHECKING:
+    from dwave.cloud.config.models import ClientConfig
+
 __all__ = ['Solvers', 'Problems', 'Regions']
 
 
diff --git a/dwave/cloud/auth/flows.py b/dwave/cloud/auth/flows.py
index bc5cf44..9dbc8a6 100644
--- a/dwave/cloud/auth/flows.py
+++ b/dwave/cloud/auth/flows.py
@@ -18,7 +18,7 @@ import logging
 import time
 import webbrowser
 from operator import sub
-from typing import Any, Callable, Dict, Optional, Union, Sequence, Literal
+from typing import Any, Callable, Dict, Optional, Union, Sequence, Literal, TYPE_CHECKING
 from urllib.parse import urljoin
 
 import click
@@ -31,11 +31,13 @@ from authlib.common.urls import add_params_to_uri
 from dwave.cloud.auth.config import OCEAN_SDK_CLIENT_ID, OCEAN_SDK_SCOPES
 from dwave.cloud.auth.creds import Credentials
 from dwave.cloud.auth.server import SingleRequestAppServer, RequestCaptureAndRedirectApp
-from dwave.cloud.config.models import ClientConfig
 from dwave.cloud.regions import resolve_endpoints
 from dwave.cloud.utils.http import default_user_agent
 from dwave.cloud.utils.logging import pretty_argvalues
 
+if TYPE_CHECKING:
+    from dwave.cloud.config.models import ClientConfig
+
 __all__ = ['AuthFlow', 'LeapAuthFlow', 'OAuthError']
 
 logger = logging.getLogger(__name__)
diff --git a/dwave/cloud/cli.py b/dwave/cloud/cli.py
index e4077d5..d53e88a 100644
--- a/dwave/cloud/cli.py
+++ b/dwave/cloud/cli.py
@@ -27,9 +27,8 @@ from typing import Callable, Dict, Optional, Tuple
 import click
 import requests.exceptions
 
-import dwave.cloud
-from dwave.cloud import Client
 from dwave.cloud import api
+from dwave.cloud.client import Client
 from dwave.cloud.solver import StructuredSolver, BaseUnstructuredSolver
 from dwave.cloud.utils.cli import default_text_input, strtrunc, CLIError
 from dwave.cloud.utils.dist import (
diff --git a/dwave/cloud/utils/decorators.py b/dwave/cloud/utils/decorators.py
index d65ae5b..0d80a82 100644
--- a/dwave/cloud/utils/decorators.py
+++ b/dwave/cloud/utils/decorators.py
@@ -28,7 +28,6 @@ import warnings
 from functools import wraps
 from secrets import token_hex
 from typing import Any, Callable, Dict, List, Mapping, Optional, Sequence, Union
-from unittest import mock
 
 from dwave.cloud.utils.time import epochnow
 
@@ -330,6 +329,10 @@ class cached:
         """
 
         def start(self):
+            # lazy import: cached.disabled is rarely used, yet mock import
+            # adds 50ms to root package import
+            from unittest import mock
+
             self.patcher = mock.patch.object(cached, '_disabled', True)
             self.patcher.start()
 
diff --git a/releasenotes/notes/speed-up-imports-d5012ac8751037b5.yaml b/releasenotes/notes/speed-up-imports-d5012ac8751037b5.yaml
new file mode 100644
index 0000000..bce2ed7
--- /dev/null
+++ b/releasenotes/notes/speed-up-imports-d5012ac8751037b5.yaml
@@ -0,0 +1,23 @@
+---
+features:
+  - |
+    Lazily import ``Client``/``Solver``/``Future`` in the top-level namespace,
+    ``dwave.cloud``. Direct imports are preferred, e.g.
+    ``from dwave.cloud.client import Client``, but the widely-used way
+    (``from dwave.cloud import Client``) is still supported for backwards
+    compatibility.
+
+    Similarly we now lazily import old namespaces with client type as submodule,
+    e.g. ``dwave.cloud.qpu``.
+
+    The main benefit of such lazy imports is import performance, as "heavy"
+    symbols are only imported when explicitly needed/asked for.
+  - |
+    Speed-up imports by slightly decoupling submodules.
+deprecations:
+  - |
+    Shorthand import paths for specialized Clients (for ``qpu``, ``sw`` and
+    ``hybrid``) are deprecated in ``dwave-cloud-client==0.12.2`` and will be
+    removed in ``dwave-cloud-client==0.14.0``. Instead, use the full import
+    path, e.g. ``from dwave.cloud.client.qpu import Client``, instead of
+    ``from dwave.cloud.qpu import Client``.


Test patch:
diff --git a/tests/__init__.py b/tests/__init__.py
index 3e79b9d..a131ba2 100644
--- a/tests/__init__.py
+++ b/tests/__init__.py
@@ -16,7 +16,7 @@ import os
 import warnings
 
 from dwave.cloud.config import load_config
-from dwave.cloud.exceptions import CanceledFutureError, ConfigFileError
+from dwave.cloud.exceptions import ConfigFileError
 
 
 # try to load client config needed for live tests on SAPI web service
diff --git a/tests/api/test_api_clients.py b/tests/api/test_api_clients.py
index 6e4ef38..acaf8e3 100644
--- a/tests/api/test_api_clients.py
+++ b/tests/api/test_api_clients.py
@@ -112,7 +112,7 @@ class TestConfig(unittest.TestCase):
         # `load_config` is already tested thoroughly in `tests.test_config`,
         # so it's ok to just mock it here
         config = dict(endpoint='https://test.com/path/')
-        with unittest.mock.patch("dwave.cloud.api.client.load_config",
+        with unittest.mock.patch("dwave.cloud.config.load_config",
                                  lambda *a, **kw: config):
             with DWaveAPIClient.from_config_file() as client:
                 self.assertEqual(client.session.base_url, config['endpoint'])
diff --git a/tests/test_client.py b/tests/test_client.py
index c835999..a920962 100644
--- a/tests/test_client.py
+++ b/tests/test_client.py
@@ -18,6 +18,7 @@ Tests the ability to connect to the SAPI server with the `dwave.cloud.qpu.Client
 test_mock_solver_loading.py duplicates some of these tests against a mock server.
 """
 
+import importlib
 import json
 import time
 import unittest
@@ -621,6 +622,21 @@ class ClientConstruction(unittest.TestCase):
                 self._verify_retry_config(retry, retry_kwargs)
 
 
+class VerifyLazyClientImport(unittest.TestCase):
+
+    def test_client_available(self):
+        import dwave.cloud
+        import dwave.cloud.client
+
+        self.assertEqual(dwave.cloud.client.Client, dwave.cloud.Client)
+
+    @parameterized.expand([("qpu", ), ("sw", ), ("hybrid", )])
+    def test_deprecation(self, name):
+        with self.assertWarns(DeprecationWarning):
+            mod = importlib.import_module(f'dwave.cloud.{name}')
+            self.assertTrue(issubclass(getattr(mod, 'Client'), Client))
+
+
 @mock.patch("dwave.cloud.regions.get_regions", get_default_regions)
 class ClientConfigIntegration(unittest.TestCase):