Repository: getmoto/moto
Pull request #7971 (2024-08-12 12:30:24)

Utility to iterate over flattened backends
This PR adds a utility method to iterate over a flattened view of a `BackendDict`.

Each record is a tuple of account ID, region, and the corresponding `BaseBackend`.

codecov[bot]: ## [Codecov](https://app.codecov.io/gh/getmoto/moto/pull/7971?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getmoto) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 94.37%. Comparing base [(`981cb2b`)](https://app.codecov.io/gh/getmoto/moto/commit/981cb2b598be3601f4a192bed6cec161fd3e46c4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getmoto) to head [(`dad8cb4`)](https://app.codecov.io/gh/getmoto/moto/commit/dad8cb4d8e3c5eab0a8ebda96d0feff68d48db53?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getmoto).

<details><summary>Additional details and impacted files</summary>


```diff
@@           Coverage Diff           @@
##           master    #7971   +/-   ##
=======================================
  Coverage   94.37%   94.37%           
=======================================
  Files        1120     1120           
  Lines       96205    96209    +4     
=======================================
+ Hits        90791    90795    +4     
  Misses       5414     5414           
```

| [Flag](https://app.codecov.io/gh/getmoto/moto/pull/7971/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getmoto) | Coverage Δ | |
|---|---|---|
| [servertests](https://app.codecov.io/gh/getmoto/moto/pull/7971/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getmoto) | `28.95% <100.00%> (+<0.01%)` | :arrow_up: |
| [unittests](https://app.codecov.io/gh/getmoto/moto/pull/7971/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getmoto) | `94.34% <100.00%> (+<0.01%)` | :arrow_up: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getmoto#carryforward-flags-in-the-pull-request-comment) to find out more.

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/getmoto/moto/pull/7971?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getmoto).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getmoto).


Patch:
diff --git a/docs/docs/configuration/index.rst b/docs/docs/configuration/index.rst
index ba0b50358..80486c6a1 100644
--- a/docs/docs/configuration/index.rst
+++ b/docs/docs/configuration/index.rst
@@ -58,9 +58,10 @@ Whitelist Services
 The `mock_aws` decorator will allow requests to all supported services. If you want to restrict this to only specific services, you can configure a service whitelist.
 For example, if you only want your application to use DynamoDB and S3:
 
-```
-mock_aws(config={"core": {"service_whitelist": ["dynamodb", "s3"]}})
-```
+.. sourcecode:: python
+
+    @mock_aws(config={"core": {"service_whitelist": ["dynamodb", "s3"]}})
+
 
 If the application under test tries to access any other services, Moto will throw a `ServiceNotWhitelisted`-exception.
 
diff --git a/docs/docs/getting_started.rst b/docs/docs/getting_started.rst
index 498237fbe..412ca1f00 100644
--- a/docs/docs/getting_started.rst
+++ b/docs/docs/getting_started.rst
@@ -231,7 +231,7 @@ You need to ensure that the mocks are actually in place.
 
 .. note:: By default, the region must be one supported by AWS, see :ref:`Can I mock the default AWS region?` for how to change this.
 
-Example on usage
+Pytest Fixtures Example Usage
 ~~~~~~~~~~~~~~~~
 If you are a user of `pytest`_, you can leverage `pytest fixtures`_ to help set up your mocks and other AWS resources that you would need.
 
@@ -249,35 +249,51 @@ Here is an example:
         os.environ["AWS_DEFAULT_REGION"] = "us-east-1"
 
     @pytest.fixture(scope="function")
-    def aws(aws_credentials):
+    def s3(aws_credentials):
+        """
+        Return a mocked S3 client
+        """
         with mock_aws():
             yield boto3.client("s3", region_name="us-east-1")
 
+    @pytest.fixture(scope="function")
+    def mocked_aws(aws_credentials):
+        """
+        Mock all AWS interactions
+        Requires you to create your own boto3 clients
+        """
+        with mock_aws():
+            yield
+
     @pytest.fixture
-    def create_bucket1(aws):
-        boto3.client("s3").create_bucket(Bucket="bb1")
+    def create_bucket1(s3):
+        s3.create_bucket(Bucket="bb1")
 
     @pytest.fixture
-    def create_bucket2(aws):
-        boto3.client("s3").create_bucket(Bucket="bb2")
+    def create_bucket2(s3):
+        s3.create_bucket(Bucket="bb2")
 
-    def test_s3_directly(aws):
+    def test_s3_bucket_creation(s3):
         s3.create_bucket(Bucket="somebucket")
 
         result = s3.list_buckets()
         assert len(result["Buckets"]) == 1
 
-    def test_bucket_creation(create_bucket1, create_bucket2):
+    def test_s3_bucket_creation_through_fixtures(create_bucket1, create_bucket2):
         result = boto3.client("s3").list_buckets()
         assert len(result["Buckets"]) == 2
 
+    def test_generic_aws_fixture(mocked_aws):
+        s3_client = boto3.client("s3")
+        s3_client.create_bucket(Bucket="somebucket")
+
 
-In the code sample above, all of the AWS/mocked fixtures take in a parameter of `aws_credentials`,
-which sets the proper fake environment variables. The fake environment variables are used so that `botocore` doesn't try to locate real
-credentials on your system.
+In the code sample above, all of the AWS/mocked fixtures (indirectly) use `aws_credentials`,
+which sets the proper fake environment variables. This is recommended to ensure that `botocore` doesn't try to use any real
+credentials.
 
 With Moto activated within the fixture, we can pass it to a test-method to ensure that any other AWS-calls are also mocked inside that test method.
-We can also combine multiple fixtures that use the same Moto-fixture.
+We can also combine multiple fixtures.
 
 Moto will delete any data after the mock ends, so the state is not shared across methods.
 
diff --git a/docs/docs/server_mode.rst b/docs/docs/server_mode.rst
index b28f83426..3e8f1fc50 100644
--- a/docs/docs/server_mode.rst
+++ b/docs/docs/server_mode.rst
@@ -62,6 +62,27 @@ By default, this server will start on port 5000, but this is configurable.
     ...
     server.stop()
 
+
+If you're using pytest, you can create this as a fixture:
+
+.. code-block:: python
+
+    # Note: pick an appropriate fixture "scope" for your use case
+    @pytest.fixture(scope="module")
+    def moto_server():
+        """Fixture to run a mocked AWS server for testing."""
+        # Note: pass `port=0` to get a random free port.
+        server = ThreadedMotoServer(port=0)
+        server.start()
+        host, port = server.get_host_and_port()
+        yield f"http://{host}:{port}"
+        server.stop()
+
+    def test_s3_using_moto_fixture(moto_server):
+        client = boto3.client("s3", endpoint_url=moto_server)
+        client.list_buckets()
+
+
 Note that the ThreadedMotoServer and the decorators act on the same state, making it possible to combine the two approaches.  :raw-html:`<br />`
 See the following example:
 
diff --git a/moto/core/base_backend.py b/moto/core/base_backend.py
index 62564f03e..59c92a851 100644
--- a/moto/core/base_backend.py
+++ b/moto/core/base_backend.py
@@ -12,6 +12,7 @@ from typing import (
     Iterator,
     List,
     Optional,
+    Tuple,
     TypeVar,
 )
 from uuid import uuid4
@@ -380,3 +381,12 @@ class BackendDict(Dict[str, AccountSpecificBackend[SERVICE_BACKEND]]):
                     additional_regions=self._additional_regions,
                 )
                 BackendDict._instances.append(self)  # type: ignore[misc]
+
+    def iter_backends(self) -> Iterator[Tuple[str, str, BaseBackend]]:
+        """
+        Iterate over a flattened view of all base backends in a BackendDict.
+        Each record is a tuple of account id, region name, and the base backend within that account and region.
+        """
+        for account_id, account_specific_backend in self.items():
+            for region_name, backend in account_specific_backend.items():
+                yield account_id, region_name, backend
diff --git a/moto/moto_server/threaded_moto_server.py b/moto/moto_server/threaded_moto_server.py
index 32f9da34e..73b3f8935 100644
--- a/moto/moto_server/threaded_moto_server.py
+++ b/moto/moto_server/threaded_moto_server.py
@@ -1,5 +1,5 @@
 from threading import Event, Thread
-from typing import Optional
+from typing import Optional, Tuple
 
 from werkzeug.serving import BaseWSGIServer, make_server
 
@@ -34,6 +34,11 @@ class ThreadedMotoServer:
         self._thread.start()
         self._server_ready_event.wait()
 
+    def get_host_and_port(self) -> Tuple[str, int]:
+        assert self._server is not None, "Make sure to call start() first"
+        host, port = self._server.server_address
+        return (str(host), port)
+
     def stop(self) -> None:
         self._server_ready_event.clear()
         if self._server:


Test patch:
diff --git a/tests/test_core/test_backenddict.py b/tests/test_core/test_backenddict.py
index 6df660b61..8f8672068 100644
--- a/tests/test_core/test_backenddict.py
+++ b/tests/test_core/test_backenddict.py
@@ -197,3 +197,16 @@ def test_global_region_defaults_to_aws() -> None:
 
     assert "aws" in s3[DEFAULT_ACCOUNT_ID]
     assert "global" in s3[DEFAULT_ACCOUNT_ID]
+
+
+def test_iterate_backend_dict() -> None:
+    ec2 = BackendDict(EC2Backend, "ec2")
+    _ = ec2[DEFAULT_ACCOUNT_ID]["us-east-1"]
+    _ = ec2[DEFAULT_ACCOUNT_ID]["us-east-2"]
+
+    regions = {"us-east-1", "us-east-2"}
+    for account, region, backend in ec2.iter_backends():
+        assert isinstance(backend, EC2Backend)
+        assert region in regions
+        regions.remove(region)
+        assert account == DEFAULT_ACCOUNT_ID
diff --git a/tests/test_core/test_server.py b/tests/test_core/test_server.py
index 2388e8f6d..bcf5eebe7 100644
--- a/tests/test_core/test_server.py
+++ b/tests/test_core/test_server.py
@@ -1,6 +1,10 @@
+from typing import Iterable
 from unittest.mock import Mock, patch
 
-from moto.server import main
+import boto3
+import pytest
+
+from moto.server import ThreadedMotoServer, main
 
 
 def test_wrong_arguments() -> None:
@@ -28,3 +32,19 @@ def test_port_argument(run_simple: Mock) -> None:  # type: ignore[misc]
     func_call = run_simple.call_args[0]
     assert func_call[0] == "127.0.0.1"
     assert func_call[1] == 8080
+
+
+@pytest.fixture(scope="module")
+def moto_server() -> Iterable[str]:
+    """Fixture to run a mocked AWS server for testing."""
+    # Note: pass `port=0` to get a random free port.
+    server = ThreadedMotoServer(port=0)
+    server.start()
+    host, port = server.get_host_and_port()
+    yield f"http://{host}:{port}"
+    server.stop()
+
+
+def test_s3_using_moto_fixture(moto_server: str) -> None:  # pylint: disable=redefined-outer-name
+    client = boto3.client("s3", endpoint_url=moto_server)
+    client.list_buckets()