Repository: getmoto/moto
Pull request #6330 (2023-05-17 11:16:18)

SNS: Cross-account access for topics
This PR introduces cross-account support for SNS topics.

This basically means that topics will now be retrieved from the appropriate backend based on the value of account ID and region in the ARN. Any account can now retrieve an SNS topic with its ARN.

Cross-account access now works for `GetTopicAttributes`, `SetTopicAttributes`, `AddPermission`, `RemovePermission`, `DeleteTopic`, `Subscribe`, `ListSubscriptionsByTopic`, `Publish`, similar to AWS. 

IAM enforcement is left out of scope.

codecov[bot]: ## [Codecov](https://app.codecov.io/gh/getmoto/moto/pull/6330?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) Report
> Merging [#6330](https://app.codecov.io/gh/getmoto/moto/pull/6330?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) (d8d89fc) into [master](https://app.codecov.io/gh/getmoto/moto/commit/61c475fef7eebc260b4369547aeea086f5208b6a?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) (61c475f) will **decrease** coverage by `0.01%`.
> The diff coverage is `100.00%`.

```diff
@@            Coverage Diff             @@
##           master    #6330      +/-   ##
==========================================
- Coverage   96.27%   96.26%   -0.01%     
==========================================
  Files         792      793       +1     
  Lines       77221    77223       +2     
==========================================
+ Hits        74341    74342       +1     
- Misses       2880     2881       +1     
```

| Flag | Coverage Δ | |
|---|---|---|
| servertests | `36.92% <17.24%> (+<0.01%)` | :arrow_up: |
| unittests | `96.21% <100.00%> (-0.01%)` | :arrow_down: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#carryforward-flags-in-the-pull-request-comment) to find out more.

| [Impacted Files](https://app.codecov.io/gh/getmoto/moto/pull/6330?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) | Coverage Δ | |
|---|---|---|
| [moto/events/models.py](https://app.codecov.io/gh/getmoto/moto/pull/6330?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-bW90by9ldmVudHMvbW9kZWxzLnB5) | `94.02% <100.00%> (-0.10%)` | :arrow_down: |
| [moto/sns/models.py](https://app.codecov.io/gh/getmoto/moto/pull/6330?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-bW90by9zbnMvbW9kZWxzLnB5) | `97.79% <100.00%> (+<0.01%)` | :arrow_up: |
| [moto/utilities/arns.py](https://app.codecov.io/gh/getmoto/moto/pull/6330?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-bW90by91dGlsaXRpZXMvYXJucy5weQ==) | `100.00% <100.00%> (ø)` | |

... and [1 file with indirect coverage changes](https://app.codecov.io/gh/getmoto/moto/pull/6330/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None)



Patch:
diff --git a/docs/docs/getting_started.rst b/docs/docs/getting_started.rst
index b11da7c83..df50a696d 100644
--- a/docs/docs/getting_started.rst
+++ b/docs/docs/getting_started.rst
@@ -1,5 +1,10 @@
 .. _getting_started:
 
+
+.. role:: raw-html(raw)
+    :format: html
+
+
 =========================
 Getting Started with Moto
 =========================
@@ -223,7 +228,7 @@ You need to ensure that the mocks are actually in place.
 
  #. **VERY IMPORTANT**: ensure that you have your mocks set up *BEFORE* your `boto3` client is established.
     This can typically happen if you import a module that has a `boto3` client instantiated outside of a function.
-    See the pesky imports section below on how to work around this.
+    See :ref:`pesky_imports_section` below on how to work around this.
 
 .. note:: By default, the region must be one supported by AWS, see :ref:`Can I mock the default AWS region?` for how to change this.
 
@@ -267,17 +272,30 @@ Next, once you need to do anything with the mocked AWS environment, do something
         assert len(result["Buckets"]) == 1
         assert result["Buckets"][0]["Name"] == "somebucket"
 
+
+.. _pesky_imports_section:
+
 What about those pesky imports
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Recall earlier, it was mentioned that mocks should be established __BEFORE__ the clients are set up. One way
-to avoid import issues is to make use of local Python imports -- i.e. import the module inside of the unit
-test you want to run vs. importing at the top of the file.
+As mentioned earlier, mocks should be established __BEFORE__ the clients are set up.
+
+Some background on why this is necessary:  :raw-html:`<br />`
+Moto intercepts HTTP requests using a custom event handler that hooks into botocore's event-system.  :raw-html:`<br />`
+When creating clients/resources, `boto3` gathers all event handlers that have been registered at that point, and injects those handlers into the created client/resource. Event handlers registered after a client is created, are not used.
+
+The `moto.core`-package registers our event handler on initialization. So to be pedantic: `moto.core` should be imported before a client is created, in order for `boto3` to call our custom handler and therefore for Moto to be active.  :raw-html:`<br />`
+The easiest way to ensure this happens, is to establish a mock before the clients are setup, as `moto.core` is imported when the mock starts.
+
+
+One way to avoid import issues is to make use of local Python imports -- i.e. import the module that creates boto3-clients inside of the unit test you want to run.
 
 Example:
 
 .. sourcecode:: python
 
     def test_something(s3):
+        # s3 is a fixture defined above that yields a boto3 s3 client.
+        
         from some.package.that.does.something.with.s3 import some_func # <-- Local import for unit test
         # ^^ Importing here ensures that the mock has been established.
 
diff --git a/docs/docs/server_mode.rst b/docs/docs/server_mode.rst
index 018fd8d04..992172688 100644
--- a/docs/docs/server_mode.rst
+++ b/docs/docs/server_mode.rst
@@ -93,6 +93,7 @@ Run using Docker
 ----------------------
 You could also use the official Docker image.
 This is stored in two different repositories:
+
  - https://hub.docker.com/r/motoserver/moto/tags
  - https://github.com/getmoto/moto/pkgs/container/motoserver
 
diff --git a/moto/core/responses.py b/moto/core/responses.py
index 8fc794261..a50aa80a4 100644
--- a/moto/core/responses.py
+++ b/moto/core/responses.py
@@ -306,7 +306,10 @@ class BaseResponse(_TemplateEnvironmentMixin, ActionAuthenticatorMixin):
         self.headers = request.headers
         if "host" not in self.headers:
             self.headers["host"] = urlparse(full_url).netloc
-        self.response_headers = {"server": "amazon.com"}
+        self.response_headers = {
+            "server": "amazon.com",
+            "date": datetime.datetime.now().strftime("%a, %d %b %Y %H:%M:%S GMT"),
+        }
 
         # Register visit with IAM
         from moto.iam.models import mark_account_as_visited
diff --git a/moto/events/models.py b/moto/events/models.py
index 767d51864..fbc5ba4a6 100644
--- a/moto/events/models.py
+++ b/moto/events/models.py
@@ -4,7 +4,6 @@ import re
 import json
 import sys
 import warnings
-from collections import namedtuple
 from datetime import datetime
 from enum import Enum, unique
 from json import JSONDecodeError
@@ -27,6 +26,7 @@ from moto.events.exceptions import (
     IllegalStatusException,
 )
 from moto.moto_api._internal import mock_random as random
+from moto.utilities.arns import parse_arn
 from moto.utilities.paginator import paginate
 from moto.utilities.tagging_service import TaggingService
 
@@ -37,10 +37,6 @@ UNDEFINED = object()
 
 
 class Rule(CloudFormationModel):
-    Arn = namedtuple(
-        "Arn", ["account", "region", "service", "resource_type", "resource_id"]
-    )
-
     def __init__(
         self,
         name: str,
@@ -123,13 +119,13 @@ class Rule(CloudFormationModel):
         # - SQS Queue + FIFO Queue
         # - Cross-region/account EventBus
         for target in self.targets:
-            arn = self._parse_arn(target["Arn"])
+            arn = parse_arn(target["Arn"])
 
             if arn.service == "logs" and arn.resource_type == "log-group":
                 self._send_to_cw_log_group(arn.resource_id, event)
             elif arn.service == "events" and not arn.resource_type:
                 input_template = json.loads(target["InputTransformer"]["InputTemplate"])
-                archive_arn = self._parse_arn(input_template["archive-arn"])
+                archive_arn = parse_arn(input_template["archive-arn"])
 
                 self._send_to_events_archive(archive_arn.resource_id, event)
             elif arn.service == "sqs":
@@ -149,33 +145,6 @@ class Rule(CloudFormationModel):
             else:
                 raise NotImplementedError(f"Expr not defined for {type(self)}")
 
-    def _parse_arn(self, arn: str) -> Arn:
-        # http://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html
-        # this method needs probably some more fine tuning,
-        # when also other targets are supported
-        _, _, service, region, account, resource = arn.split(":", 5)
-
-        if ":" in resource and "/" in resource:
-            if resource.index(":") < resource.index("/"):
-                resource_type, resource_id = resource.split(":", 1)
-            else:
-                resource_type, resource_id = resource.split("/", 1)
-        elif ":" in resource:
-            resource_type, resource_id = resource.split(":", 1)
-        elif "/" in resource:
-            resource_type, resource_id = resource.split("/", 1)
-        else:
-            resource_type = None
-            resource_id = resource
-
-        return self.Arn(
-            account=account,
-            region=region,
-            service=service,
-            resource_type=resource_type,
-            resource_id=resource_id,
-        )
-
     def _send_to_cw_log_group(self, name: str, event: Dict[str, Any]) -> None:
         from moto.logs import logs_backends
 
diff --git a/moto/iam/responses.py b/moto/iam/responses.py
index ac321c912..60ab3b508 100644
--- a/moto/iam/responses.py
+++ b/moto/iam/responses.py
@@ -1738,6 +1738,10 @@ GET_GROUP_TEMPLATE = """<GetGroupResponse>
             <UserName>{{ user.name }}</UserName>
             <UserId>{{ user.id }}</UserId>
             <Arn>{{ user.arn }}</Arn>
+            <CreateDate>{{ user.created_iso_8601 }}</CreateDate>
+            {% if user.password_last_used_iso_8601 %}
+              <PasswordLastUsed>{{ user.password_last_used_iso_8601 }}</PasswordLastUsed>
+            {% endif %}
           </member>
         {% endfor %}
       </Users>
diff --git a/moto/sns/models.py b/moto/sns/models.py
index 545d72227..4b87649aa 100644
--- a/moto/sns/models.py
+++ b/moto/sns/models.py
@@ -34,7 +34,7 @@ from .utils import (
     is_e164,
     FilterPolicyMatcher,
 )
-
+from moto.utilities.arns import parse_arn
 
 DEFAULT_PAGE_SIZE = 100
 MAXIMUM_MESSAGE_LENGTH = 262144  # 256 KiB
@@ -486,13 +486,15 @@ class SNSBackend(BaseBackend):
         try:
             topic = self.get_topic(arn)
             self.delete_topic_subscriptions(topic)
-            self.topics.pop(arn)
+            parsed_arn = parse_arn(arn)
+            sns_backends[parsed_arn.account][parsed_arn.region].topics.pop(arn, None)
         except KeyError:
             raise SNSNotFoundError(f"Topic with arn {arn} not found")
 
     def get_topic(self, arn: str) -> Topic:
+        parsed_arn = parse_arn(arn)
         try:
-            return self.topics[arn]
+            return sns_backends[parsed_arn.account][parsed_arn.region].topics[arn]
         except KeyError:
             raise SNSNotFoundError(f"Topic with arn {arn} not found")
 
@@ -932,10 +934,8 @@ class SNSBackend(BaseBackend):
         aws_account_ids: List[str],
         action_names: List[str],
     ) -> None:
-        if topic_arn not in self.topics:
-            raise SNSNotFoundError("Topic does not exist")
-
-        policy = self.topics[topic_arn]._policy_json
+        topic = self.get_topic(topic_arn)
+        policy = topic._policy_json
         statement = next(
             (
                 statement
@@ -964,18 +964,16 @@ class SNSBackend(BaseBackend):
             "Resource": topic_arn,
         }
 
-        self.topics[topic_arn]._policy_json["Statement"].append(statement)
+        topic._policy_json["Statement"].append(statement)
 
     def remove_permission(self, topic_arn: str, label: str) -> None:
-        if topic_arn not in self.topics:
-            raise SNSNotFoundError("Topic does not exist")
-
-        statements = self.topics[topic_arn]._policy_json["Statement"]
+        topic = self.get_topic(topic_arn)
+        statements = topic._policy_json["Statement"]
         statements = [
             statement for statement in statements if statement["Sid"] != label
         ]
 
-        self.topics[topic_arn]._policy_json["Statement"] = statements
+        topic._policy_json["Statement"] = statements
 
     def list_tags_for_resource(self, resource_arn: str) -> Dict[str, str]:
         if resource_arn not in self.topics:
diff --git a/moto/utilities/arns.py b/moto/utilities/arns.py
new file mode 100644
index 000000000..51a6753ac
--- /dev/null
+++ b/moto/utilities/arns.py
@@ -0,0 +1,33 @@
+from collections import namedtuple
+
+Arn = namedtuple(
+    "Arn", ["account", "region", "service", "resource_type", "resource_id"]
+)
+
+
+def parse_arn(arn: str) -> Arn:
+    # http://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html
+    # this method needs probably some more fine tuning,
+    # when also other targets are supported
+    _, _, service, region, account, resource = arn.split(":", 5)
+
+    if ":" in resource and "/" in resource:
+        if resource.index(":") < resource.index("/"):
+            resource_type, resource_id = resource.split(":", 1)
+        else:
+            resource_type, resource_id = resource.split("/", 1)
+    elif ":" in resource:
+        resource_type, resource_id = resource.split(":", 1)
+    elif "/" in resource:
+        resource_type, resource_id = resource.split("/", 1)
+    else:
+        resource_type = None
+        resource_id = resource
+
+    return Arn(
+        account=account,
+        region=region,
+        service=service,
+        resource_type=resource_type,
+        resource_id=resource_id,
+    )


Test patch:
diff --git a/tests/test_core/test_responses.py b/tests/test_core/test_responses.py
index 728f6012d..f306348ff 100644
--- a/tests/test_core/test_responses.py
+++ b/tests/test_core/test_responses.py
@@ -1,3 +1,4 @@
+import datetime
 from unittest import SkipTest, mock
 import sure  # noqa # pylint: disable=unused-import
 
@@ -8,6 +9,7 @@ from botocore.awsrequest import AWSPreparedRequest
 from moto.core.responses import AWSServiceSpec, BaseResponse
 from moto.core.responses import flatten_json_request_body
 from moto import settings
+from freezegun import freeze_time
 
 
 def test_flatten_json_request_body():
@@ -225,3 +227,21 @@ def test_jinja_render_prettify(m_env_var):
     xml_string = template.render()
     assert xml_string == expected_output
     assert m_env_var
+
+
+def test_response_metadata():
+    # Setup
+    frozen_time = datetime.datetime(
+        2023, 5, 20, 10, 20, 30, tzinfo=datetime.timezone.utc
+    )
+    request = AWSPreparedRequest("GET", "http://request", {}, None, False)
+
+    # Execute
+    with freeze_time(frozen_time):
+        bc = BaseResponse()
+        bc.setup_class(request, request.url, request.headers)
+
+    # Verify
+    assert "date" in bc.response_headers
+    if not settings.TEST_SERVER_MODE:
+        assert bc.response_headers["date"] == "Sat, 20 May 2023 10:20:30 GMT"
diff --git a/tests/test_iam/test_iam_groups.py b/tests/test_iam/test_iam_groups.py
index 332d4cf56..58976757f 100644
--- a/tests/test_iam/test_iam_groups.py
+++ b/tests/test_iam/test_iam_groups.py
@@ -6,8 +6,11 @@ import json
 
 import pytest
 from botocore.exceptions import ClientError
-from moto import mock_iam
+from moto import mock_iam, settings
 from moto.core import DEFAULT_ACCOUNT_ID as ACCOUNT_ID
+from moto.backends import get_backend
+from freezegun import freeze_time
+from dateutil.tz import tzlocal
 
 MOCK_POLICY = """
 {
@@ -107,10 +110,34 @@ def test_add_user_to_unknown_group():
 
 @mock_iam
 def test_add_user_to_group():
-    conn = boto3.client("iam", region_name="us-east-1")
-    conn.create_group(GroupName="my-group")
-    conn.create_user(UserName="my-user")
-    conn.add_user_to_group(GroupName="my-group", UserName="my-user")
+    # Setup
+    frozen_time = datetime(2023, 5, 20, 10, 20, 30, tzinfo=tzlocal())
+
+    group = "my-group"
+    user = "my-user"
+    with freeze_time(frozen_time):
+        conn = boto3.client("iam", region_name="us-east-1")
+        conn.create_group(GroupName=group)
+        conn.create_user(UserName=user)
+        conn.add_user_to_group(GroupName=group, UserName=user)
+
+        # use internal api to set password, doesn't work in servermode
+        if not settings.TEST_SERVER_MODE:
+            iam_backend = get_backend("iam")[ACCOUNT_ID]["global"]
+            iam_backend.users[user].password_last_used = datetime.utcnow()
+    # Execute
+    result = conn.get_group(GroupName=group)
+
+    # Verify
+    assert len(result["Users"]) == 1
+
+    # if in servermode then we can't test for password because we can't
+    # manipulate the backend with internal an api
+    if settings.TEST_SERVER_MODE:
+        assert "CreateDate" in result["Users"][0]
+        return
+    assert result["Users"][0]["CreateDate"] == frozen_time
+    assert result["Users"][0]["PasswordLastUsed"] == frozen_time
 
 
 @mock_iam
diff --git a/tests/test_sns/test_topics_boto3.py b/tests/test_sns/test_topics_boto3.py
index 76844c8aa..c0bcade9b 100644
--- a/tests/test_sns/test_topics_boto3.py
+++ b/tests/test_sns/test_topics_boto3.py
@@ -96,9 +96,9 @@ def test_create_topic_should_be_indempodent():
 @mock_sns
 def test_get_missing_topic():
     conn = boto3.client("sns", region_name="us-east-1")
-    conn.get_topic_attributes.when.called_with(TopicArn="a-fake-arn").should.throw(
-        ClientError
-    )
+    conn.get_topic_attributes.when.called_with(
+        TopicArn="arn:aws:sns:us-east-1:424242424242:a-fake-arn"
+    ).should.throw(ClientError)
 
 
 @mock_sns
@@ -360,7 +360,10 @@ def test_add_permission_errors():
         Label="test-2",
         AWSAccountId=["999999999999"],
         ActionName=["AddPermission"],
-    ).should.throw(ClientError, "Topic does not exist")
+    ).should.throw(
+        ClientError,
+        f"An error occurred (NotFound) when calling the AddPermission operation: Topic with arn {topic_arn + '-not-existing'} not found",
+    )
 
     client.add_permission.when.called_with(
         TopicArn=topic_arn,
@@ -383,7 +386,10 @@ def test_remove_permission_errors():
 
     client.remove_permission.when.called_with(
         TopicArn=topic_arn + "-not-existing", Label="test"
-    ).should.throw(ClientError, "Topic does not exist")
+    ).should.throw(
+        ClientError,
+        f"An error occurred (NotFound) when calling the RemovePermission operation: Topic with arn {topic_arn + '-not-existing'} not found",
+    )
 
 
 @mock_sns