Repository: getmoto/moto

Issue:
Add a @moto.mock_all
I've been writing some integration tests for a function which chains together a bunch of different AWS tasks, and the list of `@moto.mock_…` decorators just keeps on growing

```python
@moto.mock_autoscaling
@moto.mock_cloudformation
@moto.mock_ec2
@moto.mock_ecr
@moto.mock_ecs
@moto.mock_ssm
@pytest.mark.django_db
def test_do_the_thing_end_to_end(the_job):
    """End-to-end test."""
    tasks.do_the_thing.push_request(id='request_id')
    tasks.do_the_thing.run(the_job.id)
    the_job.refresh_from_db()
    assert the_job.is_finished is True
```

There's nothing wrong with this, I guess, but I think it would be nicer if I could collapse all of these decorators into one line. Also, it would be nice if I didn't need to update this test when something inside of `tasks.do_the_thing` started dealing with some other AWS service.

Is there a reason there's no `@moto.mock_everything`?

Other possible names for this decorator might be `@moto.mock_all`, `@mock.mock`, `@moto.mock_aws`.


Discussion:
Related to this - it *looks like* if any code creates a `boto3.client(someservice)` that needs to be mocked for *ALL* tests in the same pytest session
@kimvais Sorry, but I'm not sure what you mean and how it's related to this suggestion. The following code tests do what I'd expect: one sends requests to mocked S3, and the other sends requests to real S3:

```python
import boto3
import moto
import pytest


def create_and_list():
    s3 = boto3.client('s3')
    s3.create_bucket(
        Bucket='craigs-cool-bucket',
        CreateBucketConfiguration={'LocationConstraint': 'eu-west-2'},
    )
    return s3.list_buckets()


@moto.mock_s3
def test_with_mocked_s3():
    create_and_list()


def test_with_real_s3():
    create_and_list()
```

(edited to use a client instead of a resource, but the same behaviour stands)

![screenshot 2019-02-28 at 10 15 20](https://user-images.githubusercontent.com/76330/53559482-15bb7d80-3b42-11e9-8795-53739427af3e.png)
@craiga Hi, and sorry for being vague - but we are struggling with moto not working at all in some tests in a quite a bit more complicated setup:

- there's a helper function in a library (we use this in multiple projects) that that *should* do pretty much the same as the `mock_everything` above (calling mock_<service>.start()
- function is imported in `conftest.py` and called inside a `mock_aws` yield_fixture
- code being tested is Flask API endpoints that create various `boto3.client(<service>'` and do things with them
- Flask's app.get_test_client() is used to call POST/PUT/DELETE etc. on the endpoints

In debugger, the mocks get started correctly, but in the actual endpoint code the boto3.client() connects to actual AWS e.g. creates logwatch events instead of mocking them via moto.
@kimvais Ah, I see. Any chance you could share your `mock_aws` code?

I'm not familiar with how it works, but I'd guess that the Werkzeug test client you get from Flask is running your app in a place where it can no longer see your mocks. http://werkzeug.pocoo.org/docs/0.14/test/ might shed a little more light on what's happening in your situation. I'd suggest calling your functions directly if that's possible.
@kimvais please review #1926 to see if it helps your issue. I needed to add `boto3.setup_default_session()` after moto starts but before the boto3 clients are created in order for mocks to work properly.
@kgutwin - for me `boto3.setup_default_session()` fix doesn't work for S3 & @mock_s3 (moto v1.3.7, boto3 v1.9.115, botocore v1.12.115). Tests still are trying to access AWS. I have tried every possibility - different orders of moto/boto imports, `boto3.setup_default_session()` called just after boto3 import, before client regeneration and still no luck.

Anyone knows whether we can expect any upcoming moto release that would contain fixes to address the issues of moto (or rather imperfectly mocked boto in fact) accessing AWS?
A lot of issues were raised related to this topic but I cannot see any definite solution or whether any of them fixed the problem in moto itself (#1926, #2069, #2076, #1998 etc).
@pskowronek thanks for the report. Could you raise a new issue with a minimal code example that reproduces your problem? So far `boto3.setup_default_session()` is working for me, but I'd be curious to see examples where it doesn't fix the issue.
@kgutwin 
> Could you raise a new issue with a minimal code example that reproduces your problem? 

done, #2118


Yes, something like a `@moto.mock_everything` would be welcome! My preference is probably `mock_all`, but I don't have a strong opinion.

Setting this as a feature request. If anyone has time, a PR would be welcome.

Resolving patch:
diff --git a/moto/__init__.py b/moto/__init__.py
--- a/moto/__init__.py
+++ b/moto/__init__.py
@@ -1,4 +1,5 @@
 import importlib
+import sys
 
 
 def lazy_load(
@@ -165,6 +166,25 @@ def f(*args, **kwargs):
 mock_efs = lazy_load(".efs", "mock_efs")
 mock_wafv2 = lazy_load(".wafv2", "mock_wafv2")
 
+
+def mock_all():
+    dec_names = [
+        d
+        for d in dir(sys.modules["moto"])
+        if d.startswith("mock_")
+        and not d.endswith("_deprecated")
+        and not d == "mock_all"
+    ]
+
+    def deco(f):
+        for dec_name in reversed(dec_names):
+            dec = globals()[dec_name]
+            f = dec(f)
+        return f
+
+    return deco
+
+
 # import logging
 # logging.getLogger('boto').setLevel(logging.CRITICAL)
 
diff --git a/scripts/implementation_coverage.py b/scripts/implementation_coverage.py
--- a/scripts/implementation_coverage.py
+++ b/scripts/implementation_coverage.py
@@ -238,6 +238,18 @@ def test_{service_name}_behaviour:
         file.write("====================\n")
         file.write("\n")
         file.write("Please see a list of all currently supported services. Each service will have a list of the endpoints that are implemented.\n")
+        file.write("Each service will also have an example on how to mock an individual service.\n\n")
+        file.write("Note that you can mock multiple services at the same time:\n\n")
+        file.write(".. sourcecode:: python\n\n")
+        file.write("    @mock_s3\n")
+        file.write("    @mock_sqs\n")
+        file.write("    def test_both_s3_and_sqs():\n")
+        file.write("        ...\n")
+        file.write("\n\n")
+        file.write(".. sourcecode:: python\n\n")
+        file.write("    @mock_all\n")
+        file.write("    def test_all_supported_services_at_the_same_time():\n")
+        file.write("        ...\n")
         file.write("\n")
 
         file.write("\n")


Test patch:
diff --git a/tests/test_core/test_mock_all.py b/tests/test_core/test_mock_all.py
new file mode 100644
--- /dev/null
+++ b/tests/test_core/test_mock_all.py
@@ -0,0 +1,20 @@
+import boto3
+import sure  # noqa # pylint: disable=unused-import
+
+from moto import mock_all
+
+
+@mock_all()
+def test_multiple_services():
+    rgn = "us-east-1"
+    sqs = boto3.client("sqs", region_name=rgn)
+    r = sqs.list_queues()  #
+    r["ResponseMetadata"]["HTTPStatusCode"].should.equal(200)
+
+    lmbda = boto3.client("lambda", region_name=rgn)
+    r = lmbda.list_event_source_mappings()
+    r["ResponseMetadata"]["HTTPStatusCode"].should.equal(200)
+
+    ddb = boto3.client("dynamodb", region_name=rgn)
+    r = ddb.list_tables()
+    r["ResponseMetadata"]["HTTPStatusCode"].should.equal(200)