Repository: getmoto/moto

Issue:
moto imports `docker` unconditionally when SQS is used
# What
`docker` seems to be required when using SQS. The `docker` dependencies is however not intrinsic to the package.

Using `moto=2.0.0` and installed in `tox` via `pip install moto`. 

I am experiencing the following error 

```
    import docker
ModuleNotFoundError: No module named 'docker'
```

# What is expected to happen

* No such `docker` error in a clean environment
* No need for `docker` when no feature needing `docker` are used

# Workaround

Install `docker` dependency manually. `pip install docker` fixes it.

# Possible solutions

* make the `docker` dependency explicit when SQS is used or for the `all`
* move imports inside functions to load them only when really required

# Description

First of all, thank you again for this package, it is very helpful.
My code uses the `SQS extended client` and has mocked S3 and SQS. Inside a clean environment, I came up with this issue that is easy to solve but might be confusing for your users.
The error happens when simply sending a message on an SQS queue inside the clean environment.

# Traceback
The following traceback runs part of my code. This fires an error because apparently `moto` tries to run Lambda, which in turns requires `docker` even if no lambda is executed at all.

On the lines:
```
File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/moto/sqs/models.py", line 708, in send_message
    queue.add_message(message)
```

Full traceback
```
Traceback (most recent call last):
  File "/data/code/reasonal/reliability-next/ml_services/ml_reliability_workers/polling/tests/test_checks.py", line 87, in test_check_send_message
    check_send_long_msg()
  File "/data/code/reasonal/reliability-next/ml_services/ml_reliability_workers/polling/check.py", line 59, in check_send_long_msg
    send_message(queue=queue_, message=json.dumps(dict_document))
  File "/data/code/reasonal/reliability-next/ml_services/ml_reliability_workers/aws_utils/sqs_utils.py", line 95, in send_message
    response = queue.send_message(
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/sqs_extended_client/session.py", line 256, in _send_message
    return func(*args, **kwargs)
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/boto3/resources/factory.py", line 520, in do_action
    response = action(self, *args, **kwargs)
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/boto3/resources/action.py", line 83, in __call__
    response = getattr(parent.meta.client, operation_name)(*args, **params)
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/sqs_extended_client/session.py", line 256, in _send_message
    return func(*args, **kwargs)
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/botocore/client.py", line 662, in _make_api_call
    http, parsed_response = self._make_request(
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/botocore/client.py", line 682, in _make_request
    return self._endpoint.make_request(operation_model, request_dict)
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/botocore/endpoint.py", line 102, in make_request
    return self._send_request(request_dict, operation_model)
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/botocore/endpoint.py", line 136, in _send_request
    while self._needs_retry(attempts, operation_model, request_dict,
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/botocore/endpoint.py", line 253, in _needs_retry
    responses = self._event_emitter.emit(
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/botocore/hooks.py", line 356, in emit
    return self._emitter.emit(aliased_event_name, **kwargs)
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/botocore/hooks.py", line 228, in emit
    return self._emit(event_name, kwargs)
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/botocore/hooks.py", line 211, in _emit
    response = handler(**kwargs)
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/botocore/retryhandler.py", line 183, in __call__
    if self._checker(attempts, response, caught_exception):
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/botocore/retryhandler.py", line 250, in __call__
    should_retry = self._should_retry(attempt_number, response,
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/botocore/retryhandler.py", line 269, in _should_retry
    return self._checker(attempt_number, response, caught_exception)
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/botocore/retryhandler.py", line 316, in __call__
    checker_response = checker(attempt_number, response,
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/botocore/retryhandler.py", line 222, in __call__
    return self._check_caught_exception(
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/botocore/retryhandler.py", line 359, in _check_caught_exception
    raise caught_exception
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/botocore/endpoint.py", line 197, in _do_get_response
    responses = self._event_emitter.emit(event_name, request=request)
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/botocore/hooks.py", line 356, in emit
    return self._emitter.emit(aliased_event_name, **kwargs)
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/botocore/hooks.py", line 228, in emit
    return self._emit(event_name, kwargs)
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/botocore/hooks.py", line 211, in _emit
    response = handler(**kwargs)
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/moto/core/models.py", line 363, in __call__
    status, headers, body = response_callback(
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/moto/core/responses.py", line 202, in dispatch
    return cls()._dispatch(*args, **kwargs)
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/moto/core/responses.py", line 312, in _dispatch
    return self.call_action()
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/moto/core/utils.py", line 254, in _wrapper
    response = f(*args, **kwargs)
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/moto/core/utils.py", line 286, in _wrapper
    response = f(*args, **kwargs)
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/moto/sqs/responses.py", line 71, in call_action
    status_code, headers, body = super(SQSResponse, self).call_action()
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/moto/core/responses.py", line 402, in call_action
    response = method()
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/moto/sqs/responses.py", line 237, in send_message
    message = self.sqs_backend.send_message(
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/moto/sqs/models.py", line 708, in send_message
    queue.add_message(message)
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/moto/sqs/models.py", line 496, in add_message
    from moto.awslambda import lambda_backends
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/moto/awslambda/__init__.py", line 2, in <module>
    from .models import lambda_backends
  File "/data/code/reasonal/reliability-next/.tox/ml_reliability_workers/lib/python3.8/site-packages/moto/awslambda/models.py", line 10, in <module>
    import docker
ModuleNotFoundError: No module named 'docker'
```




Discussion:
Hi @raffienficiaud, the installation for moto 2.x has changed slightly - it's now required to specify the service on installation:
`pip install moto[sqs]`

That will install docker as a dependency for SQS.

It is true that it shouldn't be necessary though. Marking it as an enhancement to only require docker when actually sending a message to a Lambda function.
Thank you @bblommers I noticed the addendum to the release notes https://github.com/spulec/moto/issues/3722 but I read it like `pip install moto[all]` equivalent to `pip install moto`, which is apparently not the case!

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -4,6 +4,8 @@
 import os
 import re
 from setuptools import setup, find_packages
+import sys
+import moto.__init__ as service_list
 
 # Borrowed from pip at https://github.com/pypa/pip/blob/62c27dee45625e1b63d1e023b0656310f276e050/setup.py#L11-L15
 here = os.path.abspath(os.path.dirname(__file__))
@@ -69,10 +71,11 @@ def get_version():
 ]
 all_server_deps = all_extra_deps + ["flask", "flask-cors"]
 
-# TODO: do we want to add ALL services here?
-# i.e. even those without extra dependencies.
-# Would be good for future-compatibility, I guess.
-extras_per_service = {
+extras_per_service = {}
+for service_name in [service[5:] for service in dir(service_list) if service.startswith("mock_")]:
+    extras_per_service[service_name] = []
+extras_per_service.update(
+{
     "apigateway": [_dep_python_jose, _dep_python_jose_ecdsa_pin],
     "awslambda": [_dep_docker],
     "batch": [_dep_docker],
@@ -88,7 +91,7 @@ def get_version():
     # XRay module uses pkg_resources, but doesn't have an explicit dependency listed
     # This should be fixed in the next version: https://github.com/aws/aws-xray-sdk-python/issues/305
     "xray": [_dep_aws_xray_sdk, _setuptools],
-}
+})
 # When a Table has a Stream, we'll always need to import AWSLambda to search for a corresponding function to send the table data to
 extras_per_service["dynamodb2"] = extras_per_service["awslambda"]
 extras_per_service["dynamodbstreams"] = extras_per_service["awslambda"]


Test patch:
diff --git a/scripts/dependency_test.sh b/scripts/dependency_test.sh
--- a/scripts/dependency_test.sh
+++ b/scripts/dependency_test.sh
@@ -33,7 +33,7 @@ valid_service() {
   # Verify whether this is a valid service
   # We'll ignore metadata folders, and folders that test generic Moto behaviour
   # We'll also ignore CloudFormation, as it will always depend on other services
-  local ignore_moto_folders="core instance_metadata __pycache__ templates cloudformation utilities"
+  local ignore_moto_folders="core instance_metadata __pycache__ templates cloudformation packages utilities s3bucket_path"
   if echo $ignore_moto_folders | grep -q "$1"; then
     return 1
   else
diff --git a/tests/test_sqs/test_integration.py b/tests/test_sqs/test_integration.py
new file mode 100644
--- /dev/null
+++ b/tests/test_sqs/test_integration.py
@@ -0,0 +1,73 @@
+# -*- coding: utf-8 -*-
+from __future__ import unicode_literals
+
+import base64
+import json
+import time
+import uuid
+import hashlib
+
+import boto
+import boto3
+import sure  # noqa
+from moto import mock_sqs, mock_lambda, mock_logs
+
+from tests.test_awslambda.test_lambda import get_test_zip_file1, get_role_name
+
+
+@mock_logs
+@mock_lambda
+@mock_sqs
+def test_invoke_function_from_sqs_exception():
+    logs_conn = boto3.client("logs", region_name="us-east-1")
+    sqs = boto3.resource("sqs", region_name="us-east-1")
+    queue = sqs.create_queue(QueueName="test-sqs-queue1")
+
+    conn = boto3.client("lambda", region_name="us-east-1")
+    func = conn.create_function(
+        FunctionName="testFunction",
+        Runtime="python2.7",
+        Role=get_role_name(),
+        Handler="lambda_function.lambda_handler",
+        Code={"ZipFile": get_test_zip_file1()},
+        Description="test lambda function",
+        Timeout=3,
+        MemorySize=128,
+        Publish=True,
+    )
+
+    response = conn.create_event_source_mapping(
+        EventSourceArn=queue.attributes["QueueArn"], FunctionName=func["FunctionArn"]
+    )
+
+    assert response["EventSourceArn"] == queue.attributes["QueueArn"]
+    assert response["State"] == "Enabled"
+
+    entries = [
+        {
+            "Id": "1",
+            "MessageBody": json.dumps({"uuid": str(uuid.uuid4()), "test": "test"}),
+        }
+    ]
+
+    queue.send_messages(Entries=entries)
+
+    start = time.time()
+    while (time.time() - start) < 30:
+        result = logs_conn.describe_log_streams(logGroupName="/aws/lambda/testFunction")
+        log_streams = result.get("logStreams")
+        if not log_streams:
+            time.sleep(1)
+            continue
+        assert len(log_streams) >= 1
+
+        result = logs_conn.get_log_events(
+            logGroupName="/aws/lambda/testFunction",
+            logStreamName=log_streams[0]["logStreamName"],
+        )
+        for event in result.get("events"):
+            if "custom log event" in event["message"]:
+                return
+        time.sleep(1)
+
+    assert False, "Test Failed"
diff --git a/tests/test_sqs/test_sqs.py b/tests/test_sqs/test_sqs.py
--- a/tests/test_sqs/test_sqs.py
+++ b/tests/test_sqs/test_sqs.py
@@ -15,9 +15,10 @@
 from boto.sqs.message import Message, RawMessage
 from botocore.exceptions import ClientError
 from freezegun import freeze_time
-from moto import mock_sqs, mock_sqs_deprecated, mock_logs, settings
+from moto import mock_sqs, mock_sqs_deprecated, settings
 
 from unittest import SkipTest, mock
+
 import pytest
 from tests.helpers import requires_boto_gte
 from moto.core import ACCOUNT_ID