Repository: getmoto/moto

Issue:
Dependencies overhead
By default moto installs a lot of dependencies: https://github.com/spulec/moto/blob/ba1ceee95f289334b68330555598e80e581622ef/setup.py#L8-L28
Currently the dependency graph (created via `pipenv graph`) for moto has 84(!) positions with 43(!) unique packages. 

Those can introduce a substantial overhead when moto is used only for some AWS services. E.g. we use moto only to mock S3 service where e.g. `aws-xray-sdk` or `docker` aren't used, but we still need to install them.

My idea of cleaning this up would be to keep only core dependencies inside the `install_requires` list and extract the rest of them and organize inside the [`extras_require` mapping](https://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies). Then users could install moto only for the services they need e.g. with `pip install moto[s3]` or even for multiple services like `pip install moto[s3,xray]`.

Would you be willing to accept a PR with such solution?


Discussion:
This seems interesting.

I'm not very familiar with `extras_require`. If someone wants to install everything, is there a way to do that?
Well, you could have a special entry e.g. `all` and then you could `pip install moto[all]`. I think the more important question is: Why would you want to have that, beside being backward compatible? Is there someone that would actually want to mock out _all_ of the services that moto provides in one project?
Got it.

As an example, at my work we use dozens of services in our Moto tests so would probably prefer to include everything than listing out a dozen+ specific dependencies.
This would come in very handy for me too. We have another package that depends still on `docker-py`, so installing `moto` breaks everything due to conflict.

So, would a PR be useful?
This would also be useful to me as I mostly use for mocking out S3. Airflow has an [example of `extras_require`](https://github.com/apache/airflow/blob/master/setup.py#L378)
Yes, a PR would be welcome.

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -33,22 +33,13 @@ def get_version():
     "boto>=2.36.0",
     "boto3>=1.9.201",
     "botocore>=1.12.201",
-    "cryptography>=2.3.0",
     "requests>=2.5",
     "xmltodict",
     "six>1.9",
     "werkzeug",
-    "PyYAML>=5.1",
     "pytz",
-    "ecdsa<0.15",
     "python-dateutil<3.0.0,>=2.1",
-    "python-jose[cryptography]>=3.1.0,<4.0.0",
-    "docker>=2.5.1",
-    "jsondiff>=1.1.2",
-    "aws-xray-sdk!=0.96,>=0.93",
     "responses>=0.9.0",
-    "idna<3,>=2.5",
-    "cfn-lint>=0.4.0",
     "MarkupSafe<2.0",  # This is a Jinja2 dependency, 2.0.0a1 currently seems broken
 ]
 
@@ -72,7 +63,6 @@ def get_version():
         "mock<=3.0.5",
         "more-itertools==5.0.0",
         "setuptools==44.0.0",
-        "sshpubkeys>=3.1.0,<4.0",
         "zipp==0.6.0",
     ]
 else:
@@ -81,14 +71,57 @@ def get_version():
         "mock",
         "more-itertools",
         "setuptools",
-        "sshpubkeys>=3.1.0",
         "zipp",
     ]
 
+_dep_cryptography = "cryptography>=2.3.0"
+_dep_PyYAML = "PyYAML>=5.1"
+_dep_python_jose = "python-jose[cryptography]>=3.1.0,<4.0.0"
+_dep_python_jose_ecdsa_pin = "ecdsa<0.15"  # https://github.com/spulec/moto/pull/3263#discussion_r477404984
+_dep_docker = "docker>=2.5.1"
+_dep_jsondiff = "jsondiff>=1.1.2"
+_dep_aws_xray_sdk = "aws-xray-sdk!=0.96,>=0.93"
+_dep_idna = "idna<3,>=2.5"
+_dep_cfn_lint = "cfn-lint>=0.4.0"
+_dep_sshpubkeys_py2 = "sshpubkeys>=3.1.0,<4.0; python_version<'3'"
+_dep_sshpubkeys_py3 = "sshpubkeys>=3.1.0; python_version>'3'"
+
+all_extra_deps = [
+    _dep_cryptography,
+    _dep_PyYAML,
+    _dep_python_jose,
+    _dep_python_jose_ecdsa_pin,
+    _dep_docker,
+    _dep_jsondiff,
+    _dep_aws_xray_sdk,
+    _dep_idna,
+    _dep_cfn_lint,
+    _dep_sshpubkeys_py2,
+    _dep_sshpubkeys_py3,
+]
+
+# TODO: do we want to add ALL services here?
+# i.e. even those without extra dependencies.
+# Would be good for future-compatibility, I guess.
+extras_per_service = {
+    "ec2": [_dep_cryptography, _dep_sshpubkeys_py2, _dep_sshpubkeys_py3],
+    'acm': [_dep_cryptography],
+    'iam': [_dep_cryptography],
+    'cloudformation': [_dep_PyYAML, _dep_cfn_lint],
+    'cognitoidp': [_dep_python_jose, _dep_python_jose_ecdsa_pin],
+    'awslambda': [_dep_docker],
+    'batch': [_dep_docker],
+    'iotdata': [_dep_jsondiff],
+    'xray': [_dep_aws_xray_sdk],
+}
+
 extras_require = {
+    'all': all_extra_deps,
     'server': ['flask'],
 }
 
+extras_require.update(extras_per_service)
+
 # https://hynek.me/articles/conditional-python-dependencies/
 if int(setuptools.__version__.split(".", 1)[0]) < 18:
     if sys.version_info[0:2] < (3, 3):


Test patch: