Repository: cloud-custodian/cloud-custodian

Issue:
c7n-mailer [0.3.3] - No module named urllib3
I removed the c7n-mailer Lambda function and redeployed using the new v0.3.3. I am now seeing this error in the CloudWatch logs during runs.

[ERROR]	2018-11-14T19:01:27.535Z	afcd48a8-e83f-11e8-a7dc-6f4a34982269	Error starting mailer MailerSqsQueueProcessor(). 
Error: No module named urllib3 

Traceback (most recent call last):
File "/var/task/c7n_mailer/handle.py", line 46, in start_c7n_mailer
mailer_sqs_queue_processor.run(parallel)
File "/var/task/c7n_mailer/sqs_queue_processor.py", line 128, in run
self.process_sqs_message(sqs_message)
File "/var/task/c7n_mailer/sqs_queue_processor.py", line 173, in process_sqs_message
from .slack_delivery import SlackDelivery
File "/var/task/c7n_mailer/slack_delivery.py", line 16, in <module>
import requests
File "/var/task/requests/__init__.py", line 43, in <module>
import urllib3
ImportError: No module named urllib3


Discussion:
As an FYI, using PyPI/pip install of c7n_mailer. To troubleshoot, I updated the PythonPackageArchive in deploy.py to add urllib3. After that, I also received this same error for these modules:

chardet
certifi
idna

`def get_archive(config):
    archive = PythonPackageArchive(
        'c7n_mailer', 'ldap3', 'pyasn1', 'jinja2', 'markupsafe', 'ruamel',
        'redis', 'datadog', 'requests', 'urllib3', 'chardet', 'certifi', 'idna')`
Having the same issue with the Slack integration. Looking at the history it looks like urllib3/idna/chardet/certifi were briefly in deploy.py between May 5 and May 15.

Added here: https://github.com/capitalone/cloud-custodian/commit/ad7877888f53d58d06f2ad993a5ac173940c4945#diff-e7b16ce32846452561268dcd399eb20e

Removed here: https://github.com/capitalone/cloud-custodian/commit/99502eaeed416f5ab3f2db4c916d84e5e0bbee60#diff-e7b16ce32846452561268dcd399eb20e
Having the same issue.
Replacing code in https://github.com/capitalone/cloud-custodian/commit/99502eaeed416f5ab3f2db4c916d84e5e0bbee60#diff-e7b16ce32846452561268dcd399eb20e to the original code at https://github.com/capitalone/cloud-custodian/commit/ad7877888f53d58d06f2ad993a5ac173940c4945#diff-e7b16ce32846452561268dcd399eb20e helped me fix the problem
I can confirm what @Chan9390 says - changing the c7n-mailer's `deploy.py` to look like this:

```
 42 def get_archive(config):
 43     archive = PythonPackageArchive(
 44         'c7n_mailer',
 45         # core deps
 46         'jinja2', 'markupsafe', 'ruamel', 'ldap3', 'pyasn1', 'redis',
 47         # transport datadog - recursive deps
 48         'datadog', 'simplejson', 'decorator',
 49         # transport slack - recursive deps
 50         'slackclient', 'websocket',
 51         # requests (recursive deps), needed by datadog and slackclient
 52         'requests', 'urllib3', 'idna', 'chardet', 'certifi')
```
and doing a `pip install simplejson` did the trick for me. I now get Slack notifications as expected 👍 Thanks for this
Hoping this gets fixed soon. I'm in the middle up updating all my lamda based policies to support AWS Security Hub via the ec2 post-finding action and came across this very exact issue. Fortunately, the workaround mentioned by Chan9390 and confirmed by nerdyness saved us today and we're getting Slack notifications again.  Aside, the AWS Security Hub integration works great!

Resolving patch:
diff --git a/tools/c7n_mailer/c7n_mailer/deploy.py b/tools/c7n_mailer/c7n_mailer/deploy.py
--- a/tools/c7n_mailer/c7n_mailer/deploy.py
+++ b/tools/c7n_mailer/c7n_mailer/deploy.py
@@ -41,8 +41,13 @@ def dispatch(event, context):
 
 def get_archive(config):
     archive = PythonPackageArchive(
-        'c7n_mailer', 'ldap3', 'pyasn1', 'jinja2', 'markupsafe', 'ruamel',
-        'redis', 'datadog', 'requests')
+        'c7n_mailer',
+        # core deps
+        'jinja2', 'markupsafe', 'ruamel', 'ldap3', 'pyasn1', 'redis',
+        # transport datadog - recursive deps
+        'datadog', 'simplejson', 'decorator',
+        # requests (recursive deps), needed by datadog and slackclient
+        'requests', 'urllib3', 'idna', 'chardet', 'certifi')
 
     for d in set(config['templates_folders']):
         if not os.path.exists(d):
diff --git a/tools/c7n_mailer/setup.py b/tools/c7n_mailer/setup.py
--- a/tools/c7n_mailer/setup.py
+++ b/tools/c7n_mailer/setup.py
@@ -22,14 +22,15 @@
     "Jinja2",
     "boto3",
     "jsonschema",
+    "simplejson",  # datadog packaging dep
     "python-dateutil>=2.6",
     "ruamel.yaml==0.15.42",
     "datadog",
-    "slackclient",
     "sendgrid",
     "ldap3",
     "redis"]
 
+
 try:
     from concurrent import futures  # noqa F401
 except ImportError:


Test patch: