Repository: cloud-custodian/cloud-custodian

Issue:
Certifi python module missing when deploying cloudtrail lambda functions
The following error happens when the lambda function is triggered

`[ERROR] Runtime.ImportModuleError: Unable to import module 'custodian_policy': No module named 'certifi'`

As a workaround, I had to add the *certifi* package to the policy files.

```
policies:
  - name: my-policy
    description: my description
    resource: my-resource
    mode:
      type: cloudtrail
      packages:
        - botocore
        - boto3
        - urllib3
        - certifi
```


Discussion:
what's an example policy that triggers? ie was this already uploading modules?
The following policy has been working for months, until today. I have redacted some parts because it contained sensitive information. I hope it is not a problem.

I redeployed the lambda function because I didn't want to alert on changes made by a given IAM role. See the filter.

`c7n-org run -c config.yaml -u policy.yml  -s /tmp/`

```
policies:
  - name: ec2-changes-security-groups
    description: |
      Changes in security groups
    resource: iam-role
    mode:
      type: cloudtrail
      packages:
        - botocore
        - boto3
        - urllib3
      member-role: REDACTED
      role: REDACTED
      events:
        - source: ec2.amazonaws.com
          event: AuthorizeSecurityGroupIngress
          ids: "userIdentity.sessionContext.sessionIssuer.userName"
        - source: ec2.amazonaws.com
          event: AuthorizeSecurityGroupEgress
          ids: "userIdentity.sessionContext.sessionIssuer.userName"
        - source: ec2.amazonaws.com
          event: RevokeSecurityGroupIngress
          ids: "userIdentity.sessionContext.sessionIssuer.userName"
        - source: ec2.amazonaws.com
          event: RevokeSecurityGroupEgress
          ids: "userIdentity.sessionContext.sessionIssuer.userName"
        - source: ec2.amazonaws.com
          event: CreateSecurityGroup
          ids: "userIdentity.sessionContext.sessionIssuer.userName"
    filters:
      - not:
        - type: event
          key: "detail.userIdentity.sessionContext.sessionIssuer.userName"
          op: regex-case
          value: '^ServiceRoleForAmazonEKS-.*'
    actions:
      - type: notify
        to: ['REDACTED']
        template: default
        transport:
          type: sqs
          queue: REDACTED
```

Dockerfile running c7n-org. **The image was built today from the master branch**
```
FROM python:3.7

RUN git clone https://github.com/cloud-custodian/cloud-custodian.git /src

WORKDIR /src
RUN pip3 install .

WORKDIR /src/tools/c7n_org
RUN pip3 install .

# Setup for EntryPoint
ENV LC_ALL="C.UTF-8" LANG="C.UTF-8"
#VOLUME ["/var/log/cloud-custodian", "/etc/cloud-custodian"]
#ENTRYPOINT ["/usr/local/bin/custodian"]
WORKDIR /custodian

ADD  run.sh .
CMD ["/custodian/run.sh"]
```
so what changed is one of the transitive dependencies changed for what your uploading when you rebuilt the docker image, which picked up a new dependency on certifi. ie in this particular policy, you could have omitted the packages and it would have been fine (no newer iam features then what was avail mid 2018 when lambda froze sdks). 
there's a longer term solution here wrt to auto layer construction, but nutshell the only actionable thing i see for us to do short term is make sure that we document the additional dependency if uploading modules/packages, grepping through the docs i'm not seeing package/module uploads documented in general (although it should be).
> so what changed is one of the transitive dependencies changed for what your uploading when you rebuilt the docker image, which picked up a new dependency on certifi. ie in this particular policy, you could have omitted the packages and it would have been fine (no newer iam features then what was avail mid 2018 when lambda froze sdks).
> there's a longer term solution here wrt to auto layer construction, but nutshell the only actionable thing i see for us to do short term is make sure that we document the additional dependency if uploading modules/packages, grepping through the docs i'm not seeing package/module uploads documented in general (although it should be).

Documenting dependencies seems ok.  It's already known that we have to manually install other python modules:  `botocore,  boto3 and   urllib3`.  Adding certifi to the list won't hurt.

See #3975 and #3401 
We are hitting the same thing. We use the `c7n-org` latest (since this is the only docker tag released) docker image from docker hub and are seeing this on our lambda policies deployed by `c7n-org`. 

> you could have omitted the packages and it would have been fine

We do not use the `packages:` key in our policies and we are still seeing this.
Yeah same here, we started getting this with DynamoDB table lambda policies and are using no special packages.

Resolving patch:
diff --git a/c7n/actions/webhook.py b/c7n/actions/webhook.py
--- a/c7n/actions/webhook.py
+++ b/c7n/actions/webhook.py
@@ -14,7 +14,11 @@
 
 import json
 
-import certifi
+try:
+    import certifi
+except ImportError:
+    certifi = None
+
 import jmespath
 import urllib3
 from six.moves.urllib import parse
@@ -90,7 +94,7 @@ def __init__(self, data=None, manager=None, log_dir=None):
     def process(self, resources, event=None):
         self.http = urllib3.PoolManager(
             cert_reqs='CERT_REQUIRED',
-            ca_certs=certifi.where())
+            ca_certs=certifi and certifi.where() or None)
 
         if self.batch:
             for chunk in utils.chunks(resources, self.batch_size):


Test patch: