internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: cloud-custodian/cloud-custodian

Issue:
ci - intermittent sqlite-related failures
I've had a couple CI builds run today associated with #4101 - tests succeed locally but fail intermittently in CI for one or more ubuntu-based python environments. I see similar failures for #4035  and #4090 . The failures are all related to sqlite, which appears to be missing from the build image sometimes. That causes test results to vary across builds.

I don't know if this is a transient issue or not. Is it worth having a script step for ubuntu-based builds to ensure that sqlite is available?


Discussion:
yeah this seems like a regression in the azure pipeline build image that is hitting us intermittently.
filed upstream as https://github.com/microsoft/azure-pipelines-image-generation/issues/996

Resolving patch:
diff --git a/tools/c7n_mailer/c7n_mailer/ldap_lookup.py b/tools/c7n_mailer/c7n_mailer/ldap_lookup.py
--- a/tools/c7n_mailer/c7n_mailer/ldap_lookup.py
+++ b/tools/c7n_mailer/c7n_mailer/ldap_lookup.py
@@ -16,12 +16,7 @@
 import re
 import redis
 
-try:
-    import sqlite3
-except ImportError:
-    have_sqlite = False
-else:
-    have_sqlite = True
+import sqlite3
 from ldap3 import Connection
 from ldap3.core.exceptions import LDAPSocketOpenError
 
@@ -47,8 +42,6 @@ def __init__(self, config, logger):
             redis_port = int(config.get('redis_port', 6379))
             self.caching = self.get_redis_connection(redis_host, redis_port)
         elif self.cache_engine == 'sqlite':
-            if not have_sqlite:
-                raise RuntimeError('No sqlite available: stackoverflow.com/q/44058239')
             self.caching = LocalSqlite(config.get('ldap_cache_file', '/var/tmp/ldap.cache'), logger)
 
     def get_redis_connection(self, redis_host, redis_port):


Test patch:
Repository: cloud-custodian/cloud-custodian

Issue:
tools/mailer - import enum error
Hi,
with a 0.8.44, you have add provider(enum) on utils.py, but i can't find this on deploy.py/get_archive() and requirement.txt
(with manual fix, this working again)
with this error, i can't execute update-lambda

could you update the integration of "enum" plz ?

thanks for your help


Discussion:
i assume this is wrt to c7n_mailer?
what version of python are you using? enum has been around since python 2.7 and we don't support older versions.

what error are you seeing?
if you have a manual fix you've already done, feel free to send a pull request...
Yesterday the c7n mailer lambda function started failing for me after I redeployed it. The error message in CloudWatch logs was "Unable to import module 'periodic': No module named enum". I rolled back to 
c7n==0.8.43.1
c7n-mailer==0.5.0
and the problem went away.

Resolving patch:
diff --git a/tools/c7n_mailer/c7n_mailer/utils.py b/tools/c7n_mailer/c7n_mailer/utils.py
--- a/tools/c7n_mailer/c7n_mailer/utils.py
+++ b/tools/c7n_mailer/c7n_mailer/utils.py
@@ -15,7 +15,6 @@
 
 import base64
 from datetime import datetime, timedelta
-from enum import Enum
 import functools
 import json
 import os
@@ -29,7 +28,7 @@
 from ruamel import yaml
 
 
-class Providers(Enum):
+class Providers(object):
     AWS = 0
     Azure = 1
 
diff --git a/tools/c7n_mailer/setup.py b/tools/c7n_mailer/setup.py
--- a/tools/c7n_mailer/setup.py
+++ b/tools/c7n_mailer/setup.py
@@ -49,7 +49,7 @@
 
 setup(
     name="c7n_mailer",
-    version='0.5.1',
+    version='0.5.2',
     description="Cloud Custodian - Reference Mailer",
     long_description=long_description,
     long_description_content_type='text/markdown',


Test patch:
Repository: cloud-custodian/cloud-custodian

Issue:
orgaccounts.py (from c7n-org) regions flag puts *id001 instead of correct value
When I try to add regions as an argument with the **orgaccounts.py** script, it incorrectly puts the region with ***id001** as a value.

`python3 ./tools/c7n_org/scripts/orgaccounts.py --active true --regions ca-central-1 -f accounts.yml`

Results: 
```
- account_id: 'XXXXXXXXXXXXXXXX'
  email: email@address.com
  name: accountname
  regions: *id001
  role: arn:aws:iam::XXXXXXXXXXXXXXXX:role/OrganizationAccountAccessRole
  tags:
  - path:/OU
```
Expected: 
```
- account_id: 'XXXXXXXXXXXXXXXX'
  email: email@address.com
  name: accountname
  region:
  - ca-central-1
  role: arn:aws:iam::XXXXXXXXXXXXXXXX:role/OrganizationAccountAccessRole
  tags:
  - path:/OU
```


Discussion:
hmm.. that sounds like something magic happening from pyyaml to auto create a yaml anchor and ref. thanks for the bug report.

Resolving patch:
diff --git a/tools/c7n_org/scripts/azuresubs.py b/tools/c7n_org/scripts/azuresubs.py
--- a/tools/c7n_org/scripts/azuresubs.py
+++ b/tools/c7n_org/scripts/azuresubs.py
@@ -14,9 +14,9 @@
 
 from __future__ import print_function
 
-import yaml
 import click
 from c7n_azure.session import Session
+from c7n.utils import yaml_dump
 from azure.mgmt.resource.subscriptions import SubscriptionClient
 
 
@@ -39,11 +39,7 @@ def main(output):
         }
         results.append(sub_info)
 
-    print(
-        yaml.safe_dump(
-            {'subscriptions': results},
-            default_flow_style=False),
-        file=output)
+    print(yaml_dump({'subscriptions': results}), file=output)
 
 
 if __name__ == '__main__':
diff --git a/tools/c7n_org/scripts/orgaccounts.py b/tools/c7n_org/scripts/orgaccounts.py
--- a/tools/c7n_org/scripts/orgaccounts.py
+++ b/tools/c7n_org/scripts/orgaccounts.py
@@ -15,9 +15,9 @@
 from __future__ import print_function
 
 import click
-import yaml
 import os
 from c7n.credentials import assumed_session, SessionFactory
+from c7n.utils import yaml_dump
 
 ROLE_TEMPLATE = "arn:aws:iam::{Id}:role/OrganizationAccountAccessRole"
 
@@ -65,14 +65,10 @@ def main(role, ou, assume, profile, output, regions, active):
             'tags': tags,
             'role': role.format(**a)}
         if regions:
-            ainfo['regions'] = regions
+            ainfo['regions'] = list(regions)
         results.append(ainfo)
 
-    print(
-        yaml.safe_dump(
-            {'accounts': results},
-            default_flow_style=False),
-        file=output)
+    print(yaml_dump({'accounts': results}), file=output)
 
 
 def get_session(role, session_name, profile):


Test patch:
Repository: cloud-custodian/cloud-custodian

Issue:
gcp - cloud function deployment via policy issue
I am trying to deploy gcp cloud function via cloud-custodian policy. Everything works fine except one issue when I am trying to execut this function on gcp via pub/sub.

Here is traceback from stackdriver:

`File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 383, in run_background_function _function_handler.invoke_user_function(event_object) File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 217, in invoke_user_function return call_user_function(request_or_event) File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 214, in call_user_function event_context.Context(**request_or_event.context)) File "/user_code/main.py", line 21, in run from c7n_gcp.handler import run File "/user_code/c7n_gcp/handler.py", line 24, in <module> from c7n_gcp.entry import initialize_gcp File "/user_code/c7n_gcp/entry.py", line 20, in <module> import c7n_gcp.resources.appengine File "/user_code/c7n_gcp/resources/appengine.py", line 18, in <module> from c7n_gcp.query import QueryResourceManager, TypeInfo, ChildResourceManager, ChildTypeInfo File "/user_code/c7n_gcp/query.py", line 22, in <module> from c7n.actions import ActionRegistry File "/user_code/c7n/actions/__init__.py", line 17, in <module> from .autotag import AutoTagUser File "/user_code/c7n/actions/autotag.py", line 18, in <module> from c7n.manager import resources File "/user_code/c7n/manager.py", line 24, in <module> from c7n.resources.aws import AWS File "/user_code/c7n/resources/aws.py", line 31, in <module> import boto3 ModuleNotFoundError: No module named 'boto3'`

My assumption that somehow boto3 has been removed from requirements but it still needs to be present for gcp provider.


Discussion:
Looks like it all started from this commit: https://github.com/cloud-custodian/cloud-custodian/commit/94c60ff7d5067947026b5216c3d275518c0677ed
boto3 should not be required for runtime execution in serverless environments for azure and gcp, its a bug if it is.

Resolving patch:
diff --git a/c7n/manager.py b/c7n/manager.py
--- a/c7n/manager.py
+++ b/c7n/manager.py
@@ -24,7 +24,6 @@
     from c7n.resources.aws import AWS
     resources = AWS.resources
 except ImportError:
-    raise
     resources = PluginRegistry('resources')
 
 from c7n.utils import dumps


Test patch:
Repository: cloud-custodian/cloud-custodian

Issue:
Azure - Docs missing for VM Images



Discussion:
Seems like resource is supported, just not documented: https://cloudcustodian.io/docs/azure/resources/azure.mgmt.compute.html#azure-image
I love it when they are easy :) 

If someone is following up ensuring the quality of the auto-generated docs and replacing the manual ones then this would be covered there?  We should close after that is done I guess.

Resolving patch:
diff --git a/tools/c7n_azure/c7n_azure/resources/image.py b/tools/c7n_azure/c7n_azure/resources/image.py
--- a/tools/c7n_azure/c7n_azure/resources/image.py
+++ b/tools/c7n_azure/c7n_azure/resources/image.py
@@ -18,6 +18,23 @@
 
 @resources.register('image')
 class Image(ArmResourceManager):
+    """Virtual Machine Image
+
+    :example:
+    Returns all virtual machine images named my-test-vm-image
+
+    .. code-block:: yaml
+
+        policies:
+          - name: get-vm-image
+            resource: azure.image
+            filters:
+              - type: value
+                key: name
+                op: eq
+                value: my-test-vm-image
+
+    """
     class resource_type(ArmResourceManager.resource_type):
         service = 'azure.mgmt.compute'
         client = 'ComputeManagementClient'


Test patch:
Repository: cloud-custodian/cloud-custodian

Issue:
ci - investigate codecov upload issues
codecov has been failing to upload about 40% of the time, the issue has been reported a few times but no responses.

https://github.com/codecov/codecov-python/issues/196


Discussion:
Is it worth giving the [codecov-bash](https://docs.codecov.io/docs/about-the-codecov-bash-uploader) uploader a shot? Seems like it's the way codecov is moving, and pytest's changeover might be a useful reference - pytest-dev/pytest@739f9a4a4b0ae2efe105f3a6561088dac5096941 .
@ajkerrigan-mdsol Thanks! that seems promising and looks like it would let us also drop some of the custom frontends we had to do azure pipelines integration.

Resolving patch:
diff --git a/tools/dev/cicov.py b/tools/dev/cicov.py
deleted file mode 100755
--- a/tools/dev/cicov.py
+++ /dev/null
@@ -1,49 +0,0 @@
-#!/usr/bin/env python2
-
-import subprocess
-import os
-import logging
-
-log = logging.getLogger('cicov')
-
-# https://docs.microsoft.com/en-us/azure/devops/pipelines/build/variables?view=vsts
-
-
-config = {
-    'branch': os.environ.get('BRANCH'),
-    'pr': os.environ.get('PR'),
-    'build': os.environ.get('BUILD_ID'),
-    'commit': os.environ.get('BUILD_COMMIT')}
-
-UNSET = "system.pullRequest"
-
-
-def main():
-    logging.basicConfig(level=logging.INFO)
-
-    # Foresenics on the variables set in a context
-    for k in ('BUILD_BRANCH', 'PR', 'BUILD_ID', 'BUILD_COMMIT', 'COMMIT', 'BRANCH'):
-        v = os.environ.get(k)
-        log.info("Env var %s=%s" % (k, v))
-
-    # Defer to pull request branch (system.pullRequest), if thats not set
-    # then use Build.SourceBranchName
-    if UNSET in config['branch']:
-        config['branch'] = os.environ['BUILD_BRANCH']
-
-    args = ['codecov']
-    # Assemble cli args, skip all unset values.
-    for k, v in config.items():
-        if not v:
-            continue
-        # value not set
-        if UNSET in v:
-            continue
-        args.append("--%s" % k)
-        args.append(v)
-    log.info("Uploading CodeCoverage: %r" % (' '.join(args)))
-    subprocess.check_call(args)
-
-
-if __name__ == '__main__':
-    main()


Test patch:
Repository: cloud-custodian/cloud-custodian

Issue:
Feature Request - Add Tagging Support For storage-gateway Resource
@kapilt  Tagging is now supported with Storage Gateway service so requesting the global tagging support be pulled into it.  Thanks!


Discussion:

Resolving patch:
diff --git a/c7n/resources/storagegw.py b/c7n/resources/storagegw.py
--- a/c7n/resources/storagegw.py
+++ b/c7n/resources/storagegw.py
@@ -15,6 +15,7 @@
 
 from c7n.manager import resources
 from c7n.query import QueryResourceManager, TypeInfo
+from c7n.tags import register_universal_tags, universal_augment
 
 
 @resources.register('storage-gateway')
@@ -25,3 +26,10 @@ class resource_type(TypeInfo):
         enum_spec = ('list_gateways', 'Gateways', None)
         arn = id = 'GatewayARN'
         name = 'GatewayName'
+
+    augment = staticmethod(universal_augment)
+
+
+register_universal_tags(
+    StorageGateway.filter_registry, StorageGateway.action_registry,
+    compatibility=False)


Test patch:
Repository: cloud-custodian/cloud-custodian

Issue:
docs - schema refs have yaml anchors
this should be configured on the yaml dump for the docs


Discussion:
as an example see schema https://cloudcustodian.io/docs/azure/resources/azure-common-actions.html#azure-common-actions-tag

Resolving patch:
diff --git a/tools/c7n_sphinxext/c7n_sphinxext/docgen.py b/tools/c7n_sphinxext/c7n_sphinxext/docgen.py
--- a/tools/c7n_sphinxext/c7n_sphinxext/docgen.py
+++ b/tools/c7n_sphinxext/c7n_sphinxext/docgen.py
@@ -51,6 +51,12 @@ def get_environment():
     return env
 
 
+class SafeNoAliasDumper(yaml.SafeDumper):
+
+    def ignore_aliases(self, data):
+        return True
+
+
 class CustodianDirective(Directive):
 
     has_content = True
@@ -105,8 +111,9 @@ class CustodianSchema(CustodianDirective):
     def render_schema(cls, el):
         return cls._render(
             'schema.rst',
-            {'schema_yaml': yaml.safe_dump(
+            {'schema_yaml': yaml.dump(
                 ElementSchema.schema(cls.definitions, el),
+                Dumper=SafeNoAliasDumper,
                 default_flow_style=False)})
 
     def run(self):


Test patch:
Repository: cloud-custodian/cloud-custodian

Issue:
c7n-org not respecting policy org region restrictions
Pushing two policy at org level to be deployed in separate regions. This is what I am doing:

c7n-org run -c AWS/base/provisioning/NonProd/accounts.yml -r all -s /tmp/out -u AWS/policies/nonprod/01-common/asg-off-hours.yml

Policies look like this:

policies:
- name: policy-1
  region: us-east-1
  resource: asg
- name: policy-2
  region: us-east-2
  resource: ec2

but its pushing policy in all regions


Discussion:

Resolving patch:
diff --git a/tools/c7n_org/c7n_org/cli.py b/tools/c7n_org/c7n_org/cli.py
--- a/tools/c7n_org/c7n_org/cli.py
+++ b/tools/c7n_org/c7n_org/cli.py
@@ -538,6 +538,9 @@ def run_account(account, region, policies_config, output_path,
             p.expand_variables(p.get_variables(account.get('vars', {})))
             p.validate()
 
+            if p.region and p.region != region:
+                continue
+
             log.debug(
                 "Running policy:%s account:%s region:%s",
                 p.name, account['name'], region)


Test patch:
Repository: cloud-custodian/cloud-custodian

Issue:
c7n_mailer, AWS not installing Lambda, no logs, no errors
I have tried to setup/install the c7n_mailer lambda on our AWS account according to the docs.  I have tried it from my Mac and from Docker Images (in a Jenkins pipeline) to no avail. The kicker is I am not getting any error, or output. Is there anything I can look at to see if I have an issue from my end our something on the AWS account.  This is the command I am running:
```
c7n-mailer --config mailer.yml --update-lambda
```


Discussion:
are you checking in the same account and region that you have credentials? ie. that cli would error out w/ msg if it wasn't successfully provisioning a function.
Ah, that was what was missing, I was not setting a region in my `config.yml` so it was defaulting us-east-1, and I was looking in us-west-2.  I wonder how hard it would be to setup some basic logging info when running `--update-lambda`. Thanks for the help!


Resolving patch:
diff --git a/tools/c7n_mailer/c7n_mailer/deploy.py b/tools/c7n_mailer/c7n_mailer/deploy.py
--- a/tools/c7n_mailer/c7n_mailer/deploy.py
+++ b/tools/c7n_mailer/c7n_mailer/deploy.py
@@ -14,6 +14,7 @@
 from __future__ import absolute_import, division, print_function, unicode_literals
 
 import copy
+import logging
 import json
 import os
 
@@ -24,6 +25,8 @@
     PythonPackageArchive)
 
 
+log = logging.getLogger('custodian-mailer')
+
 entry_source = """\
 import logging
 
@@ -91,5 +94,6 @@ def provision(config, session_factory):
 
     archive = get_archive(config)
     func = LambdaFunction(func_config, archive)
+    log.info("Provisioning mailer lambda %s" % (session_factory().region_name))
     manager = LambdaManager(session_factory)
     manager.publish(func)


Test patch:
Repository: cloud-custodian/cloud-custodian

Issue:
c7n-mailer - azure sendgrid pinned version not compatible with requirements.txt
The requirements.txt is installing sendgrid==6.1.0, but we are pinned to sendgrid==6.0.5.

We should look if it's possible to remove the pin in our setup.py

## Error:
`c7n-mailer 0.5.4 has requirement sendgrid==6.0.5, but you'll have sendgrid 6.1.0 which is incompatible.`


Discussion:
doh.. I can take a look at this in tomorrow early if not addressed before then, it looks like I pissed one Datadog dependency in mailer as well that needs updating. surprised that the tox tests didn't complain on the conflict previously.

Resolving patch:
diff --git a/tools/c7n_mailer/setup.py b/tools/c7n_mailer/setup.py
--- a/tools/c7n_mailer/setup.py
+++ b/tools/c7n_mailer/setup.py
@@ -26,7 +26,7 @@
     "python-dateutil>=2.8",
     "ruamel.yaml>=0.15.93",
     "datadog",
-    "sendgrid==6.0.5",
+    "sendgrid>=6.0.5,<7.0",
     "ldap3",
     "c7n",
     "redis"]


Test patch:
Repository: cloud-custodian/cloud-custodian

Issue:
c7n-mailer - azure functions error when using SendGrid
Using a simple policy to queue a notification from the example [here](https://cloudcustodian.io/docs/tools/c7n-mailer.html#using-on-azure).

I see the following two errors:

1. Default Template
```
Invalid template reference default.j2
default.j2
```

2. Fetching contents of an object
```
local variable 'bpayload' referenced before assignment
Traceback (most recent call last):
  File "/home/site/wwwroot/c7n_mailer/azure_mailer/sendgrid_delivery.py", line 129, in _sendgrid_mail_from_email_message
    body = message.get_content()
AttributeError: 'MIMEText' object has no attribute 'get_content'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/site/wwwroot/c7n_mailer/azure_mailer/azure_queue_processor.py", line 137, in _deliver_email
    return sendgrid_delivery.sendgrid_handler(queue_message, email_messages)
  File "/home/site/wwwroot/c7n_mailer/azure_mailer/sendgrid_delivery.py", line 92, in sendgrid_handler
    mail = SendGridDelivery._sendgrid_mail_from_email_message(message)
  File "/home/site/wwwroot/c7n_mailer/azure_mailer/sendgrid_delivery.py", line 132, in _sendgrid_mail_from_email_message
    body = message.get_payload(decode=True).decode('utf-8')
  File "/usr/local/lib/python3.6/email/message.py", line 286, in get_payload
    value, defects = decode_b(b''.join(bpayload.splitlines()))
UnboundLocalError: local variable 'bpayload' referenced before assignment
```


Discussion:

Resolving patch:
diff --git a/tools/c7n_mailer/c7n_mailer/azure_mailer/deploy.py b/tools/c7n_mailer/c7n_mailer/azure_mailer/deploy.py
--- a/tools/c7n_mailer/c7n_mailer/azure_mailer/deploy.py
+++ b/tools/c7n_mailer/c7n_mailer/azure_mailer/deploy.py
@@ -70,7 +70,10 @@ def build_function_package(config, function_name, sub_id):
                 package.pkg.add_contents(function_path + '/msg-templates/%s' % t, fh.read())
 
     function_config = copy.deepcopy(config)
-    function_config['templates_folders'] = [function_path + '/msg-templates/']
+
+    functions_full_template_path = '/home/site/wwwroot/' + function_path + '/msg-templates/'
+    function_config['templates_folders'] = [functions_full_template_path]
+
     package.pkg.add_contents(
         function_path + '/config.json',
         contents=json.dumps(function_config))


Test patch:
Repository: cloud-custodian/cloud-custodian

Issue:
aws.rds-snapshot | error while filtering manual snapshots
```
- name: rds-manual-snapshot-notify
  resource: rds-snapshot
  filters:
    - type: value
       key: "SnapshotType"
       value: "manual"
    - type: age
       days: 7
       op: gte
```
Traceback:
```
File "/src/c7n/policy.py", line 928, in call
resources = PullMode(self).run()
File "/src/c7n/policy.py", line 232, in run
resources = self.policy.resource_manager.resources()
File "/src/c7n/query.py", line 419, in resources
resources = self.filter_resources(resources)
File "/src/c7n/manager.py", line 105, in filter_resources
resources = f.process(resources, event)
File "/src/c7n/filters/core.py", line 181, in process
return list(filter(self, resources))
File "/src/c7n/filters/core.py", line 621, in call
v = self.get_resource_date(i)
File "/src/c7n/filters/core.py", line 613, in get_resource_date
v = i[self.date_attribute]
KeyError: 'SnapshotCreateTime'
```
Reason: Some snapshots are in creating/error state and do not have a `SnapshotCreateTime`


Discussion:
hmmm... wondering if there's a safe default here (0), or if we need to just remove the resource from the filter match.
IMO, we should just remove from the filter match. Just thinking that setting up a default age might end up, us trying to take some action on those resources which might not be available. Your thoughts??
keep in mind a remove from filter match with a  negation `not` will end up with the same issue.
Understood your point. In that case, idk we can set a safe default time i.e. 0?
there's no differential behavior, what ever we choose will apply to both.

Resolving patch:
diff --git a/c7n/resources/rds.py b/c7n/resources/rds.py
--- a/c7n/resources/rds.py
+++ b/c7n/resources/rds.py
@@ -1039,6 +1039,9 @@ class RDSSnapshotAge(AgeFilter):
 
     date_attribute = 'SnapshotCreateTime'
 
+    def get_resource_date(self, i):
+        return i.get('SnapshotCreateTime')
+
 
 @RDSSnapshot.action_registry.register('restore')
 class RestoreInstance(BaseAction):


Test patch:
Repository: cloud-custodian/cloud-custodian

Issue:
validate exits with status 0 with structural parser errors.
It looks like the recent change to include the structural parser (#4967) skips setting the errors variable so validation exits with status 0 even when there are structural parser problems.  For instance:

```
$ custodian validate config/policies/tag-compliance-audit.yaml
2019-11-20 10:24:42,108: custodian.commands:ERROR Configuration invalid: config/policies/tag-compliance-audit.yaml
2019-11-20 10:24:42,108: custodian.commands:ERROR policy missing required keys (name, resource) data:
<trimmed for brevity>
$ echo $?
0


Discussion:

Resolving patch:
diff --git a/c7n/commands.py b/c7n/commands.py
--- a/c7n/commands.py
+++ b/c7n/commands.py
@@ -231,6 +231,7 @@ def validate(options):
         except PolicyValidationError as e:
             log.error("Configuration invalid: {}".format(config_file))
             log.error("%s" % e)
+            errors.append(e)
             continue
         errors += schema.validate(data, schm)
         conf_policy_names = {


Test patch:
diff --git a/tests/test_cli.py b/tests/test_cli.py
--- a/tests/test_cli.py
+++ b/tests/test_cli.py
@@ -115,6 +115,11 @@ def test_debug_version(self):
 
 class ValidateTest(CliTest):
 
+    def test_invalidate_structure_exit(self):
+        invalid_policies = {"policies": [{"name": "foo"}]}
+        yaml_file = self.write_policy_file(invalid_policies)
+        self.run_and_expect_failure(["custodian", "validate", yaml_file], 1)
+
     def test_validate(self):
         invalid_policies = {
             "policies": [