Repository: cloud-custodian/cloud-custodian

Issue:
Can't filter log-group using creationTime age
I'm trying, but failing to filter log groups that were created more than 30 days ago using the following filter:

```
policies:
- name: OldLogGroups
  resource: log-group
  filters:
  - type: value
    key: creationTime
    op: gt
    value_type: age
    value: 30
```

According to [AWS docs](https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_LogGroup.html), the property `creationTime` stores the creation time of the log group, expressed as the number of milliseconds after Jan 1, 1970 00:00:00 UTC.

I understand 'age' value_type already supports standard unix epoch expressed in seconds, but not milliseconds (#2051).

Support for date/time values expressed in milliseconds or adding a new filter like `log-group-age` would allow to cleanup old log groups that have been created certain time ago and either have 0 stored bytes or have not had any writes for a while.

custodian version: 0.8.42.1


Discussion:
fwiw, generally i've just gone with cleanup of those which haven't been written to for a while, via `last-write` filter, possibly with non zero status, but i agree the use case here makes sense.

i'm a little cautious about extending the existing syntax to support milliseconds (there's not a good safe mechanism here outside of detecting digits which will be wrong over time) and try/except retry is a bit intensive (this code gets invoked alot),  adding a another value_type or dedicated filter might be more appropriate.
Thanks for taking a look @kapilt. I think you are correct, extending the `last-write` syntax to also support integer milliseconds might not be worth the effort and increased load, since use of  "milliseconds since the epoch" to define age of a resource isn't that common and seems to be mostly constrained to CloudWatch.

Adding `value_type` milliseconds in general, is a good idea, since there seem to be quite a few resource properties that use it, though again some expect an integer and some expect a float (grrr).

In this particular case though, I'd say the most sensible solution would be to add a new filter to `log-group` resource.
thinking on this a bit more due to (#4187)  I think we should go a different way and just normalize these values to seconds since epoch instead of millis when we're fetching log groups.

Resolving patch:
diff --git a/c7n/resources/cw.py b/c7n/resources/cw.py
--- a/c7n/resources/cw.py
+++ b/c7n/resources/cw.py
@@ -163,7 +163,11 @@ class resource_type(object):
         dimension = 'LogGroupName'
         date = 'creationTime'
 
-    augment = universal_augment
+    def augment(self, resources):
+        resources = universal_augment(self, resources)
+        for r in resources:
+            r['creationTime'] = r['creationTime'] / 1000.0
+        return resources
 
     def get_arns(self, resources):
         # log group arn in resource describe has ':*' suffix, not all


Test patch:
diff --git a/tests/data/placebo/test_log_group_age_normalize/logs.DescribeLogGroups_1.json b/tests/data/placebo/test_log_group_age_normalize/logs.DescribeLogGroups_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_log_group_age_normalize/logs.DescribeLogGroups_1.json
@@ -0,0 +1,25 @@
+{
+    "status_code": 200,
+    "data": {
+        "logGroups": [
+            {
+                "logGroupName": "/aws/lambda/myIOTFunction",
+                "creationTime": 1548368507441,
+                "metricFilterCount": 0,
+                "arn": "arn:aws:logs:us-west-2:644160558196:log-group:/aws/lambda/myIOTFunction:*",
+                "storedBytes": 918177
+            }
+        ],
+        "ResponseMetadata": {
+            "RequestId": "81e0704d-91f7-11e9-9bca-9bc2904299e3",
+            "HTTPStatusCode": 200,
+            "HTTPHeaders": {
+                "x-amzn-requestid": "81e0704d-91f7-11e9-9bca-9bc2904299e3",
+                "content-type": "application/x-amz-json-1.1",
+                "content-length": "214",
+                "date": "Tue, 18 Jun 2019 18:33:02 GMT"
+            },
+            "RetryAttempts": 0
+        }
+    }
+}
\ No newline at end of file
diff --git a/tests/data/placebo/test_log_group_age_normalize/tagging.GetResources_1.json b/tests/data/placebo/test_log_group_age_normalize/tagging.GetResources_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_log_group_age_normalize/tagging.GetResources_1.json
@@ -0,0 +1,18 @@
+{
+    "status_code": 200,
+    "data": {
+        "PaginationToken": "",
+        "ResourceTagMappingList": [],
+        "ResponseMetadata": {
+            "RequestId": "82178597-91f7-11e9-a240-1bab5274610f",
+            "HTTPStatusCode": 200,
+            "HTTPHeaders": {
+                "x-amzn-requestid": "82178597-91f7-11e9-a240-1bab5274610f",
+                "content-type": "application/x-amz-json-1.1",
+                "content-length": "50",
+                "date": "Tue, 18 Jun 2019 18:33:03 GMT"
+            },
+            "RetryAttempts": 0
+        }
+    }
+}
\ No newline at end of file
diff --git a/tests/test_cwl.py b/tests/test_cwl.py
--- a/tests/test_cwl.py
+++ b/tests/test_cwl.py
@@ -32,6 +32,22 @@ def test_cross_account(self):
         self.assertEqual(len(resources), 1)
         self.assertEqual(resources[0]["c7n:CrossAccountViolations"], ["1111111111111"])
 
+    def test_age_normalize(self):
+        factory = self.replay_flight_data("test_log_group_age_normalize")
+        p = self.load_policy({
+            'name': 'log-age',
+            'resource': 'aws.log-group',
+            'filters': [{
+                'type': 'value',
+                'value_type': 'age',
+                'value': 30,
+                'op': 'greater-than',
+                'key': 'creationTime'}]},
+            session_factory=factory, config={'region': 'us-west-2'})
+        resources = p.run()
+        self.assertEqual(len(resources), 1)
+        self.assertEqual(resources[0]['creationTime'], 1548368507.441)
+
     def test_last_write(self):
         factory = self.replay_flight_data("test_log_group_last_write")
         p = self.load_policy(