AWS CloudWatchのログフィルタパターンを分かりやすく解説

こちらのサンプルログイベントメッセージを使用します。

```
1	[XXX] BENCHMARK : Running Start Crawl for Crawler TestCrawler2
2	[XXX] BENCHMARK : Classification complete, writing results to database mygluedatabase
3	[XXX] INFO : Crawler configured with SchemaChangePolicy {"UpdateBehavior":"UPDATE_IN_DATABASE","DeleteBehavior":"DEPRECATE_IN_DATABASE"}.
4	[XXX] INFO : Created table gluetest in database mygluedatabase
5	[XXX] BENCHMARK : Finished writing to Catalog
6	[XXX] BENCHMARK : Crawler has finished running and is in state READY
```


###検索したい語句を""で囲むまたはそのまま入力することで、一致したログイベントメッセージを取得できます。

検索条件

```
BENCHMARK
```
または、

```
"BENCHMARK"
```

検索結果

```
1	[XXX] BENCHMARK : Running Start Crawl for Crawler TestCrawler2
2	[XXX] BENCHMARK : Classification complete, writing results to database mygluedatabase
5	[XXX] BENCHMARK : Finished writing to Catalog
6	[XXX] BENCHMARK : Crawler has finished running and is in state READY
```

###""で囲むことで、スペースが存在する場合も単一文字列として認識させることが出来ます。

検索条件

```
"Running Start Crawl for Crawler"
```

検索結果

```
1	[XXX] BENCHMARK : Running Start Crawl for Crawler TestCrawler2
```

###語句を複数並べて絞り込み検索を行うことも出来ます。

検索条件

```
BENCHMARK Crawler
```

または、


```
"BENCHMARK" "Crawler"
```

検索結果

```
1	[XXX] BENCHMARK : Running Start Crawl for Crawler TestCrawler2
6	[XXX] BENCHMARK : Crawler has finished running and is in state READY
```


###語句の後に負符号 (-) をつけたあと、語句を指定すると検索条件から外すことができます。

検索条件

```
BENCHMARK - Crawler
```

または、

```
"BENCHMARK" - "Crawler"
```

検索結果

```
2	[XXX] BENCHMARK : Classification complete, writing results to database mygluedatabase
5	[XXX] BENCHMARK : Finished writing to Catalog
```

###?を使ってORパターンマッチングを行うことも出来ます。

検索条件

```
?INFO ?Finished
```

検索結果

```
3	[XXX] INFO : Crawler configured with SchemaChangePolicy {"UpdateBehavior":"UPDATE_IN_DATABASE","DeleteBehavior":"DEPRECATE_IN_DATABASE"}.
4	[XXX] INFO : Created table gluetest in database mygluedatabase
5	[XXX] BENCHMARK : Finished writing to Catalog
```

##JSON ログについて


こちらのサンプルJSONログを使用します。

```
1	{"eventType": "UpdateTrail", "sourceIPAddress": "111.111.111.111", "arrayKey": ["value", "another value"], "objectList": [{ "name": "a", "id": 1},{"name": "b", "id": 2}], "SomeObject": null, "ThisFlag": true}
2	{"eventType": "UpdateTrail2", "sourceIPAddress": "123.123.123.123", "arrayKey": ["value", "another value"], "objectList": [{ "name": "c", "id": 3},{"name": "d", "id": 4}], "SomeObject": null, "ThisFlag": false}
3	{"eventType": "UpdateTrail3", "sourceIPAddress": "120.0.0.1", "arrayKey": ["value", "another value"], "objectList": [{ "name": "e", "id": 500},{"name": "f", "id": 600}], "SomeObject": "a", "ThisFlag": true}
4	{"eventType": "UpdateTrail4", "sourceIPAddress": "120.0.0.2", "arrayKey": ["value", "another value"], "objectList": [{ "name": "e", "id": 501},{"name": "f", "id": 601}], "SomeObject": "b", "ThisFlag": true}
```

###{}で囲み、keyに$をつけて検索します。

```
{ $.eventType = "UpdateTrail" }
```

検索結果

```
1	{"eventType": "UpdateTrail", "sourceIPAddress": "111.111.111.111", "arrayKey": ["value", "another value"], "objectList": [{ "name": "a", "id": 1},{"name": "b", "id": 2}], "SomeObject": null, "ThisFlag": true}
```

###~ではない場合の条件は=に!をつけることで可能です。

```
{ $.sourceIPAddress != 123.123.* }
```

検索結果

```
1	{"eventType": "UpdateTrail", "sourceIPAddress": "111.111.111.111", "arrayKey": ["value", "another value"], "objectList": [{ "name": "a", "id": 1},{"name": "b", "id": 2}], "SomeObject": null, "ThisFlag": true}
3	{"eventType": "UpdateTrail3", "sourceIPAddress": "120.0.0.1", "arrayKey": ["value", "another value"], "objectList": [{ "name": "e", "id": 500},{"name": "f", "id": 600}], "SomeObject": "a", "ThisFlag": true}
4	{"eventType": "UpdateTrail4", "sourceIPAddress": "120.0.0.2", "arrayKey": ["value", "another value"], "objectList": [{ "name": "e", "id": 501},{"name": "f", "id": 601}], "SomeObject": "b", "ThisFlag": true}
```

###リスト内の検索も可能です。この場合は2のJSONが検索に引っかかります。

```
{ $.objectList[0].id = 3 }
```

検索結果

```
2	{"eventType": "UpdateTrail2", "sourceIPAddress": "123.123.123.123", "arrayKey": ["value", "another value"], "objectList": [{ "name": "c", "id": 3},{"name": "d", "id": 4}], "SomeObject": null, "ThisFlag": false}
```

###数値検索では、>、<、>=、<=、=、 != 演算子を使用することが出来ます。

```
{ $.objectList[0].id < 5 }
```

検索結果

```
1	{"eventType": "UpdateTrail", "sourceIPAddress": "111.111.111.111", "arrayKey": ["value", "another value"], "objectList": [{ "name": "a", "id": 1},{"name": "b", "id": 2}], "SomeObject": null, "ThisFlag": true}
2	{"eventType": "UpdateTrail2", "sourceIPAddress": "123.123.123.123", "arrayKey": ["value", "another value"], "objectList": [{ "name": "c", "id": 3},{"name": "d", "id": 4}], "SomeObject": null, "ThisFlag": false}
```


###数字ではアスタリスクも使用することが出来ます。

```
{ $.objectList[0].id = 5* }
```

検索結果

```
3	{"eventType": "UpdateTrail3", "sourceIPAddress": "120.0.0.1", "arrayKey": ["value", "another value"], "objectList": [{ "name": "e", "id": 500},{"name": "f", "id": 600}], "SomeObject": "a", "ThisFlag": true}
4	{"eventType": "UpdateTrail4", "sourceIPAddress": "120.0.0.2", "arrayKey": ["value", "another value"], "objectList": [{ "name": "e", "id": 501},{"name": "f", "id": 601}], "SomeObject": "b", "ThisFlag": true}
```


###値がNULLの場合はIS NULLで引っ掛けることが出来ます。

```
{ $.SomeObject IS NULL }
*小文字でもOK 
{ $.SomeObject is null }
```

検索結果

```
1	{"eventType": "UpdateTrail", "sourceIPAddress": "111.111.111.111", "arrayKey": ["value", "another value"], "objectList": [{ "name": "a", "id": 1},{"name": "b", "id": 2}], "SomeObject": null, "ThisFlag": true}
2	{"eventType": "UpdateTrail2", "sourceIPAddress": "123.123.123.123", "arrayKey": ["value", "another value"], "objectList": [{ "name": "c", "id": 3},{"name": "d", "id": 4}], "SomeObject": null, "ThisFlag": false}
```


###値が真偽値の場合はIS TRUE または IS FALSEで引っ掛けることが出来ます。

```
{ $.ThisFlag IS TRUE }
小文字でもOK 
{ $.ThisFlag is true }
```

検索結果

```
1	{"eventType": "UpdateTrail", "sourceIPAddress": "111.111.111.111", "arrayKey": ["value", "another value"], "objectList": [{ "name": "a", "id": 1},{"name": "b", "id": 2}], "SomeObject": null, "ThisFlag": true}
3	{"eventType": "UpdateTrail3", "sourceIPAddress": "120.0.0.1", "arrayKey": ["value", "another value"], "objectList": [{ "name": "e", "id": 500},{"name": "f", "id": 600}], "SomeObject": "a", "ThisFlag": true}
4	{"eventType": "UpdateTrail4", "sourceIPAddress": "120.0.0.2", "arrayKey": ["value", "another value"], "objectList": [{ "name": "e", "id": 501},{"name": "f", "id": 601}], "SomeObject": "b", "ThisFlag": true}
```

##JSONの複合条件について

###括弧()、OR (||) と AND (&&) を使用して複合式も可能です。
構文は演算子の標準の順序に従い、() > && > || となります。

###OR (||) 条件


```
{ ( $.SomeObject IS NULL ) || ( $.ThisFlag IS FALSE ) }
```

検索結果

```
1	{"eventType": "UpdateTrail", "sourceIPAddress": "111.111.111.111", "arrayKey": ["value", "another value"], "objectList": [{ "name": "a", "id": 1},{"name": "b", "id": 2}], "SomeObject": null, "ThisFlag": true}
2	{"eventType": "UpdateTrail2", "sourceIPAddress": "123.123.123.123", "arrayKey": ["value", "another value"], "objectList": [{ "name": "c", "id": 3},{"name": "d", "id": 4}], "SomeObject": null, "ThisFlag": false}
```


###AND (&&)条件

```
{ ( $.SomeObject IS NULL ) && ( $.ThisFlag IS TRUE ) }
```

検索結果

```
1	{"eventType": "UpdateTrail", "sourceIPAddress": "111.111.111.111", "arrayKey": ["value", "another value"], "objectList": [{ "name": "a", "id": 1},{"name": "b", "id": 2}], "SomeObject": null, "ThisFlag": true}
```

##スペース区切りで出力されるログから値を取得する

スペース区切りの項目ごとが何を示しているのか指定し、
その項目ごとに条件を指定して、値を取得することができます。

こちらのサンプルログを使用します。

```
1   frank   10/Oct/2000:13:25:15 -0700  GET /apache_pb.gif HTTP/1.0 200 1534    127.0.0.1   -
2   frank   10/Oct/2000:13:35:22 -0700  GET /apache_pb.gif HTTP/1.0 500 5324    127.0.0.1   -
3   frank   10/Oct/2000:13:50:35 -0700  GET /apache_pb.gif HTTP/1.0 200 4355    127.0.0.1   -
4   frank   10/Oct/2000:13:50:40 -0700  GET /apache_pb.gif HTTP/1.0 401 5423    127.0.0.1   -
```


###フィールド数が不明な場合は、省略符号 (…) を使用した省略通知を使用できます。

前方を省略した検索条件

```
[..., status_code, bytes]
```

検索結果

```
イベント番号	$1	$2	$3	$4	$5	$6	$7	$8	$9	$bytes	$status_code
1	127.0.0.1	-	frank	10/Oct/2000:13:25:15 -0700	GET /apache_pb.gif HTTP/1.0	1534	200
2	127.0.0.1	-	frank	10/Oct/2000:13:35:22 -0700	GET /apache_pb.gif HTTP/1.0	5324	500
3	127.0.0.1	-	frank	10/Oct/2000:13:50:35 -0700	GET /apache_pb.gif HTTP/1.0	4355	200
4	127.0.0.1	-	frank	10/Oct/2000:13:50:40 -0700	GET /apache_pb.gif HTTP/1.0	5423	401
```

間を省略した検索条件

```
[ip, user, ..., status_code, bytes]
```

検索結果

```
イベント番号	$3	$4	$5	$6	$7	$8	$9	$bytes	$ip	$status_code	$user
1	frank	10/Oct/2000:13:25:15 -0700	GET /apache_pb.gif HTTP/1.0	1534	127.0.0.1	200	-
2	frank	10/Oct/2000:13:35:22 -0700	GET /apache_pb.gif HTTP/1.0	5324	127.0.0.1	500	-
3	frank	10/Oct/2000:13:50:35 -0700	GET /apache_pb.gif HTTP/1.0	4355	127.0.0.1	200	-
4	frank	10/Oct/2000:13:50:40 -0700	GET /apache_pb.gif HTTP/1.0	5423	127.0.0.1	401	-
```

後方を省略した検索条件

```
[ip, user, ...]
```

検索結果

```
イベント番号	$3	$4	$5	$6	$7	$ip	$user
1	frank	10/Oct/2000:13:25:15 -0700	GET /apache_pb.gif HTTP/1.0	200	1534	127.0.0.1	-
2	frank	10/Oct/2000:13:35:22 -0700	GET /apache_pb.gif HTTP/1.0	500	5324	127.0.0.1	-
3	frank	10/Oct/2000:13:50:35 -0700	GET /apache_pb.gif HTTP/1.0	200	4355	127.0.0.1	-
4	frank	10/Oct/2000:13:50:40 -0700	GET /apache_pb.gif HTTP/1.0	401	5423	127.0.0.1	-
```

条件を指定して検索することも出来ます。
\>、<、>=、<=、=,  != 演算子やアスタリスク、OR (||) と AND (&&)も使用することが出来ます。

status_codeが200である場合の検索条件

```
[ip, user, username, timestamp, request, status_code = 200, bytes]
```

検索結果

```
1	1534	127.0.0.1	GET /apache_pb.gif HTTP/1.0	200	10/Oct/2000:13:25:15 -0700	-	frank
3	4355	127.0.0.1	GET /apache_pb.gif HTTP/1.0	200	10/Oct/2000:13:50:35 -0700	-	frank
```


こちらはOR (||) を試しています。
status_codeが200、または500である場合の検索条件

```
[ip, user, username, timestamp, request, status_code = 200 || status_code = 500, bytes]
```

検索結果

```
1	1534	127.0.0.1	GET /apache_pb.gif HTTP/1.0	200	10/Oct/2000:13:25:15 -0700	-	frank
2	5324	127.0.0.1	GET /apache_pb.gif HTTP/1.0	500	10/Oct/2000:13:35:22 -0700	-	frank
3	4355	127.0.0.1	GET /apache_pb.gif HTTP/1.0	200	10/Oct/2000:13:50:35 -0700	-	frank
```

##参考
[フィルターとパターンの構文](https://docs.aws.amazon.com/ja_jp/AmazonCloudWatch/latest/logs/FilterAndPatternSyntax.html)