AWS: WAF WebACL logging and Logz.io integration
Tags: aws, security, tutorial, todayilearned
Published: 2021-07-22 14:44:41

---
title: AWS: WAF WebACL logging and Logz.io integration
published: true
date: 2021-07-22 11:01:28 UTC
tags: aws,security,tutorial,todayilearned
canonical_url: https://rtfm.co.ua/en/aws-waf-webacl-logging-and-logz-io-integration/
---

![](https://cdn-images-1.medium.com/max/1024/1*8QppZuMnRQSngTE4H4UqzA.png)

In the first post — [AWS: Web Application Firewall overview, configuration, and its monitoring](https://rtfm.co.ua/en/aws-web-application-firewall-overview-configuration-and-its-monitoring/) — we spoke about its main components, created a WebACL and Rules for it, and did basic monitoring.

Also, we’ve configured WebACL’s logs collection with AWS Kinesis, but now it’s time to see them Logz.io, as CloudWatch Logs isn’t available for it yet.

So, in this post, we will configure logs sending to an AWS S3 bucket via AWS Kinesis, and then will configure Logz.io to grab those logs from S3 and will speak about logs content and how they can be used for debugging.

### **Contents**

- [AWS WAF logs](https://rtfm.co.ua/en/aws-waf-webacl-logging-and-logz-io-integration/#AWS_WAF_logs)
- [AWS Kinesis](https://rtfm.co.ua/en/aws-waf-webacl-logging-and-logz-io-integration/#AWS_Kinesis)
- [WAF ACL logging](https://rtfm.co.ua/en/aws-waf-webacl-logging-and-logz-io-integration/#WAF_ACL_logging)
- [Logz.io S3 Bucket](https://rtfm.co.ua/en/aws-waf-webacl-logging-and-logz-io-integration/#Logzio_S3_Bucket)
- [Logz.io S3 configuration](https://rtfm.co.ua/en/aws-waf-webacl-logging-and-logz-io-integration/#Logzio_S3_configuration)
- [IAM policy](https://rtfm.co.ua/en/aws-waf-webacl-logging-and-logz-io-integration/#IAM_policy)
- [IAM role](https://rtfm.co.ua/en/aws-waf-webacl-logging-and-logz-io-integration/#IAM_role)
- [WAF logs — fields](https://rtfm.co.ua/en/aws-waf-webacl-logging-and-logz-io-integration/#WAF_logs_-_fields)
- [Redacted fields — excluding data from logs](https://rtfm.co.ua/en/aws-waf-webacl-logging-and-logz-io-integration/#Redacted_fields_-_excluding_data_from_logs)
- [Logz.io alerting](https://rtfm.co.ua/en/aws-waf-webacl-logging-and-logz-io-integration/#Logzio_alerting)

### AWS WAF logs

Besides of CloudWatch metrics, we can enable logs for all requests passed via our WebACL.

AWS WAF uses AWS Kinesis to send data and AWS S3 to store logs.

Unfortunately, WAF and Kinesis can not use CloudWatch Logs (at least, at the moment of wringing, as AWS Tech Support told me that it is planned to enable them).

So, let’s do the following:

1. configure a Kinesis Firehouse stream
2. configure a WebACL to send data to that stream
3. Kinesis stream will send data to an AWS S3 bucket
4. and Logz.io will use its connector to get logs from this S3

#### AWS Kinesis

> **Note:** One AWS WAF log is equivalent to one Kinesis Data Firehose record. If you typically receive 10,000 requests per second, set a 10,000 record per second limit in Kinesis Data Firehose to enable full logging.

Go to the AWS Kinesis, create a new _Data Firehose delivery stream_, remember that its name must be prefixed as ‘_aws-waf-logs-_’.

Choose _Direct PUT or other sources_, click on the _Next_:

![](https://cdn-images-1.medium.com/max/942/0*8CkXs6H56vfB3Kfv.png)

On the next page skip data transformation, click _Next_, then for the _Destination_ select _S3_, and create a new S3 bucket:

![](https://cdn-images-1.medium.com/max/883/0*nNX9DA00IC4idovn.png)

Here, I’m using an already existing bucket. To avoid mixing logs from various ACLs — add a prefix:

![](https://cdn-images-1.medium.com/max/876/0*B-KLK9vvPu8l4wIN.png)

On the next page leave all default values. If anything will go wrong, Kinesis will send a message via CloudWatch:

![](https://cdn-images-1.medium.com/max/864/0*3qOWU7SOW0ioKFPg.png)

To have those messages, leave the _Error logging_ option enabled:

![](https://cdn-images-1.medium.com/max/879/0*0q4NFAjEdgtS1q9B.png)

Check the settings, and create the stream:

![](https://cdn-images-1.medium.com/max/879/0*PjIVS5ic4AdSQ33x.png)

Wait a couple of minutes for the stream to be created:

![](https://cdn-images-1.medium.com/max/628/0*2osM6m_gFd8hgjV2.png)

Or just go to a WebACL.

#### WAF ACL logging

Go to the ACL, on the _Logging and metrics_ tab click on the _Enable logging_:

![](https://cdn-images-1.medium.com/max/1006/0*wURFhEQCPfNATsfZ.png)

Choose the stream created above:

![](https://cdn-images-1.medium.com/max/829/0*KNif30ss8KpBn31N.png)

Here we also can enable filters, which will speak about them below.

### Logz.io S3 Bucket

Documentation is [here>>>](https://app.logz.io/#/dashboard/send-your-data/log-sources/s3-bucket).

At first, we need to create an IAM policy and a Role, that will provide access to the S3 bucket with logs. Check its documentation [here>>>](https://docs.logz.io/user-guide/give-aws-access-with-iam-roles/).

Logz.io will do [AssumeRole](https://rtfm.co.ua/aws-iam-assumerole-opisanie-primery/) via this role to get permissions to access the bucket.

#### Logz.io S3 configuration

To to the [S3 Bucket,](https://app.logz.io/#/dashboard/send-your-data/log-sources/s3-bucket) click on the _Add a bucket_:

![](https://cdn-images-1.medium.com/max/1024/0*ga7x69GXpzEeCiQM.png)

Choose _Authenticate with a role_:

![](https://cdn-images-1.medium.com/max/625/0*TdLUUD-jB5ThZid7.png)

Specify the bucket’s name, click on the _Get the role policy_:

![](https://cdn-images-1.medium.com/max/625/0*n-EEXH-mW53X9_OS.png)

#### IAM policy

Copy the policy content, go to the AWS IAM, create a new policy:

![](https://cdn-images-1.medium.com/max/578/0*ofPqYNWXdB8Df6Fh.png)

![](https://cdn-images-1.medium.com/max/1024/0*0iRXbTrFBi5VlT4t.png)

Add JSON from the Logz.io:

![](https://cdn-images-1.medium.com/max/518/0*pj-CnQkMeP8lJvrh.png)

Save the policy:

![](https://cdn-images-1.medium.com/max/658/0*_cSz4kwtXgil7Xqn.png)

#### IAM role

Go to the в AWS IAM Roles, create a new role.

Choose _Another AWS account_, specify the Account ID and External ID from the Logz.io page:

![](https://cdn-images-1.medium.com/max/640/0*GLJJxnD8fX9igxLe.png)

On the _Permissions_ page, attach the policy that we’ve created above:

![](https://cdn-images-1.medium.com/max/491/0*5wzh3ej4rBnwcGfw.png)

Save the role:

![](https://cdn-images-1.medium.com/max/694/0*QHn8-mLAgz6m1i1m.png)

In its _Trusted entities_ we can see the Account ID, that will be able to Assume that Role.

Copy role’s ARN:

![](https://cdn-images-1.medium.com/max/869/0*L4CQGmMDIle7bwe9.png)

Go back to Logz.io, set the ARN, and choose an AWS Region.

In the data type select _other_:

![](https://cdn-images-1.medium.com/max/809/0*OGXESWeD2rJms_Dk.png)

And set its value as _awswaf_ (see all types [here>>>](https://docs.logz.io/user-guide/log-shipping/built-in-log-types.html)), so ELK will be able to parse logs JSON:

![](https://cdn-images-1.medium.com/max/663/0*qEUjeVma4iJIjzFr.png)

Click _Save_, and the bucket was connected to your Logz.io account.

### WAF logs — fields

Now, need to attach an ALB for the WebACL and wait for traffic to see something in the ACL’s logs.

Attach the ACL to an Ingress — [`alb.ingress.kubernetes.io/wafv2-acl-arn`](https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.1/guide/ingress/annotations/#wafv2-acl-arn):

![](https://cdn-images-1.medium.com/max/1024/0*wtG2qkbNYrgELrkl.png)

In a couple of minutes, check the S3 — a new directory with our prefix will be created with a year, which was added by Kinesis:

![](https://cdn-images-1.medium.com/max/418/0*zs8I0hcaPXkz_nr5.png)

And the first log file:

![](https://cdn-images-1.medium.com/max/1024/0*8b0ax9TfFGTOgF8j.png)

Again, wait a couple of minutes so that the log will be delivered to Kibana.

ELK will parse its fields, and we can use them during searches.

There are two mainly used fields: type: "awsfaw", to watch all logs, or webaclId to select only a specific WebACL's log:

![](https://cdn-images-1.medium.com/max/1024/0*rA7IGYcUjByr2lAD.png)

Also, worth using the following fields in the results table (see all available in the [documentation>>>](https://docs.aws.amazon.com/waf/latest/developerguide/logging.html#logging-fields)):

- `httpRequest.clientIp`: an IP from where a request originated
- `httpRequest.uri`: URI used in a request
- `httpRequest.args`: its arguments
- `terminatingRuleId`: a WebACL's Rule, that was the final one with ALLOW or BLOCK
- `ruleGroupList`: all rules of a WebACL, that were inspected for a request

Let’s see what exactly we can find in a log message.

At first, let’s check a request, that was allowed to access our application, i.e. with the ALLOW action:

![](https://cdn-images-1.medium.com/max/1024/0*i9ENco_22D_7EsBv.png)

Here is a request that came from IP _99.***.***.101_ to the URI _/***/126/like_, and it was inspected by rules specified in the `ruleGroupList` chain, and finally it was allowed with the `Default_Action` of the WebACL.

Also, in the `ruleGroupList` we can see that one rule was applied to the request - `SignalNonBrowserUserAgent`, but as we have that rule set to the _COUNT_ action, then it was ignored:

![](https://cdn-images-1.medium.com/max/746/0*CLeIoeaAkk4W3whM.png)

And finally in the `terminatingRuleId` вwe see that the `Default_Action` was applied because of `SignalNonBrowserUserAgent` was excluded due to the COUNT action.

If a request was blocked, then in the `Default_Action` we will see a rule that was applied with the _BLOCK_ action:

![](https://cdn-images-1.medium.com/max/1024/0*9rrz7x8pMrxPYs_T.png)

#### Redacted fields — excluding data from logs

In the log, we can get some data that we don’t want to be there, such as cookies info or authentication tokens:

![](https://cdn-images-1.medium.com/max/630/0*Optxc8mFxuyAok3V.png)

To remove that data, go to the logging settings, then in the _Redacted fields_ choose fields that need to be excluded.

In that current case, it will be Headers and two fields from there  — ` authorization` and `cookie`:

![](https://cdn-images-1.medium.com/max/575/0*Hk9P0EBtRKg4DzqT.png)

Apply changes and now you’ll see their values as _REDACTED_:

![](https://cdn-images-1.medium.com/max/519/0*ow3845LTKU-TxoUP.png)

### Logz.io alerting

Also, it’s a good idea to get notifications on blocks.

Let’s create an alert via [Logz.io integration](https://support.atlassian.com/opsgenie/docs/integrate-opsgenie-with-logzio/) with Opsgenie, and Opsgenie will send us notifications to our Slack.

Describe an alert’s conditions, in the _Filters_ add `action != ALLOW`:

![](https://cdn-images-1.medium.com/max/1024/0*j3KAUVctNs6xeVJ3.png)

Choose a channel to send alerts to, and fields that you’d like to see in a message:

![](https://cdn-images-1.medium.com/max/505/0*ASDIdDy73LPYAXNn.png)

Wait for an alert in Slack:

![](https://cdn-images-1.medium.com/max/658/0*QKpUPl5R4bP8B4yL.png)

Done.

_Originally published at_ [_RTFM: Linux, DevOps, and system administration_](https://rtfm.co.ua/en/aws-waf-webacl-logging-and-logz-io-integration/)_._

* * *