How to store multiple Organization's CloudTrail Logs into an S3 Bucket of a separate account
Tags: aws, organization, cloudtrail, s3
Published: 2022-04-09 07:32:50



### This guide will help you to set up a separate S3 bucket of a separate AWS account for Aggregating CloudTrail Logs into it from multiple Regions multiple AWS Organizations & Multiple AWS Accounts.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/btasgmhvtchl4h2vxg3w.png)
 

**<u>Infrastructure Overview</u>**
Here we have two Organisations One is `MyOrg_1` & another one is `MyOrg_2`. We will set up CloudTrail Orgnation Trail for both of the Organisations and will send the Logs into a separate S3 Bucket of another AWS account.

**MyOrg_1**:
**Organisation_ID**: _o-myorg1_
**Management Account ID**: _111111111111_

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/r28uh0fd0gwsfo2bhhm4.png)

**MyOrg_2**:
**Organisation_ID**: _o-myorg2_
**Management Account ID**: _222222222222_

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6dr4vuny7fz1qkkg5oap.png)

**S3 Bucket AWS Account Info**:
**Account Name**: _My_logging_
**Account Id**: _333333333333_

> This AWS Account can be member of any Organization or can be a separate AWS Account.


**<u>Implementation</u>**
**_Step 1_**: **Create a secure S3 bucket with less privilege access in that separate AWS account. In my case, it is `My_logging (333333333333`).**

Log in to that account and go to S3 Service, then create a bucket. I am going to create a bucket named `myorg-cloudtrail-logs`.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fihyh9ytxoc3y9m3ku6a.png)
 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vzcakh9rggrr5vojl6xi.png)

- **Bucket name:** give a bucket name. In my case `myorg-cloudtrail-logs`.
- **AWS Region:** Select your AWS Region. In my case `eu-west-1`. (Note: S3 Bucket Region and CloudTrail Trail Region should be the same)
- **Object Ownership:** ACLs disabled. (you can select as per your business requirement. But you can change it letter.)
- **Block Public Access settings for this bucket**: Should Tick Block all public access.
- **Bucket Versioning:** Should Enable the bucket versioning for this bucket.
- **Default encryption:**
 > Note: Nothing is mandatory. You will select as per your business requirements.

    - _Server-side-encrytion_: Enable
    -  _Encryption key type_: AWS Key Management Service Key (SSE-KMS)
    -  _AWS KMS key_: AWS managed key (aws/s3)
    -  _Bucket Key_: Enable

- **Advanced settings:**

> This can not be changed after creation & also remember this can not be enabled after bucket creation. This should be enabled while creating the bucket. For maintaining reliable secure logs you should enable this option.
    - _Object Lock_: Enable. 

Then click the `Create bucket` button.

The bucket will be created successfully.


**_Step 2-A:_** **Update Bucket Policy To Allow Management Accounts and Organization's put logs into it.**

Go to the bucket Permission tab and Edit the Bucket policy:
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/supz0ovqvltsspzcno9i.png)
 

```
{
  "Version": "2012–10–17",
  "Statement": [
    {
      "Sid": "AWSCloudTrailAclCheck20150319",
      "Effect": "Allow",
      "Principal": {
        "Service": "cloudtrail.amazonaws.com"
      },
      "Action": "s3:GetBucketAcl",
      "Resource": "arn:aws:s3:::myorg-cloudtrail-logs"
    },
    {
      "Sid": "AWSCloudTrailWrite20150319-account",
      "Effect": "Allow",
      "Principal": {
        "Service": "cloudtrail.amazonaws.com"
      },
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::myorg-cloudtrail-logs/AWSLogs/111111111111/*",
      "Condition": {
        "StringEquals": {
          "s3:x-amz-acl": "bucket-owner-full-control"
        }
      }
    },
    {
      "Sid": "AWSCloudTrailWrite20150319-organisation",
      "Effect": "Allow",
      "Principal": {
        "Service": "cloudtrail.amazonaws.com"
      },
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::myorg-cloudtrail-logs/AWSLogs/o-myorg1/*",
      "Condition": {
        "StringEquals": {
          "s3:x-amz-acl": "bucket-owner-full-control"
        }
      }
    }
  ]
}
```
This policy will allow your `111111111111` (MyOrg_1 Management Account) Account and `0-myorg1` (MyOrg_1 Organization Id) Organization to put logs into that bucket.

> Note: Update the bucket name account Id and Organization Id with your information. 

Then Click `Save changes` Button.
Example:
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nz63jz512xctqxa40zt9.png)
 

**_Step 3-A:_** **Create Organization Trail in the Management Account of `MyOrg_1` Organization.**

Go to the Management Account (`111111111111`) console and open CloudTrail and Click `Create trail` Button.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4exj67mgo5ouv60u74ed.png)
 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ordp0kythgj5k869gysq.png)

- **Trail name**: Give a trail name. Ex: `MyOrgTrail`
- **Enable for all accounts in my organization**: Yes (By clicking this option will create a Trail of your all member accounts CloudTrail.)

> Note: This option is only visible in the management account of an Organization.

- **Storage location**: Use existing S3 bucket.
    - **Trail log bucket name**: `myorg-cloudtrail-logs` 

> Note: put your bucket name. Which you have just created in a separate AWS account

- **Log file SSE-KMS encryption**: Disable (If you don’t need it plz disable it)
- **Log file validation**: Enabled (It will validate your log file)
- **SNS notification delivery**: Disable (If you need it, You can enable it later)
- **CloudWatch Logs**: Disable (If you need it, You can enable it later)

Click `Next`
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3djxqqmt5cr6bjayy5xu.png)
 

- **Event type**: Select all events (As per your business requirements. You can change them later).
- **Management events**:
- **API activity**: Read, Write
- **Data events**: S3, DynamoDB, Lambda (As per your business requirements. You can change them later).
- **Insights events**: API call rate, API error rate (As per your business requirements. You can change them later).
Click `Next`

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/r5rt5i19i10h49cb2beq.png)
 

Review and if everything ok then Click `Create trail` Button. It will create your Organization Trail.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zcnv3p5qrccsgx5qx2sh.png)
 

It will create a trail of all member accounts all regions of this Organization. Will send all Logs into that S3 Bucket.

So you have successfully done for one Organisation. Now if you can follow the `Step-4` then you will find your `MyOrg_1` Organization Logs have been created already.

**_Step 2-B:_** **Update Bucket Policy To Allow Aonther Management Accounts and Organization's put logs into it.**

To log another Organization we need to update the S3 bucket policy as like `Step 2-A`.

Go to the `myorg_logging(333333333333)` account and open the S3 service and update the `bucket policy` of `myorg-cloudtrail-logs` bucket.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/f1clwzztg8jizwldaiqi.png)
 
     
Add this option with the existing bucket policy.
```
{
      "Sid": "AWSCloudTrailWrite20150319-account2",
      "Effect": "Allow",
      "Principal": {
        "Service": "cloudtrail.amazonaws.com"
      },
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::myorg-cloudtrail-logs/AWSLogs/222222222222/*",
      "Condition": {
        "StringEquals": {
          "s3:x-amz-acl": "bucket-owner-full-control"
        }
      }
    },
    {
      "Sid": "AWSCloudTrailWrite20150319-organisation2",
      "Effect": "Allow",
      "Principal": {
        "Service": "cloudtrail.amazonaws.com"
      },
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::myorg-cloudtrail-logs/AWSLogs/o-myorg2/*",
      "Condition": {
        "StringEquals": {
          "s3:x-amz-acl": "bucket-owner-full-control"
        }
      }
    }
```

Then your bucket policy will look like this:
```
{
  "Version": "2012–10–17",
  "Statement": [
    {
      "Sid": "AWSCloudTrailAclCheck20150319",
      "Effect": "Allow",
      "Principal": {
        "Service": "cloudtrail.amazonaws.com"
      },
      "Action": "s3:GetBucketAcl",
      "Resource": "arn:aws:s3:::myorg-cloudtrail-logs"
    },
    {
      "Sid": "AWSCloudTrailWrite20150319-account",
      "Effect": "Allow",
      "Principal": {
        "Service": "cloudtrail.amazonaws.com"
      },
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::myorg-cloudtrail-logs/AWSLogs/111111111111/*",
      "Condition": {
        "StringEquals": {
          "s3:x-amz-acl": "bucket-owner-full-control"
        }
      }
    },
    {
      "Sid": "AWSCloudTrailWrite20150319-organisation",
      "Effect": "Allow",
      "Principal": {
        "Service": "cloudtrail.amazonaws.com"
      },
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::myorg-cloudtrail-logs/AWSLogs/o-myorg1/*",
      "Condition": {
        "StringEquals": {
          "s3:x-amz-acl": "bucket-owner-full-control"
        }
      }
    },
    {
      "Sid": "AWSCloudTrailWrite20150319-account2",
      "Effect": "Allow",
      "Principal": {
        "Service": "cloudtrail.amazonaws.com"
      },
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::myorg-cloudtrail-logs/AWSLogs/222222222222/*",
      "Condition": {
        "StringEquals": {
          "s3:x-amz-acl": "bucket-owner-full-control"
        }
      }
    },
    {
      "Sid": "AWSCloudTrailWrite20150319-organisation2",
      "Effect": "Allow",
      "Principal": {
        "Service": "cloudtrail.amazonaws.com"
      },
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::myorg-cloudtrail-logs/AWSLogs/o-myorg2/*",
      "Condition": {
        "StringEquals": {
          "s3:x-amz-acl": "bucket-owner-full-control"
        }
      }
    }
  ]
}
```

Example:
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mbiyt6831jrrkapurbal.png)
 

Then click the `Save changes` button. It will update your bucket policy.

**_Step 3-B:_** **Create CloudTrail of MyOrg_2 Organization.**
Then open the management AWS account of your `MyOrg_2` Organization. And follow `Step 3-A`. It will be the same, Even the Trail name and everything.

**_Step 4:_** **Finding the logs**
Now Go to that S3 bucket you will see already some logs have been created.

There will be 4 folders. Two are MyOrg_1 Management Account Id & Organization ID and another two are MyOrg_2 Management Account Id and Organization ID.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qc3dwdvghmeskz62bhxa.png)

Now go to any organization you will see all the accounts of that organization

- Inside `MyOrg_1`
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/i8a9u1pil4yxp0ldapma.png)
 
- Inside `MyOrg_2`
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/so32kkt29v2tnjl24f0g.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l0edpnqlqc87w6rq1fox.png)
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/f72111m2bk77bry64bak.png)
  


_Congratulations!_ You have successfully created a Separate Storage Location for Your Multiple Organizations CloudTrail Logs.


> Now you got the idea of how all of this works. Now If you have to aggregate another Organization's Or AWS Account's Trail logs into that S3 bucket then you can do that by following the `Step-2A` and `Step-3A`.

Now to reduce cost you can enable S3 lifecycle policies for this bucket. To enable lifecycle policy follow this [S3 lifecycle documentation](https://dev.to/noyonict/simplify-your-data-lifecycle-optimize-storage-costs-with-amazon-s3-lifecycle-apc).


**Summary**
In this post, I showed “`Aggregating Multi Regions Multi Organizations CloudTrail Logs into an S3 Bucket`”.
To learn more, read the AWS [CloudTrail documentation](https://docs.aws.amazon.com/cloudtrail/?id=docs_gateway).


Thanks for reading! Happy Cloud Computing!

Connect with me: [Linkedin](https://www.linkedin.com/in/noyonict/)