Repository: DataDog/documentation
Pull request #29386: [DOCS-10541] Update OP AWS authentication docs
Author: login: maycmlee
Labels: name: Architecture
description: Everything related to the Doc backend
name: mergequeue-status: done
Created: 2025-05-15T20:38:10Z | Merged: 2025-05-21T17:36:23Z

<!-- *Note: Please remember to review the Datadog Documentation [Contribution Guidelines](https://github.com/DataDog/documentation/blob/master/CONTRIBUTING.md) if you have not yet done so.* -->

### What does this PR do? What is the motivation?
<!-- A brief description of the change being made with this pull request. What is your motivation for the PR? -->

### Merge instructions

<!-- 
If you're waiting for a release or there are other considerations that you want us to be aware of, list them here. 
If the PR is ready to be merged once it receives the required reviews, check the box below after you've created the PR.
-->

Merge readiness:
- [ ] Ready for merge

**For Datadog employees**:
Merge queue is enabled in this repo. Your branch name MUST follow the `<name>/<description>` convention and include the forward slash (`/`). Without this format, your pull request will not pass in CI, the GitLab pipeline will not run, and you won't get a branch preview. Getting a branch preview makes it easier for us to check any issues with your PR, such as broken links.

If your branch doesn't follow this format, rename it or create a new branch and PR.

To have your PR automatically merged after it receives the required reviews, add the following PR comment:

```
/merge
```

### Additional notes
<!-- Anything else we should know when reviewing?-->

<!-- Previewing the PR: Assuming you are a Datadog employee and named your branch `<yourname>/<description>`, a preview build will run and links to the preview output will be auto-generated and posted in the PR comments. The links will 404 until the preview build is finished running. -->



Diff:
diff --git a/content/en/observability_pipelines/destinations/amazon_s3.md b/content/en/observability_pipelines/destinations/amazon_s3.md
index 26b2ac7031b..e2ebf69dfda 100644
--- a/content/en/observability_pipelines/destinations/amazon_s3.md
+++ b/content/en/observability_pipelines/destinations/amazon_s3.md
@@ -59,6 +59,14 @@ You can route logs from Observability Pipelines to Snowflake using the Amazon S3
 
 ## How the destination works
 
+### AWS Authentication
+
+{{% observability_pipelines/aws_authentication/instructions %}}
+
+#### Permissions
+
+{{% observability_pipelines/aws_authentication/amazon_security_lake/permissions %}}
+
 ### Event batching
 
 A batch of events is flushed when one of these parameters is met. See [event batching][7] for more information.
diff --git a/content/en/observability_pipelines/destinations/amazon_security_lake.md b/content/en/observability_pipelines/destinations/amazon_security_lake.md
index 2377812464e..bca83837d95 100644
--- a/content/en/observability_pipelines/destinations/amazon_security_lake.md
+++ b/content/en/observability_pipelines/destinations/amazon_security_lake.md
@@ -23,18 +23,16 @@ Set up the Amazon Security Lake destination and its environment variables when y
 
 {{% observability_pipelines/configure_existing_pipelines/destination_env_vars/amazon_security_lake %}}
 
-## AWS Authentication
+## How the destination works
 
-{{% observability_pipelines/aws_authentication/amazon_security_lake/intro %}}
+### AWS Authentication
 
 {{% observability_pipelines/aws_authentication/instructions %}}
 
-### Permissions
+#### Permissions
 
 {{% observability_pipelines/aws_authentication/amazon_security_lake/permissions %}}
 
-## How the destination works
-
 ### Event batching
 
 A batch of events is flushed when one of these parameters is met. See [event batching][2] for more information.
diff --git a/content/en/observability_pipelines/sources/amazon_data_firehose.md b/content/en/observability_pipelines/sources/amazon_data_firehose.md
index 7b4331a0fee..a4981120bcc 100644
--- a/content/en/observability_pipelines/sources/amazon_data_firehose.md
+++ b/content/en/observability_pipelines/sources/amazon_data_firehose.md
@@ -19,4 +19,12 @@ Select and set up this source when you [set up a pipeline][1]. The information b
 
 {{% observability_pipelines/log_source_configuration/amazon_data_firehose %}}
 
+## AWS Authentication
+
+{{% observability_pipelines/aws_authentication/instructions %}}
+
+### Permissions
+
+{{% observability_pipelines/aws_authentication/amazon_s3_source/permissions %}}
+
 [1]: /observability_pipelines/set_up_pipelines/
\ No newline at end of file
diff --git a/content/en/observability_pipelines/sources/amazon_s3.md b/content/en/observability_pipelines/sources/amazon_s3.md
index adbdbff7eae..1d29e9422c3 100644
--- a/content/en/observability_pipelines/sources/amazon_s3.md
+++ b/content/en/observability_pipelines/sources/amazon_s3.md
@@ -17,8 +17,6 @@ Select and set up this source when you [set up a pipeline][1]. The information b
 
 ## AWS Authentication
 
-{{% observability_pipelines/aws_authentication/amazon_s3_source/intro %}}
-
 {{% observability_pipelines/aws_authentication/instructions %}}
 
 ### Permissions
diff --git a/layouts/shortcodes/observability_pipelines/aws_authentication/instructions.md b/layouts/shortcodes/observability_pipelines/aws_authentication/instructions.md
index 3bccc684857..71be30e8cd7 100644
--- a/layouts/shortcodes/observability_pipelines/aws_authentication/instructions.md
+++ b/layouts/shortcodes/observability_pipelines/aws_authentication/instructions.md
@@ -1,15 +1,3 @@
-To set up AWS authentication:
+The Observability Pipelines Worker uses the standard AWS credential provider chain for authentication. See [AWS SDKs and Tools standardized credential providers][10101] for more information.
 
-1. Create an IAM role if you don't have one already. The role needs, at a minimum, these [permissions](#permissions) to interact with the component. See [Create a role to delegate permissions to an IAM user][10101] for more information.
-1. In your AWS configuration file, create a new profile using the `role_arn` from the role you created in step 1.
-1. When installing the Observability Pipelines Worker, ensure you set the `AWS_PROFILE` and `AWS_CONFIG_FILE` environment variables. The `AWS_CONFIG_FILE` variable is the path to your AWS configuration file. Set `AWS_PROFILE` to the name of the profile you created in step 2. See [Configuration and credential file setting in the AWS CLI][10101] for more information. This is an example of a profile configuration:
-    ```
-    [profile profile_name]
-    region = us-east-1
-    output = json
-    role_arn = arn:aws:iam::123456789:role/MyRole
-    source_profile = default
-    ```
-
-[10101]: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user.html
-[10102]: https://docs.aws.amazon.com/cli/v1/userguide/cli-configure-files.html
\ No newline at end of file
+[10101]: https://docs.aws.amazon.com/sdkref/latest/guide/standardized-credentials.html
\ No newline at end of file
diff --git a/layouts/shortcodes/observability_pipelines/configure_existing_pipelines/destination_env_vars/datadog_archives_amazon_s3.en.md b/layouts/shortcodes/observability_pipelines/configure_existing_pipelines/destination_env_vars/datadog_archives_amazon_s3.en.md
index a9ddd7f54a5..629cbce621e 100644
--- a/layouts/shortcodes/observability_pipelines/configure_existing_pipelines/destination_env_vars/datadog_archives_amazon_s3.en.md
+++ b/layouts/shortcodes/observability_pipelines/configure_existing_pipelines/destination_env_vars/datadog_archives_amazon_s3.en.md
@@ -1,8 +1,3 @@
 #### Amazon S3
 
-- AWS access key ID of your S3 archive:
-	- Stored in the environment variable: `DD_OP_DESTINATION_DATADOG_ARCHIVES_AWS_ACCESS_KEY_ID`
-
-- AWS secret access key ID of your S3 archive:
-	- The AWS secret access key ID for the S3 archive bucket.
-	- Stored in the environment variable `DD_OP_DESTINATION_DATADOG_ARCHIVES_AWS_SECRET_KEY`.
\ No newline at end of file
+There are no environment variables to configure.
\ No newline at end of file
diff --git a/layouts/shortcodes/observability_pipelines/configure_log_archive/amazon_s3/docker.en.md b/layouts/shortcodes/observability_pipelines/configure_log_archive/amazon_s3/docker.en.md
index 3909f7ed71c..bbf96e2d743 100644
--- a/layouts/shortcodes/observability_pipelines/configure_log_archive/amazon_s3/docker.en.md
+++ b/layouts/shortcodes/observability_pipelines/configure_log_archive/amazon_s3/docker.en.md
@@ -1,19 +1,6 @@
-#### Create an IAM user
+#### Create an IAM user or role
 
-Create an IAM user and attach the IAM policy you created earlier to it.
+Create an IAM [user][301] or [role][302] and attach the policy to it.
 
-1. Navigate to the [IAM console][301].
-1. Select **Users** in the left side menu.
-1. Click **Create user**.
-1. Enter a username.
-1. Click **Next**.
-1. Select **Attach policies directly**.
-1. Choose the IAM policy you created earlier to attach to the new IAM user.
-1. Click **Next**.
-1. Optionally, add tags.
-1. Click **Create user**.
-
-Create access credentials for the new IAM user. The AWS access key and AWS secret access key are added as environment variables in the [Install the Observability Pipelines Worker](#install-the-observability-pipelines-worker) step.
-
-
-[301]: https://console.aws.amazon.com/iam/
\ No newline at end of file
+[301]: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html
+[302]: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create.html
\ No newline at end of file
diff --git a/layouts/shortcodes/observability_pipelines/configure_log_archive/amazon_s3/linux_apt.en.md b/layouts/shortcodes/observability_pipelines/configure_log_archive/amazon_s3/linux_apt.en.md
index f96acfd3602..a05f96961ff 100644
--- a/layouts/shortcodes/observability_pipelines/configure_log_archive/amazon_s3/linux_apt.en.md
+++ b/layouts/shortcodes/observability_pipelines/configure_log_archive/amazon_s3/linux_apt.en.md
@@ -1,18 +1,6 @@
-#### Create an IAM user
+#### Create an IAM user or role
 
-Create an IAM user and attach the IAM policy you created earlier to it.
+Create an IAM [user][501] or [role][502] and attach the policy to it.
 
-1. Navigate to the [IAM console][501].
-1. Select **Users** in the left side menu.
-1. Click **Create user**.
-1. Enter a username.
-1. Click **Next**.
-1. Select **Attach policies directly**.
-1. Choose the IAM policy you created earlier to attach to the new IAM user.
-1. Click **Next**.
-1. Optionally, add tags.
-1. Click **Create user**.
-
-Create access credentials for the new IAM user. The AWS access key and AWS secret access key are added later as environment variables when you install the Observability Pipelines Worker.
-
-[501]: https://console.aws.amazon.com/iam/
\ No newline at end of file
+[501]: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html
+[502]: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create.html
\ No newline at end of file
diff --git a/layouts/shortcodes/observability_pipelines/configure_log_archive/amazon_s3/linux_rpm.en.md b/layouts/shortcodes/observability_pipelines/configure_log_archive/amazon_s3/linux_rpm.en.md
index 30440d4758f..134f36ff293 100644
--- a/layouts/shortcodes/observability_pipelines/configure_log_archive/amazon_s3/linux_rpm.en.md
+++ b/layouts/shortcodes/observability_pipelines/configure_log_archive/amazon_s3/linux_rpm.en.md
@@ -1,18 +1,6 @@
-#### Create an IAM user
+#### Create an IAM user or role
 
-Create an IAM user and attach the IAM policy you created earlier to it.
+Create an IAM [user][1001] or [role][1002] and attach the policy to it.
 
-1. Navigate to the [IAM console][1001].
-1. Select **Users** in the left side menu.
-1. Click **Create user**.
-1. Enter a username.
-1. Click **Next**.
-1. Select **Attach policies directly**.
-1. Choose the IAM policy you created earlier to attach to the new IAM user.
-1. Click **Next**.
-1. Optionally, add tags.
-1. Click **Create user**.
-
-Create access credentials for the new IAM user. The AWS access key and AWS secret access key are added as environment variables in the [Install the Observability Pipelines Worker](#install-the-observability-pipelines-worker) step.
-
-[1001]: https://console.aws.amazon.com/iam/
\ No newline at end of file
+[1001]: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html
+[1002]: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create.html
\ No newline at end of file
diff --git a/layouts/shortcodes/observability_pipelines/destination_env_vars/datadog_archives_amazon_s3.en.md b/layouts/shortcodes/observability_pipelines/destination_env_vars/datadog_archives_amazon_s3.en.md
index bd8610f78f1..261918fac28 100644
--- a/layouts/shortcodes/observability_pipelines/destination_env_vars/datadog_archives_amazon_s3.en.md
+++ b/layouts/shortcodes/observability_pipelines/destination_env_vars/datadog_archives_amazon_s3.en.md
@@ -1 +1 @@
-Enter the AWS access key ID and AWS secret access key for the S3 archive bucket you created earlier.
+There are no environment variables to configure.
diff --git a/layouts/shortcodes/observability_pipelines/destination_settings/datadog_archives_amazon_s3.en.md b/layouts/shortcodes/observability_pipelines/destination_settings/datadog_archives_amazon_s3.en.md
index d0fa10940f2..b593f56ac05 100644
--- a/layouts/shortcodes/observability_pipelines/destination_settings/datadog_archives_amazon_s3.en.md
+++ b/layouts/shortcodes/observability_pipelines/destination_settings/datadog_archives_amazon_s3.en.md
@@ -4,7 +4,8 @@
     - Prefixes are useful for partitioning objects. For example, you can use a prefix as an object key to store objects under a particular directory. If using a prefix for this purpose, it must end in `/` to act as a directory path; a trailing `/` is not automatically added.
     - See [template syntax][10051] if you want to route logs to different object keys based on specific fields in your logs.
 1. Select the storage class for your S3 bucket in the **Storage Class** dropdown menu.
-
-Your AWS access key ID and AWS secret access key are set as environment variables when you install the Worker later.
+1. Optionally, select an AWS authentication option. If you select **Assume role**:
+    1. Enter the ARN of the IAM role you want to assume.
+    1. Optionally, enter the assumed role session name and external ID.
 
 [10051]: /observability_pipelines/destinations/#template-syntax
\ No newline at end of file
diff --git a/layouts/shortcodes/observability_pipelines/prerequisites/amazon_s3.md b/layouts/shortcodes/observability_pipelines/prerequisites/amazon_s3.md
index 17e997cc605..6774ba87fd3 100644
--- a/layouts/shortcodes/observability_pipelines/prerequisites/amazon_s3.md
+++ b/layouts/shortcodes/observability_pipelines/prerequisites/amazon_s3.md
@@ -1,5 +1 @@
-To use Observability Pipelines' Amazon S3 source, you need to:
-- Configure a SQS queue to receive your S3 bucket notifications, which is required to use the Amazon S3 source.
-- Set up AWS authentication using `AWS_PROFILE` and `AWS_CONFIG FILE` environment variables. Observability Pipelines uses the credentials associated with those environment variables to collect logs from Amazon S3. See [AWS Authentication][10091] for more information.
-
-[10091]: /observability_pipelines/sources/amazon_s3/#aws-authentication
\ No newline at end of file
+To use Observability Pipelines' Amazon S3 source, you must configure a SQS queue to receive your S3 bucket notifications.
\ No newline at end of file
diff --git a/layouts/shortcodes/observability_pipelines/prerequisites/amazon_security_lake.md b/layouts/shortcodes/observability_pipelines/prerequisites/amazon_security_lake.md
index d6f2ee7cf7f..7a982bcfc34 100644
--- a/layouts/shortcodes/observability_pipelines/prerequisites/amazon_security_lake.md
+++ b/layouts/shortcodes/observability_pipelines/prerequisites/amazon_security_lake.md
@@ -13,9 +13,7 @@
             - AWS account ID
             - External ID
      - Select **Create and use a new service** for service access.
-1. Set up AWS authentication using `AWS_PROFILE` and `AWS_CONFIG FILE` environment variables. Observability Pipelines uses credentials associated with those environment variables to send logs to Amazon Security Lake. See [AWS Authentication][10074] for more information.
 
 [10071]: https://docs.aws.amazon.com/security-lake/latest/userguide/getting-started.html
 [10072]: https://docs.aws.amazon.com/security-lake/latest/userguide/custom-sources.html
-[10073]: https://docs.aws.amazon.com/security-lake/latest/userguide/get-started-console.html
-[10074]: /observability_pipelines/destinations/amazon_security_lake/#aws-authentication
\ No newline at end of file
+[10073]: https://docs.aws.amazon.com/security-lake/latest/userguide/get-started-console.html
\ No newline at end of file
diff --git a/layouts/shortcodes/observability_pipelines/source_settings/amazon_data_firehose.md b/layouts/shortcodes/observability_pipelines/source_settings/amazon_data_firehose.md
index af30af19b6e..9dc7ccacf6b 100644
--- a/layouts/shortcodes/observability_pipelines/source_settings/amazon_data_firehose.md
+++ b/layouts/shortcodes/observability_pipelines/source_settings/amazon_data_firehose.md
@@ -1,7 +1,10 @@
-Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required.<br>**Note**: All file paths are made relative to the configuration data directory, which is `/var/lib/observability-pipelines-worker/config/` by default. See [Advanced Configurations][10172] for more information. The file must be owned by the `observability-pipelines-worker group` and `observability-pipelines-worker` user, or at least readable by the group or user.
-- `Server Certificate Path`: The path to the certificate file that has been signed by your Certificate Authority (CA) Root File in DER or PEM (X.509).
-- `CA Certificate Path`: The path to the certificate file that is your Certificate Authority (CA) Root File in DER or PEM (X.509).
-- `Private Key Path`: The path to the `.key` private key file that belongs to your Server Certificate Path in DER or PEM (PKCS#8) format.
+1. Optionally, select an AWS authentication option. If you select **Assume role**:
+    1. Enter the ARN of the IAM role you want to assume.
+    1. Optionally, enter the assumed role session name and external ID.
+1. Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required.<br>**Note**: All file paths are made relative to the configuration data directory, which is `/var/lib/observability-pipelines-worker/config/` by default. See [Advanced Configurations][10172] for more information. The file must be owned by the `observability-pipelines-worker group` and `observability-pipelines-worker` user, or at least readable by the group or user.
+    - `Server Certificate Path`: The path to the certificate file that has been signed by your Certificate Authority (CA) Root File in DER or PEM (X.509).
+    - `CA Certificate Path`: The path to the certificate file that is your Certificate Authority (CA) Root File in DER or PEM (X.509).
+    - `Private Key Path`: The path to the `.key` private key file that belongs to your Server Certificate Path in DER or PEM (PKCS#8) format.
 
 [10172]: /observability_pipelines/advanced_configurations/
 <!-- 10172 link is used in multiple shortcodes, so if it is changed, make sure to update those shortcodes using find and replace -->
\ No newline at end of file
diff --git a/layouts/shortcodes/observability_pipelines/source_settings/amazon_s3.md b/layouts/shortcodes/observability_pipelines/source_settings/amazon_s3.md
index 5da67cef6d3..216e6ea2c71 100644
--- a/layouts/shortcodes/observability_pipelines/source_settings/amazon_s3.md
+++ b/layouts/shortcodes/observability_pipelines/source_settings/amazon_s3.md
@@ -1,4 +1,7 @@
 1. Enter the AWS region.
+1. Optionally, select an AWS authentication option. If you select **Assume role**:
+    1. Enter the ARN of the IAM role you want to assume.
+    1. Optionally, enter the assumed role session name and external ID.
 1. Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required.<br>**Note**: All file paths are made relative to the configuration data directory, which is `/var/lib/observability-pipelines-worker/config/` by default. See [Advanced Configurations][10172] for more information. The file must be owned by the `observability-pipelines-worker group` and `observability-pipelines-worker` user, or at least readable by the group or user.
     - `Server Certificate Path`: The path to the certificate file that has been signed by your Certificate Authority (CA) Root File in DER or PEM (X.509).
     - `CA Certificate Path`: The path to the certificate file that is your Certificate Authority (CA) Root File in DER or PEM (X.509).


Reviews:
user: login: git-thuerk-done
body: Looks good just one small comment!
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-20T20:48:37Z
user: login: maycmlee
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-20T21:05:04Z

Review comments:
path: content/en/observability_pipelines/destinations/amazon_security_lake.md
user: login: git-thuerk-done
body: Just noting this file isn't in the Left hand Nav, I see it's in Preview so likely intended but just checking!
created_at: 2025-05-20T20:44:01Z
author_association: CONTRIBUTOR
path: content/en/observability_pipelines/destinations/amazon_security_lake.md
user: login: maycmlee
body: Good eyes! This doc hasn't actually been released yet (as in made available), so that's why it's not in the nav.
created_at: 2025-05-20T21:05:04Z
author_association: CONTRIBUTOR

PR comments:
user: login: github-actions[bot]
created_at: 2025-05-15T20:41:42Z
body: ## Preview links (active after the `build_preview` check completes)




### Modified Files
* https://docs-staging.datadoghq.com/may/update-op-aws-auth/observability_pipelines/destinations/amazon_s3
* https://docs-staging.datadoghq.com/may/update-op-aws-auth/observability_pipelines/destinations/amazon_security_lake
* https://docs-staging.datadoghq.com/may/update-op-aws-auth/observability_pipelines/sources/amazon_data_firehose
* https://docs-staging.datadoghq.com/may/update-op-aws-auth/observability_pipelines/sources/amazon_s3
author_association: CONTRIBUTOR
user: login: maycmlee
created_at: 2025-05-21T16:45:10Z
body: /merge
author_association: CONTRIBUTOR
user: login: dd-devflow[bot]
created_at: 2025-05-21T16:45:15Z
body: <!-- "FeedbackWorkflowId":"9c717bac-1a90-493d-806b-accd1545d476_36" -->
[View all feedbacks in Devflow UI.](https://mosaic.us1.ddbuild.io/devflow/logs/9c717bac-1a90-493d-806b-accd1545d476_36)

`2025-05-21 16:45:14 UTC` :information_source: Start processing command `/merge`

---
`2025-05-21 16:45:25 UTC` :information_source: **MergeQueue:** waiting for PR to be ready

This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals.
**Note:** if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with `/remove` command.

---
`2025-05-21 17:17:23 UTC` :information_source: **MergeQueue:** merge request added to the queue

The expected merge time in `master` is approximately 21m (p90).

---
`2025-05-21 17:36:27 UTC` :information_source: **MergeQueue:** This merge request was merged

author_association: NONE