Repository: DataDog/documentation
Pull request #28417: [DOCS-8666] Add OP Performance doc
Author: login: maycmlee
Labels: name: Architecture
description: Everything related to the Doc backend
name: editorial review
description: Waiting on a more in-depth review
name: mergequeue-status: done
Created: 2025-03-27T20:06:33Z | Merged: 2025-04-02T21:13:11Z

<!-- *Note: Please remember to review the Datadog Documentation [Contribution Guidelines](https://github.com/DataDog/documentation/blob/master/CONTRIBUTING.md) if you have not yet done so.* -->

### What does this PR do? What is the motivation?
<!-- A brief description of the change being made with this pull request. What is your motivation for the PR? -->

### Merge instructions

<!-- 
If you're waiting for a release or there are other considerations that you want us to be aware of, list them here. 
If the PR is ready to be merged once it receives the required reviews, check the box below after you've created the PR.
-->

Merge readiness:
- [ ] Ready for merge

**For Datadog employees**:
Merge queue is enabled in this repo. Your branch name MUST follow the `<name>/<description>` convention and include the forward slash (`/`). Without this format, your pull request will not pass in CI, the GitLab pipeline will not run, and you won't get a branch preview. Getting a branch preview makes it easier for us to check any issues with your PR, such as broken links.

If your branch doesn't follow this format, rename it or create a new branch and PR.

To have your PR automatically merged after it receives the required reviews, add the following PR comment:

```
/merge
```

### Additional notes
<!-- Anything else we should know when reviewing?-->

<!-- Previewing the PR: Assuming you are a Datadog employee and named your branch `<yourname>/<description>`, a preview build will run and links to the preview output will be auto-generated and posted in the PR comments. The links will 404 until the preview build is finished running. -->



Diff:
diff --git a/config/_default/menus/main.en.yaml b/config/_default/menus/main.en.yaml
index 9b6f2fc889d..df16ce4e8a3 100644
--- a/config/_default/menus/main.en.yaml
+++ b/config/_default/menus/main.en.yaml
@@ -5334,256 +5334,261 @@ menu:
       identifier: observability_pipelines_metrics
       parent: observability_pipelines_monitoring
       weight: 601
+    - name: Performance
+      identifier: observability_pipelines_performance
+      url: observability_pipelines/performance/
+      parent: observability_pipelines
+      weight: 7
     - name: Sources
       url: observability_pipelines/sources/
       parent: observability_pipelines
       identifier: observability_pipelines_sources
-      weight: 7
+      weight: 8
     - name: Amazon Data Firehose
       url: observability_pipelines/sources/amazon_data_firehose/
       parent: observability_pipelines_sources
       identifier: observability_pipelines_sources_amazon_data_firehose
-      weight: 701
+      weight: 801
     - name: Amazon S3
       url: observability_pipelines/sources/amazon_s3/
       parent: observability_pipelines_sources
       identifier: observability_pipelines_sources_amazon_s3
-      weight: 702
+      weight: 802
     - name: Datadog Agent
       url: observability_pipelines/sources/datadog_agent/
       parent: observability_pipelines_sources
       identifier: observability_pipelines_sources_datadog_agent
-      weight: 703
+      weight: 803
     - name: Fluent
       url: observability_pipelines/sources/fluent/
       parent: observability_pipelines_sources
       identifier: observability_pipelines_sources_fluent
-      weight: 704
+      weight: 804
     - name: Google Pub/Sub
       url: observability_pipelines/sources/google_pubsub/
       parent: observability_pipelines_sources
       identifier: observability_pipelines_sources_google_pubsub
-      weight: 705
+      weight: 805
     - name: HTTP Client
       url: observability_pipelines/sources/http_client/
       parent: observability_pipelines_sources
       identifier: observability_pipelines_sources_http_client
-      weight: 706
+      weight: 806
     - name: HTTP Server
       url: observability_pipelines/sources/http_server/
       parent: observability_pipelines_sources
       identifier: observability_pipelines_sources_http_server
-      weight: 707
+      weight: 807
     - name: Kafka
       url: observability_pipelines/sources/kafka/
       parent: observability_pipelines_sources
       identifier: observability_pipelines_sources_kafka
-      weight: 708
+      weight: 808
     - name: Logstash
       url: observability_pipelines/sources/logstash/
       parent: observability_pipelines_sources
       identifier: observability_pipelines_sources_logstash
-      weight: 709
+      weight: 809
     - name: Splunk HEC
       url: observability_pipelines/sources/splunk_hec/
       parent: observability_pipelines_sources
       identifier: observability_pipelines_sources_splunk_hec
-      weight: 710
+      weight: 810
     - name: Splunk TCP
       url: observability_pipelines/sources/splunk_tcp/
       parent: observability_pipelines_sources
       identifier: observability_pipelines_sources_splunk_tcp
-      weight: 711
+      weight: 811
     - name: Sumo Logic Hosted Collector
       url: observability_pipelines/sources/sumo_logic/
       parent: observability_pipelines_sources
       identifier: observability_pipelines_sources_sumo_logic
-      weight: 712
+      weight: 812
     - name: Syslog
       url: observability_pipelines/sources/syslog/
       parent: observability_pipelines_sources
       identifier: observability_pipelines_sources_syslog
-      weight: 713
+      weight: 813
     - name: Processors
       url: observability_pipelines/processors/
       parent: observability_pipelines
       identifier: observability_pipelines_processors
-      weight: 8
+      weight: 9
     - name: Add Environment Variables
       url: observability_pipelines/processors/add_environment_variables
       parent: observability_pipelines_processors
       identifier: observability_pipelines_processors_add_environment_variables
-      weight: 801
+      weight: 901
     - name: Add hostname
       url: observability_pipelines/processors/add_hostname
       parent: observability_pipelines_processors
       identifier: observability_pipelines_processors_add_hostname
-      weight: 802
+      weight: 902
     - name: Deduplicate
       url: observability_pipelines/processors/dedupe
       parent: observability_pipelines_processors
       identifier: observability_pipelines_processors_dedupe
-      weight: 803
+      weight: 903
     - name: Edit fields
       url: observability_pipelines/processors/edit_fields
       parent: observability_pipelines_processors
       identifier: observability_pipelines_processors_edit_fields
-      weight: 804
+      weight: 904
     - name: Enrichment Table
       url: observability_pipelines/processors/enrichment_table
       parent: observability_pipelines_processors
       identifier: observability_pipelines_processors_enrichment_table
-      weight: 805
+      weight: 905
     - name: Filter
       url: observability_pipelines/processors/filter
       parent: observability_pipelines_processors
       identifier: observability_pipelines_processors_filter
-      weight: 806
+      weight: 906
     - name: Generate Metrics
       url: observability_pipelines/processors/generate_metrics
       parent: observability_pipelines_processors
       identifier: observability_pipelines_processors_generate_metrics
-      weight: 807
+      weight: 907
     - name: Grok Parser
       url: observability_pipelines/processors/grok_parser
       parent: observability_pipelines_processors
       identifier: observability_pipelines_processors_grok_parser
-      weight: 808
+      weight: 908
     - name: Parse JSON
       url: observability_pipelines/processors/parse_json
       parent: observability_pipelines_processors
       identifier: observability_pipelines_processors_parse_json
-      weight: 809
+      weight: 909
     - name: Parse XML
       url: observability_pipelines/processors/parse_xml
       parent: observability_pipelines_processors
       identifier: observability_pipelines_processors_parse_xml
-      weight: 810
+      weight: 910
     - name: Quota
       url: observability_pipelines/processors/quota
       parent: observability_pipelines_processors
       identifier: observability_pipelines_processors_quota
-      weight: 811
+      weight: 911
     - name: Reduce
       url: observability_pipelines/processors/reduce
       parent: observability_pipelines_processors
       identifier: observability_pipelines_processors_reduce
-      weight: 812
+      weight: 912
     - name: Remap to OCSF
       url: observability_pipelines/processors/remap_ocsf
       parent: observability_pipelines_processors
       identifier: observability_pipelines_processors_remap_ocsf
-      weight: 813
+      weight: 913
     - name: Sample
       url: observability_pipelines/processors/sample
       parent: observability_pipelines_processors
       identifier: observability_pipelines_processors_sample
-      weight: 814
+      weight: 914
     - name: Sensitive Data Scanner
       url: observability_pipelines/processors/sensitive_data_scanner
       parent: observability_pipelines_processors
       identifier: observability_pipelines_processors_sensitive_data_scanner
-      weight: 815
+      weight: 915
     - name: Split Array
       url: observability_pipelines/processors/split_array
       parent: observability_pipelines_processors
       identifier: observability_pipelines_processors_split_array
-      weight: 816
+      weight: 916
     - name: Destinations
       url: observability_pipelines/destinations/
       parent: observability_pipelines
       identifier: observability_pipelines_destinations
-      weight: 9
+      weight: 10
     - name: Amazon OpenSearch
       url: observability_pipelines/destinations/amazon_opensearch/
       parent: observability_pipelines_destinations
       identifier: observability_pipelines_amazon_opensearch
-      weight: 901
+      weight: 1001
     - name: Amazon S3
       identifier: observability_pipelines_destinations_amazon_s3
       url: observability_pipelines/destinations/amazon_s3/
       parent: observability_pipelines_destinations
-      weight: 902
+      weight: 1002
     - name: Azure Storage
       identifier: observability_pipelines_azure_storage
       url: observability_pipelines/destinations/azure_storage/
       parent: observability_pipelines_destinations
-      weight: 903
+      weight: 1003
     - name: CrowdStrike NG-SIEM
       identifier: observability_pipelines_crowdstrike_ng_siem
       url: observability_pipelines/destinations/crowdstrike_ng_siem/
       parent: observability_pipelines_destinations
-      weight: 904
+      weight: 1004
     - name: Datadog Logs
       url: observability_pipelines/destinations/datadog_logs/
       parent: observability_pipelines_destinations
       identifier: observability_pipelines_datadog_logs
-      weight: 905
+      weight: 1005
     - name: Elasticsearch
       url: observability_pipelines/destinations/elasticsearch/
       parent: observability_pipelines_destinations
       identifier: observability_pipelines_elasticsearch
-      weight: 906
+      weight: 1006
     - name: Google Chronicle
       url: observability_pipelines/destinations/google_chronicle
       parent: observability_pipelines_destinations
       identifier: observability_pipelines_google_chronicle
-      weight: 907
+      weight: 1007
     - name: Google Cloud Storage
       identifier: observability_pipelines_google_cloud_storage
       url: /observability_pipelines/destinations/google_cloud_storage/
       parent: observability_pipelines_destinations
-      weight: 908
+      weight: 1008
     - name: Microsoft Sentinel
       identifier: observability_pipelines_microsoft_sentinel
       url: /observability_pipelines/destinations/microsoft_sentinel/
       parent: observability_pipelines_destinations
-      weight: 909
+      weight: 1009
     - name: New Relic
       identifier: observability_pipelines_new_relic
       url: /observability_pipelines/destinations/new_relic/
       parent: observability_pipelines_destinations
-      weight: 910
+      weight: 1010
     - name: OpenSearch
       url: observability_pipelines/destinations/opensearch
       parent: observability_pipelines_destinations
       identifier: observability_pipelines_opensearch
-      weight: 911
+      weight: 1011
     - name: SentinelOne
       url: observability_pipelines/destinations/sentinelone
       parent: observability_pipelines_destinations
       identifier: observability_pipelines_sentinelone
-      weight: 912
+      weight: 1012
     - name: Syslog
       url: observability_pipelines/destinations/syslog
       parent: observability_pipelines_destinations
       identifier: observability_pipelines_syslog
-      weight: 913
+      weight: 1013
     - name: Splunk HEC
       url: observability_pipelines/destinations/splunk_hec
       parent: observability_pipelines_destinations
       identifier: observability_pipelines_splunk_hec
-      weight: 914
+      weight: 1014
     - name: Sumo Logic Hosted Collector
       url: observability_pipelines/destinations/sumo_logic_hosted_collector
       parent: observability_pipelines_destinations
       identifier: observability_pipelines_sumo_logic_hosted_collector
-      weight: 915
+      weight: 1015
     - name: Best Practices for Scaling Observability Pipelines
       url: observability_pipelines/best_practices_for_scaling_observability_pipelines/
       parent: observability_pipelines
       identifier: observability_pipelines_best_practices_for_scaling_observability_pipelines
-      weight: 10
+      weight: 11
     - name: Guides
       url: observability_pipelines/guide/
       parent: observability_pipelines
       identifier: observability_pipelines_guide
-      weight: 11
+      weight: 12
     - name: Troubleshooting
       url: observability_pipelines/troubleshooting/
       identifier: observability_pipelines_troubleshooting
       parent: observability_pipelines
-      weight: 12
+      weight: 13
     - name: Log Management
       url: logs/
       pre: log
diff --git a/content/en/observability_pipelines/performance.md b/content/en/observability_pipelines/performance.md
new file mode 100644
index 00000000000..c92c3b3dc3a
--- /dev/null
+++ b/content/en/observability_pipelines/performance.md
@@ -0,0 +1,41 @@
+---
+title: Performance
+disable_toc: false
+further_reading:
+- link: "observability_pipelines/set_up_pipelines#set-up-a-pipeline"
+  tag: "Documentation"
+  text: "Set up a pipeline"
+- link: "observability_pipelines/sources"
+  tag: "Documentation"
+  text: "Sources"
+- link: "observability_pipelines/processors"
+  tag: "Documentation"
+  text: "Processors"
+- link: "observability_pipelines/destinations"
+  tag: "Documentation"
+  text: "Destinations"
+---
+
+## Overview
+
+Sometimes problems can occur even when you try to ensure your Observability Pipelines deployments are sized for the expected load. For example, an application might start generating more logs than usual, or the downstream service you are sending data to starts responding slower than expected. To address these issues:
+- Observability Pipelines propagates backpressure, which signals that the system cannot process events immediately upon receiving them.
+- Observability Pipelines components also have in-memory buffering in case the next component is busy processing incoming data.
+
+## Backpressure
+
+Backpressure is a signal that events cannot be processed as soon as they are received, and that signal is propagated up through the pipeline. For example, if a processor is sending events to a sink, and the sink cannot process it as fast as it is receiving the events, backpressure is propagated from the sink to the processor and all the way back to the source.
+
+Backpressure determines if the system should slow down the consumption or acceptance of events because it is too busy to handle more work. In some cases though, the system should not immediately propagate backpressure, because this could lead to the constant slowing down of upstream components, and potentially cause issues outside of Observability Pipelines. For example, the system prevents processes from slowing down when a component barely exceeds the saturation threshold. It can also handle temporary slowdowns and outages with external services that receive data from destinations.
+
+## In-memory buffering for components
+
+All components in Observability Pipelines have a small in-memory buffer between them. The buffer is the channel that two components communicate over. It ensures that there is a small amount of space, typically 100 events, that can be used to send events even if the component on the receiving end is busy. This allows maximizing throughput when workloads are not entirely uniform.
+
+Buffering protects against temporary overloads or outages for a given workload. The buffering model prioritizes performance when handling an excess of events, an amount that is beyond what a destination can process, by using in-memory buffers on destinations. By default, a destination's default buffer size is increased from 100 events to 500 events. The buffer capacity is increased because destinations are typically the primary source of backpressure in any given Observability Pipelines topology. They communicate to services over the network, where latency may be introduced or outages may temporarily occur.
+
+Observability Pipelines destination's buffers are configured to block events, which means it waits indefinitely to write to a buffer that is full. This is to make sure observability data is reliably processed in the order it was given. Additionally, as mentioned earlier, blocking induces backpressure and signals upstream components to slow down event acceptance or consumption. As a result, although the system retains all data, it accumulates at the edge.
+
+## Further reading
+
+{{< partial name="whats-next/whats-next.html" >}}
\ No newline at end of file


Reviews:
user: login: maycmlee
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-02T16:50:10Z
user: login: git-thuerk-done
body: Hi @maycmlee this looks good, just a few suggestions to clean up passive voice (where relevant), and also wondering if this page could use a Further reading section ?
state: CHANGES_REQUESTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-02T19:37:10Z
user: login: maycmlee
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-02T20:12:42Z
user: login: maycmlee
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-02T20:15:04Z
user: login: git-thuerk-done
body: Thanks @maycmlee looks good! Good catch on the Nav! 
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-04-02T20:51:31Z

Review comments:
diff_hunk: @@ -0,0 +1,24 @@
+---
+title: Performance
+disable_toc: false
+---
+
+## Overview
+
+Sometimes problems can occur even when you try to ensure your Observability Pipelines deployments are sized for the expected load. For example, an application might start generating more logs than usual, or the downstream service you are sending data to starts responding slower than expected. To address these issues:
+- Observability Pipelines propagates backpressure, which is a signal that events cannot be processed as quickly as they are received.
+- Observability Pipelines components also have in-memory buffering in case the component receiving the data is busy.
+
+## Backpressure
+
+Backpressure is a signal that events cannot be processed as quickly as they are received, and that signal is propagated up through the pipeline. For example, if a processor is sending events to a sink, and the sink cannot process it as fast as it is receiving the events, backpressure is propagated from the sink to the processor and all the way back to the source.
path: content/en/observability_pipelines/performance.md
user: login: maycmlee
body: ```suggestion
Backpressure is a signal that events cannot be processed as soon as they are received, and that signal is propagated up through the pipeline. For example, if a processor is sending events to a sink, and the sink cannot process it as fast as it is receiving the events, backpressure is propagated from the sink to the processor and all the way back to the source.
```
created_at: 2025-04-02T16:49:11Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,24 @@
+---
+title: Performance
+disable_toc: false
+---
+
+## Overview
+
+Sometimes problems can occur even when you try to ensure your Observability Pipelines deployments are sized for the expected load. For example, an application might start generating more logs than usual, or the downstream service you are sending data to starts responding slower than expected. To address these issues:
+- Observability Pipelines propagates backpressure, which is a signal that events cannot be processed as quickly as they are received.
path: content/en/observability_pipelines/performance.md
user: login: maycmlee
body: ```suggestion
- Observability Pipelines propagates backpressure, which is a signal that events cannot be processed as soon as they are received.
```
created_at: 2025-04-02T16:49:21Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,24 @@
+---
+title: Performance
+disable_toc: false
+---
+
+## Overview
+
+Sometimes problems can occur even when you try to ensure your Observability Pipelines deployments are sized for the expected load. For example, an application might start generating more logs than usual, or the downstream service you are sending data to starts responding slower than expected. To address these issues:
+- Observability Pipelines propagates backpressure, which is a signal that events cannot be processed as quickly as they are received.
+- Observability Pipelines components also have in-memory buffering in case the component receiving the data is busy.
+
+## Backpressure
+
+Backpressure is a signal that events cannot be processed as quickly as they are received, and that signal is propagated up through the pipeline. For example, if a processor is sending events to a sink, and the sink cannot process it as fast as it is receiving the events, backpressure is propagated from the sink to the processor and all the way back to the source.
+
+Backpressure determines if the system should slow down the consumption or acceptance of events because it is too busy to handle more work. In some cases though, the system should not immediately propagate backpressure, because this could lead to the constant slowing down of upstream components, and potentially cause issues outside of Observability Pipelines. For example, the system avoids slowing processes down when a component just crosses over the threshold of being fully saturated. The system is also able to handle temporary slowdowns and outages with external services that destinations send data to.
+
+## In-memory buffering for components
+
+All components in Observability Pipelines have a small in-memory buffer between them. The buffer is the channel that two components communicate over. It ensures that there is a small amount of space, typically 100 events, that can be used to send events even if the component on the receiving end is busy. This allows maximizing throughput when workloads are not entirely uniform.
+
+Buffering protects against temporary overloads or outages for a given workload. The buffering model prioritizes performance when handling an excess of events, an amount that is beyond what a destination can process, by using in-memory buffers on destinations. By default, a destination's default buffer size is increased from 100 events to 500 events. The buffer capacity is increased because destinations are typically the primary source of backpressure in any given Observability Pipelines topology. They communicate to services over the network, where latency may be introduced or outages may temporarily occur.
+
+Observability Pipelines destination's buffers are configured to block events, which means it waits indefinitely to write to a buffer that is currently full. This is to make sure observability data is reliably processed in the order it was given. Additionally, as mentioned earlier, blocking induces backpressure and is an important signal to upstream components that they should slow down the acceptance or consumption of events. The result is that, while no data is lost, data piles up at the edge.
path: content/en/observability_pipelines/performance.md
user: login: maycmlee
body: ```suggestion
Observability Pipelines destination's buffers are configured to block events, which means it waits indefinitely to write to a buffer that is full. This is to make sure observability data is reliably processed in the order it was given. Additionally, as mentioned earlier, blocking induces backpressure and is an important signal to upstream components that they should slow down the acceptance or consumption of events. The result is that, while no data is lost, data piles up at the edge.
```
created_at: 2025-04-02T16:50:01Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,24 @@
+---
+title: Performance
+disable_toc: false
+---
+
+## Overview
+
+Sometimes problems can occur even when you try to ensure your Observability Pipelines deployments are sized for the expected load. For example, an application might start generating more logs than usual, or the downstream service you are sending data to starts responding slower than expected. To address these issues:
+- Observability Pipelines propagates backpressure, which is a signal that events cannot be processed as soon as they are received.
path: content/en/observability_pipelines/performance.md
user: login: git-thuerk-done
body: ```suggestion
- Observability Pipelines propagates backpressure, which signals that the system cannot process events immediately upon receiving them.
```
Just cleaning up a bit to remove the passive voice part of the sentence
created_at: 2025-04-02T19:28:40Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,24 @@
+---
+title: Performance
+disable_toc: false
+---
+
+## Overview
+
+Sometimes problems can occur even when you try to ensure your Observability Pipelines deployments are sized for the expected load. For example, an application might start generating more logs than usual, or the downstream service you are sending data to starts responding slower than expected. To address these issues:
+- Observability Pipelines propagates backpressure, which is a signal that events cannot be processed as soon as they are received.
+- Observability Pipelines components also have in-memory buffering in case the component receiving the data is busy.
path: content/en/observability_pipelines/performance.md
user: login: git-thuerk-done
body: ```suggestion
- Observability Pipelines components also have in-memory buffering in case the next component is busy processing incoming data.
```
same with this one, but feel free to take or leave it, just a suggestion :) 
created_at: 2025-04-02T19:30:24Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,24 @@
+---
+title: Performance
+disable_toc: false
+---
+
+## Overview
+
+Sometimes problems can occur even when you try to ensure your Observability Pipelines deployments are sized for the expected load. For example, an application might start generating more logs than usual, or the downstream service you are sending data to starts responding slower than expected. To address these issues:
+- Observability Pipelines propagates backpressure, which is a signal that events cannot be processed as soon as they are received.
+- Observability Pipelines components also have in-memory buffering in case the component receiving the data is busy.
+
+## Backpressure
+
+Backpressure is a signal that events cannot be processed as soon as they are received, and that signal is propagated up through the pipeline. For example, if a processor is sending events to a sink, and the sink cannot process it as fast as it is receiving the events, backpressure is propagated from the sink to the processor and all the way back to the source.
+
+Backpressure determines if the system should slow down the consumption or acceptance of events because it is too busy to handle more work. In some cases though, the system should not immediately propagate backpressure, because this could lead to the constant slowing down of upstream components, and potentially cause issues outside of Observability Pipelines. For example, the system avoids slowing processes down when a component just crosses over the threshold of being fully saturated. The system is also able to handle temporary slowdowns and outages with external services that destinations send data to.
path: content/en/observability_pipelines/performance.md
user: login: git-thuerk-done
body: ```suggestion
Backpressure determines if the system should slow down the consumption or acceptance of events because it is too busy to handle more work. In some cases though, the system should not immediately propagate backpressure, because this could lead to the constant slowing down of upstream components, and potentially cause issues outside of Observability Pipelines. For example, the system prevents processes from slowing down when a component barely exceeds the saturation threshold. It can also handle temporary slowdowns and outages with external services that receive data from destinations.
```
Just suggest to re-word this a bit to simplify and make it more clear. (and remove the word "just" that vale is complaining about) :) 
created_at: 2025-04-02T19:32:43Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,24 @@
+---
+title: Performance
+disable_toc: false
+---
+
+## Overview
+
+Sometimes problems can occur even when you try to ensure your Observability Pipelines deployments are sized for the expected load. For example, an application might start generating more logs than usual, or the downstream service you are sending data to starts responding slower than expected. To address these issues:
+- Observability Pipelines propagates backpressure, which is a signal that events cannot be processed as soon as they are received.
+- Observability Pipelines components also have in-memory buffering in case the component receiving the data is busy.
+
+## Backpressure
+
+Backpressure is a signal that events cannot be processed as soon as they are received, and that signal is propagated up through the pipeline. For example, if a processor is sending events to a sink, and the sink cannot process it as fast as it is receiving the events, backpressure is propagated from the sink to the processor and all the way back to the source.
+
+Backpressure determines if the system should slow down the consumption or acceptance of events because it is too busy to handle more work. In some cases though, the system should not immediately propagate backpressure, because this could lead to the constant slowing down of upstream components, and potentially cause issues outside of Observability Pipelines. For example, the system avoids slowing processes down when a component just crosses over the threshold of being fully saturated. The system is also able to handle temporary slowdowns and outages with external services that destinations send data to.
+
+## In-memory buffering for components
+
+All components in Observability Pipelines have a small in-memory buffer between them. The buffer is the channel that two components communicate over. It ensures that there is a small amount of space, typically 100 events, that can be used to send events even if the component on the receiving end is busy. This allows maximizing throughput when workloads are not entirely uniform.
+
+Buffering protects against temporary overloads or outages for a given workload. The buffering model prioritizes performance when handling an excess of events, an amount that is beyond what a destination can process, by using in-memory buffers on destinations. By default, a destination's default buffer size is increased from 100 events to 500 events. The buffer capacity is increased because destinations are typically the primary source of backpressure in any given Observability Pipelines topology. They communicate to services over the network, where latency may be introduced or outages may temporarily occur.
+
+Observability Pipelines destination's buffers are configured to block events, which means it waits indefinitely to write to a buffer that is full. This is to make sure observability data is reliably processed in the order it was given. Additionally, as mentioned earlier, blocking induces backpressure and is an important signal to upstream components that they should slow down the acceptance or consumption of events. The result is that, while no data is lost, data piles up at the edge.
path: content/en/observability_pipelines/performance.md
user: login: git-thuerk-done
body: ```suggestion
Observability Pipelines destination's buffers are configured to block events, which means it waits indefinitely to write to a buffer that is full. This is to make sure observability data is reliably processed in the order it was given. Additionally, as mentioned earlier, blocking induces backpressure and signals upstream components to slow down event acceptance or consumption. As a result, although the system retains all data, it accumulates at the edge.
```
I was a little confused what this means "piles up at the edge" ? So just suggesting to reword again for clarity.
created_at: 2025-04-02T19:36:19Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,24 @@
+---
+title: Performance
+disable_toc: false
+---
+
+## Overview
+
+Sometimes problems can occur even when you try to ensure your Observability Pipelines deployments are sized for the expected load. For example, an application might start generating more logs than usual, or the downstream service you are sending data to starts responding slower than expected. To address these issues:
+- Observability Pipelines propagates backpressure, which is a signal that events cannot be processed as soon as they are received.
+- Observability Pipelines components also have in-memory buffering in case the component receiving the data is busy.
+
+## Backpressure
+
+Backpressure is a signal that events cannot be processed as soon as they are received, and that signal is propagated up through the pipeline. For example, if a processor is sending events to a sink, and the sink cannot process it as fast as it is receiving the events, backpressure is propagated from the sink to the processor and all the way back to the source.
+
+Backpressure determines if the system should slow down the consumption or acceptance of events because it is too busy to handle more work. In some cases though, the system should not immediately propagate backpressure, because this could lead to the constant slowing down of upstream components, and potentially cause issues outside of Observability Pipelines. For example, the system avoids slowing processes down when a component just crosses over the threshold of being fully saturated. The system is also able to handle temporary slowdowns and outages with external services that destinations send data to.
+
+## In-memory buffering for components
+
+All components in Observability Pipelines have a small in-memory buffer between them. The buffer is the channel that two components communicate over. It ensures that there is a small amount of space, typically 100 events, that can be used to send events even if the component on the receiving end is busy. This allows maximizing throughput when workloads are not entirely uniform.
+
+Buffering protects against temporary overloads or outages for a given workload. The buffering model prioritizes performance when handling an excess of events, an amount that is beyond what a destination can process, by using in-memory buffers on destinations. By default, a destination's default buffer size is increased from 100 events to 500 events. The buffer capacity is increased because destinations are typically the primary source of backpressure in any given Observability Pipelines topology. They communicate to services over the network, where latency may be introduced or outages may temporarily occur.
+
+Observability Pipelines destination's buffers are configured to block events, which means it waits indefinitely to write to a buffer that is full. This is to make sure observability data is reliably processed in the order it was given. Additionally, as mentioned earlier, blocking induces backpressure and is an important signal to upstream components that they should slow down the acceptance or consumption of events. The result is that, while no data is lost, data piles up at the edge.
path: content/en/observability_pipelines/performance.md
user: login: maycmlee
body: @aliciascott edge is a technical terminology from edge computing:
https://vector.dev/docs/setup/going-to-prod/arch/agent/#processing-at-the-edge
https://www.techtarget.com/searchdatacenter/definition/edge-computing
created_at: 2025-04-02T20:12:42Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,24 @@
+---
+title: Performance
+disable_toc: false
+---
+
+## Overview
+
+Sometimes problems can occur even when you try to ensure your Observability Pipelines deployments are sized for the expected load. For example, an application might start generating more logs than usual, or the downstream service you are sending data to starts responding slower than expected. To address these issues:
+- Observability Pipelines propagates backpressure, which is a signal that events cannot be processed as soon as they are received.
+- Observability Pipelines components also have in-memory buffering in case the component receiving the data is busy.
path: content/en/observability_pipelines/performance.md
user: login: maycmlee
body: Thanks! I'm going to edit "another" to "the next component"
created_at: 2025-04-02T20:15:04Z
author_association: CONTRIBUTOR

PR comments:
user: login: github-actions[bot]
created_at: 2025-03-27T21:00:12Z
body: ## Preview links (active after the `build_preview` check completes)

### New or renamed files
* https://docs-staging.datadoghq.com/may/op-performance/observability_pipelines/performance
author_association: CONTRIBUTOR
user: login: maycmlee
created_at: 2025-04-02T20:31:38Z
body: Thanks @aliciascott! I committed the suggestions and added a further reading section. Also realized I deleted part of the nav, so I fixed that.
author_association: CONTRIBUTOR
user: login: maycmlee
created_at: 2025-04-02T20:57:13Z
body: /merge
author_association: CONTRIBUTOR
user: login: dd-devflow[bot]
created_at: 2025-04-02T20:57:17Z
body: <!-- "FeedbackWorkflowId":"8cbe4f91-7f52-4de2-99a3-cc0d9a402f31_36" -->
[View all feedbacks in Devflow UI.](https://mosaic.us1.ddbuild.io/devflow/logs/8cbe4f91-7f52-4de2-99a3-cc0d9a402f31_36)

`2025-04-02 20:57:17 UTC` :information_source: Start processing command `/merge`

---
`2025-04-02 20:57:24 UTC` :information_source: **MergeQueue:** pull request added to the queue

The expected merge time in `master` is approximately 20m (p90).

---
`2025-04-02 21:13:17 UTC` :information_source: **MergeQueue:** This merge request was merged

author_association: NONE