Repository: DataDog/datadog-agent
Pull request #36556: [EBPF] attacher: limit multiple periodic scans of the same process
Author: login: gjulianm
Labels: name: changelog/no-changelog
description: No changelog entry needed
name: component/system-probe
name: team/ebpf-platform
name: qa/done
description: QA done before merge and regressions are covered by tests
name: medium review
description: PR review might take time
name: ask-review
description: Ask required teams to review this PR
Created: 2025-04-29T18:28:40Z | Merged: 2025-05-09T21:41:48Z

<!--
* Contributors are encouraged to read our [CONTRIBUTING](/CONTRIBUTING.md) documentation.
* Both Contributor and Reviewer Checklists are available at https://datadoghq.dev/datadog-agent/guidelines/contributing/#pull-requests.
* The pull request:
  * Should only fix one issue or add one feature at a time.
  * Must update the test suite for the relevant functionality.
  * Should pass all status checks before being reviewed or merged.
* Commit titles should be prefixed with general area of pull request's change.
* Please fill the below sections if possible with relevant information or links.
-->
### What does this PR do?

This PR ports #32400 to the uprobe attacher, which ensured that a process can be scanned more than once. While this feature existed already in the uprobe attacher, it didn't have a limit for the number of tries.

This is specially interesting for attachers with shared libraries enabled. With executable attaching, the FileRegistry ensured that we weren't repeating expensive operations for a process that already was scanned. However, with shared libraries the expensive operations are also before we call the FileRegistry (mainly scanning the process library maps files) so this limit is more interesting there.

### Motivation

https://github.com/DataDog/datadog-agent/pull/29313/files#r2066984432

Replicating behavior of previous USM library watcher.

### Describe how you validated your changes
<!--
Validate your changes before merge, ensuring that:
* Your PR is tested by static / unit / integrations / e2e tests
* Your PR description details which e2e tests cover your changes, if any
* The PR description contains details of how you validated your changes. If you validated changes manually and not through automated tests, add context on why automated tests did not fit your changes validation.

If you want additional validation by a second person, you can ask reviewers to do it. Describe how to set up an environment for manual tests in the PR description. Manual validation is expected to happen on every commit before merge.

Any manual validation step should then map to an automated test. Manual validation should not substitute automation, minus exceptions not supported by test tooling yet.
-->

Added a unit test for the new behavior. Tested in the load-test environment too with `istio` data as it had enabled periodic scans, with results showing no changes in performance nor accuracy.

<details>
<summary>Details</summary>

[Dashboard](https://dddev.datadoghq.com/dashboard/gqu-2wz-asy/usm-performance-evaluation-20?fromUser=true&refresh_mode=paused&tpl_var_base_agent-env%5B0%5D=gj-mscan-base&tpl_var_client-service%5B0%5D=golang-k6-client-http&tpl_var_compare_agent-env%5B0%5D=gj-mscan-new&tpl_var_http.version%5B0%5D=http%2F1.1&tpl_var_kube_cluster_name%5B0%5D=sp-test-env-1&tpl_var_server-service%5B0%5D=golang-httpbin&tpl_var_tls.library%5B0%5D=istio&from_ts=1746540015908&to_ts=1746542295574&live=false)
![Screenshot 2025-05-06 at 16 45 04](https://github.com/user-attachments/assets/8cb9ff8a-80f0-48ce-add6-7098fd3ec36c)

![Screenshot 2025-05-06 at 16 44 43](https://github.com/user-attachments/assets/a44624b4-cec3-4b88-97b5-ac966c9a61d1)


</details>



### Possible Drawbacks / Trade-offs

### Additional Notes
<!--
* Anything else we should know when reviewing?
* Include benchmarking information here whenever possible.
* Include info about alternatives that were considered and why the proposed
  version was chosen.
-->


Diff:
diff --git a/pkg/ebpf/uprobes/attacher.go b/pkg/ebpf/uprobes/attacher.go
index c7ec239835be..78271ce7f905 100644
--- a/pkg/ebpf/uprobes/attacher.go
+++ b/pkg/ebpf/uprobes/attacher.go
@@ -226,6 +226,11 @@ type AttacherConfig struct {
 	// events from happening. However, if it takes too long it can delay the next sync. In any case, synchronizations are
 	// usually performed every 30 seconds by default (ScanProcessesInterval), so it shouldn't be a problem.
 	OnSyncCallback func(map[uint32]struct{})
+
+	// MaxPeriodicScansPerProcess defines the maximum number of periodic scans we will perform for a given PID
+	// when EnablePeriodicScanNewProcesses is true. Useful to avoid re-scanning processes that have already
+	// been scanned, specially when shared libraries are being traced as scanning the maps file can be expensive.
+	MaxPeriodicScansPerProcess int
 }
 
 // SetDefaults configures the AttacherConfig with default values for those fields for which the compiler
@@ -242,6 +247,13 @@ func (ac *AttacherConfig) SetDefaults() {
 	if ac.EbpfConfig == nil {
 		ac.EbpfConfig = ebpf.NewConfig()
 	}
+
+	if ac.MaxPeriodicScansPerProcess == 0 {
+		// 2 seems a reasonable default, as we will give time (1 minute with the default interval)
+		// to the process to start and for any environmental errors to stop affecting the process
+		// and allow it to load the shared libraries we might be interested in.
+		ac.MaxPeriodicScansPerProcess = 2
+	}
 }
 
 // Validate checks whether the configuration is valid, returns nil if it is, an error message otherwise
@@ -360,6 +372,10 @@ type UprobeAttacher struct {
 
 	// processMonitor is the process monitor that we use to subscribe to process start and exit events
 	processMonitor ProcessMonitor
+
+	// scansPerPid is a map of PIDs to the number of times we have scanned them, to avoid re-scanning them
+	// too many times when EnablePeriodicScanNewProcesses is true
+	scansPerPid map[uint32]int
 }
 
 // NewUprobeAttacher creates a new UprobeAttacher. Receives as arguments
@@ -391,6 +407,7 @@ func NewUprobeAttacher(moduleName, name string, config AttacherConfig, mgr Probe
 		done:                   make(chan struct{}),
 		inspector:              inspector,
 		processMonitor:         processMonitor,
+		scansPerPid:            make(map[uint32]int),
 	}
 
 	utils.AddAttacher(moduleName, name, ua)
@@ -527,29 +544,51 @@ func (ua *UprobeAttacher) Sync(trackCreations, trackDeletions bool) error {
 	}
 
 	alivePIDs := make(map[uint32]struct{})
-	_ = kernel.WithAllProcs(ua.config.ProcRoot, func(pid int) error {
-		if pid == thisPID { // don't scan ourselves
+	_ = kernel.WithAllProcs(ua.config.ProcRoot, func(p int) error {
+		if p == thisPID { // don't scan ourselves
 			return nil
 		}
 
-		alivePIDs[uint32(pid)] = struct{}{}
+		pid := uint32(p)
+
+		alivePIDs[pid] = struct{}{}
 
 		if trackDeletions {
-			if _, ok := deletionCandidates[uint32(pid)]; ok {
+			if _, ok := deletionCandidates[pid]; ok {
 				// We have previously hooked into this process and it remains active,
 				// so we remove it from the deletionCandidates list, and move on to the next PID
-				delete(deletionCandidates, uint32(pid))
+				delete(deletionCandidates, pid)
 				return nil
 			}
 		}
 
-		if trackCreations {
+		if trackCreations && ua.scansPerPid[pid] < ua.config.MaxPeriodicScansPerProcess {
 			// This is a new PID so we attempt to attach SSL probes to it
-			_ = ua.AttachPID(uint32(pid))
+			ua.scansPerPid[pid]++
+			err := ua.AttachPID(pid)
+			if err == nil {
+				if ua.config.EnableDetailedLogging {
+					log.Debugf("uprobe attacher %s attached to process %d via periodic scan", ua.name, pid)
+				}
+
+				// Set the number of scans to the maximum so we don't try to scan it again
+				ua.scansPerPid[pid] = ua.config.MaxPeriodicScansPerProcess
+			} else {
+				if ua.shouldLogRegistryError(err) {
+					log.Warnf("could not attach to process %d: %v", pid, err)
+				}
+			}
 		}
 		return nil
 	})
 
+	// Clean up the scansPerPid map, removing all PIDs that are no longer alive
+	for pid := range ua.scansPerPid {
+		if _, ok := alivePIDs[pid]; !ok {
+			delete(ua.scansPerPid, pid)
+		}
+	}
+
 	if trackDeletions {
 		// At this point all entries from deletionCandidates are no longer alive, so
 		// we should detach our SSL probes from them
diff --git a/pkg/ebpf/uprobes/attacher_test.go b/pkg/ebpf/uprobes/attacher_test.go
index 31e69b262bbc..fdcc04a1b685 100644
--- a/pkg/ebpf/uprobes/attacher_test.go
+++ b/pkg/ebpf/uprobes/attacher_test.go
@@ -1089,3 +1089,133 @@ func methodHasBeenCalledWithPredicate(registry *MockFileRegistry, methodName str
 	}
 	return false
 }
+
+func TestSyncRetryAndReattach(t *testing.T) {
+	proc := FakeProcFSEntry{
+		Pid:     1,
+		Cmdline: "/bin/bash",
+		Command: "/bin/bash",
+		Exe:     "/bin/bash",
+	}
+	procFS := CreateFakeProcFS(t, []FakeProcFSEntry{proc})
+	emptyProcFS := CreateFakeProcFS(t, []FakeProcFSEntry{})
+
+	config := AttacherConfig{
+		ProcRoot: procFS,
+		Rules: []*AttachRule{
+			{
+				Targets: AttachToExecutable,
+				ProbesSelector: []manager.ProbesSelector{
+					&manager.ProbeSelector{ProbeIdentificationPair: manager.ProbeIdentificationPair{EBPFFuncName: "uprobe__SSL_connect"}},
+				},
+			},
+		},
+		EnablePeriodicScanNewProcesses: true,
+		MaxPeriodicScansPerProcess:     2,
+	}
+
+	registry := &MockFileRegistry{}
+	ua, err := NewUprobeAttacher(testModuleName, testAttacherName, config, nil, nil, nil, newMockProcessMonitor())
+	require.NoError(t, err)
+	require.NotNil(t, ua)
+
+	ua.fileRegistry = registry
+
+	// First attempt should fail, registry should report no processes
+	registry.On("Register", proc.Exe, proc.Pid, mock.Anything, mock.Anything, mock.Anything).Return(errors.New("inspection failed")).Once()
+	registry.On("GetRegisteredProcesses").Return(map[uint32]struct{}{}).Once()
+	err = ua.Sync(true, true)
+	require.NoError(t, err) // Sync itself doesn't return errors from individual attachments
+	require.Equal(t, 1, ua.scansPerPid[proc.Pid])
+	registry.AssertExpectations(t)
+
+	// Second attempt should succeed, registry should still report no processes
+	registry.On("Register", proc.Exe, proc.Pid, mock.Anything, mock.Anything, mock.Anything).Return(nil).Once()
+	registry.On("GetRegisteredProcesses").Return(map[uint32]struct{}{}).Once()
+	err = ua.Sync(true, true)
+	require.NoError(t, err)
+	require.Equal(t, 2, ua.scansPerPid[proc.Pid])
+	registry.AssertExpectations(t)
+
+	// Scan an empty procFS to simulate a process exit, the registry in this case does know about the process
+	// to simulate it has been attached correctly
+	registry.On("GetRegisteredProcesses").Return(map[uint32]struct{}{proc.Pid: {}}).Once()
+	registry.On("Unregister", proc.Pid).Return(nil).Once()
+	ua.config.ProcRoot = emptyProcFS
+	err = ua.Sync(true, true)
+	require.NoError(t, err)
+	require.Empty(t, ua.scansPerPid)
+	registry.AssertExpectations(t)
+
+	// Should be able to re-attach to same PID, registry doesn't know about the process as it was detached before
+	ua.config.ProcRoot = procFS
+	registry.On("Register", proc.Exe, proc.Pid, mock.Anything, mock.Anything, mock.Anything).Return(nil).Once()
+	registry.On("GetRegisteredProcesses").Return(map[uint32]struct{}{}).Once()
+	err = ua.Sync(true, true)
+	require.NoError(t, err)
+	require.Equal(t, config.MaxPeriodicScansPerProcess, ua.scansPerPid[proc.Pid]) // attached correctly, so marked as already scanned to the max
+	registry.AssertExpectations(t)
+}
+
+func TestSyncNoAttach(t *testing.T) {
+	proc := FakeProcFSEntry{
+		Pid:     1,
+		Cmdline: "/bin/bash",
+		Command: "/bin/bash",
+		Exe:     "/bin/bash",
+	}
+	procFS := CreateFakeProcFS(t, []FakeProcFSEntry{proc})
+	emptyProcFS := CreateFakeProcFS(t, []FakeProcFSEntry{})
+
+	config := AttacherConfig{
+		ProcRoot: procFS,
+		Rules: []*AttachRule{
+			{
+				Targets: AttachToExecutable,
+				ProbesSelector: []manager.ProbesSelector{
+					&manager.ProbeSelector{ProbeIdentificationPair: manager.ProbeIdentificationPair{EBPFFuncName: "uprobe__SSL_connect"}},
+				},
+			},
+		},
+		EnablePeriodicScanNewProcesses: true,
+		MaxPeriodicScansPerProcess:     2,
+	}
+
+	registry := &MockFileRegistry{}
+	ua, err := NewUprobeAttacher(testModuleName, testAttacherName, config, nil, nil, nil, newMockProcessMonitor())
+	require.NoError(t, err)
+	require.NotNil(t, ua)
+
+	ua.fileRegistry = registry
+
+	// All attempts should fail
+	registry.On("Register", proc.Exe, proc.Pid, mock.Anything, mock.Anything, mock.Anything).Return(errors.New("inspection failed")).Once()
+	registry.On("GetRegisteredProcesses").Return(map[uint32]struct{}{}).Once()
+	err = ua.Sync(true, true)
+	require.NoError(t, err) // Sync itself doesn't return errors from individual attachments
+	require.Equal(t, 1, ua.scansPerPid[proc.Pid])
+	registry.AssertExpectations(t)
+
+	// Second attempt should still fail
+	registry.On("Register", proc.Exe, proc.Pid, mock.Anything, mock.Anything, mock.Anything).Return(errors.New("inspection failed")).Once()
+	registry.On("GetRegisteredProcesses").Return(map[uint32]struct{}{}).Once()
+	err = ua.Sync(true, true)
+	require.NoError(t, err)
+	require.Equal(t, 2, ua.scansPerPid[proc.Pid])
+	registry.AssertExpectations(t)
+
+	// Third attempt should not even get to the registry, so we don't expect any calls to the Register method
+	registry.On("GetRegisteredProcesses").Return(map[uint32]struct{}{}).Once()
+	err = ua.Sync(true, true)
+	require.NoError(t, err)
+	require.Equal(t, 2, ua.scansPerPid[proc.Pid])
+	registry.AssertExpectations(t)
+
+	// Scan an empty procFS to simulate the process exiting, it should disapper from the scansPerPid map
+	registry.On("GetRegisteredProcesses").Return(map[uint32]struct{}{}).Once()
+	ua.config.ProcRoot = emptyProcFS
+	err = ua.Sync(true, true)
+	require.NoError(t, err)
+	require.Empty(t, ua.scansPerPid)
+	registry.AssertExpectations(t)
+}


Reviews:
user: login: copilot-pull-request-reviewer[bot]
body: ## Pull Request Overview

This pull request ports a fix from #32400 to the uprobe attacher to allow a process to be scanned multiple times, which is particularly useful when shared libraries are enabled. The key changes include adding a new unit test for periodic scanning behavior in attacher_test.go and updating the UprobeAttacher implementation in attacher.go to track scan counts via a new scansPerPid map.

### Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

| File                                    | Description                                                    |
| --------------------------------------- | -------------------------------------------------------------- |
| pkg/ebpf/uprobes/attacher_test.go       | Added unit test to validate multiple periodic scans behavior   |
| pkg/ebpf/uprobes/attacher.go             | Updated periodic scan logic and configuration defaults         |


<details>
<summary>Comments suppressed due to low confidence (1)</summary>

**pkg/ebpf/uprobes/attacher_test.go:1118**
* [nitpick] Consider adding additional test cases that verify the behavior when multiple failed attachments occur, ensuring that the scan counter increments as expected. This can help verify the boundary conditions of MaxPeriodicScansPerProcess.
```
registry.On("GetRegisteredProcesses").Return(map[uint32]struct{}{}).Once()
```
</details>


state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-30T11:22:37Z
user: login: mbertrone
body: Left a small comment.

LGTM (but I have limited context about it)
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-08T12:09:29Z
user: login: gjulianm
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-08T14:56:04Z

Review comments:
diff_hunk: @@ -527,25 +541,40 @@ func (ua *UprobeAttacher) Sync(trackCreations, trackDeletions bool) error {
 	}
 
 	alivePIDs := make(map[uint32]struct{})
-	_ = kernel.WithAllProcs(ua.config.ProcRoot, func(pid int) error {
-		if pid == thisPID { // don't scan ourselves
+	_ = kernel.WithAllProcs(ua.config.ProcRoot, func(p int) error {
+		if p == thisPID { // don't scan ourselves
 			return nil
 		}
 
-		alivePIDs[uint32(pid)] = struct{}{}
+		pid := uint32(p)
+
+		alivePIDs[pid] = struct{}{}
 
 		if trackDeletions {
-			if _, ok := deletionCandidates[uint32(pid)]; ok {
+			if _, ok := deletionCandidates[pid]; ok {
 				// We have previously hooked into this process and it remains active,
 				// so we remove it from the deletionCandidates list, and move on to the next PID
-				delete(deletionCandidates, uint32(pid))
+				delete(deletionCandidates, pid)
 				return nil
 			}
 		}
 
-		if trackCreations {
+		if trackCreations && ua.scansPerPid[pid] < ua.config.MaxPeriodicScansPerProcess {
 			// This is a new PID so we attempt to attach SSL probes to it
path: pkg/ebpf/uprobes/attacher.go
user: login: Copilot
body: Consider adding a comment clarifying why you increment the scan counter before calling AttachPID. This will help future maintainers understand that both successful and failed attachment attempts are tracked intentionally.
```suggestion
			// Increment the scan counter before attempting to attach SSL probes.
			// This ensures that both successful and failed attachment attempts are tracked.
```
created_at: 2025-04-30T11:22:37Z
author_association: CONTRIBUTOR
diff_hunk: @@ -242,6 +247,10 @@ func (ac *AttacherConfig) SetDefaults() {
 	if ac.EbpfConfig == nil {
 		ac.EbpfConfig = ebpf.NewConfig()
 	}
+
+	if ac.MaxPeriodicScansPerProcess == 0 {
+		ac.MaxPeriodicScansPerProcess = 2
path: pkg/ebpf/uprobes/attacher.go
user: login: mbertrone
body: Would you mind elaborate on the rationale behind having `2` as default? Maybe adding a comment or documenting it?
created_at: 2025-05-08T12:04:59Z
author_association: CONTRIBUTOR
diff_hunk: @@ -242,6 +247,10 @@ func (ac *AttacherConfig) SetDefaults() {
 	if ac.EbpfConfig == nil {
 		ac.EbpfConfig = ebpf.NewConfig()
 	}
+
+	if ac.MaxPeriodicScansPerProcess == 0 {
+		ac.MaxPeriodicScansPerProcess = 2
path: pkg/ebpf/uprobes/attacher.go
user: login: gjulianm
body: Added a comment with some explanation, although it comes from the default set in #32400
created_at: 2025-05-08T14:56:03Z
author_association: CONTRIBUTOR

PR comments:
user: login: agent-platform-auto-pr[bot]
created_at: 2025-04-29T19:01:19Z
body: ## Uncompressed package size comparison

Comparison with [ancestor](https://github.com/DataDog/datadog-agent/commit/2e1c7f60e53a46343005e021cb33ee8eda836155) `2e1c7f60e53a46343005e021cb33ee8eda836155`


<details open>
  <summary> Diff per package </summary>

|package|diff|status|size|ancestor|threshold|
|--|--|--|--|--|--|
|datadog-agent-amd64-deb|0.01MB|⚠️|776.35MB|776.35MB|0.50MB|
|datadog-agent-x86_64-rpm|0.00MB|✅|785.30MB|785.29MB|0.50MB|
|datadog-agent-x86_64-suse|0.00MB|✅|785.30MB|785.29MB|0.50MB|
|datadog-agent-arm64-deb|0.00MB|✅|766.14MB|766.14MB|0.50MB|
|datadog-agent-aarch64-rpm|0.00MB|✅|775.07MB|775.07MB|0.50MB|
|datadog-dogstatsd-amd64-deb|0.00MB|✅|31.20MB|31.20MB|0.50MB|
|datadog-dogstatsd-x86_64-rpm|0.00MB|✅|31.28MB|31.28MB|0.50MB|
|datadog-dogstatsd-x86_64-suse|0.00MB|✅|31.28MB|31.28MB|0.50MB|
|datadog-dogstatsd-arm64-deb|0.00MB|✅|30.03MB|30.03MB|0.50MB|
|datadog-heroku-agent-amd64-deb|0.00MB|✅|377.90MB|377.90MB|0.50MB|
|datadog-iot-agent-amd64-deb|0.00MB|✅|60.57MB|60.57MB|0.50MB|
|datadog-iot-agent-x86_64-rpm|0.00MB|✅|60.65MB|60.65MB|0.50MB|
|datadog-iot-agent-x86_64-suse|0.00MB|✅|60.65MB|60.65MB|0.50MB|
|datadog-iot-agent-arm64-deb|0.00MB|✅|57.86MB|57.86MB|0.50MB|
|datadog-iot-agent-aarch64-rpm|0.00MB|✅|57.94MB|57.94MB|0.50MB|

</details>

## Decision
⚠️ Warning



author_association: CONTRIBUTOR
user: login: cit-pr-commenter[bot]
created_at: 2025-04-29T19:21:51Z
body: ## Regression Detector

# Regression Detector Results

[Metrics dashboard](https://app.datadoghq.com/dashboard/ykh-ua8-vcu/SMP-Regression-Detector-Metrics?fromUser=true&refresh_mode=paused&tpl_var_run-id%5B0%5D=125c1410-c2d8-4f17-89bf-755f09bf5be7&view=spans&from_ts=1746817615000&to_ts=1746818215000&live=false)  
[Target profiles](https://app.datadoghq.com/profiling/explorer?query=env%3Asingle-machine-performance%20service%3Adatadog-agent%20job_id%3A125c1410-c2d8-4f17-89bf-755f09bf5be7&agg_m=count&agg_m_source=base&agg_t=count&fromUser=false&viz=stream&start=1746810415000&end=1746821815000&paused=true)  
Run ID: 125c1410-c2d8-4f17-89bf-755f09bf5be7

Baseline: 2e1c7f60e53a46343005e021cb33ee8eda836155
Comparison: 3543948e80eb8859cad4623f84ce162893d1da75
[Diff](2e1c7f60e53a46343005e021cb33ee8eda836155...3543948e80eb8859cad4623f84ce162893d1da75)


## Optimization Goals: ✅ No significant changes detected


<details>
<summary><h2>
Fine details of change detection per experiment
</h2></summary>

| perf | experiment                                   | goal               | Δ mean % | Δ mean % CI    | trials | links                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
|------|----------------------------------------------|--------------------|----------|----------------|--------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| ➖   | otlp_ingest_logs                             | memory utilization | +1.06    | [+0.93, +1.19] | 1      | [Logs](https://app.datadoghq.com/logs?query=experiment%3Aotlp_ingest_logs%20job_id%3A125c1410-c2d8-4f17-89bf-755f09bf5be7&agg_m=count&agg_m_source=base&agg_q=%40span.url&agg_q_source=base&agg_t=count&fromUser=true&index=single-machine-performance-target-logs&messageDisplay=inline&refresh_mode=paused&storage=hot&stream_sort=time%2Cdesc&top_n=100&top_o=top&viz=stream&x_missing=true&from_ts=1746810415000&to_ts=1746821815000&live=false)                                                                                                                                                                                                                                                                                                                |
| ➖   | quality_gate_idle                            | memory utilization | +0.39    | [+0.33, +0.45] | 1      | [Logs](https://app.datadoghq.com/logs?query=experiment%3Aquality_gate_idle%20job_id%3A125c1410-c2d8-4f17-89bf-755f09bf5be7&agg_m=count&agg_m_source=base&agg_q=%40span.url&agg_q_source=base&agg_t=count&fromUser=true&index=single-machine-performance-target-logs&messageDisplay=inline&refresh_mode=paused&storage=hot&stream_sort=time%2Cdesc&top_n=100&top_o=top&viz=stream&x_missing=true&from_ts=1746810415000&to_ts=1746821815000&live=false) [bounds checks dashboard](https://app.datadoghq.com/dashboard/vz3-jd5-bdi?fromUser=true&refresh_mode=paused&tpl_var_experiment%5B0%5D=quality_gate_idle&tpl_var_job_id%5B0%5D=125c1410-c2d8-4f17-89bf-755f09bf5be7&view=spans&from_ts=1746817615000&to_ts=1746818215000&live=false)                           |
| ➖   | tcp_syslog_to_blackhole                      | ingress throughput | +0.13    | [+0.06, +0.20] | 1      | [Logs](https://app.datadoghq.com/logs?query=experiment%3Atcp_syslog_to_blackhole%20job_id%3A125c1410-c2d8-4f17-89bf-755f09bf5be7&agg_m=count&agg_m_source=base&agg_q=%40span.url&agg_q_source=base&agg_t=count&fromUser=true&index=single-machine-performance-target-logs&messageDisplay=inline&refresh_mode=paused&storage=hot&stream_sort=time%2Cdesc&top_n=100&top_o=top&viz=stream&x_missing=true&from_ts=1746810415000&to_ts=1746821815000&live=false)                                                                                                                                                                                                                                                                                                         |
| ➖   | file_to_blackhole_0ms_latency                | egress throughput  | +0.10    | [-0.51, +0.71] | 1      | [Logs](https://app.datadoghq.com/logs?query=experiment%3Afile_to_blackhole_0ms_latency%20job_id%3A125c1410-c2d8-4f17-89bf-755f09bf5be7&agg_m=count&agg_m_source=base&agg_q=%40span.url&agg_q_source=base&agg_t=count&fromUser=true&index=single-machine-performance-target-logs&messageDisplay=inline&refresh_mode=paused&storage=hot&stream_sort=time%2Cdesc&top_n=100&top_o=top&viz=stream&x_missing=true&from_ts=1746810415000&to_ts=1746821815000&live=false)                                                                                                                                                                                                                                                                                                   |
| ➖   | uds_dogstatsd_20mb_12k_contexts_20_senders   | memory utilization | +0.06    | [-0.00, +0.12] | 1      | [Logs](https://app.datadoghq.com/logs?query=experiment%3Auds_dogstatsd_20mb_12k_contexts_20_senders%20job_id%3A125c1410-c2d8-4f17-89bf-755f09bf5be7&agg_m=count&agg_m_source=base&agg_q=%40span.url&agg_q_source=base&agg_t=count&fromUser=true&index=single-machine-performance-target-logs&messageDisplay=inline&refresh_mode=paused&storage=hot&stream_sort=time%2Cdesc&top_n=100&top_o=top&viz=stream&x_missing=true&from_ts=1746810415000&to_ts=1746821815000&live=false)                                                                                                                                                                                                                                                                                      |
| ➖   | file_to_blackhole_500ms_latency              | egress throughput  | +0.06    | [-0.58, +0.70] | 1      | [Logs](https://app.datadoghq.com/logs?query=experiment%3Afile_to_blackhole_500ms_latency%20job_id%3A125c1410-c2d8-4f17-89bf-755f09bf5be7&agg_m=count&agg_m_source=base&agg_q=%40span.url&agg_q_source=base&agg_t=count&fromUser=true&index=single-machine-performance-target-logs&messageDisplay=inline&refresh_mode=paused&storage=hot&stream_sort=time%2Cdesc&top_n=100&top_o=top&viz=stream&x_missing=true&from_ts=1746810415000&to_ts=1746821815000&live=false)                                                                                                                                                                                                                                                                                                 |
| ➖   | file_to_blackhole_300ms_latency              | egress throughput  | +0.02    | [-0.67, +0.71] | 1      | [Logs](https://app.datadoghq.com/logs?query=experiment%3Afile_to_blackhole_300ms_latency%20job_id%3A125c1410-c2d8-4f17-89bf-755f09bf5be7&agg_m=count&agg_m_source=base&agg_q=%40span.url&agg_q_source=base&agg_t=count&fromUser=true&index=single-machine-performance-target-logs&messageDisplay=inline&refresh_mode=paused&storage=hot&stream_sort=time%2Cdesc&top_n=100&top_o=top&viz=stream&x_missing=true&from_ts=1746810415000&to_ts=1746821815000&live=false)                                                                                                                                                                                                                                                                                                 |
| ➖   | uds_dogstatsd_to_api                         | ingress throughput | +0.02    | [-0.28, +0.32] | 1      | [Logs](https://app.datadoghq.com/logs?query=experiment%3Auds_dogstatsd_to_api%20job_id%3A125c1410-c2d8-4f17-89bf-755f09bf5be7&agg_m=count&agg_m_source=base&agg_q=%40span.url&agg_q_source=base&agg_t=count&fromUser=true&index=single-machine-performance-target-logs&messageDisplay=inline&refresh_mode=paused&storage=hot&stream_sort=time%2Cdesc&top_n=100&top_o=top&viz=stream&x_missing=true&from_ts=1746810415000&to_ts=1746821815000&live=false)                                                                                                                                                                                                                                                                                                            |
| ➖   | file_to_blackhole_1000ms_latency             | egress throughput  | +0.01    | [-0.64, +0.67] | 1      | [Logs](https://app.datadoghq.com/logs?query=experiment%3Afile_to_blackhole_1000ms_latency%20job_id%3A125c1410-c2d8-4f17-89bf-755f09bf5be7&agg_m=count&agg_m_source=base&agg_q=%40span.url&agg_q_source=base&agg_t=count&fromUser=true&index=single-machine-performance-target-logs&messageDisplay=inline&refresh_mode=paused&storage=hot&stream_sort=time%2Cdesc&top_n=100&top_o=top&viz=stream&x_missing=true&from_ts=1746810415000&to_ts=1746821815000&live=false)                                                                                                                                                                                                                                                                                                |
| ➖   | tcp_dd_logs_filter_exclude                   | ingress throughput | -0.00    | [-0.01, +0.01] | 1      | [Logs](https://app.datadoghq.com/logs?query=experiment%3Atcp_dd_logs_filter_exclude%20job_id%3A125c1410-c2d8-4f17-89bf-755f09bf5be7&agg_m=count&agg_m_source=base&agg_q=%40span.url&agg_q_source=base&agg_t=count&fromUser=true&index=single-machine-performance-target-logs&messageDisplay=inline&refresh_mode=paused&storage=hot&stream_sort=time%2Cdesc&top_n=100&top_o=top&viz=stream&x_missing=true&from_ts=1746810415000&to_ts=1746821815000&live=false)                                                                                                                                                                                                                                                                                                      |
| ➖   | file_to_blackhole_1000ms_latency_linear_load | egress throughput  | -0.00    | [-0.23, +0.23] | 1      | [Logs](https://app.datadoghq.com/logs?query=experiment%3Afile_to_blackhole_1000ms_latency_linear_load%20job_id%3A125c1410-c2d8-4f17-89bf-755f09bf5be7&agg_m=count&agg_m_source=base&agg_q=%40span.url&agg_q_source=base&agg_t=count&fromUser=true&index=single-machine-performance-target-logs&messageDisplay=inline&refresh_mode=paused&storage=hot&stream_sort=time%2Cdesc&top_n=100&top_o=top&viz=stream&x_missing=true&from_ts=1746810415000&to_ts=1746821815000&live=false)                                                                                                                                                                                                                                                                                    |
| ➖   | file_to_blackhole_100ms_latency              | egress throughput  | -0.02    | [-0.66, +0.63] | 1      | [Logs](https://app.datadoghq.com/logs?query=experiment%3Afile_to_blackhole_100ms_latency%20job_id%3A125c1410-c2d8-4f17-89bf-755f09bf5be7&agg_m=count&agg_m_source=base&agg_q=%40span.url&agg_q_source=base&agg_t=count&fromUser=true&index=single-machine-performance-target-logs&messageDisplay=inline&refresh_mode=paused&storage=hot&stream_sort=time%2Cdesc&top_n=100&top_o=top&viz=stream&x_missing=true&from_ts=1746810415000&to_ts=1746821815000&live=false)                                                                                                                                                                                                                                                                                                 |
| ➖   | file_to_blackhole_0ms_latency_http1          | egress throughput  | -0.04    | [-0.68, +0.60] | 1      | [Logs](https://app.datadoghq.com/logs?query=experiment%3Afile_to_blackhole_0ms_latency_http1%20job_id%3A125c1410-c2d8-4f17-89bf-755f09bf5be7&agg_m=count&agg_m_source=base&agg_q=%40span.url&agg_q_source=base&agg_t=count&fromUser=true&index=single-machine-performance-target-logs&messageDisplay=inline&refresh_mode=paused&storage=hot&stream_sort=time%2Cdesc&top_n=100&top_o=top&viz=stream&x_missing=true&from_ts=1746810415000&to_ts=1746821815000&live=false)                                                                                                                                                                                                                                                                                             |
| ➖   | file_to_blackhole_0ms_latency_http2          | egress throughput  | -0.08    | [-0.71, +0.55] | 1      | [Logs](https://app.datadoghq.com/logs?query=experiment%3Afile_to_blackhole_0ms_latency_http2%20job_id%3A125c1410-c2d8-4f17-89bf-755f09bf5be7&agg_m=count&agg_m_source=base&agg_q=%40span.url&agg_q_source=base&agg_t=count&fromUser=true&index=single-machine-performance-target-logs&messageDisplay=inline&refresh_mode=paused&storage=hot&stream_sort=time%2Cdesc&top_n=100&top_o=top&viz=stream&x_missing=true&from_ts=1746810415000&to_ts=1746821815000&live=false)                                                                                                                                                                                                                                                                                             |
| ➖   | docker_containers_memory                     | memory utilization | -0.13    | [-0.20, -0.05] | 1      | [Logs](https://app.datadoghq.com/logs?query=experiment%3Adocker_containers_memory%20job_id%3A125c1410-c2d8-4f17-89bf-755f09bf5be7&agg_m=count&agg_m_source=base&agg_q=%40span.url&agg_q_source=base&agg_t=count&fromUser=true&index=single-machine-performance-target-logs&messageDisplay=inline&refresh_mode=paused&storage=hot&stream_sort=time%2Cdesc&top_n=100&top_o=top&viz=stream&x_missing=true&from_ts=1746810415000&to_ts=1746821815000&live=false)                                                                                                                                                                                                                                                                                                        |
| ➖   | uds_dogstatsd_to_api_cpu                     | % cpu utilization  | -0.26    | [-1.09, +0.58] | 1      | [Logs](https://app.datadoghq.com/logs?query=experiment%3Auds_dogstatsd_to_api_cpu%20job_id%3A125c1410-c2d8-4f17-89bf-755f09bf5be7&agg_m=count&agg_m_source=base&agg_q=%40span.url&agg_q_source=base&agg_t=count&fromUser=true&index=single-machine-performance-target-logs&messageDisplay=inline&refresh_mode=paused&storage=hot&stream_sort=time%2Cdesc&top_n=100&top_o=top&viz=stream&x_missing=true&from_ts=1746810415000&to_ts=1746821815000&live=false)                                                                                                                                                                                                                                                                                                        |
| ➖   | docker_containers_cpu                        | % cpu utilization  | -0.28    | [-4.09, +3.53] | 1      | [Logs](https://app.datadoghq.com/logs?query=experiment%3Adocker_containers_cpu%20job_id%3A125c1410-c2d8-4f17-89bf-755f09bf5be7&agg_m=count&agg_m_source=base&agg_q=%40span.url&agg_q_source=base&agg_t=count&fromUser=true&index=single-machine-performance-target-logs&messageDisplay=inline&refresh_mode=paused&storage=hot&stream_sort=time%2Cdesc&top_n=100&top_o=top&viz=stream&x_missing=true&from_ts=1746810415000&to_ts=1746821815000&live=false)                                                                                                                                                                                                                                                                                                           |
| ➖   | file_tree                                    | memory utilization | -0.31    | [-0.42, -0.20] | 1      | [Logs](https://app.datadoghq.com/logs?query=experiment%3Afile_tree%20job_id%3A125c1410-c2d8-4f17-89bf-755f09bf5be7&agg_m=count&agg_m_source=base&agg_q=%40span.url&agg_q_source=base&agg_t=count&fromUser=true&index=single-machine-performance-target-logs&messageDisplay=inline&refresh_mode=paused&storage=hot&stream_sort=time%2Cdesc&top_n=100&top_o=top&viz=stream&x_missing=true&from_ts=1746810415000&to_ts=1746821815000&live=false)                                                                                                                                                                                                                                                                                                                       |
| ➖   | otlp_ingest_metrics                          | memory utilization | -0.35    | [-0.50, -0.20] | 1      | [Logs](https://app.datadoghq.com/logs?query=experiment%3Aotlp_ingest_metrics%20job_id%3A125c1410-c2d8-4f17-89bf-755f09bf5be7&agg_m=count&agg_m_source=base&agg_q=%40span.url&agg_q_source=base&agg_t=count&fromUser=true&index=single-machine-performance-target-logs&messageDisplay=inline&refresh_mode=paused&storage=hot&stream_sort=time%2Cdesc&top_n=100&top_o=top&viz=stream&x_missing=true&from_ts=1746810415000&to_ts=1746821815000&live=false)                                                                                                                                                                                                                                                                                                             |
| ➖   | quality_gate_idle_all_features               | memory utilization | -0.36    | [-0.45, -0.28] | 1      | [Logs](https://app.datadoghq.com/logs?query=experiment%3Aquality_gate_idle_all_features%20job_id%3A125c1410-c2d8-4f17-89bf-755f09bf5be7&agg_m=count&agg_m_source=base&agg_q=%40span.url&agg_q_source=base&agg_t=count&fromUser=true&index=single-machine-performance-target-logs&messageDisplay=inline&refresh_mode=paused&storage=hot&stream_sort=time%2Cdesc&top_n=100&top_o=top&viz=stream&x_missing=true&from_ts=1746810415000&to_ts=1746821815000&live=false) [bounds checks dashboard](https://app.datadoghq.com/dashboard/vz3-jd5-bdi?fromUser=true&refresh_mode=paused&tpl_var_experiment%5B0%5D=quality_gate_idle_all_features&tpl_var_job_id%5B0%5D=125c1410-c2d8-4f17-89bf-755f09bf5be7&view=spans&from_ts=1746817615000&to_ts=1746818215000&live=false) |
| ➖   | ddot_metrics                                 | memory utilization | -0.51    | [-0.62, -0.39] | 1      | [Logs](https://app.datadoghq.com/logs?query=experiment%3Addot_metrics%20job_id%3A125c1410-c2d8-4f17-89bf-755f09bf5be7&agg_m=count&agg_m_source=base&agg_q=%40span.url&agg_q_source=base&agg_t=count&fromUser=true&index=single-machine-performance-target-logs&messageDisplay=inline&refresh_mode=paused&storage=hot&stream_sort=time%2Cdesc&top_n=100&top_o=top&viz=stream&x_missing=true&from_ts=1746810415000&to_ts=1746821815000&live=false)                                                                                                                                                                                                                                                                                                                    |
| ➖   | quality_gate_logs                            | % cpu utilization  | -0.72    | [-3.48, +2.05] | 1      | [Logs](https://app.datadoghq.com/logs?query=experiment%3Aquality_gate_logs%20job_id%3A125c1410-c2d8-4f17-89bf-755f09bf5be7&agg_m=count&agg_m_source=base&agg_q=%40span.url&agg_q_source=base&agg_t=count&fromUser=true&index=single-machine-performance-target-logs&messageDisplay=inline&refresh_mode=paused&storage=hot&stream_sort=time%2Cdesc&top_n=100&top_o=top&viz=stream&x_missing=true&from_ts=1746810415000&to_ts=1746821815000&live=false) [bounds checks dashboard](https://app.datadoghq.com/dashboard/vz3-jd5-bdi?fromUser=true&refresh_mode=paused&tpl_var_experiment%5B0%5D=quality_gate_logs&tpl_var_job_id%5B0%5D=125c1410-c2d8-4f17-89bf-755f09bf5be7&view=spans&from_ts=1746817615000&to_ts=1746818215000&live=false)                           |
| ➖   | ddot_logs                                    | memory utilization | -1.40    | [-1.52, -1.28] | 1      | [Logs](https://app.datadoghq.com/logs?query=experiment%3Addot_logs%20job_id%3A125c1410-c2d8-4f17-89bf-755f09bf5be7&agg_m=count&agg_m_source=base&agg_q=%40span.url&agg_q_source=base&agg_t=count&fromUser=true&index=single-machine-performance-target-logs&messageDisplay=inline&refresh_mode=paused&storage=hot&stream_sort=time%2Cdesc&top_n=100&top_o=top&viz=stream&x_missing=true&from_ts=1746810415000&to_ts=1746821815000&live=false)                                                                                                                                                                                                                                                                                                                       |
</details>


<details>
<summary><h2>
Bounds Checks: ✅ Passed
</h2></summary>

| perf | experiment                                   | bounds_check_name  | replicates_passed | links                                                                                                                                                                                                                                                                                            |
|------|----------------------------------------------|--------------------|-------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| ✅   | docker_containers_cpu                        | simple_check_run   | 10/10             |                                                                                                                                                                                                                                                                                                  |
| ✅   | docker_containers_memory                     | memory_usage       | 10/10             |                                                                                                                                                                                                                                                                                                  |
| ✅   | docker_containers_memory                     | simple_check_run   | 10/10             |                                                                                                                                                                                                                                                                                                  |
| ✅   | file_to_blackhole_0ms_latency                | lost_bytes         | 10/10             |                                                                                                                                                                                                                                                                                                  |
| ✅   | file_to_blackhole_0ms_latency                | memory_usage       | 10/10             |                                                                                                                                                                                                                                                                                                  |
| ✅   | file_to_blackhole_0ms_latency_http1          | lost_bytes         | 10/10             |                                                                                                                                                                                                                                                                                                  |
| ✅   | file_to_blackhole_0ms_latency_http1          | memory_usage       | 10/10             |                                                                                                                                                                                                                                                                                                  |
| ✅   | file_to_blackhole_0ms_latency_http2          | lost_bytes         | 10/10             |                                                                                                                                                                                                                                                                                                  |
| ✅   | file_to_blackhole_0ms_latency_http2          | memory_usage       | 10/10             |                                                                                                                                                                                                                                                                                                  |
| ✅   | file_to_blackhole_1000ms_latency             | memory_usage       | 10/10             |                                                                                                                                                                                                                                                                                                  |
| ✅   | file_to_blackhole_1000ms_latency_linear_load | memory_usage       | 10/10             |                                                                                                                                                                                                                                                                                                  |
| ✅   | file_to_blackhole_100ms_latency              | lost_bytes         | 10/10             |                                                                                                                                                                                                                                                                                                  |
| ✅   | file_to_blackhole_100ms_latency              | memory_usage       | 10/10             |                                                                                                                                                                                                                                                                                                  |
| ✅   | file_to_blackhole_300ms_latency              | lost_bytes         | 10/10             |                                                                                                                                                                                                                                                                                                  |
| ✅   | file_to_blackhole_300ms_latency              | memory_usage       | 10/10             |                                                                                                                                                                                                                                                                                                  |
| ✅   | file_to_blackhole_500ms_latency              | lost_bytes         | 10/10             |                                                                                                                                                                                                                                                                                                  |
| ✅   | file_to_blackhole_500ms_latency              | memory_usage       | 10/10             |                                                                                                                                                                                                                                                                                                  |
| ✅   | quality_gate_idle                            | intake_connections | 10/10             | [bounds checks dashboard](https://app.datadoghq.com/dashboard/vz3-jd5-bdi?fromUser=true&refresh_mode=paused&tpl_var_experiment%5B0%5D=quality_gate_idle&tpl_var_job_id%5B0%5D=125c1410-c2d8-4f17-89bf-755f09bf5be7&view=spans&from_ts=1746817615000&to_ts=1746818215000&live=false)              |
| ✅   | quality_gate_idle                            | memory_usage       | 10/10             | [bounds checks dashboard](https://app.datadoghq.com/dashboard/vz3-jd5-bdi?fromUser=true&refresh_mode=paused&tpl_var_experiment%5B0%5D=quality_gate_idle&tpl_var_job_id%5B0%5D=125c1410-c2d8-4f17-89bf-755f09bf5be7&view=spans&from_ts=1746817615000&to_ts=1746818215000&live=false)              |
| ✅   | quality_gate_idle_all_features               | intake_connections | 10/10             | [bounds checks dashboard](https://app.datadoghq.com/dashboard/vz3-jd5-bdi?fromUser=true&refresh_mode=paused&tpl_var_experiment%5B0%5D=quality_gate_idle_all_features&tpl_var_job_id%5B0%5D=125c1410-c2d8-4f17-89bf-755f09bf5be7&view=spans&from_ts=1746817615000&to_ts=1746818215000&live=false) |
| ✅   | quality_gate_idle_all_features               | memory_usage       | 10/10             | [bounds checks dashboard](https://app.datadoghq.com/dashboard/vz3-jd5-bdi?fromUser=true&refresh_mode=paused&tpl_var_experiment%5B0%5D=quality_gate_idle_all_features&tpl_var_job_id%5B0%5D=125c1410-c2d8-4f17-89bf-755f09bf5be7&view=spans&from_ts=1746817615000&to_ts=1746818215000&live=false) |
| ✅   | quality_gate_logs                            | intake_connections | 10/10             | [bounds checks dashboard](https://app.datadoghq.com/dashboard/vz3-jd5-bdi?fromUser=true&refresh_mode=paused&tpl_var_experiment%5B0%5D=quality_gate_logs&tpl_var_job_id%5B0%5D=125c1410-c2d8-4f17-89bf-755f09bf5be7&view=spans&from_ts=1746817615000&to_ts=1746818215000&live=false)              |
| ✅   | quality_gate_logs                            | lost_bytes         | 10/10             | [bounds checks dashboard](https://app.datadoghq.com/dashboard/vz3-jd5-bdi?fromUser=true&refresh_mode=paused&tpl_var_experiment%5B0%5D=quality_gate_logs&tpl_var_job_id%5B0%5D=125c1410-c2d8-4f17-89bf-755f09bf5be7&view=spans&from_ts=1746817615000&to_ts=1746818215000&live=false)              |
| ✅   | quality_gate_logs                            | memory_usage       | 10/10             | [bounds checks dashboard](https://app.datadoghq.com/dashboard/vz3-jd5-bdi?fromUser=true&refresh_mode=paused&tpl_var_experiment%5B0%5D=quality_gate_logs&tpl_var_job_id%5B0%5D=125c1410-c2d8-4f17-89bf-755f09bf5be7&view=spans&from_ts=1746817615000&to_ts=1746818215000&live=false)              |
</details>


<details>
<summary><h2>
Explanation
</h2></summary>

**Confidence level:** 90.00%
**Effect size tolerance:** |Δ mean %| ≥ 5.00%

Performance changes are noted in the **perf** column of each table:

* ✅ = significantly better comparison variant performance
* ❌ = significantly worse comparison variant performance
* ➖ = no significant change in performance

A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".

For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:

1. Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.

2. Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that *if our statistical model is accurate*, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.

3. Its configuration does not mark it "erratic".
</details>

## CI Pass/Fail Decision

✅ **Passed.** All Quality Gates passed.

- **quality_gate_idle**, bounds check **memory_usage**: 10/10 replicas passed. Gate passed.
- **quality_gate_idle**, bounds check **intake_connections**: 10/10 replicas passed. Gate passed.
- **quality_gate_idle_all_features**, bounds check **memory_usage**: 10/10 replicas passed. Gate passed.
- **quality_gate_idle_all_features**, bounds check **intake_connections**: 10/10 replicas passed. Gate passed.
- **quality_gate_logs**, bounds check **intake_connections**: 10/10 replicas passed. Gate passed.
- **quality_gate_logs**, bounds check **memory_usage**: 10/10 replicas passed. Gate passed.
- **quality_gate_logs**, bounds check **lost_bytes**: 10/10 replicas passed. Gate passed.
author_association: NONE
user: login: agent-platform-auto-pr[bot]
created_at: 2025-04-29T19:40:46Z
body: ## Static quality checks

✅ Please find below the results from static quality gates


<details>
<summary>Successful checks</summary>

### Info

|Result|Quality gate|On disk size|On disk size limit|On wire size|On wire size limit|
|----|----|----|----|----|----|
|✅|static_quality_gate_agent_deb_amd64|751.15 MiB|778.06 MiB|185.01 MiB|191.06 MiB|
|✅|static_quality_gate_agent_deb_amd64_fips|748.99 MiB|776.09 MiB|184.7 MiB|190.72 MiB|
|✅|static_quality_gate_agent_heroku_amd64|368.46 MiB|434.99 MiB|98.03 MiB|114.34 MiB|
|✅|static_quality_gate_agent_msi|969.16 MiB|978.45 MiB|146.84 MiB|151.65 MiB|
|✅|static_quality_gate_agent_rpm_amd64|751.04 MiB|778.06 MiB|186.88 MiB|193.42 MiB|
|✅|static_quality_gate_agent_rpm_amd64_fips|748.97 MiB|776.06 MiB|185.76 MiB|192.61 MiB|
|✅|static_quality_gate_agent_rpm_arm64|741.23 MiB|768.33 MiB|169.27 MiB|174.71 MiB|
|✅|static_quality_gate_agent_rpm_arm64_fips|739.31 MiB|766.55 MiB|168.1 MiB|173.92 MiB|
|✅|static_quality_gate_agent_suse_amd64|751.18 MiB|778.08 MiB|186.88 MiB|193.42 MiB|
|✅|static_quality_gate_agent_suse_amd64_fips|748.99 MiB|776.11 MiB|185.76 MiB|192.78 MiB|
|✅|static_quality_gate_agent_suse_arm64|741.36 MiB|768.31 MiB|169.27 MiB|174.71 MiB|
|✅|static_quality_gate_agent_suse_arm64_fips|739.46 MiB|766.5 MiB|168.1 MiB|173.92 MiB|
|✅|static_quality_gate_docker_agent_amd64|845.33 MiB|862.5 MiB|285.27 MiB|292.0 MiB|
|✅|static_quality_gate_docker_agent_arm64|858.72 MiB|876.0 MiB|272.11 MiB|278.3 MiB|
|✅|static_quality_gate_docker_agent_jmx_amd64|845.33 MiB|862.5 MiB|285.27 MiB|292.0 MiB|
|✅|static_quality_gate_docker_agent_jmx_arm64|858.72 MiB|876.63 MiB|272.11 MiB|278.4 MiB|
|✅|static_quality_gate_docker_agent_windows1809|845.33 MiB|862.5 MiB|285.27 MiB|292.0 MiB|
|✅|static_quality_gate_docker_agent_windows1809_core|845.33 MiB|862.5 MiB|285.27 MiB|292.0 MiB|
|✅|static_quality_gate_docker_agent_windows1809_core_jmx|845.33 MiB|862.5 MiB|285.27 MiB|292.0 MiB|
|✅|static_quality_gate_docker_agent_windows1809_jmx|845.33 MiB|862.5 MiB|285.27 MiB|292.0 MiB|
|✅|static_quality_gate_docker_agent_windows2022|845.33 MiB|862.5 MiB|285.27 MiB|292.0 MiB|
|✅|static_quality_gate_docker_agent_windows2022_core|845.33 MiB|862.5 MiB|285.27 MiB|292.0 MiB|
|✅|static_quality_gate_docker_agent_windows2022_core_jmx|845.33 MiB|862.5 MiB|285.27 MiB|292.0 MiB|
|✅|static_quality_gate_docker_agent_windows2022_jmx|845.33 MiB|862.5 MiB|285.27 MiB|292.0 MiB|
|✅|static_quality_gate_docker_cluster_agent_amd64|259.7 MiB|263.4 MiB|101.96 MiB|104.07 MiB|
|✅|static_quality_gate_docker_cluster_agent_arm64|275.69 MiB|279.38 MiB|96.91 MiB|98.95 MiB|
|✅|static_quality_gate_docker_cws_instrumentation_amd64|6.66 MiB|7.12 MiB|2.82 MiB|3.29 MiB|
|✅|static_quality_gate_docker_cws_instrumentation_arm64|6.44 MiB|6.92 MiB|2.6 MiB|3.07 MiB|
|✅|static_quality_gate_docker_dogstatsd_amd64|37.97 MiB|46.39 MiB|14.64 MiB|17.78 MiB|
|✅|static_quality_gate_docker_dogstatsd_arm64|36.89 MiB|45.05 MiB|13.73 MiB|16.65 MiB|
|✅|static_quality_gate_dogstatsd_deb_amd64|29.83 MiB|38.4 MiB|7.89 MiB|10.26 MiB|
|✅|static_quality_gate_dogstatsd_deb_arm64|28.71 MiB|36.98 MiB|6.87 MiB|8.96 MiB|
|✅|static_quality_gate_dogstatsd_rpm_amd64|29.83 MiB|38.4 MiB|7.9 MiB|10.27 MiB|
|✅|static_quality_gate_dogstatsd_suse_amd64|29.83 MiB|38.4 MiB|7.9 MiB|10.27 MiB|
|✅|static_quality_gate_iot_agent_deb_amd64|57.85 MiB|58.51 MiB|14.55 MiB|15.02 MiB|
|✅|static_quality_gate_iot_agent_deb_arm64|55.27 MiB|55.94 MiB|12.57 MiB|13.05 MiB|
|✅|static_quality_gate_iot_agent_deb_armhf|54.03 MiB|54.32 MiB|12.59 MiB|13.05 MiB|
|✅|static_quality_gate_iot_agent_rpm_amd64|57.85 MiB|58.51 MiB|14.58 MiB|15.04 MiB|
|✅|static_quality_gate_iot_agent_rpm_arm64|55.27 MiB|55.94 MiB|12.6 MiB|13.07 MiB|
|✅|static_quality_gate_iot_agent_suse_amd64|57.85 MiB|58.51 MiB|14.58 MiB|15.04 MiB|

</details>

author_association: CONTRIBUTOR
user: login: gjulianm
created_at: 2025-05-08T14:56:55Z
body: /merge
author_association: CONTRIBUTOR
user: login: dd-devflow[bot]
created_at: 2025-05-08T14:56:59Z
body: <!-- "FeedbackWorkflowId":"1a3528e1-4746-4002-885a-abf25e80c2b8_36" -->
[View all feedbacks in Devflow UI.](https://mosaic.us1.ddbuild.io/devflow/logs/1a3528e1-4746-4002-885a-abf25e80c2b8_36)

`2025-05-08 14:56:59 UTC` :information_source: Start processing command `/merge`

---
`2025-05-08 14:57:04 UTC` :information_source: **MergeQueue:** waiting for PR to be ready

This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals.
**Note:** if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with `/remove` command.

---
`2025-05-08 18:58:03 UTC` :warning: **MergeQueue:** This merge request was unqueued

devflow unqueued this merge request: It did not become mergeable within the expected time

author_association: NONE
user: login: gjulianm
created_at: 2025-05-09T18:53:33Z
body: /merge
author_association: CONTRIBUTOR
user: login: dd-devflow[bot]
created_at: 2025-05-09T18:53:37Z
body: <!-- "FeedbackWorkflowId":"458b828b-15e1-48e3-9ab9-34e1a236d124_36" -->
[View all feedbacks in Devflow UI.](https://mosaic.us1.ddbuild.io/devflow/logs/458b828b-15e1-48e3-9ab9-34e1a236d124_36)

`2025-05-09 18:53:37 UTC` :information_source: Start processing command `/merge`

---
`2025-05-09 18:53:44 UTC` :information_source: **MergeQueue:** waiting for PR to be ready

This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals.
**Note:** if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with `/remove` command.

---
`2025-05-09 21:04:06 UTC` :information_source: **MergeQueue:** merge request added to the queue

The expected merge time in `main` is approximately 47m (p90).

---
`2025-05-09 21:41:51 UTC` :information_source: **MergeQueue:** This merge request was merged

author_association: NONE