Repository: DataDog/saluki
Pull request #636: [AGENTRUN-227] chore(checks-agent): compare check-agent in Go against Rust version
Author: login: GustavoCaso
Labels: name: area/ci
description: CI/CD, automated testing, etc.
Created: 2025-04-22T16:28:11Z | Merged: 2025-04-23T16:25:23Z

## Summary
<!-- Please provide a brief summary about what this PR does.
This should help the reviewers give feedback faster and with higher quality. -->

This PR benchmarks the Go version of the checks agent https://github.com/DataDog/datadog-agent/pull/31571 using SMP. We mirror the image created in the PR to the SMP ECR. Since the PR has few changes, we use the same commit as the baseline and comparison.

We updated the links generated to use the Go version as the comparison for the SMP dashboard. 

In a following PR, we might create a different dashboard just to focus on check agent comparison

## Change Type
- [ ] Bug fix
- [ ] New feature
- [x] Non-functional (chore, refactoring, docs)
- [ ] Performance


## How did you test this PR?
<!-- Please how you tested these changes here -->

## References

<!-- Please list any issues closed by this PR. -->

<!--
- Closes: <issue link>
-->

<!-- Any other issues or PRs relevant to this PR? Feel free to list them here. -->



Diff:
diff --git a/.ci/templates/generate-result-links-markdown.sh b/.ci/templates/generate-result-links-markdown.sh
index 0a61ea085f..b58061bb1f 100755
--- a/.ci/templates/generate-result-links-markdown.sh
+++ b/.ci/templates/generate-result-links-markdown.sh
@@ -55,6 +55,9 @@ dsd_end_time=$(cat dsd_job_end_time)
 checks_run_id=$(cat checks_run_id)
 checks_start_time=$(cat checks_job_start_time)
 checks_end_time=$(cat checks_job_end_time)
+checks_go_run_id=$(cat checks_go_run_id)
+checks_go_start_time=$(cat checks_go_job_start_time)
+checks_go_end_time=$(cat checks_go_job_end_time)
 
 # Load the job start/end times and figure out which job started first and which job ended last, which we'll use as the
 # start/end time for our dashboard, which shows both sides -- ADP and DSD -- in the same pane of glass.
@@ -74,6 +77,14 @@ else
     common_end_time=$(echo "${adp_end_time} - ${smp_negative_time_offset_secs} + ${experiment_duration_secs}" | bc)
 fi
 
+if [ "$checks_end_time" -lt "$checks_go_end_time" ]; then
+    common_checks_start_time=$(echo "${checks_end_time} - ${smp_negative_time_offset_secs}" | bc)
+    common_checks_end_time=$(echo "${checks_go_end_time} - ${smp_negative_time_offset_secs} + ${experiment_duration_secs}" | bc)
+else
+    common_checks_start_time=$(echo "${checks_go_end_time} - ${smp_negative_time_offset_secs}" | bc)
+    common_checks_end_time=$(echo "${checks_end_time} - ${smp_negative_time_offset_secs} + ${experiment_duration_secs}" | bc)
+fi
+
 # Grab the experiments for both DSD and ADP, which may or may not overlap.
 find test/smp/regression/saluki/cases -mindepth 1 -maxdepth 1 -type d | sed s#test/smp/regression/saluki/cases/##g | sort | uniq > adp-experiments
 find test/smp/regression/dogstatsd/cases -mindepth 1 -maxdepth 1 -type d | sed s#test/smp/regression/dogstatsd/cases/##g | sort | uniq > dsd-experiments
@@ -115,7 +126,7 @@ echo ""
 echo "| experiment | link(s) |"
 echo "|------------|---------|"
 
-checks_continuous_profiler_url=$(get_continuous_profiler_url "$checks_run_id" "$checks_start_time" "$checks_end_time" "quality_gates_idle_rss")
-checks_smp_dashboard_url=$(get_adp_smp_dashboard_url "$checks_run_id" "$dsd_run_id" "$checks_start_time" "$checks_end_time" "quality_gates_idle_rss")
+checks_continuous_profiler_url=$(get_continuous_profiler_url "$checks_run_id" "$common_checks_start_time" "$common_checks_end_time" "quality_gates_idle_rss")
+checks_smp_dashboard_url=$(get_adp_smp_dashboard_url "$checks_run_id" "$checks_go_run_id" "$common_checks_start_time" "$common_checks_end_time" "quality_gates_idle_rss")
 
-echo "| quality_gates_idle_rss | \\[[Profiling (Checks Agent)]($checks_continuous_profiler_url)\\] \\[[SMP Dashboard]($checks_smp_dashboard_url)\\] |"
+echo "| quality_gates_idle_rss | \\[[Profiling]($checks_continuous_profiler_url)\\] \\[[SMP Dashboard]($checks_smp_dashboard_url)\\] |"
diff --git a/.gitlab/benchmark.yml b/.gitlab/benchmark.yml
index 51c35e9349..e74ff2bc0c 100644
--- a/.gitlab/benchmark.yml
+++ b/.gitlab/benchmark.yml
@@ -5,16 +5,17 @@
   - export SMP_ECR_HOST="${SMP_ACCOUNT_ID}.dkr.ecr.us-west-2.amazonaws.com"
   - git fetch --verbose --deepen=999 origin ${CI_COMMIT_BRANCH} main
   - export BASELINE_SALUKI_SHA=$(git merge-base origin/main origin/${CI_COMMIT_BRANCH})
-  - export BASELINE_CHECKS_AGENT_SHA=$(git merge-base origin/poc-checks-agent-part-1 origin/${CI_COMMIT_BRANCH})
   - export COMPARISON_SALUKI_SHA="${CI_COMMIT_SHA}"
-  - export COMPARISON_CHECKS_AGENT_SHA="${CI_COMMIT_SHA}"
   - export BASELINE_SALUKI_IMG="${SMP_ECR_HOST}/${SMP_TEAM_ID}-saluki:adp-${CI_PIPELINE_ID}-${BASELINE_SALUKI_SHA}"
-  - export BASELINE_CHECKS_AGENT_IMG="${SMP_ECR_HOST}/${SMP_TEAM_ID}-saluki:checks-agent-${CI_PIPELINE_ID}-${BASELINE_CHECKS_AGENT_SHA}"
+  - export BASELINE_CHECKS_AGENT_IMG="${SMP_ECR_HOST}/${SMP_TEAM_ID}-saluki:checks-agent-${CI_PIPELINE_ID}-${BASELINE_SALUKI_SHA}"
   - export COMPARISON_SALUKI_IMG="${SMP_ECR_HOST}/${SMP_TEAM_ID}-saluki:adp-${CI_PIPELINE_ID}-${COMPARISON_SALUKI_SHA}"
-  - export COMPARISON_CHECKS_AGENT_IMG="${SMP_ECR_HOST}/${SMP_TEAM_ID}-saluki:checks-agent-${CI_PIPELINE_ID}-${COMPARISON_CHECKS_AGENT_SHA}"
+  - export COMPARISON_CHECKS_AGENT_IMG="${SMP_ECR_HOST}/${SMP_TEAM_ID}-saluki:checks-agent-${CI_PIPELINE_ID}-${COMPARISON_SALUKI_SHA}"
   - export BASELINE_DSD_VERSION="${BASE_DD_AGENT_VERSION}"
   - export COMPARISON_DSD_VERSION="${BASE_DD_AGENT_VERSION}"
   - export SOURCE_DSD_IMG="public.ecr.aws/datadog/dogstatsd:${BASE_DD_AGENT_VERSION}"
+  - export SOURCE_CHECK_AGENT_GO_SHA="f61d1f4e054b884cb1894254ab2714b84b4684cb"
+  - export SOURCE_CHECK_AGENT_GO_IMG="${SMP_ECR_HOST}/08450328-agent:${SOURCE_CHECK_AGENT_GO_SHA}-7-full-amd64"
+  - export BASELINE_CHECK_AGENT_GO_IMG="${SMP_ECR_HOST}/${SMP_TEAM_ID}-saluki:check-agent-go"
   - export BASELINE_DSD_IMG="${SMP_ECR_HOST}/${SMP_TEAM_ID}-saluki:dogstatsd-${BASE_DD_AGENT_VERSION}"
   - export COMPARISON_DSD_IMG="${SMP_ECR_HOST}/${SMP_TEAM_ID}-saluki:dogstatsd-${BASE_DD_AGENT_VERSION}"
 
@@ -117,11 +118,11 @@ build-checks-agent-image:
     # Initialize our AWS credentials.
     - ./test/smp/configure-smp-aws-credentials.sh
 
-    # Build baseline from `poc-checks-agent-part-1`. A more correct approach here would be to check
+    # Build baseline from `main`. A more correct approach here would be to check
     # out all but the dockerfile for saluki from the branch point and build that
     # way but this should be Good Enough for a while.
     - git fetch --all
-    - git switch poc-checks-agent-part-1
+    - git switch main
 
     # Actually build.
     - aws ecr get-login-password --region us-west-2 --profile ${AWS_NAMED_PROFILE} | docker login --username AWS --password-stdin ${SMP_ECR_HOST}
@@ -132,7 +133,7 @@ build-checks-agent-image:
       --build-arg BUILD_IMAGE=${SALUKI_BUILD_CI_IMAGE}
       --build-arg BUILD_PROFILE=${BUILD_PROFILE}
       --build-arg APP_IMAGE=${GBI_BASE_IMAGE}
-      --build-arg APP_GIT_HASH=${BASELINE_CHECKS_AGENT_SHA}
+      --build-arg APP_GIT_HASH=${BASELINE_SALUKI_SHA}
       --label git.repository=${CI_PROJECT_NAME}
       --label git.branch=${CI_COMMIT_REF_NAME}
       --label git.commit=${CI_COMMIT_SHA}
@@ -169,7 +170,7 @@ build-checks-agent-comparison-image:
       --build-arg BUILD_IMAGE=${SALUKI_BUILD_CI_IMAGE}
       --build-arg BUILD_PROFILE=${BUILD_PROFILE}
       --build-arg APP_IMAGE=${GBI_BASE_IMAGE}
-      --build-arg APP_GIT_HASH=${COMPARISON_CHECKS_AGENT_SHA}
+      --build-arg APP_GIT_HASH=${COMPARISON_SALUKI_SHA}
       --label git.repository=${CI_PROJECT_NAME}
       --label git.branch=${CI_COMMIT_REF_NAME}
       --label git.commit=${CI_COMMIT_SHA}
@@ -196,6 +197,23 @@ mirror-dsd-image-to-smp-ecr:
     - docker buildx imagetools create -t ${BASELINE_DSD_IMG} ${SOURCE_DSD_IMG}
     - docker buildx imagetools create -t ${COMPARISON_DSD_IMG} ${SOURCE_DSD_IMG}
 
+mirror-checks-agent-go-image-to-smp-ecr:
+  stage: benchmark
+  needs: []
+  image: "${SALUKI_SMP_CI_IMAGE}"
+  before_script:
+    - *setup-smp-env
+  variables:
+    AWS_NAMED_PROFILE: single-machine-performance
+  script:
+    # Initialize our AWS credentials.
+    - ./test/smp/configure-smp-aws-credentials.sh
+
+    # Mirror the Docker image build on the checks-agent Go PR https://github.com/DataDog/datadog-agent/pull/31571
+    # to the SMP ECR as SMP expects all target images to be sourceable from their ECR instance.
+    - aws ecr get-login-password --region us-west-2 --profile ${AWS_NAMED_PROFILE} | docker login --username AWS --password-stdin ${SMP_ECR_HOST}
+    - docker buildx imagetools create -t ${BASELINE_CHECK_AGENT_GO_IMG} ${SOURCE_CHECK_AGENT_GO_IMG}
+
 run-benchmarks-adp:
   stage: benchmark
   timeout: 1h
@@ -365,8 +383,8 @@ run-benchmarks-checks-agent:
             job submit
             --baseline-image ${BASELINE_CHECKS_AGENT_IMG}
             --comparison-image ${COMPARISON_CHECKS_AGENT_IMG}
-            --baseline-sha ${BASELINE_CHECKS_AGENT_SHA}
-            --comparison-sha ${COMPARISON_CHECKS_AGENT_SHA}
+            --baseline-sha ${BASELINE_SALUKI_SHA}
+            --comparison-sha ${COMPARISON_SALUKI_SHA}
             --target-config-dir ./test/smp/regression/checks-agent/
             --submission-metadata submission_metadata
     # Populate some data that we'll need later for generating links to experiment dashboards.
@@ -396,12 +414,80 @@ run-benchmarks-checks-agent:
             job result
             --submission-metadata submission_metadata
 
+run-benchmarks-checks-agent-go:
+  stage: benchmark
+  timeout: 1h
+  image: "${SALUKI_SMP_CI_IMAGE}"
+  needs:
+    - mirror-checks-agent-go-image-to-smp-ecr
+  allow_failure: true
+  before_script:
+    - *setup-smp-env
+  artifacts:
+    expire_in: 1 weeks
+    paths:
+      - submission_metadata  # for provenance, debugging
+      - checks_go_job_start_time   # for generating PR comments
+      - checks_go_job_end_time     # for generating PR comments
+      - checks_go_run_id           # for generating PR comments
+      - outputs/report.md    # for debugging, also on S3
+      - outputs/report.html  # for debugging, also on S3
+    when: always
+  variables:
+    AWS_NAMED_PROFILE: single-machine-performance
+    SMP_VERSION: 0.21.0
+    RUST_LOG: info,aws_config::profile::credentials=error
+  script:
+    # Do some pre-flight steps like creating directories, installing helper utilities, setting
+    # credentials, and so on.
+    - mkdir outputs && touch outputs/report.md outputs/report.html
+    - ./test/smp/configure-smp-aws-credentials.sh
+    # Download the SMP binary.
+    - aws --profile ${AWS_NAMED_PROFILE} s3 cp s3://smp-cli-releases/v${SMP_VERSION}/x86_64-unknown-linux-gnu/smp smp && chmod +x smp
+    # Run SMP against our converged Agent image.
+    - ./smp --team-id ${SMP_TEAM_ID} --aws-named-profile ${AWS_NAMED_PROFILE}
+            job submit
+            --baseline-image ${BASELINE_CHECK_AGENT_GO_IMG}
+            --comparison-image ${BASELINE_CHECK_AGENT_GO_IMG}
+            --baseline-sha ${SOURCE_CHECK_AGENT_GO_SHA}
+            --comparison-sha ${SOURCE_CHECK_AGENT_GO_SHA}
+            --target-config-dir ./test/smp/regression/checks-agent-go/
+            --submission-metadata submission_metadata
+    # Populate some data that we'll need later for generating links to experiment dashboards.
+    - date +%s > checks_go_job_start_time
+    - cat submission_metadata | jq -r '.jobId' > checks_go_run_id
+    # Wait for job to complete.
+    - ./smp --team-id ${SMP_TEAM_ID} --aws-named-profile ${AWS_NAMED_PROFILE}
+            job status
+            --wait
+            --wait-delay-seconds 60
+            --submission-metadata submission_metadata
+    # And populate some more data that we'll need later for generating links to experiment dashboards.
+    - date +%s > checks_go_job_end_time
+    # Pull the analysis report and output it to stdout.
+    - ./smp --team-id ${SMP_TEAM_ID} --aws-named-profile ${AWS_NAMED_PROFILE}
+            job sync
+            --submission-metadata submission_metadata
+            --output-path outputs
+    # Replace empty lines in the output with lines containing various unicode space characters.
+    #
+    # This avoids  https://gitlab.com/gitlab-org/gitlab/-/issues/217231.
+    - cat outputs/report.md | sed "s/^\$/$(echo -ne '\uFEFF\u00A0\u200B')/g"
+    # Post the HTML report to the PR.
+    - cat outputs/report.md | /usr/local/bin/pr-commenter --for-pr="$CI_COMMIT_REF_NAME" --header="Regression Detector (Checks Agent Go)"
+    # Finally, exit 1 if the job signals a regression else 0.
+    - ./smp --team-id ${SMP_TEAM_ID} --aws-named-profile ${AWS_NAMED_PROFILE}
+            job result
+            --submission-metadata submission_metadata
+
+
 comment-smp-result-links:
   stage: benchmark
   needs:
     - run-benchmarks-adp
     - run-benchmarks-dsd
     - run-benchmarks-checks-agent
+    - run-benchmarks-checks-agent-go
   image: "${SALUKI_SMP_CI_IMAGE}"
   before_script:
     - *setup-smp-env
diff --git a/test/smp/regression/checks-agent-go/cases/quality_gates_idle_rss/checks-agent-go/.gitkeep b/test/smp/regression/checks-agent-go/cases/quality_gates_idle_rss/checks-agent-go/.gitkeep
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/test/smp/regression/checks-agent-go/cases/quality_gates_idle_rss/checks-agent-go/conf.d/disk.d/conf.yaml.default b/test/smp/regression/checks-agent-go/cases/quality_gates_idle_rss/checks-agent-go/conf.d/disk.d/conf.yaml.default
new file mode 100644
index 0000000000..37f1249b2a
--- /dev/null
+++ b/test/smp/regression/checks-agent-go/cases/quality_gates_idle_rss/checks-agent-go/conf.d/disk.d/conf.yaml.default
@@ -0,0 +1,262 @@
+## All options defined here are available to all instances.
+#
+init_config:
+
+    ## @param file_system_global_exclude - list of strings - optional
+    ## Instruct the check to always add these patterns to `file_system_exclude`.
+    ##
+    ## WARNING: Overriding these defaults could negatively impact your system or
+    ## the performance of the check.
+    #
+    # file_system_global_exclude:
+    #   - iso9660$
+    #   - tracefs$
+
+    ## @param device_global_exclude - list of strings - optional
+    ## Instruct the check to always add these patterns to `device_exclude`.
+    ##
+    ## WARNING: Overriding these defaults could negatively impact your system or
+    ## the performance of the check.
+    #
+    # device_global_exclude: []
+
+    ## @param mount_point_global_exclude - list of strings - optional
+    ## Instruct the check to always add these patterns to `mount_point_exclude`.
+    ##
+    ## WARNING: Overriding these defaults could negatively impact your system or
+    ## the performance of the check.
+    #
+    # mount_point_global_exclude:
+    #   - (/host)?/proc/sys/fs/binfmt_misc$
+
+    ## @param service - string - optional
+    ## Attach the tag `service:<SERVICE>` to every metric, event, and service check emitted by this integration.
+    ##
+    ## Additionally, this sets the default `service` for every log source.
+    #
+    # service: <SERVICE>
+
+## Every instance is scheduled independently of the others.
+#
+instances:
+
+    ## @param use_mount - boolean - optional - default: false
+    ## If enabled, metrics are tagged using mount points (for example `device:/`) instead of volumes
+    ## (for example `device:/dev/disk1s5`).
+    #
+  - use_mount: false
+
+    ## @param all_partitions - boolean - optional - default: false
+    ## Instruct the check to collect from partitions even without device names.
+    ## Setting `use_mount` to true is strongly recommended in this case.
+    #
+    # all_partitions: false
+
+    ## @param tags - list of strings - optional
+    ## A list of tags to attach to every metric and service check emitted by this instance.
+    ##
+    ## Learn more about tagging at https://docs.datadoghq.com/tagging
+    #
+    # tags:
+    #   - <KEY_1>:<VALUE_1>
+    #   - <KEY_2>:<VALUE_2>
+
+    ## @param service - string - optional
+    ## Attach the tag `service:<SERVICE>` to every metric, event, and service check emitted by this integration.
+    ##
+    ## Overrides any `service` defined in the `init_config` section.
+    #
+    # service: <SERVICE>
+
+    ## @param min_collection_interval - number - optional - default: 15
+    ## This changes the collection interval of the check. For more information, see:
+    ## https://docs.datadoghq.com/developers/write_agent_check/#collection-interval
+    #
+    # min_collection_interval: 15
+
+    ## @param empty_default_hostname - boolean - optional - default: false
+    ## This forces the check to send metrics with no hostname.
+    ##
+    ## This is useful for cluster-level checks.
+    #
+    # empty_default_hostname: false
+
+    ## @param metric_patterns - mapping - optional
+    ## A mapping of metrics to include or exclude, with each entry being a regular expression.
+    ##
+    ## Metrics defined in `exclude` will take precedence in case of overlap.
+    #
+    # metric_patterns:
+    #   include:
+    #   - <INCLUDE_REGEX>
+    #   exclude:
+    #   - <EXCLUDE_REGEX>
+
+    ## @param file_system_include - list of strings - optional
+    ## Instruct the check to only collect from matching file systems.
+    ##
+    ## Character casing is ignored. For convenience, the regular expressions
+    ## start matching from the beginning and therefore to match anywhere you
+    ## must prepend `.*`. For exact matches append `$`.
+    #
+    # file_system_include:
+    #   - ext[34]$
+    #   - ntfs$
+
+    ## @param file_system_exclude - list of strings - optional
+    ## Instruct the check to not collect from matching file systems.
+    ##
+    ## Character casing is ignored. For convenience, the regular expressions
+    ## start matching from the beginning and therefore to match anywhere you
+    ## must prepend `.*`. For exact matches append `$`.
+    ##
+    ## Devices from pseudo or memory-based file systems can be excluded by disabling the
+    ## `include_all_devices` option.
+    ##
+    ## When conflicts arise, this will override `file_system_include`.
+    #
+    # file_system_exclude:
+    #   - tmpfs$
+    #   - rootfs$
+    #   - autofs$
+
+    ## @param device_include - list of strings - optional
+    ## Instruct the check to only collect from matching devices.
+    ##
+    ## Character casing is ignored on Windows. For convenience, the regular
+    ## expressions start matching from the beginning and therefore to match
+    ## anywhere you must prepend `.*`. For exact matches append `$`.
+    #
+    # device_include:
+    #   - /dev/sda[1-3]
+    #   - 'C:'
+
+    ## @param device_exclude - list of strings - optional
+    ## Instruct the check to not collect from matching devices.
+    ##
+    ## Character casing is ignored on Windows. For convenience, the regular
+    ## expressions start matching from the beginning and therefore to match
+    ## anywhere you must prepend `.*`. For exact matches append `$`.
+    ##
+    ## When conflicts arise, this will override `device_include`.
+    #
+    # device_exclude:
+    #   - /dev/sde
+    #   - '[FJ]:'
+
+    ## @param mount_point_include - list of strings - optional
+    ## Instruct the check to only collect from matching mount points.
+    ##
+    ## Character casing is ignored on Windows. For convenience, the regular
+    ## expressions start matching from the beginning and therefore to match
+    ## anywhere you must prepend `.*`. For exact matches append `$`.
+    #
+    # mount_point_include:
+    #   - /dev/sda[1-3]
+    #   - 'C:'
+
+    ## @param mount_point_exclude - list of strings - optional
+    ## Instruct the check to not collect from matching mount points.
+    ##
+    ## Character casing is ignored on Windows. For convenience, the regular
+    ## expressions start matching from the beginning and therefore to match
+    ## anywhere you must prepend `.*`. For exact matches append `$`.
+    #
+    # mount_point_exclude:
+    #   - /proc/sys/fs/binfmt_misc
+    #   - /dev/sde
+    #   - '[FJ]:'
+
+    ## @param include_all_devices - boolean - optional - default: true
+    ## Instruct the check to collect from all devices, including non-physical devices.
+    ## Set this to false to exclude pseudo, memory, duplicate or inaccessible file systems.
+    ##
+    ## For more fine-grained control, use the inclusion and exclusion options.
+    #
+    # include_all_devices: true
+
+    ## @param service_check_rw - boolean - optional - default: false
+    ## Instruct the check to notify based on partition state.
+    ##
+    ## read-write -> OK
+    ## read-only  -> CRITICAL
+    ## other      -> UNKNOWN
+    #
+    # service_check_rw: false
+
+    ## @param tag_by_filesystem - boolean - optional - default: false
+    ## Instruct the check to tag all disks with their file system e.g. filesystem:ntfs.
+    #
+    # tag_by_filesystem: false
+
+    ## @param tag_by_label - boolean - optional - default: true
+    ## Instruct the check to tag all the metrics with disk label if there is one.
+    ## Works on Linux only.
+    #
+    # tag_by_label: true
+
+    ## @param blkid_cache_file - string - optional
+    ## Instruct the check to read the labels from the blkid cache file instead of `blkid` executable.
+    ## This parameter is used only if `tag_by_label` is true. It is incompatible with `use_lsblk`.
+    ## Works on Linux only.
+    #
+    # blkid_cache_file: /run/blkid/blkid.tab
+
+    ## @param use_lsblk - boolean - optional - default: false
+    ## Instruct the check to read the labels from the `lsblk` executable instead of `blkid` executable.
+    ## This parameter is used only if `tag_by_label` is true. It is incompatible with `blkid_cache_file`.
+    ## Works on Linux only.
+    #
+    # use_lsblk: false
+
+    ## @param device_tag_re - mapping - optional
+    ## Instruct the check to apply additional tags to matching
+    ## devices (or mount points if `use_mount` is true).
+    ##
+    ## Character casing is ignored on Windows. Multiple comma-separated
+    ## tags are supported. You must properly quote the string if the
+    ## pattern contains special characters e.g. colons.
+    #
+    # device_tag_re:
+    #   /san/.*: device_type:san
+    #   /dev/sda3: role:db,disk_size:large
+    #   'c:': volume:boot
+
+    ## @param min_disk_size - number - optional - default: 0
+    ## Exclude devices with a total disk size less than a minimum value (in MiB)
+    #
+    # min_disk_size: 0
+
+    ## @param timeout - integer - optional - default: 5
+    ## Timeout of the disk query in seconds
+    #
+    # timeout: 5
+
+    ## @param create_mounts - list of mappings - optional
+    ## On Windows, instruct the check to create one or more network
+    ## mounts, and have the check collect metrics for the mounted devices.
+    ##
+    ## Uses the provided username and password (if provided and necessary)
+    ## to create an SMB or NFS mount. If `type` is not specified, then
+    ## the operating system will choose the best available network filesystem
+    ## based on the other parameters. If `type` is specified, then any type
+    ## other than `nfs` will default to an SMB file share.
+    #
+    # create_mounts:
+    #   - mountpoint: 's:'
+    #     user: auser
+    #     password: somepassword
+    #     host: smbserver
+    #     share: space
+    #   - mountpoint: 'n:'
+    #     host: nfsserver
+    #     share: /mnt/nfs_share
+    #     type: nfs
+
+    ## @param lowercase_device_tag - boolean - optional - default: false
+    ## Enable this to lowercase the "device" tags for both partition and disk metrics.
+    ## This is useful only in very specific circumstances:
+    ##   1. Your "device" tag value is uppercase and your host is running on Linux.
+    ##   2. You cannot use the "device_name" tag.
+    #
+    # lowercase_device_tag: false
diff --git a/test/smp/regression/checks-agent-go/cases/quality_gates_idle_rss/checks-agent-go/datadog.yaml b/test/smp/regression/checks-agent-go/cases/quality_gates_idle_rss/checks-agent-go/datadog.yaml
new file mode 100644
index 0000000000..f5dc33f15d
--- /dev/null
+++ b/test/smp/regression/checks-agent-go/cases/quality_gates_idle_rss/checks-agent-go/datadog.yaml
@@ -0,0 +1,18 @@
+auth_token_file_path: /tmp/agent-auth-token
+
+dd_url: http://127.0.0.1:9091
+
+telemetry.enabled: true
+logs_enabled: false
+apm_config.enabled: false
+process_config.process_collection.enabled: false
+runtime_security_config.enabled: false
+compliance_config.enabled: false
+use_dogstatsd: false
+otelcollector.enabled: false
+
+# Disable cloud detection. This stops the Agent from poking around the
+# execution environment & network. This is particularly important if the target
+# has network access.
+cloud_provider_metadata: []
+
diff --git a/test/smp/regression/checks-agent-go/cases/quality_gates_idle_rss/experiment.yaml b/test/smp/regression/checks-agent-go/cases/quality_gates_idle_rss/experiment.yaml
new file mode 100644
index 0000000000..130697fe23
--- /dev/null
+++ b/test/smp/regression/checks-agent-go/cases/quality_gates_idle_rss/experiment.yaml
@@ -0,0 +1,12 @@
+optimization_goal: memory
+erratic: false
+
+target:
+  name: checks-agent-go
+  cpu_allotment: 4
+  memory_allotment: 2GiB
+
+  environment:
+    DD_API_KEY: foo00000001
+    DD_HOSTNAME: smp-regression
+
diff --git a/test/smp/regression/checks-agent-go/cases/quality_gates_idle_rss/lading/lading.yaml b/test/smp/regression/checks-agent-go/cases/quality_gates_idle_rss/lading/lading.yaml
new file mode 100644
index 0000000000..9c34f3771d
--- /dev/null
+++ b/test/smp/regression/checks-agent-go/cases/quality_gates_idle_rss/lading/lading.yaml
@@ -0,0 +1,17 @@
+generator: []
+
+blackhole:
+  - http:
+      binding_addr: "127.0.0.1:9091"
+  - http:
+      binding_addr: "127.0.0.1:9092"
+
+target_metrics:
+  - prometheus: #core agent telemetry
+      uri: "http://127.0.0.1:5000/telemetry"
+      tags:
+        sub_agent: "core"
+  - prometheus: #check-agent telemetry
+      uri: "http://127.0.0.1:6060/telemetry"
+      tags:
+        sub_agent: "checks-agent"
diff --git a/test/smp/regression/checks-agent-go/config.yaml b/test/smp/regression/checks-agent-go/config.yaml
new file mode 100644
index 0000000000..b7ada7be46
--- /dev/null
+++ b/test/smp/regression/checks-agent-go/config.yaml
@@ -0,0 +1,2 @@
+lading:
+  version: 0.25.7


Reviews:
user: login: GustavoCaso
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-22T16:29:21Z
user: login: GustavoCaso
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-23T12:45:31Z
user: login: tobz
body: LGTM, but one non-blocking question.
state: APPROVED
author_association: MEMBER
submitted_at: 2025-04-23T12:57:48Z
user: login: tobz
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-23T13:33:57Z

Review comments:
diff_hunk: @@ -15,6 +15,8 @@
   - export BASELINE_DSD_VERSION="${BASE_DD_AGENT_VERSION}"
   - export COMPARISON_DSD_VERSION="${BASE_DD_AGENT_VERSION}"
   - export SOURCE_DSD_IMG="public.ecr.aws/datadog/dogstatsd:${BASE_DD_AGENT_VERSION}"
+  - export SOURCE_CHECK_AGENT_GO_IMG="${SMP_ECR_HOST}/08450328-agent:4c4f5e817c81a3622f1fc64959ab3dbea49c73da-7-full-amd64"
path: .gitlab/benchmark.yml
user: login: GustavoCaso
body: This image is created and published from this branch https://github.com/DataDog/datadog-agent/pull/31571
We use the latest SHA from that PR [commits](https://github.com/DataDog/datadog-agent/pull/31571/commits)
created_at: 2025-04-22T16:29:21Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,18 @@
+auth_token_file_path: /tmp/agent-auth-token
+
+dd_url: http://127.0.0.1:9091
+
+telemetry.enabled: true
+logs_enabled: false
+apm_config.enabled: false
+process_config.process_collection.enabled: false
+runtime_security_config.enabled: false
+compliance_config.enabled: false
+use_dogstatsd: false
+otelcollector.enabled: false
path: test/smp/regression/checks-agent-go/cases/quality_gates_idle_rss/checks-agent-go/datadog.yaml
user: login: GustavoCaso
body: We disable everything except the checks agent and the core agent
created_at: 2025-04-23T12:45:31Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,262 @@
+## All options defined here are available to all instances.
path: test/smp/regression/checks-agent-go/cases/quality_gates_idle_rss/checks-agent-go/conf.d/disk.d/conf.yaml.default
user: login: tobz
body: Does this file actually do anything with it having a `.default` extension?
created_at: 2025-04-23T12:57:11Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,262 @@
+## All options defined here are available to all instances.
path: test/smp/regression/checks-agent-go/cases/quality_gates_idle_rss/checks-agent-go/conf.d/disk.d/conf.yaml.default
user: login: tobz
body: From a discussion in Slack: yes, yes it does.
created_at: 2025-04-23T13:33:57Z
author_association: MEMBER

PR comments:
user: login: pr-commenter[bot]
created_at: 2025-04-22T16:46:19Z
body: ## Regression Detector (Checks Agent Go)

# Regression Detector Results



Run ID: c9f4602a-902b-40d3-8ba5-e18ae99c929b

Baseline: f61d1f4e054b884cb1894254ab2714b84b4684cb
Comparison: f61d1f4e054b884cb1894254ab2714b84b4684cb
[Diff](f61d1f4e054b884cb1894254ab2714b84b4684cb...f61d1f4e054b884cb1894254ab2714b84b4684cb)


## Optimization Goals: ✅ No significant changes detected


<details>
<summary><h2>
Fine details of change detection per experiment
</h2></summary>

| perf | experiment             | goal               | Δ mean % | Δ mean % CI    | trials | links |
|------|------------------------|--------------------|----------|----------------|--------|-------|
| ➖   | quality_gates_idle_rss | memory utilization | +0.39    | [+0.33, +0.44] | 1      |       |
</details>

<details>
<summary><h2>
Explanation
</h2></summary>

**Confidence level:** 90.00%
**Effect size tolerance:** |Δ mean %| ≥ 5.00%

Performance changes are noted in the **perf** column of each table:

* ✅ = significantly better comparison variant performance
* ❌ = significantly worse comparison variant performance
* ➖ = no significant change in performance

A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".

For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:

1. Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.

2. Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that *if our statistical model is accurate*, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.

3. Its configuration does not mark it "erratic".
</details>
author_association: NONE
user: login: pr-commenter[bot]
created_at: 2025-04-22T16:46:50Z
body: ## Regression Detector (DogStatsD)

# Regression Detector Results



Run ID: 50f42e37-61d0-4365-8a33-de1b10338744

Baseline: 7.65.0-rc.9
Comparison: 7.65.0-rc.9


## Optimization Goals: ✅ No significant changes detected


<details>
<summary><h2>
Fine details of change detection per experiment
</h2></summary>

| perf | experiment                                   | goal               | Δ mean % | Δ mean % CI    | trials | links |
|------|----------------------------------------------|--------------------|----------|----------------|--------|-------|
| ➖   | dsd_uds_100mb_3k_contexts_distributions_only | memory utilization | +0.01    | [-0.20, +0.23] | 1      |       |
| ➖   | dsd_uds_100mb_3k_contexts                    | ingress throughput | +0.01    | [-0.09, +0.12] | 1      |       |
| ➖   | dsd_uds_100mb_250k_contexts                  | ingress throughput | +0.00    | [-0.10, +0.10] | 1      |       |
| ➖   | dsd_uds_40mb_12k_contexts_40_senders         | ingress throughput | +0.00    | [-0.00, +0.00] | 1      |       |
| ➖   | dsd_uds_1mb_50k_contexts                     | ingress throughput | +0.00    | [-0.00, +0.00] | 1      |       |
| ➖   | dsd_uds_10mb_3k_contexts                     | ingress throughput | +0.00    | [-0.00, +0.00] | 1      |       |
| ➖   | dsd_uds_1mb_3k_contexts                      | ingress throughput | +0.00    | [-0.00, +0.00] | 1      |       |
| ➖   | dsd_uds_1mb_50k_contexts_memlimit            | ingress throughput | +0.00    | [-0.00, +0.00] | 1      |       |
| ➖   | dsd_uds_512kb_3k_contexts                    | ingress throughput | +0.00    | [-0.01, +0.01] | 1      |       |
| ➖   | dsd_uds_1mb_3k_contexts_dualship             | ingress throughput | -0.00    | [-0.00, +0.00] | 1      |       |
| ➖   | dsd_uds_500mb_3k_contexts                    | ingress throughput | -0.25    | [-0.33, -0.16] | 1      |       |
| ➖   | quality_gates_idle_rss                       | memory utilization | -0.30    | [-0.42, -0.19] | 1      |       |
</details>


## Bounds Checks: ❌ Failed

| perf | experiment             | bounds_check_name | replicates_passed | links |
|------|------------------------|-------------------|-------------------|-------|
| ❌   | quality_gates_idle_rss | memory_usage      | 0/10              |       |


<details>
<summary><h2>
Explanation
</h2></summary>

**Confidence level:** 90.00%
**Effect size tolerance:** |Δ mean %| ≥ 5.00%

Performance changes are noted in the **perf** column of each table:

* ✅ = significantly better comparison variant performance
* ❌ = significantly worse comparison variant performance
* ➖ = no significant change in performance

A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".

For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:

1. Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.

2. Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that *if our statistical model is accurate*, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.

3. Its configuration does not mark it "erratic".
</details>
author_association: NONE
user: login: pr-commenter[bot]
created_at: 2025-04-22T16:55:48Z
body: ## Regression Detector (Checks Agent)

# Regression Detector Results



Run ID: db53f723-7b7c-41f9-8eb2-5e388f08e370

Baseline: 55c87993b87b1a6e92be29f38a5b3bbdbfcacf24
Comparison: 43b8b832329afa7614526b3791c8dc7353f1b0f1
[Diff](55c87993b87b1a6e92be29f38a5b3bbdbfcacf24...43b8b832329afa7614526b3791c8dc7353f1b0f1)


## Optimization Goals: ✅ No significant changes detected


<details>
<summary><h2>
Fine details of change detection per experiment
</h2></summary>

| perf | experiment             | goal               | Δ mean % | Δ mean % CI    | trials | links |
|------|------------------------|--------------------|----------|----------------|--------|-------|
| ➖   | quality_gates_idle_rss | memory utilization | +0.24    | [+0.22, +0.25] | 1      |       |
</details>


<details>
<summary><h2>
Bounds Checks: ✅ Passed
</h2></summary>

| perf | experiment             | bounds_check_name | replicates_passed | links |
|------|------------------------|-------------------|-------------------|-------|
| ✅   | quality_gates_idle_rss | memory_usage      | 10/10             |       |
</details>


<details>
<summary><h2>
Explanation
</h2></summary>

**Confidence level:** 90.00%
**Effect size tolerance:** |Δ mean %| ≥ 5.00%

Performance changes are noted in the **perf** column of each table:

* ✅ = significantly better comparison variant performance
* ❌ = significantly worse comparison variant performance
* ➖ = no significant change in performance

A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".

For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:

1. Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.

2. Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that *if our statistical model is accurate*, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.

3. Its configuration does not mark it "erratic".
</details>
author_association: NONE
user: login: pr-commenter[bot]
created_at: 2025-04-22T16:59:07Z
body: ## Regression Detector (Saluki)

# Regression Detector Results



Run ID: 0f359c7d-31ae-4598-9f09-bce5b1619d40

Baseline: 55c87993b87b1a6e92be29f38a5b3bbdbfcacf24
Comparison: 43b8b832329afa7614526b3791c8dc7353f1b0f1
[Diff](55c87993b87b1a6e92be29f38a5b3bbdbfcacf24...43b8b832329afa7614526b3791c8dc7353f1b0f1)


## Optimization Goals: ✅ No significant changes detected


<details>
<summary><h2>
Fine details of change detection per experiment
</h2></summary>

| perf | experiment                                      | goal               | Δ mean % | Δ mean % CI    | trials | links |
|------|-------------------------------------------------|--------------------|----------|----------------|--------|-------|
| ➖   | quality_gates_idle_rss                          | memory utilization | +0.32    | [+0.31, +0.34] | 1      |       |
| ➖   | dsd_uds_10mb_3k_contexts                        | ingress throughput | +0.03    | [-0.01, +0.06] | 1      |       |
| ➖   | dsd_uds_100mb_250k_contexts                     | ingress throughput | +0.03    | [-0.03, +0.08] | 1      |       |
| ➖   | dsd_uds_1mb_3k_contexts                         | ingress throughput | +0.01    | [-0.00, +0.02] | 1      |       |
| ➖   | dsd_uds_50mb_10k_contexts_no_inlining           | ingress throughput | +0.00    | [-0.10, +0.11] | 1      |       |
| ➖   | dsd_uds_512kb_3k_contexts                       | ingress throughput | +0.00    | [-0.02, +0.02] | 1      |       |
| ➖   | dsd_uds_100mb_3k_contexts                       | ingress throughput | -0.00    | [-0.05, +0.05] | 1      |       |
| ➖   | dsd_uds_40mb_12k_contexts_40_senders            | ingress throughput | -0.01    | [-0.04, +0.03] | 1      |       |
| ➖   | dsd_uds_1mb_50k_contexts                        | ingress throughput | -0.01    | [-0.02, +0.01] | 1      |       |
| ➖   | dsd_uds_1mb_3k_contexts_dualship                | ingress throughput | -0.01    | [-0.02, +0.00] | 1      |       |
| ➖   | dsd_uds_50mb_10k_contexts_no_inlining_no_allocs | ingress throughput | -0.01    | [-0.13, +0.10] | 1      |       |
| ➖   | dsd_uds_100mb_3k_contexts_distributions_only    | memory utilization | -0.37    | [-0.54, -0.21] | 1      |       |
| ➖   | dsd_uds_1mb_50k_contexts_memlimit               | ingress throughput | -2.01    | [-3.60, -0.42] | 1      |       |
| ➖   | dsd_uds_500mb_3k_contexts                       | ingress throughput | -3.64    | [-3.75, -3.53] | 1      |       |
</details>


<details>
<summary><h2>
Bounds Checks: ✅ Passed
</h2></summary>

| perf | experiment             | bounds_check_name | replicates_passed | links |
|------|------------------------|-------------------|-------------------|-------|
| ✅   | quality_gates_idle_rss | memory_usage      | 10/10             |       |
</details>


<details>
<summary><h2>
Explanation
</h2></summary>

**Confidence level:** 90.00%
**Effect size tolerance:** |Δ mean %| ≥ 5.00%

Performance changes are noted in the **perf** column of each table:

* ✅ = significantly better comparison variant performance
* ❌ = significantly worse comparison variant performance
* ➖ = no significant change in performance

A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".

For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:

1. Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.

2. Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that *if our statistical model is accurate*, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.

3. Its configuration does not mark it "erratic".
</details>
author_association: NONE
user: login: pr-commenter[bot]
created_at: 2025-04-22T17:00:15Z
body: ## Regression Detector Links

## ADP Experiment Result Links

| experiment | link(s) |
|------------|---------|
| dsd_uds_100mb_250k_contexts | \[[Profiling (ADP)](https://app.datadoghq.com/profiling/explorer?query=env%3Asingle-machine-performance%20run-id%3A0f359c7d-31ae-4598-9f09-bce5b1619d40%20experiment%3Adsd_uds_100mb_250k_contexts&fromUser=true&my_code=enabled&refresh_mode=paused&viz=flame_graph&from_ts=1745423103000&to_ts=1745425323000&live=false)\] \[[Profiling (DSD)](https://app.datadoghq.com/profiling/explorer?query=env%3Asingle-machine-performance%20run-id%3A50f42e37-61d0-4365-8a33-de1b10338744%20experiment%3Adsd_uds_100mb_250k_contexts&fromUser=true&my_code=enabled&refresh_mode=paused&viz=flame_graph&from_ts=1745422420000&to_ts=1745423380000&live=false)\] \[[SMP Dashboard](https://app.datadoghq.com/dashboard/4br-nxz-khi?fromUser=true&tpl_var_dsd-run-id%5B0%5D=50f42e37-61d0-4365-8a33-de1b10338744&tpl_var_experiment%5B0%5D=dsd_uds_100mb_250k_contexts&tpl_var_adp-run-id%5B0%5D=0f359c7d-31ae-4598-9f09-bce5b1619d40&view=spans&from_ts=1745421580000&to_ts=1745424123000&live=false)\] |
| dsd_uds_100mb_3k_contexts | \[[Profiling (ADP)](https://app.datadoghq.com/profiling/explorer?query=env%3Asingle-machine-performance%20run-id%3A0f359c7d-31ae-4598-9f09-bce5b1619d40%20experiment%3Adsd_uds_100mb_3k_contexts&fromUser=true&my_code=enabled&refresh_mode=paused&viz=flame_graph&from_ts=1745423103000&to_ts=1745425323000&live=false)\] \[[Profiling (DSD)](https://app.datadoghq.com/profiling/explorer?query=env%3Asingle-machine-performance%20run-id%3A50f42e37-61d0-4365-8a33-de1b10338744%20experiment%3Adsd_uds_100mb_3k_contexts&fromUser=true&my_code=enabled&refresh_mode=paused&viz=flame_graph&from_ts=1745422420000&to_ts=1745423380000&live=false)\] \[[SMP Dashboard](https://app.datadoghq.com/dashboard/4br-nxz-khi?fromUser=true&tpl_var_dsd-run-id%5B0%5D=50f42e37-61d0-4365-8a33-de1b10338744&tpl_var_experiment%5B0%5D=dsd_uds_100mb_3k_contexts&tpl_var_adp-run-id%5B0%5D=0f359c7d-31ae-4598-9f09-bce5b1619d40&view=spans&from_ts=1745421580000&to_ts=1745424123000&live=false)\] |
| dsd_uds_100mb_3k_contexts_distributions_only | \[[Profiling (ADP)](https://app.datadoghq.com/profiling/explorer?query=env%3Asingle-machine-performance%20run-id%3A0f359c7d-31ae-4598-9f09-bce5b1619d40%20experiment%3Adsd_uds_100mb_3k_contexts_distributions_only&fromUser=true&my_code=enabled&refresh_mode=paused&viz=flame_graph&from_ts=1745423103000&to_ts=1745425323000&live=false)\] \[[Profiling (DSD)](https://app.datadoghq.com/profiling/explorer?query=env%3Asingle-machine-performance%20run-id%3A50f42e37-61d0-4365-8a33-de1b10338744%20experiment%3Adsd_uds_100mb_3k_contexts_distributions_only&fromUser=true&my_code=enabled&refresh_mode=paused&viz=flame_graph&from_ts=1745422420000&to_ts=1745423380000&live=false)\] \[[SMP Dashboard](https://app.datadoghq.com/dashboard/4br-nxz-khi?fromUser=true&tpl_var_dsd-run-id%5B0%5D=50f42e37-61d0-4365-8a33-de1b10338744&tpl_var_experiment%5B0%5D=dsd_uds_100mb_3k_contexts_distributions_only&tpl_var_adp-run-id%5B0%5D=0f359c7d-31ae-4598-9f09-bce5b1619d40&view=spans&from_ts=1745421580000&to_ts=1745424123000&live=false)\] |
| dsd_uds_10mb_3k_contexts | \[[Profiling (ADP)](https://app.datadoghq.com/profiling/explorer?query=env%3Asingle-machine-performance%20run-id%3A0f359c7d-31ae-4598-9f09-bce5b1619d40%20experiment%3Adsd_uds_10mb_3k_contexts&fromUser=true&my_code=enabled&refresh_mode=paused&viz=flame_graph&from_ts=1745423103000&to_ts=1745425323000&live=false)\] \[[Profiling (DSD)](https://app.datadoghq.com/profiling/explorer?query=env%3Asingle-machine-performance%20run-id%3A50f42e37-61d0-4365-8a33-de1b10338744%20experiment%3Adsd_uds_10mb_3k_contexts&fromUser=true&my_code=enabled&refresh_mode=paused&viz=flame_graph&from_ts=1745422420000&to_ts=1745423380000&live=false)\] \[[SMP Dashboard](https://app.datadoghq.com/dashboard/4br-nxz-khi?fromUser=true&tpl_var_dsd-run-id%5B0%5D=50f42e37-61d0-4365-8a33-de1b10338744&tpl_var_experiment%5B0%5D=dsd_uds_10mb_3k_contexts&tpl_var_adp-run-id%5B0%5D=0f359c7d-31ae-4598-9f09-bce5b1619d40&view=spans&from_ts=1745421580000&to_ts=1745424123000&live=false)\] |
| dsd_uds_1mb_3k_contexts | \[[Profiling (ADP)](https://app.datadoghq.com/profiling/explorer?query=env%3Asingle-machine-performance%20run-id%3A0f359c7d-31ae-4598-9f09-bce5b1619d40%20experiment%3Adsd_uds_1mb_3k_contexts&fromUser=true&my_code=enabled&refresh_mode=paused&viz=flame_graph&from_ts=1745423103000&to_ts=1745425323000&live=false)\] \[[Profiling (DSD)](https://app.datadoghq.com/profiling/explorer?query=env%3Asingle-machine-performance%20run-id%3A50f42e37-61d0-4365-8a33-de1b10338744%20experiment%3Adsd_uds_1mb_3k_contexts&fromUser=true&my_code=enabled&refresh_mode=paused&viz=flame_graph&from_ts=1745422420000&to_ts=1745423380000&live=false)\] \[[SMP Dashboard](https://app.datadoghq.com/dashboard/4br-nxz-khi?fromUser=true&tpl_var_dsd-run-id%5B0%5D=50f42e37-61d0-4365-8a33-de1b10338744&tpl_var_experiment%5B0%5D=dsd_uds_1mb_3k_contexts&tpl_var_adp-run-id%5B0%5D=0f359c7d-31ae-4598-9f09-bce5b1619d40&view=spans&from_ts=1745421580000&to_ts=1745424123000&live=false)\] |
| dsd_uds_1mb_3k_contexts_dualship | \[[Profiling (ADP)](https://app.datadoghq.com/profiling/explorer?query=env%3Asingle-machine-performance%20run-id%3A0f359c7d-31ae-4598-9f09-bce5b1619d40%20experiment%3Adsd_uds_1mb_3k_contexts_dualship&fromUser=true&my_code=enabled&refresh_mode=paused&viz=flame_graph&from_ts=1745423103000&to_ts=1745425323000&live=false)\] \[[Profiling (DSD)](https://app.datadoghq.com/profiling/explorer?query=env%3Asingle-machine-performance%20run-id%3A50f42e37-61d0-4365-8a33-de1b10338744%20experiment%3Adsd_uds_1mb_3k_contexts_dualship&fromUser=true&my_code=enabled&refresh_mode=paused&viz=flame_graph&from_ts=1745422420000&to_ts=1745423380000&live=false)\] \[[SMP Dashboard](https://app.datadoghq.com/dashboard/4br-nxz-khi?fromUser=true&tpl_var_dsd-run-id%5B0%5D=50f42e37-61d0-4365-8a33-de1b10338744&tpl_var_experiment%5B0%5D=dsd_uds_1mb_3k_contexts_dualship&tpl_var_adp-run-id%5B0%5D=0f359c7d-31ae-4598-9f09-bce5b1619d40&view=spans&from_ts=1745421580000&to_ts=1745424123000&live=false)\] |
| dsd_uds_1mb_50k_contexts | \[[Profiling (ADP)](https://app.datadoghq.com/profiling/explorer?query=env%3Asingle-machine-performance%20run-id%3A0f359c7d-31ae-4598-9f09-bce5b1619d40%20experiment%3Adsd_uds_1mb_50k_contexts&fromUser=true&my_code=enabled&refresh_mode=paused&viz=flame_graph&from_ts=1745423103000&to_ts=1745425323000&live=false)\] \[[Profiling (DSD)](https://app.datadoghq.com/profiling/explorer?query=env%3Asingle-machine-performance%20run-id%3A50f42e37-61d0-4365-8a33-de1b10338744%20experiment%3Adsd_uds_1mb_50k_contexts&fromUser=true&my_code=enabled&refresh_mode=paused&viz=flame_graph&from_ts=1745422420000&to_ts=1745423380000&live=false)\] \[[SMP Dashboard](https://app.datadoghq.com/dashboard/4br-nxz-khi?fromUser=true&tpl_var_dsd-run-id%5B0%5D=50f42e37-61d0-4365-8a33-de1b10338744&tpl_var_experiment%5B0%5D=dsd_uds_1mb_50k_contexts&tpl_var_adp-run-id%5B0%5D=0f359c7d-31ae-4598-9f09-bce5b1619d40&view=spans&from_ts=1745421580000&to_ts=1745424123000&live=false)\] |
| dsd_uds_1mb_50k_contexts_memlimit | \[[Profiling (ADP)](https://app.datadoghq.com/profiling/explorer?query=env%3Asingle-machine-performance%20run-id%3A0f359c7d-31ae-4598-9f09-bce5b1619d40%20experiment%3Adsd_uds_1mb_50k_contexts_memlimit&fromUser=true&my_code=enabled&refresh_mode=paused&viz=flame_graph&from_ts=1745423103000&to_ts=1745425323000&live=false)\] \[[Profiling (DSD)](https://app.datadoghq.com/profiling/explorer?query=env%3Asingle-machine-performance%20run-id%3A50f42e37-61d0-4365-8a33-de1b10338744%20experiment%3Adsd_uds_1mb_50k_contexts_memlimit&fromUser=true&my_code=enabled&refresh_mode=paused&viz=flame_graph&from_ts=1745422420000&to_ts=1745423380000&live=false)\] \[[SMP Dashboard](https://app.datadoghq.com/dashboard/4br-nxz-khi?fromUser=true&tpl_var_dsd-run-id%5B0%5D=50f42e37-61d0-4365-8a33-de1b10338744&tpl_var_experiment%5B0%5D=dsd_uds_1mb_50k_contexts_memlimit&tpl_var_adp-run-id%5B0%5D=0f359c7d-31ae-4598-9f09-bce5b1619d40&view=spans&from_ts=1745421580000&to_ts=1745424123000&live=false)\] |
| dsd_uds_40mb_12k_contexts_40_senders | \[[Profiling (ADP)](https://app.datadoghq.com/profiling/explorer?query=env%3Asingle-machine-performance%20run-id%3A0f359c7d-31ae-4598-9f09-bce5b1619d40%20experiment%3Adsd_uds_40mb_12k_contexts_40_senders&fromUser=true&my_code=enabled&refresh_mode=paused&viz=flame_graph&from_ts=1745423103000&to_ts=1745425323000&live=false)\] \[[Profiling (DSD)](https://app.datadoghq.com/profiling/explorer?query=env%3Asingle-machine-performance%20run-id%3A50f42e37-61d0-4365-8a33-de1b10338744%20experiment%3Adsd_uds_40mb_12k_contexts_40_senders&fromUser=true&my_code=enabled&refresh_mode=paused&viz=flame_graph&from_ts=1745422420000&to_ts=1745423380000&live=false)\] \[[SMP Dashboard](https://app.datadoghq.com/dashboard/4br-nxz-khi?fromUser=true&tpl_var_dsd-run-id%5B0%5D=50f42e37-61d0-4365-8a33-de1b10338744&tpl_var_experiment%5B0%5D=dsd_uds_40mb_12k_contexts_40_senders&tpl_var_adp-run-id%5B0%5D=0f359c7d-31ae-4598-9f09-bce5b1619d40&view=spans&from_ts=1745421580000&to_ts=1745424123000&live=false)\] |
| dsd_uds_500mb_3k_contexts | \[[Profiling (ADP)](https://app.datadoghq.com/profiling/explorer?query=env%3Asingle-machine-performance%20run-id%3A0f359c7d-31ae-4598-9f09-bce5b1619d40%20experiment%3Adsd_uds_500mb_3k_contexts&fromUser=true&my_code=enabled&refresh_mode=paused&viz=flame_graph&from_ts=1745423103000&to_ts=1745425323000&live=false)\] \[[Profiling (DSD)](https://app.datadoghq.com/profiling/explorer?query=env%3Asingle-machine-performance%20run-id%3A50f42e37-61d0-4365-8a33-de1b10338744%20experiment%3Adsd_uds_500mb_3k_contexts&fromUser=true&my_code=enabled&refresh_mode=paused&viz=flame_graph&from_ts=1745422420000&to_ts=1745423380000&live=false)\] \[[SMP Dashboard](https://app.datadoghq.com/dashboard/4br-nxz-khi?fromUser=true&tpl_var_dsd-run-id%5B0%5D=50f42e37-61d0-4365-8a33-de1b10338744&tpl_var_experiment%5B0%5D=dsd_uds_500mb_3k_contexts&tpl_var_adp-run-id%5B0%5D=0f359c7d-31ae-4598-9f09-bce5b1619d40&view=spans&from_ts=1745421580000&to_ts=1745424123000&live=false)\] |
| dsd_uds_512kb_3k_contexts | \[[Profiling (ADP)](https://app.datadoghq.com/profiling/explorer?query=env%3Asingle-machine-performance%20run-id%3A0f359c7d-31ae-4598-9f09-bce5b1619d40%20experiment%3Adsd_uds_512kb_3k_contexts&fromUser=true&my_code=enabled&refresh_mode=paused&viz=flame_graph&from_ts=1745423103000&to_ts=1745425323000&live=false)\] \[[Profiling (DSD)](https://app.datadoghq.com/profiling/explorer?query=env%3Asingle-machine-performance%20run-id%3A50f42e37-61d0-4365-8a33-de1b10338744%20experiment%3Adsd_uds_512kb_3k_contexts&fromUser=true&my_code=enabled&refresh_mode=paused&viz=flame_graph&from_ts=1745422420000&to_ts=1745423380000&live=false)\] \[[SMP Dashboard](https://app.datadoghq.com/dashboard/4br-nxz-khi?fromUser=true&tpl_var_dsd-run-id%5B0%5D=50f42e37-61d0-4365-8a33-de1b10338744&tpl_var_experiment%5B0%5D=dsd_uds_512kb_3k_contexts&tpl_var_adp-run-id%5B0%5D=0f359c7d-31ae-4598-9f09-bce5b1619d40&view=spans&from_ts=1745421580000&to_ts=1745424123000&live=false)\] |
| quality_gates_idle_rss | \[[Profiling (ADP)](https://app.datadoghq.com/profiling/explorer?query=env%3Asingle-machine-performance%20run-id%3A0f359c7d-31ae-4598-9f09-bce5b1619d40%20experiment%3Aquality_gates_idle_rss&fromUser=true&my_code=enabled&refresh_mode=paused&viz=flame_graph&from_ts=1745423103000&to_ts=1745425323000&live=false)\] \[[Profiling (DSD)](https://app.datadoghq.com/profiling/explorer?query=env%3Asingle-machine-performance%20run-id%3A50f42e37-61d0-4365-8a33-de1b10338744%20experiment%3Aquality_gates_idle_rss&fromUser=true&my_code=enabled&refresh_mode=paused&viz=flame_graph&from_ts=1745422420000&to_ts=1745423380000&live=false)\] \[[SMP Dashboard](https://app.datadoghq.com/dashboard/4br-nxz-khi?fromUser=true&tpl_var_dsd-run-id%5B0%5D=50f42e37-61d0-4365-8a33-de1b10338744&tpl_var_experiment%5B0%5D=quality_gates_idle_rss&tpl_var_adp-run-id%5B0%5D=0f359c7d-31ae-4598-9f09-bce5b1619d40&view=spans&from_ts=1745421580000&to_ts=1745424123000&live=false)\] |
| dsd_uds_50mb_10k_contexts_no_inlining (ADP only) | \[[Profiling (ADP)](https://app.datadoghq.com/profiling/explorer?query=env%3Asingle-machine-performance%20run-id%3A0f359c7d-31ae-4598-9f09-bce5b1619d40%20experiment%3Adsd_uds_50mb_10k_contexts_no_inlining&fromUser=true&my_code=enabled&refresh_mode=paused&viz=flame_graph&from_ts=1745423103000&to_ts=1745425323000&live=false)\] \[[SMP Dashboard](https://app.datadoghq.com/dashboard/4br-nxz-khi?fromUser=true&tpl_var_dsd-run-id%5B0%5D=50f42e37-61d0-4365-8a33-de1b10338744&tpl_var_experiment%5B0%5D=dsd_uds_50mb_10k_contexts_no_inlining&tpl_var_adp-run-id%5B0%5D=0f359c7d-31ae-4598-9f09-bce5b1619d40&view=spans&from_ts=1745423103000&to_ts=1745425323000&live=false)\] |
| dsd_uds_50mb_10k_contexts_no_inlining_no_allocs (ADP only) | \[[Profiling (ADP)](https://app.datadoghq.com/profiling/explorer?query=env%3Asingle-machine-performance%20run-id%3A0f359c7d-31ae-4598-9f09-bce5b1619d40%20experiment%3Adsd_uds_50mb_10k_contexts_no_inlining_no_allocs&fromUser=true&my_code=enabled&refresh_mode=paused&viz=flame_graph&from_ts=1745423103000&to_ts=1745425323000&live=false)\] \[[SMP Dashboard](https://app.datadoghq.com/dashboard/4br-nxz-khi?fromUser=true&tpl_var_dsd-run-id%5B0%5D=50f42e37-61d0-4365-8a33-de1b10338744&tpl_var_experiment%5B0%5D=dsd_uds_50mb_10k_contexts_no_inlining_no_allocs&tpl_var_adp-run-id%5B0%5D=0f359c7d-31ae-4598-9f09-bce5b1619d40&view=spans&from_ts=1745423103000&to_ts=1745425323000&live=false)\] |
## Checks Agent Experiment Result Links

| experiment | link(s) |
|------------|---------|
| quality_gates_idle_rss | \[[Profiling](https://app.datadoghq.com/profiling/explorer?query=env%3Asingle-machine-performance%20run-id%3Adb53f723-7b7c-41f9-8eb2-5e388f08e370%20experiment%3Aquality_gates_idle_rss&fromUser=true&my_code=enabled&refresh_mode=paused&viz=flame_graph&from_ts=1745421396000&to_ts=1745422436000&live=false)\] \[[SMP Dashboard](https://app.datadoghq.com/dashboard/4br-nxz-khi?fromUser=true&tpl_var_dsd-run-id%5B0%5D=c9f4602a-902b-40d3-8ba5-e18ae99c929b&tpl_var_experiment%5B0%5D=quality_gates_idle_rss&tpl_var_adp-run-id%5B0%5D=db53f723-7b7c-41f9-8eb2-5e388f08e370&view=spans&from_ts=1745421396000&to_ts=1745422436000&live=false)\] |

author_association: NONE