Repository: elastic/elastic-agent
Pull request #8241: [Chore] Remove `http/metric` streams for beat receivers
Author: login: khushijain21
Labels: name: skip-changelog
name: backport-9.0
description: Automated backport to the 9.0 branch
name: backport-8.19
description: Automated backport to the 8.19 branch
Created: 2025-05-26T15:27:52Z | Merged: 2025-05-28T01:17:38Z

<!-- Type of change
Please label this PR with one of the following labels, depending on the scope of your change:
- Bug
- Enhancement
- Breaking change
- Deprecation
- Cleanup
- Docs
-->

## What does this PR do?
`http/metric` stream fetches process metrics for each beat by querying the `/stats` endpoint and sending it to `metrics-elastic_agent.elastic_agent.*` index.

Since process related metrics are not applicable for beatreceivers, it is essentially a no-op. We can go ahead and drop these streams.
On inspecting a sample document ingested by such `http/stream` by beatreceiver, we can see that it contains no information from the `/stats` endpoint 

<details>
<summary>http-metric.json</summary>

```json
{
  "_index": ".ds-metrics-elastic_agent.elastic_agent-rjqmgir5ow8scl1del1iwskeuf4pbncgyoecjludi-2025.05.23-000001",
  "_id": "ClkK_ZYBjFm4DOfRpEXe",
  "_version": 1,
  "_source": {
    "data_stream": {
      "dataset": "elastic_agent.elastic_agent",
      "namespace": "rjqmgir5ow8scl1del1iwskeuf4pbncgyoecjludi",
      "type": "metrics"
    },
    "agent": {
      "id": "32587c90-4315-4a01-b761-d95a1123c1a7",
      "version": "9.1.0",
      "ephemeral_id": "9fd0df3b-1553-4e9e-a704-350317b1566b",
      "name": "Khushis-MacBook-Pro.local",
      "type": "metricbeat"
    },
    "event": {
      "duration": 1574041,
      "dataset": "elastic_agent.elastic_agent",
      "module": "http"
    },
    "metricset": {
      "name": "json",
      "period": 60000
    },
    "component": {
      "id": "beat/metrics-monitoring",
      "binary": "metricbeat"
    },
    "elastic_agent": {
      "id": "32587c90-4315-4a01-b761-d95a1123c1a7",
      "snapshot": true,
      "version": "9.1.0",
      "process": "metricbeat"
    },
    "ecs": {
      "version": "8.0.0"
    },
    "host": {
      "ip": [
        "fe80::18b5:7eaf:1ab:1211",
        "2405:201:d00a:e8b1:4ae:e140:2a0f:88b4",
        "2405:201:d00a:e8b1:9d9d:1e84:141:52fb",
        "192.168.29.170",
        "fe80::10fa:d8ff:fe06:8ea9",
        "fe80::10fa:d8ff:fe06:8ea9",
        "fe80::44af:21d5:2d6f:811e",
        "fe80::5ec6:bb77:d4e4:cfa8",
        "fe80::a88a:b533:f8f:e814",
        "fe80::ce81:b1c:bd2c:69e"
      ],
      "name": "khushis-macbook-pro.local",
      "mac": [
        "12-FA-D8-06-8E-A9",
        "1E-2B-07-D9-EC-0B",
        "1E-2B-07-D9-EC-0C",
        "1E-2B-07-D9-EC-0D",
        "1E-2B-07-D9-EC-EB",
        "1E-2B-07-D9-EC-EC",
        "1E-2B-07-D9-EC-ED",
        "36-B4-B1-E5-79-80",
        "36-B4-B1-E5-79-84",
        "36-B4-B1-E5-79-88",
        "86-97-BC-E1-6C-B4",
        "DA-D7-45-B7-43-12"
      ],
      "hostname": "Khushis-MacBook-Pro.local",
      "architecture": "arm64",
      "os": {
        "platform": "darwin",
        "version": "15.5",
        "family": "darwin",
        "name": "macOS",
        "kernel": "24.5.0",
        "build": "24F74",
        "type": "macos"
      },
      "id": "194CE554-9A32-579D-B00E-029E6412C7D4"
    },
    "@timestamp": "2025-05-23T12:07:29.556Z",
    "service": {
      "address": "http://unix/stats",
      "type": "http"
    }
  },
  "fields": {
    "elastic_agent.version": [
      "9.1.0"
    ],
    "component.binary": [
      "metricbeat"
    ],
    "elastic_agent.process": [
      "metricbeat"
    ],
    "host.hostname": [
      "Khushis-MacBook-Pro.local"
    ],
    "host.mac": [
      "12-FA-D8-06-8E-A9",
      "1E-2B-07-D9-EC-0B",
      "1E-2B-07-D9-EC-0C",
      "1E-2B-07-D9-EC-0D",
      "1E-2B-07-D9-EC-EB",
      "1E-2B-07-D9-EC-EC",
      "1E-2B-07-D9-EC-ED",
      "36-B4-B1-E5-79-80",
      "36-B4-B1-E5-79-84",
      "36-B4-B1-E5-79-88",
      "86-97-BC-E1-6C-B4",
      "DA-D7-45-B7-43-12"
    ],
    "host.os.build": [
      "24F74"
    ],
    "service.type": [
      "http"
    ],
    "host.ip": [
      "fe80::18b5:7eaf:1ab:1211",
      "2405:201:d00a:e8b1:4ae:e140:2a0f:88b4",
      "2405:201:d00a:e8b1:9d9d:1e84:141:52fb",
      "192.168.29.170",
      "fe80::10fa:d8ff:fe06:8ea9",
      "fe80::10fa:d8ff:fe06:8ea9",
      "fe80::44af:21d5:2d6f:811e",
      "fe80::5ec6:bb77:d4e4:cfa8",
      "fe80::a88a:b533:f8f:e814",
      "fe80::ce81:b1c:bd2c:69e"
    ],
    "agent.type": [
      "metricbeat"
    ],
    "component.id": [
      "beat/metrics-monitoring"
    ],
    "event.module": [
      "http"
    ],
    "host.os.version": [
      "15.5"
    ],
    "host.os.kernel": [
      "24.5.0"
    ],
    "host.os.name": [
      "macOS"
    ],
    "agent.name": [
      "Khushis-MacBook-Pro.local"
    ],
    "elastic_agent.snapshot": [
      true
    ],
    "host.name": [
      "khushis-macbook-pro.local"
    ],
    "host.id": [
      "194CE554-9A32-579D-B00E-029E6412C7D4"
    ],
    "host.os.type": [
      "macos"
    ],
    "elastic_agent.id": [
      "32587c90-4315-4a01-b761-d95a1123c1a7"
    ],
    "data_stream.namespace": [
      "rjqmgir5ow8scl1del1iwskeuf4pbncgyoecjludi"
    ],
    "metricset.period": [
      60000
    ],
    "data_stream.type": [
      "metrics"
    ],
    "event.duration": [
      1574041
    ],
    "host.architecture": [
      "arm64"
    ],
    "metricset.name": [
      "json"
    ],
    "@timestamp": [
      "2025-05-23T12:07:29.556Z"
    ],
    "agent.id": [
      "32587c90-4315-4a01-b761-d95a1123c1a7"
    ],
    "ecs.version": [
      "8.0.0"
    ],
    "host.os.platform": [
      "darwin"
    ],
    "service.address": [
      "http://unix/stats"
    ],
    "data_stream.dataset": [
      "elastic_agent.elastic_agent"
    ],
    "agent.ephemeral_id": [
      "9fd0df3b-1553-4e9e-a704-350317b1566b"
    ],
    "agent.version": [
      "9.1.0"
    ],
    "host.os.family": [
      "darwin"
    ],
    "event.dataset": [
      "elastic_agent.elastic_agent"
    ]
  }
}
```
</details>

<!-- Mandatory
Explain here the changes you made on the PR. Please explain the WHAT: patterns used, algorithms implemented, design architecture, message processing, etc.
-->



## Checklist

<!-- Mandatory
Add a checklist of things that are required to be reviewed in order to have the PR approved

List here all the items you have verified BEFORE sending this PR. Please DO NOT remove any item, striking through those that do not apply. (Just in case, strikethrough uses two tildes. ~~Scratch this.~~)
-->

- [x] I have read and understood the [pull request guidelines](https://github.com/elastic/elastic-agent/blob/main/CONTRIBUTING.md#pull-request-guidelines) of this project.
- [x] My code follows the style guidelines of this project
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] I have made corresponding change to the default configuration files
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] I have added an entry in `./changelog/fragments` using the [changelog tool](https://github.com/elastic/elastic-agent#changelog)
- [ ] I have added an integration test or an E2E test

## How to test this PR locally

1. When going from process runtime to otel runtime, we can see that only elastic-agent metrics are collected 

<img width="1512" alt="image" src="https://github.com/user-attachments/assets/91f9980e-0f75-4582-8d6d-2fd93faaca35" />

2. When going from otel runtime to process runtime, the original behavior remains consistent
Note it require us to set `_runtime_experimental:process` explicitly for it to work. 
<img width="1512" alt="image" src="https://github.com/user-attachments/assets/a8b165af-1648-46ce-a3a3-aaba9a75e9ca" />





Diff:
diff --git a/internal/pkg/agent/application/monitoring/testdata/monitoring_config_full.yaml b/internal/pkg/agent/application/monitoring/testdata/monitoring_config_full.yaml
index b4b7320ce48..ed34c1a4133 100644
--- a/internal/pkg/agent/application/monitoring/testdata/monitoring_config_full.yaml
+++ b/internal/pkg/agent/application/monitoring/testdata/monitoring_config_full.yaml
@@ -653,62 +653,6 @@ inputs:
           binary: filebeat
           id: filestream-monitoring
         target: component
-  - data_stream:
-      dataset: elastic_agent.elastic_agent
-      namespace: default
-      type: metrics
-    failure_threshold: 5
-    hosts:
-    - placeholder
-    id: metrics-monitoring-filebeat-1
-    index: metrics-elastic_agent.elastic_agent-default
-    metricsets:
-    - json
-    namespace: agent
-    path: /stats
-    period: 1m0s
-    processors:
-    - add_fields:
-        fields:
-          dataset: elastic_agent.elastic_agent
-        target: event
-    - add_fields:
-        fields:
-          id: ""
-          process: filebeat
-          snapshot: false
-          version: placeholder
-        target: elastic_agent
-    - add_fields:
-        fields:
-          id: ""
-        target: agent
-    - copy_fields:
-        fail_on_error: false
-        fields:
-        - from: http.agent.beat.cpu
-          to: system.process.cpu
-        - from: http.agent.beat.memstats.memory_sys
-          to: system.process.memory.size
-        - from: http.agent.beat.handles
-          to: system.process.fd
-        - from: http.agent.beat.cgroup
-          to: system.process.cgroup
-        - from: http.agent.apm-server
-          to: apm-server
-        - from: http.filebeat_input
-          to: filebeat_input
-        ignore_missing: true
-    - drop_fields:
-        fields:
-        - http
-        - system
-        ignore_missing: true
-    - add_fields:
-        fields:
-          binary: filebeat
-          id: filestream-otel
-        target: component
   - data_stream:
       dataset: elastic_agent.elastic_agent
       namespace: default
diff --git a/internal/pkg/agent/application/monitoring/v1_monitor.go b/internal/pkg/agent/application/monitoring/v1_monitor.go
index 78f11c35542..ccb74713c53 100644
--- a/internal/pkg/agent/application/monitoring/v1_monitor.go
+++ b/internal/pkg/agent/application/monitoring/v1_monitor.go
@@ -670,26 +670,28 @@ func (b *BeatsMonitor) getHttpStreams(
 		endpoints := []interface{}{prefixedEndpoint(utils.SocketURLWithFallback(compInfo.ID, paths.TempDir()))}
 		name := sanitizeName(binaryName)
 
-		httpStream := map[string]interface{}{
-			idKey: fmt.Sprintf("%s-%s-1", monitoringMetricsUnitID, name),
-			"data_stream": map[string]interface{}{
-				"type":      "metrics",
-				"dataset":   dataset,
-				"namespace": monitoringNamespace,
-			},
-			"metricsets": []interface{}{"json"},
-			"hosts":      endpoints,
-			"path":       "/stats",
-			"namespace":  "agent",
-			"period":     metricsCollectionIntervalString,
-			"index":      indexName,
-			"processors": processorsForHttpStream(binaryName, compInfo.ID, dataset, b.agentInfo, compInfo.RuntimeManager),
-		}
-		if failureThreshold != nil {
-			httpStream[failureThresholdKey] = *failureThreshold
+		// Do not create http streams if runtime-manager is otel and binary is of beat type
+		if compInfo.RuntimeManager != component.OtelRuntimeManager || !strings.HasSuffix(binaryName, "beat") {
+			httpStream := map[string]interface{}{
+				idKey: fmt.Sprintf("%s-%s-1", monitoringMetricsUnitID, name),
+				"data_stream": map[string]interface{}{
+					"type":      "metrics",
+					"dataset":   dataset,
+					"namespace": monitoringNamespace,
+				},
+				"metricsets": []interface{}{"json"},
+				"hosts":      endpoints,
+				"path":       "/stats",
+				"namespace":  "agent",
+				"period":     metricsCollectionIntervalString,
+				"index":      indexName,
+				"processors": processorsForHttpStream(binaryName, compInfo.ID, dataset, b.agentInfo, compInfo.RuntimeManager),
+			}
+			if failureThreshold != nil {
+				httpStream[failureThresholdKey] = *failureThreshold
+			}
+			httpStreams = append(httpStreams, httpStream)
 		}
-		httpStreams = append(httpStreams, httpStream)
-
 		// specifically for filebeat, we include input metrics
 		// disabled for filebeat receiver until https://github.com/elastic/beats/issues/43418 is resolved
 		if strings.EqualFold(name, "filebeat") && compInfo.RuntimeManager != component.OtelRuntimeManager {
@@ -1262,7 +1264,7 @@ func httpCopyRules() []interface{} {
 			"to":   "system.process.memory.size",
 		},
 
-		// I should be able to see fd usage. Am I keep too many files open?
+		// I should be able to see fd usage. Am I keeping too many files open?
 		map[string]interface{}{
 			"from": "http.agent.beat.handles",
 			"to":   "system.process.fd",


Reviews:
user: login: cmacknz
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-26T20:30:44Z
user: login: cmacknz
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-26T20:40:33Z
user: login: khushijain21
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-27T05:29:10Z
user: login: khushijain21
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-27T07:02:08Z
user: login: cmacknz
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-27T19:52:01Z
user: login: cmacknz
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-27T19:52:45Z
user: login: cmacknz
state: APPROVED
author_association: MEMBER
submitted_at: 2025-05-27T20:22:24Z
user: login: cmacknz
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-27T20:23:36Z

Review comments:
diff_hunk: @@ -670,26 +670,28 @@ func (b *BeatsMonitor) getHttpStreams(
 		endpoints := []interface{}{prefixedEndpoint(utils.SocketURLWithFallback(compInfo.ID, paths.TempDir()))}
 		name := sanitizeName(binaryName)
 
-		httpStream := map[string]interface{}{
-			idKey: fmt.Sprintf("%s-%s-1", monitoringMetricsUnitID, name),
-			"data_stream": map[string]interface{}{
-				"type":      "metrics",
-				"dataset":   dataset,
-				"namespace": monitoringNamespace,
-			},
-			"metricsets": []interface{}{"json"},
-			"hosts":      endpoints,
-			"path":       "/stats",
-			"namespace":  "agent",
-			"period":     metricsCollectionIntervalString,
-			"index":      indexName,
-			"processors": processorsForHttpStream(binaryName, compInfo.ID, dataset, b.agentInfo, compInfo.RuntimeManager),
-		}
-		if failureThreshold != nil {
-			httpStream[failureThresholdKey] = *failureThreshold
+		// Do not create http streams if runtime-manager is otel and binary is of beat type
+		if compInfo.RuntimeManager != component.OtelRuntimeManager || !strings.HasSuffix(binaryName, "beat") {
path: internal/pkg/agent/application/monitoring/v1_monitor.go
user: login: cmacknz
body: If you switch the runtime while agent is running, does this work correctly? What I would expect is:
1. Going from standard to otel runtime, the http/metrics component is dropped.
2. Going from otel to standard runtime, the http/metrics component is restored.
created_at: 2025-05-26T20:30:44Z
author_association: MEMBER
diff_hunk: @@ -670,26 +670,28 @@ func (b *BeatsMonitor) getHttpStreams(
 		endpoints := []interface{}{prefixedEndpoint(utils.SocketURLWithFallback(compInfo.ID, paths.TempDir()))}
 		name := sanitizeName(binaryName)
 
-		httpStream := map[string]interface{}{
-			idKey: fmt.Sprintf("%s-%s-1", monitoringMetricsUnitID, name),
-			"data_stream": map[string]interface{}{
-				"type":      "metrics",
-				"dataset":   dataset,
-				"namespace": monitoringNamespace,
-			},
-			"metricsets": []interface{}{"json"},
-			"hosts":      endpoints,
-			"path":       "/stats",
-			"namespace":  "agent",
-			"period":     metricsCollectionIntervalString,
-			"index":      indexName,
-			"processors": processorsForHttpStream(binaryName, compInfo.ID, dataset, b.agentInfo, compInfo.RuntimeManager),
-		}
-		if failureThreshold != nil {
-			httpStream[failureThresholdKey] = *failureThreshold
+		// Do not create http streams if runtime-manager is otel and binary is of beat type
+		if compInfo.RuntimeManager != component.OtelRuntimeManager || !strings.HasSuffix(binaryName, "beat") {
path: internal/pkg/agent/application/monitoring/v1_monitor.go
user: login: cmacknz
body: Why is the `"beat"` suffix check necessary? Shouldn't this entirely be covered by the first part of the if condition?

The beat suffix check would cover apm-server which uses libbeat but is not a beat, but it would't cover cloudbeat which will take some time to migrate into the collector as it isn't in the beats repository.
created_at: 2025-05-26T20:40:33Z
author_association: MEMBER
diff_hunk: @@ -670,26 +670,28 @@ func (b *BeatsMonitor) getHttpStreams(
 		endpoints := []interface{}{prefixedEndpoint(utils.SocketURLWithFallback(compInfo.ID, paths.TempDir()))}
 		name := sanitizeName(binaryName)
 
-		httpStream := map[string]interface{}{
-			idKey: fmt.Sprintf("%s-%s-1", monitoringMetricsUnitID, name),
-			"data_stream": map[string]interface{}{
-				"type":      "metrics",
-				"dataset":   dataset,
-				"namespace": monitoringNamespace,
-			},
-			"metricsets": []interface{}{"json"},
-			"hosts":      endpoints,
-			"path":       "/stats",
-			"namespace":  "agent",
-			"period":     metricsCollectionIntervalString,
-			"index":      indexName,
-			"processors": processorsForHttpStream(binaryName, compInfo.ID, dataset, b.agentInfo, compInfo.RuntimeManager),
-		}
-		if failureThreshold != nil {
-			httpStream[failureThresholdKey] = *failureThreshold
+		// Do not create http streams if runtime-manager is otel and binary is of beat type
+		if compInfo.RuntimeManager != component.OtelRuntimeManager || !strings.HasSuffix(binaryName, "beat") {
path: internal/pkg/agent/application/monitoring/v1_monitor.go
user: login: khushijain21
body: > Why is the "beat" suffix check necessary

Because in case of `apm-server` -  we send ["all"](https://github.com/elastic/elastic-agent/blob/main/internal/pkg/agent/application/monitoring/v1_monitor.go#L1278-L1281
) apm-server related metrics to output. This can include more than just process related metrics as I understand from here https://www.elastic.co/docs/solutions/observability/apm/use-select-metrics-emitted-directly-to-monitoring-cluster

But why we need more than process related metrics for apm-server only - is not clear to me






created_at: 2025-05-27T05:29:10Z
author_association: CONTRIBUTOR
diff_hunk: @@ -670,26 +670,28 @@ func (b *BeatsMonitor) getHttpStreams(
 		endpoints := []interface{}{prefixedEndpoint(utils.SocketURLWithFallback(compInfo.ID, paths.TempDir()))}
 		name := sanitizeName(binaryName)
 
-		httpStream := map[string]interface{}{
-			idKey: fmt.Sprintf("%s-%s-1", monitoringMetricsUnitID, name),
-			"data_stream": map[string]interface{}{
-				"type":      "metrics",
-				"dataset":   dataset,
-				"namespace": monitoringNamespace,
-			},
-			"metricsets": []interface{}{"json"},
-			"hosts":      endpoints,
-			"path":       "/stats",
-			"namespace":  "agent",
-			"period":     metricsCollectionIntervalString,
-			"index":      indexName,
-			"processors": processorsForHttpStream(binaryName, compInfo.ID, dataset, b.agentInfo, compInfo.RuntimeManager),
-		}
-		if failureThreshold != nil {
-			httpStream[failureThresholdKey] = *failureThreshold
+		// Do not create http streams if runtime-manager is otel and binary is of beat type
+		if compInfo.RuntimeManager != component.OtelRuntimeManager || !strings.HasSuffix(binaryName, "beat") {
path: internal/pkg/agent/application/monitoring/v1_monitor.go
user: login: khushijain21
body: > If you switch the runtime while agent is running, does this work correctly?

Yes, it does and I updated the issue description with more details
created_at: 2025-05-27T07:02:08Z
author_association: CONTRIBUTOR
diff_hunk: @@ -670,26 +670,28 @@ func (b *BeatsMonitor) getHttpStreams(
 		endpoints := []interface{}{prefixedEndpoint(utils.SocketURLWithFallback(compInfo.ID, paths.TempDir()))}
 		name := sanitizeName(binaryName)
 
-		httpStream := map[string]interface{}{
-			idKey: fmt.Sprintf("%s-%s-1", monitoringMetricsUnitID, name),
-			"data_stream": map[string]interface{}{
-				"type":      "metrics",
-				"dataset":   dataset,
-				"namespace": monitoringNamespace,
-			},
-			"metricsets": []interface{}{"json"},
-			"hosts":      endpoints,
-			"path":       "/stats",
-			"namespace":  "agent",
-			"period":     metricsCollectionIntervalString,
-			"index":      indexName,
-			"processors": processorsForHttpStream(binaryName, compInfo.ID, dataset, b.agentInfo, compInfo.RuntimeManager),
-		}
-		if failureThreshold != nil {
-			httpStream[failureThresholdKey] = *failureThreshold
+		// Do not create http streams if runtime-manager is otel and binary is of beat type
+		if compInfo.RuntimeManager != component.OtelRuntimeManager || !strings.HasSuffix(binaryName, "beat") {
path: internal/pkg/agent/application/monitoring/v1_monitor.go
user: login: cmacknz
body: > Because in case of apm-server - we send "all" apm-server related metrics to output.

Right, but we never plan to run `apm-server` as a receiver. I'm not sure that's even possible, why isn't the `compInfo.RuntimeManager != component.OtelRuntimeManager` check enough to handle this? Maybe I'm misunderstanding what this does, because I read it as "this component is currently configured to run the collector" which will never be true for `apm-server` AFAIK.
created_at: 2025-05-27T19:52:01Z
author_association: MEMBER
diff_hunk: @@ -670,26 +670,28 @@ func (b *BeatsMonitor) getHttpStreams(
 		endpoints := []interface{}{prefixedEndpoint(utils.SocketURLWithFallback(compInfo.ID, paths.TempDir()))}
 		name := sanitizeName(binaryName)
 
-		httpStream := map[string]interface{}{
-			idKey: fmt.Sprintf("%s-%s-1", monitoringMetricsUnitID, name),
-			"data_stream": map[string]interface{}{
-				"type":      "metrics",
-				"dataset":   dataset,
-				"namespace": monitoringNamespace,
-			},
-			"metricsets": []interface{}{"json"},
-			"hosts":      endpoints,
-			"path":       "/stats",
-			"namespace":  "agent",
-			"period":     metricsCollectionIntervalString,
-			"index":      indexName,
-			"processors": processorsForHttpStream(binaryName, compInfo.ID, dataset, b.agentInfo, compInfo.RuntimeManager),
-		}
-		if failureThreshold != nil {
-			httpStream[failureThresholdKey] = *failureThreshold
+		// Do not create http streams if runtime-manager is otel and binary is of beat type
+		if compInfo.RuntimeManager != component.OtelRuntimeManager || !strings.HasSuffix(binaryName, "beat") {
path: internal/pkg/agent/application/monitoring/v1_monitor.go
user: login: cmacknz
body: Any special handling of apm-server monitoring definitely needs to stay as is because it is potentially hooked into the monitoring of it in Elastic Cloud Hosted where it runs under agent as part of Integrations Server.
created_at: 2025-05-27T19:52:45Z
author_association: MEMBER
diff_hunk: @@ -670,26 +670,28 @@ func (b *BeatsMonitor) getHttpStreams(
 		endpoints := []interface{}{prefixedEndpoint(utils.SocketURLWithFallback(compInfo.ID, paths.TempDir()))}
 		name := sanitizeName(binaryName)
 
-		httpStream := map[string]interface{}{
-			idKey: fmt.Sprintf("%s-%s-1", monitoringMetricsUnitID, name),
-			"data_stream": map[string]interface{}{
-				"type":      "metrics",
-				"dataset":   dataset,
-				"namespace": monitoringNamespace,
-			},
-			"metricsets": []interface{}{"json"},
-			"hosts":      endpoints,
-			"path":       "/stats",
-			"namespace":  "agent",
-			"period":     metricsCollectionIntervalString,
-			"index":      indexName,
-			"processors": processorsForHttpStream(binaryName, compInfo.ID, dataset, b.agentInfo, compInfo.RuntimeManager),
-		}
-		if failureThreshold != nil {
-			httpStream[failureThresholdKey] = *failureThreshold
+		// Do not create http streams if runtime-manager is otel and binary is of beat type
+		if compInfo.RuntimeManager != component.OtelRuntimeManager || !strings.HasSuffix(binaryName, "beat") {
path: internal/pkg/agent/application/monitoring/v1_monitor.go
user: login: cmacknz
body: This works as is, it's more of a nit that this check could be redundant
created_at: 2025-05-27T20:23:36Z
author_association: MEMBER

PR comments:
user: login: mergify[bot]
created_at: 2025-05-26T15:28:31Z
body: This pull request does not have a backport label. Could you fix it @khushijain21? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:
* `backport-./d./d` is the label that automatically backports to the `8./d` branch. `/d` is the digit
* `backport-active-all` is the label that automatically backports to all active branches.
* `backport-active-8` is the label that automatically backports to all active minor branches for the 8 major.
* `backport-active-9` is the label that automatically backports to all active minor branches for the 9 major.
author_association: CONTRIBUTOR
user: login: elasticmachine
created_at: 2025-05-26T17:16:07Z
body: ## :green_heart: Build Succeeded
* [Buildkite Build](https://buildkite.com/elastic/elastic-agent/builds/21491)
* Commit: 1064667592a01d58ea892b90b2d53de3da031073

cc @khushijain21

<!--buildkite-pr-comment-elastic-agent
{"builds":[{"buildStatus":{"state":"passed","success":true,"hasRetries":false,"hasNonPreemptionRetries":false},"url":"https://buildkite.com/elastic/elastic-agent/builds/21491","number":21491,"commit":"1064667592a01d58ea892b90b2d53de3da031073"}],"number":21491}
buildkite-pr-comment-->
author_association: CONTRIBUTOR
user: login: elastic-sonarqube[bot]
created_at: 2025-05-27T11:12:11Z
body: ## [![Quality Gate passed](https://raw.githubusercontent.com/SonarSource/sonarqube-static-resources/master/v104/checks/QualityGateBadge/passed-16px.png 'Quality Gate passed')](https://sonar.elastic.dev/dashboard?id=elastic_elastic-agent_AYluowg0xMq8P7b4moiZ&pullRequest=8241) **Quality Gate passed**  
Issues  
[![](https://raw.githubusercontent.com/SonarSource/sonarqube-static-resources/master/v104/common/passed-16px.png '')](https://sonar.elastic.dev/project/issues?id=elastic_elastic-agent_AYluowg0xMq8P7b4moiZ&pullRequest=8241&resolved=false) [1 New issue](https://sonar.elastic.dev/project/issues?id=elastic_elastic-agent_AYluowg0xMq8P7b4moiZ&pullRequest=8241&resolved=false)  
[![](https://raw.githubusercontent.com/SonarSource/sonarqube-static-resources/master/v104/common/fixed-16px.png '')](https://sonar.elastic.dev/project/issues?id=elastic_elastic-agent_AYluowg0xMq8P7b4moiZ&fixedInPullRequest=8241) [0 Fixed issues](https://sonar.elastic.dev/project/issues?id=elastic_elastic-agent_AYluowg0xMq8P7b4moiZ&fixedInPullRequest=8241)  
[![](https://raw.githubusercontent.com/SonarSource/sonarqube-static-resources/master/v104/common/accepted-16px.png '')](https://sonar.elastic.dev/project/issues?id=elastic_elastic-agent_AYluowg0xMq8P7b4moiZ&pullRequest=8241&issueStatuses=ACCEPTED) [0 Accepted issues](https://sonar.elastic.dev/project/issues?id=elastic_elastic-agent_AYluowg0xMq8P7b4moiZ&pullRequest=8241&issueStatuses=ACCEPTED)

Measures  
[![](https://raw.githubusercontent.com/SonarSource/sonarqube-static-resources/master/v104/common/passed-16px.png '')](https://sonar.elastic.dev/security_hotspots?id=elastic_elastic-agent_AYluowg0xMq8P7b4moiZ&pullRequest=8241) [0 Security Hotspots](https://sonar.elastic.dev/security_hotspots?id=elastic_elastic-agent_AYluowg0xMq8P7b4moiZ&pullRequest=8241)  
[![](https://raw.githubusercontent.com/SonarSource/sonarqube-static-resources/master/v104/common/passed-16px.png '')](https://sonar.elastic.dev/component_measures?id=elastic_elastic-agent_AYluowg0xMq8P7b4moiZ&pullRequest=8241&metric=new_coverage&view=list) [100.0% Coverage on New Code](https://sonar.elastic.dev/component_measures?id=elastic_elastic-agent_AYluowg0xMq8P7b4moiZ&pullRequest=8241&metric=new_coverage&view=list)  
[![](https://raw.githubusercontent.com/SonarSource/sonarqube-static-resources/master/v104/common/passed-16px.png '')](https://sonar.elastic.dev/component_measures?id=elastic_elastic-agent_AYluowg0xMq8P7b4moiZ&pullRequest=8241&metric=new_duplicated_lines_density&view=list) [0.0% Duplication on New Code](https://sonar.elastic.dev/component_measures?id=elastic_elastic-agent_AYluowg0xMq8P7b4moiZ&pullRequest=8241&metric=new_duplicated_lines_density&view=list)  
  
[See analysis details on SonarQube](https://sonar.elastic.dev/dashboard?id=elastic_elastic-agent_AYluowg0xMq8P7b4moiZ&pullRequest=8241)


author_association: NONE
user: login: cmacknz
created_at: 2025-05-27T20:21:47Z
body: I tried this locally and I can confirm the http/metrics stats input is just duplicated for the agent process, it only contains the CPU metrics.

It also looks like this works when I turn it on, I no longer see the http module writing the stats metricset.
author_association: MEMBER