Repository: elastic/beats (Go)

Issue:
x-pack/packetbeat: System tests failure on Windows
## Failed Test

* **Test Name:** TestDevices

* **Link:** https://github.com/elastic/beats/blob/7b041ba327011d85471cfd37f1014f054ae43436/x-pack/packetbeat/tests/system/app_test.go#L55-L70

* **Branch:** main: https://github.com/elastic/beats/pull/40387

* **Artifact Link:** [beats-xpack-packetbeat_build_5706_windows-x-pack-slash-packetbeat-win-2022-system-tests.log](https://github.com/user-attachments/files/16588014/beats-xpack-packetbeat_build_5706_windows-x-pack-slash-packetbeat-win-2022-system-tests.log)

* **Notes:** Additional details about the test. e.g. theory as to failure cause
**NB!** tests should be enabled back, since those were muted while migrating from Jenkins to Buildkite. Please uncomment:
https://github.com/elastic/beats/blob/7b041ba327011d85471cfd37f1014f054ae43436/.buildkite/x-pack/pipeline.xpack.packetbeat.yml#L163
https://github.com/elastic/beats/blob/7b041ba327011d85471cfd37f1014f054ae43436/.buildkite/x-pack/pipeline.xpack.packetbeat.yml#L253

Buildkite builds: https://buildkite.com/elastic/beats-xpack-packetbeat/builds/5706#_


### Stack Trace
```
=== FAIL: x-pack/packetbeat/tests/system TestDevices (0.21s)
--
  | app_test.go:58: Output:
  | 0: \Device\NPF_{8058D1FD-C1F5-4A20-A848-0188AB321A97} (WAN Miniport (Network Monitor)) (Not assigned ip address)
  | 1: \Device\NPF_{4265A7ED-129C-497F-B1AC-A60DB44D0DC7} (WAN Miniport (IPv6)) (Not assigned ip address)
  | 2: \Device\NPF_{608080BA-7C74-441B-918A-AEA32DD991B4} (WAN Miniport (IP)) (Not assigned ip address)
  | 3: \Device\NPF_{5F3CCF6B-3314-4F07-9284-5AE0B33202A8} (Google VirtIO Ethernet Adapter) (10.224.0.241)
  | 4: \Device\NPF_Loopback (Adapter for loopback traffic capture) (::1 127.0.0.1)
  | PASS
  | coverage: 43.9% of statements in ./...
  | app_test.go:66: Expect interfaces:
  | [7:Ethernet 2:up\|broadcast\|multicast\|running 1:Loopback Pseudo-Interface 1:up\|loopback\|multicast\|running]
  | app_test.go:69:
  | Error Trace:	C:/buildkite-agent/builds/bk-agent-prod-gcp-1723456393706198410/elastic/beats-xpack-packetbeat/x-pack/packetbeat/tests/system/app_test.go:69
  | Error:      	"0: \\Device\\NPF_{8058D1FD-C1F5-4A20-A848-0188AB321A97} (WAN Miniport (Network Monitor)) (Not assigned ip address)\n1: \\Device\\NPF_{4265A7ED-129C-497F-B1AC-A60DB44D0DC7} (WAN Miniport (IPv6)) (Not assigned ip address)\n2: \\Device\\NPF_{608080BA-7C74-441B-918A-AEA32DD991B4} (WAN Miniport (IP)) (Not assigned ip address)\n3: \\Device\\NPF_{5F3CCF6B-3314-4F07-9284-5AE0B33202A8} (Google VirtIO Ethernet Adapter) (10.224.0.241)\n4: \\Device\\NPF_Loopback (Adapter for loopback traffic capture) (::1 127.0.0.1)\nPASS\ncoverage: 43.9% of statements in ./..." does not contain "Ethernet 2"
  | Test:       	TestDevices
  | app_test.go:69:
  | Error Trace:	C:/buildkite-agent/builds/bk-agent-prod-gcp-1723456393706198410/elastic/beats-xpack-packetbeat/x-pack/packetbeat/tests/system/app_test.go:69
  | Error:      	"0: \\Device\\NPF_{8058D1FD-C1F5-4A20-A848-0188AB321A97} (WAN Miniport (Network Monitor)) (Not assigned ip address)\n1: \\Device\\NPF_{4265A7ED-129C-497F-B1AC-A60DB44D0DC7} (WAN Miniport (IPv6)) (Not assigned ip address)\n2: \\Device\\NPF_{608080BA-7C74-441B-918A-AEA32DD991B4} (WAN Miniport (IP)) (Not assigned ip address)\n3: \\Device\\NPF_{5F3CCF6B-3314-4F07-9284-5AE0B33202A8} (Google VirtIO Ethernet Adapter) (10.224.0.241)\n4: \\Device\\NPF_Loopback (Adapter for loopback traffic capture) (::1 127.0.0.1)\nPASS\ncoverage: 43.9% of statements in ./..." does not contain "Loopback Pseudo-Interface 1"
  | Test:       	TestDevices
```




Discussion:
Pinging @elastic/sec-linux-platform (Team:Security-Linux Platform)
During the migration of beats-ci from Jenkins to Buildkite, a number of tests were failing consistently due to issues unrelated to the migration. Those tests were disabled to stabilize the CI, with the intent to revisit them post-migration. @oakrizan has reviewed them all in her draft PRs linked above in the description, and has opened tickets such as this one to highlight to the product teams the tests that are currently still disabled and could use some attention.
@pierrehilbert @jlind23 Can we prioritize this issue (re-enabling the test mentioned in this issue as it is currently skipped) in an upcoming sprint, please?  See https://github.com/elastic/beats/pull/41909#issuecomment-2541253385 for the rationale.
@ycombinator looks like this is owned by @elastic/sec-linux-platform.
@nfritts is there anyone on your end able to pick this up?
@jlind23 I'm not sure.  I'm doing some research to see what all is required.  Even if we picked it up though, I think having this fixed by the next feature freeze is unlikely looking at the PTO calendar.  
> I think having this fixed by the next feature freeze is unlikely looking at the PTO calendar.

It has been disabled for quite some time so I don't think we need to rush that hard to get it back on track but the sooner the better.



Patch:
diff --git a/.buildkite/scripts/gcp_auth.ps1 b/.buildkite/scripts/gcp_auth.ps1
new file mode 100644
index 000000000000..2ba5982a3e20
--- /dev/null
+++ b/.buildkite/scripts/gcp_auth.ps1
@@ -0,0 +1,20 @@
+Write-Host "~~~ Authenticating GCP"
+# Secrets must be redacted
+# https://buildkite.com/docs/pipelines/managing-log-output#redacted-environment-variables
+
+$privateCIGCSServiceAccount = "kv/ci-shared/platform-ingest/gcp-platform-ingest-ci-service-account"
+$tempFileName = "google-cloud-credentials.json"
+$secretFileLocation = Join-Path $env:TEMP $tempFileName
+
+$serviceAccountJsonSecret = Retry-Command -ScriptBlock {
+    vault kv get -field=data -format=json $privateCIGCSServiceAccount | ConvertFrom-Json
+    if ( -not $? ) { throw "Error during vault kv get" }
+}
+
+New-Item -ItemType File -Path $secretFileLocation >$null
+$serviceAccountJsonPlaintextSecret = $serviceAccountJsonSecret.plaintext | ConvertTo-Json
+Set-Content -Path $secretFileLocation -Value $serviceAccountJsonPlaintextSecret
+if ( -not $?) { throw "Error retrieving the required field from the secret" }
+
+gcloud auth activate-service-account --key-file $secretFileLocation
+Remove-Item -Path $secretFileLocation -Force
\ No newline at end of file
diff --git a/.buildkite/scripts/gcp_auth.sh b/.buildkite/scripts/gcp_auth.sh
deleted file mode 100755
index 5b92f943980e..000000000000
--- a/.buildkite/scripts/gcp_auth.sh
+++ /dev/null
@@ -1,14 +0,0 @@
-#!/usr/bin/env bash
-
-set -euo pipefail
-
-echo "~~~ Authenticating GCP"
-# Secrets must be redacted
-# https://buildkite.com/docs/pipelines/managing-log-output#redacted-environment-variables
-PRIVATE_CI_GCS_CREDENTIALS_PATH="kv/ci-shared/platform-ingest/gcp-platform-ingest-ci-service-account"
-PRIVATE_CI_GCS_CREDENTIALS_SECRET=$(vault kv get -field plaintext -format=json ${PRIVATE_CI_GCS_CREDENTIALS_PATH})
-export PRIVATE_CI_GCS_CREDENTIALS_SECRET
-echo "${PRIVATE_CI_GCS_CREDENTIALS_SECRET}" > ./gcp.json
-GOOGLE_APPLICATION_CREDENTIALS=$(realpath ./gcp.json)
-export GOOGLE_APPLICATION_CREDENTIALS
-gcloud auth activate-service-account --key-file="${GOOGLE_APPLICATION_CREDENTIALS}"
\ No newline at end of file
diff --git a/.buildkite/x-pack/pipeline.xpack.packetbeat.yml b/.buildkite/x-pack/pipeline.xpack.packetbeat.yml
index 5b3625e0fc92..1620bc2320ae 100644
--- a/.buildkite/x-pack/pipeline.xpack.packetbeat.yml
+++ b/.buildkite/x-pack/pipeline.xpack.packetbeat.yml
@@ -207,9 +207,8 @@ steps:
 
       - label: ":windows: x-pack/packetbeat: Win 2022 System Tests"
         key: "mandatory-win-2022-system-tests"
-        skip: "skipping due to elastic/beats#38142"
         command: |
-          source .buildkite/scripts/gcp_auth.sh
+          .buildkite/scripts/gcp_auth.ps1
           Set-Location -Path x-pack/packetbeat
           mage systemTest
         retry:
@@ -322,9 +321,8 @@ steps:
 
       - label: ":windows: x-pack/packetbeat: Win 10 System Tests"
         key: "extended-win-10-system-tests"
-        skip: "skipping due to elastic/beats#38142"
         command: |
-          source .buildkite/scripts/gcp_auth.sh
+          .buildkite/scripts/gcp_auth.ps1
           Set-Location -Path x-pack/packetbeat
           mage systemTest
         retry:
diff --git a/x-pack/packetbeat/magefile.go b/x-pack/packetbeat/magefile.go
index 5e938cbbda72..220c430e5d84 100644
--- a/x-pack/packetbeat/magefile.go
+++ b/x-pack/packetbeat/magefile.go
@@ -137,12 +137,7 @@ func TestPackages() error {
 }
 
 func SystemTest(ctx context.Context) error {
-	// Buildkite (CI) images have preinstalled npcap
-	if os.Getenv("CI") == "true" {
-		mg.SerialDeps(devtools.BuildSystemTestBinary)
-	} else {
-		mg.SerialDeps(xpacketbeat.GetNpcapInstaller, devtools.BuildSystemTestBinary)
-	}
+	mg.SerialDeps(xpacketbeat.GetNpcapInstaller, devtools.BuildSystemTestBinary)
 
 	args := devtools.DefaultGoTestIntegrationArgs()
 	args.Packages = []string{"./tests/system/..."}


Test patch:
diff --git a/x-pack/packetbeat/tests/system/app_test.go b/x-pack/packetbeat/tests/system/app_test.go
index 214cd9c2369a..d298b9157374 100644
--- a/x-pack/packetbeat/tests/system/app_test.go
+++ b/x-pack/packetbeat/tests/system/app_test.go
@@ -24,7 +24,7 @@ import (
 )
 
 // Keep in sync with NpcapVersion in magefile.go.
-const NpcapVersion = "1.79"
+const NpcapVersion = "1.80"
 
 func TestWindowsNpcapInstaller(t *testing.T) {
 	if runtime.GOOS != "windows" {
@@ -65,7 +65,26 @@ func TestDevices(t *testing.T) {
 	}
 	t.Log("Expect interfaces:\n", expected)
 
+ifcsLoop:
 	for _, ifc := range ifcs {
-		assert.Contains(t, stdout, ifc.Name)
+		if strings.Contains(stdout, ifc.Name) {
+			continue ifcsLoop
+		}
+		addrs, err := ifc.Addrs()
+		assert.NoError(t, err)
+		maddrs, err := ifc.MulticastAddrs()
+		assert.NoError(t, err)
+		addrs = append(addrs, maddrs...)
+		for _, addr := range addrs {
+			s := addr.String()
+			// remove the network mask suffix
+			if idx := strings.Index(s, "/"); idx > -1 {
+				s = s[:idx]
+			}
+			if strings.Contains(stdout, s) {
+				continue ifcsLoop
+			}
+		}
+		t.Errorf("interface %q not found", ifc.Name)
 	}
 }