Repository: Azure/perf-tests
Pull request #26: WireGuard Metrics Prometheus Collector
Author: login: MikeZappa87
Created: 2025-03-24T18:06:45Z | Merged: 2025-03-28T15:58:26Z

<!--  Thanks for sending a pull request!  Here are some tips for you:

1. If this is your first time, please read our contributor guidelines: https://git.k8s.io/community/contributors/guide/first-contribution.md#your-first-contribution and developer guide https://git.k8s.io/community/contributors/devel/development.md#development-guide
2. Please label this pull request according to what type of issue you are addressing, especially if this is a release targeted pull request. For reference on required PR/issue labels, read here:
https://git.k8s.io/community/contributors/devel/sig-release/release.md#issuepr-kind-label
3. Ensure you have added or ran the appropriate tests for your PR: https://git.k8s.io/community/contributors/devel/sig-testing/testing.md
4. If you want *faster* PR reviews, read how: https://git.k8s.io/community/contributors/guide/pull-requests.md#best-practices-for-faster-reviews
5. If the PR is unfinished, see how to mark it: https://git.k8s.io/community/contributors/guide/pull-requests.md#marking-unfinished-pull-requests
-->

#### What type of PR is this?

/kind feature

#### What this PR does / why we need it:

This PR is for the Cilium WireGuard effort. This collector exposes the peering counts which we will use to ensure that all the nodes have a peering with the other K8s worker nodes. 

#### Which issue(s) this PR fixes:
<!--
*Automatically closes linked issue when PR is merged.
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.
_If PR is about `failing-tests or flakes`, please post the related issues/tests in a comment and do not use `Fixes`_*
-->
Fixes #

#### Special notes for your reviewer:




Diff:
diff --git a/.github/workflows/docker-build-push-wireguard.yml b/.github/workflows/docker-build-push-wireguard.yml
new file mode 100644
index 0000000000..be75114c0a
--- /dev/null
+++ b/.github/workflows/docker-build-push-wireguard.yml
@@ -0,0 +1,41 @@
+name: Build and Push Docker Image for network/tools/wireguard-scale/collector
+
+on:
+  push:
+    paths:
+      - "network/tools/wireguard-scale/collector/**"
+    tags:
+      - "**"
+  
+jobs:
+  build-and-push:
+    permissions:
+      packages: write # Write permission is required to publish Docker images to GitHub Container Registry
+      contents: read
+    
+    runs-on: ubuntu-latest
+
+    steps:
+    - name: Checkout repository
+      uses: actions/checkout@v4
+
+    - name: Set up QEMU
+      uses: docker/setup-qemu-action@v3
+    
+    - name: Set up Docker Buildx
+      uses: docker/setup-buildx-action@v3
+
+    - name: Log in to GitHub Container Registry
+      uses: docker/login-action@v3
+      with:
+        registry: ghcr.io
+        username: ${{ github.actor }}
+        password: ${{ secrets.GITHUB_TOKEN }}
+
+    - name: Build and push Docker image
+      run: |
+        cd network/tools/wireguard-scale/collector
+        make all
+      env:
+        REPO_OWNER: ${{ github.repository_owner }}
+        IMAGE_TAG: ${{ github.ref_name }}
\ No newline at end of file
diff --git a/network/tools/wireguard-scale/collector/Dockerfile b/network/tools/wireguard-scale/collector/Dockerfile
new file mode 100644
index 0000000000..6165eea3c7
--- /dev/null
+++ b/network/tools/wireguard-scale/collector/Dockerfile
@@ -0,0 +1,24 @@
+# Use a minimal base image with Go support
+FROM golang:1.23 AS builder
+
+# Set working directory
+WORKDIR /app
+
+# Copy Go modules and install dependencies
+COPY go.mod go.sum ./
+RUN go mod download
+
+# Copy the application code
+COPY . .
+
+RUN apt update && apt install wireguard -y 
+
+# Build the WireGuard exporter binary
+RUN go build -o wg_exporter main.go
+
+# Expose the Prometheus metrics port
+EXPOSE 8080
+
+# Run the exporter
+CMD ["/app/wg_exporter"]
+ 
\ No newline at end of file
diff --git a/network/tools/wireguard-scale/collector/Makefile b/network/tools/wireguard-scale/collector/Makefile
new file mode 100644
index 0000000000..ac8d3b025a
--- /dev/null
+++ b/network/tools/wireguard-scale/collector/Makefile
@@ -0,0 +1,16 @@
+REGISTRY ?= acnpublic.azurecr.io/wireguard
+
+POD_CREATION_IMAGE_NAME ?= wgpromcollector
+POD_CREATION_VERSION ?= v0.0.5
+
+all: build push
+
+.PHONY: build
+build:
+	docker build --platform linux/amd64 -f ./Dockerfile -t $(REGISTRY)/$(POD_CREATION_IMAGE_NAME):$(POD_CREATION_VERSION) .
+
+.PHONY: push
+push: build
+	docker push "$(REGISTRY)/$(POD_CREATION_IMAGE_NAME):$(POD_CREATION_VERSION)"
+
+
diff --git a/network/tools/wireguard-scale/collector/README.md b/network/tools/wireguard-scale/collector/README.md
new file mode 100644
index 0000000000..761b9f9a8b
--- /dev/null
+++ b/network/tools/wireguard-scale/collector/README.md
@@ -0,0 +1,65 @@
+# WireGuard Prometheus Exporter
+
+## Overview
+This is a Prometheus exporter for WireGuard that collects and exposes metrics about WireGuard peers and allowed IPs. The exporter periodically polls the WireGuard interface and provides metrics that can be scraped by Prometheus.
+
+## Features
+- Exposes the total number of WireGuard peers.
+- Exposes the total number of allowed IPs across all peers.
+- Configurable polling interval and network interface.
+- Provides metrics over HTTP for Prometheus scraping.
+
+## Installation
+### Prerequisites
+- Go 1.18 or later
+- WireGuard installed and configured
+- Prometheus for scraping metrics
+
+### Build from Source
+```sh
+git clone git@github.com:Azure/perf-tests.git
+cd network/tools/wireguard-scale/collector
+go build -o wireguard-exporter
+```
+
+## Usage
+### Running the Exporter
+```sh
+./wireguard-exporter -ifname cilium_wg0 -address 0.0.0.0 -port 8080 -poll 60
+```
+
+### Command-line Flags
+| Flag         | Default       | Description                              |
+|-------------|-------------|------------------------------------------|
+| `-ifname`   | `cilium_wg0` | Name of the WireGuard interface         |
+| `-address`  | `0.0.0.0`    | Address to bind the HTTP server         |
+| `-port`     | `8080`       | Port to expose metrics                  |
+| `-poll`     | `60`         | Polling interval in seconds             |
+
+## Metrics Exposed
+The exporter exposes the following Prometheus metrics:
+
+| Metric Name                  | Type  | Description                                  |
+|------------------------------|------|----------------------------------------------|
+| `wireguard_peers_total`       | Gauge | Total number of WireGuard peers             |
+| `wireguard_allowed_ips_total` | Gauge | Total number of allowed IPs across all peers |
+
+### Accessing Metrics
+Once the exporter is running, metrics can be accessed at:
+```
+http://<address>:<port>/metrics
+```
+Example:
+```
+http://localhost:8080/metrics
+```
+
+## Example Prometheus Configuration
+To configure Prometheus to scrape metrics from this exporter, add the following job to your `prometheus.yml`:
+```yaml
+scrape_configs:
+  - job_name: 'wireguard_exporter'
+    static_configs:
+      - targets: ['localhost:8080']
+```
+
diff --git a/network/tools/wireguard-scale/collector/go.mod b/network/tools/wireguard-scale/collector/go.mod
new file mode 100644
index 0000000000..34ba4cdb49
--- /dev/null
+++ b/network/tools/wireguard-scale/collector/go.mod
@@ -0,0 +1,29 @@
+module github.com/mikezappa87/wg-scraper
+
+go 1.23.3
+
+require (
+	github.com/prometheus/client_golang v1.21.1
+	golang.zx2c4.com/wireguard/wgctrl v0.0.0-20241231184526-a9ab2273dd10
+)
+
+require (
+	github.com/beorn7/perks v1.0.1 // indirect
+	github.com/cespare/xxhash/v2 v2.3.0 // indirect
+	github.com/google/go-cmp v0.6.0 // indirect
+	github.com/josharian/native v1.1.0 // indirect
+	github.com/klauspost/compress v1.17.11 // indirect
+	github.com/mdlayher/genetlink v1.3.2 // indirect
+	github.com/mdlayher/netlink v1.7.2 // indirect
+	github.com/mdlayher/socket v0.5.1 // indirect
+	github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
+	github.com/prometheus/client_model v0.6.1 // indirect
+	github.com/prometheus/common v0.62.0 // indirect
+	github.com/prometheus/procfs v0.15.1 // indirect
+	golang.org/x/crypto v0.31.0 // indirect
+	golang.org/x/net v0.33.0 // indirect
+	golang.org/x/sync v0.10.0 // indirect
+	golang.org/x/sys v0.28.0 // indirect
+	golang.zx2c4.com/wireguard v0.0.0-20231211153847-12269c276173 // indirect
+	google.golang.org/protobuf v1.36.1 // indirect
+)
diff --git a/network/tools/wireguard-scale/collector/go.sum b/network/tools/wireguard-scale/collector/go.sum
new file mode 100644
index 0000000000..07d0d589e1
--- /dev/null
+++ b/network/tools/wireguard-scale/collector/go.sum
@@ -0,0 +1,52 @@
+github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
+github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
+github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
+github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
+github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
+github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
+github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
+github.com/josharian/native v1.1.0 h1:uuaP0hAbW7Y4l0ZRQ6C9zfb7Mg1mbFKry/xzDAfmtLA=
+github.com/josharian/native v1.1.0/go.mod h1:7X/raswPFr05uY3HiLlYeyQntB6OO7E/d2Cu7qoaN2w=
+github.com/klauspost/compress v1.17.11 h1:In6xLpyWOi1+C7tXUUWv2ot1QvBjxevKAaI6IXrJmUc=
+github.com/klauspost/compress v1.17.11/go.mod h1:pMDklpSncoRMuLFrf1W9Ss9KT+0rH90U12bZKk7uwG0=
+github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
+github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
+github.com/mdlayher/genetlink v1.3.2 h1:KdrNKe+CTu+IbZnm/GVUMXSqBBLqcGpRDa0xkQy56gw=
+github.com/mdlayher/genetlink v1.3.2/go.mod h1:tcC3pkCrPUGIKKsCsp0B3AdaaKuHtaxoJRz3cc+528o=
+github.com/mdlayher/netlink v1.7.2 h1:/UtM3ofJap7Vl4QWCPDGXY8d3GIY2UGSDbK+QWmY8/g=
+github.com/mdlayher/netlink v1.7.2/go.mod h1:xraEF7uJbxLhc5fpHL4cPe221LI2bdttWlU+ZGLfQSw=
+github.com/mdlayher/socket v0.5.1 h1:VZaqt6RkGkt2OE9l3GcC6nZkqD3xKeQLyfleW/uBcos=
+github.com/mdlayher/socket v0.5.1/go.mod h1:TjPLHI1UgwEv5J1B5q0zTZq12A/6H7nKmtTanQE37IQ=
+github.com/mikioh/ipaddr v0.0.0-20190404000644-d465c8ab6721 h1:RlZweED6sbSArvlE924+mUcZuXKLBHA35U7LN621Bws=
+github.com/mikioh/ipaddr v0.0.0-20190404000644-d465c8ab6721/go.mod h1:Ickgr2WtCLZ2MDGd4Gr0geeCH5HybhRJbonOgQpvSxc=
+github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
+github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
+github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
+github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/prometheus/client_golang v1.21.1 h1:DOvXXTqVzvkIewV/CDPFdejpMCGeMcbGCQ8YOmu+Ibk=
+github.com/prometheus/client_golang v1.21.1/go.mod h1:U9NM32ykUErtVBxdvD3zfi+EuFkkaBvMb09mIfe0Zgg=
+github.com/prometheus/client_model v0.6.1 h1:ZKSh/rekM+n3CeS952MLRAdFwIKqeY8b62p8ais2e9E=
+github.com/prometheus/client_model v0.6.1/go.mod h1:OrxVMOVHjw3lKMa8+x6HeMGkHMQyHDk9E3jmP2AmGiY=
+github.com/prometheus/common v0.62.0 h1:xasJaQlnWAeyHdUBeGjXmutelfJHWMRr+Fg4QszZ2Io=
+github.com/prometheus/common v0.62.0/go.mod h1:vyBcEuLSvWos9B1+CyL7JZ2up+uFzXhkqml0W5zIY1I=
+github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0learggepc=
+github.com/prometheus/procfs v0.15.1/go.mod h1:fB45yRUv8NstnjriLhBQLuOUt+WW4BsoGhij/e3PBqk=
+github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
+github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
+golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U=
+golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk=
+golang.org/x/net v0.33.0 h1:74SYHlV8BIgHIFC/LrYkOGIwL19eTYXQ5wc6TBuO36I=
+golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4=
+golang.org/x/sync v0.10.0 h1:3NQrjDixjgGwUOCaF8w2+VYHv0Ve/vGYSbdkTa98gmQ=
+golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
+golang.org/x/sys v0.28.0 h1:Fksou7UEQUWlKvIdsqzJmUmCX3cZuD2+P3XyyzwMhlA=
+golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
+golang.zx2c4.com/wireguard v0.0.0-20231211153847-12269c276173 h1:/jFs0duh4rdb8uIfPMv78iAJGcPKDeqAFnaLBropIC4=
+golang.zx2c4.com/wireguard v0.0.0-20231211153847-12269c276173/go.mod h1:tkCQ4FQXmpAgYVh++1cq16/dH4QJtmvpRv19DWGAHSA=
+golang.zx2c4.com/wireguard/wgctrl v0.0.0-20241231184526-a9ab2273dd10 h1:3GDAcqdIg1ozBNLgPy4SLT84nfcBjr6rhGtXYtrkWLU=
+golang.zx2c4.com/wireguard/wgctrl v0.0.0-20241231184526-a9ab2273dd10/go.mod h1:T97yPqesLiNrOYxkwmhMI0ZIlJDm+p0PMR8eRVeR5tQ=
+google.golang.org/protobuf v1.36.1 h1:yBPeRvTftaleIgM3PZ/WBIZ7XM/eEYAaEyCwvyjq/gk=
+google.golang.org/protobuf v1.36.1/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE=
+gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
+gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
diff --git a/network/tools/wireguard-scale/collector/main.go b/network/tools/wireguard-scale/collector/main.go
new file mode 100644
index 0000000000..fccb533aa9
--- /dev/null
+++ b/network/tools/wireguard-scale/collector/main.go
@@ -0,0 +1,96 @@
+package main
+
+import (
+	"flag"
+	"fmt"
+	"log"
+	"net/http"
+	"time"
+
+	"github.com/prometheus/client_golang/prometheus"
+	"github.com/prometheus/client_golang/prometheus/promhttp"
+	"golang.zx2c4.com/wireguard/wgctrl"
+)
+
+var (
+	wgPeersTotal = prometheus.NewGauge(prometheus.GaugeOpts{
+		Name: "wireguard_peers_total",
+		Help: "Total number of WireGuard peers",
+	})
+
+	wgAllowedIPsTotal = prometheus.NewGauge(prometheus.GaugeOpts{
+		Name: "wireguard_allowed_ips_total",
+		Help: "Total number of allowed IPs across all peers",
+	})
+
+	wgLastHandshake = prometheus.NewGauge(prometheus.GaugeOpts{
+		Name: "wireguard_last_handshake",
+		Help: "Verifies the handshake status of a peer",
+	})
+)
+
+// Fetch WireGuard metrics and update Prometheus metrics
+func collectMetrics(client *wgctrl.Client, ifname string) {
+	device, err := client.Device(ifname)
+	if err != nil {
+		log.Fatalf("Failed to fetch WireGuard devices: %v", err)
+	}
+
+	totalPeers := 0
+	totalAllowedIPs := 0
+
+	for _, peer := range device.Peers {
+		totalPeers++
+		totalAllowedIPs += len(peer.AllowedIPs)
+
+		duration := time.Since(peer.LastHandshakeTime)
+
+		if duration > 180*time.Second {
+			//Issue with handshake
+			wgLastHandshake.Set(0)
+		} else {
+			wgLastHandshake.Set(1)
+		}
+	}
+
+	wgPeersTotal.Set(float64(totalPeers))
+	wgAllowedIPsTotal.Set(float64(totalAllowedIPs))
+}
+
+func main() {
+	var port, poll int
+	var address, ifname string
+
+	flag.StringVar(&ifname, "ifname", "cilium_wg0", "name of the wireguard interface")
+	flag.StringVar(&address, "address", "0.0.0.0", "address for metrics endpoint")
+	flag.IntVar(&poll, "poll", 60, "polling interval in seconds")
+	flag.IntVar(&port, "port", 8080, "metrics port")
+	flag.Parse()
+
+	// Register Prometheus metrics
+	prometheus.MustRegister(wgPeersTotal)
+	prometheus.MustRegister(wgAllowedIPsTotal)
+	prometheus.MustRegister(wgLastHandshake)
+
+	// Start the metrics collection loop
+	go func() {
+		client, err := wgctrl.New()
+		if err != nil {
+			log.Fatalf("Failed to create WireGuard client: %v", err)
+		}
+		defer client.Close()
+
+		for {
+
+			fmt.Println("polling the wireguard interface")
+			collectMetrics(client, ifname)
+			time.Sleep(time.Duration(poll) * time.Second)
+		}
+	}()
+
+	// Serve Prometheus metrics
+	http.Handle("/metrics", promhttp.Handler())
+
+	fmt.Printf("WireGuard Prometheus Exporter running on :%d\n", port)
+	log.Fatal(http.ListenAndServe(fmt.Sprintf("%s:%d", address, port), nil))
+}


Reviews:
user: login: MikeZappa87
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-26T19:41:45Z
user: login: MikeZappa87
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-26T19:41:54Z
user: login: alyssa1303
body: Can we rename the PR to be more descriptive? Like "Implement prometheus exporter to collect wireguard metrics" or similar?
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-27T19:55:33Z
user: login: MikeZappa87
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-27T20:35:16Z
user: login: alyssa1303
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-03-27T21:46:00Z

Review comments:
diff_hunk: @@ -0,0 +1,14 @@
+apiVersion: v1
+kind: Service
path: clusterloader2/testing/wireguard/manifests/wg-svc.yaml
user: login: MikeZappa87
body: Move to telescope
created_at: 2025-03-26T19:41:45Z
author_association: NONE
diff_hunk: @@ -0,0 +1,32 @@
+apiVersion: apps/v1
+kind: DaemonSet
+metadata:
+  name: wireguard-exporter
+  labels:
+    app: wireguard-exporter
+  annotations:
path: clusterloader2/testing/wireguard/manifests/wireguard-ds.yaml
user: login: MikeZappa87
body: Move to telescope
created_at: 2025-03-26T19:41:54Z
author_association: NONE
diff_hunk: @@ -0,0 +1,39 @@
+name: Build and Push Docker Image for network/tools/wireguard-scale/collector
+
+on:
+  push:
+    tags:
+      - '**'
path: .github/workflows/docker-build-push-wireguard.yml
user: login: alyssa1303
body: Can we make sure this workflow only triggered when the wireguard-scale folder is touched. Example is here https://github.com/Azure/perf-tests/blob/main/.github/workflows/clusterloader2-release.yml#L5-L6
created_at: 2025-03-27T19:54:15Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,39 @@
+name: Build and Push Docker Image for network/tools/wireguard-scale/collector
+
+on:
+  push:
+    tags:
+      - '**'
path: .github/workflows/docker-build-push-wireguard.yml
user: login: MikeZappa87
body: I made that change
created_at: 2025-03-27T20:35:16Z
author_association: NONE

PR comments:
user: login: MikeZappa87
created_at: 2025-03-24T22:36:18Z
body: /assign @alyssa1303 
author_association: NONE
user: login: MikeZappa87
created_at: 2025-03-26T19:43:14Z
body: similar to https://github.com/Azure/perf-tests/pull/24/files
author_association: NONE