Repository: kiali/kiali

Issue:
Kiali uses a lot of CPU (in validations)
### Describe the bug

On a cluster with around 1600 namespaces, but a discovery selector that only selects around 380 of them and cluster wide view set to true, the CPU usage of the Kiali pod is permanently around 2500m. Although I have no reference to compare it with, this seems a bit excessive. This is even with any clients interacting with the UI.

![image](https://github.com/user-attachments/assets/7acaf779-3d2e-4291-8fa0-9586d63a27ab)

The graph shows a drop in usage. With cluster wide view disabled, CPU load is around 4000m.

### Expected Behavior

I'd like to understand if this a reasonable amount of CPU for our setup or if there is an issue with Kiali that causes it to use so much CPU.

### What are the steps to reproduce this bug?

Nothing in particular.
Cluster has around 1600 namespaces in total, but only 380 are selected with a discovery selector. Cluster wide view is enabled. Anonymous mode enabled.

### Environment

- **Kiali version:** 2.2.0
- **Istio version:** 1.23.2 asm 2
- **Kubernetes impl:** GKE
- **Kubernetes version:** 1.30
- **Other notable environmental factors:**

Issues/PRs for this Epic:
- [x] https://github.com/kiali/kiali/pull/8119
- [x] https://github.com/kiali/kiali/pull/8194
- [x] https://github.com/kiali/kiali/pull/8206
- [x] #8211 
- [x] #8210 
- [x] [pre-compute/faster validation keys for istio config objects](https://github.com/kiali/kiali/pull/8335)

Hints from issue comments:
@rolandkool: [profile.gz](https://github.com/user-attachments/files/18168277/profile.gz)

I've attached a 15 seconds CPU profile.@rolandkool: Deployed another instance on a cluster with similar amount of total namespaces, but only 25 are included in the discovery selector. CPU usage is permanently around 600m.@jshaughn: I think currently the Kiali Discovery Selector is gating the namespaces available to, or can be seen, by the user.  But I don't think it has the ability to reduce the number of k8s watches that are in place in cluster-wide, to manage caching. And so I think that is why you see more of a constant consumption based on the total namespaces.  

I think the workaround is to not use cluster-wide, despite the fact that it is often the more efficient option.  I think we will need to investigate whether we can apply Discovery Selectors to also minimize cache impact.

cc @jmazzitelli , @nrfox @rolandkool: @jshaughn thanks for your answer. I tried without cluster-wide access, but that makes the CPU usage even higher (see graph in description above. 4 cpu is without cluster wide access, 2.5 is with).
@jshaughn: OK @rolandkool , thanks for the update. I will add this to the backlog to see what we may be able to do.@frittentheke: @jshaughn  I am no expert in hunting down performance issues using pprof... but by peeking into the profile @rolandkool  provided, it seems that all of that CPU time is spent somewhere in https://github.com/kiali/kiali/blob/master/business/istio_validations.go:

![Image](https://github.com/user-attachments/assets/d5810e86-1bb9-4501-8107-83fdcfc95b05)

@rolandkool  do you happen to have more details on e.g. the reconcile time. Should be logged at `debug` level, see https://github.com/kiali/kiali/blob/cf8785962ffd76bfa861df35a3de8d102fdd2e5f/controller/validations.go#L111)@hhovsepy: This is the Istio configs validations cache reconciliation, which starts in a background every 10 seconds, does the validation of all Istio config objects and sets it into the cache to be reused by Kiali. The caching mechanism was applied for the Kiali pages loading time optimization.@jshaughn: Thanks, @frittentheke .  We will have to take a closer look at this, and see what we can do.@frittentheke: There also are all those `/metrics` exported by Kiali ... if @rolandkool was collecting them, that might also narrow down where Kiali was spending most of its time.@jshaughn: cc @nrfox 

@hhovsepy , I understand why we want to cache validation results; doing them on-demand creates UI lag for the user, and adding the new controller was a great UX perf enhancement. But I am questioning the frequency of recalculation.  In general, I don't think configuration frequently changes, and so every 10s seems quite aggressive. Of course it's very safe, and prevents obsolete validations, but it's having a noticeable effect on the Kiali pod (and possibly the cost of running Kiali).

I know we have talked about a watcher-based approach, that watches for config changes and reconciles validations only in response to a change.  Is this a feasible approach at this time?  Otherwise, I think we need to cache validations at Kiali startup and then only after a configurable period.  And the default period should likely be much greater than 10s.  Also, it doesn't have to be "all or nothing", we could use a timer and also add watchers incrementally.

@hhovsepy: Note for future work: We need to make sure that Istio configurations edited outside of the Kiali are also included into validations by a rerun or a watcher-based.@rolandkool: @frittentheke 
From the debug logging, I see this:

> 2025-02-04T15:17:54Z DBG [ValidationsReconciler] Started reconciling 
2025-02-04T15:19:34Z DBG [ValidationsReconciler] Finished reconciling in 100469ms
2025-02-04T15:19:34Z WRN [ValidationsReconciler] Reconcile took longer than the reconcile interval of [10s]. If this continues, validations will be increasingly stale. Please ensure your environment is healthy. If the issue still persists, please open an issue at www.github.com/kiali/kiali
2025-02-04T15:19:34Z DBG [ValidationsReconciler] Started reconciling 
2025-02-04T15:21:15Z DBG [ValidationsReconciler] Finished reconciling in 100436ms
2025-02-04T15:21:15Z WRN [ValidationsReconciler] Reconcile took longer than the reconcile interval of [10s]. If this continues, validations will be increasingly stale. Please ensure your environment is healthy. If the issue still persists, please open an issue at www.github.com/kiali/kiali
2025-02-04T15:21:15Z DBG [ValidationsReconciler] Started reconciling 

So it seems indeed that it is in a constant reconcile loop.
It takes about a 100 seconds to do a single reconcile (on the larger cluster), and when it has only 10s to do so, it starts a new one right away. On the smaller cluster, a reconcile takes 2 seconds. The load is around 500m on that cluster.

Feels like this 10s is a bit too aggressive. The configuration is not that volatile. It would be nice if it would only reconcile on an event or otherwise just allow a different reconcile interval. For example, 1 hour is probably sufficient for us. Most of our users are interested in the graph and dependencies between services.

Some possible interesting metrics (from a Kiali instance that has been running for about 7 minutes):
```
kiali_checker_processing_duration_seconds_sum{checker="checkers.AuthorizationPolicyChecker"} 4.570136205000005
kiali_checker_processing_duration_seconds_sum{checker="checkers.DestinationRulesChecker"} 1.3105037879999986
kiali_checker_processing_duration_seconds_sum{checker="checkers.GatewayChecker"} 0.0010832109999999995
kiali_checker_processing_duration_seconds_sum{checker="checkers.K8sGRPCRouteChecker"} 0.001104912
kiali_checker_processing_duration_seconds_sum{checker="checkers.K8sGatewayChecker"} 0.001285068999999999
kiali_checker_processing_duration_seconds_sum{checker="checkers.K8sHTTPRouteChecker"} 0.0007545829999999994
kiali_checker_processing_duration_seconds_sum{checker="checkers.K8sReferenceGrantChecker"} 0.000568709
kiali_checker_processing_duration_seconds_sum{checker="checkers.NoServiceChecker"} 43.686485464999976
kiali_checker_processing_duration_seconds_sum{checker="checkers.PeerAuthenticationChecker"} 0.0027431110000000047
kiali_checker_processing_duration_seconds_sum{checker="checkers.RequestAuthenticationChecker"} 33.374142512
kiali_checker_processing_duration_seconds_sum{checker="checkers.ServiceEntryChecker"} 2.1402931099999973
kiali_checker_processing_duration_seconds_sum{checker="checkers.SidecarChecker"} 66.75497811800005
kiali_checker_processing_duration_seconds_sum{checker="checkers.TelemetryChecker"} 0.0002582520000000002
kiali_checker_processing_duration_seconds_sum{checker="checkers.VirtualServiceChecker"} 31.964724545999957
kiali_checker_processing_duration_seconds_sum{checker="checkers.WasmPluginChecker"} 0.0002953839999999999
kiali_checker_processing_duration_seconds_sum{checker="checkers.WorkloadChecker"} 4.909717194000001
kiali_validation_processing_duration_seconds_sum{namespace="_all_",service="_all_"} 293.271228488
```
@jshaughn: Thanks @rolandkool.  Right, the fact that the reconcile is longer than the interval is not good.  Also, I'm a little curious about what we're doing that takes so long to calculate, we may want to do some detailed profiling of that validation logic.  I think event-based reconcile would be best, but otherwise we need a configurable intervale, with a much longer default.  Also, the next reconcile could depend on the length of the previous reconcile duration.@nrfox: > Some possible interesting metrics (from a Kiali instance that has been running for about 7 minutes):

This is great data, thank you!

> Feels like this 10s is a bit too aggressive. The configuration is not that volatile. It would be nice if it would only reconcile on an event or otherwise just allow a different reconcile interval. For example, 1 hour is probably sufficient for us. Most of our users are interested in the graph and dependencies between services.

Yes we should provide a config option to tune and/or disable this. The reason it was set so low to begin with was to ensure that when you make a config change, you get relatively fast feedback on whether or not that config change was valid. But there's also other tools you can use to validate your config, like `istioctl analyze`, so it's perfectly reasonable to set this higher and/or disable it altogether.

Moving to event based reconciling would help overall reconcile times after the initial validation. There's an underlying perf issue with validations that still needs to be addressed though. Moving validations to background processing was a way of improving the user experience but it did not deal with the underlying issue. Before we moved validations to being processed in the background, the validations requests would sometimes takes 1-2 minutes which would lead to significant slowdowns in the UI. It's now a better experience but obviously there's now the issue of Kiali constantly consuming resources if validations are slow. @hhovsepy: To summarize the solution directions, we can create the following sub-issues to resolve and implement:

- [ ]  Make the reconciliation interval configurable and/or provide an option to disable it.
- [ ] Implement event-based reconciliation triggered by configuration changes outside of Kiali.
- [ ] Optimize the validation process to reduce resource usage.@nrfox: Yeah and in terms of level of effort:

This one is very trivial
> Make the reconciliation interval configurable and/or provide an option to disable it.

This one is a medium-large effort
> Implement event-based reconciliation triggered by configuration changes outside of Kiali.

This one requires some investigation
> Optimize the validation process to reduce resource usage.

IMO we should prioritize 1 and 3.@jshaughn: I agree with the prioritization of 1 and 3.  Let's get the setting configurable asap and change the default to what, 60m?  Also, don't start the timer until after a reconcile completes.

Do we reconcile when Kiali itself is changing/updating config?

After that change is in, let's prioritize the 3rd task, try to understand if we can speed up the reconcile.@hhovsepy: 
> 
> Do we reconcile when Kiali itself is changing/updating config?
> 
Yeas each time Kiali edits a config, it is reconciled.

@nrfox: > Let's get the setting configurable asap and change the default to what, 60m?

60m is very large. Validations will be noticeably stale at that point. `60s`? We'll keep the warning so that if you start to go past that interval then you'll at least get a warning message telling you to adjust the config option.@rolandkool: Thanks, looking forward for the fix. That will allow me to roll out Kiali across more clusters. This high usage was holding me back.
I can help testing a pre release if you like.

@frittentheke: @nrfox @jshaughn could you possibly compile a few instructions on how to do e.g. pprof profiling that actually would help you narrow down this issue? Certainly it makes sense to also (!) tackle the underlying (potential) performance issues.
Years ago I also was involved in https://github.com/kiali/kiali/issues/4592 which was (in the end) quite easily found when looking at the profiling in the real environment.@nrfox: @frittentheke you can have a look at https://kiali.io/docs/configuration/debugging-kiali/#profiler for how to enable the profiler.@frittentheke: > [@frittentheke](https://github.com/frittentheke) you can have a look at https://kiali.io/docs/configuration/debugging-kiali/#profiler for how to enable the profiler.

@rolandkool As I don't see similar durations as you do, could you potentially run some more pprof ?@rolandkool: [profile.gz](https://github.com/user-attachments/files/18672578/profile.gz)

Here's another 15 seconds profile.

The profile command on the /debug/pprof page does not work, you need to manually add `&seconds=<nr>` to the URL.
@nrfox: @rolandkool as of v2.6 you can set `external_services.istio.validation_reconcile_interval` to have more control over how often Kiali will perform validations: https://kiali.io/docs/configuration/kialis.kiali.io/#.spec.external_services.istio.validation_reconcile_interval. You can see if that helps with the CPU usage.@jshaughn: I'm going to take a look, I'm unfamiliar with the validation logic, so it may take some time...@rolandkool: @nrfox thanks for implementing this setting. That definitely helps in our setup. I've set the reconcile interval to 15m, which is good enough for us for now. When not reconciling, the load drops to a more manageable 50-100m (depending on whether the UI is used or not).

Looking forward to a rework of this logic so that it only processes delta's instead of reconciling the whole config each time.
@frittentheke: @jshaughn we don't observe the issue in the scale as reported here, but if you need any more data (pprof) let me know. I suppose @rolandkool is just as willing to provide any debug data you require to figure out the suspected inefficiency here.@jshaughn: @frittentheke , thanks, I may accept the offer. At the moment, though, I am just familiarizing myself with the code and doing some initial re-arranging.  I'll let you know!@nrfox: If you look at the cpu profile a lot of time seems to be spent on garbage collection:

![Image](https://github.com/user-attachments/assets/897324ab-3438-46e1-ba34-d6acb0287b9e)

Reducing the number of allocations we do in validations should help with this.@nrfox: > Reducing the number of allocations we do in validations should help with this.

That being said, if this is going to take a large scale refactor, it would be nice to at least investigate this issue first: https://github.com/kiali/kiali/issues/4713 since the results of that may also require a refactor.@jshaughn: > Reducing the number of allocations we do in validations should help with this.

Is that using the old code or the new code?@frittentheke: > > Reducing the number of allocations we do in validations should help with this.
> 
> That being said, if this is going to take a large scale refactor, it would be nice to at least investigate this issue first: https://github.com/kiali/kiali/issues/4713 since the results of that may also require a refactor.

+1

I was always wondering why Kiali did his own tests and not built on top of istio analyse (even contribute missing consistency and housekeeping checks there).@nrfox: > Is that using the old code or the new code?

This was looking at the profile linked in this issue but I've seen similar results when running a local benchmark: https://github.com/kiali/kiali/pull/8208@nrfox: > I was always wondering why Kiali did his own tests and not built on top of istio analyse (even contribute missing consistency and housekeeping checks there).

iirc Kiali validations predate istio analyze but istio analyze has been around for awhile now@frittentheke: > > I was always wondering why Kiali did his own tests and not built on top of istio analyse (even contribute missing consistency and housekeeping checks there).
> 
> iirc Kiali validations predate istio analyze but istio analyze has been around for awhile now

thanks for https://github.com/kiali/kiali/issues/4713#issuecomment-2698846877 @nrfox @jshaughn: > If you look at the cpu profile a lot of time seems to be spent on garbage collection:

@nrfox The work for #8223 should help a lot with this.  See the pprof results that @hhovsepy has produced.@jshaughn: @frittentheke , @rolandkool , Kiali 2.7 will contain several changes that we hope will provide perf benefits, particularly for
validation.  It is scheduled for release Monday, Mar 17 (Happy St Patrick's Day!)  When/if you get the chance, please let us know if you are seeing improvement.@rolandkool: @jshaughn I've just deployed 2.7.1 on our clusters. With a reconcile interval of 15 minutes, I do see that each reconcile takes less time (up to a minute less). Still heavy on CPU though. @jshaughn: Thanks for the feedback, @rolandkool. I would have expected the first reconcile to be a bit faster, but assuming no config changes, I would have expected subsequent reconciles to be improved a fair amount.  This is because we added some logic to detect configuration changes.  We'll keep looking for ways to improve.@rolandkool: Here's a comparison.
On 2.6:

![Image](https://github.com/user-attachments/assets/b7ca4303-eafb-4db9-a94f-27b5e59b4dd3)

On 2.7.1:

![Image](https://github.com/user-attachments/assets/92a37545-68db-48bb-ab37-2266f1bd47aa)
@jshaughn: Thanks, @rolandkool .  What this shows me is that the CPU needed is still a spike, although the spike is generally for less time and the avg is down.  I expect that memory consumption is also reduced quite a bit during these periods.  Assuming the change detection code is working as expected (there would be trace level logs saying `"validations: no changes for cluster XXX, re-using"`), the change detection still consumes about 1/2 - 2/3 or the resources.  The savings are decent, although validation is still resource intensive; that's likely the nature of validation and all of the cross-checking that it performs.@jmazzitelli: Should Kial provide a on/off switch for validation entirely? For those environments that simply can't support it without killing resources, just have people turn it off entirely.@jshaughn: > Should Kial provide a on/off switch for validation entirely? 

I don't think so. Validations are a valuable tool and the reconcile interval is configurable.  But, if requested we could do something like have a reconcile interval <= 0 be a disable.@jshaughn: This round of validation perf enhancements is complete. It can still be intensive but we've made significant reductions to CPU and memory. We will also allow disabling validation in https://github.com/kiali/kiali/issues/8317.

Solution patch:
diff --git a/.github/workflows/build-backend.yml b/.github/workflows/build-backend.yml
index c48b2fc2e..decaf3059 100644
--- a/.github/workflows/build-backend.yml
+++ b/.github/workflows/build-backend.yml
@@ -37,7 +37,7 @@ jobs:
         # The -race flag requires CGO_ENABLED
         CGO_ENABLED: ${{ github.base_ref && '1' }}
 
-    - name: Test backend
+    - name: Run backend unit tests
       run: make -e GO_TEST_FLAGS="-race" test
 
     - name: Upload go binary
diff --git a/.github/workflows/integration-tests-backend.yml b/.github/workflows/integration-tests-backend.yml
index bdc86b8c1..5b9d7668b 100644
--- a/.github/workflows/integration-tests-backend.yml
+++ b/.github/workflows/integration-tests-backend.yml
@@ -52,16 +52,20 @@ jobs:
         path: frontend/build
 
     - name: Run backend integration tests
+      id: intTests
       run: hack/run-integration-tests.sh --test-suite backend $(if [ -n "${{ inputs.istio_version }}" ]; then echo "--istio-version ${{ inputs.istio_version }}"; fi)
 
+    - name: Run backend controller integration tests
+      run: make test-integration-controller
+
     - name: Get debug info when integration tests fail
-      if: failure()
+      if: ${{ failure() && steps.intTests.conclusion == 'failure' }}
       run: |
         mkdir debug-output
         hack/ci-get-debug-info.sh --output-directory debug-output
 
     - name: Upload debug info artifact
-      if: failure()
+      if: ${{ failure() && steps.intTests.conclusion == 'failure' }}
       uses: actions/upload-artifact@v4
       with:
         name: debug-info-${{ github.job }}
diff --git a/business/workloads.go b/business/workloads.go
index 7b37f63e6..5e01138fd 100644
--- a/business/workloads.go
+++ b/business/workloads.go
@@ -114,6 +114,7 @@ type LogEntry struct {
 }
 
 type filterOpts struct {
+	app    regexp.Regexp
 	destWk regexp.Regexp
 	destNs regexp.Regexp
 	srcWk  regexp.Regexp
@@ -817,7 +818,8 @@ func (in *WorkloadService) fetchWorkloadsFromCluster(ctx context.Context, cluste
 		return nil, err
 	}
 
-	userClient, ok := in.userClients[cluster]
+	// we've already established the user has access to the namespace; use SA client to obtain namespace resource info
+	client, ok := in.kialiSAClients[cluster]
 	if !ok {
 		return nil, fmt.Errorf("Cluster [%s] is not found or is not accessible for Kiali", cluster)
 	}
@@ -871,7 +873,7 @@ func (in *WorkloadService) fetchWorkloadsFromCluster(ctx context.Context, cluste
 		var err error
 		if in.isWorkloadIncluded(kubernetes.ReplicationControllerType) {
 			// No Cache for ReplicationControllers
-			repcon, err = userClient.GetReplicationControllers(namespace)
+			repcon, err = client.GetReplicationControllers(namespace)
 			if err != nil {
 				log.Errorf("Error fetching GetReplicationControllers per namespace %s: %s", namespace, err)
 				errChan <- err
@@ -884,9 +886,9 @@ func (in *WorkloadService) fetchWorkloadsFromCluster(ctx context.Context, cluste
 		defer wg.Done()
 
 		var err error
-		if userClient.IsOpenShift() && in.isWorkloadIncluded(kubernetes.DeploymentConfigType) {
+		if client.IsOpenShift() && in.isWorkloadIncluded(kubernetes.DeploymentConfigType) {
 			// No cache for DeploymentConfigs
-			depcon, err = userClient.GetDeploymentConfigs(ctx, namespace)
+			depcon, err = client.GetDeploymentConfigs(ctx, namespace)
 			if err != nil {
 				log.Errorf("Error fetching DeploymentConfigs per namespace %s: %s", namespace, err)
 				errChan <- err
@@ -915,7 +917,7 @@ func (in *WorkloadService) fetchWorkloadsFromCluster(ctx context.Context, cluste
 		var err error
 		if in.isWorkloadIncluded(kubernetes.CronJobType) {
 			// No cache for Cronjobs
-			conjbs, err = userClient.GetCronJobs(namespace)
+			conjbs, err = client.GetCronJobs(namespace)
 			if err != nil {
 				log.Errorf("Error fetching CronJobs per namespace %s: %s", namespace, err)
 				errChan <- err
@@ -930,7 +932,7 @@ func (in *WorkloadService) fetchWorkloadsFromCluster(ctx context.Context, cluste
 		var err error
 		if in.isWorkloadIncluded(kubernetes.JobType) {
 			// No cache for Jobs
-			jbs, err = userClient.GetJobs(namespace)
+			jbs, err = client.GetJobs(namespace)
 			if err != nil {
 				log.Errorf("Error fetching Jobs per namespace %s: %s", namespace, err)
 				errChan <- err
@@ -1488,9 +1490,10 @@ func (in *WorkloadService) fetchWorkload(ctx context.Context, criteria WorkloadC
 		return nil, err
 	}
 
-	client, ok := in.userClients[criteria.Cluster]
+	// we've already established the user has access to the namespace; use SA client to obtain namespace resource info
+	client, ok := in.kialiSAClients[criteria.Cluster]
 	if !ok {
-		return nil, fmt.Errorf("no user client for cluster [%s]", criteria.Cluster)
+		return nil, fmt.Errorf("no SA client for cluster [%s]", criteria.Cluster)
 	}
 
 	// Pods are always fetched for all workload types
@@ -2555,7 +2558,7 @@ func (in *WorkloadService) streamParsedLogs(cluster, namespace string, names []s
 	}
 
 	var engardeParser *parser.Parser
-	if opts.LogType == models.LogTypeProxy {
+	if opts.LogType == models.LogTypeProxy || opts.LogType == models.LogTypeWaypoint {
 		engardeParser = parser.New(parser.IstioProxyAccessLogsPattern)
 	}
 
@@ -2626,7 +2629,7 @@ func (in *WorkloadService) streamParsedLogs(cluster, namespace string, names []s
 			if opts.LogType == models.LogTypeZtunnel {
 				entry = parseZtunnelLine(line, name)
 			} else {
-				entry = parseLogLine(line, opts.LogType == models.LogTypeProxy, engardeParser)
+				entry = parseLogLine(line, opts.LogType == models.LogTypeProxy || opts.LogType == models.LogTypeWaypoint, engardeParser)
 			}
 
 			if entry == nil {
@@ -2637,6 +2640,10 @@ func (in *WorkloadService) streamParsedLogs(cluster, namespace string, names []s
 				continue
 			}
 
+			if opts.LogType == models.LogTypeWaypoint && !opts.filter.app.MatchString(entry.Message) {
+				continue
+			}
+
 			// If we are past the requested time window then stop processing
 			if startTime == nil {
 				startTime = &entry.OriginalTime
@@ -2707,7 +2714,8 @@ func (in *WorkloadService) streamParsedLogs(cluster, namespace string, names []s
 }
 
 // StreamPodLogs streams pod logs to an HTTP Response given the provided options
-func (in *WorkloadService) StreamPodLogs(cluster, namespace, name string, opts *LogOptions, w http.ResponseWriter) error {
+// The workload name is used to get the waypoint workloads when opts.LogType is "waypoint"
+func (in *WorkloadService) StreamPodLogs(ctx context.Context, cluster, namespace, workload, app, name string, opts *LogOptions, w http.ResponseWriter) error {
 	names := []string{}
 	if opts.LogType == models.LogTypeZtunnel {
 		// First, get ztunnel namespace and containers
@@ -2733,6 +2741,33 @@ func (in *WorkloadService) StreamPodLogs(cluster, namespace, name string, opts *
 		// They should be all in the same ns
 		return in.streamParsedLogs(cluster, pods[0].Namespace, names, opts, w)
 	}
+	if opts.LogType == models.LogTypeWaypoint {
+		wk, err := in.GetWorkload(ctx, WorkloadCriteria{Cluster: cluster, Namespace: namespace, WorkloadName: workload, IncludeServices: false})
+		if err != nil {
+			log.Errorf("Error when getting workload info: %s", err.Error())
+		} else {
+			if len(wk.WaypointWorkloads) > 0 {
+				// Waypoint filter by the app name
+				fs := filterOpts{
+					app: *regexp.MustCompile(app),
+				}
+				opts.filter = fs
+				// TODO: Get efective one
+				waypoint := wk.WaypointWorkloads[0]
+				waypointWk, errWaypoint := in.GetWorkload(ctx, WorkloadCriteria{Cluster: waypoint.Cluster, Namespace: waypoint.Namespace, WorkloadName: waypoint.Name, IncludeServices: false})
+				if errWaypoint != nil {
+					log.Errorf("Error when getting workload info: %s", err.Error())
+				} else {
+					for _, pod := range waypointWk.Pods {
+						names = append(names, pod.Name)
+					}
+					// This is needed for the K8S client
+					opts.PodLogOptions.Container = models.IstioProxy
+					return in.streamParsedLogs(cluster, waypoint.Namespace, names, opts, w)
+				}
+			}
+		}
+	}
 	names = append(names, name)
 	return in.streamParsedLogs(cluster, namespace, names, opts, w)
 }
diff --git a/config/config.go b/config/config.go
index 42accf5c7..4e6b994c2 100644
--- a/config/config.go
+++ b/config/config.go
@@ -16,6 +16,7 @@ import (
 	"github.com/kiali/kiali/config/dashboards"
 	"github.com/kiali/kiali/config/security"
 	"github.com/kiali/kiali/log"
+	"github.com/kiali/kiali/util"
 )
 
 // Files found in /kiali-override-secrets that override the ConfigMap yaml values
@@ -318,6 +319,9 @@ type IstioConfig struct {
 	Registry                     *RegistryConfig `yaml:"registry,omitempty"`
 	RootNamespace                string          `yaml:"root_namespace,omitempty"`
 	UrlServiceVersion            string          `yaml:"url_service_version"`
+	// ValidationReconcileInterval sets how often Kiali will validate Istio configuration.
+	// Validations cannot be disabled at the moment but you can set this to a long period of time.
+	ValidationReconcileInterval *time.Duration `yaml:"validation_reconcile_interval,omitempty"`
 }
 
 type ComponentStatuses struct {
@@ -748,6 +752,7 @@ func NewConfig() (c *Config) {
 				IstiodPollingIntervalSeconds:      20,
 				RootNamespace:                     "istio-system",
 				UrlServiceVersion:                 "",
+				ValidationReconcileInterval:       util.AsPtr(time.Minute),
 				GatewayAPIClasses:                 []GatewayAPIClass{},
 			},
 			Prometheus: PrometheusConfig{
diff --git a/controller/controller.go b/controller/controller.go
index af9c82e66..41e85ffe1 100644
--- a/controller/controller.go
+++ b/controller/controller.go
@@ -20,6 +20,7 @@ import (
 	metricsserver "sigs.k8s.io/controller-runtime/pkg/metrics/server"
 
 	"github.com/kiali/kiali/business"
+	"github.com/kiali/kiali/config"
 	"github.com/kiali/kiali/kubernetes"
 	"github.com/kiali/kiali/kubernetes/cache"
 	"github.com/kiali/kiali/log"
@@ -42,7 +43,7 @@ func NewScheme() (*runtime.Scheme, error) {
 }
 
 // Start creates and starts all the controllers. They'll get cancelled when the context is cancelled.
-func Start(ctx context.Context, cf kubernetes.ClientFactory, kialiCache cache.KialiCache, validationsService *business.IstioValidationsService) error {
+func Start(ctx context.Context, conf *config.Config, cf kubernetes.ClientFactory, kialiCache cache.KialiCache, validationsService *business.IstioValidationsService) error {
 	// TODO: Replace with kiali logging but if this isn't set some errors are thrown.
 	ctrl.SetLogger(zap.New())
 
@@ -72,7 +73,7 @@ func Start(ctx context.Context, cf kubernetes.ClientFactory, kialiCache cache.Ki
 		clusters = append(clusters, client.ClusterInfo().Name)
 	}
 
-	if err := NewValidationsController(ctx, clusters, kialiCache, validationsService, mgr, nil); err != nil {
+	if err := NewValidationsController(ctx, clusters, kialiCache, validationsService, mgr, conf.ExternalServices.Istio.ValidationReconcileInterval); err != nil {
 		return fmt.Errorf("error setting up ValidationsController: %s", err)
 	}
 
diff --git a/controller/validations.go b/controller/validations.go
index 49f25090d..d96a1a994 100644
--- a/controller/validations.go
+++ b/controller/validations.go
@@ -17,11 +17,8 @@ import (
 	"github.com/kiali/kiali/kubernetes/cache"
 	"github.com/kiali/kiali/log"
 	"github.com/kiali/kiali/models"
-	"github.com/kiali/kiali/util"
 )
 
-const defaultReconcileInterval = time.Second * 10
-
 // newValidationsController creates and starts a new controller for the validations.
 // It stops when the ctx is cancelled. It creates a single controller for all clusters.
 // A single controller is needed if cross clusters validations are performed in the future.
@@ -33,10 +30,7 @@ func NewValidationsController(
 	mgr ctrl.Manager,
 	reconcileInterval *time.Duration,
 ) error {
-	if reconcileInterval == nil {
-		reconcileInterval = util.AsPtr(defaultReconcileInterval)
-	}
-
+	log.Infof("Kiali will validate Istio configuration every: %s", *reconcileInterval)
 	reconciler := NewValidationsReconciler(clusters, kialiCache, validationsService, *reconcileInterval)
 
 	validationsController, err := controller.New("validations-controller", mgr, controller.Options{
@@ -108,10 +102,11 @@ func (r *ValidationsReconciler) Reconcile(ctx context.Context, req ctrl.Request)
 	startTime := time.Now()
 	defer func() {
 		totalReconcileTime := time.Since(startTime)
-		log.Debugf("[ValidationsReconciler] Finished reconciling in %dms", totalReconcileTime.Milliseconds())
+		log.Debugf("[ValidationsReconciler] Finished reconciling in %.2fs", totalReconcileTime.Seconds())
 		if totalReconcileTime > r.reconcileInterval {
 			const warningLog = "[ValidationsReconciler] Reconcile took longer than the reconcile interval of [%s]. " +
-				"If this continues, validations will be increasingly stale. Please ensure your environment is healthy. " +
+				"If this continues, validations will be increasingly stale. You can configure how often Kiali validates " +
+				"istio configuration by setting the 'external_services.istio.validation_reconcile_interval' config option. " +
 				"If the issue still persists, please open an issue at www.github.com/kiali/kiali"
 			log.Warningf(warningLog, r.reconcileInterval)
 		}
diff --git a/frontend/cypress/integration/common/istio_config.ts b/frontend/cypress/integration/common/istio_config.ts
index 80a61ab4a..b6ee31f37 100644
--- a/frontend/cypress/integration/common/istio_config.ts
+++ b/frontend/cypress/integration/common/istio_config.ts
@@ -613,6 +613,14 @@ Then('the AuthorizationPolicy should have a {string}', function (healthStatus: s
   );
 });
 
+const hexToRgb = (hex: string): string => {
+  const rValue = parseInt(hex.substring(0, 2), 16);
+  const gValue = parseInt(hex.substring(2, 4), 16);
+  const bValue = parseInt(hex.substring(4), 16);
+
+  return `rgb(${rValue}, ${gValue}, ${bValue})`;
+};
+
 function waitUntilConfigIsVisible(
   attempt: number,
   crdInstanceName: string,
@@ -626,11 +634,13 @@ function waitUntilConfigIsVisible(
   cy.request({ method: 'GET', url: `${Cypress.config('baseUrl')}/api/istio/config?refresh=0` });
   cy.get('[data-test="refresh-button"]').click();
   ensureKialiFinishedLoading();
+
   let found = false;
   cy.get('tr')
     .each($row => {
       const dataTestAttr = $row[0].attributes.getNamedItem('data-test');
       const hasNA = $row[0].innerText.includes('N/A');
+
       if (dataTestAttr !== null) {
         if (dataTestAttr.value === `VirtualItem_Ns${namespace}_${crdName}_${crdInstanceName}` && !hasNA) {
           // Check if the health status icon is correct
@@ -638,10 +648,16 @@ function waitUntilConfigIsVisible(
             .should('be.visible')
             .then(icon => {
               const colorVar = `--pf-v5-global--${healthStatus}-color--100`;
-              const color = getComputedStyle(icon[0]).getPropertyValue(colorVar);
-              if (color) {
-                found = true;
-              }
+              const statusColor = getComputedStyle(icon[0]).getPropertyValue(colorVar).replace('#', '');
+
+              cy.wrap(icon[0])
+                .invoke('css', 'color')
+                .then(iconColor => {
+                  // Convert the status color to RGB format to compare it with the icon color
+                  if (iconColor.toString() === hexToRgb(statusColor)) {
+                    found = true;
+                  }
+                });
             });
         }
       }
diff --git a/frontend/cypress/integration/common/mesh.ts b/frontend/cypress/integration/common/mesh.ts
index e950b7d93..366624dd1 100644
--- a/frontend/cypress/integration/common/mesh.ts
+++ b/frontend/cypress/integration/common/mesh.ts
@@ -45,7 +45,27 @@ When('user selects mesh node with label {string}', (label: string) => {
       assert.isTrue(controller.hasGraph());
 
       const { nodes } = elems(controller);
-      const node = nodes.find(n => n.getLabel() === label);
+      const node = nodes.find(n => n.getLabel().toLowerCase() === label.toLowerCase());
+      assert.exists(node);
+
+      const setSelectedIds = state.meshRefs.setSelectedIds as (values: string[]) => void;
+      setSelectedIds([node!.getId()]);
+    });
+});
+
+When('user selects tracing mesh node', () => {
+  cy.waitForReact();
+  cy.get('#loading_kiali_spinner').should('not.exist');
+  cy.getReact('MeshPageComponent', { state: { isReady: true } })
+    .should('have.length', 1)
+    .then($graph => {
+      const { state } = $graph[0];
+
+      const controller = state.meshRefs.getController() as Visualization;
+      assert.isTrue(controller.hasGraph());
+
+      const { nodes } = elems(controller);
+      const node = nodes.find(n => n.getLabel().toLowerCase() === 'jaeger' || n.getLabel().toLowerCase() === 'tempo');
       assert.exists(node);
 
       const setSelectedIds = state.meshRefs.setSelectedIds as (values: string[]) => void;
@@ -130,17 +150,17 @@ Then('user sees expected mesh infra', () => {
       assert.isTrue(controller.hasGraph());
 
       const { nodes, edges } = elems(controller);
-      const nodeNames = nodes.map(n => n.getLabel());
-      const nodesLength = nodeNames.some(n => n === 'External Deployments') ? 9 : 8;
+      const nodeNames = nodes.map(n => n.getLabel().toLowerCase());
+      const minNodesLength = nodeNames.some(n => n === 'external deployments') ? 9 : 8;
 
-      assert.equal(nodes.length, nodesLength, 'Unexpected number of infra nodes');
-      assert.equal(edges.length, 5, 'Unexpected number of infra edges');
-      assert.isTrue(nodeNames.some(n => n === 'Data Plane'));
-      assert.isTrue(nodeNames.some(n => n === 'Grafana'));
+      assert.isAtLeast(nodes.length, minNodesLength, 'Unexpected number of infra nodes');
+      assert.isAtLeast(edges.length, 5, 'Unexpected number of infra edges');
+      assert.isTrue(nodeNames.some(n => n === 'data plane'));
+      assert.isTrue(nodeNames.some(n => n === 'grafana'));
       assert.isTrue(nodeNames.some(n => n.startsWith('istiod')));
-      assert.isTrue(nodeNames.some(n => n === 'jaeger' || n === 'Tempo'));
+      assert.isTrue(nodeNames.some(n => n === 'jaeger' || n === 'tempo'));
       assert.isTrue(nodeNames.some(n => n === 'kiali'));
-      assert.isTrue(nodeNames.some(n => n === 'Prometheus'));
+      assert.isTrue(nodeNames.some(n => n === 'prometheus'));
     });
 });
 
@@ -220,3 +240,13 @@ Then('user sees {string} node side panel', (name: string) => {
       cy.contains(name);
     });
 });
+
+Then('user sees tracing node side panel', () => {
+  cy.waitForReact();
+  cy.get('#loading_kiali_spinner').should('not.exist');
+  cy.get('#target-panel-node')
+    .should('be.visible')
+    .within(() => {
+      cy.contains(new RegExp('jaeger|Jaeger|tempo|Tempo', 'g'));
+    });
+});
diff --git a/frontend/cypress/integration/featureFiles/mesh.feature b/frontend/cypress/integration/featureFiles/mesh.feature
index e8f9980c7..32082e6c6 100644
--- a/frontend/cypress/integration/featureFiles/mesh.feature
+++ b/frontend/cypress/integration/featureFiles/mesh.feature
@@ -33,9 +33,9 @@ Feature: Kiali Mesh page
     When user selects mesh node with label "Grafana"
     Then user sees "Grafana" node side panel
 
-  Scenario: Jaeger Infra
-    When user selects mesh node with label "jaeger"
-    Then user sees "jaeger" node side panel
+  Scenario: Tracing Infra
+    When user selects tracing mesh node
+    Then user sees tracing node side panel
 
   Scenario: Prometheus Infra
     When user selects mesh node with label "Prometheus"
diff --git a/frontend/cypress/integration/featureFiles/service_details.feature b/frontend/cypress/integration/featureFiles/service_details.feature
index 92d1a7a57..b16ba0023 100644
--- a/frontend/cypress/integration/featureFiles/service_details.feature
+++ b/frontend/cypress/integration/featureFiles/service_details.feature
@@ -59,7 +59,8 @@ Feature: Kiali Service Details page
     Then user sees trace details
 
   @bookinfo-app
-  @tracing-tracing
+  @tracing
+  @waypoint-tracing
   Scenario: See span info after selecting service span
     And user sees trace information
     When user selects a trace
diff --git a/frontend/cypress/integration/featureFiles/waypoint.feature b/frontend/cypress/integration/featureFiles/waypoint.feature
index b2cb39e97..e3756e5d0 100644
--- a/frontend/cypress/integration/featureFiles/waypoint.feature
+++ b/frontend/cypress/integration/featureFiles/waypoint.feature
@@ -10,12 +10,10 @@ Feature: Kiali Waypoint related features
   Background:
     Given user is at administrator perspective
 
-  @waypoint
   Scenario: [Setup] namespace is labeled with waypoint label
     Then "bookinfo" namespace is labeled with the waypoint label
     And the graph page has enough data
 
-  @waypoint
   Scenario: [Workload list] See the workload list of bookinfo with the correct info
     Given user is at the "workloads" list page
     When user selects the "bookinfo" namespace
@@ -27,7 +25,6 @@ Feature: Kiali Waypoint related features
     And the "Details" column on the "waypoint" row has the text "Waypoint Proxy"
     And the "Details" column on the "waypoint" row has a link ending in "bookinfo/istio/gateway.networking.k8s.io/v1/Gateway/waypoint"
 
-  @waypoint
   Scenario: [Workload details - productpage] The workload productpage is enrolled in waypoint
     Given user is at the details page for the "workload" "bookinfo/productpage-v1" located in the "" cluster
     Then user sees "ambient" badge
@@ -38,7 +35,6 @@ Feature: Kiali Waypoint related features
     And the user sees the L7 "waypoint" link
     And the link for the waypoint "waypoint" should redirect to a valid workload details
 
-  @waypoint
     Scenario: [Workload details - waypoint] The workload details for a waypoint are valid
     Given user is at the details page for the "workload" "bookinfo/waypoint" located in the "" cluster
     Then the user sees the "L7" badge
@@ -56,14 +52,12 @@ Feature: Kiali Waypoint related features
     Then user goes to the waypoint "Info" subtab
     And validates waypoint Info data
 
-  @waypoint
     Scenario: [Workload details - ztunnel] The workload details for a ztunnel are valid
     Given user is at the details page for the "workload" "istio-system/ztunnel" located in the "" cluster
     Then the user cannot see the "missing-sidecar" badge for "ztunnel" workload in "istio-system" namespace
     And the proxy status is "healthy"
     And the user validates the Ztunnel tab
 
-  @waypoint
   Scenario: [Traffic Graph] User sees ztunnel traffic
     Given user is at the "graphpf" page
     When user graphs "bookinfo" namespaces
@@ -72,7 +66,6 @@ Feature: Kiali Waypoint related features
     And user "enables" "ambientZtunnel" traffic option
     Then 7 edges appear in the graph
 
-  @waypoint
   Scenario: [Traffic Graph] User sees no Ambient traffic
     Given user is at the "graphpf" page
     When user graphs "bookinfo" namespaces
@@ -81,7 +74,6 @@ Feature: Kiali Waypoint related features
     And user "disables" "ambient" traffic option
     Then 2 edges appear in the graph
 
-  @waypoint
   Scenario: [Traffic Graph] User sees all Ambient traffic
     Given user is at the "graphpf" page
     When user graphs "bookinfo" namespaces
@@ -91,11 +83,9 @@ Feature: Kiali Waypoint related features
     And user "enables" "ambient" traffic option
     Then 16 edges appear in the graph
 
-  @waypoint
   Scenario: [Traffic Graph] User doesn't see waypoint proxy
     And the "waypoint" node "doesn't" exists
 
-  @waypoint
   Scenario: [Traffic Graph] User sees waypoint proxy
     When user opens display menu
     Then the display menu opens
@@ -105,7 +95,6 @@ Feature: Kiali Waypoint related features
     Then 16 edges appear in the graph
     And the "waypoint" node "does" exists
 
-  @waypoint
   Scenario: [Traffic Graph] User sees waypoint traffic
     Given user is at the "graphpf" page
     When user graphs "bookinfo" namespaces
@@ -114,13 +103,11 @@ Feature: Kiali Waypoint related features
     And user "enables" "ambientWaypoint" traffic option
     Then 11 edges appear in the graph
 
-  @waypoint
   Scenario: [Istio Config] Waypoint should not have validation errors
     Given user is at the "istio" page
     And user selects the "bookinfo" namespace
     Then the "K8sGateway" object in "bookinfo" namespace with "waypoint" name Istio Config is valid
 
-  @waypoint
   Scenario: [Overview] Namespace is labeled with the waypoint labels
     Given user is at the "overview" page
     When user clicks in the "LIST" view
diff --git a/frontend/package.json b/frontend/package.json
index 97052e2a9..3bc9713d4 100644
--- a/frontend/package.json
+++ b/frontend/package.json
@@ -38,21 +38,22 @@
     "build:dev": "sh -ac '. ./.env.upstream; npm run lint && npm run build:kiali'",
     "build:kiali": "npm run i18n && REACT_APP_VERSION=$npm_package_version REACT_APP_NAME=$npm_package_name REACT_APP_GIT_HASH=$(git rev-parse HEAD) GENERATE_SOURCEMAP=false EXTEND_ESLINT=true react-scripts build --profile",
     "build:prod": "sh -ac '. ./.env.downstream; npm run build:kiali'",
-    "cypress": "cypress open -e TAGS=\"not @multi-cluster and not @ambient\"",
-    "cypress:ambient": "cypress open -e TAGS=\"@ambient\" && cypress open -e TAGS=\"@waypoint\" && cypress open -e TAGS=\"@waypoint-tracing\"",
+    "cypress": "cypress open -e TAGS=\"not @multi-cluster and not @ambient and not @waypoint and (not @waypoint-tracing or @tracing)\"",
+    "cypress:ambient": "cypress open -e TAGS=\"@ambient\"",
+    "cypress:waypoint": "cypress open -e TAGS=\"@waypoint or @waypoint-tracing\"",
     "cypress:combine:reports": "jrm cypress/results/combined-report.xml \"cypress/results/*.xml\"",
     "cypress:delete:reports": "rm cypress/results/* || true",
-    "cypress:multi-cluster": "cypress open -e TAGS=\"not @crd-validation and @multi-cluster and not @multi-primary\"",
-    "cypress:multi-primary": "cypress open -e TAGS=\"not @crd-validation and @multi-cluster and @multi-primary\"",
+    "cypress:multi-cluster": "cypress open -e TAGS=\"@multi-cluster and not @multi-primary\"",
+    "cypress:multi-primary": "cypress open -e TAGS=\"@multi-cluster and @multi-primary\"",
     "cypress:perf": "cypress open --config-file cypress-perf.config.ts",
-    "cypress:run": "cypress run -e TAGS=\"@smoke\" && cypress run -e TAGS=\"not @crd-validation and not @multi-cluster and not @smoke and not @ambient and not @waypoint and not @cytoscape and not @waypoint-tracing\" && cypress run -e TAGS=\"@crd-validation and not @multi-cluster and not @smoke and not @ambient and not @waypoint and not @cytoscape and not @waypoint-tracing\"",
-    "cypress:run:ambient": "cypress run -e TAGS=\"@ambient and not @cytoscape\" && cypress run -e TAGS=\"@waypoint and not @cytoscape\" && cypress run -e TAGS=\"@waypoint-tracing and not @cytoscape\"",
-    "cypress:run:junit": "cypress run --reporter cypress-multi-reporters --reporter-options configFile=reporter-config.json -e TAGS=\"not @crd-validation and not @multi-cluster and not @ambient and not @waypoint and not @cytoscape\" && cypress run --reporter cypress-multi-reporters --reporter-options configFile=reporter-config.json -e TAGS=\"@crd-validation and not @multi-cluster and not @ambient and not @waypoint and not @cytoscape\"",
-    "cypress:run:multi-cluster": "cypress run -e TAGS=\"not @crd-validation and @multi-cluster and not @multi-primary and not @cytoscape\"",
-    "cypress:run:multi-primary": "cypress run -e TAGS=\"not @crd-validation and @multi-cluster and @multi-primary and not @cytoscape\"",
+    "cypress:run": "cypress run -e TAGS=\"@smoke\" && cypress run -e TAGS=\"not @crd-validation and not @multi-cluster and not @smoke and not @ambient and not @waypoint and not @cytoscape and (not @waypoint-tracing or @tracing)\" && cypress run -e TAGS=\"@crd-validation\"",
+    "cypress:run:ambient": "cypress run -e TAGS=\"@ambient and not @cytoscape\" && cypress run -e TAGS=\"@waypoint\" && cypress run -e TAGS=\"@waypoint-tracing\"",
+    "cypress:run:junit": "cypress run --reporter cypress-multi-reporters --reporter-options configFile=reporter-config.json -e TAGS=\"not @crd-validation and not @multi-cluster and not @ambient and not @waypoint and not @cytoscape and (not @waypoint-tracing or @tracing)\" && cypress run --reporter cypress-multi-reporters --reporter-options configFile=reporter-config.json -e TAGS=\"@crd-validation\"",
+    "cypress:run:multi-cluster": "cypress run -e TAGS=\"@multi-cluster and not @multi-primary and not @cytoscape\"",
+    "cypress:run:multi-primary": "cypress run -e TAGS=\"@multi-cluster and @multi-primary and not @cytoscape\"",
     "cypress:run:perf": "cypress run --config-file cypress-perf.config.ts",
     "cypress:run:selected": "cypress run -e TAGS=\"@selected\"",
-    "cypress:run:smoke": "cypress run -e TAGS=\"@smoke and not @cytoscape\"",
+    "cypress:run:smoke": "cypress run -e TAGS=\"@smoke\"",
     "cypress:run:test-group:junit": "cypress run --reporter cypress-multi-reporters --reporter-options configFile=reporter-config.json -e TAGS=\"${TEST_GROUP}\"",
     "cypress:run:tracing": "cypress run -e TAGS=\"@tracing and not @cytoscape\"",
     "cypress:selected": "cypress open -e TAGS=\"@selected\"",
diff --git a/frontend/src/app/History.tsx b/frontend/src/app/History.tsx
index 8cb70ffae..75f2ef6b5 100644
--- a/frontend/src/app/History.tsx
+++ b/frontend/src/app/History.tsx
@@ -95,6 +95,7 @@ export enum URLParam {
   SHOW_AVERAGE = 'avg',
   SHOW_SPANS = 'spans',
   SHOW_TRENDLINES = 'trendlines',
+  SHOW_WAYPOINT = 'waypoint',
   SHOW_ZTUNNEL = 'ztunnel',
   SORT = 'sort',
   TO = 'to',
diff --git a/frontend/src/components/Envoy/AccessLogModal.tsx b/frontend/src/components/Envoy/AccessLogModal.tsx
index c09c37757..24156602b 100644
--- a/frontend/src/components/Envoy/AccessLogModal.tsx
+++ b/frontend/src/components/Envoy/AccessLogModal.tsx
@@ -10,6 +10,7 @@ export interface AccessLogModalProps {
   accessLog: AccessLog;
   accessLogMessage: string;
   className?: string;
+  isWaypoint: boolean;
   isZtunnel: boolean;
   onClose?: () => void;
 }
@@ -835,12 +836,17 @@ export const AccessLogModal: React.FC<AccessLogModalProps> = (props: AccessLogMo
         return <>No documentation available</>;
     }
   };
-
   return (
     <Modal
       className={modalStyle}
       disableFocusTrap={true}
-      title={props.isZtunnel ? 'ztunnel Access Log Entry' : 'Envoy Access Log Entry'}
+      title={
+        props.isZtunnel
+          ? 'ztunnel Access Log Entry'
+          : props.isWaypoint
+          ? 'Waypoint Access log Entry'
+          : 'Envoy Access Log Entry'
+      }
       isOpen={true}
       onClose={props.onClose}
     >
diff --git a/frontend/src/components/Pf/PfBadges.tsx b/frontend/src/components/Pf/PfBadges.tsx
index 0ad853a85..94ba70992 100644
--- a/frontend/src/components/Pf/PfBadges.tsx
+++ b/frontend/src/components/Pf/PfBadges.tsx
@@ -88,11 +88,11 @@ export const PFBadges: { [key: string]: PFBadgeType } = Object.freeze({
   TraceStore: { badge: 'TS', tt: 'Trace Store' } as PFBadgeType,
   Unknown: { badge: 'U', tt: 'Unknown' } as PFBadgeType,
   VirtualService: { badge: 'VS', tt: 'Virtual Service' } as PFBadgeType,
-  Waypoint: { badge: 'L7', tt: 'Waypoint proxy' } as PFBadgeType,
+  Waypoint: { badge: 'L7', tt: 'Waypoint proxy', style: { backgroundColor: PFColors.Gold500 } } as PFBadgeType,
   Workload: { badge: 'W', tt: 'Workload', style: { backgroundColor: PFColors.Blue500 } } as PFBadgeType,
   WorkloadEntry: { badge: 'WE', tt: 'Workload Entry' } as PFBadgeType,
   WorkloadGroup: { badge: 'WG', tt: 'Workload Group' } as PFBadgeType,
-  Ztunnel: { badge: 'L4', tt: 'ztunnel' } as PFBadgeType
+  Ztunnel: { badge: 'L4', tt: 'ztunnel', style: { backgroundColor: PFColors.Gold300 } } as PFBadgeType
 });
 
 // This is styled for consistency with OpenShift Console.  See console: public/components/_resource.scss
diff --git a/frontend/src/components/Pf/PfColors.tsx b/frontend/src/components/Pf/PfColors.tsx
index 56e8767f7..35e12d4fd 100644
--- a/frontend/src/components/Pf/PfColors.tsx
+++ b/frontend/src/components/Pf/PfColors.tsx
@@ -22,13 +22,16 @@ export enum PFColors {
   Black900 = 'var(--pf-v5-global--palette--black-900)',
   Black1000 = 'var(--pf-v5-global--palette--black-1000)',
   Blue50 = 'var(--pf-v5-global--palette--blue-50)',
+  Blue100 = 'var(--pf-v5-global--palette--blue-100)',
   Blue200 = 'var(--pf-v5-global--palette--blue-200)',
   Blue300 = 'var(--pf-v5-global--palette--blue-300)',
   Blue400 = 'var(--pf-v5-global--palette--blue-400)',
   Blue500 = 'var(--pf-v5-global--palette--blue-500)',
   Blue600 = 'var(--pf-v5-global--palette--blue-600)',
   Cyan300 = 'var(--pf-v5-global--palette--cyan-300)',
+  Gold300 = 'var(--pf-v5-global--palette--gold-300)',
   Gold400 = 'var(--pf-v5-global--palette--gold-400)',
+  Gold500 = 'var(--pf-v5-global--palette--gold-500)',
   Green300 = 'var(--pf-v5-global--palette--green-300)',
   Green400 = 'var(--pf-v5-global--palette--green-400)',
   Green500 = 'var(--pf-v5-global--palette--green-500)',
diff --git a/frontend/src/pages/Graph/GraphToolbar/GraphTraffic.tsx b/frontend/src/pages/Graph/GraphToolbar/GraphTraffic.tsx
index 051a28b4d..c38b795db 100644
--- a/frontend/src/pages/Graph/GraphToolbar/GraphTraffic.tsx
+++ b/frontend/src/pages/Graph/GraphToolbar/GraphTraffic.tsx
@@ -127,7 +127,7 @@ const GraphTrafficComponent: React.FC<GraphTrafficProps> = (props: GraphTrafficP
       },
       {
         id: TrafficRate.AMBIENT_ZTUNNEL,
-        labelText: 'ZTunnel',
+        labelText: 'Ztunnel',
         isChecked: trafficRates.includes(TrafficRate.AMBIENT_ZTUNNEL),
         tooltip: (
           <div style={{ textAlign: 'left' }}>
diff --git a/frontend/src/pages/WorkloadDetails/WorkloadDetailsPage.tsx b/frontend/src/pages/WorkloadDetails/WorkloadDetailsPage.tsx
index 63467247d..1fd0531b8 100644
--- a/frontend/src/pages/WorkloadDetails/WorkloadDetailsPage.tsx
+++ b/frontend/src/pages/WorkloadDetails/WorkloadDetailsPage.tsx
@@ -184,11 +184,13 @@ class WorkloadDetailsPageComponent extends React.Component<WorkloadDetailsPagePr
           <Tab title="Logs" eventKey={2} key="Logs" data-test="workload-details-logs-tab">
             {hasPods ? (
               <WorkloadPodLogs
+                app={this.state.workload?.labels['app']}
                 lastRefreshAt={this.props.lastRefreshAt}
                 namespace={this.props.workloadId.namespace}
                 workload={this.props.workloadId.workload}
                 pods={this.state.workload!.pods}
                 cluster={this.state.cluster}
+                waypoints={this.state.workload!.waypointWorkloads}
               />
             ) : (
               <EmptyState variant={EmptyStateVariant.full}>
diff --git a/frontend/src/pages/WorkloadDetails/WorkloadPodLogs.tsx b/frontend/src/pages/WorkloadDetails/WorkloadPodLogs.tsx
index 9bd1e63f8..f2688e2d7 100644
--- a/frontend/src/pages/WorkloadDetails/WorkloadPodLogs.tsx
+++ b/frontend/src/pages/WorkloadDetails/WorkloadPodLogs.tsx
@@ -58,9 +58,11 @@ import { isParentKiosk, kioskContextMenuAction } from 'components/Kiosk/KioskAct
 import { TRACE_LIMIT_DEFAULT } from 'components/Metrics/TraceLimit';
 import { TraceSpansLimit } from 'components/Metrics/TraceSpansLimit';
 import { infoStyle } from 'styles/IconStyle';
+import { WaypointInfo } from '../../types/Workload';
 
-const appContainerColors = [PFColors.Blue300, PFColors.Green300, PFColors.Purple300, PFColors.Orange300];
-const proxyContainerColor = PFColors.Gold400;
+const appContainerColors = [PFColors.Blue200, PFColors.Blue300, PFColors.Blue400, PFColors.Blue100];
+const proxyContainerColor = PFColors.Gold300;
+const waypointContainerColor = PFColors.Gold500;
 const spanColor = PFColors.Cyan300;
 
 type ReduxProps = {
@@ -70,10 +72,12 @@ type ReduxProps = {
 };
 
 export type WorkloadPodLogsProps = ReduxProps & {
+  app?: string;
   cluster?: string;
   lastRefreshAt: TimeInMilliseconds;
   namespace: string;
   pods: Pod[];
+  waypoints?: WaypointInfo[];
   workload: string;
 };
 
@@ -116,6 +120,7 @@ interface WorkloadPodLogsState {
   showSpansLimit: number;
   showTimestamps: boolean;
   showToolbar: boolean;
+  showWaypoint: boolean;
   showZtunnel: boolean;
   useRegex: boolean;
 }
@@ -192,6 +197,11 @@ const checkboxStyle = kialiStyle({
   marginRight: '1rem'
 });
 
+const checkboxMarginStyle = kialiStyle({
+  marginLeft: '1rem',
+  marginRight: '1rem'
+});
+
 const logListStyle = kialiStyle({
   overflow: 'auto !important',
   paddingTop: '0.375rem',
@@ -260,6 +270,7 @@ export class WorkloadPodLogsComponent extends React.Component<WorkloadPodLogsPro
     const urlParams = new URLSearchParams(location.getSearch());
     const showSpans = urlParams.get(URLParam.SHOW_SPANS);
     const showSpansLimit = urlParams.get(URLParam.TRACING_LIMIT_TRACES);
+    const showWaypoint = urlParams.get(URLParam.SHOW_WAYPOINT);
     const showZtunnel = urlParams.get(URLParam.SHOW_ZTUNNEL);
 
     const defaultState = {
@@ -278,6 +289,7 @@ export class WorkloadPodLogsComponent extends React.Component<WorkloadPodLogsPro
       showLogValue: '',
       showSpans: showSpans === 'true',
       showSpansLimit: showSpansLimit ? parseInt(showSpansLimit) : TRACE_LIMIT_DEFAULT,
+      showWaypoint: showWaypoint === 'true',
       showZtunnel: showZtunnel === 'true',
       showTimestamps: false,
       showToolbar: true,
@@ -312,7 +324,6 @@ export class WorkloadPodLogsComponent extends React.Component<WorkloadPodLogsPro
   componentDidMount(): void {
     const screenFullAlias = screenfull as Screenfull;
     screenFullAlias.onchange(() => this.setState({ fullscreen: !this.state.fullscreen }));
-
     if (this.state.containerOptions) {
       const pod = this.props.pods[this.state.podValue!];
       this.fetchEntries(
@@ -321,6 +332,7 @@ export class WorkloadPodLogsComponent extends React.Component<WorkloadPodLogsPro
         this.state.containerOptions,
         this.state.showSpans,
         this.state.showSpansLimit,
+        this.state.showWaypoint,
         this.state.showZtunnel,
         this.state.maxLines,
         this.props.timeRange,
@@ -338,6 +350,7 @@ export class WorkloadPodLogsComponent extends React.Component<WorkloadPodLogsPro
     const showSpansChanged = prevState.showSpans !== this.state.showSpans;
     const showSpansLimitChanged = prevState.showSpansLimit !== this.state.showSpansLimit;
     const timeRangeChanged = !isEqualTimeRange(this.props.timeRange, prevProps.timeRange);
+    const showWaypoint = prevState.showWaypoint !== this.state.showWaypoint;
     const showZtunnel = prevState.showZtunnel !== this.state.showZtunnel;
 
     if (
@@ -347,6 +360,7 @@ export class WorkloadPodLogsComponent extends React.Component<WorkloadPodLogsPro
       showSpansChanged ||
       showSpansLimitChanged ||
       timeRangeChanged ||
+      showWaypoint ||
       showZtunnel
     ) {
       const pod = this.props.pods[this.state.podValue!];
@@ -356,6 +370,7 @@ export class WorkloadPodLogsComponent extends React.Component<WorkloadPodLogsPro
         newContainerOptions!,
         this.state.showSpans,
         this.state.showSpansLimit,
+        this.state.showWaypoint,
         this.state.showZtunnel,
         this.state.maxLines,
         this.props.timeRange,
@@ -551,7 +566,7 @@ export class WorkloadPodLogsComponent extends React.Component<WorkloadPodLogsPro
                   }
                   onChange={() => this.toggleSelected(c)}
                 />
-                {c.isAmbient && (
+                {c.isAmbient && i + 1 === this.state.containerOptions?.length && (
                   <>
                     <Checkbox
                       id={`ztunnel-${c.displayName}`}
@@ -579,6 +594,36 @@ export class WorkloadPodLogsComponent extends React.Component<WorkloadPodLogsPro
                     >
                       <KialiIcon.Info key={`al-i-ki`} className={checkInfoIcon} color={proxyContainerColor} />
                     </Tooltip>
+                    {this.props.waypoints && (
+                      <>
+                        <Checkbox
+                          id={`waypoint-${c.displayName}`}
+                          key={`waypoint-${i}`}
+                          className={checkboxMarginStyle}
+                          inputClassName={colorCheck(waypointContainerColor)}
+                          isChecked={this.state.showWaypoint}
+                          label={
+                            <span
+                              style={{
+                                color: waypointContainerColor,
+                                fontWeight: 'bold'
+                              }}
+                            >
+                              waypoint
+                            </span>
+                          }
+                          onChange={() => this.toggleWaypoint()}
+                        />
+                        <Tooltip
+                          key={`al-tt-tl`}
+                          position={TooltipPosition.auto}
+                          entryDelay={1000}
+                          content="A filtered - by app name - subset of log entries from the waypoint's (ambient node proxy) pod logs, relevant to the selected workload pod"
+                        >
+                          <KialiIcon.Info key={`al-i-ki`} className={checkInfoIcon} color={waypointContainerColor} />
+                        </Tooltip>
+                      </>
+                    )}
                   </>
                 )}
               </React.Fragment>
@@ -602,6 +647,14 @@ export class WorkloadPodLogsComponent extends React.Component<WorkloadPodLogsPro
     this.setState({ showZtunnel: !this.state.showZtunnel });
   };
 
+  private toggleWaypoint = (): void => {
+    const urlParams = new URLSearchParams(location.getSearch());
+    urlParams.set(URLParam.SHOW_WAYPOINT, String(!this.state.showWaypoint));
+    router.navigate(`${location.getPathname()}?${urlParams.toString()}`, { replace: true });
+
+    this.setState({ showWaypoint: !this.state.showWaypoint });
+  };
+
   private toggleTimeOptionsVisibility = (): void => {
     this.setState(prevState => ({ isTimeOptionsOpen: !prevState.isTimeOptionsOpen }));
   };
@@ -868,7 +921,8 @@ export class WorkloadPodLogsComponent extends React.Component<WorkloadPodLogsPro
           accessLog={v}
           accessLogMessage={k}
           onClose={() => this.removeAccessLogModal(k)}
-          isZtunnel={this.state.showZtunnel}
+          isZtunnel={this.state.showZtunnel && v.upstream_cluster.includes('spiffe')}
+          isWaypoint={this.state.showWaypoint && !v.upstream_cluster.includes('spiffe')}
         />
       );
     });
@@ -1087,10 +1141,8 @@ export class WorkloadPodLogsComponent extends React.Component<WorkloadPodLogsPro
       const name = c.name;
 
       const isAmbient = c.isAmbient;
-
       if (c.isProxy) {
         const proxyName = pod.name.includes('ztunnel') ? 'ztunnel' : 'sidecar-proxy';
-
         return {
           color: proxyContainerColor,
           displayName: proxyName,
@@ -1114,6 +1166,7 @@ export class WorkloadPodLogsComponent extends React.Component<WorkloadPodLogsPro
     containerOptions: ContainerOption[],
     showSpans: boolean,
     showSpansLimit: number,
+    showWaypoint: boolean,
     showZtunnel: boolean,
     maxLines: number,
     timeRange: TimeRange,
@@ -1146,10 +1199,24 @@ export class WorkloadPodLogsComponent extends React.Component<WorkloadPodLogsPro
       }
     }
 
+    if (showWaypoint) {
+      for (const c of containerOptions) {
+        if (c.isAmbient) {
+          const extracontainer = { ...c };
+          extracontainer.isAmbient = false;
+          extracontainer.color = waypointContainerColor;
+          extracontainer.displayName = 'waypoint';
+          extraContainers.push(extracontainer);
+        }
+      }
+    }
+
     const promises: Promise<ApiResponse<PodLogs | Span[]>>[] = selectedContainers.map(c => {
       return getPodLogs(
         namespace,
         podName,
+        this.props.workload,
+        this.props.app,
         c.name,
         maxLines,
         sinceTime,
@@ -1159,9 +1226,23 @@ export class WorkloadPodLogsComponent extends React.Component<WorkloadPodLogsPro
       );
     });
 
-    if (showZtunnel) {
+    if (showZtunnel || showWaypoint) {
       extraContainers.forEach(c => {
-        promises.push(getPodLogs(namespace, podName, c.name, maxLines, sinceTime, duration, LogType.ZTUNNEL, cluster));
+        const containerType = c.displayName === 'ztunnel' ? LogType.ZTUNNEL : LogType.WAYPOINT;
+        promises.push(
+          getPodLogs(
+            namespace,
+            podName,
+            this.props.workload,
+            this.props.app,
+            c.name,
+            maxLines,
+            sinceTime,
+            duration,
+            containerType,
+            cluster
+          )
+        );
       });
     }
 
diff --git a/frontend/src/services/Api.ts b/frontend/src/services/Api.ts
index ffe2dca44..9e0a35f8d 100644
--- a/frontend/src/services/Api.ts
+++ b/frontend/src/services/Api.ts
@@ -1068,6 +1068,8 @@ export const getPod = (namespace: string, name: string): Promise<ApiResponse<Pod
 export const getPodLogs = (
   namespace: string,
   name: string,
+  workload: string,
+  app?: string,
   container?: string,
   maxLines?: number,
   sinceTime?: number,
@@ -1101,6 +1103,14 @@ export const getPodLogs = (
     params.logType = logType;
   }
 
+  if (workload) {
+    params.workload = workload;
+  }
+
+  if (app) {
+    params.app = app;
+  }
+
   return newRequest<PodLogs>(HTTP_VERBS.GET, urls.podLogs(namespace, name), params, {});
 };
 
diff --git a/frontend/src/types/IstioObjects.ts b/frontend/src/types/IstioObjects.ts
index 1a3203b62..799e0fb10 100644
--- a/frontend/src/types/IstioObjects.ts
+++ b/frontend/src/types/IstioObjects.ts
@@ -278,11 +278,13 @@ export enum LogType {
 }
 
 export interface PodLogsQuery {
+  app?: string;
   container?: string;
   duration?: string;
   logType?: LogType;
   maxLines?: number;
   sinceTime?: number;
+  workload?: string;
 }
 
 export interface LogLevelQuery {
diff --git a/hack/istio/multicluster/deploy-kiali.sh b/hack/istio/multicluster/deploy-kiali.sh
index bc7fb18e9..ba2770e0e 100755
--- a/hack/istio/multicluster/deploy-kiali.sh
+++ b/hack/istio/multicluster/deploy-kiali.sh
@@ -182,6 +182,7 @@ deploy_kiali() {
     --set deployment.logger.log_level="debug"
     --set external_services.grafana.external_url="http://grafana.istio-system:3000"
     --set external_services.grafana.dashboards[0].name="Istio Mesh Dashboard"
+    --set external_services.istio.validation_reconcile_interval="5s"
     --set kiali_feature_flags.ui_defaults.graph.impl="pf"
     --set health_config.rate[0].kind="service"
     --set health_config.rate[0].name="y-server"
diff --git a/hack/setup-kind-in-ci.sh b/hack/setup-kind-in-ci.sh
index 86421e92b..48e9f4e81 100755
--- a/hack/setup-kind-in-ci.sh
+++ b/hack/setup-kind-in-ci.sh
@@ -248,6 +248,7 @@ EOF
     --set external_services.grafana.dashboards[0].name="Istio Mesh Dashboard" \
     --set external_services.tracing.enabled="true" \
     --set external_services.tracing.external_url="http://tracing.istio-system:16685/jaeger" \
+    --set external_services.istio.validation_reconcile_interval="5s" \
     --set health_config.rate[0].kind="service" \
     --set health_config.rate[0].name="y-server" \
     --set health_config.rate[0].namespace="alpha" \
@@ -325,6 +326,7 @@ setup_kind_tempo() {
     --set external_services.tracing.external_url="http://tempo-cr-query-frontend.tempo:3200" \
     --set external_services.tracing.internal_url="http://tempo-cr-query-frontend.tempo:3200" \
     --set external_services.tracing.use_grpc="false" \
+    --set external_services.istio.validation_reconcile_interval="5s" \
     --set health_config.rate[0].kind="service" \
     --set health_config.rate[0].name="y-server" \
     --set health_config.rate[0].namespace="alpha" \
diff --git a/handlers/workloads.go b/handlers/workloads.go
index 8d79f3661..391f73cd6 100644
--- a/handlers/workloads.go
+++ b/handlers/workloads.go
@@ -314,7 +314,7 @@ func PodLogs(w http.ResponseWriter, r *http.Request) {
 	}
 
 	// Fetch pod logs
-	err = business.Workload.StreamPodLogs(cluster, namespace, pod, opts, w)
+	err = business.Workload.StreamPodLogs(r.Context(), cluster, namespace, queryParams.Get("workload"), queryParams.Get("app"), pod, opts, w)
 	if err != nil {
 		handleErrorResponse(w, err)
 		return
diff --git a/kiali.go b/kiali.go
index f3f9c27ec..296224354 100644
--- a/kiali.go
+++ b/kiali.go
@@ -180,7 +180,7 @@ func main() {
 	if err != nil {
 		log.Fatalf("Error creating business layer: %s", err)
 	}
-	if err := controller.Start(ctx, clientFactory, cache, &layer.Validations); err != nil {
+	if err := controller.Start(ctx, cfg, clientFactory, cache, &layer.Validations); err != nil {
 		log.Fatalf("Error creating validations controller: %s", err)
 	}
 
diff --git a/make/Makefile.build.mk b/make/Makefile.build.mk
index ae404a0a4..3337564ca 100644
--- a/make/Makefile.build.mk
+++ b/make/Makefile.build.mk
@@ -80,11 +80,12 @@ test-integration-setup:
 	go install github.com/jstemmer/go-junit-report@latest
 
 ## test-integration: Run Integration test suite
-test-integration: test-integration-setup .ensure-envtest-bin-dir-exists
+test-integration: test-integration-setup
 	@echo Running Integration tests
 	cd tests/integration/tests && ${GO} test ${GO_TEST_FLAGS} -v -timeout 30m 2>&1 | tee >(go-junit-report > ../junit-rest-report.xml) ../int-test.log
 	@echo Test results can be found here: $$(ls -1 ${ROOTDIR}/tests/integration/junit-rest-report.xml)
 
+## test-integration-controller: Run controller integration test suite. These are not real e2e tests like the other integration tests.
 test-integration-controller: .ensure-envtest-bin-dir-exists
 	@echo Running controller integration tests
 	cd tests/integration/controller && ${GO} test -v



Test patch:
diff --git a/business/test_util.go b/business/test_util.go
index 50ad20342..5b13ff439 100644
--- a/business/test_util.go
+++ b/business/test_util.go
@@ -739,6 +739,36 @@ func FakePodSyncedWithDeployments() *core_v1.Pod {
 	}
 }
 
+func FakePodWithWaypointAndDeployments() *core_v1.Pod {
+	conf := config.NewConfig()
+
+	appLabel := conf.IstioLabels.AppLabelName
+	versionLabel := conf.IstioLabels.VersionLabelName
+	t1, _ := time.Parse(time.RFC822Z, "08 Mar 18 17:44 +0300")
+	controller := true
+	return &core_v1.Pod{
+		ObjectMeta: meta_v1.ObjectMeta{
+			Name:              "details-v1-3618568057-dnkjp",
+			Namespace:         "Namespace",
+			CreationTimestamp: meta_v1.NewTime(t1),
+			Labels:            map[string]string{appLabel: "details", versionLabel: "v1"},
+			OwnerReferences: []meta_v1.OwnerReference{{
+				Controller: &controller,
+				APIVersion: kubernetes.ReplicaSets.GroupVersion().String(),
+				Kind:       kubernetes.ReplicaSets.Kind,
+				Name:       "details-v1-3618568057",
+			}},
+			Annotations: kubetest.FakeIstioAnnotations(),
+		},
+		Spec: core_v1.PodSpec{
+			Containers: []core_v1.Container{
+				{Name: "details", Image: "whatever"},
+			},
+			InitContainers: []core_v1.Container{},
+		},
+	}
+}
+
 func FakePodLogsSyncedWithDeployments() *kubernetes.PodLogs {
 	return &kubernetes.PodLogs{
 		Logs: `2018-01-02T03:34:28+00:00 INFO #1 Log Message
@@ -764,6 +794,16 @@ func FakePodLogsZtunnel() *kubernetes.PodLogs {
 	}
 }
 
+func FakePodLogsWaypoint() *kubernetes.PodLogs {
+	content, err := readFile("../tests/data/logs/waypoint.log")
+	if err != nil {
+		log.Errorf("Error reading logs file: %s", err.Error())
+	}
+	return &kubernetes.PodLogs{
+		Logs: content,
+	}
+}
+
 func FakePodsSyncedWithDuplicated() []core_v1.Pod {
 	conf := config.NewConfig()
 
diff --git a/business/workloads_test.go b/business/workloads_test.go
index 15244717b..b4010f3b3 100644
--- a/business/workloads_test.go
+++ b/business/workloads_test.go
@@ -37,10 +37,10 @@ func setupWorkloadService(k8s kubernetes.ClientInterface, conf *config.Config) W
 	return NewWithBackends(k8sclients, k8sclients, prom, nil).Workload
 }
 
-func callStreamPodLogs(svc WorkloadService, namespace, podName string, opts *LogOptions) PodLog {
+func callStreamPodLogs(svc WorkloadService, namespace, workload, app, podName string, opts *LogOptions) PodLog {
 	w := httptest.NewRecorder()
 
-	_ = svc.StreamPodLogs(svc.config.KubernetesConfig.ClusterName, namespace, podName, opts, w)
+	_ = svc.StreamPodLogs(context.TODO(), svc.config.KubernetesConfig.ClusterName, namespace, workload, app, podName, opts, w)
 
 	response := w.Result()
 	body, _ := io.ReadAll(response.Body)
@@ -665,7 +665,7 @@ func TestGetPodLogs(t *testing.T) {
 
 	SetupBusinessLayer(t, k8s, *config.NewConfig())
 	svc := setupWorkloadService(k8s, config.NewConfig())
-	podLogs := callStreamPodLogs(svc, "Namespace", "details-v1-3618568057-dnkjp", &LogOptions{PodLogOptions: core_v1.PodLogOptions{Container: "details"}})
+	podLogs := callStreamPodLogs(svc, "Namespace", "details", "details", "details-v1-3618568057-dnkjp", &LogOptions{PodLogOptions: core_v1.PodLogOptions{Container: "details"}})
 
 	require.Equal(len(podLogs.Entries), 4)
 
@@ -704,7 +704,7 @@ func TestGetPodLogsMaxLines(t *testing.T) {
 
 	maxLines := 2
 	duration, _ := time.ParseDuration("6h")
-	podLogs := callStreamPodLogs(svc, "Namespace", "details-v1-3618568057-dnkjp", &LogOptions{PodLogOptions: core_v1.PodLogOptions{Container: "details"}, MaxLines: &maxLines, Duration: &duration})
+	podLogs := callStreamPodLogs(svc, "Namespace", "details", "details", "details-v1-3618568057-dnkjp", &LogOptions{PodLogOptions: core_v1.PodLogOptions{Container: "details"}, MaxLines: &maxLines, Duration: &duration})
 
 	require.Equal(2, len(podLogs.Entries))
 	assert.Equal("INFO #1 Log Message", podLogs.Entries[0].Message)
@@ -725,7 +725,7 @@ func TestGetPodLogsDuration(t *testing.T) {
 	svc := setupWorkloadService(k8s, config.NewConfig())
 
 	duration, _ := time.ParseDuration("59m")
-	podLogs := callStreamPodLogs(svc, "Namespace", "details-v1-3618568057-dnkjp", &LogOptions{PodLogOptions: core_v1.PodLogOptions{Container: "details"}, Duration: &duration})
+	podLogs := callStreamPodLogs(svc, "Namespace", "details", "details", "details-v1-3618568057-dnkjp", &LogOptions{PodLogOptions: core_v1.PodLogOptions{Container: "details"}, Duration: &duration})
 	require.Equal(1, len(podLogs.Entries))
 	assert.Equal("INFO #1 Log Message", podLogs.Entries[0].Message)
 
@@ -737,7 +737,7 @@ func TestGetPodLogsDuration(t *testing.T) {
 	svc = setupWorkloadService(k8s, conf)
 
 	duration, _ = time.ParseDuration("1h")
-	podLogs = callStreamPodLogs(svc, "Namespace", "details-v1-3618568057-dnkjp", &LogOptions{PodLogOptions: core_v1.PodLogOptions{Container: "details"}, Duration: &duration})
+	podLogs = callStreamPodLogs(svc, "Namespace", "details", "details", "details-v1-3618568057-dnkjp", &LogOptions{PodLogOptions: core_v1.PodLogOptions{Container: "details"}, Duration: &duration})
 	require.Equal(2, len(podLogs.Entries))
 	assert.Equal("INFO #1 Log Message", podLogs.Entries[0].Message)
 	assert.Equal("WARN #2 Log Message", podLogs.Entries[1].Message)
@@ -750,7 +750,7 @@ func TestGetPodLogsDuration(t *testing.T) {
 	svc = setupWorkloadService(k8s, conf)
 
 	duration, _ = time.ParseDuration("2h")
-	podLogs = callStreamPodLogs(svc, "Namespace", "details-v1-3618568057-dnkjp", &LogOptions{PodLogOptions: core_v1.PodLogOptions{Container: "details"}, Duration: &duration})
+	podLogs = callStreamPodLogs(svc, "Namespace", "details", "details", "details-v1-3618568057-dnkjp", &LogOptions{PodLogOptions: core_v1.PodLogOptions{Container: "details"}, Duration: &duration})
 	require.Equal(3, len(podLogs.Entries))
 	assert.Equal("INFO #1 Log Message", podLogs.Entries[0].Message)
 	assert.Equal("WARN #2 Log Message", podLogs.Entries[1].Message)
@@ -773,7 +773,7 @@ func TestGetPodLogsMaxLinesAndDurations(t *testing.T) {
 
 	maxLines := 2
 	duration, _ := time.ParseDuration("2h")
-	podLogs := callStreamPodLogs(svc, "Namespace", "details-v1-3618568057-dnkjp", &LogOptions{Duration: &duration, PodLogOptions: core_v1.PodLogOptions{Container: "details"}, MaxLines: &maxLines})
+	podLogs := callStreamPodLogs(svc, "Namespace", "details", "details", "details-v1-3618568057-dnkjp", &LogOptions{Duration: &duration, PodLogOptions: core_v1.PodLogOptions{Container: "details"}, MaxLines: &maxLines})
 	require.Equal(2, len(podLogs.Entries))
 	assert.Equal("INFO #1 Log Message", podLogs.Entries[0].Message)
 	assert.Equal("WARN #2 Log Message", podLogs.Entries[1].Message)
@@ -788,7 +788,7 @@ func TestGetPodLogsMaxLinesAndDurations(t *testing.T) {
 
 	maxLines = 3
 	duration, _ = time.ParseDuration("3h")
-	podLogs = callStreamPodLogs(svc, "Namespace", "details-v1-3618568057-dnkjp", &LogOptions{Duration: &duration, PodLogOptions: core_v1.PodLogOptions{Container: "details"}, MaxLines: &maxLines})
+	podLogs = callStreamPodLogs(svc, "Namespace", "details", "details", "details-v1-3618568057-dnkjp", &LogOptions{Duration: &duration, PodLogOptions: core_v1.PodLogOptions{Container: "details"}, MaxLines: &maxLines})
 	require.Equal(3, len(podLogs.Entries))
 	assert.Equal("INFO #1 Log Message", podLogs.Entries[0].Message)
 	assert.Equal("WARN #2 Log Message", podLogs.Entries[1].Message)
@@ -812,7 +812,7 @@ func TestGetPodLogsProxy(t *testing.T) {
 
 	maxLines := 2
 	duration, _ := time.ParseDuration("2h")
-	podLogs := callStreamPodLogs(svc, "Namespace", "details-v1-3618568057-dnkjp", &LogOptions{Duration: &duration, LogType: models.LogTypeProxy, PodLogOptions: core_v1.PodLogOptions{Container: "details"}, MaxLines: &maxLines})
+	podLogs := callStreamPodLogs(svc, "Namespace", "details", "details", "details-v1-3618568057-dnkjp", &LogOptions{Duration: &duration, LogType: models.LogTypeProxy, PodLogOptions: core_v1.PodLogOptions{Container: "details"}, MaxLines: &maxLines})
 	require.Equal(1, len(podLogs.Entries))
 	entry := podLogs.Entries[0]
 	assert.Equal(`[2021-02-01T21:34:35.533Z] "GET /hotels/Ljubljana HTTP/1.1" 200 - via_upstream - "-" 0 99 14 14 "-" "Go-http-client/1.1" "7e7e2dd0-0a96-4535-950b-e303805b7e27" "hotels.travel-agency:8000" "127.0.2021-02-01T21:34:38.761055140Z 0.1:8000" inbound|8000|| 127.0.0.1:33704 10.129.0.72:8000 10.128.0.79:39880 outbound_.8000_._.hotels.travel-agency.svc.cluster.local default`, entry.Message)
@@ -851,7 +851,7 @@ func TestGetZtunnelPodLogsProxy(t *testing.T) {
 
 	maxLines := 2
 	duration, _ := time.ParseDuration("2h")
-	podLogs := callStreamPodLogs(svc, "bookinfo", "details-v1-cf74bb974-wg44w", &LogOptions{Duration: &duration, LogType: models.LogTypeZtunnel, PodLogOptions: core_v1.PodLogOptions{Container: "details"}, MaxLines: &maxLines})
+	podLogs := callStreamPodLogs(svc, "bookinfo", "details", "details", "details-v1-cf74bb974-wg44w", &LogOptions{Duration: &duration, LogType: models.LogTypeZtunnel, PodLogOptions: core_v1.PodLogOptions{Container: "details"}, MaxLines: &maxLines})
 	require.Equal(1, len(podLogs.Entries))
 	entry := podLogs.Entries[0]
 
@@ -865,7 +865,7 @@ func TestGetZtunnelPodLogsProxy(t *testing.T) {
 	assert.Equal("details.bookinfo.svc.cluster.local", entry.AccessLog.RequestedServer)
 	assert.Equal(int64(1712917911078), entry.TimestampUnix)
 
-	podLogsQuotes := callStreamPodLogs(svc, "travel-agency", "cars-v1-6c869ff769-4hk27", &LogOptions{Duration: &duration, LogType: models.LogTypeZtunnel, PodLogOptions: core_v1.PodLogOptions{Container: "details"}, MaxLines: &maxLines})
+	podLogsQuotes := callStreamPodLogs(svc, "travel-agency", "cars", "cars", "cars-v1-6c869ff769-4hk27", &LogOptions{Duration: &duration, LogType: models.LogTypeZtunnel, PodLogOptions: core_v1.PodLogOptions{Container: "details"}, MaxLines: &maxLines})
 	require.Equal(2, len(podLogsQuotes.Entries))
 	entryQuotes := podLogsQuotes.Entries[0]
 
@@ -880,6 +880,60 @@ func TestGetZtunnelPodLogsProxy(t *testing.T) {
 	assert.Equal(int64(1719927663203), entryQuotes.TimestampUnix)
 }
 
+func TestGetWaypointPodLogsProxy(t *testing.T) {
+	assert := assert.New(t)
+	require := require.New(t)
+	conf := config.NewConfig()
+	// Setup mocks
+	kubeObjs := []runtime.Object{
+		FakePodWithWaypointAndDeployments(),
+		&osproject_v1.Project{ObjectMeta: v1.ObjectMeta{Name: "Namespace"}},
+	}
+	for _, obj := range FakeWaypointPod() {
+		o := obj
+		kubeObjs = append(kubeObjs, &o)
+	}
+
+	for _, obj := range FakeWaypointNamespaceEnrolledPods(true) {
+		o := obj
+		kubeObjs = append(kubeObjs, &o)
+	}
+
+	client := kubetest.NewFakeK8sClient(kubeObjs...)
+	client.OpenShift = true
+
+	k8s := &logStreamer{
+		logs:            FakePodLogsWaypoint().Logs,
+		ClientInterface: client,
+	}
+
+	SetupBusinessLayer(t, k8s, *conf)
+	svc := setupWorkloadService(k8s, conf)
+
+	a, v := svc.GetWorkload(context.TODO(), WorkloadCriteria{Cluster: svc.config.KubernetesConfig.ClusterName, Namespace: "Namespace", WorkloadName: "details", IncludeServices: false})
+	assert.Nil(v)
+	assert.NotNil(a)
+	maxLines := 2
+	duration, _ := time.ParseDuration("2h")
+	podLogs := callStreamPodLogs(svc, "Namespace", "details", "details", "details-v1-cf74bb974-wg44w", &LogOptions{Duration: &duration, LogType: models.LogTypeWaypoint, PodLogOptions: core_v1.PodLogOptions{Container: "details"}, MaxLines: &maxLines})
+	require.Equal(2, len(podLogs.Entries))
+	entry := podLogs.Entries[0]
+
+	assert.Equal("[2025-01-30T12:10:13.279Z] \"GET /details/0 HTTP/1.1\" 200 - via_upstream - \"-\" 0 178 0 0 \"-\" \"Go-http-client/1.1\" \"e7d5b6ce-5f7f-9731-8471-ca5bcd650a72\" \"details:9080\" \"envoy://connect_originate/10.244.0.32:9080\" inbound-vip|9080|http|details.bookinfo.svc.cluster.local envoy://internal_client_address/ 10.96.185.196:9080 10.244.0.37:52134 - default", entry.Message)
+	assert.Equal("2025-01-30 12:10:13.279", entry.Timestamp)
+	assert.Equal("INFO", entry.Severity)
+	assert.NotNil(entry.AccessLog)
+	assert.Equal("0", entry.AccessLog.BytesReceived)
+	assert.Equal("178", entry.AccessLog.BytesSent)
+	assert.Equal("0", entry.AccessLog.Duration)
+	assert.Equal("inbound-vip|9080|http|details.bookinfo.svc.cluster.local", entry.AccessLog.UpstreamCluster)
+	assert.Equal("envoy://connect_originate/10.244.0.32:9080", entry.AccessLog.UpstreamService)
+	assert.Equal("10.96.185.196:9080", entry.AccessLog.DownstreamLocal)
+	assert.Equal("/details/0", entry.AccessLog.UriPath)
+	assert.Equal("-", entry.AccessLog.RequestedServer)
+	assert.Equal(int64(1738239013279), entry.TimestampUnix)
+}
+
 func TestDuplicatedControllers(t *testing.T) {
 	assert := assert.New(t)
 	require := require.New(t)
@@ -1146,7 +1200,7 @@ func TestGetPodLogsWithoutAccessLogs(t *testing.T) {
 	SetupBusinessLayer(t, k8s, *conf)
 	svc := setupWorkloadService(k8s, conf)
 
-	podLogs := callStreamPodLogs(svc, "Namespace", "details-v1-3618568057-dnkjp", &LogOptions{LogType: models.LogTypeProxy, PodLogOptions: core_v1.PodLogOptions{Container: "istio-proxy"}})
+	podLogs := callStreamPodLogs(svc, "Namespace", "details", "details", "details-v1-3618568057-dnkjp", &LogOptions{LogType: models.LogTypeProxy, PodLogOptions: core_v1.PodLogOptions{Container: "istio-proxy"}})
 
 	assert.Equal(8, len(podLogs.Entries))
 	for _, entry := range podLogs.Entries {
diff --git a/mesh/api/testdata/test_mesh_graph.expected b/mesh/api/testdata/test_mesh_graph.expected
index cbd374c7a..0217cf41a 100644
--- a/mesh/api/testdata/test_mesh_graph.expected
+++ b/mesh/api/testdata/test_mesh_graph.expected
@@ -256,7 +256,8 @@
             "IstiodPollingIntervalSeconds": 20,
             "Registry": null,
             "RootNamespace": "istio-system",
-            "UrlServiceVersion": ""
+            "UrlServiceVersion": "",
+            "ValidationReconcileInterval": 60000000000
           }
         }
       },
diff --git a/tests/data/logs/waypoint.log b/tests/data/logs/waypoint.log
new file mode 100644
index 000000000..4298a5842
--- /dev/null
+++ b/tests/data/logs/waypoint.log
@@ -0,0 +1,16 @@
+2025-01-30T12:10:13.682034460Z [2025-01-30T12:10:13.279Z] "GET /details/0 HTTP/1.1" 200 - via_upstream - "-" 0 178 0 0 "-" "Go-http-client/1.1" "e7d5b6ce-5f7f-9731-8471-ca5bcd650a72" "details:9080" "envoy://connect_originate/10.244.0.32:9080" inbound-vip|9080|http|details.bookinfo.svc.cluster.local envoy://internal_client_address/ 10.96.185.196:9080 10.244.0.37:52134 - default
+2025-01-30T12:10:13.682034460Z [2025-01-30T10:53:16.279Z] "GET /details/0 HTTP/1.1" 200 - via_upstream - "-" 0 178 0 0 "-" "Go-http-client/1.1" "980435c4-7a97-9b78-96c9-dfa19ef8aacc" "details:9080" "envoy://connect_originate/10.244.0.32:9080" inbound-vip|9080|http|details.bookinfo.svc.cluster.local envoy://internal_client_address/ 10.96.185.196:9080 10.244.0.37:52134 - default
+2025-01-30T12:10:13.682034460Z [2025-01-30T10:53:16.283Z] "GET /ratings/0 HTTP/1.1" 200 - via_upstream - "-" 0 48 2 2 "-" "Go-http-client/1.1" "980435c4-7a97-9b78-96c9-dfa19ef8aacc" "ratings:9080" "envoy://connect_originate/10.244.0.41:9080" inbound-vip|9080|http|ratings.bookinfo.svc.cluster.local envoy://internal_client_address/ 10.107.133.104:9080 10.244.0.35:36718 - default
+2025-01-30T12:10:13.682034460Z [2025-01-30T10:53:16.282Z] "GET /reviews/0 HTTP/1.1" 200 - via_upstream - "-" 0 442 4 4 "-" "Go-http-client/1.1" "980435c4-7a97-9b78-96c9-dfa19ef8aacc" "reviews:9080" "envoy://connect_originate/10.244.0.35:9080" inbound-vip|9080|http|reviews.bookinfo.svc.cluster.local envoy://internal_client_address/ 10.111.226.218:9080 10.244.0.37:52134 - default
+2025-01-30T12:10:13.682034460Z [2025-01-30T10:53:16.275Z] "GET /productpage HTTP/1.1" 200 - via_upstream - "-" 0 9481 13 13 "-" "Go-http-client/1.1" "980435c4-7a97-9b78-96c9-dfa19ef8aacc" "bookinfo-gateway-istio.bookinfo" "10.244.0.39:80" inbound-vip|80|http|bookinfo-gateway-istio.bookinfo.svc.cluster.local 10.244.0.44:58916 10.102.76.216:80 10.244.0.45:52652 - default
+2025-01-30T12:10:13.682034460Z [2025-01-30T10:53:17.280Z] "GET /details/0 HTTP/1.1" 200 - via_upstream - "-" 0 178 0 0 "-" "Go-http-client/1.1" "e615c813-c295-9960-b327-a30483a5a57d" "details:9080" "envoy://connect_originate/10.244.0.32:9080" inbound-vip|9080|http|details.bookinfo.svc.cluster.local envoy://internal_client_address/ 10.96.185.196:9080 10.244.0.37:52134 - default
+2025-01-30T12:10:13.682034460Z [2025-01-30T10:53:17.283Z] "GET /reviews/0 HTTP/1.1" 200 - via_upstream - "-" 0 358 0 0 "-" "Go-http-client/1.1" "e615c813-c295-9960-b327-a30483a5a57d" "reviews:9080" "envoy://connect_originate/10.244.0.34:9080" inbound-vip|9080|http|reviews.bookinfo.svc.cluster.local envoy://internal_client_address/ 10.111.226.218:9080 10.244.0.37:52134 - default
+2025-01-30T12:10:13.682034460Z [2025-01-30T10:53:17.275Z] "GET /productpage HTTP/1.1" 200 - via_upstream - "-" 0 9429 11 11 "-" "Go-http-client/1.1" "e615c813-c295-9960-b327-a30483a5a57d" "bookinfo-gateway-istio.bookinfo" "10.244.0.39:80" inbound-vip|80|http|bookinfo-gateway-istio.bookinfo.svc.cluster.local 10.244.0.44:58916 10.102.76.216:80 10.244.0.45:52652 - default
+2025-01-30T12:10:13.682034460Z [2025-01-30T10:53:18.279Z] "GET /details/0 HTTP/1.1" 200 - via_upstream - "-" 0 178 0 0 "-" "Go-http-client/1.1" "900b8b64-c996-9e9e-a98c-2f3536875c96" "details:9080" "envoy://connect_originate/10.244.0.32:9080" inbound-vip|9080|http|details.bookinfo.svc.cluster.local envoy://internal_client_address/ 10.96.185.196:9080 10.244.0.37:52134 - default
+2025-01-30T12:10:13.682034460Z [2025-01-30T10:53:18.284Z] "GET /ratings/0 HTTP/1.1" 200 - via_upstream - "-" 0 48 2 1 "-" "Go-http-client/1.1" "900b8b64-c996-9e9e-a98c-2f3536875c96" "ratings:9080" "envoy://connect_originate/10.244.0.43:9080" inbound-vip|9080|http|ratings.bookinfo.svc.cluster.local envoy://internal_client_address/ 10.107.133.104:9080 10.244.0.36:36722 - default
+2025-01-30T12:10:13.682034460Z [2025-01-30T10:53:18.282Z] "GET /reviews/0 HTTP/1.1" 200 - via_upstream - "-" 0 438 4 4 "-" "Go-http-client/1.1" "900b8b64-c996-9e9e-a98c-2f3536875c96" "reviews:9080" "envoy://connect_originate/10.244.0.36:9080" inbound-vip|9080|http|reviews.bookinfo.svc.cluster.local envoy://internal_client_address/ 10.111.226.218:9080 10.244.0.37:52134 - default
+2025-01-30T12:10:13.682034460Z [2025-01-30T10:53:18.275Z] "GET /productpage HTTP/1.1" 200 - via_upstream - "-" 0 15068 13 13 "-" "Go-http-client/1.1" "900b8b64-c996-9e9e-a98c-2f3536875c96" "bookinfo-gateway-istio.bookinfo" "10.244.0.39:80" inbound-vip|80|http|bookinfo-gateway-istio.bookinfo.svc.cluster.local 10.244.0.44:58916 10.102.76.216:80 10.244.0.45:52652 - default
+2025-01-30T12:10:13.682034460Z [2025-01-30T10:53:19.279Z] "GET /details/0 HTTP/1.1" 200 - via_upstream - "-" 0 178 1 0 "-" "Go-http-client/1.1" "416f71e0-5d2e-9f92-a5b1-6527d2b7e06b" "details:9080" "envoy://connect_originate/10.244.0.32:9080" inbound-vip|9080|http|details.bookinfo.svc.cluster.local envoy://internal_client_address/ 10.96.185.196:9080 10.244.0.37:52134 - default
+2025-01-30T12:10:13.682034460Z [2025-01-30T10:53:19.288Z] "GET /ratings/0 HTTP/1.1" 200 - via_upstream - "-" 0 48 1 1 "-" "Go-http-client/1.1" "416f71e0-5d2e-9f92-a5b1-6527d2b7e06b" "ratings:9080" "envoy://connect_originate/10.244.0.33:9080" inbound-vip|9080|http|ratings.bookinfo.svc.cluster.local envoy://internal_client_address/ 10.107.133.104:9080 10.244.0.35:36718 - default
+2025-01-30T12:10:13.682034460Z [2025-01-30T10:53:19.285Z] "GET /reviews/0 HTTP/1.1" 200 - via_upstream - "-" 0 442 6 5 "-" "Go-http-client/1.1" "416f71e0-5d2e-9f92-a5b1-6527d2b7e06b" "reviews:9080" "envoy://connect_originate/10.244.0.35:9080" inbound-vip|9080|http|reviews.bookinfo.svc.cluster.local envoy://internal_client_address/ 10.111.226.218:9080 10.244.0.37:52134 - default
+2025-01-30T12:10:13.682034460Z [2025-01-30T10:53:19.275Z] "GET /productpage HTTP/1.1" 200 - via_upstream - "-" 0 15072 21 20 "-" "Go-http-client/1.1" "416f71e0-5d2e-9f92-a5b1-6527d2b7e06b" "bookinfo-gateway-istio.bookinfo" "10.244.0.39:80" inbound-vip|80|http|bookinfo-gateway-istio.bookinfo.svc.cluster.local 10.244.0.44:58916 10.102.76.216:80 10.244.0.45:52652 - default
\ No newline at end of file
diff --git a/tests/integration/controller/suite_test.go b/tests/integration/controller/suite_test.go
index be2b3bb6b..b896fa8fb 100644
--- a/tests/integration/controller/suite_test.go
+++ b/tests/integration/controller/suite_test.go
@@ -26,7 +26,6 @@ import (
 	"runtime"
 	"strings"
 	"testing"
-	"time"
 
 	. "github.com/onsi/ginkgo/v2"
 	. "github.com/onsi/gomega"
@@ -36,23 +35,19 @@ import (
 	"sigs.k8s.io/controller-runtime/pkg/envtest"
 	logf "sigs.k8s.io/controller-runtime/pkg/log"
 	"sigs.k8s.io/controller-runtime/pkg/log/zap"
+	metricsserver "sigs.k8s.io/controller-runtime/pkg/metrics/server"
 
-	"github.com/kiali/kiali/business"
-	"github.com/kiali/kiali/config"
 	"github.com/kiali/kiali/controller"
-	"github.com/kiali/kiali/kubernetes"
-	"github.com/kiali/kiali/kubernetes/cache"
 	kialicmd "github.com/kiali/kiali/tools/cmd"
-	"github.com/kiali/kiali/util"
 )
 
 var (
 	cfg        *rest.Config
 	k8sClient  client.Client
+	k8sManager ctrl.Manager
 	testEnv    *envtest.Environment
 	ctx        context.Context
 	cancel     context.CancelFunc
-	kialiCache cache.KialiCache
 )
 
 func TestControllers(t *testing.T) {
@@ -70,7 +65,6 @@ var _ = BeforeSuite(func() {
 	logf.SetLogger(zap.New(zap.WriteTo(GinkgoWriter), zap.UseDevMode(true)))
 
 	ctx, cancel = context.WithCancel(context.TODO())
-	var err error
 
 	istioVersion := os.Getenv("ISTIO_VERSION")
 	if istioVersion == "" {
@@ -107,13 +101,16 @@ var _ = BeforeSuite(func() {
 
 	By("bootstrapping test environment")
 	testEnv = &envtest.Environment{
-		// e.g. _output/istio-1.20.3/manifests/charts/base/crds/crd-all.gen.yaml
-		CRDDirectoryPaths:     []string{filepath.Join(kialiOutputFolder, istioVersion, "manifests", "charts", "base", "crds")},
+		// TODO: This method of loading in the istio CRDs needs to change. It is too reliant on the structure of the manifests dir
+		// which we don't control. Would be better to embed the CRDs if possible.
+		// e.g. _output/istio-1.24.0/manifests/charts/base/files/crd-all.gen.yaml
+		CRDDirectoryPaths:     []string{filepath.Join(kialiOutputFolder, istioVersion, "manifests", "charts", "base", "files")},
 		ErrorIfCRDPathMissing: true,
 		// e.g. "_output/k8s/1.29.1-linux-amd64"
 		BinaryAssetsDirectory: filepath.Join(kialiOutputFolder, "k8s", kubeVersion),
 	}
 
+	var err error
 	cfg, err = testEnv.Start()
 	Expect(err).NotTo(HaveOccurred())
 	Expect(cfg).NotTo(BeNil())
@@ -125,25 +122,12 @@ var _ = BeforeSuite(func() {
 	Expect(err).NotTo(HaveOccurred())
 	Expect(k8sClient).NotTo(BeNil())
 
-	k8sManager, err := ctrl.NewManager(cfg, ctrl.Options{
-		Scheme: scheme,
+	k8sManager, err = ctrl.NewManager(cfg, ctrl.Options{
+		Scheme:  scheme,
+		Metrics: metricsserver.Options{BindAddress: "0"},
 	})
 	Expect(err).ToNot(HaveOccurred())
 
-	conf := config.NewConfig()
-	kialiKubeClient, err := kubernetes.NewClientWithRemoteClusterInfo(cfg, nil)
-	Expect(err).ToNot(HaveOccurred())
-
-	saClients := map[string]kubernetes.ClientInterface{
-		conf.KubernetesConfig.ClusterName: kialiKubeClient,
-	}
-	kialiCache, err = cache.NewKialiCache(saClients, *conf)
-	Expect(err).ToNot(HaveOccurred())
-	layer, err := business.NewLayerWithSAClients(conf, kialiCache, nil, nil, nil, nil, nil, saClients)
-	Expect(err).ToNot(HaveOccurred())
-	err = controller.NewValidationsController(ctx, []string{conf.KubernetesConfig.ClusterName}, kialiCache, &layer.Validations, k8sManager, util.AsPtr(time.Millisecond*100))
-	Expect(err).ToNot(HaveOccurred())
-
 	go func() {
 		defer GinkgoRecover()
 		err = k8sManager.Start(ctx)
@@ -153,7 +137,6 @@ var _ = BeforeSuite(func() {
 
 var _ = AfterSuite(func() {
 	cancel()
-	kialiCache.Stop()
 	By("tearing down the test environment")
 	err := testEnv.Stop()
 	Expect(err).NotTo(HaveOccurred())
diff --git a/tests/integration/controller/validations_test.go b/tests/integration/controller/validations_test.go
index c3527f074..f13787d75 100644
--- a/tests/integration/controller/validations_test.go
+++ b/tests/integration/controller/validations_test.go
@@ -10,19 +10,55 @@ import (
 	corev1 "k8s.io/api/core/v1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/apimachinery/pkg/types"
+	"sigs.k8s.io/controller-runtime/pkg/client"
 
+	"github.com/kiali/kiali/business"
+	"github.com/kiali/kiali/config"
+	"github.com/kiali/kiali/controller"
+	"github.com/kiali/kiali/istio"
 	"github.com/kiali/kiali/kubernetes"
+	"github.com/kiali/kiali/kubernetes/cache"
 	"github.com/kiali/kiali/models"
+	"github.com/kiali/kiali/util"
 )
 
-var _ = Describe("Validations controller", func() {
-	const (
-		timeout  = time.Second * 10
-		interval = time.Millisecond * 250
-	)
+const (
+	timeout  = time.Second * 10
+	interval = time.Millisecond * 250
+)
 
+var _ = Describe("Validations controller", Ordered, func() {
+	var kialiCache cache.KialiCache
 	Context("When validating a VirtualService", func() {
-		It("Should create validations in the kiali cache when a new VirtualService is created", func() {
+		BeforeAll(func(specCtx SpecContext) {
+			conf := config.NewConfig()
+			kialiKubeClient, err := kubernetes.NewClientWithRemoteClusterInfo(cfg, nil)
+			Expect(err).ToNot(HaveOccurred())
+
+			saClients := map[string]kubernetes.ClientInterface{
+				conf.KubernetesConfig.ClusterName: kialiKubeClient,
+			}
+			kialiCache, err = cache.NewKialiCache(saClients, *conf)
+			Expect(err).ToNot(HaveOccurred())
+			DeferCleanup(func() {
+				kialiCache.Stop()
+			})
+
+			kialiCache.SetMesh(
+				&models.Mesh{
+					ControlPlanes: []models.ControlPlane{{Cluster: &models.KubeCluster{IsKialiHome: true}}},
+				},
+			)
+
+			discovery := istio.NewDiscovery(saClients, kialiCache, conf)
+			layer, err := business.NewLayerWithSAClients(conf, kialiCache, nil, nil, nil, nil, discovery, saClients)
+			Expect(err).ToNot(HaveOccurred())
+
+			err = controller.NewValidationsController(ctx, []string{conf.KubernetesConfig.ClusterName}, kialiCache, &layer.Validations, k8sManager, util.AsPtr(time.Millisecond*100))
+			Expect(err).ToNot(HaveOccurred())
+		})
+
+		It("Should create validations in the kiali cache when a new VirtualService is created", func(ctx SpecContext) {
 			Expect(kialiCache.Validations().Items()).Should(BeEmpty())
 
 			By("By creating a VirtualService")
@@ -31,7 +67,8 @@ var _ = Describe("Validations controller", func() {
 					Name: "istio-system",
 				},
 			}
-			Expect(k8sClient.Create(ctx, istioSystemNamespace)).Should(Succeed())
+			err := k8sClient.Create(ctx, istioSystemNamespace)
+			Expect(client.IgnoreAlreadyExists(err)).ToNot(HaveOccurred())
 			vs := &networkingv1beta1.VirtualService{
 				ObjectMeta: metav1.ObjectMeta{
 					Name:      "test-vs",
@@ -42,7 +79,8 @@ var _ = Describe("Validations controller", func() {
 					Gateways: []string{"test-gateway"},
 				},
 			}
-			Expect(k8sClient.Create(ctx, vs)).Should(Succeed())
+			err = k8sClient.Create(ctx, vs)
+			Expect(client.IgnoreAlreadyExists(err)).ToNot(HaveOccurred())
 
 			vsKey := types.NamespacedName{Name: vs.Name, Namespace: vs.Namespace}
 			createdVS := &networkingv1beta1.VirtualService{}
@@ -60,10 +98,10 @@ var _ = Describe("Validations controller", func() {
 			}, timeout, interval).Should(BeTrue())
 		})
 
-		It("Should update validations in the kiali cache when an existing VirtualService is updated", func() {
+		It("Should update validations in the kiali cache when an existing VirtualService is updated", func(ctx SpecContext) {
 			validationKey := models.IstioValidationKey{Name: "test-vs", Namespace: "default", ObjectGVK: kubernetes.VirtualServices}
-			validation, err := kialiCache.Validations().Get(validationKey)
-			Expect(err).ShouldNot(HaveOccurred())
+			validation, found := kialiCache.Validations().Get(validationKey)
+			Expect(found).To(BeTrue())
 			Expect(validation.Checks).Should(BeEmpty())
 			By("By updating a VirtualService")
 			vs := &networkingv1beta1.VirtualService{}
@@ -90,13 +128,13 @@ var _ = Describe("Validations controller", func() {
 
 			By("By checking that the validations are then updated in the kiali cache")
 			Eventually(func() bool {
-				validation, err := kialiCache.Validations().Get(validationKey)
-				Expect(err).ShouldNot(HaveOccurred())
+				validation, found := kialiCache.Validations().Get(validationKey)
+				Expect(found).To(BeTrue())
 				return len(validation.Checks) > 0
 			}, timeout, interval).Should(BeTrue())
 		})
 
-		It("Should delete validations in the kiali cache when the VirtualService is deleted", func() {
+		It("Should delete validations in the kiali cache when the VirtualService is deleted", func(ctx SpecContext) {
 			By("By deleting a VirtualService")
 			vs := &networkingv1beta1.VirtualService{}
 			vsKey := types.NamespacedName{Name: "test-vs", Namespace: "default"}
@@ -106,8 +144,8 @@ var _ = Describe("Validations controller", func() {
 			By("By checking that the validations are then deleted from the kiali cache")
 			Eventually(func() bool {
 				validationKey := models.IstioValidationKey{Name: "test-vs", Namespace: "default", ObjectGVK: kubernetes.VirtualServices}
-				_, result := kialiCache.Validations().Get(validationKey)
-				return result && len(kialiCache.Validations().Items()) == 0
+				_, found := kialiCache.Validations().Get(validationKey)
+				return !found && len(kialiCache.Validations().Items()) == 0
 			}, timeout, interval).Should(BeTrue())
 		})
 	})
diff --git a/tests/integration/tests/traces_spans_test.go b/tests/integration/tests/traces_spans_test.go
index 4f7ce5338..74fac04b4 100644
--- a/tests/integration/tests/traces_spans_test.go
+++ b/tests/integration/tests/traces_spans_test.go
@@ -104,14 +104,20 @@ func assertTraces(traces *model.TracingResponse, statusCode int, err error, requ
 }
 
 func assertSpans(spans []model.TracingSpan, statusCode int, err error, require *require.Assertions) {
+	tracing, _, errTracing := kiali.TracingConfig()
+	require.NoError(errTracing)
+
 	if statusCode == 200 {
 		require.NoError(err)
 		require.NotNil(spans)
 		if len(spans) > 0 {
 			require.NotNil(spans[0].TraceID)
-			require.NotEmpty(spans[0].References)
-			require.NotNil(spans[0].References[0].TraceID)
-			require.Equal(spans[0].TraceID, spans[0].References[0].TraceID)
+			// References in Tempo are converted from the SpanTraceId, which is not available from the /api/search in Tempo
+			if tracing.Provider == "jaeger" {
+				require.NotEmpty(spans[0].References)
+				require.NotNil(spans[0].References[0].TraceID)
+				require.Equal(spans[0].TraceID, spans[0].References[0].TraceID)
+			}
 		}
 	} else {
 		require.Fail(fmt.Sprintf("Status code should be '200' but was: %d and error: %s", statusCode, err))
diff --git a/tests/integration/utils/kiali/kiali_client.go b/tests/integration/utils/kiali/kiali_client.go
index 36fd8ca97..b5a5da45d 100644
--- a/tests/integration/utils/kiali/kiali_client.go
+++ b/tests/integration/utils/kiali/kiali_client.go
@@ -168,6 +168,18 @@ func KialiConfig() (*handlers.PublicConfig, int, error) {
 	}
 }
 
+func TracingConfig() (*models.TracingInfo, int, error) {
+	url := fmt.Sprintf("%s/api/tracing", client.kialiURL)
+	response := new(models.TracingInfo)
+
+	code, err := getRequestAndUnmarshalInto(url, response)
+	if err == nil {
+		return response, code, nil
+	} else {
+		return nil, code, err
+	}
+}
+
 func Namespaces() (*models.Namespaces, int, error) {
 	url := fmt.Sprintf("%s/api/namespaces", client.kialiURL)
 	response := new(models.Namespaces)