Code change:
Part of From For now posted as a "help wanted" issue, however if we cannot hit the timeline for 1.30 code freeze () I would take it over so we don't miss. Still issue is simple enough that it can be quickly implemented. Implementation tips: Add a metric around waitUnitlFreshAndBlock method Metric should be a histogram with latency buckets similar to api latency Metric should have label for resource version. A way to monitor watch cache impact on request latency.
/sig api-machinery /triage accepted /help
This request has been marked as needing help from a contributor. Please ensure that the issue body includes answers to the following questions: Why are we solving this issue? To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points? Does this issue have zero to low barrier of entry? How can the assignee reach out to you for help? For more details on the requirements of such an issue, please see and ensure that they are met. If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : /sig api-machinery /triage accepted /help Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details

Related discussion:
text: store.updateCache(add) } } <ins> func TestHistogramCacheReadWait(t *testing.T) { registry := k8smetrics.NewKubeRegistry() if err := registry.Register(metrics.WatchCacheReadWait); err != nil { t.Errorf("unexpected error: %v", err) } ctx := context.Background() testedMetrics := "apiserver_watch_cache_read_wait_seconds" store := newTestWatchCache(2, &cache.Indexers{}) defer store.Stop() // In background, update the store. go func() { if err := store.Add(makeTestPod("foo", 2)); err != nil { t.Errorf("unexpected error: %v", err) } if err := store.Add(makeTestPod("bar", 5)); err != nil { t.Errorf("unexpected error: %v", err) } }() testCases := []struct { desc            string resourceVersion uint64 want            string }{ { desc:            "resourceVersion is non-zero", resourceVersion: 5, want: ` # HELP apiserver_watch_cache_read_wait_seconds [ALPHA] Histogram of time spent waiting for a watch cache to become fresh. # TYPE apiserver_watch_cache_read_wait_seconds histogram apiserver_watch_cache_read_wait_seconds_bucket{resource="pods",le="0.005"} 1 apiserver_watch_cache_read_wait_seconds_bucket{resource="pods",le="0.025"} 1 apiserver_watch_cache_read_wait_seconds_bucket{resource="pods",le="0.05"} 1 apiserver_watch_cache_read_wait_seconds_bucket{resource="pods",le="0.1"} 1 apiserver_watch_cache_read_wait_seconds_bucket{resource="pods",le="0.2"} 1 apiserver_watch_cache_read_wait_seconds_bucket{resource="pods",le="0.4"} 1 apiserver_watch_cache_read_wait_seconds_bucket{resource="pods",le="0.6"} 1 apiserver_watch_cache_read_wait_seconds_bucket{resource="pods",le="0.8"} 1 apiserver_watch_cache_read_wait_seconds_bucket{resource="pods",le="1"} 1 apiserver_watch_cache_read_wait_seconds_bucket{resource="pods",le="1.25"} 1 apiserver_watch_cache_read_wait_seconds_bucket{resource="pods",le="1.5"} 1 apiserver_watch_cache_read_wait_seconds_bucket{resource="pods",le="2"} 1 apiserver_watch_cache_read_wait_seconds_bucket{resource="pods",le="3"} 1 apiserver_watch_cache_read_wait_seconds_bucket{resource="pods",le="+Inf"} 1 apiserver_watch_cache_read_wait_seconds_sum{resource="pods"} 0 apiserver_watch_cache_read_wait_seconds_count{resource="pods"} 1 `, }, { desc:            "resourceVersion is 0", resourceVersion: 0, want:            ``, }, } for _, test := range testCases { t.Run(test.desc, func(t *testing.T) { defer registry.Reset() if _, _, _, err := store.WaitUntilFreshAndGet(ctx, test.resourceVersion, "prefix/ns/bar"); err != nil { t.Errorf("unexpected error: %v", err) } if err := testutil.GatherAndCompare(registry, strings.NewReader(test.want), testedMetrics); err != nil { t.Errorf("unexpected error: %v", err) } }) } } </ins>