Code change:
07-02 and 13-02 No response No response /sig
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig scalability
I might be related to the npd.
The same test failures happening again on 25.02:
/sig node
/sig node /cc The restart of caused the flake. I also opened to track it in NPD repo.
I missed this log. It is caused by oom. Adding some memory may help. I submitted to fix it. Does mean killing metrics-server?
/remove-sig node Sorry, I misread the log. The NPD found the metric server is OOM. It is about the metrics server, not the NPD. Sorry for disturb. I may add some memory for metric-server and check why metric server memory changed. I heard that metrics server recently released a new version. I will check if it is related.
/sig node
OOM metrics shows the metric server is OOMed. This is odd, that only metrics-server-nanny is with a memory limit. It uses whose command is . metrics-server container has no memory limit, but it got killed by OOM. Does this indicate that the node is OOM? (edited)
The last run failed in node-problem-detector ContainerStateTerminated. I0227 05:03:31. ] PodStartupLatency: perc50: 14m29.018886827s, perc90: 28m44.625518807s, perc99: 32m2.19303496s; threshold 1h0m0s I0227 05:03:32. ] Loaded restart count threshold overrides: map[coredns:2 fluentd-gcp:999 konnectivity-agent:2 kube-controller-manager:1 kube-proxy:2 kube-scheduler:1 l7-lb-controller:1 metadata-proxy:2 prometheus-to-sd-exporter:2] I0227 05:03:32. ] ClusterOOMsTracker: gathering cluster OOMs tracking measurement E0227 05:03:32. ] TestMetrics: [action gather failed for SystemPodMetrics measurement: restart counts violation: RestartCount(node-problem-detector-tgl8h, node-problem-detector)=1, want <= 0; RestartCount(node-problem-detector-vpsmz, node-problem-detector)=1, want <= 0] I0227 05:03:32. ] Step "[step: 29] gathering measurements" ended W0227 05:03:32. ] Got errors during step execution: [measurement call TestMetrics - TestMetrics error: [action gather failed for SystemPodMetrics measurement: restart counts violation: RestartCount(node-problem-detector-tgl8h, node-problem-detector)=1, want <= 0; RestartCount(node-problem-detector-vpsmz, node-problem-detector)=1, want <= 0]] I0227 05:03:32. ] Waiting for the chaos monkey subroutine to end... I0227 05:03:32. ] Chaos monkey ended. I0227 05:03:32. ] I0227 05:03:32. ] Stopping DnsLookupLatency probe... I0227 05:03:47. ] Stopping InClusterNetworkLatency probe... I0227 05:04:22. ] Resources cleanup time: 50.539700215s E0227 05:04:22. ]
E0227 05:04:22. ] Test Finished E0227 05:04:22. ] Test: E0227 05:04:22. ] Status: Fail E0227 05:04:22. ] Errors: [measurement call TestMetrics - TestMetrics error: [action gather failed for SystemPodMetrics measurement: restart counts violation: RestartCount(node-problem-detector-tgl8h, node-problem-detector)=1, want <= 0; RestartCount(node-problem-detector-vpsmz, node-problem-detector)=1, want <= 0]] E0227 05:04:22. ]
I0227 05:04:22. ]
I0227 05:04:22. ] Running testing/huge- I0227 05:04:22. ]

Related discussion:
text: MINION_TAG="${INSTANCE_PREFIX}-minion" MINION_NAMES=($(eval echo ${INSTANCE_PREFIX}-minion-{1..${NUM_MINIONS}})) MINION_IP_RANGES=($(eval echo "10.245.{1..${NUM_MINIONS}}.0/24")) <del> MINION_SCOPES="" </del> <ins> MINION_SCOPES="storage-ro,compute-rw" </ins> # Increase the sleep interval value if concerned about API rate limits. 3, in seconds, is the default. POLL_SLEEP_INTERVAL=3 PORTAL_NET="10.0.0.0/16"