Code change:
<ins> #!/bin/bash # Copyright 2014 Google Inc. All rights reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # #     http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # Launches a container and verifies it can be reached. Assumes that # we're being called by hack/e2e-test.sh (we use some env vars it sets up). # TODO: Convert uses of gcutil to gcloud set -o errexit set -o nounset set -o pipefail KUBE_ROOT=$(dirname "${BASH_SOURCE}")/../.. source "${KUBE_ROOT}/cluster/kube-env.sh" source "${KUBE_ROOT}/cluster/$KUBERNETES_PROVIDER/util.sh" if [[ "$KUBERNETES_PROVIDER" != "gce" ]]; then echo "PD test is only run for GCE" return 0 fi disk_name="e2e-$(date +%H-%M-%s)" config="/tmp/${disk_name}.yaml" function teardown() { echo "Cleaning up test artifacts" ${KUBECFG} delete pods/testpd rm -rf ${config} echo "Waiting for disk to become unmounted" sleep 20 gcutil deletedisk -f --zone=${ZONE} ${disk_name} } trap "teardown" EXIT perl -p -e "s/%.*%/${disk_name}/g" ${KUBE_ROOT}/examples/gce-pd/testpd.yaml > ${config} # Create and mount the disk. gcutil adddisk --size_gb=10 --zone=${ZONE} ${disk_name} gcutil attachdisk --disk ${disk_name} ${MASTER_NAME} gcutil ssh ${MASTER_NAME} sudo rm -rf /mnt/tmp gcutil ssh ${MASTER_NAME} sudo mkdir /mnt/tmp gcutil ssh ${MASTER_NAME} sudo /usr/share/google/safe_format_and_mount /dev/disk/by-id/google-${disk_name} /mnt/tmp gcutil ssh ${MASTER_NAME} sudo umount /mnt/tmp gcloud compute instances detach-disk --disk ${disk_name} --zone ${ZONE} ${MASTER_NAME} ${KUBECFG} -c ${config} create pods pod_id_list=$($KUBECFG '-template={{range.items}}{{.id}} {{end}}' -l test=testpd list pods) # Pod turn up on a clean cluster can take a while for the docker image pull. all_running=0 for i in $(seq 1 24); do echo "Waiting for pods to come up." sleep 5 all_running=1 for id in $pod_id_list; do current_status=$($KUBECFG -template '{{.currentState.status}}' get pods/$id) || true if [[ "$current_status" != "Running" ]]; then all_running=0 break fi done if [[ "${all_running}" == 1 ]]; then break fi done if [[ "${all_running}" == 0 ]]; then echo "Pods did not come up in time" exit 1 fi </ins>

Related discussion:
text: Loaded restart count threshold overrides: map[coredns:2 fluentd-gcp:999 konnectivity-agent:2 kube-controller-manager:1 kube-proxy:2 kube-scheduler:1 l7-lb-controller:1 metadata-proxy:2 prometheus-to-sd-exporter:2] I0227 05:03:32. ] ClusterOOMsTracker: gathering cluster OOMs tracking measurement E0227 05:03:32. ] TestMetrics: [action gather failed for SystemPodMetrics measurement: restart counts violation: RestartCount(node-problem-detector-tgl8h, node-problem-detector)=1, want <= 0; RestartCount(node-problem-detector-vpsmz, node-problem-detector)=1, want <= 0] I0227 05:03:32. ] Step "[step: 29] gathering measurements" ended W0227 05:03:32. ] Got errors during step execution: [measurement call TestMetrics - TestMetrics error: [action gather failed for SystemPodMetrics measurement: restart counts violation: RestartCount(node-problem-detector-tgl8h, node-problem-detector)=1, want <= 0; RestartCount(node-problem-detector-vpsmz, node-problem-detector)=1, want <= 0]] I0227 05:03:32. ] Waiting for the chaos monkey subroutine to end... I0227 05:03:32. ] Chaos monkey ended. I0227 05:03:32. ] I0227 05:03:32. ] Stopping DnsLookupLatency probe... I0227 05:03:47. ] Stopping InClusterNetworkLatency probe... I0227 05:04:22. ] Resources cleanup time: 50.539700215s E0227 05:04:22. ]
E0227 05:04:22. ] Test Finished E0227 05:04:22. ] Test: E0227 05:04:22. ] Status: Fail E0227 05:04:22. ]