Change:
Many jobs are failing because of Some of the job logs: /priority critical-urgent /cc /cc /milestone v1.18 /sig release
Also, all the jobs report in their log the following: Do we have to take any action to change that?
After some investigation, it seems that file contains instead of
Switching from to podutils should be done by the job owners - this is out of scope for releas team (strictly speaking). And the message is just a warning, should not affect the job itself.
is this related to the workload identity switch yesterday?
Maybe? get- does seem to pass auth headers, possibly this part isn't working correctly? These jobs explicitly mount a service- though so I wouldn't expect workload identity to change things but I could be wrong...
Yes, this will not work until we attach a kubernetes service account to the job that uses workload identity:
Ideally that command should become (and let it decide how to best authenticate) rather than assuming the script runs on a VM
/assign
sent
/reopen doesn't seem to solve the issue. The jobs are still failing because of this reason. Do we need any more actions after ????
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I don't agree, the definitely helped in case of e2e scalability jobs, our presubmits and continuous jobs started passing again, e.g. https://k8s-
Looks green?
Only the 1.18 jobs are green. We need to cherry-pick the fix to the other branches as well. I will keep this issue open.
For some reason I thought kubetest always used the copy from HEAD.
in the case that it didn't we'll need this cherry picked to all active branches :|
I thought the same, but indeed the fixed only 1.18 and master jobs. Older release scalability jobs are still broken ! It's a bit confusing TBH. I thought that get- is always downloaded by kubetest from which IIUC points to the master branch. Alternatively, get- could've been embedded in the kubekins image, but if that was the case then couldn't have fixed our 1.18/master tests without kubekins version update (which didn't happen). Another options is that kubetest downloads get- from proper release branches, but I couldn't find such logic in the kubetest code. would you mind explaining how it works?
There are a lot of jobs that are failing because of this. - master - 1.18 7 jobs in 1.17 branch 5 jobs in 1.16 branch 5 jobs in 1.15 branch Can you please take a look?
/sig testing
Ping, any updates?
filed a number of cherry picks for the script
Closing... The issue has been resolved. Thanks, /close
Closing this issue. <detailsIn response to : Closing... The issue has been resolved. Thanks, /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details

Discussion:
fi } <del> # Get default service account credentials of the VM. GCE_METADATA_INTERNAL="http://metadata.google.internal/computeMetadata/v1/instance" function get-credentials { curl "${GCE_METADATA_INTERNAL}/service-accounts/default/token" -H "Metadata-Flavor: Google" -s | python -c  'import sys; import json; print(json.loads(sys.stdin.read())["access_token"])' } function valid-storage-scope { curl "${GCE_METADATA_INTERNAL}/service-accounts/default/scopes" -H "Metadata-Flavor: Google" -s | grep -E "auth/devstorage|auth/cloud-platform" } </del> function download_tarball() { local -r download_path="$1" local -r file="$2"

# if the url belongs to GCS API we should use oauth2_token in the headers curl_headers="" if { [[ "${KUBERNETES_PROVIDER:-gce}" == "gce" ]] || [[ "${KUBERNETES_PROVIDER}" == "gke" ]] ; } && <del> [[ "$url" =~ ^https://storage.googleapis.com.* ]] && valid-storage-scope ; then curl_headers="Authorization: Bearer $(get-credentials)" </del> <ins> [[ "$url" =~ ^https://storage.googleapis.com.* ]]; then curl_headers="Authorization: Bearer $(gcloud auth print-access-token)" </ins> fi curl ${curl_headers:+-H "${curl_headers}"} -fL --retry 3 --keepalive-time 2 "${url}" -o "${download_path}/${file}" elif [[ $(which wget) ]]; then

) } <del> # Get default service account credentials of the VM. GCE_METADATA_INTERNAL="http://metadata.google.internal/computeMetadata/v1/instance" function get-credentials { curl "${GCE_METADATA_INTERNAL}/service-accounts/default/token" -H "Metadata-Flavor: Google" -s | python -c  'import sys; import json; print(json.loads(sys.stdin.read())["access_token"])' } </del> function valid-storage-scope { curl "${GCE_METADATA_INTERNAL}/service-accounts/default/scopes" -H "Metadata-Flavor: Google" -s | grep -E "auth/devstorage|auth/cloud-platform" }

# if the url belongs to GCS API we should use oauth2_token in the headers curl_headers="" if { [[ "${KUBERNETES_PROVIDER:-gce}" == "gce" ]] || [[ "${KUBERNETES_PROVIDER}" == "gke" ]] ; } && <del> [[ "$kubernetes_tar_url" =~ ^https://storage.googleapis.com.* ]] && valid-storage-scope ; then curl_headers="Authorization: Bearer $(get-credentials)" </del> <ins> [[ "$kubernetes_tar_url" =~ ^https://storage.googleapis.com.* ]] ; then curl_headers="Authorization: Bearer $(gcloud auth print-access-token)" </ins> fi curl ${curl_headers:+-H "${curl_headers}"} -fL --retry 3 --keepalive-time 2 "${kubernetes_tar_url}" -o "${file}" elif [[ $(which wget) ]]; then