jiebi/CodeConvo huggingface dataset:
Code change:
return volumeLimits, nil } <del> if vmSizeList == nil { ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second) defer cancel() result, err := az.VirtualMachineSizesClient.List(ctx, az.Location) if err != nil || result.Value == nil { klog.Errorf("failed to list vm sizes in GetVolumeLimits, plugin.host: %s, location: %s", plugin.host.GetHostName(), az.Location) return volumeLimits, nil } vmSizeList = result.Value } </del> volumeLimits = map[string]int64{ <del> util.AzureVolumeLimitKey: getMaxDataDiskCount(instanceType, vmSizeList), </del> <ins> util.AzureVolumeLimitKey: getMaxDataDiskCount(instanceType), </ins> } return volumeLimits, nil } <del> func getMaxDataDiskCount(instanceType string, sizeList *[]compute.VirtualMachineSize) int64 { if sizeList == nil { return defaultAzureVolumeLimit } </del> <ins> func getMaxDataDiskCount(instanceType string) int64 { </ins> vmsize := strings.ToUpper(instanceType) <del> for _, size := range *sizeList { if size.Name == nil || size.MaxDataDiskCount == nil { klog.Errorf("failed to get vm size in getMaxDataDiskCount") continue } if strings.ToUpper(*size.Name) == vmsize { klog.V(12).Infof("got a matching size in getMaxDataDiskCount, Name: %s, MaxDataDiskCount: %d", *size.Name, *size.MaxDataDiskCount) return int64(*size.MaxDataDiskCount) } </del> <ins> maxDataDiskCount, exists := maxDataDiskCountMap[vmsize] if exists { klog.V(12).Infof("got a matching size in getMaxDataDiskCount, VM Size: %s, MaxDataDiskCount: %d", vmsize, maxDataDiskCount) return maxDataDiskCount </ins> } <ins> klog.V(12).Infof("not found a matching size in getMaxDataDiskCount, VM Size: %s, use default volume limit: %d", vmsize, defaultAzureVolumeLimit) </ins> return defaultAzureVolumeLimit }

Related discussion:
text: <!-- Please only use this template for submitting enhancement requests -- What would you like to be : related to Why is this needed: /assign /sig azure
here is the way about how to get MaxDataDiskCount: and then build kubelet, copy kubelet to agent node and then get MaxDataDiskCount data: There are 207 VM sizes totally.
Update: this command could get VM size list easily:
Code change:
"//pkg/kubelet/util/ioutils:go_default_library", "//pkg/security/apparmor:go_default_library", "//pkg/util/hash:go_default_library", <ins> "//pkg/util/parsers:go_default_library", </ins> "//pkg/util/term:go_default_library", "//vendor/github.com/blang/semver:go_default_library", "//vendor/github.com/docker/docker/api/types:go_default_library",

Related discussion:
text: <!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: What happened: We have a private registry configured (where all images including the sandbox one), are hosted. The registry credentials are being pulled by one of k8s credentials providers () With Kubernetes 1.7 sandbox image failed to be pulled with the following error message: Looks like with this only creds from docker config are being evaluated on the sandbox image pull What you expected to happen: Sandbox image pull should be attempted using credentials returned by not just the basic docker credential provider, but other credentials providers registered in the system. How to reproduce it (as minimally and precisely as possible): 1) Register private repo credentials with one of the k8s credential providers (we've tried with rancher). Configure the to use the image from the the private repo. 2) Make sure you are not logged in to this private repo on the kubelet host 3) Try to create a pod. Sandbox image pull will fail with an error mentioned above. Anything else we need to know?: Non-sandbox images from the same private repo were able to download correctly using private repo credentials provider. Environment: Kubernetes version (use ): 1.7.2 Cloud provider or hardware configuration**: OS (e.g. from /etc/os-release): Ubuntu 16.04 Kernel (e.g. ): Install tools: Others:
There are no sig labels on this issue. Please by: a sig: e.g., to notify the contributor experience sig, OR the label manually: e.g., to apply the label Note: Method 1 will trigger an email to the group. You can find the group list and label list . The in the method 1 has to be replaced with one of these: bugs, feature-requests, pr-reviews, test-failures, proposals
/sig node Just tested the same use case against k8s 1.5.x, it worked fine there and sandbox image was pulled using the credentials passed by the credential provider, not from docker config file. So it might have regressed in 1.7.x (haven't tested it on 1.6, so can't tell for sure)
Confirmed a bug introduced in dockershim. Will file a PR to fix this.
Code change:
"//pkg/kubelet/util/ioutils:go_default_library", "//pkg/security/apparmor:go_default_library", "//pkg/util/hash:go_default_library", <ins> "//pkg/util/parsers:go_default_library", </ins> "//pkg/util/term:go_default_library", "//vendor/github.com/blang/semver:go_default_library", "//vendor/github.com/docker/docker/api/types:go_default_library",

Related discussion:
text: Could you also verify whether fixed your problem?
tested against your commits cherry-picked on top of 1.8 beta, the fix seems to work
thanks.
[MILESTONENOTIFIER] Milestone Labels Complete Issue label settings: sig/node: Issue will be escalated to these SIGs if needed. priority/important-soon: Escalate to the issue owners and SIG owner; move out of milestone after several unsuccessful escalation attempts. kind/bug: Fixes a bug discovered during the current release. <detailsAdditional instructions available <a href=""here</aThe commands available for adding these labels are documented <a href=""here</a</details
Code change:
} } <del> // PingCommand is the type to hold ping command. type PingCommand string const ( // IPv4PingCommand is a ping command for IPv4. IPv4PingCommand PingCommand = "ping" // IPv6PingCommand is a ping command for IPv6. IPv6PingCommand PingCommand = "ping6" ) </del> // CheckConnectivityToHost launches a pod to test connectivity to the specified // host. An error will be returned if the host is not reachable from the pod. // // An empty nodeName will use the schedule to choose where the pod is executed. <del> func CheckConnectivityToHost(f *Framework, nodeName, podName, host string, pingCmd PingCommand, timeout int) error { </del> <ins> func CheckConnectivityToHost(f *Framework, nodeName, podName, host string, port, timeout int) error { </ins> contName := fmt.Sprintf("%s-container", podName) command := []string{ <del> string(pingCmd), "-c", "3", // send 3 pings "-W", "2", // wait at most 2 seconds for a reply </del> <ins> "nc", "-vz", </ins> "-w", strconv.Itoa(timeout), host, <ins> strconv.Itoa(port), </ins> } pod := &v1.Pod{

Related discussion:
text: <!-- Please only use this template for submitting reports about failing tests in Kubernetes CI jobs --Which jobs are failing: N/A Which test(s) are failing: Since when has it been failing: Ever Reason for failure: Vm on Azure with no public IP assigned aren't able to perform a command to the internet. Anything else we need to know: The test (see ) tries to test the internet connection for a pod by pinging 8.8.8.8. But azure does not provide ICMP support for VM without a public IP therefore this test cannot succeed.(see ) A fix would be to change the test method to a tcp request instead of a ping request (the test already performs a http get to in its BeforeSuite (see ))
/sig network /sig testing /sig azure
you may refer to (Pods don't have connectivity to external networks )
cc WDYT about the proposal to change this from an ICMP call? We probably need input from sig-architecture as well since this as well /sig architecture
/triage unresolved
An alternative is to issue a TCP/UDP request to 8.8.8.8 on port 53.
ICMP is dropped by Azure for security reasons. This is intended. Can use curl as a replacement. I will fix the corresponding test.
SGTM
Can you please review the related PR? I'll in the PR again. Thanks!
Code change:
// ServicePort contains information on service's port. type ServicePort struct { // The name of this port within the service. This must be a DNS_LABEL. <del> // All ports within a ServiceSpec must have unique names. This maps to // the 'Name' field in EndpointPort objects. </del> <ins> // All ports within a ServiceSpec must have unique names. When considering // the endpoints for a Service, this must match the 'name' field in the // EndpointPort. </ins> // Optional if only one ServicePort is defined on this service. // +optional Name string `json:"name,omitempty" protobuf:"bytes,1,opt,name=name"`

Related discussion:
text: <!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: When running a cluster service and manually created endpoints, the port name set on the cluster definition acts as an implicit selector - suitable end-points do not get mapped to the cluster service. What you expected to happen: Port name shouldn't be an implicit selector. How to reproduce it (as minimally and precisely as possible): Using the following spec file "": kubectl create -f , then describe the service. Output shows the endpoint as being correctly mapped Now run kubectl delete -f the and uncomment the name: foobar Re-run the kubectl create -f command. Now, checking the service gives the output below (end point isn't mapped): Anything else we need to know?: Adding the name: foobar to the endpoint definition makes the end point to be mapped again into the cluster. I.e. there is an implicit matching of the end-point and cluster port definitions. This is either a bug or a very surprising and undocumented feature Environment: kubectl version Client Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.7", GitCommit:"", GitTreeState:"clean", BuildDate:"2019-03-25T02:52:13Z", GoVersion:"go1.10.8", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.5", GitCommit:"", GitTreeState:"clean", BuildDate:"2019-01-16T18:14:49Z", GoVersion:"go1.10.7", Compiler:"gc", Platform:"linux/amd64"} Cloud provider or hardware configuration: OS (e.g: ): root cat /etc/os-release NAME="Ubuntu" VERSION="18.04.1 LTS (Bionic Beaver)" Kernel (e.g.
Code change:
// ServicePort contains information on service's port. type ServicePort struct { // The name of this port within the service. This must be a DNS_LABEL. <del> // All ports within a ServiceSpec must have unique names. This maps to // the 'Name' field in EndpointPort objects. </del> <ins> // All ports within a ServiceSpec must have unique names. When considering // the endpoints for a Service, this must match the 'name' field in the // EndpointPort. </ins> // Optional if only one ServicePort is defined on this service. // +optional Name string `json:"name,omitempty" protobuf:"bytes,1,opt,name=name"`

Related discussion:
text: ): uname -a Linux 4.15.0-51-generic -Ubuntu SMP Wed May 15 14:27:21 UTC 2019 x8664 x8664 x86_64 GNU/Linux Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig api-machinery
/remove-sig api-machinery /sig network The name mapping to the endpoints is intentional, but needs documentation
/triage unresolved
/assign
It may be under-documented, but it is not UNdocmented.
Also
working on a PR to fix doc
Code change:
<del> 1.2 </del> <ins> 1.3 </ins>

Related discussion:
text: E2eNode Suite.[sig-node] Node Performance Testing [Serial] [Slow] Run node performance testing with pre-defined workloads TensorFlow workload From the testgrid, it has been 1/2 flaky since the start of history (03/20/2022) and has been failing constantly (0/2) since 04/01/2022 No response /sig node
/triage accepted /priority important-soon
The failing test seems to be expecting GPU on machine and the image seems to be broken. I tried the image manually: Logs:
/assign as discussed at SIG Node CI meeting, can you please take a look? I see that pod is now running for couple minutes and then stops. I wonder if this is something environment related - maybe pod was evicted or something. I doesn't seem to be able to repro the failure of the pod locally
GitHub didn't allow me to assign the following users: xmcqueen. Note that only , repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. For more information please see <detailsIn response to : /assign Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
This is no longer a flake. It has failed every run since the start of June. Locally this test runs perfectly with a legit correct local run of the test suite code, doing the full tensorflow run of 300 epochs. The failure in the testgrid job seems to be fast (173 sec to build and fail, locally its 160 sec before it starts the tensorflow pod). I wonder if there's a blocked port or something from the data initializing python program. It needs access to: . Here's the command it runs: Is it feasible to add debugging code, like a pre-check for that http port? It would be good to at least force it to fail with a meaningful error message.
Note that HEAD is now at image version 1.2, and it is still failing per . The successful local runs of the test mentioned above used image version 1.2.
Code change:
<del> 1.2 </del> <ins> 1.3 </ins>

Related discussion:
text: Locally image version 1.1 does produce the cuda/GPU error shown above by :
More info is needed to find the RC for this test failure. I've put in a PR to get the test framework to preserve the container logs when the pod fails.
The pod logs show that it is timing out on an http connection, and it looks like it's the one where it tries to download the test data set for the tensorflow run. You can see it . Search down to . The data_download script in the image attempts to pull down a dataset from: . That url is live and healthy, so they might be blocking the gcloud.
This test depending on that external data doesn't look quite right :/
I'm discussing this test with sig-testing. I'll post an update later.
Talked to from sig-testing: he's contacting the remote to see if they are blocking us, because it looks like they are is pulling the script from a that is So I see three choices: upgrade this thing to a more recent edition delete it grab everything that's needed to run this and keep it in-house <- clear winner here what do you think: Maybe the folks over at sig-testing have a place to store the artifacts in-house? They are not massive (about 10MB see below) Size of data about 10MB: the tools and libs for the tensorflow run: the training and eval files for the testrun:
Here's a relevant issue in a related k8s repo about using/not using docker registries: Registries used in k/k CI should be on Kubernetes Community infra
looks good for the subject of putting up a new docker image containing the deps. This seems feasible and effective. I'd prefer it to deleting the test. I'll discuss it a bit more with a few others, but this seems like the way to go forward with this. TBS, I plan to take the above listed test requirements, and build a docker image containing all that is needed, and reset the test to pull the new docker image.
I've got some Dockerfile changes to fix this. I've tested it locally with no network and it does the full 300 epochs. PR coming soon.
Is it worth pushing up 182 files to get this green? It is about 6M of python, and 5M of a test dataset.
Code change:
<del> 1.2 </del> <ins> 1.3 </ins>

Related discussion:
text: Its all stashed away in the to get the docker image to contain the prereqs. It is selected subdirectories of the quite old archived version (v1.9.0) of the code from here: current version v2.7.2 and two small test data sets from here:
OK I got it down to 43 files, including the 2 datasets. That's the drivers and the datasets and the libs they need. It all gets packed into the docker image.
quick question - is there another repository that is under sig-node that can host this image?
To be clear, the test's docker image is currently in a repo, the : This PR adds artifacts to an existing docker image in the above repo.
I've cancelled the above PR. I'll try to get something else going.
There has been a decent amount of discussion in slack relating to handling the test's required artifacts. The solution that seems best is to get some practical version of the test requirements into locally owned and controlled storage, so the endpoints that provide access to the assets will at least be under internal control. This is not code, and is not under development. It is a static dataset, that is now stored in an offsite archive, and will not require maintenance. The datasets are published with the "": (see bottom right) "This allows for the sharing and adaptation of the datasets for any purpose, provided that the appropriate credit is given." Here are some links to the dataset and where its hosted: https://archive-
I'd like to get the dataset that holds the test data (about 650k) hosted in GCS where I can get it via http. With that change it is expected that the test will go green. The failures show it is failing when it tries to download the artifact from the uncontrolled remote endpoint. Somebody, please help me get this artifact into an internally controlled server. Thanks
Was the idea of downloading this only on image creation in dockerfile rejected?
Thanks for the reminder I did not fairly evaluate that proposal. I just jumped to my earlier proposal. Thinking about this proposal I see that it would mean downloading the test data for inclusion into the docker image, while the image's test driver tooling is built to pull down the dataset itself from a remote.
Code change:
<del> 1.2 </del> <ins> 1.3 </ins>

Related discussion:
text: That is a simple change to handle, while simply re-hosting the artifact on GCS would require only changing the endpoint. Also building the docker would probably still require network access to the artifact, which would probably require us to put it somewhere anyway. I assume, putting it into the docker image would best be done by downloading the test data artifact from somewhere. Would we then be storing the artifact twice - once in the docker image, and once in a locally controlled endpoint? If the docker build server has unrestricted access to the artifact, this proposal would be good. I don't have access to the servers to run tests. However, I could put up an RB for the Dockerfile to try it out. This is probably worth the effort. I'll get an PR going for trying to pull down the remote test-data artifact from the image build server, and bunde it into the docker image, and I'll link it to this ticket.
thank you. We do not change the image for a long time, so downloading it once would likely be enough. Once it is self-contained, it is even better for the continuous perf runs in tests.
/reopen
You can't reopen an issue/PR unless you authored it or you are a collaborator. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Currently the image builds for the first arch (amd64) but fails when it enters the cross compiling phase (arm64) and specifically when it tries to do apt installs from within the qemu-*-static emulator: The other images in node-perf fail to build in the same way. The jessie-dnsutils image build fails to build in the same way. I have tried pulling in newer base images up to buster, but it did not fix the problem. -Start focusing on the version of qemu. - qemu is version locked in the Makefile and has not changed in two years. There was an and the images were building at that time.
Code change:
<del> 1.2 </del> <ins> 1.3 </ins>

Related discussion:
text: Try to find clues over there.
I tried pinning many relevant versions of from but none helped. Here's a clue: This ugly hack gets it past the observed error:
Note the previous PR was missing an important change, but I do not see any feasible way that could explain the current problem with failing, so I have not felt any urgency to put up a PR for it. FTR I need to bump the VERSION. I did not bump the version in I thought maybe the CI would bump it automatically.
There's an image version in . Its still pulling the old version and failing. Bump it from 1.2 to 1.3. So one more commit is needed.
The new image is present and runs perfectly. Tested it on local linux and in google cloud: It ran to completion. Note the new image URL. Its NOT the same as the old one:
podspec references the image Here's the . tf-wide-deep's url is built The above has staging in it. . Its already built and pushed to staging. It must be promoted when this new version number goes out.
FTR: My "testing" above did not catch the current problem because I did not run the full e2e_test binary. I only tested the image via a . AND I should have read way back at the beginning.
Local run of : has to change too.
Local test using the published 1.3 image run the standard way was successful with the above: ran the test successfully. So promote the image per bump the image version in the e2e test manifest change the
After the image promotion is merged, there's STILL one more commit coming: bump the image version in the e2e test manifest change the
its promoted and is found at the expected url local containerd cleared out local test was rerun pulling from the new prod registry local test using that new registry passes
its fixed. the test is green again.
Code change:
return ConformanceContainer{containers[0], cc.Client, pod.Spec.RestartPolicy, pod.Spec.Volumes, pod.Spec.NodeName, cc.Namespace, cc.podName}, nil } <del> func (cc *ConformanceContainer) GetStatus() (api.ContainerStatus, api.PodPhase, error) { </del> <ins> func (cc *ConformanceContainer) IsReady() (bool, error) { </ins> pod, err := cc.Client.Pods(cc.Namespace).Get(cc.podName) if err != nil { <del> return api.ContainerStatus{}, api.PodUnknown, err </del> <ins> return false, err </ins> } <ins> return api.IsPodReady(pod), nil } func (cc *ConformanceContainer) GetPhase() (api.PodPhase, error) { pod, err := cc.Client.Pods(cc.Namespace).Get(cc.podName) if err != nil { return api.PodUnknown, err } return pod.Status.Phase, nil } func (cc *ConformanceContainer) GetStatus() (api.ContainerStatus, error) { pod, err := cc.Client.Pods(cc.Namespace).Get(cc.podName) if err != nil { return api.ContainerStatus{}, err } </ins> statuses := pod.Status.ContainerStatuses <del> if len(statuses) != 1 { return api.ContainerStatus{}, api.PodUnknown, errors.New("Failed to get container status") </del> <ins> if len(statuses) != 1 || statuses[0].Name != cc.Container.Name { return api.ContainerStatus{}, fmt.Errorf("unexpected container statuses %v", statuses) </ins> } <del> return statuses[0], pod.Status.Phase, nil </del> <ins> return statuses[0], nil </ins> } func (cc *ConformanceContainer) Present() (bool, error) {

Related discussion:
text: Slow conformance test has caused issues like We should make runtime/image conformance test run faster without breaking the test coverage, possibly by: Shorten consistently check interval . Remove redundant test cases such as ... /cc