jiebi/CodeConvo huggingface dataset:
Change:
I noticed that there is a huge regression on node bootup since Wednesday last couple of days: Bad one: vs. good one: Please note that the issue is not related to any kubernetes component. The node hasn't run salt client yet. This explains a lot of node not ready issue we have lately. cc/
cc/
Whoa, it shouldn't even be running at all. That implies someone a package recently.
I didn't see any new package is , and the entire package size is the same. :-(
Can you dump /var/log/apt.history (I think it is) into here?
Sorry, . Hmm. I don't see anything new on a recent-ish head node, either, but YMMV.
Yes, just about to do it ... Here you are, and please note that it is from a different node with the same bad symptom:
Yeah, no issue there. Oh. Hm. Sorry, I forgot the one directly in .
Wait. The one in configure-vm is new, it's from , and we need to take it out.
Oh no - so sorry :-( does add an apt-get update to configure-vm. But you are installing bridge-utils from apt, so I thought it was correct to apt-get update first. I can easily remove the apt-get-update and add it back into the AWS version instead. In my experience, crazy-slow apt-get problems are either due to an IPv6 address being served to an IPv4 only host, or getting a bad mirror. Is there a GCE internal mirror? Could also be the gluster repo as well - not necessarily the debian repo.
Probably the bestest thing would be to make sure bridge-utils is on the base image, only apt-get-install bridge-utils if brctl is not available, and run apt-get-update as part of apt-get-install only once per script invocation. But this isn't actually fixing the problem. Something is seriously wrong if an apt-get update takes 12 minutes.
is actually built-in on (the later is a no-op, sorry). I went through an effort at some point to make sure that never needed to or anything, because we were having troubles like this.
I can work up a PR that will probably work for all parties.
Yes, that is the root cause and we want to remove all external dependencies at bootup time. Assign this to
Thanks & sorry again to all.

Discussion:
} apt-get-install() { <ins> local -r packages=( $@ ) installed=true for package in "${packages[@]}"; do if ! dpkg -s "${package}" &>/dev/null; then installed=false break fi done if [[ "${installed}" == "true" ]]; then echo "== ${packages[@]} already installed, skipped apt-get install ${packages[@]} ==" return fi apt-get-update </ins> # Forcibly install packages (options borrowed from Salt logs). until apt-get -q -y -o DPkg::Options::=--force-confold -o DPkg::Options::=--force-confdef install $@; do echo "== install of packages $@ failed, retrying =="

echo "== Refreshing package database ==" until apt-get update; do echo "== apt-get update failed, retrying ==" <del> echo sleep 5 </del> <ins> sleep 5 </ins> done }

set-broken-motd ensure-basic-networking fix-apt-sources <del> apt-get-update </del> ensure-install-dir ensure-packages set-kube-env
Change:
The default Kubernetes release number in cluster/ubuntu/download- is 1.1.7 but 1.1.8 is out now.
Resolved by PR

Discussion:
} # k8s <del> KUBE_VERSION=${KUBE_VERSION:-"1.1.7"} </del> <ins> KUBE_VERSION=${KUBE_VERSION:-"1.1.8"} </ins> echo "Prepare kubernetes ${KUBE_VERSION} release ..." grep -q "^${KUBE_VERSION}$" binaries/.kubernetes 2>/dev/null || { curl -L https://github.com/kubernetes/kubernetes/releases/download/v${KUBE_VERSION}/kubernetes.tar.gz -o kubernetes.tar.gz
Change:
GCE minion VM OS version is The nfs server image and hits with the following dump stack
Image version now exists!
thanks!

Discussion:
RUN yum -y install /usr/bin/ps nfs-utils && yum clean all RUN mkdir -p /exports ADD run_nfs.sh /usr/local/bin/ <del> ADD index.html /exports/index.html RUN chmod 644 /exports/index.html </del> <ins> ADD index.html /tmp/index.html RUN chmod 644 /tmp/index.html </ins> <del> EXPOSE 2049/tcp </del> <ins> # expose mountd 20048/tcp and nfsd 2049/tcp EXPOSE 2049/tcp 20048/tcp </ins> <del> ENTRYPOINT ["/usr/local/bin/run_nfs.sh", "/exports"] </del> <ins> ENTRYPOINT ["/usr/local/bin/run_nfs.sh", "/exports", "/"] </ins>

# See the License for the specific language governing permissions and # limitations under the License. <del> TAG = 0.5 </del> <ins> TAG = 0.6 </ins> PREFIX = gcr.io/google_containers all: push

for i in "$@"; do # fsid=0: needed for NFSv4 echo "$i *(rw,fsid=0,insecure,no_root_squash)" >> /etc/exports <ins> # move index.html to here /bin/cp /tmp/index.html $i/ chmod 644 $i/index.html </ins> echo "Serving $i" done <ins> # start rpcbind if it is not started yet /usr/sbin/rpcinfo 127.0.0.1 > /dev/null; s=$? if [ $s -ne 0 ]; then echo "Starting rpcbind" /usr/sbin/rpcbind -w fi </ins> mount -t nfsd nfds /proc/fs/nfsd <del> # -N 2 -N 3: disable NFSv2+3 # -V 4.x: enable NFSv4 /usr/sbin/rpc.mountd -N 2 -N 3 -V 4 -V 4.1 </del> <ins> # -N 4.x: disable NFSv4 # -V 3: enable NFSv3 /usr/sbin/rpc.mountd -N 2 -V 3 -N 4 -N 4.1 </ins> /usr/sbin/exportfs -r # -G 10 to reduce grace time to 10 seconds (the lowest allowed) <del> /usr/sbin/rpc.nfsd -G 10 -N 2 -N 3 -V 4 -V 4.1 2 </del> <ins> /usr/sbin/rpc.nfsd -G 10 -N 2 -V 3 -N 4 -N 4.1 2 /usr/sbin/rpc.statd --no-notify </ins> echo "NFS started" }
Change:
Master failed to restart on Debian Jessie. Console logs showed: I have no idea where nobootwait is coming from though...
Aha... it's a bug in jessie's cloud-init. I have a fix, will produce a PR once verified.
Fix in

Discussion:
aws_account="721322707521" # TODO: we could use tags for the image if [[ -z "${AWS_IMAGE_NAME:-}" ]]; then <del> AWS_IMAGE_NAME="k8s-1.2-debian-jessie-amd64-hvm-2016-02-24-ebs" </del> <ins> AWS_IMAGE_NAME="k8s-1.2-debian-jessie-amd64-hvm-2016-03-05-ebs" </ins> fi AWS_IMAGE=`aws ec2 describe-images --owner ${aws_account} --filters Name=name,Values=${AWS_IMAGE_NAME} --query Images[].ImageId --output text` if [[ -z "${AWS_IMAGE-}" ]]; then

- [ 'chroot', '{root}', '/bin/sh', '-c', 'DEBIAN_FRONTEND=noninteractive dpkg --install /tmp/docker.deb' ] - [ 'rm', '{root}/tmp/docker.deb' ] <ins> # Fix a cloud-init bug where it uses nobootwait # see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=789884 - [ 'chroot', '{root}', '/bin/sh', '-c', 'echo "mount_default_fields: [~, ~, ''auto'', ''defaults,nofail'', ''0'', ''2'']" > /etc/cloud/cloud.cfg.d/99_kubernetes.cfg' ] </ins> # We perform a full replacement of some grub conf variables: #   GRUB_CMDLINE_LINUX_DEFAULT (add memory cgroup) #   GRUB_TIMEOUT (remove boot delay)
Change:
We should tag EBS volumes created by k8s with our tags, just like we (should) tag everything else we create/own.
Is there any documentation yet on automatic EBS volume creation?
from a user perspective, they are not AWS specific. That said, I don't know that we have clear user-facing docs. Here's an example though, from my (very) WIP Ubernetes-Lite docs: ... The PV is also tagged with the zone & region is is in (us-central1-a): Create a pod that uses the PVC:
It doesn't seem to cleanup the volume when the PV is deleted... probably intended behavior. Any way to tell it to automatically delete the volume on PV delete?
- I'm not sure about foo - hopefully that'll be resolved with docs. I think the failure to delete the volume is actually pretty important. Do you want to open an issue? It's not AWS specific (and this particular issue was auto-closed when the PR merged)
yeah I will open an issue later today. Issue seems to still be happening - the PV is never deleted. On Sat, Mar 12, 2016, 6:24 AM Justin Santa Barbara wrote:

Discussion:
type VolumeOptions struct { CapacityGB int <del> Tags       *map[string]string </del> <ins> Tags       map[string]string </ins> } // Volumes is an interface for managing cloud-provisioned volumes

volumeName := "aws://" + az + "/" + awsID // apply tags <del> if volumeOptions.Tags != nil { if err := s.createTags(awsID, *volumeOptions.Tags); err != nil { </del> <ins> tags := make(map[string]string) for k, v := range volumeOptions.Tags { tags[k] = v } if s.getClusterName() != "" { tags[TagNameKubernetesCluster] = s.getClusterName() } if len(tags) != 0 { if err := s.createTags(awsID, tags); err != nil { </ins> // delete the volume and hope it succeeds _, delerr := s.DeleteDisk(volumeName) if delerr != nil {

requestGB := int(volume.RoundUpSize(requestBytes, 1024*1024*1024)) volumeOptions := &aws.VolumeOptions{ CapacityGB: requestGB, <del> Tags:       &tags, </del> <ins> Tags:       tags, </ins> } name, err := cloud.CreateDisk(volumeOptions)
Change:
could you please update Scale Subresource section? cc
To be honest I think that we shouldn't update design docs. Overtime we'd have to update too many of them. As a workaround I'd suggest to: a warning at the top that this design doc might be updated "last update" timestamp to the design doc.
I'd rather not have obsolete docs in the tree. We need to ensure that everything in this doc that's still relevant is covered and up to date somewhere. We definitely need a doc for the scale API. (And for the rest of the API ) The kubectl details needs to be covered in kubectl user docs. Other user-facing details need to be in an autoscaling guide and example in the user guide. We need documentation for the HPA controller-manager flags and other details, in the code. Ensure there are github issues for the incomplete roadmap items. cc
SGTM. I guess it's a policy decision so if we decide to update all design docs I'm fine with this :)
Yes, please file an issue that we should update proposals and design docs. Some of them should really be converted to user docs, too.
I don't think it's a good use of time to update design docs. Features morph infinitely over time; it's not practical to keep their original design doc up to date. I don't think people expect that a design doc will even reflect what was originally implemented, much less how the feature exists at the time they are reading the design doc. Now, I do agree "We need to ensure that everything in this doc that's still relevant is covered and up to date somewhere." When something is implemented, we should write proper user guide documentation for it, which we keep up-to-date indefinitely, I don't think anything contradicts this, except "I'd rather not have obsolete docs in the tree." I think there's no way around that with design docs (and proposals), and people should just understand that's how such docs work. We can put a in proposals/ and design/ to explain this.
AFAIU you're both saying different things here :) We have already updated user guides. Should we update the ?
Please hold off on doing it until I discuss with Brian. I definitely don't think this is a good policy.
Any update here?
I talked to about this just now. He was OK with just putting a big warning at the top of any design docs that contain out-of-date information, warning that the doc is just for historical purposes and pointing to the up-to-date docs. This assumes we actually document our APIs properly and write user-guide docs for the feature that we keep up-to-date. So we need to get better at that -- I agree with him that we should not have any features that are only documented in a design doc that is not up-to-date.
And I reserve the right to gut/delete out-of-date docs at some point.
SGTM. Can you please look into this?

Discussion:
<!-- END MUNGE: UNVERSIONED_WARNING --> <ins> <h2>Warning! This document might be outdated.</h2> </ins> # Horizontal Pod Autoscaling ## Preface
Change:
Event for a pod: The cni plugins do exist at /opt/cni/bin. They cannot be executed and bash is complaining:
cc
I'm guessing this is a shared lib problem, since /usr/grte/v4/lib64/ld-linux-x86- doesn't exist on my gce vm. We should update docs to make it clear how someone would install manually (if they run into such problems again) and statically link (if the problem is indeed the shared lib) with something like Unsure how we'd support different GOOS (vend different tarballs vs checking in and building the plugin inline with our make scripts)
uggh. These binaries need to be ultra-static or else we need to containerize them.
Essentially doing would've prevented this bug :)
And see for the CNI_ARGS problems
also , re ... however, I'm not too sure if your comment was mean to be on this ticket.
cc
I have a working version. I uploaded my cni binary with bug fix on GCS. change source to change sourcehash to Welcome to give it a try.
Also see problem:

Discussion:
- user: root - name: /opt/cni - makedirs: True <del> - source: https://storage.googleapis.com/kubernetes-release/network-plugins/cni-09214926.tar.gz </del> <ins> - source: https://storage.googleapis.com/kubernetes-release/network-plugins/cni-42c4cb842dad606a84e93aad5a4484ded48e3046.tar.gz </ins> - tar_options: v <del> - source_hash: md5=58f8631f912dd88be6a0920775db7274 </del> <ins> - source_hash: md5=8cee1d59f01a27e8c2c10c120dce1e7d </ins> - archive_format: tar - if_missing: /opt/cni/bin

cidr.IP.To4()[3] += 1 json := fmt.Sprintf(NET_CONFIG_TEMPLATE, BridgeName, plugin.MTU, network.DefaultInterfaceName, podCIDR, cidr.IP.String()) <ins> glog.V(2).Infof("CNI network config set to %v", json) </ins> plugin.netConfig, err = libcni.ConfFromBytes([]byte(json)) if err == nil { glog.V(5).Infof("CNI network config:n%s", json)

return fmt.Errorf("Error building CNI config: %v", err) } <ins> glog.V(3).Infof("Calling cni plugins to add container to network with cni runtime: %+v", rt) </ins> res, err := plugin.cniConfig.AddNetwork(plugin.netConfig, rt) if err != nil { return fmt.Errorf("Error adding container to network: %v", err)

} delete(plugin.podCIDRs, id) <ins> glog.V(3).Infof("Calling cni plugins to remove container from network with cni runtime: %+v", rt) </ins> if err := plugin.cniConfig.DelNetwork(plugin.netConfig, rt); err != nil { return fmt.Errorf("Error removing container from network: %v", err) }
Change:
reported hitting this error: The instance had 4 security groups, and only one of them was tagged. However, the logic around picking the security group is incorrect.. if we get two non-tagged SGs before we see the first tagged SG, it throws an error. We'll want to cherry-pick this to 1.2
I think this one was a victim of flakiness in the auto-assigner when the author is the first-listed maintainer. Assigning to you, :-)

Discussion:
// Otherwise we will return an error. func findSecurityGroupForInstance(instance *ec2.Instance, taggedSecurityGroups map[string]*ec2.SecurityGroup) (*ec2.GroupIdentifier, error) { instanceID := aws.StringValue(instance.InstanceId) <del> var best *ec2.GroupIdentifier </del> <ins> var tagged []*ec2.GroupIdentifier var untagged []*ec2.GroupIdentifier </ins> for _, group := range instance.SecurityGroups { groupID := aws.StringValue(group.GroupId) if groupID == "" { glog.Warningf("Ignoring security group without id for instance %q: %v", instanceID, group) continue } <del> if best == nil { best = group continue </del> <ins> _, isTagged := taggedSecurityGroups[groupID] if isTagged { tagged = append(tagged, group) } else { untagged = append(untagged, group) </ins> } <ins> } </ins> <del> _, bestIsTagged := taggedSecurityGroups[*best.GroupId] _, groupIsTagged := taggedSecurityGroups[groupID] if bestIsTagged && !groupIsTagged { // best is still best } else if groupIsTagged && !bestIsTagged { best = group } else { // We create instances with one SG // If users create multiple SGs, they must tag one of them as being k8s owned return nil, fmt.Errorf("Multiple security groups found for instance (%s); ensure the k8s security group is tagged", instanceID) </del> <ins> if len(tagged) > 0 { // We create instances with one SG // If users create multiple SGs, they must tag one of them as being k8s owned if len(tagged) != 1 { return nil, fmt.Errorf("Multiple tagged security groups found for instance %s; ensure only the k8s security group is tagged", instanceID) </ins> } <ins> return tagged[0], nil </ins> } <del> if best == nil { glog.Warningf("No security group found for instance %q", instanceID) </del> <ins> if len(untagged) > 0 { // For back-compat, we will allow a single untagged SG if len(untagged) != 1 { return nil, fmt.Errorf("Multiple untagged security groups found for instance %s; ensure the k8s security group is tagged", instanceID) } return untagged[0], nil </ins> } <del> return best, nil </del> <ins> glog.Warningf("No security group found for instance %q", instanceID) return nil, nil </ins> } // Return all the security groups that are tagged as being part of our cluster
Change:
I have an existing v1.2 cluster on AWS and wanted to utilize the multi-AZ functionality. After following , the new node seems to be correctly, but I received the error below after completed. It seems like the script is not aware of the total cluster size and throwing an error, even though the instance is registering correctly.
Happening to us also :+1: nothing seems broken however, looks like the up script is getting confused.

Discussion:
if (( "${found}" == "${EXPECTED_NUM_NODES}" )) && (( "${ready}" == "${EXPECTED_NUM_NODES}")); then break elif (( "${found}" > "${EXPECTED_NUM_NODES}" )); then <del> echo -e "${color_red}Found ${found} nodes, but expected ${EXPECTED_NUM_NODES}. Your cluster may not behave correctly.${color_norm}" </del> <ins> if [[ "${KUBE_USE_EXISTING_MASTER:-}" != "true" ]]; then echo -e "${color_red}Found ${found} nodes, but expected ${EXPECTED_NUM_NODES}. Your cluster may not behave correctly.${color_norm}" fi </ins> break elif (( "${ready}" > "${EXPECTED_NUM_NODES}")); then echo -e "${color_red}Found ${ready} ready nodes, but expected ${EXPECTED_NUM_NODES}. Your cluster may not behave correctly.${color_norm}"
Change:
I keep seeing these server rejected event errors every now and then. My guess is it is trying to update an event that was expired in etcd. However I don't expect an error, since there is specific logic in to handle that use case. Any ideas why this may happening? I am using kubernetes 1.2. Here is an example, from a host getting to a cluster:
Found the problem - the recordEvent logic in was expecting the wrong status code. I've submitted a PR for it. This could be reproduced by: 1 - Update load balancers by remove a node, then adding it back 2 - Remove all events manually from etcd for that namespace 3 - Remove node and add it back again - this results in server rejected event error

Discussion:
"k8s.io/kubernetes/pkg/watch" "github.com/golang/glog" <ins> "net/http" </ins> ) const maxTriesPerEvent = 12

func isKeyNotFoundError(err error) bool { statusErr, _ := err.(*errors.StatusError) <del> // At the moment the server is returning 500 instead of a more specific // error. When changing this remember that it should be backward compatible // with old api servers that may be still returning 500. if statusErr != nil && statusErr.Status().Code == 500 { </del> <ins> if statusErr != nil && statusErr.Status().Code == http.StatusNotFound { </ins> return true } return false }

"k8s.io/kubernetes/pkg/api" "k8s.io/kubernetes/pkg/api/errors" _ "k8s.io/kubernetes/pkg/api/install" // To register api.Pod used in tests below <ins> "k8s.io/kubernetes/pkg/api/unversioned" </ins> "k8s.io/kubernetes/pkg/client/restclient" k8sruntime "k8s.io/kubernetes/pkg/runtime" "k8s.io/kubernetes/pkg/util" "k8s.io/kubernetes/pkg/util/strategicpatch" <ins> "net/http" </ins> ) type testEventSink struct {

} } <ins> func TestUpdateExpiredEvent(t *testing.T) { eventCorrelator := NewEventCorrelator(util.RealClock{}) randGen := rand.New(rand.NewSource(time.Now().UnixNano())) var createdEvent *api.Event sink := &testEventSink{ OnPatch: func(*api.Event, []byte) (*api.Event, error) { return nil, &errors.StatusError{ ErrStatus: unversioned.Status{ Code:   http.StatusNotFound, Reason: unversioned.StatusReasonNotFound, }} }, OnCreate: func(event *api.Event) (*api.Event, error) { createdEvent = event return event, nil }, } ev := &api.Event{} ev.ResourceVersion = "updated-resource-version" ev.Count = 2 recordToSink(sink, ev, eventCorrelator, randGen, 0) if createdEvent == nil { t.Error("Event did not get created after patch failed") return } if createdEvent.ResourceVersion != "" { t.Errorf("Event did not have its resource version cleared, was %s", createdEvent.ResourceVersion) } } </ins> func TestLotsOfEvents(t *testing.T) { recorderCalled := make(chan struct{}) loggerCalled := make(chan struct{})
Change:
When working on , I found that doesn't work now. Now we rely on in container inspect result to map to host directory path (See ). However, the in inspect result has been deprecated from docker 1.8.x (See ). Even though there is still field in , it is empty in fact. The new field in is a set instead of map. (See ) We should use in container inspect result or in instead.

Discussion:
Eventually(terminateContainer.Present, retryTimeout, pollInterval).Should(BeFalse()) } }) <ins> It("should report termination message if TerminationMessagePath is set [Conformance]", func() { name := "termination-message-container" terminationMessage := "DONE" terminationMessagePath := "/dev/termination-log" c := ConformanceContainer{ Container: api.Container{ Image:   ImageRegistry[busyBoxImage], Name:    name, Command: []string{"/bin/sh", "-c"}, Args:    []string{fmt.Sprintf("/bin/echo -n %s > %s", terminationMessage, terminationMessagePath)}, TerminationMessagePath: terminationMessagePath, }, Client:        f.Client, RestartPolicy: api.RestartPolicyNever, NodeName:      *nodeName, Namespace:     f.Namespace.Name, } By("create the container") Expect(c.Create()).To(Succeed()) defer c.Delete() By("wait for the container to succeed") Eventually(c.GetPhase, retryTimeout, pollInterval).Should(Equal(api.PodSucceeded)) By("get the container status") status, err := c.GetStatus() Expect(err).NotTo(HaveOccurred()) By("the container should be terminated") Expect(GetContainerState(status.State)).To(Equal(ContainerStateTerminated)) By("the termination message should be set") Expect(status.State.Terminated.Message).Should(Equal(terminationMessage)) By("delete the container") Expect(c.Delete()).To(Succeed()) }) </ins> }) Context("when running a container with a new image", func() {
Change:
After starting a container, Kubernetes places a symlink to the container's log file in . Unfortunately, rather than querying Docker for the , . This means that any change upstream in the path or filename that Docker uses to store its logs (e.g. to so that a file name transition would be a workable solution for docker/docker) would result in broken symlinks being put in place. It would be a better idea to query for the , potentially by doing something like this: This would allow Docker to put the log files anywhere and have symlinks put in the right place.
I will start work on this one, PR coming soon.

Discussion:
} } <ins> // Container information is used in adjusting OOM scores, adding ndots and getting the logPath. containerInfo, err := dm.client.InspectContainer(id.ID) if err != nil { return kubecontainer.ContainerID{}, fmt.Errorf("InspectContainer: %v", err) } </ins> // Create a symbolic link to the Docker container log file using a name which captures the // full pod name, the container name and the Docker container ID. Cluster level logging will // capture these symbolic filenames which can be used for search terms in Elasticsearch or for // labels for Cloud Logging. <del> containerLogFile := path.Join(dm.dockerRoot, "containers", id.ID, fmt.Sprintf("%s-json.log", id.ID)) </del> <ins> containerLogFile := containerInfo.LogPath </ins> symlinkFile := LogSymlink(dm.containerLogsDir, kubecontainer.GetPodFullName(pod), container.Name, id.ID) if err = dm.os.Symlink(containerLogFile, symlinkFile); err != nil { glog.Errorf("Failed to create symbolic link to the log file of pod %q container %q: %v", format.Pod(pod), container.Name, err) } <del> // Container information is used in adjusting OOM scores and adding ndots. containerInfo, err := dm.client.InspectContainer(id.ID) if err != nil { return kubecontainer.ContainerID{}, fmt.Errorf("InspectContainer: %v", err) } </del> // Check if current docker version is higher than 1.10. Otherwise, we have to apply OOMScoreAdj instead of using docker API. // TODO: Remove this logic after we stop supporting docker version < 1.10. if err = dm.applyOOMScoreAdjIfNeeded(pod, container, containerInfo); err != nil {