Change:
As of (specifically ), running against GCE without setting a node image defaults it to containervm, whereas before it defaulted it to the master image. For example, my config of: brings up a coreos master and then launches nodes with images. Before the above-referenced change this gave me my expected coreos nodes. Was this change intentional? The easiest fix that comes to mind for me is only doing the new containervm defaulting if is gci. cc
Yeah, this was something I was concerned too while making the change. However, the decision seems to have been that we decided to default to GCI, yet still default to the debian based containervm. Also, if we are to seriously start supporting hybrid clusters, i.e., master and nodes run on different OS distros, which seems true according to , this old assumption of defaults to needs to disappear anyways. cc/ Thoughts?
That sounds reasonable to me. We clearly didn't think about your use case whilst making this change.
I don't see the issue with defaulting to non-hybrid while supporting hybrid.. Defaulting seems totally reasonable even in a world where hybrid support is possible. A user could explicitly control that they want hybrid via setting them to different values regardless of the defaulting, or wherever hybrid is toggled on different defaulting could be done. I left a few questions on the referenced original issue which, if/when answered, might help me in making a PR to fix this for my usecase more cleanly.
Agreed. I am making a fix to this.

Discussion:
PREEMPTIBLE_NODE=${PREEMPTIBLE_NODE:-false} PREEMPTIBLE_MASTER=${PREEMPTIBLE_MASTER:-false} <del> # Today the cluster startup scripts asssume the master and the nodes use the # same OS distro, thus same set of initialization instructions (e.g., metadata, # startup scripts, etc.). The current workaround is the hack in util.sh that # reloads <os_distro>/helper.sh in the gap between when the master is created # and when the nodes are created. </del> # TODO(#26183): Provide a way to differentiate master OS distro and node OS # distro. OS_DISTRIBUTION=${KUBE_OS_DISTRIBUTION:-gci} <del> # For GCI, leaving it blank will auto-select the latest GCI image on the `dev` # channel. </del> <ins> # By default a cluster will be started with the master on GCI and nodes on # containervm. If you are updating the containervm version, update this # variable. CVM_VERSION=container-v1-3-v20160517 </ins> MASTER_IMAGE=${KUBE_GCE_MASTER_IMAGE:-} MASTER_IMAGE_PROJECT=${KUBE_GCE_MASTER_PROJECT:-google-containers} <del> NODE_IMAGE=${KUBE_GCE_NODE_IMAGE:-container-v1-3-v20160517} NODE_IMAGE_PROJECT=${KUBE_GCE_NODE_PROJECT:-google-containers} </del> <ins> NODE_IMAGE=${KUBE_GCE_NODE_IMAGE:-"${MASTER_IMAGE}"} NODE_IMAGE_PROJECT=${KUBE_GCE_NODE_PROJECT:-"${MASTER_IMAGE_PROJECT}"} </ins> CONTAINER_RUNTIME=${KUBE_CONTAINER_RUNTIME:-docker} RKT_VERSION=${KUBE_RKT_VERSION:-0.5.5}

PREEMPTIBLE_NODE=${PREEMPTIBLE_NODE:-false} PREEMPTIBLE_MASTER=${PREEMPTIBLE_MASTER:-false} <ins> # TODO(#26183): Provide a way to differentiate master OS distro and node OS # distro. </ins> OS_DISTRIBUTION=${KUBE_OS_DISTRIBUTION:-gci} <del> # For GCI, leaving it blank will auto-select the latest GCI image on the `dev` # channel. </del> <ins> # By default a cluster will be started with the master on GCI and nodes on # containervm. If you are updating the containervm version, update this # variable. CVM_VERSION=container-v1-3-v20160517 </ins> MASTER_IMAGE=${KUBE_GCE_MASTER_IMAGE:-} MASTER_IMAGE_PROJECT=${KUBE_GCE_MASTER_PROJECT:-google-containers} <del> NODE_IMAGE=${KUBE_GCE_NODE_IMAGE:-container-v1-3-v20160517} NODE_IMAGE_PROJECT=${KUBE_GCE_NODE_PROJECT:-google-containers} </del> <ins> NODE_IMAGE=${KUBE_GCE_NODE_IMAGE:-"${MASTER_IMAGE}"} NODE_IMAGE_PROJECT=${KUBE_GCE_NODE_PROJECT:-"${MASTER_IMAGE_PROJECT}"} </ins> CONTAINER_RUNTIME=${KUBE_CONTAINER_RUNTIME:-docker} RKT_VERSION=${KUBE_RKT_VERSION:-0.5.5}

fi if [[ "${OS_DISTRIBUTION}" == "gci" ]]; then <del> # If the master or node image is not set, we use the latest GCI dev image. # Otherwise, we respect whatever set by the user. </del> <ins> # If the master image is not set, we use the latest GCI dev image. # Otherwise, we respect whatever is set by the user. </ins> gci_images=( $(gcloud compute images list --project google-containers  --regexp='gci-dev.*' --format='value(name)') ) <del> if [[ -z "${MASTER_IMAGE:-}" ]]; then MASTER_IMAGE="${gci_images[0]}" MASTER_IMAGE_PROJECT="google-containers" fi if [[ -z "${NODE_IMAGE:-}" ]]; then NODE_IMAGE="${gci_images[0]}" NODE_IMAGE_PROJECT="google-containers" fi </del> <ins> MASTER_IMAGE=${KUBE_GCE_MASTER_IMAGE:-"${gci_images[0]}"} MASTER_IMAGE_PROJECT=${KUBE_GCE_MASTER_PROJECT:-google-containers} # The default node image when using GCI is still the Debian based ContainerVM # until GCI gets validated for node usage. NODE_IMAGE=${KUBE_GCE_NODE_IMAGE:-${CVM_VERSION}} NODE_IMAGE_PROJECT=${KUBE_GCE_NODE_PROJECT:-google-containers} </ins> fi # Verfiy cluster autoscaler configuration.

cluster/aws/templates/configure-vm-aws.sh:  kubelet_api_servers: '${KUBELET_APISERVER}' cluster/centos/util.sh:  local node_ip=${node#*@} cluster/gce/config-default.sh:# reloads <os_distro>/helper.sh in the gap between when the master is created <ins> cluster/gce/config-test.sh:# reloads <os_distro>/helper.sh in the gap between when the master is created </ins> cluster/gce/configure-vm.sh:  advertise_address: '${EXTERNAL_IP}' cluster/gce/configure-vm.sh:  api_servers: '${KUBERNETES_MASTER_NAME}' cluster/gce/configure-vm.sh:  cloud_config: ${CLOUD_CONFIG}