Change:
What happened: Upgrade using kubeadm failed because of CoreDNS migration enchancement What you expected to happen: Upgrade passed correctly How to reproduce it (as minimally and precisely as possible): any 1.15.x Kubernetes with CoreDNS that deployed with any unusual image (self built for example). We using commits counter as part of image tag, e.g - 1.6.2-100. to upgrade to 1.16.x with kubeadm. Anything else we need to know?: Logs part without ignoring preflight: Logs part with ignoring preflight: Environment: Kubernetes version (use ):Cloud provider or hardware configuration: OpenStack OS (e.g: ):Kernel (e.g. ):Install tools: Kubeadm
/sig cluster-lifecycle
The error came from It seems you should mention this issue on coredns project
True. Will notify them also.
/cc
/assign In such cases where the migration isn't supported and the user chooses to ignore the warnings from the preflight check, the migration should be skipped. wdyt? /cc
Yes. IMO, the warnings should be phrased like "CoreDNS configuration migration will not occur: reason"... so that a user expects the config migration will not be done.
/area kubeadm
SGTM
Upgrade passed correctly I think the best we can do here is to allow the upgrade to pass, but this will, in most practical cases, result in a cluster without a functioning DNS. The most common practical use-case for using a self built image of CoreDNS is to use external plugins. During the kubeadm upgrade, CoreDNS will be replaced with a stock release version of CoreDNS. When that new version of CoreDNS starts, it will not be able to load the existing Corefile's external plugins, and exit. ... or, we could finish the upgrade, leaving the existing CoreDNS deployment untouched, which I think works better.
Yea, it's kinda strong words that "Upgrade passed correctly". Probably, I think the best solution will be to make possible skip Corefile migration and make kubeadm finish the upgrade without Corefile changes. Guys that using custom images of CoreDNS should care about Corefile for new version by themselves, IMO.
Yes, I think that would work best...

Discussion:
if err != nil { return errors.Wrap(err, "unable to fetch CoreDNS current installed version and ConfigMap.") } <ins> var hasCoreDNSMigrationFailed bool </ins> if IsCoreDNSConfigMapMigrationRequired(corefile) { if err := migrateCoreDNSCorefile(client, coreDNSConfigMap, corefile, currentInstalledCoreDNSVersion); err != nil { <del> return err </del> <ins> // Errors in Corefile Migration is verified during preflight checks. This part will be executed when a user has chosen // to ignore preflight check errors. hasCoreDNSMigrationFailed = true klog.Warningf("the CoreDNS Configuration was not migrated: %v. The existing CoreDNS Corefile configuration has been retained.", err) if err := apiclient.CreateOrRetainConfigMap(client, coreDNSConfigMap, kubeadmconstants.CoreDNSConfigMap); err != nil { return err } </ins> } } else { if err := apiclient.CreateOrUpdateConfigMap(client, coreDNSConfigMap); err != nil {

return errors.Wrapf(err, "%s Deployment", unableToDecodeCoreDNS) } <del> // Create the Deployment for CoreDNS or update it in case it already exists if err := apiclient.CreateOrUpdateDeployment(client, coreDNSDeployment); err != nil { return err </del> <ins> // Create the deployment for CoreDNS or retain it in case the CoreDNS migration has failed during upgrade if hasCoreDNSMigrationFailed { if err := apiclient.CreateOrRetainDeployment(client, coreDNSDeployment, kubeadmconstants.CoreDNSDeploymentName); err != nil { return err } } else { // Create the Deployment for CoreDNS or update it in case it already exists if err := apiclient.CreateOrUpdateDeployment(client, coreDNSDeployment); err != nil { return err } </ins> } coreDNSService := &v1.Service{}

_, err = migration.Migrate(currentInstalledCoreDNSversion, kubeadmconstants.CoreDNSVersion, corefile, false) if err != nil { <del> return err </del> <ins> return errors.Wrap(err, "CoreDNS will not be upgraded") </ins> } return nil }

return nil } <ins> // CreateOrRetainDeployment creates a Deployment if the target resource doesn't exist. If the resource exists already, this function will retain the resource instead. func CreateOrRetainDeployment(client clientset.Interface, deploy *apps.Deployment, deployName string) error { if _, err := client.AppsV1().Deployments(deploy.ObjectMeta.Namespace).Get(deployName, metav1.GetOptions{}); err != nil { if !apierrors.IsNotFound(err) { return nil } if _, err := client.AppsV1().Deployments(deploy.ObjectMeta.Namespace).Create(deploy); err != nil { if !apierrors.IsAlreadyExists(err) { return errors.Wrap(err, "unable to create deployment") } } } return nil } </ins> // CreateOrUpdateDaemonSet creates a DaemonSet if the target resource doesn't exist. If the resource exists already, this function will update the resource instead. func CreateOrUpdateDaemonSet(client clientset.Interface, ds *apps.DaemonSet) error { if _, err := client.AppsV1().DaemonSets(ds.ObjectMeta.Namespace).Create(ds); err != nil {