# Declarative Management of Kubernetes Objects Using Kustomize

Declarative Management of Kubernetes Objects Using KustomizeKustomize is a standalone tool to customize Kubernetes objects through a kustomization file.Since 1.14, kubectl also supports the management of Kubernetes objects using a kustomization file. To view resources found in a directory containing a kustomization file, run the following command:kubectl kustomize <kustomization_directory> To apply those resources, run kubectl apply with --kustomize or -k flag:kubectl apply -k <kustomization_directory> Before you beginInstall kubectl.You need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. It is recommended to run this tutorial on a cluster with at least two nodes that are not acting as control plane hosts. If you do not already have a cluster, you can create one by using minikube or you can use one of these Kubernetes playgrounds:iximiuz LabsKillercodaKodeKloudPlay with KubernetesTo check the version, enter kubectl version.Overview of KustomizeKustomize is a tool for customizing Kubernetes configurations. It has the following features to manage application configuration files:generating resources from other sourcessetting cross-cutting fields for resourcescomposing and customizing collections of resourcesGenerating ResourcesConfigMaps and Secrets hold configuration or sensitive data that are used by other Kubernetes objects, such as Pods. The source of truth of ConfigMaps or Secrets are usually external to a cluster, such as a .properties file or an SSH keyfile. Kustomize has secretGenerator and configMapGenerator, which generate Secret and ConfigMap from files or literals.configMapGeneratorTo generate a ConfigMap from a file, add an entry to the files list in configMapGenerator. Here is an example of generating a ConfigMap with a data item from a .properties file:# Create a application.properties file cat <<EOF >application.properties FOO=Bar EOF cat <<EOF >./kustomization.yaml configMapGenerator: - name: example-configmap-1 files: - application.properties EOF The generated ConfigMap can be examined with the following command:kubectl kustomize ./ The generated ConfigMap is:apiVersion: v1 data: application.properties: | FOO=Bar kind: ConfigMap metadata: name: example-configmap-1-8mbdf7882g To generate a ConfigMap from an env file, add an entry to the envs list in configMapGenerator. Here is an example of generating a ConfigMap with a data item from a .env file:# Create a .env file cat <<EOF >.env FOO=Bar EOF cat <<EOF >./kustomization.yaml configMapGenerator: - name: example-configmap-1 envs: - .env EOF The generated ConfigMap can be examined with the following command:kubectl kustomize ./ The generated ConfigMap is:apiVersion: v1 data: FOO: Bar kind: ConfigMap metadata: name: example-configmap-1-42cfbf598f Note:Each variable in the .env file becomes a separate key in the ConfigMap that you generate. This is different from the previous example which embeds a file named application.properties (and all its entries) as the value for a single key.ConfigMaps can also be generated from literal key-value pairs. To generate a ConfigMap from a literal key-value pair, add an entry to the literals list in configMapGenerator. Here is an example of generating a ConfigMap with a data item from a key-value pair:cat <<EOF >./kustomization.yaml configMapGenerator: - name: example-configmap-2 literals: - FOO=Bar EOF The generated ConfigMap can be checked by the following command:kubectl kustomize ./ The generated ConfigMap is:apiVersion: v1 data: FOO: Bar kind: ConfigMap metadata: name: example-configmap-2-g2hdhfc6tk To use a generated ConfigMap in a Deployment, reference it by the name of the configMapGenerator. Kustomize will automatically replace this name with the generated name.This is an example deployment that uses a generated ConfigMap:# Create an application.properties file cat <<EOF >application.properties FOO=Bar EOF cat <<EOF >deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: my-app labels: app: my-app spec: selector: matchLabels: app: my-app template: metadata: labels: app: my-app spec: containers: - name: app image: my-app volumeMounts: - name: config mountPath: /config volumes: - name: config configMap: name: example-configmap-1 EOF cat <<EOF >./kustomization.yaml resources: - deployment.yaml configMapGenerator: - name: example-configmap-1 files: - application.properties EOF Generate the ConfigMap and Deployment:kubectl kustomize ./ The generated Deployment will refer to the generated ConfigMap by name:apiVersion: v1 data: application.properties: | FOO=Bar kind: ConfigMap metadata: name: example-configmap-1-g4hk9g2ff8 --- apiVersion: apps/v1 kind: Deployment metadata: labels: app: my-app name: my-app spec: selector: matchLabels: app: my-app template: metadata: labels: app: my-app spec: containers: - image: my-app name: app volumeMounts: - mountPath: /config name: config volumes: - configMap: name: example-configmap-1-g4hk9g2ff8 name: config secretGeneratorYou can generate Secrets from files or literal key-value pairs. To generate a Secret from a file, add an entry to the files list in secretGenerator. Here is an example of generating a Secret with a data item from a file:# Create a password.txt file cat <<EOF >./password.txt username=admin password=secret EOF cat <<EOF >./kustomization.yaml secretGenerator: - name: example-secret-1 files: - password.txt EOF The generated Secret is as follows:apiVersion: v1 data: password.txt: dXNlcm5hbWU9YWRtaW4KcGFzc3dvcmQ9c2VjcmV0Cg== kind: Secret metadata: name: example-secret-1-t2kt65hgtb type: Opaque To generate a Secret from a literal key-value pair, add an entry to literals list in secretGenerator. Here is an example of generating a Secret with a data item from a key-value pair:cat <<EOF >./kustomization.yaml secretGenerator: - name: example-secret-2 literals: - username=admin - password=secret EOF The generated Secret is as follows:apiVersion: v1 data: password: c2VjcmV0 username: YWRtaW4= kind: Secret metadata: name: example-secret-2-t52t6g96d8 type: Opaque Like ConfigMaps, generated Secrets can be used in Deployments by referring to the name of the secretGenerator:# Create a password.txt file cat <<EOF >./password.txt username=admin password=secret EOF cat <<EOF >deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: my-app labels: app: my-app spec: selector: matchLabels: app: my-app template: metadata: labels: app: my-app spec: containers: - name: app image: my-app volumeMounts: - name: password mountPath: /secrets volumes: - name: password secret: secretName: example-secret-1 EOF cat <<EOF >./kustomization.yaml resources: - deployment.yaml secretGenerator: - name: example-secret-1 files: - password.txt EOF generatorOptionsThe generated ConfigMaps and Secrets have a content hash suffix appended. This ensures that a new ConfigMap or Secret is generated when the contents are changed. To disable the behavior of appending a suffix, one can use generatorOptions. Besides that, it is also possible to specify cross-cutting options for generated ConfigMaps and Secrets.cat <<EOF >./kustomization.yaml configMapGenerator: - name: example-configmap-3 literals: - FOO=Bar generatorOptions: disableNameSuffixHash: true labels: type: generated annotations: note: generated EOF Runkubectl kustomize ./ to view the generated ConfigMap:apiVersion: v1 data: FOO: Bar kind: ConfigMap metadata: annotations: note: generated labels: type: generated name: example-configmap-3 Setting cross-cutting fieldsIt is quite common to set cross-cutting fields for all Kubernetes resources in a project. Some use cases for setting cross-cutting fields:setting the same namespace for all resourcesadding the same name prefix or suffixadding the same set of labelsadding the same set of annotationsHere is an example:# Create a deployment.yaml cat <<EOF >./deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx EOF cat <<EOF >./kustomization.yaml namespace: my-namespace namePrefix: dev- nameSuffix: "-001" labels: - pairs: app: bingo includeSelectors: true commonAnnotations: oncallPager: 800-555-1212 resources: - deployment.yaml EOF Run kubectl kustomize ./ to view those fields are all set in the Deployment Resource:apiVersion: apps/v1 kind: Deployment metadata: annotations: oncallPager: 800-555-1212 labels: app: bingo name: dev-nginx-deployment-001 namespace: my-namespace spec: selector: matchLabels: app: bingo template: metadata: annotations: oncallPager: 800-555-1212 labels: app: bingo spec: containers: - image: nginx name: nginx Composing and Customizing ResourcesIt is common to compose a set of resources in a project and manage them inside the same file or directory. Kustomize offers composing resources from different files and applying patches or other customization to them.ComposingKustomize supports composition of different resources. The resources field, in the kustomization.yaml file, defines the list of resources to include in a configuration. Set the path to a resource's configuration file in the resources list. Here is an example of an NGINX application comprised of a Deployment and a Service:# Create a deployment.yaml file cat <<EOF > deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: my-nginx spec: selector: matchLabels: run: my-nginx replicas: 2 template: metadata: labels: run: my-nginx spec: containers: - name: my-nginx image: nginx ports: - containerPort: 80 EOF # Create a service.yaml file cat <<EOF > service.yaml apiVersion: v1 kind: Service metadata: name: my-nginx labels: run: my-nginx spec: ports: - port: 80 protocol: TCP selector: run: my-nginx EOF # Create a kustomization.yaml composing them cat <<EOF >./kustomization.yaml resources: - deployment.yaml - service.yaml EOF The resources from kubectl kustomize ./ contain both the Deployment and the Service objects.CustomizingPatches can be used to apply different customizations to resources. Kustomize supports different patching mechanisms through StrategicMerge and Json6902 using the patches field. patches may be a file or an inline string, targeting a single or multiple resources.The patches field contains a list of patches applied in the order they are specified. The patch target selects resources by group, version, kind, name, namespace, labelSelector and annotationSelector.Small patches that do one thing are recommended. For example, create one patch for increasing the deployment replica number and another patch for setting the memory limit. The target resource is matched using group, version, kind, and name fields from the patch file.# Create a deployment.yaml file cat <<EOF > deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: my-nginx spec: selector: matchLabels: run: my-nginx replicas: 2 template: metadata: labels: run: my-nginx spec: containers: - name: my-nginx image: nginx ports: - containerPort: 80 EOF # Create a patch increase_replicas.yaml cat <<EOF > increase_replicas.yaml apiVersion: apps/v1 kind: Deployment metadata: name: my-nginx spec: replicas: 3 EOF # Create another patch set_memory.yaml cat <<EOF > set_memory.yaml apiVersion: apps/v1 kind: Deployment metadata: name: my-nginx spec: template: spec: containers: - name: my-nginx resources: limits: memory: 512Mi EOF cat <<EOF >./kustomization.yaml resources: - deployment.yaml patches: - path: increase_replicas.yaml - path: set_memory.yaml EOF Run kubectl kustomize ./ to view the Deployment:apiVersion: apps/v1 kind: Deployment metadata: name: my-nginx spec: replicas: 3 selector: matchLabels: run: my-nginx template: metadata: labels: run: my-nginx spec: containers: - image: nginx name: my-nginx ports: - containerPort: 80 resources: limits: memory: 512Mi Not all resources or fields support strategicMerge patches. To support modifying arbitrary fields in arbitrary resources, Kustomize offers applying JSON patch through Json6902. To find the correct Resource for a Json6902 patch, it is mandatory to specify the target field in kustomization.yaml.For example, increasing the replica number of a Deployment object can also be done through Json6902 patch. The target resource is matched using group, version, kind, and name from the target field.# Create a deployment.yaml file cat <<EOF > deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: my-nginx spec: selector: matchLabels: run: my-nginx replicas: 2 template: metadata: labels: run: my-nginx spec: containers: - name: my-nginx image: nginx ports: - containerPort: 80 EOF # Create a json patch cat <<EOF > patch.yaml - op: replace path: /spec/replicas value: 3 EOF # Create a kustomization.yaml cat <<EOF >./kustomization.yaml resources: - deployment.yaml patches: - target: group: apps version: v1 kind: Deployment name: my-nginx path: patch.yaml EOF Run kubectl kustomize ./ to see the replicas field is updated:apiVersion: apps/v1 kind: Deployment metadata: name: my-nginx spec: replicas: 3 selector: matchLabels: run: my-nginx template: metadata: labels: run: my-nginx spec: containers: - image: nginx name: my-nginx ports: - containerPort: 80 In addition to patches, Kustomize also offers customizing container images or injecting field values from other objects into containers without creating patches. For example, you can change the image used inside containers by specifying the new image in the images field in kustomization.yaml.cat <<EOF > deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: my-nginx spec: selector: matchLabels: run: my-nginx replicas: 2 template: metadata: labels: run: my-nginx spec: containers: - name: my-nginx image: nginx ports: - containerPort: 80 EOF cat <<EOF >./kustomization.yaml resources: - deployment.yaml images: - name: nginx newName: my.image.registry/nginx newTag: "1.4.0" EOF Run kubectl kustomize ./ to see that the image being used is updated:apiVersion: apps/v1 kind: Deployment metadata: name: my-nginx spec: replicas: 2 selector: matchLabels: run: my-nginx template: metadata: labels: run: my-nginx spec: containers: - image: my.image.registry/nginx:1.4.0 name: my-nginx ports: - containerPort: 80 Sometimes, the application running in a Pod may need to use configuration values from other objects. For example, a Pod from a Deployment object need to read the corresponding Service name from Env or as a command argument. Since the Service name may change as namePrefix or nameSuffix is added in the kustomization.yaml file. It is not recommended to hard code the Service name in the command argument. For this usage, Kustomize can inject the Service name into containers through replacements.# Create a deployment.yaml file (quoting the here doc delimiter) cat <<'EOF' > deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: my-nginx spec: selector: matchLabels: run: my-nginx replicas: 2 template: metadata: labels: run: my-nginx spec: containers: - name: my-nginx image: nginx command: ["start", "--host", "MY_SERVICE_NAME_PLACEHOLDER"] EOF # Create a service.yaml file cat <<EOF > service.yaml apiVersion: v1 kind: Service metadata: name: my-nginx labels: run: my-nginx spec: ports: - port: 80 protocol: TCP selector: run: my-nginx EOF cat <<EOF >./kustomization.yaml namePrefix: dev- nameSuffix: "-001" resources: - deployment.yaml - service.yaml replacements: - source: kind: Service name: my-nginx fieldPath: metadata.name targets: - select: kind: Deployment name: my-nginx fieldPaths: - spec.template.spec.containers.0.command.2 EOF Run kubectl kustomize ./ to see that the Service name injected into containers is dev-my-nginx-001:apiVersion: apps/v1 kind: Deployment metadata: name: dev-my-nginx-001 spec: replicas: 2 selector: matchLabels: run: my-nginx template: metadata: labels: run: my-nginx spec: containers: - command: - start - --host - dev-my-nginx-001 image: nginx name: my-nginx Bases and OverlaysKustomize has the concepts of bases and overlays. A base is a directory with a kustomization.yaml, which contains a set of resources and associated customization. A base could be either a local directory or a directory from a remote repo, as long as a kustomization.yaml is present inside. An overlay is a directory with a kustomization.yaml that refers to other kustomization directories as its bases. A base has no knowledge of an overlay and can be used in multiple overlays.The kustomization.yaml in an overlay directory may refer to multiple bases, combining all the resources defined in these bases into a unified configuration. Additionally, it can apply customizations on top of these resources to meet specific requirements.Here is an example of a base:# Create a directory to hold the base mkdir base # Create a base/deployment.yaml cat <<EOF > base/deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: my-nginx spec: selector: matchLabels: run: my-nginx replicas: 2 template: metadata: labels: run: my-nginx spec: containers: - name: my-nginx image: nginx EOF # Create a base/service.yaml file cat <<EOF > base/service.yaml apiVersion: v1 kind: Service metadata: name: my-nginx labels: run: my-nginx spec: ports: - port: 80 protocol: TCP selector: run: my-nginx EOF # Create a base/kustomization.yaml cat <<EOF > base/kustomization.yaml resources: - deployment.yaml - service.yaml EOF This base can be used in multiple overlays. You can add different namePrefix or other cross-cutting fields in different overlays. Here are two overlays using the same base.mkdir dev cat <<EOF > dev/kustomization.yaml resources: - ../base namePrefix: dev- EOF mkdir prod cat <<EOF > prod/kustomization.yaml resources: - ../base namePrefix: prod- EOF How to apply/view/delete objects using KustomizeUse --kustomize or -k in kubectl commands to recognize resources managed by kustomization.yaml. Note that -k should point to a kustomization directory, such askubectl apply -k <kustomization directory>/ Given the following kustomization.yaml,# Create a deployment.yaml file cat <<EOF > deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: my-nginx spec: selector: matchLabels: run: my-nginx replicas: 2 template: metadata: labels: run: my-nginx spec: containers: - name: my-nginx image: nginx ports: - containerPort: 80 EOF # Create a kustomization.yaml cat <<EOF >./kustomization.yaml namePrefix: dev- labels: - pairs: app: my-nginx includeSelectors: true resources: - deployment.yaml EOF Run the following command to apply the Deployment object dev-my-nginx:> kubectl apply -k ./ deployment.apps/dev-my-nginx created Run one of the following commands to view the Deployment object dev-my-nginx:kubectl get -k ./ kubectl describe -k ./ Run the following command to compare the Deployment object dev-my-nginx against the state that the cluster would be in if the manifest was applied:kubectl diff -k ./ Run the following command to delete the Deployment object dev-my-nginx:> kubectl delete -k ./ deployment.apps "dev-my-nginx" deleted Kustomize Feature ListFieldTypeExplanationbases[]stringEach entry in this list should resolve to a directory containing a kustomization.yaml filecommonAnnotationsmap[string]stringannotations to add to all resourcescommonLabelsmap[string]stringlabels to add to all resources and selectorsconfigMapGenerator[]ConfigMapArgsEach entry in this list generates a ConfigMapconfigurations[]stringEach entry in this list should resolve to a file containing Kustomize transformer configurationscrds[]stringEach entry in this list should resolve to an OpenAPI definition file for Kubernetes typesgeneratorOptionsGeneratorOptionsModify behaviors of all ConfigMap and Secret generatorimages[]ImageEach entry is to modify the name, tags and/or digest for one image without creating patcheslabelsmap[string]stringAdd labels without automatically injecting corresponding selectorsnamePrefixstringvalue of this field is prepended to the names of all resourcesnameSuffixstringvalue of this field is appended to the names of all resourcespatchesJson6902[]PatchEach entry in this list should resolve to a Kubernetes object and a Json PatchpatchesStrategicMerge[]stringEach entry in this list should resolve a strategic merge patch of a Kubernetes objectreplacements[]Replacementscopy the value from a resource's field into any number of specified targets.resources[]stringEach entry in this list must resolve to an existing resource configuration filesecretGenerator[]SecretArgsEach entry in this list generates a Secretvars[]VarEach entry is to capture text from one resource's fieldWhat's nextKustomizeKubectl BookKubectl Command ReferenceKubernetes API Reference