Chapter 9. Caching images for faster workspace start This section describes installing the Image Puller on a CodeReady Workspaces cluster to cache images on cluster nodes. 9.1. Image Puller overview Slow starts of Red Hat CodeReady Workspaces workspaces may be caused by waiting for the underlying cluster to pull images used in workspaces from remote registries. As such, pre-pulling images can improve start times significantly. The Image Puller can be used to pre-pull images and shorten workspace start times. The Image Puller is an additional deployment that runs alongside Red Hat CodeReady Workspaces. Given a list of images to pre-pull, the application runs inside a cluster and creates a DaemonSet that pulls the images on each node. Note The minimal requirement for an image to be pre-pulled is the availability of the sleep command, which means that FROM scratch images (for example, 'che-machine-exec') are currently not supported. Also, images that mount volumes in the dockerfile are not supported for pre-pulling on OpenShift. The application can be deployed: using OperatorHub or installing the kubernetes image puller operator by processing and applying OpenShift templates. The Image Puller pulls its configuration from a ConfigMap with the following available parameters: Table 9.1. Image Puller default parameters Parameter Usage Default CACHING_INTERVAL_HOURS Interval, in hours, between checking health of DaemonSets "1" CACHING_MEMORY_REQUEST The memory request for each cached image when the puller is running 10Mi CACHING_MEMORY_LIMIT The memory limit for each cached image when the puller is running 20Mi CACHING_CPU_REQUEST The CPU request for each cached image when the puller is running .05 CACHING_CPU_LIMIT The CPU limit for each cached image when the puller is running .2 DAEMONSET_NAME Name of DaemonSet to be created kubernetes-image-puller NAMESPACE Namespace where DaemonSet is to be created k8s-image-puller IMAGES List of images to be cached, in the format <name>=<image>;... Contains a default list of images. Before deploying, fill this with the images that fit the current requirements NODE_SELECTOR Node selector applied to the Pods created by the DaemonSet '{}' The default memory requests and limits ensure that the container has enough memory to start. When changing CACHING_MEMORY_REQUEST or CACHING_MEMORY_LIMIT , you will need to consider the total memory allocated to the DaemonSet Pods in the cluster: (memory limit) * (number of images) * (number of nodes in the cluster) For example, running the image puller that caches 5 images on 20 nodes, with a container memory limit of 20Mi requires 2000Mi of memory. 9.2. Deploying Image Puller using the operator The recommended way to deploy the Image Puller is through the operator . 9.2.1. Installing the Image Puller on OpenShift using OperatorHub First, create a namespace in your cluster to host the image puller. Our example will use the namespace "image-puller". Navigate to your OpenShift cluster's console, and select Operators . Select OperatorHub and type "image puller" into the "Filter by keyword.." search bar. Click the OpenShift Image Puller Operator , click Continue and click Install . At the Installation Mode selection, choose A specific namespace on the cluster , and use the drop-down to find the namespace you created to install the image puller. Click Subscribe . Wait for the OpenShift Image Puller Operator to install, and click the installation. Click the OpenShiftImagePuller tab, and then click Create instance . You will be taken to a screen with a YAML editor with a OpenShiftImagePuller Custom Resource. Make any modifications to the Custom resource and click Create . Navigate to the Workloads and Pods menu in the namespace that the image puller was installed, and you should see pods being created. 9.2.2. Installing the Image Puller on OpenShift using the Operator Create a namespace to host the kubernetes image puller, and apply the following manifests from the GitHub repository : export NAMESPACE=<namespace you created to host the image puller> oc apply -f https://raw.githubusercontent.com/che-incubator/kubernetes-image-puller-operator/master/deploy/crds/che.eclipse.org_kubernetesimagepullers_crd.yaml -n USDNAMESPACE oc apply -f https://raw.githubusercontent.com/che-incubator/kubernetes-image-puller-operator/master/deploy/role.yaml -n USDNAMESPACE oc apply -f https://raw.githubusercontent.com/che-incubator/kubernetes-image-puller-operator/master/deploy/role_binding.yaml -n USDNAMESPACE oc apply -f https://raw.githubusercontent.com/che-incubator/kubernetes-image-puller-operator/master/deploy/service_account.yaml -n USDNAMESPACE oc apply -f https://raw.githubusercontent.com/che-incubator/kubernetes-image-puller-operator/master/deploy/operator.yaml -n USDNAMESPACE Then create a OpenShiftImagePuller Custom Resource: apiVersion: che.eclipse.org/v1alpha1 kind: KubernetesImagePuller metadata: name: image-puller namespace: <namespace you installed the image puller in> spec: configMapName: k8s-image-puller daemonsetName: k8s-image-puller deploymentName: kubernetes-image-puller images: >- java11-maven=quay.io/eclipse/che-java11-maven:nightly;che-theia=quay.io/eclipse/che-theia: 9.3. Deploying Image Puller using OpenShift templates The Image Puller repository contains OpenShift templates for deploying on OpenShift. Prerequisites A running OpenShift cluster. The oc binary file. The following parameters are available to further configure the OpenShift templates: Table 9.2. Parameters for installing with OpenShift templates Value Usage Default DAEMONSET_NAME The value of DAEMONSET_NAME to set in the ConfigMap kubernetes-image-puller IMAGE Image used for the kubernetes-image-puller deployment registry.redhat.io/codeready-workspaces/imagepuller-rhel8 IMAGE_TAG The image tag to pull 2.1 SERVICEACCOUNT_NAME The name of the ServiceAccount used by the deployment (created as part of installation) k8s-image-puller CACHING_INTERVAL_HOURS The value of CACHING_INTERVAL_HOURS to set in the ConfigMap "1" CACHING_INTERVAL_REQUEST The value of CACHING_MEMORY_REQUEST to set in the ConfigMap "10Mi" CACHING_INTERVAL_LIMIT The value of CACHING_MEMORY_LIMIT to set in the ConfigMap "20Mi"` NODE_SELECTOR The value of NODE_SELECTOR to set in the ConfigMap "{}" See Table 9.1, "Image Puller default parameters" for more information about configuration values, such as DAEMONSET_NAME , CACHING_INTERVAL_HOURS , and CACHING_MEMORY_REQUEST . Table 9.3. List of recommended images to pre-pull Image URL Tag stacks-java-rhel8 registry.access.redhat.com/codeready-workspaces/stacks-java-rhel8 2.1 theia-rhel8 registry.access.redhat.com/codeready-workspaces/theia-rhel8 2.1 stacks-golang-rhel8 registry.access.redhat.com/codeready-workspaces/stacks-golang-rhel8 2.1 stacks-node-rhel8 registry.access.redhat.com/codeready-workspaces/stacks-node-rhel8 2.1 theia-endpoint-rhel8 registry.access.redhat.com/codeready-workspaces/theia-rhel8 2.1 pluginbroker-metadata-rhel8 registry.access.redhat.com/codeready-workspaces/pluginbroker-metadata-rhel8 2.1 pluginbroker-artifacts-rhel8 registry.access.redhat.com/codeready-workspaces/pluginbroker-artifacts-rhel8 2.1 See Table 9.1, "Image Puller default parameters" for more information about configuration values, such as DAEMONSET_NAME , CACHING_INTERVAL_HOURS , and CACHING_MEMORY_REQUEST . Procedure Installing Clone the kubernetes-image-puller repository: Create a new OpenShift project to deploy the puller into: Process and apply the templates to deploy the puller: In CodeReady Workspaces you must use custom values to deploy the image puller. To set custom values, add to the oc process an option: -p <parameterName> = <value> : Verifying the installation Confirm that a new deployment, kubernetes-image-puller , and a DaemonSet (named based on the value of the DAEMONSET_NAME parameter) exist. The DaemonSet needs to have a Pod for each node in the cluster: USD oc get deployment,daemonset,pod --namespace k8s-image-puller deployment.extensions/kubernetes-image-puller 1/1 1 1 2m19s NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE daemonset.extensions/kubernetes-image-puller 1 1 1 1 1 <none> 2m10s NAME READY STATUS RESTARTS AGE pod/kubernetes-image-puller-5495f46497-mkd4p 1/1 Running 0 2m18s pod/kubernetes-image-puller-n8bmf 3/3 Running 0 2m10s Check that the ConfigMap named k8s-image-puller has the values you specified in your parameter substitution, or that they contain the default values:

Commands:
export NAMESPACE=<namespace you created to host the image puller> apply -f https://raw.githubusercontent.com/che-incubator/kubernetes-image-puller-operator/master/deploy/crds/che.eclipse.org_kubernetesimagepullers_crd.yaml -n USDNAMESPACE apply -f https://raw.githubusercontent.com/che-incubator/kubernetes-image-puller-operator/master/deploy/role.yaml -n USDNAMESPACE apply -f https://raw.githubusercontent.com/che-incubator/kubernetes-image-puller-operator/master/deploy/role_binding.yaml -n USDNAMESPACE apply -f https://raw.githubusercontent.com/che-incubator/kubernetes-image-puller-operator/master/deploy/service_account.yaml -n USDNAMESPACE apply -f https://raw.githubusercontent.com/che-incubator/kubernetes-image-puller-operator/master/deploy/operator.yaml -n USDNAMESPACE
apiVersion: che.eclipse.org/v1alpha1 kind: KubernetesImagePuller metadata: name: image-puller namespace: <namespace you installed the image puller in> spec: configMapName: k8s-image-puller daemonsetName: k8s-image-puller deploymentName: kubernetes-image-puller images: >- java11-maven=quay.io/eclipse/che-java11-maven:nightly;che-theia=quay.io/eclipse/che-theia:next
git clone https://github.com/che-incubator/kubernetes-image-puller cd kubernetes-image-puller
oc new-project k8s-image-puller
oc process -f deploy/serviceaccount.yaml | oc apply -f - oc process -f deploy/configmap.yaml -p IMAGES='stacks-java-rhel8=registry.access.redhat.com/codeready-workspaces/stacks-java-rhel8:2.1; theia-rhel8=registry.access.redhat.com/codeready-workspaces/theia-rhel8:2.1; stacks-golang-rhel8=registry.access.redhat.com/codeready-workspaces/stacks-golang-rhel8:2.1; stacks-node-rhel8=registry.access.redhat.com/codeready-workspaces/stacks-node-rhel8:2.1; theia-endpoint-rhel8=registry.access.redhat.com/codeready-workspaces/theia-rhel8:2.1; pluginbroker-metadata-rhel8=egistry.access.redhat.com/codeready-workspaces/pluginbroker-metadata-rhel8:2.1; pluginbroker-artifacts-rhel8=registry.access.redhat.com/codeready-workspaces/pluginbroker-artifacts-rhel8:2.1;' | oc apply -f - oc process -f deploy/app.yaml -p IMAGE=registry.redhat.io/codeready-workspaces/imagepuller-rhel8 -p IMAGE_TAG='2.1' | oc apply -f -
oc get deployment,daemonset,pod --namespace k8s-image-puller deployment.extensions/kubernetes-image-puller 1/1 1 1 2m19s NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE daemonset.extensions/kubernetes-image-puller 1 1 1 1 1 <none> 2m10s NAME READY STATUS RESTARTS AGE pod/kubernetes-image-puller-5495f46497-mkd4p 1/1 Running 0 2m18s pod/kubernetes-image-puller-n8bmf 3/3 Running 0 2m10s
oc get configmap k8s-image-puller --output yaml apiVersion: v1 data: CACHING_INTERVAL_HOURS: "1" CACHING_MEMORY_LIMIT: 20Mi CACHING_MEMORY_REQUEST: 10Mi DAEMONSET_NAME: kubernetes-image-puller IMAGES: | stacks-java-rhel8=registry.access.redhat.com/codeready-workspaces/stacks-java-rhel8:2.1; theia-rhel8=registry.access.redhat.com/codeready-workspaces/theia-rhel8:2.1; stacks-golang-rhel8=registry.access.redhat.com/codeready-workspaces/stacks-golang-rhel8:2.1; stacks-node-rhel8=registry.access.redhat.com/codeready-workspaces/stacks-node-rhel8:2.1; theia-endpoint-rhel8=registry.access.redhat.com/codeready-workspaces/theia-rhel8:2.1; pluginbroker-metadata-rhel8=egistry.access.redhat.com/codeready-workspaces/pluginbroker-metadata-rhel8:2.1; pluginbroker-artifacts-rhel8=registry.access.redhat.com/codeready-workspaces/pluginbroker-artifacts-rhel8:2.1; NAMESPACE: k8s-image-puller NODE_SELECTOR: '{}' kind: ConfigMap metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"v1","data":{"CACHING_INTERVAL_HOURS":"1","CACHING_MEMORY_LIMIT":"20Mi","CACHING_MEMORY_REQUEST":"10Mi","DAEMONSET_NAME":"kubernetes-image-puller","IMAGES":"stacks-java-rhel8=registry.access.redhat.com/codeready-workspaces/stacks-java-rhel8:2.1; theia-rhel8=registry.access.redhat.com/codeready-workspaces/theia-rhel8:2.1; stacks-golang-rhel8=registry.access.redhat.com/codeready-workspaces/stacks-golang-rhel8:2.1; stacks-node-rhel8=registry.access.redhat.com/codeready-workspaces/stacks-node-rhel8:2.1; theia-endpoint-rhel8=registry.access.redhat.com/codeready-workspaces/theia-rhel8:2.1; pluginbroker-metadata-rhel8=egistry.access.redhat.com/codeready-workspaces/pluginbroker-metadata-rhel8:2.1; pluginbroker-artifacts-rhel8=registry.access.redhat.com/codeready-workspaces/pluginbroker-artifacts-rhel8:2.1;\n","NAMESPACE":"k8s-image-puller","NODE_SELECTOR":"{}"},"kind":"ConfigMap","metadata":{"annotations":{},"name":"k8s-image-puller","namespace":"k8s-image-puller"},"type":"Opaque"} creationTimestamp: 2020-02-17T22:40:13Z name: k8s-image-puller namespace: k8s-image-puller resourceVersion: "72250" selfLink: /api/v1/namespaces/k8s-image-puller/configmaps/k8s-image-puller uid: 76430ed6-51d6-11ea-9c19-52fdfc072182