Repository: kubernetes
Issue: CRI: Image IDs and references  
Labels: area/api,sig/node,kind/feature,lifecycle/frozen

CRI in its current state mostly passes an undefined image “string” when invoking image-related operations (e.g., pull, status, remove). I think it's time we revist to define/clarify better. Here are my thoughts:
 
Below is a list of terms used for 
1. **Image ID**: A canonical, content-addressable image identifier. This corresponds to the Image ID defined in the OCI image spec.
2. **Pullable reference(s)**: The syntax of a pullable reference is tied to the distribution method of the images. An image ID can map to multiple pullable references. In CRI, there are two such types of references (mainly adopted from the docker API):
  a. `RepoDigests`: `<registry>/<repository>@<manifest digest>`. E.g., `gcr.io/google-containers/nginx@sha256:01b86864da3937bf94571a816f1191c89d2ebd6c641bc692bcb3445b7d8520ef`
  b. `RepoTags`: `<registry>/<repository>:<tag>`. e.g., `gcr.io/google-containers/nginx:1.3`
3. **Manifest digest**: Digest of the manifest file. It can be part of the pullable reference (e.g., `RepoDigests`).
4. **Local reference**: What the container runtime uses to refer to the image on the node. This could be the image ID or not.
 
For CRI, we mainly care about (1) and (2). I think we should restrict pullable references to pulling, and uses the image ID for all other operations.
- **PullImage**: Pulls an image by a pullable reference and returns its ID.
- **RemoveImage**: Removes an image by its ID.
- **ImageStatus**: Inspects an image by its ID and returns information of the image, including the ID and a list of pullable references known by the runtime.
- **ResolveImage**: A new RPC call to translate a pullable reference into the image ID.
- The configuration used to create a container should include the ID of the image.
 
ResolveImage will be added to further break down the internal operations of the ImageService and make each call more focused. For example, to create a container, kubelet will need to perform the following calls.
 * Call `ResolveImage(pullableRef)` to get the Image ID
 * Call `ImageStatus(imageID)` to check if the image exists on the node.
 * Call `PullImage(pullableRef)` to pull down the image.

Without `ResolveImage`, `ImageStatus` would need to accept both pullable references and image IDs as an input.
 
Another issue is how we tackle the pullable references. These pullable references are distribution- (and can be runtime-) specific. CRI should not need to understand or distinguish pullable references such as `RepoDigests` and `RepoTags`. Unfortunately, kubernetes API reports [`RepoDigests` as `ImageID` in the `ContainerStatuses`](https://github.com/kubernetes/kubernetes/pull/34473). To continue support this, we can choose one of the options below.
1. Preserve both `RepoDigests` and `RepoTags` in CRI (w/ renaming).
2. Add hacks (regex) in kubelet to *guess* which reference is the `RepoDigests`.
3. Combine both into a `References` field in CRI. Suggest runtimes to sort pullable references based on representativeness. Change dockershim implementation to always return `RepoDigests` as the first one to be consumed by kubelet.
 
I am leaning towards (3) since there is no consensus on image distribution yet, and (3) gives us the most flexibility.

Thoughts and suggestions? @kubernetes/sig-node-api-reviews @mrunalp @stevvooe @feiskyer @Random-Liu @dchen1107 


WARNING: Some of the proposed changes will not be backward compatible, but I think it's more important to get the API in a better state, than to ensure backward compatibility right now. Others may object though.