Repository: lilydjwg/nvchecker (language: python)

Issue:
[Feature Request] Support watch update of specific docker image tag
## Background

Docker image tag is just an alias of image ID, and can be updated to refer to new image IDs.

Many docker images provide tags named by its major version (or major and minor version), so users can easily get future patch version update without updating their Dockerfile. Say `nginx:1.24` for example, whenever a new patch version 1.24.x is released, the tag will be updated.

Tags are also occasionally updated to eliminate vulnerabilities.

## Feature

I hope nvchecker can watch for updates of image tags on Docker Hub (like `nginx:1.24`, `oraclelinux:8`).  Some private base images in our intranet depend on these public images and I want to keep them update-to-date and secure.

Nvchecker's [container](https://github.com/lilydjwg/nvchecker/blob/master/nvchecker_source/container.py) plugin can only watch for new tags and is not applicable.

## Proposal

Add a container_tag plugin.

Available API to check tag update:

* Docker Hub API: https://hub.docker.com/v2/repositories/library/nginx/tags
    * Easy to use if we only support public images on hub.docker.com
* Docker Registry HTTP API: [`/v2/<name>/manifests/<reference>`](https://distribution.github.io/distribution/spec/api/#pulling-an-image-manifest)
    * ~~Unfortunately no image ID or update time is returned, so we have to use fsLayers to check whether tag is updated~~
    * Update time is returned in `history[0].v1Compatibility.created`


Discussion / hints:
Maybe we can add this functionality to the existing `container` source? @yan12125, what do you think?
> Maybe we can add this functionality to the existing `container` source? @yan12125, what do you think?

@lilydjwg I have tried extending `container` source and it works. I'll make a pull request later. 

Fix patch:
diff --git a/docs/usage.rst b/docs/usage.rst
index 9deac325..15f8efbb 100644
--- a/docs/usage.rst
+++ b/docs/usage.rst
@@ -839,7 +839,9 @@ Check container registry
 This enables you to check tags of images on a container registry like Docker.
 
 container
-  The path for the container image. For official Docker images, use namespace ``library/`` (e.g. ``library/python``).
+  The path (and tag) for the container image. For official Docker images, use namespace ``library/`` (e.g. ``library/python``).
+
+  If no tag is given, it checks latest available tag (sort by tag name), otherwise, it checks the tag's update time.
 
 registry
   The container registry host. Default: ``docker.io``
@@ -850,17 +852,23 @@ container name while this plugin requires the full name. If the host part is
 omitted, use ``docker.io``, and if there is no slash in the path, prepend
 ``library/`` to the path. Here are some examples:
 
-+----------------------------------------------+-----------+--------------------------+
-| Pull command                                 | registry  | container                |
-+==============================================+===========+==========================+
-| docker pull quay.io/prometheus/node-exporter | quay.io   | prometheus/node-exporter |
-+----------------------------------------------+-----------+--------------------------+
-| docker pull nvidia/cuda                      | docker.io | nvidia/cuda              |
-+----------------------------------------------+-----------+--------------------------+
-| docker pull python                           | docker.io | library/python           |
-+----------------------------------------------+-----------+--------------------------+
-
-This source returns tags and supports :ref:`list options`.
++-----------------------------------------------------+-----------+---------------------------------+
+| Pull        command                                 | registry  | container                       |
++=====================================================+===========+=================================+
+| docker pull quay.io/prometheus/node-exporter        | quay.io   | prometheus/node-exporter        |
++-----------------------------------------------------+-----------+---------------------------------+
+| docker pull quay.io/prometheus/node-exporter:master | quay.io   | prometheus/node-exporter:master |
++-----------------------------------------------------+-----------+---------------------------------+
+| docker pull openeuler/openeuler                     | docker.io | openeuler/openeuler             |
++-----------------------------------------------------+-----------+---------------------------------+
+| docker pull openeuler/openeuler:20.03-lts           | docker.io | openeuler/openeuler:20.03-lts   |
++-----------------------------------------------------+-----------+---------------------------------+
+| docker pull python                                  | docker.io | library/python                  |
++-----------------------------------------------------+-----------+---------------------------------+
+| docker pull python:3.11                             | docker.io | library/python:3.11             |
++-----------------------------------------------------+-----------+---------------------------------+
+
+If no tag is given, this source returns tags and supports :ref:`list options`.
 
 Check ALPM database
 ~~~~~~~~~~~~~~~~~~~
diff --git a/nvchecker_source/container.py b/nvchecker_source/container.py
index df745b8e..6fec2089 100644
--- a/nvchecker_source/container.py
+++ b/nvchecker_source/container.py
@@ -4,6 +4,7 @@
 from typing import Dict, List, NamedTuple, Optional, Tuple
 from urllib.request import parse_http_list
 from urllib.parse import urljoin
+import json
 
 from nvchecker.api import session, HTTPError
 
@@ -57,15 +58,7 @@ async def get_registry_auth_info(registry_host: str) -> AuthInfo:
 
 async def get_container_tags(info: Tuple[str, str, AuthInfo]) -> List[str]:
   image_path, registry_host, auth_info = info
-
-  auth_params = {
-    'scope': f'repository:{image_path}:pull',
-  }
-  if auth_info.service:
-    auth_params['service'] = auth_info.service
-  res = await session.get(auth_info.realm, params=auth_params)
-  token = res.json()['token']
-
+  token = await get_auth_token(auth_info, image_path)
   tags = []
   url = f'https://{registry_host}/v2/{image_path}/tags/list'
 
@@ -83,6 +76,18 @@ async def get_container_tags(info: Tuple[str, str, AuthInfo]) -> List[str]:
 
   return tags
 
+
+async def get_auth_token(auth_info, image_path):
+  auth_params = {
+    'scope': f'repository:{image_path}:pull',
+  }
+  if auth_info.service:
+    auth_params['service'] = auth_info.service
+  res = await session.get(auth_info.realm, params=auth_params)
+  token = res.json()['token']
+  return token
+
+
 def parse_next_link(value: str) -> str:
   ending = '>; rel="next"'
   if value.endswith(ending):
@@ -90,13 +95,54 @@ def parse_next_link(value: str) -> str:
   else:
     raise ValueError(value)
 
+
+async def get_container_tag_update_time(info: Tuple[str, str, str, AuthInfo]):
+  '''
+  Find the update time of a container tag.
+
+  In fact, it's the creation time of the image ID referred by the tag. Tag itself does not have any update time.
+  '''
+  image_path, image_tag, registry_host, auth_info = info
+  token = await get_auth_token(auth_info, image_path)
+
+  # HTTP headers
+  headers = {
+    'Authorization': f'Bearer {token}',
+    # Prefer Image Manifest Version 2, Schema 2: https://distribution.github.io/distribution/spec/manifest-v2-2/
+    'Accept': 'application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.container.image.v1+json, application/json',
+  }
+
+  # Get tag manifest
+  url = f'https://{registry_host}/v2/{image_path}/manifests/{image_tag}'
+  res = await session.get(url, headers=headers)
+  data = res.json()
+  # Schema 1 returns the creation time in the response
+  if data['schemaVersion'] == 1:
+    return json.loads(data['history'][0]['v1Compatibility'])['created']
+
+  # For schema 2, we have to fetch the config's blob
+  digest = data['config']['digest']
+  url = f'https://{registry_host}/v2/{image_path}/blobs/{digest}'
+  res = await session.get(url, headers=headers)
+  data = res.json()
+  return data['created']
+
+
 async def get_version(name, conf, *, cache, **kwargs):
   image_path = conf.get('container', name)
+  image_tag = None
+  # image tag is optional
+  if ':' in image_path:
+    image_path, image_tag = image_path.split(':', 1)
   registry_host = conf.get('registry', 'docker.io')
   if registry_host == 'docker.io':
     registry_host = 'registry-1.docker.io'
 
   auth_info = await cache.get(registry_host, get_registry_auth_info)
 
+  # if a tag is given, return the tag's update time, otherwise return the image's tag list
+  if image_tag:
+    key = image_path, image_tag, registry_host, auth_info
+    return await cache.get(key, get_container_tag_update_time)
   key = image_path, registry_host, auth_info
   return await cache.get(key, get_container_tags)


Test patch:
diff --git a/tests/test_container.py b/tests/test_container.py
index 98bcff6a..4429315a 100644
--- a/tests/test_container.py
+++ b/tests/test_container.py
@@ -2,6 +2,7 @@
 # Copyright (c) 2020 Chih-Hsuan Yen <yan12125 at gmail dot com>
 
 import pytest
+import datetime
 pytestmark = [pytest.mark.asyncio, pytest.mark.needs_net]
 
 async def test_container(get_version):
@@ -11,6 +12,23 @@ async def test_container(get_version):
     "include_regex": "linux",
   }) == "linux"
 
+async def test_container_with_tag(get_version):
+  update_time = await get_version("hello-world:linux", {
+    "source": "container",
+    "container": "library/hello-world:linux",
+  })
+  # the update time is changing occasionally, so we can not compare the exact time, otherwise the test will be failed in the future
+  assert datetime.date.fromisoformat(update_time.split('T')[0]) > datetime.date(2023, 1, 1)
+
+async def test_container_with_tag_and_registry(get_version):
+  update_time = await get_version("hello-world-nginx:v1.0", {
+    "source": "container",
+    "registry": "quay.io",
+    "container": "redhattraining/hello-world-nginx:v1.0",
+  })
+  # the update time probably won't be changed
+  assert datetime.date.fromisoformat(update_time.split('T')[0]) == datetime.date(2019, 6, 26)
+
 async def test_container_paging(get_version):
   assert await get_version("prometheus-operator", {
     "source": "container",