Repository: linz/geostore

Issue:
Use latest of each STAC extension version
### Enabler

So that we don't have to manually update the code to use the latest version, we want to automatically use the latest version available in the relevant Git submodule.

Need to check what happens when a file is submitted that references and old version of a stac schema

#### Acceptance Criteria

- [ ] Dependabot PRs for any of the STAC submodules run tests with the latest version of all the extensions in that submodule
- [ ] Add a note to the release documentation about notifying users which STAC extension versions are supported

#### Additional context

This avoids manual work like [this PR to use the latest LINZ STAC extensions](https://github.com/linz/geostore/pull/1444).

Caveat: We currently only support one version of each extension. When extensions release breaking changes this could affect our existing users, and we need to notify them.

#### Tasks

<!-- Tasks needed to complete this enabler -->

- [ ] ...
- [ ] ...

#### Definition of Ready

- [ ] This story is **ready** to work on
  - [ ] Negotiable (team can decide how to design and implement)
  - [ ] Valuable (from a user perspective)
  - [ ] Estimate value applied (agreed by team)
  - [ ] Small (so as to fit within an iteration)
  - [ ] Testable (in principle, even if there isn't a test for it yet)
  - [ ] Environments are ready to meet definition of done
  - [ ] Resources required to implement will be ready
  - [ ] Everyone understands and agrees with the tasks to complete the story
  - [ ] Release value (e.g. Iteration 3) applied
  - [ ] Sprint value (e.g. Aug 1 - Aug 15) applied

#### Definition of Done

- [ ] This story is **done**:
  - [ ] Acceptance criteria completed
  - [ ] Automated tests are passing
  - [ ] Code is peer reviewed and pushed to master
  - [ ] Deployed successfully to test environment
  - [ ] Checked against [CODING guidelines](https://github.com/linz/geostore/blob/master/CODING.md)
  - [ ] Relevant new tasks are added to backlog and communicated to the team
  - [ ] Important decisions recorded in the issue ticket
  - [ ] Readme/Changelog/Diagrams are updated
  - [ ] Product Owner has approved acceptance criteria as complete
  - [ ] Meets non-functional requirements:
    - [ ] Scalability (data): Can scale to 300TB of data and 100,000,000 files and ability to
          increase 10% every year
    - [ ] Scability (users): Can scale to 100 concurrent users
    - [ ] Cost: Data can be stored at < 0.5 NZD per GB per year
    - [ ] Performance: A large dataset (500 GB and 50,000 files - e.g. Akl aerial imagery) can be
          validated, imported and stored within 24 hours
    - [ ] Accessibility: Can be used from LINZ networks and the public internet
    - [ ] Availability: System available 24 hours a day and 7 days a week, this does not include
          maintenance windows < 4 hours and does not include operational support
    - [ ] Recoverability: RPO of fully imported datasets < 4 hours, RTO of a single 3 TB dataset <
          12 hours

<!-- Please add one or more of these labels: 'spike', 'refactor', 'architecture', 'infrastructure', 'compliance' -->



Discussion:

Resolving patch:
diff --git a/geostore/check_stac_metadata/stac_validators.py b/geostore/check_stac_metadata/stac_validators.py
--- a/geostore/check_stac_metadata/stac_validators.py
+++ b/geostore/check_stac_metadata/stac_validators.py
@@ -1,6 +1,9 @@
-from functools import cached_property
+from distutils.version import StrictVersion
+from functools import cached_property, lru_cache
 from json import load
+from os import scandir
 from os.path import dirname, join
+from re import fullmatch
 
 from jsonschema import Draft7Validator, FormatChecker, RefResolver
 from jsonschema._utils import URIDict
@@ -31,15 +34,28 @@ def uri(self) -> str:
         return uri_
 
 
+@lru_cache
+def get_latest_extension_schema_version(extension_path: str) -> str:
+    directories = scandir(join(dirname(__file__), extension_path))
+    versions = []
+    for directory in directories:
+        if directory.is_dir() and fullmatch(r"v\d+\.\d+\.\d+", directory.name):
+            versions.append(directory.name[1:])
+    return sorted(versions, key=StrictVersion, reverse=True)[0]
+
+
 FILE_STAC_SCHEMA_PATH = "file/v2.0.0/schema.json"
 PROJECTION_STAC_SCHEMA_PATH = "projection/v1.0.0/schema.json"
 VERSION_STAC_SCHEMA_PATH = "version/v1.0.0/schema.json"
 FILE_SCHEMA = Schema(FILE_STAC_SCHEMA_PATH)
 
-STAC_VERSION = "1.0.0"
-STAC_SPEC_PATH = f"stac-spec/v{STAC_VERSION}"
+STAC_SPEC_EXTENSION_PATH = "stac-spec"
+STAC_VERSION = get_latest_extension_schema_version(STAC_SPEC_EXTENSION_PATH)
+STAC_SPEC_PATH = f"{STAC_SPEC_EXTENSION_PATH}/v{STAC_VERSION}"
 CATALOG_SCHEMA = Schema(f"{STAC_SPEC_PATH}/catalog-spec/json-schema/catalog.json")
-LINZ_STAC_EXTENSIONS_URL_PATH = "v0.0.14"
+LINZ_STAC_EXTENSIONS_URL_PATH = (
+    f"v{get_latest_extension_schema_version(LINZ_STAC_EXTENSIONS_LOCAL_PATH)}"
+)
 LINZ_SCHEMA_URL_DIRECTORY = f"{LINZ_STAC_EXTENSIONS_URL_PATH}/linz"
 LINZ_SCHEMA_URL_PATH = f"{LINZ_SCHEMA_URL_DIRECTORY}/schema.json"
 LINZ_SCHEMA = Schema(join(LINZ_STAC_EXTENSIONS_LOCAL_PATH, LINZ_SCHEMA_URL_PATH))


Test patch:
diff --git a/tests/test_get_latest_extension_schema_version.py b/tests/test_get_latest_extension_schema_version.py
new file mode 100644
--- /dev/null
+++ b/tests/test_get_latest_extension_schema_version.py
@@ -0,0 +1,5 @@
+from geostore.check_stac_metadata.stac_validators import get_latest_extension_schema_version
+
+
+def should_get_latest_stac_spec_version() -> None:
+    assert get_latest_extension_schema_version("stac-spec") == "1.0.0"