Repository: skupperproject/skupper
Pull request #1767 (2024-11-04 17:36:23)

Updates container build tooling
Resolves Issue https://github.com/skupperproject/skupper/issues/1742

* Includes labels on container images suggested in OCI spec
* Container image executables built with without CGO for consistency
* All executables built with ldflags "-s -w" by default
* Make targets updated
  * Uses pattern rules for container image targets
  * Uses variables REGISTRY+IMAGE_TAG instead of individual per-image variables like CONTROLLER_IMAGE.
  * `docker-build` now a plain-ole' docker build for the host platform - mostly for development use
  * `multiarch-oci` now does multiarch builds with output type of oci-archive (tar files in ./oci-archives).
  * `push-multiarch-oci` uses skopeo to push oci archives to registry
  * `(docker|podman)-load-oci` conveneince method to load images from multiarch-oci archives to local image storage
  * `describe-multiarch-oci` prints fully qualified images repo:tag@sha256 for the image index and individual platforms
  * cleans up old package+release targets
* CircleCI config cleanup unused targets and rework jobs for updated container build targets.
* Dropped some platforms from the skupper cli builds: linux-i386, linux-arm32 and windows-i386. No real solid numbers behind this, but didn't feel they were likely to be used and they were contributing to the build step taking a long time.

c-kruse: Hoping to rebase this on the config-sync -> kube-adaptor rename before marking as ready for review.

Throws out lots of ideas that I thought were sensible. I'd appreciate the team's thoughts.

Also have a reference integration test (not planned to merge) open here https://github.com/skupperproject/skupper/pull/1768 to attempt to proof out the idea of the container build -> oci archive -> load in tests -> publish flow.
c-kruse: For anyone interested in inspecting the images themselves

```
$ make describe-multiarch-oci push-multiarch-oci REGISTRY=quay.io/ckruse IMAGE_TAG=$(git rev-parse --short HEAD)
index:
  - quay.io/ckruse/bootstrap:1f7e2f12@sha256:3e94c19c83336e1a22d2c415c43a4b9932b96815a9980b0ffafcdf3384048d79
  - quay.io/ckruse/config-sync:1f7e2f12@sha256:96c1b4fd316025a6348b16ca833d06a31b01a86b3a66bc1d0522ced133783a0e
  - quay.io/ckruse/controller:1f7e2f12@sha256:d09fddc9accf447570893283c8650376a75b7f598061c56dd652811b5d903cf5
  - quay.io/ckruse/network-console-collector:1f7e2f12@sha256:8a171c7a35510889217d38d6a909a670f661cf06f80d4ab0b915fdfc47a7a4d6
amd64:
  - quay.io/ckruse/bootstrap:1f7e2f12@sha256:3334fc4d3333ecb53cbe2df4172bdf8d332869b818c7e60be6e75fcfd4fdab04
  - quay.io/ckruse/config-sync:1f7e2f12@sha256:007f2d8e368a7958f678f111febf7b0753ea02b591730a20bfdf51f8a9747440
  - quay.io/ckruse/controller:1f7e2f12@sha256:c36983cafc8618b07b5d7da4cc8d199f63cee71a093c01ba29a8e3ad4313bcfd
  - quay.io/ckruse/network-console-collector:1f7e2f12@sha256:2cb26080a642ba945c43b49797423ea804e0bea76cf3dc06dd5288609ff3aa6f
arm64:
  - quay.io/ckruse/bootstrap:1f7e2f12@sha256:74006ee179d46733d38dfa9cd2600a3dd7cffa6686389b5f2d3d5008820c340e
  - quay.io/ckruse/config-sync:1f7e2f12@sha256:18e366f75cd51d7793dfcee29e84b1ebd977165a7a71c4068bd8823f8f49be6f
  - quay.io/ckruse/controller:1f7e2f12@sha256:e8d0bcdb7b2bca9a917801faeddde585fb3bd9fe235783f83ce3979ef6fb9c72
  - quay.io/ckruse/network-console-collector:1f7e2f12@sha256:973f5a562f2102225be9499b7fcf0e18230698217ac3f06924dd9d87e9f9edde
skopeo copy --all \
        oci-archive:oci-archives/bootstrap.tar \
        "docker://quay.io/ckruse/bootstrap:1f7e2f12"
....
```

Patch:
diff --git a/.circleci/config.yml b/.circleci/config.yml
index 9fa6d185..b442e31f 100644
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -14,258 +14,13 @@ executors:
       image: ubuntu-2204:2022.10.2
     resource_class: large
 
-  local_cluster_policy_test_executor:
-    machine:
-      image: ubuntu-2204:2022.10.2
-    # Policy tests run all in sequence.  So, there is no benefit in adding more
-    # CPU or memory.  Within the tests, there is paralellism, but on goroutines,
-    # so they'd not use the additional CPUs.  The only thing a running policy
-    # test competes with is the other system processes
-    # resource_class: medium (the default)
-
 commands:
-  minikube-install:
-    description: Installs the minikube executable onto the system.
-    steps:
-      - run:
-          command: >-
-            curl -Lo minikube
-            https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 &&
-            chmod +x minikube && sudo
-            mv minikube /usr/local/bin/
-          name: Install Minikube Executable
-
-  minikube-start-medium:
-    description: Starts the minikube service, with 2 CPU and 2 GiB
-    steps:
-      - run:
-          command: >-
-            minikube start --vm-driver=docker --cpus 2 --memory 2048 --insecure-registry ${LOCAL_IP}:5000
-          name: Start Minikube Cluster
-
-  # We're using only 6 GiB out of the available 15 GiB of the linux/large
-  # CircleCI executor.  This allows us some leeway to grow within this
-  # resource class when memory issues happen, so we can prepare for the
-  # next resource class with plenty of time, when required.
-  minikube-start-large:
-    description: Starts the minikube service, with 4 CPU and 6GiB
-    steps:
-      - run:
-          command: >-
-            minikube start --vm-driver=docker --cpus 4 --memory 6144  --insecure-registry ${LOCAL_IP}:5000
-          name: Start Minikube Cluster
-
-  minikube-start-load-balancer:
-    description: Starts the minikube tunnel
-    steps:
-      - run:
-          command: minikube tunnel
-          name: Start Minikube Tunnel
-          background: true
-
-  # By default, we grep for out of memory messages.  To get the full output,
-  # just set the argument 're' to '.'
-  minikube-logs:
-    description: Tail minikube logs, grepping for something (by default, OOM)
-    parameters:
-      re:
-        # This is for egrep, so you can do things like "(oom|error)"
-        default: "(out of memory|oom.?kill)"
-        type: string
-    steps:
-    - run:
-        name: Tail and grep minikube logs
-        command: minikube logs -f | egrep -i "<<parameters.re>>"
-        background: true
-
-  prepare_for_local_cluster_tests:
-    description: install right versions of go, docker, kubectl, and also build
-    steps:
-      - run:
-          name: Saving local ip
-          command: |
-            LOCAL_IP=$(ip addr show | grep inet\ | grep -vE 'lo$|docker' | awk '{print $2}' | awk -F '/' '{print $1}')
-            echo "export LOCAL_IP=${LOCAL_IP}" >> ${BASH_ENV}
-            echo "Local IP: ${LOCAL_IP}"
-            source $BASH_ENV
-      - run:
-          name: Export environment variables persistent in execution shell
-          command: |
-            echo 'export KUBECONFIG=/home/circleci/.kube/config' >> $BASH_ENV
-            echo 'export GOROOT=/usr/local/go' >> $BASH_ENV
-            echo 'export GOPATH=$HOME/go' >> $BASH_ENV
-            echo 'export PATH=$GOPATH/bin:$GOROOT/bin:$PATH' >> $BASH_ENV
-            echo "export SKUPPER_CONTROLLER_IMAGE=${LOCAL_IP}:5000/controller" >> $BASH_ENV
-            echo "export SKUPPER_CONFIG_SYNC_IMAGE=${LOCAL_IP}:5000/config-sync" >> $BASH_ENV
-            echo "export SKUPPER_NETWORK_CONSOLE_COLLECTOR_IMAGE=${LOCAL_IP}:5000/network-console-collector" >> $BASH_ENV
-            echo "export SKUPPER_BOOTSTRAP_IMAGE=${LOCAL_IP}:5000/bootstrap" >> $BASH_ENV
-            echo "export TEST_IMAGE=${LOCAL_IP}:5000/skupper-tests" >> $BASH_ENV
-            source $BASH_ENV
-      - checkout
-      - run:
-          name: cleanup previous go installation
-          command: sudo rm -rf /usr/local/go
-      - docker/install-docker
-      - go/install:
-          version: "1.22.8"
-      - kube-orb/install-kubectl
-      - run: make
-
-  local_registry_start:
-    description: prepare a local registry using pre-built images
-    steps:
-      - run:
-          name: defining insecure registry location
-          command: |
-            echo "{\"insecure-registries\": [\"0.0.0.0:5000\", \"${LOCAL_IP}:5000\"]}" | jq > /tmp/daemon.json
-            sudo cp /tmp/daemon.json /etc/docker/
-            sudo systemctl restart docker
-      - run:
-          name: create registry container
-          command: |
-            docker run --name registry -d -p 5000:5000 registry
-      - attach_workspace:
-          at: /tmp/
-      - run:
-          name: load docker images
-          command: |
-            gunzip -c /tmp/images/controller.gz | docker import - 0.0.0.0:5000/controller
-            gunzip -c /tmp/images/controller.gz | docker load
-            gunzip -c /tmp/images/config-sync.gz | docker import - 0.0.0.0:5000/config-sync
-            gunzip -c /tmp/images/config-sync.gz | docker load
-            gunzip -c /tmp/images/network-console-collector.gz | docker import - 0.0.0.0:5000/network-console-collector
-            gunzip -c /tmp/images/network-console-collector.gz | docker load
-            gunzip -c /tmp/images/bootstrap.gz | docker import - 0.0.0.0:5000/bootstrap
-            gunzip -c /tmp/images/bootstrap.gz | docker load
-      - run:
-          name: push to local registry
-          command: |
-            docker push 0.0.0.0:5000/controller
-            docker push 0.0.0.0:5000/config-sync
-            docker push 0.0.0.0:5000/network-console-collector
-            docker push 0.0.0.0:5000/bootstrap
-
-  system_monitor:
-    description: shows continuous system state
-    steps:
-    - run:
-        name: sar info
-        command: sar -h -q -r -u 60
-        background: true
-    - run:
-        name: vmstat info
-        command: vmstat -w -t 10
-        background: true
-    - run:
-        name: journalctl monitoring
-        command: journalctl -p 7 -b -f
-        background: true
-
-  system_status:
-    description: shows some point-in-time system status
-    parameters:
-      cluster_dump_re:
-        default: "(oom|error)"
-        type: string
-    steps:
-    - run:
-        name: point-in-time system status
-        command: |
-          set +e
-          cat /sys/fs/cgroup/memory/memory.max_usage_in_bytes
-          free -m
-          uptime
-          minikube kubectl cluster-info dump | grep -i "<<parameters.cluster_dump_re>>"
-          true
-        when: always
-
-  run_cluster_tests:
-    description: run all e2e tests inside the current KUBECONFIG configured cluster
-    parameters:
-      args:
-        default: ""
-        type: string
-    steps:
-      - system_status # before the tests
-      - run:
-          name: Creating artifacts directory
-          command: mkdir /tmp/artifacts
-      - run:
-          name: Creating results directory
-          command: mkdir /tmp/results
-      - run:
-          name: Run Integration Tests
-          no_output_timeout: 60m
-          command: |
-            make build-cmd && sudo install skupper /usr/local/bin
-            export PUBLIC_1_INGRESS_HOST=`minikube ip`
-            export SKUPPER_TEST_CLI_MAX_STATUS_ATTEMPTS=20
-            go test -tags=integration,podman -timeout=60m -v ./test/integration/... | tee /tmp/results/test-integration.out
-      - run:
-          name: Run client tests in real cluster
-          command: |
-            go test -v -count=1 -timeout=20m ./client -use-cluster | tee /tmp/results/test-client.out
-      - run:
-          name: Run skupper cli tests in real cluster
-          command: |
-            go test -v -count=1 ./cmd/skupper -use-cluster | tee /tmp/results/test-cmd-skupper.out
-      - run:
-          name: Run site-controller tests in real cluster
-          command: |
-            go test -v -count=1 ./cmd/site-controller -use-cluster | tee /tmp/results/test-cmd-site-controller.out
-      - system_status
-      - run:
-          name: Archiving test artifacts
-          command: |
-            find ./ -name "*.tar.gz" -o -name "*.log" -o -name "*.json" | tar -c -T - | tar -C /tmp/artifacts -xv
-          when: always
-      - run:
-          name: Generate junit reports
-          command: |
-            (cd /tmp/results && for f in *.out; do [[ -f $f ]] && cat $f | ~/go/bin/go-junit-report > ${f/.out/.xml}; done)
-          when: always
-      - store_artifacts:
-          path: /tmp/artifacts
-          destination: test-artifacts
-      - store_test_results:
-          path: /tmp/results
-
-  run_cluster_policy_tests:
-    description: run all e2e policy tests inside the current KUBECONFIG configured cluster
-    parameters:
-      args:
-        default: ""
-        type: string
+  skopeo-install:
+    description: Install Skopeo
     steps:
-      - system_status
-      - run:
-          name: Creating policy artifacts directory
-          command: mkdir /tmp/policy-artifacts
-      - run:
-          name: Creating policy results directory
-          command: mkdir /tmp/policy-results
-      - run:
-          name: Run Policy Integration Tests
-          no_output_timeout: 60m
-          command: |
-            make build-cmd && sudo install skupper /usr/local/bin
-            go test -tags=policy -timeout=60m -v ./test/integration/... | tee /tmp/policy-results/test-policy.out
-      - system_status
-      - run:
-          name: Archiving policy test artifacts
-          command: |
-            find ./ -name "*.tar.gz" -o -name "*.log" -o -name "*.json" | tar -c -T - | tar -C /tmp/policy-artifacts -xv
-          when: always
       - run:
-          name: Generate policy junit reports
-          command: |
-            (cd /tmp/policy-results && for f in *.out; do [[ -f $f ]] && cat $f | ~/go/bin/go-junit-report > ${f/.out/.xml}; done)
-          when: always
-      - store_artifacts:
-          path: /tmp/policy-artifacts
-          destination: policy-test-artifacts
-      - store_test_results:
-          path: /tmp/policy-results
+          name: Apt Install Skopeo
+          command: sudo apt-get update -y && sudo apt-get install -y skopeo
 
   compile_go_program:
     description: Compile specified platform.
@@ -296,54 +51,11 @@ commands:
             file dist/<< parameters.platform >>/skupper<< parameters.exesuffix >>;
             go version -m dist/<< parameters.platform >>/skupper<< parameters.exesuffix >>;
 
-  podman-latest:
-    description: "Install latest podman v4 or higher"
-    steps:
-      - run:
-          name: install podman
-          command: |
-            sudo mkdir -p /etc/apt/keyrings
-            curl -fsSL https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/unstable/xUbuntu_$(lsb_release -rs)/Release.key \
-              | gpg --dearmor \
-              | sudo tee /etc/apt/keyrings/devel_kubic_libcontainers_unstable.gpg > /dev/null
-            echo \
-              "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/devel_kubic_libcontainers_unstable.gpg]\
-                https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/unstable/xUbuntu_$(lsb_release -rs)/ /" \
-              | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:unstable.list > /dev/null
-            sudo apt-get update -qq
-            sudo apt-get -qq -y install podman
-            podman version
-            
-            # temporary fix for https://github.com/containers/podman/issues/21024
-            wget https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_22.04/amd64/conmon_2.1.2~0_amd64.deb -O /tmp/conmon_2.1.2.deb
-            sudo apt install /tmp/conmon_2.1.2.deb
-
-            # Bypassing CircleCI issue with user session (see: https://github.com/containers/podman/issues/16529)
-            mkdir -p ~/.config/containers/containers.conf.d
-            ( echo '[containers]' ; echo 'cgroups = "disabled"' ) > ~/.config/containers/containers.conf.d/podman-circleci-issue.conf
-
-            # Starting systemd user service
-            systemctl --user start podman.socket
-
-  go-junit:
-    description: "Install go-junit-reporter"
-    steps:
-      - run:
-          name: go junit reporter
-          command: |
-            go install github.com/jstemmer/go-junit-report@v1.0.0
-
 yaml-templates:
-  branch_filters: &run_for_all_branches_and_numeric_tags
+  branch_filters: &run_for_all_branches
     filters:
       tags:
-        only: /[0-9].*/
-
-  main_branch_filters: &run_for_main_branch
-    filters:
-      branches:
         ignore: /.*/
-        only: /main/
 
   v2_branch_filters: &run_for_v2_branch
     filters:
@@ -358,104 +70,91 @@ yaml-templates:
       tags:
         only: /[0-9].*/
 
-  release_requires: &release_requires
-    requires:
-      - build-all
-      - test-skupper-binary
-      - test
-      - main_tests_minikube_local_cluster
-      - policy_tests_minikube_local_cluster
-
   v2_release_requires: &v2_release_requires
     requires:
       - build-all
       - test-skupper-binary
       - test
+      - build-oci-images
 
 workflows:
   version: 2.1
-  build-workflow:
+  release:
     jobs:
       - build-all:
-          <<: *run_for_all_branches_and_numeric_tags
+         <<: *run_for_numeric_tags
       - test-skupper-binary:
-          <<: *run_for_all_branches_and_numeric_tags
+          <<: *run_for_numeric_tags
           matrix:
             parameters:
               image:
                 - quay.io/centos/centos:stream8
                 - quay.io/centos/centos:stream9
                 - quay.io/fedora/fedora:38
-                - quay.io/fedora/fedora:39
+                - quay.io/fedora/fedora:41
+                - docker.io/debian:bookworm
+                - docker.io/alpine:latest
           requires:
             - build-all
       - test:
-          <<: *run_for_all_branches_and_numeric_tags
-      - build_and_save_test_images:
-          <<: *run_for_all_branches_and_numeric_tags
-
-      # - main_tests_minikube_local_cluster:
-      #     <<: *run_for_all_branches_and_numeric_tags
-      #     pre-steps:
-      #       - prepare_for_local_cluster_tests
-      #     requires:
-      #       - test
-      #       - build_and_save_test_images
-
-      # - policy_tests_minikube_local_cluster:
-      #     <<: *run_for_all_branches_and_numeric_tags
-      #     pre-steps:
-      #       - prepare_for_local_cluster_tests
-      #     requires:
-      #       - test
-      #       - build_and_save_test_images
-
-      - publish-github-release-images:
-         <<: *run_for_numeric_tags
-         <<: *v2_release_requires
-         context:
-           - skupper-org
-
-      - generate-manifest:
+          <<: *run_for_numeric_tags
+      - build-oci-images:
+          <<: *run_for_numeric_tags
+          image_tag: << pipeline.git.tag >>
+      - publish-oci-images:
           <<: *run_for_numeric_tags
           <<: *v2_release_requires
-          requires:
-            - publish-github-release-images
+          image_tag: << pipeline.git.tag >>
           context:
             - skupper-org
-
+      - generate-manifest:
+          <<: *run_for_numeric_tags
+          requires:
+            - publish-oci-images
       - publish-github-release-artifacts:
           <<: *run_for_numeric_tags
-          <<: *v2_release_requires
           requires:
+            - publish-oci-images
             - generate-manifest
-            - publish-github-release-images
           context:
             - skupper-org
-
-      # - publish-github-main-artifacts:
-      #     <<: *run_for_main_branch
-      #     requires:
-      #       - publish-github-main-images
-      #     context:
-      #       - skupper-org
-
-      # - publish-github-main-images:
-      #     <<: *run_for_main_branch
-      #     <<: *release_requires
-      #     context:
-      #       - skupper-org
-
-      - publish-github-v2-artifacts:
-          <<: *run_for_v2_branch
+  build:
+    jobs:
+      - build-all:
+          <<: *run_for_all_branches
+      - test-skupper-binary:
+          <<: *run_for_all_branches
+          matrix:
+            parameters:
+              image:
+                - quay.io/centos/centos:stream8
+                - quay.io/centos/centos:stream9
+                - quay.io/fedora/fedora:38
+                - quay.io/fedora/fedora:41
+                - docker.io/debian:bookworm
+                - docker.io/alpine:latest
           requires:
-            - publish-github-v2-latest-images
+            - build-all
+      - test:
+          <<: *run_for_all_branches
+      - build-oci-images:
+          image_tag: v2-latest
+          <<: *run_for_all_branches
+      - publish-oci-images:
+          <<: *run_for_v2_branch
+          <<: *v2_release_requires
+          image_tag: v2-latest
           context:
             - skupper-org
-
-      - publish-github-v2-latest-images:
+      - generate-manifest:
+          requires:
+            - publish-oci-images
+      - publish-github-prerelease-artifacts:
           <<: *run_for_v2_branch
-          <<: *v2_release_requires
+          version: v2-release
+          requires:
+            - publish-oci-images
+            - generate-manifest
           context:
             - skupper-org
 
@@ -489,29 +188,73 @@ jobs:
           path: /tmp/artifacts
           destination: test-artifacts
 
-  build_and_save_test_images:
+  generate-manifest:
+    executor:
+      name: go_cimg
+    steps:
+      - checkout
+      - go/mod-download-cached
+      - setup_remote_docker
+      - run: make generate-manifest
+      - run: mkdir skupper-manifest
+      - run: cp ./manifest.json skupper-manifest
+      - persist_to_workspace:
+          root: .
+          paths:
+            - skupper-manifest
+
+  build-oci-images:
     executor: local_cluster_test_executor
+    parameters:
+      image_tag:
+        type: string
+      registry:
+        type: string
+        default: "quay.io/skupper"
     steps:
       - docker/install-docker
+      - skopeo-install
       - checkout
       - run: docker buildx create --use --name skupper-buildx
-      - run: make -e docker-build
+      - run: make multiarch-oci IMAGE_TAG=<< parameters.image_tag >> REGISTRY=<< parameters.registry >>
       - run:
-          name: persisting images to workspace
+          name: Inspect images
           command: |
-            mkdir /tmp/images
-            docker tag quay.io/skupper/controller:v2-latest 0.0.0.0:5000/controller
-            docker save 0.0.0.0:5000/controller | gzip > /tmp/images/controller.gz
-            docker tag quay.io/skupper/config-sync:v2-latest 0.0.0.0:5000/config-sync
-            docker save 0.0.0.0:5000/config-sync | gzip > /tmp/images/config-sync.gz
-            docker tag quay.io/skupper/network-console-collector:v2-latest 0.0.0.0:5000/network-console-collector
-            docker save 0.0.0.0:5000/network-console-collector | gzip > /tmp/images/network-console-collector.gz
-            docker tag quay.io/skupper/bootstrap:v2-latest 0.0.0.0:5000/bootstrap
-            docker save 0.0.0.0:5000/bootstrap | gzip > /tmp/images/bootstrap.gz
+            mkdir -p oci-digests
+            make describe-multiarch-oci \
+              IMAGE_TAG=<< parameters.image_tag >> \
+              REGISTRY=<< parameters.registry >> \
+              | tee ./oci-digests/digests.yaml
       - persist_to_workspace:
-          root: /tmp
+          root: ./
           paths:
-            - images/
+            - oci-archives/
+            - oci-digests/
+
+  publish-oci-images:
+    executor:
+      name: go_cimg
+    environment:
+      REGISTRY_AUTH_FILE: ~/.skopeoauth.json
+    parameters:
+      image_tag:
+        type: string
+      registry:
+        type: string
+        default: "quay.io/skupper"
+    steps:
+      - skopeo-install
+      - checkout
+      - attach_workspace:
+          at: .
+      - run:
+          name: skopeo login
+          command: |
+            skopeo login quay.io -u ${QUAY_LOGIN} -p ${QUAY_PASSWORD}
+      - run:
+          name: publish images
+          command: |
+            make describe-multiarch-oci push-multiarch-oci REGISTRY="<< parameters.registry >>" IMAGE_TAG="<< parameters.image_tag >>"
 
   build-all:
     executor:
@@ -528,17 +271,6 @@ jobs:
           goarch: amd64
           platform: linux-amd64
 
-      - compile_go_program:
-          goos: linux
-          goarch: "386"
-          platform: linux-i386
-
-      - compile_go_program:
-          goos: windows
-          goarch: "386"
-          platform: windows-i386
-          exesuffix: ".exe"
-
       - compile_go_program:
           goos: windows
           goarch: amd64
@@ -555,11 +287,6 @@ jobs:
           goarch: arm64
           platform: mac-arm64
 
-      - compile_go_program:
-          goos: linux
-          goarch: arm
-          platform: linux-arm32
-
       - compile_go_program:
           goos: linux
           goarch: arm64
@@ -591,50 +318,6 @@ jobs:
             cd dist/linux-amd64
             ./skupper site
 
-  main_tests_minikube_local_cluster:
-    executor: local_cluster_test_executor
-    steps:
-      - run: echo "skupper_image = ${SKUPPER_CONTROLLER_IMAGE}"
-      - local_registry_start
-      - minikube-install
-      - minikube-start-large
-      - minikube-start-load-balancer
-      - podman-latest
-      - go-junit
-      - minikube-logs
-      - system_monitor
-      - run: kubectl cluster-info
-      - run_cluster_tests
-
-  policy_tests_minikube_local_cluster:
-    executor: local_cluster_policy_test_executor
-    steps:
-      - run: echo "skupper_image = ${SKUPPER_CONTROLLER_IMAGE}"
-      - local_registry_start
-      - minikube-install
-      - minikube-start-medium
-      - minikube-start-load-balancer
-      - minikube-logs
-      - go-junit
-      - system_monitor
-      - run: kubectl cluster-info
-      - run_cluster_policy_tests
-
-  generate-manifest:
-    executor:
-      name: go_cimg
-    steps:
-      - checkout
-      - go/mod-download-cached
-      - setup_remote_docker
-      - run: make generate-manifest
-      - run: mkdir skupper-manifest
-      - run: cp ./manifest.json skupper-manifest
-      - persist_to_workspace:
-          root: .
-          paths:
-            - skupper-manifest
-
   publish-github-release-artifacts:
     docker:
       - image: cibuilds/github:0.10
@@ -647,6 +330,7 @@ jobs:
             VERSION="${CIRCLE_TAG}"
             BASEDIR=`pwd`
             mkdir "${BASEDIR}/archives"
+            cp ./oci-digests/* "${BASEDIR}/archives"
             for p in `ls dist` ; do
               cd "$BASEDIR/dist/$p"
               if [[ $p == windows* ]] ; then
@@ -659,98 +343,22 @@ jobs:
             cp "${BASEDIR}/skupper-manifest/manifest.json" "${BASEDIR}/archives"
             ghr -t ${GITHUB_TOKEN} -u ${CIRCLE_PROJECT_USERNAME} -r ${CIRCLE_PROJECT_REPONAME} -c ${CIRCLE_SHA1} -replace -prerelease -draft ${VERSION} "${BASEDIR}/archives"
 
-  publish-github-release-images:
-    executor:
-      name: go_cimg
-    steps:
-      - checkout
-      - setup_remote_docker
-      - run: docker login quay.io -u ${QUAY_LOGIN} -p ${QUAY_PASSWORD}
-      - run:
-          name:
-          command: |
-            echo 'export CONTROLLER_IMAGE=quay.io/skupper/controller:${CIRCLE_TAG}' >> $BASH_ENV
-            echo 'export CONFIG_SYNC_IMAGE=quay.io/skupper/config-sync:${CIRCLE_TAG}' >> $BASH_ENV
-            echo 'export NETWORK_CONSOLE_COLLECTOR_IMAGE=quay.io/skupper/network-console-collector:${CIRCLE_TAG}' >> $BASH_ENV
-            echo 'export BOOTSTRAP_IMAGE=quay.io/skupper/bootstrap:${CIRCLE_TAG}' >> $BASH_ENV
-            source $BASH_ENV
-            docker buildx create --use --name skupper-buildx
-            make -e docker-build
-            make -e docker-push
-
-  publish-github-main-images:
-    executor:
-      name: go_cimg
-    steps:
-      - checkout
-      - setup_remote_docker
-      - run: docker login quay.io -u ${QUAY_LOGIN} -p ${QUAY_PASSWORD}
-      - run:
-          name: "Publishing main images"
-          command: |
-            echo 'export CONTROLLER_IMAGE=quay.io/skupper/controller:main' >> $BASH_ENV
-            echo 'export CONFIG_SYNC_IMAGE=quay.io/skupper/config-sync:main' >> $BASH_ENV
-            echo 'export NETWORK_CONSOLE_COLLECTOR_IMAGE=quay.io/skupper/network-console-collector:main' >> $BASH_ENV
-            echo 'export BOOTSTRAP_IMAGE=quay.io/skupper/bootstrap:main' >> $BASH_ENV
-            source $BASH_ENV
-            docker buildx create --use --name skupper-buildx
-            make -e docker-build
-            make -e docker-push
-
-  publish-github-main-artifacts:
-    docker:
-      - image: cibuilds/github:0.10
-    steps:
-      - attach_workspace:
-          at: .
-      - run:
-          name: "Create a Pre-Release on GitHub"
-          command: |
-            VERSION="main-release"
-            BASEDIR=`pwd`
-            mkdir "${BASEDIR}/archives"
-            for p in `ls dist` ; do
-              cd "$BASEDIR/dist/$p"
-              if [[ $p == windows* ]] ; then
-                zip -q "${BASEDIR}/archives/skupper-cli-${VERSION}-$p.zip" *
-              else
-                tar -zcf "${BASEDIR}/archives/skupper-cli-${VERSION}-$p.tgz" *
-              fi
-            done
-            cd ${BASEDIR}
-            ghr -t ${GITHUB_TOKEN} -u ${CIRCLE_PROJECT_USERNAME} -r ${CIRCLE_PROJECT_REPONAME} -c ${CIRCLE_SHA1} -replace -prerelease ${VERSION} "${BASEDIR}/archives"
-
-  publish-github-v2-latest-images:
-    executor:
-      name: go_cimg
-    steps:
-      - checkout
-      - setup_remote_docker
-      - run: docker login quay.io -u ${QUAY_LOGIN} -p ${QUAY_PASSWORD}
-      - run:
-          name: "Publishing v2 images"
-          command: |
-            echo 'export CONTROLLER_IMAGE=quay.io/skupper/controller:v2-latest' >> $BASH_ENV
-            echo 'export CONFIG_SYNC_IMAGE=quay.io/skupper/config-sync:v2-latest' >> $BASH_ENV
-            echo 'export NETWORK_CONSOLE_COLLECTOR_IMAGE=quay.io/skupper/network-console-collector:v2-latest' >> $BASH_ENV
-            echo 'export BOOTSTRAP_IMAGE=quay.io/skupper/bootstrap:v2-latest' >> $BASH_ENV
-            source $BASH_ENV
-            docker buildx create --use --name skupper-buildx
-            make -e docker-build
-            make -e docker-push
-
-  publish-github-v2-artifacts:
+  publish-github-prerelease-artifacts:
     docker:
       - image: cibuilds/github:0.10
+    parameters:
+      version:
+        type: string
     steps:
       - attach_workspace:
           at: .
       - run:
           name: "Create a V2 Pre-Release on GitHub"
           command: |
-            VERSION="v2-release"
+            VERSION="<< parameters.version >>"
             BASEDIR=`pwd`
             mkdir "${BASEDIR}/archives"
+            cp ./oci-digests/* "${BASEDIR}/archives"
             for p in `ls dist` ; do
               cd "$BASEDIR/dist/$p"
               if [[ $p == windows* ]] ; then
@@ -760,4 +368,5 @@ jobs:
               fi
             done
             cd ${BASEDIR}
+            cp "${BASEDIR}/skupper-manifest/manifest.json" "${BASEDIR}/archives"
             ghr -t ${GITHUB_TOKEN} -u ${CIRCLE_PROJECT_USERNAME} -r ${CIRCLE_PROJECT_REPONAME} -c ${CIRCLE_SHA1} -replace -prerelease ${VERSION} "${BASEDIR}/archives"
diff --git a/.gitignore b/.gitignore
index dc689ba5..8bb8a660 100644
--- a/.gitignore
+++ b/.gitignore
@@ -10,6 +10,7 @@
 /generate-doc
 /bootstrap
 /controller
+/network-console-collector
 
 # Test binary, built with `go test -c`
 *.test
@@ -28,3 +29,4 @@
 
 # Generated files
 manifest.json
+oci-archives/
diff --git a/Dockerfile.bootstrap b/Dockerfile.bootstrap
index f1e1f547..0887295f 100644
--- a/Dockerfile.bootstrap
+++ b/Dockerfile.bootstrap
@@ -1,6 +1,5 @@
 FROM --platform=$BUILDPLATFORM golang:1.22 AS builder
 
-ARG TARGETOS
 ARG TARGETARCH
 
 WORKDIR /go/src/app
@@ -9,9 +8,13 @@ COPY go.sum .
 RUN go mod download
 COPY . .
 
-RUN make build-bootstrap GOOS=$TARGETOS GOARCH=$TARGETARCH
+ENV CGO_ENABLED=0
+RUN make build-bootstrap GOARCH=$TARGETARCH
 
 FROM --platform=$TARGETPLATFORM registry.access.redhat.com/ubi9-minimal
+LABEL \
+  org.opencontainers.image.title="Skupper Bootstrap" \
+  org.opencontainers.image.description="Produces and manges static configurations for non-kube installations of Skupper"
 
 # Create user and group and switch to user's context
 RUN microdnf -y install shadow-utils \
diff --git a/Dockerfile.config-sync b/Dockerfile.config-sync
index ae863a7e..f678aa6e 100644
--- a/Dockerfile.config-sync
+++ b/Dockerfile.config-sync
@@ -1,6 +1,5 @@
 FROM --platform=$BUILDPLATFORM golang:1.22 AS builder
 
-ARG TARGETOS
 ARG TARGETARCH
 
 WORKDIR /go/src/app
@@ -10,9 +9,13 @@ COPY go.sum .
 RUN go mod download
 
 COPY . .
-RUN make GOOS=$TARGETOS GOARCH=$TARGETARCH
+ENV CGO_ENABLED=0
+RUN make GOARCH=$TARGETARCH build-config-sync
 
 FROM --platform=$TARGETPLATFORM registry.access.redhat.com/ubi9-minimal
+LABEL \
+  org.opencontainers.image.title="Skupper Router Adaptor" \
+  org.opencontainers.image.description="Kubernetes aware interface for the skupper-router"
 
 # Create user and group and switch to user's context
 RUN microdnf -y install shadow-utils \
diff --git a/Dockerfile.controller b/Dockerfile.controller
index 1d41b421..e5929f19 100644
--- a/Dockerfile.controller
+++ b/Dockerfile.controller
@@ -1,6 +1,5 @@
 FROM --platform=$BUILDPLATFORM golang:1.22 AS builder
 
-ARG TARGETOS
 ARG TARGETARCH
 
 WORKDIR /go/src/app
@@ -9,9 +8,13 @@ COPY go.sum .
 RUN go mod download
 COPY . .
 
-RUN make build-controller GOOS=$TARGETOS GOARCH=$TARGETARCH
+ENV CGO_ENABLED=0
+RUN make build-controller GOARCH=$TARGETARCH
 
 FROM --platform=$TARGETPLATFORM registry.access.redhat.com/ubi9-minimal
+LABEL \
+  org.opencontainers.image.title="Skupper Controller" \
+  org.opencontainers.image.description="Kubernetes controller for operating skupper networks"
 
 # Create user and group and switch to user's context
 RUN microdnf -y install shadow-utils \
diff --git a/Dockerfile.network-console-collector b/Dockerfile.network-console-collector
index 7b4a0c95..1ed9c64a 100644
--- a/Dockerfile.network-console-collector
+++ b/Dockerfile.network-console-collector
@@ -1,6 +1,5 @@
 FROM --platform=$BUILDPLATFORM golang:1.22 AS builder
 
-ARG TARGETOS
 ARG TARGETARCH
 
 WORKDIR /go/src/app
@@ -11,9 +10,13 @@ RUN go mod download
 
 COPY . .
 
-RUN make GOOS=$TARGETOS GOARCH=$TARGETARCH build-network-console-collector
+ENV CGO_ENABLED=0
+RUN make GOARCH=$TARGETARCH build-network-console-collector
 
 FROM --platform=$BUILDPLATFORM node:18.18.2 AS console-builder
+LABEL \
+  org.opencontainers.image.title="Skupper Network Console Collector" \
+  org.opencontainers.image.description="Exposes Skupper network details thorugh an API, metrics and a web console"
 
 WORKDIR /skupper-console/
 ADD https://github.com/skupperproject/skupper-console/archive/v2.tar.gz .
diff --git a/Makefile b/Makefile
index b15498a7..87911363 100644
--- a/Makefile
+++ b/Makefile
@@ -1,27 +1,31 @@
 VERSION := $(shell git describe --tags --dirty=-modified --always)
-CONTROLLER_IMAGE := quay.io/skupper/controller:v2-latest
-BOOTSTRAP_IMAGE := quay.io/skupper/bootstrap:v2-latest
-CONFIG_SYNC_IMAGE := quay.io/skupper/config-sync:v2-latest
-NETWORK_CONSOLE_COLLECTOR_IMAGE := quay.io/skupper/network-console-collector:v2-latest
-TEST_IMAGE := quay.io/skupper/skupper-tests:v2-latest
-TEST_BINARIES_FOLDER := ${PWD}/test/integration/bin
-DOCKER := docker
-LDFLAGS := -X github.com/skupperproject/skupper/pkg/version.Version=${VERSION}
+REVISION := $(shell git rev-parse HEAD)
+
+LDFLAGS_EXTRA ?= -s -w # default to building stripped executables
+LDFLAGS := ${LDFLAGS_EXTRA} -X github.com/skupperproject/skupper/pkg/version.Version=${VERSION}
 TESTFLAGS := -v -race -short
-PLATFORMS ?= linux/amd64,linux/arm64
 GOOS ?= linux
 GOARCH ?= amd64
 
-all: build-cmd build-config-sync build-controller build-bootstrap build-tests build-manifest build-network-console-collector update-helm-crd
+REGISTRY := quay.io/skupper
+IMAGE_TAG := v2-latest
+PLATFORMS ?= linux/amd64,linux/arm64
+CONTAINERFILES := Dockerfile.bootstrap Dockerfile.config-sync Dockerfile.controller Dockerfile.network-console-collector
+SHARED_IMAGE_LABELS = \
+    --label "org.opencontainers.image.created=$(shell TZ=GMT date --iso-8601=seconds)" \
+	--label "org.opencontainers.image.url=https://skupper.io/" \
+	--label "org.opencontainers.image.documentation=https://skupper.io/" \
+	--label "org.opencontainers.image.source=https://github.com/skupperproject/skupper" \
+	--label "org.opencontainers.image.version=${VERSION}" \
+	--label "org.opencontainers.image.revision=${REVISION}" \
+	--label "org.opencontainers.image.licenses=Apache-2.0"
+
+
+DOCKER := docker
+SKOPEO := skopeo
+PODMAN := podman
 
-build-tests:
-	mkdir -p ${TEST_BINARIES_FOLDER}
-#	GOOS=${GOOS} GOARCH=${GOARCH} go test -c -tags=job -v ./test/integration/examples/tcp_echo/job -o ${TEST_BINARIES_FOLDER}/tcp_echo_test
-#	GOOS=${GOOS} GOARCH=${GOARCH} go test -c -tags=job -v ./test/integration/examples/http/job -o ${TEST_BINARIES_FOLDER}/http_test
-#	GOOS=${GOOS} GOARCH=${GOARCH} go test -c -tags=job -v ./test/integration/examples/bookinfo/job -o ${TEST_BINARIES_FOLDER}/bookinfo_test
-#	GOOS=${GOOS} GOARCH=${GOARCH} go test -c -tags=job -v ./test/integration/examples/mongodb/job -o ${TEST_BINARIES_FOLDER}/mongo_test
-#	GOOS=${GOOS} GOARCH=${GOARCH} go test -c -tags=job -v ./test/integration/examples/custom/hipstershop/job -o ${TEST_BINARIES_FOLDER}/grpcclient_test
-#	GOOS=${GOOS} GOARCH=${GOARCH} go test -c -tags=job -v ./test/integration/examples/tls_t/job -o ${TEST_BINARIES_FOLDER}/tls_test
+all: build-cmd build-config-sync build-controller build-bootstrap build-manifest build-network-console-collector update-helm-crd
 
 build-cmd:
 	GOOS=${GOOS} GOARCH=${GOARCH} go build -ldflags="${LDFLAGS}"  -o skupper ./cmd/skupper
@@ -30,10 +34,10 @@ build-bootstrap:
 	GOOS=${GOOS} GOARCH=${GOARCH} go build -ldflags="${LDFLAGS}"  -o bootstrap ./cmd/bootstrap
 
 build-controller:
-	go build -ldflags="${LDFLAGS}"  -o controller cmd/controller/main.go cmd/controller/controller.go
+	GOOS=${GOOS} GOARCH=${GOARCH} go build -ldflags="${LDFLAGS}"  -o controller ./cmd/controller
 
 build-config-sync:
-	GOOS=${GOOS} GOARCH=${GOARCH} go build -ldflags="${LDFLAGS}"  -o config-sync cmd/config-sync/main.go cmd/config-sync/config_sync.go cmd/config-sync/collector.go
+	GOOS=${GOOS} GOARCH=${GOARCH} go build -ldflags="${LDFLAGS}"  -o config-sync ./cmd/config-sync
 
 build-network-console-collector:
 	GOOS=${GOOS} GOARCH=${GOARCH} go build -ldflags="${LDFLAGS}"  -o network-console-collector ./cmd/network-console-collector
@@ -44,36 +48,50 @@ build-manifest:
 build-doc-generator:
 	GOOS=${GOOS} GOARCH=${GOARCH} go build -ldflags="${LDFLAGS}"  -o generate-doc ./internal/cmd/generate-doc
 
-docker-build-test-image:
-	${DOCKER} buildx build --platform ${PLATFORMS} -t ${TEST_IMAGE} -f Dockerfile.ci-test .
-	${DOCKER} buildx build --load -t ${TEST_IMAGE} -f Dockerfile.ci-test .
-
-docker-build: docker-build-test-image docker-build-bootstrap docker-build-network-console-collector
-	${DOCKER} buildx build --platform ${PLATFORMS} -t ${CONTROLLER_IMAGE} -f Dockerfile.controller .
-	${DOCKER} buildx build --load  -t ${CONTROLLER_IMAGE} -f Dockerfile.controller .
-	${DOCKER} buildx build --platform ${PLATFORMS} -t ${CONFIG_SYNC_IMAGE} -f Dockerfile.config-sync .
-	${DOCKER} buildx build --load  -t ${CONFIG_SYNC_IMAGE} -f Dockerfile.config-sync .
-
-docker-build-bootstrap:
-	${DOCKER} buildx build --platform ${PLATFORMS} -t ${BOOTSTRAP_IMAGE} -f Dockerfile.bootstrap .
-	${DOCKER} buildx build --load  -t ${BOOTSTRAP_IMAGE} -f Dockerfile.bootstrap .
-
-docker-push-bootstrap:
-	${DOCKER} buildx build --push --platform ${PLATFORMS} -t ${BOOTSTRAP_IMAGE} -f Dockerfile.bootstrap .
-
-docker-build-network-console-collector:
-	${DOCKER} buildx build --platform ${PLATFORMS} -t ${NETWORK_CONSOLE_COLLECTOR_IMAGE} -f Dockerfile.network-console-collector .
-	${DOCKER} buildx build --load  -t ${NETWORK_CONSOLE_COLLECTOR_IMAGE} -f Dockerfile.network-console-collector .
-
-docker-push-network-console-collector:
-	${DOCKER} buildx build --push --platform ${PLATFORMS} -t ${NETWORK_CONSOLE_COLLECTOR_IMAGE} -f Dockerfile.network-console-collector .
-
-docker-push-test-image:
-	${DOCKER} buildx build --push --platform ${PLATFORMS} -t ${TEST_IMAGE} -f Dockerfile.ci-test .
-
-docker-push: docker-push-test-image docker-push-bootstrap docker-push-network-console-collector
-	${DOCKER} buildx build --push --platform ${PLATFORMS} -t ${CONFIG_SYNC_IMAGE} -f Dockerfile.config-sync .
-	${DOCKER} buildx build --push --platform ${PLATFORMS} -t ${CONTROLLER_IMAGE} -f Dockerfile.controller .
+## native/default container image builds
+docker-build: $(patsubst Dockerfile.%,docker-build-%,$(CONTAINERFILES))
+docker-build-%: Dockerfile.%
+	${DOCKER} build $(SHARED_IMAGE_LABELS) -t "${REGISTRY}/$*:${IMAGE_TAG}" -f $< .
+
+podman-build: $(patsubst Dockerfile.%,podman-build-%,$(CONTAINERFILES))
+podman-build-%: Dockerfile.%
+	${PODMAN} build $(SHARED_IMAGE_LABELS) -t "${REGISTRY}/$*:${IMAGE_TAG}" -f $< .
+
+
+## multi-platform container images built in docker buildkit builder and
+# exported to oci archive format.
+multiarch-oci: $(patsubst Dockerfile.%,multiarch-oci-%,$(CONTAINERFILES))
+multiarch-oci-%: Dockerfile.% oci-archives
+	${DOCKER} buildx build \
+		"--output=type=oci,dest=$(shell pwd)/oci-archives/$*.tar" \
+		-t "${REGISTRY}/$*:${IMAGE_TAG}" \
+		$(SHARED_IMAGE_LABELS) \
+		--platform ${PLATFORMS} \
+		-f $< .
+
+## push multiarch-oci images to a registry using skopeo
+push-multiarch-oci: $(patsubst Dockerfile.%,push-multiarch-oci-%,$(CONTAINERFILES))
+push-multiarch-oci-%: ./oci-archives/%.tar
+	${SKOPEO} copy --all \
+		oci-archive:$< \
+		"docker://${REGISTRY}/$*:${IMAGE_TAG}"
+
+## Load images from oci-archive into local image storage
+podman-load-oci:
+	for archive in ./oci-archives/*.tar; do ${PODMAN} load < "$$archive"; done
+## Has unfortunate podman dependency; docker image load does not load OCI archives, while podman does.
+docker-load-oci:
+	for archive in ./oci-archives/*.tar; do \
+		img=$$(${PODMAN} load -q < "$$archive" | awk -F": " '{print $$2}') \
+		&& ${PODMAN} image save "$$img" | ${DOCKER} load; \
+	done
+
+## Print fully qualified image names by arch
+describe-multiarch-oci:
+	@scripts/oci-index-archive-info.sh amd64 arm64
+
+oci-archives:
+	mkdir -p oci-archives
 
 format:
 	go fmt ./...
@@ -98,46 +116,16 @@ cover:
 		-coverprofile cover.out \
 		./...
 
-clean:
-	rm -rf skupper controller release config-sync manifest bootstrap network-console-collector ${TEST_BINARIES_FOLDER}
-
-package: release/windows.zip release/darwin.zip release/linux.tgz release/s390x.tgz release/arm64.tgz
-
-release/linux.tgz: release/linux/skupper
-	tar -czf release/linux.tgz -C release/linux/ skupper
-
-release/linux/skupper: cmd/skupper/skupper.go
-	GOOS=linux GOARCH=amd64 go build -ldflags="${LDFLAGS}" -o release/linux/skupper ./cmd/skupper
-
-release/windows/skupper: cmd/skupper/skupper.go
-	GOOS=windows GOARCH=amd64 go build -ldflags="${LDFLAGS}" -o release/windows/skupper ./cmd/skupper
-
-release/windows.zip: release/windows/skupper
-	zip -j release/windows.zip release/windows/skupper
-
-release/darwin/skupper: cmd/skupper/skupper.go
-	GOOS=darwin GOARCH=amd64 go build -ldflags="${LDFLAGS}" -o release/darwin/skupper ./cmd/skupper
-
-release/darwin.zip: release/darwin/skupper
-	zip -j release/darwin.zip release/darwin/skupper
-
 generate-manifest: build-manifest
 	./manifest
 
 generate-doc: build-doc-generator
 	./generate-doc ./doc/cli
 
-release/s390x/skupper: cmd/skupper/skupper.go
-	GOOS=linux GOARCH=s390x go build -ldflags="${LDFLAGS}" -o release/s390x/skupper ./cmd/skupper
-
-release/s390x.tgz: release/s390x/skupper
-	tar -czf release/s390x.tgz release/s390x/skupper
-
-release/arm64/skupper: cmd/skupper/skupper.go
-	GOOS=linux GOARCH=arm64 go build -ldflags="${LDFLAGS}" -o release/arm64/skupper ./cmd/skupper
-
-release/arm64.tgz: release/arm64/skupper
-	tar -czf release/arm64.tgz release/arm64/skupper
-
 update-helm-crd:
 	./scripts/update-helm-crds.sh
+
+clean:
+	rm -rf skupper controller config-sync manifest \
+		bootstrap network-console-collector generate-doc \
+		cover.out oci-archives
diff --git a/cmd/bootstrap/README.md b/cmd/bootstrap/README.md
index da7ba160..33244c3e 100644
--- a/cmd/bootstrap/README.md
+++ b/cmd/bootstrap/README.md
@@ -65,10 +65,10 @@ files are stored, like certificates, configurations, the original sources
 (original custom resources used to bootstrap the nonkube site) and
 the runtime files generated during initialization.
 
-Namespaces are stored under ${XDG_DATA_HOME}/.local/share/skupper/namespaces
+Namespaces are stored under ${XDG_DATA_HOME}/skupper/namespaces
 for regular users when XDG_DATA_HOME environment variable is set, or under
 ${HOME}/.local/share/skupper/namespaces when it is not set.
-As the root user, namespaces are stored under: /usr/local/share/skupper/namespaces.
+As the root user, namespaces are stored under: /var/lib/skupper/namespaces.
 
 In case the path (-p) flag is omitted, Skupper will try to process
 custom resources stored at the sources directory of the default namespace,
diff --git a/cmd/bootstrap/bootstrap.go b/cmd/bootstrap/bootstrap.go
index ebfb872e..3116f9c1 100644
--- a/cmd/bootstrap/bootstrap.go
+++ b/cmd/bootstrap/bootstrap.go
@@ -43,10 +43,10 @@ files are stored, like certificates, configurations, the original sources
 (original custom resources used to bootstrap the nonkube site) and
 the runtime files generated during initialization.
 
-Namespaces are stored under ${XDG_DATA_HOME}/.local/share/skupper/namespaces
+Namespaces are stored under ${XDG_DATA_HOME}/skupper/namespaces
 for regular users when XDG_DATA_HOME environment variable is set, or under
 ${HOME}/.local/share/skupper/namespaces when it is not set.
-As the root user, namespaces are stored under: /usr/local/share/skupper/namespaces.
+As the root user, namespaces are stored under: /var/lib/skupper/namespaces.
 
 In case the path (-p) flag is omitted, Skupper will try to process
 custom resources stored at the sources directory of the default namespace,
@@ -102,7 +102,7 @@ func main() {
 	// definition based on CR files, if an input path has been provided.
 	// The /output path must be mapped to the Host's XDG_DATA_HOME/skupper
 	// or $HOME/.local/share/skupper (non-root)
-	// and /usr/local/share/skupper (root).
+	// and /var/lib/skupper (root).
 	//
 	fmt.Printf("Skupper nonkube bootstrap (version: %s)\n", version.Version)
 
diff --git a/cmd/bootstrap/bootstrap.sh b/cmd/bootstrap/bootstrap.sh
index 489f2c6a..07d7aa6e 100755
--- a/cmd/bootstrap/bootstrap.sh
+++ b/cmd/bootstrap/bootstrap.sh
@@ -79,7 +79,7 @@ container_env() {
             export CONTAINER_ENDPOINT_DEFAULT="unix:///run/podman/podman.sock"
         fi
         export USERNS=""
-        export SKUPPER_OUTPUT_PATH="/usr/local/share/skupper"
+        export SKUPPER_OUTPUT_PATH="/var/lib/skupper"
         export SERVICE_DIR="/etc/systemd/system"
     fi
     mkdir -p "${SKUPPER_OUTPUT_PATH}"
diff --git a/cmd/bootstrap/remove.sh b/cmd/bootstrap/remove.sh
index 62bd5a28..0874deea 100755
--- a/cmd/bootstrap/remove.sh
+++ b/cmd/bootstrap/remove.sh
@@ -16,7 +16,7 @@ namespaces_path="${XDG_DATA_HOME:-${HOME}/.local/share}/skupper/namespaces"
 service_path="${XDG_CONFIG_HOME:-${HOME}/.config}/systemd/user"
 systemctl="systemctl --user"
 if [ "${UID}" -eq 0 ]; then
-    namespaces_path="/usr/local/share/skupper/namespaces"
+    namespaces_path="/var/lib/skupper/namespaces"
     service_path="/etc/systemd/system"
     systemctl="systemctl"
 fi
diff --git a/internal/cmd/skupper/common/flags.go b/internal/cmd/skupper/common/flags.go
index 33befd22..97e8cfe8 100644
--- a/internal/cmd/skupper/common/flags.go
+++ b/internal/cmd/skupper/common/flags.go
@@ -189,3 +189,10 @@ type CommandListenerStatusFlags struct {
 type CommandListenerDeleteFlags struct {
 	Timeout time.Duration
 }
+
+type CommandVersionFlags struct {
+	Output string
+}
+
+type CommandDebugFlags struct {
+}
diff --git a/internal/cmd/skupper/debug/debug.go b/internal/cmd/skupper/debug/debug.go
new file mode 100644
index 00000000..c5273690
--- /dev/null
+++ b/internal/cmd/skupper/debug/debug.go
@@ -0,0 +1,43 @@
+package debug
+
+import (
+	"github.com/skupperproject/skupper/api/types"
+	"github.com/skupperproject/skupper/internal/cmd/skupper/common"
+	"github.com/skupperproject/skupper/internal/cmd/skupper/debug/kube"
+	"github.com/skupperproject/skupper/internal/cmd/skupper/debug/nonkube"
+
+	"github.com/skupperproject/skupper/pkg/config"
+	"github.com/spf13/cobra"
+)
+
+func NewCmdDebug() *cobra.Command {
+
+	cmd := CmdDebugFactory(config.GetPlatform())
+
+	return cmd
+}
+
+func CmdDebugFactory(configuredPlatform types.Platform) *cobra.Command {
+	kubeCommand := kube.NewCmdDebug()
+	nonKubeCommand := nonkube.NewCmdDebug()
+
+	cmdDebugDesc := common.SkupperCmdDescription{
+		Use:   "debug",
+		Short: "",
+		Long:  "",
+	}
+
+	cmd := common.ConfigureCobraCommand(configuredPlatform, cmdDebugDesc, kubeCommand, nonKubeCommand)
+
+	cmdFlags := common.CommandDebugFlags{}
+
+	//Add flags if necessary
+
+	kubeCommand.CobraCmd = cmd
+	kubeCommand.Flags = &cmdFlags
+	nonKubeCommand.CobraCmd = cmd
+	nonKubeCommand.Flags = &cmdFlags
+
+	return cmd
+
+}
diff --git a/internal/cmd/skupper/debug/kube/debug.go b/internal/cmd/skupper/debug/kube/debug.go
new file mode 100644
index 00000000..136050c1
--- /dev/null
+++ b/internal/cmd/skupper/debug/kube/debug.go
@@ -0,0 +1,42 @@
+package kube
+
+import (
+	"fmt"
+	"github.com/skupperproject/skupper/internal/cmd/skupper/common"
+	"github.com/skupperproject/skupper/internal/kube/client"
+	skupperv2alpha1 "github.com/skupperproject/skupper/pkg/generated/client/clientset/versioned/typed/skupper/v2alpha1"
+	"github.com/spf13/cobra"
+	"k8s.io/client-go/kubernetes"
+)
+
+type CmdDebug struct {
+	Client     skupperv2alpha1.SkupperV2alpha1Interface
+	KubeClient kubernetes.Interface
+	CobraCmd   *cobra.Command
+	Flags      *common.CommandDebugFlags
+	Namespace  string
+}
+
+func NewCmdDebug() *CmdDebug {
+
+	skupperCmd := CmdDebug{}
+
+	return &skupperCmd
+}
+
+func (cmd *CmdDebug) NewClient(cobraCommand *cobra.Command, args []string) {
+	cli, err := client.NewClient(cobraCommand.Flag("namespace").Value.String(), cobraCommand.Flag("context").Value.String(), cobraCommand.Flag("kubeconfig").Value.String())
+	if err == nil {
+		cmd.Client = cli.GetSkupperClient().SkupperV2alpha1()
+		cmd.KubeClient = cli.GetKubeClient()
+		cmd.Namespace = cli.Namespace
+	}
+}
+
+func (cmd *CmdDebug) ValidateInput(args []string) []error { return nil }
+
+func (cmd *CmdDebug) InputToOptions() {}
+
+func (cmd *CmdDebug) Run() error { fmt.Println("not yet implemented"); return nil }
+
+func (cmd *CmdDebug) WaitUntil() error { return nil }
diff --git a/internal/cmd/skupper/debug/nonkube/debug.go b/internal/cmd/skupper/debug/nonkube/debug.go
new file mode 100644
index 00000000..18cf4dc3
--- /dev/null
+++ b/internal/cmd/skupper/debug/nonkube/debug.go
@@ -0,0 +1,32 @@
+package nonkube
+
+import (
+	"fmt"
+	"github.com/skupperproject/skupper/internal/cmd/skupper/common"
+	"github.com/spf13/cobra"
+)
+
+type CmdDebug struct {
+	CobraCmd  *cobra.Command
+	Flags     *common.CommandDebugFlags
+	Namespace string
+}
+
+func NewCmdDebug() *CmdDebug {
+
+	skupperCmd := CmdDebug{}
+
+	return &skupperCmd
+}
+
+func (cmd *CmdDebug) NewClient(cobraCommand *cobra.Command, args []string) {
+
+}
+
+func (cmd *CmdDebug) ValidateInput(args []string) []error { return nil }
+
+func (cmd *CmdDebug) InputToOptions() {}
+
+func (cmd *CmdDebug) Run() error { fmt.Println("not yet implemented"); return nil }
+
+func (cmd *CmdDebug) WaitUntil() error { return nil }
diff --git a/internal/cmd/skupper/root/root.go b/internal/cmd/skupper/root/root.go
index 29b6bd0d..add52cda 100644
--- a/internal/cmd/skupper/root/root.go
+++ b/internal/cmd/skupper/root/root.go
@@ -4,10 +4,12 @@ import (
 	"github.com/skupperproject/skupper/api/types"
 	"github.com/skupperproject/skupper/internal/cmd/skupper/common"
 	"github.com/skupperproject/skupper/internal/cmd/skupper/connector"
+	"github.com/skupperproject/skupper/internal/cmd/skupper/debug"
 	"github.com/skupperproject/skupper/internal/cmd/skupper/link"
 	"github.com/skupperproject/skupper/internal/cmd/skupper/listener"
 	"github.com/skupperproject/skupper/internal/cmd/skupper/site"
 	"github.com/skupperproject/skupper/internal/cmd/skupper/token"
+	"github.com/skupperproject/skupper/internal/cmd/skupper/version"
 	"github.com/skupperproject/skupper/pkg/config"
 	"github.com/spf13/cobra"
 )
@@ -30,6 +32,8 @@ func NewSkupperRootCommand() *cobra.Command {
 	rootCmd.AddCommand(listener.NewCmdListener())
 	rootCmd.AddCommand(link.NewCmdLink())
 	rootCmd.AddCommand(connector.NewCmdConnector())
+	rootCmd.AddCommand(version.NewCmdVersion())
+	rootCmd.AddCommand(debug.NewCmdDebug())
 
 	return rootCmd
 }
diff --git a/internal/cmd/skupper/version/kube/version.go b/internal/cmd/skupper/version/kube/version.go
new file mode 100644
index 00000000..f4059276
--- /dev/null
+++ b/internal/cmd/skupper/version/kube/version.go
@@ -0,0 +1,43 @@
+package kube
+
+import (
+	"fmt"
+	"github.com/skupperproject/skupper/internal/cmd/skupper/common"
+	"github.com/skupperproject/skupper/internal/kube/client"
+	skupperv2alpha1 "github.com/skupperproject/skupper/pkg/generated/client/clientset/versioned/typed/skupper/v2alpha1"
+	"github.com/spf13/cobra"
+	"k8s.io/client-go/kubernetes"
+)
+
+type CmdVersion struct {
+	Client     skupperv2alpha1.SkupperV2alpha1Interface
+	KubeClient kubernetes.Interface
+	CobraCmd   *cobra.Command
+	Flags      *common.CommandVersionFlags
+	Namespace  string
+}
+
+func NewCmdVersion() *CmdVersion {
+
+	skupperCmd := CmdVersion{}
+
+	return &skupperCmd
+}
+
+func (cmd *CmdVersion) NewClient(cobraCommand *cobra.Command, args []string) {
+	cli, err := client.NewClient(cobraCommand.Flag("namespace").Value.String(), cobraCommand.Flag("context").Value.String(), cobraCommand.Flag("kubeconfig").Value.String())
+
+	if err == nil {
+		cmd.Client = cli.GetSkupperClient().SkupperV2alpha1()
+		cmd.KubeClient = cli.GetKubeClient()
+		cmd.Namespace = cli.Namespace
+	}
+}
+
+func (cmd *CmdVersion) ValidateInput(args []string) []error { return nil }
+
+func (cmd *CmdVersion) InputToOptions() {}
+
+func (cmd *CmdVersion) Run() error { fmt.Println("not yet implemented"); return nil }
+
+func (cmd *CmdVersion) WaitUntil() error { return nil }
diff --git a/internal/cmd/skupper/version/nonkube/version.go b/internal/cmd/skupper/version/nonkube/version.go
new file mode 100644
index 00000000..5811a908
--- /dev/null
+++ b/internal/cmd/skupper/version/nonkube/version.go
@@ -0,0 +1,32 @@
+package nonkube
+
+import (
+	"fmt"
+	"github.com/skupperproject/skupper/internal/cmd/skupper/common"
+	"github.com/spf13/cobra"
+)
+
+type CmdVersion struct {
+	CobraCmd  *cobra.Command
+	Flags     *common.CommandVersionFlags
+	Namespace string
+}
+
+func NewCmdVersion() *CmdVersion {
+
+	skupperCmd := CmdVersion{}
+
+	return &skupperCmd
+}
+
+func (cmd *CmdVersion) NewClient(cobraCommand *cobra.Command, args []string) {
+
+}
+
+func (cmd *CmdVersion) ValidateInput(args []string) []error { return nil }
+
+func (cmd *CmdVersion) InputToOptions() {}
+
+func (cmd *CmdVersion) Run() error { fmt.Println("not yet implemented"); return nil }
+
+func (cmd *CmdVersion) WaitUntil() error { return nil }
diff --git a/internal/cmd/skupper/version/version.go b/internal/cmd/skupper/version/version.go
new file mode 100644
index 00000000..b99b5d70
--- /dev/null
+++ b/internal/cmd/skupper/version/version.go
@@ -0,0 +1,44 @@
+package version
+
+import (
+	"github.com/skupperproject/skupper/api/types"
+	"github.com/skupperproject/skupper/internal/cmd/skupper/common"
+	"github.com/skupperproject/skupper/internal/cmd/skupper/version/nonkube"
+
+	"github.com/skupperproject/skupper/internal/cmd/skupper/version/kube"
+
+	"github.com/skupperproject/skupper/pkg/config"
+	"github.com/spf13/cobra"
+)
+
+func NewCmdVersion() *cobra.Command {
+
+	cmd := CmdVersionFactory(config.GetPlatform())
+
+	return cmd
+}
+
+func CmdVersionFactory(configuredPlatform types.Platform) *cobra.Command {
+	kubeCommand := kube.NewCmdVersion()
+	nonKubeCommand := nonkube.NewCmdVersion()
+
+	cmdVersionDesc := common.SkupperCmdDescription{
+		Use:   "version",
+		Short: "",
+		Long:  "",
+	}
+
+	cmd := common.ConfigureCobraCommand(configuredPlatform, cmdVersionDesc, kubeCommand, nonKubeCommand)
+
+	cmdFlags := common.CommandVersionFlags{}
+
+	cmd.Flags().StringVarP(&cmdFlags.Output, common.FlagNameOutput, "o", "", common.FlagDescOutput)
+
+	kubeCommand.CobraCmd = cmd
+	kubeCommand.Flags = &cmdFlags
+	nonKubeCommand.CobraCmd = cmd
+	nonKubeCommand.Flags = &cmdFlags
+
+	return cmd
+
+}
diff --git a/internal/nonkube/bundle/install.sh.template b/internal/nonkube/bundle/install.sh.template
index 4e279f87..833d7abe 100644
--- a/internal/nonkube/bundle/install.sh.template
+++ b/internal/nonkube/bundle/install.sh.template
@@ -30,7 +30,7 @@ export USERNS="keep-id"
 GID=$(id -g "${UID}")
 export RUNAS="${UID}:${GID}"
 if [ "${UID}" -eq 0 ]; then
-    export SKUPPER_OUTPUT_PATH="/usr/local/share/skupper"
+    export SKUPPER_OUTPUT_PATH="/var/lib/skupper"
     export SERVICE_DIR="/etc/systemd/system"
     export RUNTIME_DIR="/run"
     export SYSTEMCTL="systemctl"
diff --git a/pkg/config/platform.go b/pkg/config/platform.go
index 67dc8909..cf10f683 100644
--- a/pkg/config/platform.go
+++ b/pkg/config/platform.go
@@ -161,7 +161,7 @@ func GetPlatform() types.Platform {
 
 func getDataHome() string {
 	if os.Getuid() == 0 {
-		return "/usr/local/share/skupper"
+		return "/var/lib/skupper"
 	}
 	dataHome, ok := os.LookupEnv("XDG_DATA_HOME")
 	if !ok {
diff --git a/pkg/nonkube/api/environment.go b/pkg/nonkube/api/environment.go
index 6d8783f7..50620d4f 100644
--- a/pkg/nonkube/api/environment.go
+++ b/pkg/nonkube/api/environment.go
@@ -33,7 +33,7 @@ var getUid IdGetter = os.Getuid
 
 func GetDataHome() string {
 	if getUid() == 0 {
-		return "/usr/local/share/skupper"
+		return "/var/lib/skupper"
 	}
 	dataHome, ok := os.LookupEnv("XDG_DATA_HOME")
 	if !ok {
diff --git a/pkg/nonkube/bundle/site_state_renderer.go b/pkg/nonkube/bundle/site_state_renderer.go
index 5cca695b..0f8e3464 100644
--- a/pkg/nonkube/bundle/site_state_renderer.go
+++ b/pkg/nonkube/bundle/site_state_renderer.go
@@ -2,7 +2,9 @@ package bundle
 
 import (
 	_ "embed"
+	"encoding/json"
 	"fmt"
+	"log/slog"
 	"os"
 	"path"
 
@@ -126,12 +128,24 @@ func (s *SiteStateRenderer) prepareContainers() error {
 		//      validate whether CPU and memory thresholds can be
 		//      set to the container
 	}
+	logger := common.NewLogger()
+	if logger.Enabled(nil, slog.LevelDebug) {
+		for name, newContainer := range s.containers {
+			containerJson, _ := json.Marshal(newContainer)
+			logger.Debug("container prepared:",
+				slog.String("name", name),
+				slog.String("container", string(containerJson)),
+			)
+		}
+	}
 	return nil
 }
 
 func (s *SiteStateRenderer) createContainerScript() error {
+	logger := common.NewLogger()
 	scriptsPath := api.GetInternalBundleOutputPath(s.siteState.Site.Namespace, api.RuntimeScriptsPath)
 	scriptContent := containersToShell(s.containers)
+	logger.Debug("writing containers_create.sh", slog.String("path", scriptsPath))
 	err := os.WriteFile(path.Join(scriptsPath, "containers_create.sh"), scriptContent, 0755)
 	if err != nil {
 		return fmt.Errorf("failed to create containers script: %v", err)
@@ -140,6 +154,7 @@ func (s *SiteStateRenderer) createContainerScript() error {
 }
 
 func (s *SiteStateRenderer) createBundle() error {
+	var logger = common.NewLogger()
 	bundlesHomeDir := api.GetDefaultOutputBundlesPath()
 	siteHomeDir := api.GetDefaultBundleOutputPath(s.siteState.Site.Namespace)
 	tarball := utils.NewTarball()
@@ -162,6 +177,7 @@ func (s *SiteStateRenderer) createBundle() error {
 			OutputPath: bundlesHomeDir,
 		}
 	}
+	logger.Debug("generating bundle:", slog.String("path", bundlesHomeDir), slog.String("site", s.siteState.Site.Name))
 	err = generator.Generate(tarball, string(s.Platform))
 	if err != nil {
 		return fmt.Errorf("failed to generate site bundle (%q): %v", s.siteState.Site.Name, err)
@@ -170,7 +186,9 @@ func (s *SiteStateRenderer) createBundle() error {
 }
 
 func (s *SiteStateRenderer) removeSiteFiles() error {
+	logger := common.NewLogger()
 	siteHomeDir := api.GetDefaultBundleOutputPath(s.siteState.Site.Namespace)
+	logger.Debug("removing temporary bundle home", slog.String("path", siteHomeDir))
 	err := os.RemoveAll(siteHomeDir)
 	if err != nil {
 		return fmt.Errorf("file to remove temporary site directory %q: %v", siteHomeDir, err)
@@ -179,7 +197,9 @@ func (s *SiteStateRenderer) removeSiteFiles() error {
 }
 
 func (s *SiteStateRenderer) createFreePortScript() error {
+	logger := common.NewLogger()
 	scriptsPath := api.GetInternalBundleOutputPath(s.siteState.Site.Namespace, api.RuntimeScriptsPath)
+	logger.Debug("writing freeport.sh", slog.String("path", scriptsPath))
 	err := os.WriteFile(path.Join(scriptsPath, "router_free_port.py"), []byte(FreePortScript), 0755)
 	if err != nil {
 		return fmt.Errorf("failed to create router_free_port.py script: %v", err)
diff --git a/pkg/nonkube/bundle/systemd.go b/pkg/nonkube/bundle/systemd.go
index 1a072616..74defad0 100644
--- a/pkg/nonkube/bundle/systemd.go
+++ b/pkg/nonkube/bundle/systemd.go
@@ -4,6 +4,7 @@ import (
 	"bytes"
 	_ "embed"
 	"fmt"
+	"log/slog"
 	"os"
 	"path"
 	"text/template"
@@ -15,6 +16,7 @@ import (
 
 func CreateSystemdServices(siteState *api.SiteState) error {
 	var err error
+	var logger = common.NewLogger()
 	serviceTemplates := map[string]string{
 		"systemd":   common.SystemdServiceTemplate,
 		"container": common.SystemdContainerServiceTemplate,
@@ -36,6 +38,7 @@ func CreateSystemdServices(siteState *api.SiteState) error {
 			return fmt.Errorf("failed to execute %s service template: %w", platform, err)
 		}
 		serviceFile := path.Join(scriptsPath, fmt.Sprintf("skupper.service.%s", platform))
+		logger.Debug("writing systemd service file", slog.String("path", serviceFile))
 		err = os.WriteFile(serviceFile, buf.Bytes(), 0644)
 		if err != nil {
 			return fmt.Errorf("failed to write %s service file: %w", platform, err)
diff --git a/pkg/nonkube/common/claim_redeemer.go b/pkg/nonkube/common/claim_redeemer.go
index c1225661..e7de0d09 100644
--- a/pkg/nonkube/common/claim_redeemer.go
+++ b/pkg/nonkube/common/claim_redeemer.go
@@ -6,6 +6,7 @@ import (
 	"crypto/x509"
 	"fmt"
 	"io"
+	"log/slog"
 	"net/http"
 	"strings"
 	"time"
@@ -21,11 +22,15 @@ import (
 func RedeemClaims(siteState *api.SiteState) error {
 	var errs []error
 
+	logger := NewLogger()
 	for name, claim := range siteState.Claims {
 		err := redeemAccessToken(claim, siteState)
 		if err != nil {
 			errs = append(errs, fmt.Errorf("failed to redeem claim %s: %w", name, err))
-			fmt.Printf("RedeemClaims: failed to redeem claim: %s: %s\n", name, err)
+			logger.Error("RedeemClaims: failed to redeem claim",
+				slog.String("name", name),
+				slog.String("error", err.Error()),
+			)
 		}
 	}
 
diff --git a/pkg/nonkube/common/fs_config_renderer.go b/pkg/nonkube/common/fs_config_renderer.go
index bd609d42..bc20108d 100644
--- a/pkg/nonkube/common/fs_config_renderer.go
+++ b/pkg/nonkube/common/fs_config_renderer.go
@@ -3,7 +3,7 @@ package common
 import (
 	"errors"
 	"fmt"
-	"log"
+	"log/slog"
 	"os"
 	"path"
 	"strconv"
@@ -67,6 +67,7 @@ func NewFileSystemConfigurationRenderer(outputPath string) *FileSystemConfigurat
 // Render simply renders the given site state as configuration files.
 func (c *FileSystemConfigurationRenderer) Render(siteState *api.SiteState) error {
 	var err error
+	logger := NewLogger()
 	outputPath := c.GetOutputPath(siteState)
 	if c.SslProfileBasePath == "" {
 		c.SslProfileBasePath = DefaultSslProfileBasePath
@@ -117,7 +118,9 @@ func (c *FileSystemConfigurationRenderer) Render(siteState *api.SiteState) error
 	// Saving runtime platform
 	if !c.Bundle {
 		content := fmt.Sprintf("platform: %s\n", c.Platform)
-		err = os.WriteFile(path.Join(outputPath, string(api.RuntimeSiteStatePath), "platform.yaml"), []byte(content), 0644)
+		platformPath := path.Join(outputPath, string(api.RuntimeSiteStatePath), "platform.yaml")
+		logger.Debug("writing platform", slog.String("platform", c.Platform), slog.String("path", platformPath))
+		err = os.WriteFile(platformPath, []byte(content), 0644)
 		if err != nil {
 			return fmt.Errorf("failed to write runtime platform: %w", err)
 		}
@@ -127,6 +130,11 @@ func (c *FileSystemConfigurationRenderer) Render(siteState *api.SiteState) error
 	for raName, ra := range siteState.RouterAccesses {
 		for _, role := range ra.Spec.Roles {
 			if role.Name != "normal" {
+				logger.Debug("site endpoint configured:",
+					slog.String("host", ra.Spec.BindHost),
+					slog.String("role", role.Name),
+					slog.Int("port", role.Port),
+				)
 				endpoints = append(endpoints, v2alpha1.Endpoint{
 					Name: fmt.Sprintf("%s-%s", raName, role.Name),
 					Host: ra.Spec.BindHost,
@@ -162,23 +170,27 @@ func (c *FileSystemConfigurationRenderer) GetInputPath(siteState *api.SiteState)
 }
 
 func (c *FileSystemConfigurationRenderer) MarshalSiteStates(loadedSiteState, runtimeSiteState *api.SiteState) error {
+	logger := NewLogger()
 	if loadedSiteState != nil {
 		outputPath := c.GetOutputPath(loadedSiteState)
 		sourcesPath := path.Join(outputPath, string(api.LoadedSiteStatePath))
 		inputSourcesPath := path.Join(outputPath, string(api.InputSiteStatePath))
-		existingSources, _ := new(utils.DirectoryReader).ReadDir(inputSourcesPath, nil)
+		existingLoadedSources, _ := new(utils.DirectoryReader).ReadDir(sourcesPath, nil)
+		existingInputSources, _ := new(utils.DirectoryReader).ReadDir(inputSourcesPath, nil)
 		// when sources are already defined, we back them up
-		if len(existingSources) > 0 {
+		if len(existingLoadedSources) > 0 {
 			tb := utils.NewTarball()
-			err := tb.AddFiles(inputSourcesPath)
+			err := tb.AddFiles(sourcesPath)
 			if err != nil {
 				return fmt.Errorf("unable to backup existing sources: %s", err)
 			}
-			tbFile := path.Join(outputPath, "input", "sources.backup.tar.gz")
+			tbFile := path.Join(outputPath, "sources.backup.tar.gz")
+			logger.Debug("saving loaded state sources", slog.String("path", tbFile))
 			err = tb.Save(tbFile)
 			if err != nil {
 				return fmt.Errorf("unable to backup sources.backup.tar.gz: %s", err)
 			}
+			logger.Debug("clean up previous loaded state", slog.String("path", sourcesPath))
 			err = os.RemoveAll(sourcesPath)
 			if err != nil {
 				return fmt.Errorf("unable to remove former loaded state: %s", err)
@@ -186,17 +198,21 @@ func (c *FileSystemConfigurationRenderer) MarshalSiteStates(loadedSiteState, run
 			if err = os.Mkdir(sourcesPath, 0755); err != nil {
 				return fmt.Errorf("unable to recreate sources directory %s: %s", sourcesPath, err)
 			}
-		} else {
+		}
+		if len(existingInputSources) == 0 {
+			logger.Debug("creating input sources", slog.String("path", inputSourcesPath))
 			if err := api.MarshalSiteState(*loadedSiteState, inputSourcesPath); err != nil {
 				return err
 			}
 		}
+		logger.Debug("saving loaded state", slog.String("path", sourcesPath))
 		if err := api.MarshalSiteState(*loadedSiteState, sourcesPath); err != nil {
 			return err
 		}
 	}
 	outputPath := c.GetOutputPath(runtimeSiteState)
 	runtimeStatePath := path.Join(outputPath, string(api.RuntimeSiteStatePath))
+	logger.Debug("saving runtime state", slog.String("path", runtimeStatePath))
 	if err := api.MarshalSiteState(*runtimeSiteState, runtimeStatePath); err != nil {
 		return err
 	}
@@ -204,9 +220,12 @@ func (c *FileSystemConfigurationRenderer) MarshalSiteStates(loadedSiteState, run
 }
 
 func CleanupNamespaceForReload(namespace string) error {
+	logger := NewLogger()
+	logger.Debug("cleaning up namespace for reload")
 	// Re-create internal configuration directories
 	for _, dir := range reloadDirectories {
 		outputPath := api.GetInternalOutputPath(namespace, dir)
+		logger.Debug("recreating directory:", slog.String("path", outputPath))
 		if err := os.RemoveAll(outputPath); err != nil {
 			return fmt.Errorf("failed to remove directory %s: %w", outputPath, err)
 		}
@@ -238,6 +257,7 @@ func RestoreNamespaceData(data []byte) error {
 }
 
 func (c *FileSystemConfigurationRenderer) createTokens(siteState *api.SiteState) error {
+	logger := NewLogger()
 	tokens := make([]*api.Token, 0)
 	for name, linkAccess := range siteState.RouterAccesses {
 		noInterRouterRole := linkAccess.FindRole("inter-router") == nil
@@ -270,10 +290,11 @@ func (c *FileSystemConfigurationRenderer) createTokens(siteState *api.SiteState)
 		tokenPath := path.Join(outputPath, string(api.RuntimeTokenPath), tokenFileName)
 		tokenYaml, err := token.Marshal()
 		if err != nil {
-			return fmt.Errorf("unable to marshal token: %v", err)
+			return fmt.Errorf("unable to marshal static link: %v", err)
 		}
+		logger.Debug("writing static link", slog.String("path", tokenPath))
 		if err := os.WriteFile(tokenPath, tokenYaml, 0644); err != nil {
-			return fmt.Errorf("unable to create token file %s: %v", tokenPath, err)
+			return fmt.Errorf("unable to create static link file %s: %v", tokenPath, err)
 		}
 	}
 	return nil
@@ -289,6 +310,7 @@ func (c *FileSystemConfigurationRenderer) createRouterConfig(siteState *api.Site
 	}
 	outputPath := c.GetOutputPath(siteState)
 	routerConfigFileName := path.Join(outputPath, string(api.ConfigRouterPath), "skrouterd.json")
+	NewLogger().Debug("Writing router configuration", slog.String("path", routerConfigFileName))
 	err = os.WriteFile(routerConfigFileName, []byte(routerConfigJson), 0644)
 	if err != nil {
 		return fmt.Errorf("unable to write router config file: %v", err)
@@ -298,6 +320,7 @@ func (c *FileSystemConfigurationRenderer) createRouterConfig(siteState *api.Site
 
 func (c *FileSystemConfigurationRenderer) createTlsCertificates(siteState *api.SiteState) error {
 	var err error
+	var logger = NewLogger()
 	writeSecretFilesIgnore := func(basePath string, secret *corev1.Secret, ignoreExisting bool) error {
 		baseDir, err := os.Open(basePath)
 		if err != nil {
@@ -323,10 +346,11 @@ func (c *FileSystemConfigurationRenderer) createTlsCertificates(siteState *api.S
 				// ignoring existing certificate
 				_ = certFile.Close()
 				if ignoreExisting {
-					log.Printf("warning: %s will not be overwritten", certFileName)
+					logger.Warn("certificate will not be overwritten", slog.String("path", certFileName))
 					continue
 				}
 			}
+			logger.Debug("writing certificate", slog.String("path", certFileName))
 			err = os.WriteFile(certFileName, data, 0640)
 			if err != nil {
 				return fmt.Errorf("error writing %s: %v", certFileName, err)
diff --git a/pkg/nonkube/common/log.go b/pkg/nonkube/common/log.go
new file mode 100644
index 00000000..66357a89
--- /dev/null
+++ b/pkg/nonkube/common/log.go
@@ -0,0 +1,10 @@
+package common
+
+import (
+	"log/slog"
+)
+
+func NewLogger() *slog.Logger {
+	slog.SetLogLoggerLevel(slog.LevelInfo)
+	return slog.New(slog.Default().Handler())
+}
diff --git a/pkg/nonkube/common/site_state_renderer_common.go b/pkg/nonkube/common/site_state_renderer_common.go
index f549f6f6..543c407e 100644
--- a/pkg/nonkube/common/site_state_renderer_common.go
+++ b/pkg/nonkube/common/site_state_renderer_common.go
@@ -64,6 +64,8 @@ func copySiteStateMap[T any](m map[string]T) map[string]T {
 
 func CreateRouterAccess(siteState *api.SiteState) error {
 	if !siteState.HasRouterAccess() {
+		logger := NewLogger()
+		logger.Debug("Creating skupper-local RouterAccess")
 		name := fmt.Sprintf("skupper-local")
 		var port = 5671
 		var err error
@@ -72,6 +74,7 @@ func CreateRouterAccess(siteState *api.SiteState) error {
 			if err != nil {
 				return err
 			}
+			logger.Debug("Free TCP port discovered", "port", port)
 		}
 		siteState.CreateRouterAccess(name, port)
 	}
diff --git a/pkg/nonkube/common/site_state_validator.go b/pkg/nonkube/common/site_state_validator.go
index 4ee60996..92980b31 100644
--- a/pkg/nonkube/common/site_state_validator.go
+++ b/pkg/nonkube/common/site_state_validator.go
@@ -79,7 +79,7 @@ func (s *SiteStateValidator) validateRouterAccesses(routerAccesses map[string]*v
 			}
 		}
 		if len(routerAccess.Spec.Roles) == 0 {
-			return fmt.Errorf("invalid router access: roles are required")
+			return fmt.Errorf("invalid router access: %s - roles are required", routerAccess.Name)
 		}
 		for _, role := range routerAccess.Spec.Roles {
 			if !utils.StringSliceContains(validLinkAccessRoles, role.Name) {
@@ -135,12 +135,12 @@ func (s *SiteStateValidator) validateListeners(listeners map[string]*v2alpha1.Li
 			return fmt.Errorf("invalid listener name: %w", err)
 		}
 		if listener.Spec.Host == "" || listener.Spec.Port == 0 {
-			return fmt.Errorf("host and port are required")
+			return fmt.Errorf("invalid listener: %s - host and port are required", listener.Name)
 		}
 		ip := net.ParseIP(listener.Spec.Host)
 		validHostname := hostnameRfc1123Regex.MatchString(listener.Spec.Host)
 		if ip == nil && !validHostname {
-			return fmt.Errorf("invalid listener host: %s - a valid IP address or hostname is expected", listener.Spec.Host)
+			return fmt.Errorf("invalid listener host: %s - a valid IP address or hostname is expected (listener: %q)", listener.Spec.Host, name)
 		}
 		if utils.IntSliceContains(hostPorts[listener.Spec.Host], listener.Spec.Port) {
 			return fmt.Errorf("port %d is already mapped for host %q (listener: %q)", listener.Spec.Port, listener.Spec.Host, name)
@@ -159,12 +159,12 @@ func (s *SiteStateValidator) validateConnectors(connectors map[string]*v2alpha1.
 			return fmt.Errorf("invalid connector name: %w", err)
 		}
 		if connector.Spec.Host == "" || connector.Spec.Port == 0 {
-			return fmt.Errorf("connector host and port are required")
+			return fmt.Errorf("connector host and port are required (connector: %q)", connector.Name)
 		}
 		ip := net.ParseIP(connector.Spec.Host)
 		validHostname := hostnameRfc1123Regex.MatchString(connector.Spec.Host)
 		if ip == nil && !validHostname {
-			return fmt.Errorf("invalid connector host: %s - a valid IP address or hostname is expected", connector.Spec.Host)
+			return fmt.Errorf("invalid connector host: %s - a valid IP address or hostname is expected (connector: %q)", connector.Spec.Host, connector.Name)
 		}
 		if connector.Spec.RoutingKey == "" {
 			return fmt.Errorf("routingKey is missing for connector: %s", connector.Name)
diff --git a/pkg/nonkube/common/startup.go b/pkg/nonkube/common/startup.go
index 8ba6293a..440e7327 100644
--- a/pkg/nonkube/common/startup.go
+++ b/pkg/nonkube/common/startup.go
@@ -4,6 +4,7 @@ import (
 	"bytes"
 	_ "embed"
 	"fmt"
+	"log/slog"
 	"os"
 	"path"
 	"text/template"
@@ -72,12 +73,15 @@ func GetStartupScripts(args StartupScriptsArgs, pathProvider api.InternalPathPro
 func (s *startupScripts) Create() error {
 	var startBuf bytes.Buffer
 	var stopBuf bytes.Buffer
+	logger := NewLogger()
+	logger.Debug("creating startup scripts")
 
 	startTemplate := template.Must(template.New("start").Parse(s.StartScript))
 	if err := startTemplate.Execute(&startBuf, s); err != nil {
 		return fmt.Errorf("failed to create start script: %w", err)
 	}
 	startFileName := path.Join(s.path, s.GetStartFileName())
+	logger.Debug("writing start script", slog.String("path", startFileName))
 	err := os.WriteFile(startFileName, startBuf.Bytes(), 0755)
 	if err != nil {
 		return err
@@ -87,6 +91,7 @@ func (s *startupScripts) Create() error {
 		return fmt.Errorf("failed to create stop script: %w", err)
 	}
 	stopFileName := path.Join(s.path, s.GetStopFileName())
+	logger.Debug("writing stop script", slog.String("path", stopFileName))
 	err = os.WriteFile(stopFileName, stopBuf.Bytes(), 0755)
 	if err != nil {
 		return err
diff --git a/pkg/nonkube/common/systemd.go b/pkg/nonkube/common/systemd.go
index 2d6d0bd4..5caa64b3 100644
--- a/pkg/nonkube/common/systemd.go
+++ b/pkg/nonkube/common/systemd.go
@@ -4,6 +4,7 @@ import (
 	"bytes"
 	_ "embed"
 	"fmt"
+	"log/slog"
 	"os"
 	"os/exec"
 	"path"
@@ -83,9 +84,11 @@ func (s *systemdServiceInfo) Create() error {
 		}
 		return fmt.Errorf(msg)
 	}
-
+	var logger = NewLogger()
+	logger.Debug("creating systemd service")
 	var buf = new(bytes.Buffer)
 	var service *template.Template
+	logger.Debug("using service template for:", slog.String("platform", s.platform))
 	if s.platform == string(types.PlatformSystemd) {
 		service = template.Must(template.New(s.GetServiceName()).Parse(SystemdServiceTemplate))
 	} else {
@@ -106,6 +109,7 @@ func (s *systemdServiceInfo) Create() error {
 
 	// Saving systemd user service
 	serviceName := s.GetServiceName()
+	logger.Debug("writing service file", slog.String("path", s.getServiceFile()))
 	err = os.WriteFile(s.getServiceFile(), buf.Bytes(), 0644)
 	if err != nil {
 		return fmt.Errorf("unable to write unit file (%s): %w", s.getServiceFile(), err)
@@ -113,6 +117,7 @@ func (s *systemdServiceInfo) Create() error {
 
 	// Only enable when running locally
 	if !api.IsRunningInContainer() {
+		logger.Debug("enabling systemd service", slog.String("name", serviceName))
 		return s.enableService(serviceName)
 	}
 
@@ -134,25 +139,32 @@ func (s *systemdServiceInfo) Remove() error {
 		return fmt.Errorf("SystemD is not enabled at user level")
 	}
 
+	logger := NewLogger()
+
 	// Stopping systemd user service
 	if !api.IsRunningInContainer() {
+		logger.Debug("stopping service", slog.String("name", s.GetServiceName()))
 		cmd := s.getCmdStopSystemdService(s.GetServiceName())
 		_ = cmd.Run()
 
 		// Disabling systemd user service
+		logger.Debug("disabling service", slog.String("name", s.GetServiceName()))
 		cmd = s.getCmdDisableSystemdService(s.GetServiceName())
 		_ = cmd.Run()
 	}
 
 	// Removing the .service file
+	logger.Debug("removing service", slog.String("path", s.getServiceFile()))
 	_ = os.Remove(s.getServiceFile())
 
 	// Reloading systemd user daemon
 	if !api.IsRunningInContainer() {
+		logger.Debug("reloading systemd daemon")
 		cmd := s.getCmdReloadSystemdDaemon()
 		_ = cmd.Run()
 
 		// Resetting failed status
+		logger.Debug("resetting failed systemd service", slog.String("name", s.GetServiceName()))
 		cmd = s.getCmdResetFailedSystemService(s.GetServiceName())
 		_ = cmd.Run()
 	}
diff --git a/pkg/nonkube/compat/site_state_renderer.go b/pkg/nonkube/compat/site_state_renderer.go
index 9b250bb3..25f0f114 100644
--- a/pkg/nonkube/compat/site_state_renderer.go
+++ b/pkg/nonkube/compat/site_state_renderer.go
@@ -2,7 +2,9 @@ package compat
 
 import (
 	"context"
+	"encoding/json"
 	"fmt"
+	"log/slog"
 	"os"
 	"path"
 	"time"
@@ -28,6 +30,7 @@ type SiteStateRenderer struct {
 
 func (s *SiteStateRenderer) Render(loadedSiteState *api.SiteState, reload bool) error {
 	var err error
+	var logger = common.NewLogger()
 	var validator api.SiteStateValidator = &common.SiteStateValidator{}
 	err = validator.Validate(loadedSiteState)
 	if err != nil {
@@ -47,29 +50,43 @@ func (s *SiteStateRenderer) Render(loadedSiteState *api.SiteState, reload bool)
 		// when reload is successful, backupData must be nil
 		if backupData == nil {
 			for _, temporaryName := range s.stoppedContainers {
+				logger.Debug("Removing temporary container ", slog.String("name", temporaryName))
 				err = s.cli.ContainerRemove(temporaryName)
 				if err != nil {
-					fmt.Printf("Failed to remove temporary container %s: %v\n", temporaryName, err)
+					logger.Error("Failed to remove temporary container",
+						slog.String("name", temporaryName),
+						slog.String("error", err.Error()),
+					)
 				}
 			}
 			return
 		}
-		fmt.Println("Bootstrap failed, restoring previous state")
+		logger.Error("Bootstrap failed, restoring namespace")
 		err := common.RestoreNamespaceData(backupData)
 		if err != nil {
-			fmt.Printf("Error restoring namespace data for %q - %s\n", loadedSiteState.GetNamespace(), err)
+			logger.Error("Error restoring namespace data:",
+				slog.String("namespace", loadedSiteState.GetNamespace()),
+				slog.String("error", err.Error()),
+			)
 			return
 		}
 		for originalName, temporaryName := range s.stoppedContainers {
 			if temporaryName != originalName {
 				err = s.cli.ContainerRename(temporaryName, originalName)
 				if err != nil {
-					fmt.Printf("Error restoring container name from %q to %q - %s\n", temporaryName, originalName, err)
+					logger.Error("Error restoring container:",
+						slog.String("temporary", temporaryName),
+						slog.String("name", originalName),
+						slog.String("error", err.Error()),
+					)
 				}
 			}
 			err = s.cli.ContainerStart(originalName)
 			if err != nil {
-				fmt.Printf("Error starting container %q - %s\n", originalName, err)
+				logger.Error("Error starting container",
+					slog.String("name", originalName),
+					slog.String("error", err.Error()),
+				)
 			}
 		}
 	}()
@@ -211,12 +228,27 @@ func (s *SiteStateRenderer) prepareContainers() error {
 		//      validate whether CPU and memory thresholds can be
 		//      set to the container
 	}
+	logger := common.NewLogger()
+	if logger.Enabled(nil, slog.LevelDebug) {
+		for name, newContainer := range s.containers {
+			containerJson, _ := json.Marshal(newContainer)
+			logger.Debug("container prepared:",
+				slog.String("name", name),
+				slog.String("container", string(containerJson)),
+			)
+		}
+	}
 	return nil
 }
 
 func (s *SiteStateRenderer) pullImages(ctx context.Context) error {
 	var err error
+	var logger = common.NewLogger()
 	for component, skupperContainer := range s.containers {
+		logger.Debug("pulling:",
+			slog.String("image", skupperContainer.Image),
+			slog.String("component", component),
+		)
 		err = s.cli.ImagePull(ctx, skupperContainer.Image)
 		if err != nil {
 			return fmt.Errorf("failed to pull %s image %s: %w", component, skupperContainer.Image, err)
@@ -245,7 +277,12 @@ func (s *SiteStateRenderer) createContainers() error {
 			return fmt.Errorf("container %s already exists (component: %s)", skupperContainer.Name, component)
 		}
 	}
+	logger := common.NewLogger()
 	for component, skupperContainer := range s.containers {
+		logger.Debug("creating container",
+			slog.String("component", component),
+			slog.String("name", skupperContainer.Name),
+		)
 		err = s.cli.ContainerCreate(&skupperContainer)
 		if err != nil {
 			return fmt.Errorf("failed to create %q container (%s): %w", component, skupperContainer.Name, err)
@@ -256,8 +293,13 @@ func (s *SiteStateRenderer) createContainers() error {
 
 func (s *SiteStateRenderer) startContainers() error {
 	var err error
+	var logger = common.NewLogger()
 	defer s.cleanupContainers(err)
 	for component, skupperContainer := range s.containers {
+		logger.Debug("starting container",
+			slog.String("component", component),
+			slog.String("name", skupperContainer.Name),
+		)
 		err = s.cli.ContainerStart(skupperContainer.Name)
 		if err != nil {
 			return fmt.Errorf("failed to start %s container %q: %w", component, skupperContainer.Name, err)
diff --git a/pkg/nonkube/systemd/site_state_renderer.go b/pkg/nonkube/systemd/site_state_renderer.go
index 6e0af91a..6f1d42f9 100644
--- a/pkg/nonkube/systemd/site_state_renderer.go
+++ b/pkg/nonkube/systemd/site_state_renderer.go
@@ -2,6 +2,7 @@ package systemd
 
 import (
 	"fmt"
+	"log/slog"
 	"os"
 
 	"github.com/skupperproject/skupper/api/types"
@@ -18,6 +19,7 @@ type SiteStateRenderer struct {
 
 func (s *SiteStateRenderer) Render(loadedSiteState *api.SiteState, reload bool) error {
 	var err error
+	var logger = common.NewLogger()
 	var validator api.SiteStateValidator = &common.SiteStateValidator{}
 	err = validator.Validate(loadedSiteState)
 	if err != nil {
@@ -34,15 +36,20 @@ func (s *SiteStateRenderer) Render(loadedSiteState *api.SiteState, reload bool)
 		if backupData == nil {
 			return
 		}
-		fmt.Println("Bootstrap failed, restoring previous state")
+		logger.Error("Bootstrap failed, restoring namespace")
 		err := common.RestoreNamespaceData(backupData)
 		if err != nil {
-			fmt.Printf("Error restoring namespace data for %q - %s\n", loadedSiteState.GetNamespace(), err)
+			logger.Error("Error restoring namespace data:",
+				slog.String("namespace", loadedSiteState.GetNamespace()),
+				slog.String("error", err.Error()),
+			)
 			return
 		}
 		err = s.createSystemdService()
 		if err != nil {
-			fmt.Printf("Error recovering systemd service info - %s\n", err)
+			logger.Error("Error recovering systemd service info:",
+				slog.String("error", err.Error()),
+			)
 		}
 	}()
 
diff --git a/scripts/oci-index-archive-info.sh b/scripts/oci-index-archive-info.sh
new file mode 100755
index 00000000..66843a16
--- /dev/null
+++ b/scripts/oci-index-archive-info.sh
@@ -0,0 +1,42 @@
+#!/usr/bin/env bash
+
+REGISTRY=${REGISTRY:-quay.io/skupper}
+IMAGE_TAG=${IMAGE_TAG:-"v2-latest"}
+ARCHIVES_PATH=${ARCHIVES_PATH:-./oci-archives}
+
+skopeo::digest (){
+  skopeo inspect oci-archive:"$1" \
+    | jq -r ".Digest"
+}
+
+skopeo::digest::for::arch (){
+  skopeo inspect --raw oci-archive:"$1" \
+    | jq -r \
+	".manifests[] | select(.platform.architecture == \"$2\" and .platform.os == \"linux\") | .digest"
+}
+
+echo "index:"
+for archive in "${ARCHIVES_PATH}"/*; do
+  if [ -f "$archive" ]; then
+    fn=$(basename -- "$archive")
+    imagename="${fn%.*}"
+    digest=$(skopeo::digest "$archive" "$x")
+    if [ -n "$digest" ]; then
+      printf "  - %s/%s:%s@%s\n" "$REGISTRY" "$imagename" "$IMAGE_TAG" "$digest"
+    fi
+  fi
+done
+for x in "$@"; do
+  echo "$x:"
+  for archive in "${ARCHIVES_PATH}"/*; do
+    if [ -f "$archive" ]; then
+      fn=$(basename -- "$archive")
+      imagename="${fn%.*}"
+	  digest=$(skopeo::digest::for::arch "$archive" "$x")
+	  if [ -n "$digest" ]; then
+        printf "  - %s/%s:%s@%s\n" "$REGISTRY" "$imagename" "$IMAGE_TAG" "$digest"
+      fi
+    fi
+  done
+done
+


Test patch:
diff --git a/internal/cmd/skupper/debug/debug_test.go b/internal/cmd/skupper/debug/debug_test.go
new file mode 100644
index 00000000..8dc17c68
--- /dev/null
+++ b/internal/cmd/skupper/debug/debug_test.go
@@ -0,0 +1,1 @@
+package debug
diff --git a/internal/cmd/skupper/debug/kube/debug_test.go b/internal/cmd/skupper/debug/kube/debug_test.go
new file mode 100644
index 00000000..f84e1e4e
--- /dev/null
+++ b/internal/cmd/skupper/debug/kube/debug_test.go
@@ -0,0 +1,1 @@
+package kube
diff --git a/internal/cmd/skupper/debug/nonkube/debug_test.go b/internal/cmd/skupper/debug/nonkube/debug_test.go
new file mode 100644
index 00000000..6072c5b0
--- /dev/null
+++ b/internal/cmd/skupper/debug/nonkube/debug_test.go
@@ -0,0 +1,1 @@
+package nonkube
diff --git a/internal/cmd/skupper/version/kube/version_test.go b/internal/cmd/skupper/version/kube/version_test.go
new file mode 100644
index 00000000..f84e1e4e
--- /dev/null
+++ b/internal/cmd/skupper/version/kube/version_test.go
@@ -0,0 +1,1 @@
+package kube
diff --git a/internal/cmd/skupper/version/nonkube/version_test.go b/internal/cmd/skupper/version/nonkube/version_test.go
new file mode 100644
index 00000000..6072c5b0
--- /dev/null
+++ b/internal/cmd/skupper/version/nonkube/version_test.go
@@ -0,0 +1,1 @@
+package nonkube
diff --git a/internal/cmd/skupper/version/version_test.go b/internal/cmd/skupper/version/version_test.go
new file mode 100644
index 00000000..f37d99d0
--- /dev/null
+++ b/internal/cmd/skupper/version/version_test.go
@@ -0,0 +1,1 @@
+package version
diff --git a/pkg/nonkube/common/site_state_validator_test.go b/pkg/nonkube/common/site_state_validator_test.go
index 492b8ab4..bec91013 100644
--- a/pkg/nonkube/common/site_state_validator_test.go
+++ b/pkg/nonkube/common/site_state_validator_test.go
@@ -66,7 +66,7 @@ func TestSiteStateValidator_Validate(t *testing.T) {
 				}
 			}),
 			valid:         false,
-			errorContains: "invalid router access: roles are required",
+			errorContains: "invalid router access: link-access-one - roles are required",
 		},
 		{
 			info: "invalid-link-access-roles-invalid-role",