Repository: canonical/k8s-operator
Pull request #401: docs: Improve release docs
Author: login: HomayoonAlimohammadi
Created: 2025-03-20T08:30:12Z | Merged: 2025-03-21T13:22:43Z

### Overview
This PR improves the release docs to include manual testing steps and other necessary information.


Diff:
diff --git a/docs/releases/stable/beta-to-candidate.png b/docs/releases/stable/beta-to-candidate.png
new file mode 100644
index 000000000..a687b2370
Binary files /dev/null and b/docs/releases/stable/beta-to-candidate.png differ
diff --git a/docs/releases/stable/candidate-to-stable.png b/docs/releases/stable/candidate-to-stable.png
new file mode 100644
index 000000000..02a532bd0
Binary files /dev/null and b/docs/releases/stable/candidate-to-stable.png differ
diff --git a/docs/releases/stable/index.md b/docs/releases/stable/index.md
index 443589730..610a3f23b 100644
--- a/docs/releases/stable/index.md
+++ b/docs/releases/stable/index.md
@@ -63,31 +63,33 @@ promote to the new release.
 
 ### Pin snap channel in the release branches
 
-We need to make sure that the charms have `1.xx/<risk>` set as the default snap channel.
+The charms run the `k8s-snap` underneath, so to make sure the changes in the 
+snap are going to be available in the charm, we need to make sure the correct
+k8s snap channel is referenced. For that, the [snap_installation.yaml] file
+needs to be updated either with the correct `channel`, or with the `revision` of the
+snap.
 
-Task:
+- Example with `channel`:
 
-```sh
-git switch release_1.xx
-git checkout -b task/snap-risk/release-1.xx/<risk>
-# edit the config.options.channel.default = "1.xx/<risk>"
-edit charms/worker/k8s/charmcraft.yaml
-edit charms/worker/charmcraft.yaml
+```yaml
+amd64:
+- install-type: store
+  name: k8s
+  channel: 1.32-classic/stable
+  classic: true
 ```
 
-Where `risk` represents the channel the most stable available risk of the snap.
-
-For example) `1.30-classic/beta` is the most stable shown below
+- Example with `revision`:
 
-```sh
-⚡ snap info k8s | grep 1.30
-  1.30-classic/stable:    --
-  1.30-classic/candidate: --
-  1.30-classic/beta:      v1.30.0-beta.0 2024-03-13 (140) 107MB classic
-  1.30-classic/edge:      ^
+```yaml
+amd64:
+- install-type: store
+  name: k8s
+  revision: 2500
 ```
 
-Commit, and raise a new PR into the `release_1.xx`
+The [auto-update-snap-revision] job is also responsible for auto-updating the snap 
+revision in the [snap_installation.yaml] file. This job is triggered on a schedule.
 
 ### Pin pip versions of all python dependencies
 
@@ -130,11 +132,35 @@ pip freeze > charms/worker/k8s/requirements.txt
 
 ### Build charms from the release branches
 
+The [publish-charms] job is responsible for publishing the charms either to the
+`latest/edge` OR `<release>/beta` (e.g. `1.32/beta`) channels, depending on the
+branch that is updated. If a change is merged to the `main` branch, the charm will be
+published to the `latest/edge` channel. If a change is merged to a release branch,
+the charm will be published to the `<release>/beta` channel.
+
 Raising a PR, passing the integration tests, and merging into the release
 branch should publish the charm to the upstream `1.xx/beta` channel.
 
 ## Internal verification
 
+### Make sure all tests are passing
+
+It's assumed that tests pass on the release branch. This means that the CI for PRs
+on the release branch should be green before they are merged. **Certain tests might
+be skipped** because they rely on a specific cloud. The list of these
+tests is as follows and needs to be updated when new tests are added.
+We aim to remove this list in the future, and have all tests running in CI.
+
+- [OpenStack tests]:
+  These tests should be run manually on OpenStack by the individual responsible for
+  the release. In order to run them, make sure you have an active Juju controller with
+  an OpenStack cloud, and run:
+  ```shell
+  tox run -e integration -- -k test_openstack.py --apply-proxy --model test-openstack --keep-models
+  ```
+
+Also, make sure that all the [nightly tests] are passing.
+
 ### Run **validate-k8s-release-upgrade** job
 
 **Job**: <https://jenkins.canonical.com/k8s-ps5/job/validate-k8s-release-upgrade/>
@@ -147,6 +173,17 @@ run on:
 * multiple architectures
 * multiple clouds (aws/azure/gcp/vsphere)
 
+### Promote charms to candidate
+
+**Job**: <https://github.com/canonical/k8s-operator/actions/workflows/promote_charm.yaml>
+
+Once we've made sure that the [publish-charms] job ran successfully for the latest
+changes in the release branch, and the changes are available in `beta`, we can 
+promote the charms to `candidate`. This can be done by running the [promote-charms] job
+for the release branch:
+
+![Promote to candidate](./beta-to-candidate.png)
+
 ### Notify Solutions QA
 
 At the end of the first week and assuming all major blockers are resolved, the
@@ -189,13 +226,20 @@ Next, open an upstream PR:
 Run the workflow from a branch, select `release_1.xx`,
 
 * Choose `Charm` - `all`
-* Choose `Origin Channel`- `beta`
+* Choose `Origin Channel`- `candidate`
 * Choose `Destination Channel` - `stable`
 
-![promote charm options](promote-charm.png)
+![promote charm options](candidate-to-stable.png)
 
 ### Send announcement
 
 Email announcement to k8s-crew with any relevant information.
 
 # Fin
+
+<!-- LINKS -->
+[snap_installation.yaml]: ../../../charms/worker/k8s/templates/snap_installation.yaml
+[auto-update-snap-revision]: ../../../.github/workflows/auto-update-snap-revision.yaml
+[publish-charms]: ../../../.github/workflows/publish-charms.yaml
+[OpenStack tests]: ../../../tests/integration/test_openstack.py
+[nightly tests]: ../../../.github/workflows/nightly.yaml
diff --git a/docs/releases/stable/promote-charm.png b/docs/releases/stable/promote-charm.png
deleted file mode 100644
index 1d69696c8..000000000
Binary files a/docs/releases/stable/promote-charm.png and /dev/null differ


Reviews:
user: login: addyess
body: @HomayoonAlimohammadi thanks!  this looks great.  

Would you update the `Openstack` -> `OpenStack` suggestions when you rebase this branch?
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-20T14:07:58Z
user: login: eaudetcobello
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-20T18:52:57Z
user: login: nhennigan
body: LGTM. Just have a question on the document release notes part. Are we doing that anywhere? Besides the auto generated changelog from the github action I mean
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-20T18:54:04Z
user: login: nhennigan
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-20T18:55:08Z
user: login: eaudetcobello
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-20T18:56:33Z
user: login: eaudetcobello
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-20T18:57:14Z
user: login: eaudetcobello
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-20T18:57:36Z
user: login: HomayoonAlimohammadi
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-21T08:24:40Z

Review comments:
diff_hunk: @@ -130,11 +132,36 @@ pip freeze > charms/worker/k8s/requirements.txt
 
 ### Build charms from the release branches
 
+The [publish-charms] job is responsible for publishing the charms either to the
+`latest/edge` OR `<release>/beta` (e.g. `1.32/beta`) channels, depending on the
+branch that is updated. If a change is merged to the `main` branch, the charm will be
+published to the `latest/edge` channel. If a change is merged to a release branch,
+the charm will be published to the `<release>/beta` channel.
+
 Raising a PR, passing the integration tests, and merging into the release
 branch should publish the charm to the upstream `1.xx/beta` channel.
 
 ## Internal verification
 
+### Make sure all tests are passing
+
+It's assumed that tests pass on the release branch. This means that the CI for PRs
+on the release branch should be green before they are merged. **Certain tests might
+be skipped tho**, because of they need to be run on a specific cloud. The list of these
path: docs/releases/stable/index.md
user: login: nhennigan
body: nit: though
created_at: 2025-03-20T18:44:19Z
author_association: CONTRIBUTOR
diff_hunk: @@ -130,11 +132,36 @@ pip freeze > charms/worker/k8s/requirements.txt
 
 ### Build charms from the release branches
 
+The [publish-charms] job is responsible for publishing the charms either to the
+`latest/edge` OR `<release>/beta` (e.g. `1.32/beta`) channels, depending on the
+branch that is updated. If a change is merged to the `main` branch, the charm will be
+published to the `latest/edge` channel. If a change is merged to a release branch,
+the charm will be published to the `<release>/beta` channel.
+
 Raising a PR, passing the integration tests, and merging into the release
 branch should publish the charm to the upstream `1.xx/beta` channel.
 
 ## Internal verification
 
+### Make sure all tests are passing
+
+It's assumed that tests pass on the release branch. This means that the CI for PRs
+on the release branch should be green before they are merged. **Certain tests might
+be skipped tho**, because of they need to be run on a specific cloud. The list of these
path: docs/releases/stable/index.md
user: login: eaudetcobello
body: ```suggestion
on the release branch should be green before they are merged. **Certain tests might
be skipped** because they rely on a specific cloud. The list of these
```
created_at: 2025-03-20T18:52:57Z
author_association: CONTRIBUTOR
diff_hunk: @@ -130,11 +132,36 @@ pip freeze > charms/worker/k8s/requirements.txt
 
 ### Build charms from the release branches
 
+The [publish-charms] job is responsible for publishing the charms either to the
+`latest/edge` OR `<release>/beta` (e.g. `1.32/beta`) channels, depending on the
+branch that is updated. If a change is merged to the `main` branch, the charm will be
+published to the `latest/edge` channel. If a change is merged to a release branch,
+the charm will be published to the `<release>/beta` channel.
+
 Raising a PR, passing the integration tests, and merging into the release
 branch should publish the charm to the upstream `1.xx/beta` channel.
 
 ## Internal verification
 
+### Make sure all tests are passing
+
+It's assumed that tests pass on the release branch. This means that the CI for PRs
+on the release branch should be green before they are merged. **Certain tests might
+be skipped tho**, because of they need to be run on a specific cloud. The list of these
path: docs/releases/stable/index.md
user: login: nhennigan
body: Haha submitted almost simultaneously! 
created_at: 2025-03-20T18:55:08Z
author_association: CONTRIBUTOR
diff_hunk: @@ -130,11 +132,36 @@ pip freeze > charms/worker/k8s/requirements.txt
 
 ### Build charms from the release branches
 
+The [publish-charms] job is responsible for publishing the charms either to the
+`latest/edge` OR `<release>/beta` (e.g. `1.32/beta`) channels, depending on the
+branch that is updated. If a change is merged to the `main` branch, the charm will be
+published to the `latest/edge` channel. If a change is merged to a release branch,
+the charm will be published to the `<release>/beta` channel.
+
 Raising a PR, passing the integration tests, and merging into the release
 branch should publish the charm to the upstream `1.xx/beta` channel.
 
 ## Internal verification
 
+### Make sure all tests are passing
+
+It's assumed that tests pass on the release branch. This means that the CI for PRs
+on the release branch should be green before they are merged. **Certain tests might
+be skipped tho**, because of they need to be run on a specific cloud. The list of these
+tests is documented here and needs to be updated when new tests are added.
path: docs/releases/stable/index.md
user: login: eaudetcobello
body: Use of "here" is not clear. Could we use "in this document", "in xyz section" or something more specific?
created_at: 2025-03-20T18:56:32Z
author_association: CONTRIBUTOR
diff_hunk: @@ -130,11 +132,36 @@ pip freeze > charms/worker/k8s/requirements.txt
 
 ### Build charms from the release branches
 
+The [publish-charms] job is responsible for publishing the charms either to the
+`latest/edge` OR `<release>/beta` (e.g. `1.32/beta`) channels, depending on the
+branch that is updated. If a change is merged to the `main` branch, the charm will be
+published to the `latest/edge` channel. If a change is merged to a release branch,
+the charm will be published to the `<release>/beta` channel.
+
 Raising a PR, passing the integration tests, and merging into the release
 branch should publish the charm to the upstream `1.xx/beta` channel.
 
 ## Internal verification
 
+### Make sure all tests are passing
+
+It's assumed that tests pass on the release branch. This means that the CI for PRs
+on the release branch should be green before they are merged. **Certain tests might
+be skipped tho**, because of they need to be run on a specific cloud. The list of these
+tests is documented here and needs to be updated when new tests are added.
+Hopefully this list will be empty in the future, as we will be able to run all the
+tests in CI.
path: docs/releases/stable/index.md
user: login: eaudetcobello
body: ```suggestion
We aim to remove this list in the future, and have all tests running in CI.
```
created_at: 2025-03-20T18:57:13Z
author_association: CONTRIBUTOR
diff_hunk: @@ -130,11 +132,36 @@ pip freeze > charms/worker/k8s/requirements.txt
 
 ### Build charms from the release branches
 
+The [publish-charms] job is responsible for publishing the charms either to the
+`latest/edge` OR `<release>/beta` (e.g. `1.32/beta`) channels, depending on the
+branch that is updated. If a change is merged to the `main` branch, the charm will be
+published to the `latest/edge` channel. If a change is merged to a release branch,
+the charm will be published to the `<release>/beta` channel.
+
 Raising a PR, passing the integration tests, and merging into the release
 branch should publish the charm to the upstream `1.xx/beta` channel.
 
 ## Internal verification
 
+### Make sure all tests are passing
+
+It's assumed that tests pass on the release branch. This means that the CI for PRs
+on the release branch should be green before they are merged. **Certain tests might
+be skipped tho**, because of they need to be run on a specific cloud. The list of these
+tests is documented here and needs to be updated when new tests are added.
+Hopefully this list will be empty in the future, as we will be able to run all the
+tests in CI.
+
+- [OpenStack tests]:
+  These tests should be manually run on OpenStack by the individual responsible for
path: docs/releases/stable/index.md
user: login: eaudetcobello
body: ```suggestion
  These tests should be run manually on OpenStack by the individual responsible for
```
created_at: 2025-03-20T18:57:36Z
author_association: CONTRIBUTOR
diff_hunk: @@ -130,11 +132,36 @@ pip freeze > charms/worker/k8s/requirements.txt
 
 ### Build charms from the release branches
 
+The [publish-charms] job is responsible for publishing the charms either to the
+`latest/edge` OR `<release>/beta` (e.g. `1.32/beta`) channels, depending on the
+branch that is updated. If a change is merged to the `main` branch, the charm will be
+published to the `latest/edge` channel. If a change is merged to a release branch,
+the charm will be published to the `<release>/beta` channel.
+
 Raising a PR, passing the integration tests, and merging into the release
 branch should publish the charm to the upstream `1.xx/beta` channel.
 
 ## Internal verification
 
+### Make sure all tests are passing
+
+It's assumed that tests pass on the release branch. This means that the CI for PRs
+on the release branch should be green before they are merged. **Certain tests might
+be skipped tho**, because of they need to be run on a specific cloud. The list of these
+tests is documented here and needs to be updated when new tests are added.
path: docs/releases/stable/index.md
user: login: HomayoonAlimohammadi
body: Changed it to "The list of the tests is as follows". WDYT?
created_at: 2025-03-21T08:24:40Z
author_association: CONTRIBUTOR

PR comments:
user: login: HomayoonAlimohammadi
created_at: 2025-03-21T08:28:04Z
body: > Just have a question on the document release notes part. Are we doing that anywhere? Besides the auto generated changelog from the github action I mean

@nhennigan I'm not sure about this exactly. I didn't double check all the other steps, since in this task we aim to only cover the tests that need to be run manually before releasing. Maybe we can get back to this in the future? WDYT?


author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-03-21T08:29:49Z
body: ## Test results for commit 89d4ffb91e71327a61abeee6256b5695b311912d

Test coverage for 89d4ffb91e71327a61abeee6256b5695b311912d

  ```
coverage-report: install_deps /home/runner/work/k8s-operator/k8s-operator/charms/worker/k8s> /home/runner/.local/share/uv/tools/tox/bin/uv pip install 'coverage[toml]'
coverage-report: commands[0] /home/runner/work/k8s-operator/k8s-operator/charms/worker/k8s> coverage report
Name                                    Stmts   Miss  Cover
-----------------------------------------------------------
lib/charms/k8s/v0/k8sd_api_manager.py     366     23    94%
src/certificates.py                       183    121    34%
src/charm.py                              654    300    54%
src/cloud_integration.py                   80      3    96%
src/config/extra_args.py                   29      2    93%
src/containerd.py                         154     18    88%
src/cos_integration.py                     33     12    64%
src/endpoints.py                           40      1    98%
src/events/update_status.py                68     24    65%
src/inspector.py                           42      4    90%
src/kube_control.py                        44     32    27%
src/literals.py                            48      0   100%
src/pki.py                                 25      8    68%
src/protocols.py                           32      7    78%
src/reschedule.py                          79      4    95%
src/snap.py                               195     29    85%
src/token_distributor.py                  181    109    40%
src/upgrade.py                            109     48    56%
src/utils.py                               16     11    31%
-----------------------------------------------------------
TOTAL                                    2378    756    68%
  coverage-report: OK (0.41=setup[0.07]+cmd[0.33] seconds)
  congratulations :) (0.46 seconds)
```
  Static code analysis report

  ```
Run started:2025-03-21 08:29:06.431808

Test results:
	No issues identified.

Code scanned:
	Total lines of code: 4882
	Total lines skipped (#nosec): 3
	Total potential issues skipped due to specifically being disabled (e.g., #nosec BXXX): 0

Run metrics:
	Total issues (by severity):
		Undefined: 0
		Low: 0
		Medium: 0
		High: 0
	Total issues (by confidence):
		Undefined: 0
		Low: 0
		Medium: 0
		High: 0
Files skipped (0):
```
author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-03-21T08:29:50Z
body: ## Test results for commit 89d4ffb91e71327a61abeee6256b5695b311912d

Test coverage for 89d4ffb91e71327a61abeee6256b5695b311912d

  ```
coverage-report: install_deps /home/runner/work/k8s-operator/k8s-operator/charms/worker/k8s> /home/runner/.local/share/uv/tools/tox/bin/uv pip install 'coverage[toml]'
coverage-report: commands[0] /home/runner/work/k8s-operator/k8s-operator/charms/worker/k8s> coverage report
Name                                    Stmts   Miss  Cover
-----------------------------------------------------------
lib/charms/k8s/v0/k8sd_api_manager.py     366     23    94%
src/certificates.py                       183    121    34%
src/charm.py                              654    300    54%
src/cloud_integration.py                   80      3    96%
src/config/extra_args.py                   29      2    93%
src/containerd.py                         154     18    88%
src/cos_integration.py                     33     12    64%
src/endpoints.py                           40      1    98%
src/events/update_status.py                68     24    65%
src/inspector.py                           42      4    90%
src/kube_control.py                        44     32    27%
src/literals.py                            48      0   100%
src/pki.py                                 25      8    68%
src/protocols.py                           32      7    78%
src/reschedule.py                          79      4    95%
src/snap.py                               195     29    85%
src/token_distributor.py                  181    109    40%
src/upgrade.py                            109     48    56%
src/utils.py                               16     11    31%
-----------------------------------------------------------
TOTAL                                    2378    756    68%
  coverage-report: OK (0.36=setup[0.06]+cmd[0.30] seconds)
  congratulations :) (0.41 seconds)
```
  Static code analysis report

  ```
Run started:2025-03-21 08:29:13.470087

Test results:
	No issues identified.

Code scanned:
	Total lines of code: 4882
	Total lines skipped (#nosec): 3
	Total potential issues skipped due to specifically being disabled (e.g., #nosec BXXX): 0

Run metrics:
	Total issues (by severity):
		Undefined: 0
		Low: 0
		Medium: 0
		High: 0
	Total issues (by confidence):
		Undefined: 0
		Low: 0
		Medium: 0
		High: 0
Files skipped (0):
```
author_association: CONTRIBUTOR