Repository: openshift/appliance
Pull request #354: AGENT-1140: support interactive flow in update-hosts
Author: login: danielerez
Labels: name: approved
description: Indicates a PR has been approved by an approver from all required OWNERS files.
name: jira/valid-reference
description: Indicates that this PR references a valid Jira ticket of any type.
Created: 2025-03-16T10:48:55Z | Merged: 2025-03-18T10:49:17Z

Avoid using REQUIRED_MASTER_NODES/REQUIRED_WORKER_NODES in update-hosts.sh, since these are not available when using the interactive flow.
Instead, monitor for added hosts and update the ignition, until the cluster installation is starting.


Diff:
diff --git a/data/scripts/bin/update-hosts.sh.template b/data/scripts/bin/update-hosts.sh.template
index 2197c30d..0e9e7ab4 100644
--- a/data/scripts/bin/update-hosts.sh.template
+++ b/data/scripts/bin/update-hosts.sh.template
@@ -24,22 +24,9 @@ curl_assisted_service() {
     curl "${headers[@]}" -X "${method}" "${additional_options[@]}" "${baseURL}${endpoint}"
 }
 
-# Wait for all required hosts to be discovered
-declare host_ids
-total_required_nodes=$(( REQUIRED_MASTER_NODES + REQUIRED_WORKER_NODES ))
-while [[ "${num_of_hosts}" != "${total_required_nodes}" ]]
-do
-    echo "Waiting for ${total_required_nodes} required hosts..." 1>&2
-    host_ids=$(curl_assisted_service "/infra-envs/${INFRA_ENV_ID}/hosts" | jq -r .[].id)
-    if [[ -n ${host_ids} ]]; then
-        num_of_hosts=0
-        for id in ${host_ids}; do
-            ((num_of_hosts+=1))
-        done
-        echo "Discovered ${num_of_hosts} hosts"
-    fi
-    sleep 2
-done
+declare -A hosts
+declare cluster_id
+declare cluster_status
 
 # Set install ignition config
 ignition=$(echo '{{.InstallIgnitionConfig}}' | jq -c --raw-input)
@@ -49,13 +36,39 @@ host_env_base64=$(base64 -w 0 /etc/assisted/rendezvous-host.env)
 placeholder_base64=$(echo -n '{{.RendezvousHostEnvPlaceholder}}' | base64)
 ignition="${ignition//$placeholder_base64/$host_env_base64}"
 
-# Update installer-args and ignition for each host
-if [[ -n ${host_ids} ]]; then
+# Waiting for the cluster-id to be available
+until [[ -n ${cluster_id} ]]; do
+    echo "Querying assisted-service for cluster-id..."
+    cluster_id=$(curl_assisted_service "/clusters" GET | jq -r '.[0].id')
+    sleep 1
+done
+echo "Fetched cluster-id: $cluster_id"
+
+# Updating hosts can be done before starting the installation
+until [[ $cluster_status == "preparing-for-installation" ]]; do
+    cluster_status=$(curl_assisted_service "/clusters/${cluster_id}" | jq -r .status)
+
+    # Update ignition for each host
+    host_ids=$(curl_assisted_service "/infra-envs/${INFRA_ENV_ID}/hosts" | jq -r .[].id)
+    if [[ -z ${host_ids} ]]; then
+        sleep 2
+        continue
+    fi
+
     for id in ${host_ids}; do
+        if [[ ${hosts[$id]} == "true" ]]; then
+            # Host is already updated
+            continue
+        fi
+
         # Update host's ignition (used when booting from the installation disk after bootstrap)
-        curl_assisted_service "${BASE_URL}/infra-envs/${INFRA_ENV_ID}/hosts/${id}/ignition" \
+        curl_assisted_service "/infra-envs/${INFRA_ENV_ID}/hosts/${id}/ignition" \
             PATCH -d '{"config": '"${ignition}"'}'
-
-        echo "Updated installer-args and ignition of host with id: ${id}"
+        hosts[$id]=true
+        echo "Updated ignition of host: ${id}"
     done
-fi
+
+    sleep 1
+done
+
+echo "Done updating hosts"
diff --git a/data/services/bootstrap/update-hosts.service b/data/services/bootstrap/update-hosts.service
index 3c6a946e..4a233f62 100644
--- a/data/services/bootstrap/update-hosts.service
+++ b/data/services/bootstrap/update-hosts.service
@@ -1,5 +1,5 @@
 [Unit]
-Description=Service that updates installer-args and ignition on all hosts
+Description=Service that updates ignition on all hosts
 Wants=network-online.target
 Requires=apply-host-config.service
 After=network-online.target apply-host-config.service


Reviews:
user: login: andfasano
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T08:35:53Z
user: login: andfasano
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T08:39:47Z
user: login: danielerez
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T09:39:19Z
user: login: zaneb
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-18T10:55:10Z

Review comments:
diff_hunk: @@ -49,13 +36,39 @@ host_env_base64=$(base64 -w 0 /etc/assisted/rendezvous-host.env)
 placeholder_base64=$(echo -n '{{.RendezvousHostEnvPlaceholder}}' | base64)
 ignition="${ignition//$placeholder_base64/$host_env_base64}"
 
-# Update installer-args and ignition for each host
-if [[ -n ${host_ids} ]]; then
+# Waiting for the cluster-id to be available
+until [[ -n ${cluster_id} ]]; do
+    echo "Querying assisted-service for cluster-id..."
+    cluster_id=$(curl_assisted_service "/clusters" GET | jq -r '.[0].id')
+    sleep 1
+done
+echo "Fetched cluster-id: $cluster_id"
+
+# Updating hosts can be done before starting the installation
+until [[ $cluster_status == "preparing-for-installation" ]]; do
path: data/scripts/bin/update-hosts.sh.template
user: login: andfasano
body: In the interactive workflow there could be a (very) tiny chance to miss updating an host before the installation is started by the UI - but to address that an explicit synchronization mechanism would be required. Anyhow the patch lgtm, since it's definitely an improvement in respect to the previous version. 
created_at: 2025-03-17T08:35:53Z
author_association: CONTRIBUTOR
diff_hunk: @@ -49,13 +36,39 @@ host_env_base64=$(base64 -w 0 /etc/assisted/rendezvous-host.env)
 placeholder_base64=$(echo -n '{{.RendezvousHostEnvPlaceholder}}' | base64)
 ignition="${ignition//$placeholder_base64/$host_env_base64}"
 
-# Update installer-args and ignition for each host
-if [[ -n ${host_ids} ]]; then
+# Waiting for the cluster-id to be available
+until [[ -n ${cluster_id} ]]; do
+    echo "Querying assisted-service for cluster-id..."
+    cluster_id=$(curl_assisted_service "/clusters" GET | jq -r '.[0].id')
+    sleep 1
+done
+echo "Fetched cluster-id: $cluster_id"
+
+# Updating hosts can be done before starting the installation
+until [[ $cluster_status == "preparing-for-installation" ]]; do
+    cluster_status=$(curl_assisted_service "/clusters/${cluster_id}" | jq -r .status)
+
+    # Update ignition for each host
+    host_ids=$(curl_assisted_service "/infra-envs/${INFRA_ENV_ID}/hosts" | jq -r .[].id)
+    if [[ -z ${host_ids} ]]; then
+        sleep 2
+        continue
+    fi
+
     for id in ${host_ids}; do
+        if [[ ${hosts[$id]} == "true" ]]; then
+            # Host is already updated
+            continue
+        fi
+
         # Update host's ignition (used when booting from the installation disk after bootstrap)
-        curl_assisted_service "${BASE_URL}/infra-envs/${INFRA_ENV_ID}/hosts/${id}/ignition" \
+        curl_assisted_service "/infra-envs/${INFRA_ENV_ID}/hosts/${id}/ignition" \
             PATCH -d '{"config": '"${ignition}"'}'
-
-        echo "Updated installer-args and ignition of host with id: ${id}"
+        hosts[$id]=true
+        echo "Updated ignition of host: ${id}"
     done
-fi
+
+    sleep 2
path: data/scripts/bin/update-hosts.sh.template
user: login: andfasano
body: ```suggestion
    sleep 1
```
Just to reduce the waiting time before discovering a new host needing to be patched
created_at: 2025-03-17T08:39:46Z
author_association: CONTRIBUTOR
diff_hunk: @@ -49,13 +36,39 @@ host_env_base64=$(base64 -w 0 /etc/assisted/rendezvous-host.env)
 placeholder_base64=$(echo -n '{{.RendezvousHostEnvPlaceholder}}' | base64)
 ignition="${ignition//$placeholder_base64/$host_env_base64}"
 
-# Update installer-args and ignition for each host
-if [[ -n ${host_ids} ]]; then
+# Waiting for the cluster-id to be available
+until [[ -n ${cluster_id} ]]; do
+    echo "Querying assisted-service for cluster-id..."
+    cluster_id=$(curl_assisted_service "/clusters" GET | jq -r '.[0].id')
+    sleep 1
+done
+echo "Fetched cluster-id: $cluster_id"
+
+# Updating hosts can be done before starting the installation
+until [[ $cluster_status == "preparing-for-installation" ]]; do
path: data/scripts/bin/update-hosts.sh.template
user: login: danielerez
body: Right, I've was worried for exactly that when introducing this change:) But we indeed don't have such sync mechanism yet. So we could start with this for now.
created_at: 2025-03-17T09:05:28Z
author_association: CONTRIBUTOR
diff_hunk: @@ -49,13 +36,39 @@ host_env_base64=$(base64 -w 0 /etc/assisted/rendezvous-host.env)
 placeholder_base64=$(echo -n '{{.RendezvousHostEnvPlaceholder}}' | base64)
 ignition="${ignition//$placeholder_base64/$host_env_base64}"
 
-# Update installer-args and ignition for each host
-if [[ -n ${host_ids} ]]; then
+# Waiting for the cluster-id to be available
+until [[ -n ${cluster_id} ]]; do
+    echo "Querying assisted-service for cluster-id..."
+    cluster_id=$(curl_assisted_service "/clusters" GET | jq -r '.[0].id')
+    sleep 1
+done
+echo "Fetched cluster-id: $cluster_id"
+
+# Updating hosts can be done before starting the installation
+until [[ $cluster_status == "preparing-for-installation" ]]; do
+    cluster_status=$(curl_assisted_service "/clusters/${cluster_id}" | jq -r .status)
+
+    # Update ignition for each host
+    host_ids=$(curl_assisted_service "/infra-envs/${INFRA_ENV_ID}/hosts" | jq -r .[].id)
+    if [[ -z ${host_ids} ]]; then
+        sleep 2
+        continue
+    fi
+
     for id in ${host_ids}; do
+        if [[ ${hosts[$id]} == "true" ]]; then
+            # Host is already updated
+            continue
+        fi
+
         # Update host's ignition (used when booting from the installation disk after bootstrap)
-        curl_assisted_service "${BASE_URL}/infra-envs/${INFRA_ENV_ID}/hosts/${id}/ignition" \
+        curl_assisted_service "/infra-envs/${INFRA_ENV_ID}/hosts/${id}/ignition" \
             PATCH -d '{"config": '"${ignition}"'}'
-
-        echo "Updated installer-args and ignition of host with id: ${id}"
+        hosts[$id]=true
+        echo "Updated ignition of host: ${id}"
     done
-fi
+
+    sleep 2
path: data/scripts/bin/update-hosts.sh.template
user: login: danielerez
body: Sure, changed.
created_at: 2025-03-17T09:05:39Z
author_association: CONTRIBUTOR
diff_hunk: @@ -49,13 +36,39 @@ host_env_base64=$(base64 -w 0 /etc/assisted/rendezvous-host.env)
 placeholder_base64=$(echo -n '{{.RendezvousHostEnvPlaceholder}}' | base64)
 ignition="${ignition//$placeholder_base64/$host_env_base64}"
 
-# Update installer-args and ignition for each host
-if [[ -n ${host_ids} ]]; then
+# Waiting for the cluster-id to be available
+until [[ -n ${cluster_id} ]]; do
+    echo "Querying assisted-service for cluster-id..."
+    cluster_id=$(curl_assisted_service "/clusters" GET | jq -r '.[0].id')
+    sleep 1
+done
+echo "Fetched cluster-id: $cluster_id"
+
+# Updating hosts can be done before starting the installation
+until [[ $cluster_status == "preparing-for-installation" ]]; do
path: data/scripts/bin/update-hosts.sh.template
user: login: zaneb
body: Have we considered building this functionality directly into assisted-service?
This does seem a bit race-y.
created_at: 2025-03-18T10:55:10Z
author_association: MEMBER

PR comments:
user: login: openshift-ci-robot
created_at: 2025-03-16T10:48:58Z
body: @danielerez: No Jira issue with key GENT-1140 exists in the tracker at https://issues.redhat.com/.
Once a valid jira issue is referenced in the title of this pull request, request a refresh with <code>/jira refresh</code>.

<details>

In response to [this](https://github.com/openshift/appliance/pull/354):

>Avoid using REQUIRED_MASTER_NODES/REQUIRED_WORKER_NODES in update-hosts.sh, since these are not available when using the interactive flow.
>Instead, monitor for added hosts and update the ignition, until the cluster installation is starting.


Instructions for interacting with me using PR comments are available [here](https://prow.ci.openshift.org/command-help?repo=openshift%2Fappliance).  If you have questions or suggestions related to my behavior, please file an issue against the [openshift-eng/jira-lifecycle-plugin](https://github.com/openshift-eng/jira-lifecycle-plugin/issues/new) repository.
</details>
author_association: NONE
user: login: openshift-ci[bot]
created_at: 2025-03-16T10:49:00Z
body: Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with `/test all`
author_association: NONE
user: login: openshift-ci[bot]
created_at: 2025-03-16T10:49:08Z
body: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/openshift/appliance/pull/354#" title="Author self-approved">danielerez</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=openshift%2Fappliance).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/openshift/appliance/blob/master/OWNERS)~~ [danielerez]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
author_association: NONE
user: login: openshift-ci-robot
created_at: 2025-03-16T10:49:24Z
body: @danielerez: This pull request references [AGENT-1140](https://issues.redhat.com//browse/AGENT-1140) which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.19.0" version, but no target version was set.

<details>

In response to [this](https://github.com/openshift/appliance/pull/354):

>Avoid using REQUIRED_MASTER_NODES/REQUIRED_WORKER_NODES in update-hosts.sh, since these are not available when using the interactive flow.
>Instead, monitor for added hosts and update the ignition, until the cluster installation is starting.


Instructions for interacting with me using PR comments are available [here](https://prow.ci.openshift.org/command-help?repo=openshift%2Fappliance).  If you have questions or suggestions related to my behavior, please file an issue against the [openshift-eng/jira-lifecycle-plugin](https://github.com/openshift-eng/jira-lifecycle-plugin/issues/new) repository.
</details>
author_association: NONE
user: login: andfasano
created_at: 2025-03-17T08:36:41Z
body: /test ?
author_association: CONTRIBUTOR
user: login: openshift-ci[bot]
created_at: 2025-03-17T08:36:46Z
body: @andfasano: The following commands are available to trigger required jobs:
```
/test e2e-compact-ipv4-static
```
```
/test images
```
```
/test lint
```
```
/test unit-test
```

The following commands are available to trigger optional jobs:
```
/test e2e-sno-dualstack-dhcp
```

Use `/test all` to run all jobs.


<details>

In response to [this](https://github.com/openshift/appliance/pull/354#issuecomment-2728589562):

>/test ?


Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
</details>
author_association: NONE
user: login: andfasano
created_at: 2025-03-17T08:44:44Z
body: /test e2e-compact-ipv4-static
author_association: CONTRIBUTOR
user: login: openshift-ci[bot]
created_at: 2025-03-17T12:20:15Z
body: @danielerez: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
ci/prow/e2e-compact-ipv4-static | e64567874b8b8ea22e92234b51ea5a3224f71bff | [link](https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_appliance/354/pull-ci-openshift-appliance-master-e2e-compact-ipv4-static/1901569234970873856) | true | `/test e2e-compact-ipv4-static`

[Full PR test history](https://prow.ci.openshift.org/pr-history?org=openshift&repo=appliance&pr=354). [Your PR dashboard](https://prow.ci.openshift.org/pr?query=is:pr+state:open+author:danielerez).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
author_association: NONE