Repository: apache/ozone
Pull request #7880: HDDS-12327. Restore non-HA (to HA) upgrade test
Author: login: chiacyu
Created: 2025-02-14T15:32:08Z | Merged: 2025-03-26T08:17:55Z

## What changes were proposed in this pull request
non-ha upgrade test was removed in [HDDS-11902](https://issues.apache.org/jira/browse/HDDS-11902). However, we need to test upgrade from previous version with OM/SCM Ratis disabled to current code, where OM/SCM Ratis is always enabled. Please take a look, thanks!

## What is the link to the Apache JIRA
[HDDS-12327](https://issues.apache.org/jira/browse/HDDS-12327)

## How was this patch tested?
It can test by CI.



Diff:
diff --git a/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/.env b/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/.env
new file mode 100644
index 000000000000..babe87a492a0
--- /dev/null
+++ b/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/.env
@@ -0,0 +1,22 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+HDDS_VERSION=${hdds.version}
+OZONE_RUNNER_VERSION=${docker.ozone-runner.version}
+OZONE_RUNNER_IMAGE=apache/ozone-runner
+OZONE_TEST_IMAGE=apache/ozone-runner:${docker.ozone-runner.version}
+OZONE_DIR=/opt/hadoop
+OZONE_VOLUME=./data
diff --git a/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/docker-compose.yaml b/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/docker-compose.yaml
new file mode 100644
index 000000000000..aa59f39b503b
--- /dev/null
+++ b/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/docker-compose.yaml
@@ -0,0 +1,155 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# reusable fragments (see https://docs.docker.com/compose/compose-file/#extension-fields)
+x-common-config:
+  &common-config
+  env_file:
+    - docker-config
+  image: ${OZONE_TEST_IMAGE}
+
+x-environment:
+  &environment
+  OZONE-SITE.XML_ozone.server.default.replication: ${OZONE_REPLICATION_FACTOR:-1}
+  OZONE_UPGRADE_TO: ${OZONE_UPGRADE_TO:-0}
+  OZONE_UPGRADE_FROM: ${OZONE_UPGRADE_FROM:-0}
+
+x-datanode:
+  &datanode
+  command: ["ozone","datanode"]
+  <<: *common-config
+  environment:
+    <<: *environment
+  ports:
+    - 19864
+    - 9882
+
+x-volumes:
+    - &ozone-dir ${TEST_DIR}/../..:${OZONE_DIR}
+    - &transformation ${TEST_DIR}/../../libexec/transformation.py:/opt/hadoop/libexec/transformation.py
+
+services:
+  om:
+    command: ["ozone","om"]
+    <<: *common-config
+    environment:
+      ENSURE_OM_INITIALIZED: /data/metadata/om/current/VERSION
+      <<: *environment
+    ports:
+      - 9862
+      - 9874
+    networks:
+      net:
+        ipv4_address: 10.9.0.11
+    volumes:
+      - ${OZONE_VOLUME}/om:/data
+      - *ozone-dir
+      - *transformation
+  scm:
+    command: ["ozone","scm"]
+    <<: *common-config
+    environment:
+      ENSURE_SCM_INITIALIZED: /data/metadata/scm/current/VERSION
+      OZONE-SITE.XML_hdds.scm.safemode.min.datanode: ${OZONE_SAFEMODE_MIN_DATANODES:-1}
+      <<: *environment
+    networks:
+      net:
+        ipv4_address: 10.9.0.12
+    ports:
+      - 9876:9876
+    volumes:
+      - ${OZONE_VOLUME}/scm:/data
+      - *ozone-dir
+      - *transformation
+  dn1:
+    <<: *datanode
+    networks:
+      net:
+        ipv4_address: 10.9.0.13
+    volumes:
+      - ${OZONE_VOLUME}/dn1:/data
+      - *ozone-dir
+      - *transformation
+  dn2:
+    <<: *datanode
+    networks:
+      net:
+        ipv4_address: 10.9.0.14
+    volumes:
+      - ${OZONE_VOLUME}/dn2:/data
+      - *ozone-dir
+      - *transformation
+  dn3:
+    <<: *datanode
+    networks:
+      net:
+        ipv4_address: 10.9.0.15
+    volumes:
+      - ${OZONE_VOLUME}/dn3:/data
+      - *ozone-dir
+      - *transformation
+  dn4:
+    <<: *datanode
+    networks:
+      net:
+        ipv4_address: 10.9.0.16
+    volumes:
+      - ${OZONE_VOLUME}/dn4:/data
+      - *ozone-dir
+      - *transformation
+  dn5:
+    <<: *datanode
+    networks:
+      net:
+        ipv4_address: 10.9.0.17
+    volumes:
+      - ${OZONE_VOLUME}/dn5:/data
+      - *ozone-dir
+      - *transformation
+  recon:
+    command: ["ozone","recon"]
+    <<: *common-config
+    environment:
+      <<: *environment
+    networks:
+      net:
+        ipv4_address: 10.9.0.18
+    ports:
+      - 9888:9888
+    volumes:
+      - ${OZONE_VOLUME}/recon:/data
+      - *ozone-dir
+      - *transformation
+  s3g:
+    command: ["ozone","s3g"]
+    <<: *common-config
+    environment:
+      <<: *environment
+    networks:
+      net:
+        ipv4_address: 10.9.0.19
+    ports:
+      - 9878:9878
+    volumes:
+      - ${OZONE_VOLUME}/s3g:/data
+      - *ozone-dir
+      - *transformation
+networks:
+  net:
+    driver: bridge
+    ipam:
+      config:
+      - subnet: 10.9.0.0/16
\ No newline at end of file
diff --git a/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/docker-config b/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/docker-config
new file mode 100644
index 000000000000..d6fb398575f8
--- /dev/null
+++ b/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/docker-config
@@ -0,0 +1,50 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+OZONE-SITE.XML_ozone.metadata.dirs=/data/metadata
+
+OZONE-SITE.XML_ozone.client.failover.max.attempts=6
+
+OZONE-SITE.XML_ozone.om.address=om
+OZONE-SITE.XML_ozone.om.http-address=om:9874
+
+OZONE-SITE.XML_ozone.scm.container.size=1GB
+OZONE-SITE.XML_ozone.scm.datanode.ratis.volume.free-space.min=10MB
+OZONE-SITE.XML_ozone.scm.pipeline.creation.interval=30s
+OZONE-SITE.XML_ozone.scm.pipeline.owner.container.count=1
+OZONE-SITE.XML_ozone.scm.names=scm
+OZONE-SITE.XML_ozone.scm.datanode.id.dir=/data/metadata
+OZONE-SITE.XML_ozone.scm.block.client.address=scm
+OZONE-SITE.XML_ozone.scm.container.size=1GB
+OZONE-SITE.XML_ozone.scm.client.address=scm
+
+OZONE-SITE.XML_hdds.datanode.dir=/data/hdds
+OZONE-SITE.XML_hdds.datanode.volume.min.free.space=100MB
+
+OZONE-SITE.XML_ozone.recon.db.dir=/data/metadata/recon
+OZONE-SITE.XML_ozone.recon.om.snapshot.task.interval.delay=1m
+OZONE-SITE.XML_hdds.scmclient.max.retry.timeout=30s
+OZONE-SITE.XML_ozone.http.basedir=/tmp/ozone_http
+OZONE-SITE.XML_ozone.fs.hsync.enabled=true
+
+OZONE_CONF_DIR=/etc/hadoop
+OZONE_LOG_DIR=/var/log/hadoop
+
+no_proxy=om,scm,s3g,kdc,localhost,127.0.0.1
+
+# Explicitly enable filesystem snapshot feature for this Docker compose cluster
+# Does not take effect on Ozone versions < 1.4.0
+OZONE-SITE.XML_ozone.filesystem.snapshot.enabled=true
diff --git a/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/load.sh b/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/load.sh
new file mode 100644
index 000000000000..61f90f7017b5
--- /dev/null
+++ b/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/load.sh
@@ -0,0 +1,30 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Fail if required variables are not set.
+set -u
+: "${OZONE_VOLUME}"
+: "${TEST_DIR}"
+set +u
+
+source "$TEST_DIR/testlib.sh"
+
+export COMPOSE_FILE="$TEST_DIR/compose/non-ha/docker-compose.yaml"
+export SECURITY_ENABLED=false
+create_data_dirs "${OZONE_VOLUME}"/{om,dn1,dn2,dn3,dn4,dn5,recon,s3g,scm}
+
+echo "Using docker cluster defined in $COMPOSE_FILE"
diff --git a/hadoop-ozone/dist/src/main/compose/upgrade/test.sh b/hadoop-ozone/dist/src/main/compose/upgrade/test.sh
index 07e820477ab0..d449f14ebba5 100755
--- a/hadoop-ozone/dist/src/main/compose/upgrade/test.sh
+++ b/hadoop-ozone/dist/src/main/compose/upgrade/test.sh
@@ -35,7 +35,8 @@ RESULT_DIR="$ALL_RESULT_DIR" create_results_dir
 
 # This is the version of Ozone that should use the runner image to run the
 # code that was built. Other versions will pull images from docker hub.
-run_test ha non-rolling-upgrade 1.4.1 "$OZONE_CURRENT_VERSION"
+run_test non-ha non-rolling-upgrade 1.4.1 "$OZONE_CURRENT_VERSION"
+run_test ha     non-rolling-upgrade 1.4.1 "$OZONE_CURRENT_VERSION"
 # run_test ha non-rolling-upgrade 1.4.0 "$OZONE_CURRENT_VERSION"
 # run_test ha non-rolling-upgrade 1.3.0 "$OZONE_CURRENT_VERSION"
 # run_test ha non-rolling-upgrade 1.2.1 "$OZONE_CURRENT_VERSION"
diff --git a/hadoop-ozone/dist/src/main/smoketest/upgrade/check-finalization.robot b/hadoop-ozone/dist/src/main/smoketest/upgrade/check-finalization.robot
index 0c73a777b5ce..e80e555798af 100644
--- a/hadoop-ozone/dist/src/main/smoketest/upgrade/check-finalization.robot
+++ b/hadoop-ozone/dist/src/main/smoketest/upgrade/check-finalization.robot
@@ -16,6 +16,7 @@
 *** Settings ***
 Documentation       Finalize Upgrade of the Ozone cluster
 Resource            ../commonlib.robot
+Resource            lib.robot
 Test Timeout        10 minutes
 Test Setup          Run Keyword if    '${SECURITY_ENABLED}' == 'true'    Kinit test user     testuser     testuser.keytab
 
@@ -25,26 +26,21 @@ Check OM Finalized
     ${result} =        Execute      env
                        Log    ${result}
     Pass Execution If    '%{OZONE_UPGRADE_FROM}' == '1.1.0'    OM finalization not supported in version %{OZONE_UPGRADE_FROM}
-    ${result} =        Execute      ozone admin om finalizationstatus
-                       Log    ${result}
+    ${result} =        OM Finalization Status
                        Should Contain Any    ${result}    ALREADY_FINALIZED    FINALIZATION_DONE
 
 Check SCM Finalized
     [Tags]    scm    finalized
     Pass Execution If    '%{OZONE_UPGRADE_FROM}' == '1.1.0'    HDDS finalization not supported in version %{OZONE_UPGRADE_FROM}
-    ${result} =        Execute      ozone admin scm finalizationstatus
-                       Log    ${result}
+    ${result} =        SCM Finalization Status
                        Should Contain Any    ${result}    ALREADY_FINALIZED    FINALIZATION_DONE
 
 Check OM Pre Finalized
     [Tags]    om    pre-finalized
-    ${result} =        Execute      ozone admin om finalizationstatus
-                       Log    ${result}
+    ${result} =        OM Finalization Status
                        Should Contain Any    ${result}    FINALIZATION_REQUIRED
 
 Check SCM Pre Finalized
     [Tags]    scm    pre-finalized
-    ${result} =        Execute      ozone admin scm finalizationstatus
-                       Log    ${result}
+    ${result} =        SCM Finalization Status
                        Should Contain Any    ${result}    FINALIZATION_REQUIRED
-
diff --git a/hadoop-ozone/dist/src/main/smoketest/upgrade/finalize.robot b/hadoop-ozone/dist/src/main/smoketest/upgrade/finalize.robot
index 535b22b0a30d..4cb9caaf4943 100644
--- a/hadoop-ozone/dist/src/main/smoketest/upgrade/finalize.robot
+++ b/hadoop-ozone/dist/src/main/smoketest/upgrade/finalize.robot
@@ -16,14 +16,13 @@
 *** Settings ***
 Documentation       Finalize Upgrade of the Ozone cluster
 Resource            ../commonlib.robot
+Resource            lib.robot
 Test Timeout        10 minutes
 Test Setup          Run Keyword if    '${SECURITY_ENABLED}' == 'true'    Kinit test user     testuser     testuser.keytab
 
 *** Test Cases ***
 Finalize HDDS
-    ${result} =        Execute      ozone admin scm finalizeupgrade
-                       Log    ${result}
+    Finalize SCM
 
 Finalize OMs
-    ${result} =        Execute      ozone admin om finalizeupgrade -id %{OM_SERVICE_ID}
-                       Log    ${result}
+    Finalize OM
diff --git a/hadoop-ozone/dist/src/main/smoketest/upgrade/lib.robot b/hadoop-ozone/dist/src/main/smoketest/upgrade/lib.robot
new file mode 100644
index 000000000000..027694621372
--- /dev/null
+++ b/hadoop-ozone/dist/src/main/smoketest/upgrade/lib.robot
@@ -0,0 +1,66 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+*** Settings ***
+Documentation       Keywords for Upgrade Tests
+Library             OperatingSystem
+Resource            ../lib/os.robot
+
+*** Keywords ***
+Get OM Service ID
+    ${service_id} =     Get Environment Variable    OM_SERVICE_ID    ${EMPTY}
+    RETURN    ${service_id}
+
+
+Get OM Service Param
+    ${service_id} =     Get OM Service ID
+
+    IF    '${service_id}' == ''
+        RETURN    --service-host om
+    ELSE
+        RETURN    --service-id '${service_id}'
+    END
+
+
+OM Finalization Status
+    ${param} =     Get OM Service Param
+    ${result} =    Execute      ozone admin om finalizationstatus ${param}
+    Log       ${result}
+    RETURN    ${result}
+
+
+Finalize OM
+    ${param} =     Get OM Service Param
+    ${result} =    Execute      ozone admin om finalizeupgrade ${param}
+    Log       ${result}
+    RETURN    ${result}
+
+
+Prepare OM
+    ${param} =     Get OM Service Param
+    ${result} =       Execute     ozone admin om prepare ${param}
+    Should contain    ${result}   OM Preparation successful!
+
+
+SCM Finalization Status
+    ${result} =    Execute      ozone admin scm finalizationstatus
+    Log       ${result}
+    RETURN    ${result}
+
+
+Finalize SCM
+    ${result} =    Execute      ozone admin scm finalizeupgrade
+    Log       ${result}
+    RETURN    ${result}
diff --git a/hadoop-ozone/dist/src/main/smoketest/upgrade/prepare.robot b/hadoop-ozone/dist/src/main/smoketest/upgrade/prepare.robot
index b0cd8a5ba678..cca3ca25835a 100644
--- a/hadoop-ozone/dist/src/main/smoketest/upgrade/prepare.robot
+++ b/hadoop-ozone/dist/src/main/smoketest/upgrade/prepare.robot
@@ -16,12 +16,13 @@
 *** Settings ***
 Documentation       Prepares OMs
 Resource            ../commonlib.robot
+Resource            lib.robot
 Test Timeout        5 minutes
 Test Setup          Run Keyword if    '${SECURITY_ENABLED}' == 'true'    Kinit test user     testuser     testuser.keytab
 
 *** Test Cases ***
 Prepare Ozone Manager
     Pass Execution If    '%{OZONE_UPGRADE_FROM}' == '1.1.0'    OM prepare is skipped for version %{OZONE_UPGRADE_FROM}
-    ${result} =        Execute      ozone admin om prepare -id %{OM_SERVICE_ID}
-                       Wait Until Keyword Succeeds      3min       10sec     Should contain   ${result}   OM Preparation successful!
-
+    ${service_id} =      Get OM Service ID
+    Pass Execution If    '${service_id}' == ''    OM prepare skipped in non-HA
+    Wait Until Keyword Succeeds      3min       10sec     Prepare OM


Reviews:
user: login: adoroszlai
body: Thanks @jojochuang for picking this up.
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-25T13:47:34Z
user: login: errose28
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-25T14:39:39Z
user: login: jojochuang
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-26T03:55:27Z
user: login: jojochuang
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-26T04:00:47Z

Review comments:
diff_hunk: @@ -0,0 +1,56 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+OZONE-SITE.XML_ozone.metadata.dirs=/data/metadata
+
+OZONE-SITE.XML_ozone.client.failover.max.attempts=6
+
+OZONE-SITE.XML_ozone.om.service.ids=omservice
+OZONE-SITE.XML_ozone.om.nodes.omservice=om1
+OZONE-SITE.XML_ozone.om.address.omservice.om1=om
path: hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/docker-config
user: login: adoroszlai
body: Having to add OM service ID for `non-ha` test to pass means one of:

(1) Users with non-HA cluster will need to make config changes for the upgrade.
(2) The test is not modeling real-life upgrade steps correctly.  We need to tweak the test, not the environment.

I tend to think it's the latter.

Proposed test fix: https://github.com/apache/ozone/compare/master...adoroszlai:ozone:refs/heads/HDDS-12327
test run: https://github.com/adoroszlai/ozone/actions/runs/14060297062/job/39370001881

Please take a look.  If you agree, I'll push my additional commits to this PR's branch.
created_at: 2025-03-25T13:47:06Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,56 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+OZONE-SITE.XML_ozone.metadata.dirs=/data/metadata
+
+OZONE-SITE.XML_ozone.client.failover.max.attempts=6
+
+OZONE-SITE.XML_ozone.om.service.ids=omservice
+OZONE-SITE.XML_ozone.om.nodes.omservice=om1
+OZONE-SITE.XML_ozone.om.address.omservice.om1=om
path: hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/docker-config
user: login: errose28
body: This change makes sense to me. The Robot tests will closely resemble how the upgrade is actually executed by a user.
created_at: 2025-03-25T14:39:39Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,56 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+OZONE-SITE.XML_ozone.metadata.dirs=/data/metadata
+
+OZONE-SITE.XML_ozone.client.failover.max.attempts=6
+
+OZONE-SITE.XML_ozone.om.service.ids=omservice
+OZONE-SITE.XML_ozone.om.nodes.omservice=om1
+OZONE-SITE.XML_ozone.om.address.omservice.om1=om
path: hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/docker-config
user: login: jojochuang
body: Looks great. Feel free to push these commits in.
created_at: 2025-03-26T03:55:27Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,56 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+OZONE-SITE.XML_ozone.metadata.dirs=/data/metadata
+
+OZONE-SITE.XML_ozone.client.failover.max.attempts=6
+
+OZONE-SITE.XML_ozone.om.service.ids=omservice
+OZONE-SITE.XML_ozone.om.nodes.omservice=om1
+OZONE-SITE.XML_ozone.om.address.omservice.om1=om
path: hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/docker-config
user: login: jojochuang
body: Ok i just updated the branch with these commits. Waiting for CI
created_at: 2025-03-26T04:00:47Z
author_association: CONTRIBUTOR

PR comments:
user: login: adoroszlai
created_at: 2025-02-14T15:44:04Z
body: Thanks @chiacyu for the patch.

> It can test by CI.

It is not tested by default, only `ha` is enabled:

https://github.com/apache/ozone/blob/f2d9cc3bb40251e797ef527180b0ad74bfb112fb/hadoop-ozone/dist/src/main/compose/upgrade/test.sh#L36-L43

We'd need to add:

```
run_test non-ha non-rolling-upgrade 1.4.1 "$OZONE_CURRENT_VERSION"
```

I'd add it before the `ha` test, to fail the test earlier, since the test is currently failing (see below).

However, as I mentioned in HDDS-12327, this is currently failing near the first steps (`ozone admin om finalizationstatus`).  The problem is that the config does not have OM service ID set (since it used to be a non-HA cluster), but the command expects either the config or command-line arguments to define the OM service ID.  We need to solve this problem first (and may also encounter further errors later in the test).
author_association: CONTRIBUTOR
user: login: jojochuang
created_at: 2025-03-13T02:16:41Z
body: @chiacyu this is the final leg of Ozone 2.0.0. Are you still working on this one?
author_association: CONTRIBUTOR
user: login: chiacyu
created_at: 2025-03-13T02:26:06Z
body: @jojochuang , thanks for the reminder, if it's urgent you can have it. Thanks for the help.

author_association: CONTRIBUTOR
user: login: jojochuang
created_at: 2025-03-21T22:21:39Z
body: Added non-ha upgrade test code.

Note:
(1) Defined om.service.ids (single OM) in non-ha case.
(2) explicitly disable security (SECURITY_ENABLED=false) because non-ha docker-compose does not have kdc.
(3) Do not execute prepare command in non-ha mode. According to the [upgrade user doc](https://ozone.apache.org/docs/edge/feature/nonrolling-upgrade.html) , prepare command is not needed for non-HA mode.
author_association: CONTRIBUTOR
user: login: adoroszlai
created_at: 2025-03-26T08:18:27Z
body: Thanks @chiacyu, @jojochuang for the patch, @errose28 for the review.
author_association: CONTRIBUTOR