Project: Sermant (Orchestration & Management / Service Mesh)
File: README.md

| This document describes automated tests and how to write an automated test. ```yaml .github actions #stores all test scenarios. common #common directory for storing test scenarios. dubbo #dubbo common procedure in test scenarios entry #common procedure for action entry exit #common procedure for action exit plugin-change-check #action for check plugin dir change spring #spring common procedure in test scenarios scenarios #define specific test scenarios. dubbo #dubbo test scenarios dubbo-common #dubbo common test scenarios router #dubbo router test scenarios spring #spring test scenarios dynamic-config-nacos #spring dynacos config for nacos test scenarios dynamic-config-zk #spring dynacos config for zk test scenarios graceful #spring graceful online/offline test scenario router #spring router test scenarios spring-router #spring router test scenario spring-tag-router #spring router test scenario for springboot spring-common #spring common test scenarios ISSUE_TEMPLATE workflows #auto test workflow sermant-integration-tests #storing automated tests demo dubbo-test #dubbo automated tests demo dubbo-2-6-integration-consumer dubbo-2-6-integration-controller dubbo-2-6-integration-provider dubbo-2-7-integration-consumer dubbo-2-7-integration-controller dubbo-2-7-integration-provider dubbo-integration-api dubbo-integration-test scripts #auto test script spring-test #spring automated tests demo spring-common spring-common-demos #spring common demos spring-common-feign #common feign demo 2.x spring-common-feign-1.5.x #common feign demo, 1.5.x spring-common-gateway #common gateway for demo-gateway spring-common-resttemplate #common Resttemplate test demo spring-common-zuul #common gateway for demo-zuul spring-intergration-test #auto test ut(Junit) spring-nacos-config #nacos dynamic config demo spring-zookeeper-config #zk dynamic config demo ``` The preceding figure shows the directory structure of the current automatic test, which consists of the following three parts: workflow, this is an automatic test entry. You need to add actions to be executed in the workflow (test scenario). actionsthe specific end-to-end test procedure is defined here. sermant-integration-tests this part stores the test demo required by the action and compiles the junit test cases that require the test logic. You need to define the automatic test entry in the workflow. Currently, the workflow has test entries based on the Dubbo and Spring frameworks. The entry file is as follows: `dubbointegrationtest.yml`, Dubbo test scenario entry `springintegrationtest1`, `springintegrationtest2`, Spring test scenario entry. The latter is mainly used to test the label routing capability, while the former is used to test the capabilities of other plug-ins, including flow control, registration, graceful login and logout, and load balancing. The reason why the GitHub workflow is split is to make full use of the concurrent test of the GitHub workflow to improve the test efficiency. Currently, the automatic test is driven based on the version matrix of the test framework and depends on the ). The version matrix is set at the workflow entry. Based on the specified matrix, all actions (specific test scenarios) configured at the entry are tested to achieve the test purpose. ```yaml test-for-spring: name: Test for spring runs-on: ubuntu-latest needs: [build-agent-and-cache, download-midwares-and-cache] strategy: matrix: include: springBootVersion: "1.5.0.RELEASE" springCloudVersion: "Edgware.SR2" nacosVersion: "1.5.0.RELEASE" springBootVersion: "2.0.2.RELEASE" springCloudVersion: "Finchley.RELEASE" nacosVersion: "2.0.0.RELEASE" fail-fast: false steps: uses: actions/checkout@v3 with: fetch-depth: 100 name: common operations uses: ./.github/actions/common/spring name: (graceful) test for springboot=${{ matrix.springBootVersion }} springCloudVersion=${{ matrix.springCloudVersion }} if: env.enableGraceful == 'true' uses: ./.github/actions/scenarios/spring/graceful name: (zk dynamic config) test for springboot=${{ matrix.springBootVersion }} springCloudVersion=${{ matrix.springCloudVersion }} if: env.enableDynamicConfig == 'true' uses: ./.github/actions/scenarios/spring/dynamic-config-zk ``` As shown in the preceding figure, two matrix versions are defined. Take `springBootVersion` as an example. There are `1.5.0.RELEASE` and `2.0.2.RELEASE`. During the actual test, the actions(configured `graceful` and `zk dynamic config`) of the two versions are tested at the same time (if the concurrency is not limited).The execution does not affect each

The action path is specified by the `uses` configuration file. The automatic test is specific to a specified scenario. Take `graceful` in the preceding section as an example. The` action.yml` file is defined in the `./.github/actions/scenarios/spring/graceful` directory. This file simulates the end-to-end test process and simulates the manual graceful login and logout test process. Reproduce it during automated testing. Common capabilities are provided to simplify test scenario compilation and decouple service test scenarios from common capabilities. The common capabilities are as follows: Middleware caching/downloading/starting, agent packaging caching/downloading Action public entry processing (only logs currently) and action public exit processing (only processes are terminated and environment variables are cleared currently) Code directory change check and judgment The related component cache is defined at each test entry. The following uses Spring `graceful` as an example. Two public jobs are defined at the entry: ```yaml download-midwares-and-cache: name: download midwares and cache runs-on: ubuntu-latest steps: uses: actions/checkout@v3 name: cache local cse uses: actions/cache@v3 with: path: Local-CSE-2.1.3-linux-amd64.zip key: ${{ runner.os }}-local-cse restore-keys: | ${{ runner.os }}-local-cse name: download cse run: | export ROOT_PATH=$(pwd) bash ./sermant-integration-tests/scripts/tryDownloadMidware.sh cse name: cache zookeeper uses: actions/cache@v3 with: path: apache-zookeeper-3.6.3-bin.tar.gz key: ${{ runner.os }}-apache-zookeeper-3.6.3 restore-keys: | ${{ runner.os }}-apache-zookeeper-3.6.3 name: download zookeeper run: | export ROOT_PATH=$(pwd) bash ./sermant-integration-tests/scripts/tryDownloadMidware.sh zk name: cache nacos server uses: actions/cache@v3 with: path: nacos-server-1.4.2.tar.gz key: ${{ runner.os }}-nacos-server-1.4.2 restore-keys: | ${{ runner.os }}-nacos-server-1.4.2 name: download nacos run: | export ROOT_PATH=$(pwd) bash ./sermant-integration-tests/scripts/tryDownloadMidware.sh nacos build-agent-and-cache: name: build agent and cache runs-on: ubuntu-latest steps: uses: actions/checkout@v3 name: Set up JDK 8 uses: actions/setup-java@v3 with: java-version: '8' distribution: 'adopt' cache: maven name: cache agent uses: actions/cache@v3 with: path: sermant-agent-*/ key: ${{ runner.os }}-agent-${{ github.run_id }} name: package agent run: | sed -i '/sermant-backend/d' pom.xml sed -i '/sermant-backend-lite/d' pom.xml sed -i '/sermant-integration-tests/d' pom.xml sed -i '/sermant-injector/d' pom.xml mvn package -DskipTests -Ptest --file pom.xml ``` Use the `needs` action of GitHub action to ensure that specific action tests are processed after caching. ```shell test-for-spring: name: Test for spring runs-on: ubuntu-latest needs: [build-agent-and-cache, download-midwares-and-cache] #Depending on the front job ``` Download and start related components in the unified action. For details, see the file `./.github/actions/common/spring/action.yml` The entry and exit of each action are unified. The steps must be performed before and after each action is processed. Entry, it's path is `./.github/actions/common/entry/action.yml` ```shell name: "Entrance operations" description: "do something entrance operations for all test" inputs: log-dir: description: 'Log path for cur workflow' required: true default: './logs/default' runs: using: "composite" steps: name: create log dir shell: bash run: | log_dir=${{ inputs.log-dir }} mkdir -p ${log_dir} echo "logDir=${logdir}" >> $GITHUBENV echo "=======check log dir======" ls -l ${log_dir} ``` The log output path is unified to prevent each action from occupying the same file after the demo application is started. The input parameter `log-dir` is required. The user needs to specify the path. In addition, when the demo needs to output logs, add the log path prefix environment variable (`${{env.logDir}}`). Exit, it's path is `./.github/actions/common/exit/action.yml` ```yaml name: "Exit operations" description: "do something exit operations for all test" inputs: processor-keyword: description: 'The keyword associate to processor that you want to kill, if you have one more key word, you can separate with |,such as rest|feign' required: false default: '' runs: using: "composite" steps: name: reset shell: bash run: | echo "tailVersion=" >> $GITHUB_ENV echo "healthApi=" >> $GITHUB_ENV echo "logDir=" >> $GITHUB_ENV keyword='${{

}}' if [ $keyword != '' ];then jps -l | grep -E "${keyword}" | awk '{print $1}' | xargs -n 1 kill -9 echo "========check process========" jps -l fi ``` The exit action is used to clear environment variables and terminate processes based on the jps command. Process keywords need to be transferred. Multiple keywords are separated by vertical bars (`|`). The keywords are obtained from the name of the JAR package of the demo. To improve the efficiency and accuracy of the automatic test, determine whether to execute the corresponding scenario based on the code submission path. For example, the `graceful` code path is `sermant-plugins/sermant-service-registry`, . If the submitted code modifies the file in the path, the action test is triggered. If the submitted code is another plug-in, the scenario will not be triggered. The implementation of this function depends on the open-source action . This component can determine the change of a specified path force based on the submitted code. An independent action is extracted to detect the code change. The path is `./.github/actions/common/plugin-change-check/action.yml`. The following figure shows the `graceful` path detection process. ```yaml name: "Plugin change check" description: "check file change for all plugins" runs: using: "composite" steps: uses: marceloprado/has-changed-path@v1.0.1 id: changed-sermant-service-registry with: paths: sermant-plugins/sermant-service-registry name: env sermant-sermant-service-registry shell: bash run: | echo "sermantServiceRegistryChanged=${{ steps.changed-sermant-service-registry.outputs.changed }}" >> $GITHUB_ENV uses: marceloprado/has-changed-path@v1.0.1 id: changed-workflow-or-test with: paths: ./.github/actions ./.github/workflows sermant-integration-tests name: check push event shell: bash run: | eventName=${{ github.event_name }} if [ $eventName == 'push' ];then echo "triggerPushEvent=true" >> $GITHUB_ENV else echo "triggerPushEvent=false" >> $GITHUB_ENV fi name: statistic scenarios change env shell: bash run: | if [ ${{ env.sermantAgentCoreChanged }} == 'true' -o ${{ env.sermantServiceRegistryChanged }} == 'true' ];then echo "enableGraceful=true" >> $GITHUB_ENV fi if [ ${{ steps.changed-workflow-or-test.outputs.changed }} == 'true' ];then echo "enableGraceful=true" >> $GITHUB_ENV fi ``` In the preceding command, `paths` indicates the specified change path. After the preceding action is specified, the environment variable `env.enableGraceful` is stored. The environment variable is used in the workflow, as shown in the following figure: ```yaml name: (graceful) test for springboot=${{ matrix.springBootVersion }} springCloudVersion=${{ matrix.springCloudVersion }} if: env.enableGraceful == 'true' uses: ./.github/actions/scenarios/spring/graceful ``` The `graceful` test case is executed only when `env.enableGraceful` is true. Notices: In addition to the preceding path judgment, the entire workflow path judgment takes effect for event `pull_request`. The configuration is as follows: ```shell on: push: pull_request: branches: main develop paths: 'sermant-agentcore/' 'sermant-integration-tests/' 'sermant-plugins/sermant-dynamic-config/' 'sermant-plugins/sermant-flowcontrol/' 'sermant-plugins/sermant-loadbalancer/' 'sermant-plugins/sermant-service-registry/' 'sermant-plugins/sermant-springboot-registry/' '.github/workflows/spring_integration*.yaml' '.github/actions/' ``` You can find the set corresponding to the `paths` configuration. If the workflow takes effect, the submitted code path must be included. Therefore, if a plugin or directory is added, you must add the corresponding directory in `paths`. How to compile an automated test? The steps are as follows: Check whether the current middleware meets the requirements. (Currently, only ZooKeeper, Nacos, and LocalCse are supported.) (Optional) Add common middleware support. Coding test demos and corresponding Junit test cases for specific scenarios. Create the test action, simulate the manual test process, and reproduce the problem in the action. (Optional) Adding code path detection Add actions to the automatic test entry, that is, workflow, based on the framework type and version matrix. (Optional) Modify the paths of the working ingress. Submit the workflow for testing (1) To add the cache to the middleware, you need to add the cache steps in the corresponding automatic test

The steps are as follows: ```yaml download-midwares-and-cache: name: download midwares and cache runs-on: ubuntu-latest steps: uses: actions/checkout@v3 name: cache zookeeper uses: actions/cache@v3 with: path: apache-zookeeper-3.6.3-bin.tar.gz key: ${{ runner.os }}-apache-zookeeper-3.6.3 restore-keys: | ${{ runner.os }}-apache-zookeeper-3.6.3 name: download zookeeper run: | export ROOT_PATH=$(pwd) bash ./sermant-integration-tests/scripts/tryDownloadMidware.sh zk ``` An example. Use to cache data, and then perform the download step. When `cache-hit` is triggered, the cached ZooKeeper is directly used next time. (2) Add Startup Steps for Mideware Check whether the middleware is universal. If the middleware is universal, it is recommended that the middleware be included in the common action. Otherwise, the life cycle of the middleware is managed only in the current action. a) Add the startup step to the common action. If the application is Dubbo, add the startup steps to`./.github/actions/common/dubbo/action.yml` If the application is a Spring application, add the startup step to`./.github/actions/common/spring/action.yml` b) Add a startup step to the current action. In this case, you need to add the middleware startup step at the beginning of the action, and add the middleware termination step at the end of the action. Find the `sermant-integration-tests` directory and check whether the current test scenario is applicable to the existing demo application. If yes, add related test interfaces. Otherwise, you need to compile an independent test demo. Codeing the Junit test cases in the `sermant-integration-tests/spring-test/spring-integration-test` or `sermant-integration-tests/dubbo-test/dubbo-integration-test` directory. Create a directory in the current scenario in `./.github/actions/scenarios`. For example, the content of `./.github/actions/scenarios/test/action.yml`, is as follows: ```yaml name: "{Scenarios} Test" description: "Auto test for {your scenarios}" runs: using: "composite" steps: name: entry uses: ./.github/actions/common/entry with: log-dir: ./logs/test name: package common demos shell: bash run: | xxx name: start provider that has closed graceful ability shell: bash run: | xxx name: start applications that has opened graceful ability shell: bash env: grace.rule.enableSpring: true run: | xxx name: waiting for services start shell: bash run: | ps -ef | grep java bash ./sermant-integration-tests/scripts/checkService.sh http://127.0.0.1:8015/graceful/testGraceful 120 name: test graceful up shell: bash run: mvn test -Dsermant.integration.test.type=GRACEFUL -Dgraceful.test.type=up --file sermant-integration-tests/spring-test/pom.xml name: exit if: always() uses: ./.github/actions/common/exit with: processor-keyword: feign|rest name: if failure then upload error log uses: actions/upload-artifact@v3 if: ${{ failure() || cancelled() }} with: name: (${{ github.job }})-graceful-(${{ matrix.springBootVersion }}-${{ matrix.springCloudVersion }})-logs path: | ./*.log ./logs/ if-no-files-found: warn retention-days: 2 ``` As shown above, there are the following steps: entry package demo start test exit if failure then upload error log Upload error logs when errors occur Perform the preceding steps. Steps 2, 3, and 4 are used to simulate the end-to-end test. Referring to . If the plugin type is a new plug-in type, add the corresponding plugin path and trigger the action test using environment variables at the entrance of the automatic test. For details about the entry, see . Add the current action as follows (for details, see Graceful Online and Offline): ```yaml steps: name: (graceful) test for springboot=${{ matrix.springBootVersion }} springCloudVersion=${{ matrix.springCloudVersion }} if: env.enableGraceful == 'true' uses: ./.github/actions/scenarios/spring/graceful ``` If the matrix of the current entry is seriously inconsistent with the test scenario, you can add an entry or add a job based on the workflow of the entry to compile your own version matrix. (For details, see the` test-for-spring` job task.) Certainly, steps may be reused herein, for example, middleware caching and common processing

```yaml on: push: pull_request: branches: main develop paths: 'sermant-agentcore/' 'sermant-plugins/sermant-dynamic-config/' 'sermant-plugins/sermant-flowcontrol/' 'sermant-plugins/sermant-loadbalancer/' 'sermant-plugins/sermant-service-registry/' 'sermant-plugins/sermant-springboot-registry/' '.github/workflows/spring_integration*.yaml' '.github/actions/' ``` The preceding figure shows the triggering event of the work ingress. The workflow is triggered only when `branches` and `paths` meet the requirements at the same time. In this example, `paths` are used for path detection. The workflow is triggered only when the path is matched with the code in the paths list. Therefore, if the new code is in another directory, you must add the path to `paths`. Currently, the automatic test version is unified and defined in the workflow entry, as shown in the following figure: ```yaml name: Spring Integration Test1 env: sermantVersion: 1.0.0 ``` The following demo startup commands must use the environment variable `${{env.sermantVersion}}` as the agent version path. For example: `-javaagent:sermant-agent-${{ env.sermantVersion }}/agent/sermant-agent.jar=appName=default` Currently, a concurrent group is defined for each working inbound interface. The configuration is as follows: ```yaml concurrency: group: ${{ github.workflow }}-${{ github.event.pullrequest.number || github.ref }}-${{ github.headref }} cancel-in-progress: true ``` `${{ github.workflow }}` indicates the name of the current workflow `${{ github.event.pull_request.number }}` indicates the ID of the pull request `${{ github.ref }}` indicates the target branch of the pull request; When the event is a push event, the value is the current branch `${{ github.head_ref }}` indicates the source branch of the pull request; When the event is a push event, the value is null. If two tasks have the same group, the former will cancel the workflow. The triggering of a specific action (specific test scenario) depends on the triggering of a workflow. The triggering of a workflow depends on the `push` and `pull_request` events. Therefore, the automatic test workflow mainly depends on the preceding two events. In addition, the workflow triggering and action triggering must be preferred. Only when the workflow is triggered, the action layer can be triggered. The following describes workflow triggering and action triggering based on priorities. (1) Trigger Workflow `push` event, the commits. This event triggers all workflows by default, regardless of branches and paths. `pull_request`, the PR. This event takes into account the branch name and code path. Take chestnuts, as follows: ```yaml on: pull_request: branches: main develop paths: 'sermant-agentcore/' 'sermant-integration-tests/' 'sermant-plugins/sermant-dynamic-config/' ``` The preceding trigger conditions are as follows: The merged branch is the `main` or `develop` branch. The submitted code path must be in one of the `sermant-agentcore,sermant-integration-tests,sermant-plugins/sermant-dynamic-config`. It will take effect only when both of the preceding conditions are met. Therefore, if you add a plugin test, you must add a new directory to `paths`. 2Trigger ActionActions in specific test scenarios and non-public actions) An action can be triggered only after a workflow is triggered. The triggering of an action is more about the code path, but a specific path or event will enable full automated testing. The following will describe by using a specific event. `push` event. This event enables all action tests by default. You can find the action file for path judgment , refer to . If the event is `push`event, all action switches are enabled. `pull_request` event is used to determine the code change path. For details, see . Different test scenarios are determined based on different paths. For example, if the `sermant-plugins/sermant-router` file is changed, the test can be enabled for all associated route scenarios. In addition, the corresponding switch is added to the automatic test entry to control the execution of the automatic test.