Repository: RasaHQ/rasa
Pull request: Separate docs-tests from CI workflow
**Proposed changes**:
- [X] Separate `docs-tests` job from CI workflow. 
`docs-tests` now becomes a single workflow such that we can re-run it if it is failed
- [x] Let CI workflow wait for `docs-tests` workflow

**Status (please check what you already did)**:
- [ ] added some tests for the functionality
- [ ] updated the documentation
- [ ] updated the changelog (please check [changelog](https://github.com/RasaHQ/rasa/tree/main/changelog) for instructions)
- [ ] reformat files using `black` (please check [Readme](https://github.com/RasaHQ/rasa#code-style) for instructions)


Problem statement:
Documentation checks - retry if a link is not reachable 
One of the checks in the Publish Documentation workflow checks if links in the docs are still reachable, but sometimes it happens that one of the links didn't respond on time, in such a scenario the workflow will finish with failure. 

Things to improve:
- increase the timeout for connection
- retry if a connection failed

Hints from issue discussion:
I looked [publish documentation workflow](https://github.com/RasaHQ/rasa/blob/master/.github/workflows/documentation.yml). Not sure about which check is reaching links. @tczekajlo Do you have an idea? 
> I looked [publish documentation workflow](https://github.com/RasaHQ/rasa/blob/master/.github/workflows/documentation.yml). Not sure about which check is reaching links. @tczekajlo Do you have an idea?

Everything is in the Makefile, the `test-docs` section.
The test-docs workflow from CI uses the command `remark --frail docs/*.mdx docs/*/*.mdx` (see [here](https://github.com/RasaHQ/rasa/blob/5549b07d70a39819bf793017de3badd7ea903bc6/docs/package.json#L24)) We can look at the [remark cli ](https://github.com/remarkjs/remark/tree/main/packages/remark-cli)for timeout or retry option. If remark does not privde such option, we can certainly separate the test-docs workflow from CI.

Commit message: Merge branch 'main' into separate-workflow-for-docs-tests

Patch to review:
diff --git a/.github/workflows/ci-docs-tests.yml b/.github/workflows/ci-docs-tests.yml
new file mode 100644
index 000000000000..c1e0572e10c0
--- /dev/null
+++ b/.github/workflows/ci-docs-tests.yml
@@ -0,0 +1,95 @@
+name: Docs Tests
+on:
+  push:
+    branches:
+    - main
+    tags:
+    - '*'
+  pull_request:
+    types: [opened, synchronize, labeled]
+
+env:
+  GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
+jobs:
+  changes:
+    name: Check for file changes
+    runs-on: ubuntu-latest
+    outputs:
+      backend: ${{ steps.filter.outputs.backend }}
+      docker: ${{ steps.filter.outputs.docker }}
+      docs: ${{ steps.filter.outputs.docs }}
+    steps:
+      - uses: actions/checkout@v2
+      - uses: RasaHQ/pr-changed-files-filter@c4f7116a04b8a4596313469429e2ad235f59d9c4
+        id: filter
+        with:
+          token: ${{ secrets.GITHUB_TOKEN }}
+          filters: .github/change_filters.yml
+
+  test_documentation:
+    name: Test Documentation
+    runs-on: ubuntu-latest
+    needs: [ changes ]
+
+    steps:
+    - name: Checkout git repository 🕝
+      if: needs.changes.outputs.docs == 'true'
+      uses: actions/checkout@v2
+
+    - name: Set up Python 3.7 🐍
+      if: needs.changes.outputs.docs == 'true'
+      uses: actions/setup-python@v1
+      with:
+        python-version: 3.7
+
+    - name: Set up Node 12.x 🦙
+      if: needs.changes.outputs.docs == 'true'
+      uses: actions/setup-node@v1
+      with:
+        node-version: '12.x'
+
+    - name: Read Poetry Version 🔢
+      if: needs.changes.outputs.docs == 'true'
+      run: |
+        echo "POETRY_VERSION=$(scripts/poetry-version.sh)" >> $GITHUB_ENV
+      shell: bash
+
+    - name: Install poetry 🦄
+      if: needs.changes.outputs.docs == 'true'
+      uses: Gr1N/setup-poetry@v4
+      with:
+        poetry-version: ${{ env.POETRY_VERSION }}
+
+    - name: Load Poetry Cached Libraries ⬇
+      if: needs.changes.outputs.docs == 'true'
+      uses: actions/cache@v2
+      with:
+        path: ~/.cache/pypoetry/virtualenvs
+        key: ${{ runner.os }}-poetry-${{ env.POETRY_VERSION }}-3.7-${{ hashFiles('**/poetry.lock') }}-${{ secrets.POETRY_CACHE_VERSION }}
+        restore-keys: ${{ runner.os }}-poetry-3.7
+
+    - name: Load Yarn Cached Packages ⬇
+      if: needs.changes.outputs.docs == 'true'
+      uses: actions/cache@v2
+      with:
+        path: docs/node_modules
+        key: ${{ runner.os }}-yarn-12.x-${{ hashFiles('docs/yarn.lock') }}
+        restore-keys: ${{ runner.os }}-yarn-12.x
+
+    - name: Install Dependencies 📦
+      if: needs.changes.outputs.docs == 'true'
+      run: |
+        sudo apt-get -y install libpq-dev
+        make install-full install-docs
+
+    - name: Run Swagger 🕵️‍♀️
+      if: needs.changes.outputs.docs == 'true'
+      run: |
+        npm install -g swagger-cli
+        swagger-cli validate docs/static/spec/action-server.yml
+        swagger-cli validate docs/static/spec/rasa.yml
+
+    - name: Test Docs 🕸
+      if: needs.changes.outputs.docs == 'true'
+      run: make test-docs
\ No newline at end of file
diff --git a/.github/workflows/continous-integration.yml b/.github/workflows/continous-integration.yml
index 79f485c4e554..1f2b666f22d0 100644
--- a/.github/workflows/continous-integration.yml
+++ b/.github/workflows/continous-integration.yml
@@ -28,9 +28,15 @@ env:
   # needed to fix issues with boto during testing:
   # https://github.com/travis-ci/travis-ci/issues/7940
   BOTO_CONFIG: /dev/null
+
   IS_TAG_BUILD: ${{ startsWith(github.event.ref, 'refs/tags') }}
   DOCKERHUB_USERNAME: tmbo
 
+  # for wait_for_xx jobs
+  WAIT_TIMEOUT_SECS: 3000
+  WAIT_INTERVAL_SECS: 60
+
+
 jobs:
   changes:
     name: Check for file changes
@@ -47,72 +53,28 @@ jobs:
           token: ${{ secrets.GITHUB_TOKEN }}
           filters: .github/change_filters.yml
 
-  documentation:
-    name: Test Documentation
+
+  wait_for_docs_tests:
+    # Looks for doc test workflows and waits for it to complete successfully
+    # Runs on tag pushes and pushes to main exclusively, as it is a dependency of release jobs
+    name: Wait for docs tests
     runs-on: ubuntu-latest
-    needs: [changes]
+    if: github.event_name == 'push' && (startsWith(github.event.ref, 'refs/tags') || github.ref == 'refs/heads/main')
 
     steps:
-    - name: Checkout git repository 🕝
-      if: needs.changes.outputs.docs == 'true'
-      uses: actions/checkout@v2
+      - name: Checkout git repository 🕝
+        uses: actions/checkout@v2
 
-    - name: Set up Python 3.7 🐍
-      if: needs.changes.outputs.docs == 'true'
-      uses: actions/setup-python@v1
-      with:
-        python-version: 3.7
-
-    - name: Set up Node 12.x 🦙
-      if: needs.changes.outputs.docs == 'true'
-      uses: actions/setup-node@v1
-      with:
-        node-version: '12.x'
-
-    - name: Read Poetry Version 🔢
-      if: needs.changes.outputs.docs == 'true'
-      run: |
-        echo "POETRY_VERSION=$(scripts/poetry-version.sh)" >> $GITHUB_ENV
-      shell: bash
-
-    - name: Install poetry 🦄
-      if: needs.changes.outputs.docs == 'true'
-      uses: Gr1N/setup-poetry@v4
-      with:
-        poetry-version: ${{ env.POETRY_VERSION }}
-
-    - name: Load Poetry Cached Libraries ⬇
-      if: needs.changes.outputs.docs == 'true'
-      uses: actions/cache@v2
-      with:
-        path: ~/.cache/pypoetry/virtualenvs
-        key: ${{ runner.os }}-poetry-${{ env.POETRY_VERSION }}-3.7-${{ hashFiles('**/poetry.lock') }}-${{ secrets.POETRY_CACHE_VERSION }}
-        restore-keys: ${{ runner.os }}-poetry-3.7
-
-    - name: Load Yarn Cached Packages ⬇
-      if: needs.changes.outputs.docs == 'true'
-      uses: actions/cache@v2
-      with:
-        path: docs/node_modules
-        key: ${{ runner.os }}-yarn-12.x-${{ hashFiles('docs/yarn.lock') }}
-        restore-keys: ${{ runner.os }}-yarn-12.x
-
-    - name: Install Dependencies 📦
-      if: needs.changes.outputs.docs == 'true'
-      run: |
-        sudo apt-get -y install libpq-dev
-        make install-full install-docs
-
-    - name: Run Swagger 🕵️‍♀️
-      if: needs.changes.outputs.docs == 'true'
-      run: |
-        npm install -g swagger-cli
-        swagger-cli validate docs/static/spec/action-server.yml
-        swagger-cli validate docs/static/spec/rasa.yml
+      - name: Wait for doc tests
+        uses: fountainhead/action-wait-for-check@4699210ccc66e2a13260803fadbb77085421b891
+        id: wait-for-doc-tests
+        with:
+          token: ${{ secrets.GITHUB_TOKEN }}
+          checkName: Test documentation
+          ref: ${{ github.event.pull_request.head.sha || github.sha }}
+          timeoutSeconds: ${{ env.WAIT_TIMEOUT_SECS }}
+          intervalSeconds: ${{ env.WAIT_INTERVAL_SECS }}
 
-    - name: Test Docs 🕸
-      if: needs.changes.outputs.docs == 'true'
-      run: make test-docs
 
   quality:
     name: Code Quality
@@ -505,7 +467,7 @@ jobs:
 
     # deploy will only be run when there is a tag available
     if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags') && github.repository == 'RasaHQ/rasa'
-    needs: [quality, test, documentation, docker]  # only run after all other stages succeeded
+    needs: [quality, test, wait_for_docs_tests, docker]  # only run after all other stages succeeded
 
     steps:
     - name: Checkout git repository 🕝
@@ -569,7 +531,7 @@ jobs:
 
   mergepal-merge: # run merge pal in the end
     runs-on: ubuntu-latest
-    needs: [quality, test, documentation, docker]
+    needs: [quality, test,  wait_for_docs_tests, docker]
     steps:
       - uses: actions/checkout@v1
       - name: Make sure there is a github token


Code review comments:
diff_hunk: @@ -47,72 +53,28 @@ jobs:
           token: ${{ secrets.GITHUB_TOKEN }}
           filters: .github/change_filters.yml
 
-  documentation:
-    name: Test Documentation
+
+  wait_for_docs_tests:
+    # Looks for doc test workflows and waits for it to complete successfully
+    # Runs on tag pushes and pushes to main exclusively, as it is a dependency of release jobs
+    name: Wait for docs tests
     runs-on: ubuntu-latest
-    needs: [changes]
+    if: github.event_name == 'push' && (startsWith(github.event.ref, 'refs/tags') || github.ref == 'refs/heads/main')
path: .github/workflows/continous-integration.yml
text: @user1:
Why do we need it? I see that the `Test Documentation` runs always, thus we should have this `if` here 

@author:
You mean we don't need the `if` in line 62? I agree. This is a copy-and-paste error. I didn't mean to include line 62. I will delete it.