Repository: Sage-Bionetworks/schematic (language: python)

Issue:
[FDS-1725] Missing `entityId` handling testing
**Description:**
This PR follows up on #1456, where the `filenameExists` validation rule was implemented. This rule handles cases where a file path present in a manifest does not exist in a dataset and cases where the entityId provided in a manifest row does not match with its corresponding file path. 

This PR adds additional error handling for situations including:
- The entityId field in a manifest row is empty
- The entityId provided in a manifest row does not exist in the dataset

Previously, an outer join was used in the validation rule followed by a line that handled cases where there were more rows in the dataset than the manifest. The outer join implementation did not always maintain the order of manifest rows resulting in incorrect errors on manifest rows. I changed this outer join to a left join on the manifest to handle both issues.

The row number indexing for `filename_validation` is also updated to match the other validation rules for consistency.

I have also updated `generate_filename_error` to handle the new error cases.

**Testing:**
- New integration test cases for `filename_validation` have been added as needed.
- Unit tests for `filename_validation` and `generate_filename_error` have been added


Discussion / hints:
For the new cases where we're checking if entityIds don't exist at all or are missing from the manifest we should also add validation error messages to [`generate_filename_error`](https://github.com/Sage-Bionetworks/schematic/blob/develop/schematic/models/validate_attribute.py#L482) that specify to the user what the issue with their metadata was. And then the validation integration test should be updated to check for those error messages for the test cases you add to the manifest
Thanks for the comments @GiaJordan! I am going to revert this back to being a draft for now and work on those updates.

Fix patch:
diff --git a/.github/workflows/docker.yml b/.github/workflows/docker.yml
index 58f903f76..12b355dc0 100644
--- a/.github/workflows/docker.yml
+++ b/.github/workflows/docker.yml
@@ -17,16 +17,16 @@ jobs:
     steps:
       -
         name: Checkout
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
       -
         name: Set up QEMU
-        uses: docker/setup-qemu-action@v2
+        uses: docker/setup-qemu-action@v3
       -
         name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v2
+        uses: docker/setup-buildx-action@v3
       -
         name: Login to DockerHub
-        uses: docker/login-action@v2
+        uses: docker/login-action@v3
         with:
           username: schematicbot
           password: ${{ secrets.DOCKER_HUB_TOKEN }}
@@ -36,7 +36,7 @@ jobs:
         run: echo "::set-output name=sha_short::$(git rev-parse --short HEAD)"
       -
         name: Build and push (tagged release)
-        uses: docker/build-push-action@v3
+        uses: docker/build-push-action@v6
         if: ${{ github.event_name == 'push' }}
         with:
           platforms: linux/amd64,linux/arm64
@@ -48,7 +48,7 @@ jobs:
             ${{ env.DOCKER_ORG }}/${{ env.DOCKER_REPO }}:commit-${{ steps.vars.outputs.sha_short }}
       -
         name: Build and push (manual release)
-        uses: docker/build-push-action@v3
+        uses: docker/build-push-action@v6
         if: ${{ github.event_name == 'workflow_dispatch' }}
         with:
           platforms: linux/amd64,linux/arm64
diff --git a/.github/workflows/docker_build.yml b/.github/workflows/docker_build.yml
index 50596b7db..f1beb3489 100644
--- a/.github/workflows/docker_build.yml
+++ b/.github/workflows/docker_build.yml
@@ -23,13 +23,13 @@ jobs:
 
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v2
+        uses: actions/checkout@v4
           
       - name: Set env variable for version tag
         run: echo "RELEASE_VERSION=${GITHUB_REF#refs/*/}" >> $GITHUB_ENV
 
       - name: Log in to the Container registry
-        uses: docker/login-action@f054a8b539a109f9f41c372932f1ae047eff08c9
+        uses: docker/login-action@v3
         with:
           registry: ${{ env.REGISTRY }}
           username: ${{ github.actor }}
@@ -37,7 +37,7 @@ jobs:
 
       - name: Extract metadata (tags, labels) for Docker
         id: meta
-        uses: docker/metadata-action@98669ae865ea3cffbcbaa878cf57c20bbf1c6c38
+        uses: docker/metadata-action@v5
         with:
           images: ${{ env.IMAGE_PATH }}
           tags: |
@@ -46,7 +46,7 @@ jobs:
             type=semver,pattern={{raw}}
 
       - name: Build and push Docker image
-        uses: docker/build-push-action@ad44023a93711e3deb337508980b4b5e9bcdc5dc
+        uses: docker/build-push-action@v6
         with:
           file: schematic_api/Dockerfile
           push: true
diff --git a/.github/workflows/pdoc.yml b/.github/workflows/pdoc.yml
index 51924c4b4..fdf0be220 100644
--- a/.github/workflows/pdoc.yml
+++ b/.github/workflows/pdoc.yml
@@ -37,10 +37,10 @@ jobs:
       #       check-out repo and set-up python     
       #----------------------------------------------
       - name: Check out repository
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
 
       - name: Set up Python ${{ matrix.python-version }}
-        uses: actions/setup-python@v3
+        uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python-version }}
       
@@ -59,10 +59,10 @@ jobs:
       #----------------------------------------------
       - name: Load cached venv
         id: cached-poetry-dependencies
-        uses: actions/cache@v3
+        uses: actions/cache@v4
         with:
           path: .venv
-          key: venv-${{ runner.os }}-${{ hashFiles('**/poetry.lock') }}
+          key: venv-${{ runner.os }}-${{ hashFiles('**/poetry.lock') }}-${{ matrix.python-version }}
 
       #----------------------------------------------
       # install dependencies and root project
@@ -76,7 +76,7 @@ jobs:
       - run: poetry show pdoc
       - run: poetry run pdoc --docformat google --mermaid -o docs/schematic schematic/manifest schematic/models schematic/schemas schematic/store schematic/utils schematic/visualization
 
-      - uses: actions/upload-pages-artifact@v1
+      - uses: actions/upload-pages-artifact@v3
         with:
           path: docs/schematic
 
@@ -93,4 +93,4 @@ jobs:
       url: ${{ steps.deployment.outputs.page_url }}
     steps:
       - id: deployment
-        uses: actions/deploy-pages@v1
+        uses: actions/deploy-pages@v4
diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml
index eb2ebafcb..c3225661e 100644
--- a/.github/workflows/publish.yml
+++ b/.github/workflows/publish.yml
@@ -16,10 +16,10 @@ jobs:
       #       check-out repo and set-up python     
       #----------------------------------------------
       - name: Check out repository
-        uses: actions/checkout@v2
+        uses: actions/checkout@v4
 
       - name: Set up Python ${{ matrix.python-version }}
-        uses: actions/setup-python@v2
+        uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python-version }}
 
@@ -38,10 +38,10 @@ jobs:
       #----------------------------------------------
       - name: Load cached venv
         id: cached-poetry-dependencies
-        uses: actions/cache@v2
+        uses: actions/cache@v4
         with:
           path: .venv
-          key: venv-${{ runner.os }}-${{ hashFiles('**/poetry.lock') }}
+          key: venv-${{ runner.os }}-${{ hashFiles('**/poetry.lock') }}-${{ matrix.python-version }}
 
       #----------------------------------------------
       # install dependencies and root project
diff --git a/.github/workflows/scan_repo.yml b/.github/workflows/scan_repo.yml
index 6a93beee7..56c7ac35a 100644
--- a/.github/workflows/scan_repo.yml
+++ b/.github/workflows/scan_repo.yml
@@ -14,7 +14,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout code
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
 
       - name: Run Trivy vulnerability scanner in repo mode
         uses: aquasecurity/trivy-action@master
@@ -32,4 +32,4 @@ jobs:
         uses: github/codeql-action/upload-sarif@v3
         with:
           sarif_file: 'trivy-results.sarif'
-          category: Git Repository
\ No newline at end of file
+          category: Git Repository


Test patch:
diff --git a/.github/workflows/api_test.yml b/.github/workflows/api_test.yml
index 354cbe3ba..31b8e7eff 100644
--- a/.github/workflows/api_test.yml
+++ b/.github/workflows/api_test.yml
@@ -27,10 +27,10 @@ jobs:
       #       check-out repo and set-up python
       #----------------------------------------------
       - name: Check out repository
-        uses: actions/checkout@v2
+        uses: actions/checkout@v4
 
       - name: Set up Python ${{ matrix.python-version }}
-        uses: actions/setup-python@v2
+        uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python-version }}
 
@@ -48,10 +48,10 @@ jobs:
       #----------------------------------------------
       - name: Load cached venv
         id: cached-poetry-dependencies
-        uses: actions/cache@v2
+        uses: actions/cache@v4
         with:
           path: .venv
-          key: venv-${{ runner.os }}-${{ hashFiles('**/poetry.lock') }}
+          key: venv-${{ runner.os }}-${{ hashFiles('**/poetry.lock') }}-${{ matrix.python-version }}
 
       #----------------------------------------------
       # install dependencies and root project
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
index c17a4981d..4df1cb2b9 100644
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -39,10 +39,10 @@ jobs:
       #       check-out repo and set-up python
       #----------------------------------------------
       - name: Check out repository
-        uses: actions/checkout@v2
+        uses: actions/checkout@v4
 
       - name: Set up Python ${{ matrix.python-version }}
-        uses: actions/setup-python@v2
+        uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python-version }}
 
@@ -72,10 +72,10 @@ jobs:
       #----------------------------------------------
       - name: Load cached venv
         id: cached-poetry-dependencies
-        uses: actions/cache@v2
+        uses: actions/cache@v4
         with:
           path: .venv
-          key: venv-${{ runner.os }}-${{ hashFiles('**/poetry.lock') }}
+          key: venv-${{ runner.os }}-${{ hashFiles('**/poetry.lock') }}-${{ matrix.python-version }}
 
       #----------------------------------------------
       # install dependencies and root project