Repository: PowerGridModel/power-grid-model-io
Pull request #307: Github release workflow fix: Rename job call
Author: login: nitbharambe
Created: 2025-05-27T14:26:08Z | Merged: 2025-05-30T07:16:00Z

**Fixes**: 
publishing post v1.2.123

### Changes proposed in this PR include:

* Rename call of build-test-release
* Homogenize workflows in PGM and PGM-IO to make life easier

### Checks

- [x] If changes are related to CI/CD, are they verified via a manual run on this branch? (https://github.com/PowerGridModel/power-grid-model-io/actions/runs/15303577121)



Diff:
diff --git a/.github/workflows/build-test-release.yml b/.github/workflows/build-test-release.yml
index f5c1dfeb..90027210 100644
--- a/.github/workflows/build-test-release.yml
+++ b/.github/workflows/build-test-release.yml
@@ -3,16 +3,9 @@
 # SPDX-License-Identifier: MPL-2.0
 
 
-name: Build, Test, Sonar
+name: Build and Test
 
 on:
-  push:
-    branches:
-      - main
-  # run pipeline on pull request
-  pull_request:
-  # run pipeline on merge queue
-  merge_group:
   # run pipeline from another workflow
   workflow_call:
     inputs:
@@ -71,41 +64,6 @@ jobs:
           name: power-grid-model-io
           path: wheelhouse/
 
-  sonar-cloud:
-    permissions:
-      contents: write
-    runs-on: ubuntu-latest
-    steps:
-
-      - name: Checkout source code
-        uses: actions/checkout@v4
-        with:
-          fetch-depth: 0  # Shallow clones should be disabled for a better relevancy of analysis
-
-      - name: Setup Python 3.11
-        uses: actions/setup-python@v5
-        with:
-          python-version: "3.11"
-
-      - name: Install in develop mode
-        run: |
-          pip install -e .[dev]
-
-      - name: Test and Coverage
-        run: |
-          pytest
-
-          # Fix relative paths in coverage file
-          # Known bug: https://community.sonarsource.com/t/sonar-on-github-actions-with-python-coverage-source-issue/36057
-          sed -i 's@/home/runner/work/power-grid-model-io/power-grid-model-io@/github/workspace@g' python_coverage.xml
-
-      - name: SonarCloud Scan
-        if: ${{ (github.event_name == 'push') || (github.event.pull_request.head.repo.owner.login == 'PowerGridModel') }}
-        uses: SonarSource/sonarqube-scan-action@master
-        env:
-          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-          SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
-
   unit-tests:
     needs: build-python
     strategy:
@@ -183,7 +141,6 @@ jobs:
       - build-python
       - unit-tests
       - validation-tests
-      - sonar-cloud
     permissions:
       contents: write
     runs-on: ubuntu-latest
@@ -207,7 +164,7 @@ jobs:
         run: echo "${{ steps.tag.outputs.tag }}"
 
       - name: Create GitHub release
-        if: (github.event_name == 'push') || ((github.event_name == 'workflow_dispatch') && (github.event.inputs.create_release == 'true'))
+        if: (inputs.create_release)
         uses: softprops/action-gh-release@v2
         with:
           files: |
diff --git a/.github/workflows/check-code-quality.yml b/.github/workflows/check-code-quality.yml
index cfc25e1c..9e38edf2 100644
--- a/.github/workflows/check-code-quality.yml
+++ b/.github/workflows/check-code-quality.yml
@@ -6,13 +6,6 @@
 name: Check Code Quality
 
 on:
-  push:
-    branches:
-      - main
-  # run pipeline on pull request
-  pull_request:
-  # run pipeline on merge queue
-  merge_group:
   # run pipeline from another workflow
   workflow_call:
 
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 69cf0219..a49ecc45 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -3,7 +3,7 @@
 # SPDX-License-Identifier: MPL-2.0
 
 
-name: Build, Test, Sonar and Publish
+name: CI Build
 
 on:
   push:
@@ -13,14 +13,6 @@ on:
   pull_request:
   # run pipeline on merge queue
   merge_group:
-  # run pipeline from another workflow
-  workflow_call:
-    inputs:
-      create_release:
-        type: boolean
-        description: Create a (pre-)release when CI passes
-        default: false
-        required: false
   # run this workflow manually from the Actions tab
   workflow_dispatch:
     inputs:
@@ -31,10 +23,15 @@ on:
         required: true
 
 concurrency:
-  group: ${{ github.workflow }}-${{ github.ref }}-main
+  group: ${{ github.workflow }}-${{ github.ref }}-ci-build
   cancel-in-progress: true
 
 jobs:
+  ci-started:
+    runs-on: ubuntu-latest
+    steps:
+      - run: echo "ci started"
+
   build-test-release:
     name: build-test-release
     uses: "./.github/workflows/build-test-release.yml"
@@ -46,14 +43,27 @@ jobs:
       # otherwise create_release becomes false
       create_release: ${{ (github.event_name == 'workflow_dispatch' && inputs.create_release) || github.event_name == 'push'}}
 
+  check-code-quality:
+    uses: "./.github/workflows/check-code-quality.yml"
+
+  reuse-compliance:
+    uses: "./.github/workflows/reuse-compliance.yml"
+
+  ci-passed:
+    runs-on: ubuntu-latest
+    needs: [ci-started, build-test-release, check-code-quality, reuse-compliance]
+
+    steps:
+      - run: echo "ci passed"
+      
   publish:
     name: Publish to PyPI
     runs-on: ubuntu-latest
     permissions:
       contents: write
       id-token: write  # Required for Trusted Publishing
-    needs: github-release
-    if: (github.event_name == 'workflow_dispatch') || github.event_name == 'push'
+    needs: build-test-release
+    if: (github.event_name == 'workflow_dispatch' && inputs.create_release) || github.event_name == 'push'
 
     steps:
       - name: Download assets from GitHub release
diff --git a/.github/workflows/nightly.yml b/.github/workflows/nightly.yml
index d69e070b..aa767627 100644
--- a/.github/workflows/nightly.yml
+++ b/.github/workflows/nightly.yml
@@ -15,7 +15,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  main:
+  build-test-release:
     uses: "./.github/workflows/build-test-release.yml"
     permissions:
       contents: write
diff --git a/.github/workflows/reuse-compliance.yml b/.github/workflows/reuse-compliance.yml
index dbed5882..5c342b73 100644
--- a/.github/workflows/reuse-compliance.yml
+++ b/.github/workflows/reuse-compliance.yml
@@ -6,13 +6,6 @@
 name: REUSE Compliance Check
 
 on:
-  push:
-    branches:
-      - main
-  # run pipeline on pull request
-  pull_request:
-  # run pipeline on merge queue
-  merge_group:
   # run pipeline from another workflow
   workflow_call:
   # run this workflow manually from the Actions tab
diff --git a/.github/workflows/sonar.yml b/.github/workflows/sonar.yml
new file mode 100644
index 00000000..fbbd9aab
--- /dev/null
+++ b/.github/workflows/sonar.yml
@@ -0,0 +1,55 @@
+# SPDX-FileCopyrightText: Contributors to the Power Grid Model project <powergridmodel@lfenergy.org>
+#
+# SPDX-License-Identifier: MPL-2.0
+
+name: Sonar Cloud
+
+on:
+  # run pipeline on push event of main branch
+  push:
+    branches:
+      - main
+  # run pipeline on pull request
+  pull_request:
+  # run pipeline on merge queue
+  merge_group:
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}-sonar
+  cancel-in-progress: true
+
+jobs:
+  sonar-cloud:
+    permissions:
+      contents: write
+    runs-on: ubuntu-latest
+    steps:
+
+      - name: Checkout source code
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0  # Shallow clones should be disabled for a better relevancy of analysis
+
+      - name: Setup Python 3.11
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+
+      - name: Install in develop mode
+        run: |
+          pip install -e .[dev]
+
+      - name: Test and Coverage
+        run: |
+          pytest
+
+          # Fix relative paths in coverage file
+          # Known bug: https://community.sonarsource.com/t/sonar-on-github-actions-with-python-coverage-source-issue/36057
+          sed -i 's@/home/runner/work/power-grid-model-io/power-grid-model-io@/github/workspace@g' python_coverage.xml
+
+      - name: SonarCloud Scan
+        if: ${{ (github.event_name == 'push') || (github.event.pull_request.head.repo.owner.login == 'PowerGridModel') }}
+        uses: SonarSource/sonarqube-scan-action@master
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
\ No newline at end of file
diff --git a/README.md b/README.md
index 515edd24..61182a3f 100644
--- a/README.md
+++ b/README.md
@@ -10,9 +10,7 @@ SPDX-License-Identifier: MPL-2.0
 [![Downloads](https://static.pepy.tech/badge/power-grid-model-io)](https://pepy.tech/project/power-grid-model-io)
 [![Downloads](https://static.pepy.tech/badge/power-grid-model-io/month)](https://pepy.tech/project/power-grid-model-io)
 
-[![Build and Test Python](https://github.com/PowerGridModel/power-grid-model-io/actions/workflows/build-test-and-sonar.yml/badge.svg)](https://github.com/PowerGridModel/power-grid-model-io/actions/workflows/build-test-and-sonar.yml)
-[![Check Code Quality](https://github.com/PowerGridModel/power-grid-model-io/actions/workflows/check-code-quality.yml/badge.svg)](https://github.com/PowerGridModel/power-grid-model-io/actions/workflows/check-code-quality.yml)
-[![REUSE Compliance Check](https://github.com/PowerGridModel/power-grid-model-io/actions/workflows/reuse-compliance.yml/badge.svg)](https://github.com/PowerGridModel/power-grid-model-io/actions/workflows/reuse-compliance.yml)
+[![CI Build](https://github.com/PowerGridModel/power-grid-model-io/actions/workflows/ci.yml/badge.svg)](https://github.com/PowerGridModel/power-grid-model-io/actions/workflows/ci.yml)
 [![docs](https://readthedocs.org/projects/power-grid-model-io/badge/)](https://power-grid-model-io.readthedocs.io/en/stable/)
 
 [![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=PowerGridModel_power-grid-model-io&metric=alert_status)](https://sonarcloud.io/summary/new_code?id=PowerGridModel_power-grid-model-io)
@@ -26,7 +24,7 @@ SPDX-License-Identifier: MPL-2.0
 
 [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.8059257.svg)](https://zenodo.org/record/8059257)
 
-[![](https://github.com/PowerGridModel/.github/blob/main/artwork/svg/color.svg)](#)
+[![Power Grid Model logo](https://github.com/PowerGridModel/.github/blob/main/artwork/svg/color.svg)](#)
 
 # Power Grid Model Input/Output
 


Reviews:
user: login: mgovers
body: I guess everything works except:
* [x] the required status check.
* [x] Test the upload by publishing to TestPyPI (https://test.pypi.org/legacy/) cfr. the inline comments on the upload action
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-28T08:54:49Z
user: login: mgovers
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-28T09:34:28Z
user: login: mgovers
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-28T12:16:47Z
user: login: figueroa1395
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-28T15:23:55Z
user: login: figueroa1395
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-28T15:29:02Z
user: login: mgovers
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-30T06:21:35Z
user: login: mgovers
state: APPROVED
author_association: MEMBER
submitted_at: 2025-05-30T07:09:22Z
user: login: mgovers
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-30T07:58:56Z

Review comments:
diff_hunk: @@ -46,13 +43,26 @@ jobs:
       # otherwise create_release becomes false
       create_release: ${{ (github.event_name == 'workflow_dispatch' && inputs.create_release) || github.event_name == 'push'}}
 
+  check-code-quality:
+    uses: "./.github/workflows/check-code-quality.yml"
+
+  reuse-compliance:
+    uses: "./.github/workflows/reuse-compliance.yml"
+
+  ci-passed:
+    runs-on: ubuntu-latest
+    needs: [ci-started, build-test-release, check-code-quality, reuse-compliance]
+
+    steps:
+      - run: echo "ci passed"
+      
   publish:
     name: Publish to PyPI
     runs-on: ubuntu-latest
     permissions:
       contents: write
       id-token: write  # Required for Trusted Publishing
-    needs: github-release
+    needs: build-test-release
     if: (github.event_name == 'workflow_dispatch') || github.event_name == 'push'
path: .github/workflows/ci.yml
user: login: mgovers
body: ```suggestion
    if: (github.event_name == 'workflow_dispatch' && inputs.create_release) || github.event_name == 'push'
```
created_at: 2025-05-28T09:33:59Z
author_association: MEMBER
diff_hunk: @@ -80,7 +90,7 @@ jobs:
         uses: pypa/gh-action-pypi-publish@release/v1
         with:
           # To test, use the TestPyPI:
-          # repository-url: https://test.pypi.org/legacy/
+          repository-url: https://test.pypi.org/legacy/
path: .github/workflows/ci.yml
user: login: mgovers
body: TODO: remove
created_at: 2025-05-28T12:16:47Z
author_association: MEMBER
diff_hunk: @@ -46,14 +43,27 @@ jobs:
       # otherwise create_release becomes false
       create_release: ${{ (github.event_name == 'workflow_dispatch' && inputs.create_release) || github.event_name == 'push'}}
 
+  check-code-quality:
+    uses: "./.github/workflows/check-code-quality.yml"
+
+  reuse-compliance:
+    uses: "./.github/workflows/reuse-compliance.yml"
+
+  ci-passed:
+    runs-on: ubuntu-latest
+    needs: [ci-started, build-test-release, check-code-quality, reuse-compliance]
+
+    steps:
+      - run: echo "ci passed"
+      
   publish:
     name: Publish to PyPI
     runs-on: ubuntu-latest
     permissions:
       contents: write
       id-token: write  # Required for Trusted Publishing
-    needs: github-release
-    if: (github.event_name == 'workflow_dispatch') || github.event_name == 'push'
+    needs: build-test-release
+    if: (github.event_name == 'workflow_dispatch' && inputs.create_release) || github.event_name == 'push'
path: .github/workflows/ci.yml
user: login: figueroa1395
body: This line is different from PGM. 
```suggestion
    if: (github.event_name == 'workflow_dispatch' || github.event_name == 'push'
```
The suggestion corresponds to the PGM version. This allows to test in Test PyPI without having to create an extra release, but potentially can lead to publishing something unintentionally to the main PyPI. I perhaps recall a conversation about this with @mgovers, any thoughts?

Both are correct, it is just a matter of how we want to proceed with this one. But I am inclined to say that we keep this one. Thoughts @nitbharambe ?
created_at: 2025-05-28T15:23:55Z
author_association: MEMBER
diff_hunk: @@ -46,14 +43,27 @@ jobs:
       # otherwise create_release becomes false
       create_release: ${{ (github.event_name == 'workflow_dispatch' && inputs.create_release) || github.event_name == 'push'}}
 
+  check-code-quality:
+    uses: "./.github/workflows/check-code-quality.yml"
+
+  reuse-compliance:
+    uses: "./.github/workflows/reuse-compliance.yml"
+
+  ci-passed:
+    runs-on: ubuntu-latest
+    needs: [ci-started, build-test-release, check-code-quality, reuse-compliance]
+
+    steps:
+      - run: echo "ci passed"
+      
   publish:
     name: Publish to PyPI
     runs-on: ubuntu-latest
     permissions:
       contents: write
       id-token: write  # Required for Trusted Publishing
-    needs: github-release
-    if: (github.event_name == 'workflow_dispatch') || github.event_name == 'push'
+    needs: build-test-release
+    if: (github.event_name == 'workflow_dispatch' && inputs.create_release) || github.event_name == 'push'
path: .github/workflows/ci.yml
user: login: figueroa1395
body: After a couple of minutes of thought, perhaps the best would be to add another input which is separate from the one to create a pre-release. One that is only for publishing purposes and that is always true when pushing to main. That way it is always intended when we publish something via manual trigger and doesn't necessarily involve creating a pre-release. Conversely, creating a pre-release shouldn't involve publishing something.

However, it is probably out of scope of this PR and something to add in a separate PR, if agreed upon of course.
created_at: 2025-05-28T15:29:02Z
author_association: MEMBER
diff_hunk: @@ -80,7 +90,7 @@ jobs:
         uses: pypa/gh-action-pypi-publish@release/v1
         with:
           # To test, use the TestPyPI:
-          # repository-url: https://test.pypi.org/legacy/
+          repository-url: https://test.pypi.org/legacy/
path: .github/workflows/ci.yml
user: login: mgovers
body: ```suggestion
          # repository-url: https://test.pypi.org/legacy/
```
created_at: 2025-05-30T06:20:01Z
author_association: MEMBER
diff_hunk: @@ -46,14 +43,27 @@ jobs:
       # otherwise create_release becomes false
       create_release: ${{ (github.event_name == 'workflow_dispatch' && inputs.create_release) || github.event_name == 'push'}}
 
+  check-code-quality:
+    uses: "./.github/workflows/check-code-quality.yml"
+
+  reuse-compliance:
+    uses: "./.github/workflows/reuse-compliance.yml"
+
+  ci-passed:
+    runs-on: ubuntu-latest
+    needs: [ci-started, build-test-release, check-code-quality, reuse-compliance]
+
+    steps:
+      - run: echo "ci passed"
+      
   publish:
     name: Publish to PyPI
     runs-on: ubuntu-latest
     permissions:
       contents: write
       id-token: write  # Required for Trusted Publishing
-    needs: github-release
-    if: (github.event_name == 'workflow_dispatch') || github.event_name == 'push'
+    needs: build-test-release
+    if: (github.event_name == 'workflow_dispatch' && inputs.create_release) || github.event_name == 'push'
path: .github/workflows/ci.yml
user: login: mgovers
body: indeed out of scope of this PR.

i also think that it would be good to add publishing to test.pypi.org as an optional input parameter (i.e.: on `release`, select which pypi repo to release to)
created_at: 2025-05-30T06:21:02Z
author_association: MEMBER
diff_hunk: @@ -15,7 +15,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  main:
+  build-test-release:
path: .github/workflows/nightly.yml
user: login: mgovers
body: we should re-add building sonar in the nightly @nitbharambe , also in PGM
created_at: 2025-05-30T07:58:56Z
author_association: MEMBER

PR comments:
user: login: figueroa1395
created_at: 2025-05-28T05:08:31Z
body: Maybe a good idea is to first just fix the current issue, as it was initially done in [54e3dbd](https://github.com/PowerGridModel/power-grid-model-io/pull/307/commits/54e3dbd00209474b6eee9698c121047281e1edc0), and make sure that works. Then do the homogenization in a separate PR. Even though it is a "simple" PR, it is a dangerous one, and if we break something during homogenization, then it may be harder to track.

In addition, if you want to test things before merging, the Test PyPI repo for pgm-io is still alive: https://test.pypi.org/project/power-grid-model-io/. Just follow the comments in the publish job to see how to use it.
author_association: MEMBER
user: login: nitbharambe
created_at: 2025-05-28T08:40:45Z
body: > Maybe a good idea is to first just fix the current issue, as it was initially done in [54e3dbd](https://github.com/PowerGridModel/power-grid-model-io/pull/307/commits/54e3dbd00209474b6eee9698c121047281e1edc0), and make sure that works. Then do the homogenization in a separate PR. Even though it is a "simple" PR, it is a dangerous one, and if we break something during homogenization, then it may be harder to track.
> 
> In addition, if you want to test things before merging, the Test PyPI repo for pgm-io is still alive: https://test.pypi.org/project/power-grid-model-io/. Just follow the comments in the publish job to see how to use it.

That was the original intention. However the fix requires homogenization (or some more steps more than renaming). Hence the upcoming commits include homogenization too.
author_association: MEMBER
user: login: figueroa1395
created_at: 2025-05-28T15:06:05Z
body: Publishing has now been tested and works as expected: https://test.pypi.org/project/power-grid-model-io/#history.

Manual CI trigger (successful): https://github.com/PowerGridModel/power-grid-model-io/actions/runs/15303577121

One more TODO, is to clean up the "testing" pre-releases that were generated to keep it clean. This was also done before.
author_association: MEMBER
user: login: sonarqubecloud[bot]
created_at: 2025-05-30T06:23:56Z
body: ## [![Quality Gate Passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-passed-20px.png 'Quality Gate Passed')](https://sonarcloud.io/dashboard?id=PowerGridModel_power-grid-model-io&pullRequest=307) **Quality Gate passed**  
Issues  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 New issues](https://sonarcloud.io/project/issues?id=PowerGridModel_power-grid-model-io&pullRequest=307&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/accepted-16px.png '') [0 Accepted issues](https://sonarcloud.io/project/issues?id=PowerGridModel_power-grid-model-io&pullRequest=307&issueStatuses=ACCEPTED)

Measures  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=PowerGridModel_power-grid-model-io&pullRequest=307&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=PowerGridModel_power-grid-model-io&pullRequest=307&metric=new_coverage&view=list)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=PowerGridModel_power-grid-model-io&pullRequest=307&metric=new_duplicated_lines_density&view=list)  
  
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=PowerGridModel_power-grid-model-io&pullRequest=307)


author_association: NONE