Repository: Clinical-Genomics/BALSAMIC

Issue:
Verification Report [v16.0.1]
# Balsamic Verification Report

## Release Information

- Version: `16.0.1`
- Release Type: `Patch`
- Release Commit: https://github.com/Clinical-Genomics/BALSAMIC/commit/bac844057beb341eebdf88260fb36e59999faa61
- Release Date: 2025-02-10

## Person(s) involved in the development of new release
- Developer/s: MJ
- Verification operator/s: MJ
- Verification reviewer/s: KSV

## Summary

<!--
Provide a brief summary of the verification performed for the release, including the objectives and scope.
-->

This bug report https://github.com/Clinical-Genomics/BALSAMIC/issues/1531 describes an issue with storing balsamic cases in production because a report file can no longer be generated by snakemake. This file is not needed and can be removed.

It requires changes in hermes: https://github.com/Clinical-Genomics/hermes/pull/133 and balsamic: https://github.com/Clinical-Genomics/BALSAMIC/pull/1532

These are the goals of this verification:
1. Verify that integrity of the workflows are intact
2. That previously failing cases can be stored

## Changelog

<!--
Provide a summary of the changes and updates included in this minor/patch release.
-->

| Type    | Pull Request | Performed tests              |
|---------|--------------|------------------------------|
| Removed   |  https://github.com/Clinical-Genomics/BALSAMIC/pull/1532     | Tested that case that failed to be stored in production can be stored |


## Pre-Verification Checklist

Before proceeding with the verification process, ensure that the following tasks have been completed:

- [x] [Install Balsamic in stage and production environments in hasta and build its cache](https://atlas.scilifelab.se/infrastructure/BALSAMIC/balsamic/#instructions-for-installation). (**NOTE: Cache was instead copied from 16.0.0 to 16.0.1 and all paths in config-files were changed from 16.0.0 to 16.0.1**)

- [x] Confirm the availability of necessary resources, such as test cases.
- [x] Review the changelog and ensure all changes and updates are documented:

    | Document               | Sections to Be Updated | Pull Request |
    |------------------------|------------------------|--------------|
    | Balsamic Documentation | N/A   |  N/A       |
    | Atlas Documentation    | N/A        | N/A        |

- [x] Set up the stage environment with the necessary software and configurations:

    | Software | Current Version | Pull Request with the Required Updates |
    |----------|-----------------|----------------------------------------|
    | Hermes   | Version 5.1.3   | https://github.com/Clinical-Genomics/hermes/pull/133                               |
    | Servers   | Version 19.19.5   | https://github.com/Clinical-Genomics/servers/pull/1574                              |

## Verification Results

<!--
List the specific test cases that were executed during the verification process. Include the test case ID,
description, and status (Pass/Fail).
-->

### Workflow Integrity Verification Cases

| Case ID        | Analysis type      | Expected QC                                           | Status    |
|----------------|--------------------|-------------------------------------------------------|-----------|
| `setamoeba`    | TGA, tumor-only    | Pass                                                  | Pass|

- [x] This section has been verified successfully

### Storage, Delivery and Upload Integrity Verifications

<!--
Table of potentially affected steps in storage, clinical delivery, and upload of files.
-->

| Processes                           | Affected in current version | Affected workflows     | 
|-------------------------------------|-----------------------------|------------------------|
| New files to be stored              | No                      | ALL / TGA / WGS / UMI  |
| New files to be delivered           | No                      | ALL / TGA / WGS / UMI  | 
| New files to be uploaded to Scout   | No                      | ALL / TGA / WGS / UMI  | 
| Changes to Housekeeper IDs          | No                      | ALL / TGA / WGS / UMI  | 
| Changes to Scout upload             | No                      | ALL / TGA / WGS / UMI  |

<!--
For each affected workflow above, verify that all required files are stored, delivered and uploaded.
If none are affected, this step can be ignored. 
-->

| Case ID        | Analysis type       | Storage status | Delivery status | Upload  status | 
|----------------|---------------------|----------------|----------------|----------------|
| `acetuna`      | QC, tumor-only      | NA   | NA   | NA   | 
| `civilsole`    | WGS, tumor-only     | NA   | NA   | NA  |
| `fleetjay`     | WGS, tumor-normal   | NA   | NA   | NA  | 
| `setamoeba`    | TGA, tumor-only     | NA   | NA   | NA  | 
| `unitedbeagle` | TGA, tumor-normal   | NA   | NA   | NA  | 
| `uphippo`      | UMI, tumor-only     | NA   | NA   | NA   | 
| `equalbug`     | UMI, tumor-normal   | NA   | NA  | NA   | 

- [x] This section has been verified successfully, or been identified as irrelevant for the current verification

### Version Specific Verification Cases

See tests in Hermes PR: https://github.com/Clinical-Genomics/hermes/pull/133

- [x] WGS TN case that failed to store in production was copied over to stage, where the hermes PR and balsamic PR was deployed, the case could then be stored in housekeeper.


- [x] This section has been verified successfully, or been identified as irrelevant for the current verification

### Test cases can be stored in housekeeper after removing snakemake report creation

<!--
Provide detailed results for the specific test, including observations, passing criteria, and any relevant metrics.
-->

| Case ID  | Description   | Expected outcome | Status    |
|----------|---------------|------------------|-----------|
| `apparentarachnid`  | Case failed to store in production due to failing to create snakemake report  | After applying the hermes and balsamic PR in stage, the case should be able to be stored       | Pass |
| `bettergorilla`  | Case had successfully delivered in production before the error appeared  | After applying the hermes and balsamic PR in stage, the case should still be able to be stored       | Pass |

### Issues Identified

<!--
Document any issues or defects identified during the verification process. Include the issue ID, description, severity, 
and status (Open/Closed).
-->

N/A 

### Recommendations

<!--
Provide any recommendations or suggestions for improvement based on the verification results and observations.
-->

This patch is only affecting the storing of cases and does not have any impact on the analysis. Because of this only this limited testing is sufficient to ensure that the code is working, and as the tests have succeeded I recommend that the release be put into production //MJ 2025-02-10


### Conclusion

<!--
Summarize the overall outcome of the verification for the minor or patch release. Include any significant findings, 
achievements, or areas requiring further attention.
-->

---

## Approval
- Verification review: KSV
- Date: 2025-02-10

## Deployment
- Responsible: MJ
- Date: 2025-02-10

Discussion:
@Karl-Svard: Looks good! Signed ✍️ @mathiasbio: Deployed in production:
- hermes https://github.com/Clinical-Genomics/hermes/pull/133 
- server pr1 https://github.com/Clinical-Genomics/servers/pull/1574
- server pr2 https://github.com/Clinical-Genomics/servers/pull/1575
- balsamic https://github.com/Clinical-Genomics/BALSAMIC/pull/1532

Solution patch:
diff --git a/BALSAMIC/commands/report/deliver.py b/BALSAMIC/commands/report/deliver.py
index 38f0ef12..904d7f64 100644
--- a/BALSAMIC/commands/report/deliver.py
+++ b/BALSAMIC/commands/report/deliver.py
@@ -1,22 +1,17 @@
 """Report deliver CLI command."""
 import logging
-import subprocess
-import sys
 from pathlib import Path
-from typing import Any, Dict, List, Optional
+from typing import Any, Dict, List
 
 import click
 import snakemake
 from BALSAMIC.constants.constants import FileType
 
 from BALSAMIC.commands.options import (
-    OPTION_DISABLE_VARIANT_CALLER,
     OPTION_RULES_TO_DELIVER,
     OPTION_SAMPLE_CONFIG,
 )
-from BALSAMIC.constants.analysis import RunMode
 from BALSAMIC.models.config import ConfigModel
-from BALSAMIC.models.snakemake import SnakemakeExecutable
 from BALSAMIC.utils.cli import (
     get_snakefile,
     convert_deliverables_tags,
@@ -31,13 +26,11 @@ LOG = logging.getLogger(__name__)
 @click.command(
     "deliver", short_help="Create a <case_id>.hk file with output analysis files"
 )
-@OPTION_DISABLE_VARIANT_CALLER
 @OPTION_RULES_TO_DELIVER
 @OPTION_SAMPLE_CONFIG
 @click.pass_context
 def deliver(
     context: click.Context,
-    disable_variant_caller: Optional[str],
     rules_to_deliver: List[str],
     sample_config: str,
 ):
@@ -48,33 +41,12 @@ def deliver(
     config_model: ConfigModel = ConfigModel(**config)
     output_dir: Path = Path(config_model.analysis.result, "delivery_report")
     output_dir.mkdir(exist_ok=True)
-    working_dir: Path = Path(
-        config_model.analysis.analysis_dir,
-        config_model.analysis.case_id,
-        "BALSAMIC_run",
-    )
-    html_report: Path = Path(output_dir, f"{config_model.analysis.case_id}_report.html")
+
     snakefile: Path = get_snakefile(
         analysis_type=config_model.analysis.analysis_type,
         analysis_workflow=config_model.analysis.analysis_workflow,
     )
 
-    LOG.info(f"Creating HTML report file: {html_report.as_posix()}")
-    snakemake_executable: SnakemakeExecutable = SnakemakeExecutable(
-        case_id=config_model.analysis.case_id,
-        config_path=sample_config,
-        disable_variant_caller=disable_variant_caller,
-        report_path=html_report,
-        run_analysis=True,
-        run_mode=RunMode.LOCAL,
-        snakefile=snakefile,
-        snakemake_options=["--quiet"],
-        working_dir=working_dir,
-    )
-    subprocess.check_output(
-        f"{sys.executable} -m {snakemake_executable.get_command()}".split(), shell=False
-    )
-
     LOG.info(f"Delivering analysis workflow: {config_model.analysis.analysis_workflow}")
     hk_file: Path = Path(output_dir, f"{config_model.analysis.case_id}.hk")
     delivery_ready_file: Path = Path(
@@ -92,17 +64,6 @@ def deliver(
         delivery_json=hk_deliverables, sample_config_dict=config
     )
 
-    # HTML analysis report
-    hk_deliverables.append(
-        {
-            "path": html_report.as_posix(),
-            "step": "balsamic_delivery",
-            "format": get_file_extension(html_report.as_posix()),
-            "tag": ["balsamic-report"],
-            "id": config_model.analysis.case_id,
-        }
-    )
-
     # Sample configuration file
     hk_deliverables.append(
         {
diff --git a/BALSAMIC/models/snakemake.py b/BALSAMIC/models/snakemake.py
index 979e0523..577e7387 100644
--- a/BALSAMIC/models/snakemake.py
+++ b/BALSAMIC/models/snakemake.py
@@ -41,7 +41,6 @@ class SnakemakeExecutable(BaseModel):
         profile (Optional[ClusterProfile])                           : Cluster profile to submit jobs.
         qos (Optional[QOS])                                          : QOS for sbatch jobs.
         quiet (Optional[bool])                                       : Quiet mode for snakemake.
-        report_path (Optional[Path])                                 : Snakemake generated report path.
         run_analysis (bool)                                          : Flag to run the actual analysis.
         run_mode (RunMode)                                           : Cluster run mode to execute analysis.
         script_dir (Optional[DirectoryPath])                         : Cluster profile scripts directory.
@@ -66,7 +65,6 @@ class SnakemakeExecutable(BaseModel):
     profile: Optional[ClusterProfile] = None
     qos: Optional[QOS] = None
     quiet: bool = False
-    report_path: Optional[Path] = None
     run_analysis: bool = False
     run_mode: RunMode
     script_dir: Optional[DirectoryPath] = None
@@ -113,12 +111,6 @@ class SnakemakeExecutable(BaseModel):
             return "--quiet"
         return ""
 
-    def get_report_path_option(self) -> str:
-        """Return string representation of the report_path option."""
-        if self.report_path:
-            return f"--report {self.report_path.as_posix()}"
-        return ""
-
     def get_run_analysis_flag(self) -> str:
         """Return string representation of the run_analysis flag."""
         if not self.run_analysis:
@@ -154,7 +146,6 @@ class SnakemakeExecutable(BaseModel):
             f"{self.get_force_flag()} "
             f"{self.get_run_analysis_flag()} "
             f"{self.get_snakemake_cluster_options()} "
-            f"{self.get_report_path_option()} "
             f"{self.get_config_options()} "
             f"{self.get_snakemake_options_command()}"
         )
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
index a79aa97f..c5fea554 100644
--- a/CHANGELOG.rst
+++ b/CHANGELOG.rst
@@ -1,3 +1,11 @@
+[16.0.1]
+-------
+
+Removed:
+^^^^^^^^
+* remove snakemake case_report.html report creation https://github.com/Clinical-Genomics/BALSAMIC/pull/1530
+
+
 [16.0.0]
 -------
 


Test patch:
diff --git a/tests/commands/report/test_deliver.py b/tests/commands/report/test_deliver.py
index 7e55385c..02d5a503 100644
--- a/tests/commands/report/test_deliver.py
+++ b/tests/commands/report/test_deliver.py
@@ -35,8 +35,6 @@ def test_deliver_tumor_only_panel(
                 "deliver",
                 "--sample-config",
                 tumor_only_config,
-                "--disable-variant-caller",
-                "cnvkit",
             ]
         )
 
diff --git a/tests/conftest.py b/tests/conftest.py
index 0f60f36f..b2aef745 100644
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -2469,7 +2469,6 @@ def fixture_snakemake_executable_validated_data(
         "profile": ClusterProfile.SLURM,
         "qos": QOS.HIGH,
         "quiet": True,
-        "report_path": None,
         "run_analysis": True,
         "run_mode": RunMode.CLUSTER,
         "script_dir": session_tmp_path,
diff --git a/tests/models/test_snakemake_models.py b/tests/models/test_snakemake_models.py
index 373763bb..e85dd669 100644
--- a/tests/models/test_snakemake_models.py
+++ b/tests/models/test_snakemake_models.py
@@ -122,22 +122,6 @@ def test_quiet_flag(snakemake_executable: SnakemakeExecutable):
     assert quiet_flag == "--quiet"
 
 
-def test_get_report_path_option(
-    session_tmp_path: Path, snakemake_executable: SnakemakeExecutable
-):
-    """Test formatting of the report path option."""
-
-    # GIVEN a snakemake executable model with a report path option
-    snakemake_model: SnakemakeExecutable = copy.deepcopy(snakemake_executable)
-    snakemake_model.report_path = session_tmp_path
-
-    # WHEN calling the method
-    report_path_option: str = snakemake_model.get_report_path_option()
-
-    # THEN the expected format should be returned
-    assert report_path_option == f"--report {session_tmp_path.as_posix()}"
-
-
 def test_get_run_analysis_flag(snakemake_executable: SnakemakeExecutable):
     """Test formatting of the run analysis flag."""