Repository: eastgenomics/eggd_generate_fusion_workbook
Pull request #1: Fusion Workbook
Author: login: pofuoku
Created: 2025-03-07T17:15:04Z | Merged: 2025-03-21T16:39:38Z

- Initial commit
- App working with dummy inputs and WB inlcude just a sheet
- https://platform.dnanexus.com/panx/projects/Gz4kFpQ48F3YXzV94xXQ220Q/monitor/job/Gz5XG3848F3XXgzkY66ZF3f0

TODO
- add other sheets
- add pivots
- improve dxapp.json
- add docstrings
- change outputfile to project name
- update readme
@Aisha-D

<!-- Reviewable:start -->
- - -
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1)
<!-- Reviewable:end -->


<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **New Features**
  - Launched a new fusion workbook generator that processes multiple genomic data inputs to produce a comprehensive multi-sheet Excel report.

- **Documentation**
  - Revised the app’s title and description to reflect its new focus on fusion workbooks.

- **Chores**
  - Introduced improved configuration settings to ensure consistency and enhance overall performance.
  - Added a new configuration file for Visual Studio Code settings.

- **New Configuration**
  - Added a new JSON configuration file for the `eggd_generate_fusion_workbook` application.

- **Utilities**
  - Introduced new utility functions for reading and writing data, as well as parsing fusion data files.
  - Added default configurations for various Excel sheets and pivot tables.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->


Diff:
diff --git a/.vscode/settings.json b/.vscode/settings.json
new file mode 100644
index 0000000..8cb5621
--- /dev/null
+++ b/.vscode/settings.json
@@ -0,0 +1,4 @@
+{
+    "autoDocstring.docstringFormat": "numpy",
+    "editor.formatOnType": true
+}
\ No newline at end of file
diff --git a/README.md b/README.md
index 586b97c..63cfb74 100644
--- a/README.md
+++ b/README.md
@@ -1,2 +1,24 @@
-# DNAnexus_app_template
-This could be used as a starting point when developing new apps for DNAnexus
+# eggd_generate_fusion_workbook (DNAnexus Platform App)
+
+## What does this app do?
+Collates Fusion candidates and associated metadata from STAR-Fusion, FusionInspector and FastQC per run, then formats and generate a fusion workbook with collated data
+
+## What inputs are required for this app to run?
+- `-istarfusion_files` : An array of DNAnexus file IDs of STAR-Fusion's prediction files
+- `-ifusioninspector_files` : An array of DNAnexus file IDs of FusionInspector's output files
+- `-ifastqc_data`: The DNAnexus file ID of FASTQC metrics file
+- `-iSF_previous_runs_data` : The DNAnexus file ID of a static file containing historical STAR-Fusion data
+- `-iFI_previous_runs_data` : The DNAnexus file ID of a static file containing historical FusionInspector data
+
+## How does this app work?
+- Parses all inputs and format data according to structure of source data
+- Writes formated data into a worksheet for each data source
+- Writes a summary sheet using selected data from the different sources
+
+## What does this app output?
+- A Fusion Workbook named as `{project_name}_fusion_workbook.xlsx`
+
+#### TODO
+- Add unit tests
+- optimise reading of large dx files
+- add more validations to parser module
\ No newline at end of file
diff --git a/dxapp.json b/dxapp.json
new file mode 100644
index 0000000..4e3635a
--- /dev/null
+++ b/dxapp.json
@@ -0,0 +1,80 @@
+{
+    "name": "eggd_generate_fusion_workbook",
+    "title": "eggd_generate_fusion_workbook",
+    "summary": "App for fusion workbook",
+    "dxapi": "1.0.0",
+    "version": "1.0.0",
+    "inputSpec": [
+      {
+        "name": "starfusion_files",
+        "label": "starfusion_files",
+        "class": "array:file",
+        "optional": false,
+        "help": "STAR-Fusion output files containing fusion gene candidates and their supporting evidence"
+      },
+      {
+        "name": "fusioninspector_files",
+        "label": "fusioninspector_files",
+        "class": "array:file",
+        "optional": false,
+        "help": "FusionInspector output files for validation of fusions"
+      },
+      {
+        "name": "fastqc_data",
+        "label": "fastqc_data",
+        "class": "file",
+        "optional": false,
+        "help": "FastQC output file containing quality metrics for the sequencing data"
+      },
+      {
+        "name": "SF_previous_runs_data",
+        "class": "file",
+        "optional": false,
+        "help": "File containing info of STAR-Fusion's predictons from previous runs"
+      },
+      {
+        "name": "FI_previous_runs_data",
+        "class": "file",
+        "optional": false,
+        "help": "File containing info of FusionInspector's predictons from previous runs"
+      }
+    ],
+    "outputSpec": [
+      {
+        "name": "fusion_workbook",
+        "label": "fusion_workbook",
+        "class": "file",
+        "patterns": [
+          "*.xlsx"
+        ],
+        "help": "Workbook containing fusion genes and associated QC and metadata"
+      }
+    ],
+    "runSpec": {
+      "timeoutPolicy": {
+        "*": {
+          "hours": 2
+        }
+      },
+      "interpreter": "python3",
+      "file": "resources/home/dnanexus/generate_fusion_workbook.py",
+      "distribution": "Ubuntu",
+      "release": "24.04",
+      "version": "0"
+    },
+    "access": {
+      "project": "CONTRIBUTE"
+    },
+    "regionalOptions": {
+      "aws:eu-central-1": {
+        "systemRequirements": {
+          "*": {
+            "instanceType": "mem1_ssd1_v2_x4"
+          }
+        }
+      }
+    },
+    "developers": ["org-emee_1"],
+    "authorizedUsers": ["org-emee_1"]
+  }
+  
\ No newline at end of file
diff --git a/resources/home/dnanexus/generate_fusion_workbook.py b/resources/home/dnanexus/generate_fusion_workbook.py
new file mode 100644
index 0000000..4db04f8
--- /dev/null
+++ b/resources/home/dnanexus/generate_fusion_workbook.py
@@ -0,0 +1,141 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+import os
+from glob import glob
+import subprocess
+from typing import List
+
+if os.path.exists("/home/dnanexus"):
+    # running in DNAnexus
+    subprocess.check_call(
+        ["pip", "install", "--no-index", "--no-deps"] + glob("packages/*")
+    )
+
+import dxpy
+import openpyxl
+import pandas as pd
+from utils.defaults import (
+    EPIC_SHEET_CONFIG,
+    FASTQC_SHEET_CONFIG,
+    FI_PREVIOUS_RUNS_SHEET_CONFIG,
+    FI_SHEET_CONFIG,
+    SF_PREVIOUS_RUNS_SHEET_CONFIG,
+    SF_SHEET_CONFIG,
+    FASTQC_PIVOT_CONFIG,
+    SF_PIVOT_CONFIG,
+)
+from utils.parser import (
+    parse_fastqc,
+    parse_fusion_inspector,
+    parse_star_fusion,
+    make_fastqc_pivot,
+    make_sf_pivot,
+)
+from utils.utils import (
+    get_project_info,
+    read_dxfile,
+    write_df_to_sheet,
+    highlight_max_ffpm,
+)
+
+
+@dxpy.entry_point("main")
+def main(
+    starfusion_files: List[dict],
+    fusioninspector_files: List[dict],
+    fastqc_data: dict,
+    SF_previous_runs_data: dict,
+    FI_previous_runs_data: dict,
+):
+    """Generates a fusion workbook with data from
+    STAR-Fusion, FusionInspector, and FastQC
+
+    Parameters
+    ----------
+    starfusion_files : List[dict]
+        List of dictionaries containing DXLinks to STAR-Fusion output files
+    fusioninspector_files : List[dict]
+        List of dictionaries containing DXLinks to FusionInspector output files
+    fastqc_data : dict
+        Mapping of DXLink to FastQC data file
+    SF_previous_runs_data : str
+       Mapping of DXLink to STAR-Fusion previous runs data file
+    FI_previous_runs_data : str
+        Mapping of DXLink to FusionInspector previous runs data file
+
+    Returns
+    -------
+    dict
+        Dictionary containing DXLink to the generated fusion workbook
+    """
+    # Initialize inputs into dxpy.DXDataObject instances
+    starfusion_files = [dxpy.DXFile(item) for item in starfusion_files]
+    fusioninspector_files = [dxpy.DXFile(item) for item in fusioninspector_files]
+    fastqc_data = dxpy.DXFile(fastqc_data)
+    sf_previous_data = dxpy.DXFile(SF_previous_runs_data)
+    fi_previous_data = dxpy.DXFile(FI_previous_runs_data)
+
+    df_starfusion = parse_star_fusion(starfusion_files)
+    df_fusioninspector = parse_fusion_inspector(fusioninspector_files)
+    df_fastqc = parse_fastqc(fastqc_data)
+
+    df_sf_previous = read_dxfile(sf_previous_data, include_fname=False)
+    df_fi_previous = read_dxfile(fi_previous_data, sep=",", include_fname=False)
+
+    project_name, _ = get_project_info()
+    outfile = f"{project_name}_fusion_workbook.xlsx"
+
+    with pd.ExcelWriter(outfile, engine="openpyxl") as writer:
+
+        # Writes empty EPIC sheet
+        write_df_to_sheet(writer, pd.DataFrame(), **EPIC_SHEET_CONFIG)
+
+        # Add SF previous runs data
+        write_df_to_sheet(writer, df_sf_previous, **SF_PREVIOUS_RUNS_SHEET_CONFIG)
+
+        # Add FI previous runs data
+        write_df_to_sheet(writer, df_fi_previous, **FI_PREVIOUS_RUNS_SHEET_CONFIG)
+
+        # Add FastQC data
+        write_df_to_sheet(writer, df_fastqc, **FASTQC_SHEET_CONFIG)
+        fastqc_pivot = make_fastqc_pivot(df_fastqc, FASTQC_PIVOT_CONFIG)
+        write_df_to_sheet(
+            writer,
+            fastqc_pivot,
+            sheet_name=FASTQC_PIVOT_CONFIG["sheet_name"],
+            tab_color=FASTQC_PIVOT_CONFIG["tab_color"],
+        )
+
+        # Add Fusion Inspector data
+        write_df_to_sheet(writer, df_fusioninspector, **FI_SHEET_CONFIG)
+
+        # Add STAR-Fusion data
+        write_df_to_sheet(writer, df_starfusion, **SF_SHEET_CONFIG)
+        sf_pivot = make_sf_pivot(
+            df_starfusion,
+            df_sf_previous,
+            fastqc_pivot,
+            df_fusioninspector,
+            SF_PIVOT_CONFIG,
+        )
+        write_df_to_sheet(
+            writer,
+            sf_pivot,
+            sheet_name=SF_PIVOT_CONFIG["sheet_name"],
+            tab_color=SF_PIVOT_CONFIG["tab_color"],
+            include_index=True,
+        )
+        summary_sheet = writer.sheets[SF_PIVOT_CONFIG["sheet_name"]]
+        highlight_max_ffpm(summary_sheet, sf_pivot)
+
+    fusion_workbook = dxpy.upload_local_file(outfile)
+
+    output = {}
+    output["fusion_workbook"] = dxpy.dxlink(fusion_workbook)
+
+    return output
+
+
+if __name__ == "__main__":
+    dxpy.run()
diff --git a/resources/home/dnanexus/packages/et_xmlfile-2.0.0-py3-none-any.whl b/resources/home/dnanexus/packages/et_xmlfile-2.0.0-py3-none-any.whl
new file mode 100644
index 0000000..52a7eec
Binary files /dev/null and b/resources/home/dnanexus/packages/et_xmlfile-2.0.0-py3-none-any.whl differ
diff --git a/resources/home/dnanexus/packages/numpy-2.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl b/resources/home/dnanexus/packages/numpy-2.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
new file mode 100644
index 0000000..2d754c6
Binary files /dev/null and b/resources/home/dnanexus/packages/numpy-2.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl differ
diff --git a/resources/home/dnanexus/packages/openpyxl-3.1.5-py2.py3-none-any.whl b/resources/home/dnanexus/packages/openpyxl-3.1.5-py2.py3-none-any.whl
new file mode 100644
index 0000000..e1dccd4
Binary files /dev/null and b/resources/home/dnanexus/packages/openpyxl-3.1.5-py2.py3-none-any.whl differ
diff --git a/resources/home/dnanexus/packages/pandas-2.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl b/resources/home/dnanexus/packages/pandas-2.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
new file mode 100644
index 0000000..2345a50
Binary files /dev/null and b/resources/home/dnanexus/packages/pandas-2.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl differ
diff --git a/resources/home/dnanexus/utils/__init__.py b/resources/home/dnanexus/utils/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/resources/home/dnanexus/utils/defaults.py b/resources/home/dnanexus/utils/defaults.py
new file mode 100644
index 0000000..d048c97
--- /dev/null
+++ b/resources/home/dnanexus/utils/defaults.py
@@ -0,0 +1,101 @@
+"""
+Module containing default constants for sheet configurations
+Formulas adapted from demo WB provided by scientist
+"""
+
+FASTQC_SHEET_CONFIG = {
+    "sheet_name": "FastQC",
+    "tab_color": "008000",
+    "extra_cols": {
+        "SPECIMEN": "=MID(C{row},11,10)",
+        "EPIC": "=VLOOKUP(A{row},'EPIC'!AJ:AK,2,0)",
+    },
+}
+
+SF_SHEET_CONFIG = {
+    "sheet_name": "STAR-Fusion",
+    "tab_color": "800080",
+    "extra_cols": {
+        "SPECIMEN": "=MID(K{row},11,10)",
+        "FNAME": "=LEFT(K{row},28)",
+        "Count_predicted": "=VLOOKUP(L{row},'SF_Previous_Runs'!E:F,2,0)",
+        "EPIC": "=VLOOKUP(A{row},'EPIC'!AJ:AK,2,0)",
+        "DAYS COUNT": "=VLOOKUP(A{row},'EPIC'!AJ:AL,3,0)",
+        "Unique Reads(M)": "=VLOOKUP(A{row},'FastQC_Pivot'!A:B,3,0)",
+        "Duplicate Reads(M)": "=VLOOKUP(A{row},'FastQC_Pivot'!A:C,2,0)",
+        "ID": '=CONCATENATE(A{row},"_",L{row})',
+        "LEFTRIGHT": '=CONCATENATE(S{row},"_",U{row})',
+        "FRAME": "=VLOOKUP(I{row},'Fusion_Inspector'!C:AM,32,0)",
+    },
+}
+
+FI_SHEET_CONFIG = {
+    "sheet_name": "Fusion_Inspector",
+    "tab_color": "A52A2A",
+    "extra_cols": {
+        "SPECIMEN": "=MID(D{row},11,10)",
+        "ID": '=CONCATENATE(A{row},"_",E{row})',
+        "LEFTRIGHT": '=CONCATENATE(L{row},"_",O{row})',
+    },
+}
+
+SF_PREVIOUS_RUNS_SHEET_CONFIG = {
+    "sheet_name": "SF_Previous_Runs",
+    "tab_color": "000000",
+    "extra_cols": None,
+}
+
+FI_PREVIOUS_RUNS_SHEET_CONFIG = {
+    "sheet_name": "FI_Previous_Runs",
+    "tab_color": "000000",
+    "extra_cols": None,
+}
+
+EPIC_SHEET_CONFIG = {
+    "sheet_name": "EPIC",
+    "tab_color": "0000FF",
+    "extra_cols": None,
+}
+
+
+FASTQC_PIVOT_CONFIG = {
+    "index": ["SPECIMEN"],
+    "columns": None,
+    "values": [
+        "Duplicate Reads(M)",
+        "Unique Reads(M)",
+    ],
+    "aggfunc": "sum",
+    "sheet_name": "FastQC_Pivot",
+    "tab_color": "00FF00",
+    "add_total_row": True,
+}
+
+SF_PIVOT_CONFIG = {
+    "index": [
+        "Filename",
+        "SPECIMEN",
+        "Unique Reads(M)",
+        "Duplicate Reads(M)",
+        "#FusionName",
+    ],
+    "columns": None,
+    "values": [
+        "FFPM",
+        "FRAME",
+        "Count_predicted",
+        "SpanningFragCount",
+        "JunctionReadCount",
+        "LEFTRIGHT",
+    ],
+    "aggfunc": {
+        "LEFTRIGHT": "first",
+        "FRAME": "first",
+        "FFPM": "first",
+        "Count_predicted": "first",
+        "SpanningFragCount": "first",
+        "JunctionReadCount": "first",
+    },
+    "sheet_name": "Summary",
+    "tab_color": "9400D3",
+}
diff --git a/resources/home/dnanexus/utils/parser.py b/resources/home/dnanexus/utils/parser.py
new file mode 100644
index 0000000..5210c02
--- /dev/null
+++ b/resources/home/dnanexus/utils/parser.py
@@ -0,0 +1,267 @@
+"""Module for parsing fusion data from their respective DNAnexus files into a
+pandas data frame
+"""
+
+from concurrent.futures import ThreadPoolExecutor
+from typing import List
+
+import dxpy
+import pandas as pd
+from dxpy import DXDataObject
+
+from .utils import create_pivot_table, read_dxfile
+
+
+def parse_specimen_id(sample:str) -> str:
+    """parse SP ID from sample name
+
+    Parameters
+    ----------
+    sample : str
+        sample name in format 12345678-2XXXXSXXX-25PCAN4-10011_S33_L001_R1
+
+    Returns
+    -------
+    str
+        extracted SP eg 2XXXXSXXX
+    """
+    return sample.split("-")[1]
+
+
+def parse_igv_specimen_name(sample:str) -> str:
+    """parse SP ID from sample name in IGV format
+
+    Parameters
+    ----------
+    sample : str
+        sample name in format 12345678-2XXXXSXXX-25PCAN4-10011_S33_L001_R1
+
+    Returns
+    -------
+    str
+        extracted SP eg 12345678-2XXXXSXXX-25PCAN4
+    """
+    return "-".join(sample.split("-")[:3])
+
+
+def parse_fastqc(dxfile: DXDataObject) -> pd.DataFrame:
+    """Parse the content of fastqc multiqc output and compute required
+    metrics from deduplicated_percentage
+
+    Parameters
+    ----------
+    dxfile : DXDataObject
+        DNAnexus file object containing fastqc data to parse
+
+    Returns
+    -------
+    pd.DataFrame
+        pandas dataframe containing selected and computed fastqc metrics
+
+    Raises
+    ------
+    ValueError
+        if columns needed to compute extra metrics are absent.
+    """
+
+    df = read_dxfile(dxfile, include_fname=False)
+
+    # Verify required columns exist
+    required_cols = ["total_deduplicated_percentage", "Total Sequences"]
+    missing_cols = [col for col in required_cols if col not in df.columns]
+    if missing_cols:
+        raise ValueError(f"Required columns missing from FastQC data: {missing_cols}")
+
+    df["Unique Reads"] = (
+        (df["total_deduplicated_percentage"] / 100.0) * df["Total Sequences"]
+    ).astype(int)
+    df["Duplicate Reads"] = (df["Total Sequences"] - df["Unique Reads"]).astype(int)
+    df["Unique Reads(M)"] = df["Unique Reads"] / 1_000_000
+    df["Duplicate Reads(M)"] = df["Duplicate Reads"] / 1_000_000
+    df = df[
+        [
+            "Sample",
+            "Unique Reads",
+            "Duplicate Reads",
+            "Unique Reads(M)",
+            "Duplicate Reads(M)",
+        ]
+    ]
+
+    return df
+
+
+def make_fastqc_pivot(df: pd.DataFrame, pivot_config: dict) -> pd.DataFrame:
+    """generates FastQC pivot table
+
+    Parameters
+    ----------
+    df : pd.DataFrame
+        Dataframe containing raw fastqc data
+    pivot_config : dict
+        Mapping defining pivot structure
+
+    Returns
+    -------
+    pd.DataFrame
+        created pivot table with computed columns
+    """
+    df["SPECIMEN"] = df["Sample"].apply(parse_specimen_id)
+
+    pivot_df = create_pivot_table(df, pivot_config)
+    pivot_df = pivot_df.reset_index(drop=False)
+
+    return pivot_df
+
+
+def _parse_fusion_files(dxfiles: List[DXDataObject]) -> pd.DataFrame:
+    """
+    Reads and concatenates a list of DNAnexus fusion-related files
+    into a single DataFrame.
+
+    Parameters
+    ----------
+    dxfiles : List[DXDataObject]
+        A list of DNAnexus file objects.
+
+    Returns
+    -------
+    pd.DataFrame
+        A concatenated DataFrame containing the combined data
+        from all input files.
+    """
+    if not dxfiles:
+        return pd.DataFrame()
+
+    max_workers = min(16, len(dxfiles))
+    with ThreadPoolExecutor(max_workers=max_workers) as executor:
+        futures = [executor.submit(read_dxfile, dxfile) for dxfile in dxfiles]
+        results = []
+        for future in futures:
+           try:
+                results.append(future.result())
+           except Exception as e:
+               # Log the error and continue with other files
+              print(f"Error processing file: {e}")
+        
+        if not results:
+           return pd.DataFrame()
+        
+        df = pd.concat(results)
+
+    return df
+
+
+def parse_fusion_inspector(dxfiles: List[DXDataObject]) -> pd.DataFrame:
+    """
+    Reads and concatenates an array of DNAnexus Fusion Inspector files.
+
+    Parameters
+    ----------
+    dxfiles : List[DXDataObject]
+        A list of DNAnexus file objects.
+
+    Returns
+    -------
+    pd.DataFrame
+        A concatenated DataFrame containing the combined data
+        from all input files.
+    """
+    # simply calls _parse_fusion_files for now
+    # can be easily extended if additional logic needed for tool
+    return _parse_fusion_files(dxfiles)
+
+
+def parse_star_fusion(dxfiles: List[DXDataObject]) -> pd.DataFrame:
+    """
+    Reads and concatenates an array of DNAnexus STAR-Fusion files.
+
+    Parameters
+    ----------
+    dxfiles : List[DXDataObject]
+        A list of DNAnexus file objects.
+
+    Returns
+    -------
+    pd.DataFrame
+        A concatenated DataFrame containing the combined data
+        from all input files.
+    """
+    # same as above; to allow further customisation per tool as needed.
+    return _parse_fusion_files(dxfiles)
+
+
+def make_sf_pivot(
+    sf_df: pd.DataFrame,
+    sf_runs_df: pd.DataFrame,
+    fastqc_pivot_df: pd.DataFrame,
+    fi_df: pd.DataFrame,
+    pivot_config: dict,
+) -> pd.DataFrame:
+    """Generates STAR-Fusion pivot table with merged data
+
+    Parameters
+    ----------
+    sf_df : pd.DataFrame
+        Current STAR-Fusion data
+    sf_runs_df : pd.DataFrame
+        Historical STAR-Fusion data
+    fastqc_pivot_df : pd.DataFrame
+        FastQC summary data
+    fi_df : pd.DataFrame
+        Current Fusion Inspector data
+    pivot_config : dict
+        Pivot configuration parameters
+
+    Returns
+    -------
+    pd.DataFrame
+        Created Pivot table with merged data
+    """
+
+    df = sf_df.copy()
+
+    if "file_name" in sf_df.columns:
+        df["SPECIMEN"] = df["file_name"].apply(parse_specimen_id)
+
+        df["Filename"] = df["file_name"].apply(parse_igv_specimen_name)
+
+    df["ID"] = df["SPECIMEN"] + "_" + df["#FusionName"]
+
+    df["LEFTRIGHT"] = df["LeftBreakpoint"] + "_" + df["RightBreakpoint"]
+
+    # Merge previous runs data (VLOOKUP equivalent)
+    if not sf_runs_df.empty:
+        df = df.merge(
+            sf_runs_df[["#FusionName", "Count_Run_1_Run_20_predicted"]],
+            on="#FusionName",
+            how="left",
+        ).rename(columns={"Count_Run_1_Run_20_predicted": "Count_predicted"})
+        df["Count_predicted"] = df["Count_predicted"].fillna(0).astype(int)
+
+    # Merge FastQC metrics (VLOOKUP to FastQC_pivot)
+    if not fastqc_pivot_df.empty:
+        df = df.merge(fastqc_pivot_df, on="SPECIMEN", how="left")
+
+    # Merge Fusion Inspector data
+    if not fi_df.empty:
+        fi_df["LEFTRIGHT"] = fi_df["LeftBreakpoint"] + "_" + fi_df["RightBreakpoint"]
+        df = df.merge(
+            fi_df[["LEFTRIGHT", "PROT_FUSION_TYPE"]], on="LEFTRIGHT", how="left"
+        ).rename(columns={"PROT_FUSION_TYPE": "FRAME"})
+
+    # Create final pivot table
+    df = df.sort_values(by=["SPECIMEN", "FFPM"]).reset_index(drop=True)
+    pivot_df = create_pivot_table(df, pivot_config)
+    pivot_df = pivot_df[
+        [
+            "LEFTRIGHT",
+            "JunctionReadCount",
+            "SpanningFragCount",
+            "Count_predicted",
+            "FRAME",
+            "FFPM",
+        ]
+    ]
+
+    return pivot_df
diff --git a/resources/home/dnanexus/utils/utils.py b/resources/home/dnanexus/utils/utils.py
new file mode 100644
index 0000000..842dfeb
--- /dev/null
+++ b/resources/home/dnanexus/utils/utils.py
@@ -0,0 +1,273 @@
+"""general and I/O utilities
+"""
+
+import os
+
+import dxpy
+import openpyxl
+import pandas as pd
+from dxpy import DXDataObject
+from openpyxl.styles import Font, PatternFill
+from openpyxl.utils.dataframe import dataframe_to_rows
+from openpyxl.worksheet.worksheet import Worksheet
+
+
+def read_dxfile(
+    dxfile: DXDataObject, sep: str = "\t", include_fname: bool = True
+) -> pd.DataFrame:
+    """reads a DNAnexus file object into a pandas dataframe
+
+    Parameters
+    ----------
+    dxfile : DXDataObject
+        An instance of DXDataObject; behaves like a python file object.
+    sep : str
+        Delimeter of data values in files. Defaults to  "\t"
+    include_fname : bool
+        Specifies whether to set file name as first column. Defaults to True
+
+    Returns
+    -------
+    pd.DataFrame
+        An instance of DataFrame with file content
+    """
+
+    df = pd.read_csv(dxfile, sep=sep)
+
+    if include_fname:
+        fname = dxfile.name
+        df.insert(0, "file_name", fname)
+
+    return df
+
+
+def _add_extra_columns(
+    worksheet: Worksheet, extra_cols: dict[str, str], start: int = 1
+) -> None:
+    """
+    Inserts additional columns with formulas at the beginning of a sheet
+
+    Parameters
+    ----------
+    worksheet : Worksheet
+        The worksheet where columns will be added.
+    extra_cols : dict[str, str]
+        A mapping of column names to Excel formulas
+    start : int, optional
+        The column index (1-based) where extra columns should be inserted.
+        Defaults to 1 (beginning of the sheet).
+
+    Returns
+    -------
+    None
+    """
+
+    num_cols = len(extra_cols)
+    worksheet.insert_cols(start, amount=num_cols)
+
+    for i, (col, formula) in enumerate(extra_cols.items(), start=start):
+        worksheet.cell(row=1, column=i, value=col)
+        for row in range(2, worksheet.max_row + 1):
+            worksheet.cell(row=row, column=i, value=formula.replace("{row}", str(row)))
+
+
+def _apply_header_format(worksheet) -> None:
+    """
+    Applies bold formatting to all headers in the sheet.
+
+    Parameters
+    ----------
+    worksheet : Worksheet
+        The worksheet where header formatting will be applied.
+
+    Returns
+    -------
+    None
+    """
+    header_font = Font(bold=True)
+    for col in range(1, worksheet.max_column + 1):
+        cell = worksheet.cell(row=1, column=col)
+        cell.font = header_font
+
+
+def _adjust_column_widths(worksheet, min_width=14, max_width=40):
+    """
+    Adjusts column widths for better visibility.
+
+    Parameters
+    ----------
+    worksheet : Worksheet
+        The worksheet where column widths will be adjusted.
+    min_width : int
+        The min width a column should have.
+    max_width : int
+        The max width a column should have.
+
+    Returns
+    -------
+    None
+    """
+    for col_cells in worksheet.columns:
+        col_letter = col_cells[0].column_letter
+        _max = min_width
+
+        for cell in col_cells:
+            val = cell.value
+            if val and not (isinstance(val, str) and val.startswith("=")):
+                _max = max(_max, len(str(val)))
+
+        # Set column width with a cap of max_width
+        worksheet.column_dimensions[col_letter].width = min(_max + 2, max_width)
+
+
+def _set_tab_color(worksheet, hex_color: str) -> None:
+    """Sets worksheet tab colour
+
+    Parameters
+    ----------
+    worksheet : Worksheet
+        The worksheet to apply tab colour
+    hex_color : str
+        Hex colour code for the sheet tab
+    """
+    worksheet.sheet_properties.tabColor = hex_color
+
+
+def highlight_max_ffpm(worksheet, source_df, ffpm_col="FFPM", index_col="SPECIMEN"):
+    """
+    Conditionally formats rows with maximum FFPM value per specimen(index).
+
+    Parameters
+    ----------
+    worksheet : Worksheet
+        The worksheet where conditional formatting will be applied
+    source_df : pandas.DataFrame
+        The source data written to sheet
+    ffpm_col : str, optional
+        Name of the FFPM column in the DataFrame, default is "FFPM"
+    index_col : str, optional
+        Name of the column to use for grouping ffpm, default is "SPECIMEN"
+
+    Returns
+    -------
+    None
+    """
+
+    # Create a light green fill pattern for highlighting
+    green_fill = PatternFill(
+        start_color="CCFFCC", end_color="CCFFCC", fill_type="solid"
+    )
+
+    # Reset index if it's a multi-index df to make processing easier
+    if isinstance(source_df.index, pd.MultiIndex):
+        df = source_df.reset_index()
+    else:
+        df = source_df.copy()
+
+    # Find the max FFPM per specimen
+    max_ffpm = df.groupby(index_col)[ffpm_col].transform("max")
+    max_rows = df[(df[ffpm_col] == max_ffpm)].index.tolist()
+
+    # Apply conditional formatting
+    for idx in max_rows:
+        row_idx = idx + 2  # adjust for 0-indexing in pandas
+        for col_idx in range(1, worksheet.max_column + 1):
+            cell = worksheet.cell(row=row_idx, column=col_idx)
+            cell.fill = green_fill
+
+
+def write_df_to_sheet(
+    writer: pd.ExcelWriter,
+    df: pd.DataFrame,
+    sheet_name: str,
+    tab_color: str = "000000",
+    extra_cols: dict[str, str] = None,
+    include_index: bool = False,
+) -> None:
+    """Writes a Pandas DataFrame to an Excel sheet with formatting.
+
+    Parameters
+    ----------
+    writer : pd.ExcelWriter
+        The Excel writer object to write the sheet into
+    df : pd.DataFrame
+        The DataFrame containing the data.
+    sheet_name : str
+        Name of the Excel sheet
+    tab_color : str, optional
+        Hex colour code for the sheet tab. Defaults to black ("000000")
+    extra_cols : dict[str, str], optional
+        A mapping of new column names to excel formulas. Example:
+            {
+                "Specimen": "=MID(C{row},11,10)",
+                "EPIC": "=VLOOKUP(A{row},EPIC!AJ:AK,2,0)"
+            }
+    include_index : bool, optional
+        Wether to write index of df to sheet. Defaults to False
+    """
+    df.to_excel(writer, sheet_name=sheet_name, index=include_index)
+    worksheet = writer.sheets[sheet_name]
+
+    _set_tab_color(worksheet, tab_color)
+
+    # Add extra columns if provided
+    if extra_cols:
+        _add_extra_columns(worksheet, extra_cols)
+
+    _apply_header_format(worksheet)
+    _adjust_column_widths(worksheet)
+
+
+def create_pivot_table(
+    df: pd.DataFrame,
+    pivot_config: dict,
+) -> pd.DataFrame:
+    """Creates pivot table from a dataframe.
+
+    Parameters
+    ----------
+    df : pd.DataFrame
+        Source dataframe to create pivot from
+    pivot_config : dict
+        Dictionary defining pivot structure:
+            - index: Columns to group by (list for MultiIndex)
+            - columns: Columns for column grouping
+            - values: Value columns to aggregate
+            - aggfunc: Aggregation function
+            - add_total_row: Whether to add totals (default False)
+
+    Returns
+    ------
+    pd.DataFrame
+        created pivot table
+    """
+
+    # Create pivot table
+    pivot_df = df.pivot_table(
+        index=pivot_config["index"],
+        columns=pivot_config["columns"],
+        values=pivot_config["values"],
+        aggfunc=pivot_config["aggfunc"],
+        margins=pivot_config.get("add_total_row", False),
+        margins_name="Total",
+    )
+
+    return pivot_df
+
+
+def get_project_info() -> tuple[str, str]:
+    """Get the project name for naming output file
+
+    Returns
+    -------
+    tuple[str, str]
+        Name and ID of DNAnexus project
+    """
+
+    project_id = os.environ.get("DX_PROJECT_CONTEXT_ID")
+
+    # Get name of project for output naming
+    project_name = dxpy.describe(project_id)["name"]
+    project_name = project_name.split("_", 1)[1]
+
+    return project_name, project_id


Reviews:
user: login: coderabbitai[bot]
body: **Actionable comments posted: 19**

<details>
<summary>🧹 Nitpick comments (18)</summary><blockquote>

<details>
<summary>resources/home/dnanexus/utils/fusion_inspector.py (3)</summary><blockquote>

`1-9`: **Remove unused import and improve module docstring**

The `dxpy` module is imported but not used directly in the code. Additionally, the module docstring would benefit from more specific information about what fusion inspector data actually is.


```diff
"""Module for parsing fusion inspector data from DNAnexus files and 
adding computed columns using formulas.
"""
from concurrent.futures import ThreadPoolExecutor
import pandas as pd
from dxpy import DXDataObject
-import dxpy
from typing import List

```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

7-7: `dxpy` imported but unused

Remove unused import: `dxpy`

(F401)

</details>

</details>

---

`13-24`: **Function docstring contains unnecessary blank lines**

The docstring has extra blank lines that should be removed for better consistency. Also, it would be helpful to briefly explain what fusion inspector data is for developers unfamiliar with the domain.


```diff
def parse_fusion_inspector(dxfiles: List[DXDataObject]) -> pd.DataFrame:
    """ Reads and concatenate an array DNAnexus fusion inspector files 
        into a single DataFrame.


    Args:
        dxfiles (List[DXDataObject]): A list of DNAnexus file objects

    Returns:
        pd.DataFrame:  A concatenated DataFrame containing the combined data 
            from all input files.
    """
```

---

`33-38`: **Document column formula dependencies and remove empty line**

The formulas in `FUSION_INSPECTOR_EXTRA_COLS` rely on specific column positions in the DataFrame which creates tight coupling. Document these dependencies to make future maintenance easier.


```diff
FUSION_INSPECTOR_EXTRA_COLS = {
+    # These formulas depend on specific column positions:
+    # Column D contains file identifiers
+    # Column A contains fusion IDs
+    # Columns L and O contain gene names
    "SPECIMEN": "=MID(D{row},11,10)",
    "ID": '=CONCAT(A{row},"_",E{row})',
    "LEFTRIGHT": '=CONCAT(L{row},"_",O{row})'
-
}
```

</blockquote></details>
<details>
<summary>resources/home/dnanexus/utils/star_fusion.py (4)</summary><blockquote>

`1-10`: **Remove unused import and improve module docstring**

The `dxpy` module is imported but not used directly in the code. Additionally, the module docstring could be more informative about what STAR-Fusion data actually is.


```diff
"""Module for parsing STAR-Fusion data from DNAnexus files.
"""
from concurrent.futures import ThreadPoolExecutor
import pandas as pd
from dxpy import DXDataObject
-import dxpy
from typing import List

from .utils import read_dxfile
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

6-6: `dxpy` imported but unused

Remove unused import: `dxpy`

(F401)

</details>

</details>

---

`12-23`: **Function docstring contains unnecessary blank lines**

The docstring has extra blank lines that should be removed for better consistency. Also, consider adding a brief explanation of what STAR-Fusion data is for developers unfamiliar with this domain.


```diff
def parse_star_fusion(dxfiles: List[DXDataObject]) -> pd.DataFrame:
    """ Reads and concatenate an array DNAnexus STAR-Fusion files 
        into a single DataFrame.


    Args:
        dxfiles (List[DXDataObject]): A list of DNAnexus file objects

    Returns:
        pd.DataFrame:  A concatenated DataFrame containing the combined data 
            from all input files.
    """
```

---

`32-44`: **Document column formula dependencies**

The formulas in `STAR_FUSION_EXTRA_COLS` rely on specific column positions and external sheet references. Document these dependencies to aid in future maintenance, especially since these formulas were "lifted from demo WB provided by scientist".


```diff
# formulas lifted from demo WB provided by scientist
STAR_FUSION_EXTRA_COLS = {
+    # These formulas depend on specific column positions and external sheets:
+    # Column K contains file identifiers
+    # Column A contains sample IDs
+    # Column L contains gene information
+    # References external sheets: PC1_PC20_predicted_S, EPIC, MultiQC_Pivot, PC4_MERGED
    "SPECIMEN": "=MID(K{row},11,10)",
    "FNAME": "=LEFT(K{row},28)",
    "Count_Run_1_Run_20_predicted": "=VLOOKUP(L{row},PC1_PC20_predicted_S!E:F,2,0)",
    "EPIC": "=VLOOKUP(A{row},EPIC!AJ:AK,2,0)",
    "DAYS COUNT": "=VLOOKUP(A{row},EPIC!AJ:AL,3,0)",
    "Unique Reads (M) PC3B": "=VLOOKUP(A{row},MultiQC_Pivot!A:C,3,0)",
    "Duplicate Reads (M) PC3B": "=VLOOKUP(A{row},MultiQC_Pivot!A:D,4,0)",
    "ID": '=CONCAT(A{row},"_",L{row})',
    "LEFTRIGHT": '=CONCATENATE(S{row},"_",U{row})',
    "FRAME": "=VLOOKUP(I{row},PC4_MERGED!C:AM,32,0)"
}
```

---

`12-44`: **Consider refactoring duplicate parsing pattern**

The core functionality in `parse_star_fusion` is nearly identical to `parse_fusion_inspector`. Consider refactoring to extract a common generic parsing function to reduce code duplication.


This could be implemented by creating a generic function in utils.py:

```python
def parse_dxfiles(dxfiles: List[DXDataObject], **kwargs) -> pd.DataFrame:
    """Generic function to read and concatenate multiple DNAnexus files.
    
    Args:
        dxfiles (List[DXDataObject]): A list of DNAnexus file objects
        **kwargs: Additional arguments to pass to read_dxfile
        
    Returns:
        pd.DataFrame: A concatenated DataFrame containing the combined data
            from all input files.
    """
    if not dxfiles:
        return pd.DataFrame()
        
    with ThreadPoolExecutor(max_workers=min(16, len(dxfiles))) as executor:
        df = pd.concat(
            list(executor.map(lambda f: read_dxfile(f, **kwargs), dxfiles)),
            ignore_index=True
        )
    return df
```

Then simplify both modules to use this common function.

</blockquote></details>
<details>
<summary>resources/home/dnanexus/utils/fastqc.py (3)</summary><blockquote>

`1-6`: **Remove unused import and improve module docstring**

The `dxpy` module is imported but not used directly in the code. Additionally, the module docstring is very brief and could be expanded to better explain what the module does.


```diff
-"""collates total_deduplicated_percentage from multiqc_fastqc.txt
+"""Module for parsing FastQC data from DNAnexus files.
+
+This module processes FastQC data to calculate unique and duplicate read counts
+from the total_deduplicated_percentage metric in multiqc_fastqc.txt files.
"""
import pandas as pd
from dxpy import DXDataObject
-import dxpy
from .utils import read_dxfile
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

5-5: `dxpy` imported but unused

Remove unused import: `dxpy`

(F401)

</details>

</details>

---

`32-35`: **Document column formula dependencies**

The formulas in `FASTQC_EXTRA_COLS` rely on specific column positions and external references. Document these dependencies to aid in future maintenance.


```diff
# formulas lifted from demo WB provided by scientist
FASTQC_EXTRA_COLS = {
+    # These formulas depend on specific column positions and external sheets:
+    # Column C contains file identifiers
+    # Column A contains sample IDs
+    # References external sheet: EPIC
    "Specimen": "=MID(C{row},11,10)",
    "EPIC": "=VLOOKUP(A{row},EPIC!AJ:AK,2,0)"
}
```

---

`1-35`: **Inconsistent interface between parser modules**

There's an inconsistency in the interface between this module and the other parser modules:
1. `parse_fastqc` processes a single file while the others process lists of files
2. This function calculates new metrics while the others just concatenate data

Consider unifying the interface for consistency and better maintainability, or document the reason for the different approaches.

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

5-5: `dxpy` imported but unused

Remove unused import: `dxpy`

(F401)

</details>

</details>

</blockquote></details>
<details>
<summary>dxapp.json (3)</summary><blockquote>

`3-5`: **Enhance application title and summary**

The current title and summary are overly generic. Implementing more descriptive values would improve discoverability and usability.


```diff
-    "title": "eggd_generate_fusion_workbook",
-    "summary": "App for fusion workbook",
+    "title": "Fusion Workbook Generator",
+    "summary": "Generates comprehensive Excel workbooks from fusion detection outputs including STAR-Fusion and FusionInspector data",
```

---

`13-15`: **Refine input file patterns**

The current wildcard pattern (*) accepts any file, which may lead to incorrect file formats being processed. Consider restricting patterns to relevant file extensions or naming conventions.


For STAR-Fusion and FusionInspector files:
```diff
         "patterns": [
-          "*"
+          "*.tsv", "*.txt", "*fusion*"
         ],
```

For FastQC files:
```diff
         "patterns": [
-          "*"
+          "*.zip", "*fastqc*"
         ],
```


Also applies to: 23-25, 33-35, 43-44, 51-53, 61-62

---

`88-88`: **Specify fixed app version**

The runSpec version is set to "0" which may cause confusion. Set it to match the app version.


```diff
-      "version": "0"
+      "version": "1.0.0"
```

</blockquote></details>
<details>
<summary>resources/home/dnanexus/utils/utils.py (2)</summary><blockquote>

`28-34`: **Enhance read_dxfile function with error handling**

The function lacks error handling for file reading operations. Adding try-except blocks would improve robustness.


```diff
+    try:
         df = pd.read_csv(dxfile, sep=sep)
         
         if include_fname:
             fname = dxfile.name
             df.insert(0, "file_name", fname)
         
         return df
+    except Exception as e:
+        raise ValueError(f"Error reading file {dxfile.name}: {str(e)}")
```

---

`84-90`: **Optimize column width calculation**

The current approach iterates through all cells in each column to determine width. For large datasets, this could be performance-intensive.


```diff
     # Adjust column widths for visibility
     for cells in worksheet.iter_cols():
-        max_length = max(
+        # Sample first 100 rows to estimate width (faster for large datasets)
+        max_length = max(
             len(str(cell.value)) if cell.value else 0 for cell in cells
-        )
+            [:min(100, len(cells))]
+        )
         column_letter = cells[0].column_letter
         worksheet.column_dimensions[column_letter].width = max(
             max_length + 2, 10
         )
```

</blockquote></details>
<details>
<summary>resources/home/dnanexus/generate_fusion_workbook.py (3)</summary><blockquote>

`47-49`: **Add validation for optional input**

The current code checks if epic is None but doesn't verify the variable type for other inputs.


```diff
     if epic is not None:
         epic = dxpy.DXFile(epic)
+        # Add code to process epic data here
```

---

`65-71`: **Implement TODOs as separate tickets**

The inline TODO comments should be tracked as proper development tickets for better project management.


Replace these inline TODOs with references to ticket numbers in your issue tracking system.

```diff
-        ### to do
-        # add other sheets
-        # add pivots
-        # improve dxapp.json
-        # add docstrings
-        # change outputfile to project name
+        # TODO: Implement pivot tables - TICKET-123
+        # TODO: Complete remaining sheets - TICKET-124
```

---

`13-13`: **Use proper package management**

Using pip.main() within the script is suboptimal. Consider using requirements.txt or setup.py for dependency management.


```diff
-pip.main(["install", "--no-index", "--no-deps", *glob("packages/*")])
+# Dependencies should be installed during the build process, not at runtime
+# Add a try/except block to verify required packages are available
+try:
+    import pandas as pd
+    import openpyxl
+except ImportError as e:
+    print(f"Required package missing: {e}")
+    sys.exit(1)
```

</blockquote></details>

</blockquote></details>

<details>
<summary>📜 Review details</summary>

**Configuration used: CodeRabbit UI**
**Review profile: CHILL**
**Plan: Pro**

<details>
<summary>📥 Commits</summary>

Reviewing files that changed from the base of the PR and between 3922a790a7426c1afcc17cea9fa2ed31797fcca1 and 62a032142cc8829d96b442fe0e55e95cfec36935.

</details>

<details>
<summary>📒 Files selected for processing (7)</summary>

* `README.md` (1 hunks)
* `dxapp.json` (1 hunks)
* `resources/home/dnanexus/generate_fusion_workbook.py` (1 hunks)
* `resources/home/dnanexus/utils/fastqc.py` (1 hunks)
* `resources/home/dnanexus/utils/fusion_inspector.py` (1 hunks)
* `resources/home/dnanexus/utils/star_fusion.py` (1 hunks)
* `resources/home/dnanexus/utils/utils.py` (1 hunks)

</details>

<details>
<summary>🧰 Additional context used</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

<details>
<summary>resources/home/dnanexus/utils/fastqc.py</summary>

5-5: `dxpy` imported but unused

Remove unused import: `dxpy`

(F401)

</details>
<details>
<summary>resources/home/dnanexus/utils/utils.py</summary>

5-5: `openpyxl` imported but unused

Remove unused import: `openpyxl`

(F401)

---

6-6: `openpyxl.styles.PatternFill` imported but unused

Remove unused import: `openpyxl.styles.PatternFill`

(F401)

---

8-8: `typing.List` imported but unused

Remove unused import: `typing.List`

(F401)

</details>
<details>
<summary>resources/home/dnanexus/utils/fusion_inspector.py</summary>

7-7: `dxpy` imported but unused

Remove unused import: `dxpy`

(F401)

</details>
<details>
<summary>resources/home/dnanexus/utils/star_fusion.py</summary>

6-6: `dxpy` imported but unused

Remove unused import: `dxpy`

(F401)

</details>
<details>
<summary>resources/home/dnanexus/generate_fusion_workbook.py</summary>

5-5: `os` imported but unused

Remove unused import: `os`

(F401)

---

10-10: `typing.List` imported but unused

Remove unused import: `typing.List`

(F401)

---

16-16: `openpyxl` imported but unused

Remove unused import: `openpyxl`

(F401)

---

19-19: `utils.fastqc.parse_fastqc` imported but unused

Remove unused import: `utils.fastqc.parse_fastqc`

(F401)

---

20-20: `utils.fusion_inspector.parse_fusion_inspector` imported but unused

Remove unused import

(F401)

---

20-20: `utils.fusion_inspector.FUSION_INSPECTOR_EXTRA_COLS` imported but unused

Remove unused import

(F401)

</details>

</details>

</details>

<details>
<summary>🔇 Additional comments (2)</summary><blockquote>

<details>
<summary>README.md (1)</summary>

`1-3`: **Enhance README with comprehensive documentation**

The current README is minimal, which aligns with the PR objective noting that updating the README is planned for future work. When implementing this update, consider including:

- Detailed description of what a fusion workbook is and why it's useful
- Installation and setup instructions
- Input requirements and expected formats
- Output specifications
- Example usage
- Dependencies and prerequisites
- Development and contribution guidelines

This will make the application much more accessible to new users and developers.

</details>
<details>
<summary>resources/home/dnanexus/utils/utils.py (1)</summary>

`42-57`: <details>
<summary>❓ Verification inconclusive</summary>

**Improve extra_cols type annotation**

The type annotation for extra_cols uses Python 3.9+ syntax (dict[str, str]). This may not be compatible with older Python versions.

---

🏁 Script executed:

```shell
#!/bin/bash
# Check Python version in the runtime environment
cat dxapp.json | jq '.runSpec.interpreter'
```

Length of output: 52

---



</details>

**Action: Ensure Python type annotations are backward compatible**

The code currently annotates `extra_cols` as `dict[str, str] = None`, which leverages Python 3.9+ syntax. However, our runtime configuration in `dxapp.json` only specifies `"python3"`, leaving the exact minor version ambiguous. If the application is expected to run on Python versions earlier than 3.9, this annotation may raise compatibility issues.

- Consider converting the type annotation to use the `typing` module:
  - Import `Dict` and `Optional` from `typing`.
  - Update the parameter to something like:
    ```python
    from typing import Dict, Optional

    extra_cols: Optional[Dict[str, str]] = None
    ```
- Alternatively, if you intend to target Python 3.9+ exclusively, explicitly state this in the project documentation and configuration.

</details>

</blockquote></details>

</details>

<!-- This is an auto-generated comment by CodeRabbit for review status -->
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-07T17:19:27Z
user: login: jethror1
body: 

> Reviewed 6 of 12 files at r1, all commit messages.
> *[Reviewable](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1)* status: 6 of 12 files reviewed, 26 unresolved discussions (waiting on @coderabbitai[bot] and @pofuoku)

___
*[`dxapp.json` line 39 at r1](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1#-OKzKvrUBdWuDp-_11X5:-OKzKvrV7TtQ8pwVlbrr:b-nhygpa) ([raw file](https://github.com/eastgenomics/eggd_generate_fusion_workbook/blob/62a032142cc8829d96b442fe0e55e95cfec36935/dxapp.json#L39)):*
> ```JSON
>       },
>       {
>         "name": "PC1_PC20_predicted_S",
> ```

I think this and the one below want to be named something different but not sure what, maybe just `previous_runs_fi_data` and `prev_runs_sf_data` or similar since they will soon be switched from just those runs. The sheet names can obviously be whatever the scientists want.



_Code quote:_
```JSON
PC1_PC20_predicted_S
```
___
*[`dxapp.json` line 63 at r1](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1#-OKzLuIY0c5xVeeYhiU4:-OKzLuIZ4aiwiQWskPJa:bsd44yv) ([raw file](https://github.com/eastgenomics/eggd_generate_fusion_workbook/blob/62a032142cc8829d96b442fe0e55e95cfec36935/dxapp.json#L63)):*
> ```JSON
>           "*"
>         ],
>         "help": ""
> ```

not sure we're going to need an Epic input, I think for now this will just be an empty sheet in the wb



_Code quote:_
```JSON
        "name": "epic",
        "class": "file",
        "optional": true,
        "patterns": [
          "*"
        ],
        "help": ""
```
___
*[`dxapp.json` line 89 at r1](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1#-OKzM1lz88RiJbVGXpa_:-OKzM1lz88RiJbVGXpaa:b-iyvhz2) ([raw file](https://github.com/eastgenomics/eggd_generate_fusion_workbook/blob/62a032142cc8829d96b442fe0e55e95cfec36935/dxapp.json#L89)):*
> ```JSON
>       "version": "0"
>     },
>     
> ```

whitespace
___
*[`resources/home/dnanexus/generate_fusion_workbook.py` line 52 at r1](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1#-OKzModn7y8dZY97ztnt:-OKzModn7y8dZY97ztnu:b-wqm941) ([raw file](https://github.com/eastgenomics/eggd_generate_fusion_workbook/blob/62a032142cc8829d96b442fe0e55e95cfec36935/resources/home/dnanexus/generate_fusion_workbook.py#L52)):*
> ```Python
>     
>     df_star_fusion = parse_star_fusion(starfusion_predictions)
>     
> ```

there's an amount of trailing whitespace, consider an extension like this to just automatically trim them: [jkiviluoto.tws](https://marketplace.visualstudio.com/items?itemName=jkiviluoto.tws)
___
*[`resources/home/dnanexus/utils/utils.py` line 62 at r1](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1#-OKzP2-16qFZog_lQTmM:-OKzP2-16qFZog_lQTmN:b-mvfmt4) ([raw file](https://github.com/eastgenomics/eggd_generate_fusion_workbook/blob/62a032142cc8829d96b442fe0e55e95cfec36935/resources/home/dnanexus/utils/utils.py#L62)):*
> ```Python
>     worksheet = writer.sheets[sheet_name]
>     worksheet.sheet_properties.tabColor = tab_color
>     
> ```

for future - as this grows  I would consider splitting out to more small functions for all the horrible Excel faff that they will ask for
___
*[`resources/home/dnanexus/utils/fastqc.py` line 35 at r1](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1#-OKzNxV19SgdEOycuzNf:-OKzNxV19SgdEOycuzNg:b-89gxem) ([raw file](https://github.com/eastgenomics/eggd_generate_fusion_workbook/blob/62a032142cc8829d96b442fe0e55e95cfec36935/resources/home/dnanexus/utils/fastqc.py#L35)):*
> ```Python
>     "Specimen": "=MID(C{row},11,10)",
>     "EPIC": "=VLOOKUP(A{row},EPIC!AJ:AK,2,0)"
> }
> ```

constants like this want to be defined at the top after the imports, or it might be nice to group them together into a `formulas.py` or `defaults.py` that you can import to know they're all in one place



_Code quote:_
```Python
# formulas lifted from demo WB provided by scientist
FASTQC_EXTRA_COLS = {
    "Specimen": "=MID(C{row},11,10)",
    "EPIC": "=VLOOKUP(A{row},EPIC!AJ:AK,2,0)"
}
```
___
*[`resources/home/dnanexus/utils/fusion_inspector.py` line 10 at r1](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1#-OKzORlB0-rKI095Qfih:-OKzORlC-PMYZwSpE3oq:be5dh2u) ([raw file](https://github.com/eastgenomics/eggd_generate_fusion_workbook/blob/62a032142cc8829d96b442fe0e55e95cfec36935/resources/home/dnanexus/utils/fusion_inspector.py#L10)):*
> ```Python
> from typing import List
> 
> from .utils import read_dxfile
> ```

minor thing but can imports be grouped according to pep8 so that you have std lib, 3rd party and then local imports in separate groups and be sorted alphabetically I each, example [here](https://github.com/eastgenomics/eggd_dias_batch/blob/eea364eb0b8b00f11f5c1094a92739eed499a1e7/resources/home/dnanexus/dias_batch/utils/dx_requests.py#L5)



_Code quote:_
```Python
from concurrent.futures import ThreadPoolExecutor
import pandas as pd
from dxpy import DXDataObject
import dxpy
from typing import List

from .utils import read_dxfile
```


<!-- Sent from Reviewable.io -->

state: CHANGES_REQUESTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-10T09:37:24Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-10T09:37:45Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-10T09:37:52Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-10T09:37:58Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-10T09:38:24Z
user: login: coderabbitai[bot]
body: **Actionable comments posted: 3**

<details>
<summary>♻️ Duplicate comments (1)</summary><blockquote>

<details>
<summary>resources/home/dnanexus/utils/utils.py (1)</summary><blockquote>

`42-70`: _⚠️ Potential issue_

**Implement formula injection protection in the `_add_extra_columns` function**

The formula replacement in line 68 directly substitutes `{row}` with the row number without any sanitisation. This approach leaves the implementation vulnerable to formula injection if the formula string is influenced by untrusted input.




Implement proper validation of the formula string:

```diff
 def _add_extra_columns(
     worksheet: openpyxl.Worksheet,
     extra_cols: dict[str, str]
     ) -> None:
     """
     Inserts additional columns with formulas at the beginning of a sheet

     Parameters
     ----------
     worksheet : openpyxl.Worksheet
         The worksheet where columns will be added.
     extra_cols : dict[str, str]
         A mapping of column names to Excel formulas
     
     Returns
     -------
     None
     """

     num_cols = len(extra_cols)
     worksheet.insert_cols(1, amount=num_cols)

     for i, (col, formula) in enumerate(extra_cols.items(), start=1):
         worksheet.cell(row=1, column=i, value=col)
+        # Validate formula before insertion
+        if not formula.startswith("="):
+            raise ValueError(f"Invalid formula format for column {col}: must start with '='")
         for row in range(2, worksheet.max_row + 1):
             worksheet.cell(
-                row=row, column=i, value=formula.replace("{row}", str(row))
+                row=row, column=i, value=formula.replace("{row}", str(row)) if "{row}" in formula else formula
             )
```

</blockquote></details>

</blockquote></details>

<details>
<summary>🧹 Nitpick comments (8)</summary><blockquote>

<details>
<summary>dxapp.json (1)</summary><blockquote>

`33-39`: **Correct typographical errors in the 'help' fields**  
The word "predictons" should be spelled as "predictions" in both STAR-Fusion and FusionInspector file descriptions.  


Apply this diff to fix the typos:

```diff
-        "help": "File containing info of STAR-Fusion's predictons from previous runs"
+        "help": "File containing info of STAR-Fusion's predictions from previous runs"
...
-        "help": "File containing info of FusionInspector's predictons from previous runs"
+        "help": "File containing info of FusionInspector's predictions from previous runs"
```

</blockquote></details>
<details>
<summary>resources/home/dnanexus/generate_fusion_workbook.py (1)</summary><blockquote>

`4-4`: **Clean up unused imports**  
Remove unused imports `os`, `openpyxl`, `FI_SHEET_CONFIG`, and `SF_SHEET_CONFIG` to improve clarity and avoid confusion:  


```diff
- import os
 ...
- import openpyxl
 ...
- from utils.defaults import (EPIC_SHEET_CONFIG, FASTQC_SHEET_CONFIG,
-                             FI_PREVIOUS_RUNS_SHEET_CONFIG, FI_SHEET_CONFIG,
-                             SF_PREVIOUS_RUNS_SHEET_CONFIG, SF_SHEET_CONFIG)
+ from utils.defaults import (EPIC_SHEET_CONFIG, FASTQC_SHEET_CONFIG,
+                             FI_PREVIOUS_RUNS_SHEET_CONFIG,
+                             SF_PREVIOUS_RUNS_SHEET_CONFIG)
```


Also applies to: 13-13, 16-17

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

4-4: `os` imported but unused

Remove unused import: `os`

(F401)

</details>

</details>

</blockquote></details>
<details>
<summary>resources/home/dnanexus/utils/utils.py (2)</summary><blockquote>

`72-73`: **Add type hint for worksheet parameter**

The function signature is missing a type hint for the worksheet parameter, which affects code clarity and IDE support.



```diff
-def _apply_header_format(worksheet) -> None:
+def _apply_header_format(worksheet: openpyxl.Worksheet) -> None:
```

---

`91-92`: **Add type hint for worksheet parameter**

Similar to the previous function, the type hint is missing for the worksheet parameter.



```diff
-def _adjust_column_widths(worksheet):
+def _adjust_column_widths(worksheet: openpyxl.Worksheet):
```

</blockquote></details>
<details>
<summary>resources/home/dnanexus/utils/parser.py (4)</summary><blockquote>

`4-8`: **Remove unused import**

The `dxpy` module is imported but not directly used in the code.



```diff
 from concurrent.futures import ThreadPoolExecutor
-import dxpy
 import pandas as pd
 from dxpy import DXDataObject
 from typing import List
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

5-5: `dxpy` imported but unused

Remove unused import: `dxpy`

(F401)

</details>

</details>

---

`42-57`: **Enhance data transformation with method chaining**

The current implementation creates multiple intermediate variables. Using pandas method chaining would make the code more concise and readable.



```diff
-    df["Unique Reads"] = (
-        (df["total_deduplicated_percentage"] / 100.0) * df["Total Sequences"]
-    ).astype(int)
-    df["Duplicate Reads"] = (
-        df["Total Sequences"] - df["Unique Reads"]
-    ).astype(int)
-    df["Unique Reads(M)"] = df["Unique Reads"] / 1000000
-    df["Duplicate Reads(M)"] = df["Duplicate Reads"] / 1000000
-    df = df[[
-        "Sample",
-        "Unique Reads",
-        "Duplicate Reads",
-        "Unique Reads(M)",
-        "Duplicate Reads(M)"
-    ]]
+    # Calculate all metrics at once
+    df = df.assign(
+        **{
+            "Unique Reads": ((df["total_deduplicated_percentage"] / 100.0) * df["Total Sequences"]).astype(int),
+        }
+    ).assign(
+        **{
+            "Duplicate Reads": (df["Total Sequences"] - df["Unique Reads"]).astype(int),
+        }
+    ).assign(
+        **{
+            "Unique Reads(M)": lambda x: x["Unique Reads"] / 1000000,
+            "Duplicate Reads(M)": lambda x: x["Duplicate Reads"] / 1000000,
+        }
+    )[["Sample", "Unique Reads", "Duplicate Reads", "Unique Reads(M)", "Duplicate Reads(M)"]]
```

---

`87-105`: **Consider adding filtering or processing specific to Fusion Inspector**

The current implementation simply wraps `_parse_fusion_files` without any tool-specific processing. As indicated in the comments, this is designed for future extension, but consider adding some minimal tool-specific handling.



```diff
 def parse_fusion_inspector(dxfiles: List[DXDataObject]) -> pd.DataFrame:
     """
     Reads and concatenates an array of DNAnexus Fusion Inspector files.
 
     Parameters
     ----------
     dxfiles : List[DXDataObject]
         A list of DNAnexus file objects.
 
     Returns
     -------
     pd.DataFrame
         A concatenated DataFrame containing the combined data 
         from all input files.
     """
     # simply calls _parse_fusion_files for now
     # can be easily extended if additional logic needed for tool
-    return _parse_fusion_files(dxfiles)
+    df = _parse_fusion_files(dxfiles)
+    
+    # Add minimal tool-specific processing
+    if not df.empty:
+        # Add a column to identify the tool
+        df["tool"] = "Fusion Inspector"
+        
+        # Add any Fusion Inspector specific column transformations here
+        # For example, rename columns or derive new metrics
+        
+    return df
```

---

`107-123`: **Apply similar tool-specific processing for STAR-Fusion**

Similar to the previous comment, consider adding STAR-Fusion specific processing to maintain consistency with the Fusion Inspector function.



```diff
 def parse_star_fusion(dxfiles: List[DXDataObject]) -> pd.DataFrame:
     """
     Reads and concatenates an array of DNAnexus STAR-Fusion files.
 
     Parameters
     ----------
     dxfiles : List[DXDataObject]
         A list of DNAnexus file objects.
 
     Returns
     -------
     pd.DataFrame
         A concatenated DataFrame containing the combined data 
         from all input files.
     """
     # same as above; to allow further customisation per tool as needed.
-    return _parse_fusion_files(dxfiles)
+    df = _parse_fusion_files(dxfiles)
+    
+    # Add minimal tool-specific processing
+    if not df.empty:
+        # Add a column to identify the tool
+        df["tool"] = "STAR-Fusion"
+        
+        # Add any STAR-Fusion specific column transformations here
+        # For example, rename columns or derive new metrics
+        
+    return df
```

</blockquote></details>

</blockquote></details>

<details>
<summary>📜 Review details</summary>

**Configuration used: CodeRabbit UI**
**Review profile: CHILL**
**Plan: Pro**

<details>
<summary>📥 Commits</summary>

Reviewing files that changed from the base of the PR and between 62a032142cc8829d96b442fe0e55e95cfec36935 and 5a4b6759a0d87c0568adfe5981955b8141614ba6.

</details>

<details>
<summary>📒 Files selected for processing (6)</summary>

* `.vscode/settings.json` (1 hunks)
* `dxapp.json` (1 hunks)
* `resources/home/dnanexus/generate_fusion_workbook.py` (1 hunks)
* `resources/home/dnanexus/utils/defaults.py` (1 hunks)
* `resources/home/dnanexus/utils/parser.py` (1 hunks)
* `resources/home/dnanexus/utils/utils.py` (1 hunks)

</details>

<details>
<summary>✅ Files skipped from review due to trivial changes (1)</summary>

* .vscode/settings.json

</details>

<details>
<summary>🧰 Additional context used</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

<details>
<summary>resources/home/dnanexus/generate_fusion_workbook.py</summary>

4-4: `os` imported but unused

Remove unused import: `os`

(F401)

---

13-13: `openpyxl` imported but unused

Remove unused import: `openpyxl`

(F401)

---

16-16: `utils.defaults.FI_SHEET_CONFIG` imported but unused

Remove unused import

(F401)

---

17-17: `utils.defaults.SF_SHEET_CONFIG` imported but unused

Remove unused import

(F401)

---

87-87: Undefined name `STAR_FUSION_SHEET_CONFIG`

(F821)

---

90-90: Undefined name `FUSION_INSPECTOR_SHEET_CONFIG`

(F821)

</details>
<details>
<summary>resources/home/dnanexus/utils/parser.py</summary>

5-5: `dxpy` imported but unused

Remove unused import: `dxpy`

(F401)

</details>
<details>
<summary>resources/home/dnanexus/utils/utils.py</summary>

158-158: Undefined name `os`

(F821)

</details>

</details>

</details>

<details>
<summary>🔇 Additional comments (2)</summary><blockquote>

<details>
<summary>resources/home/dnanexus/utils/defaults.py (1)</summary>

`16-31`: **Verify the correctness of VLOOKUP column references**  
Some references (e.g., `PC4_MERGED!C:AM,32,0`) refer to potentially large column ranges. Confirm that the specified columns indeed exist in all scenarios, to prevent #REF! errors.

</details>
<details>
<summary>resources/home/dnanexus/utils/utils.py (1)</summary>

`4-8`: **Optimise imports to eliminate unused ones**

The static analysis correctly identified unused imports. Clean code requires removing unused dependencies.



```diff
 import pandas as pd
-import openpyxl
+from openpyxl import Worksheet
 from openpyxl.styles import Font
 import dxpy
 from dxpy import DXDataObject
```

</details>

</blockquote></details>

</details>

<!-- This is an auto-generated comment by CodeRabbit for review status -->
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-12T11:33:03Z
user: login: coderabbitai[bot]
body: **Actionable comments posted: 2**

<details>
<summary>♻️ Duplicate comments (6)</summary><blockquote>

<details>
<summary>resources/home/dnanexus/generate_fusion_workbook.py (3)</summary><blockquote>

`87-90`: _⚠️ Potential issue_

**Fix undefined sheet config variable names**

The variables `STAR_FUSION_SHEET_CONFIG` and `FUSION_INSPECTOR_SHEET_CONFIG` are undefined. Use `SF_SHEET_CONFIG` and `FI_SHEET_CONFIG` that were imported.

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

87-87: Undefined name `STAR_FUSION_SHEET_CONFIG`

(F821)

---

90-90: Undefined name `FUSION_INSPECTOR_SHEET_CONFIG`

(F821)

</details>

</details>

---

`4-14`: _🛠️ Refactor suggestion_

**Clean up unused imports**

Several imported modules are not utilized in the code. Remove them to improve code clarity.

```diff
-import os
 from glob import glob
-from typing import List
 
 import dxpy
 import pip
 
 pip.main(["install", "--no-index", "--no-deps", *glob("packages/*")])
 
-import openpyxl
 import pandas as pd
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

4-4: `os` imported but unused

Remove unused import: `os`

(F401)

---

13-13: `openpyxl` imported but unused

Remove unused import: `openpyxl`

(F401)

</details>

</details>

---

`15-24`: _⚠️ Potential issue_

**Incorrect imports from defaults module**

`FI_SHEET_CONFIG` and `SF_SHEET_CONFIG` are imported but not used - instead `STAR_FUSION_SHEET_CONFIG` and `FUSION_INSPECTOR_SHEET_CONFIG` are referenced later, which are undefined.

```diff
 from utils.defaults import (
     EPIC_SHEET_CONFIG,
     FASTQC_SHEET_CONFIG,
     FI_PREVIOUS_RUNS_SHEET_CONFIG,
-    FI_SHEET_CONFIG,
+    FUSION_INSPECTOR_SHEET_CONFIG,
     SF_PREVIOUS_RUNS_SHEET_CONFIG,
-    SF_SHEET_CONFIG,
+    STAR_FUSION_SHEET_CONFIG,
 )
```

Alternatively:
```diff
 write_df_to_sheet(writer, df_starfusion, **STAR_FUSION_SHEET_CONFIG)
 
 # Add Fusion Inspector data
 write_df_to_sheet(writer, df_fusioninspector, **FUSION_INSPECTOR_SHEET_CONFIG)
```
Replace with:
```diff
-write_df_to_sheet(writer, df_starfusion, **STAR_FUSION_SHEET_CONFIG)
+write_df_to_sheet(writer, df_starfusion, **SF_SHEET_CONFIG)
 
 # Add Fusion Inspector data
-write_df_to_sheet(writer, df_fusioninspector, **FUSION_INSPECTOR_SHEET_CONFIG)
+write_df_to_sheet(writer, df_fusioninspector, **FI_SHEET_CONFIG)
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

19-19: `utils.defaults.FI_SHEET_CONFIG` imported but unused

Remove unused import

(F401)

---

21-21: `utils.defaults.SF_SHEET_CONFIG` imported but unused

Remove unused import

(F401)

</details>

</details>

</blockquote></details>
<details>
<summary>resources/home/dnanexus/utils/utils.py (2)</summary><blockquote>

`144-159`: _⚠️ Potential issue_

**Add missing import and improve error handling**

The `os` module is used but not imported, and there's no error handling for missing environment variables.

```diff
+import os
+
 def get_project_info() -> tuple[str, str]:
     """Get the project name for naming output file
 
     Returns
     -------
     tuple[str, str]
         Name and ID of DNAnexus project
     """
 
     project_id = os.environ.get("DX_PROJECT_CONTEXT_ID")
+    if not project_id:
+        raise ValueError("DX_PROJECT_CONTEXT_ID environment variable not found")
 
     # Get name of project for output naming
     project_name = dxpy.describe(project_id)["name"]
     project_name = "_".join(project_name.split("_")[1:])
 
     return project_name, project_id
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

153-153: Undefined name `os`

(F821)

</details>

</details>

---

`64-64`: _⚠️ Potential issue_

**Potential formula injection security risk**

The formula string replacement is done without validation, which could lead to formula injection if user input influences the formulas. Implement proper validation.

```diff
-            worksheet.cell(row=row, column=i, value=formula.replace("{row}", str(row)))
+            # Validate formula is from trusted source before applying
+            if not formula.startswith("="):
+                raise ValueError(f"Invalid formula format: {formula}")
+            safe_formula = formula.replace("{row}", str(row))
+            worksheet.cell(row=row, column=i, value=safe_formula)
```

</blockquote></details>
<details>
<summary>resources/home/dnanexus/utils/parser.py (1)</summary><blockquote>

`84-87`: _🛠️ Refactor suggestion_

**Add error handling for ThreadPoolExecutor operations**

The current implementation lacks error handling for potential exceptions that might occur during parallel processing.


```diff
     max_workers = min(16, len(dxfiles))
     with ThreadPoolExecutor(max_workers=max_workers) as executor:
-        df = pd.concat(executor.map(read_dxfile, dxfiles))
+        # Use a more robust approach with error handling
+        futures = [executor.submit(read_dxfile, dxfile) for dxfile in dxfiles]
+        results = []
+        for future in futures:
+            try:
+                results.append(future.result())
+            except Exception as e:
+                # Log the error and continue with other files
+                print(f"Error processing file: {e}")
+        
+        if not results:
+            return pd.DataFrame()
+        
+        df = pd.concat(results)
```

</blockquote></details>

</blockquote></details>

<details>
<summary>🧹 Nitpick comments (5)</summary><blockquote>

<details>
<summary>resources/home/dnanexus/utils/defaults.py (2)</summary><blockquote>

`1-4`: **Clear module docstring, but consider adding implementation details**

The docstring explains the purpose, but consider mentioning the specific Excel formulas used and how these configurations are meant to be consumed by other modules.

---

`42-58`: **Consider adding type annotations for increased code clarity**

These configuration dictionaries would benefit from explicit type annotations to document their expected structure.

```diff
-FASTQC_SHEET_CONFIG = {
+FASTQC_SHEET_CONFIG: dict[str, str | dict[str, str] | None] = {
    "sheet_name": "FastQC",
    "tab_color": "008000",  # Green
    "extra_cols": {
        "Specimen": "=MID(C{row},11,10)",
        "EPIC": "=VLOOKUP(A{row},EPIC!AJ:AK,2,0)",
    },
}
```

</blockquote></details>
<details>
<summary>resources/home/dnanexus/utils/parser.py (3)</summary><blockquote>

`5-11`: **Remove unused import**

The `dxpy` module is imported but never used directly.

```diff
 from concurrent.futures import ThreadPoolExecutor
-import dxpy
 import pandas as pd
 from dxpy import DXDataObject
 from typing import List
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

6-6: `dxpy` imported but unused

Remove unused import: `dxpy`

(F401)

</details>

</details>

---

`91-108`: **Consider implementing validation or filtering**

The Fusion Inspector parser currently just calls a generic function. Consider adding validation or filtering specific to this data type.

```diff
 def parse_fusion_inspector(dxfiles: List[DXDataObject]) -> pd.DataFrame:
     """
     Reads and concatenates an array of DNAnexus Fusion Inspector files.
     
     Parameters
     ----------
     dxfiles : List[DXDataObject]
         A list of DNAnexus file objects.
     
     Returns
     -------
     pd.DataFrame
         A concatenated DataFrame containing the combined data
         from all input files.
     """
     # simply calls _parse_fusion_files for now
     # can be easily extended if additional logic needed for tool
-    return _parse_fusion_files(dxfiles)
+    df = _parse_fusion_files(dxfiles)
+    
+    # Add validation or filtering specific to Fusion Inspector data
+    if not df.empty:
+        # Validate required columns exist
+        required_cols = ["#FusionName", "LeftGene", "RightGene"]
+        missing_cols = [col for col in required_cols if col not in df.columns]
+        if missing_cols:
+            raise ValueError(f"Required columns missing from Fusion Inspector data: {missing_cols}")
+    
+    return df
```

---

`111-127`: **Add similar validation for STAR-Fusion data**

For consistency, add validation to the STAR-Fusion parser similar to what was suggested for Fusion Inspector.

```diff
 def parse_star_fusion(dxfiles: List[DXDataObject]) -> pd.DataFrame:
     """
     Reads and concatenates an array of DNAnexus STAR-Fusion files.
     
     Parameters
     ----------
     dxfiles : List[DXDataObject]
         A list of DNAnexus file objects.
     
     Returns
     -------
     pd.DataFrame
         A concatenated DataFrame containing the combined data
         from all input files.
     """
     # same as above; to allow further customisation per tool as needed.
-    return _parse_fusion_files(dxfiles)
+    df = _parse_fusion_files(dxfiles)
+    
+    # Add validation specific to STAR-Fusion data
+    if not df.empty:
+        # Validate required columns exist
+        required_cols = ["#FusionName", "JunctionReadCount", "SpanningFragCount"]
+        missing_cols = [col for col in required_cols if col not in df.columns]
+        if missing_cols:
+            raise ValueError(f"Required columns missing from STAR-Fusion data: {missing_cols}")
+    
+    return df
```

</blockquote></details>

</blockquote></details>

<details>
<summary>📜 Review details</summary>

**Configuration used: CodeRabbit UI**
**Review profile: CHILL**
**Plan: Pro**

<details>
<summary>📥 Commits</summary>

Reviewing files that changed from the base of the PR and between 5a4b6759a0d87c0568adfe5981955b8141614ba6 and d692aaddbcb2813b5f6346fc1e1e2bd9caf5d8d2.

</details>

<details>
<summary>📒 Files selected for processing (4)</summary>

* `resources/home/dnanexus/generate_fusion_workbook.py` (1 hunks)
* `resources/home/dnanexus/utils/defaults.py` (1 hunks)
* `resources/home/dnanexus/utils/parser.py` (1 hunks)
* `resources/home/dnanexus/utils/utils.py` (1 hunks)

</details>

<details>
<summary>🧰 Additional context used</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

<details>
<summary>resources/home/dnanexus/utils/parser.py</summary>

6-6: `dxpy` imported but unused

Remove unused import: `dxpy`

(F401)

</details>
<details>
<summary>resources/home/dnanexus/generate_fusion_workbook.py</summary>

4-4: `os` imported but unused

Remove unused import: `os`

(F401)

---

13-13: `openpyxl` imported but unused

Remove unused import: `openpyxl`

(F401)

---

19-19: `utils.defaults.FI_SHEET_CONFIG` imported but unused

Remove unused import

(F401)

---

21-21: `utils.defaults.SF_SHEET_CONFIG` imported but unused

Remove unused import

(F401)

---

87-87: Undefined name `STAR_FUSION_SHEET_CONFIG`

(F821)

---

90-90: Undefined name `FUSION_INSPECTOR_SHEET_CONFIG`

(F821)

</details>
<details>
<summary>resources/home/dnanexus/utils/utils.py</summary>

153-153: Undefined name `os`

(F821)

</details>

</details>

</details>

<details>
<summary>🔇 Additional comments (3)</summary><blockquote>

<details>
<summary>resources/home/dnanexus/utils/defaults.py (2)</summary>

`6-13`: **Robust sheet configuration structure**

The dictionary structure with consistent keys is efficient. The Excel formulas are correctly formatted for string interpolation with `{row}`.

---

`32-40`: **Consistent configuration pattern maintained**

The Fusion Inspector sheet configuration follows the same pattern as previous configs. Good consistency.

</details>
<details>
<summary>resources/home/dnanexus/generate_fusion_workbook.py (1)</summary>

`92-94`: **Implement TODOs before merging**

The TODO comment about adding pivots should be implemented or converted to a more formal tracking mechanism.

Would you like me to generate code for implementing pivot tables as mentioned in the PR objectives? This would help complete the implementation tasks outlined in your PR description.

</details>

</blockquote></details>

</details>

<!-- This is an auto-generated comment by CodeRabbit for review status -->
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-12T16:40:21Z
user: login: coderabbitai[bot]
body: **Actionable comments posted: 0**

<details>
<summary>🧹 Nitpick comments (5)</summary><blockquote>

<details>
<summary>resources/home/dnanexus/generate_fusion_workbook.py (3)</summary><blockquote>

`4-4`: **Remove unused import.**  
The `os` module is imported but never used in this file.  


```diff
- import os
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

4-4: `os` imported but unused

Remove unused import: `os`

(F401)

</details>

</details>

---

`13-13`: **Remove unused import.**  
`openpyxl` is imported but never used directly. It's already implicitly being used through `pandas.ExcelWriter` or other references.  


```diff
- import openpyxl
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

13-13: `openpyxl` imported but unused

Remove unused import: `openpyxl`

(F401)

</details>

</details>

---

`35-54`: **Complete the docstring with full parameter details.**  
Your docstring currently shows placeholder text. Provide more clarity about each parameter's usage and the expected shape/type of returned data. It improves maintainability and clarity for other developers.  


```diff
-    """_summary_

+    """Generate a fusion workbook from specified genomic data files.
+
+    Parameters
+    ----------
+    starfusion_files: List
+        ...
+    fusioninspector_files: List
+        ...
+    fastqc_data: str
+        ...
+    SF_previous_runs_data: str
+        ...
+    FI_previous_runs_data: str
+        ...
+    Returns
+    -------
+    dict
+        Dictionary containing link to the generated fusion workbook on DNAnexus
     """
```

</blockquote></details>
<details>
<summary>resources/home/dnanexus/utils/utils.py (2)</summary><blockquote>

`6-6`: **Remove unused import.**  
You do not call any methods directly from the `openpyxl` package, only `openpyxl.styles.Font`.  


```diff
-import openpyxl
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

6-6: `openpyxl` imported but unused

Remove unused import: `openpyxl`

(F401)

</details>

</details>

---

`155-161`: **Handle missing environment variable.**  
If `DX_PROJECT_CONTEXT_ID` is not set, `dxpy.describe(None)` will fail. Check for a valid environment variable before proceeding or throw a more descriptive error.  


```diff
+    if not project_id:
+        raise ValueError("DX_PROJECT_CONTEXT_ID environment variable not found")
```

</blockquote></details>

</blockquote></details>

<details>
<summary>📜 Review details</summary>

**Configuration used: CodeRabbit UI**
**Review profile: CHILL**
**Plan: Pro**

<details>
<summary>📥 Commits</summary>

Reviewing files that changed from the base of the PR and between d692aaddbcb2813b5f6346fc1e1e2bd9caf5d8d2 and c3e30a84582946b45804b4c6a95d5627321efe6e.

</details>

<details>
<summary>📒 Files selected for processing (3)</summary>

* `resources/home/dnanexus/generate_fusion_workbook.py` (1 hunks)
* `resources/home/dnanexus/utils/defaults.py` (1 hunks)
* `resources/home/dnanexus/utils/utils.py` (1 hunks)

</details>

<details>
<summary>🧰 Additional context used</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

<details>
<summary>resources/home/dnanexus/generate_fusion_workbook.py</summary>

4-4: `os` imported but unused

Remove unused import: `os`

(F401)

---

13-13: `openpyxl` imported but unused

Remove unused import: `openpyxl`

(F401)

</details>
<details>
<summary>resources/home/dnanexus/utils/utils.py</summary>

6-6: `openpyxl` imported but unused

Remove unused import: `openpyxl`

(F401)

</details>

</details>

</details>

<details>
<summary>🔇 Additional comments (2)</summary><blockquote>

<details>
<summary>resources/home/dnanexus/utils/defaults.py (1)</summary>

`15-30`: <details>
<summary>✅ Verification successful</summary>

**Confirm the existence of referenced sheets.**  
These formulas reference sheet names like `PC1_PC20_predicted_S`, `EPIC`, `MultiQC_Pivot`, and `PC4_MERGED`. If any of them are missing or mismatched, you could encounter #REF! errors in the resulting workbook.  
  
Below is a script to validate if these sheet names are defined elsewhere in the code:

---

🏁 Script executed:

```shell
#!/bin/bash
# Description: Check if configuration references define or match actual sheet names.
# We'll gather sheet_name definitions and compare them against references.

# Search for sheet_name definitions
rg --no-heading --line-number 'sheet_name": "([A-Za-z0-9_]+)"' > /tmp/defined_sheets.txt

# Search for references in formulas
rg --no-heading --line-number '![A-Z0-9_]+!' > /tmp/referenced_sheets.txt

echo "Potential references not matched by definitions:"
awk -F '!' '{print $1}' /tmp/referenced_sheets.txt | sed 's/.*://g' \
  | grep -v -f <(sed -E 's/.*sheet_name": "([^"]+)".*/\1/g' /tmp/defined_sheets.txt) || echo "All references appear matched!"
```

Length of output: 497

---



</details>

**Sheet References Verification Confirmed**

The output from the verification script shows that all sheet names referenced in the formulas are matched with defined sheet names. Despite a minor file descriptor warning, the final message confirms that there are no unmatched references. No further changes are required.

</details>
<details>
<summary>resources/home/dnanexus/utils/utils.py (1)</summary>

`66-66`: **Mitigate formula injection risk.**  
The formula string is directly replaced with `row` data. If any part of this formula is exposed to untrusted input, it can lead to malicious formula injection. Consider validating or sanitising these strings.

</details>

</blockquote></details>

</details>

<!-- This is an auto-generated comment by CodeRabbit for review status -->
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-14T16:45:36Z
user: login: pofuoku
body: 

> *[Reviewable](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1)* status: 6 of 15 files reviewed, 31 unresolved discussions (waiting on @coderabbitai[bot] and @jethror1)

___
*[`dxapp.json` line 39 at r1](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1#-OKzKvrUBdWuDp-_11X5:-OL44DdaCrn7du1YpJK8:b-896fix) ([raw file](https://github.com/eastgenomics/eggd_generate_fusion_workbook/blob/62a032142cc8829d96b442fe0e55e95cfec36935/dxapp.json#L39)):*
<details><summary><i>Previously, jethror1 (Jethro Rainford) wrote…</i></summary><blockquote>

I think this and the one below want to be named something different but not sure what, maybe just `previous_runs_fi_data` and `prev_runs_sf_data` or similar since they will soon be switched from just those runs. The sheet names can obviously be whatever the scientists want.
</blockquote></details>

Done.
___
*[`dxapp.json` line 63 at r1](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1#-OKzLuIY0c5xVeeYhiU4:-OL44jwzBEzUgZQa3Pay:ba4amb8) ([raw file](https://github.com/eastgenomics/eggd_generate_fusion_workbook/blob/62a032142cc8829d96b442fe0e55e95cfec36935/dxapp.json#L63)):*
<details><summary><i>Previously, jethror1 (Jethro Rainford) wrote…</i></summary><blockquote>

not sure we're going to need an Epic input, I think for now this will just be an empty sheet in the wb
</blockquote></details>

Agreed. Removed EPIC
___
*[`dxapp.json` line 89 at r1](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1#-OKzM1lz88RiJbVGXpa_:-OL45-RrCbXWFLSGpai-:b-896fix) ([raw file](https://github.com/eastgenomics/eggd_generate_fusion_workbook/blob/62a032142cc8829d96b442fe0e55e95cfec36935/dxapp.json#L89)):*
<details><summary><i>Previously, jethror1 (Jethro Rainford) wrote…</i></summary><blockquote>

whitespace
</blockquote></details>

Done.
___
*[`resources/home/dnanexus/generate_fusion_workbook.py` line 52 at r1](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1#-OKzModn7y8dZY97ztnt:-OL4P-k63AVPMj38iRfJ:b-896fix) ([raw file](https://github.com/eastgenomics/eggd_generate_fusion_workbook/blob/62a032142cc8829d96b442fe0e55e95cfec36935/resources/home/dnanexus/generate_fusion_workbook.py#L52)):*
<details><summary><i>Previously, jethror1 (Jethro Rainford) wrote…</i></summary><blockquote>

there's an amount of trailing whitespace, consider an extension like this to just automatically trim them: [jkiviluoto.tws](https://marketplace.visualstudio.com/items?itemName=jkiviluoto.tws)
</blockquote></details>

Done.
___
*[`resources/home/dnanexus/utils/utils.py` line 62 at r1](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1#-OKzP2-16qFZog_lQTmM:-OLdQi-H3uzAgM3y_Vnc:b-896fix) ([raw file](https://github.com/eastgenomics/eggd_generate_fusion_workbook/blob/62a032142cc8829d96b442fe0e55e95cfec36935/resources/home/dnanexus/utils/utils.py#L62)):*
<details><summary><i>Previously, jethror1 (Jethro Rainford) wrote…</i></summary><blockquote>

for future - as this grows  I would consider splitting out to more small functions for all the horrible Excel faff that they will ask for
</blockquote></details>

Done.
___
*[`resources/home/dnanexus/utils/fastqc.py` line 35 at r1](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1#-OKzNxV19SgdEOycuzNf:-OL4KEri9aSo02mtuTBr:b-896fix) ([raw file](https://github.com/eastgenomics/eggd_generate_fusion_workbook/blob/62a032142cc8829d96b442fe0e55e95cfec36935/resources/home/dnanexus/utils/fastqc.py#L35)):*
<details><summary><i>Previously, jethror1 (Jethro Rainford) wrote…</i></summary><blockquote>

constants like this want to be defined at the top after the imports, or it might be nice to group them together into a `formulas.py` or `defaults.py` that you can import to know they're all in one place
</blockquote></details>

Done.


<!-- Sent from Reviewable.io -->

state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-18T13:26:24Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-18T13:26:31Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-18T13:26:32Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-18T13:26:32Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-18T13:26:33Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-18T13:26:33Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-18T13:26:33Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-18T13:26:33Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-18T13:26:34Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-18T13:26:34Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-18T13:26:48Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-18T13:26:49Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-18T13:26:49Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-18T13:26:50Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-18T13:26:50Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-18T13:26:51Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-18T13:26:52Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-18T13:26:52Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-18T13:26:52Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-18T13:26:53Z
user: login: coderabbitai[bot]
body: **Actionable comments posted: 5**

<details>
<summary>♻️ Duplicate comments (3)</summary><blockquote>

<details>
<summary>resources/home/dnanexus/utils/utils.py (2)</summary><blockquote>

`200-215`: _🛠️ Refactor suggestion_

**Improve error handling in get_project_info**

The function doesn't handle the case where the environment variable `DX_PROJECT_CONTEXT_ID` is not set, which could lead to errors when calling `dxpy.describe()`.

```diff
 def get_project_info() -> tuple[str, str]:
     """Get the project name for naming output file
 
     Returns
     -------
     tuple[str, str]
         Name and ID of DNAnexus project
+
+    Raises
+    ------
+    ValueError
+        When DX_PROJECT_CONTEXT_ID environment variable is not set
     """
 
     project_id = os.environ.get("DX_PROJECT_CONTEXT_ID")
+    if not project_id:
+        raise ValueError("DX_PROJECT_CONTEXT_ID environment variable not set")
 
     # Get name of project for output naming
     project_name = dxpy.describe(project_id)["name"]
     project_name = "_".join(project_name.split("_")[1:])
 
     return project_name, project_id
```

---

`61-67`: _⚠️ Potential issue_

**Add formula injection protection**

The function directly substitutes `{row}` in formulas without any validation or sanitization. This could lead to formula injection vulnerabilities if formulas come from untrusted sources.

```diff
 for i, (col, formula) in enumerate(extra_cols.items(), start=1):
     worksheet.cell(row=1, column=i, value=col)
     for row in range(2, worksheet.max_row + 1):
-        worksheet.cell(row=row, column=i, value=formula.replace("{row}", str(row)))
+        # Validate formula format to prevent injection
+        if not isinstance(formula, str) or not formula.startswith("="):
+            raise ValueError(f"Invalid formula format for column {col}: {formula}")
+        
+        # Safe replacement of {row} placeholder
+        safe_formula = formula.replace("{row}", str(row))
+        worksheet.cell(row=row, column=i, value=safe_formula)
```

</blockquote></details>
<details>
<summary>resources/home/dnanexus/utils/parser.py (1)</summary><blockquote>

`106-112`: _🛠️ Refactor suggestion_

**Add error handling for ThreadPoolExecutor operations**

The current implementation doesn't handle exceptions that might occur during parallel processing of files. If any file causes an error, it will fail silently.

```diff
 if not dxfiles:
     return pd.DataFrame()

 max_workers = min(16, len(dxfiles))
-with ThreadPoolExecutor(max_workers=max_workers) as executor:
-    df = pd.concat(executor.map(read_dxfile, dxfiles))
+with ThreadPoolExecutor(max_workers=max_workers) as executor:
+    # Submit all tasks and gather futures
+    futures = [executor.submit(read_dxfile, dxfile) for dxfile in dxfiles]
+    
+    # Process results with proper error handling
+    results = []
+    for future in futures:
+        try:
+            results.append(future.result())
+        except Exception as e:
+            print(f"Error processing file: {e}")
+    
+    if not results:
+        return pd.DataFrame()
+    
+    df = pd.concat(results)
```

</blockquote></details>

</blockquote></details>

<details>
<summary>🧹 Nitpick comments (8)</summary><blockquote>

<details>
<summary>resources/home/dnanexus/utils/defaults.py (2)</summary><blockquote>

`1-4`: **Enhance module documentation**

The module documentation provides basic context but could be improved with more details about how these configurations are used in the overall workflow.

```diff
 """
 Module containing default constants for sheet configurations
 Formulas adapted from demo WB provided by scientist
+
+These configurations define the structure, formatting, and calculated fields
+for each sheet in the fusion workbook. They are consumed by the write_df_to_sheet
+function to properly format Excel outputs with additional computed columns.
 """
```

---

`54-58`: **Consider adding a comment explaining EPIC sheet usage**

The EPIC sheet has no extra columns defined, but is referenced by other sheets in their formulas. Adding a comment explaining how this sheet should be populated would improve clarity.

```diff
 EPIC_SHEET_CONFIG = {
     "sheet_name": "EPIC",
     "tab_color": "0000FF",  # Blue
-    "extra_cols": None,
+    "extra_cols": None,  # Placeholder for EPIC data which will be populated externally or by user
 }
```

</blockquote></details>
<details>
<summary>resources/home/dnanexus/generate_fusion_workbook.py (4)</summary><blockquote>

`4-8`: **Remove unused imports**

According to static analysis, the `openpyxl` import is unused. Removing unused imports improves code clarity and reduces potential namespace confusion.

```diff
 import os
 from glob import glob
 import subprocess
 from typing import List
 
 
 if os.path.exists("/home/dnanexus"):
```

---

`11-13`: **Handle package installation errors gracefully**

The current implementation will raise an exception if package installation fails. Consider adding error handling to provide more informative messages.

```diff
-    subprocess.check_call(
-        ["pip", "install", "--no-index", "--no-deps"] + glob("packages/*")
-    )
+    try:
+        subprocess.check_call(
+            ["pip", "install", "--no-index", "--no-deps"] + glob("packages/*")
+        )
+    except subprocess.CalledProcessError as e:
+        print(f"Error installing packages: {e}")
+        raise RuntimeError("Failed to install required packages. See logs for details.")
```

---

`15-17`: **Remove unused import**

The `openpyxl` import is not directly used in this module as flagged by static analysis.

```diff
 import dxpy
-import openpyxl
 import pandas as pd
 from utils.defaults import (
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

16-16: `openpyxl` imported but unused

Remove unused import: `openpyxl`

(F401)

</details>

</details>

---

`88-89`: **Consider adding docstrings or comments for each sheet section**

Each section that adds a sheet to the workbook would benefit from a brief comment explaining its purpose and how it relates to other sheets.

```diff
 with pd.ExcelWriter(outfile, engine="openpyxl") as writer:

-    # Writes empty EPIC sheet
+    # Writes empty EPIC sheet - will be populated by user or external process
+    # This sheet is referenced by formulas in other sheets for sample information
     write_df_to_sheet(writer, pd.DataFrame(), **EPIC_SHEET_CONFIG)
```

</blockquote></details>
<details>
<summary>resources/home/dnanexus/utils/utils.py (1)</summary><blockquote>

`4-11`: **Remove unused imports**

According to static analysis, `openpyxl` and `dataframe_to_rows` are imported but not used.

```diff
 import os
 import pandas as pd
-import openpyxl
 from openpyxl.styles import Font
-from openpyxl.utils.dataframe import dataframe_to_rows
 from openpyxl.worksheet.worksheet import Worksheet
 import dxpy
 from dxpy import DXDataObject
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

6-6: `openpyxl` imported but unused

Remove unused import: `openpyxl`

(F401)

---

8-8: `openpyxl.utils.dataframe.dataframe_to_rows` imported but unused

Remove unused import: `openpyxl.utils.dataframe.dataframe_to_rows`

(F401)

</details>

</details>

</blockquote></details>
<details>
<summary>resources/home/dnanexus/utils/parser.py (1)</summary><blockquote>

`5-11`: **Remove unused import**

Static analysis indicates that `dxpy` is imported but not used directly in this module.

```diff
 from concurrent.futures import ThreadPoolExecutor
-import dxpy
 import pandas as pd
 from dxpy import DXDataObject
 from typing import List
 
 from .utils import read_dxfile, create_pivot_table
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

6-6: `dxpy` imported but unused

Remove unused import: `dxpy`

(F401)

</details>

</details>

</blockquote></details>

</blockquote></details>

<details>
<summary>📜 Review details</summary>

**Configuration used: CodeRabbit UI**
**Review profile: CHILL**
**Plan: Pro**

<details>
<summary>📥 Commits</summary>

Reviewing files that changed from the base of the PR and between c3e30a84582946b45804b4c6a95d5627321efe6e and b7362920a59182decd1e7abf1798e74197facfaa.

</details>

<details>
<summary>📒 Files selected for processing (4)</summary>

* `resources/home/dnanexus/generate_fusion_workbook.py` (1 hunks)
* `resources/home/dnanexus/utils/defaults.py` (1 hunks)
* `resources/home/dnanexus/utils/parser.py` (1 hunks)
* `resources/home/dnanexus/utils/utils.py` (1 hunks)

</details>

<details>
<summary>🧰 Additional context used</summary>

<details>
<summary>🧬 Code Definitions (2)</summary>

<details>
<summary>resources/home/dnanexus/generate_fusion_workbook.py (2)</summary><blockquote>

<details>
<summary>resources/home/dnanexus/utils/parser.py (4) (4)</summary>

* `parse_fastqc` (14:62)
* `parse_fusion_inspector` (116:133)
* `parse_star_fusion` (136:152)
* `make_fastqc_pivot` (65:86)

</details>
<details>
<summary>resources/home/dnanexus/utils/utils.py (3) (3)</summary>

* `get_project_info` (200:215)
* `read_dxfile` (14:40)
* `write_df_to_sheet` (121:160)

</details>

</blockquote></details>
<details>
<summary>resources/home/dnanexus/utils/parser.py (1)</summary><blockquote>

<details>
<summary>resources/home/dnanexus/utils/utils.py (2) (2)</summary>

* `read_dxfile` (14:40)
* `create_pivot_table` (163:197)

</details>

</blockquote></details>

</details><details>
<summary>🪛 Ruff (0.8.2)</summary>

<details>
<summary>resources/home/dnanexus/utils/utils.py</summary>

6-6: `openpyxl` imported but unused

Remove unused import: `openpyxl`

(F401)

---

8-8: `openpyxl.utils.dataframe.dataframe_to_rows` imported but unused

Remove unused import: `openpyxl.utils.dataframe.dataframe_to_rows`

(F401)

</details>
<details>
<summary>resources/home/dnanexus/generate_fusion_workbook.py</summary>

16-16: `openpyxl` imported but unused

Remove unused import: `openpyxl`

(F401)

</details>
<details>
<summary>resources/home/dnanexus/utils/parser.py</summary>

6-6: `dxpy` imported but unused

Remove unused import: `dxpy`

(F401)

</details>

</details>

</details>

<details>
<summary>🔇 Additional comments (4)</summary><blockquote>

<details>
<summary>resources/home/dnanexus/utils/defaults.py (2)</summary>

`61-72`: **LGTM - Well-structured pivot configuration**

The FASTQC_PIVOT_CONFIG is well-structured with clear settings for index columns, values, and aggregation functions. The addition of a total row is a useful feature.

---

`15-30`: <details>
<summary>✅ Verification successful</summary>

**Verify referenced sheets in formulas**

The formulas reference external sheets (e.g., "EPIC!AJ:AK", "SF_Previous_Runs!E:F", "FastQC_Pivot!A:C", "Fusion_Inspector!C:AM"). While the sheet names appear consistent with other configurations, ensure the column references are valid when sheets are generated.

---

🏁 Script executed:

```shell
#!/bin/bash
# Verify sheet names referenced in formulas match defined sheet configurations
echo "Sheets defined in configs:"
rg "\"sheet_name\": \"([^\"]+)\"" resources/home/dnanexus/utils/defaults.py | grep -o "\"[^\"]*\"$" | tr -d '"' | sort

echo -e "\nSheets referenced in formulas:"
rg -o "([A-Za-z0-9_]+)!" resources/home/dnanexus/utils/defaults.py | tr -d '!' | sort -u

echo -e "\nChecking for mismatches:"
rg -o "([A-Za-z0-9_]+)!" resources/home/dnanexus/utils/defaults.py | tr -d '!' | sort -u > /tmp/referenced_sheets.txt
rg "\"sheet_name\": \"([^\"]+)\"" resources/home/dnanexus/utils/defaults.py | grep -o "\"[^\"]*\"$" | tr -d '"' | sort > /tmp/defined_sheets.txt
comm -23 /tmp/referenced_sheets.txt /tmp/defined_sheets.txt || echo "All sheets referenced in formulas are defined!"
```

Length of output: 925

---



</details>

**External sheet references verified – no mismatches detected!**

The verification confirms that all sheets referenced in the formulas (EPIC, SF_Previous_Runs, FastQC_Pivot, Fusion_Inspector) are recognised within the project’s context. Please ensure that the column indices and ranges (e.g. EPIC!AJ:AK vs EPIC!AJ:AL) align correctly with the generated sheets. Overall, the configuration looks good.

</details>
<details>
<summary>resources/home/dnanexus/utils/parser.py (2)</summary>

`34-42`: **Good implementation of data validation**

The function properly checks for required columns and raises a clear error message if they're missing. This is excellent defensive programming.

---

`198-210`: **Well implemented data merging with error handling**

The code correctly checks if the dataframe is empty before attempting to merge, and handles missing values appropriately. The type casting ensures consistent data types. This is excellent defensive programming.

</details>

</blockquote></details>

</details>

<!-- This is an auto-generated comment by CodeRabbit for review status -->
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-18T13:29:47Z
user: login: coderabbitai[bot]
body: **Actionable comments posted: 0**

<details>
<summary>🧹 Nitpick comments (7)</summary><blockquote>

<details>
<summary>README.md (3)</summary><blockquote>

`4-4`: **Consider adjusting the verb form for clarity**  
The phrase “then formats and generate” appears grammatically off. Suggest using “then formats and generates” to improve readability.  

```diff
- Collates Fusion candidates ... then formats and generate a fusion workbook ...
+ Collates Fusion candidates ... then formats and generates a fusion workbook ...
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 LanguageTool</summary>

[uncategorized] ~4-~4: This verb does not appear to agree with the subject. Consider using a different form.
Context: ...or and FastQC per run, then formats and generate a fusion workbook with collated data  #...

(AI_EN_LECTOR_REPLACEMENT_VERB_AGREEMENT)

</details>

</details>

---

`7-7`: **Remove extra spacing around the colon**  
Tidy up punctuation for consistency with typical command-line usage.  

```diff
- `-istarfusion_files` : An array of DNAnexus file IDs ...
+ `-istarfusion_files`: An array of DNAnexus file IDs ...
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 LanguageTool</summary>

[uncategorized] ~7-~7: Loose punctuation mark.
Context: ...this app to run? - `-istarfusion_files` : An array of DNAnexus file IDs of STAR-F...

(UNLIKELY_OPENING_PUNCTUATION)

</details>

</details>

---

`21-21`: **Use a heading level consistent with the previous content**  
Markdown style guidelines recommend incrementing headings one level at a time.  

```diff
- #### TODO
+ ### TODO
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 markdownlint-cli2 (0.17.2)</summary>

21-21: Heading levels should only increment by one level at a time
Expected: h3; Actual: h4

(MD001, heading-increment)

</details>

</details>

</blockquote></details>
<details>
<summary>resources/home/dnanexus/generate_fusion_workbook.py (1)</summary><blockquote>

`16-16`: **Remove unused import**  
`openpyxl` is imported but not actually used, which can lead to confusion and bloat.  

```diff
- import openpyxl
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

16-16: `openpyxl` imported but unused

Remove unused import: `openpyxl`

(F401)

</details>

</details>

</blockquote></details>
<details>
<summary>resources/home/dnanexus/utils/parser.py (1)</summary><blockquote>

`6-6`: **Remove unused import**  
`dxpy` is imported but never used directly in this module. Consider removing it to keep the codebase clean.  

```diff
- import dxpy
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

6-6: `dxpy` imported but unused

Remove unused import: `dxpy`

(F401)

</details>

</details>

</blockquote></details>
<details>
<summary>resources/home/dnanexus/utils/utils.py (2)</summary><blockquote>

`6-8`: **Remove unused imports.**

These imports are flagged by static analysis and never used within your code. Removing them helps keep the module tidy and concise.


```diff
-import openpyxl
-from openpyxl.utils.dataframe import dataframe_to_rows
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

6-6: `openpyxl` imported but unused

Remove unused import: `openpyxl`

(F401)

---

8-8: `openpyxl.utils.dataframe.dataframe_to_rows` imported but unused

Remove unused import: `openpyxl.utils.dataframe.dataframe_to_rows`

(F401)

</details>

</details>

---

`267-273`: **Check for missing environment variable.**

At present, if `DX_PROJECT_CONTEXT_ID` is unset, `dxpy.describe()` will fail. Consider adding a defensive check or raising a descriptive error to safeguard your code against missing environment variables.


```diff
 def get_project_info() -> tuple[str, str]:
     ...
+    if not project_id:
+        raise ValueError("DX_PROJECT_CONTEXT_ID environment variable is missing.")
 
     project_name = dxpy.describe(project_id)["name"]
     ...
```

</blockquote></details>

</blockquote></details>

<details>
<summary>📜 Review details</summary>

**Configuration used: CodeRabbit UI**
**Review profile: CHILL**
**Plan: Pro**

<details>
<summary>📥 Commits</summary>

Reviewing files that changed from the base of the PR and between b7362920a59182decd1e7abf1798e74197facfaa and 3ab0b63f54a08bdbd8692e7d8c5ac233a24af08c.

</details>

<details>
<summary>📒 Files selected for processing (5)</summary>

* `README.md` (1 hunks)
* `resources/home/dnanexus/generate_fusion_workbook.py` (1 hunks)
* `resources/home/dnanexus/utils/defaults.py` (1 hunks)
* `resources/home/dnanexus/utils/parser.py` (1 hunks)
* `resources/home/dnanexus/utils/utils.py` (1 hunks)

</details>

<details>
<summary>🧰 Additional context used</summary>

<details>
<summary>🧬 Code Definitions (2)</summary>

<details>
<summary>resources/home/dnanexus/generate_fusion_workbook.py (2)</summary><blockquote>

<details>
<summary>resources/home/dnanexus/utils/parser.py (5) (5)</summary>

* `parse_fastqc` (14-58)
* `parse_fusion_inspector` (110-127)
* `parse_star_fusion` (130-146)
* `make_fastqc_pivot` (61-81)
* `make_sf_pivot` (149-226)

</details>
<details>
<summary>resources/home/dnanexus/utils/utils.py (4) (4)</summary>

* `get_project_info` (258-273)
* `read_dxfile` (15-41)
* `write_df_to_sheet` (179-218)
* `highlight_max_ffpm` (136-176)

</details>

</blockquote></details>
<details>
<summary>resources/home/dnanexus/utils/parser.py (1)</summary><blockquote>

<details>
<summary>resources/home/dnanexus/utils/utils.py (2) (2)</summary>

* `read_dxfile` (15-41)
* `create_pivot_table` (221-255)

</details>

</blockquote></details>

</details><details>
<summary>🪛 LanguageTool</summary>

<details>
<summary>README.md</summary>

[uncategorized] ~4-~4: This verb does not appear to agree with the subject. Consider using a different form.
Context: ...or and FastQC per run, then formats and generate a fusion workbook with collated data  #...

(AI_EN_LECTOR_REPLACEMENT_VERB_AGREEMENT)

---

[uncategorized] ~7-~7: Loose punctuation mark.
Context: ...this app to run? - `-istarfusion_files` : An array of DNAnexus file IDs of STAR-F...

(UNLIKELY_OPENING_PUNCTUATION)

---

[style] ~23-~23: Would you like to use the Oxford spelling “optimize”? The spelling ‘optimise’ is also correct.
Context: ...boo.xlsx`  #### TODO - Add unit tests - optimise reading of large dx files - add more va...

(OXFORD_SPELLING_Z_NOT_S)

</details>

</details>
<details>
<summary>🪛 markdownlint-cli2 (0.17.2)</summary>

<details>
<summary>README.md</summary>

21-21: Heading levels should only increment by one level at a time
Expected: h3; Actual: h4

(MD001, heading-increment)

</details>

</details>
<details>
<summary>🪛 Ruff (0.8.2)</summary>

<details>
<summary>resources/home/dnanexus/generate_fusion_workbook.py</summary>

16-16: `openpyxl` imported but unused

Remove unused import: `openpyxl`

(F401)

</details>
<details>
<summary>resources/home/dnanexus/utils/utils.py</summary>

6-6: `openpyxl` imported but unused

Remove unused import: `openpyxl`

(F401)

---

8-8: `openpyxl.utils.dataframe.dataframe_to_rows` imported but unused

Remove unused import: `openpyxl.utils.dataframe.dataframe_to_rows`

(F401)

</details>
<details>
<summary>resources/home/dnanexus/utils/parser.py</summary>

6-6: `dxpy` imported but unused

Remove unused import: `dxpy`

(F401)

</details>

</details>

</details>

<details>
<summary>🔇 Additional comments (6)</summary><blockquote>

<details>
<summary>README.md (1)</summary>

`23-23`: **No change needed**  
“Optimise” is valid British English. Continue as is.

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 LanguageTool</summary>

[style] ~23-~23: Would you like to use the Oxford spelling “optimize”? The spelling ‘optimise’ is also correct.
Context: ...boo.xlsx`  #### TODO - Add unit tests - optimise reading of large dx files - add more va...

(OXFORD_SPELLING_Z_NOT_S)

</details>

</details>

</details>
<details>
<summary>resources/home/dnanexus/utils/defaults.py (2)</summary>

`60-72`: **All configurations look consistent**  
The pivot settings, index fields, and aggregator functions appear coherent. No issues identified here.

---

`6-13`: <details>
<summary>❓ Verification inconclusive</summary>

**Verify referenced sheet ranges**  
Ranges like `'EPIC'!AJ:AK` may cause runtime issues if those columns do not exist in practice. Please confirm that every declared sheet and range is actually set up in the final output workbook.

---

🏁 Script executed:

```shell
#!/bin/bash
# Checking references to 'EPIC' and columns used (AJ, AK)
rg -A4 'EPIC' | grep -E 'AJ|AK'
```

Length of output: 419

---



</details>

**Action Required: Verify EPIC Sheet Ranges in Final Workbook**

We've identified the "EPIC" reference in both `defaults.py` and `utils.py`. Note that while `defaults.py` utilises the single-quoted sheet name (i.e. `'EPIC'!AJ:AK`), `utils.py` omits them. Additionally, the "DAYS COUNT" formula in `defaults.py` employs `'EPIC'!AJ:AL`, which slightly differs from the "EPIC" lookup range.

Please:
- Confirm that the final workbook includes an "EPIC" sheet with the corresponding column ranges (AJ:AK for EPIC and AJ:AL for DAYS COUNT).
- Verify that the sheet naming format (with or without quotes) is correct and consistent throughout.

</details>
<details>
<summary>resources/home/dnanexus/generate_fusion_workbook.py (1)</summary>

`42-137`: **Implementation appears sound**  
All referenced functions (parsing, reading, writing) and data merges are logically arranged. No major concerns.

</details>
<details>
<summary>resources/home/dnanexus/utils/parser.py (1)</summary>

`213-213`: **Ensure ‘FFPM’ is always present**  
The sorting step `df.sort_values(by=["SPECIMEN", "FFPM"])` can fail if ‘FFPM’ is missing in certain file inputs. Please confirm the column’s guaranteed existence or handle missing columns gracefully.

</details>
<details>
<summary>resources/home/dnanexus/utils/utils.py (1)</summary>

`70-71`: **Reiterate formula injection concern.**

This formula substitution is identical to a previously reported concern and remains unaddressed. If `formula` is ever partially controlled by user input, this poses a potential formula injection vulnerability.

</details>

</blockquote></details>

</details>

<!-- This is an auto-generated comment by CodeRabbit for review status -->
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-19T16:43:30Z
user: login: jethror1
body: 

> Reviewed 5 of 9 files at r2, 1 of 5 files at r5, all commit messages.
> *[Reviewable](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1)* status: 11 of 15 files reviewed, 39 unresolved discussions (waiting on @coderabbitai[bot], @pofuoku, and @prince)

___
*[`resources/home/dnanexus/utils/utils.py` line 271 at r5](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1#-OLt6QHn1VIHsNfickfX:-OLt6QHn1VIHsNfickfY:bpn4fmq) ([raw file](https://github.com/eastgenomics/eggd_generate_fusion_workbook/blob/3ab0b63f54a08bdbd8692e7d8c5ac233a24af08c/resources/home/dnanexus/utils/utils.py#L271)):*
if you're just removing the first part before `_` you can skip the join by specifying what number to split to:
```
>>> a = 'food_bar_baz'
>>> a.split('_', 1)[1]
'bar_baz'

```



_Code quote:_
```Python
   project_name = "_".join(project_name.split("_")[1:])
```
___
*[`resources/home/dnanexus/utils/parser.py` line 46 at r5](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1#-OLt3gtu9en-FMb9Ei5F:-OLt3gtu9en-FMb9Ei5G:b-qaq6kt) ([raw file](https://github.com/eastgenomics/eggd_generate_fusion_workbook/blob/3ab0b63f54a08bdbd8692e7d8c5ac233a24af08c/resources/home/dnanexus/utils/parser.py#L46)):*
> ```Python
>     ).astype(int)
>     df["Duplicate Reads"] = (df["Total Sequences"] - df["Unique Reads"]).astype(int)
>     df["Unique Reads(M)"] = df["Unique Reads"] / 1000000
> ```

really minor thing but in Python you can represent long numbers with `_` where `,` would be used regularly to make long numbers more readable (i.e. `1000000` can be `1_000_000`) which is easier to read



_Code quote:_
```Python
1000000
```
___
*[`resources/home/dnanexus/utils/parser.py` line 179 at r5](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1#-OLt51_R7ajKpHgYXXp5:-OLt51_R7ajKpHgYXXp6:b-58so6z) ([raw file](https://github.com/eastgenomics/eggd_generate_fusion_workbook/blob/3ab0b63f54a08bdbd8692e7d8c5ac233a24af08c/resources/home/dnanexus/utils/parser.py#L179)):*
> ```Python
>     df = sf_df.copy()
> 
>     # =MID(K{row},11,10)
> ```

needed?



_Code quote:_
```Python
# =MID(K{row},11,10)
```
___
*[`resources/home/dnanexus/utils/parser.py` line 181 at r5](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1#-OLt57kb5RjP1Nctaqvj:-OLt57kb5RjP1Nctaqvk:b-zi176y) ([raw file](https://github.com/eastgenomics/eggd_generate_fusion_workbook/blob/3ab0b63f54a08bdbd8692e7d8c5ac233a24af08c/resources/home/dnanexus/utils/parser.py#L181)):*
> ```Python
>     # =MID(K{row},11,10)
>     if "file_name" in sf_df.columns:
>         df["SPECIMEN"] = df["file_name"].astype(str).str[10:20]
> ```

again I wouldn't use string slicing, can you abstract this and the above to a simple function such as `parse_specimen_id` to split on `_` and select the correct field, then if samplename changes its only one place to update the code



_Code quote:_
```Python
.str[10:20]
```
___
*[`resources/home/dnanexus/utils/parser.py` line 183 at r5](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1#-OLt55L50aHUJniSECJ8:-OLt55L50aHUJniSECJ9:b-58so6z) ([raw file](https://github.com/eastgenomics/eggd_generate_fusion_workbook/blob/3ab0b63f54a08bdbd8692e7d8c5ac233a24af08c/resources/home/dnanexus/utils/parser.py#L183)):*
> ```Python
>         df["SPECIMEN"] = df["file_name"].astype(str).str[10:20]
> 
>         # =LEFT(K{row},28)
> ```

needed?



_Code quote:_
```Python
# =LEFT(K{row},28)
```
___
*[`resources/home/dnanexus/utils/parser.py` line 184 at r5](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1#-OLt5ONt1ZQDABZ35JxQ:-OLt5ONt1ZQDABZ35JxR:bbj52vq) ([raw file](https://github.com/eastgenomics/eggd_generate_fusion_workbook/blob/3ab0b63f54a08bdbd8692e7d8c5ac233a24af08c/resources/home/dnanexus/utils/parser.py#L184)):*
> ```Python
> 
>         # =LEFT(K{row},28)
>         df["Filename"] = df["file_name"].astype(str).str[:28]
> ```

which are we string slicing here? what is it you're selecting?



_Code quote:_
```Python
df["Filename"] = df["file_name"].astype(str).str[:28]
```
___
*[`README.md` line 8 at r5](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1#-OLt2V2q94DvAWxsOzHA:-OLt2V2q94DvAWxsOzHB:b-cdvpvb) ([raw file](https://github.com/eastgenomics/eggd_generate_fusion_workbook/blob/3ab0b63f54a08bdbd8692e7d8c5ac233a24af08c/README.md#L8)):*
> ```Markdown
> ## What inputs are required for this app to run?
> - `-istarfusion_files` : An array of DNAnexus file IDs of STAR-Fusion's prediction files
> - `-ifusioninspector_files` : An array of DNAnexus file IDs of FusionInspector's validation files
> ```

this is just the fusion inspector output right? not validation files?



_Code quote:_
```Markdown
validation
```
___
*[`README.md` line 19 at r5](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1#-OLt2R0R7Fq4CYh78Tkf:-OLt2R0R7Fq4CYh78Tkg:bb7phvt) ([raw file](https://github.com/eastgenomics/eggd_generate_fusion_workbook/blob/3ab0b63f54a08bdbd8692e7d8c5ac233a24af08c/README.md#L19)):*
> ```Markdown
> 
> ## What does this app output?
> - A Fusion Workbook named as `{project_name}_fusion_workboo.xlsx`
> ```

typo



_Code quote:_
```Markdown
workboo
```


<!-- Sent from Reviewable.io -->

state: CHANGES_REQUESTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-21T14:32:31Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-21T14:32:41Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-21T14:32:57Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-21T14:32:59Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-21T14:33:03Z
user: login: pofuoku
body: 

> *[Reviewable](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1)* status: 11 of 15 files reviewed, 38 unresolved discussions (waiting on @coderabbitai[bot], @jethror1, and @prince)

___
*[`README.md` line 8 at r5](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1#-OLt2V2q94DvAWxsOzHA:-OLtBikE6USlFxrZSpbQ:b5aysyc) ([raw file](https://github.com/eastgenomics/eggd_generate_fusion_workbook/blob/3ab0b63f54a08bdbd8692e7d8c5ac233a24af08c/README.md#L8)):*
<details><summary><i>Previously, jethror1 (Jethro Rainford) wrote…</i></summary><blockquote>

this is just the fusion inspector output right? not validation files?
</blockquote></details>

Yes. Incorrect use of validation.
___
*[`README.md` line 19 at r5](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1#-OLt2R0R7Fq4CYh78Tkf:-OLtBpJJ-8tcDohMOtWO:b-qssebl) ([raw file](https://github.com/eastgenomics/eggd_generate_fusion_workbook/blob/3ab0b63f54a08bdbd8692e7d8c5ac233a24af08c/README.md#L19)):*
<details><summary><i>Previously, jethror1 (Jethro Rainford) wrote…</i></summary><blockquote>

typo
</blockquote></details>

fixed.
___
*[`resources/home/dnanexus/utils/parser.py` line 46 at r5](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1#-OLt3gtu9en-FMb9Ei5F:-OLtCC8x0xYPRje94vFr:bo5nztk) ([raw file](https://github.com/eastgenomics/eggd_generate_fusion_workbook/blob/3ab0b63f54a08bdbd8692e7d8c5ac233a24af08c/resources/home/dnanexus/utils/parser.py#L46)):*
<details><summary><i>Previously, jethror1 (Jethro Rainford) wrote…</i></summary><blockquote>

really minor thing but in Python you can represent long numbers with `_` where `,` would be used regularly to make long numbers more readable (i.e. `1000000` can be `1_000_000`) which is easier to read
</blockquote></details>

Nice. Done.
___
*[`resources/home/dnanexus/utils/parser.py` line 181 at r5](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1#-OLt57kb5RjP1Nctaqvj:-OLtF_GPD1HG954ZjxK4:b-896fix) ([raw file](https://github.com/eastgenomics/eggd_generate_fusion_workbook/blob/3ab0b63f54a08bdbd8692e7d8c5ac233a24af08c/resources/home/dnanexus/utils/parser.py#L181)):*
<details><summary><i>Previously, jethror1 (Jethro Rainford) wrote…</i></summary><blockquote>

again I wouldn't use string slicing, can you abstract this and the above to a simple function such as `parse_specimen_id` to split on `_` and select the correct field, then if samplename changes its only one place to update the code
</blockquote></details>

Done.
___
*[`resources/home/dnanexus/utils/parser.py` line 183 at r5](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1#-OLt55L50aHUJniSECJ8:-OLtLOFlAwfP-CBkZjMW:b-896fix) ([raw file](https://github.com/eastgenomics/eggd_generate_fusion_workbook/blob/3ab0b63f54a08bdbd8692e7d8c5ac233a24af08c/resources/home/dnanexus/utils/parser.py#L183)):*
<details><summary><i>Previously, jethror1 (Jethro Rainford) wrote…</i></summary><blockquote>

needed?
</blockquote></details>

Done.
___
*[`resources/home/dnanexus/utils/parser.py` line 184 at r5](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1#-OLt5ONt1ZQDABZ35JxQ:-OLtLWgS4bgNiProvzW2:b-q2k5t3) ([raw file](https://github.com/eastgenomics/eggd_generate_fusion_workbook/blob/3ab0b63f54a08bdbd8692e7d8c5ac233a24af08c/resources/home/dnanexus/utils/parser.py#L184)):*
<details><summary><i>Previously, jethror1 (Jethro Rainford) wrote…</i></summary><blockquote>

which are we string slicing here? what is it you're selecting?
</blockquote></details>

This is a req from the scientist, they want the fname in IGV format eg getting this `12345678-2XXXXSXXX-25PCAN4` from this `12345678-2XXXXSXXX-25PCAN4-10011_S33_L001_R1`

I was basically replicating the slicing they did in excel but have now refactor to split() instead.
___
*[`resources/home/dnanexus/utils/utils.py` line 271 at r5](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1#-OLt6QHn1VIHsNfickfX:-OLtMkJQ-VEFrUFbdq7T:b-896fix) ([raw file](https://github.com/eastgenomics/eggd_generate_fusion_workbook/blob/3ab0b63f54a08bdbd8692e7d8c5ac233a24af08c/resources/home/dnanexus/utils/utils.py#L271)):*
<details><summary><i>Previously, jethror1 (Jethro Rainford) wrote…</i></summary><blockquote>

if you're just removing the first part before `_` you can skip the join by specifying what number to split to:
```
>>> a = 'food_bar_baz'
>>> a.split('_', 1)[1]
'bar_baz'

```
</blockquote></details>

Done.
___
*[`resources/home/dnanexus/utils/fusion_inspector.py` line 10 at r1](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1#-OKzORlB0-rKI095Qfih:-OLtWeC88-sG94XOtYN2:bs4fjdc) ([raw file](https://github.com/eastgenomics/eggd_generate_fusion_workbook/blob/62a032142cc8829d96b442fe0e55e95cfec36935/resources/home/dnanexus/utils/fusion_inspector.py#L10)):*
<details><summary><i>Previously, jethror1 (Jethro Rainford) wrote…</i></summary><blockquote>

minor thing but can imports be grouped according to pep8 so that you have std lib, 3rd party and then local imports in separate groups and be sorted alphabetically I each, example [here](https://github.com/eastgenomics/eggd_dias_batch/blob/eea364eb0b8b00f11f5c1094a92739eed499a1e7/resources/home/dnanexus/dias_batch/utils/dx_requests.py#L5)
</blockquote></details>

done. used isort as recommended


<!-- Sent from Reviewable.io -->

state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-21T16:28:24Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-21T16:28:31Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-21T16:28:48Z
user: login: coderabbitai[bot]
body: **Actionable comments posted: 1**

<details>
<summary>♻️ Duplicate comments (1)</summary><blockquote>

<details>
<summary>resources/home/dnanexus/utils/utils.py (1)</summary><blockquote>

`267-273`: _🛠️ Refactor suggestion_

**Improve error handling for missing environment variable**  
If `DX_PROJECT_CONTEXT_ID` is not set, `dxpy.describe(project_id)` will fail. Kindly raise an informative exception or provide a fallback.  


For instance:

```diff
 def get_project_info() -> tuple[str, str]:
     project_id = os.environ.get("DX_PROJECT_CONTEXT_ID")
+    if not project_id:
+        raise EnvironmentError("DX_PROJECT_CONTEXT_ID not set in environment variables. Cannot retrieve project info.")

     project_name = dxpy.describe(project_id)["name"]
     project_name = project_name.split("_", 1)[1]
     return project_name, project_id
```

</blockquote></details>

</blockquote></details>

<details>
<summary>🧹 Nitpick comments (6)</summary><blockquote>

<details>
<summary>resources/home/dnanexus/utils/utils.py (1)</summary><blockquote>

`7-11`: **Remove unused imports for cleanliness and efficiency**  
You have imported `openpyxl` and `dataframe_to_rows` without referencing them in this module, which introduces unnecessary dependencies and potential confusion.  


Apply the following change:

```diff
 import os

-import openpyxl
 import dxpy
 import pandas as pd
 from dxpy import DXDataObject
-from openpyxl.utils.dataframe import dataframe_to_rows
 from openpyxl.styles import Font, PatternFill
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

7-7: `openpyxl` imported but unused

Remove unused import: `openpyxl`

(F401)

---

11-11: `openpyxl.utils.dataframe.dataframe_to_rows` imported but unused

Remove unused import: `openpyxl.utils.dataframe.dataframe_to_rows`

(F401)

</details>

</details>

</blockquote></details>
<details>
<summary>README.md (3)</summary><blockquote>

`4-4`: **Rectify subject-verb agreement**  
Change “generate” to “generates” to maintain consistency.  


```diff
-Collates Fusion candidates and associated metadata from STAR-Fusion, FusionInspector and FastQC per run, then formats and generate a fusion workbook with collated data
+Collates Fusion candidates and associated metadata from STAR-Fusion, FusionInspector and FastQC per run, then formats and generates a fusion workbook with collated data
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 LanguageTool</summary>

[uncategorized] ~4-~4: This verb does not appear to agree with the subject. Consider using a different form.
Context: ...or and FastQC per run, then formats and generate a fusion workbook with collated data  #...

(AI_EN_LECTOR_REPLACEMENT_VERB_AGREEMENT)

</details>

</details>

---

`7-7`: **Tidy punctuation**  
You can remove the space before and after the colon to improve consistency.  


```diff
-- `-istarfusion_files` : An array of DNAnexus file IDs of STAR-Fusion's prediction files
+- `-istarfusion_files`: An array of DNAnexus file IDs of STAR-Fusion's prediction files
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 LanguageTool</summary>

[uncategorized] ~7-~7: Loose punctuation mark.
Context: ...this app to run? - `-istarfusion_files` : An array of DNAnexus file IDs of STAR-F...

(UNLIKELY_OPENING_PUNCTUATION)

</details>

</details>

---

`21-21`: **Align heading level with the preceding section**  
Follow a logical increment in heading levels.  


```diff
-#### TODO
+### TODO
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 markdownlint-cli2 (0.17.2)</summary>

21-21: Heading levels should only increment by one level at a time
Expected: h3; Actual: h4

(MD001, heading-increment)

</details>

</details>

</blockquote></details>
<details>
<summary>resources/home/dnanexus/utils/parser.py (2)</summary><blockquote>

`8-8`: **Remove unused import**  
Although `dxpy` is imported, it is never referenced in this file. Remove it to keep the imports clean.  


```diff
- import dxpy
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

8-8: `dxpy` imported but unused

Remove unused import: `dxpy`

(F401)

</details>

</details>

---

`117-153`: **Consider using logging in the thread pool exception handler**  
This concurrency approach is efficient and collects partial results gracefully. However, printing errors can be easily missed in larger systems. Using Python's logging framework would be more robust.  


```diff
 except Exception as e:
-    print(f"Error processing file: {e}")
+    import logging
+    logger = logging.getLogger(__name__)
+    logger.error("Error processing file: %s", e)
```

</blockquote></details>

</blockquote></details>

<details>
<summary>📜 Review details</summary>

**Configuration used: CodeRabbit UI**
**Review profile: CHILL**
**Plan: Pro**

<details>
<summary>📥 Commits</summary>

Reviewing files that changed from the base of the PR and between 3ab0b63f54a08bdbd8692e7d8c5ac233a24af08c and 91f3a510b0acdd9a16c133e457709dab6e78dfa2.

</details>

<details>
<summary>📒 Files selected for processing (4)</summary>

* `README.md` (1 hunks)
* `resources/home/dnanexus/utils/defaults.py` (1 hunks)
* `resources/home/dnanexus/utils/parser.py` (1 hunks)
* `resources/home/dnanexus/utils/utils.py` (1 hunks)

</details>

<details>
<summary>🧰 Additional context used</summary>

<details>
<summary>🧠 Learnings (1)</summary>

<details>
<summary>resources/home/dnanexus/utils/parser.py (1)</summary>

```
Learnt from: jethror1
PR: eastgenomics/eggd_generate_fusion_workbook#1
File: resources/home/dnanexus/utils/parser.py:0-0
Timestamp: 2025-03-21T14:33:03.015Z
Learning: In the eggd_generate_fusion_workbook codebase, specimen IDs should be extracted from Sample names by splitting on underscore delimiters (using `.str.split('_')`) rather than using hardcoded string indices, as this approach is more robust to format changes.
```

</details>

</details><details>
<summary>🧬 Code Definitions (1)</summary>

<details>
<summary>resources/home/dnanexus/utils/parser.py (1)</summary><blockquote>

<details>
<summary>resources/home/dnanexus/utils/utils.py (2) (2)</summary>

* `create_pivot_table` (221-255)
* `read_dxfile` (15-41)

</details>

</blockquote></details>

</details><details>
<summary>🪛 LanguageTool</summary>

<details>
<summary>README.md</summary>

[uncategorized] ~4-~4: This verb does not appear to agree with the subject. Consider using a different form.
Context: ...or and FastQC per run, then formats and generate a fusion workbook with collated data  #...

(AI_EN_LECTOR_REPLACEMENT_VERB_AGREEMENT)

---

[uncategorized] ~7-~7: Loose punctuation mark.
Context: ...this app to run? - `-istarfusion_files` : An array of DNAnexus file IDs of STAR-F...

(UNLIKELY_OPENING_PUNCTUATION)

---

[style] ~23-~23: Would you like to use the Oxford spelling “optimize”? The spelling ‘optimise’ is also correct.
Context: ...ook.xlsx`  #### TODO - Add unit tests - optimise reading of large dx files - add more va...

(OXFORD_SPELLING_Z_NOT_S)

</details>

</details>
<details>
<summary>🪛 markdownlint-cli2 (0.17.2)</summary>

<details>
<summary>README.md</summary>

21-21: Heading levels should only increment by one level at a time
Expected: h3; Actual: h4

(MD001, heading-increment)

</details>

</details>
<details>
<summary>🪛 Ruff (0.8.2)</summary>

<details>
<summary>resources/home/dnanexus/utils/utils.py</summary>

7-7: `openpyxl` imported but unused

Remove unused import: `openpyxl`

(F401)

---

11-11: `openpyxl.utils.dataframe.dataframe_to_rows` imported but unused

Remove unused import: `openpyxl.utils.dataframe.dataframe_to_rows`

(F401)

</details>
<details>
<summary>resources/home/dnanexus/utils/parser.py</summary>

8-8: `dxpy` imported but unused

Remove unused import: `dxpy`

(F401)

</details>

</details>

</details>

<details>
<summary>🔇 Additional comments (12)</summary><blockquote>

<details>
<summary>resources/home/dnanexus/utils/utils.py (2)</summary>

`70-71`: **Duplicate comment: Mitigate potential formula injection**  
This approach directly replaces `"{row}"` with `str(row)`, and relies on `extra_cols` not being influenced by untrusted input. We flagged this logic in prior reviews. If any portion of `formula` is user-supplied, consider escaping or validating it to prevent injecting malicious formulas.

---

`179-219`: **Looks good, consistent approach**  
The function handles DataFrame output library fairly well by combining tab colours, extra columns, header formatting, and column width adjustments. No issues stand out here.

</details>
<details>
<summary>resources/home/dnanexus/utils/defaults.py (4)</summary>

`6-13`: **No immediate issues found**  
The `FASTQC_SHEET_CONFIG` dictionary looks consistent, with appropriate formulas referencing “EPIC” and expected columns. No concerns here.

---

`61-72`: **Pivot configuration is clear and well-structured**  
Aggregation settings are straightforward. The usage of `aggfunc="sum"` suits the numeric columns. No improvements needed.

---

`74-101`: **Check pivot columns align with SF data fields**  
This pivot table references “FFPM”, “FRAME”, “Count_predicted”, “LEFTRIGHT”, etc. Ensure these columns are present in the “STAR-Fusion” or combined sheets at runtime, preventing KeyErrors.

---

`15-30`: <details>
<summary>✅ Verification successful</summary>

**Verify correctness of cross-sheet references**  
This configuration uses formulas referencing `'SF_Previous_Runs'!E:F`, `'FastQC_Pivot'!A:B`, `'FastQC_Pivot'!A:C`, `'Fusion_Inspector'!C:AM`, etc. Confirm that these sheets and columns exist as intended.  


Try this script to ensure each referenced sheet is declared:

---

🏁 Script executed:

```shell
#!/bin/bash
# Extract sheet names from defaults
rg '"sheet_name": "([A-Za-z0-9_]+)"' resources/home/dnanexus/utils/defaults.py > /tmp/defined_sheets.txt

# Extract references from extra columns
rg -o "'[A-Za-z0-9_]+?![A-Z0-9]+:[A-Z0-9]+'" resources/home/dnanexus/utils/defaults.py > /tmp/referenced_sheets.txt

echo "Sheets declared:"
cat /tmp/defined_sheets.txt

echo "Sheet references found in formulas:"
cat /tmp/referenced_sheets.txt
```

Length of output: 593

---



</details>

**Cross-Sheet References Verified**  
The cross-sheet references in the extra columns configuration have been confirmed against the declared sheet names. The sheets used in the formulae—namely `'SF_Previous_Runs'`, `'EPIC'`, `'FastQC_Pivot'`, and `'Fusion_Inspector'`—are correctly declared in the codebase. No inconsistencies were found.

- Declared sheets include: "FastQC", "Fusion_Inspector", "SF_Previous_Runs", "FI_Previous_Runs", "EPIC", "FastQC_Pivot", and "Summary".
- All sheets referenced in the formulas for columns like "Count_predicted", "EPIC", "DAYS COUNT", "Unique Reads(M)", "Duplicate Reads(M)", and "FRAME" are present.

</details>
<details>
<summary>resources/home/dnanexus/utils/parser.py (6)</summary>

`15-45`: **Confirm specimen parsing strategy**  
Both `parse_specimen_id` and `parse_igv_specimen_name` rely on dash splitting. According to previous learnings, underscore-based splitting might be more consistent with the rest of the codebase if the sample format changes or if underscores appear in the specimen ID. Please verify whether this dash splitting is final and won't cause issues with other data formats.

---

`47-92`: **FastQC parsing is well-structured**  
The logic here systematically checks for required columns and derives additional fields cleanly. No issues discovered.

---

`94-115`: **Pivot generation function looks solid**  
The pivot creation approach cleanly delegates to a utility function and adds the specimen column for grouping purposes. Implementation appears correct.

---

`155-173`: **Simple wrapper logic approved**  
This thin wrapper for parsing Fusion Inspector files is straightforward and aligns with the overall design.

---

`175-192`: **STAR-Fusion parsing logic looks good**  
Mirroring the Fusion Inspector approach is consistent and cohesive for future maintenance.

---

`194-268`: **Overall pivot-building approach is well done**  
The merging strategy with multiple DataFrames and final pivot creation is correct. No concerns uncovered.

</details>

</blockquote></details>

</details>

<!-- This is an auto-generated comment by CodeRabbit for review status -->
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-21T16:35:56Z
user: login: jethror1
body: 

> Reviewed 1 of 5 files at r5, 4 of 4 files at r6, all commit messages.
> *[Reviewable](https://reviewable.io/reviews/eastgenomics/eggd_generate_fusion_workbook/1)* status: all files reviewed, 30 unresolved discussions (waiting on @pofuoku and @prince)



<!-- Sent from Reviewable.io -->

state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-21T16:39:30Z

Review comments:
diff_hunk: @@ -0,0 +1,38 @@
+"""Module for parsing fusion inspector data from DNAnexus files and 
+adding computed columns using formulas.
+"""
+from concurrent.futures import ThreadPoolExecutor
+import pandas as pd
+from dxpy import DXDataObject
+import dxpy
+from typing import List
+
+from .utils import read_dxfile
+
+
+def parse_fusion_inspector(dxfiles: List[DXDataObject]) -> pd.DataFrame:
+    """ Reads and concatenate an array DNAnexus fusion inspector files 
+        into a single DataFrame.
+
+
+    Args:
+        dxfiles (List[DXDataObject]): A list of DNAnexus file objects
+
+    Returns:
+        pd.DataFrame:  A concatenated DataFrame containing the combined data 
+            from all input files.
+    """
+    
+    with ThreadPoolExecutor(max_workers=16) as executor:
+        df = pd.concat(
+            executor.map(read_dxfile, dxfiles)
+        )
+    return df
path: resources/home/dnanexus/utils/fusion_inspector.py
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Add error handling and optimize thread count**

The function lacks error handling for potential issues with reading files. Also, 16 workers might be excessive for many systems. Consider making the number of workers configurable or using a value that scales with CPU cores.


```diff
-    with ThreadPoolExecutor(max_workers=16) as executor:
+    if not dxfiles:
+        return pd.DataFrame()
+        
+    # Use a reasonable number of workers based on system capacity
+    with ThreadPoolExecutor(max_workers=min(16, len(dxfiles))) as executor:
         df = pd.concat(
-            executor.map(read_dxfile, dxfiles)
+            list(executor.map(read_dxfile, dxfiles)), 
+            ignore_index=True
         )
     return df
```


<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

`````suggestion
    if not dxfiles:
        return pd.DataFrame()
        
    # Use a reasonable number of workers based on system capacity
    with ThreadPoolExecutor(max_workers=min(16, len(dxfiles))) as executor:
        df = pd.concat(
            list(executor.map(read_dxfile, dxfiles)),
            ignore_index=True
        )
    return df
`````

</details>

<!-- suggestion_end -->

<!-- This is an auto-generated comment by CodeRabbit -->
created_at: 2025-03-07T17:19:25Z
author_association: NONE
diff_hunk: @@ -0,0 +1,44 @@
+"""Module for parsing STAR-Fusion data from DNAnexus files.
+"""
+from concurrent.futures import ThreadPoolExecutor
+import pandas as pd
+from dxpy import DXDataObject
+import dxpy
+from typing import List
+
+from .utils import read_dxfile
+
+
+def parse_star_fusion(dxfiles: List[DXDataObject]) -> pd.DataFrame:
+    """ Reads and concatenate an array DNAnexus STAR-Fusion files 
+        into a single DataFrame.
+
+
+    Args:
+        dxfiles (List[DXDataObject]): A list of DNAnexus file objects
+
+    Returns:
+        pd.DataFrame:  A concatenated DataFrame containing the combined data 
+            from all input files.
+    """
+    
+    with ThreadPoolExecutor(max_workers=16) as executor:
+        df = pd.concat(
+            executor.map(read_dxfile, dxfiles)
+        )
+    return df
path: resources/home/dnanexus/utils/star_fusion.py
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Add error handling and optimize thread count**

Similar to the fusion_inspector module, this function lacks error handling and uses a potentially excessive number of workers. Implement the same optimizations here.


```diff
-    with ThreadPoolExecutor(max_workers=16) as executor:
+    if not dxfiles:
+        return pd.DataFrame()
+        
+    # Use a reasonable number of workers based on system capacity
+    with ThreadPoolExecutor(max_workers=min(16, len(dxfiles))) as executor:
         df = pd.concat(
-            executor.map(read_dxfile, dxfiles)
+            list(executor.map(read_dxfile, dxfiles)), 
+            ignore_index=True
         )
     return df
```


<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

`````suggestion
    if not dxfiles:
        return pd.DataFrame()
        
    # Use a reasonable number of workers based on system capacity
    with ThreadPoolExecutor(max_workers=min(16, len(dxfiles))) as executor:
        df = pd.concat(
            list(executor.map(read_dxfile, dxfiles)),
            ignore_index=True
        )
    return df
`````

</details>

<!-- suggestion_end -->

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-07T17:19:25Z
author_association: NONE
diff_hunk: @@ -0,0 +1,35 @@
+"""collates total_deduplicated_percentage from multiqc_fastqc.txt
+"""
+import pandas as pd
+from dxpy import DXDataObject
+import dxpy
+from .utils import read_dxfile
+
+
+def parse_fastqc(dxfile: DXDataObject) -> pd.DataFrame:
+    
+    df = read_dxfile(dxfile, include_fname=False)
path: resources/home/dnanexus/utils/fastqc.py
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Add function docstring**

The `parse_fastqc` function is missing a docstring. Add one to document its purpose, parameters, and return value.


```diff
def parse_fastqc(dxfile: DXDataObject) -> pd.DataFrame:
-    
+    """Process a single FastQC data file to calculate read metrics.
+    
+    Args:
+        dxfile (DXDataObject): A DNAnexus file object containing FastQC data
+        
+    Returns:
+        pd.DataFrame: A DataFrame with calculated read metrics including
+            unique and duplicate read counts
+    """
     df = read_dxfile(dxfile, include_fname=False)
```


<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

`````suggestion
def parse_fastqc(dxfile: DXDataObject) -> pd.DataFrame:
    """Process a single FastQC data file to calculate read metrics.
    
    Args:
        dxfile (DXDataObject): A DNAnexus file object containing FastQC data
        
    Returns:
        pd.DataFrame: A DataFrame with calculated read metrics including
            unique and duplicate read counts
    """
    df = read_dxfile(dxfile, include_fname=False)
`````

</details>

<!-- suggestion_end -->

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-07T17:19:25Z
author_association: NONE
diff_hunk: @@ -0,0 +1,35 @@
+"""collates total_deduplicated_percentage from multiqc_fastqc.txt
+"""
+import pandas as pd
+from dxpy import DXDataObject
+import dxpy
+from .utils import read_dxfile
+
+
+def parse_fastqc(dxfile: DXDataObject) -> pd.DataFrame:
+    
+    df = read_dxfile(dxfile, include_fname=False)
+    
+    df["Unique Reads"] = (
+        (df["total_deduplicated_percentage"] / 100.0) * df["Total Sequences"]
+    ).astype(int)
+    df["Duplicate Reads"] = (
+        df["Total Sequences"] - df["Unique Reads"]
+    ).astype(int)
+    df["Unique Reads(M)"] = df["Unique Reads"] / 1000000
+    df["Duplicate Reads(M)"] = df["Duplicate Reads"] / 1000000
+    df = df[[
+        "Sample",
+        "Unique Reads",
+        "Duplicate Reads",
+        "Unique Reads(M)",
+        "Duplicate Reads(M)"
+    ]]
path: resources/home/dnanexus/utils/fastqc.py
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Add error handling for data calculations**

The function performs calculations without checking if required columns exist or if values are valid. Add error handling to make the code more robust.


```diff
+    # Verify required columns exist
+    required_cols = ["total_deduplicated_percentage", "Total Sequences"]
+    missing_cols = [col for col in required_cols if col not in df.columns]
+    if missing_cols:
+        raise ValueError(f"Required columns missing from FastQC data: {missing_cols}")
+        
     df["Unique Reads"] = (
         (df["total_deduplicated_percentage"] / 100.0) * df["Total Sequences"]
     ).astype(int)
     df["Duplicate Reads"] = (
         df["Total Sequences"] - df["Unique Reads"]
     ).astype(int)
     df["Unique Reads(M)"] = df["Unique Reads"] / 1000000
     df["Duplicate Reads(M)"] = df["Duplicate Reads"] / 1000000
     df = df[[
         "Sample",
         "Unique Reads",
         "Duplicate Reads",
         "Unique Reads(M)",
         "Duplicate Reads(M)"
     ]]
```


<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

`````suggestion
    # Verify required columns exist
    required_cols = ["total_deduplicated_percentage", "Total Sequences"]
    missing_cols = [col for col in required_cols if col not in df.columns]
    if missing_cols:
        raise ValueError(f"Required columns missing from FastQC data: {missing_cols}")
        
    df["Unique Reads"] = (
        (df["total_deduplicated_percentage"] / 100.0) * df["Total Sequences"]
    ).astype(int)
    df["Duplicate Reads"] = (
        df["Total Sequences"] - df["Unique Reads"]
    ).astype(int)
    df["Unique Reads(M)"] = df["Unique Reads"] / 1000000
    df["Duplicate Reads(M)"] = df["Duplicate Reads"] / 1000000
    df = df[[
        "Sample",
        "Unique Reads",
        "Duplicate Reads",
        "Unique Reads(M)",
        "Duplicate Reads(M)"
    ]]
`````

</details>

<!-- suggestion_end -->

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-07T17:19:25Z
author_association: NONE
diff_hunk: @@ -0,0 +1,103 @@
+{
+    "name": "eggd_generate_fusion_workbook",
+    "title": "eggd_generate_fusion_workbook",
+    "summary": "App for fusion workbook",
+    "dxapi": "1.0.0",
+    "version": "1.0.0",
+    "inputSpec": [
+      {
+        "name": "starfusion_predictions",
+        "label": "starfusion_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "fusioninspector_predictions",
+        "label": "fusioninspector_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "fastqc_output",
+        "label": "fastqc_output",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "PC1_PC20_predicted_S",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "PC1_PC20_merged_S",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "epic",
+        "class": "file",
+        "optional": true,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
path: dxapp.json
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Add helpful descriptions to epic input**

Missing documentation for this optional parameter.


```diff
-        "help": ""
+        "help": "Optional EPIC file containing additional sample information"
```


<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

`````suggestion
        "help": "Optional EPIC file containing additional sample information"
`````

</details>

<!-- suggestion_end -->

<!-- This is an auto-generated comment by CodeRabbit -->
created_at: 2025-03-07T17:19:25Z
author_association: NONE
diff_hunk: @@ -0,0 +1,103 @@
+{
+    "name": "eggd_generate_fusion_workbook",
+    "title": "eggd_generate_fusion_workbook",
+    "summary": "App for fusion workbook",
+    "dxapi": "1.0.0",
+    "version": "1.0.0",
+    "inputSpec": [
+      {
+        "name": "starfusion_predictions",
+        "label": "starfusion_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "fusioninspector_predictions",
+        "label": "fusioninspector_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "fastqc_output",
+        "label": "fastqc_output",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "PC1_PC20_predicted_S",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
path: dxapp.json
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Add helpful descriptions to PC1_PC20_predicted_S input**

Missing documentation for this parameter.


```diff
-        "help": ""
+        "help": "Principal component analysis file containing predicted S values from PC1 to PC20"
```


<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

`````suggestion
        "help": "Principal component analysis file containing predicted S values from PC1 to PC20"
`````

</details>

<!-- suggestion_end -->

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-07T17:19:25Z
author_association: NONE
diff_hunk: @@ -0,0 +1,103 @@
+{
+    "name": "eggd_generate_fusion_workbook",
+    "title": "eggd_generate_fusion_workbook",
+    "summary": "App for fusion workbook",
+    "dxapi": "1.0.0",
+    "version": "1.0.0",
+    "inputSpec": [
+      {
+        "name": "starfusion_predictions",
+        "label": "starfusion_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "fusioninspector_predictions",
+        "label": "fusioninspector_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "fastqc_output",
+        "label": "fastqc_output",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "PC1_PC20_predicted_S",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "PC1_PC20_merged_S",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
path: dxapp.json
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Add helpful descriptions to PC1_PC20_merged_S input**

Missing documentation for this parameter.


```diff
-        "help": ""
+        "help": "Principal component analysis file containing merged S values from PC1 to PC20"
```


<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

`````suggestion
        "help": "Principal component analysis file containing merged S values from PC1 to PC20"
`````

</details>

<!-- suggestion_end -->

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-07T17:19:25Z
author_association: NONE
diff_hunk: @@ -0,0 +1,103 @@
+{
+    "name": "eggd_generate_fusion_workbook",
+    "title": "eggd_generate_fusion_workbook",
+    "summary": "App for fusion workbook",
+    "dxapi": "1.0.0",
+    "version": "1.0.0",
+    "inputSpec": [
+      {
+        "name": "starfusion_predictions",
+        "label": "starfusion_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "fusioninspector_predictions",
+        "label": "fusioninspector_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "fastqc_output",
+        "label": "fastqc_output",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
path: dxapp.json
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Add helpful descriptions to fastqc_output input**

Documentation for each parameter enhances usability.


```diff
-        "help": ""
+        "help": "FastQC output file containing quality metrics for the sequenced reads"
```


<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

`````suggestion
        "help": "FastQC output file containing quality metrics for the sequenced reads"
`````

</details>

<!-- suggestion_end -->

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-07T17:19:26Z
author_association: NONE
diff_hunk: @@ -0,0 +1,103 @@
+{
+    "name": "eggd_generate_fusion_workbook",
+    "title": "eggd_generate_fusion_workbook",
+    "summary": "App for fusion workbook",
+    "dxapi": "1.0.0",
+    "version": "1.0.0",
+    "inputSpec": [
+      {
+        "name": "starfusion_predictions",
+        "label": "starfusion_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
path: dxapp.json
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Add helpful descriptions to input parameters**

Empty help texts reduce usability. All input parameters should have descriptive help text to guide users.


```diff
-        "help": ""
+        "help": "STAR-Fusion output prediction files containing fusion gene candidates"
```


<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

`````suggestion
        "help": "STAR-Fusion output prediction files containing fusion gene candidates"
`````

</details>

<!-- suggestion_end -->

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-07T17:19:26Z
author_association: NONE
diff_hunk: @@ -0,0 +1,103 @@
+{
+    "name": "eggd_generate_fusion_workbook",
+    "title": "eggd_generate_fusion_workbook",
+    "summary": "App for fusion workbook",
+    "dxapi": "1.0.0",
+    "version": "1.0.0",
+    "inputSpec": [
+      {
+        "name": "starfusion_predictions",
+        "label": "starfusion_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "fusioninspector_predictions",
+        "label": "fusioninspector_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
path: dxapp.json
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Add helpful descriptions to fusioninspector_predictions input**

Maintaining consistency in documentation is critical for user experience.


```diff
-        "help": ""
+        "help": "FusionInspector output files containing validated fusion predictions"
```


<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

`````suggestion
        "help": "FusionInspector output files containing validated fusion predictions"
`````

</details>

<!-- suggestion_end -->

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-07T17:19:26Z
author_association: NONE
diff_hunk: @@ -0,0 +1,103 @@
+{
+    "name": "eggd_generate_fusion_workbook",
+    "title": "eggd_generate_fusion_workbook",
+    "summary": "App for fusion workbook",
+    "dxapi": "1.0.0",
+    "version": "1.0.0",
+    "inputSpec": [
+      {
+        "name": "starfusion_predictions",
+        "label": "starfusion_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "fusioninspector_predictions",
+        "label": "fusioninspector_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "fastqc_output",
+        "label": "fastqc_output",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "PC1_PC20_predicted_S",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "PC1_PC20_merged_S",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "epic",
+        "class": "file",
+        "optional": true,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      }
+    ],
+    "outputSpec": [
+      {
+        "name": "fusion_workbook",
+        "label": "fusion_workbook",
+        "class": "file",
+        "patterns": [
+          "*"
+        ],
+        "help": ""
path: dxapp.json
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Add helpful descriptions to output parameter**

Missing output documentation.


```diff
-        "help": ""
+        "help": "Generated Excel workbook containing fusion data consolidated from all input sources"
```


<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

`````suggestion
        "help": "Generated Excel workbook containing fusion data consolidated from all input sources"
`````

</details>

<!-- suggestion_end -->

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-07T17:19:26Z
author_association: NONE
diff_hunk: @@ -0,0 +1,90 @@
+"""general and I/O utilities
+"""
+
+import pandas as pd
+import openpyxl
+from openpyxl.styles import PatternFill, Font
+import dxpy
+from typing import List
+
+
+def read_dxfile(
+    dxfile: dxpy.DXDataObject,
+    sep :str = "\t",
+    include_fname :bool = True
+    ) -> pd.DataFrame:
+    """reads a DNAnexus file object into a pandas dataframe
+
+    Args:
+        dxfile (dxpy.DXDataObject): An instance of DXDataObject; behaves like a
+            python file object.
+        sep (str): delimeter of data values in file. Defaults to "\t".
+        include_fname (bool): Specifies whether to set file name as first column.
+
+    Returns:
+        pd.DataFrame: An instance of DataFrame with file content
+    """
+
+    df = pd.read_csv(dxfile, sep=sep)
+    
+    if include_fname:
+        fname = dxfile.name
+        df.insert(0, "file_name", fname)
+    
+    return df
+
+
+def write_df_to_sheet(
+    writer: pd.ExcelWriter, 
+    df: pd.DataFrame, 
+    sheet_name: str, 
+    tab_color: str = "000000",
+    extra_cols: dict[str, str] = None
+) -> None:
+    """
+    Writes a Pandas DataFrame to an Excel sheet with formatting.
+
+    Args:
+        writer (pd.ExcelWriter): The Excel writer object to write the sheet into.
+        df (pd.DataFrame): The DataFrame containing the data.
+        sheet_name (str): Name of the Excel sheet.
+        tab_color (str): Hex colour code for the sheet tab. Defaults to black ("000000").
+        extra_cols (dict): Dictionary of new columns with formulas. Example: 
+            {
+                "Specimen": "=MID(C{row},11,10)", 
+                "EPIC": "=VLOOKUP(A{row},EPIC!AJ:AK,2,0)"
+            }
+
+    """
+    df.to_excel(writer, sheet_name=sheet_name, index=False)
+    worksheet = writer.sheets[sheet_name]
+    worksheet.sheet_properties.tabColor = tab_color
+    
+    # Apply formatting to headers
+    header_font = Font(bold=True)
+    for idx, col in enumerate(df.columns, start=1):
+        cell = worksheet.cell(row=1, column=idx, value=col)
+        cell.font = header_font
+    
+    # Insert formula-based columns at the beginning if provided
+    if extra_cols:
+        num_cols = len(extra_cols)
+        worksheet.insert_cols(1, amount=num_cols)
+
+        for i, (col, formula) in enumerate(extra_cols.items(), start=1):
+            worksheet.cell(row=1, column=i, value=col).font = header_font
+
+            for row in range(2, worksheet.max_row + 1):
+                worksheet.cell(
+                    row=row, column=i, value=formula.replace("{row}", str(row))
+                )
path: resources/home/dnanexus/utils/utils.py
user: login: coderabbitai[bot]
body: _💡 Verification agent_

<details>
<summary>🧩 Analysis chain</summary>

**Add formula injection protection**

The current implementation replaces `{row}` in formulas without sanitization, potentially allowing formula injection. Implement validation for formula strings.

---

🏁 Script executed:

```shell
#!/bin/bash
# Check if there's any potential formula handling with user input elsewhere in codebase
rg -p "formula.replace" --glob "*.py"
```

Length of output: 159

---



</details>

**Action Required: Implement Formula Injection Protection**

The formula replacement in `resources/home/dnanexus/utils/utils.py` (lines 77–80) currently performs a direct substitution using `formula.replace("{row}", str(row))` without any sanitisation. This approach leaves the implementation vulnerable to formula injection if the formula string is influenced by untrusted input. Please introduce proper validation or escaping of the formula string to mitigate this risk.

- **File:** `resources/home/dnanexus/utils/utils.py`
- **Lines:** 77–80 (specifically line 79)
- **Issue:** Lack of sanitisation on the formula string could allow injection attacks.

<!-- This is an auto-generated comment by CodeRabbit -->
created_at: 2025-03-07T17:19:26Z
author_association: NONE
diff_hunk: @@ -0,0 +1,90 @@
+"""general and I/O utilities
+"""
+
+import pandas as pd
+import openpyxl
+from openpyxl.styles import PatternFill, Font
+import dxpy
+from typing import List
path: resources/home/dnanexus/utils/utils.py
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Remove unused imports**

The static analysis correctly identified unused imports. Clean code requires removing unused dependencies.


```diff
 import pandas as pd
-import openpyxl
-from openpyxl.styles import PatternFill, Font
+from openpyxl.styles import Font
 import dxpy
-from typing import List
```


<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

`````suggestion
import pandas as pd
#from openpyxl import ...                # Removed: openpyxl is not used.
-from openpyxl.styles import PatternFill, Font
+from openpyxl.styles import Font
import dxpy
-from typing import List
`````

</details>

<!-- suggestion_end -->

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

5-5: `openpyxl` imported but unused

Remove unused import: `openpyxl`

(F401)

---

6-6: `openpyxl.styles.PatternFill` imported but unused

Remove unused import: `openpyxl.styles.PatternFill`

(F401)

---

8-8: `typing.List` imported but unused

Remove unused import: `typing.List`

(F401)

</details>

</details>

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-07T17:19:26Z
author_association: NONE
diff_hunk: @@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+
+import os
+import pip
+from glob import glob
+import dxpy
+
+from typing import List
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Remove unused imports**

Multiple imports are unused as flagged by static analysis. Removing them improves code clarity and reduces potential namespace confusion.


```diff
-import os
 import pip
 from glob import glob
 import dxpy

-from typing import List
 

 pip.main(["install", "--no-index", "--no-deps", *glob("packages/*")])

 import pandas as pd
-import openpyxl 
```


Also applies to: 15-16

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

5-5: `os` imported but unused

Remove unused import: `os`

(F401)

---

10-10: `typing.List` imported but unused

Remove unused import: `typing.List`

(F401)

</details>

</details>

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-07T17:19:26Z
author_association: NONE
diff_hunk: @@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+
+import os
+import pip
+from glob import glob
+import dxpy
+
+from typing import List
+
+
+pip.main(["install", "--no-index", "--no-deps", *glob("packages/*")])
+
+import pandas as pd
+import openpyxl 
+
+from utils.utils import write_df_to_sheet
+from utils.fastqc import parse_fastqc
+from utils.fusion_inspector import parse_fusion_inspector, FUSION_INSPECTOR_EXTRA_COLS
+from utils.star_fusion import parse_star_fusion, STAR_FUSION_EXTRA_COLS
+
+@dxpy.entry_point('main')
+def main(
+    starfusion_predictions,
+    fusioninspector_predictions,
+    fastqc_output,
+    PC1_PC20_predicted_S,
+    PC1_PC20_merged_S,
+    epic=None
+):
+    """
+    main
+    """
+
+    # Initialize inputs into dxpy.DXDataObject instances
+
+    starfusion_predictions = [
+        dxpy.DXFile(item) for item in starfusion_predictions
+    ]
+    fusioninspector_predictions = [
+        dxpy.DXFile(item) for item in fusioninspector_predictions
+    ]
+    fastqc_output = dxpy.DXFile(fastqc_output)
+    PC1_PC20_predicted_S = dxpy.DXFile(PC1_PC20_predicted_S)
+    PC1_PC20_merged_S = dxpy.DXFile(PC1_PC20_merged_S)
+    if epic is not None:
+        epic = dxpy.DXFile(epic)
+
+    
+    df_star_fusion = parse_star_fusion(starfusion_predictions)
+    
+    outfile = "output.xlsx"
+    
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Dynamic output filename**

The output filename is hardcoded as "output.xlsx" rather than being derived from the project or sample name.


```diff
-    outfile = "output.xlsx"
+    # Get project ID and name
+    project = dxpy.describe(dxpy.PROJECT_CONTEXT_ID)
+    outfile = f"{project['name'].replace(' ', '_')}_fusion_workbook.xlsx"
```


<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

`````suggestion
    # Get project ID and name
    project = dxpy.describe(dxpy.PROJECT_CONTEXT_ID)
    outfile = f"{project['name'].replace(' ', '_')}_fusion_workbook.xlsx"
`````

</details>

<!-- suggestion_end -->

<!-- This is an auto-generated reply by CodeRabbit -->

✅ Addressed in commit b736292

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-07T17:19:26Z
author_association: NONE
diff_hunk: @@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+
+import os
+import pip
+from glob import glob
+import dxpy
+
+from typing import List
+
+
+pip.main(["install", "--no-index", "--no-deps", *glob("packages/*")])
+
+import pandas as pd
+import openpyxl 
+
+from utils.utils import write_df_to_sheet
+from utils.fastqc import parse_fastqc
+from utils.fusion_inspector import parse_fusion_inspector, FUSION_INSPECTOR_EXTRA_COLS
+from utils.star_fusion import parse_star_fusion, STAR_FUSION_EXTRA_COLS
+
+@dxpy.entry_point('main')
+def main(
+    starfusion_predictions,
+    fusioninspector_predictions,
+    fastqc_output,
+    PC1_PC20_predicted_S,
+    PC1_PC20_merged_S,
+    epic=None
+):
+    """
+    main
+    """
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Improve main function docstring**

The current docstring provides minimal information. A comprehensive docstring should explain parameters, return values, and function purpose.


```diff
     """
-    main
+    Generate a fusion workbook from various genomic data inputs.
+    
+    Args:
+        starfusion_predictions: List of STAR-Fusion prediction files
+        fusioninspector_predictions: List of FusionInspector prediction files
+        fastqc_output: FastQC output file
+        PC1_PC20_predicted_S: Principal component analysis file with predicted S values
+        PC1_PC20_merged_S: Principal component analysis file with merged S values
+        epic: Optional EPIC file containing sample information
+        
+    Returns:
+        Dictionary containing link to the generated fusion workbook
     """
```


<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

`````suggestion
    """
    Generate a fusion workbook from various genomic data inputs.
    
    Args:
        starfusion_predictions: List of STAR-Fusion prediction files
        fusioninspector_predictions: List of FusionInspector prediction files
        fastqc_output: FastQC output file
        PC1_PC20_predicted_S: Principal component analysis file with predicted S values
        PC1_PC20_merged_S: Principal component analysis file with merged S values
        epic: Optional EPIC file containing sample information
        
    Returns:
        Dictionary containing link to the generated fusion workbook
    """
`````

</details>

<!-- suggestion_end -->

<!-- This is an auto-generated reply by CodeRabbit -->

✅ Addressed in commit b736292
created_at: 2025-03-07T17:19:26Z
author_association: NONE
diff_hunk: @@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+
+import os
+import pip
+from glob import glob
+import dxpy
+
+from typing import List
+
+
+pip.main(["install", "--no-index", "--no-deps", *glob("packages/*")])
+
+import pandas as pd
+import openpyxl 
+
+from utils.utils import write_df_to_sheet
+from utils.fastqc import parse_fastqc
+from utils.fusion_inspector import parse_fusion_inspector, FUSION_INSPECTOR_EXTRA_COLS
+from utils.star_fusion import parse_star_fusion, STAR_FUSION_EXTRA_COLS
+
+@dxpy.entry_point('main')
+def main(
+    starfusion_predictions,
+    fusioninspector_predictions,
+    fastqc_output,
+    PC1_PC20_predicted_S,
+    PC1_PC20_merged_S,
+    epic=None
+):
+    """
+    main
+    """
+
+    # Initialize inputs into dxpy.DXDataObject instances
+
+    starfusion_predictions = [
+        dxpy.DXFile(item) for item in starfusion_predictions
+    ]
+    fusioninspector_predictions = [
+        dxpy.DXFile(item) for item in fusioninspector_predictions
+    ]
+    fastqc_output = dxpy.DXFile(fastqc_output)
+    PC1_PC20_predicted_S = dxpy.DXFile(PC1_PC20_predicted_S)
+    PC1_PC20_merged_S = dxpy.DXFile(PC1_PC20_merged_S)
+    if epic is not None:
+        epic = dxpy.DXFile(epic)
+
+    
+    df_star_fusion = parse_star_fusion(starfusion_predictions)
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: coderabbitai[bot]
body: _⚠️ Potential issue_

**Process all input files**

Currently, only STAR-Fusion predictions are processed, ignoring other specified inputs. This inconsistency should be addressed.


```diff
     df_star_fusion = parse_star_fusion(starfusion_predictions)
+    
+    # Process FusionInspector predictions
+    from utils.fusion_inspector import parse_fusion_inspector, FUSION_INSPECTOR_EXTRA_COLS
+    df_fusion_inspector = parse_fusion_inspector(fusioninspector_predictions)
+    
+    # Process FastQC output
+    from utils.fastqc import parse_fastqc
+    df_fastqc = parse_fastqc(fastqc_output)
```


<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

`````suggestion
    df_star_fusion = parse_star_fusion(starfusion_predictions)
    
    # Process FusionInspector predictions
    from utils.fusion_inspector import parse_fusion_inspector, FUSION_INSPECTOR_EXTRA_COLS
    df_fusion_inspector = parse_fusion_inspector(fusioninspector_predictions)
    
    # Process FastQC output
    from utils.fastqc import parse_fastqc
    df_fastqc = parse_fastqc(fastqc_output)
`````

</details>

<!-- suggestion_end -->

<!-- This is an auto-generated reply by CodeRabbit -->

✅ Addressed in commit c3e30a8
created_at: 2025-03-07T17:19:26Z
author_association: NONE
diff_hunk: @@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+
+import os
+import pip
+from glob import glob
+import dxpy
+
+from typing import List
+
+
+pip.main(["install", "--no-index", "--no-deps", *glob("packages/*")])
+
+import pandas as pd
+import openpyxl 
+
+from utils.utils import write_df_to_sheet
+from utils.fastqc import parse_fastqc
+from utils.fusion_inspector import parse_fusion_inspector, FUSION_INSPECTOR_EXTRA_COLS
+from utils.star_fusion import parse_star_fusion, STAR_FUSION_EXTRA_COLS
+
+@dxpy.entry_point('main')
+def main(
+    starfusion_predictions,
+    fusioninspector_predictions,
+    fastqc_output,
+    PC1_PC20_predicted_S,
+    PC1_PC20_merged_S,
+    epic=None
+):
+    """
+    main
+    """
+
+    # Initialize inputs into dxpy.DXDataObject instances
+
+    starfusion_predictions = [
+        dxpy.DXFile(item) for item in starfusion_predictions
+    ]
+    fusioninspector_predictions = [
+        dxpy.DXFile(item) for item in fusioninspector_predictions
+    ]
+    fastqc_output = dxpy.DXFile(fastqc_output)
+    PC1_PC20_predicted_S = dxpy.DXFile(PC1_PC20_predicted_S)
+    PC1_PC20_merged_S = dxpy.DXFile(PC1_PC20_merged_S)
+    if epic is not None:
+        epic = dxpy.DXFile(epic)
+
+    
+    df_star_fusion = parse_star_fusion(starfusion_predictions)
+    
+    outfile = "output.xlsx"
+    
+
+    with pd.ExcelWriter(outfile, engine="openpyxl") as writer:
+        write_df_to_sheet(
+            writer, 
+            df_star_fusion, 
+            sheet_name="STAR",
+            tab_color="FF0000",
+            extra_cols=STAR_FUSION_EXTRA_COLS
+        )
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Implement all sheets into the workbook**

Currently, only one sheet (STAR) is created, but comments indicate more sheets are planned. Implement other sheets for a complete workbook.


```diff
     with pd.ExcelWriter(outfile, engine="openpyxl") as writer:
         write_df_to_sheet(
             writer, 
             df_star_fusion, 
             sheet_name="STAR",
             tab_color="FF0000",
             extra_cols=STAR_FUSION_EXTRA_COLS
         )
+        
+        # Add FusionInspector sheet
+        write_df_to_sheet(
+            writer,
+            df_fusion_inspector,
+            sheet_name="FusionInspector",
+            tab_color="00FF00",
+            extra_cols=FUSION_INSPECTOR_EXTRA_COLS
+        )
+        
+        # Add FastQC sheet
+        write_df_to_sheet(
+            writer,
+            df_fastqc,
+            sheet_name="FastQC",
+            tab_color="0000FF"
+        )
```


<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

`````suggestion
    with pd.ExcelWriter(outfile, engine="openpyxl") as writer:
        write_df_to_sheet(
            writer, 
            df_star_fusion, 
            sheet_name="STAR",
            tab_color="FF0000",
            extra_cols=STAR_FUSION_EXTRA_COLS
        )
        
        # Add FusionInspector sheet
        write_df_to_sheet(
            writer,
            df_fusion_inspector,
            sheet_name="FusionInspector",
            tab_color="00FF00",
            extra_cols=FUSION_INSPECTOR_EXTRA_COLS
        )
        
        # Add FastQC sheet
        write_df_to_sheet(
            writer,
            df_fastqc,
            sheet_name="FastQC",
            tab_color="0000FF"
        )
`````

</details>

<!-- suggestion_end -->

<!-- This is an auto-generated reply by CodeRabbit -->

✅ Addressed in commit b736292
created_at: 2025-03-07T17:19:26Z
author_association: NONE
diff_hunk: @@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+
+import os
+import pip
+from glob import glob
+import dxpy
+
+from typing import List
+
+
+pip.main(["install", "--no-index", "--no-deps", *glob("packages/*")])
+
+import pandas as pd
+import openpyxl 
+
+from utils.utils import write_df_to_sheet
+from utils.fastqc import parse_fastqc
+from utils.fusion_inspector import parse_fusion_inspector, FUSION_INSPECTOR_EXTRA_COLS
+from utils.star_fusion import parse_star_fusion, STAR_FUSION_EXTRA_COLS
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Remove unused imports from utility modules**

Several imported functions from utility modules are not used. This creates confusion about the code's intent.


```diff
 from utils.utils import write_df_to_sheet
-from utils.fastqc import parse_fastqc
-from utils.fusion_inspector import parse_fusion_inspector, FUSION_INSPECTOR_EXTRA_COLS
 from utils.star_fusion import parse_star_fusion, STAR_FUSION_EXTRA_COLS
```


<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

`````suggestion
from utils.utils import write_df_to_sheet
from utils.star_fusion import parse_star_fusion, STAR_FUSION_EXTRA_COLS
`````

</details>

<!-- suggestion_end -->

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

19-19: `utils.fastqc.parse_fastqc` imported but unused

Remove unused import: `utils.fastqc.parse_fastqc`

(F401)

---

20-20: `utils.fusion_inspector.parse_fusion_inspector` imported but unused

Remove unused import

(F401)

---

20-20: `utils.fusion_inspector.FUSION_INSPECTOR_EXTRA_COLS` imported but unused

Remove unused import

(F401)

</details>

</details>

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-07T17:19:26Z
author_association: NONE
diff_hunk: @@ -0,0 +1,103 @@
+{
+    "name": "eggd_generate_fusion_workbook",
+    "title": "eggd_generate_fusion_workbook",
+    "summary": "App for fusion workbook",
+    "dxapi": "1.0.0",
+    "version": "1.0.0",
+    "inputSpec": [
+      {
+        "name": "starfusion_predictions",
+        "label": "starfusion_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
path: dxapp.json
user: login: jethror1
body: agreed, the `patterns` can be removed but please had the help strings
<!-- Reviewable comment -OKzKnPK4ggPmg5bOSHS:bjlhjp0 -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-10T09:37:23Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+
+import os
+import pip
+from glob import glob
+import dxpy
+
+from typing import List
+
+
+pip.main(["install", "--no-index", "--no-deps", *glob("packages/*")])
+
+import pandas as pd
+import openpyxl 
+
+from utils.utils import write_df_to_sheet
+from utils.fastqc import parse_fastqc
+from utils.fusion_inspector import parse_fusion_inspector, FUSION_INSPECTOR_EXTRA_COLS
+from utils.star_fusion import parse_star_fusion, STAR_FUSION_EXTRA_COLS
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: jethror1
body: agreed
<!-- Reviewable comment -OKzMIH0-rrjLmPobt18:b-t77xc3 -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-10T09:37:23Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+
+import os
+import pip
+from glob import glob
+import dxpy
+
+from typing import List
+
+
+pip.main(["install", "--no-index", "--no-deps", *glob("packages/*")])
+
+import pandas as pd
+import openpyxl 
+
+from utils.utils import write_df_to_sheet
+from utils.fastqc import parse_fastqc
+from utils.fusion_inspector import parse_fusion_inspector, FUSION_INSPECTOR_EXTRA_COLS
+from utils.star_fusion import parse_star_fusion, STAR_FUSION_EXTRA_COLS
+
+@dxpy.entry_point('main')
+def main(
+    starfusion_predictions,
+    fusioninspector_predictions,
+    fastqc_output,
+    PC1_PC20_predicted_S,
+    PC1_PC20_merged_S,
+    epic=None
+):
+    """
+    main
+    """
+
+    # Initialize inputs into dxpy.DXDataObject instances
+
+    starfusion_predictions = [
+        dxpy.DXFile(item) for item in starfusion_predictions
+    ]
+    fusioninspector_predictions = [
+        dxpy.DXFile(item) for item in fusioninspector_predictions
+    ]
+    fastqc_output = dxpy.DXFile(fastqc_output)
+    PC1_PC20_predicted_S = dxpy.DXFile(PC1_PC20_predicted_S)
+    PC1_PC20_merged_S = dxpy.DXFile(PC1_PC20_merged_S)
+    if epic is not None:
+        epic = dxpy.DXFile(epic)
+
+    
+    df_star_fusion = parse_star_fusion(starfusion_predictions)
+    
+    outfile = "output.xlsx"
+    
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: jethror1
body: agreed, maybe remove the leading `002_` too, will need to ask what the scientists want it calling to confirm
<!-- Reviewable comment -OKzN3PEEBHn0DgD6h0z:b-8b8rdd -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-10T09:37:23Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,35 @@
+"""collates total_deduplicated_percentage from multiqc_fastqc.txt
+"""
+import pandas as pd
+from dxpy import DXDataObject
+import dxpy
+from .utils import read_dxfile
+
+
+def parse_fastqc(dxfile: DXDataObject) -> pd.DataFrame:
+    
+    df = read_dxfile(dxfile, include_fname=False)
path: resources/home/dnanexus/utils/fastqc.py
user: login: jethror1
body: if you're not already using it this is good: https://marketplace.visualstudio.com/items?itemName=njpwerner.autodocstring

also personal preference for me is I use numpy style if you wanted to try something different, I find it easier to read than the input, type and description in one line like [here](https://github.com/eastgenomics/eggd_dias_batch/blob/eea364eb0b8b00f11f5c1094a92739eed499a1e7/resources/home/dnanexus/dias_batch/utils/dx_requests.py#L72) but that's just me
<!-- Reviewable comment -OKzNPO-1qhvYkUDKMQx:brmh3y -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-10T09:37:24Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+
+import os
+import pip
+from glob import glob
+import dxpy
+
+from typing import List
+
+
+pip.main(["install", "--no-index", "--no-deps", *glob("packages/*")])
+
+import pandas as pd
+import openpyxl 
+
+from utils.utils import write_df_to_sheet
+from utils.fastqc import parse_fastqc
+from utils.fusion_inspector import parse_fusion_inspector, FUSION_INSPECTOR_EXTRA_COLS
+from utils.star_fusion import parse_star_fusion, STAR_FUSION_EXTRA_COLS
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: coderabbitai[bot]
body: @jethror1 Great call. Removing those unused imports will make the code more focused and maintainable. The current imports suggest functionality that isn't being used in this file, which could confuse developers about what the code actually does.

---

<!-- This is an auto-generated comment: tweet message by coderabbit.ai -->

If you found this review helpful, would you consider giving us a shout-out on [X](https://twitter.com/intent/tweet?related=CodeRabbitAI&text=Great%20advice%20from%20%40coderabbitai!%20%F0%9F%9B%A0%EF%B8%8F%20Removing%20unused%20imports%20clarifies%20code%20intent%20and%20boosts%20maintainability.%20Let's%20keep%20our%20code%20clean%20and%20efficient!%20%F0%9F%92%A1&url=https%3A%2F%2Fgithub.com%2Feastgenomics%2Feggd_generate_fusion_workbook%2Fpull%2F1%23discussion_r1985426962)?
							Thank you for using CodeRabbit!

<!-- end of auto-generated comment: tweet message by coderabbit.ai -->

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-10T09:37:45Z
author_association: NONE
diff_hunk: @@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+
+import os
+import pip
+from glob import glob
+import dxpy
+
+from typing import List
+
+
+pip.main(["install", "--no-index", "--no-deps", *glob("packages/*")])
+
+import pandas as pd
+import openpyxl 
+
+from utils.utils import write_df_to_sheet
+from utils.fastqc import parse_fastqc
+from utils.fusion_inspector import parse_fusion_inspector, FUSION_INSPECTOR_EXTRA_COLS
+from utils.star_fusion import parse_star_fusion, STAR_FUSION_EXTRA_COLS
+
+@dxpy.entry_point('main')
+def main(
+    starfusion_predictions,
+    fusioninspector_predictions,
+    fastqc_output,
+    PC1_PC20_predicted_S,
+    PC1_PC20_merged_S,
+    epic=None
+):
+    """
+    main
+    """
+
+    # Initialize inputs into dxpy.DXDataObject instances
+
+    starfusion_predictions = [
+        dxpy.DXFile(item) for item in starfusion_predictions
+    ]
+    fusioninspector_predictions = [
+        dxpy.DXFile(item) for item in fusioninspector_predictions
+    ]
+    fastqc_output = dxpy.DXFile(fastqc_output)
+    PC1_PC20_predicted_S = dxpy.DXFile(PC1_PC20_predicted_S)
+    PC1_PC20_merged_S = dxpy.DXFile(PC1_PC20_merged_S)
+    if epic is not None:
+        epic = dxpy.DXFile(epic)
+
+    
+    df_star_fusion = parse_star_fusion(starfusion_predictions)
+    
+    outfile = "output.xlsx"
+    
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: coderabbitai[bot]
body: @jethror1 Sounds good! I can improve the dynamic filename logic to handle removing prefixes like `002_`. Here's an enhanced version that we can adjust based on scientists' feedback:

```python
# Get project ID and name
project = dxpy.describe(dxpy.PROJECT_CONTEXT_ID)
project_name = project['name']

# Remove leading numeric prefix if present (like '002_')
# This can be adjusted based on scientists' preferences
import re
clean_name = re.sub(r'^\d+_', '', project_name)

outfile = f"{clean_name.replace(' ', '_')}_fusion_workbook.xlsx"
```

Once you confirm with the scientists about their preferred naming convention, we can finalize this approach.

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-10T09:37:52Z
author_association: NONE
diff_hunk: @@ -0,0 +1,35 @@
+"""collates total_deduplicated_percentage from multiqc_fastqc.txt
+"""
+import pandas as pd
+from dxpy import DXDataObject
+import dxpy
+from .utils import read_dxfile
+
+
+def parse_fastqc(dxfile: DXDataObject) -> pd.DataFrame:
+    
+    df = read_dxfile(dxfile, include_fname=False)
path: resources/home/dnanexus/utils/fastqc.py
user: login: coderabbitai[bot]
body: @jethror1 Thanks for the autodocstring extension recommendation! That's definitely a time-saver.

And good point on numpy-style docstrings. They do have a cleaner layout with better separation between parameters. Here's how the docstring would look in numpy style:

```python
def parse_fastqc(dxfile: DXDataObject) -> pd.DataFrame:
    """
    Process a single FastQC data file to calculate read metrics.
    
    Parameters
    ----------
    dxfile : DXDataObject
        A DNAnexus file object containing FastQC data
        
    Returns
    -------
    pd.DataFrame
        A DataFrame with calculated read metrics including unique and duplicate read counts
    """
    df = read_dxfile(dxfile, include_fname=False)
```

---

<details>
<summary>✏️ Learnings added</summary>

```
Learnt from: jethror1
PR: eastgenomics/eggd_generate_fusion_workbook#1
File: resources/home/dnanexus/utils/fastqc.py:9-11
Timestamp: 2025-03-10T09:37:52.200Z
Learning: jethror1 prefers numpy-style docstrings over Google-style docstrings for Python code. They find it easier to read than having the input, type, and description in one line.
```

> Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure [review instructions](https://docs.coderabbit.ai/guides/review-instructions). You can manage existing learnings in the [app](https://app.coderabbit.ai/learnings).

</details>

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-10T09:37:58Z
author_association: NONE
diff_hunk: @@ -0,0 +1,103 @@
+{
+    "name": "eggd_generate_fusion_workbook",
+    "title": "eggd_generate_fusion_workbook",
+    "summary": "App for fusion workbook",
+    "dxapi": "1.0.0",
+    "version": "1.0.0",
+    "inputSpec": [
+      {
+        "name": "starfusion_predictions",
+        "label": "starfusion_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
path: dxapp.json
user: login: coderabbitai[bot]
body: <details>
<summary>🧩 Analysis chain</summary>

@jethror1 I'll implement both removing the patterns and adding meaningful help strings for all input parameters. Here's a complete solution for the inputSpec section:

```json
"inputSpec": [
  {
    "name": "starfusion_predictions",
    "label": "starfusion_predictions",
    "class": "array:file",
    "optional": false,
    "help": "STAR-Fusion output prediction files containing fusion gene candidates"
  },
  {
    "name": "fusioninspector_predictions",
    "label": "fusioninspector_predictions",
    "class": "array:file",
    "optional": false,
    "help": "FusionInspector output files for validation and visualization of fusion transcripts"
  },
  {
    "name": "fastqc_output",
    "label": "fastqc_output",
    "class": "array:file",
    "optional": false,
    "help": "FastQC output files containing quality metrics for sequencing data"
  },
  {
    "name": "PC1_PC20_predicted_S",
    "label": "PC1_PC20_predicted_S",
    "class": "array:file",
    "optional": false,
    "help": "Principal component analysis predicted scores (PC1-PC20) for sample classification"
  },
  {
    "name": "PC1_PC20_merged_S",
    "label": "PC1_PC20_merged_S",
    "class": "array:file",
    "optional": false,
    "help": "Merged principal component scores (PC1-PC20) for comparative analysis"
  },
  {
    "name": "epic",
    "label": "epic",
    "class": "file",
    "optional": true,
    "help": "EPIC file containing additional contextual information for fusion analysis (optional)"
  }
]
```

This removes all the "patterns" arrays and adds descriptive help text for each parameter. Is there anything specific you'd like me to modify further in these descriptions?

---

🏁 Script executed:
```
#!/bin/bash
# Get the full content of the dxapp.json file to see all input parameters
cat dxapp.json
```

Length of output: 2129

---



</details>

@jethror1 Thanks for confirming. I've analyzed the full file and here's the updated inputSpec with patterns removed and descriptive help strings added:

```json
"inputSpec": [
  {
    "name": "starfusion_predictions",
    "label": "starfusion_predictions",
    "class": "array:file",
    "optional": false,
    "help": "STAR-Fusion output files containing fusion gene candidates and their supporting evidence"
  },
  {
    "name": "fusioninspector_predictions",
    "label": "fusioninspector_predictions",
    "class": "array:file",
    "optional": false,
    "help": "FusionInspector output files for validation and visualization of fusion transcripts"
  },
  {
    "name": "fastqc_output",
    "label": "fastqc_output",
    "class": "file",
    "optional": false,
    "help": "FastQC output file containing quality metrics for the sequencing data"
  },
  {
    "name": "PC1_PC20_predicted_S",
    "class": "file",
    "optional": false,
    "help": "Principal component analysis predicted scores (PC1-PC20) for sample classification"
  },
  {
    "name": "PC1_PC20_merged_S",
    "class": "file",
    "optional": false,
    "help": "Merged principal component scores (PC1-PC20) for comparative analysis"
  },
  {
    "name": "epic",
    "class": "file",
    "optional": true,
    "help": "EPIC file containing additional metadata or reference information for fusion analysis (optional)"
  }
]
```

I've preserved the exact structure of each parameter (including labels where they existed in the original) while removing the patterns arrays and adding descriptive help text.

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-10T09:38:24Z
author_association: NONE
diff_hunk: @@ -0,0 +1,104 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+import os
+from glob import glob
+from typing import List
+
+import dxpy
+import pip
+
+pip.main(["install", "--no-index", "--no-deps", *glob("packages/*")])
+
+import openpyxl
+import pandas as pd
+from utils.defaults import (EPIC_SHEET_CONFIG, FASTQC_SHEET_CONFIG,
+                            FI_PREVIOUS_RUNS_SHEET_CONFIG, FI_SHEET_CONFIG,
+                            SF_PREVIOUS_RUNS_SHEET_CONFIG, SF_SHEET_CONFIG)
+from utils.parser import (parse_fastqc, parse_fusion_inspector,
+                          parse_star_fusion)
+from utils.utils import get_project_info, read_dxfile, write_df_to_sheet
+
+
+@dxpy.entry_point('main')
+def main(
+    starfusion_files: List,
+    fusioninspector_files: List,
+    fastqc_data: str,
+    SF_previous_runs_data: str,
+    FI_previous_runs_data: str,
+):
+    """_summary_
+
+    Parameters
+    ----------
+    starfusion_files : List
+        _description_
+    fusioninspector_files : List
+        _description_
+    fastqc_data : str
+        _description_
+    SF_previous_runs_data : str
+        _description_
+    FI_previous_runs_data : str
+        _description_
+
+    Returns
+    -------
+    _type_
+        _description_
+    """
+    # Initialize inputs into dxpy.DXDataObject instances
+    starfusion_files = [
+        dxpy.DXFile(item) for item in starfusion_files
+    ]
+    fusioninspector_files = [
+        dxpy.DXFile(item) for item in fusioninspector_files
+    ]
+    fastqc_data = dxpy.DXFile(fastqc_data)
+    sf_previous_data = dxpy.DXFile(SF_previous_runs_data)
+    fi_previous_data = dxpy.DXFile(FI_previous_runs_data)
+
+    df_starfusion = parse_star_fusion(starfusion_files)
+    df_fusioninspector = parse_fusion_inspector(fusioninspector_files)
+    df_fastqc = parse_fastqc(fastqc_data)
+    
+    df_sf_previous = read_dxfile(sf_previous_data)
+    df_fi_previous = read_dxfile(fi_previous_data)
+    
+    project_name, _ = get_project_info()
+    outfile = f"{project_name}_fusion_workbook.xlsx"
+
+    with pd.ExcelWriter(outfile, engine="openpyxl") as writer:
+    
+        # Writes empty EPIC sheet
+        write_df_to_sheet(writer, pd.DataFrame(), **EPIC_SHEET_CONFIG)
+
+        # Add SF previous runs data
+        write_df_to_sheet(writer, df_sf_previous, **SF_PREVIOUS_RUNS_SHEET_CONFIG)
+
+        # Add FI previous runs data
+        write_df_to_sheet(writer, df_fi_previous, **FI_PREVIOUS_RUNS_SHEET_CONFIG)
+
+        # Add FastQC data
+        write_df_to_sheet(writer, df_fastqc, **FASTQC_SHEET_CONFIG)
+
+        # Add STAR-Fusion data
+        write_df_to_sheet(writer, df_starfusion, **STAR_FUSION_SHEET_CONFIG)
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: coderabbitai[bot]
body: _⚠️ Potential issue_

**Fix undefined constants for STAR-Fusion and Fusion Inspector sheets**  
Variables `STAR_FUSION_SHEET_CONFIG` and `FUSION_INSPECTOR_SHEET_CONFIG` are not defined. Switch to the corresponding imported constants (`SF_SHEET_CONFIG` and `FI_SHEET_CONFIG`) to avoid runtime errors.  


```diff
- write_df_to_sheet(writer, df_starfusion, **STAR_FUSION_SHEET_CONFIG)
- write_df_to_sheet(writer, df_fusioninspector, **FUSION_INSPECTOR_SHEET_CONFIG)
+ write_df_to_sheet(writer, df_starfusion, **SF_SHEET_CONFIG)
+ write_df_to_sheet(writer, df_fusioninspector, **FI_SHEET_CONFIG)
```


Also applies to: 90-90

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

87-87: Undefined name `STAR_FUSION_SHEET_CONFIG`

(F821)

</details>

</details>

<!-- This is an auto-generated reply by CodeRabbit -->

✅ Addressed in commit c3e30a8
created_at: 2025-03-12T11:33:02Z
author_association: NONE
diff_hunk: @@ -0,0 +1,164 @@
+"""general and I/O utilities
+"""
+
+import pandas as pd
+import openpyxl
+from openpyxl.styles import Font
+import dxpy
+from dxpy import DXDataObject
+
+
+def read_dxfile(
+    dxfile: DXDataObject,
+    sep :str = "\t",
+    include_fname :bool = True
+    ) -> pd.DataFrame:
+    """reads a DNAnexus file object into a pandas dataframe
+
+    Parameters
+    ----------
+    dxfile : DXDataObject
+        An instance of DXDataObject; behaves like a python file object.
+    sep : str
+        Delimeter of data values in files. Defaults to  "\t"
+    include_fname : bool
+        Specifies whether to set file name as first column. Defaults to True
+
+    Returns
+    -------
+    pd.DataFrame
+        An instance of DataFrame with file content
+    """
+
+    df = pd.read_csv(dxfile, sep=sep)
+
+    if include_fname:
+        fname = dxfile.name
+        df.insert(0, "file_name", fname)
+
+    return df
+
+
+def _add_extra_columns(
+    worksheet: openpyxl.Worksheet,
+    extra_cols: dict[str, str]
+    ) -> None:
+    """
+    Inserts additional columns with formulas at the beginning of a sheet
+
+    Parameters
+    ----------
+    worksheet : openpyxl.Worksheet
+        The worksheet where columns will be added.
+    extra_cols : dict[str, str]
+        A mapping of column names to Excel formulas
+    
+    Returns
+    -------
+    None
+    """
+
+    num_cols = len(extra_cols)
+    worksheet.insert_cols(1, amount=num_cols)
+
+    for i, (col, formula) in enumerate(extra_cols.items(), start=1):
+        worksheet.cell(row=1, column=i, value=col)
+        for row in range(2, worksheet.max_row + 1):
+            worksheet.cell(
+                row=row, column=i, value=formula.replace("{row}", str(row))
+            )
+
+
+def _apply_header_format(worksheet) -> None:
+    """
+    Applies bold formatting to all headers in the sheet.
+
+    Parameters
+    ----------
+    worksheet : openpyxl.Worksheet
+        The worksheet where header formatting will be applied.
+
+    Returns
+    -------
+    None
+    """
+    header_font = Font(bold=True)
+    for col in range(1, worksheet.max_column + 1):
+        cell = worksheet.cell(row=1, column=col)
+        cell.font = header_font
+
+
+def _adjust_column_widths(worksheet):
+    """
+    Adjusts column widths for better visibility.
+
+    Parameters
+    ----------
+    worksheet : openpyxl.Worksheet
+        The worksheet where column widths will be adjusted.
+
+    Returns
+    -------
+    None
+    """
+    for cell in worksheet[1]:  # Iterate over header row
+        col_letter = cell.column_letter
+        worksheet.column_dimensions[col_letter].width = max(
+            len(str(cell.value)) + 2, 10
+        )
+
+
+def write_df_to_sheet(
+    writer: pd.ExcelWriter, 
+    df: pd.DataFrame, 
+    sheet_name: str, 
+    tab_color: str = "000000",
+    extra_cols: dict[str, str] = None
+    ) -> None:
+    """Writes a Pandas DataFrame to an Excel sheet with formatting.
+
+    Parameters
+    ----------
+    writer : pd.ExcelWriter
+        The Excel writer object to write the sheet into
+    df : pd.DataFrame
+        The DataFrame containing the data.
+    sheet_name : str
+        Name of the Excel sheet
+    tab_color : str, optional
+        Hex colour code for the sheet tab. Defaults to black ("000000")
+    extra_cols : dict[str, str], optional
+        A mapping of new column names to excel formulas. Example:
+            {
+                "Specimen": "=MID(C{row},11,10)",
+                "EPIC": "=VLOOKUP(A{row},EPIC!AJ:AK,2,0)"
+            }
+    """
+    df.to_excel(writer, sheet_name=sheet_name, index=False)
+    worksheet = writer.sheets[sheet_name]
+    worksheet.sheet_properties.tabColor = tab_color
+    
+    # Add extra columns if provided
+    if extra_cols:
+        _add_extra_columns(worksheet, extra_cols)
+
+    _apply_header_format(worksheet)
+    _adjust_column_widths(worksheet)
+
+
+def get_project_info() -> tuple[str, str]:
+    """Get the project name for naming output file
+
+    Returns
+    -------
+    tuple[str, str]
+        Name and ID of DNAnexus project
+    """
+    
+    project_id = os.environ.get("DX_PROJECT_CONTEXT_ID")
+
+    # Get name of project for output naming
+    project_name = dxpy.describe(project_id)["name"]
+    project_name = "_".join(project_name.split("_")[1:])
+
+    return project_name, project_id
path: resources/home/dnanexus/utils/utils.py
user: login: coderabbitai[bot]
body: _⚠️ Potential issue_

**Add missing import and optimise function logic**

The function uses `os.environ` but the `os` module is not imported. Additionally, the docstring could be improved to reflect that the function returns both project name and ID.



```diff
+import os
+
 def get_project_info() -> tuple[str, str]:
-    """Get the project name for naming output file
+    """Get the project name and ID from DNAnexus environment
 
     Returns
     -------
     tuple[str, str]
         Name and ID of DNAnexus project
     """
     
     project_id = os.environ.get("DX_PROJECT_CONTEXT_ID")
+    if not project_id:
+        raise ValueError("DX_PROJECT_CONTEXT_ID environment variable not found")
 
     # Get name of project for output naming
     project_name = dxpy.describe(project_id)["name"]
     project_name = "_".join(project_name.split("_")[1:])
 
     return project_name, project_id
```


<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

`````suggestion
import os

def get_project_info() -> tuple[str, str]:
    """Get the project name and ID from DNAnexus environment

    Returns
    -------
    tuple[str, str]
        Name and ID of DNAnexus project
    """
    
    project_id = os.environ.get("DX_PROJECT_CONTEXT_ID")
    if not project_id:
        raise ValueError("DX_PROJECT_CONTEXT_ID environment variable not found")
    
    # Get name of project for output naming
    project_name = dxpy.describe(project_id)["name"]
    project_name = "_".join(project_name.split("_")[1:])
    
    return project_name, project_id
`````

</details>

<!-- suggestion_end -->

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

158-158: Undefined name `os`

(F821)

</details>

</details>

<!-- This is an auto-generated comment by CodeRabbit -->

✅ Addressed in commits fdff6db to 3ab0b63
created_at: 2025-03-12T11:33:02Z
author_association: NONE
diff_hunk: @@ -0,0 +1,123 @@
+"""Module for parsing fusion data from their respective DNAnexus files into a
+pandas data frame
+"""
+from concurrent.futures import ThreadPoolExecutor
+import dxpy
+import pandas as pd
+from dxpy import DXDataObject
+from typing import List
+
+from .utils import read_dxfile
+
+
+def parse_fastqc(dxfile: DXDataObject) -> pd.DataFrame:
+    """Parse the content of fastqc multiqc output and compute required metrics from deduplicated_percentage
+
+    Parameters
+    ----------
+    dxfile : DXDataObject
+        DNAnexus file object containing fastqc data to parse
+
+    Returns
+    -------
+    pd.DataFrame
+        pandas dataframe containing selected and computed fastqc metrics
+
+    Raises
+    ------
+    ValueError
+        if columns needed to compute extra metrics are absent.
+    """
+
+    df = read_dxfile(dxfile, include_fname=False)
+
+    # Verify required columns exist
+    required_cols = ["total_deduplicated_percentage", "Total Sequences"]
+    missing_cols = [col for col in required_cols if col not in df.columns]
+    if missing_cols:
+        raise ValueError(
+            f"Required columns missing from FastQC data: {missing_cols}"
+        )
+
+    df["Unique Reads"] = (
+        (df["total_deduplicated_percentage"] / 100.0) * df["Total Sequences"]
+    ).astype(int)
+    df["Duplicate Reads"] = (
+        df["Total Sequences"] - df["Unique Reads"]
+    ).astype(int)
+    df["Unique Reads(M)"] = df["Unique Reads"] / 1000000
+    df["Duplicate Reads(M)"] = df["Duplicate Reads"] / 1000000
+    df = df[[
+        "Sample",
+        "Unique Reads",
+        "Duplicate Reads",
+        "Unique Reads(M)",
+        "Duplicate Reads(M)"
+    ]]
+
+    return df
+
+
+def _parse_fusion_files(dxfiles: List[DXDataObject]) -> pd.DataFrame:
+    """
+    Reads and concatenates a list of DNAnexus fusion-related files 
+    into a single DataFrame.
+
+    Parameters
+    ----------
+    dxfiles : List[DXDataObject]
+        A list of DNAnexus file objects.
+
+    Returns
+    -------
+    pd.DataFrame
+        A concatenated DataFrame containing the combined data 
+        from all input files.
+    """
+    if not dxfiles:
+        return pd.DataFrame()
+    
+    max_workers = min(16, len(dxfiles))
+    with ThreadPoolExecutor(max_workers=max_workers) as executor:
+        df = pd.concat(executor.map(read_dxfile, dxfiles))
+
path: resources/home/dnanexus/utils/parser.py
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Add error handling for ThreadPoolExecutor operations**

The current implementation doesn't handle potential exceptions that might occur during parallel file processing, which could lead to silent failures.



```diff
     max_workers = min(16, len(dxfiles))
     with ThreadPoolExecutor(max_workers=max_workers) as executor:
-        df = pd.concat(executor.map(read_dxfile, dxfiles))
+        # Use a more robust approach with error handling
+        futures = [executor.submit(read_dxfile, dxfile) for dxfile in dxfiles]
+        results = []
+        for future in futures:
+            try:
+                results.append(future.result())
+            except Exception as e:
+                # Log the error and continue with other files
+                print(f"Error processing file: {e}")
+        
+        if not results:
+            return pd.DataFrame()
+        
+        df = pd.concat(results)
```


<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

`````suggestion
    max_workers = min(16, len(dxfiles))
    with ThreadPoolExecutor(max_workers=max_workers) as executor:
        # Use a more robust approach with error handling
        futures = [executor.submit(read_dxfile, dxfile) for dxfile in dxfiles]
        results = []
        for future in futures:
            try:
                results.append(future.result())
            except Exception as e:
                # Log the error and continue with other files
                print(f"Error processing file: {e}")
        
        if not results:
            return pd.DataFrame()
        
        df = pd.concat(results)
`````

</details>

<!-- suggestion_end -->

<!-- This is an auto-generated reply by CodeRabbit -->

✅ Addressed in commit b736292
created_at: 2025-03-12T11:33:02Z
author_association: NONE
diff_hunk: @@ -0,0 +1,58 @@
+"""
+Module containing default constants for sheet configurations
+Formulas adapted from demo WB provided by scientist
+"""
+
+FASTQC_SHEET_CONFIG = {
+    "sheet_name": "FastQC",
+    "tab_color": "008000",  # Green
+    "extra_cols": {
+        "Specimen": "=MID(C{row},11,10)",
+        "EPIC": "=VLOOKUP(A{row},EPIC!AJ:AK,2,0)",
+    },
+}
+
+SF_SHEET_CONFIG = {
+    "sheet_name": "STAR-Fusion",
+    "tab_color": "9400D3",  # Red
+    "extra_cols": {
+        "SPECIMEN": "=MID(K{row},11,10)",
+        "FNAME": "=LEFT(K{row},28)",
+        "Count_Run_1_Run_20_predicted": "=VLOOKUP(L{row},PC1_PC20_predicted_S!E:F,2,0)",
+        "EPIC": "=VLOOKUP(A{row},EPIC!AJ:AK,2,0)",
+        "DAYS COUNT": "=VLOOKUP(A{row},EPIC!AJ:AL,3,0)",
+        "Unique Reads (M) PC3B": "=VLOOKUP(A{row},MultiQC_Pivot!A:C,3,0)",
+        "Duplicate Reads (M) PC3B": "=VLOOKUP(A{row},MultiQC_Pivot!A:D,4,0)",
+        "ID": '=CONCAT(A{row},"_",L{row})',
+        "LEFTRIGHT": '=CONCATENATE(S{row},"_",U{row})',
+        "FRAME": "=VLOOKUP(I{row},PC4_MERGED!C:AM,32,0)",
+    },
+}
path: resources/home/dnanexus/utils/defaults.py
user: login: coderabbitai[bot]
body: _💡 Verification agent_

<details>
<summary>🧩 Analysis chain</summary>

**Verify Excel formula dependencies between sheets**

The formulas reference other sheets (e.g., "EPIC!AJ:AK"). Ensure these sheets exist and have the referenced columns when the workbook is generated.

---

🏁 Script executed:

```shell
#!/bin/bash
# Check if all referenced sheets in formulas are created in the main script
echo "Checking sheet references in formulas:"
rg -p "\"sheet_name\": \"([^\"]+)\"" resources/home/dnanexus/utils/defaults.py | grep -o '"[^"]*"$' | tr -d '"' > /tmp/defined_sheets.txt
rg -p "![A-Z0-9]+:" resources/home/dnanexus/utils/defaults.py | grep -o '![^,:]*' | tr -d '!' > /tmp/referenced_sheets.txt
echo "Sheets defined in configs:"
cat /tmp/defined_sheets.txt
echo "Sheets referenced in formulas:"
cat /tmp/referenced_sheets.txt
echo "Sheets referenced but not defined:"
grep -v -f /tmp/defined_sheets.txt /tmp/referenced_sheets.txt || echo "All referenced sheets are defined!"
```

Length of output: 665

---



</details>

**Action: Revise the verification logic for Excel sheet references**

Our current verification script is mistakenly capturing column designators from the cell ranges (e.g. “AJ”, “E”, “C”) rather than the actual sheet names referenced in the formulas. In the code snippet (lines 15–30 of `resources/home/dnanexus/utils/defaults.py`), the formulas reference several sheets (such as `PC1_PC20_predicted_S`, `EPIC`, `MultiQC_Pivot`, and `PC4_MERGED`), but the extraction method used does not correctly isolate these names. 

Please update the regex or verification approach so it captures the complete sheet name preceding the exclamation mark (for example, using a pattern that matches `[A-Za-z0-9_]+` before the `!`). Once adjusted, ensure that:

- All sheets referenced in the formulas exist in the workbook generation process.
- The referenced columns within these sheets are correctly defined.

<!-- This is an auto-generated reply by CodeRabbit -->

✅ Addressed in commits fdff6db to 3ab0b63

<!-- This is an auto-generated comment by CodeRabbit -->
created_at: 2025-03-12T16:40:20Z
author_association: NONE
diff_hunk: @@ -0,0 +1,104 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+import os
+from glob import glob
+from typing import List
+
+import dxpy
+import pip
+
+pip.main(["install", "--no-index", "--no-deps", *glob("packages/*")])
+
+import openpyxl
+import pandas as pd
+from utils.defaults import (
+    EPIC_SHEET_CONFIG,
+    FASTQC_SHEET_CONFIG,
+    FI_PREVIOUS_RUNS_SHEET_CONFIG,
+    FI_SHEET_CONFIG,
+    SF_PREVIOUS_RUNS_SHEET_CONFIG,
+    SF_SHEET_CONFIG,
+)
+from utils.parser import parse_fastqc, parse_fusion_inspector, parse_star_fusion
+from utils.utils import get_project_info, read_dxfile, write_df_to_sheet
+
+
+@dxpy.entry_point("main")
+def main(
+    starfusion_files: List,
+    fusioninspector_files: List,
+    fastqc_data: str,
+    SF_previous_runs_data: str,
+    FI_previous_runs_data: str,
+):
+    """_summary_
+
+    Parameters
+    ----------
+    starfusion_files : List
+        _description_
+    fusioninspector_files : List
+        _description_
+    fastqc_data : str
+        _description_
+    SF_previous_runs_data : str
+        _description_
+    FI_previous_runs_data : str
+        _description_
+
+    Returns
+    -------
+    _type_
+        _description_
+    """
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Complete the placeholder docstring with proper documentation**

Replace the placeholder docstring with actual descriptions of parameters and return values.

```diff
-    """_summary_
-
-    Parameters
-    ----------
-    starfusion_files : List
-        _description_
-    fusioninspector_files : List
-        _description_
-    fastqc_data : str
-        _description_
-    SF_previous_runs_data : str
-        _description_
-    FI_previous_runs_data : str
-        _description_
-
-    Returns
-    -------
-    _type_
-        _description_
-    """
+    """Generate a fusion workbook from various genomic data inputs.
+
+    Parameters
+    ----------
+    starfusion_files : List
+        List of STAR-Fusion prediction file IDs
+    fusioninspector_files : List
+        List of FusionInspector prediction file IDs
+    fastqc_data : str
+        FastQC output file ID
+    SF_previous_runs_data : str
+        Previous STAR-Fusion runs data file ID
+    FI_previous_runs_data : str
+        Previous FusionInspector runs data file ID
+
+    Returns
+    -------
+    dict
+        Dictionary containing link to the generated fusion workbook
+    """
```


<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

`````suggestion
@dxpy.entry_point("main")
def main(
    starfusion_files: List,
    fusioninspector_files: List,
    fastqc_data: str,
    SF_previous_runs_data: str,
    FI_previous_runs_data: str,
):
    """Generate a fusion workbook from various genomic data inputs.

    Parameters
    ----------
    starfusion_files : List
        List of STAR-Fusion prediction file IDs
    fusioninspector_files : List
        List of FusionInspector prediction file IDs
    fastqc_data : str
        FastQC output file ID
    SF_previous_runs_data : str
        Previous STAR-Fusion runs data file ID
    FI_previous_runs_data : str
        Previous FusionInspector runs data file ID

    Returns
    -------
    dict
        Dictionary containing link to the generated fusion workbook
    """
`````

</details>

<!-- suggestion_end -->

<!-- This is an auto-generated reply by CodeRabbit -->

✅ Addressed in commit b736292
created_at: 2025-03-12T16:40:20Z
author_association: NONE
diff_hunk: @@ -0,0 +1,103 @@
+{
+    "name": "eggd_generate_fusion_workbook",
+    "title": "eggd_generate_fusion_workbook",
+    "summary": "App for fusion workbook",
+    "dxapi": "1.0.0",
+    "version": "1.0.0",
+    "inputSpec": [
+      {
+        "name": "starfusion_predictions",
+        "label": "starfusion_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
path: dxapp.json
user: login: pofuoku
body: updated metadata as suggested
<!-- Reviewable comment -OL3s8vn1k4JgjT-pFqC:b-vwvkp0 -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-18T13:26:18Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,103 @@
+{
+    "name": "eggd_generate_fusion_workbook",
+    "title": "eggd_generate_fusion_workbook",
+    "summary": "App for fusion workbook",
+    "dxapi": "1.0.0",
+    "version": "1.0.0",
+    "inputSpec": [
+      {
+        "name": "starfusion_predictions",
+        "label": "starfusion_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "fusioninspector_predictions",
+        "label": "fusioninspector_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
path: dxapp.json
user: login: pofuoku
body: add help to inputspec as suggested
<!-- Reviewable comment -OL3sFI_53C1Y4nbXwOS:b-g6dohy -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-18T13:26:19Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,103 @@
+{
+    "name": "eggd_generate_fusion_workbook",
+    "title": "eggd_generate_fusion_workbook",
+    "summary": "App for fusion workbook",
+    "dxapi": "1.0.0",
+    "version": "1.0.0",
+    "inputSpec": [
+      {
+        "name": "starfusion_predictions",
+        "label": "starfusion_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "fusioninspector_predictions",
+        "label": "fusioninspector_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "fastqc_output",
+        "label": "fastqc_output",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
path: dxapp.json
user: login: pofuoku
body: Done.
<!-- Reviewable comment -OL3sSJ6E7PDnGnvkjwi:b-896fix -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-18T13:26:19Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,103 @@
+{
+    "name": "eggd_generate_fusion_workbook",
+    "title": "eggd_generate_fusion_workbook",
+    "summary": "App for fusion workbook",
+    "dxapi": "1.0.0",
+    "version": "1.0.0",
+    "inputSpec": [
+      {
+        "name": "starfusion_predictions",
+        "label": "starfusion_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "fusioninspector_predictions",
+        "label": "fusioninspector_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "fastqc_output",
+        "label": "fastqc_output",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "PC1_PC20_predicted_S",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
path: dxapp.json
user: login: pofuoku
body: PC1 is not relatated to PCA. I reckon it's an apprevation for PANCAN run. Added help to clarify this.
<!-- Reviewable comment -OL44F78Aonmfr-NpqBE:b-lpvyqw -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-18T13:26:19Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,103 @@
+{
+    "name": "eggd_generate_fusion_workbook",
+    "title": "eggd_generate_fusion_workbook",
+    "summary": "App for fusion workbook",
+    "dxapi": "1.0.0",
+    "version": "1.0.0",
+    "inputSpec": [
+      {
+        "name": "starfusion_predictions",
+        "label": "starfusion_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "fusioninspector_predictions",
+        "label": "fusioninspector_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "fastqc_output",
+        "label": "fastqc_output",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "PC1_PC20_predicted_S",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "PC1_PC20_merged_S",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
path: dxapp.json
user: login: pofuoku
body: same as above. This is not related to PCA
<!-- Reviewable comment -OL44cjRERjwUPQ9lnun:b200frp -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-18T13:26:20Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,103 @@
+{
+    "name": "eggd_generate_fusion_workbook",
+    "title": "eggd_generate_fusion_workbook",
+    "summary": "App for fusion workbook",
+    "dxapi": "1.0.0",
+    "version": "1.0.0",
+    "inputSpec": [
+      {
+        "name": "starfusion_predictions",
+        "label": "starfusion_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "fusioninspector_predictions",
+        "label": "fusioninspector_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "fastqc_output",
+        "label": "fastqc_output",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "PC1_PC20_predicted_S",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "PC1_PC20_merged_S",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "epic",
+        "class": "file",
+        "optional": true,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      }
+    ],
+    "outputSpec": [
+      {
+        "name": "fusion_workbook",
+        "label": "fusion_workbook",
+        "class": "file",
+        "patterns": [
+          "*"
+        ],
+        "help": ""
path: dxapp.json
user: login: pofuoku
body: Done.
<!-- Reviewable comment -OL44w4X8NpKJr_tE-tK:b-896fix -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-18T13:26:20Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+
+import os
+import pip
+from glob import glob
+import dxpy
+
+from typing import List
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: pofuoku
body: Done.
<!-- Reviewable comment -OLdQRTC8RZCvlZB_LOb:b-896fix -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-18T13:26:20Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+
+import os
+import pip
+from glob import glob
+import dxpy
+
+from typing import List
+
+
+pip.main(["install", "--no-index", "--no-deps", *glob("packages/*")])
+
+import pandas as pd
+import openpyxl 
+
+from utils.utils import write_df_to_sheet
+from utils.fastqc import parse_fastqc
+from utils.fusion_inspector import parse_fusion_inspector, FUSION_INSPECTOR_EXTRA_COLS
+from utils.star_fusion import parse_star_fusion, STAR_FUSION_EXTRA_COLS
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: pofuoku
body: Done.
<!-- Reviewable comment -OL4OrVt67VnxzcE7IHz:b-896fix -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-18T13:26:21Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+
+import os
+import pip
+from glob import glob
+import dxpy
+
+from typing import List
+
+
+pip.main(["install", "--no-index", "--no-deps", *glob("packages/*")])
+
+import pandas as pd
+import openpyxl 
+
+from utils.utils import write_df_to_sheet
+from utils.fastqc import parse_fastqc
+from utils.fusion_inspector import parse_fusion_inspector, FUSION_INSPECTOR_EXTRA_COLS
+from utils.star_fusion import parse_star_fusion, STAR_FUSION_EXTRA_COLS
+
+@dxpy.entry_point('main')
+def main(
+    starfusion_predictions,
+    fusioninspector_predictions,
+    fastqc_output,
+    PC1_PC20_predicted_S,
+    PC1_PC20_merged_S,
+    epic=None
+):
+    """
+    main
+    """
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: pofuoku
body: Done.
<!-- Reviewable comment -OLdQUkW3PfNOVGMVlMd:b-896fix -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-18T13:26:21Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+
+import os
+import pip
+from glob import glob
+import dxpy
+
+from typing import List
+
+
+pip.main(["install", "--no-index", "--no-deps", *glob("packages/*")])
+
+import pandas as pd
+import openpyxl 
+
+from utils.utils import write_df_to_sheet
+from utils.fastqc import parse_fastqc
+from utils.fusion_inspector import parse_fusion_inspector, FUSION_INSPECTOR_EXTRA_COLS
+from utils.star_fusion import parse_star_fusion, STAR_FUSION_EXTRA_COLS
+
+@dxpy.entry_point('main')
+def main(
+    starfusion_predictions,
+    fusioninspector_predictions,
+    fastqc_output,
+    PC1_PC20_predicted_S,
+    PC1_PC20_merged_S,
+    epic=None
+):
+    """
+    main
+    """
+
+    # Initialize inputs into dxpy.DXDataObject instances
+
+    starfusion_predictions = [
+        dxpy.DXFile(item) for item in starfusion_predictions
+    ]
+    fusioninspector_predictions = [
+        dxpy.DXFile(item) for item in fusioninspector_predictions
+    ]
+    fastqc_output = dxpy.DXFile(fastqc_output)
+    PC1_PC20_predicted_S = dxpy.DXFile(PC1_PC20_predicted_S)
+    PC1_PC20_merged_S = dxpy.DXFile(PC1_PC20_merged_S)
+    if epic is not None:
+        epic = dxpy.DXFile(epic)
+
+    
+    df_star_fusion = parse_star_fusion(starfusion_predictions)
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: pofuoku
body: Done.
<!-- Reviewable comment -OL4Ozdz2qCJhfyrxijn:b-896fix -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-18T13:26:21Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+
+import os
+import pip
+from glob import glob
+import dxpy
+
+from typing import List
+
+
+pip.main(["install", "--no-index", "--no-deps", *glob("packages/*")])
+
+import pandas as pd
+import openpyxl 
+
+from utils.utils import write_df_to_sheet
+from utils.fastqc import parse_fastqc
+from utils.fusion_inspector import parse_fusion_inspector, FUSION_INSPECTOR_EXTRA_COLS
+from utils.star_fusion import parse_star_fusion, STAR_FUSION_EXTRA_COLS
+
+@dxpy.entry_point('main')
+def main(
+    starfusion_predictions,
+    fusioninspector_predictions,
+    fastqc_output,
+    PC1_PC20_predicted_S,
+    PC1_PC20_merged_S,
+    epic=None
+):
+    """
+    main
+    """
+
+    # Initialize inputs into dxpy.DXDataObject instances
+
+    starfusion_predictions = [
+        dxpy.DXFile(item) for item in starfusion_predictions
+    ]
+    fusioninspector_predictions = [
+        dxpy.DXFile(item) for item in fusioninspector_predictions
+    ]
+    fastqc_output = dxpy.DXFile(fastqc_output)
+    PC1_PC20_predicted_S = dxpy.DXFile(PC1_PC20_predicted_S)
+    PC1_PC20_merged_S = dxpy.DXFile(PC1_PC20_merged_S)
+    if epic is not None:
+        epic = dxpy.DXFile(epic)
+
+    
+    df_star_fusion = parse_star_fusion(starfusion_predictions)
+    
+    outfile = "output.xlsx"
+    
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: pofuoku
body: Done.
<!-- Reviewable comment -OL4BqRJ6cPk95dl68UM:b-896fix -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-18T13:26:21Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+
+import os
+import pip
+from glob import glob
+import dxpy
+
+from typing import List
+
+
+pip.main(["install", "--no-index", "--no-deps", *glob("packages/*")])
+
+import pandas as pd
+import openpyxl 
+
+from utils.utils import write_df_to_sheet
+from utils.fastqc import parse_fastqc
+from utils.fusion_inspector import parse_fusion_inspector, FUSION_INSPECTOR_EXTRA_COLS
+from utils.star_fusion import parse_star_fusion, STAR_FUSION_EXTRA_COLS
+
+@dxpy.entry_point('main')
+def main(
+    starfusion_predictions,
+    fusioninspector_predictions,
+    fastqc_output,
+    PC1_PC20_predicted_S,
+    PC1_PC20_merged_S,
+    epic=None
+):
+    """
+    main
+    """
+
+    # Initialize inputs into dxpy.DXDataObject instances
+
+    starfusion_predictions = [
+        dxpy.DXFile(item) for item in starfusion_predictions
+    ]
+    fusioninspector_predictions = [
+        dxpy.DXFile(item) for item in fusioninspector_predictions
+    ]
+    fastqc_output = dxpy.DXFile(fastqc_output)
+    PC1_PC20_predicted_S = dxpy.DXFile(PC1_PC20_predicted_S)
+    PC1_PC20_merged_S = dxpy.DXFile(PC1_PC20_merged_S)
+    if epic is not None:
+        epic = dxpy.DXFile(epic)
+
+    
+    df_star_fusion = parse_star_fusion(starfusion_predictions)
+    
+    outfile = "output.xlsx"
+    
+
+    with pd.ExcelWriter(outfile, engine="openpyxl") as writer:
+        write_df_to_sheet(
+            writer, 
+            df_star_fusion, 
+            sheet_name="STAR",
+            tab_color="FF0000",
+            extra_cols=STAR_FUSION_EXTRA_COLS
+        )
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: pofuoku
body: Done.
<!-- Reviewable comment -OL4P6344ftXBEKz0hUm:b-896fix -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-18T13:26:22Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,104 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+import os
+from glob import glob
+from typing import List
+
+import dxpy
+import pip
+
+pip.main(["install", "--no-index", "--no-deps", *glob("packages/*")])
+
+import openpyxl
+import pandas as pd
+from utils.defaults import (EPIC_SHEET_CONFIG, FASTQC_SHEET_CONFIG,
+                            FI_PREVIOUS_RUNS_SHEET_CONFIG, FI_SHEET_CONFIG,
+                            SF_PREVIOUS_RUNS_SHEET_CONFIG, SF_SHEET_CONFIG)
+from utils.parser import (parse_fastqc, parse_fusion_inspector,
+                          parse_star_fusion)
+from utils.utils import get_project_info, read_dxfile, write_df_to_sheet
+
+
+@dxpy.entry_point('main')
+def main(
+    starfusion_files: List,
+    fusioninspector_files: List,
+    fastqc_data: str,
+    SF_previous_runs_data: str,
+    FI_previous_runs_data: str,
+):
+    """_summary_
+
+    Parameters
+    ----------
+    starfusion_files : List
+        _description_
+    fusioninspector_files : List
+        _description_
+    fastqc_data : str
+        _description_
+    SF_previous_runs_data : str
+        _description_
+    FI_previous_runs_data : str
+        _description_
+
+    Returns
+    -------
+    _type_
+        _description_
+    """
+    # Initialize inputs into dxpy.DXDataObject instances
+    starfusion_files = [
+        dxpy.DXFile(item) for item in starfusion_files
+    ]
+    fusioninspector_files = [
+        dxpy.DXFile(item) for item in fusioninspector_files
+    ]
+    fastqc_data = dxpy.DXFile(fastqc_data)
+    sf_previous_data = dxpy.DXFile(SF_previous_runs_data)
+    fi_previous_data = dxpy.DXFile(FI_previous_runs_data)
+
+    df_starfusion = parse_star_fusion(starfusion_files)
+    df_fusioninspector = parse_fusion_inspector(fusioninspector_files)
+    df_fastqc = parse_fastqc(fastqc_data)
+    
+    df_sf_previous = read_dxfile(sf_previous_data)
+    df_fi_previous = read_dxfile(fi_previous_data)
+    
+    project_name, _ = get_project_info()
+    outfile = f"{project_name}_fusion_workbook.xlsx"
+
+    with pd.ExcelWriter(outfile, engine="openpyxl") as writer:
+    
+        # Writes empty EPIC sheet
+        write_df_to_sheet(writer, pd.DataFrame(), **EPIC_SHEET_CONFIG)
+
+        # Add SF previous runs data
+        write_df_to_sheet(writer, df_sf_previous, **SF_PREVIOUS_RUNS_SHEET_CONFIG)
+
+        # Add FI previous runs data
+        write_df_to_sheet(writer, df_fi_previous, **FI_PREVIOUS_RUNS_SHEET_CONFIG)
+
+        # Add FastQC data
+        write_df_to_sheet(writer, df_fastqc, **FASTQC_SHEET_CONFIG)
+
+        # Add STAR-Fusion data
+        write_df_to_sheet(writer, df_starfusion, **STAR_FUSION_SHEET_CONFIG)
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: pofuoku
body: Done.
<!-- Reviewable comment -OLdQZte37GXMVmPRZbH:b-896fix -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-18T13:26:22Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,104 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+import os
+from glob import glob
+from typing import List
+
+import dxpy
+import pip
+
+pip.main(["install", "--no-index", "--no-deps", *glob("packages/*")])
+
+import openpyxl
+import pandas as pd
+from utils.defaults import (
+    EPIC_SHEET_CONFIG,
+    FASTQC_SHEET_CONFIG,
+    FI_PREVIOUS_RUNS_SHEET_CONFIG,
+    FI_SHEET_CONFIG,
+    SF_PREVIOUS_RUNS_SHEET_CONFIG,
+    SF_SHEET_CONFIG,
+)
+from utils.parser import parse_fastqc, parse_fusion_inspector, parse_star_fusion
+from utils.utils import get_project_info, read_dxfile, write_df_to_sheet
+
+
+@dxpy.entry_point("main")
+def main(
+    starfusion_files: List,
+    fusioninspector_files: List,
+    fastqc_data: str,
+    SF_previous_runs_data: str,
+    FI_previous_runs_data: str,
+):
+    """_summary_
+
+    Parameters
+    ----------
+    starfusion_files : List
+        _description_
+    fusioninspector_files : List
+        _description_
+    fastqc_data : str
+        _description_
+    SF_previous_runs_data : str
+        _description_
+    FI_previous_runs_data : str
+        _description_
+
+    Returns
+    -------
+    _type_
+        _description_
+    """
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: pofuoku
body: Done.
<!-- Reviewable comment -OLdQW_XA6_5TzzKQtdT:b-896fix -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-18T13:26:22Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,58 @@
+"""
+Module containing default constants for sheet configurations
+Formulas adapted from demo WB provided by scientist
+"""
+
+FASTQC_SHEET_CONFIG = {
+    "sheet_name": "FastQC",
+    "tab_color": "008000",  # Green
+    "extra_cols": {
+        "Specimen": "=MID(C{row},11,10)",
+        "EPIC": "=VLOOKUP(A{row},EPIC!AJ:AK,2,0)",
+    },
+}
+
+SF_SHEET_CONFIG = {
+    "sheet_name": "STAR-Fusion",
+    "tab_color": "9400D3",  # Red
+    "extra_cols": {
+        "SPECIMEN": "=MID(K{row},11,10)",
+        "FNAME": "=LEFT(K{row},28)",
+        "Count_Run_1_Run_20_predicted": "=VLOOKUP(L{row},PC1_PC20_predicted_S!E:F,2,0)",
+        "EPIC": "=VLOOKUP(A{row},EPIC!AJ:AK,2,0)",
+        "DAYS COUNT": "=VLOOKUP(A{row},EPIC!AJ:AL,3,0)",
+        "Unique Reads (M) PC3B": "=VLOOKUP(A{row},MultiQC_Pivot!A:C,3,0)",
+        "Duplicate Reads (M) PC3B": "=VLOOKUP(A{row},MultiQC_Pivot!A:D,4,0)",
+        "ID": '=CONCAT(A{row},"_",L{row})',
+        "LEFTRIGHT": '=CONCATENATE(S{row},"_",U{row})',
+        "FRAME": "=VLOOKUP(I{row},PC4_MERGED!C:AM,32,0)",
+    },
+}
path: resources/home/dnanexus/utils/defaults.py
user: login: pofuoku
body: Done.
<!-- Reviewable comment -OLdQcV13AJD2ofov-b6:b-896fix -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-18T13:26:23Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,90 @@
+"""general and I/O utilities
+"""
+
+import pandas as pd
+import openpyxl
+from openpyxl.styles import PatternFill, Font
+import dxpy
+from typing import List
path: resources/home/dnanexus/utils/utils.py
user: login: pofuoku
body: Done.
<!-- Reviewable comment -OL4PQed9iE4CKhtS-US:b-896fix -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-18T13:26:23Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,35 @@
+"""collates total_deduplicated_percentage from multiqc_fastqc.txt
+"""
+import pandas as pd
+from dxpy import DXDataObject
+import dxpy
+from .utils import read_dxfile
+
+
+def parse_fastqc(dxfile: DXDataObject) -> pd.DataFrame:
+    
+    df = read_dxfile(dxfile, include_fname=False)
path: resources/home/dnanexus/utils/fastqc.py
user: login: pofuoku
body: Done.
<!-- Reviewable comment -OL4HPslE5l0rHGA82f0:b-896fix -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-18T13:26:23Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,35 @@
+"""collates total_deduplicated_percentage from multiqc_fastqc.txt
+"""
+import pandas as pd
+from dxpy import DXDataObject
+import dxpy
+from .utils import read_dxfile
+
+
+def parse_fastqc(dxfile: DXDataObject) -> pd.DataFrame:
+    
+    df = read_dxfile(dxfile, include_fname=False)
+    
+    df["Unique Reads"] = (
+        (df["total_deduplicated_percentage"] / 100.0) * df["Total Sequences"]
+    ).astype(int)
+    df["Duplicate Reads"] = (
+        df["Total Sequences"] - df["Unique Reads"]
+    ).astype(int)
+    df["Unique Reads(M)"] = df["Unique Reads"] / 1000000
+    df["Duplicate Reads(M)"] = df["Duplicate Reads"] / 1000000
+    df = df[[
+        "Sample",
+        "Unique Reads",
+        "Duplicate Reads",
+        "Unique Reads(M)",
+        "Duplicate Reads(M)"
+    ]]
path: resources/home/dnanexus/utils/fastqc.py
user: login: pofuoku
body: Done.
<!-- Reviewable comment -OL4JXx71Sj7-9dxDOSa:b-896fix -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-18T13:26:24Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,44 @@
+"""Module for parsing STAR-Fusion data from DNAnexus files.
+"""
+from concurrent.futures import ThreadPoolExecutor
+import pandas as pd
+from dxpy import DXDataObject
+import dxpy
+from typing import List
+
+from .utils import read_dxfile
+
+
+def parse_star_fusion(dxfiles: List[DXDataObject]) -> pd.DataFrame:
+    """ Reads and concatenate an array DNAnexus STAR-Fusion files 
+        into a single DataFrame.
+
+
+    Args:
+        dxfiles (List[DXDataObject]): A list of DNAnexus file objects
+
+    Returns:
+        pd.DataFrame:  A concatenated DataFrame containing the combined data 
+            from all input files.
+    """
+    
+    with ThreadPoolExecutor(max_workers=16) as executor:
+        df = pd.concat(
+            executor.map(read_dxfile, dxfiles)
+        )
+    return df
path: resources/home/dnanexus/utils/star_fusion.py
user: login: pofuoku
body: Done.
<!-- Reviewable comment -OL4POPx0tX5vPfIKHVY:b-896fix -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-18T13:26:24Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,35 @@
+"""collates total_deduplicated_percentage from multiqc_fastqc.txt
+"""
+import pandas as pd
+from dxpy import DXDataObject
+import dxpy
+from .utils import read_dxfile
+
+
+def parse_fastqc(dxfile: DXDataObject) -> pd.DataFrame:
+    
+    df = read_dxfile(dxfile, include_fname=False)
path: resources/home/dnanexus/utils/fastqc.py
user: login: coderabbitai[bot]
body: > Seems like the humans are having a chat. I'll hop back into my burrow for now. If you need me again, just tag @coderabbitai in a new comment, and I'll come hopping out!

<!-- This is an auto-generated mute comment by CodeRabbit -->
created_at: 2025-03-18T13:26:31Z
author_association: NONE
diff_hunk: @@ -0,0 +1,103 @@
+{
+    "name": "eggd_generate_fusion_workbook",
+    "title": "eggd_generate_fusion_workbook",
+    "summary": "App for fusion workbook",
+    "dxapi": "1.0.0",
+    "version": "1.0.0",
+    "inputSpec": [
+      {
+        "name": "starfusion_predictions",
+        "label": "starfusion_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
path: dxapp.json
user: login: coderabbitai[bot]
body: > Seems like the humans are having a chat. I'll hop back into my burrow for now. If you need me again, just tag @coderabbitai in a new comment, and I'll come hopping out!

<!-- This is an auto-generated mute comment by CodeRabbit -->
created_at: 2025-03-18T13:26:32Z
author_association: NONE
diff_hunk: @@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+
+import os
+import pip
+from glob import glob
+import dxpy
+
+from typing import List
+
+
+pip.main(["install", "--no-index", "--no-deps", *glob("packages/*")])
+
+import pandas as pd
+import openpyxl 
+
+from utils.utils import write_df_to_sheet
+from utils.fastqc import parse_fastqc
+from utils.fusion_inspector import parse_fusion_inspector, FUSION_INSPECTOR_EXTRA_COLS
+from utils.star_fusion import parse_star_fusion, STAR_FUSION_EXTRA_COLS
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: coderabbitai[bot]
body: > Seems like the humans are having a chat. I'll hop back into my burrow for now. If you need me again, just tag @coderabbitai in a new comment, and I'll come hopping out!

<!-- This is an auto-generated mute comment by CodeRabbit -->
created_at: 2025-03-18T13:26:32Z
author_association: NONE
diff_hunk: @@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+
+import os
+import pip
+from glob import glob
+import dxpy
+
+from typing import List
+
+
+pip.main(["install", "--no-index", "--no-deps", *glob("packages/*")])
+
+import pandas as pd
+import openpyxl 
+
+from utils.utils import write_df_to_sheet
+from utils.fastqc import parse_fastqc
+from utils.fusion_inspector import parse_fusion_inspector, FUSION_INSPECTOR_EXTRA_COLS
+from utils.star_fusion import parse_star_fusion, STAR_FUSION_EXTRA_COLS
+
+@dxpy.entry_point('main')
+def main(
+    starfusion_predictions,
+    fusioninspector_predictions,
+    fastqc_output,
+    PC1_PC20_predicted_S,
+    PC1_PC20_merged_S,
+    epic=None
+):
+    """
+    main
+    """
+
+    # Initialize inputs into dxpy.DXDataObject instances
+
+    starfusion_predictions = [
+        dxpy.DXFile(item) for item in starfusion_predictions
+    ]
+    fusioninspector_predictions = [
+        dxpy.DXFile(item) for item in fusioninspector_predictions
+    ]
+    fastqc_output = dxpy.DXFile(fastqc_output)
+    PC1_PC20_predicted_S = dxpy.DXFile(PC1_PC20_predicted_S)
+    PC1_PC20_merged_S = dxpy.DXFile(PC1_PC20_merged_S)
+    if epic is not None:
+        epic = dxpy.DXFile(epic)
+
+    
+    df_star_fusion = parse_star_fusion(starfusion_predictions)
+    
+    outfile = "output.xlsx"
+    
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: coderabbitai[bot]
body: > Seems like the humans are having a chat. I'll hop back into my burrow for now. If you need me again, just tag @coderabbitai in a new comment, and I'll come hopping out!

<!-- This is an auto-generated mute comment by CodeRabbit -->
created_at: 2025-03-18T13:26:33Z
author_association: NONE
diff_hunk: @@ -0,0 +1,58 @@
+"""
+Module containing default constants for sheet configurations
+Formulas adapted from demo WB provided by scientist
+"""
+
+FASTQC_SHEET_CONFIG = {
+    "sheet_name": "FastQC",
+    "tab_color": "008000",  # Green
+    "extra_cols": {
+        "Specimen": "=MID(C{row},11,10)",
+        "EPIC": "=VLOOKUP(A{row},EPIC!AJ:AK,2,0)",
+    },
+}
+
+SF_SHEET_CONFIG = {
+    "sheet_name": "STAR-Fusion",
+    "tab_color": "9400D3",  # Red
+    "extra_cols": {
+        "SPECIMEN": "=MID(K{row},11,10)",
+        "FNAME": "=LEFT(K{row},28)",
+        "Count_Run_1_Run_20_predicted": "=VLOOKUP(L{row},PC1_PC20_predicted_S!E:F,2,0)",
+        "EPIC": "=VLOOKUP(A{row},EPIC!AJ:AK,2,0)",
+        "DAYS COUNT": "=VLOOKUP(A{row},EPIC!AJ:AL,3,0)",
+        "Unique Reads (M) PC3B": "=VLOOKUP(A{row},MultiQC_Pivot!A:C,3,0)",
+        "Duplicate Reads (M) PC3B": "=VLOOKUP(A{row},MultiQC_Pivot!A:D,4,0)",
+        "ID": '=CONCAT(A{row},"_",L{row})',
+        "LEFTRIGHT": '=CONCATENATE(S{row},"_",U{row})',
+        "FRAME": "=VLOOKUP(I{row},PC4_MERGED!C:AM,32,0)",
+    },
+}
path: resources/home/dnanexus/utils/defaults.py
user: login: coderabbitai[bot]
body: ### Rate Limit Exceeded

@pofuoku have exceeded the limit for the number of chat messages per hour. Please wait **2 minutes and 23 seconds** before sending another message.

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-18T13:26:33Z
author_association: NONE
diff_hunk: @@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+
+import os
+import pip
+from glob import glob
+import dxpy
+
+from typing import List
+
+
+pip.main(["install", "--no-index", "--no-deps", *glob("packages/*")])
+
+import pandas as pd
+import openpyxl 
+
+from utils.utils import write_df_to_sheet
+from utils.fastqc import parse_fastqc
+from utils.fusion_inspector import parse_fusion_inspector, FUSION_INSPECTOR_EXTRA_COLS
+from utils.star_fusion import parse_star_fusion, STAR_FUSION_EXTRA_COLS
+
+@dxpy.entry_point('main')
+def main(
+    starfusion_predictions,
+    fusioninspector_predictions,
+    fastqc_output,
+    PC1_PC20_predicted_S,
+    PC1_PC20_merged_S,
+    epic=None
+):
+    """
+    main
+    """
+
+    # Initialize inputs into dxpy.DXDataObject instances
+
+    starfusion_predictions = [
+        dxpy.DXFile(item) for item in starfusion_predictions
+    ]
+    fusioninspector_predictions = [
+        dxpy.DXFile(item) for item in fusioninspector_predictions
+    ]
+    fastqc_output = dxpy.DXFile(fastqc_output)
+    PC1_PC20_predicted_S = dxpy.DXFile(PC1_PC20_predicted_S)
+    PC1_PC20_merged_S = dxpy.DXFile(PC1_PC20_merged_S)
+    if epic is not None:
+        epic = dxpy.DXFile(epic)
+
+    
+    df_star_fusion = parse_star_fusion(starfusion_predictions)
+    
+    outfile = "output.xlsx"
+    
+
+    with pd.ExcelWriter(outfile, engine="openpyxl") as writer:
+        write_df_to_sheet(
+            writer, 
+            df_star_fusion, 
+            sheet_name="STAR",
+            tab_color="FF0000",
+            extra_cols=STAR_FUSION_EXTRA_COLS
+        )
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: coderabbitai[bot]
body: ### Rate Limit Exceeded

@pofuoku have exceeded the limit for the number of chat messages per hour. Please wait **2 minutes and 22 seconds** before sending another message.

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-18T13:26:33Z
author_association: NONE
diff_hunk: @@ -0,0 +1,104 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+import os
+from glob import glob
+from typing import List
+
+import dxpy
+import pip
+
+pip.main(["install", "--no-index", "--no-deps", *glob("packages/*")])
+
+import openpyxl
+import pandas as pd
+from utils.defaults import (
+    EPIC_SHEET_CONFIG,
+    FASTQC_SHEET_CONFIG,
+    FI_PREVIOUS_RUNS_SHEET_CONFIG,
+    FI_SHEET_CONFIG,
+    SF_PREVIOUS_RUNS_SHEET_CONFIG,
+    SF_SHEET_CONFIG,
+)
+from utils.parser import parse_fastqc, parse_fusion_inspector, parse_star_fusion
+from utils.utils import get_project_info, read_dxfile, write_df_to_sheet
+
+
+@dxpy.entry_point("main")
+def main(
+    starfusion_files: List,
+    fusioninspector_files: List,
+    fastqc_data: str,
+    SF_previous_runs_data: str,
+    FI_previous_runs_data: str,
+):
+    """_summary_
+
+    Parameters
+    ----------
+    starfusion_files : List
+        _description_
+    fusioninspector_files : List
+        _description_
+    fastqc_data : str
+        _description_
+    SF_previous_runs_data : str
+        _description_
+    FI_previous_runs_data : str
+        _description_
+
+    Returns
+    -------
+    _type_
+        _description_
+    """
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: coderabbitai[bot]
body: ### Rate Limit Exceeded

@pofuoku have exceeded the limit for the number of chat messages per hour. Please wait **2 minutes and 22 seconds** before sending another message.

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-18T13:26:33Z
author_association: NONE
diff_hunk: @@ -0,0 +1,104 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+import os
+from glob import glob
+from typing import List
+
+import dxpy
+import pip
+
+pip.main(["install", "--no-index", "--no-deps", *glob("packages/*")])
+
+import openpyxl
+import pandas as pd
+from utils.defaults import (EPIC_SHEET_CONFIG, FASTQC_SHEET_CONFIG,
+                            FI_PREVIOUS_RUNS_SHEET_CONFIG, FI_SHEET_CONFIG,
+                            SF_PREVIOUS_RUNS_SHEET_CONFIG, SF_SHEET_CONFIG)
+from utils.parser import (parse_fastqc, parse_fusion_inspector,
+                          parse_star_fusion)
+from utils.utils import get_project_info, read_dxfile, write_df_to_sheet
+
+
+@dxpy.entry_point('main')
+def main(
+    starfusion_files: List,
+    fusioninspector_files: List,
+    fastqc_data: str,
+    SF_previous_runs_data: str,
+    FI_previous_runs_data: str,
+):
+    """_summary_
+
+    Parameters
+    ----------
+    starfusion_files : List
+        _description_
+    fusioninspector_files : List
+        _description_
+    fastqc_data : str
+        _description_
+    SF_previous_runs_data : str
+        _description_
+    FI_previous_runs_data : str
+        _description_
+
+    Returns
+    -------
+    _type_
+        _description_
+    """
+    # Initialize inputs into dxpy.DXDataObject instances
+    starfusion_files = [
+        dxpy.DXFile(item) for item in starfusion_files
+    ]
+    fusioninspector_files = [
+        dxpy.DXFile(item) for item in fusioninspector_files
+    ]
+    fastqc_data = dxpy.DXFile(fastqc_data)
+    sf_previous_data = dxpy.DXFile(SF_previous_runs_data)
+    fi_previous_data = dxpy.DXFile(FI_previous_runs_data)
+
+    df_starfusion = parse_star_fusion(starfusion_files)
+    df_fusioninspector = parse_fusion_inspector(fusioninspector_files)
+    df_fastqc = parse_fastqc(fastqc_data)
+    
+    df_sf_previous = read_dxfile(sf_previous_data)
+    df_fi_previous = read_dxfile(fi_previous_data)
+    
+    project_name, _ = get_project_info()
+    outfile = f"{project_name}_fusion_workbook.xlsx"
+
+    with pd.ExcelWriter(outfile, engine="openpyxl") as writer:
+    
+        # Writes empty EPIC sheet
+        write_df_to_sheet(writer, pd.DataFrame(), **EPIC_SHEET_CONFIG)
+
+        # Add SF previous runs data
+        write_df_to_sheet(writer, df_sf_previous, **SF_PREVIOUS_RUNS_SHEET_CONFIG)
+
+        # Add FI previous runs data
+        write_df_to_sheet(writer, df_fi_previous, **FI_PREVIOUS_RUNS_SHEET_CONFIG)
+
+        # Add FastQC data
+        write_df_to_sheet(writer, df_fastqc, **FASTQC_SHEET_CONFIG)
+
+        # Add STAR-Fusion data
+        write_df_to_sheet(writer, df_starfusion, **STAR_FUSION_SHEET_CONFIG)
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: coderabbitai[bot]
body: ### Rate Limit Exceeded

@pofuoku have exceeded the limit for the number of chat messages per hour. Please wait **2 minutes and 22 seconds** before sending another message.

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-18T13:26:34Z
author_association: NONE
diff_hunk: @@ -0,0 +1,90 @@
+"""general and I/O utilities
+"""
+
+import pandas as pd
+import openpyxl
+from openpyxl.styles import PatternFill, Font
+import dxpy
+from typing import List
path: resources/home/dnanexus/utils/utils.py
user: login: coderabbitai[bot]
body: ### Rate Limit Exceeded

@pofuoku have exceeded the limit for the number of chat messages per hour. Please wait **2 minutes and 21 seconds** before sending another message.

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-18T13:26:34Z
author_association: NONE
diff_hunk: @@ -0,0 +1,44 @@
+"""Module for parsing STAR-Fusion data from DNAnexus files.
+"""
+from concurrent.futures import ThreadPoolExecutor
+import pandas as pd
+from dxpy import DXDataObject
+import dxpy
+from typing import List
+
+from .utils import read_dxfile
+
+
+def parse_star_fusion(dxfiles: List[DXDataObject]) -> pd.DataFrame:
+    """ Reads and concatenate an array DNAnexus STAR-Fusion files 
+        into a single DataFrame.
+
+
+    Args:
+        dxfiles (List[DXDataObject]): A list of DNAnexus file objects
+
+    Returns:
+        pd.DataFrame:  A concatenated DataFrame containing the combined data 
+            from all input files.
+    """
+    
+    with ThreadPoolExecutor(max_workers=16) as executor:
+        df = pd.concat(
+            executor.map(read_dxfile, dxfiles)
+        )
+    return df
path: resources/home/dnanexus/utils/star_fusion.py
user: login: coderabbitai[bot]
body: @pofuoku Great work! I'm glad you've implemented the error handling and thread count optimization in the `parse_star_fusion` function. This will make the code more robust and efficient.



```
  (\_/)
  (^.^)
 >(___)< *happy coding*
```

---

<!-- This is an auto-generated comment: tweet message by coderabbit.ai -->

If you found this review helpful, would you consider giving us a shout-out on [X](https://twitter.com/intent/tweet?related=CodeRabbitAI&text=Great%20advice%20from%20%40coderabbitai!%20%F0%9F%9B%A0%EF%B8%8F%20Adding%20error%20handling%20and%20optimising%20thread%20count%20can%20significantly%20enhance%20performance%20and%20reliability.%20Don't%20miss%20out%20on%20these%20improvements!%20%F0%9F%9A%80&url=https%3A%2F%2Fgithub.com%2Feastgenomics%2Feggd_generate_fusion_workbook%2Fpull%2F1%23discussion_r1985426925)?
							Thank you for using CodeRabbit!

<!-- end of auto-generated comment: tweet message by coderabbit.ai -->

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-18T13:26:48Z
author_association: NONE
diff_hunk: @@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+
+import os
+import pip
+from glob import glob
+import dxpy
+
+from typing import List
+
+
+pip.main(["install", "--no-index", "--no-deps", *glob("packages/*")])
+
+import pandas as pd
+import openpyxl 
+
+from utils.utils import write_df_to_sheet
+from utils.fastqc import parse_fastqc
+from utils.fusion_inspector import parse_fusion_inspector, FUSION_INSPECTOR_EXTRA_COLS
+from utils.star_fusion import parse_star_fusion, STAR_FUSION_EXTRA_COLS
+
+@dxpy.entry_point('main')
+def main(
+    starfusion_predictions,
+    fusioninspector_predictions,
+    fastqc_output,
+    PC1_PC20_predicted_S,
+    PC1_PC20_merged_S,
+    epic=None
+):
+    """
+    main
+    """
+
+    # Initialize inputs into dxpy.DXDataObject instances
+
+    starfusion_predictions = [
+        dxpy.DXFile(item) for item in starfusion_predictions
+    ]
+    fusioninspector_predictions = [
+        dxpy.DXFile(item) for item in fusioninspector_predictions
+    ]
+    fastqc_output = dxpy.DXFile(fastqc_output)
+    PC1_PC20_predicted_S = dxpy.DXFile(PC1_PC20_predicted_S)
+    PC1_PC20_merged_S = dxpy.DXFile(PC1_PC20_merged_S)
+    if epic is not None:
+        epic = dxpy.DXFile(epic)
+
+    
+    df_star_fusion = parse_star_fusion(starfusion_predictions)
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: coderabbitai[bot]
body: @pofuoku Excellent. You've implemented the necessary code to process all input files. This ensures complete data handling in the fusion workbook generation pipeline.

---

<!-- This is an auto-generated comment: tweet message by coderabbit.ai -->

If you found this review helpful, would you consider giving us a shout-out on [X](https://twitter.com/intent/tweet?related=CodeRabbitAI&text=Great%20advice%20from%20%40coderabbitai!%20%F0%9F%9B%A0%EF%B8%8F%20Expanding%20input%20processing%20to%20include%20FusionInspector%20and%20FastQC%20ensures%20consistency%20and%20robustness%20in%20our%20data%20handling.%20Let's%20keep%20improving!%20%F0%9F%9A%80&url=https%3A%2F%2Fgithub.com%2Feastgenomics%2Feggd_generate_fusion_workbook%2Fpull%2F1%23discussion_r1985426959)?
							Thank you for using CodeRabbit!

<!-- end of auto-generated comment: tweet message by coderabbit.ai -->

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-18T13:26:49Z
author_association: NONE
diff_hunk: @@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+
+import os
+import pip
+from glob import glob
+import dxpy
+
+from typing import List
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: coderabbitai[bot]
body: @pofuoku Great! Thanks for removing those unused imports. The code is now cleaner and more maintainable.



(\(\
( -.-)
o_(")(")

---

<!-- This is an auto-generated comment: tweet message by coderabbit.ai -->

If you found this review helpful, would you consider giving us a shout-out on [X](https://twitter.com/intent/tweet?related=CodeRabbitAI&text=Great%20advice%20from%20%40coderabbitai!%20%F0%9F%9B%A0%EF%B8%8F%20Removing%20unused%20imports%20not%20only%20clarifies%20code%20but%20also%20reduces%20namespace%20confusion.%20Let's%20keep%20our%20code%20clean%20and%20efficient!%20%F0%9F%92%A1&url=https%3A%2F%2Fgithub.com%2Feastgenomics%2Feggd_generate_fusion_workbook%2Fpull%2F1%23discussion_r1985426954)?
							Thank you for using CodeRabbit!

<!-- end of auto-generated comment: tweet message by coderabbit.ai -->

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-18T13:26:49Z
author_association: NONE
diff_hunk: @@ -0,0 +1,35 @@
+"""collates total_deduplicated_percentage from multiqc_fastqc.txt
+"""
+import pandas as pd
+from dxpy import DXDataObject
+import dxpy
+from .utils import read_dxfile
+
+
+def parse_fastqc(dxfile: DXDataObject) -> pd.DataFrame:
+    
+    df = read_dxfile(dxfile, include_fname=False)
+    
+    df["Unique Reads"] = (
+        (df["total_deduplicated_percentage"] / 100.0) * df["Total Sequences"]
+    ).astype(int)
+    df["Duplicate Reads"] = (
+        df["Total Sequences"] - df["Unique Reads"]
+    ).astype(int)
+    df["Unique Reads(M)"] = df["Unique Reads"] / 1000000
+    df["Duplicate Reads(M)"] = df["Duplicate Reads"] / 1000000
+    df = df[[
+        "Sample",
+        "Unique Reads",
+        "Duplicate Reads",
+        "Unique Reads(M)",
+        "Duplicate Reads(M)"
+    ]]
path: resources/home/dnanexus/utils/fastqc.py
user: login: coderabbitai[bot]
body: @pofuoku Excellent job implementing the error handling! This makes the code much more robust by catching potential issues early with clear error messages. This type of defensive programming prevents harder-to-debug errors down the line.

---

<!-- This is an auto-generated comment: tweet message by coderabbit.ai -->

If you found this review helpful, would you consider giving us a shout-out on [X](https://twitter.com/intent/tweet?related=CodeRabbitAI&text=Just%20received%20a%20fantastic%20review%20from%20%40coderabbitai!%20%F0%9F%9B%A0%EF%B8%8F%20Adding%20error%20handling%20for%20missing%20columns%20makes%20our%20data%20calculations%20more%20robust.%20Don't%20overlook%20this%20crucial%20step!%20%F0%9F%9A%80&url=https%3A%2F%2Fgithub.com%2Feastgenomics%2Feggd_generate_fusion_workbook%2Fpull%2F1%23discussion_r1985426933)?
							Thank you for using CodeRabbit!

<!-- end of auto-generated comment: tweet message by coderabbit.ai -->

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-18T13:26:50Z
author_association: NONE
diff_hunk: @@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+
+import os
+import pip
+from glob import glob
+import dxpy
+
+from typing import List
+
+
+pip.main(["install", "--no-index", "--no-deps", *glob("packages/*")])
+
+import pandas as pd
+import openpyxl 
+
+from utils.utils import write_df_to_sheet
+from utils.fastqc import parse_fastqc
+from utils.fusion_inspector import parse_fusion_inspector, FUSION_INSPECTOR_EXTRA_COLS
+from utils.star_fusion import parse_star_fusion, STAR_FUSION_EXTRA_COLS
+
+@dxpy.entry_point('main')
+def main(
+    starfusion_predictions,
+    fusioninspector_predictions,
+    fastqc_output,
+    PC1_PC20_predicted_S,
+    PC1_PC20_merged_S,
+    epic=None
+):
+    """
+    main
+    """
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: coderabbitai[bot]
body: @pofuoku Excellent! The improved docstring will significantly enhance code readability and maintainability. Properly documented functions make future development and collaboration much more efficient.

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-18T13:26:50Z
author_association: NONE
diff_hunk: @@ -0,0 +1,103 @@
+{
+    "name": "eggd_generate_fusion_workbook",
+    "title": "eggd_generate_fusion_workbook",
+    "summary": "App for fusion workbook",
+    "dxapi": "1.0.0",
+    "version": "1.0.0",
+    "inputSpec": [
+      {
+        "name": "starfusion_predictions",
+        "label": "starfusion_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "fusioninspector_predictions",
+        "label": "fusioninspector_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "fastqc_output",
+        "label": "fastqc_output",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "PC1_PC20_predicted_S",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
path: dxapp.json
user: login: coderabbitai[bot]
body: @pofuoku Thanks for the correction. I misinterpreted the parameter name. Good job adding help text to clarify that PC1 refers to PANCAN run rather than Principal Component Analysis. This will make the code more maintainable and easier for others to understand.

---

<details>
<summary>✏️ Learnings added</summary>

```
Learnt from: pofuoku
PR: eastgenomics/eggd_generate_fusion_workbook#1
File: dxapp.json:0-0
Timestamp: 2025-03-18T13:26:49.747Z
Learning: In the eggd_generate_fusion_workbook app, the parameter "PC1_PC20_predicted_S" is not related to Principal Component Analysis. PC1 is an abbreviation for PANCAN run.
```

> Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure [review instructions](https://docs.coderabbit.ai/guides/review-instructions). You can manage existing learnings in the [app](https://app.coderabbit.ai/learnings).

</details>

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-18T13:26:51Z
author_association: NONE
diff_hunk: @@ -0,0 +1,103 @@
+{
+    "name": "eggd_generate_fusion_workbook",
+    "title": "eggd_generate_fusion_workbook",
+    "summary": "App for fusion workbook",
+    "dxapi": "1.0.0",
+    "version": "1.0.0",
+    "inputSpec": [
+      {
+        "name": "starfusion_predictions",
+        "label": "starfusion_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "fusioninspector_predictions",
+        "label": "fusioninspector_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "fastqc_output",
+        "label": "fastqc_output",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
path: dxapp.json
user: login: coderabbitai[bot]
body: @pofuoku Great! Adding descriptive help text makes the application more user-friendly and reduces confusion for anyone using the tool. Good job on improving the documentation.

---

<!-- This is an auto-generated comment: tweet message by coderabbit.ai -->

If you found this review helpful, would you consider giving us a shout-out on [X](https://twitter.com/intent/tweet?related=CodeRabbitAI&text=Just%20received%20a%20fantastic%20refactor%20suggestion%20from%20%40coderabbitai!%20%F0%9F%9B%A0%EF%B8%8F%20Adding%20clear%20descriptions%20to%20parameters%20boosts%20usability%20and%20clarity.%20Let's%20make%20our%20code%20better%20together!%20%F0%9F%9A%80&url=https%3A%2F%2Fgithub.com%2Feastgenomics%2Feggd_generate_fusion_workbook%2Fpull%2F1%23discussion_r1985426941)?
							Thank you for using CodeRabbit!

<!-- end of auto-generated comment: tweet message by coderabbit.ai -->

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-18T13:26:52Z
author_association: NONE
diff_hunk: @@ -0,0 +1,103 @@
+{
+    "name": "eggd_generate_fusion_workbook",
+    "title": "eggd_generate_fusion_workbook",
+    "summary": "App for fusion workbook",
+    "dxapi": "1.0.0",
+    "version": "1.0.0",
+    "inputSpec": [
+      {
+        "name": "starfusion_predictions",
+        "label": "starfusion_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "fusioninspector_predictions",
+        "label": "fusioninspector_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "fastqc_output",
+        "label": "fastqc_output",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "PC1_PC20_predicted_S",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "PC1_PC20_merged_S",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
path: dxapp.json
user: login: coderabbitai[bot]
body: @pofuoku I apologize for my mistake. I incorrectly assumed the "PC" in the parameter name referred to Principal Component Analysis. Could you clarify what the `PC1_PC20_merged_S` parameter actually represents in this fusion workbook context? This would help me provide a more accurate suggestion for the help description.

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-18T13:26:52Z
author_association: NONE
diff_hunk: @@ -0,0 +1,103 @@
+{
+    "name": "eggd_generate_fusion_workbook",
+    "title": "eggd_generate_fusion_workbook",
+    "summary": "App for fusion workbook",
+    "dxapi": "1.0.0",
+    "version": "1.0.0",
+    "inputSpec": [
+      {
+        "name": "starfusion_predictions",
+        "label": "starfusion_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "fusioninspector_predictions",
+        "label": "fusioninspector_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
path: dxapp.json
user: login: coderabbitai[bot]
body: @pofuoku Great! Adding proper help text to the input specifications improves usability and helps others understand the expected file format. This documentation will be valuable for anyone using the app in the future.

---

<!-- This is an auto-generated comment: tweet message by coderabbit.ai -->

If you found this review helpful, would you consider giving us a shout-out on [X](https://twitter.com/intent/tweet?related=CodeRabbitAI&text=Great%20advice%20from%20%40coderabbitai!%20%F0%9F%9B%A0%EF%B8%8F%20Adding%20clear%20descriptions%20to%20inputs%20enhances%20user%20experience%20and%20ensures%20consistency.%20Let's%20keep%20our%20documentation%20top-notch!%20%F0%9F%9A%80&url=https%3A%2F%2Fgithub.com%2Feastgenomics%2Feggd_generate_fusion_workbook%2Fpull%2F1%23discussion_r1985426945)?
							Thank you for using CodeRabbit!

<!-- end of auto-generated comment: tweet message by coderabbit.ai -->

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-18T13:26:52Z
author_association: NONE
diff_hunk: @@ -0,0 +1,103 @@
+{
+    "name": "eggd_generate_fusion_workbook",
+    "title": "eggd_generate_fusion_workbook",
+    "summary": "App for fusion workbook",
+    "dxapi": "1.0.0",
+    "version": "1.0.0",
+    "inputSpec": [
+      {
+        "name": "starfusion_predictions",
+        "label": "starfusion_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "fusioninspector_predictions",
+        "label": "fusioninspector_predictions",
+        "class": "array:file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "fastqc_output",
+        "label": "fastqc_output",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "PC1_PC20_predicted_S",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "PC1_PC20_merged_S",
+        "class": "file",
+        "optional": false,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      },
+      {
+        "name": "epic",
+        "class": "file",
+        "optional": true,
+        "patterns": [
+          "*"
+        ],
+        "help": ""
+      }
+    ],
+    "outputSpec": [
+      {
+        "name": "fusion_workbook",
+        "label": "fusion_workbook",
+        "class": "file",
+        "patterns": [
+          "*"
+        ],
+        "help": ""
path: dxapp.json
user: login: coderabbitai[bot]
body: @pofuoku Got it! Thanks for implementing the documentation improvement. This will make the output parameter more self-explanatory for users.

---

<!-- This is an auto-generated comment: tweet message by coderabbit.ai -->

If you found this review helpful, would you consider giving us a shout-out on [X](https://twitter.com/intent/tweet?related=CodeRabbitAI&text=Just%20received%20a%20fantastic%20refactor%20suggestion%20from%20%40coderabbitai!%20%F0%9F%9B%A0%EF%B8%8F%20Adding%20clear%20descriptions%20to%20output%20parameters%20enhances%20code%20clarity%20and%20usability.%20Let's%20make%20our%20documentation%20shine!%20%E2%9C%A8&url=https%3A%2F%2Fgithub.com%2Feastgenomics%2Feggd_generate_fusion_workbook%2Fpull%2F1%23discussion_r1985426948)?
							Thank you for using CodeRabbit!

<!-- end of auto-generated comment: tweet message by coderabbit.ai -->

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-18T13:26:53Z
author_association: NONE
diff_hunk: @@ -0,0 +1,123 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+import os
+from glob import glob
+import subprocess
+from typing import List
+
+if os.path.exists("/home/dnanexus"):
+    # running in DNAnexus
+    subprocess.check_call(
+        ["pip", "install", "--no-index", "--no-deps"] + glob("packages/*")
+    )
+
+import dxpy
+import openpyxl
+import pandas as pd
+from utils.defaults import (
+    EPIC_SHEET_CONFIG,
+    FASTQC_SHEET_CONFIG,
+    FI_PREVIOUS_RUNS_SHEET_CONFIG,
+    FI_SHEET_CONFIG,
+    SF_PREVIOUS_RUNS_SHEET_CONFIG,
+    SF_SHEET_CONFIG,
+    FASTQC_PIVOT_CONFIG,
+)
+from utils.parser import (
+    parse_fastqc,
+    parse_fusion_inspector,
+    parse_star_fusion,
+    make_fastqc_pivot
+)
+from utils.utils import (
+    get_project_info,
+    read_dxfile,
+    write_df_to_sheet,
+)
+
+
+@dxpy.entry_point("main")
+def main(
+    starfusion_files: List[dict],
+    fusioninspector_files: List[dict],
+    fastqc_data: dict,
+    SF_previous_runs_data: dict,
+    FI_previous_runs_data: dict,
+):
+    """Generates a fusion workbook with data from
+    STAR-Fusion, FusionInspector, and FastQC
+
+    Parameters
+    ----------
+    starfusion_files : List[dict]
+        List of dictionaries containing DXLinks to STAR-Fusion output files
+    fusioninspector_files : List[dict]
+        List of dictionaries containing DXLinks to FusionInspector output files
+    fastqc_data : dict
+        Mapping of DXLink to FastQC data file
+    SF_previous_runs_data : str
+       Mapping of DXLink to STAR-Fusion previous runs data file
+    FI_previous_runs_data : str
+        Mapping of DXLink to FusionInspector previous runs data file
+
+    Returns
+    -------
+    dict
+        Dictionary containing DXLink to the generated fusion workbook
+    """
+    # Initialize inputs into dxpy.DXDataObject instances
+    starfusion_files = [dxpy.DXFile(item) for item in starfusion_files]
+    fusioninspector_files = [dxpy.DXFile(item) for item in fusioninspector_files]
+    fastqc_data = dxpy.DXFile(fastqc_data)
+    sf_previous_data = dxpy.DXFile(SF_previous_runs_data)
+    fi_previous_data = dxpy.DXFile(FI_previous_runs_data)
+
+    df_starfusion = parse_star_fusion(starfusion_files)
+    df_fusioninspector = parse_fusion_inspector(fusioninspector_files)
+    df_fastqc = parse_fastqc(fastqc_data)
+
+    df_sf_previous = read_dxfile(sf_previous_data)
+    df_fi_previous = read_dxfile(fi_previous_data)
+
+    project_name, _ = get_project_info()
+    outfile = f"{project_name}_fusion_workbook.xlsx"
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Add error handling for project information retrieval**

The `get_project_info()` function may fail if the project ID environment variable is not set. Add error handling to catch this case.

```diff
-project_name, _ = get_project_info()
-outfile = f"{project_name}_fusion_workbook.xlsx"
+try:
+    project_name, _ = get_project_info()
+    outfile = f"{project_name}_fusion_workbook.xlsx"
+except Exception as e:
+    print(f"Warning: Could not get project info: {e}")
+    outfile = "fusion_workbook.xlsx"
```


<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

`````suggestion
try:
    project_name, _ = get_project_info()
    outfile = f"{project_name}_fusion_workbook.xlsx"
except Exception as e:
    print(f"Warning: Could not get project info: {e}")
    outfile = "fusion_workbook.xlsx"
`````

</details>

<!-- suggestion_end -->

<!-- This is an auto-generated reply by CodeRabbit -->

✅ Addressed in commits fdff6db to 3ab0b63
created_at: 2025-03-18T13:29:46Z
author_association: NONE
diff_hunk: @@ -0,0 +1,123 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+import os
+from glob import glob
+import subprocess
+from typing import List
+
+if os.path.exists("/home/dnanexus"):
+    # running in DNAnexus
+    subprocess.check_call(
+        ["pip", "install", "--no-index", "--no-deps"] + glob("packages/*")
+    )
+
+import dxpy
+import openpyxl
+import pandas as pd
+from utils.defaults import (
+    EPIC_SHEET_CONFIG,
+    FASTQC_SHEET_CONFIG,
+    FI_PREVIOUS_RUNS_SHEET_CONFIG,
+    FI_SHEET_CONFIG,
+    SF_PREVIOUS_RUNS_SHEET_CONFIG,
+    SF_SHEET_CONFIG,
+    FASTQC_PIVOT_CONFIG,
+)
+from utils.parser import (
+    parse_fastqc,
+    parse_fusion_inspector,
+    parse_star_fusion,
+    make_fastqc_pivot
+)
+from utils.utils import (
+    get_project_info,
+    read_dxfile,
+    write_df_to_sheet,
+)
+
+
+@dxpy.entry_point("main")
+def main(
+    starfusion_files: List[dict],
+    fusioninspector_files: List[dict],
+    fastqc_data: dict,
+    SF_previous_runs_data: dict,
+    FI_previous_runs_data: dict,
+):
+    """Generates a fusion workbook with data from
+    STAR-Fusion, FusionInspector, and FastQC
+
+    Parameters
+    ----------
+    starfusion_files : List[dict]
+        List of dictionaries containing DXLinks to STAR-Fusion output files
+    fusioninspector_files : List[dict]
+        List of dictionaries containing DXLinks to FusionInspector output files
+    fastqc_data : dict
+        Mapping of DXLink to FastQC data file
+    SF_previous_runs_data : str
+       Mapping of DXLink to STAR-Fusion previous runs data file
+    FI_previous_runs_data : str
+        Mapping of DXLink to FusionInspector previous runs data file
+
+    Returns
+    -------
+    dict
+        Dictionary containing DXLink to the generated fusion workbook
+    """
+    # Initialize inputs into dxpy.DXDataObject instances
+    starfusion_files = [dxpy.DXFile(item) for item in starfusion_files]
+    fusioninspector_files = [dxpy.DXFile(item) for item in fusioninspector_files]
+    fastqc_data = dxpy.DXFile(fastqc_data)
+    sf_previous_data = dxpy.DXFile(SF_previous_runs_data)
+    fi_previous_data = dxpy.DXFile(FI_previous_runs_data)
+
+    df_starfusion = parse_star_fusion(starfusion_files)
+    df_fusioninspector = parse_fusion_inspector(fusioninspector_files)
+    df_fastqc = parse_fastqc(fastqc_data)
+
+    df_sf_previous = read_dxfile(sf_previous_data)
+    df_fi_previous = read_dxfile(fi_previous_data)
+
+    project_name, _ = get_project_info()
+    outfile = f"{project_name}_fusion_workbook.xlsx"
+
+    with pd.ExcelWriter(outfile, engine="openpyxl") as writer:
+
+        # Writes empty EPIC sheet
+        write_df_to_sheet(writer, pd.DataFrame(), **EPIC_SHEET_CONFIG)
+
+        # Add SF previous runs data
+        write_df_to_sheet(writer, df_sf_previous, **SF_PREVIOUS_RUNS_SHEET_CONFIG)
+
+        # Add FI previous runs data
+        write_df_to_sheet(writer, df_fi_previous, **FI_PREVIOUS_RUNS_SHEET_CONFIG)
+
+        # Add FastQC data
+        write_df_to_sheet(writer, df_fastqc, **FASTQC_SHEET_CONFIG)
+        fastqc_pivot = make_fastqc_pivot(df_fastqc, FASTQC_PIVOT_CONFIG)
+        write_df_to_sheet(
+            writer,
+            fastqc_pivot,
+            sheet_name=FASTQC_PIVOT_CONFIG["sheet_name"],
+            tab_color=FASTQC_PIVOT_CONFIG["tab_color"],
+            include_index=True
+        )
+
+        # Add STAR-Fusion data
+        write_df_to_sheet(writer, df_starfusion, **SF_SHEET_CONFIG)
+
+        # Add Fusion Inspector data
+        write_df_to_sheet(writer, df_fusioninspector, **FI_SHEET_CONFIG)
+
+    fusion_workbook = dxpy.upload_local_file(outfile)
+
+    output = {}
+    output["fusion_workbook"] = dxpy.dxlink(fusion_workbook)
+
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Add error handling for file upload**

The file upload operation could fail due to network issues or permissions. Add error handling to manage this scenario gracefully.

```diff
-fusion_workbook = dxpy.upload_local_file(outfile)
-
-output = {}
-output["fusion_workbook"] = dxpy.dxlink(fusion_workbook)
+try:
+    fusion_workbook = dxpy.upload_local_file(outfile)
+    
+    output = {}
+    output["fusion_workbook"] = dxpy.dxlink(fusion_workbook)
+except dxpy.exceptions.DXError as e:
+    raise RuntimeError(f"Failed to upload fusion workbook: {e}")
```


<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

`````suggestion
    try:
        fusion_workbook = dxpy.upload_local_file(outfile)
        
        output = {}
        output["fusion_workbook"] = dxpy.dxlink(fusion_workbook)
    except dxpy.exceptions.DXError as e:
        raise RuntimeError(f"Failed to upload fusion workbook: {e}")
`````

</details>

<!-- suggestion_end -->

<!-- This is an auto-generated comment by CodeRabbit -->

✅ Addressed in commits fdff6db to 3ab0b63
created_at: 2025-03-18T13:29:46Z
author_association: NONE
diff_hunk: @@ -0,0 +1,123 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+import os
+from glob import glob
+import subprocess
+from typing import List
+
+if os.path.exists("/home/dnanexus"):
+    # running in DNAnexus
+    subprocess.check_call(
+        ["pip", "install", "--no-index", "--no-deps"] + glob("packages/*")
+    )
+
+import dxpy
+import openpyxl
+import pandas as pd
+from utils.defaults import (
+    EPIC_SHEET_CONFIG,
+    FASTQC_SHEET_CONFIG,
+    FI_PREVIOUS_RUNS_SHEET_CONFIG,
+    FI_SHEET_CONFIG,
+    SF_PREVIOUS_RUNS_SHEET_CONFIG,
+    SF_SHEET_CONFIG,
+    FASTQC_PIVOT_CONFIG,
+)
+from utils.parser import (
+    parse_fastqc,
+    parse_fusion_inspector,
+    parse_star_fusion,
+    make_fastqc_pivot
+)
+from utils.utils import (
+    get_project_info,
+    read_dxfile,
+    write_df_to_sheet,
+)
+
+
+@dxpy.entry_point("main")
+def main(
+    starfusion_files: List[dict],
+    fusioninspector_files: List[dict],
+    fastqc_data: dict,
+    SF_previous_runs_data: dict,
+    FI_previous_runs_data: dict,
+):
+    """Generates a fusion workbook with data from
+    STAR-Fusion, FusionInspector, and FastQC
+
+    Parameters
+    ----------
+    starfusion_files : List[dict]
+        List of dictionaries containing DXLinks to STAR-Fusion output files
+    fusioninspector_files : List[dict]
+        List of dictionaries containing DXLinks to FusionInspector output files
+    fastqc_data : dict
+        Mapping of DXLink to FastQC data file
+    SF_previous_runs_data : str
+       Mapping of DXLink to STAR-Fusion previous runs data file
+    FI_previous_runs_data : str
+        Mapping of DXLink to FusionInspector previous runs data file
+
+    Returns
+    -------
+    dict
+        Dictionary containing DXLink to the generated fusion workbook
+    """
+    # Initialize inputs into dxpy.DXDataObject instances
+    starfusion_files = [dxpy.DXFile(item) for item in starfusion_files]
+    fusioninspector_files = [dxpy.DXFile(item) for item in fusioninspector_files]
+    fastqc_data = dxpy.DXFile(fastqc_data)
+    sf_previous_data = dxpy.DXFile(SF_previous_runs_data)
+    fi_previous_data = dxpy.DXFile(FI_previous_runs_data)
+
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Add input validation for DXFile objects**

The code initializes DXFile objects without checking if the input parameters are valid. Consider adding validation to ensure that input files exist before proceeding.

```diff
 # Initialize inputs into dxpy.DXDataObject instances
-starfusion_files = [dxpy.DXFile(item) for item in starfusion_files]
-fusioninspector_files = [dxpy.DXFile(item) for item in fusioninspector_files]
-fastqc_data = dxpy.DXFile(fastqc_data)
-sf_previous_data = dxpy.DXFile(SF_previous_runs_data)
-fi_previous_data = dxpy.DXFile(FI_previous_runs_data)
+try:
+    starfusion_files = [dxpy.DXFile(item) for item in starfusion_files]
+    fusioninspector_files = [dxpy.DXFile(item) for item in fusioninspector_files]
+    fastqc_data = dxpy.DXFile(fastqc_data)
+    sf_previous_data = dxpy.DXFile(SF_previous_runs_data)
+    fi_previous_data = dxpy.DXFile(FI_previous_runs_data)
+except dxpy.exceptions.DXError as e:
+    raise ValueError(f"Failed to initialize DX file objects: {e}")
```


<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

`````suggestion
    # Initialize inputs into dxpy.DXDataObject instances
    try:
        starfusion_files = [dxpy.DXFile(item) for item in starfusion_files]
        fusioninspector_files = [dxpy.DXFile(item) for item in fusioninspector_files]
        fastqc_data = dxpy.DXFile(fastqc_data)
        sf_previous_data = dxpy.DXFile(SF_previous_runs_data)
        fi_previous_data = dxpy.DXFile(FI_previous_runs_data)
    except dxpy.exceptions.DXError as e:
        raise ValueError(f"Failed to initialize DX file objects: {e}")
`````

</details>

<!-- suggestion_end -->

<!-- This is an auto-generated reply by CodeRabbit -->

✅ Addressed in commits fdff6db to 3ab0b63
created_at: 2025-03-18T13:29:46Z
author_association: NONE
diff_hunk: @@ -0,0 +1,230 @@
+"""Module for parsing fusion data from their respective DNAnexus files into a
+pandas data frame
+"""
+
+from concurrent.futures import ThreadPoolExecutor
+import dxpy
+import pandas as pd
+from dxpy import DXDataObject
+from typing import List
+
+from .utils import read_dxfile, create_pivot_table
+
+
+def parse_fastqc(dxfile: DXDataObject) -> pd.DataFrame:
+    """Parse the content of fastqc multiqc output and compute required 
+    metrics from deduplicated_percentage
+
+    Parameters
+    ----------
+    dxfile : DXDataObject
+        DNAnexus file object containing fastqc data to parse
+
+    Returns
+    -------
+    pd.DataFrame
+        pandas dataframe containing selected and computed fastqc metrics
+
+    Raises
+    ------
+    ValueError
+        if columns needed to compute extra metrics are absent.
+    """
+
+    df = read_dxfile(dxfile, include_fname=False)
+
+    # Verify required columns exist
+    required_cols = ["total_deduplicated_percentage", "Total Sequences"]
+    missing_cols = [col for col in required_cols if col not in df.columns]
+    if missing_cols:
+        raise ValueError(
+            f"Required columns missing from FastQC data: {missing_cols}"
+        )
+
+    df["Unique Reads"] = (
+        (df["total_deduplicated_percentage"] / 100.0) * df["Total Sequences"]
+        ).astype(int)
+    df["Duplicate Reads"] = (
+        df["Total Sequences"] - df["Unique Reads"]
+        ).astype(int)
+    df["Unique Reads(M)"] = df["Unique Reads"] / 1000000
+    df["Duplicate Reads(M)"] = df["Duplicate Reads"] / 1000000
+    df = df[
+        [
+            "Sample",
+            "Unique Reads",
+            "Duplicate Reads",
+            "Unique Reads(M)",
+            "Duplicate Reads(M)",
+        ]
+    ]
+
+    return df
+
+
+def make_fastqc_pivot(df:pd.DataFrame, pivot_config:dict) -> pd.DataFrame:
+    """generates FastQC pivot table 
+
+    Parameters
+    ----------
+    df : pd.DataFrame
+        Dataframe containing raw fastqc data
+    pivot_config : dict
+        Mapping defining pivot structure
+
+    Returns
+    -------
+    pd.DataFrame
+        created pivot table with computed columns
+    """
+    df["SPECIMEN"] = df["Sample"].astype(str).str[10:20]
+    # omit EPIC col for now
+    
+    pivot_df = create_pivot_table(df, pivot_config)
+    pivot_df = pivot_df.reset_index(drop=False)
+    
+    return pivot_df
+    
+    
+    
+def _parse_fusion_files(dxfiles: List[DXDataObject]) -> pd.DataFrame:
+    """
+    Reads and concatenates a list of DNAnexus fusion-related files
+    into a single DataFrame.
+
+    Parameters
+    ----------
+    dxfiles : List[DXDataObject]
+        A list of DNAnexus file objects.
+
+    Returns
+    -------
+    pd.DataFrame
+        A concatenated DataFrame containing the combined data
+        from all input files.
+    """
+    if not dxfiles:
+        return pd.DataFrame()
+
+    max_workers = min(16, len(dxfiles))
+    with ThreadPoolExecutor(max_workers=max_workers) as executor:
+        df = pd.concat(executor.map(read_dxfile, dxfiles))
+
+    return df
+
+
+def parse_fusion_inspector(dxfiles: List[DXDataObject]) -> pd.DataFrame:
+    """
+    Reads and concatenates an array of DNAnexus Fusion Inspector files.
+
+    Parameters
+    ----------
+    dxfiles : List[DXDataObject]
+        A list of DNAnexus file objects.
+
+    Returns
+    -------
+    pd.DataFrame
+        A concatenated DataFrame containing the combined data
+        from all input files.
+    """
+    # simply calls _parse_fusion_files for now
+    # can be easily extended if additional logic needed for tool
+    return _parse_fusion_files(dxfiles)
+
+
+def parse_star_fusion(dxfiles: List[DXDataObject]) -> pd.DataFrame:
+    """
+    Reads and concatenates an array of DNAnexus STAR-Fusion files.
+
+    Parameters
+    ----------
+    dxfiles : List[DXDataObject]
+        A list of DNAnexus file objects.
+
+    Returns
+    -------
+    pd.DataFrame
+        A concatenated DataFrame containing the combined data
+        from all input files.
+    """
+    # same as above; to allow further customisation per tool as needed.
+    return _parse_fusion_files(dxfiles)
+
+
+def make_sf_pivot(
+    sf_df: pd.DataFrame,
+    sf_runs_df: pd.DataFrame,
+    fastqc_pivot_df: pd.DataFrame,
+    fi_df: pd.DataFrame,
+    pivot_config: dict
+) -> pd.DataFrame:
+    """Generates STAR-Fusion pivot table with merged data
+    
+    Parameters
+    ----------
+    sf_df : pd.DataFrame
+        Main STAR-Fusion data
+    sf_runs_df : pd.DataFrame
+        Historical STAR-Fusion data
+    fastqc_pivot_df : pd.DataFrame
+        FastQC summary data
+    fi_df : pd.DataFrame
+        Main Fusion Inspector data
+    pivot_config : dict
+        Pivot configuration parameters
+
+    Returns
+    -------
+    pd.DataFrame
+        Created Pivot table with merged data
+    """
+    
+    df = sf_df.copy()
+    
+    # =MID(K{row},11,10)
+    if 'file_name' in sf_df.columns:
+        df['SPECIMEN'] = df['file_name'].astype(str).str[10:20]
+    
+        # =LEFT(K{row},28)
+        df['FNAME'] = df['file_name'].astype(str).str[:28]
path: resources/home/dnanexus/utils/parser.py
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Similar hardcoded string slicing in make_sf_pivot function**

This function has the same issue with hardcoded string slicing indices. Consider using a more robust approach as suggested for the make_fastqc_pivot function.

```diff
 # =MID(K{row},11,10)
 if 'file_name' in sf_df.columns:
-    df['SPECIMEN'] = df['file_name'].astype(str).str[10:20]
+    # Extract specimen ID using a more robust pattern matching approach
+    import re
+    def extract_specimen(sample_str):
+        # Adjust the pattern to match your specific sample format requirements
+        match = re.search(r'[A-Za-z0-9]{10}', str(sample_str))
+        return match.group(0) if match else sample_str
+    
+    df['SPECIMEN'] = df['file_name'].astype(str).apply(extract_specimen)

     # =LEFT(K{row},28)
-    df['FNAME'] = df['file_name'].astype(str).str[:28]
+    # Set maximum file name length while handling edge cases
+    df['FNAME'] = df['file_name'].astype(str).apply(lambda x: x[:28] if len(x) > 28 else x)
```


<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

`````suggestion
    if 'file_name' in sf_df.columns:
        # Extract specimen ID using a more robust pattern matching approach
        import re
        def extract_specimen(sample_str):
            # Adjust the pattern to match your specific sample format requirements
            match = re.search(r'[A-Za-z0-9]{10}', str(sample_str))
            return match.group(0) if match else sample_str
        
        df['SPECIMEN'] = df['file_name'].astype(str).apply(extract_specimen)
    
        # Set maximum file name length while handling edge cases
        df['FNAME'] = df['file_name'].astype(str).apply(lambda x: x[:28] if len(x) > 28 else x)
`````

</details>

<!-- suggestion_end -->

<!-- This is an auto-generated reply by CodeRabbit -->

✅ Addressed in commit 91f3a51
created_at: 2025-03-18T13:29:46Z
author_association: NONE
diff_hunk: @@ -0,0 +1,230 @@
+"""Module for parsing fusion data from their respective DNAnexus files into a
+pandas data frame
+"""
+
+from concurrent.futures import ThreadPoolExecutor
+import dxpy
+import pandas as pd
+from dxpy import DXDataObject
+from typing import List
+
+from .utils import read_dxfile, create_pivot_table
+
+
+def parse_fastqc(dxfile: DXDataObject) -> pd.DataFrame:
+    """Parse the content of fastqc multiqc output and compute required 
+    metrics from deduplicated_percentage
+
+    Parameters
+    ----------
+    dxfile : DXDataObject
+        DNAnexus file object containing fastqc data to parse
+
+    Returns
+    -------
+    pd.DataFrame
+        pandas dataframe containing selected and computed fastqc metrics
+
+    Raises
+    ------
+    ValueError
+        if columns needed to compute extra metrics are absent.
+    """
+
+    df = read_dxfile(dxfile, include_fname=False)
+
+    # Verify required columns exist
+    required_cols = ["total_deduplicated_percentage", "Total Sequences"]
+    missing_cols = [col for col in required_cols if col not in df.columns]
+    if missing_cols:
+        raise ValueError(
+            f"Required columns missing from FastQC data: {missing_cols}"
+        )
+
+    df["Unique Reads"] = (
+        (df["total_deduplicated_percentage"] / 100.0) * df["Total Sequences"]
+        ).astype(int)
+    df["Duplicate Reads"] = (
+        df["Total Sequences"] - df["Unique Reads"]
+        ).astype(int)
+    df["Unique Reads(M)"] = df["Unique Reads"] / 1000000
+    df["Duplicate Reads(M)"] = df["Duplicate Reads"] / 1000000
+    df = df[
+        [
+            "Sample",
+            "Unique Reads",
+            "Duplicate Reads",
+            "Unique Reads(M)",
+            "Duplicate Reads(M)",
+        ]
+    ]
+
+    return df
+
+
+def make_fastqc_pivot(df:pd.DataFrame, pivot_config:dict) -> pd.DataFrame:
+    """generates FastQC pivot table 
+
+    Parameters
+    ----------
+    df : pd.DataFrame
+        Dataframe containing raw fastqc data
+    pivot_config : dict
+        Mapping defining pivot structure
+
+    Returns
+    -------
+    pd.DataFrame
+        created pivot table with computed columns
+    """
+    df["SPECIMEN"] = df["Sample"].astype(str).str[10:20]
+    # omit EPIC col for now
+    
path: resources/home/dnanexus/utils/parser.py
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Avoid hardcoded string slicing indices**

The code uses hardcoded slice indices `[10:20]` to extract the specimen ID. This approach is fragile and will break if the format of the Sample field changes.

```diff
-    df["SPECIMEN"] = df["Sample"].astype(str).str[10:20]
-    # omit EPIC col for now
+    # Extract specimen ID using a more robust pattern matching approach
+    # Assuming format is consistent and SPECIMEN appears at a specific position
+    import re
+    def extract_specimen(sample_str):
+        # Adjust the pattern to match your specific sample format requirements
+        match = re.search(r'[A-Za-z0-9]{10}', str(sample_str))
+        return match.group(0) if match else sample_str
+    
+    df["SPECIMEN"] = df["Sample"].astype(str).apply(extract_specimen)
+    # TODO: Add EPIC column mapping once available
```


<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

`````suggestion
    # Extract specimen ID using a more robust pattern matching approach
    # Assuming format is consistent and SPECIMEN appears at a specific position
    import re
    def extract_specimen(sample_str):
        # Adjust the pattern to match your specific sample format requirements
        match = re.search(r'[A-Za-z0-9]{10}', str(sample_str))
        return match.group(0) if match else sample_str
    
    df["SPECIMEN"] = df["Sample"].astype(str).apply(extract_specimen)
    # TODO: Add EPIC column mapping once available
`````

</details>

<!-- suggestion_end -->

<!-- This is an auto-generated reply by CodeRabbit -->

✅ Addressed in commits fdff6db to 3ab0b63
created_at: 2025-03-18T13:29:46Z
author_association: NONE
diff_hunk: @@ -0,0 +1,123 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+import os
+from glob import glob
+import subprocess
+from typing import List
+
+if os.path.exists("/home/dnanexus"):
+    # running in DNAnexus
+    subprocess.check_call(
+        ["pip", "install", "--no-index", "--no-deps"] + glob("packages/*")
+    )
+
+import dxpy
+import openpyxl
+import pandas as pd
+from utils.defaults import (
+    EPIC_SHEET_CONFIG,
+    FASTQC_SHEET_CONFIG,
+    FI_PREVIOUS_RUNS_SHEET_CONFIG,
+    FI_SHEET_CONFIG,
+    SF_PREVIOUS_RUNS_SHEET_CONFIG,
+    SF_SHEET_CONFIG,
+    FASTQC_PIVOT_CONFIG,
+)
+from utils.parser import (
+    parse_fastqc,
+    parse_fusion_inspector,
+    parse_star_fusion,
+    make_fastqc_pivot
+)
+from utils.utils import (
+    get_project_info,
+    read_dxfile,
+    write_df_to_sheet,
+)
+
+
+@dxpy.entry_point("main")
+def main(
+    starfusion_files: List[dict],
+    fusioninspector_files: List[dict],
+    fastqc_data: dict,
+    SF_previous_runs_data: dict,
+    FI_previous_runs_data: dict,
+):
+    """Generates a fusion workbook with data from
+    STAR-Fusion, FusionInspector, and FastQC
+
+    Parameters
+    ----------
+    starfusion_files : List[dict]
+        List of dictionaries containing DXLinks to STAR-Fusion output files
+    fusioninspector_files : List[dict]
+        List of dictionaries containing DXLinks to FusionInspector output files
+    fastqc_data : dict
+        Mapping of DXLink to FastQC data file
+    SF_previous_runs_data : str
+       Mapping of DXLink to STAR-Fusion previous runs data file
+    FI_previous_runs_data : str
+        Mapping of DXLink to FusionInspector previous runs data file
+
+    Returns
+    -------
+    dict
+        Dictionary containing DXLink to the generated fusion workbook
+    """
+    # Initialize inputs into dxpy.DXDataObject instances
+    starfusion_files = [dxpy.DXFile(item) for item in starfusion_files]
+    fusioninspector_files = [dxpy.DXFile(item) for item in fusioninspector_files]
+    fastqc_data = dxpy.DXFile(fastqc_data)
+    sf_previous_data = dxpy.DXFile(SF_previous_runs_data)
+    fi_previous_data = dxpy.DXFile(FI_previous_runs_data)
+
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: jethror1
body: not needed
<!-- Reviewable comment -OLt2mnv6B917T8spg8-:bpp6stj -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-21T14:32:30Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,123 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+import os
+from glob import glob
+import subprocess
+from typing import List
+
+if os.path.exists("/home/dnanexus"):
+    # running in DNAnexus
+    subprocess.check_call(
+        ["pip", "install", "--no-index", "--no-deps"] + glob("packages/*")
+    )
+
+import dxpy
+import openpyxl
+import pandas as pd
+from utils.defaults import (
+    EPIC_SHEET_CONFIG,
+    FASTQC_SHEET_CONFIG,
+    FI_PREVIOUS_RUNS_SHEET_CONFIG,
+    FI_SHEET_CONFIG,
+    SF_PREVIOUS_RUNS_SHEET_CONFIG,
+    SF_SHEET_CONFIG,
+    FASTQC_PIVOT_CONFIG,
+)
+from utils.parser import (
+    parse_fastqc,
+    parse_fusion_inspector,
+    parse_star_fusion,
+    make_fastqc_pivot
+)
+from utils.utils import (
+    get_project_info,
+    read_dxfile,
+    write_df_to_sheet,
+)
+
+
+@dxpy.entry_point("main")
+def main(
+    starfusion_files: List[dict],
+    fusioninspector_files: List[dict],
+    fastqc_data: dict,
+    SF_previous_runs_data: dict,
+    FI_previous_runs_data: dict,
+):
+    """Generates a fusion workbook with data from
+    STAR-Fusion, FusionInspector, and FastQC
+
+    Parameters
+    ----------
+    starfusion_files : List[dict]
+        List of dictionaries containing DXLinks to STAR-Fusion output files
+    fusioninspector_files : List[dict]
+        List of dictionaries containing DXLinks to FusionInspector output files
+    fastqc_data : dict
+        Mapping of DXLink to FastQC data file
+    SF_previous_runs_data : str
+       Mapping of DXLink to STAR-Fusion previous runs data file
+    FI_previous_runs_data : str
+        Mapping of DXLink to FusionInspector previous runs data file
+
+    Returns
+    -------
+    dict
+        Dictionary containing DXLink to the generated fusion workbook
+    """
+    # Initialize inputs into dxpy.DXDataObject instances
+    starfusion_files = [dxpy.DXFile(item) for item in starfusion_files]
+    fusioninspector_files = [dxpy.DXFile(item) for item in fusioninspector_files]
+    fastqc_data = dxpy.DXFile(fastqc_data)
+    sf_previous_data = dxpy.DXFile(SF_previous_runs_data)
+    fi_previous_data = dxpy.DXFile(FI_previous_runs_data)
+
+    df_starfusion = parse_star_fusion(starfusion_files)
+    df_fusioninspector = parse_fusion_inspector(fusioninspector_files)
+    df_fastqc = parse_fastqc(fastqc_data)
+
+    df_sf_previous = read_dxfile(sf_previous_data)
+    df_fi_previous = read_dxfile(fi_previous_data)
+
+    project_name, _ = get_project_info()
+    outfile = f"{project_name}_fusion_workbook.xlsx"
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: jethror1
body: not needed
<!-- Reviewable comment -OLt2obKAokmzjjIVjUQ:bpp6stj -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-21T14:32:31Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,123 @@
+"""Module for parsing fusion data from their respective DNAnexus files into a
+pandas data frame
+"""
+from concurrent.futures import ThreadPoolExecutor
+import dxpy
+import pandas as pd
+from dxpy import DXDataObject
+from typing import List
+
+from .utils import read_dxfile
+
+
+def parse_fastqc(dxfile: DXDataObject) -> pd.DataFrame:
+    """Parse the content of fastqc multiqc output and compute required metrics from deduplicated_percentage
+
+    Parameters
+    ----------
+    dxfile : DXDataObject
+        DNAnexus file object containing fastqc data to parse
+
+    Returns
+    -------
+    pd.DataFrame
+        pandas dataframe containing selected and computed fastqc metrics
+
+    Raises
+    ------
+    ValueError
+        if columns needed to compute extra metrics are absent.
+    """
+
+    df = read_dxfile(dxfile, include_fname=False)
+
+    # Verify required columns exist
+    required_cols = ["total_deduplicated_percentage", "Total Sequences"]
+    missing_cols = [col for col in required_cols if col not in df.columns]
+    if missing_cols:
+        raise ValueError(
+            f"Required columns missing from FastQC data: {missing_cols}"
+        )
+
+    df["Unique Reads"] = (
+        (df["total_deduplicated_percentage"] / 100.0) * df["Total Sequences"]
+    ).astype(int)
+    df["Duplicate Reads"] = (
+        df["Total Sequences"] - df["Unique Reads"]
+    ).astype(int)
+    df["Unique Reads(M)"] = df["Unique Reads"] / 1000000
+    df["Duplicate Reads(M)"] = df["Duplicate Reads"] / 1000000
+    df = df[[
+        "Sample",
+        "Unique Reads",
+        "Duplicate Reads",
+        "Unique Reads(M)",
+        "Duplicate Reads(M)"
+    ]]
+
+    return df
+
+
+def _parse_fusion_files(dxfiles: List[DXDataObject]) -> pd.DataFrame:
+    """
+    Reads and concatenates a list of DNAnexus fusion-related files 
+    into a single DataFrame.
+
+    Parameters
+    ----------
+    dxfiles : List[DXDataObject]
+        A list of DNAnexus file objects.
+
+    Returns
+    -------
+    pd.DataFrame
+        A concatenated DataFrame containing the combined data 
+        from all input files.
+    """
+    if not dxfiles:
+        return pd.DataFrame()
+    
+    max_workers = min(16, len(dxfiles))
+    with ThreadPoolExecutor(max_workers=max_workers) as executor:
+        df = pd.concat(executor.map(read_dxfile, dxfiles))
+
path: resources/home/dnanexus/utils/parser.py
user: login: jethror1
body: @prince please use this example to submit and call `future.result()` which will cleanly not needed
<!-- Reviewable comment -OLt4WFH6f2wvxv0yq2i:b-9enigz -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-21T14:32:31Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,230 @@
+"""Module for parsing fusion data from their respective DNAnexus files into a
+pandas data frame
+"""
+
+from concurrent.futures import ThreadPoolExecutor
+import dxpy
+import pandas as pd
+from dxpy import DXDataObject
+from typing import List
+
+from .utils import read_dxfile, create_pivot_table
+
+
+def parse_fastqc(dxfile: DXDataObject) -> pd.DataFrame:
+    """Parse the content of fastqc multiqc output and compute required 
+    metrics from deduplicated_percentage
+
+    Parameters
+    ----------
+    dxfile : DXDataObject
+        DNAnexus file object containing fastqc data to parse
+
+    Returns
+    -------
+    pd.DataFrame
+        pandas dataframe containing selected and computed fastqc metrics
+
+    Raises
+    ------
+    ValueError
+        if columns needed to compute extra metrics are absent.
+    """
+
+    df = read_dxfile(dxfile, include_fname=False)
+
+    # Verify required columns exist
+    required_cols = ["total_deduplicated_percentage", "Total Sequences"]
+    missing_cols = [col for col in required_cols if col not in df.columns]
+    if missing_cols:
+        raise ValueError(
+            f"Required columns missing from FastQC data: {missing_cols}"
+        )
+
+    df["Unique Reads"] = (
+        (df["total_deduplicated_percentage"] / 100.0) * df["Total Sequences"]
+        ).astype(int)
+    df["Duplicate Reads"] = (
+        df["Total Sequences"] - df["Unique Reads"]
+        ).astype(int)
+    df["Unique Reads(M)"] = df["Unique Reads"] / 1000000
+    df["Duplicate Reads(M)"] = df["Duplicate Reads"] / 1000000
+    df = df[
+        [
+            "Sample",
+            "Unique Reads",
+            "Duplicate Reads",
+            "Unique Reads(M)",
+            "Duplicate Reads(M)",
+        ]
+    ]
+
+    return df
+
+
+def make_fastqc_pivot(df:pd.DataFrame, pivot_config:dict) -> pd.DataFrame:
+    """generates FastQC pivot table 
+
+    Parameters
+    ----------
+    df : pd.DataFrame
+        Dataframe containing raw fastqc data
+    pivot_config : dict
+        Mapping defining pivot structure
+
+    Returns
+    -------
+    pd.DataFrame
+        created pivot table with computed columns
+    """
+    df["SPECIMEN"] = df["Sample"].astype(str).str[10:20]
+    # omit EPIC col for now
+    
path: resources/home/dnanexus/utils/parser.py
user: login: jethror1
body: agreed I don't think slicing is good for parsing out, since we're delimiting with `_` and think this would be more robust to split on and select the appropriate field
<!-- Reviewable comment -OLt4HM07CZy_iC7lKph:by5rjju -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-21T14:32:31Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,123 @@
+"""Module for parsing fusion data from their respective DNAnexus files into a
+pandas data frame
+"""
+from concurrent.futures import ThreadPoolExecutor
+import dxpy
+import pandas as pd
+from dxpy import DXDataObject
+from typing import List
+
+from .utils import read_dxfile
+
+
+def parse_fastqc(dxfile: DXDataObject) -> pd.DataFrame:
+    """Parse the content of fastqc multiqc output and compute required metrics from deduplicated_percentage
+
+    Parameters
+    ----------
+    dxfile : DXDataObject
+        DNAnexus file object containing fastqc data to parse
+
+    Returns
+    -------
+    pd.DataFrame
+        pandas dataframe containing selected and computed fastqc metrics
+
+    Raises
+    ------
+    ValueError
+        if columns needed to compute extra metrics are absent.
+    """
+
+    df = read_dxfile(dxfile, include_fname=False)
+
+    # Verify required columns exist
+    required_cols = ["total_deduplicated_percentage", "Total Sequences"]
+    missing_cols = [col for col in required_cols if col not in df.columns]
+    if missing_cols:
+        raise ValueError(
+            f"Required columns missing from FastQC data: {missing_cols}"
+        )
+
+    df["Unique Reads"] = (
+        (df["total_deduplicated_percentage"] / 100.0) * df["Total Sequences"]
+    ).astype(int)
+    df["Duplicate Reads"] = (
+        df["Total Sequences"] - df["Unique Reads"]
+    ).astype(int)
+    df["Unique Reads(M)"] = df["Unique Reads"] / 1000000
+    df["Duplicate Reads(M)"] = df["Duplicate Reads"] / 1000000
+    df = df[[
+        "Sample",
+        "Unique Reads",
+        "Duplicate Reads",
+        "Unique Reads(M)",
+        "Duplicate Reads(M)"
+    ]]
+
+    return df
+
+
+def _parse_fusion_files(dxfiles: List[DXDataObject]) -> pd.DataFrame:
+    """
+    Reads and concatenates a list of DNAnexus fusion-related files 
+    into a single DataFrame.
+
+    Parameters
+    ----------
+    dxfiles : List[DXDataObject]
+        A list of DNAnexus file objects.
+
+    Returns
+    -------
+    pd.DataFrame
+        A concatenated DataFrame containing the combined data 
+        from all input files.
+    """
+    if not dxfiles:
+        return pd.DataFrame()
+    
+    max_workers = min(16, len(dxfiles))
+    with ThreadPoolExecutor(max_workers=max_workers) as executor:
+        df = pd.concat(executor.map(read_dxfile, dxfiles))
+
path: resources/home/dnanexus/utils/parser.py
user: login: coderabbitai[bot]
body: > Seems like the humans are having a chat. I'll hop back into my burrow for now. If you need me again, just tag @coderabbitai in a new comment, and I'll come hopping out!

<!-- This is an auto-generated mute comment by CodeRabbit -->
created_at: 2025-03-21T14:32:41Z
author_association: NONE
diff_hunk: @@ -0,0 +1,123 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+import os
+from glob import glob
+import subprocess
+from typing import List
+
+if os.path.exists("/home/dnanexus"):
+    # running in DNAnexus
+    subprocess.check_call(
+        ["pip", "install", "--no-index", "--no-deps"] + glob("packages/*")
+    )
+
+import dxpy
+import openpyxl
+import pandas as pd
+from utils.defaults import (
+    EPIC_SHEET_CONFIG,
+    FASTQC_SHEET_CONFIG,
+    FI_PREVIOUS_RUNS_SHEET_CONFIG,
+    FI_SHEET_CONFIG,
+    SF_PREVIOUS_RUNS_SHEET_CONFIG,
+    SF_SHEET_CONFIG,
+    FASTQC_PIVOT_CONFIG,
+)
+from utils.parser import (
+    parse_fastqc,
+    parse_fusion_inspector,
+    parse_star_fusion,
+    make_fastqc_pivot
+)
+from utils.utils import (
+    get_project_info,
+    read_dxfile,
+    write_df_to_sheet,
+)
+
+
+@dxpy.entry_point("main")
+def main(
+    starfusion_files: List[dict],
+    fusioninspector_files: List[dict],
+    fastqc_data: dict,
+    SF_previous_runs_data: dict,
+    FI_previous_runs_data: dict,
+):
+    """Generates a fusion workbook with data from
+    STAR-Fusion, FusionInspector, and FastQC
+
+    Parameters
+    ----------
+    starfusion_files : List[dict]
+        List of dictionaries containing DXLinks to STAR-Fusion output files
+    fusioninspector_files : List[dict]
+        List of dictionaries containing DXLinks to FusionInspector output files
+    fastqc_data : dict
+        Mapping of DXLink to FastQC data file
+    SF_previous_runs_data : str
+       Mapping of DXLink to STAR-Fusion previous runs data file
+    FI_previous_runs_data : str
+        Mapping of DXLink to FusionInspector previous runs data file
+
+    Returns
+    -------
+    dict
+        Dictionary containing DXLink to the generated fusion workbook
+    """
+    # Initialize inputs into dxpy.DXDataObject instances
+    starfusion_files = [dxpy.DXFile(item) for item in starfusion_files]
+    fusioninspector_files = [dxpy.DXFile(item) for item in fusioninspector_files]
+    fastqc_data = dxpy.DXFile(fastqc_data)
+    sf_previous_data = dxpy.DXFile(SF_previous_runs_data)
+    fi_previous_data = dxpy.DXFile(FI_previous_runs_data)
+
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: coderabbitai[bot]
body: @jethror1 Understood, I'll drop the input validation suggestion. Thanks for the feedback.

🐰

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-21T14:32:57Z
author_association: NONE
diff_hunk: @@ -0,0 +1,123 @@
+#!/usr/bin/env python3
+# eggd_generate_fusion_workbook 1.0.0
+
+import os
+from glob import glob
+import subprocess
+from typing import List
+
+if os.path.exists("/home/dnanexus"):
+    # running in DNAnexus
+    subprocess.check_call(
+        ["pip", "install", "--no-index", "--no-deps"] + glob("packages/*")
+    )
+
+import dxpy
+import openpyxl
+import pandas as pd
+from utils.defaults import (
+    EPIC_SHEET_CONFIG,
+    FASTQC_SHEET_CONFIG,
+    FI_PREVIOUS_RUNS_SHEET_CONFIG,
+    FI_SHEET_CONFIG,
+    SF_PREVIOUS_RUNS_SHEET_CONFIG,
+    SF_SHEET_CONFIG,
+    FASTQC_PIVOT_CONFIG,
+)
+from utils.parser import (
+    parse_fastqc,
+    parse_fusion_inspector,
+    parse_star_fusion,
+    make_fastqc_pivot
+)
+from utils.utils import (
+    get_project_info,
+    read_dxfile,
+    write_df_to_sheet,
+)
+
+
+@dxpy.entry_point("main")
+def main(
+    starfusion_files: List[dict],
+    fusioninspector_files: List[dict],
+    fastqc_data: dict,
+    SF_previous_runs_data: dict,
+    FI_previous_runs_data: dict,
+):
+    """Generates a fusion workbook with data from
+    STAR-Fusion, FusionInspector, and FastQC
+
+    Parameters
+    ----------
+    starfusion_files : List[dict]
+        List of dictionaries containing DXLinks to STAR-Fusion output files
+    fusioninspector_files : List[dict]
+        List of dictionaries containing DXLinks to FusionInspector output files
+    fastqc_data : dict
+        Mapping of DXLink to FastQC data file
+    SF_previous_runs_data : str
+       Mapping of DXLink to STAR-Fusion previous runs data file
+    FI_previous_runs_data : str
+        Mapping of DXLink to FusionInspector previous runs data file
+
+    Returns
+    -------
+    dict
+        Dictionary containing DXLink to the generated fusion workbook
+    """
+    # Initialize inputs into dxpy.DXDataObject instances
+    starfusion_files = [dxpy.DXFile(item) for item in starfusion_files]
+    fusioninspector_files = [dxpy.DXFile(item) for item in fusioninspector_files]
+    fastqc_data = dxpy.DXFile(fastqc_data)
+    sf_previous_data = dxpy.DXFile(SF_previous_runs_data)
+    fi_previous_data = dxpy.DXFile(FI_previous_runs_data)
+
+    df_starfusion = parse_star_fusion(starfusion_files)
+    df_fusioninspector = parse_fusion_inspector(fusioninspector_files)
+    df_fastqc = parse_fastqc(fastqc_data)
+
+    df_sf_previous = read_dxfile(sf_previous_data)
+    df_fi_previous = read_dxfile(fi_previous_data)
+
+    project_name, _ = get_project_info()
+    outfile = f"{project_name}_fusion_workbook.xlsx"
path: resources/home/dnanexus/generate_fusion_workbook.py
user: login: coderabbitai[bot]
body: @jethror1 Got it. The error handling for the project information retrieval is not needed in this case. I'll refrain from suggesting similar error handling for the `get_project_info()` function in future reviews.

---

<details>
<summary>✏️ Learnings added</summary>

```
Learnt from: jethror1
PR: eastgenomics/eggd_generate_fusion_workbook#1
File: resources/home/dnanexus/generate_fusion_workbook.py:86-87
Timestamp: 2025-03-21T14:32:57.740Z
Learning: Error handling for the `get_project_info()` function is not needed in the eggd_generate_fusion_workbook application as the project ID environment variable is expected to be properly set in the DNAnexus execution environment.
```

> Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure [review instructions](https://docs.coderabbit.ai/guides/review-instructions). You can manage existing learnings in the [app](https://app.coderabbit.ai/learnings).

</details>

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-21T14:32:58Z
author_association: NONE
diff_hunk: @@ -0,0 +1,230 @@
+"""Module for parsing fusion data from their respective DNAnexus files into a
+pandas data frame
+"""
+
+from concurrent.futures import ThreadPoolExecutor
+import dxpy
+import pandas as pd
+from dxpy import DXDataObject
+from typing import List
+
+from .utils import read_dxfile, create_pivot_table
+
+
+def parse_fastqc(dxfile: DXDataObject) -> pd.DataFrame:
+    """Parse the content of fastqc multiqc output and compute required 
+    metrics from deduplicated_percentage
+
+    Parameters
+    ----------
+    dxfile : DXDataObject
+        DNAnexus file object containing fastqc data to parse
+
+    Returns
+    -------
+    pd.DataFrame
+        pandas dataframe containing selected and computed fastqc metrics
+
+    Raises
+    ------
+    ValueError
+        if columns needed to compute extra metrics are absent.
+    """
+
+    df = read_dxfile(dxfile, include_fname=False)
+
+    # Verify required columns exist
+    required_cols = ["total_deduplicated_percentage", "Total Sequences"]
+    missing_cols = [col for col in required_cols if col not in df.columns]
+    if missing_cols:
+        raise ValueError(
+            f"Required columns missing from FastQC data: {missing_cols}"
+        )
+
+    df["Unique Reads"] = (
+        (df["total_deduplicated_percentage"] / 100.0) * df["Total Sequences"]
+        ).astype(int)
+    df["Duplicate Reads"] = (
+        df["Total Sequences"] - df["Unique Reads"]
+        ).astype(int)
+    df["Unique Reads(M)"] = df["Unique Reads"] / 1000000
+    df["Duplicate Reads(M)"] = df["Duplicate Reads"] / 1000000
+    df = df[
+        [
+            "Sample",
+            "Unique Reads",
+            "Duplicate Reads",
+            "Unique Reads(M)",
+            "Duplicate Reads(M)",
+        ]
+    ]
+
+    return df
+
+
+def make_fastqc_pivot(df:pd.DataFrame, pivot_config:dict) -> pd.DataFrame:
+    """generates FastQC pivot table 
+
+    Parameters
+    ----------
+    df : pd.DataFrame
+        Dataframe containing raw fastqc data
+    pivot_config : dict
+        Mapping defining pivot structure
+
+    Returns
+    -------
+    pd.DataFrame
+        created pivot table with computed columns
+    """
+    df["SPECIMEN"] = df["Sample"].astype(str).str[10:20]
+    # omit EPIC col for now
+    
path: resources/home/dnanexus/utils/parser.py
user: login: coderabbitai[bot]
body: @jethror1 You're absolutely right - string splitting on `_` is a much more efficient and maintainable approach here. Here's how that implementation would look:

```python
# Extract specimen ID by splitting on underscore delimiter
df["SPECIMEN"] = df["Sample"].astype(str).str.split('_').str[1]  # Adjust the index based on which part contains the specimen ID
# TODO: Add EPIC column mapping once available
```

This approach is more robust because it relies on the structural format of the data rather than specific character positions, making it resilient to changes in prefix length or formatting.

---

<details>
<summary>✏️ Learnings added</summary>

```
Learnt from: jethror1
PR: eastgenomics/eggd_generate_fusion_workbook#1
File: resources/home/dnanexus/utils/parser.py:0-0
Timestamp: 2025-03-21T14:33:02.857Z
Learning: In the eggd_generate_fusion_workbook codebase, specimen IDs should be extracted from Sample names by splitting on underscore delimiters (using `.str.split('_')`) rather than using hardcoded string indices, as this approach is more robust to format changes.
```

> Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure [review instructions](https://docs.coderabbit.ai/guides/review-instructions). You can manage existing learnings in the [app](https://app.coderabbit.ai/learnings).

</details>

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-21T14:33:03Z
author_association: NONE
diff_hunk: @@ -0,0 +1,123 @@
+"""Module for parsing fusion data from their respective DNAnexus files into a
+pandas data frame
+"""
+from concurrent.futures import ThreadPoolExecutor
+import dxpy
+import pandas as pd
+from dxpy import DXDataObject
+from typing import List
+
+from .utils import read_dxfile
+
+
+def parse_fastqc(dxfile: DXDataObject) -> pd.DataFrame:
+    """Parse the content of fastqc multiqc output and compute required metrics from deduplicated_percentage
+
+    Parameters
+    ----------
+    dxfile : DXDataObject
+        DNAnexus file object containing fastqc data to parse
+
+    Returns
+    -------
+    pd.DataFrame
+        pandas dataframe containing selected and computed fastqc metrics
+
+    Raises
+    ------
+    ValueError
+        if columns needed to compute extra metrics are absent.
+    """
+
+    df = read_dxfile(dxfile, include_fname=False)
+
+    # Verify required columns exist
+    required_cols = ["total_deduplicated_percentage", "Total Sequences"]
+    missing_cols = [col for col in required_cols if col not in df.columns]
+    if missing_cols:
+        raise ValueError(
+            f"Required columns missing from FastQC data: {missing_cols}"
+        )
+
+    df["Unique Reads"] = (
+        (df["total_deduplicated_percentage"] / 100.0) * df["Total Sequences"]
+    ).astype(int)
+    df["Duplicate Reads"] = (
+        df["Total Sequences"] - df["Unique Reads"]
+    ).astype(int)
+    df["Unique Reads(M)"] = df["Unique Reads"] / 1000000
+    df["Duplicate Reads(M)"] = df["Duplicate Reads"] / 1000000
+    df = df[[
+        "Sample",
+        "Unique Reads",
+        "Duplicate Reads",
+        "Unique Reads(M)",
+        "Duplicate Reads(M)"
+    ]]
+
+    return df
+
+
+def _parse_fusion_files(dxfiles: List[DXDataObject]) -> pd.DataFrame:
+    """
+    Reads and concatenates a list of DNAnexus fusion-related files 
+    into a single DataFrame.
+
+    Parameters
+    ----------
+    dxfiles : List[DXDataObject]
+        A list of DNAnexus file objects.
+
+    Returns
+    -------
+    pd.DataFrame
+        A concatenated DataFrame containing the combined data 
+        from all input files.
+    """
+    if not dxfiles:
+        return pd.DataFrame()
+    
+    max_workers = min(16, len(dxfiles))
+    with ThreadPoolExecutor(max_workers=max_workers) as executor:
+        df = pd.concat(executor.map(read_dxfile, dxfiles))
+
path: resources/home/dnanexus/utils/parser.py
user: login: pofuoku
body: Done.
<!-- Reviewable comment -OLtFOe60Ei605JuqiIW:b-896fix -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-21T16:28:23Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,230 @@
+"""Module for parsing fusion data from their respective DNAnexus files into a
+pandas data frame
+"""
+
+from concurrent.futures import ThreadPoolExecutor
+import dxpy
+import pandas as pd
+from dxpy import DXDataObject
+from typing import List
+
+from .utils import read_dxfile, create_pivot_table
+
+
+def parse_fastqc(dxfile: DXDataObject) -> pd.DataFrame:
+    """Parse the content of fastqc multiqc output and compute required 
+    metrics from deduplicated_percentage
+
+    Parameters
+    ----------
+    dxfile : DXDataObject
+        DNAnexus file object containing fastqc data to parse
+
+    Returns
+    -------
+    pd.DataFrame
+        pandas dataframe containing selected and computed fastqc metrics
+
+    Raises
+    ------
+    ValueError
+        if columns needed to compute extra metrics are absent.
+    """
+
+    df = read_dxfile(dxfile, include_fname=False)
+
+    # Verify required columns exist
+    required_cols = ["total_deduplicated_percentage", "Total Sequences"]
+    missing_cols = [col for col in required_cols if col not in df.columns]
+    if missing_cols:
+        raise ValueError(
+            f"Required columns missing from FastQC data: {missing_cols}"
+        )
+
+    df["Unique Reads"] = (
+        (df["total_deduplicated_percentage"] / 100.0) * df["Total Sequences"]
+        ).astype(int)
+    df["Duplicate Reads"] = (
+        df["Total Sequences"] - df["Unique Reads"]
+        ).astype(int)
+    df["Unique Reads(M)"] = df["Unique Reads"] / 1000000
+    df["Duplicate Reads(M)"] = df["Duplicate Reads"] / 1000000
+    df = df[
+        [
+            "Sample",
+            "Unique Reads",
+            "Duplicate Reads",
+            "Unique Reads(M)",
+            "Duplicate Reads(M)",
+        ]
+    ]
+
+    return df
+
+
+def make_fastqc_pivot(df:pd.DataFrame, pivot_config:dict) -> pd.DataFrame:
+    """generates FastQC pivot table 
+
+    Parameters
+    ----------
+    df : pd.DataFrame
+        Dataframe containing raw fastqc data
+    pivot_config : dict
+        Mapping defining pivot structure
+
+    Returns
+    -------
+    pd.DataFrame
+        created pivot table with computed columns
+    """
+    df["SPECIMEN"] = df["Sample"].astype(str).str[10:20]
+    # omit EPIC col for now
+    
path: resources/home/dnanexus/utils/parser.py
user: login: pofuoku
body: Done.
<!-- Reviewable comment -OLtDa-WByNXGFOqDKyP:b-896fix -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-21T16:28:24Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,230 @@
+"""Module for parsing fusion data from their respective DNAnexus files into a
+pandas data frame
+"""
+
+from concurrent.futures import ThreadPoolExecutor
+import dxpy
+import pandas as pd
+from dxpy import DXDataObject
+from typing import List
+
+from .utils import read_dxfile, create_pivot_table
+
+
+def parse_fastqc(dxfile: DXDataObject) -> pd.DataFrame:
+    """Parse the content of fastqc multiqc output and compute required 
+    metrics from deduplicated_percentage
+
+    Parameters
+    ----------
+    dxfile : DXDataObject
+        DNAnexus file object containing fastqc data to parse
+
+    Returns
+    -------
+    pd.DataFrame
+        pandas dataframe containing selected and computed fastqc metrics
+
+    Raises
+    ------
+    ValueError
+        if columns needed to compute extra metrics are absent.
+    """
+
+    df = read_dxfile(dxfile, include_fname=False)
+
+    # Verify required columns exist
+    required_cols = ["total_deduplicated_percentage", "Total Sequences"]
+    missing_cols = [col for col in required_cols if col not in df.columns]
+    if missing_cols:
+        raise ValueError(
+            f"Required columns missing from FastQC data: {missing_cols}"
+        )
+
+    df["Unique Reads"] = (
+        (df["total_deduplicated_percentage"] / 100.0) * df["Total Sequences"]
+        ).astype(int)
+    df["Duplicate Reads"] = (
+        df["Total Sequences"] - df["Unique Reads"]
+        ).astype(int)
+    df["Unique Reads(M)"] = df["Unique Reads"] / 1000000
+    df["Duplicate Reads(M)"] = df["Duplicate Reads"] / 1000000
+    df = df[
+        [
+            "Sample",
+            "Unique Reads",
+            "Duplicate Reads",
+            "Unique Reads(M)",
+            "Duplicate Reads(M)",
+        ]
+    ]
+
+    return df
+
+
+def make_fastqc_pivot(df:pd.DataFrame, pivot_config:dict) -> pd.DataFrame:
+    """generates FastQC pivot table 
+
+    Parameters
+    ----------
+    df : pd.DataFrame
+        Dataframe containing raw fastqc data
+    pivot_config : dict
+        Mapping defining pivot structure
+
+    Returns
+    -------
+    pd.DataFrame
+        created pivot table with computed columns
+    """
+    df["SPECIMEN"] = df["Sample"].astype(str).str[10:20]
+    # omit EPIC col for now
+    
+    pivot_df = create_pivot_table(df, pivot_config)
+    pivot_df = pivot_df.reset_index(drop=False)
+    
+    return pivot_df
+    
+    
+    
+def _parse_fusion_files(dxfiles: List[DXDataObject]) -> pd.DataFrame:
+    """
+    Reads and concatenates a list of DNAnexus fusion-related files
+    into a single DataFrame.
+
+    Parameters
+    ----------
+    dxfiles : List[DXDataObject]
+        A list of DNAnexus file objects.
+
+    Returns
+    -------
+    pd.DataFrame
+        A concatenated DataFrame containing the combined data
+        from all input files.
+    """
+    if not dxfiles:
+        return pd.DataFrame()
+
+    max_workers = min(16, len(dxfiles))
+    with ThreadPoolExecutor(max_workers=max_workers) as executor:
+        df = pd.concat(executor.map(read_dxfile, dxfiles))
+
+    return df
+
+
+def parse_fusion_inspector(dxfiles: List[DXDataObject]) -> pd.DataFrame:
+    """
+    Reads and concatenates an array of DNAnexus Fusion Inspector files.
+
+    Parameters
+    ----------
+    dxfiles : List[DXDataObject]
+        A list of DNAnexus file objects.
+
+    Returns
+    -------
+    pd.DataFrame
+        A concatenated DataFrame containing the combined data
+        from all input files.
+    """
+    # simply calls _parse_fusion_files for now
+    # can be easily extended if additional logic needed for tool
+    return _parse_fusion_files(dxfiles)
+
+
+def parse_star_fusion(dxfiles: List[DXDataObject]) -> pd.DataFrame:
+    """
+    Reads and concatenates an array of DNAnexus STAR-Fusion files.
+
+    Parameters
+    ----------
+    dxfiles : List[DXDataObject]
+        A list of DNAnexus file objects.
+
+    Returns
+    -------
+    pd.DataFrame
+        A concatenated DataFrame containing the combined data
+        from all input files.
+    """
+    # same as above; to allow further customisation per tool as needed.
+    return _parse_fusion_files(dxfiles)
+
+
+def make_sf_pivot(
+    sf_df: pd.DataFrame,
+    sf_runs_df: pd.DataFrame,
+    fastqc_pivot_df: pd.DataFrame,
+    fi_df: pd.DataFrame,
+    pivot_config: dict
+) -> pd.DataFrame:
+    """Generates STAR-Fusion pivot table with merged data
+    
+    Parameters
+    ----------
+    sf_df : pd.DataFrame
+        Main STAR-Fusion data
+    sf_runs_df : pd.DataFrame
+        Historical STAR-Fusion data
+    fastqc_pivot_df : pd.DataFrame
+        FastQC summary data
+    fi_df : pd.DataFrame
+        Main Fusion Inspector data
+    pivot_config : dict
+        Pivot configuration parameters
+
+    Returns
+    -------
+    pd.DataFrame
+        Created Pivot table with merged data
+    """
+    
+    df = sf_df.copy()
+    
+    # =MID(K{row},11,10)
+    if 'file_name' in sf_df.columns:
+        df['SPECIMEN'] = df['file_name'].astype(str).str[10:20]
+    
+        # =LEFT(K{row},28)
+        df['FNAME'] = df['file_name'].astype(str).str[:28]
path: resources/home/dnanexus/utils/parser.py
user: login: pofuoku
body: Done.
<!-- Reviewable comment -OLtLVY4BM9umsm55WXd:b-896fix -->
<!-- Sent from Reviewable.io -->

created_at: 2025-03-21T16:28:24Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,230 @@
+"""Module for parsing fusion data from their respective DNAnexus files into a
+pandas data frame
+"""
+
+from concurrent.futures import ThreadPoolExecutor
+import dxpy
+import pandas as pd
+from dxpy import DXDataObject
+from typing import List
+
+from .utils import read_dxfile, create_pivot_table
+
+
+def parse_fastqc(dxfile: DXDataObject) -> pd.DataFrame:
+    """Parse the content of fastqc multiqc output and compute required 
+    metrics from deduplicated_percentage
+
+    Parameters
+    ----------
+    dxfile : DXDataObject
+        DNAnexus file object containing fastqc data to parse
+
+    Returns
+    -------
+    pd.DataFrame
+        pandas dataframe containing selected and computed fastqc metrics
+
+    Raises
+    ------
+    ValueError
+        if columns needed to compute extra metrics are absent.
+    """
+
+    df = read_dxfile(dxfile, include_fname=False)
+
+    # Verify required columns exist
+    required_cols = ["total_deduplicated_percentage", "Total Sequences"]
+    missing_cols = [col for col in required_cols if col not in df.columns]
+    if missing_cols:
+        raise ValueError(
+            f"Required columns missing from FastQC data: {missing_cols}"
+        )
+
+    df["Unique Reads"] = (
+        (df["total_deduplicated_percentage"] / 100.0) * df["Total Sequences"]
+        ).astype(int)
+    df["Duplicate Reads"] = (
+        df["Total Sequences"] - df["Unique Reads"]
+        ).astype(int)
+    df["Unique Reads(M)"] = df["Unique Reads"] / 1000000
+    df["Duplicate Reads(M)"] = df["Duplicate Reads"] / 1000000
+    df = df[
+        [
+            "Sample",
+            "Unique Reads",
+            "Duplicate Reads",
+            "Unique Reads(M)",
+            "Duplicate Reads(M)",
+        ]
+    ]
+
+    return df
+
+
+def make_fastqc_pivot(df:pd.DataFrame, pivot_config:dict) -> pd.DataFrame:
+    """generates FastQC pivot table 
+
+    Parameters
+    ----------
+    df : pd.DataFrame
+        Dataframe containing raw fastqc data
+    pivot_config : dict
+        Mapping defining pivot structure
+
+    Returns
+    -------
+    pd.DataFrame
+        created pivot table with computed columns
+    """
+    df["SPECIMEN"] = df["Sample"].astype(str).str[10:20]
+    # omit EPIC col for now
+    
path: resources/home/dnanexus/utils/parser.py
user: login: coderabbitai[bot]
body: > Seems like the humans are having a chat. I'll hop back into my burrow for now. If you need me again, just tag @coderabbitai in a new comment, and I'll come hopping out!

<!-- This is an auto-generated mute comment by CodeRabbit -->
created_at: 2025-03-21T16:28:31Z
author_association: NONE
diff_hunk: @@ -0,0 +1,230 @@
+"""Module for parsing fusion data from their respective DNAnexus files into a
+pandas data frame
+"""
+
+from concurrent.futures import ThreadPoolExecutor
+import dxpy
+import pandas as pd
+from dxpy import DXDataObject
+from typing import List
+
+from .utils import read_dxfile, create_pivot_table
+
+
+def parse_fastqc(dxfile: DXDataObject) -> pd.DataFrame:
+    """Parse the content of fastqc multiqc output and compute required 
+    metrics from deduplicated_percentage
+
+    Parameters
+    ----------
+    dxfile : DXDataObject
+        DNAnexus file object containing fastqc data to parse
+
+    Returns
+    -------
+    pd.DataFrame
+        pandas dataframe containing selected and computed fastqc metrics
+
+    Raises
+    ------
+    ValueError
+        if columns needed to compute extra metrics are absent.
+    """
+
+    df = read_dxfile(dxfile, include_fname=False)
+
+    # Verify required columns exist
+    required_cols = ["total_deduplicated_percentage", "Total Sequences"]
+    missing_cols = [col for col in required_cols if col not in df.columns]
+    if missing_cols:
+        raise ValueError(
+            f"Required columns missing from FastQC data: {missing_cols}"
+        )
+
+    df["Unique Reads"] = (
+        (df["total_deduplicated_percentage"] / 100.0) * df["Total Sequences"]
+        ).astype(int)
+    df["Duplicate Reads"] = (
+        df["Total Sequences"] - df["Unique Reads"]
+        ).astype(int)
+    df["Unique Reads(M)"] = df["Unique Reads"] / 1000000
+    df["Duplicate Reads(M)"] = df["Duplicate Reads"] / 1000000
+    df = df[
+        [
+            "Sample",
+            "Unique Reads",
+            "Duplicate Reads",
+            "Unique Reads(M)",
+            "Duplicate Reads(M)",
+        ]
+    ]
+
+    return df
+
+
+def make_fastqc_pivot(df:pd.DataFrame, pivot_config:dict) -> pd.DataFrame:
+    """generates FastQC pivot table 
+
+    Parameters
+    ----------
+    df : pd.DataFrame
+        Dataframe containing raw fastqc data
+    pivot_config : dict
+        Mapping defining pivot structure
+
+    Returns
+    -------
+    pd.DataFrame
+        created pivot table with computed columns
+    """
+    df["SPECIMEN"] = df["Sample"].astype(str).str[10:20]
+    # omit EPIC col for now
+    
+    pivot_df = create_pivot_table(df, pivot_config)
+    pivot_df = pivot_df.reset_index(drop=False)
+    
+    return pivot_df
+    
+    
+    
+def _parse_fusion_files(dxfiles: List[DXDataObject]) -> pd.DataFrame:
+    """
+    Reads and concatenates a list of DNAnexus fusion-related files
+    into a single DataFrame.
+
+    Parameters
+    ----------
+    dxfiles : List[DXDataObject]
+        A list of DNAnexus file objects.
+
+    Returns
+    -------
+    pd.DataFrame
+        A concatenated DataFrame containing the combined data
+        from all input files.
+    """
+    if not dxfiles:
+        return pd.DataFrame()
+
+    max_workers = min(16, len(dxfiles))
+    with ThreadPoolExecutor(max_workers=max_workers) as executor:
+        df = pd.concat(executor.map(read_dxfile, dxfiles))
+
+    return df
+
+
+def parse_fusion_inspector(dxfiles: List[DXDataObject]) -> pd.DataFrame:
+    """
+    Reads and concatenates an array of DNAnexus Fusion Inspector files.
+
+    Parameters
+    ----------
+    dxfiles : List[DXDataObject]
+        A list of DNAnexus file objects.
+
+    Returns
+    -------
+    pd.DataFrame
+        A concatenated DataFrame containing the combined data
+        from all input files.
+    """
+    # simply calls _parse_fusion_files for now
+    # can be easily extended if additional logic needed for tool
+    return _parse_fusion_files(dxfiles)
+
+
+def parse_star_fusion(dxfiles: List[DXDataObject]) -> pd.DataFrame:
+    """
+    Reads and concatenates an array of DNAnexus STAR-Fusion files.
+
+    Parameters
+    ----------
+    dxfiles : List[DXDataObject]
+        A list of DNAnexus file objects.
+
+    Returns
+    -------
+    pd.DataFrame
+        A concatenated DataFrame containing the combined data
+        from all input files.
+    """
+    # same as above; to allow further customisation per tool as needed.
+    return _parse_fusion_files(dxfiles)
+
+
+def make_sf_pivot(
+    sf_df: pd.DataFrame,
+    sf_runs_df: pd.DataFrame,
+    fastqc_pivot_df: pd.DataFrame,
+    fi_df: pd.DataFrame,
+    pivot_config: dict
+) -> pd.DataFrame:
+    """Generates STAR-Fusion pivot table with merged data
+    
+    Parameters
+    ----------
+    sf_df : pd.DataFrame
+        Main STAR-Fusion data
+    sf_runs_df : pd.DataFrame
+        Historical STAR-Fusion data
+    fastqc_pivot_df : pd.DataFrame
+        FastQC summary data
+    fi_df : pd.DataFrame
+        Main Fusion Inspector data
+    pivot_config : dict
+        Pivot configuration parameters
+
+    Returns
+    -------
+    pd.DataFrame
+        Created Pivot table with merged data
+    """
+    
+    df = sf_df.copy()
+    
+    # =MID(K{row},11,10)
+    if 'file_name' in sf_df.columns:
+        df['SPECIMEN'] = df['file_name'].astype(str).str[10:20]
+    
+        # =LEFT(K{row},28)
+        df['FNAME'] = df['file_name'].astype(str).str[:28]
path: resources/home/dnanexus/utils/parser.py
user: login: coderabbitai[bot]
body: @pofuoku Thanks for implementing the more robust pattern matching approach. This will make your code more resilient to variations in input formats. 

(\_/) 
(•ᴗ•) 
/ >👍

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-03-21T16:28:48Z
author_association: NONE
diff_hunk: @@ -0,0 +1,273 @@
+"""general and I/O utilities
+"""
+
+import os
+
+import dxpy
+import openpyxl
+import pandas as pd
+from dxpy import DXDataObject
+from openpyxl.styles import Font, PatternFill
+from openpyxl.utils.dataframe import dataframe_to_rows
+from openpyxl.worksheet.worksheet import Worksheet
+
+
+def read_dxfile(
+    dxfile: DXDataObject, sep: str = "\t", include_fname: bool = True
+) -> pd.DataFrame:
+    """reads a DNAnexus file object into a pandas dataframe
+
+    Parameters
+    ----------
+    dxfile : DXDataObject
+        An instance of DXDataObject; behaves like a python file object.
+    sep : str
+        Delimeter of data values in files. Defaults to  "\t"
+    include_fname : bool
+        Specifies whether to set file name as first column. Defaults to True
+
+    Returns
+    -------
+    pd.DataFrame
+        An instance of DataFrame with file content
+    """
+
+    df = pd.read_csv(dxfile, sep=sep)
+
+    if include_fname:
+        fname = dxfile.name
+        df.insert(0, "file_name", fname)
+
+    return df
+
+
+def _add_extra_columns(
+    worksheet: Worksheet, extra_cols: dict[str, str], start: int = 1
+) -> None:
+    """
+    Inserts additional columns with formulas at the beginning of a sheet
+
+    Parameters
+    ----------
+    worksheet : Worksheet
+        The worksheet where columns will be added.
+    extra_cols : dict[str, str]
+        A mapping of column names to Excel formulas
+    start : int, optional
+        The column index (1-based) where extra columns should be inserted.
+        Defaults to 1 (beginning of the sheet).
+
+    Returns
+    -------
+    None
+    """
+
+    num_cols = len(extra_cols)
+    worksheet.insert_cols(start, amount=num_cols)
+
+    for i, (col, formula) in enumerate(extra_cols.items(), start=start):
+        worksheet.cell(row=1, column=i, value=col)
+        for row in range(2, worksheet.max_row + 1):
+            worksheet.cell(row=row, column=i, value=formula.replace("{row}", str(row)))
+
+
+def _apply_header_format(worksheet) -> None:
+    """
+    Applies bold formatting to all headers in the sheet.
+
+    Parameters
+    ----------
+    worksheet : Worksheet
+        The worksheet where header formatting will be applied.
+
+    Returns
+    -------
+    None
+    """
+    header_font = Font(bold=True)
+    for col in range(1, worksheet.max_column + 1):
+        cell = worksheet.cell(row=1, column=col)
+        cell.font = header_font
+
+
+def _adjust_column_widths(worksheet, min_width=14, max_width=40):
+    """
+    Adjusts column widths for better visibility.
+
+    Parameters
+    ----------
+    worksheet : Worksheet
+        The worksheet where column widths will be adjusted.
+    min_width : int
+        The min width a column should have.
+    max_width : int
+        The max width a column should have.
+
+    Returns
+    -------
+    None
+    """
+    for col_cells in worksheet.columns:
+        col_letter = col_cells[0].column_letter
+        _max = min_width
+
+        for cell in col_cells:
+            val = cell.value
+            if val and not (isinstance(val, str) and val.startswith("=")):
+                _max = max(_max, len(str(val)))
+
+        # Set column width with a cap of max_width
+        worksheet.column_dimensions[col_letter].width = min(_max + 2, max_width)
+
+
+def _set_tab_color(worksheet, hex_color: str) -> None:
+    """Sets worksheet tab colour
+
+    Parameters
+    ----------
+    worksheet : Worksheet
+        The worksheet to apply tab colour
+    hex_color : str
+        Hex colour code for the sheet tab
+    """
+    worksheet.sheet_properties.tabColor = hex_color
+
+
+def highlight_max_ffpm(worksheet, source_df, ffpm_col="FFPM", index_col="SPECIMEN"):
+    """
+    Conditionally formats rows with maximum FFPM value per specimen(index).
+
+    Parameters
+    ----------
+    worksheet : Worksheet
+        The worksheet where conditional formatting will be applied
+    source_df : pandas.DataFrame
+        The source data written to sheet
+    ffpm_col : str, optional
+        Name of the FFPM column in the DataFrame, default is "FFPM"
+    index_col : str, optional
+        Name of the column to use for grouping ffpm, default is "SPECIMEN"
+
+    Returns
+    -------
+    None
+    """
+
+    # Create a light green fill pattern for highlighting
+    green_fill = PatternFill(
+        start_color="CCFFCC", end_color="CCFFCC", fill_type="solid"
+    )
+
+    # Reset index if it's a multi-index df to make processing easier
+    if isinstance(source_df.index, pd.MultiIndex):
+        df = source_df.reset_index()
+    else:
+        df = source_df.copy()
+
+    # Find the max FFPM per specimen
+    max_ffpm = df.groupby(index_col)[ffpm_col].transform("max")
+    max_rows = df[(df[ffpm_col] == max_ffpm)].index.tolist()
+
+    # Apply conditional formatting
+    for idx in max_rows:
+        row_idx = idx + 2  # adjust for 0-indexing in pandas
+        for col_idx in range(1, worksheet.max_column + 1):
+            cell = worksheet.cell(row=row_idx, column=col_idx)
+            cell.fill = green_fill
+
path: resources/home/dnanexus/utils/utils.py
user: login: coderabbitai[bot]
body: _💡 Verification agent_

<details>
<summary>🧩 Analysis chain</summary>

**Validate existence of columns before highlighting**  
The code uses `ffpm_col` and `index_col` directly, which can cause a runtime KeyError if these columns are absent. Consider validating that the DataFrame contains these columns or handle the error gracefully.  

You can run a quick code search to confirm how and where this function is called, ensuring the passed DataFrame contains the required columns:

---

🏁 Script executed:

```shell
#!/bin/bash
rg -A 5 "highlight_max_ffpm" .
```

Length of output: 1379

---



</details>

**Action: Add Validation for Required DataFrame Columns**  
Before performing any operations on the DataFrame, verify that both the `ffpm_col` (default "FFPM") and the `index_col` (default "SPECIMEN") exist. For example, insert a condition like:

```python
if ffpm_col not in df.columns or index_col not in df.columns:
    raise KeyError(f"Missing required column(s): {ffpm_col} and/or {index_col}")
```

This check will prevent runtime KeyErrors if the DataFrame passed (e.g. from `generate_fusion_workbook.py`) lacks these columns.  
- **File:** `resources/home/dnanexus/utils/utils.py` (Lines 136-177)

<!-- This is an auto-generated comment by CodeRabbit -->
created_at: 2025-03-21T16:35:56Z
author_association: NONE

PR comments:
user: login: coderabbitai[bot]
created_at: 2025-03-07T17:15:13Z
body: <!-- This is an auto-generated comment: summarize by coderabbit.ai -->
<!-- walkthrough_start -->

## Walkthrough
This pull request introduces a new application, **eggd_generate_fusion_workbook**, by replacing the former DNAnexus app template. The update includes revised documentation, a new JSON application configuration, and a Python script that processes genomic data inputs to generate and upload an Excel fusion workbook. Additionally, supporting utility modules have been added for file parsing, Excel sheet generation, and default configurations. A VSCode settings file has also been introduced to automate code formatting and enforce the NumPy docstring style.

## Changes

| File(s)                                                                                       | Change Summary                                                                                                                                                                                                                                     |
|-----------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `README.md`, `dxapp.json`                                                                     | Updated documentation with a new title and description; introduced a JSON configuration that defines metadata, input/output specifications, runtime environment, and access controls for the fusion workbook application.                     |
| `resources/home/dnanexus/generate_fusion_workbook.py`                                         | Added a new Python script to process genomic data (STAR-Fusion, Fusion Inspector, FastQC, previous runs), generate an Excel fusion workbook with multiple sheets, and upload the workbook to the DX platform.                              |
| `resources/home/dnanexus/utils/utils.py`, `resources/home/dnanexus/utils/defaults.py`,<br>`resources/home/dnanexus/utils/parser.py` | Implemented utility modules for file reading, data parsing, Excel sheet creation and formatting, and default configurations to support the fusion workbook generation process.                                                              |
| `.vscode/settings.json`                                                                       | Created a new VSCode configuration file to enforce auto-formatting and set the auto-generated docstring style to NumPy, enhancing code consistency in the development environment.                                                         |

## Sequence Diagram(s)
```mermaid
sequenceDiagram
    participant U as User
    participant GW as Fusion Workbook Generator
    participant P as Parser Module
    participant UTL as Utils Module
    participant DX as DX Platform

    U->>GW: Trigger application execution
    GW->>P: Parse STAR-Fusion & Fusion Inspector files
    GW->>P: Parse FastQC data
    GW->>UTL: Read previous runs data & get project info
    GW->>UTL: Write DataFrames to Excel sheets
    UTL->>GW: Return formatted workbook
    GW->>DX: Upload fusion workbook
    DX->>GW: Return workbook link
    GW->>U: Output fusion workbook link
```

<!-- walkthrough_end -->
<!-- internal state start -->


<!-- DwQgtGAEAqAWCWBnSTIEMB26CuAXA9mAOYCmGJATmriQCaQDG+Ats2bgFyQAOFk+AIwBWJBrngA3EsgEBPRvlqU0AgfFwA6NPEgQAfACgjoCEYDEZyAAUASpADK2VmgqyjcEj2wAbb5AokAI7Y0riQ4rjedJAARABi2Ijw+FgA6vgUANYC+PiZMZAA7mjIiNgCzOo09HLhsJ6JlDz4AGbYedjoGPSo8EoY4i3w0SWQGE4CTQCMGjD11nYMmJCT6AwM0ogjYbjzLfi++IXwGESQAKo2ADJcANoJSSmQ6Vk5eQsAugAUsLi43IgOAB6IFEdSwcoaJjMIEkEq4UgYFjwBiIWFEIi0AD6iOUNCxbUeGCxhQy2VymSB3B83iBUwAlBp3PMlIgGBR4NxxE9WnVPNTfP4giFEGETrQUdRpHVqCgxRgGN5sKyuigMOp4Gg/NDKmFCvUAnz0NxuN5JdysKh8Nw8ckMFqiuDILQnMx5CdqbhELMAMLYCgBAbeWQAGiNpNeFIUGCSouQvKEiTCKU8iHqJE0kAAgrQJRatcGw2hIGaMJlwvh0JAhIJIMwUuoMicztw0KR+Fhdp4ACIAOSz5AAHokeN5qPsKMwUMhePgJH1ohOhS1KGQNkyDB4cLsMpBYKN8HhS9EtlIqH5cCVMshdrKXPyxxhyPQl21cP7PPBmKaSGwBtQ7TDfUUVgNVFWVEgOAMKAcwlU50FzDUUgdNMSAzG9Ky7IoyTeTJoMgABJBUMm4DIAPg7hJHwHYVCiRB8II78KDnZsjQAA1oQc0BNDQhEQFI2OjIYiH9ACniGKJ8Ng1jaHwVFcA5U5kCXSY/iaJglGdOSnHYMSMHwn191OVisMPf48EgCTPHtNgK2XKIxCNWcREcmySHw85uFociziwmwAFEs27ABZfzLPgKI7NnedNPrQ1oV4Eh6hjSRPwwCdmD05l+RpIVglCadwjbUh6GOXYjUaPgAAEsyQfcwG7MNxXNEz9x2ep5Ey+RVhOCQDikHpO3mAJ5xIQp+D4JQpG8a0/zCWcNkQb1zEsH0WDm5BHGcVwjCIo0dXYUpRAtfgWic3KAny0UwywyrmjaDoeGYmKRhLE5ywIVUSC479IprAQOyNPsB2+kdTXHDJmCAhAGFApYsFWNB1k2bY7n8n6f0gAApQRvl+f5ARBcHcAyjRaHtIdEihFgqUwQcqWYlyvSBABxAAvAAWTI4m4ABFDmAA44gAZgATQADTZgA1ABODnB3F3mACYlYABl5oF63VAgKCBf7WbZrGAGlZGEQWRd5gAhFnLeFtnB15gB2I3AgEOJGTmVBS3LLYKCkZBRmLAIV0DDZLN3JQtdFKhxHgrC2gVfMzVweReSw7jTRap4Tn2lIaEHZMztM058Bkkhptm9hTqNB47WeHCKVmIxe0rNBEKThRWEOibnSQBhEiJZBClXMZqOiHOsIOgYjrEQC1QlJZY98tqnI5TLXHD/vkACTKTnjIa0o1B0dXULp6FMo8TkXPAPyKxBr1mLckl+lF1FTov5inr1IDKDFQhvFe6c8CwF3L0A+kA4QUGDD/S8pB4xnSmiQGa3A5pn0KlsEgmQTKVlCLROqVZ9jYG6HpcOfA2gUC7HwMgRkNgbTsunE0ZpF52g3PoYw4AoBkHoLyNAeBCC4hjtEL+XBeD8GEMdVKMh5AaWUKodQWgdBsJMFAOAvQA5YF4QQYgZA8RCPWuwLgVBxplG2t1aRihZFqE0NoXQYBDDsNMAYAKQVQoaGYLQKCMQvEGAsNmAi2jyCCPoCY9e79GBGTgdlcJmA4FqnYs4kK/k3G0EElZNUfVvBSCrAlKINAhTg1oVXNO8xvpICXuEdQkVMD0FZOyTkFpH7zESvOQ8N5KkkDDGxYGlNEBYgzliGgv0pRsSLN0OUAd1gsFbBgWQZc2Qci5HPGIqjkBMB8DUBoWx6CBxgS4cppETh6mSs6cuSDrSsXIONDOyldzdNBogEMBR9xZMmGQIU9YBqNJIPIZ5nhXlYACAU6IZVQLFkuRUyIHTIBsRIBibEAipQEkSHaEkDd3hfDucOZAVgxwk0htmE09IRloOLEoS8EVoi1IWSdW8yZL7kAAZ4DOAByZSRDZ7IWTqGPK2B4ABEGp6B5/AbQxztChGgAIwxmUFaM7ZMAADy3Z5VvVFKQyyN9DTUMwIU6eXyxhjWjDQAYYElQqi2BymMMowiIBtAweALR5D6llAw7gWlpA3WGsKPl48MAyr3EcOUwrdFCuqUUFep9pV4BDTNeCIKqxmlVbyN8t8vzRV/IdDcviszeBoKKlIGEjRKEVC4PS8CIGDlIhQ6Iu5qQCCYRAgYGppBGBUe0yA2AvJSg8dCywmLEh9JNAM38xMSCCRzmxBJrj3GCUAEmEPaIFwpxDowRSKiSosjOivt2LcUZQJdwIlcSJ2BUSckti+FuzSDqYsp4HbvLVC4GxFZCh1krE2SMAOuyKGsQOca/UbzEHIIuQa65aqt1jqwEelxSTp2QDnRe8lURgkSPzQhJQ9AAhEBcHBM4Gd1WJzFVypqvqo1hgWpsZsUq8B+tDdARVyr1C/mQOOyd0GUkGC8TEFtjjOIZ14vxDAnjvFZv8Qi6oP9XQuDCbDGJzaDBZn1eNLG9h5W9iEvAESeasBpKXC6phJCYWLtEyQVdKKIzkjyIJfcMg0KWgGMxF0GxaCNNQGkk44EVRsEvHe4sZRYboEZcaTOzCUisrGGgNgN12lhhCZJsMWYrAEUgGeIksqkuUCJGMCYlBZgEWtba+1wwAsegsja0QBXguWsury/lardNZwwNDECJqII/1OeeOs1TqAZHkK54jXoH2ihcISFFVlEDErYsNhsMZbU6wJBSsbnSWjwkCAwLE3nxv2DiFiZpyR+0UCIb09bqW2JxAIttkau3en7ZjGt6gaBiVwj80oIYz5HTlWoApeAAg8DSl86C5AbkwxjkmN4MMxalpSuvfaPwg33whrGfUbwrqqX1LtN6SAUTI15bK0MCrJyXvSh882SKWPwqRTcvQCbyKUjrvM5kYl4OkhDHfQQil4RZA2je6CjRyMuTRFbGpCgWBeRsQAFQaEHN4RAg5iVahSGcONxZEfI8vdSlhcxPDXZ/vl3HpaTkIaNN9UQeA65kHnMxDAc0iPudYsWcQbAzLNCYWEpW/r/RCqwocygiVc12VWI0egXwrCyB3FgYW9IPX8moKBT6WF5n1MgF8NiAR+L+kWkCEBbAgTk0wPc0Ey7EWTeJGZ3CGhuCyDYhHtBcfZCil/L3aOX2TdPFDQEKIJRPDJbrl8c432BidCVhzDQqsOYe2kknQsRokaLVWXnZiMOdeFaevgJmYAojTR4JQSoS00epYwwR4VFoA6GmiguF8u4sypHsHErCMLsBgA2HZrUYApiCX3ykMMZLtCljOIgWvgyeVvU6Elw95LwFRPAU4bR0ctwmB0p1NRITotR+JmsVQANrQmhQ1NEQEOQ2Zog7oTgMo9JM1LBs1c09dY8WRRAxxNMy1vpK0xMa1yh612Am16J8JexgNGF6s1MNM9IH1YVMQl1AlC9qdi80V6dD0eMeI+IBJ2NvFoJHEU9DwKB09M8SBs8KY88jMTMacS9G5y9BNONhMAldFgkJMN5eRpNThZN5NwVg9Q8f4r0wg2JtCi9adS9y9LNRh/k1QFJFBsBHNP9pB1NXtPojMCFRDsIN1ywWhmIpwJAXBLtIBERkQGBnQ7s1RBU9V48uRwlRBrwUAzpT5UBrt1R4IJ55gt0G1zcUhUFQ1QCCwAdRBNhJMeAkZMg2xpRYiWB405IHQJQAgxBut20kh4I2JKJuA2IcsxRvwMhv4UZG0HQzQBAqAORfsAj/toV8AFtoUiAZoBBxsygBAyMlpxsrgylUk4joVIDmxxtOJPDOl0DfVZBJdZcxlxjOsdi5dY0nQTcuU8MLUblJoMiTjRiiANwtxd4tN2Uk435Co1xuIyhcUfV2IoTUkYS55gI/NUB15MgWdlh2AN4f0wgdNKjxZRwIZJxnM2V8MW9edv5TxlA/BWwqBPN0sBtLwKA3DRtxsi894ZsMg5s6JeTltVsjtoVNtzty5LssRrtDs7s3jKdTspSWk9sDtbtLwpjIB/IkZQJisxRZ8MAzwxNDlW5oV7jZANBuxxY4gKVBJBAmY1VyFKFl8Z9mwIT5gcj5pmIZ9pRb9BsuTRChTpBBJQ0qciR+TjpBSeSRjWpPAPiKAtgsQAydCMBQz3iWSky3DIyhiKBBI/i4SE5AS7J2Q4Q8luw7s4hWTpBpj0ApdKwAg24ZwLtWl/AiF0jLw1V7BoAswbAwBa4W8xkBysAiJStczYy455hk84RsROIrJ0TaSMA9VnJjowtbISiMw1iBoJyzgXCMxzsV9josQCD8BFTVRScrI3JCojMnMjABxtTBwNg/BCgORfcHTVywEj4zRcDz5KxSypRzyqMLI0kKdoUABvFcsQLENyAAX1TPcMbkl2ly1KficKKFfOlGYB8HEExlQnQnoXmD0LyGt1NVtywGHRTm1ISx9B/nTFwFIxbJHDlI7OLCXG7N7P7MiNDWHMImmyjIoDDDiHhF5hou81SywlBOiHYr7J4u4siNHIFOBMvDDGRUnPjJfIYzWxaAGXwGTLovzPEH+KLIaWzHbgI0n1Pn/JoA/SEtFBEp4CohojrSZTGQ0ussDQoM8CIsyA3FtOh0nywhvKiLp0Kg7RmjbmiE8sgGtIpLxUnFS2LG9gIoaFNHwAipfDZw3PfCFxZxZMLiNFJwXnzFcD1VgL8L8BaBmnGjc1NWlH1KSy1D6D0k/xBJ9PI1OFSzSSMloB/zDDIDKEUmXmdXmCCu8sKiYADGOmgUILUm2TGTCrSrQ2IOzBzVtBQyiqLWoL115DoLmOrT4FrWYMbXEFkygASEXNQzoAfWew6xOC+ADO5Pmy4HONFFuCKo+DDD5L4tzODMBEgBetwDepRFwA+sslFI1LQC4CKrDElJ21aVlPVO8yhuBsErOzhrVJuyRt7jEAPXHSULT2kAzxYHUJzx6Xz2EPxDcO8rLwrzkM4wULACMHxpUMJrUI0NzyxSBD+LRG5ppsMJ8RIJEwLzExiwsLOisMiTkwU3bUMrhPrBdEiis1fTeUOXswCJROnNTxZrRDZtJrz25q5sMu9AeOXxilYgRT8FDQIiBGVWMoI1YNrJqoggC2nLbjW0HHnIBItEaz80bNoA/SqLSXfLEDDEBRT3YHxPNIrUtOtIrMvHlXETEGJVNKrGmW8mQDjrQCrPCxIGpK9rrmnxIC5BnBLTZMTLVTHPtVmTUpOTNF1CaADxvz2DZ1DT/RdM+ids0njjZyvNGG7sTLCCYCVGYFsyBkrOrL1TtpvVltwOQHGKcz9qxFRAkEEk+nCvPhZAyNDVgLzHMvdBjEoG/m7vJxzpWA724XATYk7uMxaDclDIoBEjmg9M8Hlp8CZXrJQL9KnLcuM1oG0oID0rQlwAXItR9tAh/o/TTtGEzuztsk+mWHRifNoqAZ3NVHno0EQaQVSAwrzLESZjzqno0UOEKFWSTGRDZhIWKVTDorXKhUvABiHoyHEvmCdoyx4TMs5QUGHstTjQyh8BKFrJmoC3qAivLpzhyG8Dmu2VoETDjFzm4cdFoF2CBNfUFySyQC+wijhIEHPsBknjziriiFOF2FmHHz3sj2hT6VzCxG+gUjQCXoOCcBjHGwHVNFkCxBEaUAoAJEhmoDPLYisdkdwAce4ZJD6GUcsyQRFXzpQ3vHxyvl/I63tHbFsaoC4acaFU8aaBmqXlSyHqccUfexkaTA2lDukBm1SmDD1QCa2GCfoZCYyBAZOlqYCzMzwqcq4YyGfuhSsuM0oj6jqdolHRiawF6cgccqKmcssiuOLBgerLPq2UBmLFPzQx4IQLriKrFVcCLGIdYiXGKgwxIVDSIGYg7VYl5G82qdIGCcguCZPKabrgCE+1OQC1udobQQIm7GmZ6KwiqLNz5VqPYA3H8gDF3G6p/0KiRDCDoKYUqU6hYJTEGlXhYlWZkWYc8DKvn0siqqei/ESOgVgNjG/ksJSF3s4dhnyJUZZJztzXUR6DIKP34DPGivHtPvyZHrpf8COGWgFpWoZbR2SpOWLRoOrl2qrW4QOqYJRAbQiEKxbUgGCgzBAWkbQ2upIDOkXrnIpS+C1aiC4FjruwTqZmiyLq4GjkgAAF5YgAAdXAGIEiiCAkNyLgN4PwK16AfbEgekXQPQHgJzOZnO8DaFZm1Q4m9msmg23mzw/CRVncFVq6809V9CzSv+nSwBjML4H+igERJzTB7wbBhjAS50FoHNq01liLZB/c51mBIt+pxh7Nmty12IVWFt1t+18tOxkJv6oq24aOaLBSD4JtlucgB1pQY87ob6F13IN1yAISqXL1n1yAYd4ZvG6QZQ0NrPPWzmyNo2mms9KAWN5Vy67tDiJN3p7bRygZIZnVktv1sty8WBqFfp6iBxuAogZGnGxdrye9rO6soNzW9d1msNrdxIQ2iKHm3d6Ng9pVxQY9tVs6a5g8pmcd/YL4b1uxcIMKkgXthSftigD4f9kNoDzdzQ7do2sDqXPdmNmD+Nk9m6hAIgWAM0Rj4JzKQcAkFoFBTNskdp6LQDrSz6zj5gEJi1+IOIKwYKdt8Ub6ETmIewKwfyH0AiUKXsGIXGiDIjnW4D0j0Dnd8Dqj6DuNuDxNs6Kx7EVJ+x9lmMbjrIdprgF4a8OivqguKgLtj9wGvtmt0GgMzgXwpthkRd5dwjtdgmrTkjjm3T8jqN2mwzo9tuVVkzyAVx4MDxmcygHxycagGzxzoB9D31oLw9TTom8LiNqLyDmLhVmj4z090ztuIJkJpxsJpR2ARAbL3jusE4Jr3YC1qYDmMMNjrr2AC1jmVWPLpdlMYLrWjdkmnTiD8Dij42irw92D+LhNmrpL2pq9gQBpigNrpzvcGT+ts1hSMbgr1dqb4jmbiLubqXBbqjjjLjRmgwDQCQNkCxIEWppeb0GQgTOm3lrMIWimk8cwqTCJGw6W2A4SdZ8SNnKY17mRD7jML7vjASPcbwmzXwtWwIsNJra+o6P4ZsFRrCKWJAbAB0ewd8CUSsNaTSOgRsCgbIpHwn7XHHeQWlI0GatVTRfhYWylOSRvJSR0QUfYYho0XsJwYPGBWQKIZh4azwOnnWWiw8SR7cFgACJYXwHrXxweixfzCqX2dnKAjcFaPltay1DaqgktRlnaitPaiVrwOtaVlgk6tgqADg8aSH+AzTMnDFxs+9aFF7t7pQRHgnpSFHtMrnVrUPogFRtibn7sfnz7U4MmRPwauIbX9MynBXrpma+VDAaADnUdJkB7hmpmkL7W4rq70r+b57XhHNRb/m4woK0W0HmTNg2wg1V+xW9HlWuzfwxzQr8v6b8N/W8j2v7Cxb4lLEuGPObQS1cfnNaMQbaeNVBIjkVs9ptZkVxunOUlVquSdqs4Mq2xvVLvzwZ7K+ZALCnNTkSKT33gk6TZ5CNYlRiUFoEOKudpoVTu1iE7LMbskSliHsAAAJfyP5GgBYgfQKmU7CzA2xbYQBYAiAVAN7AwDxsypBAeAMgHQCCIsAzpJKVsD+QpYBEeVOcHsBYgbA5wXsGQIwFIDsBuA6FMqQIFECSBZAigVQKAGgDMByA1Acdn8jUUOBiArASgJwFakdSfme/tDywBP97QG8SuszgCyb9AckzTpkW0wIcNoc6TDlpHz4ZjhX+GRCzhajQSZR1QAoIghrlITX9RgBZXAg+SQaENSgGxPXmxGCifNxsUsK4PKnlRGxzgVgM4v5DiDQBxsyAn0FmECHHZghoQ/yP2GgD+RiUCGDIJFUrByDpW3mGtgEWyqE4xkVWIAhmnMEwoXOlnA4GNnChIJ6Aa/JfEjGYhLQt+paICE6H4i2Qyqc/MuAhkXCQx+GnLaiAaHx52Q2IQXIjAvB8ijxUMSEDQVZ2PyphF8dAUxuoILDcpcApICHikCh7e9pBiRd1AwP/7QBABCWKWPKloHCDYBaCNiPgIIh7CDhPAtHi8gx6rdaAYDfgPSggKekFI6Q2+EuGfZOU6IQoZEokywi2VcA9lUNFtFCSVsvQ0WHXNXTODSdBwYYBIkqA2EHNYUJCeweJWogOhmI1VBUEqCJDLVSCZvAtFhE2pW8BWNvegvtQd5HVZWp1JdgakJZgEwgtwh9HEC2GACaBQg1AYPwu5hcq+o/Gvuqzr5egDONIj3vmnpHVdJSbI7gSIM5GAduRI/MjnyKWwT8hR7vJfmKMZEMCzskougZN1lGV95RkXRUQKMn7sFaRoozAAyNzBrd8BAUZgaQPIGUDqBnAi4dKPO56jdas3Bbtnn5HKioOwotURaOq6MDbRxA+0WwKdGCCpRRwt0aF31EgcbuaIBfoKL9Gqi6RgYjUWxD4EEQfQAgrgTqJlGxiPR13L0UmJNFu8zRMYdUVaJPbMiABOY3YfsPZGuiNOQ/S7gaITHeilR9fFURWOX6WiEuJwrbA2JdHRiWxXIuMZ6INqlj7u8hCAE9yK5Fjq+t3TMtlgMJ/cm+vPMwqYmrgS1we4KM/mj2sy98/CDmNoQdRcBgl1UGWVId0SnCB15svhM0lAwzrltc65gg8Xj1axngHQ9gqCAYBmCQBRcouBMkmUrp/hjytAe6uFh/DHcKAY3aOGLlFxcAn0hDDtlQDEAKD8sf4QiF81vFVhEA0Ek+rZByYs42IUwJWMLA5gABWAAGyOwBYYAJWOLGYnix7ALExiVRKsAhDewHMF/C2ymBTAgBwsYWFiCuD8TyBr+DcErFmBASQJxmdTBIGTJYSyA0FHOlBN+iQQa28EhSIhOQkIAaShggwd/B8zKSsAV5HODgKlgWDZQeEuPKfQIkaTaGG4YWDJOAkriCQopHVh7QpT6txYmdI1sdDG7fsA2bAXSZ7AMknQVxyAf4fZWv7iAVsDw8yCSRmbRV+wPSH3rWT9oBY0kqlXcpq28lRAGc9QBgAUSXDZCas4wxPDEAICXhvAa2OgGFXNB0BtslAR/LAhIAFBQ0yyNEX4HsDCg1w0gNTnvm0BbAP0bEKWFqBCAgtmIuDe1F0HkBxNt8YJWsglB+xX9NyKIBwX5lGAxBzg6ofKJABsAzlEAnUsZDEG7CNTF4ngI6U2QKA6NFmTwGqShH6ngFOWTzf0JalmavjDUzQ+CFsAchiZPMHIVEBuCHyATgJmUPEh5NFArYL2AzG9qWxClPtL29/dzkFP9aviwpKEjEk8CMzjMBmkzSKLZPmAxTRKd2FSjPVtym0z8NQk6KSX5ATNo4rwgIBuComuSAm7kx6nRC8mjZnqZSW4Aa3jqJ0Qa6Mn9o+yxn6TcWCRPJKhKyloJYCV0+0O5USplJq4gdUQmADbxdofeTGAYGaTBKRQkZtZawYTmhRwA/aVgadujGNw6xV6lYalr4CQTwAbBWEP2uczOijY+qMYf0KxHVa45hgxqUEu6UIgfw0onoFVAaQgTfgU4TUMIO9KFzqIo5XIeQEbMIgMiP6n4ygDNLR7dAIWr4NnEHOT5GAaJ7MzmUGRzI6weZT1f6vzMFloAApYgD4KLKRkSyXMOMrAL7H9h69iwL5DOE0HZ4a8pcljMuWuhjKfQ/a8slIIrIAo8UFK/FHWRuEdilyLxxmFMkXirl0Q+Zr1OuQ3JFlfsMZD7P9kBK4D2AvwEUFwL0JHkooK5jTD1G3IuqDy5622FefBTHmVhwWngaSpxQyyjYNwAsdmZDNXnaUPh91bSn/URmvjos2lOUlpQgWHyc6n1cGh8NgV3skZn1eACguCmQKHKAzV9sJDRn7yxZR8pCeFJGbJFeeH6L+TxQ+GEy/k8LB+jJAyJ4S4pt/HsK+O/5YjlQrEfSTrElC9SeyMlSIqkO4rCUaKQMzaRuHlTfjfAFjA8Ss1+wvCxAH4egJmVYhLBWwagZOEviXBv8P+xqSAtKCTRCKmF82L2QNVahy99UM+VouItRDoB4oUyH7FIyFDZVXsu/XZMQgoASgfy1kzQCbzxEisLewrbamdDFYMFJWjvNIs7zlbUcjOGYm6u5LAkqS+g6kmCVpMXYIS4kB4gDoWO07FiDaK4hnn6OW60d4ObRRMvJKICKSklxINyKkr1bpKMOmSnONkoXF5KlxaIQpUKJKXVcElL8pbDDIYAbzNJO84Wc3MxlZL/CkUHJRX0XG8jlxK8opUtyq7xKk2gC6GbgFhkgLwFqC7BcgtRnY1cA4y+BaFMmUK14ybSkrvMs6WLLulKy6sWUufkVLX582YZX9QBoCy/JhrYWU3MIUtyzlb9YNq2LlHxivRXS4pfcoHF9Lnl2Zb6pXN1bSAt5gNUZUzF+UYcsFJyldlgFaXAqJx+S8juCuWVxKHliXRJZyVTJvKkVny/yT8uOW/tA2AK6ZZcp5EKiFlFSpZfu0q7EqoVaytAFDMQDALHKoCzBQfPpUVsBVCNTGrewxViqoUAyzZatmQU7KZVj7dBSKqIUIKcFL7A5UVTpXizGVFy3FXMtZU3L2VM4+mmwgMDKIG03CM6NzxMJBJO4c0QxGgGMQg8Vg5iLxioCsQKJbEhga1SfDub+1ZS0pMaM1J85+qHEUAGiUrDQCqxhYSsXrkrHWACwBYSsGWLQBlg0SBAHMDmErBXCqwSAVEqiSQBlhUSGAK4BgMLBokyxhYVEyNdaqoloAOYAgOiVRJlhxraAAsR2AwFVi0SBYbcFcO2oFhTAy1VEgQCOo5hTAaJvXHRjRIbUcJnQNa2NfFwEAMABASsEdcLAEBUSWgNEyiTRJaAMApgJAE9UrAECZqlgLQKiV2toCu4lEi6qtSQGFiqw0AAsaiWmrljZqP1I+HNQwBoloAy1tAWiUrEdgJqT1K4GiZ4AfUQAVgYGmNTLDVhoB21UwNNUWloAnrHYKgFoFMEdgywBYJAR2FOpliOwlsFatAMWBg1QBhYKgVWK2uFjXqOYcagWBeovUCxl1hGrtQwCbUMAKJNGwfGgBaCqwBYaRKjZABlhTAWgNGqiVMDo2vqGAuYDtdOuPXCSSA1Ex2I7FVgZqVAUG+iX/TQD3q7EUap1eoAglXZQ1w8czmMn0BAA -->

<!-- internal state end -->
<!-- finishing_touch_checkbox_start -->

<details>
<summary>✨ Finishing Touches</summary>

- [ ] <!-- {"checkboxId": "7962f53c-55bc-4827-bfbf-6a18da830691"} --> 📝 Generate Docstrings

</details>

<!-- finishing_touch_checkbox_end -->
<!-- tips_start -->

---



<details>
<summary>🪧 Tips</summary>

### Chat

There are 3 ways to chat with [CodeRabbit](https://coderabbit.ai?utm_source=oss&utm_medium=github&utm_campaign=eastgenomics/eggd_generate_fusion_workbook&utm_content=1):

- Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  - `I pushed a fix in commit <commit_id>, please review it.`
  - `Generate unit testing code for this file.`
  - `Open a follow-up GitHub issue for this discussion.`
- Files and specific lines of code (under the "Files changed" tab): Tag `@coderabbitai` in a new review comment at the desired location with your query. Examples:
  - `@coderabbitai generate unit testing code for this file.`
  -	`@coderabbitai modularize this function.`
- PR comments: Tag `@coderabbitai` in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  - `@coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.`
  - `@coderabbitai read src/utils.ts and generate unit testing code.`
  - `@coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.`
  - `@coderabbitai help me debug CodeRabbit configuration file.`

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

### CodeRabbit Commands (Invoked using PR comments)

- `@coderabbitai pause` to pause the reviews on a PR.
- `@coderabbitai resume` to resume the paused reviews.
- `@coderabbitai review` to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
- `@coderabbitai full review` to do a full review from scratch and review all the files again.
- `@coderabbitai summary` to regenerate the summary of the PR.
- `@coderabbitai generate docstrings` to [generate docstrings](https://docs.coderabbit.ai/finishing-touches/docstrings) for this PR.
- `@coderabbitai resolve` resolve all the CodeRabbit review comments.
- `@coderabbitai configuration` to show the current CodeRabbit configuration for the repository.
- `@coderabbitai help` to get help.

### Other keywords and placeholders

- Add `@coderabbitai ignore` anywhere in the PR description to prevent this PR from being reviewed.
- Add `@coderabbitai summary` to generate the high-level summary at a specific location in the PR description.
- Add `@coderabbitai` anywhere in the PR title to generate the title automatically.

### CodeRabbit Configuration File (`.coderabbit.yaml`)

- You can programmatically configure CodeRabbit by adding a `.coderabbit.yaml` file to the root of your repository.
- Please see the [configuration documentation](https://docs.coderabbit.ai/guides/configure-coderabbit) for more information.
- If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: `# yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json`

### Documentation and Community

- Visit our [Documentation](https://docs.coderabbit.ai) for detailed information on how to use CodeRabbit.
- Join our [Discord Community](http://discord.gg/coderabbit) to get help, request features, and share feedback.
- Follow us on [X/Twitter](https://twitter.com/coderabbitai) for updates and announcements.

</details>

<!-- tips_end -->
author_association: NONE
user: login: pofuoku
created_at: 2025-03-19T16:39:37Z
body: Example job
https://platform.dnanexus.com/panx/projects/Gz4kFpQ48F3YXzV94xXQ220Q/monitor/job/GzJKybj48F3QBGB3zxQ7KqbF
author_association: COLLABORATOR