Repository: MannLabs/alphadia (language: python)

Issue:
Enable running E2E tests on cluster
WHO
devs

WHAT
can run alphaDIA tests on cluster

WHY
To leverage full processing power

Acceptance Criteria
The run_e2e_tests script can be run on the SLURM cluster



Discussion / hints:

Fix patch:
diff --git a/docs/installation.md b/docs/installation.md
index 329f9104..405f923a 100644
--- a/docs/installation.md
+++ b/docs/installation.md
@@ -221,3 +221,6 @@ A exit code of 127 indicates that there was an unknown error. All other nonzero
 'business errors', i.e. those caused most likely by user input (data and/or configuration).
 
 Further details on such errors can be found in the `events.jsonl` file in the `.progress` folder(s) of the output directory.
+
+### Slurm script example
+You can find an example of a Slurm script here: [./tests/e2e_tests/e2e_slurm.sh](./tests/e2e_tests/e2e_slurm.sh).


Test patch:
diff --git a/tests/e2e_tests/e2e_slurm.sh b/tests/e2e_tests/e2e_slurm.sh
new file mode 100644
index 00000000..ee4db7e8
--- /dev/null
+++ b/tests/e2e_tests/e2e_slurm.sh
@@ -0,0 +1,54 @@
+#!/usr/bin/env bash
+# A simple SLURM script to run AlphaDIA end2end tests on a SLURM cluster.
+# Prerequisites:
+# - conda environment with working AlphaDIA installation
+# - input parameters (see below) set to desired values
+# - (optional) SBATCH directives adapted to current use case
+#SBATCH --job-name=alphadia
+#SBATCH --nodes=1
+#SBATCH --ntasks=1
+#SBATCH --cpus-per-task=24
+#SBATCH --mem=128G
+#SBATCH --time=01:00:00
+
+# input parameters:
+CONDA_ENV=alphadia
+BRANCH=development  # branch to take test cases from
+TEST_CASE=basic
+
+#
+set -u -e
+OUTPUT_FOLDER=output_${CONDA_ENV}_${TEST_CASE}_${SLURM_JOB_ID}
+URL=https://raw.githubusercontent.com/MannLabs/alphadia/${BRANCH}/tests
+export TQDM_MININTERVAL=10  # avoid lots of tqdm outputs
+
+echo CONDA_ENV=$CONDA_ENV
+echo BRANCH=$BRANCH
+echo OUTPUT_FOLDER=$OUTPUT_FOLDER
+
+echo CONDA_ENV ">>>>>>"
+conda info
+conda run -n $CONDA_ENV pip freeze
+echo "<<<<<<"
+
+echo MONO_VERSION ">>>>>>"
+conda run -n $CONDA_ENV mono --version
+echo "<<<<<<"
+
+echo "Preparing test data.."
+
+mkdir -p $OUTPUT_FOLDER && cd $OUTPUT_FOLDER
+
+# note: if the locations of these files change, this script will need to be updated
+wget $URL/run_e2e_tests.sh
+
+mkdir -p e2e_tests && cd e2e_tests
+wget $URL/e2e_tests/e2e_test_cases.yaml
+wget $URL/e2e_tests/prepare_test_data.py
+wget $URL/e2e_tests/calc_metrics.py
+cd ..
+
+chmod +x ./run_e2e_tests.sh
+
+echo "Running alphadia.."
+./run_e2e_tests.sh $TEST_CASE $CONDA_ENV