Repository: nf-core/differentialabundance
Pull request #448: NOMERGE simplify toolsheethandling
Author: login: pinin4fjords
Created: 2025-03-31T16:35:32Z | Merged: 2025-04-08T08:53:44Z

<!--
# nf-core/differentialabundance pull request

Many thanks for contributing to nf-core/differentialabundance!

Please fill in the appropriate checklist below (delete whatever is not relevant).
These are the most common things requested on pull requests (PRs).

Remember that PRs should be made against the dev branch, unless you're preparing a pipeline release.

Learn more about contributing: [CONTRIBUTING.md](https://github.com/nf-core/differentialabundance/tree/master/.github/CONTRIBUTING.md)
-->

My reworking of https://github.com/nf-core/differentialabundance/pull/443, for illustration in PR review there. 

Note: **I've stripped out a lot of workflow code, and am progressively re-adding to get to the state of #443, but with the new formulation **


## PR checklist

- [ ] This comment contains a description of changes (with reason).
- [ ] If you've fixed a bug or added code that should be tested, add tests!
- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/differentialabundance/tree/master/.github/CONTRIBUTING.md)
- [ ] If necessary, also make a PR on the nf-core/differentialabundance _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.
- [ ] Make sure your code lints (`nf-core lint`).
- [ ] Ensure the test suite passes (`nf-test test main.nf.test -profile test,docker`).
- [ ] Check for unexpected warnings in debug mode (`nextflow run . -profile debug,test,docker --outdir <OUTDIR>`).
- [ ] Usage Documentation in `docs/usage.md` is updated.
- [ ] Output Documentation in `docs/output.md` is updated.
- [ ] `CHANGELOG.md` is updated.
- [ ] `README.md` is updated (including new tool citations and authors/contributors).



Diff:
diff --git a/assets/toolsheet.csv b/assets/toolsheet.csv
new file mode 100644
index 000000000..2d1544731
--- /dev/null
+++ b/assets/toolsheet.csv
@@ -0,0 +1,11 @@
+analysis_name,study_type,differential_method,limma_use_voom,functional_method
+deseq2,rnaseq,deseq2,,
+deseq2_gsea,rnaseq,deseq2,,gsea
+deseq2_gprofiler2,rnaseq,deseq2,,gprofiler2
+limma_rnaseq,rnaseq,limma,true,
+limma_rnaseq_gsea,rnaseq,limma,true,gsea
+limma_rnaseq_gprofiler2,rnaseq,limma,true,gprofiler2
+limma_affy,affy_array,limma,,
+limma_affy_gsea,affy_array,limma,,gsea
+limma_affy_gprofiler2,affy_array,limma,,gprofiler2
+limma_maxquant,maxquant,limma,,
diff --git a/assets/toolsheet_affy.csv b/assets/toolsheet_affy.csv
deleted file mode 100644
index f2b03b434..000000000
--- a/assets/toolsheet_affy.csv
+++ /dev/null
@@ -1,4 +0,0 @@
-analysis_name,diff_method,diff_args,func_method,func_args
-limma,limma,,,
-limma_gsea,limma,,gsea,
-limma_gprofiler2,limma,,gprofiler2,
diff --git a/assets/toolsheet_maxquant.csv b/assets/toolsheet_maxquant.csv
deleted file mode 100644
index 8c8b2487e..000000000
--- a/assets/toolsheet_maxquant.csv
+++ /dev/null
@@ -1,2 +0,0 @@
-analysis_name,diff_method,diff_args,func_method,func_args
-limma,limma,,,
diff --git a/assets/toolsheet_rnaseq.csv b/assets/toolsheet_rnaseq.csv
deleted file mode 100644
index fdb649ed4..000000000
--- a/assets/toolsheet_rnaseq.csv
+++ /dev/null
@@ -1,7 +0,0 @@
-analysis_name,diff_method,diff_args,func_method,func_args
-deseq2,deseq2,,,
-deseq2_gsea,deseq2,,gsea,
-deseq2_gprofiler2,deseq2,,gprofiler2,
-limma,limma,--limma_use_voom true,,
-limma_gsea,limma,--limma_use_voom true,gsea,
-limma_gprofiler2,limma,--limma_use_voom true,gprofiler2,
diff --git a/conf/modules.config b/conf/modules.config
index 2d27a7838..188cef0f6 100644
--- a/conf/modules.config
+++ b/conf/modules.config
@@ -172,22 +172,22 @@ process {
         ext.args = { [
             "--gene_id_col \"${params.features_id_col}\"",
             "--sample_id_col \"${params.observations_id_col}\"",
-            "--test $meta.args_differential.deseq2_test",
-            "--fit_type $meta.args_differential.deseq2_fit_type",
-            "--sf_type $meta.args_differential.deseq2_sf_type",
-            "--min_replicates_for_replace $meta.args_differential.deseq2_min_replicates_for_replace",
-            "--use_t $meta.args_differential.deseq2_use_t",
-            "--lfc_threshold $meta.args_differential.deseq2_lfc_threshold",
-            "--alt_hypothesis $meta.args_differential.deseq2_alt_hypothesis",
-            "--independent_filtering $meta.args_differential.deseq2_independent_filtering",
-            "--p_adjust_method $meta.args_differential.deseq2_p_adjust_method",
-            "--alpha $meta.args_differential.deseq2_alpha",
-            "--minmu $meta.args_differential.deseq2_minmu",
-            "--vs_method $meta.args_differential.deseq2_vs_method",
-            "--vs_blind $meta.args_differential.deseq2_vs_blind",
-            "--vst_nsub $meta.args_differential.deseq2_vst_nsub",
-            "--shrink_lfc $meta.args_differential.deseq2_shrink_lfc",
-            "--cores $meta.args_differential.deseq2_cores",
+            "--test $meta.deseq2_test",
+            "--fit_type $meta.deseq2_fit_type",
+            "--sf_type $meta.deseq2_sf_type",
+            "--min_replicates_for_replace $meta.deseq2_min_replicates_for_replace",
+            "--use_t $meta.deseq2_use_t",
+            "--lfc_threshold $meta.deseq2_lfc_threshold",
+            "--alt_hypothesis $meta.deseq2_alt_hypothesis",
+            "--independent_filtering $meta.deseq2_independent_filtering",
+            "--p_adjust_method $meta.deseq2_p_adjust_method",
+            "--alpha $meta.deseq2_alpha",
+            "--minmu $meta.deseq2_minmu",
+            "--vs_method $meta.deseq2_vs_method",
+            "--vs_blind $meta.deseq2_vs_blind",
+            "--vst_nsub $meta.deseq2_vst_nsub",
+            "--shrink_lfc $meta.deseq2_shrink_lfc",
+            "--cores $meta.deseq2_cores",
             "--round_digits $params.report_round_digits",
             ((meta.blocking == null) ? '' : "--blocking_variables $meta.blocking"),
         ].join(' ').trim() }
@@ -216,24 +216,24 @@ process {
         ext.args = { [
             "--gene_id_col \"${params.features_id_col}\"",
             "--sample_id_col \"${params.observations_id_col}\"",
-            "--test $meta.args_differential.deseq2_test",
-            "--fit_type $meta.args_differential.deseq2_fit_type",
+            "--test $meta.deseq2_test",
+            "--fit_type $meta.deseq2_fit_type",
             "--sizefactors_from_controls $params.sizefactors_from_controls",
-            "--sf_type $meta.args_differential.deseq2_sf_type",
-            "--min_replicates_for_replace $meta.args_differential.deseq2_min_replicates_for_replace",
-            "--use_t $meta.args_differential.deseq2_use_t",
-            "--lfc_threshold $meta.args_differential.deseq2_lfc_threshold",
-            "--alt_hypothesis $meta.args_differential.deseq2_alt_hypothesis",
-            "--independent_filtering $meta.args_differential.deseq2_independent_filtering",
-            "--p_adjust_method $meta.args_differential.deseq2_p_adjust_method",
-            "--alpha $meta.args_differential.deseq2_alpha",
-            "--minmu $meta.args_differential.deseq2_minmu",
-            "--vs_method $meta.args_differential.deseq2_vs_method",
-            "--vs_blind $meta.args_differential.deseq2_vs_blind",
-            "--vst_nsub $meta.args_differential.deseq2_vst_nsub",
-            "--shrink_lfc $meta.args_differential.deseq2_shrink_lfc",
-            "--cores $meta.args_differential.deseq2_cores",
-            "--subset_to_contrast_samples $meta.args_differential.differential_subset_to_contrast_samples",
+            "--sf_type $meta.deseq2_sf_type",
+            "--min_replicates_for_replace $meta.deseq2_min_replicates_for_replace",
+            "--use_t $meta.deseq2_use_t",
+            "--lfc_threshold $meta.deseq2_lfc_threshold",
+            "--alt_hypothesis $meta.deseq2_alt_hypothesis",
+            "--independent_filtering $meta.deseq2_independent_filtering",
+            "--p_adjust_method $meta.deseq2_p_adjust_method",
+            "--alpha $meta.deseq2_alpha",
+            "--minmu $meta.deseq2_minmu",
+            "--vs_method $meta.deseq2_vs_method",
+            "--vs_blind $meta.deseq2_vs_blind",
+            "--vst_nsub $meta.deseq2_vst_nsub",
+            "--shrink_lfc $meta.deseq2_shrink_lfc",
+            "--cores $meta.deseq2_cores",
+            "--subset_to_contrast_samples $meta.differential_subset_to_contrast_samples",
             "--round_digits $params.report_round_digits",
             ((meta.blocking == null) ? '' : "--blocking_variables $meta.blocking"),
             ((meta.exclude_samples_col == null) ? '' : "--exclude_samples_col $meta.exclude_samples_col"),
@@ -251,24 +251,24 @@ process {
                 pattern: '*.{normalised_counts}.tsv'
             ],
         ]
-        ext.when = { meta.args_differential.limma_use_voom }
+        ext.when = { meta.limma_use_voom }
         ext.args = { [
             "--probe_id_col \"${params.features_id_col}\"",
             "--sample_id_col \"${params.observations_id_col}\"",
-            "--ndups \"${meta.args_differential.limma_ndups}\"",
-            "--spacing \"${meta.args_differential.limma_spacing}\"",
-            "--block \"${meta.args_differential.limma_block}\"",
-            "--correlation ${meta.args_differential.limma_correlation}",
-            "--method \"${meta.args_differential.limma_method}\"",
-            "--proportion ${meta.args_differential.limma_proportion}",
-            "--stdev.coef.lim \"${meta.args_differential.limma_stdev_coef_lim}\"",
-            "--trend ${meta.args_differential.limma_trend}",
-            "--robust ${meta.args_differential.limma_robust}",
-            "--winsor.tail.p \"${meta.args_differential.limma_winsor_tail_p}\"",
-            "--p.value ${meta.args_differential.limma_p_value}",
-            "--lfc ${meta.args_differential.limma_lfc}",
-            "--confint ${meta.args_differential.limma_confint}",
-            "--use_voom \"${meta.args_differential.limma_use_voom}\"",
+            "--ndups \"${meta.limma_ndups}\"",
+            "--spacing \"${meta.limma_spacing}\"",
+            "--block \"${meta.limma_block}\"",
+            "--correlation ${meta.limma_correlation}",
+            "--method \"${meta.limma_method}\"",
+            "--proportion ${meta.limma_proportion}",
+            "--stdev.coef.lim \"${meta.limma_stdev_coef_lim}\"",
+            "--trend ${meta.limma_trend}",
+            "--robust ${meta.limma_robust}",
+            "--winsor.tail.p \"${meta.limma_winsor_tail_p}\"",
+            "--p.value ${meta.limma_p_value}",
+            "--lfc ${meta.limma_lfc}",
+            "--confint ${meta.limma_confint}",
+            "--use_voom \"${meta.limma_use_voom}\"",
             ((meta.blocking == null) ? '' : "--blocking_variables $meta.blocking"),
         ].join(' ').trim() }
     }
@@ -296,21 +296,21 @@ process {
         ext.args = { [
             "--probe_id_col \"${params.features_id_col}\"",
             "--sample_id_col \"${params.observations_id_col}\"",
-            "--ndups \"${meta.args_differential.limma_ndups}\"",
-            "--spacing \"${meta.args_differential.limma_spacing}\"",
-            "--block \"${meta.args_differential.limma_block}\"",
-            "--correlation ${meta.args_differential.limma_correlation}",
-            "--method \"${meta.args_differential.limma_method}\"",
-            "--proportion ${meta.args_differential.limma_proportion}",
-            "--stdev.coef.lim \"${meta.args_differential.limma_stdev_coef_lim}\"",
-            "--trend ${meta.args_differential.limma_trend}",
-            "--robust ${meta.args_differential.limma_robust}",
-            "--winsor.tail.p \"${meta.args_differential.limma_winsor_tail_p}\"",
-            "--p.value ${meta.args_differential.limma_p_value}",
-            "--lfc ${meta.args_differential.limma_lfc}",
-            "--confint ${meta.args_differential.limma_confint}",
-            "--subset_to_contrast_samples $meta.args_differential.differential_subset_to_contrast_samples",
-            "--use_voom \"${meta.args_differential.limma_use_voom}\"",
+            "--ndups \"${meta.limma_ndups}\"",
+            "--spacing \"${meta.limma_spacing}\"",
+            "--block \"${meta.limma_block}\"",
+            "--correlation ${meta.limma_correlation}",
+            "--method \"${meta.limma_method}\"",
+            "--proportion ${meta.limma_proportion}",
+            "--stdev.coef.lim \"${meta.limma_stdev_coef_lim}\"",
+            "--trend ${meta.limma_trend}",
+            "--robust ${meta.limma_robust}",
+            "--winsor.tail.p \"${meta.limma_winsor_tail_p}\"",
+            "--p.value ${meta.limma_p_value}",
+            "--lfc ${meta.limma_lfc}",
+            "--confint ${meta.limma_confint}",
+            "--subset_to_contrast_samples $meta.differential_subset_to_contrast_samples",
+            "--use_voom \"${meta.limma_use_voom}\"",
             ((meta.blocking == null) ? '' : "--blocking_variables $meta.blocking"),
             ((meta.exclude_samples_col == null) ? '' : "--exclude_samples_col $meta.exclude_samples_col"),
             ((meta.exclude_samples_values == null) ? '' : "--exclude_samples_values $meta.exclude_samples_values")
@@ -354,23 +354,23 @@ process {
             ]
         ]
         ext.args = { [
-            "-nperm $meta.args_functional.gsea_nperm",
-            "-permute $meta.args_functional.gsea_permute",
-            "-scoring_scheme $meta.args_functional.gsea_scoring_scheme",
-            "-metric $meta.args_functional.gsea_metric",
-            "-sort $meta.args_functional.gsea_sort",
-            "-order $meta.args_functional.gsea_order",
-            "-set_max $meta.args_functional.gsea_set_max",
-            "-set_min $meta.args_functional.gsea_set_min",
-            "-norm $meta.args_functional.gsea_norm",
-            "-rnd_type $meta.args_functional.gsea_rnd_type",
-            "-make_sets $meta.args_functional.gsea_make_sets",
-            "-median $meta.args_functional.gsea_median",
-            "-num $meta.args_functional.gsea_num",
-            "-plot_top_x $meta.args_functional.gsea_plot_top_x",
-            "-rnd_seed $meta.args_functional.gsea_rnd_seed",
-            "-save_rnd_lists $meta.args_functional.gsea_save_rnd_lists",
-            "-zip_report $meta.args_functional.gsea_zip_report"
+            "-nperm $meta.gsea_nperm",
+            "-permute $meta.gsea_permute",
+            "-scoring_scheme $meta.gsea_scoring_scheme",
+            "-metric $meta.gsea_metric",
+            "-sort $meta.gsea_sort",
+            "-order $meta.gsea_order",
+            "-set_max $meta.gsea_set_max",
+            "-set_min $meta.gsea_set_min",
+            "-norm $meta.gsea_norm",
+            "-rnd_type $meta.gsea_rnd_type",
+            "-make_sets $meta.gsea_make_sets",
+            "-median $meta.gsea_median",
+            "-num $meta.gsea_num",
+            "-plot_top_x $meta.gsea_plot_top_x",
+            "-rnd_seed $meta.gsea_rnd_seed",
+            "-save_rnd_lists $meta.gsea_save_rnd_lists",
+            "-zip_report $meta.gsea_zip_report"
         ].join(' ').trim() }
     }
 
@@ -398,20 +398,20 @@ process {
             ]
         ]
         ext.args = { [
-            "--significant \"${meta.args_functional.gprofiler2_significant}\"",
-            "--measure_underrepresentation \"${meta.args_functional.gprofiler2_measure_underrepresentation}\"",
-            "--correction_method \"${meta.args_functional.gprofiler2_correction_method}\"",
-            "--evcodes \"${meta.args_functional.gprofiler2_evcodes}\"",
-            "--pval_threshold \"${meta.args_functional.gprofiler2_max_qval}\"",
-            "--domain_scope ${meta.args_functional.gprofiler2_domain_scope}",
-            "--min_diff \"${meta.args_functional.gprofiler2_min_diff}\"",
-            "--round_digits ${meta.args_functional.report_round_digits}",
-            "--palette_name \"${meta.args_functional.gprofiler2_palette_name}\"",
-            ((meta.args_differential.differential_feature_id_column == null) ? '' : "--de_id_column \"${meta.args_differential.differential_feature_id_column}\""),
-            ((meta.args_functional.gprofiler2_token == null) ? '' : "--token \"${meta.args_functional.gprofiler2_token}\""),
-            ((meta.args_functional.gprofiler2_organism == null) ? '' : "--organism \"${meta.args_functional.gprofiler2_organism}\""),
-            ((meta.args_functional.gprofiler2_background_column == null) ? '' : "--background_column \"${meta.args_functional.gprofiler2_background_column}\""),
-            ((meta.args_functional.gprofiler2_sources == null) ? '' : "--sources \"${meta.args_functional.gprofiler2_sources}\"")
+            "--significant \"${meta.gprofiler2_significant}\"",
+            "--measure_underrepresentation \"${meta.gprofiler2_measure_underrepresentation}\"",
+            "--correction_method \"${meta.gprofiler2_correction_method}\"",
+            "--evcodes \"${meta.gprofiler2_evcodes}\"",
+            "--pval_threshold \"${meta.gprofiler2_max_qval}\"",
+            "--domain_scope ${meta.gprofiler2_domain_scope}",
+            "--min_diff \"${meta.gprofiler2_min_diff}\"",
+            "--round_digits ${meta.report_round_digits}",
+            "--palette_name \"${meta.gprofiler2_palette_name}\"",
+            ((meta.differential_feature_id_column == null) ? '' : "--de_id_column \"${meta.differential_feature_id_column}\""),
+            ((meta.gprofiler2_token == null) ? '' : "--token \"${meta.gprofiler2_token}\""),
+            ((meta.gprofiler2_organism == null) ? '' : "--organism \"${meta.gprofiler2_organism}\""),
+            ((meta.gprofiler2_background_column == null) ? '' : "--background_column \"${meta.gprofiler2_background_column}\""),
+            ((meta.gprofiler2_sources == null) ? '' : "--sources \"${meta.gprofiler2_sources}\"")
         ].join(' ').trim() }
     }
 
@@ -422,13 +422,13 @@ process {
         ]
         memory = { 12.GB * task.attempt }
         ext.args = { [
-            "--sample_id_col \"${params.observations_id_col}\"",
-            "--feature_id_col \"${params.features_id_col}\"",
-            "--assay_names \"${params.exploratory_assay_names}\"",
-            "--final_assay \"${params.exploratory_final_assay}\"",
-            "--outlier_mad_threshold ${params.exploratory_mad_threshold}",
-            "--palette_name \"${params.exploratory_palette_name}\"",
-            ( (params.study_type == 'maxquant') ? "--log2_assays ''" : (((params.exploratory_log2_assays == null) ? '' : "--log2_assays \"$params.exploratory_log2_assays\"".replace('[', '').replace(']', ''))) )
+            "--sample_id_col \"${meta.study_meta.observations_id_col}\"",
+            "--feature_id_col \"${meta.study_meta.features_id_col}\"",
+            "--assay_names \"${meta.study_meta.exploratory_assay_names}\"",
+            "--final_assay \"${meta.study_meta.exploratory_final_assay}\"",
+            "--outlier_mad_threshold ${meta.study_meta.exploratory_mad_threshold}",
+            "--palette_name \"${meta.study_meta.exploratory_palette_name}\"",
+            ( (meta.study_meta.study_type == 'maxquant') ? "--log2_assays ''" : (((meta.study_meta.exploratory_log2_assays == null) ? '' : "--log2_assays \"$meta.study_meta.exploratory_log2_assays\"".replace('[', '').replace(']', ''))) )
         ].join(' ').trim() }
     }
 
@@ -442,13 +442,13 @@ process {
             "--feature_id_col \"${params.features_id_col}\"",
             "--reference_level \"$meta.reference\"",
             "--treatment_level \"$meta.target\"",
-            "--fold_change_col \"${meta.args_differential.differential_fc_column}\"",
-            "--p_value_column \"${meta.args_differential.differential_qval_column}\"",
-            "--diff_feature_id_col \"${meta.args_differential.differential_feature_id_column}\"",
-            "--fold_change_threshold \"${meta.args_differential.differential_min_fold_change}\"",
-            "--p_value_threshold \"${meta.args_differential.differential_max_qval}\"",
-            "--unlog_foldchanges \"${meta.args_differential.differential_foldchanges_logged}\"",
-            "--palette_name \"${meta.args_differential.differential_palette_name}\""
+            "--fold_change_col \"${meta.study_meta.differential_fc_column}\"",
+            "--p_value_column \"${meta.study_meta.differential_qval_column}\"",
+            "--diff_feature_id_col \"${meta.study_meta.differential_feature_id_column}\"",
+            "--fold_change_threshold \"${meta.study_meta.differential_min_fold_change}\"",
+            "--p_value_threshold \"${meta.study_meta.differential_max_qval}\"",
+            "--unlog_foldchanges \"${meta.study_meta.differential_foldchanges_logged}\"",
+            "--palette_name \"${meta.study_meta.differential_palette_name}\""
         ].join(' ').trim() }
     }
 
@@ -465,11 +465,11 @@ process {
             "--sample_id_col \"${params.observations_id_col}\"",
             "--feature_id_col \"${params.features_id_col}\"",
             "--feature_name_col \"" + (params.features_name_col == null ? params.features_id_col : params.features_name_col) + "\"",
-            "--diff_feature_id_col \"${meta.args_differential.differential_feature_id_column}\"",
-            "--fold_change_column \"${meta.args_differential.differential_fc_column}\"",
-            "--pval_column \"${meta.args_differential.differential_pval_column}\"",
-            "--qval_column \"${meta.args_differential.differential_qval_column}\"",
-            "--unlog_foldchanges \"${meta.args_differential.differential_foldchanges_logged}\"",
+            "--diff_feature_id_col \"${meta.differential_feature_id_column}\"",
+            "--fold_change_column \"${meta.differential_fc_column}\"",
+            "--pval_column \"${meta.differential_pval_column}\"",
+            "--qval_column \"${meta.differential_qval_column}\"",
+            "--unlog_foldchanges \"${meta.differential_foldchanges_logged}\"",
             ((params.report_title == null) ? '' : "--title \"$params.report_title\""),
             ((params.report_author == null) ? '' : "--author \"$params.report_author\""),
             ((params.report_description == null) ? '' : "--description \"$params.report_description\""),
@@ -506,15 +506,15 @@ process {
             enabled: false
         ]
         ext.prefix = { "${meta.id}" }
-        ext.args   = [
-            "--sample_id_col \"${params.observations_id_col}\"",
-            "--minimum_samples ${params.filtering_min_samples}",
-            "--minimum_abundance ${params.filtering_min_abundance}",
-            (params.filtering_min_proportion ? "--minimum_proportion ${params.filtering_min_proportion}" : ''),
-            (params.filtering_grouping_var ? "--grouping_variable \"${params.filtering_grouping_var}\"" : ''),
-            (params.filtering_min_proportion_not_na ? "--minimum_proportion_not_na \"${params.filtering_min_proportion_not_na}\"" : ''),
-            (params.filtering_min_samples_not_na ? "--minimum_samples_not_na \"${params.filtering_min_samples_not_na}\"" : '')
-        ].join(' ').trim()
+        ext.args   = {[
+            "--sample_id_col \"${meta.observations_id_col}\"",
+            "--minimum_samples ${meta.filtering_min_samples}",
+            "--minimum_abundance ${meta.filtering_min_abundance}",
+            (meta.filtering_min_proportion ? "--minimum_proportion ${meta.filtering_min_proportion}" : ''),
+            (meta.filtering_grouping_var ? "--grouping_variable \"${meta.filtering_grouping_var}\"" : ''),
+            (meta.filtering_min_proportion_not_na ? "--minimum_proportion_not_na \"${meta.filtering_min_proportion_not_na}\"" : ''),
+            (meta.filtering_min_samples_not_na ? "--minimum_samples_not_na \"${meta.filtering_min_samples_not_na}\"" : '')
+        ].join(' ').trim()}
     }
 
     withName: 'CUSTOM_TABULARTOGSEACLS' {
diff --git a/main.nf b/main.nf
index 87db9071d..7b3246254 100644
--- a/main.nf
+++ b/main.nf
@@ -78,7 +78,7 @@ workflow {
     //
 
     NFCORE_DIFFERENTIALABUNDANCE (
-        PIPELINE_INITIALISATION.out.tools
+        PIPELINE_INITIALISATION.out.paramsets
     )
 
     //
diff --git a/modules/nf-core/custom/matrixfilter/main.nf b/modules/nf-core/custom/matrixfilter/main.nf
index 47a99fad3..f8cd05aa8 100644
--- a/modules/nf-core/custom/matrixfilter/main.nf
+++ b/modules/nf-core/custom/matrixfilter/main.nf
@@ -1,5 +1,5 @@
 process CUSTOM_MATRIXFILTER {
-    tag "$meta"
+    tag "${meta.id}"
     label 'process_single'
     conda "${moduleDir}/environment.yml"
     container "${workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container
diff --git a/subworkflows/local/utils_nfcore_differentialabundance_pipeline/main.nf b/subworkflows/local/utils_nfcore_differentialabundance_pipeline/main.nf
index 4f7fc0d54..ea4a55d8b 100644
--- a/subworkflows/local/utils_nfcore_differentialabundance_pipeline/main.nf
+++ b/subworkflows/local/utils_nfcore_differentialabundance_pipeline/main.nf
@@ -35,6 +35,11 @@ workflow PIPELINE_INITIALISATION {
 
     main:
 
+    // Check that params is available
+    if (!params) {
+        error("Pipeline parameters not initialized. This is a critical error.")
+    }
+
     ch_versions = Channel.empty()
 
     //
@@ -68,13 +73,15 @@ workflow PIPELINE_INITIALISATION {
     //
     validateInputParameters()
 
-    //
+    // ===========================================================================
+    // Handle toolsheet
+    // ===========================================================================
+
     // Define tool settings based on analysis name or default parameters
-    //
-    ch_tools = Channel.of(getToolConfigurations())
+    ch_paramsets = Channel.fromList(getToolConfigurations())
 
     emit:
-    tools       = ch_tools
+    paramsets = ch_paramsets
     versions    = ch_versions
 }
 
@@ -125,6 +132,38 @@ workflow PIPELINE_COMPLETION {
     }
 }
 
+/*
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+    SUBWORKFLOW FOR GROUPING BY ANALYSIS
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Toolsheet usage means that we sometimes group multiple analyses into a single run
+where the input parameterisation would have been the same. There may also be multiple
+results per group where we have multiple contrasts. To use these results downstream
+we must 'expand' results back to the original groupings, but also group by the
+individual analysis such that e.g. the results from multiple contrasts travel
+through the workflow together.
+*/
+
+workflow GROUP_BY_ANALYSIS {
+
+    take:
+    channel_in
+    paramsets_by_name
+
+    main:
+
+    channel_out = channel_in
+        .map{meta, differential_results -> [meta.analysis_names, meta, differential_results]}
+        .transpose()
+        .join(paramsets_by_name)
+        .map{analysis_name, meta, differential_results, paramset -> [paramset, meta, differential_results]}
+
+
+    emit:
+    with_study_meta = channel_out
+}
+
 /*
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     FUNCTIONS
@@ -135,6 +174,63 @@ workflow PIPELINE_COMPLETION {
 //
 def validateInputParameters() {
     genomeExistsError()
+
+    // Check file existence for input parameters
+    def checkPathParamList = [ params.input ]
+    for (param in checkPathParamList) {
+        if (param) {
+            file(param, checkIfExists: true)
+        }
+    }
+
+    // Check mandatory parameters
+    if (!params.input) {
+        error("Input samplesheet not specified!")
+    }
+
+    // Validate study type specific parameters
+    if (params.study_type == 'affy_array') {
+        if (!params.affy_cel_files_archive) {
+            error("CEL files archive not specified!")
+        }
+    } else if (params.study_type == 'maxquant') {
+        if (params.functional_method) {
+            error("Functional analysis is not yet possible with maxquant input data; please set --functional_method to null and rerun pipeline!")
+        }
+        if (!params.matrix) {
+            error("Input matrix not specified!")
+        }
+    } else if (params.study_type == 'geo_soft_file') {
+        if (!params.querygse || !params.features_metadata_cols) {
+            error("Query GSE not specified or features metadata columns not specified")
+        }
+    } else if (params.study_type == "rnaseq") {
+        if (!params.matrix) {
+            error("Input matrix not specified!")
+        }
+    }
+
+    // Validate functional analysis parameters
+    if (params.functional_method) {
+        if (params.functional_method == 'gsea' && !params.gene_sets_files) {
+            error("GSEA activated but gene set file not specified!")
+        } else if (params.functional_method == 'gprofiler2') {
+            if (!params.gprofiler2_token && !params.gprofiler2_organism) {
+                error("To run gprofiler2, please provide a run token, GMT file or organism!")
+            }
+            if (params.gene_sets_files && params.gene_sets_files.split(",").size() > 1) {
+                error("gprofiler2 can currently only work with a single gene set file")
+            }
+        }
+    }
+
+    // Validate contrasts parameters
+    if (params.contrasts_yml && params.contrasts) {
+        error("Both '--contrasts' and '--contrasts_yml' parameters are set. Please specify only one of these options to define contrasts.")
+    }
+    if (!(params.contrasts_yml || params.contrasts)) {
+        error("Either '--contrasts' and '--contrasts_yml' must be set. Please specify one of these options to define contrasts.")
+    }
 }
 
 //
@@ -240,167 +336,238 @@ def methodsDescriptionText(mqc_methods_yaml) {
     return description_html.toString()
 }
 
-/**
-* Parse a string of arguments into a map.
-* @param argsStr The string of arguments to parse.
-* @return A map of parameters.
-* @example
-* parseStrArgsToMap("--arg1 aa --arg2 bb --arg4 cc") => [arg1: aa, arg2: bb, arg4: cc]
-*/
-def parseStrArgsToMap(String argsStr) {
-    if (!argsStr) return [:]                     // if null return empty
-    def tokens = argsStr.split().findAll { it }  // Split and remove empty strings
-    def pairs = tokens.collate(2)                // Group into pairs
-    return pairs.collectEntries {
-        [(it[0].replaceAll('^-+', '')): it[1]]   // Remove leading dashes
+// Split parameters into different sets based on prefixes and analysis type
+def splitParameters(params) {
+    // Base parameters are everything not in other sets
+    def baseParams = [:]
+    def differentialParams = [:]
+    def functionalParams = [:]
+
+    // First collect all differential parameters
+    params.each { key, value ->
+        if (key.startsWith('differential_') || key.startsWith('limma_') || key.startsWith('deseq2_')) {
+            differentialParams[key] = value
+        }
     }
-}
 
-/**
-* Parse the parameters for a specific analysis type and method.
-* @param analysisType The analysis type. Eg. 'differential', 'functional'.
-* @param method The method to use for the analysis.
-* @param argsStr The string of arguments to parse.
-* @return A map of parameters.
-*/
-def parseParams(String analysisType, String method, String argsStr){
-    def parsed_params = [:]
-
-    // parse the params specific to the analysis type from the pipeline params
-    parsed_params += params.findAll { k, v -> k.matches(~/(${analysisType}).*/) }
-
-    // parse the params specific to the method
-    // First it gets the default values as specified by the pipeline params scope,
-    // and then overwrite with the values specified in the argsStr (highest priority),
-    // when provided
-    if (method) {
-        parsed_params += params.findAll { k, v -> k.matches(~/(${method}).*/) }
-        parsed_params["${analysisType}_method"] = method
+    // Then collect functional parameters based on method
+    if (params.functional_method == 'gprofiler2') {
+        // For gprofiler2, include both base and differential params
+        params.each { key, value ->
+            if (key.startsWith('functional_') || key.startsWith('gprofiler2_')) {
+                functionalParams[key] = value
+            }
+        }
+        // Add all differential params to functional params for gprofiler2
+        functionalParams.putAll(differentialParams)
+    } else if (params.functional_method == 'gsea') {
+        // For GSEA, only include functional params
+        params.each { key, value ->
+            if (key.startsWith('functional_') || key.startsWith('gsea_')) {
+                functionalParams[key] = value
+            }
+        }
     }
-    if (argsStr) {
-        parsed_params += parseStrArgsToMap(argsStr)
+
+    // Base params are everything not in other sets
+    params.each { key, value ->
+        if (!differentialParams.containsKey(key) && !functionalParams.containsKey(key)) {
+            baseParams[key] = value
+        }
     }
 
-    return parsed_params
+    // Add base params to both differential and functional
+    differentialParams.putAll(baseParams)
+    functionalParams.putAll(baseParams)
+
+    return [
+        base: baseParams,
+        differential: differentialParams,
+        functional: functionalParams
+    ]
 }
 
-/**
-* Get tool configurations based on whether analysis_name is provided
-* @return A channel with tool configurations.
-*/
+// Get tool configurations based on whether analysis_name is provided
 def getToolConfigurations() {
-    if (params.analysis_name) {
-        return getToolsheetConfigurations()
-    } else {
-        return getDefaultConfigurations()
-    }
+    // Use toolsheet if either analysis_name or custom toolsheet is provided, otherwise use default params
+    return (params.analysis_name || params.toolsheet_custom) ?
+        getToolsheetConfigurations() :
+        getDefaultConfigurations()
 }
 
-/**
-* Get configurations from toolsheet
-* @return A list of tool configurations by parsing toolsheet.
-*/
-def getToolsheetConfigurations() {
-    // Load appropriate toolsheet based on study type
-    def toolsheet_path = getToolsheetPath()
+// Create a temporary schema file for toolsheet validation
+// This schema is derived from the pipeline's nextflow_schema.json by:
+// 1. Reading the toolsheet headers to determine which fields to include
+// 2. Extracting only those properties from the pipeline schema
+// 3. Creating a schema that allows for an array of objects
+// 4. Making analysis_name the only required field
+// 5. Adding meta fields to all properties
+// @return The absolute path to the temporary schema file
+def createToolsheetSchema() {
+    // Load toolsheet and get headers
+    def toolsheet_path = params.toolsheet_custom ?: "${projectDir}/assets/toolsheet.csv"
+    def toolsheet_lines = new File(toolsheet_path).readLines()
+    def headers = toolsheet_lines[0].split(',')
+
+    // Ensure analysis_name is in headers
+    if (!headers.contains('analysis_name')) {
+        error("The toolsheet must contain an 'analysis_name' column")
+    }
 
-    // Load and filter toolsheet for specific analysis
-    def toolsheet = samplesheetToList(toolsheet_path, "${projectDir}/assets/schema_tools.json")
-        .find { it[0].analysis_name == params.analysis_name }
-        ?: error("Analysis '${params.analysis_name}' not found in toolsheet")
+    // Load and parse pipeline schema
+    def pipeline_schema = new File("${projectDir}/nextflow_schema.json").text
+    def schema_json = new groovy.json.JsonSlurper().parseText(pipeline_schema)
+
+    // Extract only the properties that are present in the toolsheet
+    def all_properties = [:]
+    schema_json.$defs.each { group_name, group_def ->
+        if (group_def.properties) {
+            group_def.properties.each { prop_name, prop_def ->
+                // Only include properties that are in the toolsheet headers
+                if (headers.contains(prop_name)) {
+                    // Add meta field to each property
+                    def prop_with_meta = prop_def + [meta: [prop_name]]
+                    all_properties[prop_name] = prop_with_meta
+                }
+            }
+        }
+    }
 
-    // Construct tool configurations
-    def tool_config = toolsheet[0]
-    return [
-        getNormalizationConfig(tool_config),
-        getDifferentialConfig(tool_config),
-        getFunctionalConfig(tool_config)
-    ]
-}
+    // Ensure analysis_name property exists in schema
+    if (!all_properties.containsKey('analysis_name')) {
+        // Find analysis_name definition in pipeline schema
+        def analysis_name_def = null
+        schema_json.$defs.each { group_name, group_def ->
+            if (group_def.properties && group_def.properties.containsKey('analysis_name')) {
+                analysis_name_def = group_def.properties['analysis_name']
+            }
+        }
 
-/**
-* Get default configurations from pipeline parameters
-* @return A list of default tool configurations.
-*/
-def getDefaultConfigurations() {
-    return [
-        getNormalizationConfig(null),
-        getDifferentialConfig(null),
-        getFunctionalConfig(null)
+        if (!analysis_name_def) {
+            error("Could not find analysis_name definition in pipeline schema")
+        }
+
+        // Add analysis_name to properties with meta field
+        all_properties['analysis_name'] = analysis_name_def + [meta: ['analysis_name']]
+    }
+
+    // Create samplesheet schema with filtered properties and analysis_name as required
+    def samplesheet_schema = [
+        '$schema': 'https://json-schema.org/draft/2020-12/schema',
+        'title': 'nf-core/differentialabundance - toolsheet schema',
+        'description': 'Schema for validating the toolsheet configuration',
+        'type': 'array',
+        'items': [
+            'type': 'object',
+            'properties': all_properties,
+            'required': ['analysis_name']
+        ]
     ]
+
+    // Write temporary schema file
+    def temp_schema = File.createTempFile("samplesheet_schema", ".json")
+    def schema_string = new groovy.json.JsonBuilder(samplesheet_schema).toPrettyString()
+    temp_schema.text = schema_string
+
+    return temp_schema.absolutePath
 }
 
-/**
-* Get the appropriate toolsheet path.
-* @return The path to the toolsheet.
-*/
-def getToolsheetPath() {
-    // use user-provided toolsheet if available
-    if (params.toolsheet) {
-        return params.toolsheet
+// Simplify meta map to only include parameters relevant for a specific analysis type
+def subsetMeta(meta, analysis_type) {
+    def relevantParams = [:]
+
+    // Define static lists of parameter prefixes for different analysis types
+    def differentialPrefixes = ['differential_', 'limma_', 'deseq2_']
+    def functionalPrefixes = ['functional_']
+    def gseaPrefixes = ['gsea_']
+    def gprofiler2Prefixes = ['gprofiler2_']
+    def basicFields = ['id', 'study_name', 'study_type', 'analysis_name']
+
+    // Always include basic fields
+    basicFields.each { field ->
+        if (meta.containsKey(field)) {
+            relevantParams[field] = meta[field]
+        }
     }
-    // use default toolsheet based on study type
-    switch (params.study_type) {
-        case 'rnaseq':
-            return "${projectDir}/assets/toolsheet_rnaseq.csv"
-        case ['affy_array', 'geo_soft_file']:
-            return "${projectDir}/assets/toolsheet_affy.csv"
-        case 'maxquant':
-            return "${projectDir}/assets/toolsheet_maxquant.csv"
-        default:
-            error("Invalid study_type. Must be one of: 'rnaseq', 'affy_array', 'geo_soft_file', 'maxquant'")
+
+    // Add parameters based on analysis type
+    switch(analysis_type) {
+        case 'differential':
+            // Include differential-specific parameters
+            meta.each { key, value ->
+                if (differentialPrefixes.any { key.startsWith(it) }) {
+                    relevantParams[key] = value
+                }
+            }
+            break
+
+        case 'functional':
+            // Include functional-specific parameters
+            meta.each { key, value ->
+                if (functionalPrefixes.any { key.startsWith(it) } ||
+                    (meta.functional_method == 'gsea' && gseaPrefixes.any { key.startsWith(it) }) ||
+                    (meta.functional_method == 'gprofiler2' && gprofiler2Prefixes.any { key.startsWith(it) })) {
+                    relevantParams[key] = value
+                }
+            }
+
+            // For gprofiler2, also include differential parameters
+            if (meta.functional_method == 'gprofiler2') {
+                meta.each { key, value ->
+                    if (differentialPrefixes.any { key.startsWith(it) }) {
+                        relevantParams[key] = value
+                    }
+                }
+            }
+            break
     }
+    return relevantParams
 }
 
-/**
-* Get normalization tool configuration
-* @param tool_config The tool configuration.
-* @return A map of normalization configuration.
-*/
-def getNormalizationConfig(tool_config) {
-    // non-rnaseq studies use directly the normalized matrix output from the VALIDATOR module
-    if (params.study_type != 'rnaseq') {
-        return [method: 'validator', args: [:]]
+// Get configurations from toolsheet
+def getToolsheetConfigurations() {
+    // Load toolsheet
+    def toolsheet_path = params.toolsheet_custom ?: "${projectDir}/assets/toolsheet.csv"
+
+    // Create temporary schema for validation
+    def schema_path = createToolsheetSchema()
+
+    // Load toolsheet and validate each row against the transformed schema
+    def raw_toolsheet = samplesheetToList(toolsheet_path, schema_path)
+
+    def toolsheet = raw_toolsheet
+        .findAll { row ->
+            // If analysis_name is set, only keep matching rows
+            if (params.analysis_name) {
+                return row[0].analysis_name == params.analysis_name
+            }
+            return true
+        }
+
+    if (toolsheet.isEmpty()) {
+        if (params.analysis_name) {
+            error("No configuration found in toolsheet for analysis_name '${params.analysis_name}'")
+        } else {
+            error("No valid configurations found in toolsheet")
+        }
+    }
+
+    def staticparams = params.findAll { k, v -> k != 'trace_report_suffix' } as Map
+    return toolsheet.collect{ row ->
+        ['id': staticparams.study_name] +staticparams + row[0]
     }
-    // for rna-seq studies, use the differential method specified in the toolsheet
-    // or the default differential method specified in the pipeline parameters
-    def method = tool_config?.diff_method ?: params.differential_method
-    def args = parseParams('differential', method, tool_config?.diff_args ?: null)
-    return [method: method, args: args]
 }
 
-/**
-* Get differential analysis tool configuration
-* @param tool_config The tool configuration.
-* @return A map of differential analysis configuration.
-*/
-def getDifferentialConfig(tool_config) {
-    def method = tool_config?.diff_method ?: params.differential_method
-    def args = parseParams('differential', method, tool_config?.diff_args ?: null)
-    def fc_threshold = args.differential_min_fold_change
-    def stat_threshold = args.differential_max_qval
-    return [
-        method: method,
-        args: args,
-        fc_threshold: fc_threshold,
-        stat_threshold: stat_threshold
-    ]
+// Get default configurations from pipeline parameters
+def getDefaultConfigurations() {
+    return [params + ['id': params.study_name]]
 }
 
-/**
-* Get functional analysis tool configuration
-* @param tool_config The tool configuration.
-* @return A map of functional analysis configuration.
-*/
-def getFunctionalConfig(tool_config) {
-    def method = tool_config?.func_method ?: params.functional_method
-    def args = parseParams('functional', method, tool_config?.func_args ?: null)
-    def input_type = method == 'gprofiler2' ? 'filtered' :
-                    method == 'gsea' ? 'norm' :
-                    null
-    return [
-        method: method,
-        args: args,
-        input_type: input_type
-    ]
+// Helper function to re-apply full parameters to deduplicated results
+def reapplyParams(channel, paramsets_by_name) {
+    channel
+        .map{meta, differential_results -> [meta.analysis_names, meta, differential_results]}
+        .transpose()
+        .join(paramsets_by_name)
+        .map{analysis_names, meta, differential_results, paramset -> [meta + ['study_meta': paramset], differential_results]}
 }
diff --git a/subworkflows/nf-core/abundance_differential_filter/main.nf b/subworkflows/nf-core/abundance_differential_filter/main.nf
index 3d0d0c22d..4bffa369a 100644
--- a/subworkflows/nf-core/abundance_differential_filter/main.nf
+++ b/subworkflows/nf-core/abundance_differential_filter/main.nf
@@ -12,7 +12,11 @@ include { CUSTOM_FILTERDIFFERENTIALTABLE      } from '../../../modules/nf-core/c
 // Combine meta maps, including merging non-identical values of shared keys (e.g. 'id')
 def mergeMaps(meta, meta2){
     (meta + meta2).collectEntries { k, v ->
-        meta[k] && meta[k] != v ? [k, "${meta[k]}_${v}"] : [k, v]
+        if (!(v instanceof Map) && meta[k] && meta[k] != v) {
+            [k, "${meta[k]}_${v}"]
+        } else {
+            [k, v]
+        }
     }
 }
 
@@ -25,14 +29,14 @@ workflow ABUNDANCE_DIFFERENTIAL_FILTER {
     ch_samplesheet           // [ meta_exp, samplesheet ]
     ch_transcript_lengths    // [ meta_exp, transcript_lengths]
     ch_control_features      // [meta_exp, control_features]
-    ch_contrasts             // [[ meta_contrast, contrast_variable, reference, target ]]
+    ch_contrasts             // [[ [meta_contrast], [contrast_variable], [reference], [target] ]]
 
     main:
 
     // Set up how the channels crossed below will be used to generate channels for processing
     def criteria = multiMapCriteria { meta_input, abundance, analysis_method, fc_threshold, stat_threshold, meta_exp, samplesheet, meta_contrasts, variable, reference, target ->
-        def meta_for_diff = mergeMaps(meta_contrasts, meta_input) + [ 'method_differential': analysis_method ]
-        def meta_input_new = meta_input + [ 'method_differential': analysis_method ]
+        def meta_for_diff = meta_contrasts + meta_input + [ 'differential_method': analysis_method ]
+        def meta_input_new = meta_input + [ 'differential_method': analysis_method ]
         samples_and_matrix:
             [ meta_input_new, samplesheet, abundance ]
         contrasts_for_diff:
@@ -47,7 +51,7 @@ workflow ABUNDANCE_DIFFERENTIAL_FILTER {
     // run differential analysis for every combination of matrix and contrast
     inputs = ch_input
         .combine(ch_samplesheet)
-        .combine(ch_contrasts)
+        .combine(ch_contrasts.transpose())
         .multiMap(criteria)
 
     // We only need a normalised matrix from one contrast. The reason we don't
@@ -56,7 +60,7 @@ workflow ABUNDANCE_DIFFERENTIAL_FILTER {
     // not returning the full normalized matrix as NORM modules would do.
     norm_inputs = ch_input
         .combine(ch_samplesheet)
-        .combine(ch_contrasts.first()) // Just taking the first contrast
+        .combine(ch_contrasts.transpose().first()) // Just taking the first contrast
         .multiMap(criteria)
 
     // ----------------------------------------------------
@@ -71,13 +75,13 @@ workflow ABUNDANCE_DIFFERENTIAL_FILTER {
     // LIMMA_NORM directly. It internally runs normalization + DE analysis.
 
     LIMMA_NORM(
-        norm_inputs.contrasts_for_norm.filter{it[0].method_differential == 'limma'},
-        norm_inputs.samples_and_matrix.filter{it[0].method_differential == 'limma'}
+        norm_inputs.contrasts_for_norm.filter{it[0].differential_method == 'limma'},
+        norm_inputs.samples_and_matrix.filter{it[0].differential_method == 'limma'}
     )
 
     LIMMA_DIFFERENTIAL(
-        inputs.contrasts_for_diff.filter{ it[0].method_differential == 'limma' },
-        inputs.samples_and_matrix.filter{ it[0].method_differential == 'limma' }
+        inputs.contrasts_for_diff.filter{ it[0].differential_method == 'limma' },
+        inputs.samples_and_matrix.filter{ it[0].differential_method == 'limma' }
     )
 
     // ----------------------------------------------------
@@ -92,15 +96,15 @@ workflow ABUNDANCE_DIFFERENTIAL_FILTER {
     // DESEQ2_NORM directly. It internally runs normalization + DE analysis.
 
     DESEQ2_NORM(
-        norm_inputs.contrasts_for_norm.filter{it[0].method_differential == 'deseq2'},
-        norm_inputs.samples_and_matrix.filter{it[0].method_differential == 'deseq2'},
+        norm_inputs.contrasts_for_norm.filter{it[0].differential_method == 'deseq2'},
+        norm_inputs.samples_and_matrix.filter{it[0].differential_method == 'deseq2'},
         ch_control_features.first(),
         ch_transcript_lengths.first()
     )
 
     DESEQ2_DIFFERENTIAL(
-        inputs.contrasts_for_diff.filter{it[0].method_differential == 'deseq2'},
-        inputs.samples_and_matrix.filter{it[0].method_differential == 'deseq2'},
+        inputs.contrasts_for_diff.filter{it[0].differential_method == 'deseq2'},
+        inputs.samples_and_matrix.filter{it[0].differential_method == 'deseq2'},
         ch_control_features.first(),
         ch_transcript_lengths.first()
     )
@@ -113,8 +117,8 @@ workflow ABUNDANCE_DIFFERENTIAL_FILTER {
     // not produce a normalized matrix.
 
     PROPR_PROPD(
-        inputs.contrasts_for_diff.filter{it[0].method_differential == 'propd'},
-        inputs.samples_and_matrix.filter { it[0].method_differential == 'propd' }
+        inputs.contrasts_for_diff.filter{it[0].differential_method == 'propd'},
+        inputs.samples_and_matrix.filter { it[0].differential_method == 'propd' }
     )
 
     // ----------------------------------------------------
@@ -162,14 +166,14 @@ workflow ABUNDANCE_DIFFERENTIAL_FILTER {
             ]
             filter_input: [meta + filter_meta, results]
             fc_input: [
-                method_params[meta.method_differential].fc_column,
+                method_params[meta.differential_method].fc_column,
                 filter_meta.fc_threshold,
-                method_params[meta.method_differential].fc_cardinality
+                method_params[meta.differential_method].fc_cardinality
             ]
             stat_input: [
-                method_params[meta.method_differential].stat_column,
+                method_params[meta.differential_method].stat_column,
                 filter_meta.stat_threshold,
-                method_params[meta.method_differential].stat_cardinality
+                method_params[meta.differential_method].stat_cardinality
             ]
         }
 
diff --git a/subworkflows/nf-core/differential_functional_enrichment/main.nf b/subworkflows/nf-core/differential_functional_enrichment/main.nf
index 802812a5d..279eaec73 100644
--- a/subworkflows/nf-core/differential_functional_enrichment/main.nf
+++ b/subworkflows/nf-core/differential_functional_enrichment/main.nf
@@ -41,7 +41,7 @@ workflow DIFFERENTIAL_FUNCTIONAL_ENRICHMENT {
     ch_input = ch_input
         .multiMap {
             meta_input, file, genesets, background, analysis_method ->
-            def meta_new = meta_input + [ 'method_functional': analysis_method ]
+            def meta_new = meta_input + [ 'functional_method': analysis_method ]
             input:
                 [ meta_new, file ]
             genesets:
@@ -69,9 +69,9 @@ workflow DIFFERENTIAL_FUNCTIONAL_ENRICHMENT {
     }
     ch_preinput_for_gsea = ch_input.input
         .join(ch_input.genesets)
-        .filter{ it[0].method_functional == 'gsea' }
+        .filter{ it[0].functional_method == 'gsea' }
         .combine(ch_samplesheet.join(ch_featuresheet))
-        .combine(ch_contrasts)
+        .combine(ch_contrasts.transpose())
         .multiMap(criteria)
 
     // ----------------------------------------------------
@@ -79,9 +79,9 @@ workflow DIFFERENTIAL_FUNCTIONAL_ENRICHMENT {
     // ----------------------------------------------------
 
     GPROFILER2_GOST(
-        ch_input.input.filter{ it[0].method_functional == 'gprofiler2' },
-        ch_input.genesets.filter{ it[0].method_functional == 'gprofiler2'},
-        ch_input.background.filter{ it[0].method_functional == 'gprofiler2'}
+        ch_input.input.filter{ it[0].functional_method == 'gprofiler2' },
+        ch_input.genesets.filter{ it[0].functional_method == 'gprofiler2'},
+        ch_input.background.filter{ it[0].functional_method == 'gprofiler2'}
     )
 
     // ----------------------------------------------------
@@ -116,8 +116,8 @@ workflow DIFFERENTIAL_FUNCTIONAL_ENRICHMENT {
     // ----------------------------------------------------
 
     PROPR_GREA(
-        ch_input.input.filter{ it[0].method_functional == 'grea' },
-        ch_input.genesets.filter{ it[0].method_functional == 'grea' }
+        ch_input.input.filter{ it[0].functional_method == 'grea' },
+        ch_input.genesets.filter{ it[0].functional_method == 'grea' }
     )
 
     // collect versions info
diff --git a/workflows/differentialabundance.nf b/workflows/differentialabundance.nf
index 9c3d7e65a..132482c46 100644
--- a/workflows/differentialabundance.nf
+++ b/workflows/differentialabundance.nf
@@ -1,99 +1,14 @@
-/*
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-    HANDLE INPUT PARAMETERS
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-*/
-
-def checkPathParamList = [ params.input ]
-for (param in checkPathParamList) { if (param) { file(param, checkIfExists: true) } }
-
-// Check mandatory parameters
-def exp_meta = [ "id": params.study_name  ]
-if (params.input) { ch_input = Channel.of([ exp_meta, file(params.input, checkIfExists: true) ]) } else { exit 1, 'Input samplesheet not specified!' }
-
-if (params.study_type == 'affy_array') {
-    if (params.affy_cel_files_archive) {
-        ch_celfiles = Channel.of([ exp_meta, file(params.affy_cel_files_archive, checkIfExists: true) ])
-    } else {
-        error("CEL files archive not specified!")
-    }
-} else if (params.study_type == 'maxquant') {
-
-    // Should the user have enabled functional analysis, and throw an error
-    if (params.functional_method ){
-        error("Functional analysis is not yet possible with maxquant input data; please set --functional_method to null and rerun pipeline!")
-    }
-    if (!params.matrix) {
-        error("Input matrix not specified!")
-    }
-    matrix_file = file(params.matrix, checkIfExists: true)
-
-    // Make channel for proteus
-    proteus_in = Channel.of([ file(params.input), matrix_file ])
-} else if (params.study_type == 'geo_soft_file') {
-
-    // To pull SOFT files from a GEO a GSE study identifer must be provided
-
-    if (params.querygse && params.features_metadata_cols) {
-        ch_querygse = Channel.of([exp_meta, params.querygse])
-    } else {
-        error("Query GSE not specified or features metadata columns not specified")
-    }
-} else {
-    // If this is not microarray data or maxquant output, and this an RNA-seq dataset,
-    // then assume we're reading from a matrix
-
-    if (params.study_type == "rnaseq" && params.matrix) {
-        matrix_file = file(params.matrix, checkIfExists: true)
-        ch_in_raw = Channel.of([ exp_meta, matrix_file])
-    } else {
-        error("Input matrix not specified!")
-    }
-
-}
-
-// Check optional parameters
-if (params.transcript_length_matrix) { ch_transcript_lengths = Channel.of([ exp_meta, file(params.transcript_length_matrix, checkIfExists: true)]).first() } else { ch_transcript_lengths = Channel.of([[],[]]) }
-if (params.control_features) { ch_control_features = Channel.of([ exp_meta, file(params.control_features, checkIfExists: true)]).first() } else { ch_control_features = Channel.of([[],[]]) }
-
-if (params.gene_sets_files) {
-    gene_sets_files = params.gene_sets_files.split(",")
-    ch_gene_sets = Channel.of(gene_sets_files).map { file(it, checkIfExists: true) }
-    if (params.functional_method == 'gprofiler2' && (!params.gprofiler2_token && !params.gprofiler2_organism) && gene_sets_files.size() > 1) {
-        error("gprofiler2 can currently only work with a single gene set file")
-    }
-} else {
-    ch_gene_sets = Channel.of([[]])
-
-    // TODO these checks should be done considering ch_tools instead
-    if (params.functional_method == 'gsea') {
-        error("GSEA activated but gene set file not specified!")
-    } else if (params.functional_method == 'gprofiler2') {
-        if (!params.gprofiler2_token && !params.gprofiler2_organism) {
-            error("To run gprofiler2, please provide a run token, GMT file or organism!")
-        }
-    }
-}
-
-// Report related files
-report_file = file(params.report_file, checkIfExists: true)
-logo_file = file(params.logo_file, checkIfExists: true)
-css_file = file(params.css_file, checkIfExists: true)
-citations_file = file(params.citations_file, checkIfExists: true)
-
-/*
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-    CONFIG FILES
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-*/
-
-
 /*
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     IMPORT LOCAL MODULES/SUBWORKFLOWS
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 */
 
+include { subsetMeta } from '../subworkflows/local/utils_nfcore_differentialabundance_pipeline/main'
+include { GROUP_BY_ANALYSIS as GROUP_BY_ANALYSIS_DIFFERENTIAL_RESULTS } from '../subworkflows/local/utils_nfcore_differentialabundance_pipeline/main'
+include { GROUP_BY_ANALYSIS as GROUP_BY_ANALYSIS_DIFFERENTIAL_FILTERED } from '../subworkflows/local/utils_nfcore_differentialabundance_pipeline/main'
+include { GROUP_BY_ANALYSIS as GROUP_BY_ANALYSIS_DIFFERENTIAL_NORM } from '../subworkflows/local/utils_nfcore_differentialabundance_pipeline/main'
+include { GROUP_BY_ANALYSIS as GROUP_BY_ANALYSIS_DIFFERENTIAL_VARSTAB } from '../subworkflows/local/utils_nfcore_differentialabundance_pipeline/main'
 
 /*
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -136,49 +51,102 @@ include { DIFFERENTIAL_FUNCTIONAL_ENRICHMENT                } from '../subworkfl
 workflow DIFFERENTIALABUNDANCE {
 
     take:
-    ch_tools
+    ch_paramsets
 
     main:
 
     ch_versions = Channel.empty()
+    ch_paramsets_by_name = ch_paramsets.map{paramset -> [paramset.analysis_name, paramset]}
+
+    // ========================================================================
+    // Handle input parameters
+    // ========================================================================
+
+    // Get the sample sheets
+    ch_samplesheet = ch_paramsets
+        .map { paramset ->
+            [ paramset, file(paramset.input, checkIfExists: true) ]
+        }
+
+    // Create input channels based on study type
+    ch_input = ch_samplesheet
+        .branch { meta, input ->
+            affy_array: meta.study_type == 'affy_array'
+            maxquant: meta.study_type == 'maxquant'
+            geo_soft_file: meta.study_type == 'geo_soft_file'
+            rnaseq: meta.study_type == 'rnaseq'
+        }
+
+    // Handle Affy array inputs
+    ch_celfiles = ch_input.affy_array
+        .map { meta, input ->
+            [ meta, file(meta.affy_cel_files_archive, checkIfExists: true) ]
+        }
+
+    // Handle Maxquant inputs
+    ch_maxquant_inputs = ch_input.maxquant
+        .map { meta, input ->
+            matrix_file = file(meta.matrix, checkIfExists: true)
+            [ meta, file(meta.input), matrix_file ]
+        }
+
+    // Handle GEO soft file inputs
+    ch_querygse = ch_input.geo_soft_file
+        .map { meta, input ->
+            [ meta, meta.querygse ]
+        }
+
+    // Create optional parameter channels based on ch_paramsets
+    ch_transcript_lengths = ch_paramsets
+        .map { paramset -> [ paramset, paramset.transcript_length_matrix ? file(paramset.transcript_length_matrix, checkIfExists: true) : [] ] }
+
+    ch_control_features = ch_paramsets
+        .map { paramset -> [ paramset, paramset.control_features ? file(paramset.control_features, checkIfExists: true) : [] ] }
+
+    ch_gene_sets = ch_paramsets
+        .map { paramset -> [ paramset, paramset.gene_sets_files ? paramset.gene_sets_files.split(",").collect { file(it, checkIfExists: true) } : [] ] }
 
     // ========================================================================
     // Handle contrasts
     // ========================================================================
 
-    // Channel for the contrasts file
-    if (params.contrasts_yml && params.contrasts) {
-        error("Both '--contrasts' and '--contrasts_yml' parameters are set. Please specify only one of these options to define contrasts.")
-    }
-    if (!(params.contrasts_yml || params.contrasts)) {
-        error("Either '--contrasts' and '--contrasts_yml' must be set. Please specify one of these options to define contrasts.")
-    }
+    // Create contrasts channels
+    ch_contrasts_file = ch_paramsets
+        .map { paramset ->
+            [ paramset, file(paramset.contrasts_yml ?: paramset.contrasts, checkIfExists: true) ]
+        }
 
-    // SUPPORT BOTH YAML AND CSV CONTRASTS FILE
-    if (params.contrasts_yml) {
-        //yaml contrasts file processing
-        ch_contrasts_file = Channel.from([[exp_meta, file(params.contrasts_yml)]])
-        ch_contrast_variables = ch_contrasts_file
-            .map { entry ->
-                def yaml_file = entry[1]
-                def yaml_data = new groovy.yaml.YamlSlurper().parse(yaml_file)
-                yaml_data.contrasts.collect { contrast ->
-                    tuple('id': contrast.comparison[0])
-                }
-            }
-            .flatten()
-            .unique() // Uniquify to keep each contrast variable only once (in case it exists in multiple lines for blocking etc.)
-    } else if (params.contrasts) {
-        //csv contrasts file processing
-        ch_contrasts_file = Channel.from([[exp_meta, file(params.contrasts)]])
-        ch_contrast_variables = ch_contrasts_file
-            .splitCsv(header:true, sep:(params.contrasts.endsWith('csv') ? ',' : '\t'))
-            .map{ it.tail().first() }
-            .map{
-                tuple('id': it.variable)
+    ch_contrasts_file_with_extension = ch_contrasts_file
+        .map {
+            paramset, file -> [paramset, file, file.extension]
+        }
+
+    ch_contrast_variables_input = ch_contrasts_file_with_extension
+        .branch{ paramset, file, extension ->
+            yml: extension == 'yml' || extension == 'yaml'
+            csv: extension == 'csv'
+            tsv: extension == 'tsv'
+        }
+
+    ch_contrasts_variables_from_yml = ch_contrast_variables_input.yml
+        .map { paramset, yaml_file, ext ->
+            def yaml_data = new groovy.yaml.YamlSlurper().parse(yaml_file)
+            yaml_data.contrasts.collect { contrast ->
+                tuple('id': contrast.comparison[0], 'study_meta': paramset)
             }
-            .unique()
-    }
+        }
+        .flatten()
+        .unique() // Uniquify to keep each contrast variable only once (in case it exists in multiple lines for blocking etc.)
+
+    ch_contrasts_variables_from_other = ch_contrast_variables_input.csv.splitCsv(header:true)
+        .mix(ch_contrast_variables_input.tsv.splitCsv(header:true, sep:'\t'))
+        .map { paramset, row, ext ->
+            ['id': row.variable, study_meta: paramset]
+        }
+        .unique()
+
+    ch_contrast_variables = ch_contrasts_variables_from_yml
+        .mix(ch_contrasts_variables_from_other)
 
     // ========================================================================
     // Data preprocessing
@@ -187,182 +155,221 @@ workflow DIFFERENTIALABUNDANCE {
     // If we have affy array data in the form of CEL files we'll be deriving
     // matrix and annotation from them
 
-    if (params.study_type == 'affy_array'){
+    // Uncompress the CEL files archive
+    UNTAR ( ch_celfiles )
 
-        // Uncompress the CEL files archive
+    ch_affy_input = ch_input.affy_array
+        .join(UNTAR.out.untar)
 
-        UNTAR ( ch_celfiles )
+    // Run affy to derive the matrix. Reset the meta so it can be used to
+    // define a prefix for different matrix flavours
 
-        ch_affy_input = ch_input
-            .join(UNTAR.out.untar)
+    AFFY_JUSTRMA_RAW (
+        ch_affy_input,
+        [[],[]]
+    )
+    AFFY_JUSTRMA_NORM (
+        ch_affy_input,
+        [[],[]]
+    )
 
-        // Run affy to derive the matrix. Reset the meta so it can be used to
-        // define a prefix for different matrix flavours
+    ch_affy_platform_features = AFFY_JUSTRMA_RAW.out.annotation
 
-        AFFY_JUSTRMA_RAW (
-            ch_affy_input,
-            [[],[]]
-        )
-        AFFY_JUSTRMA_NORM (
-            ch_affy_input,
-            [[],[]]
-        )
+    ch_versions = ch_versions
+        .mix(AFFY_JUSTRMA_RAW.out.versions)
 
-        // Fetch affy outputs and reset the meta
+    // We'll be running Proteus once per unique contrast variable to generate plots
 
-        ch_in_raw = AFFY_JUSTRMA_RAW.out.expression
-        ch_in_norm = AFFY_JUSTRMA_NORM.out.expression
+    // Run proteus to import protein abundances
+    PROTEUS(
+        ch_contrast_variables.combine(ch_maxquant_inputs)
+    )
 
-        ch_affy_platform_features = AFFY_JUSTRMA_RAW.out.annotation
+    ch_versions = ch_versions.mix(PROTEUS.out.versions)
 
-        ch_versions = ch_versions
-            .mix(AFFY_JUSTRMA_RAW.out.versions)
+    // Run GEO query to get the annotation
 
-    } else if (params.study_type == 'maxquant'){
+    GEOQUERY_GETGEO(ch_querygse)
+    ch_soft_features = GEOQUERY_GETGEO.out.annotation
 
-        // We'll be running Proteus once per unique contrast variable to generate plots
-        // TODO: there should probably be a separate plotting module in proteus to simplify this
+    ch_versions = ch_versions
+        .mix(GEOQUERY_GETGEO.out.versions)
 
-        // Run proteus to import protein abundances
-        PROTEUS(
-            ch_contrast_variables.combine(proteus_in)
-        )
+    // Raw inputs
+    // Note that we re-map the proteus output tables to the study ID as the
+    // tables are the same across contrasts, only one norm table will be necessary
 
-        // Re-map the proteus output tables to the study ID as the tables are the same across contrasts, only one norm table will be necessary
-        ch_in_raw = PROTEUS.out.raw_tab
-            .first()
-            .map{ meta, matrix -> tuple(exp_meta, matrix) }
-        ch_in_norm = PROTEUS.out.norm_tab
-            .first()
-            .map{ meta, matrix -> tuple(exp_meta, matrix) }
+    ch_in_raw = ch_input.rnaseq
+        .map{meta, file -> [meta, meta.matrix]}
+        .mix(AFFY_JUSTRMA_RAW.out.expression)
+        .mix(PROTEUS.out.raw_tab.first().map{ meta, matrix -> tuple(meta.study_meta, matrix) })
 
-        ch_versions = ch_versions.mix(PROTEUS.out.versions)
-    } else if(params.study_type == 'geo_soft_file'){
+    // Normalised inputs
 
-        GEOQUERY_GETGEO(ch_querygse)
-        ch_in_norm = GEOQUERY_GETGEO.out.expression
-        ch_soft_features = GEOQUERY_GETGEO.out.annotation
+    ch_in_norm = AFFY_JUSTRMA_NORM.out.expression
+        .mix(PROTEUS.out.norm_tab.first().map{ meta, matrix -> tuple(meta.study_meta, matrix) })
+        .mix(GEOQUERY_GETGEO.out.expression)
 
-        ch_versions = ch_versions
-            .mix(GEOQUERY_GETGEO.out.versions)
-    }
     //// Fetch or derive a feature annotation table
 
-    // If user has provided a feature annotation table, use that
-    if (params.features){
-        ch_features = Channel.of([ exp_meta, file(params.features, checkIfExists: true)])
-    } else if (params.study_type == 'affy_array'){
-        ch_features = ch_affy_platform_features
-    } else if(params.study_type == 'geo_soft_file') {
-        ch_features = ch_soft_features
-    } else if (params.gtf){
-        // Get feature annotations from a GTF file, gunzip if necessary
-
-        file_gtf_in = file(params.gtf)
-        file_gtf = [ [ "id": file_gtf_in.simpleName ], file_gtf_in ]
-
-        if ( params.gtf.endsWith('.gz') ){
-            GUNZIP_GTF(file_gtf)
-            file_gtf = GUNZIP_GTF.out.gunzip
-            ch_versions = ch_versions.mix(GUNZIP_GTF.out.versions)
+    // Branch ch_paramsets based on feature source
+    ch_feature_sources = ch_paramsets
+        .branch {
+            user_features: it.features
+            affy_features: it.study_type == 'affy_array'
+            geo_features: it.study_type == 'geo_soft_file'
+            gtf_features: it.gtf
+            matrix_features: true
         }
 
-        // Get a features table from the GTF and combine with the matrix and sample
-        // annotation (fom = features/ observations/ matrix)
+    // Handle user-provided feature annotations
+    ch_user_features = ch_feature_sources.user_features
+        .map { paramset -> [ paramset, file(paramset.features, checkIfExists: true) ] }
 
-        GTF_TO_TABLE( file_gtf, [[ "id":""], []])
-        ch_features = GTF_TO_TABLE.out.feature_annotation
-            .map{
-                tuple( exp_meta, it[1])
-            }
+    // Handle Affy array platform features
+    ch_affy_features = ch_feature_sources.affy_features
+        .map { paramset -> ch_affy_platform_features }
 
-        // Record the version of the GTF -> table tool
+    // Handle GEO soft file features
+    ch_geo_features = ch_feature_sources.geo_features
+        .map { paramset -> ch_soft_features }
 
-        ch_versions = ch_versions
-            .mix(GTF_TO_TABLE.out.versions)
-    }
-    else{
-
-        // Otherwise we can just use the matrix input; save it to the workdir so that it does not
-        // just appear wherever the user runs the pipeline
-        matrix_as_anno_filename = "${workflow.workDir}/${matrix_file.getBaseName()}_as_anno.${matrix_file.getExtension()}"
-        if (params.study_type == 'maxquant'){
-            ch_features_matrix = ch_in_norm
-        } else {
-            ch_features_matrix = ch_in_raw
+    // Handle GTF-based feature annotations
+    ch_gtf_files = ch_feature_sources.gtf_features
+        .map { paramset -> [ paramset, file(paramset.gtf, checkIfExists: true) ] }
+
+    // Process GTF files if needed
+    ch_gtf_for_processing = ch_gtf_files
+        .branch {
+            compressed: it[1].name.endsWith('.gz')
+            uncompressed: !it[1].name.endsWith('.gz')
         }
-        ch_features = ch_features_matrix
-            .map{ meta, matrix ->
-                matrix_copy = file(matrix_as_anno_filename)
-                matrix_copy.exists() && matrix.getText().md5().equals(matrix_copy.getText().md5()) ?: matrix.copyTo(matrix_as_anno_filename)
-                [ meta, file(matrix_as_anno_filename) ]
-            }
+
+    // Decompress GTF files if needed
+    GUNZIP_GTF(ch_gtf_for_processing.compressed)
+    ch_versions = ch_versions.mix(GUNZIP_GTF.out.versions)
+
+    // Combine compressed and uncompressed GTF files
+    ch_gtf_processed = ch_gtf_for_processing.uncompressed
+        .mix(GUNZIP_GTF.out.gunzip)
+
+    // Convert GTF to feature annotation table
+    GTF_TO_TABLE(ch_gtf_processed, [tuple('id':""), []])
+    ch_gtf_features = GTF_TO_TABLE.out.feature_annotation
+    ch_versions = ch_versions.mix(GTF_TO_TABLE.out.versions)
+
+    ch_pre_matrix_features = ch_feature_sources.matrix_features.branch{ paramset ->
+        maxquant: paramset.study_type == 'maxquant'
+        other: true
     }
+    ch_matrix_features = ch_pre_matrix_features.maxquant.join(ch_in_norm)
+        .mix(ch_pre_matrix_features.other.join(ch_in_raw))
+        .map{ meta, matrix ->
+            matrix_as_anno_filename = "${workflow.workDir}/${matrix.getBaseName()}_as_anno.${matrix.getExtension()}"
+            matrix_copy = file(matrix_as_anno_filename)
+            matrix_copy.exists() && matrix.getText().md5().equals(matrix_copy.getText().md5()) ?: matrix.copyTo(matrix_as_anno_filename)
+            [ meta, file(matrix_as_anno_filename) ]
+        }
+
+    // Combine all feature annotation channels
+    ch_features = ch_user_features
+        .mix(ch_affy_features)
+        .mix(ch_geo_features)
+        .mix(ch_gtf_features)
+        .mix(ch_matrix_features)
+
 
     // Check compatibility of FOM elements and contrasts
-    if (params.study_type == 'affy_array' || params.study_type == 'maxquant'){
-        ch_matrices_for_validation = ch_in_raw
-            .join(ch_in_norm)
-            .map{tuple(it[0], [it[1], it[2]])}
-    }
-    else if (params.study_type == 'geo_soft_file'){
-        ch_matrices_for_validation = ch_in_norm
-    }
-    else{
-        ch_matrices_for_validation = ch_in_raw
+
+    ch_matrix_sources = ch_paramsets.branch{
+        affy_or_maxquant: params.study_type == 'affy_array' || params.study_type == 'maxquant'
+        geo_soft_file: params.study_type == 'geo_soft_file'
+        other: true
     }
 
+    // Create a channel with the matrices for validation. For Affy and MaxQuant
+    // we have both raw and norm matrices, for GEO soft file we only have norm,
+    // for other we only have raw
+
+    ch_matrices_for_validation = ch_matrix_sources.affy_or_maxquant
+        .join(ch_in_raw)
+        .join(ch_in_norm)
+        .map{tuple(it[0], [it[1], it[2]])}
+        .mix(ch_matrix_sources.geo_soft_file.join(ch_in_norm)
+        .mix(ch_matrix_sources.other.join(ch_in_raw)))
+
+    // Validate the components agains one another. We try not to assume too
+    // much about chanel order, so we deploy the multiMap workaround.
+
+    validator_input = ch_samplesheet
+        .join(ch_matrices_for_validation)
+        .join(ch_features)
+        .join(ch_contrasts_file)
+        .multiMap{meta, samplesheet, matrices, features_file, contrasts_file ->
+            samplesheet_matrices: [meta, samplesheet, matrices]
+            features_file: [meta, features_file]
+            contrasts_file: [meta, contrasts_file]
+        }
+
     VALIDATOR(
-        ch_input.join(ch_matrices_for_validation),
-        ch_features,
-        ch_contrasts_file
+        validator_input.samplesheet_matrices,
+        validator_input.features_file,
+        validator_input.contrasts_file
     )
 
     // For Affy, we've validated multiple input matrices for raw and norm,
     // we'll separate them out again here
 
-    if (params.study_type == 'affy_array' || params.study_type == 'maxquant'){
-        ch_validated_assays = VALIDATOR.out.assays
-            .transpose()
-            .branch {
-                raw: it[1].name.contains('raw')
-                normalised: it[1].name =~ /normali[sz]ed/
-            }
-        ch_raw = ch_validated_assays.raw
-        ch_norm = ch_validated_assays.normalised
-    }
-    else if (params.study_type == 'geo_soft_file') {
-        ch_norm = VALIDATOR.out.assays
-    }
+    ch_multi_validated_assays = VALIDATOR.out.assays
+        .filter{meta, assays -> meta.study_type == 'affy_array' || meta.study_type == 'maxquant'}
+        .transpose()
+        .branch { meta, assay ->
+            raw: assay.name.contains('raw')
+            normalised: assay.name =~ /normali[sz]ed/
+        }
 
-    if(params.study_type != 'rnaseq') {
-        ch_matrix_for_differential = ch_norm
-    }
-    else{
-        ch_raw = VALIDATOR.out.assays
-        ch_matrix_for_differential = ch_raw
-    }
+    // Get raw matrices from the validation
+
+    ch_raw = ch_multi_validated_assays.raw
+        .mix(VALIDATOR.out.assays.filter{meta, assay -> meta.study_type== 'rnaseq'})
+
+    // For RNASeq and GEO soft file we've validated a single matrix, raw in the
+    // case of RNASeq and norm in the case of GEO soft file, and these are the
+    // matrices we'll use for differential abundance testing. For affy or maxquant
+    // we'll use the normalised matrices.
+
+    ch_matrix_for_differential = ch_multi_validated_assays.normalised
+        .mix(VALIDATOR.out.assays.filter{meta, assay -> meta.study_type == 'rnaseq' || meta.study_type == 'geo_soft_file'})
 
     // Split the contrasts up so we can run differential analyses and
     // downstream plots separately.
     // Replace NA strings that might have snuck into the blocking column
 
     ch_contrasts = VALIDATOR.out.contrasts
-        .map{it[1]}
         .splitCsv ( header:true, sep:'\t' )
-        .map{
-            it.blocking = it.blocking.replaceAll('^NA$', '')
-            if (!it.id){
-                it.id = it.values().join('_')
+        .map{study_meta, contrast ->
+            contrast.blocking = contrast.blocking.replaceAll('^NA$', '')
+            if (!contrast.id){
+                contrast.id = contrast.values().join('_')
             }
-            tuple(it, it.variable, it.reference, it.target)
+            meta = contrast + [study_meta: study_meta]
+            tuple(study_meta, meta,contrast.variable, contrast.reference, contrast.target)
         }
+        .groupTuple() // [study_meta, [meta], [variable], [reference], [target]]
+        .map{ study_meta, meta, variable, reference, target -> [study_meta, [meta, variable, reference, target]]}
 
     // Firstly Filter the input matrix
 
+    ch_matrixfilter_input = ch_matrix_for_differential
+        .join(VALIDATOR.out.sample_meta)
+        .multiMap{meta, matrix, samplesheet ->
+            matrix: [meta, matrix]
+            samplesheet: [meta, samplesheet]
+        }
+
     CUSTOM_MATRIXFILTER(
-        ch_matrix_for_differential,
-        VALIDATOR.out.sample_meta
+        ch_matrixfilter_input.matrix,
+        ch_matrixfilter_input.samplesheet
     )
 
     // ========================================================================
@@ -370,62 +377,95 @@ workflow DIFFERENTIALABUNDANCE {
     // ========================================================================
 
     // Prepare inputs for differential processes
-
     ch_differential_input = CUSTOM_MATRIXFILTER.out.filtered
-        .combine(ch_tools)
-        .map { meta, matrix, tools_norm, tools_diff, tools_func ->
-            [
-                // as the differential subworkflow will automatically add
-                // method_differential into meta, here we only need to add
-                // args_differential into meta. In this way, the differential
-                // outputs should always have in the meta the method_differential
-                // and args_differential
-                meta + [args_differential: tools_diff.args],
-                matrix,
-                tools_diff.method,
-                tools_diff.fc_threshold,
-                tools_diff.stat_threshold
-            ]
+        .join(VALIDATOR.out.sample_meta)
+        .join(ch_transcript_lengths)
+        .join(ch_control_features)
+        .join(ch_contrasts)
+
+    // Deduplicate differential analysis runs
+    ch_deduplicated_differential_input = ch_differential_input
+        .map{ meta, matrix, samplesheet, transcript_lengths, control_features, contrasts ->
+            def simplified_meta = subsetMeta(meta, 'differential')
+            [simplified_meta, meta.analysis_name, matrix, samplesheet, transcript_lengths, control_features, contrasts]
+        }
+        .groupTuple()
+        .map{ diff_meta, analysis_names, matrix, samplesheet, transcript_lengths, control_features, contrasts ->
+            diff_meta.analysis_names = analysis_names
+            [diff_meta, matrix[0], samplesheet[0], transcript_lengths[0], control_features[0], contrasts[0]]
+        }
+
+    // Use a multiMap to generate synched channels for differential analysis
+    ch_differential_input = ch_deduplicated_differential_input
+        .multiMap{ meta, matrix, samplesheet, transcript_lengths, control_features, contrasts ->
+            input: [meta, matrix, meta.differential_method, meta.differential_min_fold_change, meta.differential_max_qval]
+            samplesheet: [meta, samplesheet]
+            transcript_lengths: [meta, transcript_lengths]
+            control_features: [meta, control_features]
+            contrasts: [meta, contrasts]
         }
 
     // Run differential analysis
 
     ABUNDANCE_DIFFERENTIAL_FILTER(
-        ch_differential_input,
-        VALIDATOR.out.sample_meta,
-        ch_transcript_lengths,
-        ch_control_features,
-        ch_contrasts
+        ch_differential_input.input,
+        ch_differential_input.samplesheet,
+        ch_differential_input.transcript_lengths,
+        ch_differential_input.control_features,
+        ch_differential_input.contrasts.map{ study_meta, contrasts -> contrasts }
+    )
+
+    // Collect differential results
+    // S
+
+    GROUP_BY_ANALYSIS_DIFFERENTIAL_RESULTS(
+        ABUNDANCE_DIFFERENTIAL_FILTER.out.results_genewise,
+        ch_paramsets_by_name
+    )
+    ch_differential_results = GROUP_BY_ANALYSIS_DIFFERENTIAL_RESULTS.out.with_study_meta
+
+    GROUP_BY_ANALYSIS_DIFFERENTIAL_FILTERED(
+        ABUNDANCE_DIFFERENTIAL_FILTER.out.results_genewise_filtered,
+        ch_paramsets_by_name
     )
+    ch_differential_results_filtered = GROUP_BY_ANALYSIS_DIFFERENTIAL_FILTERED.out.with_study_meta
 
-    // collect differential results
+    // Processed matrices, one pers study/ toolsheet row, so we can reset the meta to the study_meta
+
+    GROUP_BY_ANALYSIS_DIFFERENTIAL_NORM(
+        ABUNDANCE_DIFFERENTIAL_FILTER.out.normalised_matrix,
+        ch_paramsets_by_name
+    )
+    ch_differential_norm = GROUP_BY_ANALYSIS_DIFFERENTIAL_NORM.out.with_study_meta
+        .map{study_meta, metas, matrix -> [study_meta, matrix]}
+
+    GROUP_BY_ANALYSIS_DIFFERENTIAL_VARSTAB(
+        ABUNDANCE_DIFFERENTIAL_FILTER.out.variance_stabilised_matrix,
+        ch_paramsets_by_name
+    )
+    ch_differential_varstab = GROUP_BY_ANALYSIS_DIFFERENTIAL_VARSTAB.out.with_study_meta
+        .map{study_meta, metas, matrix -> [study_meta, matrix]}
 
-    ch_differential_results = ABUNDANCE_DIFFERENTIAL_FILTER.out.results_genewise
-    ch_differential_results_filtered = ABUNDANCE_DIFFERENTIAL_FILTER.out.results_genewise_filtered
     ch_differential_model = ABUNDANCE_DIFFERENTIAL_FILTER.out.model
-    ch_differential_norm = ABUNDANCE_DIFFERENTIAL_FILTER.out.normalised_matrix
-    ch_differential_varstab = ABUNDANCE_DIFFERENTIAL_FILTER.out.variance_stabilised_matrix
 
     ch_versions = ch_versions
         .mix(ABUNDANCE_DIFFERENTIAL_FILTER.out.versions)
 
-    // when the study type is rnaseq, we use the normalised matrix from the differential subworkflow
-    if (params.study_type == 'rnaseq') {
-        ch_norm = ch_differential_norm
-    } else {
-        // when the study type is not rnaseq, the normalised matrix comes directly from VALIDATOR.out
-        // we need to update the meta so that it can match with tools_norm
-        ch_norm = ch_norm
-            .map { meta, norm ->
-                def meta_new = meta + [method_differential: 'validator', args_differential: [:]]
-                [meta_new, norm]
-            }
-    }
+    // Derive a channel of normalised matrices
+    // - from differential analysis for RNASeq
+    // - from normalisedvalidated assays for Affy and MaxQuant
+    // - from validated assays for GEO soft file
+
+    ch_norm = ch_differential_norm
+        .filter{meta, matrix -> meta.study_type == 'rnaseq'}
+        .mix(ch_multi_validated_assays.normalised)
+        .mix(VALIDATOR.out.assays.filter{meta, assay -> meta.study_type == 'geo_soft_file'})
+
+    // Prepare channel with normalized matrix, and variance stabilized matrices when available
 
-    // prepare channel with normalized matrix, and variance stabilized matrices when available
     ch_processed_matrices = ch_norm.join(ch_differential_varstab, remainder: true)
         .map { meta, norm, vs ->
-            def matrices = vs ? [norm] + vs : [norm]
+            def matrices = vs ? [norm, vs] : [norm]
             [meta, matrices]
         }
 
@@ -434,57 +474,51 @@ workflow DIFFERENTIALABUNDANCE {
     // ========================================================================
 
     // Prepare background file - for the moment it is only needed for gprofiler2
-    // TODO: prepare this channel based on ch_tools
-
-    ch_background = Channel.of([[]])
-    if (params.functional_method == 'gprofiler2') {
-        if (params.gprofiler2_background_file == "auto") {
-            // If auto, use input matrix as background
-            ch_background = CUSTOM_MATRIXFILTER.out.filtered.map{it.tail()}.first()
-        } else {
-            ch_background = Channel.from(file(params.gprofiler2_background_file, checkIfExists: true))
-        }
-    }
+
+    ch_background = CUSTOM_MATRIXFILTER.out.filtered
+        .filter{meta, matrix -> meta.functional_method == 'gprofiler2' && params.gprofiler2_background_file == "auto"}
+        .mix(
+            ch_paramsets
+                .filter{paramset -> paramset.functional_method == 'gprofiler2' && ! paramset.gprofiler2_background_file }
+                .map{paramset -> [paramset, file(paramset.gprofiler2_background_file, checkIfExists: true)]}
+        )
+        .mix(
+            ch_paramsets
+                .filter{ paramset -> paramset.functional_method != 'gprofiler2'}
+                .map{paramset -> [paramset, []]}
+        )
 
     // Prepare input for functional analysis
 
-    // Some functional analysis methods act directly on the differential analysis results, some on the normalised matrix.
-    // By crossing with ch_tools, we pair the correct input file with the correct functional analysis method, taking into
-    // account the upstream differential method, args and data type (normalized matrix or filtered differential analysis
-    // results).
-    ch_functional_input = ch_norm.map { meta, input ->
-            [[method: meta.method_differential, args: meta.args_differential, type: 'norm'], meta, input]
-        }
+    ch_functional_analysis_matrices = ch_norm
+        .filter{meta, matrix -> meta.functional_method == 'gsea'}
         .mix(
-            ch_differential_results_filtered.map { meta, input ->
-                [[method: meta.method_differential, args: meta.args_differential, type: 'filtered'], meta, input]
-            }
-        )
-        .cross(
-            ch_tools.map { tools_norm, tools_diff, tools_func ->
-                [[method: tools_norm.method, args: tools_norm.args, type: tools_func.input_type], tools_func]
-            }
+            ch_differential_results_filtered
+                .filter{study_meta, meta, results -> meta.functional_method == 'gprofiler2'}
+                .map{study_meta, metas, results -> [study_meta, input]}
+                .transpose()
         )
-        .map { input, tools ->
-            // the functional analysis subworkflow will automatically add method_functional into meta
-            // so here we only need to add args_functional into meta. In this way, the functional outputs
-            // will have in the meta: method_differential, args_differential, method_functional and args_functional
-            def meta = input[1] + [args_functional: tools[1].args]
-            [meta, input[2], tools[1].method]  // meta, input, functional analysis method
-        }
-        .combine(ch_gene_sets)
-        .combine(ch_background)
-        .map { meta, input, method, gene_sets, background ->
-            [meta, input, gene_sets, background, method]
+
+    ch_functional_input = ch_functional_analysis_matrices
+        .join(ch_gene_sets)
+        .join(ch_background)
+        .join(ch_contrasts)
+        .join(VALIDATOR.out.sample_meta)
+        .join(VALIDATOR.out.feature_meta)
+        .multiMap{meta, input, gene_sets, background, contrasts, samplesheet, features ->
+            input: [meta, input, gene_sets, background, meta.functional_method]
+            contrasts: [meta, contrasts]
+            samplesheet: [meta, samplesheet]
+            features: [meta, features, meta.features_id_col, meta.features_name_col]
         }
 
     // Run functional analysis
 
     DIFFERENTIAL_FUNCTIONAL_ENRICHMENT(
-        ch_functional_input,
-        ch_contrasts,
-        VALIDATOR.out.sample_meta,
-        VALIDATOR.out.feature_meta.combine(Channel.of([params.features_id_col, params.features_name_col]))
+        ch_functional_input.input,
+        ch_functional_input.contrasts.map{study_meta, contrasts -> contrasts},
+        ch_functional_input.samplesheet,
+        ch_functional_input.features
     )
 
     // Collect functional analysis results
@@ -503,51 +537,78 @@ workflow DIFFERENTIALABUNDANCE {
         .mix(DIFFERENTIAL_FUNCTIONAL_ENRICHMENT.out.versions)
 
     // ========================================================================
-    // Plot figures
+    // Other analyses
     // ========================================================================
 
-    // The exploratory plots are made by coloring by every unique variable used
-    // to define contrasts
+    // Run IMMUNEDECONV
 
-    ch_contrast_variables = ch_contrasts
-        .map{
-            [ "id": it[1] ]
+    ch_immunedeconv_input = ch_raw
+        .filter{meta, raw -> meta.immunedeconv_run}
+        .multiMap{meta, raw ->
+            input: [meta, raw, meta.immunedeconv_method, meta.immunedeconv_function]
+            name_col: meta.features_name_col
         }
-        .unique()
+
+    IMMUNEDECONV(
+        ch_immunedeconv_input.input,
+        ch_immunedeconv_input.name_col
+    )
+
+    ch_versions = ch_versions
+        .mix(IMMUNEDECONV.out.versions)
+
+    // ========================================================================
+    // Plot figures
+    // ========================================================================
 
     // For geoquery we've done no matrix processing and been supplied with the
     // normalised matrix, which can be passed through to downstream analysis
-    if (params.study_type == "geo_soft_file") {
-        ch_mat = ch_norm.map {meta, norm -> [meta, [norm]]}
-    } else {
-        // otherwise, we take all the matrices: raw, normalized matrix,
-        // and variance stabilized matrices if available
-        ch_mat = ch_raw.map{ it[1] }
-            .combine(ch_processed_matrices)
-            .map { raw, meta, matrices ->
-                [meta, [raw] + matrices]
-            }
-    }
 
-    ch_all_matrices = VALIDATOR.out.sample_meta                 // meta_exp, samples
-        .join(VALIDATOR.out.feature_meta)                       // meta_exp, samples, features
-        .combine(ch_mat)                                        // meta_mat, list of matrices (raw, norm, variance stabilized)
-        .map { meta_exp, samples, features, meta_mat, matrices ->
-            return [meta_mat, samples, features, matrices]
+    ch_mat = ch_norm.filter{meta, matrix -> meta.study_type == 'geo_soft_file'}
+        .map{meta, norm -> [meta, [norm]]}
+        .mix(
+            ch_raw
+                .filter{meta, raw -> meta.study_type != 'geo_soft_file'}
+                .join(ch_processed_matrices)
+                .map { meta, raw, matrices ->
+                    [meta, [raw] + matrices]
+                }
+        )
+
+    ch_all_matrices = VALIDATOR.out.sample_meta              // meta_exp, samples
+        .join(VALIDATOR.out.feature_meta)                    // meta_exp, samples, features
+        .join(ch_mat)                                        // meta_mat, list of matrices (raw, norm, variance stabilized)
+        .map { meta, samples, features, matrices ->
+            return [meta, samples, features, matrices]
         }
 
     // Exploratory analysis
 
+    ch_exploratory_input = ch_contrast_variables         // [meta]
+        .map{meta -> [ meta.study_meta, meta.id] }       // [study_meta, variable]
+        .groupTuple()                                    // [study_meta, [variable]]
+        .combine(ch_all_matrices.groupTuple(), by: 0)    // [study_meta, [variable], [samples, features, [matrices]]]
+        .transpose()                                     // [study_meta, variable, samples, features, [matrices]]
+        .map{study_meta, variable, samples, features, matrices ->
+            [['id': variable] + ['study_meta': study_meta], samples, features, matrices]
+        }
+
     PLOT_EXPLORATORY(
-        ch_contrast_variables
-            .combine(ch_all_matrices.map{ it.tail() })
+        ch_exploratory_input
     )
 
     // Plot differential analysis results
 
+    ch_plot_differential_input = ch_differential_results
+        .combine(ch_all_matrices, by: 0)
+        .multiMap{study_meta, meta, differential_results, samples, features, matrices ->
+            differential_results: [meta, differential_results]
+            samples_features_matrices: [meta, samples, features, matrices]
+        }
+
     PLOT_DIFFERENTIAL(
-        ch_differential_results,
-        ch_all_matrices.first()
+        ch_plot_differential_input.differential_results,
+        ch_plot_differential_input.samples_features_matrices
     )
 
     // Gather software versions
@@ -574,196 +635,26 @@ workflow DIFFERENTIALABUNDANCE {
     ch_collated_versions = softwareVersionsToYAML(ch_versions)
         .collectFile(storeDir: "${params.outdir}/pipeline_info", name: 'collated_versions.yml', sort: true, newLine: true)
 
-    // Generate a list of files that will be used by the markdown report
-
-    ch_report_file = Channel.from(report_file)
-        .map{ tuple(exp_meta, it) }
-
-    ch_logo_file = Channel.from(logo_file)
-    ch_css_file = Channel.from(css_file)
-    ch_citations_file = Channel.from(citations_file)
-
-    // create a list of input files for the report
-    // This list will be created considering the tools used in the pipeline.
-    // So all the files that were produced following a concrete set of tools
-    // will go together to the report. This is needed later on for running
-    // multiple tools in parallel (future implementation). It is also needed
-    // to transfer the args in ch_tools properly to the report module.
-
-    // first we parse the matrices channel with a tool-based key
-    ch_matrices_with_key = ch_all_matrices
-        .map { meta, samples, features, matrices ->
-            [[method: meta.method_differential, args: meta.args_differential], samples, features, matrices]
-        }
-        .cross (
-            ch_tools.map { tools_norm, tools_diff, tools_func -> [
-                [method: tools_norm.method, args: tools_norm.args],
-                [method_differential: tools_diff.method, args_differential: tools_diff.args, method_functional: tools_func.method, args_functional: tools_func.args]
-            ]}
-        )
-        .map { files, tools ->
-            [tools[1], files[1], files[2], files[3]]  // key, samples, features, [ matrices ]
-        }
-
-    // we parse the differential analysis outputs channel with a tool-based key
-    ch_differential_with_key = ch_differential_results
-        .join(ch_differential_model, remainder: true)
-        .map { meta, results, model ->
-            [[method: meta.method_differential, args: meta.args_differential], results, model]
-        }
-        .groupTuple()   // group all the files produced for different contrasts together
-        .cross (
-            ch_tools.map { tools_norm, tools_diff, tools_func -> [
-                [method: tools_diff.method, args: tools_diff.args],
-                [method_differential: tools_diff.method, args_differential: tools_diff.args, method_functional: tools_func.method, args_functional: tools_func.args]
-            ]}
-        )
-        .map { files, tools ->
-            [tools[1], files[1], files[2]]  // key, [ results ], [ models ]
-        }
-
-    // we parse the functional analysis outputs channel with a tool-based key
-    ch_functional_with_key = ch_functional_results
-        .map {[
-            [method_differential: it[0].method_differential, args_differential: it[0].args_differential, method_functional: it[0].method_functional, args_functional: it[0].args_functional],
-            it.tail()
-        ]}
-        .groupTuple()   // group all the files produced for different contrasts together
-        .cross (
-            ch_tools.map { tools_norm, tools_diff, tools_func -> [
-                // note that the normalized matrix is used to run gsea, whereas differential results are used to run gprofiler2
-                [method_differential: (tools_func.method == 'gsea' ? tools_norm.method : tools_diff.method), args_differential: (tools_func.method == 'gsea' ? tools_norm.args : tools_diff.args), method_functional: tools_func.method, args_functional: tools_func.args],
-                [method_differential: tools_diff.method, args_differential: tools_diff.args, method_functional: tools_func.method, args_functional: tools_func.args]
-            ]}
-        )
-        .map { files, tools ->
-            [tools[1], files.tail()]    // key, [ files ]
-        }
-
-    // we combine the channels defined above with the contrasts, collated versions,
-    // logo, css and citations files. We also combine with the outputs from differential
-    // and functional analysis using the tool-based key:
-    // [method_differential, args_differential, method_functional, args_functional]
-    ch_report_input_files = ch_matrices_with_key
-        .combine(VALIDATOR.out.contrasts.map{it.tail()})
-        .combine(ch_collated_versions)
-        .combine(ch_logo_file)
-        .combine(ch_css_file)
-        .combine(ch_citations_file)
-        .join(ch_differential_with_key)
-        .join(ch_functional_with_key, remainder: true)
-        .map {
-            // functional analysis is optional. When it is not run, joining an empty
-            // ch_functional_with_key channel with remainder:true will result in a
-            // channel with a meta containing null values for method_functional and
-            // args_functional, and null files. Since args_functional should be a
-            // map, here we update meta to keep args_functional as a map [:]. We
-            // also remove the null files using grep().
-            def meta = (it[0].method_functional) ? it[0] : it[0] + [args_functional: [:]]
-            [meta, it.tail().grep().flatten()]    // meta, files
-        }
-
-
-    // Run IMMUNEDECONV
-    if (params.immunedeconv_run){
-        matrix_file = file(params.matrix, checkIfExists:true)
-        IMMUNEDECONV(
-            [ [id:matrix_file.baseName], matrix_file, params.immunedeconv_method, params.immunedeconv_function ],
-            params.features_name_col
-        )
-        ch_versions = ch_versions
-            .mix(IMMUNEDECONV.out.versions)
-    }
-
-    if (params.shinyngs_build_app){
-
-        // Make (and optionally deploy) the shinyngs app
-
-        // parse matrices and differential results considering the tool-based key
-        ch_matrices_with_differential = ch_matrices_with_key
-            .join(ch_differential_with_key)
-            .multiMap { key, samples, features, matrices, results, models ->
-                def meta = exp_meta + key
-                matrices:
-                [meta, samples, features, matrices]   // meta, samples, features, [  matrices ]
-                results:
-                [meta, results]                       // meta, [ results ]
-            }
-
-        // Make a new contrasts file from the differential metas to guarantee the
-        // same order as the differential results
-        def contrast_columns = ['variable','reference','target','blocking']
-        ch_app_differential = ch_differential_results.first().map{contrast_columns.join(',')}
-            .concat(
-                ch_differential_results.map{it[0].subMap(contrast_columns).values().join(',')}
-            )
-            .collectFile(name: 'contrasts.csv', newLine: true, sort: false)
-            // combine with differential results
-            .combine (ch_matrices_with_differential.results)
-            .map { contrasts, meta, results ->
-                [meta, contrasts, results]
-            }
-
-        SHINYNGS_APP(
-            ch_matrices_with_differential.matrices, // meta, samples, features, [  matrices ]
-            ch_app_differential, // meta, contrasts, [differential results]
-            params.exploratory_assay_names.split(',').findIndexOf { it == params.exploratory_final_assay } + 1
-        )
-        ch_versions = ch_versions.mix(SHINYNGS_APP.out.versions)
-    }
-
-    // Make a params list - starting with the input matrices and the relevant
-    // params to use in reporting
+    // Derive the report file
 
-    def report_file_names = [ 'observations', 'features' ] +
-        params.exploratory_assay_names.split(',').collect { "${it}_matrix".toString() } +
-        [ 'contrasts_file', 'versions_file', 'logo', 'css', 'citations' ]
+    ch_report_file = ch_paramsets
+        .map{ paramset -> tuple(paramset, paramset.report_file) }
 
-    // Condition params reported on study type
-    def params_pattern = "report|gene_sets|study|observations|features|filtering|exploratory|differential"
-    if (params.study_type == 'affy_array' || params.study_type == 'geo_soft_file'){
-        params_pattern += "|affy"
-    } else if (params.study_type == 'maxquant'){
-        params_pattern += "|proteus"
-    }
+    // Generate a list of files that will be used by the markdown report
 
-    // prepare the final report params
-    ch_report_params = ch_report_input_files
-        .map{ meta, files ->
-            // the params scope don't necessarily include the args parsed from toolsheet.
-            // Hence we need to consider both params and the args parsed from toolsheet
-            // (args_differential and args_functional) to create the final report params.
-            def params_with_tools = params + meta.args_differential + meta.args_functional
-
-            // update the pattern with tools-specific info
-            def pattern_with_tools = params_pattern +
-                "|${meta.method_differential}" +
-                (meta.method_functional ? "|functional|${meta.method_functional}" : "")
-
-            // Extract and return parameters for the report by creating a map where
-            // the keys are parameter names matching the pattern and the values are
-            // their corresponding parameter values. And add this map to the report
-            // files map.
-            params_with_tools.findAll{ k,v -> k.matches(~/(${pattern_with_tools}).*/) } +
-            [report_file_names, files.collect{ f -> f.name}].transpose().collectEntries()
+    ch_report_files = ch_paramsets
+        .map { paramset ->
+            [ paramset, [
+                file(paramset.report_file, checkIfExists: true),
+                file(paramset.logo_file, checkIfExists: true),
+                file(paramset.css_file, checkIfExists: true),
+                file(paramset.citations_file, checkIfExists: true)
+            ]]
         }
+        .join(ch_all_matrices)
+        .join(VALIDATOR.out.contrasts)
+        .combine(ch_differential_results, by: 0)
 
-    // Render the final report
-
-    RMARKDOWNNOTEBOOK(
-        ch_report_file,
-        ch_report_params,
-        ch_report_input_files.map{it.tail().flatten()}
-    )
-
-    // Make a report bundle comprising the markdown document and all necessary
-    // input files
-
-    MAKE_REPORT_BUNDLE(
-        RMARKDOWNNOTEBOOK.out.parameterised_notebook
-            .combine(ch_report_input_files.map{it.tail().flatten()})
-            .map{[it[0], it[1..-1]]}
-    )
 
 }
 


Reviews:
user: login: grst
body: Hi @pinin4fjords and @suzannejin, 

I like the direction this is going! 

>  I propose we simplify this by making the toolsheet directly represent pipeline parameters. 
> Each row represents a complete set of pipeline parameters

This sounds a whole lot like nextflow profiles. With the difference being that multiple "profiles" can be run in a single pipeline invokation? 

I'm also still trying to understand how the toolsheet would look like for my use-cases. If I want to run RNA-seq analysis with limma/voom + apply different functional analysis tools (e.g. gsea on gene ontology + decoupler on 3 different signatures), would that look somewhat like

```csv
analysis_name,study_type,differential_method,limma_use_voom,functional_method,signature_file
limma_rnaseq,rnaseq,limma,true,gsea,gene_ontology.gmt
limma_rnaseq,rnaseq,limma,true,decoupler,progeny.tsv
limma_rnaseq,rnaseq,limma,true,decoupler,collectri.tsv
limma_rnaseq,rnaseq,limma,true,decoupler,hallmark.tsv
```

?


Would that then run limma four times with the same settings, or run it once and reuse the results for the different functional analyses? 
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-04T06:47:27Z
user: login: pinin4fjords
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-04T08:30:55Z
user: login: pinin4fjords
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-04T08:35:49Z
user: login: pinin4fjords
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-04T08:36:52Z
user: login: pinin4fjords
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-04T08:38:29Z
user: login: suzannejin
state: COMMENTED
author_association: NONE
submitted_at: 2025-04-04T10:15:55Z
user: login: pinin4fjords
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-04T10:19:59Z
user: login: suzannejin
state: COMMENTED
author_association: NONE
submitted_at: 2025-04-04T10:21:05Z
user: login: suzannejin
state: COMMENTED
author_association: NONE
submitted_at: 2025-04-04T10:25:57Z
user: login: pinin4fjords
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-04T10:30:24Z
user: login: suzannejin
state: COMMENTED
author_association: NONE
submitted_at: 2025-04-04T11:55:32Z
user: login: suzannejin
state: COMMENTED
author_association: NONE
submitted_at: 2025-04-07T10:10:37Z
user: login: suzannejin
state: COMMENTED
author_association: NONE
submitted_at: 2025-04-07T10:12:24Z
user: login: suzannejin
state: COMMENTED
author_association: NONE
submitted_at: 2025-04-07T10:31:33Z
user: login: suzannejin
state: COMMENTED
author_association: NONE
submitted_at: 2025-04-07T10:37:43Z
user: login: pinin4fjords
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-07T10:57:19Z
user: login: pinin4fjords
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-07T11:00:52Z
user: login: pinin4fjords
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-07T11:05:32Z
user: login: pinin4fjords
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-07T11:07:17Z
user: login: pinin4fjords
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-07T11:07:50Z
user: login: suzannejin
state: APPROVED
author_association: NONE
submitted_at: 2025-04-07T13:53:41Z

Review comments:
diff_hunk: @@ -135,6 +142,63 @@ workflow PIPELINE_COMPLETION {
 //
 def validateInputParameters() {
     genomeExistsError()
+
+    // Check file existence for input parameters
+    def checkPathParamList = [ params.input ]
+    for (param in checkPathParamList) {
+        if (param) {
+            file(param, checkIfExists: true)
+        }
+    }
+
+    // Check mandatory parameters
+    if (!params.input) {
+        error("Input samplesheet not specified!")
+    }
+
+    // Validate study type specific parameters
+    if (params.study_type == 'affy_array') {
+        if (!params.affy_cel_files_archive) {
+            error("CEL files archive not specified!")
+        }
+    } else if (params.study_type == 'maxquant') {
+        if (params.functional_method) {
path: subworkflows/local/utils_nfcore_differentialabundance_pipeline/main.nf
user: login: grst
body: Also not directly related: I don't know a lot about Proteomics data, but why would it not be possible to run a functional method? In the end it produces also a matrix with expression values, isn't it? 
created_at: 2025-04-03T14:20:59Z
author_association: MEMBER
diff_hunk: @@ -135,6 +142,63 @@ workflow PIPELINE_COMPLETION {
 //
 def validateInputParameters() {
     genomeExistsError()
+
+    // Check file existence for input parameters
+    def checkPathParamList = [ params.input ]
+    for (param in checkPathParamList) {
+        if (param) {
+            file(param, checkIfExists: true)
+        }
+    }
+
+    // Check mandatory parameters
+    if (!params.input) {
+        error("Input samplesheet not specified!")
+    }
+
+    // Validate study type specific parameters
+    if (params.study_type == 'affy_array') {
path: subworkflows/local/utils_nfcore_differentialabundance_pipeline/main.nf
user: login: grst
body: Not directly related to this PR, but I haven't fully understood the scope of `study_type`. Is it only to specify which data input type to use? Or is it also to define which methods to run on which data? If the latter, wouldn't that be the point of the new toolsheet? 

I'm mostly asking because I think I'd have to choose "rnaseq" for running limma/dream on an input matrix, that doesn't necessarily need to be rnaseq/count data. 
created_at: 2025-04-03T14:24:22Z
author_association: MEMBER
diff_hunk: @@ -240,167 +304,123 @@ def methodsDescriptionText(mqc_methods_yaml) {
     return description_html.toString()
 }
 
-/**
-* Parse a string of arguments into a map.
-* @param argsStr The string of arguments to parse.
-* @return A map of parameters.
-* @example
-* parseStrArgsToMap("--arg1 aa --arg2 bb --arg4 cc") => [arg1: aa, arg2: bb, arg4: cc]
-*/
-def parseStrArgsToMap(String argsStr) {
-    if (!argsStr) return [:]                     // if null return empty
-    def tokens = argsStr.split().findAll { it }  // Split and remove empty strings
-    def pairs = tokens.collate(2)                // Group into pairs
-    return pairs.collectEntries {
-        [(it[0].replaceAll('^-+', '')): it[1]]   // Remove leading dashes
-    }
+// Get tool configurations based on whether analysis_name is provided
+def getToolConfigurations() {
+    // Use toolsheet if either analysis_name or custom toolsheet is provided, otherwise use default params
+    return (params.analysis_name || params.toolsheet_custom) ?
+        getToolsheetConfigurations() :
+        getDefaultConfigurations()
 }
 
-/**
-* Parse the parameters for a specific analysis type and method.
-* @param analysisType The analysis type. Eg. 'differential', 'functional'.
-* @param method The method to use for the analysis.
-* @param argsStr The string of arguments to parse.
-* @return A map of parameters.
-*/
-def parseParams(String analysisType, String method, String argsStr){
-    def parsed_params = [:]
-
-    // parse the params specific to the analysis type from the pipeline params
-    parsed_params += params.findAll { k, v -> k.matches(~/(${analysisType}).*/) }
-
-    // parse the params specific to the method
-    // First it gets the default values as specified by the pipeline params scope,
-    // and then overwrite with the values specified in the argsStr (highest priority),
-    // when provided
-    if (method) {
-        parsed_params += params.findAll { k, v -> k.matches(~/(${method}).*/) }
-        parsed_params["${analysisType}_method"] = method
+// Create a temporary schema file for toolsheet validation
+// This schema is derived from the pipeline's nextflow_schema.json by:
+// 1. Reading the toolsheet headers to determine which fields to include
+// 2. Extracting only those properties from the pipeline schema
+// 3. Creating a schema that allows for an array of objects
+// 4. Making analysis_name the only required field
+// 5. Adding meta fields to all properties
+// @return The absolute path to the temporary schema file
+def createToolsheetSchema() {
+    // Load toolsheet and get headers
+    def toolsheet_path = params.toolsheet_custom ?: "${projectDir}/assets/toolsheet.csv"
+    def toolsheet_lines = new File(toolsheet_path).readLines()
+    def headers = toolsheet_lines[0].split(',')
+
+    // Ensure analysis_name is in headers
+    if (!headers.contains('analysis_name')) {
+        error("The toolsheet must contain an 'analysis_name' column")
     }
-    if (argsStr) {
-        parsed_params += parseStrArgsToMap(argsStr)
+
+    // Load and parse pipeline schema
+    def pipeline_schema = new File("${projectDir}/nextflow_schema.json").text
+    def schema_json = new groovy.json.JsonSlurper().parseText(pipeline_schema)
+
+    // Extract only the properties that are present in the toolsheet
+    def all_properties = [:]
+    schema_json.$defs.each { group_name, group_def ->
+        if (group_def.properties) {
+            group_def.properties.each { prop_name, prop_def ->
+                // Only include properties that are in the toolsheet headers
+                if (headers.contains(prop_name)) {
+                    // Add meta field to each property
+                    def prop_with_meta = prop_def + [meta: [prop_name]]
+                    all_properties[prop_name] = prop_with_meta
+                }
+            }
+        }
     }
 
-    return parsed_params
-}
+    // Ensure analysis_name property exists in schema
+    if (!all_properties.containsKey('analysis_name')) {
+        // Find analysis_name definition in pipeline schema
+        def analysis_name_def = null
+        schema_json.$defs.each { group_name, group_def ->
+            if (group_def.properties && group_def.properties.containsKey('analysis_name')) {
+                analysis_name_def = group_def.properties['analysis_name']
+            }
+        }
 
-/**
-* Get tool configurations based on whether analysis_name is provided
-* @return A channel with tool configurations.
-*/
-def getToolConfigurations() {
-    if (params.analysis_name) {
-        return getToolsheetConfigurations()
-    } else {
-        return getDefaultConfigurations()
+        if (!analysis_name_def) {
+            error("Could not find analysis_name definition in pipeline schema")
+        }
+
+        // Add analysis_name to properties with meta field
+        all_properties['analysis_name'] = analysis_name_def + [meta: ['analysis_name']]
path: subworkflows/local/utils_nfcore_differentialabundance_pipeline/main.nf
user: login: grst
body: Why is this part necessary? I understood you already fail if the toolsheet header doesn't contain `analysis_name` and copy over all properties that are in the header from the pipeline_schema into `all_properties`. 

Couldn't you just fail if `analysis_name` is not in `all_properties`? 
created_at: 2025-04-03T14:33:29Z
author_association: MEMBER
diff_hunk: @@ -240,167 +304,123 @@ def methodsDescriptionText(mqc_methods_yaml) {
     return description_html.toString()
 }
 
-/**
-* Parse a string of arguments into a map.
-* @param argsStr The string of arguments to parse.
-* @return A map of parameters.
-* @example
-* parseStrArgsToMap("--arg1 aa --arg2 bb --arg4 cc") => [arg1: aa, arg2: bb, arg4: cc]
-*/
-def parseStrArgsToMap(String argsStr) {
-    if (!argsStr) return [:]                     // if null return empty
-    def tokens = argsStr.split().findAll { it }  // Split and remove empty strings
-    def pairs = tokens.collate(2)                // Group into pairs
-    return pairs.collectEntries {
-        [(it[0].replaceAll('^-+', '')): it[1]]   // Remove leading dashes
-    }
+// Get tool configurations based on whether analysis_name is provided
+def getToolConfigurations() {
+    // Use toolsheet if either analysis_name or custom toolsheet is provided, otherwise use default params
+    return (params.analysis_name || params.toolsheet_custom) ?
+        getToolsheetConfigurations() :
+        getDefaultConfigurations()
 }
 
-/**
-* Parse the parameters for a specific analysis type and method.
-* @param analysisType The analysis type. Eg. 'differential', 'functional'.
-* @param method The method to use for the analysis.
-* @param argsStr The string of arguments to parse.
-* @return A map of parameters.
-*/
-def parseParams(String analysisType, String method, String argsStr){
-    def parsed_params = [:]
-
-    // parse the params specific to the analysis type from the pipeline params
-    parsed_params += params.findAll { k, v -> k.matches(~/(${analysisType}).*/) }
-
-    // parse the params specific to the method
-    // First it gets the default values as specified by the pipeline params scope,
-    // and then overwrite with the values specified in the argsStr (highest priority),
-    // when provided
-    if (method) {
-        parsed_params += params.findAll { k, v -> k.matches(~/(${method}).*/) }
-        parsed_params["${analysisType}_method"] = method
+// Create a temporary schema file for toolsheet validation
+// This schema is derived from the pipeline's nextflow_schema.json by:
+// 1. Reading the toolsheet headers to determine which fields to include
+// 2. Extracting only those properties from the pipeline schema
+// 3. Creating a schema that allows for an array of objects
+// 4. Making analysis_name the only required field
+// 5. Adding meta fields to all properties
+// @return The absolute path to the temporary schema file
+def createToolsheetSchema() {
+    // Load toolsheet and get headers
+    def toolsheet_path = params.toolsheet_custom ?: "${projectDir}/assets/toolsheet.csv"
+    def toolsheet_lines = new File(toolsheet_path).readLines()
+    def headers = toolsheet_lines[0].split(',')
+
+    // Ensure analysis_name is in headers
+    if (!headers.contains('analysis_name')) {
+        error("The toolsheet must contain an 'analysis_name' column")
     }
-    if (argsStr) {
-        parsed_params += parseStrArgsToMap(argsStr)
+
+    // Load and parse pipeline schema
+    def pipeline_schema = new File("${projectDir}/nextflow_schema.json").text
+    def schema_json = new groovy.json.JsonSlurper().parseText(pipeline_schema)
+
+    // Extract only the properties that are present in the toolsheet
+    def all_properties = [:]
+    schema_json.$defs.each { group_name, group_def ->
+        if (group_def.properties) {
+            group_def.properties.each { prop_name, prop_def ->
+                // Only include properties that are in the toolsheet headers
+                if (headers.contains(prop_name)) {
+                    // Add meta field to each property
+                    def prop_with_meta = prop_def + [meta: [prop_name]]
+                    all_properties[prop_name] = prop_with_meta
+                }
+            }
+        }
     }
 
-    return parsed_params
-}
+    // Ensure analysis_name property exists in schema
+    if (!all_properties.containsKey('analysis_name')) {
+        // Find analysis_name definition in pipeline schema
+        def analysis_name_def = null
+        schema_json.$defs.each { group_name, group_def ->
+            if (group_def.properties && group_def.properties.containsKey('analysis_name')) {
+                analysis_name_def = group_def.properties['analysis_name']
+            }
+        }
 
-/**
-* Get tool configurations based on whether analysis_name is provided
-* @return A channel with tool configurations.
-*/
-def getToolConfigurations() {
-    if (params.analysis_name) {
-        return getToolsheetConfigurations()
-    } else {
-        return getDefaultConfigurations()
+        if (!analysis_name_def) {
+            error("Could not find analysis_name definition in pipeline schema")
+        }
+
+        // Add analysis_name to properties with meta field
+        all_properties['analysis_name'] = analysis_name_def + [meta: ['analysis_name']]
     }
-}
 
-/**
-* Get configurations from toolsheet
-* @return A list of tool configurations by parsing toolsheet.
-*/
-def getToolsheetConfigurations() {
-    // Load appropriate toolsheet based on study type
-    def toolsheet_path = getToolsheetPath()
-
-    // Load and filter toolsheet for specific analysis
-    def toolsheet = samplesheetToList(toolsheet_path, "${projectDir}/assets/schema_tools.json")
-        .find { it[0].analysis_name == params.analysis_name }
-        ?: error("Analysis '${params.analysis_name}' not found in toolsheet")
-
-    // Construct tool configurations
-    def tool_config = toolsheet[0]
-    return [
-        getNormalizationConfig(tool_config),
-        getDifferentialConfig(tool_config),
-        getFunctionalConfig(tool_config)
+    // Create samplesheet schema with filtered properties and analysis_name as required
+    def samplesheet_schema = [
+        '$schema': 'https://json-schema.org/draft/2020-12/schema',
+        'title': 'nf-core/differentialabundance - toolsheet schema',
+        'description': 'Schema for validating the toolsheet configuration',
+        'type': 'array',
+        'items': [
+            'type': 'object',
+            'properties': all_properties,
+            'required': ['analysis_name']
+        ]
     ]
-}
 
-/**
-* Get default configurations from pipeline parameters
-* @return A list of default tool configurations.
-*/
-def getDefaultConfigurations() {
-    return [
-        getNormalizationConfig(null),
-        getDifferentialConfig(null),
-        getFunctionalConfig(null)
-    ]
-}
+    // Write temporary schema file
+    def temp_schema = File.createTempFile("samplesheet_schema", ".json")
+    def schema_string = new groovy.json.JsonBuilder(samplesheet_schema).toPrettyString()
+    temp_schema.text = schema_string
 
-/**
-* Get the appropriate toolsheet path.
-* @return The path to the toolsheet.
-*/
-def getToolsheetPath() {
-    // use user-provided toolsheet if available
-    if (params.toolsheet) {
-        return params.toolsheet
-    }
-    // use default toolsheet based on study type
-    switch (params.study_type) {
-        case 'rnaseq':
-            return "${projectDir}/assets/toolsheet_rnaseq.csv"
-        case ['affy_array', 'geo_soft_file']:
-            return "${projectDir}/assets/toolsheet_affy.csv"
-        case 'maxquant':
-            return "${projectDir}/assets/toolsheet_maxquant.csv"
-        default:
-            error("Invalid study_type. Must be one of: 'rnaseq', 'affy_array', 'geo_soft_file', 'maxquant'")
-    }
+    return temp_schema.absolutePath
 }
 
-/**
-* Get normalization tool configuration
-* @param tool_config The tool configuration.
-* @return A map of normalization configuration.
-*/
-def getNormalizationConfig(tool_config) {
-    // non-rnaseq studies use directly the normalized matrix output from the VALIDATOR module
-    if (params.study_type != 'rnaseq') {
-        return [method: 'validator', args: [:]]
+// Get configurations from toolsheet
+def getToolsheetConfigurations() {
+    // Load toolsheet
+    def toolsheet_path = params.toolsheet_custom ?: "${projectDir}/assets/toolsheet.csv"
+
+    // Create temporary schema for validation
+    def schema_path = createToolsheetSchema()
+
+    // Load toolsheet and validate each row against the transformed schema
+    def raw_toolsheet = samplesheetToList(toolsheet_path, schema_path)
+
+    def toolsheet = raw_toolsheet
+        .findAll { row ->
+            // If analysis_name is set, only keep matching rows
+            if (params.analysis_name) {
+                return row[0].analysis_name == params.analysis_name
+            }
+            return true
+        }
+
+    if (toolsheet.isEmpty()) {
+        if (params.analysis_name) {
+            error("No configuration found in toolsheet for analysis_name '${params.analysis_name}'")
+        } else {
+            error("No valid configurations found in toolsheet")
+        }
     }
-    // for rna-seq studies, use the differential method specified in the toolsheet
-    // or the default differential method specified in the pipeline parameters
-    def method = tool_config?.diff_method ?: params.differential_method
-    def args = parseParams('differential', method, tool_config?.diff_args ?: null)
-    return [method: method, args: args]
-}
 
-/**
-* Get differential analysis tool configuration
-* @param tool_config The tool configuration.
-* @return A map of differential analysis configuration.
-*/
-def getDifferentialConfig(tool_config) {
-    def method = tool_config?.diff_method ?: params.differential_method
-    def args = parseParams('differential', method, tool_config?.diff_args ?: null)
-    def fc_threshold = args.differential_min_fold_change
-    def stat_threshold = args.differential_max_qval
-    return [
-        method: method,
-        args: args,
-        fc_threshold: fc_threshold,
-        stat_threshold: stat_threshold
-    ]
+    return toolsheet.collect{ row -> (params.findAll { k, v -> k != 'trace_report_suffix' } + row[0]) }
path: subworkflows/local/utils_nfcore_differentialabundance_pipeline/main.nf
user: login: grst
body: I don't follow what this does, consider adding a comment. 
created_at: 2025-04-03T14:39:56Z
author_association: MEMBER
diff_hunk: @@ -135,6 +142,63 @@ workflow PIPELINE_COMPLETION {
 //
 def validateInputParameters() {
     genomeExistsError()
+
+    // Check file existence for input parameters
+    def checkPathParamList = [ params.input ]
+    for (param in checkPathParamList) {
+        if (param) {
+            file(param, checkIfExists: true)
+        }
+    }
+
+    // Check mandatory parameters
+    if (!params.input) {
+        error("Input samplesheet not specified!")
+    }
+
+    // Validate study type specific parameters
+    if (params.study_type == 'affy_array') {
path: subworkflows/local/utils_nfcore_differentialabundance_pipeline/main.nf
user: login: pinin4fjords
body: That's a very good point. We could reduce the usage of `study_type` in some contexts, where a parameter to drive a particular decision point might be more appropriate.
created_at: 2025-04-04T08:30:55Z
author_association: MEMBER
diff_hunk: @@ -240,167 +304,123 @@ def methodsDescriptionText(mqc_methods_yaml) {
     return description_html.toString()
 }
 
-/**
-* Parse a string of arguments into a map.
-* @param argsStr The string of arguments to parse.
-* @return A map of parameters.
-* @example
-* parseStrArgsToMap("--arg1 aa --arg2 bb --arg4 cc") => [arg1: aa, arg2: bb, arg4: cc]
-*/
-def parseStrArgsToMap(String argsStr) {
-    if (!argsStr) return [:]                     // if null return empty
-    def tokens = argsStr.split().findAll { it }  // Split and remove empty strings
-    def pairs = tokens.collate(2)                // Group into pairs
-    return pairs.collectEntries {
-        [(it[0].replaceAll('^-+', '')): it[1]]   // Remove leading dashes
-    }
+// Get tool configurations based on whether analysis_name is provided
+def getToolConfigurations() {
+    // Use toolsheet if either analysis_name or custom toolsheet is provided, otherwise use default params
+    return (params.analysis_name || params.toolsheet_custom) ?
+        getToolsheetConfigurations() :
+        getDefaultConfigurations()
 }
 
-/**
-* Parse the parameters for a specific analysis type and method.
-* @param analysisType The analysis type. Eg. 'differential', 'functional'.
-* @param method The method to use for the analysis.
-* @param argsStr The string of arguments to parse.
-* @return A map of parameters.
-*/
-def parseParams(String analysisType, String method, String argsStr){
-    def parsed_params = [:]
-
-    // parse the params specific to the analysis type from the pipeline params
-    parsed_params += params.findAll { k, v -> k.matches(~/(${analysisType}).*/) }
-
-    // parse the params specific to the method
-    // First it gets the default values as specified by the pipeline params scope,
-    // and then overwrite with the values specified in the argsStr (highest priority),
-    // when provided
-    if (method) {
-        parsed_params += params.findAll { k, v -> k.matches(~/(${method}).*/) }
-        parsed_params["${analysisType}_method"] = method
+// Create a temporary schema file for toolsheet validation
+// This schema is derived from the pipeline's nextflow_schema.json by:
+// 1. Reading the toolsheet headers to determine which fields to include
+// 2. Extracting only those properties from the pipeline schema
+// 3. Creating a schema that allows for an array of objects
+// 4. Making analysis_name the only required field
+// 5. Adding meta fields to all properties
+// @return The absolute path to the temporary schema file
+def createToolsheetSchema() {
+    // Load toolsheet and get headers
+    def toolsheet_path = params.toolsheet_custom ?: "${projectDir}/assets/toolsheet.csv"
+    def toolsheet_lines = new File(toolsheet_path).readLines()
+    def headers = toolsheet_lines[0].split(',')
+
+    // Ensure analysis_name is in headers
+    if (!headers.contains('analysis_name')) {
+        error("The toolsheet must contain an 'analysis_name' column")
     }
-    if (argsStr) {
-        parsed_params += parseStrArgsToMap(argsStr)
+
+    // Load and parse pipeline schema
+    def pipeline_schema = new File("${projectDir}/nextflow_schema.json").text
+    def schema_json = new groovy.json.JsonSlurper().parseText(pipeline_schema)
+
+    // Extract only the properties that are present in the toolsheet
+    def all_properties = [:]
+    schema_json.$defs.each { group_name, group_def ->
+        if (group_def.properties) {
+            group_def.properties.each { prop_name, prop_def ->
+                // Only include properties that are in the toolsheet headers
+                if (headers.contains(prop_name)) {
+                    // Add meta field to each property
+                    def prop_with_meta = prop_def + [meta: [prop_name]]
+                    all_properties[prop_name] = prop_with_meta
+                }
+            }
+        }
     }
 
-    return parsed_params
-}
+    // Ensure analysis_name property exists in schema
+    if (!all_properties.containsKey('analysis_name')) {
+        // Find analysis_name definition in pipeline schema
+        def analysis_name_def = null
+        schema_json.$defs.each { group_name, group_def ->
+            if (group_def.properties && group_def.properties.containsKey('analysis_name')) {
+                analysis_name_def = group_def.properties['analysis_name']
+            }
+        }
 
-/**
-* Get tool configurations based on whether analysis_name is provided
-* @return A channel with tool configurations.
-*/
-def getToolConfigurations() {
-    if (params.analysis_name) {
-        return getToolsheetConfigurations()
-    } else {
-        return getDefaultConfigurations()
+        if (!analysis_name_def) {
+            error("Could not find analysis_name definition in pipeline schema")
+        }
+
+        // Add analysis_name to properties with meta field
+        all_properties['analysis_name'] = analysis_name_def + [meta: ['analysis_name']]
path: subworkflows/local/utils_nfcore_differentialabundance_pipeline/main.nf
user: login: pinin4fjords
body: We can definitely look at tidying up this function- consider this a first go to demonstrate the principle.
created_at: 2025-04-04T08:35:48Z
author_association: MEMBER
diff_hunk: @@ -240,167 +304,123 @@ def methodsDescriptionText(mqc_methods_yaml) {
     return description_html.toString()
 }
 
-/**
-* Parse a string of arguments into a map.
-* @param argsStr The string of arguments to parse.
-* @return A map of parameters.
-* @example
-* parseStrArgsToMap("--arg1 aa --arg2 bb --arg4 cc") => [arg1: aa, arg2: bb, arg4: cc]
-*/
-def parseStrArgsToMap(String argsStr) {
-    if (!argsStr) return [:]                     // if null return empty
-    def tokens = argsStr.split().findAll { it }  // Split and remove empty strings
-    def pairs = tokens.collate(2)                // Group into pairs
-    return pairs.collectEntries {
-        [(it[0].replaceAll('^-+', '')): it[1]]   // Remove leading dashes
-    }
+// Get tool configurations based on whether analysis_name is provided
+def getToolConfigurations() {
+    // Use toolsheet if either analysis_name or custom toolsheet is provided, otherwise use default params
+    return (params.analysis_name || params.toolsheet_custom) ?
+        getToolsheetConfigurations() :
+        getDefaultConfigurations()
 }
 
-/**
-* Parse the parameters for a specific analysis type and method.
-* @param analysisType The analysis type. Eg. 'differential', 'functional'.
-* @param method The method to use for the analysis.
-* @param argsStr The string of arguments to parse.
-* @return A map of parameters.
-*/
-def parseParams(String analysisType, String method, String argsStr){
-    def parsed_params = [:]
-
-    // parse the params specific to the analysis type from the pipeline params
-    parsed_params += params.findAll { k, v -> k.matches(~/(${analysisType}).*/) }
-
-    // parse the params specific to the method
-    // First it gets the default values as specified by the pipeline params scope,
-    // and then overwrite with the values specified in the argsStr (highest priority),
-    // when provided
-    if (method) {
-        parsed_params += params.findAll { k, v -> k.matches(~/(${method}).*/) }
-        parsed_params["${analysisType}_method"] = method
+// Create a temporary schema file for toolsheet validation
+// This schema is derived from the pipeline's nextflow_schema.json by:
+// 1. Reading the toolsheet headers to determine which fields to include
+// 2. Extracting only those properties from the pipeline schema
+// 3. Creating a schema that allows for an array of objects
+// 4. Making analysis_name the only required field
+// 5. Adding meta fields to all properties
+// @return The absolute path to the temporary schema file
+def createToolsheetSchema() {
+    // Load toolsheet and get headers
+    def toolsheet_path = params.toolsheet_custom ?: "${projectDir}/assets/toolsheet.csv"
+    def toolsheet_lines = new File(toolsheet_path).readLines()
+    def headers = toolsheet_lines[0].split(',')
+
+    // Ensure analysis_name is in headers
+    if (!headers.contains('analysis_name')) {
+        error("The toolsheet must contain an 'analysis_name' column")
     }
-    if (argsStr) {
-        parsed_params += parseStrArgsToMap(argsStr)
+
+    // Load and parse pipeline schema
+    def pipeline_schema = new File("${projectDir}/nextflow_schema.json").text
+    def schema_json = new groovy.json.JsonSlurper().parseText(pipeline_schema)
+
+    // Extract only the properties that are present in the toolsheet
+    def all_properties = [:]
+    schema_json.$defs.each { group_name, group_def ->
+        if (group_def.properties) {
+            group_def.properties.each { prop_name, prop_def ->
+                // Only include properties that are in the toolsheet headers
+                if (headers.contains(prop_name)) {
+                    // Add meta field to each property
+                    def prop_with_meta = prop_def + [meta: [prop_name]]
+                    all_properties[prop_name] = prop_with_meta
+                }
+            }
+        }
     }
 
-    return parsed_params
-}
+    // Ensure analysis_name property exists in schema
+    if (!all_properties.containsKey('analysis_name')) {
+        // Find analysis_name definition in pipeline schema
+        def analysis_name_def = null
+        schema_json.$defs.each { group_name, group_def ->
+            if (group_def.properties && group_def.properties.containsKey('analysis_name')) {
+                analysis_name_def = group_def.properties['analysis_name']
+            }
+        }
 
-/**
-* Get tool configurations based on whether analysis_name is provided
-* @return A channel with tool configurations.
-*/
-def getToolConfigurations() {
-    if (params.analysis_name) {
-        return getToolsheetConfigurations()
-    } else {
-        return getDefaultConfigurations()
+        if (!analysis_name_def) {
+            error("Could not find analysis_name definition in pipeline schema")
+        }
+
+        // Add analysis_name to properties with meta field
+        all_properties['analysis_name'] = analysis_name_def + [meta: ['analysis_name']]
     }
-}
 
-/**
-* Get configurations from toolsheet
-* @return A list of tool configurations by parsing toolsheet.
-*/
-def getToolsheetConfigurations() {
-    // Load appropriate toolsheet based on study type
-    def toolsheet_path = getToolsheetPath()
-
-    // Load and filter toolsheet for specific analysis
-    def toolsheet = samplesheetToList(toolsheet_path, "${projectDir}/assets/schema_tools.json")
-        .find { it[0].analysis_name == params.analysis_name }
-        ?: error("Analysis '${params.analysis_name}' not found in toolsheet")
-
-    // Construct tool configurations
-    def tool_config = toolsheet[0]
-    return [
-        getNormalizationConfig(tool_config),
-        getDifferentialConfig(tool_config),
-        getFunctionalConfig(tool_config)
+    // Create samplesheet schema with filtered properties and analysis_name as required
+    def samplesheet_schema = [
+        '$schema': 'https://json-schema.org/draft/2020-12/schema',
+        'title': 'nf-core/differentialabundance - toolsheet schema',
+        'description': 'Schema for validating the toolsheet configuration',
+        'type': 'array',
+        'items': [
+            'type': 'object',
+            'properties': all_properties,
+            'required': ['analysis_name']
+        ]
     ]
-}
 
-/**
-* Get default configurations from pipeline parameters
-* @return A list of default tool configurations.
-*/
-def getDefaultConfigurations() {
-    return [
-        getNormalizationConfig(null),
-        getDifferentialConfig(null),
-        getFunctionalConfig(null)
-    ]
-}
+    // Write temporary schema file
+    def temp_schema = File.createTempFile("samplesheet_schema", ".json")
+    def schema_string = new groovy.json.JsonBuilder(samplesheet_schema).toPrettyString()
+    temp_schema.text = schema_string
 
-/**
-* Get the appropriate toolsheet path.
-* @return The path to the toolsheet.
-*/
-def getToolsheetPath() {
-    // use user-provided toolsheet if available
-    if (params.toolsheet) {
-        return params.toolsheet
-    }
-    // use default toolsheet based on study type
-    switch (params.study_type) {
-        case 'rnaseq':
-            return "${projectDir}/assets/toolsheet_rnaseq.csv"
-        case ['affy_array', 'geo_soft_file']:
-            return "${projectDir}/assets/toolsheet_affy.csv"
-        case 'maxquant':
-            return "${projectDir}/assets/toolsheet_maxquant.csv"
-        default:
-            error("Invalid study_type. Must be one of: 'rnaseq', 'affy_array', 'geo_soft_file', 'maxquant'")
-    }
+    return temp_schema.absolutePath
 }
 
-/**
-* Get normalization tool configuration
-* @param tool_config The tool configuration.
-* @return A map of normalization configuration.
-*/
-def getNormalizationConfig(tool_config) {
-    // non-rnaseq studies use directly the normalized matrix output from the VALIDATOR module
-    if (params.study_type != 'rnaseq') {
-        return [method: 'validator', args: [:]]
+// Get configurations from toolsheet
+def getToolsheetConfigurations() {
+    // Load toolsheet
+    def toolsheet_path = params.toolsheet_custom ?: "${projectDir}/assets/toolsheet.csv"
+
+    // Create temporary schema for validation
+    def schema_path = createToolsheetSchema()
+
+    // Load toolsheet and validate each row against the transformed schema
+    def raw_toolsheet = samplesheetToList(toolsheet_path, schema_path)
+
+    def toolsheet = raw_toolsheet
+        .findAll { row ->
+            // If analysis_name is set, only keep matching rows
+            if (params.analysis_name) {
+                return row[0].analysis_name == params.analysis_name
+            }
+            return true
+        }
+
+    if (toolsheet.isEmpty()) {
+        if (params.analysis_name) {
+            error("No configuration found in toolsheet for analysis_name '${params.analysis_name}'")
+        } else {
+            error("No valid configurations found in toolsheet")
+        }
     }
-    // for rna-seq studies, use the differential method specified in the toolsheet
-    // or the default differential method specified in the pipeline parameters
-    def method = tool_config?.diff_method ?: params.differential_method
-    def args = parseParams('differential', method, tool_config?.diff_args ?: null)
-    return [method: method, args: args]
-}
 
-/**
-* Get differential analysis tool configuration
-* @param tool_config The tool configuration.
-* @return A map of differential analysis configuration.
-*/
-def getDifferentialConfig(tool_config) {
-    def method = tool_config?.diff_method ?: params.differential_method
-    def args = parseParams('differential', method, tool_config?.diff_args ?: null)
-    def fc_threshold = args.differential_min_fold_change
-    def stat_threshold = args.differential_max_qval
-    return [
-        method: method,
-        args: args,
-        fc_threshold: fc_threshold,
-        stat_threshold: stat_threshold
-    ]
+    return toolsheet.collect{ row -> (params.findAll { k, v -> k != 'trace_report_suffix' } + row[0]) }
path: subworkflows/local/utils_nfcore_differentialabundance_pipeline/main.nf
user: login: pinin4fjords
body: Think of `collect` like a `map` in Nextflow. `trace_report_suffix` is generated at runtime and it was invalidating the caches every time, so it's import to not include it. Will add comment.
created_at: 2025-04-04T08:36:52Z
author_association: MEMBER
diff_hunk: @@ -135,6 +142,63 @@ workflow PIPELINE_COMPLETION {
 //
 def validateInputParameters() {
     genomeExistsError()
+
+    // Check file existence for input parameters
+    def checkPathParamList = [ params.input ]
+    for (param in checkPathParamList) {
+        if (param) {
+            file(param, checkIfExists: true)
+        }
+    }
+
+    // Check mandatory parameters
+    if (!params.input) {
+        error("Input samplesheet not specified!")
+    }
+
+    // Validate study type specific parameters
+    if (params.study_type == 'affy_array') {
+        if (!params.affy_cel_files_archive) {
+            error("CEL files archive not specified!")
+        }
+    } else if (params.study_type == 'maxquant') {
+        if (params.functional_method) {
path: subworkflows/local/utils_nfcore_differentialabundance_pipeline/main.nf
user: login: pinin4fjords
body: @WackerO established this logic. Proteomics data is a bit weird in its sparsity etc, maybe the assumptions associated with GSEA etc don't hold.
created_at: 2025-04-04T08:38:29Z
author_association: MEMBER
diff_hunk: @@ -329,29 +247,123 @@ def methodsDescriptionText(mqc_methods_yaml) {
     return description_html.toString()
 }
 
-/**
-* Parse a string of arguments into a map.
-* @param argsStr The string of arguments to parse.
-* @return A map of parameters.
-* @example
-* parseArgs("--arg1 aa --arg2 bb --arg4 cc") => [arg1: aa, arg2: bb, arg4: cc]
-*/
-def parseArgs(String argsStr) {
-    if (!argsStr) return [:]                     // if null return empty
-    def tokens = argsStr.split().findAll { it }  // Split and remove empty strings
-    def pairs = tokens.collate(2)                // Group into pairs
-    return pairs.collectEntries {
-        [(it[0].replaceAll('^-+', '')): it[1]]   // Remove leading dashes
+// Get tool configurations based on whether analysis_name is provided
+def getToolConfigurations() {
+    // Use toolsheet if either analysis_name or custom toolsheet is provided, otherwise use default params
+    return (params.analysis_name || params.toolsheet_custom) ?
+        getToolsheetConfigurations() :
+        getDefaultConfigurations()
+}
+
+// Create a temporary schema file for toolsheet validation
+// This schema is derived from the pipeline's nextflow_schema.json by:
+// 1. Reading the toolsheet headers to determine which fields to include
+// 2. Extracting only those properties from the pipeline schema
+// 3. Creating a schema that allows for an array of objects
+// 4. Making analysis_name the only required field
+// 5. Adding meta fields to all properties
+// @return The absolute path to the temporary schema file
+def createToolsheetSchema() {
+    // Load toolsheet and get headers
+    def toolsheet_path = params.toolsheet_custom ?: "${projectDir}/assets/toolsheet.csv"
+    def toolsheet_lines = new File(toolsheet_path).readLines()
+    def headers = toolsheet_lines[0].split(',')
+
+    // Ensure analysis_name is in headers
+    if (!headers.contains('analysis_name')) {
+        error("The toolsheet must contain an 'analysis_name' column")
     }
+
+    // Load and parse pipeline schema
+    def pipeline_schema = new File("${projectDir}/nextflow_schema.json").text
+    def schema_json = new groovy.json.JsonSlurper().parseText(pipeline_schema)
+
+    // Extract only the properties that are present in the toolsheet
+    def all_properties = [:]
+    schema_json.$defs.each { group_name, group_def ->
+        if (group_def.properties) {
+            group_def.properties.each { prop_name, prop_def ->
+                // Only include properties that are in the toolsheet headers
+                if (headers.contains(prop_name)) {
+                    // Add meta field to each property
+                    def prop_with_meta = prop_def + [meta: [prop_name]]
+                    all_properties[prop_name] = prop_with_meta
+                }
+            }
+        }
+    }
+
+    // Ensure analysis_name property exists in schema
+    if (!all_properties.containsKey('analysis_name')) {
+        // Find analysis_name definition in pipeline schema
+        def analysis_name_def = null
+        schema_json.$defs.each { group_name, group_def ->
+            if (group_def.properties && group_def.properties.containsKey('analysis_name')) {
+                analysis_name_def = group_def.properties['analysis_name']
+            }
+        }
+
+        if (!analysis_name_def) {
+            error("Could not find analysis_name definition in pipeline schema")
+        }
+
+        // Add analysis_name to properties with meta field
+        all_properties['analysis_name'] = analysis_name_def + [meta: ['analysis_name']]
+    }
+
+    // Create samplesheet schema with filtered properties and analysis_name as required
+    def samplesheet_schema = [
+        '$schema': 'https://json-schema.org/draft/2020-12/schema',
+        'title': 'nf-core/differentialabundance - toolsheet schema',
+        'description': 'Schema for validating the toolsheet configuration',
+        'type': 'array',
+        'items': [
+            'type': 'object',
+            'properties': all_properties,
+            'required': ['analysis_name']
+        ]
+    ]
+
+    // Write temporary schema file
+    def temp_schema = File.createTempFile("samplesheet_schema", ".json")
+    def schema_string = new groovy.json.JsonBuilder(samplesheet_schema).toPrettyString()
+    temp_schema.text = schema_string
+
+    return temp_schema.absolutePath
 }
 
-/**
-* Get params from the pipeline params scope.
-* @param pattern The pattern to match.
-* @return A map of params.
-*/
-def getParams(String basePattern, String method) {
-    if (!method) return [:]
-    pattern = "$basePattern|$method"
-    return params.findAll { k, v -> k.matches(~/(${pattern}).*/) }
+// Get configurations from toolsheet
+def getToolsheetConfigurations() {
+    // Load toolsheet
+    def toolsheet_path = params.toolsheet_custom ?: "${projectDir}/assets/toolsheet.csv"
+
+    // Create temporary schema for validation
+    def schema_path = createToolsheetSchema()
+
+    // Load toolsheet and validate each row against the transformed schema
+    def raw_toolsheet = samplesheetToList(toolsheet_path, schema_path)
+
+    def toolsheet = raw_toolsheet
+        .findAll { row ->
+            // If analysis_name is set, only keep matching rows
+            if (params.analysis_name) {
+                return row[0].analysis_name == params.analysis_name
+            }
+            return true
+        }
+
+    if (toolsheet.isEmpty()) {
+        if (params.analysis_name) {
+            error("No configuration found in toolsheet for analysis_name '${params.analysis_name}'")
+        } else {
+            error("No valid configurations found in toolsheet")
+        }
+    }
+
+    return toolsheet.map{it + params}
path: subworkflows/local/utils_nfcore_differentialabundance_pipeline/main.nf
user: login: suzannejin
body: maybe params + it insted, so that toolsheet params stay in the highest priority?
created_at: 2025-04-04T10:15:55Z
author_association: NONE
diff_hunk: @@ -329,29 +247,123 @@ def methodsDescriptionText(mqc_methods_yaml) {
     return description_html.toString()
 }
 
-/**
-* Parse a string of arguments into a map.
-* @param argsStr The string of arguments to parse.
-* @return A map of parameters.
-* @example
-* parseArgs("--arg1 aa --arg2 bb --arg4 cc") => [arg1: aa, arg2: bb, arg4: cc]
-*/
-def parseArgs(String argsStr) {
-    if (!argsStr) return [:]                     // if null return empty
-    def tokens = argsStr.split().findAll { it }  // Split and remove empty strings
-    def pairs = tokens.collate(2)                // Group into pairs
-    return pairs.collectEntries {
-        [(it[0].replaceAll('^-+', '')): it[1]]   // Remove leading dashes
+// Get tool configurations based on whether analysis_name is provided
+def getToolConfigurations() {
+    // Use toolsheet if either analysis_name or custom toolsheet is provided, otherwise use default params
+    return (params.analysis_name || params.toolsheet_custom) ?
+        getToolsheetConfigurations() :
+        getDefaultConfigurations()
+}
+
+// Create a temporary schema file for toolsheet validation
+// This schema is derived from the pipeline's nextflow_schema.json by:
+// 1. Reading the toolsheet headers to determine which fields to include
+// 2. Extracting only those properties from the pipeline schema
+// 3. Creating a schema that allows for an array of objects
+// 4. Making analysis_name the only required field
+// 5. Adding meta fields to all properties
+// @return The absolute path to the temporary schema file
+def createToolsheetSchema() {
+    // Load toolsheet and get headers
+    def toolsheet_path = params.toolsheet_custom ?: "${projectDir}/assets/toolsheet.csv"
+    def toolsheet_lines = new File(toolsheet_path).readLines()
+    def headers = toolsheet_lines[0].split(',')
+
+    // Ensure analysis_name is in headers
+    if (!headers.contains('analysis_name')) {
+        error("The toolsheet must contain an 'analysis_name' column")
     }
+
+    // Load and parse pipeline schema
+    def pipeline_schema = new File("${projectDir}/nextflow_schema.json").text
+    def schema_json = new groovy.json.JsonSlurper().parseText(pipeline_schema)
+
+    // Extract only the properties that are present in the toolsheet
+    def all_properties = [:]
+    schema_json.$defs.each { group_name, group_def ->
+        if (group_def.properties) {
+            group_def.properties.each { prop_name, prop_def ->
+                // Only include properties that are in the toolsheet headers
+                if (headers.contains(prop_name)) {
+                    // Add meta field to each property
+                    def prop_with_meta = prop_def + [meta: [prop_name]]
+                    all_properties[prop_name] = prop_with_meta
+                }
+            }
+        }
+    }
+
+    // Ensure analysis_name property exists in schema
+    if (!all_properties.containsKey('analysis_name')) {
+        // Find analysis_name definition in pipeline schema
+        def analysis_name_def = null
+        schema_json.$defs.each { group_name, group_def ->
+            if (group_def.properties && group_def.properties.containsKey('analysis_name')) {
+                analysis_name_def = group_def.properties['analysis_name']
+            }
+        }
+
+        if (!analysis_name_def) {
+            error("Could not find analysis_name definition in pipeline schema")
+        }
+
+        // Add analysis_name to properties with meta field
+        all_properties['analysis_name'] = analysis_name_def + [meta: ['analysis_name']]
+    }
+
+    // Create samplesheet schema with filtered properties and analysis_name as required
+    def samplesheet_schema = [
+        '$schema': 'https://json-schema.org/draft/2020-12/schema',
+        'title': 'nf-core/differentialabundance - toolsheet schema',
+        'description': 'Schema for validating the toolsheet configuration',
+        'type': 'array',
+        'items': [
+            'type': 'object',
+            'properties': all_properties,
+            'required': ['analysis_name']
+        ]
+    ]
+
+    // Write temporary schema file
+    def temp_schema = File.createTempFile("samplesheet_schema", ".json")
+    def schema_string = new groovy.json.JsonBuilder(samplesheet_schema).toPrettyString()
+    temp_schema.text = schema_string
+
+    return temp_schema.absolutePath
 }
 
-/**
-* Get params from the pipeline params scope.
-* @param pattern The pattern to match.
-* @return A map of params.
-*/
-def getParams(String basePattern, String method) {
-    if (!method) return [:]
-    pattern = "$basePattern|$method"
-    return params.findAll { k, v -> k.matches(~/(${pattern}).*/) }
+// Get configurations from toolsheet
+def getToolsheetConfigurations() {
+    // Load toolsheet
+    def toolsheet_path = params.toolsheet_custom ?: "${projectDir}/assets/toolsheet.csv"
+
+    // Create temporary schema for validation
+    def schema_path = createToolsheetSchema()
+
+    // Load toolsheet and validate each row against the transformed schema
+    def raw_toolsheet = samplesheetToList(toolsheet_path, schema_path)
+
+    def toolsheet = raw_toolsheet
+        .findAll { row ->
+            // If analysis_name is set, only keep matching rows
+            if (params.analysis_name) {
+                return row[0].analysis_name == params.analysis_name
+            }
+            return true
+        }
+
+    if (toolsheet.isEmpty()) {
+        if (params.analysis_name) {
+            error("No configuration found in toolsheet for analysis_name '${params.analysis_name}'")
+        } else {
+            error("No valid configurations found in toolsheet")
+        }
+    }
+
+    return toolsheet.map{it + params}
path: subworkflows/local/utils_nfcore_differentialabundance_pipeline/main.nf
user: login: pinin4fjords
body: Yep, I spotted that too- corrected now!

created_at: 2025-04-04T10:19:58Z
author_association: MEMBER
diff_hunk: @@ -240,167 +304,123 @@ def methodsDescriptionText(mqc_methods_yaml) {
     return description_html.toString()
 }
 
-/**
-* Parse a string of arguments into a map.
-* @param argsStr The string of arguments to parse.
-* @return A map of parameters.
-* @example
-* parseStrArgsToMap("--arg1 aa --arg2 bb --arg4 cc") => [arg1: aa, arg2: bb, arg4: cc]
-*/
-def parseStrArgsToMap(String argsStr) {
-    if (!argsStr) return [:]                     // if null return empty
-    def tokens = argsStr.split().findAll { it }  // Split and remove empty strings
-    def pairs = tokens.collate(2)                // Group into pairs
-    return pairs.collectEntries {
-        [(it[0].replaceAll('^-+', '')): it[1]]   // Remove leading dashes
-    }
+// Get tool configurations based on whether analysis_name is provided
+def getToolConfigurations() {
+    // Use toolsheet if either analysis_name or custom toolsheet is provided, otherwise use default params
+    return (params.analysis_name || params.toolsheet_custom) ?
+        getToolsheetConfigurations() :
+        getDefaultConfigurations()
 }
 
-/**
-* Parse the parameters for a specific analysis type and method.
-* @param analysisType The analysis type. Eg. 'differential', 'functional'.
-* @param method The method to use for the analysis.
-* @param argsStr The string of arguments to parse.
-* @return A map of parameters.
-*/
-def parseParams(String analysisType, String method, String argsStr){
-    def parsed_params = [:]
-
-    // parse the params specific to the analysis type from the pipeline params
-    parsed_params += params.findAll { k, v -> k.matches(~/(${analysisType}).*/) }
-
-    // parse the params specific to the method
-    // First it gets the default values as specified by the pipeline params scope,
-    // and then overwrite with the values specified in the argsStr (highest priority),
-    // when provided
-    if (method) {
-        parsed_params += params.findAll { k, v -> k.matches(~/(${method}).*/) }
-        parsed_params["${analysisType}_method"] = method
+// Create a temporary schema file for toolsheet validation
+// This schema is derived from the pipeline's nextflow_schema.json by:
+// 1. Reading the toolsheet headers to determine which fields to include
+// 2. Extracting only those properties from the pipeline schema
+// 3. Creating a schema that allows for an array of objects
+// 4. Making analysis_name the only required field
+// 5. Adding meta fields to all properties
+// @return The absolute path to the temporary schema file
+def createToolsheetSchema() {
+    // Load toolsheet and get headers
+    def toolsheet_path = params.toolsheet_custom ?: "${projectDir}/assets/toolsheet.csv"
+    def toolsheet_lines = new File(toolsheet_path).readLines()
+    def headers = toolsheet_lines[0].split(',')
path: subworkflows/local/utils_nfcore_differentialabundance_pipeline/main.nf
user: login: suzannejin
body: maybe also consider csv/tsv?
created_at: 2025-04-04T10:21:05Z
author_association: NONE
diff_hunk: @@ -25,14 +29,14 @@ workflow ABUNDANCE_DIFFERENTIAL_FILTER {
     ch_samplesheet           // [ meta_exp, samplesheet ]
     ch_transcript_lengths    // [ meta_exp, transcript_lengths]
     ch_control_features      // [meta_exp, control_features]
-    ch_contrasts             // [[ meta_contrast, contrast_variable, reference, target ]]
+    ch_contrasts             // [[ [meta_contrast], [contrast_variable], [reference], [target] ]]
path: subworkflows/nf-core/abundance_differential_filter/main.nf
user: login: suzannejin
body: why change it to this list structure?
created_at: 2025-04-04T10:25:57Z
author_association: NONE
diff_hunk: @@ -25,14 +29,14 @@ workflow ABUNDANCE_DIFFERENTIAL_FILTER {
     ch_samplesheet           // [ meta_exp, samplesheet ]
     ch_transcript_lengths    // [ meta_exp, transcript_lengths]
     ch_control_features      // [meta_exp, control_features]
-    ch_contrasts             // [[ meta_contrast, contrast_variable, reference, target ]]
+    ch_contrasts             // [[ [meta_contrast], [contrast_variable], [reference], [target] ]]
path: subworkflows/nf-core/abundance_differential_filter/main.nf
user: login: pinin4fjords
body: This is because I wanted to allow for the possibility of a _different_ set of contrasts on each row of the toolsheet. I realise it's not your use case, but it is something this setup allows for. 

To do that, the contrasts have to be passed as one unit, in this structure. Then there's a quick `transpose()` you'll see further down to expand the contrasts again. 
created_at: 2025-04-04T10:30:24Z
author_association: MEMBER
diff_hunk: @@ -187,182 +153,221 @@ workflow DIFFERENTIALABUNDANCE {
     // If we have affy array data in the form of CEL files we'll be deriving
     // matrix and annotation from them
 
-    if (params.study_type == 'affy_array'){
+    // Uncompress the CEL files archive
+    UNTAR ( ch_celfiles )
 
-        // Uncompress the CEL files archive
+    ch_affy_input = ch_input.affy_array
+        .join(UNTAR.out.untar)
 
-        UNTAR ( ch_celfiles )
+    // Run affy to derive the matrix. Reset the meta so it can be used to
+    // define a prefix for different matrix flavours
 
-        ch_affy_input = ch_input
-            .join(UNTAR.out.untar)
+    AFFY_JUSTRMA_RAW (
+        ch_affy_input,
+        [[],[]]
+    )
+    AFFY_JUSTRMA_NORM (
+        ch_affy_input,
+        [[],[]]
+    )
 
-        // Run affy to derive the matrix. Reset the meta so it can be used to
-        // define a prefix for different matrix flavours
+    ch_affy_platform_features = AFFY_JUSTRMA_RAW.out.annotation
 
-        AFFY_JUSTRMA_RAW (
-            ch_affy_input,
-            [[],[]]
-        )
-        AFFY_JUSTRMA_NORM (
-            ch_affy_input,
-            [[],[]]
-        )
+    ch_versions = ch_versions
+        .mix(AFFY_JUSTRMA_RAW.out.versions)
 
-        // Fetch affy outputs and reset the meta
+    // We'll be running Proteus once per unique contrast variable to generate plots
 
-        ch_in_raw = AFFY_JUSTRMA_RAW.out.expression
-        ch_in_norm = AFFY_JUSTRMA_NORM.out.expression
+    // Run proteus to import protein abundances
+    PROTEUS(
+        ch_contrast_variables.combine(ch_maxquant_inputs)
+    )
 
-        ch_affy_platform_features = AFFY_JUSTRMA_RAW.out.annotation
+    ch_versions = ch_versions.mix(PROTEUS.out.versions)
 
-        ch_versions = ch_versions
-            .mix(AFFY_JUSTRMA_RAW.out.versions)
+    // Run GEO query to get the annotation
 
-    } else if (params.study_type == 'maxquant'){
+    GEOQUERY_GETGEO(ch_querygse)
+    ch_soft_features = GEOQUERY_GETGEO.out.annotation
 
-        // We'll be running Proteus once per unique contrast variable to generate plots
-        // TODO: there should probably be a separate plotting module in proteus to simplify this
+    ch_versions = ch_versions
+        .mix(GEOQUERY_GETGEO.out.versions)
 
-        // Run proteus to import protein abundances
-        PROTEUS(
-            ch_contrast_variables.combine(proteus_in)
-        )
+    // Raw inputs
+    // Note that we re-map the proteus output tables to the study ID as the
+    // tables are the same across contrasts, only one norm table will be necessary
 
-        // Re-map the proteus output tables to the study ID as the tables are the same across contrasts, only one norm table will be necessary
-        ch_in_raw = PROTEUS.out.raw_tab
-            .first()
-            .map{ meta, matrix -> tuple(exp_meta, matrix) }
-        ch_in_norm = PROTEUS.out.norm_tab
-            .first()
-            .map{ meta, matrix -> tuple(exp_meta, matrix) }
+    ch_in_raw = ch_input.rnaseq
+        .map{meta, file -> [meta, meta.matrix]}
path: workflows/differentialabundance.nf
user: login: suzannejin
body: this is not clear to me why you take meta.matrix (the path str?) instead of the file?
created_at: 2025-04-04T11:55:32Z
author_association: NONE
diff_hunk: @@ -304,6 +304,57 @@ def methodsDescriptionText(mqc_methods_yaml) {
     return description_html.toString()
 }
 
+// Split parameters into different sets based on prefixes and analysis type
+def splitParameters(params) {
+    // Base parameters are everything not in other sets
+    def baseParams = [:]
+    def differentialParams = [:]
+    def functionalParams = [:]
+
+    // First collect all differential parameters
+    params.each { key, value ->
+        if (key.startsWith('differential_') || key.startsWith('limma_') || key.startsWith('deseq2_')) {
+            differentialParams[key] = value
+        }
+    }
+
+    // Then collect functional parameters based on method
+    if (params.functional_method == 'gprofiler2') {
+        // For gprofiler2, include both base and differential params
path: subworkflows/local/utils_nfcore_differentialabundance_pipeline/main.nf
user: login: suzannejin
body: i guess this is because of the fact that gprofiler2 module needs the functional params but also some differential params, as reflected in modules.config. May you add comment too?
created_at: 2025-04-07T10:10:37Z
author_association: NONE
diff_hunk: @@ -304,6 +304,57 @@ def methodsDescriptionText(mqc_methods_yaml) {
     return description_html.toString()
 }
 
+// Split parameters into different sets based on prefixes and analysis type
+def splitParameters(params) {
+    // Base parameters are everything not in other sets
+    def baseParams = [:]
+    def differentialParams = [:]
+    def functionalParams = [:]
+
+    // First collect all differential parameters
+    params.each { key, value ->
+        if (key.startsWith('differential_') || key.startsWith('limma_') || key.startsWith('deseq2_')) {
+            differentialParams[key] = value
+        }
+    }
+
+    // Then collect functional parameters based on method
+    if (params.functional_method == 'gprofiler2') {
+        // For gprofiler2, include both base and differential params
+        params.each { key, value ->
+            if (key.startsWith('functional_') || key.startsWith('gprofiler2_')) {
+                functionalParams[key] = value
+            }
+        }
+        // Add all differential params to functional params for gprofiler2
+        functionalParams.putAll(differentialParams)
+    } else if (params.functional_method == 'gsea') {
+        // For GSEA, only include functional params
+        params.each { key, value ->
+            if (key.startsWith('functional_') || key.startsWith('gsea_')) {
+                functionalParams[key] = value
+            }
+        }
+    }
+
+    // Base params are everything not in other sets
+    params.each { key, value ->
+        if (!differentialParams.containsKey(key) && !functionalParams.containsKey(key)) {
+            baseParams[key] = value
+        }
+    }
+
+    // Add base params to both differential and functional
path: subworkflows/local/utils_nfcore_differentialabundance_pipeline/main.nf
user: login: suzannejin
body: base params is returned, why add it to differential params and functional params too? May you explain or add comment?
created_at: 2025-04-07T10:12:24Z
author_association: NONE
diff_hunk: @@ -389,6 +440,58 @@ def createToolsheetSchema() {
     return temp_schema.absolutePath
 }
 
+// Simplify meta map to only include parameters relevant for a specific analysis type
+def subsetMeta(meta, analysis_type) {
+    def relevantParams = [:]
+
+    // Define static lists of parameter prefixes for different analysis types
+    def differentialPrefixes = ['differential_', 'limma_', 'deseq2_']
path: subworkflows/local/utils_nfcore_differentialabundance_pipeline/main.nf
user: login: suzannejin
body: i know that this is only a proof of concept and you are cleaning the code later, I just add a note here for later :)
I think the method specific prefixes could be added dynamically based on the meta.differential_method or meta.functional method. Otherwise relevantParams include params from non relevant methods.
created_at: 2025-04-07T10:31:33Z
author_association: NONE
diff_hunk: @@ -390,6 +389,27 @@ workflow DIFFERENTIALABUNDANCE {
         .join(ch_transcript_lengths)
         .join(ch_control_features)
         .join(ch_contrasts)
+
+    // Deduplicate differential analysis runs by:
+    // 1. Subsetting meta to only relevant parameters
+    // 2. Grouping by the simplified meta
+    // 2. Preserving original analysis names for later parameter re-application
+    // This prevents redundant runs when multiple tool sheet rows require the same analysis
+
+    ch_deduplicated_differential_input = ch_pre_differential_input
+        .map{meta, matrix, diff_method, diff_fc_threshold, diff_qval_threshold, samplesheet, transcript_lengths, control_features, contrasts ->
+            def simplified_meta = subsetMeta(meta, 'differential')
+            [simplified_meta, meta.analysis_name, matrix, diff_method, diff_fc_threshold, diff_qval_threshold, samplesheet, transcript_lengths, control_features, contrasts]
+        }
+        .groupTuple()
+        .map{diff_meta, analysis_names, matrix, diff_method, diff_fc_threshold, diff_qval_threshold, samplesheet, transcript_lengths, control_features, contrasts ->
+            diff_meta.analysis_names = analysis_names
path: workflows/differentialabundance.nf
user: login: suzannejin
body: ahhh nice, i like the trick of adding all the analysis names shared by the same diff set, to facilitate later meta expansion while avoiding repeated computation :)
created_at: 2025-04-07T10:37:42Z
author_association: NONE
diff_hunk: @@ -304,6 +304,57 @@ def methodsDescriptionText(mqc_methods_yaml) {
     return description_html.toString()
 }
 
+// Split parameters into different sets based on prefixes and analysis type
+def splitParameters(params) {
+    // Base parameters are everything not in other sets
+    def baseParams = [:]
+    def differentialParams = [:]
+    def functionalParams = [:]
+
+    // First collect all differential parameters
+    params.each { key, value ->
+        if (key.startsWith('differential_') || key.startsWith('limma_') || key.startsWith('deseq2_')) {
+            differentialParams[key] = value
+        }
+    }
+
+    // Then collect functional parameters based on method
+    if (params.functional_method == 'gprofiler2') {
+        // For gprofiler2, include both base and differential params
path: subworkflows/local/utils_nfcore_differentialabundance_pipeline/main.nf
user: login: pinin4fjords
body: Yep, this is because changes in differential params effect gprofiler2, in a way that is not true for GSEA (as currently implemented). Feel free to suggest a rewording?
created_at: 2025-04-07T10:57:19Z
author_association: MEMBER
diff_hunk: @@ -304,6 +304,57 @@ def methodsDescriptionText(mqc_methods_yaml) {
     return description_html.toString()
 }
 
+// Split parameters into different sets based on prefixes and analysis type
+def splitParameters(params) {
+    // Base parameters are everything not in other sets
+    def baseParams = [:]
+    def differentialParams = [:]
+    def functionalParams = [:]
+
+    // First collect all differential parameters
+    params.each { key, value ->
+        if (key.startsWith('differential_') || key.startsWith('limma_') || key.startsWith('deseq2_')) {
+            differentialParams[key] = value
+        }
+    }
+
+    // Then collect functional parameters based on method
+    if (params.functional_method == 'gprofiler2') {
+        // For gprofiler2, include both base and differential params
+        params.each { key, value ->
+            if (key.startsWith('functional_') || key.startsWith('gprofiler2_')) {
+                functionalParams[key] = value
+            }
+        }
+        // Add all differential params to functional params for gprofiler2
+        functionalParams.putAll(differentialParams)
+    } else if (params.functional_method == 'gsea') {
+        // For GSEA, only include functional params
+        params.each { key, value ->
+            if (key.startsWith('functional_') || key.startsWith('gsea_')) {
+                functionalParams[key] = value
+            }
+        }
+    }
+
+    // Base params are everything not in other sets
+    params.each { key, value ->
+        if (!differentialParams.containsKey(key) && !functionalParams.containsKey(key)) {
+            baseParams[key] = value
+        }
+    }
+
+    // Add base params to both differential and functional
path: subworkflows/local/utils_nfcore_differentialabundance_pipeline/main.nf
user: login: pinin4fjords
body: So, the way I'm thinking of this, we can iterate on ANY parameter, but we need to control when we can share results across rows, and when we can't. 

By adding base parameters to e.g. the differential set, I ensure that any variation in e.g. the input matrix would prevent sharing of results, even if the differential parameterisation is identical. 

I'll try and add a comment to explain
created_at: 2025-04-07T11:00:52Z
author_association: MEMBER
diff_hunk: @@ -240,167 +304,238 @@ def methodsDescriptionText(mqc_methods_yaml) {
     return description_html.toString()
 }
 
-/**
-* Parse a string of arguments into a map.
-* @param argsStr The string of arguments to parse.
-* @return A map of parameters.
-* @example
-* parseStrArgsToMap("--arg1 aa --arg2 bb --arg4 cc") => [arg1: aa, arg2: bb, arg4: cc]
-*/
-def parseStrArgsToMap(String argsStr) {
-    if (!argsStr) return [:]                     // if null return empty
-    def tokens = argsStr.split().findAll { it }  // Split and remove empty strings
-    def pairs = tokens.collate(2)                // Group into pairs
-    return pairs.collectEntries {
-        [(it[0].replaceAll('^-+', '')): it[1]]   // Remove leading dashes
+// Split parameters into different sets based on prefixes and analysis type
+def splitParameters(params) {
+    // Base parameters are everything not in other sets
+    def baseParams = [:]
+    def differentialParams = [:]
+    def functionalParams = [:]
+
+    // First collect all differential parameters
+    params.each { key, value ->
+        if (key.startsWith('differential_') || key.startsWith('limma_') || key.startsWith('deseq2_')) {
+            differentialParams[key] = value
+        }
     }
-}
 
-/**
-* Parse the parameters for a specific analysis type and method.
-* @param analysisType The analysis type. Eg. 'differential', 'functional'.
-* @param method The method to use for the analysis.
-* @param argsStr The string of arguments to parse.
-* @return A map of parameters.
-*/
-def parseParams(String analysisType, String method, String argsStr){
-    def parsed_params = [:]
-
-    // parse the params specific to the analysis type from the pipeline params
-    parsed_params += params.findAll { k, v -> k.matches(~/(${analysisType}).*/) }
-
-    // parse the params specific to the method
-    // First it gets the default values as specified by the pipeline params scope,
-    // and then overwrite with the values specified in the argsStr (highest priority),
-    // when provided
-    if (method) {
-        parsed_params += params.findAll { k, v -> k.matches(~/(${method}).*/) }
-        parsed_params["${analysisType}_method"] = method
+    // Then collect functional parameters based on method
+    if (params.functional_method == 'gprofiler2') {
+        // For gprofiler2, include both base and differential params
+        params.each { key, value ->
+            if (key.startsWith('functional_') || key.startsWith('gprofiler2_')) {
+                functionalParams[key] = value
+            }
+        }
+        // Add all differential params to functional params for gprofiler2
+        functionalParams.putAll(differentialParams)
+    } else if (params.functional_method == 'gsea') {
+        // For GSEA, only include functional params
+        params.each { key, value ->
+            if (key.startsWith('functional_') || key.startsWith('gsea_')) {
+                functionalParams[key] = value
+            }
+        }
     }
-    if (argsStr) {
-        parsed_params += parseStrArgsToMap(argsStr)
+
+    // Base params are everything not in other sets
+    params.each { key, value ->
+        if (!differentialParams.containsKey(key) && !functionalParams.containsKey(key)) {
+            baseParams[key] = value
+        }
     }
 
-    return parsed_params
+    // Add base params to both differential and functional
path: subworkflows/local/utils_nfcore_differentialabundance_pipeline/main.nf
user: login: pinin4fjords
body: ```suggestion
    // Add base params to both differential and functional. We'll be comparing metas
    // to determine when we can share results across parameter iterations, so we need 
    // to make sure that anything which could lead to differences in e,g. differential 
    // results is recorded. This is the things upstream of differential, and well as 
    //differential parameters themselves.
```
created_at: 2025-04-07T11:05:31Z
author_association: MEMBER
diff_hunk: @@ -389,6 +440,58 @@ def createToolsheetSchema() {
     return temp_schema.absolutePath
 }
 
+// Simplify meta map to only include parameters relevant for a specific analysis type
+def subsetMeta(meta, analysis_type) {
+    def relevantParams = [:]
+
+    // Define static lists of parameter prefixes for different analysis types
+    def differentialPrefixes = ['differential_', 'limma_', 'deseq2_']
path: subworkflows/local/utils_nfcore_differentialabundance_pipeline/main.nf
user: login: pinin4fjords
body: True, though there's no cost to that.
created_at: 2025-04-07T11:07:17Z
author_association: MEMBER
diff_hunk: @@ -187,245 +152,299 @@ workflow DIFFERENTIALABUNDANCE {
     // If we have affy array data in the form of CEL files we'll be deriving
     // matrix and annotation from them
 
-    if (params.study_type == 'affy_array'){
+    // Uncompress the CEL files archive
+    UNTAR ( ch_celfiles )
 
-        // Uncompress the CEL files archive
+    ch_affy_input = ch_input.affy_array
+        .join(UNTAR.out.untar)
 
-        UNTAR ( ch_celfiles )
+    // Run affy to derive the matrix. Reset the meta so it can be used to
+    // define a prefix for different matrix flavours
 
-        ch_affy_input = ch_input
-            .join(UNTAR.out.untar)
+    AFFY_JUSTRMA_RAW (
+        ch_affy_input,
+        [[],[]]
+    )
+    AFFY_JUSTRMA_NORM (
+        ch_affy_input,
+        [[],[]]
+    )
 
-        // Run affy to derive the matrix. Reset the meta so it can be used to
-        // define a prefix for different matrix flavours
+    ch_affy_platform_features = AFFY_JUSTRMA_RAW.out.annotation
 
-        AFFY_JUSTRMA_RAW (
-            ch_affy_input,
-            [[],[]]
-        )
-        AFFY_JUSTRMA_NORM (
-            ch_affy_input,
-            [[],[]]
-        )
+    ch_versions = ch_versions
+        .mix(AFFY_JUSTRMA_RAW.out.versions)
 
-        // Fetch affy outputs and reset the meta
+    // We'll be running Proteus once per unique contrast variable to generate plots
 
-        ch_in_raw = AFFY_JUSTRMA_RAW.out.expression
-        ch_in_norm = AFFY_JUSTRMA_NORM.out.expression
+    // Run proteus to import protein abundances
+    PROTEUS(
+        ch_contrast_variables.combine(ch_maxquant_inputs)
+    )
 
-        ch_affy_platform_features = AFFY_JUSTRMA_RAW.out.annotation
+    ch_versions = ch_versions.mix(PROTEUS.out.versions)
 
-        ch_versions = ch_versions
-            .mix(AFFY_JUSTRMA_RAW.out.versions)
+    // Run GEO query to get the annotation
 
-    } else if (params.study_type == 'maxquant'){
+    GEOQUERY_GETGEO(ch_querygse)
+    ch_soft_features = GEOQUERY_GETGEO.out.annotation
 
-        // We'll be running Proteus once per unique contrast variable to generate plots
-        // TODO: there should probably be a separate plotting module in proteus to simplify this
+    ch_versions = ch_versions
+        .mix(GEOQUERY_GETGEO.out.versions)
 
-        // Run proteus to import protein abundances
-        PROTEUS(
-            ch_contrast_variables.combine(proteus_in)
-        )
+    // Raw inputs
+    // Note that we re-map the proteus output tables to the study ID as the
+    // tables are the same across contrasts, only one norm table will be necessary
 
-        // Re-map the proteus output tables to the study ID as the tables are the same across contrasts, only one norm table will be necessary
-        ch_in_raw = PROTEUS.out.raw_tab
-            .first()
-            .map{ meta, matrix -> tuple(exp_meta, matrix) }
-        ch_in_norm = PROTEUS.out.norm_tab
-            .first()
-            .map{ meta, matrix -> tuple(exp_meta, matrix) }
+    ch_in_raw = ch_input.rnaseq
+        .map{meta, file -> [meta, meta.matrix]}
+        .mix(AFFY_JUSTRMA_RAW.out.expression)
+        .mix(PROTEUS.out.raw_tab.first().map{ meta, matrix -> tuple(meta.study_meta, matrix) })
 
-        ch_versions = ch_versions.mix(PROTEUS.out.versions)
-    } else if(params.study_type == 'geo_soft_file'){
+    // Normalised inputs
 
-        GEOQUERY_GETGEO(ch_querygse)
-        ch_in_norm = GEOQUERY_GETGEO.out.expression
-        ch_soft_features = GEOQUERY_GETGEO.out.annotation
+    ch_in_norm = AFFY_JUSTRMA_NORM.out.expression
+        .mix(PROTEUS.out.norm_tab.first().map{ meta, matrix -> tuple(meta.study_meta, matrix) })
+        .mix(GEOQUERY_GETGEO.out.expression)
 
-        ch_versions = ch_versions
-            .mix(GEOQUERY_GETGEO.out.versions)
-    }
     //// Fetch or derive a feature annotation table
 
-    // If user has provided a feature annotation table, use that
-    if (params.features){
-        ch_features = Channel.of([ exp_meta, file(params.features, checkIfExists: true)])
-    } else if (params.study_type == 'affy_array'){
-        ch_features = ch_affy_platform_features
-    } else if(params.study_type == 'geo_soft_file') {
-        ch_features = ch_soft_features
-    } else if (params.gtf){
-        // Get feature annotations from a GTF file, gunzip if necessary
-
-        file_gtf_in = file(params.gtf)
-        file_gtf = [ [ "id": file_gtf_in.simpleName ], file_gtf_in ]
-
-        if ( params.gtf.endsWith('.gz') ){
-            GUNZIP_GTF(file_gtf)
-            file_gtf = GUNZIP_GTF.out.gunzip
-            ch_versions = ch_versions.mix(GUNZIP_GTF.out.versions)
+    // Branch ch_paramsets based on feature source
+    ch_feature_sources = ch_paramsets
+        .branch {
+            user_features: it.features
+            affy_features: it.study_type == 'affy_array'
+            geo_features: it.study_type == 'geo_soft_file'
+            gtf_features: it.gtf
+            matrix_features: true
         }
 
-        // Get a features table from the GTF and combine with the matrix and sample
-        // annotation (fom = features/ observations/ matrix)
+    // Handle user-provided feature annotations
+    ch_user_features = ch_feature_sources.user_features
+        .map { paramset -> [ paramset, file(paramset.features, checkIfExists: true) ] }
 
-        GTF_TO_TABLE( file_gtf, [[ "id":""], []])
-        ch_features = GTF_TO_TABLE.out.feature_annotation
-            .map{
-                tuple( exp_meta, it[1])
-            }
+    // Handle Affy array platform features
+    ch_affy_features = ch_feature_sources.affy_features
+        .map { paramset -> ch_affy_platform_features }
 
-        // Record the version of the GTF -> table tool
+    // Handle GEO soft file features
+    ch_geo_features = ch_feature_sources.geo_features
+        .map { paramset -> ch_soft_features }
 
-        ch_versions = ch_versions
-            .mix(GTF_TO_TABLE.out.versions)
-    }
-    else{
-
-        // Otherwise we can just use the matrix input; save it to the workdir so that it does not
-        // just appear wherever the user runs the pipeline
-        matrix_as_anno_filename = "${workflow.workDir}/${matrix_file.getBaseName()}_as_anno.${matrix_file.getExtension()}"
-        if (params.study_type == 'maxquant'){
-            ch_features_matrix = ch_in_norm
-        } else {
-            ch_features_matrix = ch_in_raw
+    // Handle GTF-based feature annotations
+    ch_gtf_files = ch_feature_sources.gtf_features
+        .map { paramset -> [ paramset, file(paramset.gtf, checkIfExists: true) ] }
+
+    // Process GTF files if needed
+    ch_gtf_for_processing = ch_gtf_files
+        .branch {
+            compressed: it[1].name.endsWith('.gz')
+            uncompressed: !it[1].name.endsWith('.gz')
         }
-        ch_features = ch_features_matrix
-            .map{ meta, matrix ->
-                matrix_copy = file(matrix_as_anno_filename)
-                matrix_copy.exists() && matrix.getText().md5().equals(matrix_copy.getText().md5()) ?: matrix.copyTo(matrix_as_anno_filename)
-                [ meta, file(matrix_as_anno_filename) ]
-            }
+
+    // Decompress GTF files if needed
+    GUNZIP_GTF(ch_gtf_for_processing.compressed)
+    ch_versions = ch_versions.mix(GUNZIP_GTF.out.versions)
+
+    // Combine compressed and uncompressed GTF files
+    ch_gtf_processed = ch_gtf_for_processing.uncompressed
+        .mix(GUNZIP_GTF.out.gunzip)
+
+    // Convert GTF to feature annotation table
+    GTF_TO_TABLE(ch_gtf_processed, [tuple('id':""), []])
+    ch_gtf_features = GTF_TO_TABLE.out.feature_annotation
+    ch_versions = ch_versions.mix(GTF_TO_TABLE.out.versions)
+
+    ch_pre_matrix_features = ch_feature_sources.matrix_features.branch{ paramset ->
+        maxquant: paramset.study_type == 'maxquant'
+        other: true
     }
+    ch_matrix_features = ch_pre_matrix_features.maxquant.join(ch_in_norm)
+        .mix(ch_pre_matrix_features.other.join(ch_in_raw))
+        .map{ meta, matrix ->
+            matrix_as_anno_filename = "${workflow.workDir}/${matrix.getBaseName()}_as_anno.${matrix.getExtension()}"
+            matrix_copy = file(matrix_as_anno_filename)
+            matrix_copy.exists() && matrix.getText().md5().equals(matrix_copy.getText().md5()) ?: matrix.copyTo(matrix_as_anno_filename)
+            [ meta, file(matrix_as_anno_filename) ]
+        }
+
+    // Combine all feature annotation channels
+    ch_features = ch_user_features
+        .mix(ch_affy_features)
+        .mix(ch_geo_features)
+        .mix(ch_gtf_features)
+        .mix(ch_matrix_features)
+
 
     // Check compatibility of FOM elements and contrasts
-    if (params.study_type == 'affy_array' || params.study_type == 'maxquant'){
-        ch_matrices_for_validation = ch_in_raw
-            .join(ch_in_norm)
-            .map{tuple(it[0], [it[1], it[2]])}
-    }
-    else if (params.study_type == 'geo_soft_file'){
-        ch_matrices_for_validation = ch_in_norm
-    }
-    else{
-        ch_matrices_for_validation = ch_in_raw
+
+    ch_matrix_sources = ch_paramsets.branch{
+        affy_or_maxquant: params.study_type == 'affy_array' || params.study_type == 'maxquant'
+        geo_soft_file: params.study_type == 'geo_soft_file'
+        other: true
     }
 
+    // Create a channel with the matrices for validation. For Affy and MaxQuant
+    // we have both raw and norm matrices, for GEO soft file we only have norm,
+    // for other we only have raw
+
+    ch_matrices_for_validation = ch_matrix_sources.affy_or_maxquant
+        .join(ch_in_raw)
+        .join(ch_in_norm)
+        .map{tuple(it[0], [it[1], it[2]])}
+        .mix(ch_matrix_sources.geo_soft_file.join(ch_in_norm)
+        .mix(ch_matrix_sources.other.join(ch_in_raw)))
+
+    // Validate the components agains one another. We try not to assume too
+    // much about chanel order, so we deploy the multiMap workaround.
+
+    validator_input = ch_samplesheet
+        .join(ch_matrices_for_validation)
+        .join(ch_features)
+        .join(ch_contrasts_file)
+        .multiMap{meta, samplesheet, matrices, features_file, contrasts_file ->
+            samplesheet_matrices: [meta, samplesheet, matrices]
+            features_file: [meta, features_file]
+            contrasts_file: [meta, contrasts_file]
+        }
+
     VALIDATOR(
-        ch_input.join(ch_matrices_for_validation),
-        ch_features,
-        ch_contrasts_file
+        validator_input.samplesheet_matrices,
+        validator_input.features_file,
+        validator_input.contrasts_file
     )
 
     // For Affy, we've validated multiple input matrices for raw and norm,
     // we'll separate them out again here
 
-    if (params.study_type == 'affy_array' || params.study_type == 'maxquant'){
-        ch_validated_assays = VALIDATOR.out.assays
-            .transpose()
-            .branch {
-                raw: it[1].name.contains('raw')
-                normalised: it[1].name =~ /normali[sz]ed/
-            }
-        ch_raw = ch_validated_assays.raw
-        ch_norm = ch_validated_assays.normalised
-    }
-    else if (params.study_type == 'geo_soft_file') {
-        ch_norm = VALIDATOR.out.assays
-    }
+    ch_multi_validated_assays = VALIDATOR.out.assays
+        .filter{meta, assays -> meta.study_type == 'affy_array' || meta.study_type == 'maxquant'}
+        .transpose()
+        .branch { meta, assay ->
+            raw: assay.name.contains('raw')
+            normalised: assay.name =~ /normali[sz]ed/
+        }
 
-    if(params.study_type != 'rnaseq') {
-        ch_matrix_for_differential = ch_norm
-    }
-    else{
-        ch_raw = VALIDATOR.out.assays
-        ch_matrix_for_differential = ch_raw
-    }
+    // Get raw matrices from the validation
+
+    ch_raw = ch_multi_validated_assays.raw
+        .mix(VALIDATOR.out.assays.filter{meta, assay -> meta.study_type== 'rnaseq'})
+
+    // For RNASeq and GEO soft file we've validated a single matrix, raw in the
+    // case of RNASeq and norm in the case of GEO soft file, and these are the
+    // matrices we'll use for differential abundance testing. For affy or maxquant
+    // we'll use the normalised matrices.
+
+    ch_matrix_for_differential = ch_multi_validated_assays.normalised
+        .mix(VALIDATOR.out.assays.filter{meta, assay -> meta.study_type == 'rnaseq' || meta.study_type == 'geo_soft_file'})
 
     // Split the contrasts up so we can run differential analyses and
     // downstream plots separately.
     // Replace NA strings that might have snuck into the blocking column
 
     ch_contrasts = VALIDATOR.out.contrasts
-        .map{it[1]}
         .splitCsv ( header:true, sep:'\t' )
-        .map{
-            it.blocking = it.blocking.replaceAll('^NA$', '')
-            if (!it.id){
-                it.id = it.values().join('_')
+        .map{study_meta, contrast ->
+            contrast.blocking = contrast.blocking.replaceAll('^NA$', '')
+            if (!contrast.id){
+                contrast.id = contrast.values().join('_')
             }
-            tuple(it, it.variable, it.reference, it.target)
+            meta = contrast + [study_meta: study_meta]
+            tuple(study_meta, meta,contrast.variable, contrast.reference, contrast.target)
         }
+        .groupTuple() // [study_meta, [meta], [variable], [reference], [target]]
+        .map{ study_meta, meta, variable, reference, target -> [study_meta, [meta, variable, reference, target]]}
 
     // Firstly Filter the input matrix
 
+    ch_matrixfilter_input = ch_matrix_for_differential
+        .join(VALIDATOR.out.sample_meta)
+        .multiMap{meta, matrix, samplesheet ->
+            matrix: [meta, matrix]
+            samplesheet: [meta, samplesheet]
+        }
+
     CUSTOM_MATRIXFILTER(
-        ch_matrix_for_differential,
-        VALIDATOR.out.sample_meta
+        ch_matrixfilter_input.matrix,
+        ch_matrixfilter_input.samplesheet
     )
 
     // ========================================================================
     // Differential analysis
     // ========================================================================
 
     // Prepare inputs for differential processes
-
     ch_differential_input = CUSTOM_MATRIXFILTER.out.filtered
-        .combine(ch_tools)
-        .map { meta, matrix, tools_norm, tools_diff, tools_func ->
-            [
-                // as the differential subworkflow will automatically add
-                // method_differential into meta, here we only need to add
-                // args_differential into meta. In this way, the differential
-                // outputs should always have in the meta the method_differential
-                // and args_differential
-                meta + [args_differential: tools_diff.args],
-                matrix,
-                tools_diff.method,
-                tools_diff.fc_threshold,
-                tools_diff.stat_threshold
-            ]
+        .join(VALIDATOR.out.sample_meta)
+        .join(ch_transcript_lengths)
+        .join(ch_control_features)
+        .join(ch_contrasts)
+
+    // Deduplicate differential analysis runs
+    ch_deduplicated_differential_input = ch_differential_input
+        .map{ meta, matrix, samplesheet, transcript_lengths, control_features, contrasts ->
+            def simplified_meta = subsetMeta(meta, 'differential')
+            [simplified_meta, meta.analysis_name, matrix, samplesheet, transcript_lengths, control_features, contrasts]
+        }
+        .groupTuple()
+        .map{ diff_meta, analysis_names, matrix, samplesheet, transcript_lengths, control_features, contrasts ->
+            diff_meta.analysis_names = analysis_names
+            [diff_meta, matrix[0], samplesheet[0], transcript_lengths[0], control_features[0], contrasts[0]]
+        }
+
+    // Use a multiMap to generate synched channels for differential analysis
+    ch_differential_input = ch_deduplicated_differential_input
+        .multiMap{ meta, matrix, samplesheet, transcript_lengths, control_features, contrasts ->
+            input: [meta, matrix, meta.differential_method, meta.differential_min_fold_change, meta.differential_max_qval]
+            samplesheet: [meta, samplesheet]
+            transcript_lengths: [meta, transcript_lengths]
+            control_features: [meta, control_features]
+            contrasts: [meta, contrasts]
         }
 
     // Run differential analysis
 
     ABUNDANCE_DIFFERENTIAL_FILTER(
-        ch_differential_input,
-        VALIDATOR.out.sample_meta,
-        ch_transcript_lengths,
-        ch_control_features,
-        ch_contrasts
+        ch_differential_input.input,
+        ch_differential_input.samplesheet,
+        ch_differential_input.transcript_lengths,
+        ch_differential_input.control_features,
+        ch_differential_input.contrasts.map{ study_meta, contrasts -> contrasts }
     )
 
-    // collect differential results
+    // Collect differential results
+    // reapplyParams fetches back the study_meta from the paramset
+
+    ch_differential_results = reapplyParams(ABUNDANCE_DIFFERENTIAL_FILTER.out.results_genewise, ch_paramsets_by_name)
+    ch_differential_results_filtered = reapplyParams(ABUNDANCE_DIFFERENTIAL_FILTER.out.results_genewise_filtered, ch_paramsets_by_name)
+
+    // Processed matrices, one pers study/ toolsheet row, so we can reset the meta to the study_meta
+
+    ch_differential_norm = reapplyParams(ABUNDANCE_DIFFERENTIAL_FILTER.out.normalised_matrix, ch_paramsets_by_name)
path: workflows/differentialabundance.nf
user: login: pinin4fjords
body: I don't like this, by the way, I'm trying to think of something nicer
created_at: 2025-04-07T11:07:50Z
author_association: MEMBER

PR comments:
user: login: github-actions[bot]
created_at: 2025-03-31T16:36:50Z
body: ## `nf-core pipelines lint` overall result: Passed :white_check_mark: :warning:

Posted for pipeline commit ea0545d

```diff
+| ✅ 310 tests passed       |+
#| ❔   6 tests were ignored |#
!| ❗   6 tests had warnings |!
```

<details>

### :heavy_exclamation_mark: Test warnings:

* [pipeline_todos](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/pipeline_todos) - TODO string in `nextflow.config`: _Update the field with the details of the contributors to your pipeline. New with Nextflow version 24.10.0_
* [pipeline_todos](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/pipeline_todos) - TODO string in `ro-crate-metadata.json`: _"description": "<h1>\n  <picture>\n    <source media=\"(prefers-color-scheme: dark)\" srcset=\"docs/images/nf-core-differentialabundance_logo_dark.png\">\n    <img alt=\"nf-core/differentialabundance\" src=\"docs/images/nf-core-differentialabundance_logo_light.png\">\n  </picture>\n</h1>\n\n[![GitHub Actions CI Status](https://github.com/nf-core/differentialabundance/actions/workflows/ci.yml/badge.svg)](https://github.com/nf-core/differentialabundance/actions/workflows/ci.yml)\n[![GitHub Actions Linting Status](https://github.com/nf-core/differentialabundance/actions/workflows/linting.yml/badge.svg)](https://github.com/nf-core/differentialabundance/actions/workflows/linting.yml)[![AWS CI](https://img.shields.io/badge/CI%20tests-full%20size-FF9900?labelColor=000000&logo=Amazon%20AWS)](https://nf-co.re/differentialabundance/results)[![Cite with Zenodo](http://img.shields.io/badge/DOI-10.5281/zenodo.XXXXXXX-1073c8?labelColor=000000)](https://doi.org/10.5281/zenodo.XXXXXXX)\n[![nf-test](https://img.shields.io/badge/unit_tests-nf--test-337ab7.svg)](https://www.nf-test.com)\n\n[![Nextflow](https://img.shields.io/badge/nextflow%20DSL2-%E2%89%A524.04.2-23aa62.svg)](https://www.nextflow.io/)\n[![run with conda](http://img.shields.io/badge/run%20with-conda-3EB049?labelColor=000000&logo=anaconda)](https://docs.conda.io/en/latest/)\n[![run with docker](https://img.shields.io/badge/run%20with-docker-0db7ed?labelColor=000000&logo=docker)](https://www.docker.com/)\n[![run with singularity](https://img.shields.io/badge/run%20with-singularity-1d355c.svg?labelColor=000000)](https://sylabs.io/docs/)\n[![Launch on Seqera Platform](https://img.shields.io/badge/Launch%20%F0%9F%9A%80-Seqera%20Platform-%234256e7)](https://cloud.seqera.io/launch?pipeline=https://github.com/nf-core/differentialabundance)\n\n[![Get help on Slack](http://img.shields.io/badge/slack-nf--core%20%23differentialabundance-4A154B?labelColor=000000&logo=slack)](https://nfcore.slack.com/channels/differentialabundance)[![Follow on Twitter](http://img.shields.io/badge/twitter-%40nf__core-1DA1F2?labelColor=000000&logo=twitter)](https://twitter.com/nf_core)[![Follow on Mastodon](https://img.shields.io/badge/mastodon-nf__core-6364ff?labelColor=FFFFFF&logo=mastodon)](https://mstdn.science/@nf_core)[![Watch on YouTube](http://img.shields.io/badge/youtube-nf--core-FF0000?labelColor=000000&logo=youtube)](https://www.youtube.com/c/nf-core)\n\n## Introduction\n\n**nf-core/differentialabundance** is a bioinformatics pipeline that ...\n\n TODO nf-core:\n   Complete this sentence with a 2-3 sentence summary of what types of data the pipeline ingests, a brief overview of the\n   major pipeline sections and the types of output it produces. You're giving an overview to someone new\n   to nf-core here, in 15-20 seconds. For an example, see https://github.com/nf-core/rnaseq/blob/master/README.md#introduction\n\n\n Include a figure that guides the user through the major workflow steps. Many nf-core\n     workflows use the \"tube map\" design for that. See https://nf-co.re/docs/contributing/design_guidelines#examples for examples.   \n Fill in short bullet-pointed list of the default steps in the pipeline \n\n## Usage\n\n> [!NOTE]\n> If you are new to Nextflow and nf-core, please refer to [this page](https://nf-co.re/docs/usage/installation) on how to set-up Nextflow. Make sure to [test your setup](https://nf-co.re/docs/usage/introduction#how-to-run-a-pipeline) with `-profile test` before running the workflow on actual data.\n\n Describe the minimum required steps to execute the pipeline, e.g. how to prepare samplesheets.\n     Explain what rows and columns represent. For instance (please edit as appropriate):\n\nFirst, prepare a samplesheet with your input data that looks as follows:\n\n`samplesheet.csv`:\n\n```csv\nsample,fastq_1,fastq_2\nCONTROL_REP1,AEG588A1_S1_L002_R1_001.fastq.gz,AEG588A1_S1_L002_R2_001.fastq.gz\n```\n\nEach row represents a fastq file (single-end) or a pair of fastq files (paired end).\n\n\n\nNow, you can run the pipeline using:\n\n update the following command to include all required parameters for a minimal example \n\n```bash\nnextflow run nf-core/differentialabundance \\\n   -profile <docker/singularity/.../institute> \\\n   --input samplesheet.csv \\\n   --outdir <OUTDIR>\n```\n\n> [!WARNING]\n> Please provide pipeline parameters via the CLI or Nextflow `-params-file` option. Custom config files including those provided by the `-c` Nextflow option can be used to provide any configuration _**except for parameters**_; see [docs](https://nf-co.re/docs/usage/getting_started/configuration#custom-configuration-files).\n\nFor more details and further functionality, please refer to the [usage documentation](https://nf-co.re/differentialabundance/usage) and the [parameter documentation](https://nf-co.re/differentialabundance/parameters).\n\n## Pipeline output\n\nTo see the results of an example test run with a full size dataset refer to the [results](https://nf-co.re/differentialabundance/results) tab on the nf-core website pipeline page.\nFor more details about the output files and reports, please refer to the\n[output documentation](https://nf-co.re/differentialabundance/output).\n\n## Credits\n\nnf-core/differentialabundance was originally written by Oskar Wacker, Jonathan Manning.\n\nWe thank the following people for their extensive assistance in the development of this pipeline:\n\n If applicable, make list of people who have also contributed \n\n## Contributions and Support\n\nIf you would like to contribute to this pipeline, please see the [contributing guidelines](.github/CONTRIBUTING.md).\n\nFor further information or help, don't hesitate to get in touch on the [Slack `#differentialabundance` channel](https://nfcore.slack.com/channels/differentialabundance) (you can join with [this invite](https://nf-co.re/join/slack)).\n\n## Citations\n\n Add citation for pipeline after first release. Uncomment lines below and update Zenodo doi and badge at the top of this file. \n If you use nf-core/differentialabundance for your analysis, please cite it using the following doi: [10.5281/zenodo.XXXXXX](https://doi.org/10.5281/zenodo.XXXXXX) \n\n Add bibliography of tools and data used in your pipeline \n\nAn extensive list of references for the tools used by the pipeline can be found in the [`CITATIONS.md`](CITATIONS.md) file.\n\nYou can cite the `nf-core` publication as follows:\n\n> **The nf-core framework for community-curated bioinformatics pipelines.**\n>\n> Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen.\n>\n> _Nat Biotechnol._ 2020 Feb 13. doi: [10.1038/s41587-020-0439-x](https://dx.doi.org/10.1038/s41587-020-0439-x).\n",_
* [pipeline_todos](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/pipeline_todos) - TODO string in `main.nf`: _Optionally add in-text citation tools to this list._
* [pipeline_todos](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/pipeline_todos) - TODO string in `main.nf`: _Optionally add bibliographic entries to this list._
* [pipeline_todos](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/pipeline_todos) - TODO string in `main.nf`: _Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!_
* [pipeline_todos](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/pipeline_todos) - TODO string in `base.config`: _Check the defaults for all processes_

### :grey_question: Tests ignored:

* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File is ignored: `assets/multiqc_config.yml`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default ignored: params.report_file
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default ignored: params.logo_file
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default ignored: params.css_file
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default ignored: params.citations_file
* [multiqc_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/multiqc_config) - multiqc_config

### :white_check_mark: Tests passed:

* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `.gitattributes`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `.gitignore`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `.nf-core.yml`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `.editorconfig`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `.prettierignore`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `.prettierrc.yml`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `CHANGELOG.md`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `CITATIONS.md`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `CODE_OF_CONDUCT.md`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `LICENSE` or `LICENSE.md` or `LICENCE` or `LICENCE.md`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `nextflow_schema.json`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `nextflow.config`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `README.md`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `.github/.dockstore.yml`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `.github/CONTRIBUTING.md`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `.github/ISSUE_TEMPLATE/bug_report.yml`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `.github/ISSUE_TEMPLATE/config.yml`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `.github/ISSUE_TEMPLATE/feature_request.yml`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `.github/PULL_REQUEST_TEMPLATE.md`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `.github/workflows/branch.yml`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `.github/workflows/ci.yml`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `.github/workflows/linting_comment.yml`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `.github/workflows/linting.yml`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `assets/email_template.html`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `assets/email_template.txt`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `assets/sendmail_template.txt`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `assets/nf-core-differentialabundance_logo_light.png`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `conf/modules.config`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `conf/test.config`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `conf/test_full.config`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `docs/images/nf-core-differentialabundance_logo_light.png`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `docs/images/nf-core-differentialabundance_logo_dark.png`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `docs/output.md`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `docs/README.md`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `docs/README.md`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `docs/usage.md`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `main.nf`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `conf/base.config`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `conf/igenomes.config`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `conf/igenomes_ignored.config`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `.github/workflows/awstest.yml`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `.github/workflows/awsfulltest.yml`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `modules.json`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File found: `ro-crate-metadata.json`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File not found check: `.github/ISSUE_TEMPLATE/bug_report.md`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File not found check: `.github/ISSUE_TEMPLATE/feature_request.md`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File not found check: `.github/workflows/push_dockerhub.yml`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File not found check: `.markdownlint.yml`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File not found check: `.nf-core.yaml`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File not found check: `.yamllint.yml`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File not found check: `bin/markdown_to_html.r`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File not found check: `conf/aws.config`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File not found check: `docs/images/nf-core-differentialabundance_logo.png`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File not found check: `lib/Checks.groovy`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File not found check: `lib/Completion.groovy`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File not found check: `lib/NfcoreTemplate.groovy`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File not found check: `lib/Utils.groovy`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File not found check: `lib/Workflow.groovy`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File not found check: `lib/WorkflowMain.groovy`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File not found check: `lib/WorkflowDifferentialabundance.groovy`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File not found check: `parameters.settings.json`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File not found check: `pipeline_template.yml`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File not found check: `Singularity`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File not found check: `lib/nfcore_external_java_deps.jar`
* [files_exist](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_exist) - File not found check: `.travis.yml`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Found nf-schema plugin
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable found: `manifest.name`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable found: `manifest.nextflowVersion`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable found: `manifest.description`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable found: `manifest.version`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable found: `manifest.homePage`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable found: `timeline.enabled`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable found: `trace.enabled`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable found: `report.enabled`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable found: `dag.enabled`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable found: `process.cpus`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable found: `process.memory`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable found: `process.time`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable found: `params.outdir`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable found: `params.input`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable found: `validation.help.enabled`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable found: `manifest.mainScript`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable found: `timeline.file`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable found: `trace.file`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable found: `report.file`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable found: `dag.file`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable found: `validation.help.beforeText`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable found: `validation.help.afterText`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable found: `validation.help.command`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable found: `validation.summary.beforeText`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable found: `validation.summary.afterText`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable (correctly) not found: `params.nf_required_version`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable (correctly) not found: `params.container`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable (correctly) not found: `params.singleEnd`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable (correctly) not found: `params.igenomesIgnore`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable (correctly) not found: `params.name`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable (correctly) not found: `params.enable_conda`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable (correctly) not found: `params.max_cpus`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable (correctly) not found: `params.max_memory`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable (correctly) not found: `params.max_time`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable (correctly) not found: `params.validationFailUnrecognisedParams`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable (correctly) not found: `params.validationLenientMode`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable (correctly) not found: `params.validationSchemaIgnoreParams`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable (correctly) not found: `params.validationShowHiddenParams`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config ``timeline.enabled`` had correct value: ``true``
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config ``report.enabled`` had correct value: ``true``
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config ``trace.enabled`` had correct value: ``true``
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config ``dag.enabled`` had correct value: ``true``
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config ``manifest.name`` began with ``nf-core/``
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable ``manifest.homePage`` began with https://github.com/nf-core/
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config ``dag.file`` ended with ``.html``
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config variable ``manifest.nextflowVersion`` started with >= or !>=
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config ``manifest.version`` ends in ``dev``: ``1.6.0dev``
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config `params.custom_config_version` is set to `master`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config `params.custom_config_base` is set to `https://raw.githubusercontent.com/nf-core/configs/master`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Lines for loading custom profiles found
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - nextflow.config contains configuration profile `test`
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.study_name= study
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.study_type= rnaseq
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.study_abundance_type= counts
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.affy_cel_files_archive= null
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.querygse= null
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.observations_id_col= sample
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.observations_type= sample
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.features_id_col= gene_id
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.features_name_col= gene_name
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.features_type= gene
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.features_metadata_cols= gene_id,gene_name,gene_biotype
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.features_gtf_feature_type= transcript
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.features_gtf_table_first_field= gene_id
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.affy_file_name_col= file
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.affy_background= true
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.affy_bgversion= 2
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.affy_cdfname= null
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.affy_build_annotation= true
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.proteus_measurecol_prefix= LFQ intensity
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.proteus_norm_function= normalizeMedian
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.proteus_plotsd_method= violin
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.proteus_plotmv_loess= true
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.proteus_palette_name= Set1
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.filtering_min_abundance= 1.0
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.filtering_min_samples= 1.0
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.filtering_min_proportion_not_na= 0.5
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.exploratory_clustering_method= ward.D2
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.exploratory_cor_method= spearman
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.exploratory_n_features= 500
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.exploratory_whisker_distance= 1.5
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.exploratory_mad_threshold= -5
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.exploratory_main_variable= auto_pca
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.exploratory_assay_names= raw,normalised,variance_stabilised
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.exploratory_final_assay= variance_stabilised
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.exploratory_palette_name= Set1
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.toolsheet= null
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.differential_method= deseq2
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.differential_feature_id_column= gene_id
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.differential_fc_column= log2FoldChange
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.differential_pval_column= pvalue
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.differential_qval_column= padj
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.differential_min_fold_change= 2.0
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.differential_max_pval= 1.0
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.differential_max_qval= 0.05
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.differential_feature_name_column= gene_name
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.differential_foldchanges_logged= true
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.differential_palette_name= Set1
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.deseq2_test= Wald
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.deseq2_fit_type= parametric
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.deseq2_sf_type= ratio
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.deseq2_min_replicates_for_replace= 7
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.deseq2_independent_filtering= true
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.deseq2_lfc_threshold= 0
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.deseq2_alt_hypothesis= greaterAbs
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.deseq2_p_adjust_method= BH
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.deseq2_alpha= 0.1
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.deseq2_minmu= 0.5
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.deseq2_vs_method= vst
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.deseq2_shrink_lfc= true
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.deseq2_cores= 1
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.deseq2_vs_blind= true
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.deseq2_vst_nsub= 1000
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.limma_spacing= null
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.limma_block= null
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.limma_correlation= null
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.limma_method= ls
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.limma_proportion= 0.01
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.limma_stdev_coef_lim= 0.1,4
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.limma_winsor_tail_p= 0.05,0.1
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.limma_lfc= 0
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.limma_adjust_method= BH
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.limma_p_value= 1.0
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.limma_use_voom= false
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.immunedeconv_run= false
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.immunedeconv_method= quantiseq
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.immunedeconv_function= deconvolute
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.functional_method= null
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.gene_sets_files= null
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.gsea_permute= phenotype
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.gsea_nperm= 1000
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.gsea_scoring_scheme= weighted
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.gsea_metric= Signal2Noise
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.gsea_sort= real
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.gsea_order= descending
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.gsea_set_max= 500
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.gsea_set_min= 15
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.gsea_norm= meandiv
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.gsea_rnd_type= no_balance
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.gsea_make_sets= true
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.gsea_num= 100
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.gsea_plot_top_x= 20
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.gsea_rnd_seed= timestamp
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.gprofiler2_significant= true
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.gprofiler2_measure_underrepresentation= false
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.gprofiler2_evcodes= false
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.gprofiler2_max_qval= 0.05
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.gprofiler2_domain_scope= annotated
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.gprofiler2_min_diff= 1
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.gprofiler2_palette_name= Blues
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.shinyngs_build_app= true
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.shinyngs_shinyapps_account= null
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.shinyngs_shinyapps_app_name= null
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.report_title= null
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.report_author= null
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.report_description= null
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.report_scree= true
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.report_round_digits= 4
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.igenomes_base= s3://ngi-igenomes/igenomes/
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.custom_config_version= master
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.custom_config_base= https://raw.githubusercontent.com/nf-core/configs/master
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.publish_dir_mode= copy
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.validate_params= true
* [nextflow_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nextflow_config) - Config default value correct: params.pipelines_testdata_base_path= https://raw.githubusercontent.com/nf-core/test-datasets/
* [files_unchanged](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_unchanged) - `.gitattributes` matches the template
* [files_unchanged](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_unchanged) - `.prettierrc.yml` matches the template
* [files_unchanged](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_unchanged) - `CODE_OF_CONDUCT.md` matches the template
* [files_unchanged](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_unchanged) - `LICENSE` matches the template
* [files_unchanged](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_unchanged) - `.github/.dockstore.yml` matches the template
* [files_unchanged](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_unchanged) - `.github/CONTRIBUTING.md` matches the template
* [files_unchanged](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_unchanged) - `.github/ISSUE_TEMPLATE/bug_report.yml` matches the template
* [files_unchanged](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_unchanged) - `.github/ISSUE_TEMPLATE/config.yml` matches the template
* [files_unchanged](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_unchanged) - `.github/ISSUE_TEMPLATE/feature_request.yml` matches the template
* [files_unchanged](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_unchanged) - `.github/PULL_REQUEST_TEMPLATE.md` matches the template
* [files_unchanged](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_unchanged) - `.github/workflows/branch.yml` matches the template
* [files_unchanged](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_unchanged) - `.github/workflows/linting_comment.yml` matches the template
* [files_unchanged](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_unchanged) - `.github/workflows/linting.yml` matches the template
* [files_unchanged](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_unchanged) - `assets/email_template.html` matches the template
* [files_unchanged](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_unchanged) - `assets/email_template.txt` matches the template
* [files_unchanged](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_unchanged) - `assets/sendmail_template.txt` matches the template
* [files_unchanged](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_unchanged) - `assets/nf-core-differentialabundance_logo_light.png` matches the template
* [files_unchanged](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_unchanged) - `docs/images/nf-core-differentialabundance_logo_light.png` matches the template
* [files_unchanged](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_unchanged) - `docs/images/nf-core-differentialabundance_logo_dark.png` matches the template
* [files_unchanged](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_unchanged) - `docs/README.md` matches the template
* [files_unchanged](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_unchanged) - `.gitignore` matches the template
* [files_unchanged](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/files_unchanged) - `.prettierignore` matches the template
* [actions_ci](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/actions_ci) - '.github/workflows/ci.yml' is triggered on expected events
* [actions_ci](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/actions_ci) - '.github/workflows/ci.yml' checks minimum NF version
* [actions_awstest](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/actions_awstest) - '.github/workflows/awstest.yml' is triggered correctly
* [actions_awsfulltest](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/actions_awsfulltest) - `.github/workflows/awsfulltest.yml` is triggered correctly
* [actions_awsfulltest](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/actions_awsfulltest) - `.github/workflows/awsfulltest.yml` does not use `-profile test`
* [readme](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/readme) - README Nextflow minimum version badge matched config. Badge: `24.04.2`, Config: `24.04.2`
* [readme](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/readme) - README Zenodo placeholder was replaced with DOI.
* [plugin_includes](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/plugin_includes) - No wrong validation plugin imports have been found
* [pipeline_name_conventions](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/pipeline_name_conventions) - Name adheres to nf-core convention
* [template_strings](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/template_strings) - Did not find any Jinja template strings (0 files)
* [schema_lint](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/schema_lint) - Schema lint passed
* [schema_lint](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/schema_lint) - Schema title + description lint passed
* [schema_lint](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/schema_lint) - Input mimetype lint passed: 'text/csv'
* [schema_params](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/schema_params) - Schema matched params returned from nextflow config
* [system_exit](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/system_exit) - No `System.exit` calls found
* [actions_schema_validation](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/actions_schema_validation) - Workflow validation passed: download_pipeline.yml
* [actions_schema_validation](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/actions_schema_validation) - Workflow validation passed: ci.yml
* [actions_schema_validation](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/actions_schema_validation) - Workflow validation passed: release-announcements.yml
* [actions_schema_validation](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/actions_schema_validation) - Workflow validation passed: branch.yml
* [actions_schema_validation](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/actions_schema_validation) - Workflow validation passed: linting_comment.yml
* [actions_schema_validation](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/actions_schema_validation) - Workflow validation passed: template_version_comment.yml
* [actions_schema_validation](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/actions_schema_validation) - Workflow validation passed: awstest.yml
* [actions_schema_validation](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/actions_schema_validation) - Workflow validation passed: linting.yml
* [actions_schema_validation](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/actions_schema_validation) - Workflow validation passed: awsfulltest.yml
* [actions_schema_validation](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/actions_schema_validation) - Workflow validation passed: clean-up.yml
* [actions_schema_validation](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/actions_schema_validation) - Workflow validation passed: fix-linting.yml
* [merge_markers](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/merge_markers) - No merge markers found in pipeline files
* [modules_json](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/modules_json) - Only installed modules found in `modules.json`
* [modules_structure](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/modules_structure) - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'
* [local_component_structure](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/local_component_structure) - local subworkflows directory structure is correct 'subworkflows/local/TOOL/SUBTOOL'
* [base_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/base_config) - `conf/base.config` found and not ignored.
* [modules_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/modules_config) - `conf/modules.config` found and not ignored.
* [modules_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/modules_config) - `GUNZIP_GTF` found in `conf/modules.config` and Nextflow scripts.
* [modules_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/modules_config) - `GTF_TO_TABLE` found in `conf/modules.config` and Nextflow scripts.
* [modules_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/modules_config) - `VALIDATOR` found in `conf/modules.config` and Nextflow scripts.
* [modules_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/modules_config) - `AFFY_JUSTRMA_RAW` found in `conf/modules.config` and Nextflow scripts.
* [modules_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/modules_config) - `AFFY_JUSTRMA_NORM` found in `conf/modules.config` and Nextflow scripts.
* [modules_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/modules_config) - `PROTEUS` found in `conf/modules.config` and Nextflow scripts.
* [modules_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/modules_config) - `GEOQUERY_GETGEO` found in `conf/modules.config` and Nextflow scripts.
* [modules_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/modules_config) - `DESEQ2_NORM` found in `conf/modules.config` and Nextflow scripts.
* [modules_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/modules_config) - `DESEQ2_DIFFERENTIAL` found in `conf/modules.config` and Nextflow scripts.
* [modules_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/modules_config) - `LIMMA_NORM` found in `conf/modules.config` and Nextflow scripts.
* [modules_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/modules_config) - `LIMMA_DIFFERENTIAL` found in `conf/modules.config` and Nextflow scripts.
* [modules_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/modules_config) - `CUSTOM_FILTERDIFFERENTIALTABLE` found in `conf/modules.config` and Nextflow scripts.
* [modules_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/modules_config) - `IMMUNEDECONV` found in `conf/modules.config` and Nextflow scripts.
* [modules_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/modules_config) - `GSEA_GSEA` found in `conf/modules.config` and Nextflow scripts.
* [modules_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/modules_config) - `GPROFILER2_GOST` found in `conf/modules.config` and Nextflow scripts.
* [modules_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/modules_config) - `PLOT_EXPLORATORY` found in `conf/modules.config` and Nextflow scripts.
* [modules_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/modules_config) - `PLOT_DIFFERENTIAL` found in `conf/modules.config` and Nextflow scripts.
* [modules_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/modules_config) - `SHINYNGS_APP` found in `conf/modules.config` and Nextflow scripts.
* [modules_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/modules_config) - `RMARKDOWNNOTEBOOK` found in `conf/modules.config` and Nextflow scripts.
* [modules_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/modules_config) - `MAKE_REPORT_BUNDLE` found in `conf/modules.config` and Nextflow scripts.
* [modules_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/modules_config) - `CUSTOM_MATRIXFILTER` found in `conf/modules.config` and Nextflow scripts.
* [modules_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/modules_config) - `CUSTOM_TABULARTOGSEACLS` found in `conf/modules.config` and Nextflow scripts.
* [modules_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/modules_config) - `CUSTOM_TABULARTOGSEAGCT` found in `conf/modules.config` and Nextflow scripts.
* [modules_config](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/modules_config) - `CUSTOM_TABULARTOGSEACHIP` found in `conf/modules.config` and Nextflow scripts.
* [nfcore_yml](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nfcore_yml) - Repository type in `.nf-core.yml` is valid: `pipeline`
* [nfcore_yml](https://nf-co.re/tools/docs/3.2.0/pipeline_lint_tests/nfcore_yml) - nf-core version in `.nf-core.yml` is set to the latest version: `3.2.0`

### Run details

* nf-core/tools version 3.2.0
* Run at `2025-04-07 12:44:09`

</details>

<!-- Sticky Pull Request Comment -->
author_association: NONE
user: login: pinin4fjords
created_at: 2025-04-04T09:03:07Z
body: > This sounds a whole lot like nextflow profiles. With the difference being that multiple "profiles" can be run in a single pipeline invokation?

Yep, pretty much.

> I'm also still trying to understand how the toolsheet would look like for my use-cases. If I want to run RNA-seq analysis with limma/voom + apply different functional analysis tools (e.g. gsea on gene ontology + decoupler on 3 different signatures), would that look somewhat like
> 
> ```
> analysis_name,study_type,differential_method,limma_use_voom,functional_method,signature_file
> limma_rnaseq,rnaseq,limma,true,gsea,gene_ontology.gmt
> limma_rnaseq,rnaseq,limma,true,decoupler,progeny.tsv
> limma_rnaseq,rnaseq,limma,true,decoupler,collectri.tsv
> limma_rnaseq,rnaseq,limma,true,decoupler,hallmark.tsv
> ```
> 
> ?
> 
> Would that then run limma four times with the same settings, or run it once and reuse the results for the different functional analyses?

As coded in this first pass, it would run Limma 4x, because the `meta` for the process will be different. Once solution I'm thinking of will be to use subsets of the metadata to collapse inputs for certain steps (using a groupTuple), and then re-expand afterwards (using a `join()` and `transpose()`) so we maintain the association between results and paramsets without duplicating work. That will require us to maintain a dictionary of the params pertinent to the various steps. I'll add something to illustrate that today.

This is related to what @suzannejin is proposing in separating differential, functional params etc, but I think we can implement it more cleanly in this setup. 








author_association: MEMBER
user: login: grst
created_at: 2025-04-04T09:16:38Z
body: > As coded in this first pass, it would run Limma 4x, because the meta for the process will be different. Once solution I'm thinking of will be to use subsets of the metadata to collapse inputs for certain steps (using a groupTuple), and then re-expand afterwards (using a join() and transpose()) so we maintain the association between results and paramsets without duplicating work. That will require us to maintain a dictionary of the params pertinent to the various steps. I'll add something to illustrate that today.

One other approach could be to not fundamentally distinguish between differential and functional methods, i.e. both are just methods and use an input value which can either be the input matrix, or the output of another matrix. This would also accomodate that certain functional methods like decoupler can be executed both, on the gene expression matrix, and the differential expression results. 


```bash
analysis_name,method,source,limma_use_voom,signature_file
# run limma on gene expression matrix
limma_rnaseq,limma,GEX_MATRIX,true,
# run decoupler on limma result. `source` references another "analysis name"
decoupler_rnaseq_de_progeny,decoupler,limma_rnaseq,,progeny.tsv
decoupler_rnaseq_de_collectri,decoupler,limma_rnaseq,,collectri.tsv
# run decoupler on gene expression matrix
decoupler_rnaseq_progeny,decoupler,GEX_MATRIX,,progeny.tsv
# run limma on decoupler results (if decoupler is run on GEX matrix, it generates separate values for each sample, we still need to do stats)
limma_progeny,limma,decoupler_rnaseq_progeny,false,
```



author_association: MEMBER
user: login: pinin4fjords
created_at: 2025-04-04T09:26:00Z
body: > > As coded in this first pass, it would run Limma 4x, because the meta for the process will be different. Once solution I'm thinking of will be to use subsets of the metadata to collapse inputs for certain steps (using a groupTuple), and then re-expand afterwards (using a join() and transpose()) so we maintain the association between results and paramsets without duplicating work. That will require us to maintain a dictionary of the params pertinent to the various steps. I'll add something to illustrate that today.
> 
> One other approach could be to not fundamentally distinguish between differential and functional methods, i.e. both are just methods and use an input value which can either be the input matrix, or the output of another matrix. This would also accomodate that certain functional methods like decoupler can be executed both, on the gene expression matrix, and the differential expression results.
> 
> ```shell
> analysis_name,method,source,limma_use_voom,signature_file
> # run limma on gene expression matrix
> limma_rnaseq,limma,GEX_MATRIX,true,
> # run decoupler on limma result. `source` references another "analysis name"
> decoupler_rnaseq_de_progeny,decoupler,limma_rnaseq,,progeny.tsv
> decoupler_rnaseq_de_collectri,decoupler,limma_rnaseq,,collectri.tsv
> # run decoupler on gene expression matrix
> decoupler_rnaseq_progeny,decoupler,GEX_MATRIX,,progeny.tsv
> # run limma on decoupler results (if decoupler is run on GEX matrix, it generates separate values for each sample, we still need to do stats)
> limma_progeny,limma,decoupler_rnaseq_progeny,false,
> ```

That's moving us back to a 'special thing' again, rather than a paramter set, which is Suzanne's solution, and adds a whole bunch of overheads, complex joins etc, which I really don't like.

By constraining this to be a simple parameter set, it's easy to understand and we open up a whole bunch of other use cases. We just have to mitigate the impact on repeated operations. 
author_association: MEMBER
user: login: grst
created_at: 2025-04-04T09:33:36Z
body: > That's moving us back to a 'special thing' again, rather than a paramter set

not necessarily, it just depends on how you define the parameters. E.g. the "source" would correspond to the `--matrix` flag with the only difference that it can reference another `analysis_name` in addition to a TSV/CSV file. Everything else could be standard parameters as fas as i can tell. 

> and adds a whole bunch of overheads, complex joins etc, which I really don't like.

this is probably the case, I haven't really thought it through how bad it would be. 
author_association: MEMBER
user: login: grst
created_at: 2025-04-04T11:09:01Z
body: >> and adds a whole bunch of overheads, complex joins etc, which I really don't like.
> 
> this is probably the case, I haven't really thought it through how bad it would be.

Given the *constraint that only two methods can be chained*, I think it would not be that bad. 

(Very simplified) Pseudocode
```
toolsheet_step1 = ch_toolsheet.filter( it['input'] == "INPUT_MATRIX" }
toolsheet_step2 = ch_toolsheet.filter( it['input'] != "INPUT_MATRIX }

result_step1 = apply_tools(toolsheet_step1)
result_step2 = apply_tools(toolsheet_step2.join(result_step1, by={ x, y => x["tool_name"] == y["input"] }))
```


author_association: MEMBER
user: login: suzannejin
created_at: 2025-04-04T14:45:14Z
body: 


> > This sounds a whole lot like nextflow profiles. With the difference being that multiple "profiles" can be run in a single pipeline invokation?
> 
> Yep, pretty much.
> 
> > I'm also still trying to understand how the toolsheet would look like for my use-cases. If I want to run RNA-seq analysis with limma/voom + apply different functional analysis tools (e.g. gsea on gene ontology + decoupler on 3 different signatures), would that look somewhat like
> > ```
> > analysis_name,study_type,differential_method,limma_use_voom,functional_method,signature_file
> > limma_rnaseq,rnaseq,limma,true,gsea,gene_ontology.gmt
> > limma_rnaseq,rnaseq,limma,true,decoupler,progeny.tsv
> > limma_rnaseq,rnaseq,limma,true,decoupler,collectri.tsv
> > limma_rnaseq,rnaseq,limma,true,decoupler,hallmark.tsv
> > ```
> > 
> > 
> >     
> >       
> >     
> > 
> >       
> >     
> > 
> >     
> >   
> > ?
> > Would that then run limma four times with the same settings, or run it once and reuse the results for the different functional analyses?
> 
> As coded in this first pass, it would run Limma 4x, because the `meta` for the process will be different. Once solution I'm thinking of will be to use subsets of the metadata to collapse inputs for certain steps (using a groupTuple), and then re-expand afterwards (using a `join()` and `transpose()`) so we maintain the association between results and paramsets without duplicating work. That will require us to maintain a dictionary of the params pertinent to the various steps. I'll add something to illustrate that today.
> 
> This is related to what @suzannejin is proposing in separating differential, functional params etc, but I think we can implement it more cleanly in this setup.

Exactly. One can achieve that by playing a bit with the joining/extension of channels based on the common paramset.

A downside, however, is that when a distinguishable row id (ie. analysis_name) is present inside the meta (and hence having different meta will run limma 4x), one can always save the output based on that id, otherwise the alternative is to save the output based on the row paramset, which can become not very readable for the user. Although I guess this can also be solved by not saving with publishdir at each module level, but use a final gather module to gather all the files generated for a given paramset row, and save the files at the end using the analysis_name as id. 
author_association: NONE
user: login: suzannejin
created_at: 2025-04-04T15:05:38Z
body: > > As coded in this first pass, it would run Limma 4x, because the meta for the process will be different. Once solution I'm thinking of will be to use subsets of the metadata to collapse inputs for certain steps (using a groupTuple), and then re-expand afterwards (using a join() and transpose()) so we maintain the association between results and paramsets without duplicating work. That will require us to maintain a dictionary of the params pertinent to the various steps. I'll add something to illustrate that today.
> 
> One other approach could be to not fundamentally distinguish between differential and functional methods, i.e. both are just methods and use an input value which can either be the input matrix, or the output of another matrix. This would also accomodate that certain functional methods like decoupler can be executed both, on the gene expression matrix, and the differential expression results.
> 
> ```shell
> analysis_name,method,source,limma_use_voom,signature_file
> # run limma on gene expression matrix
> limma_rnaseq,limma,GEX_MATRIX,true,
> # run decoupler on limma result. `source` references another "analysis name"
> decoupler_rnaseq_de_progeny,decoupler,limma_rnaseq,,progeny.tsv
> decoupler_rnaseq_de_collectri,decoupler,limma_rnaseq,,collectri.tsv
> # run decoupler on gene expression matrix
> decoupler_rnaseq_progeny,decoupler,GEX_MATRIX,,progeny.tsv
> # run limma on decoupler results (if decoupler is run on GEX matrix, it generates separate values for each sample, we still need to do stats)
> limma_progeny,limma,decoupler_rnaseq_progeny,false,
> ```

Actually, even the currently already implemented code has two main chain routes, which assembles to the cases you mention:
1. differential -> de results -> functional
2. (normalization or not) -> matrix -> functional
The first is the case for deseq2+gprofiler2 for example, and the second for deseq2+gsea

What about making this two chain route options explicit?
Something like:
```
analysis_name,study_type,matrix_method,differential_method,functional_method,functional_input,limma_use_voom
deseq2_gprofiler2,rnaseq,,deseq2,gprofiler2,differential,  # get de results from deseq2_diff and run gprofiler2
deseq2_gsea,rnaseq,deseq2,,gsea,matrix,                    # get normalized matrix from deseq2_norm and run gsea
limma_gsea_affy,affy,norm,,gsea,matrix,false               # get default normalized matrix from validator, and run gsea
limma_decoupler,rnaseq,,limma,decoupler,differential,true  # run decoupler on limma-voom results
decoupler,rnaseq,raw,,decoupler,matrix,                    # run decoupler on input gene expression data
```
author_association: NONE
user: login: suzannejin
created_at: 2025-04-04T15:18:19Z
body: > Hi @pinin4fjords and @suzannejin,
> 
> I like the direction this is going!
> 
> > I propose we simplify this by making the toolsheet directly represent pipeline parameters.
> > Each row represents a complete set of pipeline parameters
> 
> This sounds a whole lot like nextflow profiles. With the difference being that multiple "profiles" can be run in a single pipeline invokation?
> 

Exactly! Although, another thing that we would like to explore at some point is the possibility of integrating methods of the same class (for example different DE results, along the line of this: https://pmc.ncbi.nlm.nih.gov/articles/PMC11614847/). Then the value of multiply running methods would go beyond simply parallelize at run time, or beyond benchmark individual methods.
author_association: NONE
user: login: pinin4fjords
created_at: 2025-04-04T17:05:24Z
body: [7996f76](https://github.com/nf-core/differentialabundance/pull/448/commits/7996f764468f76b45a723f3738cd54684b64871d) shows what I mean with the grouping and transposing.

Some improvement possible, but shows how I think this should go. A function can be used to simplify metadata dynamically before certain operations, alllowing the use of groupTuple to collapse channels and prevent duplicated analysis. The deduplicated metas store the 'analysis_name' values, so a transpose operation can be used to restore channel structure after the fact. 

This is still a lot clearer I think



author_association: MEMBER
user: login: grst
created_at: 2025-04-07T12:31:49Z
body:  
> Actually, even the currently already implemented code has two main chain routes, which assembles to the cases you mention:
> 
> 1. differential -> de results -> functional
> 2. (normalization or not) -> matrix -> functional
>    The first is the case for deseq2+gprofiler2 for example, and the second for deseq2+gsea
> 
> What about making this two chain route options explicit? Something like:
> 
> ```
> analysis_name,study_type,matrix_method,differential_method,functional_method,functional_input,limma_use_voom
> deseq2_gprofiler2,rnaseq,,deseq2,gprofiler2,differential,  # get de results from deseq2_diff and run gprofiler2
> deseq2_gsea,rnaseq,deseq2,,gsea,matrix,                    # get normalized matrix from deseq2_norm and run gsea
> limma_gsea_affy,affy,norm,,gsea,matrix,false               # get default normalized matrix from validator, and run gsea
> limma_decoupler,rnaseq,,limma,decoupler,differential,true  # run decoupler on limma-voom results
> decoupler,rnaseq,raw,,decoupler,matrix,                    # run decoupler on input gene expression data
> ```

That could work, although I could also see it being useful to run the differential expression on the result of the functional analyses. In that case, we'd need a `differential_input` column too. 


Note that given the number of possible arguments, redundancies in the toolsheet could be a lot uglier than the example above. There's ~50 possible parameters that can be passed to differential methods at the moment. 
author_association: MEMBER
user: login: pinin4fjords
created_at: 2025-04-07T12:45:09Z
body: > > Actually, even the currently already implemented code has two main chain routes, which assembles to the cases you mention:
> > 
> > 1. differential -> de results -> functional
> > 2. (normalization or not) -> matrix -> functional
> >    The first is the case for deseq2+gprofiler2 for example, and the second for deseq2+gsea
> > 
> > What about making this two chain route options explicit? Something like:
> > ```
> > analysis_name,study_type,matrix_method,differential_method,functional_method,functional_input,limma_use_voom
> > deseq2_gprofiler2,rnaseq,,deseq2,gprofiler2,differential,  # get de results from deseq2_diff and run gprofiler2
> > deseq2_gsea,rnaseq,deseq2,,gsea,matrix,                    # get normalized matrix from deseq2_norm and run gsea
> > limma_gsea_affy,affy,norm,,gsea,matrix,false               # get default normalized matrix from validator, and run gsea
> > limma_decoupler,rnaseq,,limma,decoupler,differential,true  # run decoupler on limma-voom results
> > decoupler,rnaseq,raw,,decoupler,matrix,                    # run decoupler on input gene expression data
> > ```
> 
> That could work, although I could also see it being useful to run the differential expression on the result of the functional analyses. In that case, we'd need a `differential_input` column too.
> 
> Note that given the number of possible arguments, redundancies in the toolsheet could be a lot uglier than the example above. There's ~50 possible parameters that can be passed to differential methods at the moment.

Yeah, I REALLY think a direct mapping from toolsheet columns to parameters is the simplest path forwards right now, even if we can refine the backend parts a bit.
author_association: MEMBER
user: login: grst
created_at: 2025-04-07T12:58:08Z
body: > Yeah, I REALLY think a direct mapping from toolsheet columns to parameters is the simplest path forwards right now, even if we can refine the backend parts a bit.

I don't disagree with this. Both the suggestion to add a `input` column or a `functional_input` column are fully compatible with being a parameter. 
author_association: MEMBER
user: login: pinin4fjords
created_at: 2025-04-07T13:00:03Z
body: > > Yeah, I REALLY think a direct mapping from toolsheet columns to parameters is the simplest path forwards right now, even if we can refine the backend parts a bit.
> 
> I don't disagree with this. Both the suggestion to add a `input` column or a `functional_input` column are fully compatible with being a parameter.

Ahh yes, see what you mean now
author_association: MEMBER
user: login: pinin4fjords
created_at: 2025-04-07T13:27:35Z
body: @suzannejin I'm going to need to put most of my time on other projects soon, which is going to slow me down on this. Would you be up for developing on from here? 

You can merge this into your branch and go from there, or just apply the same philosophy if you prefer. This branch does kind of work (though you have to turn off `immundeconv` until we fix that), it just needs the last bits re-added to do with the reporting etc.

Key things from this POC I think we need to keep:

1. Everything from channels via `filter` etc (no `if (params.x`). 
2. modules.config just pulls from meta.param rather than params.param
3. toolsheets represent single iterations of pipeline configuration - no 'special' columns

I can come back and refine a bit, but I'll be slower, so depends how hurried you are!





author_association: MEMBER
user: login: suzannejin
created_at: 2025-04-07T13:52:53Z
body: > @suzannejin I'm going to need to put most of my time on other projects soon, which is going to slow me down on this. Would you be up for developing on from here?
> 
> You can merge this into your branch and go from there, or just apply the same philosophy if you prefer. This branch does kind of work (though you have to turn off `immundeconv` until we fix that), it just needs the last bits re-added to do with the reporting etc.
> 
> Key things from this POC I think we need to keep:
> 
> 1. Everything from channels via `filter` etc (no `if (params.x`).
> 2. modules.config just pulls from meta.param rather than params.param
> 3. toolsheets represent single iterations of pipeline configuration - no 'special' columns
> 
> I can come back and refine a bit, but I'll be slower, so depends how hurried you are!

Sure! Thank you for the work! I can take the relay baton ;) 
Actually like this dynamic :)
author_association: NONE
user: login: pinin4fjords
created_at: 2025-04-08T08:11:38Z
body: I saw you approved @suzannejin - do you want this merged?
author_association: MEMBER