Repository: INCATools/ontology-development-kit
Pull request #1194: Simply the "Imports" section in the Makefile template
Author: login: gouttegd
Created: 2025-03-04T19:14:55Z | Merged: 2025-03-25T12:43:31Z

This PR attempts to make the "Imports" section in `Makefile.jinja2` easier to read and maintain.

First, it moves some of the logic into the `odk.py` script. Notably, the script now takes care of automatically assigning the default module type to all individual modules that do not have an explicit `module_type` parameter, so that we do not have to go fetch the default value in the template. That is, instead of having to do things like that:

```Jinja2
{% if ('slme' == ont.module_type) or (ont.module_type is none and 'slme' == project.import_group.module_type) %}
```

(testing if the module is a SLME module _or_ if it does not have an explicit type and the default module type is SLME), we can do simply:

```Jinja2
{% if 'slme' == ont.module_type %}
```

That change alone should reduce the risks of blood clots forming in the brain of ODK developers trying to read the template by at least a few percent.

Second, it removes rules that were generated for nothing: when `use_base_merging` is used, there is no need to generate rules for the individual imports (see #1189).

Third, it removes the “fast track” for “large“ SLME modules. This was a special rule invoked for (1) SLME modules marked as “large” or (2) module using the (undocumented) type `fast_slme`. The only difference compared to the standard SLME rule is that the “preprocessing step” (removing original ontology annotations and inserting a `dc:source` annotation instead) is skipped in the fast track. This was because the preprocessing step used to be performed with SPARQL queries, which was memory-intensive for large modules. That step is now performed by the ODK plugin using pure Java code, and has no excessive memory requirement. Therefore, it is no longer needed to treat large modules differently.

Fourth, and though it is not really a “simplification” (on the contrary, it’s a new feature), it adds the possibility of disabling the extraction of a seed from the -edit file (option `scan_signature = False`) (#638).

Fifth, it re-formats the template according to a few rules:

* break all lines so that no lines is longer than 100 characters (max) or 72 characters (preferred);
* consistently use long option names (e.g., `--input` instead of `-i`, `--term-file` instead of `-T`, etc);
* indent Jinja2 conditionals;
* add comments at the end of conditional blocks to help identify which block is closed (e.g., `{% endif %}{# !use_base_merging -#}`).


Diff:
diff --git a/odk/odk.py b/odk/odk.py
index 5f1293c3..1574d4bd 100755
--- a/odk/odk.py
+++ b/odk/odk.py
@@ -290,6 +290,11 @@ def fill_missing(self):
             for id in self.ids:
                 if id not in [p.id for p in self.products]:
                     self._add_stub(id)
+        self._derive_fields()
+
+    def _derive_fields(self):
+        pass
+
 @dataclass_json
 @dataclass
 class SubsetGroup(ProductGroup):
@@ -364,11 +369,36 @@ class ImportGroup(ProductGroup):
        When imports are not merged, the annotation is added during the release process to the full release artefact.
     """
 
+    scan_signature : bool = True
+    """If true, the edit file is scanned for additional terms to import.
+       Otherwise, imports are seeded solely from the manually maintained
+       *_terms.txt files. Note that setting this option to False makes
+       Protégé-based declarations of terms to import impossible.
+    """
+
     def _add_stub(self, id : OntologyHandle):
         if self.products is None:
             self.products = []
         self.products.append(ImportProduct(id=id))
 
+    def _derive_fields(self):
+        self.special_products = []
+        for p in self.products:
+            if p.module_type is None:
+                # Use group-level parameters
+                p.module_type = self.module_type
+                p.module_type_slme = self.module_type_slme
+                p.slme_individuals = self.slme_individuals
+            elif p.module_type == 'fast_slme':
+                # Accept fast_slme as a synonym for slme, for backwards
+                # compatibility
+                p.module_type = 'slme'
+            if p.base_iris is None:
+                p.base_iris = [ 'http://purl.obolibrary.org/obo/' + p.id.upper() ]
+            if p.is_large or p.module_type != self.module_type:
+                # This module will require a distinct rule
+                self.special_products.append(p)
+
 @dataclass_json
 @dataclass
 class ReportConfig(JsonSchemaMixin):
diff --git a/template/src/ontology/Makefile.jinja2 b/template/src/ontology/Makefile.jinja2
index 2705525f..90078a0e 100644
--- a/template/src/ontology/Makefile.jinja2
+++ b/template/src/ontology/Makefile.jinja2
@@ -447,21 +447,23 @@ $(REPORTDIR)/release-diff.md: $(ONT).owl $(TMPDIR)/current-release.owl
 # Imports: Seeding system 
 # ------------------------
 
-# seed.txt contains all referenced entities
-IMPORTSEED=$(TMPDIR)/seed.txt
+{% if project.import_group.scan_signature -%}
+# pre_seed.txt contains all entities referenced from the -edit file
+# and its components
 PRESEED=$(TMPDIR)/pre_seed.txt
+$(PRESEED): $(SRCMERGED)
+	$(ROBOT) query --input $< --format --csv \
+		       --query $(SPARQLDIR)/terms.sparql $@
 
+{% endif -%}
 $(SRCMERGED): $(EDIT_PREPROCESSED) $(OTHER_SRC)
 	$(ROBOT) remove --input $< --select imports --trim false \
-		merge  $(patsubst %, -i %, $(OTHER_SRC)) -o $@
+		 merge $(foreach src, $(OTHER_SRC), --input $(src)) \
+		       --output $@
 
 $(EDIT_PREPROCESSED): $(SRC)
 	$(ROBOT) convert --input $< --format ofn --output $@
 
-$(PRESEED): $(SRCMERGED)
-	$(ROBOT) query -f csv -i $< --query ../sparql/terms.sparql $@.tmp &&\
-	cat $@.tmp | sort | uniq >  $@
-
 {% if 'basic' in project.release_artefacts or 'simple' in project.release_artefacts or 'simple-non-classified' in project.release_artefacts or project.primary_release == 'basic' or project.primary_release == 'simple-non-classified' or project.primary_release == 'simple' -%}
 SIMPLESEED=$(TMPDIR)/simple_seed.txt
 
@@ -470,8 +472,9 @@ $(SIMPLESEED): $(SRCMERGED) $(ONTOLOGYTERMS)
 	cat $@.tmp $(ONTOLOGYTERMS) | sort | uniq >  $@ &&\
 	echo "http://www.geneontology.org/formats/oboInOwl#SubsetProperty" >> $@ &&\
 	echo "http://www.geneontology.org/formats/oboInOwl#SynonymTypeProperty" >> $@
+
 {% endif -%}
-{% if project.use_custom_import_module %} 
+{% if project.use_custom_import_module -%}
 IMPORT_MODULE_TEMPLATE=$(TEMPLATEDIR)/external_import.tsv
 IMPORT_MODULE_SIGNATURE=$(TMPDIR)/external_import_terms.txt
 IMPORT_MODULE=$(IMPORTDIR)/external_import.owl
@@ -484,13 +487,23 @@ $(IMPORT_MODULE): $(IMPORT_MODULE_TEMPLATE) | $(TMPDIR)
 $(IMPORT_MODULE_SIGNATURE): $(IMPORT_MODULE) | $(TMPDIR)
 	$(ROBOT) query -f csv -i $< --query ../sparql/terms.sparql $@.tmp &&\
 	cat $@.tmp | sort | uniq >  $@
-{% endif %}
 
-ALLSEED = $(PRESEED) {% if project.use_dosdps %} $(TMPDIR)/all_pattern_terms.txt {% endif %}\
-{% if project.use_custom_import_module %} $(IMPORT_MODULE_SIGNATURE) {% endif %}
+{% endif -%}
+
+{% if project.import_group.scan_signature or project.use_custom_import_module -%}
+# seed.txt contains all entities to import in addition to those defined
+# in the individual _terms.txt files.
+IMPORTSEED = $(TMPDIR)/seed.txt
+$(IMPORTSEED):
+{%-  if project.import_group.scan_signature %} $(PRESEED)
+{%-    if project.use_dosdps %} $(TMPDIR)/all_pattern_terms.txt{% endif -%}
+{%-  endif -%}
+{%-if project.use_custom_import_module %} $(IMPORT_MODULE_SIGNATURE){% endif %} | $(TMPDIR)
+	cat $^ | sort | uniq > $@
+
+T_IMPORTSEED = --term-file $(IMPORTSEED)
 
-$(IMPORTSEED): $(ALLSEED) | $(TMPDIR)
-	if [ $(IMP) = true ]; then cat $(ALLSEED) | sort | uniq > $@; fi
+{% endif -%}
 
 {% if project.import_group is defined -%}
 ANNOTATION_PROPERTIES={% for p in project.import_group.annotation_properties %}{{ p }} {% endfor %}
@@ -503,122 +516,190 @@ ANNOTATION_PROPERTIES={% for p in project.import_group.annotation_properties %}{
 # This pattern uses ROBOT to generate an import module
 
 ifeq ($(IMP),true)
-# Should be able to drop this if robot can just take a big messy list of terms as input.
-$(IMPORTDIR)/%_terms_combined.txt: $(IMPORTSEED) $(IMPORTDIR)/%_terms.txt
-	cat $^ | grep -v ^# | sort | uniq >  $@
 
-{% if project.import_group.use_base_merging %}
-ALL_TERMS_COMBINED = $(patsubst %, $(IMPORTDIR)/%_terms_combined.txt, $(IMPORTS))
-$(IMPORTDIR)/merged_terms_combined.txt: $(ALL_TERMS_COMBINED)
-	cat $^ | grep -v ^# | sort | uniq >  $@
-
-{% if 'slme' == project.import_group.module_type %}
-$(IMPORTDIR)/merged_import.owl: $(MIRRORDIR)/merged.owl $(IMPORTDIR)/merged_terms_combined.txt | all_robot_plugins
-	$(ROBOT) merge -i $< \{% if project.import_group.exclude_iri_patterns is not none %}
-		{% for pattern in project.import_group.exclude_iri_patterns %}remove  --select "{{ pattern }}" {% endfor %} \{% endif %}
-		extract -T $(IMPORTDIR)/merged_terms_combined.txt --force true --copy-ontology-annotations true --individuals {{ project.import_group.slme_individuals }} --method {{ project.import_group.module_type_slme }} \
-		remove $(patsubst %, --term %, $(ANNOTATION_PROPERTIES)) -T $(IMPORTDIR)/merged_terms_combined.txt --select complement --select annotation-properties \
-		odk:normalize --base-iri {{ project.uribase }} --subset-decls true --synonym-decls true --merge-axioms true \
-		$(ANNOTATE_CONVERT_FILE)
-{% else %}
-$(IMPORTDIR)/merged_import.owl: $(MIRRORDIR)/merged.owl $(IMPORTDIR)/merged_terms_combined.txt
-	echo "ERROR: You have configured your default module type to be {{project.import_group.module_type}}; this behavior needs to be overwritten in {{ project.id }}.Makefile!" && false
-{% endif %}
-{% endif %}
-
-{% if 'slme' == project.import_group.module_type %}
-$(IMPORTDIR)/%_import.owl: {% if project.import_group.use_base_merging %}$(MIRRORDIR)/merged.owl{% else %}$(MIRRORDIR)/%.owl{% endif %} $(IMPORTDIR)/%_terms_combined.txt | all_robot_plugins
-	$(ROBOT) annotate -i $< --remove-annotations \
-		odk:normalize --add-source true \
-		extract -T $(IMPORTDIR)/$*_terms_combined.txt --force true --copy-ontology-annotations false --individuals {{ project.import_group.slme_individuals }} --method {{ project.import_group.module_type_slme }} \
-		odk:normalize --base-iri {{ project.uribase }} --subset-decls true --synonym-decls true --merge-axioms true \
-		$(ANNOTATE_CONVERT_FILE)
-{% elif 'minimal' == project.import_group.module_type %}
-$(IMPORTDIR)/%_import.owl: {% if project.import_group.use_base_merging %}$(MIRRORDIR)/merged.owl{% else %}$(MIRRORDIR)/%.owl{% endif %} $(IMPORTDIR)/%_terms_combined.txt | all_robot_plugins
-	$(ROBOT) annotate -i $< --remove-annotations \
-		odk:normalize --add-source true \
-		extract -T $(IMPORTDIR)/$*_terms_combined.txt --force true --copy-ontology-annotations true --method BOT \
-		remove --base-iri $(OBOBASE)"/$(shell echo $* | tr a-z A-Z)_" --axioms external --preserve-structure false --trim false \
-		odk:normalize --base-iri {{ project.uribase }} --subset-decls true --synonym-decls true --merge-axioms true \
-		remove $(patsubst %, --term %, $(ANNOTATION_PROPERTIES)) -T $(IMPORTDIR)/$*_terms_combined.txt --select complement --select "classes individuals annotation-properties" \
-		$(ANNOTATE_CONVERT_FILE)
-{% elif 'mirror' == project.import_group.module_type %}
-$(IMPORTDIR)/%_import.owl: {% if project.import_group.use_base_merging %}$(MIRRORDIR)/merged.owl{% else %}$(MIRRORDIR)/%.owl{% endif %} $(IMPORTDIR)/%_terms_combined.txt | all_robot_plugins
-	$(ROBOT) annotate -i $< --remove-annotations \
-		odk:normalize --base-iri {{ project.uribase }} --subset-decls true --synonym-decls true --merge-axioms true --add-source true \
-		$(ANNOTATE_CONVERT_FILE)
-{% elif 'filter' == project.import_group.module_type %}
-$(IMPORTDIR)/%_import.owl: {% if project.import_group.use_base_merging %}$(MIRRORDIR)/merged.owl{% else %}$(MIRRORDIR)/%.owl{% endif %} $(IMPORTDIR)/%_terms_combined.txt | all_robot_plugins
-	$(ROBOT) merge -i $< \
-		annotate --remove-annotations \
-		odk:normalize --add-source true \
-		remove --base-iri $(OBOBASE)"/$(shell echo $* | tr a-z A-Z)_" --axioms external --preserve-structure false --trim false \
-		remove $(patsubst %, --term %, $(ANNOTATION_PROPERTIES)) -T $(IMPORTDIR)/$*_terms_combined.txt --select complement \
-		odk:normalize --base-iri {{ project.uribase }} --subset-decls true --synonym-decls true --merge-axioms true \
-		$(ANNOTATE_CONVERT_FILE)
-{% elif 'custom' == project.import_group.module_type %}
-$(IMPORTDIR)/%_import.owl: {% if project.import_group.use_base_merging %}$(MIRRORDIR)/merged.owl{% else %}$(MIRRORDIR)/%.owl{% endif %}
-	echo "ERROR: You have configured your default module type to be custom; this behavior needs to be overwritten in {{ project.id }}.Makefile!" && false
-{% endif %}
+{%   if project.import_group.use_base_merging -%}
+ALL_TERMS = $(foreach imp, $(IMPORTS), $(IMPORTDIR)/$(imp)_terms.txt)
+
+{%     if 'slme' == project.import_group.module_type -%}
+$(IMPORTDIR)/merged_import.owl: $(MIRRORDIR)/merged.owl $(ALL_TERMS) \
+				$(IMPORTSEED) | all_robot_plugins
+	$(ROBOT) merge --input $<{% if project.import_group.exclude_iri_patterns is not none -%}
+		 {%- for pattern in project.import_group.exclude_iri_patterns %} \
+		 remove --select "{{ pattern }}"{% endfor %}{% endif %} \
+		 extract $(foreach f, $(ALL_TERMS), --term-file $(f)) $(T_IMPORTSEED) \
+		         --force true --copy-ontology-annotations false \
+		         --individuals {{ project.import_group.slme_individuals }} \
+		         --method {{ project.import_group.module_type_slme }} \
+		 remove $(foreach p, $(ANNOTATION_PROPERTIES), --term $(p)) \
+		        $(foreach f, $(ALL_TERMS), --term-file $(f)) $(T_IMPORTSEED) \
+		        --select complement --select annotation-properties \
+		 odk:normalize --base-iri {{ project.uribase }} \
+		               --subset-decls true --synonym-decls true \
+		               --merge-axioms true \
+		 $(ANNOTATE_CONVERT_FILE)
+
+{%     else -%}
+$(IMPORTDIR)/merged_import.owl: $(MIRRORDIR)/merged.owl $(ALL_TERMS) \
+				$(IMPORTSEED)
+	@echo "ERROR: You have configured the merged import as a custom module;"
+	@echo "       This rule needs to be overwritten in {{ project.id }}.Makefile!"
+	@false
+
+{%     endif -%}
+{%   else %}{# !project.import_group.use_base_merging -#}
+
+## Default module type ({{ project.import_group.module_type }})
+{%     if 'slme' == project.import_group.module_type -%}
+$(IMPORTDIR)/%_import.owl: $(MIRRORDIR)/%.owl $(IMPORTDIR)/%_terms.txt \
+			   $(IMPORTSEED) | all_robot_plugins
+	$(ROBOT) annotate --input $< --remove-annotations \
+		 odk:normalize --add-source true \
+		 extract --term-file $(IMPORTDIR)/$*_terms.txt $(T_IMPORTSEED) \
+		         --force true --copy-ontology-annotations true \
+		         --individuals {{ project.import_group.slme_individuals }} \
+		         --method {{ project.import_group.module_type_slme }} \
+		 odk:normalize --base-iri {{ project.uribase }} \
+		               --subset-decls true --synonym-decls true \
+		               --merge-axioms true \
+		 $(ANNOTATE_CONVERT_FILE)
+
+{%     elif 'minimal' == project.import_group.module_type -%}
+$(IMPORTDIR)/%_import.owl: $(MIRRORDIR)/%.owl $(IMPORTDIR)/%_terms.txt \
+			   $(IMPORTSEED) | all_robot_plugins
+	$(ROBOT) annotate --input $< --remove-annotations \
+		 odk:normalize --add-source true \
+		 extract --term-file $(IMPORTDIR)/$*_terms.txt $(T_IMPORTSEED) \
+		         --force true --copy-ontology-annotations true \
+		         --method BOT \
+		 remove --base-iri $(OBOBASE)"/$(shell echo $* | tr a-z A-Z)_" \
+		        --axioms external \
+		        --preserve-structure false --trim false \
+		 odk:normalize --base-iri {{ project.uribase }} \
+		               --subset-decls true --synonym-decls true \
+		               --merge-axioms true \
+		 remove $(foreach p, $(ANNOTATION_PROPERTIES), --term $(p)) \
+		        --term-file $(IMPORTDIR)/$*_terms.txt $(T_IMPORTSEED) \
+		        --select complement \
+		        --select "classes individuals annotation-properties" \
+		 $(ANNOTATE_CONVERT_FILE)
+
+{%     elif 'mirror' == project.import_group.module_type -%}
+$(IMPORTDIR)/%_import.owl: $(MIRRORDIR)/%.owl | all_robot_plugins
+	$(ROBOT) annotate --input $< --remove-annotations \
+		 odk:normalize --base-iri {{ project.uribase }} \
+		               --subset-decls true --synonym-decls true \
+		               --merge-axioms true --add-source true \
+		 $(ANNOTATE_CONVERT_FILE)
+
+{%     elif 'filter' == project.import_group.module_type -%}
+$(IMPORTDIR)/%_import.owl: $(MIRRORDIR)/%.owl $(IMPORTDIR)/%_terms.txt \
+			   $(IMPORTSEED) | all_robot_plugins
+	$(ROBOT) merge --input $< \
+		 annotate --remove-annotations \
+		 odk:normalize --add-source true \
+		 remove --base-iri $(OBOBASE)"/$(shell echo $* | tr a-z A-Z)_" \
+		        --axioms external \
+		        --preserve-structure false --trim false \
+		 remove $(foreach p, $(ANNOTATION_PROPERTIES), --term $(p)) \
+		        --term-file $(IMPORTDIR)/$*_terms.txt $(T_IMPORTSEED) \
+		        --select complement \
+		 odk:normalize --base-iri {{ project.uribase }} \
+		               --subset-decls true --synonym-decls true \
+		               --merge-axioms true \
+		 $(ANNOTATE_CONVERT_FILE)
+
+{%     elif 'custom' == project.import_group.module_type -%}
+$(IMPORTDIR)/%_import.owl: $(MIRRORDIR)/%.owl
+	@echo "ERROR: You have configured the default module type to be custom;"
+	@echo "       This rule needs to be overwritten in {{ project.id }}.Makefile!"
+	@false
+
+{%     endif -%}
 .PRECIOUS: $(IMPORTDIR)/%_import.owl
-{% if project.import_group.export_obo %}
-# convert imports to obo.
-# this can be useful for spot-checks and diffs.
-# we set strict mode to false by default. For discussion see https://github.com/owlcs/owlapi/issues/752
-$(IMPORTDIR)/%_import.obo: $(IMPORTDIR)/%_import.owl
-	$(ROBOT) convert --check false -i $< -f obo -o $@.tmp.obo && mv $@.tmp.obo $@
-{% endif -%}
 
-{%- for ont in project.import_group.products -%}
-{% if ont.is_large or ont.module_type is not none %}
-## Module for ontology: {{ ont.id }}
-{% if ont.is_large -%}
+{#
+ # An import module of a different type than the default type needs its
+ # own specific rule to override the pattern rule above.
+ # A module that is marked as "large" (regardless of its type) also
+ # needs its own rule, so that it can be selectively excluded when
+ # running under IMP_LARGE=false.
+ # We generate those rules here.
+-#}
+{%     for ont in project.import_group.special_products -%}
+## Module for ontology: {{ ont.id }} ({{ ont.module_type }})
+{%       if ont.is_large -%}
 ifeq ($(IMP_LARGE),true)
-{% endif -%}
-{% if (ont.is_large and ('slme' == ont.module_type or (ont.module_type is none and 'slme' == project.import_group.module_type))) or ('fast_slme' == ont.module_type) or (ont.module_type is none and 'fast_slme' == project.import_group.module_type) %}
-$(IMPORTDIR)/{{ ont.id }}_import.owl: {% if project.import_group.use_base_merging %}$(MIRRORDIR)/merged.owl{% else %}$(MIRRORDIR)/{{ ont.id }}.owl{% endif %} $(IMPORTDIR)/{{ ont.id }}_terms_combined.txt | all_robot_plugins
-	$(ROBOT) extract -i $< -T $(IMPORTDIR)/{{ ont.id }}_terms_combined.txt --force true  --copy-ontology-annotations false --individuals {% if ont.module_type is none %}{{ project.import_group.slme_individuals }} --method {{ project.import_group.module_type_slme }}{% else %}{{ ont.slme_individuals }} --method {{ ont.module_type_slme }}{% endif %} \
-		odk:normalize --base-iri {{ project.uribase }} --subset-decls true --synonym-decls true --merge-axioms true \
-		$(ANNOTATE_CONVERT_FILE)
-{% elif ('slme' == ont.module_type) or (ont.module_type is none and 'slme' == project.import_group.module_type) %}
-$(IMPORTDIR)/{{ ont.id }}_import.owl: {% if project.import_group.use_base_merging %}$(MIRRORDIR)/merged.owl{% else %}$(MIRRORDIR)/{{ ont.id }}.owl{% endif %} $(IMPORTDIR)/{{ ont.id }}_terms_combined.txt | all_robot_plugins
-	$(ROBOT) annotate -i $< --remove-annotations \
-		odk:normalize --add-source true \
-		extract -T $(IMPORTDIR)/{{ ont.id }}_terms_combined.txt --copy-ontology-annotations true --force true --individuals {% if ont.module_type is none %}{{ project.import_group.slme_individuals }} --method {{ project.import_group.module_type_slme }}{% else %}{{ ont.slme_individuals }} --method {{ ont.module_type_slme }}{% endif %} \
-		odk:normalize --base-iri {{ project.uribase }} --subset-decls true --synonym-decls true --merge-axioms true \
-		$(ANNOTATE_CONVERT_FILE)
-{% elif ('filter' == ont.module_type) or (ont.module_type is none and 'filter' == project.import_group.module_type) %}
-$(IMPORTDIR)/{{ ont.id }}_import.owl: {% if project.import_group.use_base_merging %}$(MIRRORDIR)/merged.owl{% else %}$(MIRRORDIR)/{{ ont.id }}.owl{% endif %} $(IMPORTDIR)/{{ ont.id }}_terms_combined.txt | all_robot_plugins
-	$(ROBOT) annotate -i $< --remove-annotations \
-		odk:normalize --add-source true \
-		extract -T $(IMPORTDIR)/{{ ont.id }}_terms_combined.txt --copy-ontology-annotations true --force true --method BOT \
-		remove {% if ont.base_iris is not none %}{% for iri in ont.base_iris %}--base-iri {{ iri }} {% endfor %}{% else %}--base-iri $(OBOBASE)/{{ ont.id.upper() }} {% endif %}--axioms external --preserve-structure false --trim false \
-		remove $(patsubst %, --term %, $(ANNOTATION_PROPERTIES)) {% if ont.annotation_properties is defined %}{% for p in ont.annotation_properties %}--term {{ p }} {% endfor %}{% endif %} -T $(IMPORTDIR)/{{ ont.id }}_terms_combined.txt --select complement \
-		odk:normalize --base-iri {{ project.uribase }} --subset-decls true --synonym-decls true --merge-axioms true \
-		$(ANNOTATE_CONVERT_FILE)
-{% elif ('mirror' == ont.module_type) or (ont.module_type is none and 'mirror' == project.import_group.module_type) %}
-$(IMPORTDIR)/{{ ont.id }}_import.owl: {% if project.import_group.use_base_merging %}$(MIRRORDIR)/merged.owl{% else %}$(MIRRORDIR)/{{ ont.id }}.owl{% endif %} $(IMPORTDIR)/{{ ont.id }}_terms_combined.txt | all_robot_plugins
-	$(ROBOT) annotate -i $< --remove-annotations \
-		odk:normalize --base-iri {{ project.uribase }} --subset-decls true --synonym-decls true --merge-axioms true --add-source true \
-		$(ANNOTATE_CONVERT_FILE)
-{% elif ('minimal' == ont.module_type) or (ont.module_type is none and 'minimal' == project.import_group.module_type) %}
-$(IMPORTDIR)/{{ ont.id }}_import.owl: {% if project.import_group.use_base_merging %}$(MIRRORDIR)/merged.owl{% else %}$(MIRRORDIR)/{{ ont.id }}.owl{% endif %} $(IMPORTDIR)/{{ ont.id }}_terms_combined.txt | all_robot_plugins
-	$(ROBOT) annotate -i $< --remove-annotations \
-		odk:normalize --add-source true \
-		extract -T $(IMPORTDIR)/{{ ont.id }}_terms_combined.txt --copy-ontology-annotations true --force true --method BOT \
-		remove {% if ont.base_iris is not none %}{% for iri in ont.base_iris %}--base-iri {{iri}} {% endfor %}{% else %}--base-iri $(OBOBASE)/{{ ont.id.upper() }} {% endif %}--axioms external --preserve-structure false --trim false \
-		odk:normalize --base-iri {{ project.uribase }} --subset-decls true --synonym-decls true --merge-axioms true \
-		remove $(patsubst %, --term %, $(ANNOTATION_PROPERTIES)) {% if ont.annotation_properties is defined %}{% for p in ont.annotation_properties %}--term {{ p }} {% endfor %}{% endif %} -T $(IMPORTDIR)/{{ ont.id }}_terms_combined.txt --select complement --select "classes individuals annotation-properties" \
-		$(ANNOTATE_CONVERT_FILE)
-{% elif ('custom' == ont.module_type) or (ont.module_type is none and 'custom' == project.import_group.module_type) %}
-$(IMPORTDIR)/{{ ont.id }}_import.owl: {% if project.import_group.use_base_merging %}$(MIRRORDIR)/merged.owl{% elif 'no_mirror' != ont.mirror_type %}$(MIRRORDIR)/{{ ont.id }}.owl{% endif %}
-	echo "ERROR: You have configured your default module type to be custom; this behavior needs to be overwritten in {{ project.id }}.Makefile!" && false
-{%- endif %}
-{% if ont.is_large -%}
+{%       endif -%}
+{%       if 'slme' == ont.module_type -%}
+$(IMPORTDIR)/{{ ont.id }}_import.owl: $(MIRRORDIR)/{{ ont.id }}.owl $(IMPORTDIR)/{{ ont.id }}_terms.txt $(IMPORTSEED) | all_robot_plugins
+	$(ROBOT) annotate --input $< --remove-annotations \
+		 odk:normalize --add-source true \
+		 extract --term-file $(IMPORTDIR)/{{ ont.id }}_terms.txt $(T_IMPORTSEED) \
+		         --copy-ontology-annotations true --force true \
+		         --individuals {{ ont.slme_individuals }} \
+		         --method {{ ont.module_type_slme }} \
+		 odk:normalize --base-iri {{ project.uribase }} \
+		               --subset-decls true --synonym-decls true \
+		               --merge-axioms true \
+		 $(ANNOTATE_CONVERT_FILE)
+{%       elif 'filter' == ont.module_type -%}
+$(IMPORTDIR)/{{ ont.id }}_import.owl: $(MIRRORDIR)/{{ ont.id }}.owl $(IMPORTDIR)/{{ ont.id }}_terms.txt $(IMPORTSEED) | all_robot_plugins
+	$(ROBOT) annotate --input $< --remove-annotations \
+		 odk:normalize --add-source true \
+		 extract --term-file $(IMPORTDIR)/{{ ont.id }}_terms.txt $(T_IMPORTSEED) \
+		         --copy-ontology-annotations true --force true --method BOT \
+		 remove --axioms external --preserve-structure false --trim false \{% for iri in ont.base_iris %}
+		        --base-iri {{ iri }} \{% endfor %}
+		 remove $(foreach p, $(ANNOTATION_PROPERTIES), --term $(p)) \{% for p in ont.annotation_properties %}
+		        --term {{ p }} \{% endfor %}
+		        --term-file $(IMPORTDIR)/{{ ont.id }}_terms.txt $(T_IMPORTSEED) \
+		        --select complement \
+		 odk:normalize --base-iri {{ project.uribase }} \
+		               --subset-decls true --synonym-decls true \
+		               --merge-axioms true \
+		 $(ANNOTATE_CONVERT_FILE)
+{%       elif 'mirror' == ont.module_type -%}
+$(IMPORTDIR)/{{ ont.id }}_import.owl: $(MIRRORDIR)/{{ ont.id }}.owl | all_robot_plugins
+	$(ROBOT) annotate --input $< --remove-annotations \
+		 odk:normalize --base-iri {{ project.uribase }} \
+		               --subset-decls true --synonym-decls true \
+		               --merge-axioms true --add-source true \
+		 $(ANNOTATE_CONVERT_FILE)
+{%       elif 'minimal' == ont.module_type -%}
+$(IMPORTDIR)/{{ ont.id }}_import.owl: $(MIRRORDIR)/{{ ont.id }}.owl $(IMPORTDIR)/{{ ont.id }}_terms.txt $(IMPORTSEED) | all_robot_plugins
+	$(ROBOT) annotate --input $< --remove-annotations \
+		 odk:normalize --add-source true \
+		 extract --term-file $(IMPORTDIR)/{{ ont.id }}_terms.txt $(T_IMPORTSEED) \
+		         --copy-ontology-annotations true --force true --method BOT \
+		 remove --axioms external --preserve-structure false --trim false \{% for iri in ont.base_iris %}
+		        --base-iri {{ iri }} \{% endfor %}
+		 odk:normalize --base-iri {{ project.uribase }} --subset-decls true \
+		               --synonym-decls true --merge-axioms true \
+		 remove $(foreach p, $(ANNOTATION_PROPERTIES), --term $(p)) \{% for p in ont.annotation_properties %}
+		        --term {{ p }} \{% endfor %}
+		        --term-file $(IMPORTDIR)/{{ ont.id }}_terms.txt $(T_IMPORTSEED) \
+		        --select complement \
+		        --select "classes individual annotation-properties" \
+		 $(ANNOTATE_CONVERT_FILE)
+{%       elif 'custom' -%}
+$(IMPORTDIR)/{{ ont.id }}_import.owl: {% if 'no_mirror' != ont.mirror_type %}$(MIRRORDIR)/{{ ont.id }}.owl{% endif %}
+	@echo "ERROR: You have configured {{ ont.id }} as a custom module;"
+	@echo "       This rule needs to be overwritten in {{ project.id }}.Makefile!"
+	@false
+{%       endif -%}
+{%       if ont.is_large -%}
 endif # IMP_LARGE=true
-{% endif -%}
-{%- endif %}
-{%- endfor %}
+{%       endif %}
+{%     endfor -%}
+{%   endif %}{# !use_base_merging -#}
+
+{%   if project.import_group.export_obo -%}
+# Convert any import module to OBO (this can be useful for spot-checks
+# and diffs).
+$(IMPORTDIR)/%_import.obo: $(IMPORTDIR)/%_import.owl
+	$(ROBOT) convert --input $< --check false --format obo --output $@
+
+{%   endif -%}
 endif # IMP=true
 
 .PHONY: refresh-imports
@@ -640,7 +721,7 @@ refresh-%:
 .PHONY: no-mirror-refresh-%
 no-mirror-refresh-%:
 	$(MAKE) IMP=true IMP_LARGE=true MIR=false PAT=false $(IMPORTDIR)/$*_import.owl -B
-{%- endif %}
+{%- endif %}{# !project.import_group is defined #}
 
 {% if project.components is not none %}
 # ----------------------------------------


Reviews:
user: login: matentzn
body: I went through the code line by line and LGTM; I made some mostly irrelevant comments, and currently running some tests. This may take a bit, but just wanted to ensure you I am working in it :)
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-13T07:43:15Z
user: login: gouttegd
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-13T09:47:27Z
user: login: gouttegd
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-13T09:47:48Z
user: login: gouttegd
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-13T10:02:20Z
user: login: matentzn
body: I tested with three ontologies I care about (HPO, RO and CHR) and it the behaviour looks fine to me. Awesome! THANK you for this!
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-25T12:30:56Z

Review comments:
diff_hunk: @@ -484,13 +487,23 @@ $(IMPORT_MODULE): $(IMPORT_MODULE_TEMPLATE) | $(TMPDIR)
 $(IMPORT_MODULE_SIGNATURE): $(IMPORT_MODULE) | $(TMPDIR)
 	$(ROBOT) query -f csv -i $< --query ../sparql/terms.sparql $@.tmp &&\
 	cat $@.tmp | sort | uniq >  $@
-{% endif %}
 
-ALLSEED = $(PRESEED) {% if project.use_dosdps %} $(TMPDIR)/all_pattern_terms.txt {% endif %}\
-{% if project.use_custom_import_module %} $(IMPORT_MODULE_SIGNATURE) {% endif %}
+{% endif -%}
+
+{% if project.import_group.scan_signature or project.use_custom_import_module -%}
+# seed.txt contains all entities to import in addition to those defined
+# in the individual _terms.txt files.
+IMPORTSEED = $(TMPDIR)/seed.txt
+$(IMPORTSEED):
+{%-  if project.import_group.scan_signature %} $(PRESEED)
+{%-    if project.use_dosdps %} $(TMPDIR)/all_pattern_terms.txt{% endif -%}
+{%-  endif -%}
+{%-if project.use_custom_import_module %} $(IMPORT_MODULE_SIGNATURE){% endif %} | $(TMPDIR)
+	cat $^ | sort | uniq > $@
path: template/src/ontology/Makefile.jinja2
user: login: matentzn
body: why keep sort | uniq; just because we need cat for concatenating anyways, so, why not also...?
created_at: 2025-03-11T14:49:07Z
author_association: CONTRIBUTOR
diff_hunk: @@ -503,122 +516,190 @@ ANNOTATION_PROPERTIES={% for p in project.import_group.annotation_properties %}{
 # This pattern uses ROBOT to generate an import module
 
 ifeq ($(IMP),true)
-# Should be able to drop this if robot can just take a big messy list of terms as input.
-$(IMPORTDIR)/%_terms_combined.txt: $(IMPORTSEED) $(IMPORTDIR)/%_terms.txt
-	cat $^ | grep -v ^# | sort | uniq >  $@
 
-{% if project.import_group.use_base_merging %}
-ALL_TERMS_COMBINED = $(patsubst %, $(IMPORTDIR)/%_terms_combined.txt, $(IMPORTS))
-$(IMPORTDIR)/merged_terms_combined.txt: $(ALL_TERMS_COMBINED)
-	cat $^ | grep -v ^# | sort | uniq >  $@
-
-{% if 'slme' == project.import_group.module_type %}
-$(IMPORTDIR)/merged_import.owl: $(MIRRORDIR)/merged.owl $(IMPORTDIR)/merged_terms_combined.txt | all_robot_plugins
-	$(ROBOT) merge -i $< \{% if project.import_group.exclude_iri_patterns is not none %}
-		{% for pattern in project.import_group.exclude_iri_patterns %}remove  --select "{{ pattern }}" {% endfor %} \{% endif %}
-		extract -T $(IMPORTDIR)/merged_terms_combined.txt --force true --copy-ontology-annotations true --individuals {{ project.import_group.slme_individuals }} --method {{ project.import_group.module_type_slme }} \
-		remove $(patsubst %, --term %, $(ANNOTATION_PROPERTIES)) -T $(IMPORTDIR)/merged_terms_combined.txt --select complement --select annotation-properties \
-		odk:normalize --base-iri {{ project.uribase }} --subset-decls true --synonym-decls true --merge-axioms true \
-		$(ANNOTATE_CONVERT_FILE)
-{% else %}
-$(IMPORTDIR)/merged_import.owl: $(MIRRORDIR)/merged.owl $(IMPORTDIR)/merged_terms_combined.txt
-	echo "ERROR: You have configured your default module type to be {{project.import_group.module_type}}; this behavior needs to be overwritten in {{ project.id }}.Makefile!" && false
-{% endif %}
-{% endif %}
-
-{% if 'slme' == project.import_group.module_type %}
-$(IMPORTDIR)/%_import.owl: {% if project.import_group.use_base_merging %}$(MIRRORDIR)/merged.owl{% else %}$(MIRRORDIR)/%.owl{% endif %} $(IMPORTDIR)/%_terms_combined.txt | all_robot_plugins
-	$(ROBOT) annotate -i $< --remove-annotations \
-		odk:normalize --add-source true \
-		extract -T $(IMPORTDIR)/$*_terms_combined.txt --force true --copy-ontology-annotations false --individuals {{ project.import_group.slme_individuals }} --method {{ project.import_group.module_type_slme }} \
-		odk:normalize --base-iri {{ project.uribase }} --subset-decls true --synonym-decls true --merge-axioms true \
-		$(ANNOTATE_CONVERT_FILE)
-{% elif 'minimal' == project.import_group.module_type %}
-$(IMPORTDIR)/%_import.owl: {% if project.import_group.use_base_merging %}$(MIRRORDIR)/merged.owl{% else %}$(MIRRORDIR)/%.owl{% endif %} $(IMPORTDIR)/%_terms_combined.txt | all_robot_plugins
-	$(ROBOT) annotate -i $< --remove-annotations \
-		odk:normalize --add-source true \
-		extract -T $(IMPORTDIR)/$*_terms_combined.txt --force true --copy-ontology-annotations true --method BOT \
-		remove --base-iri $(OBOBASE)"/$(shell echo $* | tr a-z A-Z)_" --axioms external --preserve-structure false --trim false \
-		odk:normalize --base-iri {{ project.uribase }} --subset-decls true --synonym-decls true --merge-axioms true \
-		remove $(patsubst %, --term %, $(ANNOTATION_PROPERTIES)) -T $(IMPORTDIR)/$*_terms_combined.txt --select complement --select "classes individuals annotation-properties" \
-		$(ANNOTATE_CONVERT_FILE)
-{% elif 'mirror' == project.import_group.module_type %}
-$(IMPORTDIR)/%_import.owl: {% if project.import_group.use_base_merging %}$(MIRRORDIR)/merged.owl{% else %}$(MIRRORDIR)/%.owl{% endif %} $(IMPORTDIR)/%_terms_combined.txt | all_robot_plugins
-	$(ROBOT) annotate -i $< --remove-annotations \
-		odk:normalize --base-iri {{ project.uribase }} --subset-decls true --synonym-decls true --merge-axioms true --add-source true \
-		$(ANNOTATE_CONVERT_FILE)
-{% elif 'filter' == project.import_group.module_type %}
-$(IMPORTDIR)/%_import.owl: {% if project.import_group.use_base_merging %}$(MIRRORDIR)/merged.owl{% else %}$(MIRRORDIR)/%.owl{% endif %} $(IMPORTDIR)/%_terms_combined.txt | all_robot_plugins
-	$(ROBOT) merge -i $< \
-		annotate --remove-annotations \
-		odk:normalize --add-source true \
-		remove --base-iri $(OBOBASE)"/$(shell echo $* | tr a-z A-Z)_" --axioms external --preserve-structure false --trim false \
-		remove $(patsubst %, --term %, $(ANNOTATION_PROPERTIES)) -T $(IMPORTDIR)/$*_terms_combined.txt --select complement \
-		odk:normalize --base-iri {{ project.uribase }} --subset-decls true --synonym-decls true --merge-axioms true \
-		$(ANNOTATE_CONVERT_FILE)
-{% elif 'custom' == project.import_group.module_type %}
-$(IMPORTDIR)/%_import.owl: {% if project.import_group.use_base_merging %}$(MIRRORDIR)/merged.owl{% else %}$(MIRRORDIR)/%.owl{% endif %}
-	echo "ERROR: You have configured your default module type to be custom; this behavior needs to be overwritten in {{ project.id }}.Makefile!" && false
-{% endif %}
+{%   if project.import_group.use_base_merging -%}
+ALL_TERMS = $(foreach imp, $(IMPORTS), $(IMPORTDIR)/$(imp)_terms.txt)
+
+{%     if 'slme' == project.import_group.module_type -%}
+$(IMPORTDIR)/merged_import.owl: $(MIRRORDIR)/merged.owl $(ALL_TERMS) \
+				$(IMPORTSEED) | all_robot_plugins
+	$(ROBOT) merge --input $<{% if project.import_group.exclude_iri_patterns is not none -%}
+		 {%- for pattern in project.import_group.exclude_iri_patterns %} \
+		 remove --select "{{ pattern }}"{% endfor %}{% endif %} \
+		 extract $(foreach f, $(ALL_TERMS), --term-file $(f)) $(T_IMPORTSEED) \
+		         --force true --copy-ontology-annotations false \
+		         --individuals {{ project.import_group.slme_individuals }} \
+		         --method {{ project.import_group.module_type_slme }} \
+		 remove $(foreach p, $(ANNOTATION_PROPERTIES), --term $(p)) \
+		        $(foreach f, $(ALL_TERMS), --term-file $(f)) $(T_IMPORTSEED) \
+		        --select complement --select annotation-properties \
+		 odk:normalize --base-iri {{ project.uribase }} \
+		               --subset-decls true --synonym-decls true \
+		               --merge-axioms true \
+		 $(ANNOTATE_CONVERT_FILE)
+
+{%     else -%}
+$(IMPORTDIR)/merged_import.owl: $(MIRRORDIR)/merged.owl $(ALL_TERMS) \
+				$(IMPORTSEED)
+	@echo "ERROR: You have configured the merged import as a custom module;"
+	@echo "       This rule needs to be overwritten in {{ project.id }}.Makefile!"
+	@false
+
+{%     endif -%}
+{%   else %}{# !project.import_group.use_base_merging -#}
path: template/src/ontology/Makefile.jinja2
user: login: matentzn
body: super helpful to use jinja comments here, thanks
created_at: 2025-03-11T14:51:36Z
author_association: CONTRIBUTOR
diff_hunk: @@ -484,13 +487,23 @@ $(IMPORT_MODULE): $(IMPORT_MODULE_TEMPLATE) | $(TMPDIR)
 $(IMPORT_MODULE_SIGNATURE): $(IMPORT_MODULE) | $(TMPDIR)
 	$(ROBOT) query -f csv -i $< --query ../sparql/terms.sparql $@.tmp &&\
 	cat $@.tmp | sort | uniq >  $@
-{% endif %}
 
-ALLSEED = $(PRESEED) {% if project.use_dosdps %} $(TMPDIR)/all_pattern_terms.txt {% endif %}\
-{% if project.use_custom_import_module %} $(IMPORT_MODULE_SIGNATURE) {% endif %}
+{% endif -%}
+
+{% if project.import_group.scan_signature or project.use_custom_import_module -%}
+# seed.txt contains all entities to import in addition to those defined
+# in the individual _terms.txt files.
+IMPORTSEED = $(TMPDIR)/seed.txt
+$(IMPORTSEED):
+{%-  if project.import_group.scan_signature %} $(PRESEED)
+{%-    if project.use_dosdps %} $(TMPDIR)/all_pattern_terms.txt{% endif -%}
+{%-  endif -%}
+{%-if project.use_custom_import_module %} $(IMPORT_MODULE_SIGNATURE){% endif %} | $(TMPDIR)
+	cat $^ | sort | uniq > $@
+
+T_IMPORTSEED = --term-file $(IMPORTSEED)
path: template/src/ontology/Makefile.jinja2
user: login: matentzn
body: Why factor this out as a variable? Not against it, just curious of the motivation.
created_at: 2025-03-11T14:53:10Z
author_association: CONTRIBUTOR
diff_hunk: @@ -484,13 +487,23 @@ $(IMPORT_MODULE): $(IMPORT_MODULE_TEMPLATE) | $(TMPDIR)
 $(IMPORT_MODULE_SIGNATURE): $(IMPORT_MODULE) | $(TMPDIR)
 	$(ROBOT) query -f csv -i $< --query ../sparql/terms.sparql $@.tmp &&\
 	cat $@.tmp | sort | uniq >  $@
-{% endif %}
 
-ALLSEED = $(PRESEED) {% if project.use_dosdps %} $(TMPDIR)/all_pattern_terms.txt {% endif %}\
-{% if project.use_custom_import_module %} $(IMPORT_MODULE_SIGNATURE) {% endif %}
+{% endif -%}
+
+{% if project.import_group.scan_signature or project.use_custom_import_module -%}
+# seed.txt contains all entities to import in addition to those defined
+# in the individual _terms.txt files.
+IMPORTSEED = $(TMPDIR)/seed.txt
+$(IMPORTSEED):
+{%-  if project.import_group.scan_signature %} $(PRESEED)
+{%-    if project.use_dosdps %} $(TMPDIR)/all_pattern_terms.txt{% endif -%}
+{%-  endif -%}
+{%-if project.use_custom_import_module %} $(IMPORT_MODULE_SIGNATURE){% endif %} | $(TMPDIR)
+	cat $^ | sort | uniq > $@
path: template/src/ontology/Makefile.jinja2
user: login: gouttegd
body: When we do need an intermediate seed file (like here), we might as well sort it. It’s redundant with what ROBOT will later do with the file, but if users decide to have a look at this file (maybe to debug an issue with the seed generation process, or if they can’t understand why a given term ends up in the seed), it’d be nice to have a clean, sorted file.
created_at: 2025-03-13T09:47:27Z
author_association: CONTRIBUTOR
diff_hunk: @@ -503,122 +516,190 @@ ANNOTATION_PROPERTIES={% for p in project.import_group.annotation_properties %}{
 # This pattern uses ROBOT to generate an import module
 
 ifeq ($(IMP),true)
-# Should be able to drop this if robot can just take a big messy list of terms as input.
-$(IMPORTDIR)/%_terms_combined.txt: $(IMPORTSEED) $(IMPORTDIR)/%_terms.txt
-	cat $^ | grep -v ^# | sort | uniq >  $@
 
-{% if project.import_group.use_base_merging %}
-ALL_TERMS_COMBINED = $(patsubst %, $(IMPORTDIR)/%_terms_combined.txt, $(IMPORTS))
-$(IMPORTDIR)/merged_terms_combined.txt: $(ALL_TERMS_COMBINED)
-	cat $^ | grep -v ^# | sort | uniq >  $@
-
-{% if 'slme' == project.import_group.module_type %}
-$(IMPORTDIR)/merged_import.owl: $(MIRRORDIR)/merged.owl $(IMPORTDIR)/merged_terms_combined.txt | all_robot_plugins
-	$(ROBOT) merge -i $< \{% if project.import_group.exclude_iri_patterns is not none %}
-		{% for pattern in project.import_group.exclude_iri_patterns %}remove  --select "{{ pattern }}" {% endfor %} \{% endif %}
-		extract -T $(IMPORTDIR)/merged_terms_combined.txt --force true --copy-ontology-annotations true --individuals {{ project.import_group.slme_individuals }} --method {{ project.import_group.module_type_slme }} \
-		remove $(patsubst %, --term %, $(ANNOTATION_PROPERTIES)) -T $(IMPORTDIR)/merged_terms_combined.txt --select complement --select annotation-properties \
-		odk:normalize --base-iri {{ project.uribase }} --subset-decls true --synonym-decls true --merge-axioms true \
-		$(ANNOTATE_CONVERT_FILE)
-{% else %}
-$(IMPORTDIR)/merged_import.owl: $(MIRRORDIR)/merged.owl $(IMPORTDIR)/merged_terms_combined.txt
-	echo "ERROR: You have configured your default module type to be {{project.import_group.module_type}}; this behavior needs to be overwritten in {{ project.id }}.Makefile!" && false
-{% endif %}
-{% endif %}
-
-{% if 'slme' == project.import_group.module_type %}
-$(IMPORTDIR)/%_import.owl: {% if project.import_group.use_base_merging %}$(MIRRORDIR)/merged.owl{% else %}$(MIRRORDIR)/%.owl{% endif %} $(IMPORTDIR)/%_terms_combined.txt | all_robot_plugins
-	$(ROBOT) annotate -i $< --remove-annotations \
-		odk:normalize --add-source true \
-		extract -T $(IMPORTDIR)/$*_terms_combined.txt --force true --copy-ontology-annotations false --individuals {{ project.import_group.slme_individuals }} --method {{ project.import_group.module_type_slme }} \
-		odk:normalize --base-iri {{ project.uribase }} --subset-decls true --synonym-decls true --merge-axioms true \
-		$(ANNOTATE_CONVERT_FILE)
-{% elif 'minimal' == project.import_group.module_type %}
-$(IMPORTDIR)/%_import.owl: {% if project.import_group.use_base_merging %}$(MIRRORDIR)/merged.owl{% else %}$(MIRRORDIR)/%.owl{% endif %} $(IMPORTDIR)/%_terms_combined.txt | all_robot_plugins
-	$(ROBOT) annotate -i $< --remove-annotations \
-		odk:normalize --add-source true \
-		extract -T $(IMPORTDIR)/$*_terms_combined.txt --force true --copy-ontology-annotations true --method BOT \
-		remove --base-iri $(OBOBASE)"/$(shell echo $* | tr a-z A-Z)_" --axioms external --preserve-structure false --trim false \
-		odk:normalize --base-iri {{ project.uribase }} --subset-decls true --synonym-decls true --merge-axioms true \
-		remove $(patsubst %, --term %, $(ANNOTATION_PROPERTIES)) -T $(IMPORTDIR)/$*_terms_combined.txt --select complement --select "classes individuals annotation-properties" \
-		$(ANNOTATE_CONVERT_FILE)
-{% elif 'mirror' == project.import_group.module_type %}
-$(IMPORTDIR)/%_import.owl: {% if project.import_group.use_base_merging %}$(MIRRORDIR)/merged.owl{% else %}$(MIRRORDIR)/%.owl{% endif %} $(IMPORTDIR)/%_terms_combined.txt | all_robot_plugins
-	$(ROBOT) annotate -i $< --remove-annotations \
-		odk:normalize --base-iri {{ project.uribase }} --subset-decls true --synonym-decls true --merge-axioms true --add-source true \
-		$(ANNOTATE_CONVERT_FILE)
-{% elif 'filter' == project.import_group.module_type %}
-$(IMPORTDIR)/%_import.owl: {% if project.import_group.use_base_merging %}$(MIRRORDIR)/merged.owl{% else %}$(MIRRORDIR)/%.owl{% endif %} $(IMPORTDIR)/%_terms_combined.txt | all_robot_plugins
-	$(ROBOT) merge -i $< \
-		annotate --remove-annotations \
-		odk:normalize --add-source true \
-		remove --base-iri $(OBOBASE)"/$(shell echo $* | tr a-z A-Z)_" --axioms external --preserve-structure false --trim false \
-		remove $(patsubst %, --term %, $(ANNOTATION_PROPERTIES)) -T $(IMPORTDIR)/$*_terms_combined.txt --select complement \
-		odk:normalize --base-iri {{ project.uribase }} --subset-decls true --synonym-decls true --merge-axioms true \
-		$(ANNOTATE_CONVERT_FILE)
-{% elif 'custom' == project.import_group.module_type %}
-$(IMPORTDIR)/%_import.owl: {% if project.import_group.use_base_merging %}$(MIRRORDIR)/merged.owl{% else %}$(MIRRORDIR)/%.owl{% endif %}
-	echo "ERROR: You have configured your default module type to be custom; this behavior needs to be overwritten in {{ project.id }}.Makefile!" && false
-{% endif %}
+{%   if project.import_group.use_base_merging -%}
+ALL_TERMS = $(foreach imp, $(IMPORTS), $(IMPORTDIR)/$(imp)_terms.txt)
+
+{%     if 'slme' == project.import_group.module_type -%}
+$(IMPORTDIR)/merged_import.owl: $(MIRRORDIR)/merged.owl $(ALL_TERMS) \
+				$(IMPORTSEED) | all_robot_plugins
+	$(ROBOT) merge --input $<{% if project.import_group.exclude_iri_patterns is not none -%}
+		 {%- for pattern in project.import_group.exclude_iri_patterns %} \
+		 remove --select "{{ pattern }}"{% endfor %}{% endif %} \
+		 extract $(foreach f, $(ALL_TERMS), --term-file $(f)) $(T_IMPORTSEED) \
+		         --force true --copy-ontology-annotations false \
+		         --individuals {{ project.import_group.slme_individuals }} \
+		         --method {{ project.import_group.module_type_slme }} \
+		 remove $(foreach p, $(ANNOTATION_PROPERTIES), --term $(p)) \
+		        $(foreach f, $(ALL_TERMS), --term-file $(f)) $(T_IMPORTSEED) \
+		        --select complement --select annotation-properties \
+		 odk:normalize --base-iri {{ project.uribase }} \
+		               --subset-decls true --synonym-decls true \
+		               --merge-axioms true \
+		 $(ANNOTATE_CONVERT_FILE)
+
+{%     else -%}
+$(IMPORTDIR)/merged_import.owl: $(MIRRORDIR)/merged.owl $(ALL_TERMS) \
+				$(IMPORTSEED)
+	@echo "ERROR: You have configured the merged import as a custom module;"
+	@echo "       This rule needs to be overwritten in {{ project.id }}.Makefile!"
+	@false
+
+{%     endif -%}
+{%   else %}{# !project.import_group.use_base_merging -#}
path: template/src/ontology/Makefile.jinja2
user: login: gouttegd
body: Over time I plan to generalise that to the entire template.
created_at: 2025-03-13T09:47:47Z
author_association: CONTRIBUTOR
diff_hunk: @@ -484,13 +487,23 @@ $(IMPORT_MODULE): $(IMPORT_MODULE_TEMPLATE) | $(TMPDIR)
 $(IMPORT_MODULE_SIGNATURE): $(IMPORT_MODULE) | $(TMPDIR)
 	$(ROBOT) query -f csv -i $< --query ../sparql/terms.sparql $@.tmp &&\
 	cat $@.tmp | sort | uniq >  $@
-{% endif %}
 
-ALLSEED = $(PRESEED) {% if project.use_dosdps %} $(TMPDIR)/all_pattern_terms.txt {% endif %}\
-{% if project.use_custom_import_module %} $(IMPORT_MODULE_SIGNATURE) {% endif %}
+{% endif -%}
+
+{% if project.import_group.scan_signature or project.use_custom_import_module -%}
+# seed.txt contains all entities to import in addition to those defined
+# in the individual _terms.txt files.
+IMPORTSEED = $(TMPDIR)/seed.txt
+$(IMPORTSEED):
+{%-  if project.import_group.scan_signature %} $(PRESEED)
+{%-    if project.use_dosdps %} $(TMPDIR)/all_pattern_terms.txt{% endif -%}
+{%-  endif -%}
+{%-if project.use_custom_import_module %} $(IMPORT_MODULE_SIGNATURE){% endif %} | $(TMPDIR)
+	cat $^ | sort | uniq > $@
+
+T_IMPORTSEED = --term-file $(IMPORTSEED)
path: template/src/ontology/Makefile.jinja2
user: login: gouttegd
body: Consider what happens when we do _not_ need an auto-generated seed file (i.e. when `scan_signature` and `use_custom_import_module` are both `False`): then the auto-generated seed file pointed by `$(IMPORTSEED)` will never exist.

This means that wherever we need to give that seed to ROBOT with `--term-file` (in _all_ the rules that produce the import modules), we would need to guard the entire option with Jinja2 conditionals, for example:

```Jinja2
$(IMPORTDIR)/%_import.owl: $(MIRRORDIR)/%.owl $(IMPORTDIR)/%_terms.txt \
			   $(IMPORTSEED) | all_robot_plugins
	$(ROBOT) [ ... rest of the rule omitted for brevity ... ] \
		 extract --term-file $(IMPORTDIR)/$*_terms.txt \
{% if project.scan_signature or project.use_custom_import_module -%}
                         --term-file $(IMPORTSEED){% endif %} \
		         --force true --copy-ontology-annotations true \
                 [ ... rest of the rule omitted for brevity ... ] \
		 $(ANNOTATE_CONVERT_FILE)
```

By putting the entire option in a variable that is only defined when needed, we can instead do simply:

```Jinja2
$(IMPORTDIR)/%_import.owl: $(MIRRORDIR)/%.owl $(IMPORTDIR)/%_terms.txt \
			   $(IMPORTSEED) | all_robot_plugins
	$(ROBOT) [ ... rest of the rule omitted for brevity ... ] \
		 extract --term-file $(IMPORTDIR)/$*_terms.txt $(T_IMPORTSEED) \
		         --force true --copy-ontology-annotations true \
                 [ ... rest of the rule omitted for brevity ... ] \
		 $(ANNOTATE_CONVERT_FILE)
```

When `scan_signature` and `use_custom_import_module` are both `False`, `T_IMPORTSEED` will not be defined, so in the rule above it will simply be expanded to nothing.

The alternative would have been to do something like this:

```Jinja2
{% if project.import_group.scan_signature or project.use_custom_import_module -%}
# seed.txt contains all entities to import in addition to those defined
# in the individual _terms.txt files.
IMPORTSEED = $(TMPDIR)/seed.txt
$(IMPORTSEED):
{%-  if project.import_group.scan_signature %} $(PRESEED)
{%-    if project.use_dosdps %} $(TMPDIR)/all_pattern_terms.txt{% endif -%}
{%-  endif -%}
{%-if project.use_custom_import_module %} $(IMPORT_MODULE_SIGNATURE){% endif %} | $(TMPDIR)
	cat $^ | sort | uniq > $@

{% else %}{# We don't actually need a seed file, but we ensure an empty one always exists no matter what. -#}
IMPORTSEED = $(TMPDIR)/seed.txt
$(IMPORTSEED):
        touch $@
{% endif -%}
```

and then to unconditionally use `--term-file $(IMPORTSEED)` in all the rules that produce the import modules (pretty sure that ROBOT accepts an empty term file).

But why bother with a dummy file when we can easily avoid it?
created_at: 2025-03-13T10:02:20Z
author_association: CONTRIBUTOR

PR comments:
user: login: gouttegd
created_at: 2025-03-13T10:04:48Z
body: > currently running some tests

FYI I tested it on all FlyBase ontologies (FBdv, FBbt, DPO, FBcv) without issues. But of course they are all using a merged import module, so this did not test all the code paths.
author_association: CONTRIBUTOR
user: login: gouttegd
created_at: 2025-03-22T15:08:58Z
body: @matentzn Are you still testing this?
author_association: CONTRIBUTOR
user: login: matentzn
created_at: 2025-03-23T09:59:53Z
body: Sorry, I was buried in a backlog, now I am running some tests. 
author_association: CONTRIBUTOR