Repository: microbiomedata/nmdc-schema
Pull request #2445: Make `data_object_type` slot required on `DataObject` and implement migrator
Author: login: kheal
Created: 2025-05-07T18:53:33Z | Merged: 2025-05-21T20:47:07Z

This MR will close #2431 

It will make the slot `data_object_type` required on `DataObject`s.  I've updated all valid and invalid tests accordingly and added an invalid test that is missing this field.

Note that _in theory_ this could make data that were compliant now not compliant; though our NMDC mongo data are fully populated with regards to this field (see https://github.com/microbiomedata/issues/issues/1175).  Therefore, the migrator implemented here simply raises an error if it encounters data that do not meet this expectation. 



Linked issue #2431: Updated modeling for data_object_type
Make this required for all data_object_set records. 

Depends on:
- [x] https://github.com/microbiomedata/issues/issues/1175

Linked issue comments: user: login: kheal
created_at: 2025-04-18T19:31:13Z
body: Possible repeat of https://github.com/microbiomedata/nmdc-schema/issues/2346
author_association: CONTRIBUTOR

Diff:
diff --git a/nmdc_schema/migrators/migrator_from_11_7_0_to_11_8_0.py b/nmdc_schema/migrators/migrator_from_11_7_0_to_11_8_0.py
new file mode 100644
index 0000000000..18de134f6a
--- /dev/null
+++ b/nmdc_schema/migrators/migrator_from_11_7_0_to_11_8_0.py
@@ -0,0 +1,25 @@
+from nmdc_schema.migrators.migrator_base import MigratorBase
+
+class Migrator(MigratorBase):
+    r"""Migrates a database between two schemas."""
+
+    _from_version = "11.7.0"
+    _to_version = "11.8.0"
+
+    def upgrade(self) -> None:
+        r"""Migrates the database from conforming to the original schema, to conforming to the new schema."""
+        self.adapter.do_for_each_document("data_object_set", self.validate_data_object_type)
+
+    def validate_data_object_type(self, data_object: dict) -> None:
+        r"""
+        Raises an exception if the document lacks a `data_object_type` field.
+
+        >>> m = Migrator()
+        >>> m.validate_data_object_type({"id": 123, "type": "nmdc:DataObject", "data_object_type": "Type"})
+        >>> m.validate_data_object_type({"id": 123, "type": "nmdc:DataObject"})
+        Traceback (most recent call last):
+        ...
+        ValueError: data_object_type is required and is not present in the data object 123
+        """
+        if "data_object_type" not in data_object:
+            raise ValueError(f"data_object_type is required and is not present in the data object {data_object.get('id')}")
\ No newline at end of file
diff --git a/src/data/invalid/DataObject-invalid_id-1.yaml b/src/data/invalid/DataObject-invalid_id-1.yaml
index 228ce206fc..678efaf560 100644
--- a/src/data/invalid/DataObject-invalid_id-1.yaml
+++ b/src/data/invalid/DataObject-invalid_id-1.yaml
@@ -4,3 +4,4 @@ name: mapping_stats.txt
 description: Metagenome Contig Coverage Stats for gold:Gp0061273
 file_size_bytes: 32787380
 type: nmdc:DataObject
+data_object_type: Metagenome Raw Read 1
diff --git a/src/data/invalid/DataObject-invalid_id-2.yaml b/src/data/invalid/DataObject-invalid_id-2.yaml
index f3b6bfd77a..83e14ce866 100644
--- a/src/data/invalid/DataObject-invalid_id-2.yaml
+++ b/src/data/invalid/DataObject-invalid_id-2.yaml
@@ -4,3 +4,4 @@ name: mapping_stats.txt
 description: Metagenome Contig Coverage Stats for gold:Gp0061273
 file_size_bytes: 32787380
 type: nmdc:DataObject
+data_object_type: Metagenome Bins Info File
diff --git a/src/data/invalid/DataObject-invalid_was_generated_by.yaml b/src/data/invalid/DataObject-invalid_was_generated_by.yaml
index c01630e936..f04f89c7f8 100644
--- a/src/data/invalid/DataObject-invalid_was_generated_by.yaml
+++ b/src/data/invalid/DataObject-invalid_was_generated_by.yaml
@@ -5,3 +5,4 @@ description: Sequencing from project ABDC description
 file_size_bytes: 32787380
 type: nmdc:DataObject
 was_generated_by: nmdc:omprc-11-abd4sg.1
+data_object_type: Metagenome Raw Read 1
diff --git a/src/data/invalid/DataObject-no-data_object_type.yaml b/src/data/invalid/DataObject-no-data_object_type.yaml
new file mode 100644
index 0000000000..fe99bf02da
--- /dev/null
+++ b/src/data/invalid/DataObject-no-data_object_type.yaml
@@ -0,0 +1,6 @@
+# This example is invalid because the data_object_type field is missing.
+id: nmdc:dobj-99-izwYW6
+type: nmdc:DataObject
+name: mapping_stats.txt
+description: Metagenome Contig Coverage Stats for gold:Gp0061273
+file_size_bytes: 32787380
\ No newline at end of file
diff --git a/src/data/invalid/DataObject-no-id.yaml b/src/data/invalid/DataObject-no-id.yaml
index dde4c69598..c4024a0535 100644
--- a/src/data/invalid/DataObject-no-id.yaml
+++ b/src/data/invalid/DataObject-no-id.yaml
@@ -1,3 +1,4 @@
 description: "Crispr Terms for nmdc:ann0vx38"
 name: "Crispr Terms for nmdc:ann0vx38"
-type: nmdc:DataObject
\ No newline at end of file
+type: nmdc:DataObject
+data_object_type: Crispr Terms
\ No newline at end of file
diff --git a/src/data/invalid/DataObject-no-name.yaml b/src/data/invalid/DataObject-no-name.yaml
index 6e3ecafc98..43123dae7e 100644
--- a/src/data/invalid/DataObject-no-name.yaml
+++ b/src/data/invalid/DataObject-no-name.yaml
@@ -1,3 +1,4 @@
 description: "Crispr Terms for nmdc:ann0vx38"
 id: nmdc:dobj-99-abc123
 type: nmdc:DataObject
+data_object_type: Crispr Terms
diff --git a/src/data/invalid/DataObject-no-type.yaml b/src/data/invalid/DataObject-no-type.yaml
index 9211ca164c..8b225e54d9 100644
--- a/src/data/invalid/DataObject-no-type.yaml
+++ b/src/data/invalid/DataObject-no-type.yaml
@@ -1,3 +1,4 @@
 description: "Crispr Terms for nmdc:ann0vx38"
 id: nmdc:dobj-99-abc123
 name: "Crispr Terms for nmdc:ann0vx38"
+data_object_type: Crispr Terms
diff --git a/src/data/valid/DataObject-1.yaml b/src/data/valid/DataObject-1.yaml
index 80e0e28613..4f8f388a98 100644
--- a/src/data/valid/DataObject-1.yaml
+++ b/src/data/valid/DataObject-1.yaml
@@ -3,3 +3,4 @@ type: nmdc:DataObject
 name: mapping_stats.txt
 description: Metagenome Contig Coverage Stats for gold:Gp0061273
 file_size_bytes: 32787380
+data_object_type: Metagenome Bins
diff --git a/src/data/valid/DataObject-2.yaml b/src/data/valid/DataObject-2.yaml
index 1bc57cc097..4c9c23f6ce 100644
--- a/src/data/valid/DataObject-2.yaml
+++ b/src/data/valid/DataObject-2.yaml
@@ -2,4 +2,5 @@ id: nmdc:dobj-99-PqBJvW
 name: assembly_scaffolds.fna
 description: Assembled scaffold fasta for gold:Gp0061273
 file_size_bytes: 205297945
-type: nmdc:DataObject
\ No newline at end of file
+type: nmdc:DataObject
+data_object_type: Metagenome Bins
\ No newline at end of file
diff --git a/src/data/valid/DataObject-3.yaml b/src/data/valid/DataObject-3.yaml
index b8a5d8ca4f..6835227b79 100644
--- a/src/data/valid/DataObject-3.yaml
+++ b/src/data/valid/DataObject-3.yaml
@@ -2,4 +2,5 @@ id: nmdc:dobj-99-izwYW6
 name: mapping_stats.txt
 description: Metagenome Contig Coverage Stats for gold:Gp0061273
 file_size_bytes: 32787380
-type: nmdc:DataObject
\ No newline at end of file
+type: nmdc:DataObject
+data_object_type: Metagenome Bins
\ No newline at end of file
diff --git a/src/data/valid/DataObject-Crisper-Terms-data_object_type.yaml b/src/data/valid/DataObject-Crisper-Terms-data_object_type.yaml
index 07598aea17..c2fb49e040 100644
--- a/src/data/valid/DataObject-Crisper-Terms-data_object_type.yaml
+++ b/src/data/valid/DataObject-Crisper-Terms-data_object_type.yaml
@@ -6,3 +6,4 @@ md5_checksum: "22afa3d49b73eaec2e9787a6b88fbdc3"
 name: Crispr Terms
 type: nmdc:DataObject
 url: http://example.com
+data_object_type: CheckM Statistics
diff --git a/src/data/valid/DataObject-minimal.yaml b/src/data/valid/DataObject-minimal.yaml
index 37b1ad3228..7d25f1eb25 100644
--- a/src/data/valid/DataObject-minimal.yaml
+++ b/src/data/valid/DataObject-minimal.yaml
@@ -2,3 +2,4 @@ description: "Crispr Terms for nmdc:ann0vx38"
 id: "nmdc:dobj-11-dtTMNb"
 name: Crispr Terms
 type: nmdc:DataObject
+data_object_type: Crispr Terms
\ No newline at end of file
diff --git a/src/data/valid/Database-interleaved.yaml b/src/data/valid/Database-interleaved.yaml
index 68bffb792e..135d67d195 100644
--- a/src/data/valid/Database-interleaved.yaml
+++ b/src/data/valid/Database-interleaved.yaml
@@ -3076,16 +3076,19 @@ data_object_set:
     name: 11340.8.202049.GTCTCCT-AAGGAGA.fastq.gz
     description: Raw sequencer read data
     file_size_bytes: 9208349052
+    data_object_type: Metagenome Raw Reads
   - id: nmdc:dobj-99-7zrm55
     type: nmdc:DataObject
     name: 11839.4.222578.GAGCTCA-TTGAGCT.fastq.gz
     description: Raw sequencer read data
     file_size_bytes: 34243309819
+    data_object_type: Metagenome Raw Reads
   - id: nmdc:dobj-99-sQQw0I
     type: nmdc:DataObject
     name: 12660.4.274923.GATCGTAC-GTACGATC.fastq.gz
     description: Raw sequencer read data
     file_size_bytes: 7580035314
+    data_object_type: Metagenome Raw Reads
   - id: nmdc:dobj-70-izwYW6
     type: nmdc:DataObject
     name: CoreMS.toml
@@ -3111,6 +3114,7 @@ data_object_set:
     type: nmdc:DataObject
     name: emsl_calibration_run.raw
     description: GCMS data file
+    data_object_type: GC-MS Raw Data
     data_category: instrument_data
     file_size_bytes: 9208349052
     url: https://emsl_site/emsl_calibration_run.raw
diff --git a/src/data/valid/Database-mass_spectrometry_gc.yaml b/src/data/valid/Database-mass_spectrometry_gc.yaml
index eb14a6f363..7a7e808b3e 100644
--- a/src/data/valid/Database-mass_spectrometry_gc.yaml
+++ b/src/data/valid/Database-mass_spectrometry_gc.yaml
@@ -65,6 +65,7 @@ data_object_set:
     file_size_bytes: 9208349052
     in_manifest:
       - nmdc:manif-99-krhtest                                   #defined below
+    data_object_type: GC-MS Raw Data
   - id: nmdc:dobj-11-9n9n9n
     type: nmdc:DataObject
     data_category: instrument_data
@@ -74,3 +75,4 @@ data_object_set:
     file_size_bytes: 9208349052
     in_manifest:
       - nmdc:manif-99-krhtest                                   #defined below
+    data_object_type: GC-MS Raw Data
\ No newline at end of file
diff --git a/src/data/valid/Database-nmdc-example.yaml b/src/data/valid/Database-nmdc-example.yaml
index dfd715a800..8e57f9b355 100644
--- a/src/data/valid/Database-nmdc-example.yaml
+++ b/src/data/valid/Database-nmdc-example.yaml
@@ -282,13 +282,16 @@ data_object_set:
     description: Raw sequencer read data
     file_size_bytes: 9208349052
     type: nmdc:DataObject
+    data_object_type: Metagenome Raw Read 1
   - id: nmdc:dobj-99-7zrm55
     name: 11839.4.222578.GAGCTCA-TTGAGCT.fastq.gz
     description: Raw sequencer read data
     file_size_bytes: 34243309819
     type: nmdc:DataObject
+    data_object_type: Metagenome Raw Read 1
   - id: nmdc:dobj-99-sQQw0I
     name: 12660.4.274923.GATCGTAC-GTACGATC.fastq.gz
     description: Raw sequencer read data
     file_size_bytes: 7580035314
     type: nmdc:DataObject
+    data_object_type: Metagenome Raw Read 1
diff --git a/src/schema/basic_classes.yaml b/src/schema/basic_classes.yaml
index 263630e9e0..800f5cc602 100644
--- a/src/schema/basic_classes.yaml
+++ b/src/schema/basic_classes.yaml
@@ -374,6 +374,8 @@ classes:
           syntax: "{id_nmdc_prefix}:(wfmag|wfmb|wfmgan|wfmgas|wfmsa|wfmp|wfmt|wfmtan|wfmtas|wfnom|wfrbt|wfrqc)-{id_shoulder}-{id_blade}{id_version}$|{id_nmdc_prefix}:(omprc|dgms|dgns)-{id_shoulder}-{id_blade}$"
           interpolated: true
         range: DataEmitterProcess
+      data_object_type:
+        required: true
   DataEmitterProcess:
     class_uri: nmdc:DataEmitterProcess
     is_a: PlannedProcess


Reviews:
user: login: eecavanna
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-08T19:11:37Z
user: login: eecavanna
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-08T19:12:24Z
user: login: eecavanna
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-08T19:12:37Z
user: login: eecavanna
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-08T19:13:31Z
user: login: eecavanna
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-08T19:16:23Z
user: login: kheal
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-08T19:22:13Z
user: login: hesspnnl
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-08T19:34:54Z
user: login: eecavanna
body: The migrator looks good to me, and I like the approach of raising an exception when the original data does not meet our expectations.
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-05-08T20:21:10Z
user: login: eecavanna
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-09T20:30:35Z
user: login: hesspnnl
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-13T13:16:22Z
user: login: eecavanna
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-13T18:09:39Z
user: login: aclum
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-21T20:17:16Z
user: login: turbomam
state: APPROVED
author_association: MEMBER
submitted_at: 2025-05-21T20:23:33Z

Review comments:
diff_hunk: @@ -0,0 +1,28 @@
+from nmdc_schema.migrators.migrator_base import MigratorBase
+
+class Migrator(MigratorBase):
+    r"""Migrates a database between two schemas."""
+
+    _from_version = "11.7.0"
+    _to_version = "11.8.0"
+
+    def upgrade(self) -> None:
+        r"""Migrates the database from conforming to the original schema, to conforming to the new schema."""
+        self.adapter.process_each_document("data_object_set", [self.validate_data_object_type])
path: nmdc_schema/migrators/migrator_from_11_7_0_to_11_8_0.py
user: login: eecavanna
body: I recommend using the `do_for_each_document` adapter method instead of this `process_each_document` adapter method. The reason is that the former does something for each document, without modifying the document, itself; whereas the latter modifies the document (in this case, overwriting it with a copy of itself). In practice, the former will be faster during the real migration.

Switching to the former would involve making the following changes: replace `process_each_document` with `do_for_each_document` and then remove the surrounding `[`...`]` from `[self.validate_data_object_type]`.
created_at: 2025-05-08T19:11:37Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,28 @@
+from nmdc_schema.migrators.migrator_base import MigratorBase
+
+class Migrator(MigratorBase):
+    r"""Migrates a database between two schemas."""
+
+    _from_version = "11.7.0"
+    _to_version = "11.8.0"
+
+    def upgrade(self) -> None:
+        r"""Migrates the database from conforming to the original schema, to conforming to the new schema."""
+        self.adapter.process_each_document("data_object_set", [self.validate_data_object_type])
+
+    def validate_data_object_type(self, data_object: dict) -> dict:
+        r"""
+        If the data object does not have data_category field, add a value based on the data_object_type.
+        If the data object has any other value for data_object_type; ignore it.
+
+        >>> m = Migrator()
+        >>> m.validate_data_object_type({"id":123, "type":"nmdc:DataObject", "data_object_type":"Type"})
path: nmdc_schema/migrators/migrator_from_11_7_0_to_11_8_0.py
user: login: eecavanna
body: ```suggestion
        >>> m.validate_data_object_type({"id": 123, "type": "nmdc:DataObject", "data_object_type": "Type"})
```
created_at: 2025-05-08T19:12:23Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,28 @@
+from nmdc_schema.migrators.migrator_base import MigratorBase
+
+class Migrator(MigratorBase):
+    r"""Migrates a database between two schemas."""
+
+    _from_version = "11.7.0"
+    _to_version = "11.8.0"
+
+    def upgrade(self) -> None:
+        r"""Migrates the database from conforming to the original schema, to conforming to the new schema."""
+        self.adapter.process_each_document("data_object_set", [self.validate_data_object_type])
+
+    def validate_data_object_type(self, data_object: dict) -> dict:
+        r"""
+        If the data object does not have data_category field, add a value based on the data_object_type.
+        If the data object has any other value for data_object_type; ignore it.
+
+        >>> m = Migrator()
+        >>> m.validate_data_object_type({"id":123, "type":"nmdc:DataObject", "data_object_type":"Type"})
+        {'id': 123, 'type': 'nmdc:DataObject', 'data_object_type': 'Type'}
+        >>> m.validate_data_object_type({"id":123, "type":"nmdc:DataObject"})
path: nmdc_schema/migrators/migrator_from_11_7_0_to_11_8_0.py
user: login: eecavanna
body: ```suggestion
        >>> m.validate_data_object_type({"id": 123, "type": "nmdc:DataObject"})
```
created_at: 2025-05-08T19:12:37Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,28 @@
+from nmdc_schema.migrators.migrator_base import MigratorBase
+
+class Migrator(MigratorBase):
+    r"""Migrates a database between two schemas."""
+
+    _from_version = "11.7.0"
+    _to_version = "11.8.0"
+
+    def upgrade(self) -> None:
+        r"""Migrates the database from conforming to the original schema, to conforming to the new schema."""
+        self.adapter.process_each_document("data_object_set", [self.validate_data_object_type])
+
+    def validate_data_object_type(self, data_object: dict) -> dict:
+        r"""
+        If the data object does not have data_category field, add a value based on the data_object_type.
+        If the data object has any other value for data_object_type; ignore it.
path: nmdc_schema/migrators/migrator_from_11_7_0_to_11_8_0.py
user: login: eecavanna
body: I think this docstring may have been left over from copy/pasting a method from a different migrator.

```suggestion
        Raises an exception if the document either lacks a `data_object_type`
        field or it has a `data_object_type` field whose value is falsey.
```
created_at: 2025-05-08T19:13:31Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,28 @@
+from nmdc_schema.migrators.migrator_base import MigratorBase
+
+class Migrator(MigratorBase):
+    r"""Migrates a database between two schemas."""
+
+    _from_version = "11.7.0"
+    _to_version = "11.8.0"
+
+    def upgrade(self) -> None:
+        r"""Migrates the database from conforming to the original schema, to conforming to the new schema."""
+        self.adapter.process_each_document("data_object_set", [self.validate_data_object_type])
+
+    def validate_data_object_type(self, data_object: dict) -> dict:
+        r"""
+        If the data object does not have data_category field, add a value based on the data_object_type.
+        If the data object has any other value for data_object_type; ignore it.
+
+        >>> m = Migrator()
+        >>> m.validate_data_object_type({"id":123, "type":"nmdc:DataObject", "data_object_type":"Type"})
+        {'id': 123, 'type': 'nmdc:DataObject', 'data_object_type': 'Type'}
+        >>> m.validate_data_object_type({"id":123, "type":"nmdc:DataObject"})
+        Traceback (most recent call last):
+        ...
+        ValueError: data_object_type is required and is not present in the data object 123
+        """
+        if not data_object.get("data_object_type"):
+            raise ValueError(f"data_object_type is required and is not present in the data object {data_object.get('id')}")
+        return data_object
path: nmdc_schema/migrators/migrator_from_11_7_0_to_11_8_0.py
user: login: eecavanna
body: If you switch from using the `process_each_document` adapter method to using the `do_for_each_document` adapter method, then returning the document from this function will not be necessary anymore (if removed, I'd also recommend updating this function's return value type hint from `-> dict` to `-> None`).
created_at: 2025-05-08T19:16:22Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,28 @@
+from nmdc_schema.migrators.migrator_base import MigratorBase
+
+class Migrator(MigratorBase):
+    r"""Migrates a database between two schemas."""
+
+    _from_version = "11.7.0"
+    _to_version = "11.8.0"
+
+    def upgrade(self) -> None:
+        r"""Migrates the database from conforming to the original schema, to conforming to the new schema."""
+        self.adapter.process_each_document("data_object_set", [self.validate_data_object_type])
+
+    def validate_data_object_type(self, data_object: dict) -> dict:
+        r"""
+        If the data object does not have data_category field, add a value based on the data_object_type.
+        If the data object has any other value for data_object_type; ignore it.
+
+        >>> m = Migrator()
+        >>> m.validate_data_object_type({"id":123, "type":"nmdc:DataObject", "data_object_type":"Type"})
+        {'id': 123, 'type': 'nmdc:DataObject', 'data_object_type': 'Type'}
+        >>> m.validate_data_object_type({"id":123, "type":"nmdc:DataObject"})
+        Traceback (most recent call last):
+        ...
+        ValueError: data_object_type is required and is not present in the data object 123
+        """
+        if not data_object.get("data_object_type"):
+            raise ValueError(f"data_object_type is required and is not present in the data object {data_object.get('id')}")
+        return data_object
path: nmdc_schema/migrators/migrator_from_11_7_0_to_11_8_0.py
user: login: kheal
body: Excellent feedback, thanks @eecavanna.  I haven't used the `do_for_each_document` method before.
created_at: 2025-05-08T19:22:13Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,28 @@
+from nmdc_schema.migrators.migrator_base import MigratorBase
+
+class Migrator(MigratorBase):
+    r"""Migrates a database between two schemas."""
+
+    _from_version = "11.7.0"
+    _to_version = "11.8.0"
+
+    def upgrade(self) -> None:
+        r"""Migrates the database from conforming to the original schema, to conforming to the new schema."""
+        self.adapter.process_each_document("data_object_set", [self.validate_data_object_type])
+
+    def validate_data_object_type(self, data_object: dict) -> dict:
+        r"""
+        If the data object does not have data_category field, add a value based on the data_object_type.
+        If the data object has any other value for data_object_type; ignore it.
+
+        >>> m = Migrator()
+        >>> m.validate_data_object_type({"id":123, "type":"nmdc:DataObject", "data_object_type":"Type"})
+        {'id': 123, 'type': 'nmdc:DataObject', 'data_object_type': 'Type'}
+        >>> m.validate_data_object_type({"id":123, "type":"nmdc:DataObject"})
+        Traceback (most recent call last):
+        ...
+        ValueError: data_object_type is required and is not present in the data object 123
+        """
+        if not data_object.get("data_object_type"):
+            raise ValueError(f"data_object_type is required and is not present in the data object {data_object.get('id')}")
+        return data_object
path: nmdc_schema/migrators/migrator_from_11_7_0_to_11_8_0.py
user: login: hesspnnl
body: That's perfect. I'll implement that thank you!
created_at: 2025-05-08T19:34:54Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,26 @@
+from nmdc_schema.migrators.migrator_base import MigratorBase
+
+class Migrator(MigratorBase):
+    r"""Migrates a database between two schemas."""
+
+    _from_version = "11.7.0"
+    _to_version = "11.8.0"
+
+    def upgrade(self) -> None:
+        r"""Migrates the database from conforming to the original schema, to conforming to the new schema."""
+        self.adapter.do_for_each_document("data_object_set", self.validate_data_object_type)
+
+    def validate_data_object_type(self, data_object: dict) -> None:
+        r"""
+        Raises an exception if the document either lacks a `data_object_type`
+        field or it has a `data_object_type` field whose value is falsey.
+
+        >>> m = Migrator()
+        >>> m.validate_data_object_type({"id": 123, "type": "nmdc:DataObject", "data_object_type": "Type"})
+        >>> m.validate_data_object_type({"id": 123, "type": "nmdc:DataObject"})
+        Traceback (most recent call last):
+        ...
+        ValueError: data_object_type is required and is not present in the data object 123
+        """
+        if not data_object.get("data_object_type"):
path: nmdc_schema/migrators/migrator_from_11_7_0_to_11_8_0.py
user: login: eecavanna
body: I want to point out that this condition will be satisfied by not only the `data_object_type` field not existing in the dictionary, but also the field _existing_ and having a falsey value.

_If_ you want to use a condition that is only satisfied be the field not existing (I'm thinking that might be the case, based on the message being passed to `ValueError()`), I'd recommend changing the condition to:

```py
if "data_object_type" not in data_object:
```

An example dictionary (albeit, not a schema 11.7.0-compliant one) for which this condition would evaluate to something different from the original condition is:

```py
{
  "id": "nmdc:dobj-00-000001",
  "data_object_type": "",  # <-- field exists, but value is falsey
}
```
created_at: 2025-05-09T20:30:35Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,26 @@
+from nmdc_schema.migrators.migrator_base import MigratorBase
+
+class Migrator(MigratorBase):
+    r"""Migrates a database between two schemas."""
+
+    _from_version = "11.7.0"
+    _to_version = "11.8.0"
+
+    def upgrade(self) -> None:
+        r"""Migrates the database from conforming to the original schema, to conforming to the new schema."""
+        self.adapter.do_for_each_document("data_object_set", self.validate_data_object_type)
+
+    def validate_data_object_type(self, data_object: dict) -> None:
+        r"""
+        Raises an exception if the document either lacks a `data_object_type`
+        field or it has a `data_object_type` field whose value is falsey.
+
+        >>> m = Migrator()
+        >>> m.validate_data_object_type({"id": 123, "type": "nmdc:DataObject", "data_object_type": "Type"})
+        >>> m.validate_data_object_type({"id": 123, "type": "nmdc:DataObject"})
+        Traceback (most recent call last):
+        ...
+        ValueError: data_object_type is required and is not present in the data object 123
+        """
+        if not data_object.get("data_object_type"):
path: nmdc_schema/migrators/migrator_from_11_7_0_to_11_8_0.py
user: login: hesspnnl
body: Sounds good. Thanks for the suggestion!
created_at: 2025-05-13T13:16:22Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,26 @@
+from nmdc_schema.migrators.migrator_base import MigratorBase
+
+class Migrator(MigratorBase):
+    r"""Migrates a database between two schemas."""
+
+    _from_version = "11.7.0"
+    _to_version = "11.8.0"
+
+    def upgrade(self) -> None:
+        r"""Migrates the database from conforming to the original schema, to conforming to the new schema."""
+        self.adapter.do_for_each_document("data_object_set", self.validate_data_object_type)
+
+    def validate_data_object_type(self, data_object: dict) -> None:
+        r"""
+        Raises an exception if the document either lacks a `data_object_type`
+        field or it has a `data_object_type` field whose value is falsey.
path: nmdc_schema/migrators/migrator_from_11_7_0_to_11_8_0.py
user: login: eecavanna
body: Now that the condition has been updated, I recommend updating the docstring to match it. The current docstring reflects the previous behavior.

```suggestion
        Raises an exception if the document lacks a `data_object_type` field.
```
created_at: 2025-05-13T18:09:39Z
author_association: COLLABORATOR

PR comments:
user: login: github-actions[bot]
created_at: 2025-05-07T18:56:16Z
body: [PR Preview Action](https://github.com/rossjrw/pr-preview-action) v1.6.1
:---:
| <p></p> :rocket: View preview at <br> https://microbiomedata.github.io/nmdc-schema/pr-preview/pr-2445/ <br><br>
| <h6>Built to branch [`gh-pages`](https://github.com/microbiomedata/nmdc-schema/tree/gh-pages) at 2025-05-13 19:33 UTC. <br> Preview will be ready when the [GitHub Pages deployment](https://github.com/microbiomedata/nmdc-schema/deployments) is complete. <br><br> </h6>
<!-- Sticky Pull Request Commentpr-preview -->
author_association: NONE
user: login: kheal
created_at: 2025-05-07T20:52:51Z
body: todo - write a migrator that throws an error if there is a data object record with this field not popualted.
author_association: CONTRIBUTOR
user: login: kheal
created_at: 2025-05-07T21:02:28Z
body: @hesspnnl - do you want to give this migrator a go?  If so, you can check out this branch and add it.
author_association: CONTRIBUTOR
user: login: hesspnnl
created_at: 2025-05-08T12:28:29Z
body: > todo - write a migrator that throws an error if there is a data object record with this field not popualted.

yes 
author_association: CONTRIBUTOR
user: login: hesspnnl
created_at: 2025-05-08T16:04:10Z
body: > @hesspnnl - do you want to give this migrator a go? If so, you can check out this branch and add it.

I wonder if this should be combined with https://github.com/microbiomedata/nmdc-schema/issues/2444
author_association: CONTRIBUTOR