Repository: bids-standard/pybids

Issue:
BUG: null metadata results in indexing failure
```python
sdcflows/conftest.py:39: in <module>
    layouts = {
sdcflows/conftest.py:40: in <dictcomp>
    p.name: BIDSLayout(str(p), validate=False, derivatives=True)
/usr/share/miniconda/envs/test/lib/python3.10/site-packages/bids/layout/layout.py:177: in __init__
    _indexer(self)
/usr/share/miniconda/envs/test/lib/python3.10/site-packages/bids/layout/index.py:154: in __call__
    self._index_metadata()
/usr/share/miniconda/envs/test/lib/python3.10/site-packages/bids/layout/index.py:484: in _index_metadata
    tag = _create_tag_dict(bf, all_entities[md_key], md_val, is_metadata=True)
/usr/share/miniconda/envs/test/lib/python3.10/site-packages/bids/layout/models.py:720: in _create_tag_dict
    raise ValueError(
E   ValueError: Passed value has an invalid dtype (NoneType). Must be one of int, float, bool, or str.
```

This is caused by the metadata (snipped for brevity):

```json
{
    "Rows": 256,
    "InversionTime": 1.0,
    "InplanePhaseEncodingDirection": "ROW",
    "SeriesTime": "102035.540000",
    "NumberOfAverages": 1.0,
    "ManufacturersModelName": "Skyra",
    "PATModeText": "p2",
    "PhaseEncodingDirection": "i-",
    "BandwidthPerPixelPhaseEncode": null
}
```

Question is: Do we drop `null`s or encode them in the database? My inclination is that the simplest thing is to drop, as we treat `None` queries as absent anyway.

cc @oesteban for opinions.


Discussion:
Either of the following patches will fix this:

```diff
diff --git a/bids/layout/index.py b/bids/layout/index.py
index 62cbec26..20030c47 100644
--- a/bids/layout/index.py
+++ b/bids/layout/index.py
@@ -465,6 +465,8 @@ class BIDSLayoutIndexer:
 
             # Create Tag <-> Entity mappings, and any newly discovered Entities
             for md_key, md_val in file_md.items():
+                if md_val is None:
+                    continue
                 tag_string = '{}_{}'.format(bf.path, md_key)
                 # Skip pairs that were already found in the filenames
                 if tag_string in all_tags:
```

```diff
diff --git a/bids/layout/models.py b/bids/layout/models.py
index 4221ab3c..ddde2300 100644
--- a/bids/layout/models.py
+++ b/bids/layout/models.py
@@ -713,6 +713,9 @@ def _create_tag_dict(file, entity, value, dtype=None, is_metadata=False):
         if dtype in ['list', 'dict']:
             _dtype = 'json'
             _value = json.dumps(value)
+        elif value is None:
+            _dtype = 'json'
+            _value = 'null'
         else:
             _dtype = dtype
             _value = str(value)
```

Resolving patch:
diff --git a/bids/layout/index.py b/bids/layout/index.py
--- a/bids/layout/index.py
+++ b/bids/layout/index.py
@@ -465,6 +465,10 @@ def create_association_pair(src, dst, kind, kind2=None):
 
             # Create Tag <-> Entity mappings, and any newly discovered Entities
             for md_key, md_val in file_md.items():
+                # Treat null entries (deserialized to None) as absent
+                # Alternative is to cast None to null in layout.models._create_tag_dict
+                if md_val is None:
+                    continue
                 tag_string = '{}_{}'.format(bf.path, md_key)
                 # Skip pairs that were already found in the filenames
                 if tag_string in all_tags:


Test patch:
diff --git a/bids/tests/data/ds005/task-mixedgamblestask_bold.json b/bids/tests/data/ds005/task-mixedgamblestask_bold.json
--- a/bids/tests/data/ds005/task-mixedgamblestask_bold.json
+++ b/bids/tests/data/ds005/task-mixedgamblestask_bold.json
@@ -1,5 +1,6 @@
 {
     "RepetitionTime": 2.0,
     "TaskName": "mixed-gambles task",
-    "SliceTiming": [0.0, 0.0571, 0.1143, 0.1714, 0.2286, 0.2857]
+    "SliceTiming": [0.0, 0.0571, 0.1143, 0.1714, 0.2286, 0.2857],
+    "NullTestMetadata": null
 }