Repository: bids-standard/pybids

Issue:
Unable to use `Query.*` on extension
On the following dataset:

```
sub-0
  sub-0
  sub-0_suff.0
```

I am unable to use any of the `bids.layout.Query` items to filter `extension` with the `BIDSLayout.get()`. All such attempts return empty lists. eg:

```
layout.get(extension=Query.NONE) === []
layout.get(extension=Query.ANY) === []
layout.get(extension=Query.OPTIONAL) === []
```

The function behaves as expected for `subject` and `suffix`.

The code likely responsible is a bunch of SQL queries, and I have very little experience in such things, so I'm afraid I have no start to a solution.

Admittedly, such filtering over extension probably has little use in the real world... the above example was discovered while input fuzzing (as in #991). So if this issue will be resolve anyway in Pybids 1.0, it may not be worth spending time addressing, in which case feel free to close.

Unable to use `Query.*` on extension
On the following dataset:

```
sub-0
  sub-0
  sub-0_suff.0
```

I am unable to use any of the `bids.layout.Query` items to filter `extension` with the `BIDSLayout.get()`. All such attempts return empty lists. eg:

```
layout.get(extension=Query.NONE) === []
layout.get(extension=Query.ANY) === []
layout.get(extension=Query.OPTIONAL) === []
```

The function behaves as expected for `subject` and `suffix`.

The code likely responsible is a bunch of SQL queries, and I have very little experience in such things, so I'm afraid I have no start to a solution.

Admittedly, such filtering over extension probably has little use in the real world... the above example was discovered while input fuzzing (as in #991). So if this issue will be resolve anyway in Pybids 1.0, it may not be worth spending time addressing, in which case feel free to close.



Discussion:
Extensions should not be treated differently than other path components, so I'm not sure why this is happening. A quick glance at the code that normalizes the initial dot doesn't seem like it should have this effect. I agree, though, that this is a pretty niche case. If someone figures it out, or it goes away with the 1.0 change, then cool. We can leave this open until then.
Figured out what was going on here when researching #993. In `bids.layout.BIDSLayout.get()`, the function does some validation of `extension` to ensure all entries start with a leading `'.'`. This ends up turning the filter into a list of values, even if only one value (`Query.NONE`) is passed. The function then fails for the same reason as in #993. So resolving #993 will also resolve this issue
Extensions should not be treated differently than other path components, so I'm not sure why this is happening. A quick glance at the code that normalizes the initial dot doesn't seem like it should have this effect. I agree, though, that this is a pretty niche case. If someone figures it out, or it goes away with the 1.0 change, then cool. We can leave this open until then.
Figured out what was going on here when researching #993. In `bids.layout.BIDSLayout.get()`, the function does some validation of `extension` to ensure all entries start with a leading `'.'`. This ends up turning the filter into a list of values, even if only one value (`Query.NONE`) is passed. The function then fails for the same reason as in #993. So resolving #993 will also resolve this issue

Resolving patch:
diff --git a/bids/layout/layout.py b/bids/layout/layout.py
--- a/bids/layout/layout.py
+++ b/bids/layout/layout.py
@@ -293,7 +293,10 @@ def _sanitize_query_dtypes(self, entities):
                 continue
             try:
                 if isinstance(val, (list, tuple)):
-                    entities[name] = [ents[name]._astype(v) for v in val]
+                    entities[name] = [
+                        v if isinstance(v, Query) else ents[name]._astype(v)
+                        for v in val
+                    ]
                 else:
                     entities[name] = ents[name]._astype(val)
             except Exception:


Test patch:
diff --git a/bids/layout/tests/conftest.py b/bids/layout/tests/conftest.py
--- a/bids/layout/tests/conftest.py
+++ b/bids/layout/tests/conftest.py
@@ -25,6 +25,12 @@ def layout_ds005():
     return BIDSLayout(data_dir)
 
 
+@pytest.fixture(scope="module")
+def layout_ds005_no_validate():
+    data_dir = join(get_test_data_path(), 'ds005')
+    return BIDSLayout(data_dir, validate=False)
+
+
 @pytest.fixture(scope="module")
 def layout_ds117():
     data_dir = join(get_test_data_path(), 'ds000117')
diff --git a/bids/layout/tests/test_layout.py b/bids/layout/tests/test_layout.py
--- a/bids/layout/tests/test_layout.py
+++ b/bids/layout/tests/test_layout.py
@@ -981,6 +981,33 @@ def test_get_with_query_constants_in_match_list(layout_ds005):
     assert set(get_none_and_any) == set(get_none) | set(get_any)
 
 
+def test_get_non_run_entity_with_query_constants_in_match_list(layout_ds005):
+    l = layout_ds005
+    get1 = l.get(subject='01', acquisition="MPRAGE", suffix='T1w')
+    get_none = l.get(subject='01', acquisition=None, suffix='T1w')
+    get_any = l.get(subject='01', acquisition=Query.ANY, suffix='T1w')
+    get1_and_none = l.get(subject='01', acquisition=[None, "MPRAGE"], suffix='T1w')
+    get1_and_any = l.get(subject='01', acquisition=[Query.ANY, "MPRAGE"], suffix='T1w')
+    get_none_and_any = l.get(
+        subject='01', acquisition=[Query.ANY, Query.NONE], suffix='T1w'
+    )
+    assert set(get1_and_none) == set(get1) | set(get_none)
+    assert set(get1_and_any) == set(get1) | set(get_any)
+    assert set(get_none_and_any) == set(get_none) | set(get_any)
+
+
+def test_query_constants_work_on_extension(layout_ds005_no_validate):
+    l = layout_ds005_no_validate
+    get_both = l.get(subject='11', datatype='dwi', extension=Query.OPTIONAL)
+    get_ext = l.get(subject='11', datatype='dwi', extension=Query.REQUIRED)
+    get_no_ext = l.get(subject='11', datatype='dwi', extension=Query.NONE)
+    assert len(get_both) == 2
+    assert len(get_ext) == 1
+    assert len(get_no_ext) == 1
+    assert 'extension' in get_ext[0].get_entities()
+    assert 'extension' not in get_no_ext[0].get_entities()
+
+
 def test_load_layout(layout_synthetic_nodb, db_dir):
     db_path = str(db_dir / 'tmp_db')
     layout_synthetic_nodb.save(db_path)
diff --git a/bids/tests/data/ds005/sub-11/dwi/sub-11 b/bids/tests/data/ds005/sub-11/dwi/sub-11
new file mode 100644
diff --git a/bids/tests/data/ds005/sub-12/anat/sub-12_acq-MP2RAGE_T1w.nii.gz b/bids/tests/data/ds005/sub-12/anat/sub-12_acq-MP2RAGE_T1w.nii.gz
new file mode 100644
diff --git a/bids/tests/data/ds005/sub-12/anat/sub-12_acq-MPRAGE_T1w.nii.gz b/bids/tests/data/ds005/sub-12/anat/sub-12_acq-MPRAGE_T1w.nii.gz
new file mode 100644