Repository: bids-standard/pybids
Pull request #819 (2022-02-19 00:17:33)

ENH: Retain zero-padding in run entities while preserving integer queries and comparisons
This introduces a new type `PaddedInt` which is an int that retains its original string representation. The main purpose of this is to enable downstream tools to preserve the original representation when reconstructing paths.

xref #381 nipreps/fmriprep#2666 nipreps/fmriprep#2655

codecov[bot]: # [Codecov](https://codecov.io/gh/bids-standard/pybids/pull/819?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=bids-standard) Report
> Merging [#819](https://codecov.io/gh/bids-standard/pybids/pull/819?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=bids-standard) (e4503b7) into [master](https://codecov.io/gh/bids-standard/pybids/commit/f422263c9ddce2e3fd03cb1bb7885aa7224bbab4?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=bids-standard) (f422263) will **decrease** coverage by `0.10%`.
> The diff coverage is `93.33%`.

[![Impacted file tree graph](https://codecov.io/gh/bids-standard/pybids/pull/819/graphs/tree.svg?width=650&height=150&src=pr&token=0d39OR1fhx&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=bids-standard)](https://codecov.io/gh/bids-standard/pybids/pull/819?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=bids-standard)

```diff
@@            Coverage Diff             @@
##           master     #819      +/-   ##
==========================================
- Coverage   85.93%   85.83%   -0.11%     
==========================================
  Files          34       34              
  Lines        3883     3868      -15     
  Branches      998     1001       +3     
==========================================
- Hits         3337     3320      -17     
- Misses        352      355       +3     
+ Partials      194      193       -1     
```


| [Impacted Files](https://codecov.io/gh/bids-standard/pybids/pull/819?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=bids-standard) | Coverage Δ | |
|---|---|---|
| [bids/layout/utils.py](https://codecov.io/gh/bids-standard/pybids/pull/819/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=bids-standard#diff-Ymlkcy9sYXlvdXQvdXRpbHMucHk=) | `96.29% <87.50%> (-3.71%)` | :arrow_down: |
| [bids/layout/layout.py](https://codecov.io/gh/bids-standard/pybids/pull/819/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=bids-standard#diff-Ymlkcy9sYXlvdXQvbGF5b3V0LnB5) | `88.20% <100.00%> (-0.11%)` | :arrow_down: |
| [bids/layout/models.py](https://codecov.io/gh/bids-standard/pybids/pull/819/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=bids-standard#diff-Ymlkcy9sYXlvdXQvbW9kZWxzLnB5) | `93.06% <100.00%> (+0.33%)` | :arrow_up: |
| [bids/\_\_main\_\_.py](https://codecov.io/gh/bids-standard/pybids/pull/819/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=bids-standard#diff-Ymlkcy9fX21haW5fXy5weQ==) | `0.00% <0.00%> (-33.34%)` | :arrow_down: |
| [bids/cli.py](https://codecov.io/gh/bids-standard/pybids/pull/819/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=bids-standard#diff-Ymlkcy9jbGkucHk=) | `48.35% <0.00%> (-1.65%)` | :arrow_down: |
| [bids/modeling/model\_spec.py](https://codecov.io/gh/bids-standard/pybids/pull/819/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=bids-standard#diff-Ymlkcy9tb2RlbGluZy9tb2RlbF9zcGVjLnB5) | `81.88% <0.00%> (-0.69%)` | :arrow_down: |
| [bids/modeling/statsmodels.py](https://codecov.io/gh/bids-standard/pybids/pull/819/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=bids-standard#diff-Ymlkcy9tb2RlbGluZy9zdGF0c21vZGVscy5weQ==) | `85.08% <0.00%> (-0.25%)` | :arrow_down: |
| [bids/layout/db.py](https://codecov.io/gh/bids-standard/pybids/pull/819/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=bids-standard#diff-Ymlkcy9sYXlvdXQvZGIucHk=) | `95.83% <0.00%> (-0.22%)` | :arrow_down: |
| [bids/variables/variables.py](https://codecov.io/gh/bids-standard/pybids/pull/819/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=bids-standard#diff-Ymlkcy92YXJpYWJsZXMvdmFyaWFibGVzLnB5) | `89.60% <0.00%> (-0.13%)` | :arrow_down: |
| ... and [5 more](https://codecov.io/gh/bids-standard/pybids/pull/819/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=bids-standard) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/bids-standard/pybids/pull/819?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=bids-standard).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=bids-standard)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/bids-standard/pybids/pull/819?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=bids-standard). Last update [f422263...e4503b7](https://codecov.io/gh/bids-standard/pybids/pull/819?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=bids-standard). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=bids-standard).


Patch:
diff --git a/bids/layout/config/bids.json b/bids/layout/config/bids.json
index 6bcc2132..44f6b636 100644
--- a/bids/layout/config/bids.json
+++ b/bids/layout/config/bids.json
@@ -34,7 +34,7 @@
         },
         {
             "name": "run",
-            "pattern": "[_/\\\\]+run-0*(\\d+)",
+            "pattern": "[_/\\\\]+run-(\\d+)",
             "dtype": "int"
         },
         {
diff --git a/bids/layout/layout.py b/bids/layout/layout.py
index 21e68a3a..6763d111 100644
--- a/bids/layout/layout.py
+++ b/bids/layout/layout.py
@@ -12,6 +12,7 @@ from pathlib import Path
 
 import sqlalchemy as sa
 from sqlalchemy.orm import aliased
+from sqlalchemy.sql.expression import cast
 from bids_validator import BIDSValidator
 
 from ..utils import listify, natural_sort
@@ -772,10 +773,11 @@ class BIDSLayout(object):
                     else:
                         val_clause = tag_alias._value.op('REGEXP')(str(val))
                 else:
-                    if isinstance(val, (list, tuple)):
-                        val_clause = tag_alias._value.in_(val)
+                    vals = listify(val)
+                    if isinstance(vals[0], int):
+                        val_clause = cast(tag_alias._value, sa.Integer).in_(vals)
                     else:
-                        val_clause = tag_alias._value == val
+                        val_clause = tag_alias._value.in_(vals)
 
                 query = join_method(
                     tag_alias,
diff --git a/bids/layout/models.py b/bids/layout/models.py
index bf0ca674..be1534bb 100644
--- a/bids/layout/models.py
+++ b/bids/layout/models.py
@@ -18,7 +18,7 @@ from sqlalchemy.orm import reconstructor, relationship, backref, object_session
 from ..utils import listify
 from .writing import build_path, write_to_file
 from ..config import get_option
-from .utils import BIDSMetadata
+from .utils import BIDSMetadata, PaddedInt
 
 Base = declarative_base()
 
@@ -71,6 +71,9 @@ class LayoutInfo(Base):
 
         return kwargs
 
+    def __repr__(self):
+        return f"<LayoutInfo {self.root}>"
+
 
 class Config(Base):
     """Container for BIDS configuration information.
@@ -162,6 +165,9 @@ class Config(Base):
 
         return Config(session=session, **config)
 
+    def __repr__(self):
+        return f"<Config {self.name}>"
+
 
 class BIDSFile(Base):
     """Represents a single file or directory in a BIDS dataset.
@@ -531,12 +537,18 @@ class Entity(Base):
 
         self._init_on_load()
 
+    def __repr__(self):
+        return f"<Entity {self.name} (pattern={self.pattern}, dtype={self.dtype})>"
+
     @reconstructor
     def _init_on_load(self):
         if self._dtype not in ('str', 'float', 'int', 'bool'):
             raise ValueError("Invalid dtype '{}'. Must be one of 'int', "
                              "'float', 'bool', or 'str'.".format(self._dtype))
-        self.dtype = eval(self._dtype)
+        if self._dtype == "int":
+            self.dtype = PaddedInt
+        else:
+            self.dtype = eval(self._dtype)
         self.regex = re.compile(self.pattern) if self.pattern is not None else None
 
     def __iter__(self):
@@ -684,6 +696,9 @@ class Tag(Base):
         if self._dtype == 'json':
             self.value = json.loads(self._value)
             self.dtype = 'json'
+        elif self._dtype == 'int':
+            self.dtype = PaddedInt
+            self.value = self.dtype(self._value)
         else:
             self.dtype = eval(self._dtype)
             self.value = self.dtype(self._value)
diff --git a/bids/layout/utils.py b/bids/layout/utils.py
index 5d30d094..1655aeca 100644
--- a/bids/layout/utils.py
+++ b/bids/layout/utils.py
@@ -20,6 +20,74 @@ class BIDSMetadata(dict):
                 "Metadata term {!r} unavailable for file {}.".format(key, self._source_file))
 
 
+class PaddedInt(int):
+    """ Integer type that preserves zero-padding
+
+    Acts like an int in almost all ways except that string formatting
+    will keep the original zero-padding. Numeric format specifiers
+
+    >>> PaddedInt(1)
+    1
+    >>> p2 = PaddedInt("02")
+    >>> p2
+    02
+    >>> str(p2)
+    '02'
+    >>> p2 == 2
+    True
+    >>> p2 in range(3)
+    True
+    >>> f"{p2}"
+    '02'
+    >>> f"{p2:s}"
+    '02'
+    >>> f"{p2!s}"
+    '02'
+    >>> f"{p2!r}"
+    '02'
+    >>> f"{p2:d}"
+    '2'
+    >>> f"{p2:03d}"
+    '002'
+    >>> f"{p2:f}"
+    '2.000000'
+    >>> {2: "val"}.get(p2)
+    'val'
+    >>> {p2: "val"}.get(2)
+    'val'
+
+    Note that arithmetic will break the padding.
+
+    >>> str(p2 + 1)
+    '3'
+    """
+    def __init__(self, val):
+        self.sval = str(val)
+
+    def __eq__(self, val):
+        return val == self.sval or super().__eq__(val)
+
+    def __str__(self):
+        return self.sval
+
+    def __repr__(self):
+        return self.sval
+
+    def __format__(self, format_spec):
+        """ Format a padded integer
+
+        If a format spec can be used on a string, apply it to the zero-padded string.
+        Otherwise format as an integer.
+        """
+        try:
+            return format(self.sval, format_spec)
+        except:
+            return super().__format__(format_spec)
+
+    def __hash__(self):
+        return super().__hash__()
+
+
 def parse_file_entities(filename, entities=None, config=None,
                         include_unmatched=False):
     """Parse the passed filename for entity/value pairs.


Test patch:
diff --git a/bids/layout/tests/test_layout.py b/bids/layout/tests/test_layout.py
index bfbe9fe9..c19d6329 100644
--- a/bids/layout/tests/test_layout.py
+++ b/bids/layout/tests/test_layout.py
@@ -14,6 +14,7 @@ import pytest
 from bids.layout import BIDSLayout, Query
 from bids.layout.models import Config
 from bids.layout.index import BIDSLayoutIndexer
+from bids.layout.utils import PaddedInt
 from bids.tests import get_test_data_path
 from bids.utils import natural_sort
 
@@ -711,7 +712,7 @@ def test_get_with_wrong_dtypes(layout_7t_trt):
     ''' Test automatic dtype sanitization. '''
     l = layout_7t_trt
     assert (l.get(run=1) == l.get(run='1') == l.get(run=np.int64(1)) ==
-            l.get(run=[1, '15']))
+            l.get(run=[1, '15']) == l.get(run='01'))
     assert not l.get(run='not_numeric')
     assert l.get(session=1) == l.get(session='1')
 
@@ -807,4 +808,17 @@ def test_load_layout_config_not_overwritten(layout_synthetic_nodb, tmpdir):
     for attr in ['root', 'absolute_paths', 'derivatives']:
         assert getattr(cm1.layout_info, attr) == getattr(cm2.layout_info, attr)
 
-    assert cm1.layout_info.config != cm2.layout_info.config
\ No newline at end of file
+    assert cm1.layout_info.config != cm2.layout_info.config
+
+
+def test_padded_run_roundtrip(layout_ds005):
+    for run in (1, "1", "01"):
+        res = layout_ds005.get(subject="01", task="mixedgamblestask",
+                               run=run, extension=".nii.gz")
+        assert len(res) == 1
+    boldfile = res[0]
+    ents = boldfile.get_entities()
+    assert isinstance(ents["run"], PaddedInt)
+    assert ents["run"] == 1
+    newpath = layout_ds005.build_path(ents, absolute_paths=False)
+    assert newpath == "sub-01/func/sub-01_task-mixedgamblestask_run-01_bold.nii.gz"
diff --git a/bids/layout/tests/test_models.py b/bids/layout/tests/test_models.py
index 6b9ace2f..2f7862fa 100644
--- a/bids/layout/tests/test_models.py
+++ b/bids/layout/tests/test_models.py
@@ -13,6 +13,7 @@ import numpy as np
 
 from bids.layout.models import (BIDSFile, Entity, Tag, Base, Config,
                                 FileAssociation, BIDSImageFile, LayoutInfo)
+from bids.layout.utils import PaddedInt
 from bids.tests import get_test_data_path
 
 
@@ -129,9 +130,11 @@ def test_tag_dtype(sample_bidsfile, subject_entity):
         Tag(f, e, '4', 'int'),
         Tag(f, e, '4', int),
         Tag(f, e, 4),
-        Tag(file=f, entity=e, dtype=int, value='4')
+        Tag(file=f, entity=e, dtype=int, value='4'),
+        Tag(file=f, entity=e, dtype=int, value='04'),
     ]
-    assert all([t.dtype == int for t in tags])
+    assert all(t.dtype == PaddedInt for t in tags)
+    assert all(t.value == 4 for t in tags)
 
 
 def test_entity_add_file(sample_bidsfile):
diff --git a/bids/tests/data/ds005/sub-01/func/sub-01_task-mixedgamblestask_run-01_events.json b/bids/tests/data/ds005/sub-01/func/sub-01_task-mixedgamblestask_run-01_events.json
index 552bd68e..3aeeae62 100644
--- a/bids/tests/data/ds005/sub-01/func/sub-01_task-mixedgamblestask_run-01_events.json
+++ b/bids/tests/data/ds005/sub-01/func/sub-01_task-mixedgamblestask_run-01_events.json
@@ -1,3 +1,3 @@
 {
-    "run": 1
-}
\ No newline at end of file
+    "run": "01"
+}