Repository: bids-standard/pybids

Issue:
Analysis: Block.get_design_matrix not grouping by run level
The following bug is coming up when using fitlins on the `generic_object_decoding` dataset on OpenNeuro. (cc: @effigies )

I'm attempting to fit the following model.

```
{
  "name": "test_god_simple",
  "input": {
    "task": "perception",
    "run": "1"
  },
  "blocks": [
    {
      "level": "run",
      "model": {
        "variables": [
          "response_time"
        ],
        "HRF_variables": [
          "response_time"
        ]
      }
    }
  ],
  "description": "Simple GOD"
}
```

fitlins does something like following to set up the model:

    analysis = Analysis(model='/analysis/model_orig.json',
                           layout=BIDSLayout(['/datasets/generic_object_decoding/'])
    analysis.setup(**analysis.model['input'])

The problem comes when trying to get the design matrices in the first 'run' block (in the `first_model` function of `fitlins`). Presumably, the following line should return a list of design matrices and entities for every run that matches "input":

    dms = analysis[0].get_design_matrix(analysis[0].model['HRF_variables'], mode='sparse')

This returns 40 design matrices, only for run=1, but the entity `run` is not listed among the entities. For example, the entities for dms[0] = `{'session': 'perceptionTest01', 'subject': '01', 'task': 'perception'}`
This causes an error in fitlins because preproc files are not subsequently filtered correctly.

Confusing, if I change the json model to omit `run` from input, I get a similar result from `get_design_matrix` with 40 design matrices being return. But now, the design matrix dataframes include events for all runs (10 in this case). I would have expected a design matrix per run, even if I was not filtering on runs. In this case, entities also does not include `run` as a key. 

Any ideas? Am I correct in thinking `get_design_matrix` should always return a result for each run individually? 

I'll do more digging and see if I can replicate this result on another dataset
Analysis: Block.get_design_matrix not grouping by run level
The following bug is coming up when using fitlins on the `generic_object_decoding` dataset on OpenNeuro. (cc: @effigies )

I'm attempting to fit the following model.

```
{
  "name": "test_god_simple",
  "input": {
    "task": "perception",
    "run": "1"
  },
  "blocks": [
    {
      "level": "run",
      "model": {
        "variables": [
          "response_time"
        ],
        "HRF_variables": [
          "response_time"
        ]
      }
    }
  ],
  "description": "Simple GOD"
}
```

fitlins does something like following to set up the model:

    analysis = Analysis(model='/analysis/model_orig.json',
                           layout=BIDSLayout(['/datasets/generic_object_decoding/'])
    analysis.setup(**analysis.model['input'])

The problem comes when trying to get the design matrices in the first 'run' block (in the `first_model` function of `fitlins`). Presumably, the following line should return a list of design matrices and entities for every run that matches "input":

    dms = analysis[0].get_design_matrix(analysis[0].model['HRF_variables'], mode='sparse')

This returns 40 design matrices, only for run=1, but the entity `run` is not listed among the entities. For example, the entities for dms[0] = `{'session': 'perceptionTest01', 'subject': '01', 'task': 'perception'}`
This causes an error in fitlins because preproc files are not subsequently filtered correctly.

Confusing, if I change the json model to omit `run` from input, I get a similar result from `get_design_matrix` with 40 design matrices being return. But now, the design matrix dataframes include events for all runs (10 in this case). I would have expected a design matrix per run, even if I was not filtering on runs. In this case, entities also does not include `run` as a key. 

Any ideas? Am I correct in thinking `get_design_matrix` should always return a result for each run individually? 

I'll do more digging and see if I can replicate this result on another dataset


Discussion:
I'm not able to look at this just now, but one thought is that IIRC, `get_design_matrix` doesn't return a run entity value if run is not explicitly included in the filename. So that might be the issue. But I think there was a good reason for doing it that way, so I wouldn't call this a bug (though we might need a workaround here). If that's not the problem, then I'm not sure...
Oh and you're right that you should always be getting a single run per list element. Will look into that when I can if Chris doesn't figure it out first.
In this dataset, run values are explicitly in the filenames, and the it passes BIDS validaton: https://openneuro.org/datasets/ds001246/versions/00002?app=FMRIPREP&version=54&job=5ab2f56ed5151800014bcc3a
Okay I've found the bug. It is happening in `kollekshuns.BIDSVariableCollection._index_entities`

That function omits entities if they vary across the collection. It turns out run was being represented as both a string and an integer within a set of variables. Specifically, `SparseRunVariables` represented run as an int, and `DenseRunVariables` as a string.

This commission then propagated all the way down, and caused the issues I outlined above. In the case of a more than one type of run, it caused incorrect grouping of variables at higher levels (grouping them together, rather than one per run).  

Seems as they should be a string in both correct?
Ah, thanks, that makes sense. They should actually all be integers; the spec requires `run` values to be ints. What's not clear is why they're being assigned as strings; can you track that down? In grabbit, the `run` entity now has type `int`, and conversions to int should be internally forced when first reading the files in, so this shouldn't be happening at all.
It's actually coming from `BIDSVariable._index_entities`. In the case of a Dense variable, it uses the index to build a list of entities:

https://github.com/INCF/pybids/blob/d481dda98c7e3b38b4a812124daaf0d450d64e19/bids/variables/variables.py#L210

The dtype for self.index['run'] is object, so its coming out as a string rather than an integer.

I'll track down a bit further, but I'm guessing making sure the dtype is int when building the DenseIndex should fix the issue. 
I'm not able to look at this just now, but one thought is that IIRC, `get_design_matrix` doesn't return a run entity value if run is not explicitly included in the filename. So that might be the issue. But I think there was a good reason for doing it that way, so I wouldn't call this a bug (though we might need a workaround here). If that's not the problem, then I'm not sure...
Oh and you're right that you should always be getting a single run per list element. Will look into that when I can if Chris doesn't figure it out first.
In this dataset, run values are explicitly in the filenames, and the it passes BIDS validaton: https://openneuro.org/datasets/ds001246/versions/00002?app=FMRIPREP&version=54&job=5ab2f56ed5151800014bcc3a
Okay I've found the bug. It is happening in `kollekshuns.BIDSVariableCollection._index_entities`

That function omits entities if they vary across the collection. It turns out run was being represented as both a string and an integer within a set of variables. Specifically, `SparseRunVariables` represented run as an int, and `DenseRunVariables` as a string.

This commission then propagated all the way down, and caused the issues I outlined above. In the case of a more than one type of run, it caused incorrect grouping of variables at higher levels (grouping them together, rather than one per run).  

Seems as they should be a string in both correct?
Ah, thanks, that makes sense. They should actually all be integers; the spec requires `run` values to be ints. What's not clear is why they're being assigned as strings; can you track that down? In grabbit, the `run` entity now has type `int`, and conversions to int should be internally forced when first reading the files in, so this shouldn't be happening at all.
It's actually coming from `BIDSVariable._index_entities`. In the case of a Dense variable, it uses the index to build a list of entities:

https://github.com/INCF/pybids/blob/d481dda98c7e3b38b4a812124daaf0d450d64e19/bids/variables/variables.py#L210

The dtype for self.index['run'] is object, so its coming out as a string rather than an integer.

I'll track down a bit further, but I'm guessing making sure the dtype is int when building the DenseIndex should fix the issue. 

Resolving patch:
diff --git a/bids/variables/variables.py b/bids/variables/variables.py
--- a/bids/variables/variables.py
+++ b/bids/variables/variables.py
@@ -398,8 +398,7 @@ def _build_entity_index(self, run_info, sampling_rate):
         for run in run_info:
             reps = int(math.ceil(run.duration * sampling_rate))
             ent_vals = list(run.entities.values())
-            data = np.broadcast_to(ent_vals, (reps, len(ent_vals)))
-            df = pd.DataFrame(data, columns=list(run.entities.keys()))
+            df = pd.DataFrame([ent_vals] * reps, columns=list(run.entities.keys()))
             ts = pd.date_range(0, periods=len(df), freq='%sms' % sr)
             _timestamps.append(ts.to_series())
             index.append(df)


Test patch: