anon-iclr-submission/benchname-bug-localization huggingface dataset:
Repository: nf-core/tools
Issue: nf-core modules install has duplicated text in message

### Description of the bug

nf-core modules install gives a message with the text
"Recomputing commit SHAs for modules **which which were were** missing from"
The duplicated words should be removed.

### Command used and terminal output

_No response_

### System information

_No response_

Changed files:
nf_core/modules/modules_command.py

diff --git a/nf_core/modules/modules_command.py b/nf_core/modules/modules_command.py
index dbfff111..a60291fc 100644
--- a/nf_core/modules/modules_command.py
+++ b/nf_core/modules/modules_command.py
@@ -207,7 +207,7 @@ class ModuleCommand:
                 log.info(f"Recomputing commit SHA for module {format_missing[0]} which was missing from 'modules.json'")
             else:
                 log.info(
-                    f"Recomputing commit SHAs for modules which which were were missing from 'modules.json': {', '.join(format_missing)}"
+                    f"Recomputing commit SHAs for modules which were missing from 'modules.json': {', '.join(format_missing)}"
                 )
             failed_to_find_commit_sha = []
             for repo, modules in missing_from_modules_json.items():
Repository: nf-core/tools
Issue: Use of future functionality

### Description of the bug

The `shutil.copytree` argument [`dirs_exist_ok`](https://docs.python.org/3/library/shutil.html#shutil.copytree) is only available in Python version 3.8+ but nf-core tool still support Python 3.7

https://github.com/nf-core/tools/blob/b5400d84d307343353b6ab09aad204231f74fb0e/nf_core/modules/lint/module_changes.py#L28

Two options: drop Python 3.7 support or change that code.


Changed files:
nf_core/modules/lint/module_changes.py

diff --git a/nf_core/modules/lint/module_changes.py b/nf_core/modules/lint/module_changes.py
index 04a250d7..b6e99781 100644
--- a/nf_core/modules/lint/module_changes.py
+++ b/nf_core/modules/lint/module_changes.py
@@ -25,7 +25,7 @@ def module_changes(module_lint_object, module):
         # If the module is patched, we need to apply
         # the patch in reverse before comparing with the remote
         tempdir = Path(tempfile.mkdtemp())
-        shutil.copytree(module.module_dir, tempdir, dirs_exist_ok=True)
+        shutil.copytree(module.module_dir, tempdir)
         try:
             new_lines = ModulesDiffer.try_apply_patch(
                 module.module_name, module_lint_object.modules_repo.fullname, module.patch_path, tempdir, reverse=True
Repository: nf-core/tools
Issue: nf-core lint throws exception when --dir is used

Running `nf-core lint` throws the following exception when using the `--dir` option

## Steps to reproduce

```bash
$ nf-core lint --dir nf-core-testpipeline

                                          ,--./,-.
          ___     __   __   __   ___     /,-._.--~\\
    |\\ | |__  __ /  ` /  \\ |__) |__         }  {
    | \\| |       \\__, \\__/ |  \\ |___     \\`-._,-`-,
                                          `._,._,'

    nf-core/tools version 2.2.dev0



ERROR    Could not determine repository type of 'nf-core-testpipeline'                                                                                       __main__.py:345
```
If the lint is run from the pipeline folder it works.

## Expected behaviour

<!-- A clear and concise description of what you expected to happen. -->

## System

- OS: macOS
- Version of nf-core/tools: 2.2.dev0
- Python version: Tested on 3.7 and 3.8

## Additional context

The reason behind the error is that in #1322 the `os.path.join` was removed from the code.


Changed files:
nf_core/modules/module_utils.py

diff --git a/nf_core/modules/module_utils.py b/nf_core/modules/module_utils.py
index 6ef675bf..ce8a8027 100644
--- a/nf_core/modules/module_utils.py
+++ b/nf_core/modules/module_utils.py
@@ -344,8 +344,8 @@ def get_repo_type(dir):
         raise LookupError("Could not find directory: {}".format(dir))
 
     # Determine repository type
-    if os.path.exists("README.md"):
-        with open("README.md") as fh:
+    if os.path.exists(os.path.join(dir, "README.md")):
+        with open(os.path.join(dir, "README.md")) as fh:
             if fh.readline().rstrip().startswith("# ![nf-core/modules]"):
                 return "modules"
             else:
Repository: nf-core/tools
Issue: nf-code modules lint failing when all modules up to date

nf-code modules lint failing when all modules up to date

This is due to a failed messages that is not turned into a `LintResult` object

Changed files:
tests/test_modules.py, nf_core/modules/lint.py

diff --git a/nf_core/modules/lint.py b/nf_core/modules/lint.py
index 909dd911..c5dc501a 100644
--- a/nf_core/modules/lint.py
+++ b/nf_core/modules/lint.py
@@ -375,7 +375,6 @@ class ModuleLint(object):
         Downloads the 'main.nf', 'functions.nf' and 'meta.yml' files for every module
         and compare them to the local copies
         """
-        all_modules_up_to_date = True
         files_to_check = ["main.nf", "functions.nf", "meta.yml"]
 
         progress_bar = rich.progress.Progress(
@@ -420,7 +419,6 @@ class ModuleLint(object):
                             remote_copy = r.content.decode("utf-8")
 
                             if local_copy != remote_copy:
-                                all_modules_up_to_date = False
                                 self.warned.append(
                                     LintResult(
                                         mod,
@@ -429,6 +427,15 @@ class ModuleLint(object):
                                         f"{os.path.join(mod.module_dir, f)}",
                                     )
                                 )
+                            else:
+                                self.passed.append(
+                                    LintResult(
+                                        mod,
+                                        "check_local_copy",
+                                        "Local copy of module up to date",
+                                        f"{os.path.join(mod.module_dir, f)}",
+                                    )
+                                )
                         except UnicodeDecodeError as e:
                             self.warned.append(
                                 LintResult(
@@ -439,9 +446,6 @@ class ModuleLint(object):
                                 )
                             )
 
-        if all_modules_up_to_date:
-            self.passed.append("All modules are up to date!")
-
 
 class NFCoreModule(object):
     """
diff --git a/tests/test_modules.py b/tests/test_modules.py
index 52962700..b4809dd2 100644
--- a/tests/test_modules.py
+++ b/tests/test_modules.py
@@ -83,7 +83,7 @@ class TestModules(unittest.TestCase):
         self.mods.install("fastqc")
         module_lint = nf_core.modules.ModuleLint(dir=self.pipeline_dir)
         module_lint.lint(print_results=False, all_modules=True)
-        assert len(module_lint.passed) == 16
+        assert len(module_lint.passed) == 18
         assert len(module_lint.warned) == 0
         assert len(module_lint.failed) == 0
Repository: oemof/tespy
Issue: ZeroDivisionError in iteration speed calculation

When simulating a very simple network, a ZeroDivisionError can occur within the calculation of the iteration speed (precisely: calculating iterations per second). This is due to the discrete nature of computational time keeping. When a simulation finishes quick enough, by chance the time stamps at the start and end are equal and subtraction yields a zero, which in turn results in a ZeroDivisionError.

Changed files:
tespy/networks/networks.py

diff --git a/tespy/networks/networks.py b/tespy/networks/networks.py
index 6084fc7e..137ca33e 100644
--- a/tespy/networks/networks.py
+++ b/tespy/networks/networks.py
@@ -1889,9 +1889,12 @@ class network:
             msg = ('Total iterations: ' + str(self.iter + 1) + ', '
                    'Calculation time: ' +
                    str(round(self.end_time - self.start_time, 1)) + ' s, '
-                   'Iterations per second: ' +
-                   str(round((self.iter + 1) /
-                             (self.end_time - self.start_time), 2)))
+                   'Iterations per second: ')
+            ips = 'inf'
+            if self.end_time != self.start_time:
+                ips = str(round(
+                    (self.iter + 1) / (self.end_time - self.start_time), 2))
+            msg += ips
             logging.debug(msg)
             if self.iterinfo:
                 print(msg)
Repository: bids-standard/pybids
Issue: BIDSLayoutIndexer not passed to derivative layouts

The following will not respect the indexer preference for the derivative layouts

````
    indexer = BIDSLayoutIndexer(index_metadata=False)
    layout = BIDSLayout(preproc_dir, derivatives=preproc_dir, indexer=indexer)
```


Changed files:
bids/layout/layout.py

diff --git a/bids/layout/layout.py b/bids/layout/layout.py
index e960987a..eaf09cbd 100644
--- a/bids/layout/layout.py
+++ b/bids/layout/layout.py
@@ -169,11 +169,12 @@ class BIDSLayout:
             self.connection_manager = ConnectionManager(
                 database_path, reset_database, config, init_args)
 
-            if indexer is None:
-                indexer = BIDSLayoutIndexer(
-                    validate=validate and not is_derivative, **indexer_kwargs
-                )
-            indexer(self)
+            # Do not overwrite indexer variable, so the same configuration is passed to
+            # add_derivatives() below
+            _indexer = indexer or BIDSLayoutIndexer(
+                validate=validate and not is_derivative, **indexer_kwargs
+            )
+            _indexer(self)
 
         # Add derivatives if any are found
         if derivatives:
@@ -184,7 +185,7 @@ class BIDSLayout:
                 validate=validate, absolute_paths=absolute_paths,
                 derivatives=None, sources=self, config=None,
                 regex_search=regex_search, reset_database=reset_database,
-                **indexer_kwargs)
+                indexer=indexer, **indexer_kwargs)
 
     @property
     def root(self):
Repository: bids-standard/pybids
Issue: Scale transformation shouldn't return NAs in event of constant input

If constant input is passed to the `Scale` transformation, a column of N/A values is returned. This should probably just fail with an exception (or at minimum, return a column of `0`).

Changed files:
bids/analysis/tests/test_transformations.py, bids/analysis/transformations/compute.py

diff --git a/bids/analysis/tests/test_transformations.py b/bids/analysis/tests/test_transformations.py
index cc3126dd..63eecaa3 100644
--- a/bids/analysis/tests/test_transformations.py
+++ b/bids/analysis/tests/test_transformations.py
@@ -128,7 +128,7 @@ def test_sum(collection):
                       output='sum', weights=[1, 1, 1])
 
 
-def test_scale(collection):
+def test_scale(collection, sparse_run_variable_with_missing_values):
     transform.Scale(collection, variables=['RT', 'parametric gain'],
                     output=['RT_Z', 'gain_Z'], groupby=['run', 'subject'])
     groupby = collection['RT'].get_grouper(['run', 'subject'])
@@ -137,6 +137,16 @@ def test_scale(collection):
         groupby).apply(lambda x: (x - x.mean()) / x.std())
     assert np.allclose(z1, z2)
 
+    # Test constant input
+    coll = sparse_run_variable_with_missing_values
+    coll['var'].values.fillna(1)
+    with pytest.raises(ValueError, match="Cannot scale.*1.0"):
+        transform.Scale(coll, 'var').values
+    with pytest.raises(ValueError, match="Cannot scale.*1.0"):
+        transform.Scale(coll, 'var', replace_na='before').values
+    transform.Scale(coll, 'var', replace_na='after', output='zero')
+    assert coll['zero'].values.unique() == 0
+
 
 def test_demean(collection):
     transform.Demean(collection, variables=['RT'], output=['RT_dm'])
diff --git a/bids/analysis/transformations/compute.py b/bids/analysis/transformations/compute.py
index 80f6232f..4c06a354 100644
--- a/bids/analysis/transformations/compute.py
+++ b/bids/analysis/transformations/compute.py
@@ -138,9 +138,19 @@ class Scale(Transformation):
         None, no replacement is performed. If 'before', missing values are
         replaced with 0's before scaling. If 'after', missing values are
         replaced with 0 after scaling.
+
+    Notes
+    -----
+    If a constant column is passed in, and replace_na is None or 'before', an
+    exception will be raised.
     """
 
     def _transform(self, data, demean=True, rescale=True, replace_na=None):
+        if data.nunique() == 1 and replace_na in {None, 'before'}:
+            val = data.unique()[0]
+            raise ValueError("Cannot scale a column with constant value ({})! "
+                             "If you want a constant column of 0's returned, "
+                             "set replace_na to 'after'.".format(val))
         if replace_na == 'before':
             data = data.fillna(0.)
         if demean:
Repository: bids-standard/pybids
Issue: model: JSON to dict modified key values for transformation

In ` Replace` transformation, you specify as a dict which variables to transform.

e.g.:

```
{'LIKELY': "5"}
```

However, the parser from JSON to dict to convert BIDS Stats Models modifies keys to lower case, which in the case of specific case sensitive values modifies the transformation itself.

Changed files:
bids/utils.py

diff --git a/bids/utils.py b/bids/utils.py
index 902f1e57..b4f2463c 100644
--- a/bids/utils.py
+++ b/bids/utils.py
@@ -44,9 +44,10 @@ def natural_sort(l, field=None):
 
 def convert_JSON(j):
     """ Recursively convert CamelCase keys to snake_case.
-    From: https://stackoverflow.com/questions/17156078/converting-identifier-naming-between-camelcase-and-underscores-during-json-seria
+    From: https://stackoverflow.com/questions/17156078/
+    converting-identifier-naming-between-camelcase-and-
+    underscores-during-json-seria
     """
-
     def camel_to_snake(s):
         a = re.compile('((?<=[a-z0-9])[A-Z]|(?!^)[A-Z](?=[a-z]))')
         return a.sub(r'_\\1', s).lower()
@@ -54,7 +55,7 @@ def convert_JSON(j):
     def convertArray(a):
         newArr = []
         for i in a:
-            if isinstance(i,list):
+            if isinstance(i, list):
                 newArr.append(convertArray(i))
             elif isinstance(i, dict):
                 newArr.append(convert_JSON(i))
@@ -66,7 +67,8 @@ def convert_JSON(j):
     for k, value in j.items():
         newK = camel_to_snake(k)
 
-        if isinstance(value, dict):
+        # Replace transformation uses a dict, so skip lower-casing
+        if isinstance(value, dict) and k != 'Replace':
             out[newK] = convert_JSON(value)
         elif isinstance(value, list):
             out[newK] = convertArray(value)
Repository: bids-standard/pybids
Issue: BIDS validation fails because of missing leading /

The BIDS validator regular expressions expect absolute paths, but we've been feeding relative ones. Most files have been passing due to a separate bug in the validator regular expressions, but that was purely luck (or lack thereof). The `BIDSValidator` class should use absolute paths when checking against all rules.

See #339 for context.

Changed files:
bids/layout/layout.py

diff --git a/bids/layout/layout.py b/bids/layout/layout.py
index b18dc962..8c799a08 100644
--- a/bids/layout/layout.py
+++ b/bids/layout/layout.py
@@ -305,17 +305,14 @@ class BIDSLayout(Layout):
         if not self.validate:
             return True
 
-        # For derivatives, we need to cheat a bit and construct a fake
-        # derivatives path--prepend 'derivatives' and the pipeline name
-        to_check = os.path.relpath(f, self.root)
+        # Derivatives are currently not validated.
         if 'derivatives' in self.domains:
-            to_check = os.path.join(
-                'derivatives', self.description['PipelineDescription']['Name'],
-                to_check)
+            return True
 
-        sep = os.path.sep
-        if to_check[:len(sep)] != sep:
-            to_check = sep + to_check
+        # BIDS validator expects absolute paths, but really these are relative
+        # to the BIDS project root.
+        to_check = os.path.relpath(f, self.root)
+        to_check = os.path.join(os.path.sep, to_check)
 
         return self.validator.is_bids(to_check)
Repository: bids-standard/pybids
Issue: BIDS Validator data not installed

When installing pybids through pip using an archive (e.g. tarball, zip, instead of from github with `-e`), the data for the `BIDSValidator` is not installed, causing `BIDSLayout` to fail initating
```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/dist-packages/bids/layout/layout.py", line 143, in __init__
    **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/grabbit/core.py", line 419, in __init__
    self.index()
  File "/usr/local/lib/python3.6/dist-packages/grabbit/core.py", line 610, in index
    _index_dir(path, list(domains))
  File "/usr/local/lib/python3.6/dist-packages/grabbit/core.py", line 603, in _index_dir
    elif self._validate_file(full_path):
  File "/usr/local/lib/python3.6/dist-packages/bids/layout/layout.py", line 204, in _validate_file
    return self.validator.is_bids(to_check)
  File "/usr/local/lib/python3.6/dist-packages/bids/layout/validation.py", line 64, in is_bids
    conditions.append(self.is_top_level(path))
  File "/usr/local/lib/python3.6/dist-packages/bids/layout/validation.py", line 75, in is_top_level
    with open(join(self.rule_dir, 'fixed_top_level_names.json'), 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/lib/python3.6/dist-packages/bids/layout/config/validator/fixed_top_level_names.json'
```

Changed files:
bids/version.py, bids/layout/validation.py

diff --git a/bids/layout/validation.py b/bids/layout/validation.py
index 7f10ac77..6bd5ef18 100644
--- a/bids/layout/validation.py
+++ b/bids/layout/validation.py
@@ -35,8 +35,7 @@ class BIDSValidator():
     """
 
     def __init__(self, index_associated=True):
-        self.rule_dir = join(dirname(abspath(__file__)), 'config',
-                              'validator')
+        self.rule_dir = join(dirname(abspath(__file__)),'config', 'validator')
         self.index_associated = index_associated
 
     def is_bids(self, path):
diff --git a/bids/version.py b/bids/version.py
index aa881794..57b957e2 100644
--- a/bids/version.py
+++ b/bids/version.py
@@ -65,7 +65,7 @@ def package_files(directory):
 
 extra_files = package_files('path_to/extra_files_dir')
 PACKAGE_DATA = {
-    'bids.layout': ['config/*.json'],
+    'bids.layout': ['config/*.json', 'config/validator/*.json'],
     'bids.reports': ['config/*.json'],
     'bids': package_files('bids/tests/data')
 }
Repository: bids-standard/pybids
Issue: variables: BIDSVariableCollection.to_df loses n/as

This relates to: https://github.com/poldracklab/fitlins/issues/56

If there are `n/a`s in a `Sparse` variable, and `get_design_collection` is called with `force=True`, missing values are ommited (even if `Analysis.setup` was called with `drop_na=False`, and the variables originally had the `n/a`s)

`BIDSVariableCollection.to_df` does have a `fillna` argument (which by default is `nan`), but the following line removes `n/a`s prior to this: 

https://github.com/INCF/pybids/blob/87015d897ab881d0db47994595d63bba06daa6ab/bids/variables/kollekshuns.py#L116


Changed files:
bids/variables/kollekshuns.py

diff --git a/bids/variables/kollekshuns.py b/bids/variables/kollekshuns.py
index fdaf8f92..ab76c83d 100644
--- a/bids/variables/kollekshuns.py
+++ b/bids/variables/kollekshuns.py
@@ -113,9 +113,11 @@ class BIDSVariableCollection(object):
             return df.reset_index(drop=True).fillna(fillna)
 
         ind_cols = list(set(df.columns) - {'condition', 'amplitude'})
+
+        df['amplitude'] = df['amplitude'].fillna('n/a')
         df = df.pivot_table(index=ind_cols, columns='condition',
                             values='amplitude', aggfunc='first')
-        df = df.reset_index().fillna(fillna)
+        df = df.reset_index().replace('n/a', fillna)
         df.columns.name = None
         return df
Repository: bids-standard/pybids
Issue: analysis: groupby defaults to included "run", but could be None

When applying transformations, it seems the default groupby is `['run', 'subject']`.

However, this is can be a problem because some runs dont have a run number (if there is only one run, it seems to be optional under the BIDS spec). In this case, a pandas `KeyError` will occur. 

Changed files:
bids/analysis/tests/test_transformations.py, bids/analysis/transformations/base.py

diff --git a/bids/analysis/tests/test_transformations.py b/bids/analysis/tests/test_transformations.py
index a08d5d6d..cad35f84 100644
--- a/bids/analysis/tests/test_transformations.py
+++ b/bids/analysis/tests/test_transformations.py
@@ -39,7 +39,7 @@ def test_product(collection):
 
 def test_scale(collection):
     transform.scale(collection, variables=['RT', 'parametric gain'],
-                    output=['RT_Z', 'gain_Z'])
+                    output=['RT_Z', 'gain_Z'], groupby=['run', 'subject'])
     ents = collection['RT'].index
     groupby = pd.core.groupby._get_grouper(ents, ['run', 'subject'])[0]
     z1 = collection['RT_Z'].values
@@ -79,7 +79,7 @@ def test_orthogonalize_sparse(collection):
     pg_pre = collection['parametric gain'].values
     rt = collection['RT'].values
     transform.orthogonalize(collection, variables='parametric gain',
-                            other='RT')
+                            other='RT', groupby=['run', 'subject'])
     pg_post = collection['parametric gain'].values
     vals = np.c_[rt.values, pg_pre.values, pg_post.values]
     df = pd.DataFrame(vals, columns=['rt', 'pre', 'post'])
diff --git a/bids/analysis/transformations/base.py b/bids/analysis/transformations/base.py
index 58d739e6..ca2965f4 100644
--- a/bids/analysis/transformations/base.py
+++ b/bids/analysis/transformations/base.py
@@ -91,7 +91,7 @@ class Transformation(object):
         >= 3.3, we can't override both new and init. '''
         self.collection = collection
         self.variables = listify(variables)
-        self.groupby = kwargs.pop('groupby', ['run', 'subject'])
+        self.groupby = kwargs.pop('groupby', None)
         self.output = listify(kwargs.pop('output', None))
         self.output_prefix = kwargs.pop('output_prefix', None)
         self.output_suffix = kwargs.pop('output_suffix', None)
Repository: bids-standard/pybids
Issue: BIDSLayout.get_nearest_helper() does not anticipate absent 'type' entity

Happens with openfmri ds000002 due to contained derivatives:

```
  File "/home/mih/.local/lib/python2.7/site-packages/bids/grabbids/bids_layout.py", line 45, in get_metadata
    potentialJSONs = self.get_nearest_helper(path, '.json', **kwargs)
  File "/home/mih/.local/lib/python2.7/site-packages/bids/grabbids/bids_layout.py", line 37, in get_nearest_helper
    type = self.files[path].entities['type']
KeyError: 'type'
()
> /home/mih/.local/lib/python2.7/site-packages/bids/grabbids/bids_layout.py(37)get_nearest_helper()
-> type = self.files[path].entities['type']
(Pdb) self.files[path].entities
{u'subject': '01'}
(Pdb) path
u'/tmp/openfmri/ds000002/derivatives/mriqc/anatomical_sub-01.pdf'
```

Changed files:
bids/grabbids/bids_layout.py

diff --git a/bids/grabbids/bids_layout.py b/bids/grabbids/bids_layout.py
index b2b27730..64f0dc37 100644
--- a/bids/grabbids/bids_layout.py
+++ b/bids/grabbids/bids_layout.py
@@ -62,7 +62,11 @@ class BIDSLayout(Layout):
                 "BIDS project." % path)
 
         if not type:
-            # Constrain the search to .json files with the same type as target
+            if 'type' not in self.files[path].entities:
+                raise ValueError(
+                    "File '%s' does not have a valid type definition, most "
+                    "likely because it is not a valid BIDS file." % path
+                )
             type = self.files[path].entities['type']
 
         tmp = self.get_nearest(path, extensions=extension, all_=True,
Repository: bids-standard/pybids
Issue: Fieldmaps - _magnitude2.nii[.gz] is optional

As per these lines:

https://github.com/INCF/pybids/blob/800d15053952991c9cd4a00cf0039288d489ca12/bids/grabbids/bids_layout.py#L143-L148

when a fieldmap type is identified as `'phasediff'` then pybids understands that there is always a `_magnitude2.nii[.gz]` file. However, this is maked as optional in the specification.

Changed files:
bids/grabbids/bids_layout.py

diff --git a/bids/grabbids/bids_layout.py b/bids/grabbids/bids_layout.py
index a4b3dc34..e6a0df65 100644
--- a/bids/grabbids/bids_layout.py
+++ b/bids/grabbids/bids_layout.py
@@ -143,9 +143,11 @@ class BIDSLayout(Layout):
                         cur_fieldmap = {"phasediff": file.filename,
                                         "magnitude1": file.filename.replace(
                                             "phasediff", "magnitude1"),
-                                        "magnitude2": file.filename.replace(
-                                            "phasediff", "magnitude2"),
                                         "type": "phasediff"}
+                        magnitude2 = file.filename.replace(
+                            "phasediff", "magnitude2")
+                        if os.path.isfile(magnitude2):
+                            cur_fieldmap['magnitude2'] = magnitude2
                     elif file.type == "phase1":
                         cur_fieldmap["phase1"] = file.filename
                         cur_fieldmap["magnitude1"] = \\
Repository: choderalab/openmmtools
Issue: SamplerState.apply_to_context() erroneously applies box-vectors *after* positions are set

A major bug is present in [the code for `SamplerState.apply_to_context()`](https://github.com/choderalab/openmmtools/blob/master/openmmtools/states.py#L1701-L1721):
```python
        context.setPositions(self._positions)
        if self._velocities is not None and not ignore_velocities:
            context.setVelocities(self._velocities)
        if self.box_vectors is not None:
            context.setPeriodicBoxVectors(*self.box_vectors)
```
On Reference, this doesn't cause problems, but on OpenCL, the [`setPositions`](OpenCLUpdateStateDataKernel) method causes a call to [`reorderAtoms()](https://github.com/pandegroup/openmm/blob/7cdd6d16348b051e97f162580f908ace13be0e25/platforms/opencl/src/OpenCLKernels.cpp#L267-L269) which [uses the periodic box vectors](https://github.com/pandegroup/openmm/blob/33e2c3cd52af27ae13df06e4769bf168cae882d5/platforms/cuda/src/CudaContext.cpp#L1256-L1289). CUDA is presumably the same as OpenCL.

Changed files:
openmmtools/states.py

diff --git a/openmmtools/states.py b/openmmtools/states.py
index b5498f4..c185e12 100644
--- a/openmmtools/states.py
+++ b/openmmtools/states.py
@@ -1714,11 +1714,12 @@ class SamplerState(object):
             Context only to compute energies.
 
         """
+        # NOTE: Box vectors MUST be updated before positions are set.
+        if self.box_vectors is not None:
+            context.setPeriodicBoxVectors(*self.box_vectors)
         context.setPositions(self._positions)
         if self._velocities is not None and not ignore_velocities:
             context.setVelocities(self._velocities)
-        if self.box_vectors is not None:
-            context.setPeriodicBoxVectors(*self.box_vectors)
 
     def has_nan(self):
         """Check that energies and positions are finite.
Repository: coronasafe/care
Issue: Local developer setup: Add example District.id value to the README

On trying to deploy this application locally for development, I'm not able to proceed with creating a superuser because it asks for a `District (District.id):` Please add an example value  to the README.

Changed files:
care/users/models.py

diff --git a/care/users/models.py b/care/users/models.py
index 7b2c6718..20a3a294 100644
--- a/care/users/models.py
+++ b/care/users/models.py
@@ -81,6 +81,12 @@ class CustomUserManager(UserManager):
         qs = super().get_queryset()
         return qs.filter(deleted=False)
 
+    def create_superuser(self, username, email, password, **extra_fields):
+        district_id = extra_fields["district"]
+        district = District.objects.get(id=district_id)
+        extra_fields["district"] = district
+        return super().create_superuser(username, email, password, **extra_fields)
+
 
 class Skill(models.Model):
     name = models.CharField(max_length=255)
Repository: choderalab/openmmtools
Issue: CustomExternalForce should use periodicdistance to avoid major artifacts when heavy atoms are restrained to fixed points in space

cc: https://github.com/pandegroup/openmm/issues/1854#issuecomment-346202857

In order to avoid major artifacts when using restraints to fixed points in space via `CustomExternalForce` (in [`restrain_atoms`](https://github.com/choderalab/openmmtools/blob/f50f0844b28106978973e6374613bc8d24a3c4b9/openmmtools/forcefactories.py#L101)), for periodic systems, we should replace
```python
restraint_force = mm.CustomExternalForce('(K/2)*((x-x0)^2 + (y-y0)^2 + (z-z0)^2)')
```
with
```python
restraint_force = mm.CustomExternalForce('(K/2)*periodicdistance(x, y, z, x0, y0, z0)^2')
```
as suggested in [this comment](https://github.com/pandegroup/openmm/issues/1854#issuecomment-346493353).

The relevant code is [here](https://github.com/choderalab/openmmtools/blob/f50f0844b28106978973e6374613bc8d24a3c4b9/openmmtools/forcefactories.py#L158).

Changed files:
openmmtools/forcefactories.py

diff --git a/openmmtools/forcefactories.py b/openmmtools/forcefactories.py
index 8c59aef..3ebca36 100644
--- a/openmmtools/forcefactories.py
+++ b/openmmtools/forcefactories.py
@@ -155,7 +155,11 @@ def restrain_atoms(thermodynamic_state, sampler_state, restrained_atoms, sigma=3
         sampler_state.positions -= centroid * distance_unit
 
     # Create a CustomExternalForce to restrain all atoms.
-    restraint_force = openmm.CustomExternalForce('(K/2)*((x-x0)^2 + (y-y0)^2 + (z-z0)^2)')
+    if thermodynamic_state.is_periodic:
+        energy_expression = '(K/2)*periodicdistance(x, y, z, x0, y0, z0)^2' # periodic distance
+    else:
+        energy_expression = '(K/2)*((x-x0)^2 + (y-y0)^2 + (z-z0)^2)' # non-periodic distance
+    restraint_force = openmm.CustomExternalForce(energy_expression)
     # Adding the spring constant as a global parameter allows us to turn it off if desired
     restraint_force.addGlobalParameter('K', K)
     restraint_force.addPerParticleParameter('x0')
Repository: pythonindia/junction
Issue: edit proposal view - post request is throwing 405 error

Just tried to edit a proposal on http://junction.pybelgaum.org, it throwed 405 error? 


Changed files:
junction/proposals/proposal_urls.py

diff --git a/junction/proposals/proposal_urls.py b/junction/proposals/proposal_urls.py
index c47dd27..16afe15 100644
--- a/junction/proposals/proposal_urls.py
+++ b/junction/proposals/proposal_urls.py
@@ -11,7 +11,7 @@ urlpatterns = patterns(
 
     url(r'^$', views.list_proposals, name='proposals-list'),
     url(r'^create/$', views.create_proposal, name='proposal-create'),
-    url(r'^(?P<slug>[\\w-]+)/update$',
+    url(r'^(?P<slug>[\\w-]+)/update/$',
         views.update_proposal, name='proposal-update'),
     url(r'^(?P<slug>[\\w-]+)/$', views.detail_proposal, name='proposal-detail'),
     url(r'^(?P<slug>[\\w-]+)/delete/$',
Repository: pythonindia/junction
Issue: Bug: Proposals To Be Reviewed displays even drafted proposal

Proposals To Be Reviewed section filter need to correct to display on public proposal.


Changed files:
junction/proposals/views.py

diff --git a/junction/proposals/views.py b/junction/proposals/views.py
index cee16bf..df3eb1a 100644
--- a/junction/proposals/views.py
+++ b/junction/proposals/views.py
@@ -69,7 +69,8 @@ def list_proposals(request, conference_slug):
                                       ProposalSectionReviewer.objects.filter(
                                           conference_reviewer__reviewer=request.user)]
         proposals_to_review = [p for p in proposals_qs
-                               if p.proposal_section in proposal_reviewer_sections]
+                               if p.proposal_section in proposal_reviewer_sections and
+                               p.status == PROPOSAL_STATUS_PUBLIC]
 
     # Filtering
     proposal_section_filter = request.GET.getlist('proposal_section')
Repository: missionpinball/mpf
Issue: Fix crash in high_score mode

Crash when starting high acores mode after game enderd:

```
Traceback (most recent call last):
  File "c:\\users\\wilder\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\mpf\\commands\\game.py", line 220, in __init__
    MachineController(mpf_path, machine_path, vars(self.args)).run()
  File "c:\\users\\wilder\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\mpf\\core\\machine.py", line 709, in run
    self._run_loop()
  File "c:\\users\\wilder\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\mpf\\core\\machine.py", line 769, in _run_loop
    raise self._exception['exception']
  File "c:\\users\\wilder\\appdata\\local\\programs\\python\\python36\\lib\\asyncio\\events.py", line 145, in _run
    self._callback(*self._args)
  File "c:\\users\\wilder\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\mpf\\core\\async_mode.py", line 39, in _done
    future.result()
  File "c:\\users\\wilder\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\mpf\\modes\\high_score\\code\\high_score.py", line 127, in _run
    if not self.machine.game.player_list:
AttributeError: 'NoneType' object has no attribute 'player_list'
```

Changed files:
mpf/tests/test_HighScoreMode.py, mpf/modes/high_score/code/high_score.py

diff --git a/mpf/modes/high_score/code/high_score.py b/mpf/modes/high_score/code/high_score.py
index cbaf6c6a8..088ca061d 100644
--- a/mpf/modes/high_score/code/high_score.py
+++ b/mpf/modes/high_score/code/high_score.py
@@ -124,7 +124,8 @@ class HighScore(AsyncMode):
     @asyncio.coroutine
     def _run(self) -> Generator[int, None, None]:
         """Run high score mode."""
-        if not self.machine.game.player_list:
+        if not self.machine.game or not self.machine.game.player_list:
+            self.log.warning("High Score started but there was no game. Will not start.")
             return
 
         new_high_score_list = {}
diff --git a/mpf/tests/test_HighScoreMode.py b/mpf/tests/test_HighScoreMode.py
index 3549565a0..b30082df8 100644
--- a/mpf/tests/test_HighScoreMode.py
+++ b/mpf/tests/test_HighScoreMode.py
@@ -29,6 +29,15 @@ class TestHighScoreMode(MpfBcpTestCase):
             self.machine.game.request_player_add()
             self.advance_time_and_run()
 
+    def test_high_score_without_game(self):
+        # try to start high score mode without a running game
+        self.assertIn("high_score", self.machine.modes)
+        self.assertModeNotRunning("high_score")
+        self.machine.modes["high_score"].start()
+        self.advance_time_and_run()
+        # mode should not start (neither crash)
+        self.assertModeNotRunning("high_score")
+
     def test_default_high_scores(self):
         new_score_data = OrderedDict()
         new_score_data['score'] = [('BRI', 4242),
Repository: missionpinball/mpf
Issue: Runtime crash with bogus score entry

MPF crashes during runtime. We should either refuse a broken score entry on boot/validate or it should work.

```
DEBUG : Events : ============== EVENTS QUEUE =============
DEBUG : Events : sw_playfield_active, None, None, {}
DEBUG : Events : s_stage_target_inactive, None, None, {}
DEBUG : Events : player_stage_jackpot, None, None, {'change': 1000, 'player_num': 1, 'value': 1100, 'prev_value': 100}
DEBUG : Events : mode_stage_stage_jackpot_score, None, None, {'change': 0, 'value': 1000, 'prev_value': 1000}
DEBUG : Events : =========================================
ERROR : root : need more than 1 value to unpack
Traceback (most recent call last):
 File "/home/pi/mpf/mpf/commands/core.py", line 133, in __init__
   MachineController(mpf_path, machine_path, vars(args)).run()
 File "/home/pi/mpf/mpf/core/machine.py", line 544, in run
   self._run_loop()
 File "/home/pi/mpf/mpf/core/machine.py", line 565, in _run_loop
   self.process_frame()
 File "/home/pi/mpf/mpf/core/machine.py", line 580, in process_frame
   self.events.process_event_queue()
 File "/home/pi/mpf/mpf/core/events.py", line 477, in process_event_queue
   **event[3])
 File "/home/pi/mpf/mpf/core/events.py", line 412, in _process_event
   result = handler[0](**merged_kwargs)
 File "/home/pi/mpf/mpf/core/scoring.py", line 70, in _score_event_callback
   value, block = value.split('|')
ValueError: need more than 1 value to unpack
```


Changed files:
mpf/core/scoring.py

diff --git a/mpf/core/scoring.py b/mpf/core/scoring.py
index e4878f2dd..0e0d29cad 100644
--- a/mpf/core/scoring.py
+++ b/mpf/core/scoring.py
@@ -25,6 +25,22 @@ class ScoreController(object):
         self.mode_configs = OrderedDict()
         self.mode_scores = dict()
 
+    def validate_entry(self, entry, mode):
+        # TODO: better validation at mode loading time
+        for value in entry.values():
+            if isinstance(value, int):
+                continue
+            elif isinstance(value, str):
+                try:
+                    value, block = value.split('|')
+                    int(value)
+                    if block != "block":
+                        raise AssertionError("Invalid action in scoring entry: {} in mode {}".format(
+                            entry, mode.name))
+                except ValueError:
+                    raise AssertionError("Invalid scoring entry: {} in mode {}".format(
+                        entry, mode.name))
+
     def mode_start(self, config, mode, priority, **kwargs):
         del kwargs
         self.mode_configs[mode] = config
@@ -32,7 +48,10 @@ class ScoreController(object):
         self.mode_configs = OrderedDict(sorted(iter(self.mode_configs.items()),
                                                key=lambda x: x[0].priority,
                                                reverse=True))
+
         for event in list(config.keys()):
+            self.validate_entry(config[event], mode)
+
             mode.add_mode_event_handler(event, self._score_event_callback,
                                         priority, event_name=event)
Repository: missionpinball/mpf
Issue: Asset pools not getting created

Both video pools and sound pools are not getting created, however image pools are working just fine (when all are specified in a configuration file).


Changed files:
mpf/core/assets.py, mpf/core/config_validator.py

diff --git a/mpf/core/assets.py b/mpf/core/assets.py
index 4b2c38699..fc089e6de 100644
--- a/mpf/core/assets.py
+++ b/mpf/core/assets.py
@@ -458,13 +458,11 @@ class AssetManager(object):
         for ac in [x for x in self._asset_classes
                    if x['pool_config_section']]:
 
-            if (ac['pool_config_section']) not in config:
-                return
-
-            for name, settings in config[ac['pool_config_section']].items():
-                getattr(self.machine, ac['attribute'])[name] = (
-                    ac['cls'].asset_group_class(self.machine, name, settings,
-                                                ac['cls']))
+            if (ac['pool_config_section']) in config:
+                for name, settings in config[ac['pool_config_section']].items():
+                    getattr(self.machine, ac['attribute'])[name] = (
+                        ac['cls'].asset_group_class(self.machine, name, settings,
+                                                    ac['cls']))
 
     def _load_mode_assets(self, config, priority, mode):
         # Called on mode start to load the assets that are set to automatically
diff --git a/mpf/core/config_validator.py b/mpf/core/config_validator.py
index 587bc7544..cb352fc5d 100644
--- a/mpf/core/config_validator.py
+++ b/mpf/core/config_validator.py
@@ -780,7 +780,6 @@ smartmatrix:
 sound_player:
     __valid_in__: machine, mode, show
     action: single|enum(play,stop)|play
-    track: single|str|None
     volume: single|gain|None
     loops: single|int|None
     priority: single|int|None
@@ -803,6 +802,8 @@ sounds:
     loops: single|int|0
     priority: single|int|0
     max_queue_time: single|secs|None
+    events_when_played: list|str|None
+    events_when_stopped: list|str|None
     ducking:
         target: single|str|
         delay: single|str|0
Repository: missionpinball/mpf
Issue: update "both" command to use actual mpf call

Currently "mpf both" uses os.fork() to call "mpf mc" and "mpf game". But if you have mpf installed differently where you need to run, for example, "python3 -m mpf both", then it will fail since it calls "mpf mc" instead of "python3 -m mpf mc", etc.


Changed files:
mpf/commands/both.py

diff --git a/mpf/commands/both.py b/mpf/commands/both.py
index c1921ba27..678a220cd 100644
--- a/mpf/commands/both.py
+++ b/mpf/commands/both.py
@@ -1,11 +1,11 @@
 import os
 import platform
 import subprocess
+from importlib import import_module
 
 
 class Command(object):
     def __init__(self, mpf_path, machine_path, args):
-        del mpf_path
         if platform.system() == 'Windows':
             subprocess.Popen('mpf game {} {}'.format(machine_path,
                                                      ' '.join(args)))
@@ -13,8 +13,8 @@ class Command(object):
 
         else:
             if os.fork():
-                os.system('mpf game {} {}'.format(
-                    machine_path, ' '.join(args)))
+                module = import_module('mpf.commands.game')
+                module.Command(mpf_path, machine_path, args)
             else:
-                os.system('mpf mc {} {}'.format(
-                    machine_path, ' '.join(args)))
+                module = import_module('mpfmc.commands.mc')
+                module.Command(mpf_path, machine_path, args)
Repository: missionpinball/mpf
Issue: Actually call _pulse_done in ScoreReel

The call is documented in advance.
- Either delete method or call it


Changed files:
mpf/devices/score_reel.py

diff --git a/mpf/devices/score_reel.py b/mpf/devices/score_reel.py
index c907a66f7..59d0821fd 100644
--- a/mpf/devices/score_reel.py
+++ b/mpf/devices/score_reel.py
@@ -104,7 +104,6 @@ class ScoreReel(SystemWideDevice):
 
         This method also schedules delays to post the following events:
 
-        `reel_<name>_pulse_done`: When the coil is done pulsing
         `reel_<name>_ready`: When the config['repeat_pulse_time'] time is up
         `reel_<name>_hw_value`: When the config['hw_confirm_time'] time is up
 
@@ -171,13 +170,6 @@ class ScoreReel(SystemWideDevice):
                              "cannot roll over")
             return False
 
-    def _pulse_done(self):
-        # automatically called (via a delay) after the reel fires to post an
-        # event that the reel's coil is done pulsing
-        self.machine.events.post('reel_' + self.name + "_pulse_done")
-        '''event: reel_(name)_pulse_done
-        desc: The score real (name) is done pulsing.'''
-
     def _ready_to_fire(self):
         # automatically called (via a delay) after the reel fires to post an
         # event that the reel is ready to fire again
Repository: missionpinball/mpf
Issue: Score reels get stuck

https://groups.google.com/forum/#!topic/mpf-users/VUbET9o8K0s

Changed files:
mpf/devices/score_reel_group.py

diff --git a/mpf/devices/score_reel_group.py b/mpf/devices/score_reel_group.py
index 251f9df5f..0633595b9 100644
--- a/mpf/devices/score_reel_group.py
+++ b/mpf/devices/score_reel_group.py
@@ -639,6 +639,9 @@ class ScoreReelGroup(SystemWideDevice):
         # Watch for these reels going out of sync so we can turn off the lights
         # while they're resyncing
 
+        if self.unlight_on_resync_key:
+            self.machine.events.remove_handler_by_key(self.unlight_on_resync_key)
+
         self.unlight_on_resync_key = self.machine.events.add_handler(
             'scorereelgroup_' + self.name + '_resync',
             self.unlight,
@@ -656,6 +659,8 @@ class ScoreReelGroup(SystemWideDevice):
             light.off()
 
         if relight_on_valid:
+            if self.light_on_valid_key:
+                self.machine.events.remove_handler_by_key(self.light_on_valid_key)
             self.light_on_valid_key = self.machine.events.add_handler(
                 'scorereelgroup_' + self.name + '_valid',
                 self.light,
Repository: missionpinball/mpf
Issue: Improve flipper sw_flip/release

Support dual wound coils (maybe only via dual wound coil device?) 

Add sw_flip_and_release(ms) 


Changed files:
mpf/devices/flipper.py, mpf/tests/test_Flippers.py

diff --git a/mpf/devices/flipper.py b/mpf/devices/flipper.py
index 58eedae23..8951e38a3 100644
--- a/mpf/devices/flipper.py
+++ b/mpf/devices/flipper.py
@@ -195,14 +195,17 @@ class Flipper(SystemWideDevice):
         Note this method will keep this flipper enabled until you call
         sw_release().
         """
-        # todo add support for other types of flipper coils
         # Send the activation switch press to the switch controller
         self.machine.switch_controller.process_switch(
             name=self.config['activation_switch'].name,
             state=1,
             logical=True)
 
-        self.config['main_coil'].enable()
+        if self.config['hold_coil']:
+            self.config['main_coil'].pulse()
+            self.config['hold_coil'].enable()
+        else:
+            self.config['main_coil'].enable()
 
     def sw_release(self):
         """Deactive the flipper via software as if the flipper button was released.
@@ -217,3 +220,6 @@ class Flipper(SystemWideDevice):
 
         # disable the flipper coil(s)
         self.config['main_coil'].disable()
+
+        if self.config['hold_coil']:
+            self.config['hold_coil'].disable()
diff --git a/mpf/tests/test_Flippers.py b/mpf/tests/test_Flippers.py
index a35f12dac..d62074cfb 100644
--- a/mpf/tests/test_Flippers.py
+++ b/mpf/tests/test_Flippers.py
@@ -91,8 +91,20 @@ class TestFlippers(MpfTestCase):
 
         self.machine.coils.c_flipper_main.enable = MagicMock()
         self.machine.coils.c_flipper_main.disable = MagicMock()
-        self.machine.flippers.f_test_hold_eos.sw_flip()
+        self.machine.flippers.f_test_single.sw_flip()
         self.machine.coils.c_flipper_main.enable.assert_called_once_with()
 
+        self.machine.flippers.f_test_single.sw_release()
+        self.machine.coils.c_flipper_main.disable.assert_called_once_with()
+
+        self.machine.coils.c_flipper_main.pulse = MagicMock()
+        self.machine.coils.c_flipper_main.disable = MagicMock()
+        self.machine.coils.c_flipper_hold.enable = MagicMock()
+        self.machine.coils.c_flipper_hold.disable = MagicMock()
+        self.machine.flippers.f_test_hold_eos.sw_flip()
+        self.machine.coils.c_flipper_main.pulse.assert_called_once_with()
+        self.machine.coils.c_flipper_hold.enable.assert_called_once_with()
+
         self.machine.flippers.f_test_hold_eos.sw_release()
         self.machine.coils.c_flipper_main.disable.assert_called_once_with()
+        self.machine.coils.c_flipper_hold.disable.assert_called_once_with()