Repository: glotzerlab/signac
Pull request #156 (2019-02-28 17:25:44)

Fix/schema subset
The schema subsetting needs to be able to loop over the provided `jobs_or_statepoints` once per key.

csadorf: I'm afraid this will have to go into the next patch release.
codecov-io: # [Codecov](https://codecov.io/gh/glotzerlab/signac/pull/156?src=pr&el=h1) Report
> Merging [#156](https://codecov.io/gh/glotzerlab/signac/pull/156?src=pr&el=desc) into [release/1.0.0](https://codecov.io/gh/glotzerlab/signac/commit/616293cacb5d8aa6d684c6b310a093afd8ec8e4f?src=pr&el=desc) will **increase** coverage by `0.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/glotzerlab/signac/pull/156/graphs/tree.svg?width=650&token=FghbjTyzgM&height=150&src=pr)](https://codecov.io/gh/glotzerlab/signac/pull/156?src=pr&el=tree)

```diff
@@               Coverage Diff                @@
##           release/1.0.0    #156      +/-   ##
================================================
+ Coverage          65.09%   65.1%   +0.01%     
================================================
  Files                 37      37              
  Lines               5503    5505       +2     
================================================
+ Hits                3582    3584       +2     
  Misses              1921    1921
```


| [Impacted Files](https://codecov.io/gh/glotzerlab/signac/pull/156?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [signac/contrib/schema.py](https://codecov.io/gh/glotzerlab/signac/pull/156/diff?src=pr&el=tree#diff-c2lnbmFjL2NvbnRyaWIvc2NoZW1hLnB5) | `89.16% <100%> (+0.18%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/glotzerlab/signac/pull/156?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/glotzerlab/signac/pull/156?src=pr&el=footer). Last update [616293c...5b45548](https://codecov.io/gh/glotzerlab/signac/pull/156?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

csadorf: Additional minimal example for reproducing the issue: https://gist.github.com/berceanu/80f002aad0714a47bbcacf205949ce42

Patch:
diff --git a/changelog.txt b/changelog.txt
index ebb8073e..3737bc57 100644
--- a/changelog.txt
+++ b/changelog.txt
@@ -50,6 +50,7 @@ Fixed
  - Fix ``Collection.update()`` behavior such that existing documents with identical primary key are updated. Previously, a ``KeyError`` would be raised.
  - Fix issue where the ``Job.move()`` would trigger a confusing ``DestinationExists`` exception when trying to move jobs across devices / file systems.
  - Fix issue that caused failures when the ``python-rapidjson`` package is installed. The ``python-rapidjson`` package is used as the primary JSON-backend when installed.
+ - Fix issue where schema with multiple keys would subset incorrectly if the list of jobs or statepoints was provided as an iterator rather than a sequence.
 
 Removed
 +++++++
diff --git a/signac/contrib/schema.py b/signac/contrib/schema.py
index ef1210c5..0d60120a 100644
--- a/signac/contrib/schema.py
+++ b/signac/contrib/schema.py
@@ -4,6 +4,7 @@
 from pprint import pformat
 from collections import defaultdict as ddict
 from numbers import Number
+import itertools
 
 from ..common import six
 if six.PY2:
@@ -179,9 +180,10 @@ class ProjectSchema(object):
     def __call__(self, jobs_or_statepoints):
         "Evaluate the schema for the given state points."
         s = dict()
-        for key in self:
+        iterators = itertools.tee(jobs_or_statepoints, len(self))
+        for key, it in zip(self, iterators):
             values = []
-            for sp in jobs_or_statepoints:
+            for sp in it:
                 if not isinstance(sp, Mapping):
                     sp = sp.statepoint
                 v = sp[key[0]]


Test patch:
diff --git a/tests/test_project.py b/tests/test_project.py
index 40e63d43..5cfeb573 100644
--- a/tests/test_project.py
+++ b/tests/test_project.py
@@ -690,10 +690,13 @@ class ProjectTest(BaseProjectTest):
 
     def test_schema_eval(self):
         for i in range(10):
-            self.project.open_job(dict(a=i)).init()
+            for j in range(10):
+                self.project.open_job(dict(a=i, b=j)).init()
         s = self.project.detect_schema()
         self.assertEqual(s, s(self.project))
         self.assertEqual(s, s([job.sp for job in self.project]))
+        # Check that it works with iterables that can only be consumed once
+        self.assertEqual(s, s((job.sp for job in self.project)))
 
     def test_schema_difference(self):
         def get_sp(i):