Repository: glotzerlab/signac
Pull request #200 (2019-06-04 18:56:06)

Feature/issue 104 implement json
## Description
This change introduces the read_json and to_json methods to the Collection class for easy conversion between Collections and JSON strings

## Motivation and Context
This change addresses the issue raised in: https://github.com/glotzerlab/signac/issues/104#issuecomment-472898475

## Types of Changes
- [ ] Documentation update
- [ ] Bug fix
- [x] New feature

## Checklist:
- [x] I am familiar with the [**Contributing Guidelines**](https://github.com/glotzerlab/signac/blob/master/CONTRIBUTING.md).
- [x] I agree with the terms of the [**Contributor Agreement**](https://github.com/glotzerlab/signac/blob/master/ContributorAgreement.md).
- [ ] My name is on the [list of contributors](https://github.com/glotzerlab/signac/blob/master/contributors.txt).
- [x] My code follows the [code style guideline](https://github.com/glotzerlab/signac/blob/master/CONTRIBUTING.md#code-style) of this project.
- [x] The changes introduced by this pull request are covered by existing or newly introduced tests.

If necessary:
- [ ] I have updated the API documentation as part of the package doc-strings.
- [ ] I have created a separate pull request to update the [framework documentation](https://docs.signac.io/) on [signac-docs](https://github.com/glotzerlab/signac-docs) and linked it here.
- [x] I have updated the [changelog](https://github.com/glotzerlab/signac/blob/master/changelog.txt).

I was unsure if and how to update the API documentation, but believe I have covered the remaining points from the Contributing Guidelines. Thanks in advance for reviewing this!

codecov[bot]: # [Codecov](https://codecov.io/gh/glotzerlab/signac/pull/200?src=pr&el=h1) Report
> Merging [#200](https://codecov.io/gh/glotzerlab/signac/pull/200?src=pr&el=desc) into [master](https://codecov.io/gh/glotzerlab/signac/commit/bbf2b74912118264013294ab42799ed605b24265?src=pr&el=desc) will **increase** coverage by `0.05%`.
> The diff coverage is `85.71%`.

[![Impacted file tree graph](https://codecov.io/gh/glotzerlab/signac/pull/200/graphs/tree.svg?width=650&token=FghbjTyzgM&height=150&src=pr)](https://codecov.io/gh/glotzerlab/signac/pull/200?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master     #200      +/-   ##
==========================================
+ Coverage    64.9%   64.96%   +0.05%     
==========================================
  Files          37       37              
  Lines        5557     5575      +18     
==========================================
+ Hits         3607     3622      +15     
- Misses       1950     1953       +3
```


| [Impacted Files](https://codecov.io/gh/glotzerlab/signac/pull/200?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [signac/core/json.py](https://codecov.io/gh/glotzerlab/signac/pull/200/diff?src=pr&el=tree#diff-c2lnbmFjL2NvcmUvanNvbi5weQ==) | `95% <100%> (+0.26%)` | :arrow_up: |
| [signac/contrib/collection.py](https://codecov.io/gh/glotzerlab/signac/pull/200/diff?src=pr&el=tree#diff-c2lnbmFjL2NvbnRyaWIvY29sbGVjdGlvbi5weQ==) | `88.68% <84.61%> (-0.23%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/glotzerlab/signac/pull/200?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/glotzerlab/signac/pull/200?src=pr&el=footer). Last update [bbf2b74...e4f63b2](https://codecov.io/gh/glotzerlab/signac/pull/200?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

codecov[bot]: # [Codecov](https://codecov.io/gh/glotzerlab/signac/pull/200?src=pr&el=h1) Report
> Merging [#200](https://codecov.io/gh/glotzerlab/signac/pull/200?src=pr&el=desc) into [master](https://codecov.io/gh/glotzerlab/signac/commit/bbf2b74912118264013294ab42799ed605b24265?src=pr&el=desc) will **increase** coverage by `0.05%`.
> The diff coverage is `85.71%`.

[![Impacted file tree graph](https://codecov.io/gh/glotzerlab/signac/pull/200/graphs/tree.svg?width=650&token=FghbjTyzgM&height=150&src=pr)](https://codecov.io/gh/glotzerlab/signac/pull/200?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master     #200      +/-   ##
==========================================
+ Coverage    64.9%   64.96%   +0.05%     
==========================================
  Files          37       37              
  Lines        5557     5575      +18     
==========================================
+ Hits         3607     3622      +15     
- Misses       1950     1953       +3
```


| [Impacted Files](https://codecov.io/gh/glotzerlab/signac/pull/200?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [signac/core/json.py](https://codecov.io/gh/glotzerlab/signac/pull/200/diff?src=pr&el=tree#diff-c2lnbmFjL2NvcmUvanNvbi5weQ==) | `95% <100%> (+0.26%)` | :arrow_up: |
| [signac/contrib/collection.py](https://codecov.io/gh/glotzerlab/signac/pull/200/diff?src=pr&el=tree#diff-c2lnbmFjL2NvbnRyaWIvY29sbGVjdGlvbi5weQ==) | `88.68% <84.61%> (-0.23%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/glotzerlab/signac/pull/200?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/glotzerlab/signac/pull/200?src=pr&el=footer). Last update [bbf2b74...e4f63b2](https://codecov.io/gh/glotzerlab/signac/pull/200?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

codecov[bot]: # [Codecov](https://codecov.io/gh/glotzerlab/signac/pull/200?src=pr&el=h1) Report
> Merging [#200](https://codecov.io/gh/glotzerlab/signac/pull/200?src=pr&el=desc) into [master](https://codecov.io/gh/glotzerlab/signac/commit/bbf2b74912118264013294ab42799ed605b24265?src=pr&el=desc) will **increase** coverage by `0.05%`.
> The diff coverage is `85.71%`.

[![Impacted file tree graph](https://codecov.io/gh/glotzerlab/signac/pull/200/graphs/tree.svg?width=650&token=FghbjTyzgM&height=150&src=pr)](https://codecov.io/gh/glotzerlab/signac/pull/200?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master     #200      +/-   ##
==========================================
+ Coverage    64.9%   64.96%   +0.05%     
==========================================
  Files          37       37              
  Lines        5557     5575      +18     
==========================================
+ Hits         3607     3622      +15     
- Misses       1950     1953       +3
```


| [Impacted Files](https://codecov.io/gh/glotzerlab/signac/pull/200?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [signac/core/json.py](https://codecov.io/gh/glotzerlab/signac/pull/200/diff?src=pr&el=tree#diff-c2lnbmFjL2NvcmUvanNvbi5weQ==) | `95% <100%> (+0.26%)` | :arrow_up: |
| [signac/contrib/collection.py](https://codecov.io/gh/glotzerlab/signac/pull/200/diff?src=pr&el=tree#diff-c2lnbmFjL2NvbnRyaWIvY29sbGVjdGlvbi5weQ==) | `88.68% <84.61%> (-0.23%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/glotzerlab/signac/pull/200?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/glotzerlab/signac/pull/200?src=pr&el=footer). Last update [bbf2b74...e4f63b2](https://codecov.io/gh/glotzerlab/signac/pull/200?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

codecov[bot]: # [Codecov](https://codecov.io/gh/glotzerlab/signac/pull/200?src=pr&el=h1) Report
> Merging [#200](https://codecov.io/gh/glotzerlab/signac/pull/200?src=pr&el=desc) into [master](https://codecov.io/gh/glotzerlab/signac/commit/bbf2b74912118264013294ab42799ed605b24265?src=pr&el=desc) will **increase** coverage by `0.05%`.
> The diff coverage is `85.71%`.

[![Impacted file tree graph](https://codecov.io/gh/glotzerlab/signac/pull/200/graphs/tree.svg?width=650&token=FghbjTyzgM&height=150&src=pr)](https://codecov.io/gh/glotzerlab/signac/pull/200?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master     #200      +/-   ##
==========================================
+ Coverage    64.9%   64.96%   +0.05%     
==========================================
  Files          37       37              
  Lines        5557     5575      +18     
==========================================
+ Hits         3607     3622      +15     
- Misses       1950     1953       +3
```


| [Impacted Files](https://codecov.io/gh/glotzerlab/signac/pull/200?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [signac/core/json.py](https://codecov.io/gh/glotzerlab/signac/pull/200/diff?src=pr&el=tree#diff-c2lnbmFjL2NvcmUvanNvbi5weQ==) | `95% <100%> (+0.26%)` | :arrow_up: |
| [signac/contrib/collection.py](https://codecov.io/gh/glotzerlab/signac/pull/200/diff?src=pr&el=tree#diff-c2lnbmFjL2NvbnRyaWIvY29sbGVjdGlvbi5weQ==) | `88.68% <84.61%> (-0.23%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/glotzerlab/signac/pull/200?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/glotzerlab/signac/pull/200?src=pr&el=footer). Last update [bbf2b74...e4f63b2](https://codecov.io/gh/glotzerlab/signac/pull/200?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

bdice: @xadams Overall this looks good! I see there are a lot of changes to whitespace - did you format the code somehow? It looks like the changes are flake8-compliant but it made it a little confusing to read the diff.
xadams: Thank you both for the comments! The whitespace was done automatically by my IDE and should now be removed. I wouldn't necessarily trust IntelliJ's built in style guide as its imposed line length at least is outdated according to several forums.
bdice: Forgot to mention this earlier (wrote a comment and didn't submit it) - the `to_json` and `from_json` methods are missing docstrings. Those would be good to add.
csadorf: @xadams Thank you very much for this contribution? Do you have an ORCID that I could add to the `contributors.yaml` file?

Patch:
diff --git a/changelog.txt b/changelog.txt
index 7d364fb2..8efc41a1 100644
--- a/changelog.txt
+++ b/changelog.txt
@@ -21,6 +21,7 @@ next
 ----
 
  - Keep signac shell command history on a per-project basis.
+ - Add ``read_json()`` and ``to_json()`` methods to Collection class (#104).
  - Fix issue with incorrect detection of dict-like files managed with the ``DictManager`` class (e.g. ``job.stores``) (#203).
 
 [1.1.0] -- 2019-05-19
diff --git a/contributors.yaml b/contributors.yaml
index 985e092f..64820b5c 100644
--- a/contributors.yaml
+++ b/contributors.yaml
@@ -48,4 +48,8 @@ contributors:
     family-names: Wang
     given-names: Kelly
     affiliation: "University of Michigan"
+  -
+    family-names: Adams
+    given-names: Alexander
+    affiliation: "University of Michigan"
 ...
diff --git a/signac/contrib/collection.py b/signac/contrib/collection.py
index 20af2ae8..5b970dc9 100644
--- a/signac/contrib/collection.py
+++ b/signac/contrib/collection.py
@@ -938,6 +938,41 @@ class Collection(object):
         else:
             self._dump(file)
 
+    def to_json(self, file=None):
+        """Dump the collection as a JSON file.
+
+        This function returns the JSON-string directly if the
+        file argument is None.
+
+        :param file:
+            The file to write the JSON string to.
+        """
+        json_string = json.dumps(list(self.find()))
+        if file is None:
+            return json_string
+        elif isinstance(file, six.string_types):
+            with open(file, 'w') as json_file:
+                json_file.write(json_string)
+        else:
+            file.write(json_string)
+
+    @classmethod
+    def read_json(cls, file=None):
+        """Construct an instance of Collection from a JSON file.
+
+        :param file:
+            The json file to read, provided as either a filename or a
+            file-like object.
+        :return:
+            A Collection containing the JSON file
+        """
+        if isinstance(file, six.string_types):
+            with open(file, 'r') as json_file:
+                json_data = json.load(json_file)
+        else:
+            json_data = json.load(file)
+        return Collection(json_data)
+
     @classmethod
     def _open(cls, file, compresslevel=0):
         try:
diff --git a/signac/core/json.py b/signac/core/json.py
index 3de9943d..3d1545b6 100644
--- a/signac/core/json.py
+++ b/signac/core/json.py
@@ -70,8 +70,12 @@ def loads(s):
     return json.loads(s)
 
 
+def load(s):
+    return json.load(s)
+
+
 def dumps(o, sort_keys=False, indent=None):
     return CustomJSONEncoder(sort_keys=sort_keys, indent=indent).encode(o)
 
 
-__all__ = ['loads', 'dumps', 'JSONDecodeError']
+__all__ = ['loads', 'load', 'dumps', 'JSONDecodeError']


Test patch:
diff --git a/tests/test_collection.py b/tests/test_collection.py
index 8431545c..366739f7 100644
--- a/tests/test_collection.py
+++ b/tests/test_collection.py
@@ -614,6 +614,26 @@ class FileCollectionTestBadJson(unittest.TestCase):
                 pass
 
 
+class CollectionTestToFromJson(unittest.TestCase):
+
+    def setUp(self):
+        self._tmp_dir = TemporaryDirectory(prefix='signac_collection_')
+        self._fn_json = os.path.join(self._tmp_dir.name, 'test.json')
+        self.addCleanup(self._tmp_dir.cleanup)
+        self.c = Collection.open(filename=':memory:')
+        docs = [dict(_id=str(i)) for i in range(10)]
+        self.c.update(docs)
+        self.c.flush()
+
+    def test_write_and_read(self):
+        self.c.to_json(self._fn_json)
+        self.assertGreater(os.path.getsize(self._fn_json), 0)
+        c = Collection.read_json(self._fn_json)
+        self.assertEqual(len(list(c)), 10)
+        self.assertEqual(len(c.find()), 10)
+        c.close()
+
+
 class FileCollectionTestReadOnly(unittest.TestCase):
 
     def setUp(self):