Repository: glotzerlab/signac
Pull request #453 (2020-12-30 01:42:27)

Feature/synced collection/optimization
<!-- Provide a general summary of your changes in the Title above -->

## Description
<!-- Describe your changes in detail -->

This PR optimizes all JSON-based SyncedCollections, including the buffered mode. I benchmarked a number of operations comparing the new JSONDict to the old one on two machines, a 2020 Macbook Pro and our Linux workstations using a slow network file system (as opposed to faster scratch space). I'm happy to share some plots for those interested. Brief summary: depending on the operation being benchmarked, whether buffering is enabled, and the operating system, I see performance improvements in the range of 1.3x-2.7x. In general the biggest performance improvements are observed for operations that change the dict since they touch all of the code paths that I optimized. Some more detailed information:

- Overall I see bigger improvements on the Mac than on the Linux machines. I have not benchmarked the exact bottlenecks on the Linux machines, but I assume that these differences are mostly related to file I/O; the Mac has an SSD, so the I/O costs will be much lower and we can more directly observe the differences in the code complexity.
- Operations that mutate dicts (__setitem__ or update) are 2.2-2.7x faster on the Mac, whereas they are 1.2-2.2x faster on the Linux workstation. 
- Operations that only access data without changing it (__getitem__, keys, values, __call__) are 1.4-2.2x faster on the Mac. These operations gain a little less performance than the mutating operations because they require only a save, not a load call. On the Linux workstations this drops to more like a 1.2-1.8x speed bump.
- On the Mac, buffering only increases performance by about 20% for most operations. Moreover, for some operations it is actually slower. The new JSONDict is also faster than the old JSONDict, even when the new JSONDict does not have buffering activated. Again, this is largely a function of the fast SSD. However, it reveals serious bottlenecks with our buffering strategy that we could exploit; I'll make another issue to discuss that and link it when I do.
- On the Linux workstations, buffering increases performance by up to an order of magnitude, so that's more as expected. The old buffered JSONDict is 4x faster than the new unbuffered JSONDict, but once buffering is enabled the new JSONDict wins by about 2.5x again.

## Motivation and Context
<!-- Why is this change required? What problem does it solve? -->
<!-- If it fixes an open issue, please link to the issue here. -->

## Types of Changes
<!-- Please select all items that apply either now or after creating the pull request: -->
- [ ] Documentation update
- [ ] Bug fix
- [ ] New feature
- [ ] Breaking change<sup>1</sup>

<sup>1</sup>The change breaks (or has the potential to break) existing functionality.

## Checklist:
<!-- Please select all items that apply either now or after creating the pull request. -->
<!-- If you are unsure about any of these items, do not hesitate to ask! -->
- [ ] I am familiar with the [**Contributing Guidelines**](https://github.com/glotzerlab/signac/blob/master/CONTRIBUTING.md).
- [ ] I agree with the terms of the [**Contributor Agreement**](https://github.com/glotzerlab/signac/blob/master/ContributorAgreement.md).
- [ ] My name is on the [list of contributors](https://github.com/glotzerlab/signac/blob/master/contributors.yaml).
- [ ] My code follows the [code style guideline](https://github.com/glotzerlab/signac/blob/master/CONTRIBUTING.md#code-style) of this project.
- [ ] The changes introduced by this pull request are covered by existing or newly introduced tests.

If necessary:
- [ ] I have updated the API documentation as part of the package doc-strings.
- [ ] I have created a separate pull request to update the [framework documentation](https://docs.signac.io/) on [signac-docs](https://github.com/glotzerlab/signac-docs) and linked it here.
- [ ] I have updated the [changelog](https://github.com/glotzerlab/signac/blob/master/changelog.txt) and added all related issue and pull request numbers for future reference (if applicable). See example below.

<!-- Example for a changelog entry: `Fix issue with launching rockets to the moon (#101, #212).` -->


codecov[bot]: # [Codecov](https://codecov.io/gh/glotzerlab/signac/pull/453?src=pr&el=h1) Report
> Merging [#453](https://codecov.io/gh/glotzerlab/signac/pull/453?src=pr&el=desc) (6924adb) into [feature/synced_collections](https://codecov.io/gh/glotzerlab/signac/commit/d11d05deb765223ee95ae3d73ca897d67ac4189f?el=desc) (d11d05d) will **increase** coverage by `0.04%`.
> The diff coverage is `89.85%`.

[![Impacted file tree graph](https://codecov.io/gh/glotzerlab/signac/pull/453/graphs/tree.svg?width=650&height=150&src=pr&token=FghbjTyzgM)](https://codecov.io/gh/glotzerlab/signac/pull/453?src=pr&el=tree)

```diff
@@                      Coverage Diff                       @@
##           feature/synced_collections     #453      +/-   ##
==============================================================
+ Coverage                       77.81%   77.85%   +0.04%     
==============================================================
  Files                              55       56       +1     
  Lines                            6517     6575      +58     
  Branches                         1226     1232       +6     
==============================================================
+ Hits                             5071     5119      +48     
- Misses                           1147     1155       +8     
- Partials                          299      301       +2     
```


| [Impacted Files](https://codecov.io/gh/glotzerlab/signac/pull/453?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [signac/core/synced\_collections/utils.py](https://codecov.io/gh/glotzerlab/signac/pull/453/diff?src=pr&el=tree#diff-c2lnbmFjL2NvcmUvc3luY2VkX2NvbGxlY3Rpb25zL3V0aWxzLnB5) | `72.50% <72.50%> (ø)` | |
| [...nac/core/synced\_collections/buffered\_collection.py](https://codecov.io/gh/glotzerlab/signac/pull/453/diff?src=pr&el=tree#diff-c2lnbmFjL2NvcmUvc3luY2VkX2NvbGxlY3Rpb25zL2J1ZmZlcmVkX2NvbGxlY3Rpb24ucHk=) | `90.90% <89.28%> (+0.36%)` | :arrow_up: |
| [signac/core/synced\_collections/collection\_json.py](https://codecov.io/gh/glotzerlab/signac/pull/453/diff?src=pr&el=tree#diff-c2lnbmFjL2NvcmUvc3luY2VkX2NvbGxlY3Rpb25zL2NvbGxlY3Rpb25fanNvbi5weQ==) | `98.64% <100.00%> (+0.01%)` | :arrow_up: |
| [...ore/synced\_collections/file\_buffered\_collection.py](https://codecov.io/gh/glotzerlab/signac/pull/453/diff?src=pr&el=tree#diff-c2lnbmFjL2NvcmUvc3luY2VkX2NvbGxlY3Rpb25zL2ZpbGVfYnVmZmVyZWRfY29sbGVjdGlvbi5weQ==) | `85.45% <100.00%> (+0.13%)` | :arrow_up: |
| [signac/core/synced\_collections/synced\_attr\_dict.py](https://codecov.io/gh/glotzerlab/signac/pull/453/diff?src=pr&el=tree#diff-c2lnbmFjL2NvcmUvc3luY2VkX2NvbGxlY3Rpb25zL3N5bmNlZF9hdHRyX2RpY3QucHk=) | `100.00% <100.00%> (ø)` | |
| [...ignac/core/synced\_collections/synced\_collection.py](https://codecov.io/gh/glotzerlab/signac/pull/453/diff?src=pr&el=tree#diff-c2lnbmFjL2NvcmUvc3luY2VkX2NvbGxlY3Rpb25zL3N5bmNlZF9jb2xsZWN0aW9uLnB5) | `92.37% <100.00%> (+0.13%)` | :arrow_up: |
| [signac/core/synced\_collections/synced\_list.py](https://codecov.io/gh/glotzerlab/signac/pull/453/diff?src=pr&el=tree#diff-c2lnbmFjL2NvcmUvc3luY2VkX2NvbGxlY3Rpb25zL3N5bmNlZF9saXN0LnB5) | `99.05% <100.00%> (+0.94%)` | :arrow_up: |
| [signac/core/synced\_collections/validators.py](https://codecov.io/gh/glotzerlab/signac/pull/453/diff?src=pr&el=tree#diff-c2lnbmFjL2NvcmUvc3luY2VkX2NvbGxlY3Rpb25zL3ZhbGlkYXRvcnMucHk=) | `95.12% <100.00%> (+0.67%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/glotzerlab/signac/pull/453?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/glotzerlab/signac/pull/453?src=pr&el=footer). Last update [d11d05d...6924adb](https://codecov.io/gh/glotzerlab/signac/pull/453?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

vyasr: @bdice suggested that @klywang @cbkerr might be interested in seeing these results.
cbkerr: Thanks for tagging me @vyasr. I'm trying to understand `SyncedCollection` before proceeding on this review. I tried to clarify some of my confusion [here](https://github.com/glotzerlab/signac/issues/249#issuecomment-753122358).

Patch:
diff --git a/setup.cfg b/setup.cfg
index 0cbe2f99..bb86feb7 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -20,7 +20,7 @@ ignore = E123,E126,E203,E226,E241,E704,W503,W504
 match = ^((?!\.sync-zenodo-metadata|setup|benchmark|mpipool|connection|crypt|host|filesystems|indexing).)*\.py$
 match-dir = ^((?!\.|tests|configobj|db).)*$
 ignore-decorators = "deprecated"
-ignore = D105, D107, D203, D204, D213
+add-ignore = D105, D107, D203, D204, D213
 
 [mypy]
 ignore_missing_imports = True
diff --git a/signac/core/synced_collections/buffered_collection.py b/signac/core/synced_collections/buffered_collection.py
index 94c03801..e5a3aed4 100644
--- a/signac/core/synced_collections/buffered_collection.py
+++ b/signac/core/synced_collections/buffered_collection.py
@@ -16,7 +16,7 @@ heavy I/O. The specific buffering mechanism must be implemented by each backend
 since it depends on the nature of the underlying data format.
 
 All buffered collections expose a local context manager for buffering. In addition,
-this module exposes a global context manager :func:`buffer_reads_writes` that
+this module exposes a global context manager :func:`buffer_all` that
 indicates to all buffered collections irrespective of data type or backend that
 they should enter buffered mode. These context managers may be nested freely, and
 buffer flushes will occur when all such managers have been exited.
@@ -24,7 +24,7 @@ buffer flushes will occur when all such managers have been exited.
 .. code-block::
 
     with collection1.buffered:
-        with buffer_reads_writes:
+        with buffer_all:
             collection2['foo'] = 1
             collection1['bar'] = 1
             # collection2 will flush when this context exits.
@@ -44,56 +44,6 @@ from .synced_collection import SyncedCollection
 
 logger = logging.getLogger(__name__)
 
-_BUFFERED_MODE = 0
-_BUFFERED_BACKENDS: List[Any] = []
-
-
-@contextmanager
-def buffer_reads_writes():
-    """Enter a globally buffer context for all BufferedCollection instances.
-
-    All future operations use the buffer whenever possible. Write operations
-    are deferred until the context is exited, at which point all buffered
-    backends will flush their buffers. Individual backends may flush their
-    buffers within this context if the implementation requires it; this context
-    manager represents a promise to buffer whenever possible, but does not
-    guarantee that no writes will occur under all circumstances.
-    """
-    global _BUFFERED_MODE
-    assert _BUFFERED_MODE >= 0
-
-    _BUFFERED_MODE += 1
-    try:
-        yield
-    finally:
-        _BUFFERED_MODE -= 1
-        if _BUFFERED_MODE == 0:
-            _flush_all_backends()
-
-
-def _flush_all_backends():
-    """Execute all deferred write operations.
-
-    Raises
-    ------
-    BufferedError
-        If there are any issues with flushing any backend.
-
-    """
-    global _BUFFERED_BACKENDS
-    logger.debug("Flushing buffer...")
-    issues = {}
-    for backend in _BUFFERED_BACKENDS:
-        try:
-            # try to sync the data to backend
-            issue = backend._flush_buffer()
-            issues.update(issue)
-        except OSError as error:
-            logger.error(str(error))
-            issues[backend] = error
-    if issues:
-        raise BufferedError(issues)
-
 
 class BufferedCollection(SyncedCollection):
     """A :class:`SyncedCollection` defining an interface for buffering.
@@ -122,6 +72,9 @@ class BufferedCollection(SyncedCollection):
           is to simply call :meth:`~SyncedCollection._save_to_resource`
     """
 
+    _BUFFERED_MODE = 0
+    _BUFFERED_BACKENDS: List[Any] = []
+
     # TODO: Include statement about thread-safety, and perhaps see if we can
     # improve it. The main possible limitation is that while multithreaded
     # access to different synced collections is safe in non-buffered mode
@@ -147,8 +100,52 @@ class BufferedCollection(SyncedCollection):
         Each subclass has its own means of buffering and must be flushed.
         """
         super().__init_subclass__()
-        global _BUFFERED_BACKENDS
-        _BUFFERED_BACKENDS.append(cls)
+        BufferedCollection._BUFFERED_BACKENDS.append(cls)
+
+    @staticmethod
+    @contextmanager
+    def buffer_all():
+        """Enter a globally buffer context for all BufferedCollection instances.
+
+        All future operations use the buffer whenever possible. Write operations
+        are deferred until the context is exited, at which point all buffered
+        backends will flush their buffers. Individual backends may flush their
+        buffers within this context if the implementation requires it; this context
+        manager represents a promise to buffer whenever possible, but does not
+        guarantee that no writes will occur under all circumstances.
+        """
+        assert BufferedCollection._BUFFERED_MODE >= 0
+
+        BufferedCollection._BUFFERED_MODE += 1
+        try:
+            yield
+        finally:
+            BufferedCollection._BUFFERED_MODE -= 1
+            if BufferedCollection._BUFFERED_MODE == 0:
+                BufferedCollection._flush_all_backends()
+
+    @staticmethod
+    def _flush_all_backends():
+        """Execute all deferred write operations.
+
+        Raises
+        ------
+        BufferedError
+            If there are any issues with flushing any backend.
+
+        """
+        logger.debug("Flushing buffer...")
+        issues = {}
+        for backend in BufferedCollection._BUFFERED_BACKENDS:
+            try:
+                # try to sync the data to backend
+                issue = backend._flush_buffer()
+                issues.update(issue)
+            except OSError as error:
+                logger.error(str(error))
+                issues[backend] = error
+        if issues:
+            raise BufferedError(issues)
 
     def _save(self):
         """Synchronize data with the backend but buffer if needed.
@@ -222,8 +219,7 @@ class BufferedCollection(SyncedCollection):
     @property
     def _is_buffered(self):
         """Check if we should write to the buffer or not."""
-        global _BUFFERED_MODE
-        return self._buffered + _BUFFERED_MODE > 0
+        return self._buffered + BufferedCollection._BUFFERED_MODE > 0
 
     def _flush(self):
         """Flush data associated with this instance from the buffer."""
@@ -233,3 +229,6 @@ class BufferedCollection(SyncedCollection):
     def _flush_buffer(self):
         """Flush all data in this class's buffer."""
         pass
+
+
+buffer_all = BufferedCollection.buffer_all
diff --git a/signac/core/synced_collections/collection_json.py b/signac/core/synced_collections/collection_json.py
index 1ba735dd..0d0a54a6 100644
--- a/signac/core/synced_collections/collection_json.py
+++ b/signac/core/synced_collections/collection_json.py
@@ -13,9 +13,11 @@ from .file_buffered_collection import FileBufferedCollection
 from .synced_attr_dict import SyncedAttrDict
 from .synced_collection import SyncedCollection
 from .synced_list import SyncedList
+from .utils import SCJSONEncoder
 from .validators import json_format_validator
 
 
+# TODO: This method should be removed in signac 2.0.
 def _convert_key_to_str(data):
     """Recursively convert non-string keys to strings in dicts.
 
@@ -27,8 +29,12 @@ def _convert_key_to_str(data):
     for dicts. These inputs were silently converted to string keys and stored
     since JSON does not support integer keys. This behavior is deprecated and
     will become an error in signac 2.0.
+
+    Note for developers: this method is designed for use as a validator in the
+    synced collections framework, but due to the backwards compatibility requirement
+    it violates the general behavior of validators by modifying the data in place.
+    This behavior can be removed in signac 2.0 once non-str keys become an error.
     """
-    # TODO: This method should be removed in signac 2.0.
     if isinstance(data, dict):
 
         def _str_key(key):
@@ -41,12 +47,13 @@ def _convert_key_to_str(data):
                 key = str(key)
             return key
 
-        return {
-            _str_key(key): _convert_key_to_str(value) for key, value in data.items()
-        }
+        # Get a list of keys a priori to support modification in place.
+        for key in list(data):
+            _convert_key_to_str(data[key])
+            data[_str_key(key)] = data.pop(key)
     elif isinstance(data, list):
-        return [_convert_key_to_str(value) for value in data]
-    return data
+        for i, value in enumerate(data):
+            _convert_key_to_str(value)
 
 
 class JSONCollection(SyncedCollection):
@@ -97,11 +104,8 @@ class JSONCollection(SyncedCollection):
 
     def _save_to_resource(self):
         """Write the data to JSON file."""
-        data = self._to_base()
-        # Converting non-string keys to string
-        data = _convert_key_to_str(data)
         # Serialize data
-        blob = json.dumps(data).encode()
+        blob = json.dumps(self, cls=SCJSONEncoder).encode()
         # When write_concern flag is set, we write the data into dummy file and then
         # replace that file with original file.
         if self._write_concern:
@@ -120,7 +124,7 @@ class JSONCollection(SyncedCollection):
         return self._filename
 
 
-JSONCollection.add_validator(json_format_validator)
+JSONCollection.add_validator(json_format_validator, _convert_key_to_str)
 
 
 class BufferedJSONCollection(FileBufferedCollection, JSONCollection):
diff --git a/signac/core/synced_collections/file_buffered_collection.py b/signac/core/synced_collections/file_buffered_collection.py
index 3240c457..f416197a 100644
--- a/signac/core/synced_collections/file_buffered_collection.py
+++ b/signac/core/synced_collections/file_buffered_collection.py
@@ -16,6 +16,7 @@ from typing import Dict, Tuple, Union
 
 from .buffered_collection import BufferedCollection
 from .errors import MetadataError
+from .utils import SCJSONEncoder
 
 
 class FileBufferedCollection(BufferedCollection):
@@ -48,6 +49,12 @@ class FileBufferedCollection(BufferedCollection):
 
     """
 
+    # Note for developers: since all subclasses share a single cache, all
+    # references to cache-related class variables in the code use the class
+    # name explicitly rather than using cls (in classmethods) or self (in
+    # methods). This usage avoids any possibility for confusion regarding
+    # backend-specific caches.
+
     _cache: Dict[str, Dict[str, Union[bytes, str, Tuple[int, float]]]] = {}
     _cached_collections: Dict[int, BufferedCollection] = {}
     _BUFFER_CAPACITY = 32 * 2 ** 20  # 32 MB
@@ -163,14 +170,14 @@ class FileBufferedCollection(BufferedCollection):
         """
         if not self._is_buffered or force:
             try:
-                cached_data = self._cache[self._filename]
+                cached_data = FileBufferedCollection._cache[self._filename]
             except KeyError:
                 # There are valid reasons for nothing to be in the cache (the
                 # object was never actually accessed during global buffering,
                 # multiple collections pointing to the same file, etc).
                 pass
             else:
-                blob = self._encode(self._to_base())
+                blob = self._encode(self._data)
 
                 # If the contents have not been changed since the initial read,
                 # we don't need to rewrite it.
@@ -185,7 +192,7 @@ class FileBufferedCollection(BufferedCollection):
                 finally:
                     # Whether or not an error was raised, the cache must be
                     # cleared to ensure a valid final buffer state.
-                    del self._cache[self._filename]
+                    del FileBufferedCollection._cache[self._filename]
                     data_size = len(cached_data["contents"])
                     FileBufferedCollection._CURRENT_BUFFER_SIZE -= data_size
 
@@ -207,7 +214,7 @@ class FileBufferedCollection(BufferedCollection):
             The underlying encoded data.
 
         """
-        return json.dumps(data).encode()
+        return json.dumps(data, cls=SCJSONEncoder).encode()
 
     @staticmethod
     def _decode(blob):
@@ -235,9 +242,9 @@ class FileBufferedCollection(BufferedCollection):
         See :meth:`~._initialize_data_in_cache` for details on the data stored
         in the buffer and the integrity checks performed.
         """
-        if self._filename in self._cache:
-            blob = self._encode(self._to_base())
-            cached_data = self._cache[self._filename]
+        if self._filename in FileBufferedCollection._cache:
+            blob = self._encode(self._data)
+            cached_data = FileBufferedCollection._cache[self._filename]
             buffer_size_change = len(blob) - len(cached_data["contents"])
             FileBufferedCollection._CURRENT_BUFFER_SIZE += buffer_size_change
             cached_data["contents"] = blob
@@ -253,7 +260,9 @@ class FileBufferedCollection(BufferedCollection):
             # the data to initialize the cache with.
             self._initialize_data_in_cache()
             disk_data = self._load_from_resource()
-            self._cache[self._filename]["hash"] = self._hash(self._encode(disk_data))
+            FileBufferedCollection._cache[self._filename]["hash"] = self._hash(
+                self._encode(disk_data)
+            )
 
         if (
             FileBufferedCollection._CURRENT_BUFFER_SIZE
@@ -280,7 +289,7 @@ class FileBufferedCollection(BufferedCollection):
             underlying file.
 
         """
-        if self._filename in self._cache:
+        if self._filename in FileBufferedCollection._cache:
             # Need to check if we have multiple collections pointing to the
             # same file, and if so, track it.
             if id(self) not in FileBufferedCollection._cached_collections:
@@ -289,7 +298,7 @@ class FileBufferedCollection(BufferedCollection):
             self._initialize_data_in_cache()
 
         # Load from buffer
-        blob = self._cache[self._filename]["contents"]
+        blob = FileBufferedCollection._cache[self._filename]["contents"]
 
         if (
             FileBufferedCollection._CURRENT_BUFFER_SIZE
@@ -316,16 +325,16 @@ class FileBufferedCollection(BufferedCollection):
         additional check helps prevent or transparently error on otherwise
         unsafe access patterns.
         """
-        blob = self._encode(self._to_base())
+        blob = self._encode(self._data)
         metadata = self._get_file_metadata()
 
-        self._cache[self._filename] = {
+        FileBufferedCollection._cache[self._filename] = {
             "contents": blob,
             "hash": self._hash(blob),
             "metadata": metadata,
         }
         FileBufferedCollection._CURRENT_BUFFER_SIZE += len(
-            self._cache[self._filename]["contents"]
+            FileBufferedCollection._cache[self._filename]["contents"]
         )
         FileBufferedCollection._cached_collections[id(self)] = self
 
diff --git a/signac/core/synced_collections/synced_attr_dict.py b/signac/core/synced_collections/synced_attr_dict.py
index a1d904a1..8a76377e 100644
--- a/signac/core/synced_collections/synced_attr_dict.py
+++ b/signac/core/synced_collections/synced_attr_dict.py
@@ -12,9 +12,17 @@ named by keys, including nested keys.
 from collections.abc import Mapping, MutableMapping
 from typing import Tuple
 
-from .synced_collection import SyncedCollection
+from .synced_collection import SyncedCollection, _sc_resolver
+from .utils import AbstractTypeResolver
 from .validators import no_dot_in_key
 
+# Identifies mappings, which are the base type for this class.
+_mapping_resolver = AbstractTypeResolver(
+    {
+        "MAPPING": lambda obj: isinstance(obj, Mapping),
+    }
+)
+
 
 class SyncedAttrDict(SyncedCollection, MutableMapping):
     """Implement the dict data structure along with values access through attributes named as keys.
@@ -70,7 +78,8 @@ class SyncedAttrDict(SyncedCollection, MutableMapping):
         """
         converted = {}
         for key, value in self._data.items():
-            if isinstance(value, SyncedCollection):
+            switch_type = _sc_resolver.get_type(value)
+            if switch_type == "SYNCEDCOLLECTION":
                 converted[key] = value._to_base()
             else:
                 converted[key] = value
@@ -90,9 +99,7 @@ class SyncedAttrDict(SyncedCollection, MutableMapping):
         bool
 
         """
-        if isinstance(data, Mapping):
-            return True
-        return False
+        return _mapping_resolver.get_type(data) == "MAPPING"
 
     def _update(self, data=None):
         """Update the in-memory representation to match the provided data.
@@ -114,27 +121,43 @@ class SyncedAttrDict(SyncedCollection, MutableMapping):
 
         """
         if data is None:
-            data = {}
-        if isinstance(data, Mapping):
-            self._validate(data)
+            self._data.clear()
+        elif _mapping_resolver.get_type(data) == "MAPPING":
             with self._suspend_sync():
-                # This loop avoids rebuilding existing synced collections for performance.
-                for key in data:
-                    if key in self._data:
-                        if data[key] == self._data[key]:
+                for key, new_value in data.items():
+                    try:
+                        # The most common usage of SyncedCollections is with a
+                        # single object referencing an underlying resource at a
+                        # time, so we should almost always find that elements
+                        # of data are already contained in self._data, so EAFP
+                        # is the best choice for performance.
+                        existing = self._data[key]
+                    except KeyError:
+                        # If the item wasn't present at all, we can simply
+                        # assign it.
+                        self._validate({key: new_value})
+                        self._data[key] = self._from_base(new_value, parent=self)
+                    else:
+                        if new_value == existing:
                             continue
-                        if isinstance(self._data[key], SyncedCollection):
+                        if _sc_resolver.get_type(existing) == "SYNCEDCOLLECTION":
                             try:
-                                self._data[key]._update(data[key])
+                                existing._update(new_value)
                                 continue
                             except ValueError:
                                 pass
-                    self._data[key] = self._from_base(data[key], parent=self)
-                remove = set()
-                for key in self._data:
-                    if key not in data:
-                        remove.add(key)
-                for key in remove:
+
+                        # Fall through if the new value is not identical to the
+                        # existing value and
+                        #     1) The existing value is not a SyncedCollection
+                        #        (in which case we would have tried to update it), OR
+                        #     2) The existing value is a SyncedCollection, but
+                        #       the new value is not a compatible type for _update.
+                        self._validate({key: new_value})
+                        self._data[key] = self._from_base(new_value, parent=self)
+
+                to_remove = [key for key in self._data if key not in data]
+                for key in to_remove:
                     del self._data[key]
         else:
             raise ValueError(
@@ -144,10 +167,17 @@ class SyncedAttrDict(SyncedCollection, MutableMapping):
             )
 
     def __setitem__(self, key, value):
-        self._validate({key: value})
+        # TODO: Remove in signac 2.0, currently we're constructing a dict to
+        # allow in-place modification by _convert_key_to_str, but validators
+        # should not have side effects once that backwards compatibility layer
+        # is removed, so we can validate a temporary dict {key: value} and
+        # directly set using those rather than looping over data.
+        data = {key: value}
+        self._validate(data)
         self._load()
         with self._suspend_sync():
-            self._data[key] = self._from_base(value, parent=self)
+            for key, value in data.items():
+                self._data[key] = self._from_base(value, parent=self)
         self._save()
 
     def reset(self, data=None):
@@ -166,7 +196,7 @@ class SyncedAttrDict(SyncedCollection, MutableMapping):
         """
         if data is None:
             data = {}
-        if isinstance(data, Mapping):
+        if _mapping_resolver.get_type(data) == "MAPPING":
             self._validate(data)
             with self._suspend_sync():
                 self._data = {
@@ -216,7 +246,7 @@ class SyncedAttrDict(SyncedCollection, MutableMapping):
     def update(self, other=None, **kwargs):  # noqa: D102
         if other is not None:
             # Convert sequence of key, value pairs to dict before validation
-            if not isinstance(other, Mapping):
+            if _mapping_resolver.get_type(other) != "MAPPING":
                 other = dict(other)
         else:
             other = {}
@@ -232,10 +262,18 @@ class SyncedAttrDict(SyncedCollection, MutableMapping):
         if key in self._data:
             ret = self._data[key]
         else:
-            self._validate({key: default})
             ret = self._from_base(default, parent=self)
+            # TODO: Remove in signac 2.0, currently we're constructing a dict
+            # to allow in-place modification by _convert_key_to_str, but
+            # validators should not have side effects once that backwards
+            # compatibility layer is removed, so we can validate a temporary
+            # dict {key: value} and directly set using those rather than
+            # looping over data.
+            data = {key: ret}
+            self._validate(data)
             with self._suspend_sync():
-                self._data[key] = ret
+                for key, value in data.items():
+                    self._data[key] = value
             self._save()
         return ret
 
diff --git a/signac/core/synced_collections/synced_collection.py b/signac/core/synced_collections/synced_collection.py
index 881a84f0..7fb47623 100644
--- a/signac/core/synced_collections/synced_collection.py
+++ b/signac/core/synced_collections/synced_collection.py
@@ -9,6 +9,8 @@ from contextlib import contextmanager
 from inspect import isabstract
 from typing import Any, Callable, DefaultDict, List
 
+from .utils import AbstractTypeResolver
+
 try:
     import numpy
 
@@ -16,6 +18,13 @@ try:
 except ImportError:
     NUMPY = False
 
+# Identifies types of SyncedCollection, which are the base type for this class.
+_sc_resolver = AbstractTypeResolver(
+    {
+        "SYNCEDCOLLECTION": lambda obj: isinstance(obj, SyncedCollection),
+    }
+)
+
 
 class SyncedCollection(Collection):
     """An abstract :class:`Collection` type that is synced with a backend.
@@ -66,7 +75,7 @@ class SyncedCollection(Collection):
     callables that accept different data types as input and raise Exceptions if the
     data does not conform to the requirements of a particular backend. For
     example, a JSON validator would raise Exceptions if it detected non-string
-    keys in a dict.
+    keys in a dict. Validators should have no side effects.
 
     Parameters
     ----------
diff --git a/signac/core/synced_collections/synced_list.py b/signac/core/synced_collections/synced_list.py
index 2d7395bc..932347c5 100644
--- a/signac/core/synced_collections/synced_list.py
+++ b/signac/core/synced_collections/synced_list.py
@@ -9,12 +9,25 @@ implementing the convert method `_to_base` for lists.
 
 from collections.abc import MutableSequence, Sequence
 
-from .synced_collection import NUMPY, SyncedCollection
+from .synced_collection import NUMPY, SyncedCollection, _sc_resolver
+from .utils import AbstractTypeResolver
 
 if NUMPY:
     import numpy
 
 
+# Identifies sequences, which are the base type for this class.
+_sequence_resolver = AbstractTypeResolver(
+    {
+        "SEQUENCE": (
+            lambda obj: isinstance(obj, Sequence)
+            and not isinstance(obj, str)
+            or (NUMPY and isinstance(obj, numpy.ndarray))
+        ),
+    }
+)
+
+
 class SyncedList(SyncedCollection, MutableSequence):
     """Implementation of list data structure.
 
@@ -22,15 +35,16 @@ class SyncedList(SyncedCollection, MutableSequence):
     and :class:`~collections.abc.MutableSequence`. Therefore, it behaves similar
     to a :class:`list`.
 
-    .. warning::
+    Warnings
+    --------
+    While the SyncedList object behaves like a :class:`list`, there are
+    important distinctions to remember. In particular, because operations
+    are reflected as changes to an underlying backend, copying (even deep
+    copying) a SyncedList instance may exhibit unexpected behavior. If a
+    true copy is required, you should use the `_to_base()` method to get a
+    :class:`list` representation, and if necessary construct a new
+    SyncedList.
 
-        While the SyncedList object behaves like a :class:`list`, there are
-        important distinctions to remember. In particular, because operations
-        are reflected as changes to an underlying backend, copying (even deep
-        copying) a SyncedList instance may exhibit unexpected behavior. If a
-        true copy is required, you should use the `_to_base()` method to get a
-        :class:`list` representation, and if necessary construct a new
-        SyncedList.
     """
 
     def __init__(self, data=None, *args, **kwargs):
@@ -61,12 +75,7 @@ class SyncedList(SyncedCollection, MutableSequence):
         bool
 
         """
-        if isinstance(data, Sequence) and not isinstance(data, str):
-            return True
-        elif NUMPY:
-            if isinstance(data, numpy.ndarray):
-                return True
-        return False
+        return _sequence_resolver.get_type(data) == "SEQUENCE"
 
     def _to_base(self):
         """Convert the SyncedList object to list.
@@ -79,7 +88,8 @@ class SyncedList(SyncedCollection, MutableSequence):
         """
         converted = []
         for value in self._data:
-            if isinstance(value, SyncedCollection):
+            switch_type = _sc_resolver.get_type(value)
+            if switch_type == "SYNCEDCOLLECTION":
                 converted.append(value._to_base())
             else:
                 converted.append(value)
@@ -105,26 +115,33 @@ class SyncedList(SyncedCollection, MutableSequence):
 
         """
         if data is None:
-            data = []
-        self._validate(data)
-        if isinstance(data, Sequence) and not isinstance(data, str):
+            self._data.clear()
+        elif _sequence_resolver.get_type(data) == "SEQUENCE":
             with self._suspend_sync():
-                # This loop avoids rebuilding existing synced collections for performance.
-                # TODO: Potential improvements to this code: Remove order constraints.
+                # This loop is optimized based on common usage patterns:
+                # insertion and removal at the end of a list. Inserting or
+                # removing in the middle will result in extra conversion
+                # operations for all subsequent items. In the worst case,
+                # inserting at the beginning will require reconverting all
+                # elements of the data.
                 for i in range(min(len(self), len(data))):
                     if data[i] == self._data[i]:
                         continue
-                    if isinstance(self._data[i], SyncedCollection):
+                    if _sc_resolver.get_type(self._data[i]) == "SYNCEDCOLLECTION":
                         try:
                             self._data[i]._update(data[i])
                             continue
                         except ValueError:
                             pass
-                    self._data[i] = self._from_base(data=data[i], parent=self)
+                    self._validate(data[i])
+                    self._data[i] = self._from_base(data[i], parent=self)
+
                 if len(self._data) > len(data):
                     self._data = self._data[: len(data)]
                 else:
-                    self.extend(data[len(self) :])
+                    new_data = data[len(self) :]
+                    self._validate(new_data)
+                    self.extend(new_data)
         else:
             raise ValueError(
                 "Unsupported type: {}. The data must be a non-string sequence or None.".format(
@@ -151,7 +168,7 @@ class SyncedList(SyncedCollection, MutableSequence):
         elif NUMPY and isinstance(data, numpy.ndarray):
             data = data.tolist()
         self._validate(data)
-        if isinstance(data, Sequence) and not isinstance(data, str):
+        if _sequence_resolver.get_type(data) == "SEQUENCE":
             with self._suspend_sync():
                 self._data = [
                     self._from_base(data=value, parent=self) for value in data
diff --git a/signac/core/synced_collections/utils.py b/signac/core/synced_collections/utils.py
new file mode 100644
index 00000000..1c7cecd0
--- /dev/null
+++ b/signac/core/synced_collections/utils.py
@@ -0,0 +1,138 @@
+# Copyright (c) 2020 The Regents of the University of Michigan
+# All rights reserved.
+# This software is licensed under the BSD 3-Clause License.
+"""Define common utilities."""
+
+from json import JSONEncoder
+from typing import Any, Dict
+
+try:
+    import numpy
+
+    NUMPY = True
+except ImportError:
+    NUMPY = False
+
+
+class AbstractTypeResolver:
+    """Mapping between recognized types and their abstract parents.
+
+    Synced collections are heavily reliant on checking the types of objects to
+    determine the appropriate type of behavior in various scenarios. For maximum
+    generality, most of these checks use the ABCs defined in :py:mod:`collections.abc`.
+    The price of this flexibility is that `isinstance` checks with these classes
+    are very slow because the ``__instancecheck__`` hooks are implemented in pure
+    Python and require checking many different cases.
+
+    Rather than attempting to directly optimize this behavior, this class provides
+    a workaround by which we can amortize the cost of type checks. Given a set
+    of types that must be resolved and a way to identify each of these (which
+    may be expensive), it maintains a local cache of all instances of a given
+    type that have previously been observed. This reduces the cost of type checking
+    to a simple dict lookup, except for the first time a new type is observed.
+
+    Parameters
+    ----------
+    abstract_type_identifiers : collections.abc.Mapping
+        A mapping from a string identifier for a group of types (e.g. ``MAPPING``)
+        to a callable that can be used to identify that type. Due to insertion order
+        guarantees of dictionaries in Python>=3.6 (officially 3.7), it is beneficial
+        to order this dictionary with the most frequently occuring types first.
+
+    Attributes
+    ----------
+    abstract_type_identifiers : Dict[str, Callable[Any, bool]]
+        A mapping from string identifiers for an abstract type to callables that
+        accepts an object and returns True if the object is of the key type and
+        False if not.
+    type_map : Dict[Type, str]
+        A mapping from concrete types to the corresponding named abstract type
+        from :attr:`type_enum`.
+
+    """
+
+    def __init__(self, abstract_type_identifiers):
+        self.abstract_type_identifiers = abstract_type_identifiers
+        self.type_map = {}
+
+    def get_type(self, obj):
+        """Get the type string corresponding to this data type.
+
+        Parameters
+        ----------
+        obj : Any
+            Any object whose type to check
+
+        Returns
+        -------
+        str
+            The name of the type, where valid types are the keys of the dict
+            argument to the constructor. If the object's type cannot be identified,
+            will return ``None``.
+
+        """
+        obj_type = type(obj)
+        enum_type = None
+        try:
+            enum_type = self.type_map[obj_type]
+        except KeyError:
+            for data_type, id_func in self.abstract_type_identifiers.items():
+                if id_func(obj):
+                    enum_type = self.type_map[obj_type] = data_type
+                    break
+            self.type_map[obj_type] = enum_type
+
+        return enum_type
+
+
+def default(o: Any) -> Dict[str, Any]:  # noqa: D102
+    """Get a JSON-serializable version of compatible types.
+
+    This function is suitable for use with JSON-serialization tools as a way
+    to serialize :class:`SyncedCollection` objects and NumPy arrays.
+
+    Warnings
+    --------
+    JSON encoding of numpy arrays is not invertible; once encoded, reloading
+    the data will result in converting arrays to lists and numpy numbers into
+    ints or floats.
+
+    """
+    if NUMPY:
+        if isinstance(o, numpy.number):
+            return o.item()
+        elif isinstance(o, numpy.ndarray):
+            return o.tolist()
+    try:
+        return o._data
+    except AttributeError as e:
+        raise TypeError from e
+
+
+class SCJSONEncoder(JSONEncoder):
+    """A JSONEncoder capable of encoding SyncedCollections and other supported types.
+
+    This encoder will attempt to obtain a JSON-serializable representation of
+    an object that is otherwise not serializable by attempting to access its
+    _data attribute. In addition, it supports direct writing of numpy arrays.
+
+    Warnings
+    --------
+    JSON encoding of numpy arrays is not invertible; once encoded, reloading
+    the data will result in converting arrays to lists and numpy numbers into
+    ints or floats.
+
+    """
+
+    # TODO: If a user tries to access this encoder to manually dump and calls a
+    # dump before any operation, the data won't have been initialized. This
+    # isn't in itself important, since we'll make this private, but consider
+    # whether there are any issues with that. I assume not, since we're
+    # considering making these objects lazy altogether.
+    def default(self, o: Any) -> Dict[str, Any]:  # noqa: D102
+        try:
+            return default(o)
+        except TypeError:
+            # Call the super method, which raises a TypeError if it cannot
+            # encode the object.
+            return super().default(o)
diff --git a/signac/core/synced_collections/validators.py b/signac/core/synced_collections/validators.py
index 05be930b..a67f4c69 100644
--- a/signac/core/synced_collections/validators.py
+++ b/signac/core/synced_collections/validators.py
@@ -13,6 +13,7 @@ types as needed.
 from collections.abc import Mapping, Sequence
 
 from ...errors import InvalidKeyError, KeyTypeError
+from .utils import AbstractTypeResolver
 
 try:
     import numpy
@@ -22,6 +23,15 @@ except ImportError:
     NUMPY = False
 
 
+_no_dot_in_key_type_resolver = AbstractTypeResolver(
+    {
+        "MAPPING": lambda obj: isinstance(obj, Mapping),
+        "NON_STR_SEQUENCE": lambda obj: isinstance(obj, Sequence)
+        and not isinstance(obj, str),
+    }
+)
+
+
 def no_dot_in_key(data):
     """Raise an exception if there is a dot (``.``) in a mapping's key.
 
@@ -40,7 +50,9 @@ def no_dot_in_key(data):
     """
     VALID_KEY_TYPES = (str, int, bool, type(None))
 
-    if isinstance(data, Mapping):
+    switch_type = _no_dot_in_key_type_resolver.get_type(data)
+
+    if switch_type == "MAPPING":
         for key, value in data.items():
             if isinstance(key, str):
                 if "." in key:
@@ -52,11 +64,21 @@ def no_dot_in_key(data):
                     f"Mapping keys must be str, int, bool or None, not {type(key).__name__}"
                 )
             no_dot_in_key(value)
-    elif isinstance(data, Sequence) and not isinstance(data, str):
+    elif switch_type == "NON_STR_SEQUENCE":
         for value in data:
             no_dot_in_key(value)
 
 
+_json_format_validator_type_resolver = AbstractTypeResolver(
+    {
+        "BASE": lambda obj: isinstance(obj, (str, int, float, bool, type(None))),
+        "MAPPING": lambda obj: isinstance(obj, Mapping),
+        "SEQUENCE": lambda obj: isinstance(obj, Sequence),
+        "NUMPY": lambda obj: NUMPY and isinstance(obj, (numpy.ndarray, numpy.number)),
+    }
+)
+
+
 def json_format_validator(data):
     """Validate input data can be serialized to JSON.
 
@@ -73,9 +95,11 @@ def json_format_validator(data):
         If the data type of ``data`` is not supported.
 
     """
-    if isinstance(data, (str, int, float, bool, type(None))):
+    switch_type = _json_format_validator_type_resolver.get_type(data)
+
+    if switch_type == "BASE":
         return
-    elif isinstance(data, Mapping):
+    elif switch_type == "MAPPING":
         for key, value in data.items():
             # Support for non-str keys will be removed in version 2.0.
             # See issue: https://github.com/glotzerlab/signac/issues/316.
@@ -84,10 +108,10 @@ def json_format_validator(data):
                     f"Keys must be str, int, bool or None, not {type(key).__name__}"
                 )
             json_format_validator(value)
-    elif isinstance(data, Sequence):
+    elif switch_type == "SEQUENCE":
         for value in data:
             json_format_validator(value)
-    elif NUMPY and isinstance(data, (numpy.ndarray, numpy.number)):
+    elif switch_type == "NUMPY":
         if numpy.iscomplex(data).any():
             raise TypeError(
                 "NumPy object with complex value(s) is not JSON serializable"


Test patch:
diff --git a/tests/test_synced_collections/synced_collection_test.py b/tests/test_synced_collections/synced_collection_test.py
index 1fea5efb..c5553d3d 100644
--- a/tests/test_synced_collections/synced_collection_test.py
+++ b/tests/test_synced_collections/synced_collection_test.py
@@ -6,7 +6,6 @@ from copy import deepcopy
 
 import pytest
 
-from signac.core.synced_collections.collection_json import JSONDict
 from signac.core.synced_collections.synced_collection import SyncedCollection
 from signac.errors import InvalidKeyError, KeyTypeError
 
@@ -388,17 +387,6 @@ class SyncedDictTest(SyncedCollectionTest):
             assert key in synced_collection
             assert synced_collection[key] == testdata
 
-    # The following test tests the support for non-str keys
-    # for JSON backend which will be removed in version 2.0.
-    # See issue: https://github.com/glotzerlab/signac/issues/316.
-    def test_keys_non_str_valid_type(self, synced_collection, testdata):
-        if isinstance(synced_collection, JSONDict):
-            for key in (0, None, True):
-                with pytest.deprecated_call(match="Use of.+as key is deprecated"):
-                    synced_collection[key] = testdata
-                assert str(key) in synced_collection
-                assert synced_collection[str(key)] == testdata
-
     def test_keys_invalid_type(self, synced_collection, testdata):
         class A:
             pass
diff --git a/tests/test_synced_collections/test_json_buffered_collection.py b/tests/test_synced_collections/test_json_buffered_collection.py
index d173e13a..e1ccba5e 100644
--- a/tests/test_synced_collections/test_json_buffered_collection.py
+++ b/tests/test_synced_collections/test_json_buffered_collection.py
@@ -10,7 +10,7 @@ from tempfile import TemporaryDirectory
 import pytest
 from test_json_collection import JSONCollectionTest, TestJSONDict, TestJSONList
 
-from signac.core.synced_collections.buffered_collection import buffer_reads_writes
+from signac.core.synced_collections.buffered_collection import buffer_all
 from signac.core.synced_collections.collection_json import (
     BufferedJSONCollection,
     BufferedJSONDict,
@@ -131,7 +131,7 @@ class TestBufferedJSONDict(BufferedJSONCollectionTest, TestJSONDict):
         synced_collection["buffered"] = testdata
         assert "buffered" in synced_collection
         assert synced_collection["buffered"] == testdata
-        with buffer_reads_writes():
+        with buffer_all():
             assert "buffered" in synced_collection
             assert synced_collection["buffered"] == testdata
             synced_collection["buffered2"] = 1
@@ -140,7 +140,7 @@ class TestBufferedJSONDict(BufferedJSONCollectionTest, TestJSONDict):
         assert len(synced_collection) == 2
         assert "buffered2" in synced_collection
         assert synced_collection["buffered2"] == 1
-        with buffer_reads_writes():
+        with buffer_all():
             del synced_collection["buffered"]
             assert len(synced_collection) == 1
             assert "buffered" not in synced_collection
@@ -150,7 +150,7 @@ class TestBufferedJSONDict(BufferedJSONCollectionTest, TestJSONDict):
         assert synced_collection["buffered2"] == 1
 
         with pytest.raises(BufferedError):
-            with buffer_reads_writes():
+            with buffer_all():
                 synced_collection["buffered2"] = 2
                 self.store({"test": 1})
                 assert synced_collection["buffered2"] == 2
@@ -162,7 +162,7 @@ class TestBufferedJSONDict(BufferedJSONCollectionTest, TestJSONDict):
         assert len(synced_collection) == 0
 
         for outer_buffer, inner_buffer in itertools.product(
-            [synced_collection.buffered, buffer_reads_writes], repeat=2
+            [synced_collection.buffered, buffer_all], repeat=2
         ):
             err_msg = (
                 f"outer_buffer: {outer_buffer.__qualname__}, "
@@ -200,7 +200,7 @@ class TestBufferedJSONDict(BufferedJSONCollectionTest, TestJSONDict):
             assert "inside_first" in synced_collection2
             assert "inside_first" in on_disk_dict2
 
-            with buffer_reads_writes():
+            with buffer_all():
                 synced_collection["inside_second"] = 3
                 synced_collection2["inside_second"] = 3
 
@@ -248,7 +248,7 @@ class TestBufferedJSONDict(BufferedJSONCollectionTest, TestJSONDict):
             assert synced_collection["inside_first"] == 2
             assert "inside_first" not in on_disk_dict
 
-            with buffer_reads_writes():
+            with buffer_all():
                 synced_collection["inside_second"] = 3
                 synced_collection2["inside_second"] = 4
 
@@ -295,7 +295,7 @@ class TestBufferedJSONDict(BufferedJSONCollectionTest, TestJSONDict):
                 assert synced_collection["inside_first"] == 2
                 assert on_disk_dict["inside_first"] == 3
 
-                with buffer_reads_writes():
+                with buffer_all():
                     synced_collection["inside_second"] = 3
                     synced_collection2["inside_second"] = 4
 
@@ -351,12 +351,12 @@ class TestBufferedJSONList(BufferedJSONCollectionTest, TestJSONList):
 
     def test_global_buffered(self, synced_collection):
         assert len(synced_collection) == 0
-        with buffer_reads_writes():
+        with buffer_all():
             synced_collection.reset([1, 2, 3])
             assert len(synced_collection) == 3
         assert len(synced_collection) == 3
         assert synced_collection == [1, 2, 3]
-        with buffer_reads_writes():
+        with buffer_all():
             assert len(synced_collection) == 3
             assert synced_collection == [1, 2, 3]
             synced_collection[0] = 4
@@ -367,7 +367,7 @@ class TestBufferedJSONList(BufferedJSONCollectionTest, TestJSONList):
 
         # metacheck failure
         with pytest.raises(BufferedError):
-            with buffer_reads_writes():
+            with buffer_all():
                 synced_collection.reset([1])
                 assert synced_collection == [1]
                 # Unfortunately the resolution of os.stat is
diff --git a/tests/test_synced_collections/test_json_collection.py b/tests/test_synced_collections/test_json_collection.py
index 7a3c8167..a5037db5 100644
--- a/tests/test_synced_collections/test_json_collection.py
+++ b/tests/test_synced_collections/test_json_collection.py
@@ -53,6 +53,16 @@ class TestJSONDict(JSONCollectionTest, SyncedDictTest):
 
     _collection_type = JSONDict
 
+    # The following test tests the support for non-str keys
+    # for JSON backend which will be removed in version 2.0.
+    # See issue: https://github.com/glotzerlab/signac/issues/316.
+    def test_keys_non_str_valid_type(self, synced_collection, testdata):
+        for key in (0, None, True):
+            with pytest.deprecated_call(match="Use of.+as key is deprecated"):
+                synced_collection[key] = testdata
+            assert str(key) in synced_collection
+            assert synced_collection[str(key)] == testdata
+
 
 class TestJSONList(JSONCollectionTest, SyncedListTest):
 
diff --git a/tests/test_synced_collections/test_utils.py b/tests/test_synced_collections/test_utils.py
new file mode 100644
index 00000000..d0b39857
--- /dev/null
+++ b/tests/test_synced_collections/test_utils.py
@@ -0,0 +1,73 @@
+# Copyright (c) 2020 The Regents of the University of Michigan
+# All rights reserved.
+# This software is licensed under the BSD 3-Clause License.
+import json
+import os
+from collections.abc import Collection, MutableSequence
+from tempfile import TemporaryDirectory
+
+import pytest
+
+from signac.core.synced_collections.collection_json import JSONDict
+from signac.core.synced_collections.synced_list import SyncedList
+from signac.core.synced_collections.utils import AbstractTypeResolver, SCJSONEncoder
+
+try:
+    import numpy
+
+    NUMPY = True
+except ImportError:
+    NUMPY = False
+
+
+def test_type_resolver():
+    resolver = AbstractTypeResolver(
+        {
+            "dict": lambda obj: isinstance(obj, dict),
+            "tuple": lambda obj: isinstance(obj, tuple),
+            "str": lambda obj: isinstance(obj, str),
+            "mutablesequence": lambda obj: isinstance(obj, MutableSequence),
+            "collection": lambda obj: isinstance(obj, Collection),
+            "set": lambda obj: isinstance(obj, set),
+        }
+    )
+
+    assert resolver.get_type({}) == "dict"
+    assert resolver.get_type((0, 1)) == "tuple"
+    assert resolver.get_type("abc") == "str"
+    assert resolver.get_type([]) == "mutablesequence"
+
+    # Make sure that order matters; collection should be found before list.
+    assert resolver.get_type(set()) == "collection"
+
+
+def test_json_encoder():
+    def encode_flat_dict(d):
+        """A limited JSON-encoding method for a flat dict or SyncedDict of ints."""
+        if hasattr(d, "_data"):
+            d = d._data
+
+        return "{" + ", ".join(f'"{k}": {v}' for k, v in d.items()) + "}"
+
+    # Raw dictionaries should be encoded transparently.
+    data = {"foo": 1, "bar": 2, "baz": 3}
+    assert json.dumps(data) == encode_flat_dict(data)
+    assert json.dumps(data, cls=SCJSONEncoder) == json.dumps(data)
+
+    with TemporaryDirectory() as tmp_dir:
+        fn = os.path.join(tmp_dir, "test_json_encoding.json")
+        synced_data = JSONDict(fn)
+        synced_data.update(data)
+        with pytest.raises(TypeError):
+            json.dumps(synced_data)
+        assert json.dumps(synced_data, cls=SCJSONEncoder) == encode_flat_dict(
+            synced_data
+        )
+
+        if NUMPY:
+            array = numpy.random.rand(3)
+            synced_data["foo"] = array
+            assert isinstance(synced_data["foo"], SyncedList)
+            assert (
+                json.loads(json.dumps(synced_data, cls=SCJSONEncoder)) == synced_data()
+            )