Repository: AgnostiqHQ/covalent
Pull request #1576 (2023-04-14 20:45:24)

Update transportable object
More details: https://github.com/AgnostiqHQ/covalent-cloud-server/issues/215

<!--
⚠️ Your pull request title should be short detailed and understandable for all.
⚠️ Increment the version number in the /VERSION file. If this is a bugfix, increment the patch version (the rightmost number), for example 0.18.2 becomes 0.18.3. If it's a new feature, increment the minor version (the middle number), for example 0.18.2 becomes 0.19.0.
⚠️ Add a note to /CHANGELOG.md with your version number and the date, summarizing the changes.
⚠️ If your pull request fixes an open issue, please link to the issue.
⚠️ Rebase the latest changes from the develop branch. Assuming origin points to https://github.com/AgnostiqHQ/covalent, you should run git rebase origin/develop.
-->

- [ ] I have added the tests to cover my changes.
- [ ] I have updated the documentation and CHANGELOG accordingly.
- [ ] I have read the CONTRIBUTING document.


github-actions[bot]: Hello. You may have forgotten to update the changelog!
Please edit [`CHANGELOG.md`](/AgnostiqHQ/covalent/blob/master/CHANGELOG.md) with a one-to-two sentence description of the change in the UNRELEASED section. You may include a small working example for new features.
codecov[bot]: ## [Codecov](https://codecov.io/gh/AgnostiqHQ/covalent/pull/1576?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=AgnostiqHQ) Report
> Merging [#1576](https://codecov.io/gh/AgnostiqHQ/covalent/pull/1576?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=AgnostiqHQ) (8cc98fe) into [develop](https://codecov.io/gh/AgnostiqHQ/covalent/commit/43d69485d71458816d1499cc13c74111c5161de9?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=AgnostiqHQ) (43d6948) will **increase** coverage by `0.95%`.
> The diff coverage is `95.68%`.

<details><summary>Additional details and impacted files</summary>


[![Impacted file tree graph](https://codecov.io/gh/AgnostiqHQ/covalent/pull/1576/graphs/tree.svg?width=650&height=150&src=pr&token=YGHCB3DE4P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=AgnostiqHQ)](https://codecov.io/gh/AgnostiqHQ/covalent/pull/1576?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=AgnostiqHQ)

```diff
@@             Coverage Diff             @@
##           develop    #1576      +/-   ##
===========================================
+ Coverage    85.67%   86.62%   +0.95%     
===========================================
  Files          158       83      -75     
  Lines         6568     4943    -1625     
  Branches       145        0     -145     
===========================================
- Hits          5627     4282    -1345     
+ Misses         839      661     -178     
+ Partials       102        0     -102     
```

| Flag | Coverage Δ | | *Carryforward flag |
|---|---|---|---|
| Dispatcher | `87.28% <ø> (ø)` | | Carriedforward from [43d6948](https://codecov.io/gh/AgnostiqHQ/covalent/commit/43d69485d71458816d1499cc13c74111c5161de9?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=AgnostiqHQ) |
| Functional_Tests | `?` | | |
| SDK | `86.22% <95.68%> (+0.23%)` | :arrow_up: | |
| UI_Frontend | `?` | | |

*This pull request uses carry forward flags. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=AgnostiqHQ) to find out more.

</details>

Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index b2559de7..4bb8dccd 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [UNRELEASED]
 
+### Changed
+
+- Moved TransportableObject from transport.py to a new file transportable_object.py.
+
 ## [0.220.0-rc.0] - 2023-04-14
 
 ### Authors
diff --git a/covalent/_workflow/transport.py b/covalent/_workflow/transport.py
index ecdba864..9b5c722e 100644
--- a/covalent/_workflow/transport.py
+++ b/covalent/_workflow/transport.py
@@ -20,9 +20,7 @@
 
 """Class implementation of the transport graph in the workflow graph."""
 
-import base64
 import json
-import platform
 from copy import deepcopy
 from typing import Any, Callable, Dict
 
@@ -31,207 +29,10 @@ import networkx as nx
 
 from .._shared_files.defaults import parameter_prefix
 from .._shared_files.util_classes import RESULT_STATUS
-
-
-class TransportableObject:
-    """
-    A function is converted to a transportable object by serializing it using cloudpickle
-    and then whenever executing it, the transportable object is deserialized. The object
-    will also contain additional info like the python version used to serialize it.
-
-    Attributes:
-        _object: The serialized object.
-        python_version: The python version used on the client's machine.
-    """
-
-    def __init__(self, obj: Any) -> None:
-        self._object = base64.b64encode(cloudpickle.dumps(obj)).decode("utf-8")
-        self.python_version = platform.python_version()
-
-        self.object_string = str(obj)
-
-        try:
-            self._json = json.dumps(obj)
-
-        except TypeError as ex:
-            self._json = ""
-
-        self.attrs = {"doc": getattr(obj, "__doc__", ""), "name": getattr(obj, "__name__", "")}
-
-    def __eq__(self, obj) -> bool:
-        return self.__dict__ == obj.__dict__ if isinstance(obj, TransportableObject) else False
-
-    def get_deserialized(self) -> Callable:
-        """
-        Get the deserialized transportable object.
-
-        Args:
-            None
-
-        Returns:
-            function: The deserialized object/callable function.
-
-        """
-
-        return cloudpickle.loads(base64.b64decode(self._object.encode("utf-8")))
-
-    @property
-    def json(self):
-        return self._json
-
-    def to_dict(self) -> dict:
-        """Return a JSON-serializable dictionary representation of self"""
-        return {"type": "TransportableObject", "attributes": self.__dict__.copy()}
-
-    @staticmethod
-    def from_dict(object_dict) -> "TransportableObject":
-        """Rehydrate a dictionary representation
-
-        Args:
-            object_dict: a dictionary representation returned by `to_dict`
-
-        Returns:
-            A `TransportableObject` represented by `object_dict`
-        """
-
-        sc = TransportableObject(None)
-        sc.__dict__ = object_dict["attributes"]
-        return sc
-
-    def get_serialized(self) -> str:
-        """
-        Get the serialized transportable object.
-
-        Args:
-            None
-
-        Returns:
-            object: The serialized transportable object.
-        """
-
-        return self._object
-
-    def serialize(self) -> bytes:
-        """
-        Serialize the transportable object.
-
-        Args:
-            None
-
-        Returns:
-            pickled_object: The serialized object alongwith the python version.
-        """
-
-        return cloudpickle.dumps(
-            {
-                "object": self.get_serialized(),
-                "object_string": self.object_string,
-                "json": self._json,
-                "attrs": self.attrs,
-                "py_version": self.python_version,
-            }
-        )
-
-    def serialize_to_json(self) -> str:
-        """
-        Serialize the transportable object to JSON.
-
-        Args:
-            None
-
-        Returns:
-            A JSON string representation of the transportable object
-        """
-
-        return json.dumps(self.to_dict())
-
-    @staticmethod
-    def deserialize_from_json(json_string: str) -> str:
-        """
-        Reconstruct a transportable object from JSON
-
-        Args:
-            json_string: A JSON string representation of a TransportableObject
-
-        Returns:
-            A TransportableObject instance
-        """
-
-        object_dict = json.loads(json_string)
-        return TransportableObject.from_dict(object_dict)
-
-    @staticmethod
-    def make_transportable(obj) -> "TransportableObject":
-        if isinstance(obj, TransportableObject):
-            return obj
-        else:
-            return TransportableObject(obj)
-
-    @staticmethod
-    def deserialize(data: bytes) -> "TransportableObject":
-        """
-        Deserialize the transportable object.
-
-        Args:
-            data: Cloudpickled function.
-
-        Returns:
-            object: The deserialized transportable object.
-        """
-
-        obj = cloudpickle.loads(data)
-        sc = TransportableObject(None)
-        sc._object = obj["object"]
-        sc._json = obj["json"]
-        sc.attrs = obj["attrs"]
-        sc.python_version = obj["py_version"]
-        return sc
-
-    @staticmethod
-    def deserialize_list(collection: list) -> list:
-        """
-        Recursively deserializes a list of TransportableObjects. More
-        precisely, `collection` is a list, each of whose entries is
-        assumed to be either a `TransportableObject`, a list, or dict`
-        """
-
-        new_list = []
-        for item in collection:
-            if isinstance(item, TransportableObject):
-                new_list.append(item.get_deserialized())
-            elif isinstance(item, list):
-                new_list.append(TransportableObject.deserialize_list(item))
-            elif isinstance(item, dict):
-                new_list.append(TransportableObject.deserialize_dict(item))
-            else:
-                raise TypeError("Couldn't deserialize collection")
-        return new_list
-
-    @staticmethod
-    def deserialize_dict(collection: dict) -> dict:
-        """
-        Recursively deserializes a dict of TransportableObjects. More
-        precisely, `collection` is a dict, each of whose entries is
-        assumed to be either a `TransportableObject`, a list, or dict`
-
-        """
-
-        new_dict = {}
-        for k, item in collection.items():
-            if isinstance(item, TransportableObject):
-                new_dict[k] = item.get_deserialized()
-            elif isinstance(item, list):
-                new_dict[k] = TransportableObject.deserialize_list(item)
-            elif isinstance(item, dict):
-                new_dict[k] = TransportableObject.deserialize_dict(item)
-            else:
-                raise TypeError("Couldn't deserialize collection")
-        return new_dict
+from .transportable_object import TransportableObject
 
 
 # Functions for encoding the transport graph
-
-
 def encode_metadata(metadata: dict) -> dict:
     # Idempotent
     # Special handling required for: executor, workflow_executor, deps, call_before/after, triggers
diff --git a/covalent/_workflow/transportable_object.py b/covalent/_workflow/transportable_object.py
new file mode 100644
index 00000000..4b796d57
--- /dev/null
+++ b/covalent/_workflow/transportable_object.py
@@ -0,0 +1,423 @@
+# Copyright 2021 Agnostiq Inc.
+#
+# This file is part of Covalent.
+#
+# Licensed under the GNU Affero General Public License 3.0 (the "License").
+# A copy of the License may be obtained with this software package or at
+#
+#      https://www.gnu.org/licenses/agpl-3.0.en.html
+#
+# Use of this file is prohibited except in compliance with the License. Any
+# modifications or derivative works of this file must retain this copyright
+# notice, and modified files must contain a notice indicating that they have
+# been altered from the originals.
+#
+# Covalent is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+# FITNESS FOR A PARTICULAR PURPOSE. See the License for more details.
+#
+# Relief from the License may be granted by purchasing a commercial license.
+
+"""Transportable object module."""
+
+import base64
+import json
+import platform
+from typing import Any, Callable
+
+import cloudpickle
+
+#  [string offset (8 bytes), big][data offset (8 bytes), big][header][string][data]
+
+STRING_OFFSET_BYTES = 8
+DATA_OFFSET_BYTES = 8
+HEADER_OFFSET = STRING_OFFSET_BYTES + DATA_OFFSET_BYTES
+BYTE_ORDER = "big"
+
+
+class _TOArchive:
+    """Archived transportable object."""
+
+    def __init__(self, header: bytes, object_string: bytes, data: bytes):
+        """Initialize TOArchive.
+
+        Args:
+            header: Archived transportable object header.
+            object_string: Archived transportable object string.
+            data: Archived transportable object data.
+
+        """
+        self.header = header
+        self.object_string = object_string
+        self.data = data
+
+    def cat(self) -> bytes:
+        """Concatenate TOArchive.
+
+        Returns:
+            Concatenated TOArchive.
+
+        """
+        header_size = len(self.header)
+        string_size = len(self.object_string)
+        data_offset = STRING_OFFSET_BYTES + DATA_OFFSET_BYTES + header_size + string_size
+        string_offset = STRING_OFFSET_BYTES + DATA_OFFSET_BYTES + header_size
+
+        data_offset = data_offset.to_bytes(DATA_OFFSET_BYTES, BYTE_ORDER, signed=False)
+        string_offset = string_offset.to_bytes(STRING_OFFSET_BYTES, BYTE_ORDER, signed=False)
+
+        return string_offset + data_offset + self.header + self.object_string + self.data
+
+    def load(self, header_only: bool, string_only: bool) -> "_TOArchive":
+        """Load TOArchive object.
+
+        Args:
+            header_only: Load header only.
+            string_only: Load string only.
+
+        Returns:
+            Archived transportable object.
+
+        """
+        string_offset = _TOArchiveUtils.string_offset(self)
+        header = _TOArchiveUtils.parse_header(self, string_offset)
+        object_string = b""
+        data = b""
+
+        if not header_only:
+            data_offset = _TOArchiveUtils.data_offset(self)
+            object_string = _TOArchiveUtils.parse_string(self, string_offset, data_offset)
+
+            if not string_only:
+                data = _TOArchiveUtils.parse_data(self, data_offset)
+        return _TOArchive(header, object_string, data)
+
+    def _to_transportable_object(self) -> "TransportableObject":
+        """Convert a _TOArchive to a TransportableObject.
+
+        Args:
+            ar: Archived transportable object to be converted.
+
+        Returns:
+            Transportable object.
+
+        """
+        decoded_object_str = self.object_string.decode("utf-8")
+        decoded_data = self.data.decode("utf-8")
+        decoded_header = json.loads(self.header.decode("utf-8"))
+        to = TransportableObject(None)
+        to._header = decoded_header
+        to._object_string = decoded_object_str or ""
+        to._object = decoded_data or ""
+        return to
+
+
+class _TOArchiveUtils:
+    """TOArchive utilities object."""
+
+    @staticmethod
+    def data_offset(serialized: bytes) -> int:
+        """Get data offset.
+
+        Args:
+            serialized: Serialized TOArchive.
+
+        Returns:
+            Data offset.
+
+        """
+        size64 = serialized[STRING_OFFSET_BYTES : STRING_OFFSET_BYTES + DATA_OFFSET_BYTES]
+        return int.from_bytes(size64, BYTE_ORDER, signed=False)
+
+    @staticmethod
+    def string_offset(serialized: bytes) -> int:
+        """String offset.
+
+        Args:
+            serialized: Serialized TOArchive.
+
+        Returns:
+            String offset.
+
+        """
+        size64 = serialized[:STRING_OFFSET_BYTES]
+        return int.from_bytes(size64, BYTE_ORDER, signed=False)
+
+    @staticmethod
+    def parse_header(serialized: bytes, string_offset: int) -> bytes:
+        """Parse TOArchive header.
+
+        Args:
+            serialized: Serialized TOArchive.
+            string_offset: String offset.
+
+        Returns:
+            Serialized TOArchive header.
+
+        """
+        return serialized[HEADER_OFFSET:string_offset]
+
+    @staticmethod
+    def parse_string(serialized: bytes, string_offset: int, data_offset: int) -> bytes:
+        """Parse string.
+
+        Args:
+            serialized: Serialized TOArchive.
+            string_offset: String offset.
+            data_offset: Data offset.
+
+        Returns:
+            Serialized TOArchive object string.
+
+        """
+        return serialized[string_offset:data_offset]
+
+    @staticmethod
+    def parse_data(serialized: bytes, data_offset: int) -> bytes:
+        """Parse data.
+
+        Args:
+            serialized: Serialized TOArchive.
+            data_offset: Data offset.
+
+        Returns:
+            Serialized TOArchive data.
+
+        """
+        return serialized[data_offset:]
+
+
+class TransportableObject:
+    """
+    A function is converted to a transportable object by serializing it using cloudpickle
+    and then whenever executing it, the transportable object is deserialized. The object
+    will also contain additional info like the python version used to serialize it.
+    """
+
+    def __init__(self, obj: Any) -> None:
+        """Initialize TransportableObject.
+
+        Args:
+            obj: Object to be serialized.
+
+        Attributes:
+            _object: The serialized object.
+            _object_string: The string representation of the object.
+            _header: The header of the object with python version (python version used on the client's machine), doc (Object doc string) and name attributes.
+
+        Returns:
+            None
+
+        """
+        b64object = base64.b64encode(cloudpickle.dumps(obj))
+        object_string_u8 = str(obj).encode("utf-8")
+
+        self._object = b64object.decode("utf-8")
+        self._object_string = object_string_u8.decode("utf-8")
+
+        self._header = {
+            "py_version": platform.python_version(),
+            "attrs": {
+                "doc": getattr(obj, "__doc__", ""),
+                "name": getattr(obj, "__name__", ""),
+            },
+        }
+
+    @property
+    def python_version(self):
+        return self._header["py_version"]
+
+    @property
+    def attrs(self):
+        return self._header["attrs"]
+
+    @property
+    def object_string(self):
+        return self._object_string
+
+    def __eq__(self, obj) -> bool:
+        if not isinstance(obj, TransportableObject):
+            return False
+        return self.__dict__ == obj.__dict__
+
+    def get_deserialized(self) -> Callable:
+        """
+        Get the deserialized transportable object.
+
+        Note that this method is different from the `deserialize` method which deserializes from the `archived` transportable object.
+
+        Returns:
+            function: The deserialized object/callable function.
+
+        """
+        return cloudpickle.loads(base64.b64decode(self._object.encode("utf-8")))
+
+    def to_dict(self) -> dict:
+        """Return a JSON-serializable dictionary representation of self.
+
+        Returns:
+            dict: A JSON-serializable dictionary representation of self.
+
+        """
+        return {"type": "TransportableObject", "attributes": self.__dict__.copy()}
+
+    @staticmethod
+    def from_dict(object_dict) -> "TransportableObject":
+        """Rehydrate a dictionary representation.
+
+        Args:
+            object_dict: a dictionary representation returned by `to_dict`.
+
+        Returns:
+            A `TransportableObject` represented by `object_dict`.
+
+        """
+        sc = TransportableObject(None)
+        sc.__dict__ = object_dict["attributes"]
+        return sc
+
+    def get_serialized(self) -> str:
+        """
+        Get the serialized transportable object.
+
+        Note that this is different from the `serialize` method which serializes the `archived` transportable object.
+
+        Returns:
+            object: The serialized transportable object.
+
+        """
+        return self._object
+
+    def serialize(self) -> bytes:
+        """
+        Serialize the transportable object to the archived transportable object.
+
+        Returns:
+            The serialized object along with the python version.
+
+        """
+        return self._to_archive().cat()
+
+    def serialize_to_json(self) -> str:
+        """
+        Serialize the transportable object to JSON.
+
+        Returns:
+            A JSON string representation of the transportable object.
+
+        """
+        return json.dumps(self.to_dict())
+
+    @staticmethod
+    def deserialize_from_json(json_string: str) -> str:
+        """
+        Reconstruct a transportable object from JSON
+
+        Args:
+            json_string: A JSON string representation of a TransportableObject.
+
+        Returns:
+            A TransportableObject instance.
+
+        """
+        object_dict = json.loads(json_string)
+        return TransportableObject.from_dict(object_dict)
+
+    @staticmethod
+    def make_transportable(obj: Any) -> "TransportableObject":
+        """Make an object transportable.
+
+        Args:
+            obj: The object to make transportable.
+
+        Returns:
+            Transportable object.
+
+        """
+        if isinstance(obj, TransportableObject):
+            return obj
+        else:
+            return TransportableObject(obj)
+
+    @staticmethod
+    def deserialize(
+        serialized: bytes, *, header_only: bool = False, string_only: bool = False
+    ) -> "TransportableObject":
+        """Deserialize the transportable object from the archived transportable object.
+
+        Args:
+            data: Serialized transportable object
+
+        Returns:
+            The deserialized transportable object.
+
+        """
+        ar = _TOArchive.load(serialized, header_only, string_only)
+        return ar._to_transportable_object()
+
+    @staticmethod
+    def deserialize_list(collection: list) -> list:
+        """
+        Recursively deserializes a list of TransportableObjects. More
+        precisely, `collection` is a list, each of whose entries is
+        assumed to be either a `TransportableObject`, a list, or dict`
+
+        Args:
+            collection: A list of TransportableObjects.
+
+        Returns:
+            A list of deserialized objects.
+
+        """
+        new_list = []
+        for item in collection:
+            if isinstance(item, TransportableObject):
+                new_list.append(item.get_deserialized())
+            elif isinstance(item, list):
+                new_list.append(TransportableObject.deserialize_list(item))
+            elif isinstance(item, dict):
+                new_list.append(TransportableObject.deserialize_dict(item))
+            else:
+                raise TypeError("Couldn't deserialize collection")
+        return new_list
+
+    @staticmethod
+    def deserialize_dict(collection: dict) -> dict:
+        """
+        Recursively deserializes a dict of TransportableObjects. More
+        precisely, `collection` is a dict, each of whose entries is
+        assumed to be either a `TransportableObject`, a list, or dict`
+
+        Args:
+            collection: A dictionary of TransportableObjects.
+
+        Returns:
+            A dictionary of deserialized objects.
+
+        """
+        new_dict = {}
+        for k, item in collection.items():
+            if isinstance(item, TransportableObject):
+                new_dict[k] = item.get_deserialized()
+            elif isinstance(item, list):
+                new_dict[k] = TransportableObject.deserialize_list(item)
+            elif isinstance(item, dict):
+                new_dict[k] = TransportableObject.deserialize_dict(item)
+            else:
+                raise TypeError("Couldn't deserialize collection")
+        return new_dict
+
+    def _to_archive(self) -> _TOArchive:
+        """Convert a TransportableObject to a _TOArchive.
+
+        Args:
+            to: Transportable object to be converted.
+
+        Returns:
+            Archived transportable object.
+
+        """
+        header = json.dumps(self._header).encode("utf-8")
+        object_string = self._object_string.encode("utf-8")
+        data = self._object.encode("utf-8")
+        return _TOArchive(header=header, object_string=object_string, data=data)
diff --git a/doc/source/how_to/execution/creating_custom_executors.ipynb b/doc/source/how_to/execution/creating_custom_executors.ipynb
index 2d846522..31e335bb 100644
--- a/doc/source/how_to/execution/creating_custom_executors.ipynb
+++ b/doc/source/how_to/execution/creating_custom_executors.ipynb
@@ -149,7 +149,7 @@
     "                f\"Node {task_metadata['node_id']} in dispatch {task_metadata['dispatch_id']} took {time_taken}s of CPU time.\\n\"\n",
     "            )\n",
     "\n",
-    "        return result"
+    "        return result\n"
    ]
   },
   {
@@ -185,6 +185,7 @@
    "source": [
     "import covalent as ct\n",
     "import sys\n",
+    "\n",
     "sys.path.append(str(Path(\"./custom_executors\").resolve()))\n",
     "from custom_executors import TimingExecutor\n",
     "\n",
@@ -277,13 +278,6 @@
     "\n",
     "[SlurmPlugin (GitHub)](https://github.com/AgnostiqHQ/covalent-slurm-plugin)"
    ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": []
   }
  ],
  "metadata": {


Test patch:
diff --git a/tests/covalent_tests/workflow/transport_test.py b/tests/covalent_tests/workflow/transport_test.py
index e8361eeb..833f48a9 100644
--- a/tests/covalent_tests/workflow/transport_test.py
+++ b/tests/covalent_tests/workflow/transport_test.py
@@ -20,6 +20,7 @@
 
 """Unit tests for transport graph."""
 
+import copy
 import platform
 from unittest.mock import call
 
@@ -86,28 +87,15 @@ def test_transportable_object_python_version(transportable_object):
     assert to.python_version == platform.python_version()
 
 
-def test_transportable_object_json_property(transportable_object):
-    """Test that TO stores a json rep of json-serializable objects"""
-
-    import json
-
-    jsonable_obj = {"a": 1, "b": 2}
-    t = TransportableObject(jsonable_obj)
-    assert t.json == json.dumps(jsonable_obj)
-
-    new_t = TransportableObject(t)
-    assert new_t.json == ""
-
-
 def test_transportable_object_eq(transportable_object):
     """Test the __eq__ magic method of TransportableObject"""
 
     to = transportable_object
     to_new = TransportableObject(None)
-    to_new.__dict__ = to.__dict__.copy()
+    to_new.__dict__ = copy.deepcopy(to.__dict__)
     assert to.__eq__(to_new)
 
-    to_new.python_version = "3.5.1"
+    to_new._header["py_version"] = "3.5.1"
     assert not to.__eq__(to_new)
 
     assert not to.__eq__({})
@@ -127,7 +115,41 @@ def test_transportable_object_get_deserialized(transportable_object):
     assert to.get_deserialized()(x=2) == subtask(x=2)
 
 
+def test_transportable_object_serialize_deserialize_string_only():
+    """Test extracting string only from serialized transportable object."""
+
+    x = 123
+    to = TransportableObject(x)
+
+    ser = to.serialize()
+    new_to = TransportableObject.deserialize(ser, string_only=True)
+    assert new_to.object_string == to.object_string
+    assert new_to._object == ""
+    assert new_to._header["py_version"] == platform.python_version()
+    assert new_to._header["attrs"]["name"] == ""
+    assert "doc" in new_to._header["attrs"]
+    assert new_to._header["attrs"] == new_to.attrs
+
+
+def test_transportable_object_serialize_deserialize_header_only():
+    """Test extracting header only from serialized transportable object."""
+
+    x = 123
+    to = TransportableObject(x)
+
+    ser = to.serialize()
+    new_to = TransportableObject.deserialize(ser, header_only=True)
+
+    assert new_to.object_string == ""
+    assert new_to._header
+    assert new_to._header["py_version"] == platform.python_version()
+    assert new_to._header["attrs"]["name"] == ""
+    assert "doc" in new_to._header["attrs"]
+
+
 def test_transportable_object_from_dict(transportable_object):
+    """Test transportable object creation from dictionary."""
+
     to = transportable_object
 
     object_dict = to.to_dict()
@@ -137,15 +159,17 @@ def test_transportable_object_from_dict(transportable_object):
 
 
 def test_transportable_object_to_dict_attributes(transportable_object):
-    """Test attributes from `to_dict` contain correct name and docstrings"""
+    """Test attributes from `to_dict` contain correct name and doc strings."""
 
     tr_dict = transportable_object.to_dict()
 
-    assert tr_dict["attributes"]["attrs"]["doc"] == subtask.__doc__
-    assert tr_dict["attributes"]["attrs"]["name"] == subtask.__name__
+    assert tr_dict["attributes"]["_header"]["attrs"]["doc"] == subtask.__doc__
+    assert tr_dict["attributes"]["_header"]["attrs"]["name"] == subtask.__name__
 
 
 def test_transportable_object_serialize_to_json(transportable_object):
+    """Test the transportable object can be serialized to JSON."""
+
     import json
 
     to = transportable_object
@@ -153,7 +177,7 @@ def test_transportable_object_serialize_to_json(transportable_object):
 
 
 def test_transportable_object_deserialize_from_json(transportable_object):
-    import json
+    """Test the transportable object can be deserialized from JSON."""
 
     to = transportable_object
     json_string = to.serialize_to_json()
@@ -162,7 +186,7 @@ def test_transportable_object_deserialize_from_json(transportable_object):
 
 
 def test_transportable_object_make_transportable_idempotent(transportable_object):
-    """Test that `make_transportable` is idempotent"""
+    """Test that `make_transportable` is idempotent."""
 
     to = transportable_object
     assert TransportableObject.make_transportable(to) == to
@@ -177,10 +201,17 @@ def test_transportable_object_serialize_deserialize(transportable_object):
 
     assert new_to.get_deserialized()(x=3) == subtask(x=3)
     assert new_to.python_version == to.python_version
+    assert new_to._header["py_version"] == platform.python_version()
+    assert new_to._header["attrs"] == {
+        "name": "subtask",
+        "doc": "Workflow subtask.",
+    }
 
 
-def test_transportable_object_deserialize_list(transportable_object):
-    deserialized = [1, 2, {"a": 3, "b": [4, 5]}]
+def test_transportable_object_deserialize_list():
+    """Test deserialization of a list of transportable objects."""
+
+    deserialized = [1, 2, {"a": 3, "b": [4, 5]}, [6, 7]]
     serialized_list = [
         TransportableObject.make_transportable(1),
         TransportableObject.make_transportable(2),
@@ -191,24 +222,48 @@ def test_transportable_object_deserialize_list(transportable_object):
                 TransportableObject.make_transportable(5),
             ],
         },
+        [
+            TransportableObject.make_transportable(6),
+            TransportableObject.make_transportable(7),
+        ],
     ]
 
     assert TransportableObject.deserialize_list(serialized_list) == deserialized
 
 
-def test_transportable_object_deserialize_dict(transportable_object):
-    deserialized = {"a": 1, "b": [2, {"c": 3}]}
+def test_transportable_object_deserialize_list_exception():
+    """Test exception when deserializing of a list of transportable objects."""
+
+    with pytest.raises(TypeError):
+        TransportableObject.deserialize_list([lambda x: x])
+
+
+def test_transportable_object_deserialize_dict():
+    """Test deserialization of a dictionary of transportable objects."""
+
+    deserialized = {"a": 1, "b": [2, {"c": 3}], "c": {"d": 4, "e": 5}}
     serialized_dict = {
         "a": TransportableObject.make_transportable(1),
         "b": [
             TransportableObject.make_transportable(2),
             {"c": TransportableObject.make_transportable(3)},
         ],
+        "c": {
+            "d": TransportableObject.make_transportable(4),
+            "e": TransportableObject.make_transportable(5),
+        },
     }
 
     assert TransportableObject.deserialize_dict(serialized_dict) == deserialized
 
 
+def test_transportable_object_deserialize_dict_exception():
+    """Test exception when deserializing a dictionary of transportable objects."""
+
+    with pytest.raises(TypeError):
+        TransportableObject.deserialize_dict({"a": lambda x: x})
+
+
 def test_transport_graph_initialization():
     """Test the initialization of an empty transport graph."""