Repository: PennyLaneAI/pennylane
Pull request #1355 (2021-05-21 22:25:54)

Tracking context
This PR introduces tracking to devices so that users can keep tracking of device executions, regardless of whether they are buried in gradients or optimizations steps or not.

Users can manually control the `dev.tracker.tracking` attribute to turn on execution printing.

For a demonstration, try:

```python
import pennylane as qml

dev = qml.device('default.qubit', wires=2)
@qml.qnode(dev)
def circ():
    qml.PauliX(1)
    return qml.state()

with qml.Tracker(circ.device) as tracker:
    for _ in range(10):
        circ(shots=10)
```

By default, the tracking context resets the number of executions and total shots.  Users can use `dev.tracking(persistent=True)` to keep current execution numbers.

Build of a tutorial page [here](https://pennylane--1355.org.readthedocs.build/en/1355/code/qml_Tracker.html)

github-actions[bot]: Hello. You may have forgotten to update the changelog!
Please edit [.github/CHANGELOG.md](/PennyLaneAI/pennylane/blob/master/.github/CHANGELOG.md) with:
* A one-to-two sentence description of the change. You may include a small working example for new features.
* A link back to this PR.
* Your name (or GitHub username) in the contributors section.
codecov[bot]: # [Codecov](https://codecov.io/gh/PennyLaneAI/pennylane/pull/1355?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PennyLaneAI) Report
> Merging [#1355](https://codecov.io/gh/PennyLaneAI/pennylane/pull/1355?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PennyLaneAI) (cff99f3) into [master](https://codecov.io/gh/PennyLaneAI/pennylane/commit/0965bb0c97c4de2dc2408751ca3a61dd47074264?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PennyLaneAI) (0965bb0) will **decrease** coverage by `0.14%`.
> The diff coverage is `46.87%`.

[![Impacted file tree graph](https://codecov.io/gh/PennyLaneAI/pennylane/pull/1355/graphs/tree.svg?width=650&height=150&src=pr&token=9S3YzlUgGk&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PennyLaneAI)](https://codecov.io/gh/PennyLaneAI/pennylane/pull/1355?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PennyLaneAI)

```diff
@@            Coverage Diff             @@
##           master    #1355      +/-   ##
==========================================
- Coverage   98.25%   98.11%   -0.15%     
==========================================
  Files         146      146              
  Lines       11146    11178      +32     
==========================================
+ Hits        10952    10967      +15     
- Misses        194      211      +17     
```


| [Impacted Files](https://codecov.io/gh/PennyLaneAI/pennylane/pull/1355?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PennyLaneAI) | Coverage Δ | |
|---|---|---|
| [pennylane/\_device.py](https://codecov.io/gh/PennyLaneAI/pennylane/pull/1355/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PennyLaneAI#diff-cGVubnlsYW5lL19kZXZpY2UucHk=) | `91.17% <42.85%> (-5.55%)` | :arrow_down: |
| [pennylane/\_qubit\_device.py](https://codecov.io/gh/PennyLaneAI/pennylane/pull/1355/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PennyLaneAI#diff-cGVubnlsYW5lL19xdWJpdF9kZXZpY2UucHk=) | `98.16% <75.00%> (-0.35%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/PennyLaneAI/pennylane/pull/1355?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PennyLaneAI).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PennyLaneAI)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/PennyLaneAI/pennylane/pull/1355?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PennyLaneAI). Last update [0965bb0...cff99f3](https://codecov.io/gh/PennyLaneAI/pennylane/pull/1355?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PennyLaneAI). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PennyLaneAI).

albi3ro: 
> I think I only have one main question, and it might be out of scope -- is it possible to do the following somehow?
> 
> ```python
> with qml.Tracker(dev1, dev2) as tracker:
> ```
> 
> that is, share the _same_ tracker among multiple devices?

I think we can think about that for later PR's, but keep this initial PR minimal.

Patch:
diff --git a/.github/CHANGELOG.md b/.github/CHANGELOG.md
index cc8a8f52a..b32923633 100644
--- a/.github/CHANGELOG.md
+++ b/.github/CHANGELOG.md
@@ -2,6 +2,51 @@
 
 <h3>New features since last release</h3>
 
+* The new Device Tracker capabilities allows for flexible and versatile tracking of executions,
+  even inside parameter-shift gradients. This functionality will improve the ease of monitoring
+  large batches and remote jobs.
+  [(#1355)](https://github.com/PennyLaneAI/pennylane/pull/1355)
+
+  ```python
+  dev = qml.device('default.qubit', wires=1, shots=100)
+
+  @qml.qnode(dev, diff_method="parameter-shift")
+  def circuit(x):
+      qml.RX(x, wires=0)
+      return qml.expval(qml.PauliZ(0))
+
+  x = np.array(0.1)
+
+  with qml.Tracker(circuit.device) as tracker:
+      qml.grad(circuit)(x)
+  ```
+
+  ```pycon
+  >>> tracker.totals
+ {'executions': 3, 'shots': 300, 'batches': 1, 'batch_len': 2}
+  >>> tracker.history
+  {'executions': [1, 1, 1],
+   'shots': [100, 100, 100],
+   'batches': [1],
+   'batch_len': [2]}
+  >>> tracker.latest
+  {'batches': 1, 'batch_len': 2}
+  ```
+
+  Users can also provide a custom function to the `callback` keyword that gets called each time
+  the information is updated.  This functionality allows users to monitor remote jobs or large
+  parameter-shift batches.
+
+  ```pycon
+  >>> def shots_info(totals, history, latest):
+  ...     print("Total shots: ", totals['shots'])
+  >>> with qml.Tracker(circuit.device, callback=shots_info) as tracker:
+  ...     qml.grad(circuit)(0.1)
+  Total shots:  100
+  Total shots:  200
+  Total shots:  300
+  ```
+
 * VQE problems can now intuitively been set up by passing the Hamiltonian 
   as an observable. [(#1474)](https://github.com/PennyLaneAI/pennylane/pull/1474)
 
diff --git a/doc/development/plugins.rst b/doc/development/plugins.rst
index 154f3fbcf..6d420a9ff 100644
--- a/doc/development/plugins.rst
+++ b/doc/development/plugins.rst
@@ -118,6 +118,9 @@ as well as potential further capabilities, by providing the following class attr
   *  ``'supports_tensor_observables'`` (*bool*): ``True`` if the device supports observables composed from tensor
      products such as ``PauliZ(wires=0) @ PauliZ(wires=1)``.
 
+  *  ``'supports_tracker'`` (*bool*): ``True`` if it has a device tracker attribute and updates information with
+     it.
+
   Some capabilities are queried by PennyLane core to make decisions on how to best run computations, others are used
   by external apps built on top of the device ecosystem.
 
@@ -139,9 +142,9 @@ following arguments:
 
 * ``wires`` (*int* or *Iterable[Number, str]*): The number of subsystems represented by the device,
   or iterable that contains unique labels for the subsystems as numbers (i.e., ``[-1, 0, 2]``)
-  and/or strings (``['ancilla', 'q1', 'q2']``).
+  and/or strings (``['auxiliary', 'q1', 'q2']``).
 
-* ``shots=1000`` (*None*, *int* or *List[iint]*): number of circuit
+* ``shots=1000`` (*None*, *int* or *List[int]*): number of circuit
   evaluations/random samples used to estimate probabilities, expectation
   values, variances  of observables in non-analytic mode. If ``None``, the device
   calculates probability, expectation values, and variances analytically.  If an
@@ -275,8 +278,6 @@ this may have unintended side-effects and is not recommended.
 
 :html:`</div></div>`
 
-.. _installing_plugin:
-
 
 Wire handling
 -------------
@@ -292,9 +293,9 @@ For example:
 
     from pennylane.wires import Wires
 
-    wires = Wires(['ancilla', 0, 1])
-    print(wires[0]) # <Wires = ['ancilla']>
-    print(wires.labels) # ('ancilla', 0, 1)
+    wires = Wires(['auxiliary', 0, 1])
+    print(wires[0]) # <Wires = ['auxiliary']>
+    print(wires.labels) # ('auxiliary', 0, 1)
 
 As shown in the section on :doc:`/introduction/circuits`, a device can be created with custom wire labels:
 
@@ -358,6 +359,56 @@ object.
 As a convention, devices should do the translation and unpacking as late as possible in the function tree, and
 where possible pass the original :class:`~.wires.Wires` objects around.
 
+Device tracker support
+----------------------
+
+The device tracker stores and records information when tracking mode is turned on. Devices can store data like
+the number of executions, number of shots, number of batches, or remote simulator cost for users to interact with
+in a customizable way.
+
+Three aspects of the :class:`~.Tracker` class are relevant to plugin designers:
+
+* The boolean ``active`` attribute that denotes whether or not to update and record
+* ``update`` method which accepts keyword-value pairs and stores the information
+* ``record`` method which users can customize to log, print, or otherwise do something with the stored information
+
+To gain any of the device tracker functionality, a device should initialize with a placeholder 
+:class:`~.Tracker` instance. Users can overwrite this attribute by initializing a new instance with
+the device as an argument.
+
+We recommend placing the following code near the end of the ``execute`` method:
+
+.. code-block:: python
+
+  if self.tracker.active:
+    self.tracker.update(executions=1, shots=self._shots)
+    self.tracker.record()
+
+And similar code in the ``batch_execute`` method:
+
+.. code-block:: python
+
+  if self.tracker.active:
+    self.tracker.update(batches=1, batch_len=len(circuits))
+    self.tracker.record()
+
+These functions are called in base :class:`~.Device` and :class:`~.QubitDevice` devices. Unless you are
+overriding the ``execute`` and ``batch_execute`` methods or want to customize the stored
+information, you do not need to add any new code.
+
+While this is the recommended usage, the ``update`` and ``record`` methods can be called at any location
+within the device. While the above example tracks executions, shots, and batches, the 
+:meth:`~.Tracker.update` method can accept any combination of
+keyword-value pairs.  For example, a device could also track cost and a job ID via:
+
+.. code-block:: python
+
+  price_for_execution = 0.10
+  job_id = "abcde"
+  self.tracker.update(price=price_for_execution, job_id=job_id)
+
+.. _installing_plugin:
+
 Identifying and installing your device
 --------------------------------------
 
diff --git a/pennylane/__init__.py b/pennylane/__init__.py
index 3da94819a..9164dd479 100644
--- a/pennylane/__init__.py
+++ b/pennylane/__init__.py
@@ -29,7 +29,6 @@ import pennylane.gradients
 import pennylane.kernels
 import pennylane.math
 import pennylane.operation
-import pennylane.qaoa as qaoa
 import pennylane.qnn
 import pennylane.templates
 from pennylane._device import Device, DeviceError
@@ -39,10 +38,12 @@ from pennylane._version import __version__
 from pennylane.about import about
 from pennylane.circuit_graph import CircuitGraph
 from pennylane.configuration import Configuration
+from pennylane.tracker import Tracker
 from pennylane.io import *
 from pennylane.measure import density_matrix, expval, probs, sample, state, var
 from pennylane.ops import *
 from pennylane.optimize import *
+from pennylane import qaoa
 from pennylane.qnode import QNode, qnode
 from pennylane.templates import broadcast, layer, template
 from pennylane.transforms import (
diff --git a/pennylane/_device.py b/pennylane/_device.py
index a9e3d601b..637c79cc6 100644
--- a/pennylane/_device.py
+++ b/pennylane/_device.py
@@ -14,7 +14,7 @@
 """
 This module contains the :class:`Device` abstract base class.
 """
-# pylint: disable=too-many-format-args
+# pylint: disable=too-many-format-args, use-maxsplit-arg
 import abc
 from collections.abc import Iterable, Sequence
 from collections import OrderedDict, namedtuple
@@ -110,9 +110,7 @@ class Device(abc.ABC):
     """
 
     # pylint: disable=too-many-public-methods
-    _capabilities = {
-        "model": None,
-    }
+    _capabilities = {"model": None}
     """The capabilities dictionary stores the properties of a device. Devices can add their
     own custom properties and overwrite existing ones by overriding the ``capabilities()`` method."""
 
@@ -140,6 +138,8 @@ class Device(abc.ABC):
         self._obs_queue = None
         self._parameters = None
 
+        self.tracker = qml.Tracker()
+
     def __repr__(self):
         """String representation."""
         return "<{} device (wires={}, shots={}) at {}>".format(
@@ -375,6 +375,7 @@ class Device(abc.ABC):
         """
         return cls._capabilities
 
+    # pylint: disable=too-many-branches
     def execute(self, queue, observables, parameters={}, **kwargs):
         """Execute a queue of quantum operations on the device and then measure the given observables.
 
@@ -452,6 +453,10 @@ class Device(abc.ABC):
             # increment counter for number of executions of device
             self._num_executions += 1
 
+            if self.tracker.active:
+                self.tracker.update(executions=1, shots=self._shots)
+                self.tracker.record()
+
             # Ensures that a combination with sample does not put
             # expvals and vars in superfluous arrays
             if all(obs.return_type is Sample for obs in observables):
@@ -485,6 +490,10 @@ class Device(abc.ABC):
             res = self.execute(circuit.operations, circuit.observables)
             results.append(res)
 
+        if self.tracker.active:
+            self.tracker.update(batches=1, batch_len=len(circuits))
+            self.tracker.record()
+
         return results
 
     @property
diff --git a/pennylane/_qubit_device.py b/pennylane/_qubit_device.py
index 77143d606..5561d9cca 100644
--- a/pennylane/_qubit_device.py
+++ b/pennylane/_qubit_device.py
@@ -17,7 +17,7 @@ This module contains the :class:`QubitDevice` abstract base class.
 
 # For now, arguments may be different from the signatures provided in Device
 # e.g. instead of expval(self, observable, wires, par) have expval(self, observable)
-# pylint: disable=arguments-differ, abstract-method, no-value-for-parameter,too-many-instance-attributes,too-many-branches
+# pylint: disable=arguments-differ, abstract-method, no-value-for-parameter,too-many-instance-attributes,too-many-branches, arguments-renamed
 import abc
 from collections import OrderedDict
 import itertools
@@ -239,6 +239,10 @@ class QubitDevice(Device):
         # increment counter for number of executions of qubit device
         self._num_executions += 1
 
+        if self.tracker.active:
+            self.tracker.update(executions=1, shots=self._shots)
+            self.tracker.record()
+
         return results
 
     @property
@@ -274,6 +278,10 @@ class QubitDevice(Device):
             res = self.execute(circuit)
             results.append(res)
 
+        if self.tracker.active:
+            self.tracker.update(batches=1, batch_len=len(circuits))
+            self.tracker.record()
+
         return results
 
     @abc.abstractmethod
diff --git a/pennylane/tracker.py b/pennylane/tracker.py
new file mode 100644
index 000000000..27115d4ba
--- /dev/null
+++ b/pennylane/tracker.py
@@ -0,0 +1,195 @@
+# Copyright 2018-2021 Xanadu Quantum Technologies Inc.
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+r"""
+This module contains a class for updating and recording information about device executions.
+"""
+
+# pylint: disable=attribute-defined-outside-init
+
+from numbers import Number
+
+
+class Tracker:
+    """This class stores information about device executions and allows users to interact with that
+    data upon individual executions and batches, even within parameter-shift gradients and
+    optimization steps.
+
+    The information is stored in three class attribute dictionaries: ``totals``, ``history``,
+    and ``latest``:
+
+    * ``latest`` tracks the last set of information passed to the tracker.
+    * ``history`` stores a list of values passed for each keyword.
+    * ``totals`` keeps a running sum per keyword when the values are numeric.
+
+    Standard devices will track the number of executions, number of shots, number of batch
+    executions, and batch execution length, but plugins may store additional information with
+    no changes to this class.
+
+    Information is only stored when the class attribute ``active`` is set to ``True``. This
+    attribute can be toggled via a context manager and Python's ``with`` statement. Upon entering a
+    context, the stored information is reset, unless ``persistent=True``. Tracking mode can also be
+    manually triggered by setting ``tracker.active = True`` without the use of a context manager.
+
+    Args:
+        dev (Device): A PennyLane compatible device
+        callback=None (callable or None): A function of the keywords ``totals``,
+            ``history`` and ``latest``.  Run on each ``record`` call with current values of
+            the corresponding attributes.
+        persistent=False (bool): Whether to reset stored information upon
+            entering a runtime context.
+
+
+    **Example**
+
+    Using a ``with`` statement to toggle the active mode, we can see the number of executions
+    and shots used to calculate a parameter-shift derivative.
+
+    .. code-block:: python
+
+        dev = qml.device('default.qubit', wires=1, shots=100)
+
+        @qml.qnode(dev, diff_method="parameter-shift")
+        def circuit(x):
+            qml.RX(x, wires=0)
+            return qml.expval(qml.PauliZ(0))
+
+        x = np.array(0.1)
+
+        with qml.Tracker(dev) as tracker:
+            qml.grad(circuit)(x)
+
+    You can then access the tabulated information through ``totals``, ``history``, and ``latest``:
+
+    >>> tracker.totals
+    {'executions': 3, 'shots': 300, 'batches': 1, 'batch_len': 2}
+    >>> tracker.history
+    {'executions': [1, 1, 1],
+     'shots': [100, 100, 100],
+     'batches': [1],
+     'batch_len': [2]}
+    >>> tracker.latest
+    {'batches': 1, 'batch_len': 2}
+
+    We can see that calculating the gradient of ``circuit`` takes three total evaluations: one
+    forward pass and one batch of length two for the derivative of ``qml.RX``.
+
+    .. UsageDetails::
+
+        .. note::
+            With backpropagation, this function should take ``qnode.device``
+            instead of the device used to create the QNode.
+
+        Users can pass a custom callback function to the ``callback`` keyword. This
+        function is run each time the ``record()`` method is called, which occurs near
+        the end of a device's ``execute`` and ``batch_execute`` methods. Using ``print``
+        or logging, users can monitor completion during a long set of jobs.
+
+        The function passed must accept ``totals``, ``history``, and ``latest`` as
+        keyword arguments. The dictionary ``latest`` will contain different keywords based on whether
+        whether ``execute`` or ``batch_execute`` last performed an update.
+
+        >>> def shots_info(totals, history, latest):
+        ...     if 'shots' in latest.keys():
+        ...         print("Total shots: ", totals['shots'])
+        >>> with qml.Tracker(circuit.device, callback=shots_info) as tracker:
+        ...     qml.grad(circuit)(0.1)
+        Total shots:  100
+        Total shots:  200
+        Total shots:  300
+
+        By specifying ``persistent=False``, you can reuse the same tracker across
+        multiple contexts.
+
+        >>> with qml.Tracker(circuit.device, persistent=False) as tracker:
+        ...     circuit(0.1)
+        >>> with tracker:
+        ...     circuit(0.2)
+        >>> tracker.totals['executions']
+        2
+
+    """
+
+    def __init__(self, dev=None, callback=None, persistent=False):
+        self.persistent = persistent
+
+        self.callback = callback
+
+        self.reset()
+
+        self.active = False
+
+        if dev is not None:
+            if not hasattr(dev, "tracker"):
+                raise Exception(f"Device '{dev.short_name}' does not support device tracking")
+            dev.tracker = self
+
+    def __enter__(self):
+        if not self.persistent:
+            self.reset()
+
+        self.active = True
+        return self
+
+    def __exit__(self, exc_type, exc_value, exc_traceback):
+        self.active = False
+
+    def update(self, **kwargs):
+        """Store passed keyword-value pairs into ``totals``,``history``, and ``latest`` attributes.
+
+        There is no restriction on the key-value pairs passed, but in the standard devices, the
+        device ``execute`` method will pass ``executions`` and ``shots``, and the ``batch_execute``
+        method will pass ``batches`` and ``batch_len``.
+
+        Only numeric values will be added to ``totals``.
+
+        >>> tracker.update(a=1, b=2, c="c")
+        >>> tracker.latest
+        {"a":1, "b":2, "c":"c"}
+        >>> tracker.history
+        {"a": [1], "b": [2], "c": ["c"]}
+        >>> tracker.totals
+        {"a": 1, "b": 2}
+
+        """
+        self.latest = kwargs
+
+        for key, value in kwargs.items():
+            # update history
+            if key in self.history.keys():
+                self.history[key].append(value)
+            else:
+                self.history[key] = [value]
+
+            # updating totals
+            if value is not None:
+                # Only total numeric values
+                if isinstance(value, Number):
+                    self.totals[key] = value + self.totals.get(key, 0)
+
+    def reset(self):
+        """Resets stored information."""
+        self.totals = dict()
+        self.history = dict()
+        self.latest = dict()
+
+    def record(self):
+        """This method allows users to interact with the stored data.  While it's intended purpose
+        is monitoring large jobs through ``print`` statements or logging, the function is
+        completely flexible and customizable.
+
+        If a user provided a ``callback`` function during initialization, that function is called
+        with the current ``totals``, ``history``, and ``latest`` data variables as keyword arguments.
+        """
+        if self.callback is not None:
+            self.callback(totals=self.totals, history=self.history, latest=self.latest)


Test patch:
diff --git a/pennylane/devices/tests/test_tracker.py b/pennylane/devices/tests/test_tracker.py
new file mode 100644
index 000000000..55704e71e
--- /dev/null
+++ b/pennylane/devices/tests/test_tracker.py
@@ -0,0 +1,81 @@
+# Copyright 2018-2021 Xanadu Quantum Technologies Inc.
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Tests that the device uses a tracker properly if it is supported."""
+# pylint: disable=no-self-use,no-member
+import pytest
+
+import pennylane as qml
+
+
+class TestTracker:
+    """Tests that the device uses a tracker attribute properly"""
+
+    def test_tracker_initialization(self, device):
+        """Tests a tracker instance is assigned at initialization."""
+
+        dev = device(1)
+
+        if not dev.capabilities().get("supports_tracker", False):
+            pytest.skip("Device does not support a tracker")
+
+        assert isinstance(dev.tracker, qml.Tracker)
+
+    def test_tracker_updated_in_execution_mode(self, device, mocker):
+        """Tests that device update and records during tracking mode"""
+
+        dev = device(1)
+
+        if not dev.capabilities().get("supports_tracker", False):
+            pytest.skip("Device does not support a tracker")
+
+        @qml.qnode(dev, diff_method="parameter-shift")
+        def circ():
+            return qml.expval(qml.PauliX(wires=[0]))
+
+        spy_update = mocker.spy(dev.tracker, "update")
+        spy_record = mocker.spy(dev.tracker, "record")
+
+        dev.tracker.active = False
+        circ()
+
+        assert spy_update.call_count == 0
+        assert spy_record.call_count == 0
+
+        dev.tracker.active = True
+        circ()
+
+        assert spy_update.call_count == 1
+        assert spy_record.call_count == 1
+
+    def test_tracker_batch_execute(self, device, mocker):
+        """Asserts tracker updates and records upon both batch execute and standard execute."""
+
+        dev = device(1)
+
+        if not dev.capabilities().get("supports_tracker", False):
+            pytest.skip("Device does not support a tracker")
+
+        @qml.qnode(dev, diff_method="parameter-shift")
+        def circ():
+            return qml.expval(qml.PauliX(wires=[0]))
+
+        spy_update = mocker.spy(dev.tracker, "update")
+        spy_record = mocker.spy(dev.tracker, "record")
+
+        circ()
+        dev.tracker.active = True
+        dev.batch_execute([circ.qtape])
+
+        assert spy_update.call_count == 2
+        assert spy_record.call_count == 2
diff --git a/tests/test_tracker.py b/tests/test_tracker.py
new file mode 100644
index 000000000..d826a932c
--- /dev/null
+++ b/tests/test_tracker.py
@@ -0,0 +1,274 @@
+# Copyright 2018-2021 Xanadu Quantum Technologies Inc.
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""
+Unit tests for the Tracker and constructor
+"""
+import pytest
+
+import pennylane as qml
+from pennylane import Tracker
+
+
+class TestTrackerCoreBehavior:
+    """Unittests for the tracker class"""
+
+    def test_default_initialization(self):
+        """Tests default initializalition"""
+
+        tracker = Tracker()
+
+        assert tracker.persistent == False
+        assert tracker.callback is None
+
+        assert tracker.history == dict()
+        assert tracker.totals == dict()
+        assert tracker.latest == dict()
+
+        assert tracker.active == False
+
+    def test_device_assignment(self):
+        """Assert gets assigned to device"""
+
+        class TempDevice:
+            short_name = "temp"
+
+            def __init__(self):
+                self.tracker = Tracker()
+
+        temp = TempDevice()
+
+        tracker = Tracker(dev=temp)
+
+        assert id(temp.tracker) == id(tracker)
+
+    def test_incompatible_device_assignment_no_tracker(self):
+        """Assert exception raised when `supports_tracker` not True"""
+
+        class TempDevice:
+            short_name = "temp"
+
+            def capabilities(self):
+                return dict()
+
+        temp = TempDevice()
+
+        with pytest.raises(Exception, match=r"Device 'temp' does not support device tracking"):
+            Tracker(dev=temp)
+
+    def test_incompatible_device_assignment_explicit_false(self):
+        """Assert exception raised when `supports_tracker` is False"""
+
+        class TempDevice:
+            short_name = "temp"
+
+        temp = TempDevice()
+
+        with pytest.raises(Exception, match=r"Device 'temp' does not support device tracking"):
+            Tracker(dev=temp)
+
+    def test_reset(self):
+        """Assert reset empties totals, history and latest"""
+
+        tracker = Tracker()
+
+        tracker.totals = {"a": 1}
+        tracker.history = {"a": [1]}
+        tracker.latest = {"a": 1}
+
+        tracker.reset()
+
+        assert tracker.totals == dict()
+        assert tracker.history == dict()
+        assert tracker.latest == dict()
+
+    def test_enter_and_exit(self):
+        """Assert entering and exit work as expected"""
+
+        tracker = Tracker()
+        tracker.totals = {"a": 1}
+        tracker.history = {"a": [1]}
+        tracker.latest = {"a": 1}
+
+        returned = tracker.__enter__()
+
+        assert id(tracker) == id(returned)
+        assert tracker.active == True
+
+        assert tracker.totals == dict()
+        assert tracker.history == dict()
+        assert tracker.latest == dict()
+
+        tracker.__exit__(None, None, None)
+
+        assert tracker.active == False
+
+    def test_context(self):
+        """Assert works with runtime context"""
+
+        with Tracker() as tracker:
+            assert isinstance(tracker, Tracker)
+            assert tracker.active == True
+
+        assert tracker.active == False
+
+    def test_update(self):
+        """Checks update stores to history and totals"""
+
+        tracker = Tracker()
+
+        tracker.update(a=1, b="b", c=None)
+        tracker.update(a=2, b="b2", c=1)
+
+        assert tracker.history == {"a": [1, 2], "b": ["b", "b2"], "c": [None, 1]}
+
+        assert tracker.totals == {"a": 3, "c": 1}
+
+        assert tracker.latest == {"a": 2, "b": "b2", "c": 1}
+
+    def test_record_callback(self, mocker):
+        class callback_wrapper:
+            @staticmethod
+            def callback(totals, history, latest):
+                pass
+
+        wrapper = callback_wrapper()
+        spy = mocker.spy(wrapper, "callback")
+
+        tracker = Tracker(callback=wrapper.callback)
+
+        tracker.totals = {"a": 1, "b": 2}
+        tracker.history = {"a": [1], "b": [1, 1]}
+        tracker.latest = {"a": 1, "b": 1}
+
+        tracker.record()
+
+        _, kwargs_called = spy.call_args_list[-1]
+
+        assert kwargs_called["totals"] == tracker.totals
+        assert kwargs_called["history"] == tracker.history
+        assert kwargs_called["latest"] == tracker.latest
+
+
+# Integration test definitions
+
+dev_qubit = qml.device("default.qubit", wires=1)
+
+
+@qml.qnode(dev_qubit)
+def circuit_qubit():
+    return qml.expval(qml.PauliZ(0))
+
+
+dev_gaussian = qml.device("default.gaussian", wires=1)
+
+
+@qml.qnode(dev_gaussian)
+def circuit_gaussian():
+    return qml.expval(qml.X(0))
+
+
+@pytest.mark.parametrize("circuit", (circuit_qubit, circuit_gaussian))
+class TestDefaultTrackerIntegration:
+    """Tests integration behavior with 'default.gaussian'.
+
+    Integration with several `QubitDevice`-inherited devices are tested in the
+    device suite. Using `default.gaussian`, we test one that inherits from `Device`.
+    """
+
+    def test_single_execution_default(self, circuit, mocker):
+        """Test correct behavior with single circuit execution"""
+
+        class callback_wrapper:
+            @staticmethod
+            def callback(totals=dict(), history=dict(), latest=dict()):
+                pass
+
+        wrapper = callback_wrapper()
+        spy = mocker.spy(wrapper, "callback")
+
+        with Tracker(circuit.device, callback=wrapper.callback) as tracker:
+            circuit()
+            circuit()
+
+        assert tracker.totals == {"executions": 2}
+        assert tracker.history == {"executions": [1, 1], "shots": [None, None]}
+        assert tracker.latest == {"executions": 1, "shots": None}
+
+        _, kwargs_called = spy.call_args_list[-1]
+
+        assert kwargs_called["totals"] == {"executions": 2}
+        assert kwargs_called["history"] == {"executions": [1, 1], "shots": [None, None]}
+        assert kwargs_called["latest"] == {"executions": 1, "shots": None}
+
+    def test_shots_execution_default(self, circuit, mocker):
+        """Test correct tracks shots as well."""
+
+        class callback_wrapper:
+            @staticmethod
+            def callback(totals=dict(), history=dict(), latest=dict()):
+                pass
+
+        wrapper = callback_wrapper()
+        spy = mocker.spy(wrapper, "callback")
+
+        with Tracker(circuit.device, callback=wrapper.callback) as tracker:
+            circuit(shots=10)
+            circuit(shots=20)
+
+        assert tracker.totals == {"executions": 2, "shots": 30}
+        assert tracker.history == {"executions": [1, 1], "shots": [10, 20]}
+        assert tracker.latest == {"executions": 1, "shots": 20}
+
+        assert spy.call_count == 2
+
+        _, kwargs_called = spy.call_args_list[-1]
+        assert kwargs_called["totals"] == {"executions": 2, "shots": 30}
+        assert kwargs_called["history"] == {"executions": [1, 1], "shots": [10, 20]}
+        assert kwargs_called["latest"] == {"executions": 1, "shots": 20}
+
+    def test_batch_execution(self, circuit, mocker):
+        """Tests that batch execute also updates information stored."""
+
+        class callback_wrapper:
+            @staticmethod
+            def callback(totals=dict(), history=dict(), latest=dict()):
+                pass
+
+        wrapper = callback_wrapper()
+        spy = mocker.spy(wrapper, "callback")
+
+        # initial execution to get qtape
+        circuit()
+
+        with Tracker(circuit.device, callback=wrapper.callback) as tracker:
+            circuit.device.batch_execute([circuit.qtape, circuit.qtape])
+
+        assert tracker.totals == {"executions": 2, "batches": 1, "batch_len": 2}
+        assert tracker.history == {
+            "executions": [1, 1],
+            "shots": [None, None],
+            "batches": [1],
+            "batch_len": [2],
+        }
+        assert tracker.latest == {"batches": 1, "batch_len": 2}
+
+        _, kwargs_called = spy.call_args_list[-1]
+        assert kwargs_called["totals"] == {"executions": 2, "batches": 1, "batch_len": 2}
+        assert kwargs_called["history"] == {
+            "executions": [1, 1],
+            "shots": [None, None],
+            "batches": [1],
+            "batch_len": [2],
+        }
+        assert kwargs_called["latest"] == {"batches": 1, "batch_len": 2}