Repository: PennyLaneAI/pennylane
Pull request #3401 (2022-11-21 20:37:56)

Refactor process queue; AnnotatedQueue inherits OrderedDict
**Context:**
One day, we will not depend on QuantumTape internally. To get there, we need a standard workflow to create what a tape represents today. This will be to record operations etc. to an AnnotatedQueue, and then to construct a QuantumScript from said queue. The function to help with this is called `_process_queue` today, and it is a method of the QuantumTape class.

**Description of the Change:**
1. The `QuantumTape._process_queue()` method is separated out to the `queuing` module (as a standalone function in the module).
2. `AnnotatedQueue` used to just contain an `OrderedDict` (called `_queue`) and every method of the class just operated on that dict. Now, it inherits `OrderedDict` and operates on itself! ~Consequentially, QuantumTape no longer inherits `AnnotatedQueue`, but now IT has an `AnnotatedQueue` attribute called `_queue` 😄.~ It still inherits `AnnotatedQueue`, but there is no more `tape._queue` attribute.
    - To make up for this, some dunders in `QuantumTape` have been updated to re-route to `AnnotatedQueue` dunders. This is fine because we don't want to support QuantumTape long-term anyway.
    - All instances of `._queue` have been deleted in the codebase. This is an internal-only "breaking change"
4. Added `qml.queuing` to the Internal section of the docs. Check it out [here](https://xanaduai-pennylane--3401.com.readthedocs.build/en/3401/code/qml_queuing.html). Note that `apply` and `QueuingManager` are also on the main `qml` page because they are public-facing.

**Benefits:**
This allows us to move forward with the tape refactor, and will make it easier to remove internal usage of the `QuantumTape` class in subsequent PRs.

**Possible Drawbacks:**
None I can see. Brought up some small things in the inline comments.

github-actions[bot]: Hello. You may have forgotten to update the changelog!
Please edit [`doc/releases/changelog-dev.md`](/PennyLaneAI/pennylane/blob/master/doc/releases/changelog-dev.md) with:
* A one-to-two sentence description of the change. You may include a small working example for new features.
* A link back to this PR.
* Your name (or GitHub username) in the contributors section.
codecov[bot]: # [Codecov](https://codecov.io/gh/PennyLaneAI/pennylane/pull/3401?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PennyLaneAI) Report
> Merging [#3401](https://codecov.io/gh/PennyLaneAI/pennylane/pull/3401?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PennyLaneAI) (76712d7) into [master](https://codecov.io/gh/PennyLaneAI/pennylane/commit/ffa6e46aae3ab1c94c8c9c8d2761a900ca91abd0?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PennyLaneAI) (ffa6e46) will **increase** coverage by `0.00%`.
> The diff coverage is `100.00%`.

```diff
@@           Coverage Diff           @@
##           master    #3401   +/-   ##
=======================================
  Coverage   99.70%   99.70%           
=======================================
  Files         298      298           
  Lines       26188    26191    +3     
=======================================
+ Hits        26110    26113    +3     
  Misses         78       78           
```


| [Impacted Files](https://codecov.io/gh/PennyLaneAI/pennylane/pull/3401?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PennyLaneAI) | Coverage Δ | |
|---|---|---|
| [pennylane/operation.py](https://codecov.io/gh/PennyLaneAI/pennylane/pull/3401/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PennyLaneAI#diff-cGVubnlsYW5lL29wZXJhdGlvbi5weQ==) | `97.03% <100.00%> (ø)` | |
| [pennylane/queuing.py](https://codecov.io/gh/PennyLaneAI/pennylane/pull/3401/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PennyLaneAI#diff-cGVubnlsYW5lL3F1ZXVpbmcucHk=) | `100.00% <100.00%> (ø)` | |
| [pennylane/tape/tape.py](https://codecov.io/gh/PennyLaneAI/pennylane/pull/3401/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PennyLaneAI#diff-cGVubnlsYW5lL3RhcGUvdGFwZS5weQ==) | `97.36% <100.00%> (-0.16%)` | :arrow_down: |
| [pennylane/transforms/qfunc\_transforms.py](https://codecov.io/gh/PennyLaneAI/pennylane/pull/3401/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PennyLaneAI#diff-cGVubnlsYW5lL3RyYW5zZm9ybXMvcWZ1bmNfdHJhbnNmb3Jtcy5weQ==) | `100.00% <100.00%> (ø)` | |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PennyLaneAI). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PennyLaneAI)


Patch:
diff --git a/doc/code/qml_queuing.rst b/doc/code/qml_queuing.rst
new file mode 100644
index 000000000..5d9ffe77d
--- /dev/null
+++ b/doc/code/qml_queuing.rst
@@ -0,0 +1,14 @@
+qml.queuing
+=============
+
+Overview
+--------
+
+.. currentmodule:: pennylane.queuing
+
+.. automodapi:: pennylane.queuing
+    :no-heading:
+    :include-all-objects:
+    :no-inheritance-diagram:
+    :no-inherited-members:
+    :skip: contextmanager, OrderedDict
diff --git a/doc/index.rst b/doc/index.rst
index 094bdf0e8..4f739e4b7 100644
--- a/doc/index.rst
+++ b/doc/index.rst
@@ -200,6 +200,7 @@ PennyLane is **free** and **open source**, released under the Apache License, Ve
    code/qml_devices
    code/qml_interfaces
    code/qml_operation
+   code/qml_queuing
    code/qml_tape
    code/qml_utils
    code/qml_wires
\ No newline at end of file
diff --git a/doc/releases/changelog-dev.md b/doc/releases/changelog-dev.md
index e99dfa660..9ebae6325 100644
--- a/doc/releases/changelog-dev.md
+++ b/doc/releases/changelog-dev.md
@@ -113,6 +113,10 @@
 * Remove private `_wires` setter from the `Controlled.map_wires` method.
   [3405](https://github.com/PennyLaneAI/pennylane/pull/3405)
 
+* `QuantumTape._process_queue` has been moved to `qml.queuing.process_queue` to disentangle
+  its functionality from the `QuantumTape` class.
+  [(#3401)](https://github.com/PennyLaneAI/pennylane/pull/3401)
+
 <h3>Breaking changes</h3>
 
 * The `log_base` attribute has been moved from `MeasurementProcess` to the new `_VnEntropy` and
@@ -122,6 +126,11 @@
 * Python 3.7 support is no longer maintained.
   [(#3276)](https://github.com/PennyLaneAI/pennylane/pull/3276)
 
+* Instead of having an `OrderedDict` attribute called `_queue`, `AnnotatedQueue` now inherits from
+  `OrderedDict` and encapsulates the queue. Consequentially, this also applies to the `QuantumTape`
+  class which inherits from `AnnotatedQueue`.
+  [(#3401)](https://github.com/PennyLaneAI/pennylane/pull/3401)
+
 <h3>Deprecations</h3>
 
 Deprecations cycles are tracked at [doc/developement/deprecations.rst](https://docs.pennylane.ai/en/latest/development/deprecations.html).
diff --git a/pennylane/operation.py b/pennylane/operation.py
index 6c39468d5..d369e7fdd 100644
--- a/pennylane/operation.py
+++ b/pennylane/operation.py
@@ -1860,7 +1860,7 @@ class Tensor(Observable):
         else:
             raise ValueError("Can only perform tensor products between observables.")
 
-        if QueuingManager.recording() and self not in QueuingManager.active_context()._queue:
+        if QueuingManager.recording() and self not in QueuingManager.active_context():
             QueuingManager.append(self)
 
         QueuingManager.update_info(self, owns=tuple(self.obs))
diff --git a/pennylane/queuing.py b/pennylane/queuing.py
index 0a8145e99..ee64dae0d 100644
--- a/pennylane/queuing.py
+++ b/pennylane/queuing.py
@@ -12,7 +12,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 """
-This module contains the :class:`QueuingManager`.
+This module contains :func:`qml.apply`, :class:`QueuingManager`, and internal code to handle queuing.
 """
 
 import copy
@@ -183,13 +183,10 @@ class QueuingManager:
         return cls.active_context().get_info(obj) if cls.recording() else None
 
 
-class AnnotatedQueue:
+class AnnotatedQueue(OrderedDict):
     """Lightweight class that maintains a basic queue of operations, in addition
     to metadata annotations."""
 
-    def __init__(self):
-        self._queue = OrderedDict()
-
     def __enter__(self):
         """Adds this instance to the global list of active contexts.
 
@@ -206,28 +203,28 @@ class AnnotatedQueue:
 
     def append(self, obj, **kwargs):
         """Append ``obj`` into the queue with ``kwargs`` metadata."""
-        self._queue[obj] = kwargs
+        self[obj] = kwargs
 
     def remove(self, obj):
         """Remove ``obj`` from the queue.  Raises ``KeyError`` if ``obj`` is not already in the queue."""
-        del self._queue[obj]
+        del self[obj]
 
     def update_info(self, obj, **kwargs):
         """Update ``obj``'s metadata with ``kwargs`` if it exists in the queue."""
-        if obj in self._queue:
-            self._queue[obj].update(kwargs)
+        if obj in self:
+            self[obj].update(kwargs)
 
     def get_info(self, obj):
         """Retrieve the metadata for ``obj``.  Raises a ``QueuingError`` if obj is not in the queue."""
-        if obj not in self._queue:
+        if obj not in self:
             raise QueuingError(f"Object {obj} not in the queue.")
 
-        return self._queue[obj]
+        return self[obj]
 
     @property
     def queue(self):
         """Returns a list of objects in the annotated queue"""
-        return list(self._queue.keys())
+        return list(self.keys())
 
 
 def apply(op, context=QueuingManager):
@@ -347,3 +344,34 @@ def apply(op, context=QueuingManager):
         context.append(op)
 
     return op
+
+
+# pylint: disable=protected-access
+def process_queue(queue: AnnotatedQueue):
+    """Process the annotated queue, creating a list of quantum
+    operations and measurement processes.
+
+    Args:
+        queue (.AnnotatedQueue): The queue to be processed into individual lists
+
+    Returns:
+        tuple[list(.Operation), list(.MeasurementProcess)], list(.Operation):
+        The list of main tape operations, the list of tape measurements, and the list of preparation operations
+    """
+    lists = {"_prep": [], "_ops": [], "_measurements": []}
+    list_order = {"_prep": 0, "_ops": 1, "_measurements": 2}
+    current_list = "_prep"
+
+    for obj, info in queue.items():
+
+        if "owner" not in info and getattr(obj, "_queue_category", None) is not None:
+            if list_order[obj._queue_category] > list_order[current_list]:
+                current_list = obj._queue_category
+            elif list_order[obj._queue_category] < list_order[current_list]:
+                raise ValueError(
+                    f"{obj._queue_category[1:]} operation {obj} must occur prior "
+                    f"to {current_list[1:]}. Please place earlier in the queue."
+                )
+            lists[obj._queue_category].append(obj)
+
+    return lists["_ops"], lists["_measurements"], lists["_prep"]
diff --git a/pennylane/tape/operation_recorder.py b/pennylane/tape/operation_recorder.py
index 267c2d0fb..16437a4d9 100644
--- a/pennylane/tape/operation_recorder.py
+++ b/pennylane/tape/operation_recorder.py
@@ -55,7 +55,7 @@ class OperationRecorder(QuantumTape):
     def _process_queue(self):
         super()._process_queue()
 
-        for obj, info in self._queue.items():
+        for obj, info in self.items():
             QueuingManager.append(obj, **info)
 
         new_tape = self.expand(depth=5, stop_at=lambda obj: not isinstance(obj, QuantumTape))
diff --git a/pennylane/tape/tape.py b/pennylane/tape/tape.py
index f09e5935a..683465afc 100644
--- a/pennylane/tape/tape.py
+++ b/pennylane/tape/tape.py
@@ -21,7 +21,7 @@ from threading import RLock
 import pennylane as qml
 from pennylane.measurements import AllCounts, Counts, Probability, Sample
 from pennylane.operation import DecompositionUndefinedError, Operator
-from pennylane.queuing import AnnotatedQueue, QueuingManager
+from pennylane.queuing import AnnotatedQueue, QueuingManager, process_queue
 
 from .qscript import QuantumScript
 
@@ -361,7 +361,7 @@ class QuantumTape(QuantumScript, AnnotatedQueue):
         try:
             if self.do_queue:
                 QueuingManager.append(self)
-            return super().__enter__()
+            return AnnotatedQueue.__enter__(self)
         except Exception as _:
             QuantumTape._lock.release()
             raise
@@ -397,22 +397,21 @@ class QuantumTape(QuantumScript, AnnotatedQueue):
 
         Also calls `_update()` which sets many attributes.
         """
-        self._prep = []
-        self._ops = []
-        self._measurements = []
-        list_order = {"_prep": 0, "_ops": 1, "_measurements": 2}
-        current_list = "_prep"
-
-        for obj, info in self._queue.items():
-
-            if "owner" not in info and getattr(obj, "_queue_category", None) is not None:
-                if list_order[obj._queue_category] > list_order[current_list]:
-                    current_list = obj._queue_category
-                elif list_order[obj._queue_category] < list_order[current_list]:
-                    raise ValueError(
-                        f"{obj._queue_category[1:]} operation {obj} must occur prior "
-                        f"to {current_list[1:]}. Please place earlier in the queue."
-                    )
-                getattr(self, obj._queue_category).append(obj)
-
+        self._ops, self._measurements, self._prep = process_queue(self)
         self._update()
+
+    def __getitem__(self, key):
+        """
+        Overrides the default because QuantumTape is both a QuantumScript and an AnnotatedQueue.
+        If key is an int, the caller is likely indexing the backing QuantumScript. Otherwise, the
+        caller is likely indexing the backing AnnotatedQueue.
+        """
+        if isinstance(key, int):
+            return QuantumScript.__getitem__(self, key)
+        return AnnotatedQueue.__getitem__(self, key)
+
+    def __setitem__(self, key, val):
+        AnnotatedQueue.__setitem__(self, key, val)
+
+    def __hash__(self):
+        return QuantumScript.__hash__(self)
diff --git a/pennylane/transforms/qfunc_transforms.py b/pennylane/transforms/qfunc_transforms.py
index 5378f6093..b698f47a4 100644
--- a/pennylane/transforms/qfunc_transforms.py
+++ b/pennylane/transforms/qfunc_transforms.py
@@ -83,7 +83,7 @@ class NonQueuingTape(qml.queuing.AnnotatedQueue):
         # accesses the parent of the target class that NonQueingTape is mixed into
         super()._process_queue()  # pylint:disable=no-member
 
-        for obj, info in self._queue.items():
+        for obj, info in self.items():
             qml.queuing.QueuingManager.append(obj, **info)
 
         qml.queuing.QueuingManager.remove(self)


Test patch:
diff --git a/tests/measurements/test_measurements.py b/tests/measurements/test_measurements.py
index 623061581..3b689774a 100644
--- a/tests/measurements/test_measurements.py
+++ b/tests/measurements/test_measurements.py
@@ -177,7 +177,7 @@ class TestStatisticsQueuing:
             tensor_op = A @ B
             stat_func(tensor_op)
 
-        assert len(q._queue) == 2  # pylint: disable=protected-access
+        assert len(q) == 2
 
         assert q.queue[0] is tensor_op
         meas_proc = q.queue[-1]
diff --git a/tests/ops/functions/test_map_wires.py b/tests/ops/functions/test_map_wires.py
index 1b60e8f07..97365c7cb 100644
--- a/tests/ops/functions/test_map_wires.py
+++ b/tests/ops/functions/test_map_wires.py
@@ -66,7 +66,7 @@ class TestMapWiresOperators:
             _ = qml.map_wires(op, wire_map=wire_map, queue=False)
         assert len(tape.circuit) == 1
         assert tape.circuit[0] is op
-        assert tape._queue[op].get("owner", None) is None
+        assert tape.get_info(op).get("owner", None) is None
 
     def test_map_wires_with_queuing_and_without_replacing(self):
         """Test the map_wires method while queuing with `queue=True` and `replace=False`."""
@@ -77,7 +77,7 @@ class TestMapWiresOperators:
         assert len(tape.circuit) == 2
         assert tape.circuit[0] is op
         assert tape.circuit[1] is m_op
-        assert tape._queue[op].get("owner", None) is None
+        assert tape.get_info(op).get("owner", None) is None
 
     def test_map_wires_with_queuing_and_with_replacing(self):
         """Test the map_wires method while queuing with `queue = True` and `replace=True`."""
@@ -87,7 +87,7 @@ class TestMapWiresOperators:
             m_op = qml.map_wires(op, wire_map=wire_map, queue=True, replace=True)
         assert len(tape.circuit) == 1
         assert tape.circuit[0] is m_op
-        assert tape._queue[op].get("owner", None) is m_op
+        assert tape.get_info(op).get("owner", None) is m_op
 
     def test_map_wires_unsupported_object_raises_error(self):
         """Test that an error is raised when trying to map the wires of an unsupported object."""
diff --git a/tests/ops/functions/test_simplify.py b/tests/ops/functions/test_simplify.py
index fcb946884..dac56e423 100644
--- a/tests/ops/functions/test_simplify.py
+++ b/tests/ops/functions/test_simplify.py
@@ -64,7 +64,7 @@ class TestSimplifyOperators:
             s_op = qml.simplify(op)
         assert len(tape.circuit) == 1
         assert tape.circuit[0] is s_op
-        assert tape._queue[op]["owner"] is s_op
+        assert tape.get_info(op)["owner"] is s_op
 
     def test_simplify_unsupported_object_raises_error(self):
         """Test that an error is raised when trying to simplify an unsupported object."""
diff --git a/tests/ops/op_math/test_adjoint_op.py b/tests/ops/op_math/test_adjoint_op.py
index a49a1be74..7005b9b03 100644
--- a/tests/ops/op_math/test_adjoint_op.py
+++ b/tests/ops/op_math/test_adjoint_op.py
@@ -574,8 +574,8 @@ class TestQueueing:
             base = qml.Rot(1.2345, 2.3456, 3.4567, wires="b")
             op = Adjoint(base)
 
-        assert tape._queue[base]["owner"] is op
-        assert tape._queue[op]["owns"] is base
+        assert tape.get_info(base)["owner"] is op
+        assert tape.get_info(op)["owns"] is base
         assert tape.operations == [op]
 
     def test_queueing_base_defined_outside(self):
@@ -585,8 +585,8 @@ class TestQueueing:
         with qml.tape.QuantumTape() as tape:
             op = Adjoint(base)
 
-        assert len(tape._queue) == 1
-        assert tape._queue[op]["owns"] is base
+        assert len(tape) == 1
+        assert tape.get_info(op)["owns"] is base
         assert tape.operations == [op]
 
     def test_do_queue_False(self):
diff --git a/tests/ops/op_math/test_adjoint_transform.py b/tests/ops/op_math/test_adjoint_transform.py
index 798cc6aad..1e4676029 100644
--- a/tests/ops/op_math/test_adjoint_transform.py
+++ b/tests/ops/op_math/test_adjoint_transform.py
@@ -47,9 +47,9 @@ class TestPreconstructedOp:
         assert isinstance(out, Adjoint)
         assert out.base is base
         assert len(tape) == 1
-        assert len(tape._queue) == 2
-        assert tape._queue[base] == {"owner": out}
-        assert tape._queue[out] == {"owns": base}
+        assert qml.queuing.AnnotatedQueue.__len__(tape) == 2
+        assert tape.get_info(base) == {"owner": out}
+        assert tape.get_info(out) == {"owns": base}
 
     def test_single_op_defined_outside_queue_eager(self):
         """Test if base is defined outside context and the function eagerly simplifies
@@ -63,8 +63,8 @@ class TestPreconstructedOp:
         assert len(tape) == 1
         assert tape[0] is out
 
-        assert len(tape._queue) == 1
-        assert tape._queue[out] == {"owns": base}
+        assert qml.queuing.AnnotatedQueue.__len__(tape) == 1
+        assert tape.get_info(out) == {"owns": base}
 
     def test_single_observable(self):
         """Test passing a single preconstructed observable in a queuing context."""
@@ -161,8 +161,8 @@ class TestNonLazyExecution:
 
         assert len(tape) == 1
         assert out is tape[0]
-        assert tape._queue[base] == {"owner": out}
-        assert tape._queue[out] == {"owns": base}
+        assert tape.get_info(base) == {"owner": out}
+        assert tape.get_info(out) == {"owns": base}
 
         assert isinstance(out, qml.RX)
         assert out.data == [-1.23]
@@ -175,9 +175,9 @@ class TestNonLazyExecution:
 
         assert len(tape) == 1
         assert out is tape[0]
-        assert len(tape._queue) == 2
-        assert tape._queue[base] == {"owner": out}
-        assert tape._queue[out] == {"owns": base}
+        assert qml.queuing.AnnotatedQueue.__len__(tape) == 2
+        assert tape.get_info(base) == {"owner": out}
+        assert tape.get_info(out) == {"owns": base}
 
         assert isinstance(out, Adjoint)
         assert isinstance(out.base, qml.S)
diff --git a/tests/ops/op_math/test_controlled_op.py b/tests/ops/op_math/test_controlled_op.py
index 9f9e3317c..dee1f2b0f 100644
--- a/tests/ops/op_math/test_controlled_op.py
+++ b/tests/ops/op_math/test_controlled_op.py
@@ -607,8 +607,8 @@ class TestQueuing:
             base = qml.Rot(1.234, 2.345, 3.456, wires=2)
             op = Controlled(base, (0, 1))
 
-        assert tape._queue[base]["owner"] is op
-        assert tape._queue[op]["owns"] is base
+        assert tape.get_info(base)["owner"] is op
+        assert tape.get_info(op)["owns"] is base
         assert tape.operations == [op]
 
     def test_queuing_base_defined_outside(self):
@@ -618,8 +618,8 @@ class TestQueuing:
         with qml.tape.QuantumTape() as tape:
             op = Controlled(base, ("a", "b"))
 
-        assert len(tape._queue) == 1
-        assert tape._queue[op]["owns"] is base
+        assert qml.queuing.AnnotatedQueue.__len__(tape) == 1
+        assert tape.get_info(op)["owns"] is base
         assert tape.operations == [op]
 
     def test_do_queue_false(self):
@@ -629,7 +629,7 @@ class TestQueuing:
         with qml.tape.QuantumTape() as tape:
             op = Controlled(base, 1, do_queue=False)
 
-        assert len(tape._queue) == 0
+        assert qml.queuing.AnnotatedQueue.__len__(tape) == 0
 
 
 base_num_control_mats = [
diff --git a/tests/ops/op_math/test_pow_op.py b/tests/ops/op_math/test_pow_op.py
index 3a81a8349..fd0b0c51a 100644
--- a/tests/ops/op_math/test_pow_op.py
+++ b/tests/ops/op_math/test_pow_op.py
@@ -85,8 +85,8 @@ class TestConstructor:
             original_op = qml.PauliX(0)
             new_op = qml.pow(original_op, 0.5, lazy=False)
 
-        assert q._queue[original_op]["owner"] is new_op
-        assert q._queue[new_op]["owns"] is original_op
+        assert q[original_op]["owner"] is new_op
+        assert q[new_op]["owns"] is original_op
 
 
 @pytest.mark.parametrize("power_method", [Pow, pow_using_dunder_method, qml.pow])
@@ -534,8 +534,8 @@ class TestQueueing:
             base = qml.Rot(1.2345, 2.3456, 3.4567, wires="b")
             op = Pow(base, 1.2)
 
-        assert tape._queue[base]["owner"] is op
-        assert tape._queue[op]["owns"] is base
+        assert tape.get_info(base)["owner"] is op
+        assert tape.get_info(op)["owns"] is base
         assert tape.operations == [op]
 
     def test_queueing_base_defined_outside(self):
@@ -546,7 +546,7 @@ class TestQueueing:
             op = Pow(base, 3.4)
 
         assert len(tape.queue) == 1
-        assert tape._queue[op]["owns"] is base
+        assert tape.get_info(op)["owns"] is base
         assert tape.operations == [op]
 
     def test_do_queue_False(self):
@@ -605,7 +605,7 @@ class TestMatrix:
 
     @pytest.mark.tf
     @pytest.mark.parametrize("z", (2, -2, 1.23, -0.5))
-    def test_matrix_against_shortcut_jax(self, z):
+    def test_matrix_against_shortcut_tf(self, z):
         """Test the matrix using a tf variable parameter."""
         import tensorflow as tf
 
diff --git a/tests/ops/op_math/test_symbolic_op.py b/tests/ops/op_math/test_symbolic_op.py
index 3bf4b84ec..f7f919c88 100644
--- a/tests/ops/op_math/test_symbolic_op.py
+++ b/tests/ops/op_math/test_symbolic_op.py
@@ -171,8 +171,8 @@ class TestQueuing:
             base = TempOperator("a")
             op = SymbolicOp(base)
 
-        assert tape._queue[base]["owner"] is op
-        assert tape._queue[op]["owns"] is base
+        assert tape.get_info(base)["owner"] is op
+        assert tape.get_info(op)["owns"] is base
         assert tape.operations == [op]
 
     def test_queuing_base_defined_outside(self):
@@ -182,8 +182,8 @@ class TestQueuing:
         with qml.tape.QuantumTape() as tape:
             op = SymbolicOp(base)
 
-        assert len(tape._queue) == 1
-        assert tape._queue[op]["owns"] is base
+        assert qml.queuing.AnnotatedQueue.__len__(tape) == 1
+        assert tape.get_info(op)["owns"] is base
         assert tape.operations == [op]
 
     def test_do_queue_false(self):
diff --git a/tests/ops/qubit/test_hamiltonian.py b/tests/ops/qubit/test_hamiltonian.py
index 42feedf52..1130c8e8a 100644
--- a/tests/ops/qubit/test_hamiltonian.py
+++ b/tests/ops/qubit/test_hamiltonian.py
@@ -738,7 +738,7 @@ class TestHamiltonian:
         # check that H is simplified
         assert H.ops == [a, b]
         # check that the simplified Hamiltonian is in the queue
-        assert H in tape._queue
+        assert tape.get_info(H) is not None
 
     def test_data(self):
         """Tests the obs_data method"""
diff --git a/tests/test_operation.py b/tests/test_operation.py
index deb7c24fe..341b20efa 100644
--- a/tests/test_operation.py
+++ b/tests/test_operation.py
@@ -1189,7 +1189,7 @@ class TestTensor:
         assert len(tape.queue) == 1
         assert tape.queue[0] is T
 
-        assert tape._queue[T] == {"owns": (op1, op2)}
+        assert tape[T] == {"owns": (op1, op2)}
 
     def test_queuing(self):
         """Test the queuing of a Tensor object."""
@@ -1204,9 +1204,9 @@ class TestTensor:
         assert tape.queue[1] is op2
         assert tape.queue[2] is T
 
-        assert tape._queue[op1] == {"owner": T}
-        assert tape._queue[op2] == {"owner": T}
-        assert tape._queue[T] == {"owns": (op1, op2)}
+        assert tape[op1] == {"owner": T}
+        assert tape[op2] == {"owner": T}
+        assert tape[T] == {"owns": (op1, op2)}
 
     def test_queuing_observable_matmul(self):
         """Test queuing when tensor constructed with matmul."""
@@ -1217,9 +1217,9 @@ class TestTensor:
             t = op1 @ op2
 
         assert len(tape.queue) == 3
-        assert tape._queue[op1] == {"owner": t}
-        assert tape._queue[op2] == {"owner": t}
-        assert tape._queue[t] == {"owns": (op1, op2)}
+        assert tape[op1] == {"owner": t}
+        assert tape[op2] == {"owner": t}
+        assert tape[t] == {"owns": (op1, op2)}
 
     def test_queuing_tensor_matmul(self):
         """Tests the tensor-specific matmul method updates queuing metadata."""
@@ -1232,8 +1232,8 @@ class TestTensor:
             op3 = qml.PauliZ(2)
             t2 = t @ op3
 
-        assert tape._queue[t2] == {"owns": (op1, op2, op3)}
-        assert tape._queue[op3] == {"owner": t2}
+        assert tape[t2] == {"owns": (op1, op2, op3)}
+        assert tape[op3] == {"owner": t2}
 
     def test_queuing_tensor_matmul_components_outside(self):
         """Tests the tensor-specific matmul method when components are defined outside the
@@ -1243,13 +1243,13 @@ class TestTensor:
         op2 = qml.PauliY(1)
         t1 = Tensor(op1, op2)
 
-        with qml.tape.QuantumTape() as tape:
+        with qml.queuing.AnnotatedQueue() as q:
             op3 = qml.PauliZ(2)
             t2 = t1 @ op3
 
-        assert len(tape._queue) == 2
-        assert tape._queue[op3] == {"owner": t2}
-        assert tape._queue[t2] == {"owns": (op1, op2, op3)}
+        assert len(q) == 2
+        assert q[op3] == {"owner": t2}
+        assert q[t2] == {"owns": (op1, op2, op3)}
 
     def test_queuing_tensor_rmatmul(self):
         """Tests tensor-specific rmatmul updates queuing metatadata."""
@@ -1264,8 +1264,8 @@ class TestTensor:
 
             t2 = op3 @ t1
 
-        assert tape._queue[op3] == {"owner": t2}
-        assert tape._queue[t2] == {"owns": (op3, op1, op2)}
+        assert tape[op3] == {"owner": t2}
+        assert tape[t2] == {"owns": (op3, op1, op2)}
 
     def test_name(self):
         """Test that the names of the observables are
@@ -1653,7 +1653,7 @@ class TestTensor:
             T_pruned = T.prune()
             m = qml.expval(T_pruned)
 
-        ann_queue = tape._queue
+        ann_queue = tape
 
         # the pruned tensor became the owner of Paulis
         assert ann_queue[a]["owner"] == T_pruned
@@ -1684,7 +1684,7 @@ class TestTensor:
             T_pruned = T.prune()
             m = qml.expval(T_pruned)
 
-        ann_queue = tape._queue
+        ann_queue = tape
 
         # the pruned tensor is the Pauli observable
         assert T_pruned == a