Repository: PennyLaneAI/pennylane

Issue:
Documentation-friendly technical details
Currently we have some compiled literature and some internal notes on the subject of quantum gradient circuits. It's early days and this is also a research project, so things are still a bit unorganized.

Knowing that we are going to open source the software, let's begin to organize and clean these things up a bit. We will begin with establishing a document similar to the [conventions page](https://strawberryfields.readthedocs.io/en/latest/op_conventions.html) that we made while developing Strawberry Fields. Here we will refine our ideas and notation, with an eye that it will eventually become part of the documentation


Discussion:
Some updates to the docs have been done in af14a144, but these will still need to updated and polished further before release
Putting a note here so that we remember to include each of the employed gradient formulas on the conventions page
I read through the documentation, and there is already a lot there. 
But what I don't understand is the concept of a quantum node as opposed to a quantum function.
(Sorry, keep hitting the wrong enter key). Is it necessary to have both concepts? Aren't they somewhat the same, and we can use quantum function which controls a quantum device? 

In short, this is confusing to the reader and I'm not sure we need it.
Yes, this is a discussion @josh146 and I have had previously. At some point, I had the same thoughts about these two similar concepts being redundant.

One distinction is that a quantum function is a just mathematical object which happens to be efficiently computable on a quantum device. On the other hand, the QNode is a particular abstraction, with fixed rules about how classical information is converted to quantum information, and back again. 
My suggestion is to rename quantum function to quantum node and use only one concept. What do you think @josh146 @co9olguy ? 

Either way, it would be good to know for the manuscript how we frame this.
I'm very torn on the qfunc vs. qnode. On the one hand, I get the sort of abstraction Nathan mentions (a qfunc is a quantum function that can be run _on_ a particular qnode, that is attached to a hardware device. One qfunc may be used to initialise multiple qnodes).

But, on the other hand, I am guilty of using qfunc/qnode interchangeably in my communication! Which indicates just how close a concept they are.

Using the `QNode` class initializer, this abstraction is easy to visualise:
```python
def quantum_function(x):
  qm.RX(x, 0)
  return qm.expectation.Z(0)

qnode1 = QNode(quantum_function, dev1)
qnode2 = QNode(quantum_function, dev2)
```

This is not possible currently if you use the `qfunc` decorator, as this binds the following function to a QNode/device on function definition, and after that cannot be re-used (as there is no longer any access to the underlying qfunc).

I think we should keep both abstractions in the documentation. However, to avoid confusion, we should rename the decorator, since it essentially binds a qfunc to a qnode for the user.

**Summary:**

* A quantum function is any Python function that conforms to our restricted OpenQML qfunc definition; i.e. can only use `qm.ops`, one per line, and must return an expectation value.

* A qfunc can be bound to a QNode on a device, which is used to build the hybrid computational graph. The same qfunc can be used in multiple QNodes.

@mariaschuld and I were discussing alternate names for the decorator. Options are:

* `@to_qnode`/`@toqnode`
* `@make_qnode`/@makeqnode`
* `@qnode` (as opposed to the class `QNode`, which is in uppercase)
* `@QNode.bind` (make the decorator a class method of `QNode`)

My preference I think would be `@qnode`, thoughts?

Seconded `@qnode`.

So I will use the concept of a qnode in the paper for now, and a quantum function is basically a wrapper for a variational or fixed circuit.

Can we somewhere write down the requirements of a quantum function so that the decorator works? 
I've removed the conventions page, and shifted a lot of what it was meant to cover to other parts of the library (e.g., under Supported Operations). Some further points that were written there are no longer important (e.g., since we put expvals always at the end of a circuit, there is no need to discuss subtleties of non-commuting measurements)

Resolving patch:
diff --git a/pennylane/optimize/lie_algebra.py b/pennylane/optimize/lie_algebra.py
--- a/pennylane/optimize/lie_algebra.py
+++ b/pennylane/optimize/lie_algebra.py
@@ -288,7 +288,9 @@ def step_and_cost(self):
             lie_gradient, self.stepsize, self.trottersteps, self.exact
         )(self.circuit.func)
 
-        self.circuit = qml.QNode(new_circuit, self.circuit.device)
+        # we can set diff_method=None because the gradient of the QNode is computed
+        # directly in this optimizer
+        self.circuit = qml.QNode(new_circuit, self.circuit.device, diff_method=None)
         return self.circuit, cost
 
     def get_su_n_operators(self, restriction):


Test patch:
diff --git a/tests/returntypes/test_new_lie_algebra_optimizer.py b/tests/returntypes/test_new_lie_algebra_optimizer.py
new file mode 100644
--- /dev/null
+++ b/tests/returntypes/test_new_lie_algebra_optimizer.py
@@ -0,0 +1,371 @@
+# Copyright 2018-2021 Xanadu Quantum Technologies Inc.
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""
+Unit tests for the ``LieAlgebraOptimizer``.
+"""
+import pytest
+
+from scipy.sparse.linalg import expm
+import numpy as np
+
+import pennylane as qml
+from pennylane.optimize import LieAlgebraOptimizer
+
+
+def circuit_1():
+    """Simple circuit."""
+    qml.Hadamard(wires=[0])
+    qml.Hadamard(wires=[1])
+
+
+def circuit_2():
+    """Simply parameterized circuit."""
+    qml.RX(0.1, wires=[0])
+    qml.RY(0.5, wires=[1])
+    qml.CNOT(wires=[0, 1])
+    qml.RY(0.6, wires=[0])
+
+
+def circuit_3():
+    """Three-qubit circuit."""
+    qml.RY(0.5, wires=[0])
+    qml.RY(0.6, wires=[1])
+    qml.RY(0.7, wires=[2])
+    qml.CNOT(wires=[0, 1])
+    qml.CNOT(wires=[1, 2])
+    qml.RX(-0.6, wires=[0])
+    qml.RX(-0.3, wires=[1])
+    qml.RX(-0.2, wires=[2])
+
+
+hamiltonian_1 = qml.Hamiltonian(
+    coeffs=[-1.0] * 3,
+    observables=[qml.PauliX(0), qml.PauliZ(1), qml.PauliY(0) @ qml.PauliX(1)],
+)
+
+hamiltonian_2 = qml.Hamiltonian(
+    coeffs=[-0.2, 0.3, -0.15],
+    observables=[
+        qml.PauliY(1),
+        qml.PauliZ(0) @ qml.PauliZ(1),
+        qml.PauliX(0) @ qml.PauliX(1),
+    ],
+)
+
+hamiltonian_3 = qml.Hamiltonian(
+    coeffs=[-2.0], observables=[qml.PauliY(0) @ qml.PauliY(1) @ qml.PauliY(2)]
+)
+
+
+@pytest.mark.parametrize(
+    "circuit,hamiltonian",
+    [
+        (circuit_1, hamiltonian_1),
+        (circuit_1, hamiltonian_2),
+        (circuit_2, hamiltonian_1),
+        (circuit_2, hamiltonian_2),
+        (circuit_3, hamiltonian_3),
+    ],
+)
+def test_lie_algebra_omegas(circuit, hamiltonian):
+    """Test that we calculate the Riemannian gradient coefficients Tr{[rho, H] P_j} correctly."""
+    # pylint: disable=no-member
+
+    nqubits = max([max(ps.wires) for ps in hamiltonian.ops]) + 1
+    wires = range(nqubits)
+    dev = qml.device("default.qubit", wires=nqubits)
+
+    @qml.qnode(dev)
+    def get_state():
+        circuit()
+        return qml.state()
+
+    @qml.qnode(dev)
+    def lie_circuit():
+        circuit()
+        return qml.expval(hamiltonian)
+
+    phi = get_state()
+    rho = np.outer(phi, phi.conj())
+    hamiltonian_np = qml.utils.sparse_hamiltonian(hamiltonian, wires).toarray()
+    lie_algebra_np = hamiltonian_np @ rho - rho @ hamiltonian_np
+    opt = LieAlgebraOptimizer(circuit=lie_circuit)
+    ops = opt.get_su_n_operators(None)[0]
+    omegas_np = []
+    for op in ops:
+        op = qml.math.expand_matrix(op.matrix(), op.wires, wires)
+        omegas_np.append(1j * np.trace(lie_algebra_np @ op))
+    omegas = opt.get_omegas()
+    assert np.allclose(omegas, omegas_np)
+
+
+@pytest.mark.parametrize(
+    "circuit,hamiltonian",
+    [
+        (circuit_1, hamiltonian_1),
+        (circuit_1, hamiltonian_2),
+        (circuit_2, hamiltonian_1),
+        (circuit_2, hamiltonian_2),
+        (circuit_3, hamiltonian_3),
+    ],
+)
+def test_lie_algebra_omegas_restricted(circuit, hamiltonian):
+    """Test that we calculate the (restricted) Riemannian gradient coefficients correctly."""
+    # pylint: disable=no-member
+    nqubits = max([max(ps.wires) for ps in hamiltonian.ops]) + 1
+    wires = range(nqubits)
+    dev = qml.device("default.qubit", wires=nqubits)
+
+    @qml.qnode(dev)
+    def get_state():
+        circuit()
+        return qml.state()
+
+    @qml.qnode(dev)
+    def lie_circuit():
+        circuit()
+        return qml.expval(hamiltonian)
+
+    phi = get_state()
+    rho = np.outer(phi, phi.conj())
+    hamiltonian_np = qml.utils.sparse_hamiltonian(hamiltonian, wires).toarray()
+    lie_algebra_np = hamiltonian_np @ rho - rho @ hamiltonian_np
+
+    restriction = qml.Hamiltonian(
+        coeffs=[1.0] * 3,
+        observables=[qml.PauliX(0), qml.PauliY(1), qml.PauliY(0) @ qml.PauliY(1)],
+    )
+
+    opt = LieAlgebraOptimizer(circuit=lie_circuit, restriction=restriction)
+    ops = opt.get_su_n_operators(restriction)[0]
+    omegas_np = []
+    for op in ops:
+        op = qml.math.expand_matrix(op.matrix(), op.wires, wires)
+        omegas_np.append(1j * np.trace(lie_algebra_np @ op))
+    omegas = opt.get_omegas()
+
+    assert np.allclose(omegas, omegas_np)
+
+
+@pytest.mark.parametrize(
+    "circuit,hamiltonian",
+    [
+        (circuit_1, hamiltonian_1),
+        (circuit_1, hamiltonian_2),
+        (circuit_2, hamiltonian_1),
+        (circuit_3, hamiltonian_3),
+    ],
+)
+def test_lie_algebra_evolution(circuit, hamiltonian):
+    """Test that the optimizer produces the correct unitary to append."""
+    # pylint: disable=no-member
+    nqubits = max([max(ps.wires) for ps in hamiltonian.ops]) + 1
+    wires = range(nqubits)
+    dev = qml.device("default.qubit", wires=nqubits)
+
+    @qml.qnode(dev)
+    def get_state():
+        circuit()
+        return qml.state()
+
+    @qml.qnode(dev)
+    def lie_circuit():
+        circuit()
+        return qml.expval(hamiltonian)
+
+    phi = get_state()
+    rho = np.outer(phi, phi.conj())
+    hamiltonian_np = qml.utils.sparse_hamiltonian(hamiltonian, wires).toarray()
+    lie_algebra_np = hamiltonian_np @ rho - rho @ hamiltonian_np
+
+    phi_exact = expm(-0.1 * lie_algebra_np * 2**nqubits) @ phi
+    rho_exact = np.outer(phi_exact, phi_exact.conj())
+    opt = LieAlgebraOptimizer(circuit=lie_circuit, stepsize=0.1, exact=True)
+    opt.step_and_cost()
+    cost_pl = opt.circuit()
+    cost_exact = np.trace(rho_exact @ hamiltonian_np)
+    assert np.allclose(cost_pl, cost_exact, atol=1e-4)
+
+
+@pytest.mark.parametrize(
+    "circuit,hamiltonian",
+    [
+        (circuit_1, hamiltonian_1),
+        (circuit_1, hamiltonian_2),
+        (circuit_2, hamiltonian_1),
+        (circuit_2, hamiltonian_2),
+        (circuit_3, hamiltonian_3),
+    ],
+)
+def test_lie_algebra_step(circuit, hamiltonian):
+    """Test that we can take subsequent steps with the optimizer."""
+    nqubits = max([max(ps.wires) for ps in hamiltonian.ops]) + 1
+
+    dev = qml.device("default.qubit", wires=nqubits)
+
+    @qml.qnode(dev)
+    def lie_circuit():
+        circuit()
+        return qml.expval(hamiltonian)
+
+    opt = LieAlgebraOptimizer(circuit=lie_circuit)
+    opt.step()
+    opt.step()
+
+
+@pytest.mark.parametrize(
+    "circuit,hamiltonian",
+    [
+        (circuit_1, hamiltonian_1),
+        (circuit_1, hamiltonian_2),
+        (circuit_2, hamiltonian_1),
+        (circuit_2, hamiltonian_2),
+        (circuit_3, hamiltonian_3),
+    ],
+)
+def test_lie_algebra_step_trotterstep(circuit, hamiltonian):
+    """Test that we can take subsequent steps with the optimizer."""
+    nqubits = max([max(ps.wires) for ps in hamiltonian.ops]) + 1
+
+    dev = qml.device("default.qubit", wires=nqubits)
+
+    @qml.qnode(dev)
+    def lie_circuit():
+        circuit()
+        return qml.expval(hamiltonian)
+
+    opt = LieAlgebraOptimizer(circuit=lie_circuit, trottersteps=3)
+    opt.step()
+    opt.step()
+
+
+def test_lie_algebra_circuit_input_1_check():
+    """Test that a type error is raise for non-QNode circuits."""
+
+    def circuit():
+        qml.RY(0.5, wires=0)
+
+    with pytest.raises(TypeError, match="circuit must be a QNode"):
+        LieAlgebraOptimizer(circuit=circuit, stepsize=0.001)
+
+
+def test_lie_algebra_hamiltonian_input_1_check():
+    """Test that a type error is raise for non-QNode circuits."""
+
+    @qml.qnode(qml.device("default.qubit", wires=3))
+    def circuit():
+        qml.RY(0.5, wires=0)
+        return qml.state()
+
+    with pytest.raises(
+        TypeError,
+        match="circuit must return the expectation value of a Hamiltonian",
+    ):
+        LieAlgebraOptimizer(circuit=circuit, stepsize=0.001)
+
+
+def test_lie_algebra_nqubits_check():
+    """Test that we warn if the system is too big."""
+
+    @qml.qnode(qml.device("default.qubit", wires=5))
+    def circuit():
+        qml.RY(0.5, wires=0)
+        return qml.expval(qml.Hamiltonian(coeffs=[-1.0], observables=[qml.PauliX(0)]))
+
+    with pytest.warns(UserWarning, match="The exact Riemannian gradient is exponentially"):
+        LieAlgebraOptimizer(circuit=circuit, stepsize=0.001)
+
+
+def test_lie_algebra_restriction_check():
+    """Test that a type error is raise for non-QNode circuits."""
+
+    @qml.qnode(qml.device("default.qubit", wires=3))
+    def circuit():
+        qml.RY(0.5, wires=0)
+        return qml.expval(qml.Hamiltonian(coeffs=[-1.0], observables=[qml.PauliX(0)]))
+
+    restriction = "not_a_hamiltonian"
+    with pytest.raises(
+        TypeError,
+        match="restriction must be a Hamiltonian",
+    ):
+        LieAlgebraOptimizer(circuit=circuit, restriction=restriction, stepsize=0.001)
+
+
+def test_docstring_example():
+    """Test the docstring example with Trotterized evolution."""
+    hamiltonian = qml.Hamiltonian(
+        coeffs=[-1.0] * 3,
+        observables=[qml.PauliX(0), qml.PauliZ(1), qml.PauliY(0) @ qml.PauliX(1)],
+    )
+
+    @qml.qnode(qml.device("default.qubit", wires=2))
+    def quant_fun():
+        qml.RX(0.1, wires=[0])
+        qml.RY(0.5, wires=[1])
+        qml.CNOT(wires=[0, 1])
+        qml.RY(0.6, wires=[0])
+        return qml.expval(hamiltonian)
+
+    opt = LieAlgebraOptimizer(circuit=quant_fun, stepsize=0.1)
+
+    for _ in range(12):
+        circuit, cost = opt.step_and_cost()
+    circuit()
+    assert np.isclose(cost, -2.236068, atol=1e-3)
+
+
+def test_docstring_example_exact():
+    """Test that the optimizer works with matrix exponential."""
+
+    hamiltonian = qml.Hamiltonian(
+        coeffs=[-1.0] * 3,
+        observables=[qml.PauliX(0), qml.PauliZ(1), qml.PauliY(0) @ qml.PauliX(1)],
+    )
+
+    @qml.qnode(qml.device("default.qubit", wires=2))
+    def quant_fun():
+        qml.RX(0.1, wires=[0])
+        qml.RY(0.5, wires=[1])
+        qml.CNOT(wires=[0, 1])
+        qml.RY(0.6, wires=[0])
+        return qml.expval(hamiltonian)
+
+    opt = LieAlgebraOptimizer(circuit=quant_fun, stepsize=0.1, exact=True)
+
+    for _ in range(12):
+        circuit, cost = opt.step_and_cost()
+    circuit()
+    assert np.isclose(cost, -2.236068, atol=1e-3)
+
+
+def test_example_shots():
+    """Test that the optimizer works with finite shots."""
+    hamiltonian = qml.Hamiltonian(
+        coeffs=[-1.0] * 3,
+        observables=[qml.PauliX(0), qml.PauliZ(1), qml.PauliY(0) @ qml.PauliX(1)],
+    )
+
+    @qml.qnode(qml.device("default.qubit", wires=2, shots=1000), diff_method=None)
+    def quant_fun():
+        qml.RX(0.1, wires=[0])
+        qml.RY(0.5, wires=[1])
+        qml.CNOT(wires=[0, 1])
+        qml.RY(0.6, wires=[0])
+        return qml.expval(hamiltonian)
+
+    opt = LieAlgebraOptimizer(circuit=quant_fun, stepsize=0.1, exact=False)
+
+    for _ in range(3):
+        opt.step_and_cost()