Repository: PennyLaneAI/pennylane

Issue:
First-order finite-diff differentiation with float32 parameters results in floating-point errors
Currently, parameters of type `float32` will incur floating-point errors with finite-diff gradients.  The JAX interface defaults to `float32`, so the issue is most prominent there, but the problem could occur with any interface.

Inside `JacobianTape.numeric_pd`, the relevant parameter does get converted to `np.float64` when [the shift is added](https://github.com/PennyLaneAI/pennylane/blob/23b194ea61dd4158478ce06ef7f4719105f20b57/pennylane/tape/jacobian_tape.py#L265).  But for the calculation of [`"y0"`](https://github.com/PennyLaneAI/pennylane/blob/23b194ea61dd4158478ce06ef7f4719105f20b57/pennylane/tape/jacobian_tape.py#L536), the parameter values are not converted.

Because of this, finite differences will be accurate when `order=2`, but not when `order=1` (the default).

I propose fixing this by casting parameters to `np.float64` for calculation of `"y0"`:

```
                params_f64 = np.array(params, dtype=np.float64)
                options["y0"] = np.asarray(self.execute_device(params_f64, device))
```

This fix assumes every value in `params` is a single scalar. In other words, `params` is not a ragged array. Are we willing to make that assumption?


Discussion:
I think your fix makes sense, especially if we are already converting the parameters to `float64`.

> This fix assumes every value in params is a single scalar. In other words, params is not a ragged array. Are we willing to make that assumption?

I am willing to make that assumption, yes. I believe a lot of our abstractions will break if we consider a non-scalar as a 'differentiable parameter'.

I think it should be as simple as editing the failing test in `default.mixed` to ensure that the Hermitian matrix created has `requires_grad=False`.


Resolving patch:
diff --git a/pennylane/tape/jacobian_tape.py b/pennylane/tape/jacobian_tape.py
--- a/pennylane/tape/jacobian_tape.py
+++ b/pennylane/tape/jacobian_tape.py
@@ -538,7 +538,9 @@ def jacobian(self, device, params=None, **options):
                 # First order (forward) finite-difference will be performed.
                 # Compute the value of the tape at the current parameters here. This ensures
                 # this computation is only performed once, for all parameters.
-                options["y0"] = np.asarray(self.execute_device(params, device))
+                # convert to float64 to eliminate floating point errors when params float32
+                params_f64 = np.array(params, dtype=np.float64)
+                options["y0"] = np.asarray(self.execute_device(params_f64, device))
 
         # some gradient methods need the device or the device wires
         options["device"] = device


Test patch:
diff --git a/pennylane/devices/tests/test_compare_default_qubit.py b/pennylane/devices/tests/test_compare_default_qubit.py
--- a/pennylane/devices/tests/test_compare_default_qubit.py
+++ b/pennylane/devices/tests/test_compare_default_qubit.py
@@ -52,6 +52,7 @@ def test_hermitian_expectation(self, device, tol):
                 [-5 - 2j, -5 - 4j, -4 - 3j, -6],
             ]
         )
+        A_.requires_grad = False
 
         def circuit(theta, phi):
             qml.RX(theta, wires=[0])
diff --git a/tests/tape/test_qnode.py b/tests/tape/test_qnode.py
--- a/tests/tape/test_qnode.py
+++ b/tests/tape/test_qnode.py
@@ -956,3 +956,31 @@ def circuit(a):
         res = circuit(0.8, shots=2)
         assert len(res) == 2
         assert dev.shots == 3
+
+
+def test_finitediff_float32(tol):
+    """Tests that float32 parameters do not effect order 1 finite-diff results.
+
+    Checks bugfix.  Problem occured with StronglyEntanglingLayers, but not simpler circuits.
+    """
+
+    n_wires = 2
+    n_layers = 2
+
+    shape = qml.templates.StronglyEntanglingLayers.shape(n_wires=n_wires, n_layers=n_layers)
+
+    rng = np.random.default_rng(seed=42)
+    params = rng.random(shape)
+    params_f32 = np.array(params, dtype=np.float32)
+
+    dev = qml.device("default.qubit", n_wires)
+
+    @qml.qnode(dev, diff_method="finite-diff", order=1)
+    def circuit(params):
+        qml.templates.StronglyEntanglingLayers(params, wires=range(n_wires))
+        return qml.expval(qml.PauliZ(0))
+
+    grad64 = qml.grad(circuit)(params)
+    grad32 = qml.grad(circuit)(params_f32)
+
+    assert np.allclose(grad64, grad32, atol=tol, rtol=0)