Repository: PennyLaneAI/pennylane

Issue:
[BUG] imaginary part of expectation values of observables with complex coefficients is silently discarded
### Expected behavior

The PL operator arithmetic allows for operators with imaginary/complex  coefficients. If such expectation values of such operators are measured I expect to either get a imaginary/complex value (the real expectation value of the Hermitian operator times the complex coefficient) or an exception that only Hermitian observables can be measured.

### Actual behavior

Instead PennyLane silently discards the imaginary part, as the attached example code shows. 

### Additional information

_No response_

### Source code

```shell
import pennylane as qml

obs = (0.2 + 0.7j) * qml.PauliZ(0)

@qml.qnode(qml.device("default.qubit", 1))
def qnode():
    return qml.expval(obs)

qnode()
```


### Tracebacks

_No response_

### System information

```shell
Name: PennyLane
Version: 0.35.1
Summary: PennyLane is a cross-platform Python library for quantum computing, quantum machine learning, and quantum chemistry. Train a quantum computer the same way as a neural network.
Home-page: https://github.com/PennyLaneAI/pennylane
Author:
Author-email:
License: Apache License 2.0
Location: /home/cvjjm/miniforge3/envs/[...]/lib/python3.11/site-packages
Requires: appdirs, autograd, autoray, cachetools, networkx, numpy, pennylane-lightning, requests, rustworkx, scipy, semantic-version, toml, typing-extensions
Required-by: matchgate_shadows, PennyLane_Lightning

Platform info:           Linux-5.15.153.1-microsoft-standard-WSL2-x86_64-with-glibc2.31
Python version:          3.11.9
Numpy version:           1.23.5
Scipy version:           1.13.0
Installed devices:
- lightning.qubit (PennyLane_Lightning-0.35.1)
- default.clifford (PennyLane-0.35.1)
- default.gaussian (PennyLane-0.35.1)
- default.mixed (PennyLane-0.35.1)
- default.qubit (PennyLane-0.35.1)
- default.qubit.autograd (PennyLane-0.35.1)
- default.qubit.jax (PennyLane-0.35.1)
- default.qubit.legacy (PennyLane-0.35.1)
- default.qubit.tf (PennyLane-0.35.1)
- default.qubit.torch (PennyLane-0.35.1)
- default.qutrit (PennyLane-0.35.1)
- null.qubit (PennyLane-0.35.1)
```


### Existing GitHub issues

- [X] I have searched existing GitHub issues to make sure the issue does not already exist.

Discussion hints:
github-actions[bot]: Hello. You may have forgotten to update the changelog!
Please edit [`doc/releases/changelog-dev.md`](/PennyLaneAI/pennylane/blob/master/doc/releases/changelog-dev.md) with:
* A one-to-two sentence description of the change. You may include a small working example for new features.
* A link back to this PR.
* Your name (or GitHub username) in the contributors section.
codecov[bot]: ## [Codecov](https://app.codecov.io/gh/PennyLaneAI/pennylane/pull/6939?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PennyLaneAI) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 99.59%. Comparing base [(`0ecf816`)](https://app.codecov.io/gh/PennyLaneAI/pennylane/commit/0ecf816e93da558599087e282d01ebac64ff4715?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PennyLaneAI) to head [(`e3a953c`)](https://app.codecov.io/gh/PennyLaneAI/pennylane/commit/e3a953c4ae15da43886ee5f63f0500b4660f9962?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PennyLaneAI).

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##           master    #6939      +/-   ##
==========================================
- Coverage   99.59%   99.59%   -0.01%     
==========================================
  Files         480      480              
  Lines       45505    45504       -1     
==========================================
- Hits        45320    45319       -1     
  Misses        185      185              
```

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/PennyLaneAI/pennylane/pull/6939?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PennyLaneAI).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PennyLaneAI).

andrijapau: >  @JerryChen97: Its still worth adding another sys test in maybe devices/qubit/measure or simulate, where there should be lots of test suite starting from a Hamiltonian and expecting certain output.
andrijapau: > > @JerryChen97: Its still worth adding another sys test in maybe devices/qubit/measure or simulate, where there should be lots of test suite starting from a Hamiltonian and expecting certain output.
> 
> Sure, let me add a system test

Lol, turns out there was another place that we were converting to real silently. Added a test to cover that scenario as well.
andrijapau: > > > @JerryChen97: Its still worth adding another sys test in maybe devices/qubit/measure or simulate, where there should be lots of test suite starting from a Hamiltonian and expecting certain output.
> > 
> > 
> > Sure, let me add a system test
> 
> Lol, turns out there was another place that we were converting to real silently. Added a test to cover that scenario as well. Although, idk if the change to `measure.py` is beyond the scope of this PR ... what do we think @mudit2812 @JerryChen97?

Nevermind, I had to revert it, it affected some gradient stuff because it needs a scalar output. To get this all fixed goes beyond the scope of this bug fix.
JerryChen97: Then this is very interesting... Why in the very beginning the system test was running well without unexpected results?
JerryChen97: > Then this is very interesting... Why in the very beginning the system test was running well without unexpected results?

With a glance at the diff of two tests, I guess `Hamiltonian` made the difference?
andrijapau: > > Then this is very interesting... Why in the very beginning the system test was running well without unexpected results?
> 
> With a glance at the diff of two tests, I guess `Hamiltonian` made the difference?

Indeed, because the observable is a `Hamiltonian`, the test calls `csr_dot_products` in `measure.py` which subsequently converts the result to real. So I thought I could remove that cast to real but it broke the gradient tests.
JerryChen97: > > > Then this is very interesting... Why in the very beginning the system test was running well without unexpected results?
> > 
> > 
> > With a glance at the diff of two tests, I guess `Hamiltonian` made the difference?
> 
> Indeed, because the observable is a `Hamiltonian`, the test calls `csr_dot_products` in `measure.py` which subsequently converts the result to real. So I thought I could remove that cast to real but it broke the gradient tests.

Thanks for experimenting with this; it happens to be crucial for me since I might need to touch those `csr_dot` very soon lool
JerryChen97: How about add the fixed usecase into `test_simulate.py` as a new test class or a single test suite? For example:
```python
@pytest.mark.parametrize("coeffs", [1, 0.5, 0.5j, 0.5 + 0.5j])
def test_expected_dtype_unchanged(coeffs):
    """Test that the return type of the expval function is correct, not brute-forcely converted into real"""

    @qml.qnode(qml.device("default.qubit"))
    def circuit(coeffs):
        return qml.expval(coeffs * qml.PauliZ(0))

    res = circuit(coeffs)
    assert np.allclose(res, coeffs)
```

Solution patch:
diff --git a/doc/releases/changelog-dev.md b/doc/releases/changelog-dev.md
index 1e56914c5..b5eb82fb2 100644
--- a/doc/releases/changelog-dev.md
+++ b/doc/releases/changelog-dev.md
@@ -282,6 +282,9 @@
 
 <h3>Bug fixes 🐛</h3>
 
+* `qml.expval` no longer silently casts to a real number when observable coefficients are imaginary.
+  [(#6939)](https://github.com/PennyLaneAI/pennylane/pull/6939)
+
 * Fixed `qml.wires.Wires` initialization to disallow `Wires` objects as wires labels.
   Now, `Wires` is idempotent, e.g. `Wires([Wires([0]), Wires([1])])==Wires([0, 1])`.
   [(#6933)](https://github.com/PennyLaneAI/pennylane/pull/6933)
diff --git a/pennylane/measurements/expval.py b/pennylane/measurements/expval.py
index b6ef68544..d21c58192 100644
--- a/pennylane/measurements/expval.py
+++ b/pennylane/measurements/expval.py
@@ -162,5 +162,4 @@ class ExpectationMP(SampleMeasurement, StateMeasurement):
         Args:
             probabilities (array): the probabilities of collapsing to eigen states
         """
-        eigvals = qml.math.cast_like(self.eigvals(), 1.0)
-        return qml.math.dot(probabilities, eigvals)
+        return qml.math.dot(probabilities, self.eigvals())


Test patch:
diff --git a/tests/measurements/test_expval.py b/tests/measurements/test_expval.py
index 4700fd82f..30f191888 100644
--- a/tests/measurements/test_expval.py
+++ b/tests/measurements/test_expval.py
@@ -45,9 +45,52 @@ def test_expval_identity_nowires_LQ():
         _ = qnode()
 
 
+# pylint: disable=too-many-public-methods
 class TestExpval:
     """Tests for the expval function"""
 
+    @pytest.mark.parametrize("coeffs", [1, 1j, 1 + 1j])
+    def test_process_counts_dtype(self, coeffs):
+        """Test that the return type of the process_counts function is correct"""
+        counts = {"000": 100, "100": 100}
+        wire_order = qml.wires.Wires((0, 1, 2))
+        res = qml.expval(coeffs * qml.Z(1)).process_counts(counts=counts, wire_order=wire_order)
+        assert np.allclose(res, coeffs)
+
+    @pytest.mark.parametrize("coeffs", [1, 1j, 1 + 1j])
+    def test_process_state_dtype(self, coeffs):
+        """Test that the return type of the process_state function is correct"""
+        res = qml.measurements.ExpectationMP(obs=coeffs * qml.Z(0)).process_state(
+            state=[1, 0], wire_order=qml.wires.Wires(0)
+        )
+        assert np.allclose(res, coeffs)
+
+    @pytest.mark.parametrize("coeffs", [1, 1j, 1 + 1j])
+    @pytest.mark.parametrize(
+        "state,expected",
+        [
+            ([[1.0, 0.0], [0.0, 0.0]], 1.0),  # Pure |0⟩ state
+            ([[0.0, 0.0], [0.0, 1.0]], -1.0),  # Pure |1⟩ state
+            ([[0.5, 0.5], [0.5, 0.5]], 0.0),  # Mixed state
+            ([[0.75, 0.0], [0.0, 0.25]], 0.5),  # Another mixed state
+        ],
+    )
+    def test_process_density_matrix_dtype(self, coeffs, state, expected):
+        mp = ExpectationMP(obs=coeffs * qml.PauliZ(0))
+        result = mp.process_density_matrix(state, wire_order=qml.wires.Wires([0]))
+        assert qml.math.allclose(result, expected * coeffs)
+
+    @pytest.mark.parametrize("coeffs", [1, 1j, 1 + 1j])
+    def test_process_samples_dtype(self, coeffs, seed):
+        """Test that the return type of the process_samples function is correct"""
+        shots = 100
+        rng = np.random.default_rng(seed)
+        samples = rng.choice([0, 1], size=(shots, 2)).astype(np.int64)
+        obs = coeffs * qml.PauliZ(0)
+        expected = qml.expval(obs).process_samples(samples, [0, 1])
+        result = ExpectationMP(obs=obs).process_samples(samples, [0, 1])
+        assert qml.math.allclose(result, expected)
+
     @pytest.mark.parametrize("shots", [None, 1111, [1111, 1111]])
     def test_value(self, tol, shots, seed):
         """Test that the expval interface works"""
@@ -358,3 +401,15 @@ class TestExpval:
         res = qml.expval(qml.Z(wire)).process_counts(counts=counts, wire_order=wire_order)
 
         assert np.allclose(res, expected)
+
+
+@pytest.mark.parametrize("coeffs", [1, 1j, 1 + 1j])
+def test_qnode_expval_dtype(coeffs):
+    """System level test to ensure dtype is correctly preserved."""
+
+    @qml.qnode(qml.device("default.qubit"))
+    def circuit(coeffs):
+        return qml.expval(coeffs * qml.PauliZ(0))
+
+    res = circuit(coeffs)
+    assert np.allclose(res, coeffs)