Repository: Qiskit/qiskit

Issue:
Speedup sparse matrix construction in SparsePauliOp
### What should we add?

Just putting this at the top: link to qrusty: https://github.com/chetmurthy/qrusty

When constructing sparse matrixes from SparsePauliOp, the current code constructs a sparse matrix from each BasePauli (which is a string of form 'IIIXYZIZYIII') and then combines then in linear combination with complex coefficients.  This is inefficient and space-wasting on many levels:

1. the BasePauli construction is Python code, and it's not the fastest.
2. for large problems, we can tens of thousands of BasePaulis, each of which takes tens of megabytes, so adding them up entails considerable construction of intermediate datastructures that are nearly-immediately discarded.

Concretely, I'm working with some people who have 24-qubit problems with >20k BasePaulis -- the construction problem alone takes effectively interminable time (heck, the 20-qubit problem runs longer than a weekend).  Their largest problem (so far) has 92B nonzero complex entries, and another person I'm working with has 27-qubit problems.

There's another way to solve this problem, but let's review how the current solution works:
```
sumMatrix := empty
for each (basePauli,coeff) in sparsePauliOp:
    spmat := convert basePauli to sparse matrix
    sumMatrix += coeff * spmat
```
And that convert line can be written as:

```
  basemat := empty
  for each row in 0..2^#qubits:
    basemat = basemat <append> (row,col, value for the nonzero value in this basePauli)
```
Viewed this way, the "sumMatrix +=" (when the matrices are represented as sparse matrices) is really an insertion sort: each row of "basemat" is insertion-sorted into "sumMatrix" and if there's a nonzero value there, then it's added.

But we can swap the order of the iterations, yielding:

```
sumMatrix := empty
for each row in 0..2^#qubits:
  baseRow := empty
  for each (basePauli, coeff) in sparsePauliOp:
    baseRow = baseRow <append> (col, value for nonzero value in this basePauli row)
  sort baseRow by col#, adding value for columns with more than one value
  add baseRow to sumMatrix
```

This code has a natural parallelized implementation by taking the "for each row" loop and running it in parallel.

I have code that can do this in this repo: https://github.com/chetmurthy/qrusty

and specifically this function: https://github.com/chetmurthy/qrusty/blob/ea11318ff4c92d33f2141841b6e28eb4cd1df8b3/qrusty/src/accel.rs#L267
which constructs the three vectors required to build a CSR matrix.

I've built code around this with classes that match "BasePauli" and "SparsePauliOp", so that this Rust code can be used from Python outside of Qiskit, and also a Python wrapper of the `sprs` sparse-matrix type, because the people I'm working with also find that Python Scipy is not parallelized for many operations (e.g. matrix-vector multiply, "a*x+b*y", evaluating "c*M*x + d*M*x" for complex "c", vector "x") that need to be parallelized when matrix dimension reaches 2^20 and above.

There are several ways one might import the basic function of "fast construction of matrices from SparsePauliOp" into Qiskit:

1. just pull in this function (link above) and write enough wrapper to use it from Qiskit

2. refer to Qrusty's implementation (== "construct the qrusty SparsePauliOp, then call the function to construct a SpMat, then convert to a Scipy CSR matrix") but leave Qrusty as-is, so that people who need to do more parallelized matrix operations can do so with Qrusty

3. integrate Qrusty into Qiskit, slowly making qrusty's version of BasePauli and SparsePauliOp into the implementation of Qiskit's versions of those objects.

It seems clear that #3 is a lot of work, maybe even not desirable.  But #1 has the problem that for anybody who actually wants to work with high # of qubits, once you have the matrix you can't do much with it without converting it back into a form where you can then apply Rust parallelism to it.   B/c the matrix is so bloody big.

My belief is, #3 is the way to go for this reason.



Discussion:
While this is probably going to speed things up, it is likely not an ideal solution; you still have big matrices.  Rather, it is easy to directly compute matrix elements given a row and column index and a list of Paulis.  Thus you can do things like matrix-vector operations without constructing a matrix at all.  Generally these are known as "matrix-free methods"
Two thoughts:
1. true enough, but you're suggesting a larger change in the way people write codes in this domain, where this issue is just trying to accelerate the codes as-is.
2. In  the case I mention with 24 qubits and 20k Pauli strings, computing the value of any particular matrix cell means computing whether each of those 20k Pauli strings contributes a nonzero value: while sure, you can do this matrix-free, it's still much more inefficient than computing the matrix once,  I would think.  Though I could be wrong.
It depends really.  But agreed that it is more work than modifying the current operators. 
I *am* curious how one might apply matrix-free methods to computing sparse-matrix x dense-vector product.  Musing on it, it seems like, you want to compute each row sparsely, which means executing the per-row algorithm I described in the issue text.  So you compute the 20k nonzero positions in the 2^24-length row, multiply by the value in the dense-vector for each one, and add 'em all up.  But in the instance, the 24 qubit matrix has 5B nonzero entries -- so that means between 2^8 and 2^9 nonzero entries per row (2^33 / 2^24) so it seems like that's a lot of work being done to avoid .... memoizing the row.

But OTOH, I do take your point, in the sense that, if by doing this you can avoid moving around massive amounts of memory, then maybe it pays for itself just in locality.

I think I should code it up and see! Is the above description of what happens on each row of the sparse matrix x dense vecxtor product an accurate reflection of what you were imagining?
So the basic idea follows Eq.4 from here: https://journals.aps.org/prxquantum/abstract/10.1103/PRXQuantum.2.040326

Namely given a `row` and `col` expressed as integers, you can convert them to their bit-string values and use those values to index the elements of each 1Q operator in the operator string.

Now ideally you would not do this for `I` because those do not matter.  In addition, one can usually break the element computation early if you hit a zero element since the equation is a product.

You can do it in parallel over the rows (or cols), which is commonly done.  In your case, if you did a large sparse matrix dense vector computation (spMV as it is known) you would find that the benefits of parallel iteration diminish with the number of nonzero elements in the matrix (NNZ).  This is because such operations are memory bandwidth limited, and the floating-point capabilities of the CPU are waiting on the memory IO.  The limit is basically the cache size of your processor.

Now ideally all of this would be done as part of a someone larger effort on `SparseOperators` as @mtreinish and myself have been discussing recently.  As you have seen, there are efficiency improvements to be found from the Qiskit operators, not just in terms of dropping them down to lower-level languages, but also just in terms of structure and algorithms.
In general, we would welcome the speedup with Rust as long as there are no API changes.

The bottleneck of the `SparsePauliOp.to_matrix()` method is the following line:
https://github.com/Qiskit/qiskit-terra/blob/5c19a1f34935cc0a230878fc67539af59cc0ab2a/qiskit/quantum_info/operators/symplectic/base_pauli.py#L434
I think it is easiest and most acceptable to rustify this line.

`SparsePauliOp`'s constructor is slow: I think we can rustify https://github.com/Qiskit/qiskit-terra/blob/5c19a1f34935cc0a230878fc67539af59cc0ab2a/qiskit/quantum_info/operators/symplectic/pauli_list.py#L170-L183.

Should I make everything rust?: I don't know if it's possible or not, and isn't it too hard since there are many methods? And we may want to support `jax` in the future (not definitive...)
Hamamura-san,

1. Indeed, I first rustified that line, and it was worth a speedup.  But you still construct massive matrices for each Pauli string, only to "add them up" to get your matrix for the SparsePauliOp.

2. The next step is to observe that the "add them up" is a kind of insertion-sort (viewing each row of the matrix as a list of col/value pairs, adding two rows as vectors is like merging them) where "sum += mat_i" "inserts" the values of "mat_i" into "sum".  So you can speed that up by turning the insertion-sort into a mergesort in the obvious way.

These are worth a good speedup, and make 20-qubit problems tractable (instead of "cannot wait for the matrix to be constructed"), it takes some hours.  But it still takes hours.

3. To do better, you have to build the final matrix row-by-row, directly from the list of Pauli-strings (or their bit-vector equivalents) and coefficients.  And if you do that, you can get the time down for a 20-qubit problem to *negligible* and for 24-qubit problems to ... ten minutes.  Something like that.   Fast enough that it's inconsequential compared to (for example) extracting the lowest eigenvalue of the matrix using the PySCF Davidson solver.
> the basic idea follows Eq.4

I'd like to continue to try to understand your proposal.  I'm going to describe what I think is the situation and hopefully I get it right.  I'm not saying you're wrong,  but rather, that I don't quite understand how to do this efficiently at the sizes we're contemplating.  I'm going to describe the ramifications of your observation/proposal, and if you think I've got it right, then I'll off and code it up.

That method for constructing the matrix that corresponds to a Pauli string, is what's in Qiskit, and what I've implemented in the Rust code of qrusty (directly copying Qiskit's Python code).  Or, at least, I believe so.  Now, the matrix corresponding to a Pauli string has one nonzero value on each row, so when we multiple such a matrix by a dense vector, that row x vector dot-product becomes a vector-lookup and complex-multiply, right?

But when we have 20k or 30k such Pauli strings, then (for each row) we can either

1. iterate over the Pauli-strings, computing the (col,value) nonzero from each, then doing a vector-lookup, to compute the dot-product (which means random-access to the vector
2. or iterate over the Pauli-strings, compute the (col, value) nonzeros, sort by col, construct what is effectively a sparse-vector, and then do a sparse-vector x dense-vector dot-product.

For the *largest* 24-qubit problem I'm working on, we've got 92B nonzeros.  That's ~2^37.  So 2^13 nonzeros per row.  So those 30k Pauli strings produce some (col,value) with the same column-number.  We can either coalesce, constructing (effectively) the (sparse) row of the matrix, or we can just generate the nonzeros one-by-one as we dot-product.  If we do the former, we can march down the 2^24 entry dense-vector linearly; if we do the latter, we have to random-access into the vector.

It seems like the efficient plan is to construct the row, sorting it into a sparse-vector.  But at that point, it comes down to which is more efficient:

a. construct the row and save it (e.g., build the sparse-matrix),  reusing it for each spMV
b. construct the row each time we perform an spMV, use it and then discard it

Is that about correct?  If so, I think this is testable.

ETA: 2^13 nonzeros in a sparse format is 2^18 bytes (2^5 b/c complex + 2 longs, maybe a little less); 2^24 dense vector is 2^28 bytes.  So we're talking larger than cache size regardless.

And I should add that there are people I'm working with who are doing similar things for 2^27 qubits (but with only 55 Pauli strings, so vastly smaller matrices; still, the dense vector is 2^27 entries, so 2^31 bytes).
Yeah, I forgot that the restriction to Paulis simplifies things a bit.

It is easy to compute each row given the form of the Pauli operator.  Expressing your row index in as a bit-string you can get the corresponding column value by flipping the bits that correspond to non-diagonal Paulis.  Computing the element value itself is linear in the number of qubits.

So then yeah, is it better to form a matrix representation of the whole, or go on by one?  Typically matrix-free things are not the fastest, but rather are meant to scale.  I think, after realizing I was missing the Pauli simplification, that it is probably best to do like you are now.  The CSR matrix is efficient to construct here because you can build row by row already.
Ah,  well.  I was hoping for a little (ahem) hunting trip thru the land of code.  Oh, well.  Thank you for the sightseeing though!
Since this issue has been idle for a while, just wanted to say I recently hit the same bottleneck described in this issue, and found the linked `qrusty` code still gives a valuable speed-up.

I am naively doing the conversion by calling these two functions in order:
```
def spo_to_sparse_qrusty(spo):
    paulis = [qrusty.Pauli(l.to_label()) for l in spo.paulis]
    coeffs = spo.coeffs
    return qrusty.SparsePauliOp(paulis, coeffs).to_matrix_mode("RowwiseUnsafeChunked/1000")

def sparse_qrusty_to_sparse_mat(spop):
    shape, data, indices, indptr = spop.export()
    return csr_matrix((data, indices, indptr), shape=shape)
```

While this is probably not optimal, I found for large SPO's this was vastly faster than using Qiskit's existing `SparsePauliOp.to_matrix(sparse=True)`. So it seems like it would still be very worthwhile to update Qiskit to use this routine or something similar if feasible.
I'll have a look at bringing some upgraded SPO-to-matrix routines in-house if it's currently being a bit of a bottleneck for people.  The current version of the code isn't intended to be performant, just simple to get working.  I've got to take a transatlantic flight on Saturday, which is as good an excuse as any to do a bit of Rust acceleration.

Writing an largely new `quantum_info` module based more around matrix-free methods would be a much larger undertaking that we'd need to set aside a lot more time for, and of course any time we expose stuff to Python is going to suffer the same problems that heavy Numpy-based code has (and that things like tensorflow / jax / numba solve in various ways) - everything has to be constructed as "vectorised" loops over simple primitive kernels, which leads to additional loop overhead and temporary allocations.
@aeddins-ibm: just for benchmarking purposes, do you have a rough estimate for the numbers of qubits in your operators and the number of Pauli terms?

I wrote an implementation of both dense-matrix and CSR-matrix construction methods that directly act on Qiskit's `SparsePauliOp`, and the performance is pretty promising (a shade faster than `qrusty` for the problem domains I tried, even with qrusty threaded and the new one as-yet running single-threaded), but I could do with checking more closely to the problem domains you care about.
Thanks Jake, that's very cool you can match the performance even single-threaded. It's not pressing but I'd be interested in trying your method at some point. The convenience/portability of being able to stay within standard Qiskit is definitely a plus as well.

Resolving patch:
diff --git a/qiskit/quantum_info/operators/symplectic/sparse_pauli_op.py b/qiskit/quantum_info/operators/symplectic/sparse_pauli_op.py
--- a/qiskit/quantum_info/operators/symplectic/sparse_pauli_op.py
+++ b/qiskit/quantum_info/operators/symplectic/sparse_pauli_op.py
@@ -23,7 +23,13 @@
 import numpy as np
 import rustworkx as rx
 
-from qiskit._accelerate.sparse_pauli_op import unordered_unique, decompose_dense
+from qiskit._accelerate.sparse_pauli_op import (
+    ZXPaulis,
+    decompose_dense,
+    to_matrix_dense,
+    to_matrix_sparse,
+    unordered_unique,
+)
 from qiskit.circuit.parameter import Parameter
 from qiskit.circuit.parameterexpression import ParameterExpression
 from qiskit.circuit.parametertable import ParameterView
@@ -919,24 +925,39 @@ def to_list(self, array: bool = False):
             return labels
         return labels.tolist()
 
-    def to_matrix(self, sparse: bool = False) -> np.ndarray:
+    def to_matrix(self, sparse: bool = False, force_serial: bool = False) -> np.ndarray:
         """Convert to a dense or sparse matrix.
 
         Args:
-            sparse (bool): if True return a sparse CSR matrix, otherwise
-                           return dense Numpy array (Default: False).
+            sparse: if ``True`` return a sparse CSR matrix, otherwise return dense Numpy
+                array (the default).
+            force_serial: if ``True``, use an unthreaded implementation, regardless of the state of
+                the `Qiskit threading-control environment variables
+                <https://docs.quantum.ibm.com/start/configure-qiskit-local#environment-variables>`__.
+                By default, this will use threaded parallelism over the available CPUs.
 
         Returns:
             array: A dense matrix if `sparse=False`.
             csr_matrix: A sparse matrix in CSR format if `sparse=True`.
         """
-        mat = None
-        for i in self.matrix_iter(sparse=sparse):
-            if mat is None:
-                mat = i
-            else:
-                mat += i
-        return mat
+        if self.coeffs.dtype == object:
+            # Fallback to slow Python-space method.
+            return sum(self.matrix_iter(sparse=sparse))
+
+        pauli_list = self.paulis
+        zx = ZXPaulis(
+            pauli_list.x.astype(np.bool_),
+            pauli_list.z.astype(np.bool_),
+            pauli_list.phase.astype(np.uint8),
+            self.coeffs.astype(np.complex128),
+        )
+        if sparse:
+            from scipy.sparse import csr_matrix
+
+            data, indices, indptr = to_matrix_sparse(zx, force_serial=force_serial)
+            side = 1 << self.num_qubits
+            return csr_matrix((data, indices, indptr), shape=(side, side))
+        return to_matrix_dense(zx, force_serial=force_serial)
 
     def to_operator(self) -> Operator:
         """Convert to a matrix Operator object"""


Test patch:
diff --git a/crates/accelerate/src/test.rs b/crates/accelerate/src/test.rs
new file mode 100644
--- /dev/null
+++ b/crates/accelerate/src/test.rs
@@ -0,0 +1,24 @@
+// This code is part of Qiskit.
+//
+// (C) Copyright IBM 2023
+//
+// This code is licensed under the Apache License, Version 2.0. You may
+// obtain a copy of this license in the LICENSE.txt file in the root directory
+// of this source tree or at http://www.apache.org/licenses/LICENSE-2.0.
+//
+// Any modifications or derivative works of this code must retain this
+// copyright notice, and modified files need to carry a notice indicating
+// that they have been altered from the originals.
+
+/// Helper for tests that involve calling Rayon code from within Miri.  This runs the given
+/// function in a scoped threadpool, which is then immediately dropped.  This means that Miri will
+/// not complain about the global (static) threads that are not joined when the process exits,
+/// which is deliberate.
+pub fn in_scoped_thread_pool<F, T>(worker: F) -> Result<T, ::rayon::ThreadPoolBuildError>
+where
+    T: Send,
+    F: FnOnce() -> T + Send,
+{
+    ::rayon::ThreadPoolBuilder::new()
+        .build_scoped(::rayon::ThreadBuilder::run, |pool| pool.install(worker))
+}
diff --git a/test/python/quantum_info/operators/symplectic/test_sparse_pauli_op.py b/test/python/quantum_info/operators/symplectic/test_sparse_pauli_op.py
--- a/test/python/quantum_info/operators/symplectic/test_sparse_pauli_op.py
+++ b/test/python/quantum_info/operators/symplectic/test_sparse_pauli_op.py
@@ -15,9 +15,11 @@
 import itertools as it
 import unittest
 import numpy as np
+import scipy.sparse
 import rustworkx as rx
 from ddt import ddt
 
+
 from qiskit import QiskitError
 from qiskit.circuit import ParameterExpression, Parameter, ParameterVector
 from qiskit.circuit.parametertable import ParameterView
@@ -259,6 +261,36 @@ def test_to_matrix_large(self):
         np.testing.assert_array_equal(spp_op.to_matrix(), target)
         np.testing.assert_array_equal(spp_op.to_matrix(sparse=True).toarray(), target)
 
+    def test_to_matrix_zero(self):
+        """Test `to_matrix` with a zero operator."""
+        num_qubits = 4
+        zero_numpy = np.zeros((2**num_qubits, 2**num_qubits), dtype=np.complex128)
+        zero = SparsePauliOp.from_list([], num_qubits=num_qubits)
+
+        zero_dense = zero.to_matrix(sparse=False)
+        np.testing.assert_array_equal(zero_dense, zero_numpy)
+
+        zero_sparse = zero.to_matrix(sparse=True)
+        self.assertIsInstance(zero_sparse, scipy.sparse.csr_matrix)
+        np.testing.assert_array_equal(zero_sparse.A, zero_numpy)
+
+    def test_to_matrix_parallel_vs_serial(self):
+        """Parallel execution should produce the same results as serial execution up to
+        floating-point associativity effects."""
+        # Using powers-of-two coefficients to make floating-point arithmetic associative so we can
+        # do bit-for-bit assertions.  Choose labels that have at least few overlapping locations.
+        labels = ["XZIXYX", "YIIYXY", "ZZZIIZ", "IIIIII"]
+        coeffs = [0.25, 0.125j, 0.5 - 0.25j, -0.125 + 0.5j]
+        op = SparsePauliOp(labels, coeffs)
+        np.testing.assert_array_equal(
+            op.to_matrix(sparse=True, force_serial=False).toarray(),
+            op.to_matrix(sparse=True, force_serial=True).toarray(),
+        )
+        np.testing.assert_array_equal(
+            op.to_matrix(sparse=False, force_serial=False),
+            op.to_matrix(sparse=False, force_serial=True),
+        )
+
     def test_to_matrix_parameters(self):
         """Test to_matrix method for parameterized SparsePauliOp."""
         labels = ["XI", "YZ", "YY", "ZZ"]