Repository: cupy/cupy

Issue:
Fix `cupyx.scipy.sparse.csgraph` module to work with the latest RAPIDS cuGraph
Follow the API change introduced in RAPIDS 22.12.
https://github.com/cupy/cupy/pull/7647#discussion_r1244820097

cc/ @pentschev 


Discussion:
Thanks @kmaehashi for the ping.

@rlratzel are those API changes something we can keep CuPy devs informed when they happen, or better yet, could the graph team commit to keep CuPy up-to-date when needed?
> @rlratzel are those API changes something we can keep CuPy devs informed when they happen, or better yet, could the graph team commit to keep CuPy up-to-date when needed?

Yes, this was an oversight and we should have realized this would affect cupy.  We'll plan on submitting a PR to cupy to update it to use the latest connected components APIs.  cc @jnke2016
Thanks for letting me know @Rick ***@***.***> . I am investigating the issue, running tests then I will push a PR. I will keep you posted.

Joseph

From: Rick Ratzel ***@***.***>
Date: Monday, July 3, 2023 at 10:27 AM
To: cupy/cupy ***@***.***>
Cc: Joseph Nke ***@***.***>, Mention ***@***.***>
Subject: Re: [cupy/cupy] Fix `cupyx.scipy.sparse.csgraph` module to work with the latest RAPIDS cuGraph (Issue #7672)

@rlratzel<https://github.com/rlratzel> are those API changes something we can keep CuPy devs informed when they happen, or better yet, could the graph team commit to keep CuPy up-to-date when needed?

Yes, this was an oversight and we should have realized this would affect cupy. We'll plan on submitting a PR to cupy to update it to use the latest connected components APIs. cc @jnke2016<https://github.com/jnke2016>

—
Reply to this email directly, view it on GitHub<https://github.com/cupy/cupy/issues/7672#issuecomment-1618656300>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ASD4LHE6WCLDWBTCKJOEFDTXOLQG3ANCNFSM6AAAAAAZX662NQ>.
You are receiving this because you were mentioned.Message ID: ***@***.***>

Thank you all for your help! :pray:

Resolving patch:
diff --git a/cupyx/scipy/sparse/csgraph/_traversal.py b/cupyx/scipy/sparse/csgraph/_traversal.py
--- a/cupyx/scipy/sparse/csgraph/_traversal.py
+++ b/cupyx/scipy/sparse/csgraph/_traversal.py
@@ -51,9 +51,9 @@ def connected_components(csgraph, directed=True, connection='weak',
         raise ValueError('graph should be a square array')
     if csgraph.nnz == 0:
         return m, cupy.arange(m, dtype=csgraph.indices.dtype)
-    labels = cupy.empty(m, dtype=csgraph.indices.dtype)
 
     if connection == 'strong':
+        labels = cupy.empty(m, dtype=csgraph.indices.dtype)
         pylibcugraph.strongly_connected_components(
             offsets=csgraph.indptr, indices=csgraph.indices, weights=None,
             num_verts=m, num_edges=csgraph.nnz, labels=labels)
@@ -61,12 +61,15 @@ def connected_components(csgraph, directed=True, connection='weak',
         csgraph += csgraph.T
         if not cupyx.scipy.sparse.isspmatrix_csr(csgraph):
             csgraph = cupyx.scipy.sparse.csr_matrix(csgraph)
-        pylibcugraph.weakly_connected_components(
-            offsets=csgraph.indptr, indices=csgraph.indices, weights=None,
-            num_verts=m, num_edges=csgraph.nnz, labels=labels)
-        # Note: In the case of weak connection, cuGraph creates labels with a
-        # start number of 1, so decrement the label number.
-        labels -= 1
+        _, labels = pylibcugraph.weakly_connected_components(
+            resource_handle=None,
+            graph=None,
+            indices=csgraph.indices,
+            offsets=csgraph.indptr,
+            weights=None,
+            labels=None,
+            do_expensive_check=False,
+        )
 
     count = cupy.zeros((1,), dtype=csgraph.indices.dtype)
     root_labels = cupy.empty((m,), dtype=csgraph.indices.dtype)


Test patch:
diff --git a/.pfnci/linux/tests/cuda-rapids.Dockerfile b/.pfnci/linux/tests/cuda-rapids.Dockerfile
--- a/.pfnci/linux/tests/cuda-rapids.Dockerfile
+++ b/.pfnci/linux/tests/cuda-rapids.Dockerfile
@@ -1,4 +1,4 @@
-ARG BASE_IMAGE="rapidsai/rapidsai-core-dev:22.10-cuda11.5-devel-ubuntu20.04-py3.9"
+ARG BASE_IMAGE="rapidsai/rapidsai-core-dev:23.06-cuda11.8-devel-ubuntu22.04-py3.10"
 FROM ${BASE_IMAGE}
 
 RUN export DEBIAN_FRONTEND=noninteractive && \
diff --git a/tests/cupyx_tests/scipy_tests/sparse_tests/csgraph_tests/test_traversal.py b/tests/cupyx_tests/scipy_tests/sparse_tests/csgraph_tests/test_traversal.py
--- a/tests/cupyx_tests/scipy_tests/sparse_tests/csgraph_tests/test_traversal.py
+++ b/tests/cupyx_tests/scipy_tests/sparse_tests/csgraph_tests/test_traversal.py
@@ -44,16 +44,18 @@ def test_connected_components(self, xp, sp):
             n, labels = sp.csgraph.connected_components(
                 a, directed=self.directed, connection=self.connection,
                 return_labels=self.return_labels)
-            if xp == numpy and self.directed and self.connection == 'strong':
-                # Note: Label IDs are adjusted as SciPy returns un-ordered
-                # labels in strong connection case.
-                table = xp.zeros((n,), dtype=numpy.int32) - 1
-                j = 0
-                for i in range(labels.size):
-                    if table[labels[i]] < 0:
-                        table[labels[i]] = j
-                        j = j + 1
-                    labels[i] = table[labels[i]]
+            # Note: CuPy returns un-ordered results in both strong and
+            # weak connection case while SciPy returns un-ordered labels
+            # in strong connection case therefore. Since in most cases
+            # the labels returned by both cuPy and Scipy are un-ordered
+            # therefore, always adjust the labels.
+            table = xp.zeros((n,), dtype=numpy.int32) - 1
+            j = 0
+            for i in range(labels.size):
+                if table[labels[i]] < 0:
+                    table[labels[i]] = j
+                    j = j + 1
+                labels[i] = table[labels[i]]
             return n, labels
         else:
             return sp.csgraph.connected_components(