Repository: dmlc/dgl

Issue:
Sanity check in subgraph calls
## 🚀 Feature
Currently the subgraph functions such as `node_subgraph`, `edge_subgraph`, `in_subgraph` and `out_subgraph` does not allow users to pass in duplicated values.  But our functions do not check for those duplicated values, which may result in incomprehensible errors like https://discuss.dgl.ai/t/c10-cudaoutofmemoryerror/2996/5


Discussion:
Related discussion here: https://github.com/dmlc/dgl/issues/3853
I tested the `node_subgraph` interface but I couldn't reproduce the OOM issue as in [https://discuss.dgl.ai/t/c10-cudaoutofmemoryerror/2996](url). Overall, duplicated node ids for subgraph induction would lead to undefined behavior (i.e., make the output graph structure incorrect), but this API itself would not have an OOM issue. Particularly, the result of the CPU version is quite different from the GPU version, where the CPU version induces a totally incorrect subgraph. For example, 
```
device='cpu'
G = dgl.DGLGraph(([100, 101, 102], [101, 102, 100])).to(device)
SG = G.subgraph(torch.tensor([100, 100, 101, 101], device=device))
print('Subraph edges:', SG.edges())
print('Subgraph nid:', SG.ndata[dgl.NID])
print('Subgraph eid:', SG.edata[dgl.EID])
```
produces
```
Subraph edges: (tensor([0, 1]), tensor([1, 1]))
Subgraph nid: tensor([100, 100, 101, 101])
Subgraph eid: tensor([0, 0])
```
, which is totally incorrect. This is because it builds a `IdHashMap` for the nodes when building the edge destination IDs, which will skip deduplicated values. If we change the device to `cuda`, it would be
```
Subraph edges: (tensor([0, 1], device='cuda:0'), tensor([2, 2], device='cuda:0'))
Subgraph nid: tensor([100, 100, 101, 101], device='cuda:0')
Subgraph eid: tensor([0, 0], device='cuda:0')
```
It looks more correct except the edge destination will point to the first occurrence of the deduplicate nodes. This is because it sorts the nodes and use a binary search to look for a destination node, which keeps the deduplicate values.

@jermainewang The deduplicated nodes in subgraph induction would not make a program crash problem. Obviously, we cannot have a light weight sanity check for it. I think we may change our CPU implementation to make it consistent with the GPU code, which induces a subgraph with existing edges.
@BarclayII What do you think?

Several thoughts:
* Sometimes invalid memory access such as out-of-bound error may lead to OOM but it depends on the environment. I suspect when there are duplicate values, our code may have issues like that. 
* If the GPU implementation has sorted the values, then for each thread `i` that handles node `v[i]`, you can check duplicates by simply look at nodes `v[i+1]` or `v[i-1]`. This should be quite lightweighted and can be fused with the kernel that exacts subgraph.
> I think we may change our CPU implementation to make it consistent with the GPU code, which induces a subgraph with existing edges.

This reminds me that we should define the semantics of subgraphing more precisely.  Say, if we view
```python
g.subgraph(nids)
```
as
```python
g.adj()[nids, :][:, nids]
```
Then it can handle duplicated nodes without heavy sanity checks, it also has well-defined semantics, and it will not break existing code.  The GPU implementation is still wrong in this regard though.
> `g.adj()[nids, :][:, nids]`

I still cannot understand why it would not be heavy. Do we guarantee `nids` is sorted here?

> I still cannot understand why it would not be heavy. Do we guarantee `nids` is sorted here?

In this case we do not need sanity checks on the uniqueness at all.  However, we can no longer use `IdHashMap` to do the mapping.  So we either need a multimap (on CPU), or sort the indices ourselves, slice the matrix, and then permute the indices back to the order given by the user (on GPU).
So what about we make a minor fix on the CPU implementation and later we make a feature request to support duplicate values? It also reminds me that we may also need to support inducing a sub bipartite graph, which is used in layer-wise sampling as in [https://discuss.dgl.ai/t/difficulties-with-layer-based-sampling-implementation-ladies-sampler/3163/10](url). The syntax may like `dgl.bipartite_subgraph(g, lhs, rhs)`.

Resolving patch:
diff --git a/python/dgl/subgraph.py b/python/dgl/subgraph.py
--- a/python/dgl/subgraph.py
+++ b/python/dgl/subgraph.py
@@ -34,7 +34,8 @@ def node_subgraph(graph, nodes, *, relabel_nodes=True, store_ids=True, output_de
     graph : DGLGraph
         The graph to extract subgraphs from.
     nodes : nodes or dict[str, nodes]
-        The nodes to form the subgraph. The allowed nodes formats are:
+        The nodes to form the subgraph, which cannot have any duplicate value. The result
+        will be undefined otherwise. The allowed nodes formats are:
 
         * Int Tensor: Each element is a node ID. The tensor must have the same device type
           and ID data type as the graph's.
@@ -333,7 +334,8 @@ def in_subgraph(graph, nodes, *, relabel_nodes=False, store_ids=True, output_dev
     graph : DGLGraph
         The input graph.
     nodes : nodes or dict[str, nodes]
-        The nodes to form the subgraph. The allowed nodes formats are:
+        The nodes to form the subgraph, which cannot have any duplicate value. The result
+        will be undefined otherwise. The allowed nodes formats are:
 
         * Int Tensor: Each element is a node ID. The tensor must have the same device type
           and ID data type as the graph's.
@@ -460,7 +462,8 @@ def out_subgraph(graph, nodes, *, relabel_nodes=False, store_ids=True, output_de
     graph : DGLGraph
         The input graph.
     nodes : nodes or dict[str, nodes]
-        The nodes to form the subgraph. The allowed nodes formats are:
+        The nodes to form the subgraph, which cannot have any duplicate value. The result
+        will be undefined otherwise. The allowed nodes formats are:
 
         * Int Tensor: Each element is a node ID. The tensor must have the same device type
           and ID data type as the graph's.
@@ -588,7 +591,8 @@ def khop_in_subgraph(graph, nodes, k, *, relabel_nodes=True, store_ids=True, out
     graph : DGLGraph
         The input graph.
     nodes : nodes or dict[str, nodes]
-        The starting node(s) to expand. The allowed formats are:
+        The starting node(s) to expand, which cannot have any duplicate value. The result
+        will be undefined otherwise. The allowed formats are:
 
         * Int: ID of a single node.
         * Int Tensor: Each element is a node ID. The tensor must have the same device
@@ -749,7 +753,8 @@ def khop_out_subgraph(graph, nodes, k, *, relabel_nodes=True, store_ids=True, ou
     graph : DGLGraph
         The input graph.
     nodes : nodes or dict[str, nodes]
-        The starting node(s) to expand. The allowed formats are:
+        The starting node(s) to expand, which cannot have any duplicate value. The result
+        will be undefined otherwise. The allowed formats are:
 
         * Int: ID of a single node.
         * Int Tensor: Each element is a node ID. The tensor must have the same device


Test patch: