Repository: dmlc/dgl
Pull request: [DistDGL] add log when load graph and data
## Description
<!-- Brief description. Refer to the related issues if existed.
It'll be great if relevant reviewers can be assigned as well.-->
fix for https://github.com/dmlc/dgl/issues/5295
## Checklist
Please feel free to remove inapplicable items for your PR.
- [ ] The PR title starts with [$CATEGORY] (such as [NN], [Model], [Doc], [Feature]])
- [ ] I've leverage the [tools](https://docs.google.com/document/d/1iHyj7zlmygKSk5gBPsqIqL5ASPzJSPREaNT_QdsiYA4/edit) to beautify the python and c++ code.
- [ ] The PR is complete and small, read the [Google eng practice (CL equals to PR)](https://google.github.io/eng-practices/review/developer/small-cls.html) to understand more about small PR. In DGL, we consider PRs with less than 200 lines of core code change are small (example, test and documentation could be exempted).
- [ ] All changes have test coverage
- [ ] Code is well-documented
- [ ] To the best of my knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change
- [ ] Related issue is referred in this PR
- [ ] If the PR is for a new model/paper, I've updated the example index [here](../examples/README.md).

## Changes
<!-- You could use following template
- [ ] Feature1, tests, (and when applicable, API doc)
- [ ] Feature2, tests, (and when applicable, API doc)
-->


Problem statement:
Fix print statements in distDGL
## 📚 Documentation /Development
### Misleading print statements
1. When training on large graphs, distDGL takes a while  to load the dataset and convert it to different storage formats (up to 1.5 hours in my experience). It does not show any log in the mean time which makes the user think that either the code got stuck or there is some issue with socket connection. 

For example, RGCN on papers100M dataset waits after this step for a long time:

```
[21:50:21] /opt/dgl/src/rpc/rpc.cc:159: Receiver with NetType~socket is created.
[21:50:21] /opt/dgl/src/rpc/rpc.cc:159: Receiver with NetType~socket is created.
```
 
We can add more statements such as:
```
Loading dataset...
Converting graph format...
Storing graph or embedding tables...
```
etc.

### Too many optional messages

Also, the current code prints too many optional messages like padding info (`Pad nids from 69525 to 69524`), partition info (`part 0, train: 69525, val: 8685, test: 5506`), etc. Most of the messages should be printed only if a user turns on the verbose mode.

@Rhett-Ying

Hints from issue discussion:
confirmed. thanks for reporting this...
Also, I think the current code prints too many optional messages like padding info (`Pad nids from 69525 to 69524`), partition info (`part 0, train: 69525, val: 8685, test: 5506`), etc. Most of the messages should be printed only if a user turns on the verbose mode. 
I changed the issue description to add these new info.
@isratnisa  could you help fix it? just print what you think that are essential messages.
```
Also, I think the current code prints too many optional messages like padding info (Pad nids from 69525 to 69524), partition info (part 0, train: 69525, val: 8685, test: 5506), etc. Most of the messages should be printed only if a user turns on the verbose mode.
```

@isratnisa These log are printed by the example script? `//examples/pytorch/grpahsage/dist/trian_dist.py`? Actually there's no `verbose` option in DistDGL for now.

Commit message: fix lint

Patch to review:
diff --git a/python/dgl/distributed/dist_graph.py b/python/dgl/distributed/dist_graph.py
index c8690f0eb64b..1925fa9dce55 100644
--- a/python/dgl/distributed/dist_graph.py
+++ b/python/dgl/distributed/dist_graph.py
@@ -391,8 +391,13 @@ def __init__(
                         self.client_g.edata[k], dtype
                     )
             # Create the graph formats specified the users.
+            print(
+                "Start to create specified graph formats which may take "
+                "non-trivial time."
+            )
             self.client_g = self.client_g.formats(graph_format)
             self.client_g.create_formats_()
+            print("Finished creating specified graph formats.")
             if not disable_shared_mem:
                 self.client_g = _copy_graph_to_shared_mem(
                     self.client_g, graph_name, graph_format
diff --git a/python/dgl/distributed/partition.py b/python/dgl/distributed/partition.py
index cc71d591179d..36d70a1a9cda 100644
--- a/python/dgl/distributed/partition.py
+++ b/python/dgl/distributed/partition.py
@@ -190,7 +190,14 @@ def load_partition(part_config, part_id, load_feats=True):
     assert (
         "part_graph" in part_files
     ), "the partition does not contain graph structure."
-    graph = load_graphs(relative_to_config(part_files["part_graph"]))[0][0]
+    partition_path = relative_to_config(part_files["part_graph"])
+    print(
+        f"Start to load partition from {partition_path} which is "
+        f"{os.path.getsize(partition_path)} bytes. It may take non-trivial "
+        "time for large partition."
+    )
+    graph = load_graphs(partition_path)[0][0]
+    print("Finished loading partition...")
 
     assert (
         NID in graph.ndata
@@ -294,10 +301,24 @@ def load_partition_feats(
     ), "the partition does not contain edge feature."
     node_feats = None
     if load_nodes:
-        node_feats = load_tensors(relative_to_config(part_files["node_feats"]))
+        feat_path = relative_to_config(part_files["node_feats"])
+        print(
+            f"Start to load node data from {feat_path} which is "
+            f"{os.path.getsize(feat_path)} bytes. It may take non-trivial "
+            "time for large data."
+        )
+        node_feats = load_tensors(feat_path)
+        print("Finished loading node data...")
     edge_feats = None
     if load_edges:
-        edge_feats = load_tensors(relative_to_config(part_files["edge_feats"]))
+        feat_path = relative_to_config(part_files["edge_feats"])
+        print(
+            f"Start to load edge data from {feat_path} which is "
+            f"{os.path.getsize(feat_path)} bytes. It may take non-trivial "
+            "time for large data."
+        )
+        edge_feats = load_tensors(feat_path)
+        print("Finished loading node data...")
     # In the old format, the feature name doesn't contain node/edge type.
     # For compatibility, let's add node/edge types to the feature names.
     if node_feats is not None:


Code review comments:
diff_hunk: @@ -294,10 +301,24 @@ def load_partition_feats(
     ), "the partition does not contain edge feature."
     node_feats = None
     if load_nodes:
-        node_feats = load_tensors(relative_to_config(part_files["node_feats"]))
+        feat_path = relative_to_config(part_files["node_feats"])
+        print(
+            f"Start to load node data from {feat_path} which is "
+            f"{os.path.getsize(feat_path)} bytes. It may take non-trivial "
+            "time for large data."
+        )
+        node_feats = load_tensors(feat_path)
+        print("Finished loading node data...")
     edge_feats = None
     if load_edges:
-        edge_feats = load_tensors(relative_to_config(part_files["edge_feats"]))
+        feat_path = relative_to_config(part_files["edge_feats"])
+        print(
+            f"Start to load edge data from {feat_path} which is "
+            f"{os.path.getsize(feat_path)} bytes. It may take non-trivial "
+            "time for large data."
+        )
+        edge_feats = load_tensors(feat_path)
+        print("Finished loading node data...")
path: python/dgl/distributed/partition.py
text: @user1:
Typo `print("Finished loading edge data...")`

diff_hunk: @@ -190,7 +190,14 @@ def load_partition(part_config, part_id, load_feats=True):
     assert (
         "part_graph" in part_files
     ), "the partition does not contain graph structure."
-    graph = load_graphs(relative_to_config(part_files["part_graph"]))[0][0]
+    partition_path = relative_to_config(part_files["part_graph"])
+    print(
+        f"Start to load partition from {partition_path} which is "
+        f"{os.path.getsize(partition_path)} bytes. It may take non-trivial "
+        "time for large partition."
+    )
+    graph = load_graphs(partition_path)[0][0]
+    print("Finished loading partition...")
path: python/dgl/distributed/partition.py
text: @user1:
I think we don't need `...` here as we are not waiting for anything. 

@author:
removed

diff_hunk: @@ -294,10 +301,24 @@ def load_partition_feats(
     ), "the partition does not contain edge feature."
     node_feats = None
     if load_nodes:
-        node_feats = load_tensors(relative_to_config(part_files["node_feats"]))
+        feat_path = relative_to_config(part_files["node_feats"])
+        print(
+            f"Start to load node data from {feat_path} which is "
+            f"{os.path.getsize(feat_path)} bytes. It may take non-trivial "
+            "time for large data."
+        )
+        node_feats = load_tensors(feat_path)
+        print("Finished loading node data...")
path: python/dgl/distributed/partition.py
text: @user1:
Same. May be no `...`