internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: dmlc/dgl

Issue:
[Docs] `num_traces` in `dgl.sampling.random_walk`
## 📚 Documentation

This is a API reference part.

In the doc of `dgl.sampling.random_walk`: [page link](https://docs.dgl.ai/generated/dgl.sampling.random_walk.html?highlight=random%20walk#dgl.sampling.random_walk) , it says: `For a single starting node, num_traces traces would be generated`. However, I can't find `num_traces` in the following parameters list nor function explaination and examples. And calling of `dgl.sampling.random_walk(g, nodes, num_traces=n)` will raise an Error: `TypeError: random_walk() got an unexpected keyword argument 'num_traces'`.



Discussion:
What's your dgl version? We've just released 0.7

Resolving patch:
diff --git a/python/dgl/sampling/randomwalks.py b/python/dgl/sampling/randomwalks.py
--- a/python/dgl/sampling/randomwalks.py
+++ b/python/dgl/sampling/randomwalks.py
@@ -15,12 +15,15 @@ def random_walk(g, nodes, *, metapath=None, length=None, prob=None, restart_prob
                 return_eids=False):
     """Generate random walk traces from an array of starting nodes based on the given metapath.
 
-    For a single starting node, ``num_traces`` traces would be generated.  A trace would
+    Each starting node will have one trace generated, which
 
     1. Start from the given node and set ``t`` to 0.
     2. Pick and traverse along edge type ``metapath[t]`` from the current node.
     3. If no edge can be found, halt.  Otherwise, increment ``t`` and go to step 2.
 
+    To generate multiple traces for a single node, you can specify the same node multiple
+    times.
+
     The returned traces all have length ``len(metapath) + 1``, where the first node
     is the starting node itself.
 


Test patch:
Repository: dmlc/dgl

Issue:
Small typo in GAT tutorial
## 📚 Documentation

Hi there,

there's a small typo in this tutorial page for GAT: https://docs.dgl.ai/en/v0.7.0/tutorials/models/1_gnn/9_gat.html

```
**GAN** uses weighting neighbor features with feature dependent and structure-free normalization, in the style of attention.
```

I think you mean _GAT_.


Discussion:
Thanks! Would you like to push a patch? :)

Resolving patch:
diff --git a/tutorials/models/1_gnn/9_gat.py b/tutorials/models/1_gnn/9_gat.py
--- a/tutorials/models/1_gnn/9_gat.py
+++ b/tutorials/models/1_gnn/9_gat.py
@@ -29,7 +29,7 @@
 the research paper `GraphSAGE
 <https://www-cs-faculty.stanford.edu/people/jure/pubs/graphsage-nips17.pdf>`_.
 However, `Graph Attention Network <https://arxiv.org/abs/1710.10903>`_ proposes a
-different type of aggregation. GAN uses weighting neighbor features with feature dependent and
+different type of aggregation. GAT uses weighting neighbor features with feature dependent and
 structure-free normalization, in the style of attention.
 """
 ###############################################################


Test patch:
Repository: dmlc/dgl

Issue:
Edge dataloader exclude parameter not working
## 🐛 Bug

When set "exclude" parameter in EdgeDataLoader, the returned block will not exclude the sampled edges and reverse edges

## To Reproduce

Steps to reproduce the behavior:
A simple script to reproduce the behaviour
```python
 import torch
 import dgl
 src = torch.tensor([1,2,3,4])
 dst = torch.tensor([0,0,0,0])
 graph = dgl.graph((torch.cat((src, dst)), torch.cat((dst, src))))
 train_eid = torch.tensor([i for i in range(graph.edges()[0].shape[0])])
 reverse_eids = torch.tensor([4,5,6,7,0,1,2,3])
 sampler = dgl.dataloading.MultiLayerFullNeighborSampler(1)
 dataloader = dgl.dataloading.EdgeDataLoader(graph, train_eid, sampler, exclude = 'reverse_id',         reverse_eids = reverse_eids, batch_size = 1)
 
 for input_nodes, pair_graph, blocks in dataloader:
     print(pair_graph.edges(form = 'uv'))
     print(blocks[0].edges(form = 'uv'))
     break
```
The result is:
(tensor([0]), tensor([1]))
(tensor([1,0,2,3,4]), tensor([0,1,1,1,1]))

The block does not exclude the sampled edges and reverse edge

## Expected behavior
From my perspective, the sampled edge and its reverse edge should be removed:
(tensor([0]), tensor([1]))
(tensor([2,3,4]), tensor([1,1,1]))

## Environment

 - DGL Version : 0.7.1
 - Backend Library & Version (e.g., PyTorch 0.4.1, MXNet/Gluon 1.3): Pytorch 1.9.1
 - OS (e.g., Linux): Linux
 - How you installed DGL (`conda`, `pip`, source): pip
 - Python version: 3.8.8




Discussion:

Resolving patch:
diff --git a/python/dgl/dataloading/dataloader.py b/python/dgl/dataloading/dataloader.py
--- a/python/dgl/dataloading/dataloader.py
+++ b/python/dgl/dataloading/dataloader.py
@@ -108,6 +108,7 @@ def __call__(self, frontier):
                         frontier, v, etype=k, store_ids=True)
                     new_eids[k] = F.gather_row(parent_eids[k], frontier.edges[k].data[EID])
             frontier.edata[EID] = new_eids
+        return frontier
 
 
 def _create_eid_excluder(exclude_eids, device):
@@ -388,7 +389,7 @@ def sample_blocks(self, g, seed_nodes, exclude_eids=None):
             if not self.exclude_edges_in_frontier:
                 eid_excluder = _create_eid_excluder(exclude_eids, self.output_device)
                 if eid_excluder is not None:
-                    eid_excluder(frontier)
+                    frontier = eid_excluder(frontier)
 
             block = transform.to_block(frontier, seed_nodes_out)
             if self.return_eids:


Test patch:
Repository: dmlc/dgl

Issue:
Edge dataloader exclude parameter not working
## 🐛 Bug

When set "exclude" parameter in EdgeDataLoader, the returned block will not exclude the sampled edges and reverse edges

## To Reproduce

Steps to reproduce the behavior:
A simple script to reproduce the behaviour
```python
 import torch
 import dgl
 src = torch.tensor([1,2,3,4])
 dst = torch.tensor([0,0,0,0])
 graph = dgl.graph((torch.cat((src, dst)), torch.cat((dst, src))))
 train_eid = torch.tensor([i for i in range(graph.edges()[0].shape[0])])
 reverse_eids = torch.tensor([4,5,6,7,0,1,2,3])
 sampler = dgl.dataloading.MultiLayerFullNeighborSampler(1)
 dataloader = dgl.dataloading.EdgeDataLoader(graph, train_eid, sampler, exclude = 'reverse_id',         reverse_eids = reverse_eids, batch_size = 1)
 
 for input_nodes, pair_graph, blocks in dataloader:
     print(pair_graph.edges(form = 'uv'))
     print(blocks[0].edges(form = 'uv'))
     break
```
The result is:
(tensor([0]), tensor([1]))
(tensor([1,0,2,3,4]), tensor([0,1,1,1,1]))

The block does not exclude the sampled edges and reverse edge

## Expected behavior
From my perspective, the sampled edge and its reverse edge should be removed:
(tensor([0]), tensor([1]))
(tensor([2,3,4]), tensor([1,1,1]))

## Environment

 - DGL Version : 0.7.1
 - Backend Library & Version (e.g., PyTorch 0.4.1, MXNet/Gluon 1.3): Pytorch 1.9.1
 - OS (e.g., Linux): Linux
 - How you installed DGL (`conda`, `pip`, source): pip
 - Python version: 3.8.8




Discussion:

Resolving patch:
diff --git a/python/dgl/dataloading/dataloader.py b/python/dgl/dataloading/dataloader.py
--- a/python/dgl/dataloading/dataloader.py
+++ b/python/dgl/dataloading/dataloader.py
@@ -386,7 +386,7 @@ def sample_blocks(self, g, seed_nodes, exclude_eids=None):
 
             # Removing edges from the frontier for link prediction training falls
             # into the category of frontier postprocessing
-            if not self.exclude_edges_in_frontier:
+            if not self.exclude_edges_in_frontier(g):
                 eid_excluder = _create_eid_excluder(exclude_eids, self.output_device)
                 if eid_excluder is not None:
                     frontier = eid_excluder(frontier)
diff --git a/python/dgl/dataloading/neighbor.py b/python/dgl/dataloading/neighbor.py
--- a/python/dgl/dataloading/neighbor.py
+++ b/python/dgl/dataloading/neighbor.py
@@ -165,3 +165,7 @@ class MultiLayerFullNeighborSampler(MultiLayerNeighborSampler):
     """
     def __init__(self, n_layers, return_eids=False):
         super().__init__([None] * n_layers, return_eids=return_eids)
+
+    @classmethod
+    def exclude_edges_in_frontier(cls, g):
+        return False


Test patch:
Repository: dmlc/dgl

Issue:
Error in "Link Prediction using Graph Neural Networks"
## 📚 Documentation
<!-- Please specify whether it's tutorial part or API reference part-->
When split negative edges to train and test set, we may have use wrong number.
https://docs.dgl.ai/en/latest/tutorials/blitz/4_link_predict.html

```
# Split edge set for training and testing
u, v = g.edges()

eids = np.arange(g.number_of_edges())
eids = np.random.permutation(eids)
test_size = int(len(eids) * 0.1)
train_size = g.number_of_edges() - test_size
test_pos_u, test_pos_v = u[eids[:test_size]], v[eids[:test_size]]
train_pos_u, train_pos_v = u[eids[test_size:]], v[eids[test_size:]]

# Find all negative edges and split them for training and testing
adj = sp.coo_matrix((np.ones(len(u)), (u.numpy(), v.numpy())))
adj_neg = 1 - adj.todense() - np.eye(g.number_of_nodes())
neg_u, neg_v = np.where(adj_neg != 0)

neg_eids = np.random.choice(len(neg_u), g.number_of_edges())
test_neg_u, test_neg_v = neg_u[neg_eids[:test_size]], neg_v[neg_eids[:test_size]]

##################
train_neg_u, train_neg_v = neg_u[neg_eids[train_size:]], neg_v[neg_eids[train_size:]]
# "train_size" may should be "test_size"?
##################
```



<!-- Describe the issue.-->



Discussion:

Resolving patch:
diff --git a/tutorials/blitz/4_link_predict.py b/tutorials/blitz/4_link_predict.py
--- a/tutorials/blitz/4_link_predict.py
+++ b/tutorials/blitz/4_link_predict.py
@@ -99,7 +99,7 @@
 
 neg_eids = np.random.choice(len(neg_u), g.number_of_edges())
 test_neg_u, test_neg_v = neg_u[neg_eids[:test_size]], neg_v[neg_eids[:test_size]]
-train_neg_u, train_neg_v = neg_u[neg_eids[train_size:]], neg_v[neg_eids[train_size:]]
+train_neg_u, train_neg_v = neg_u[neg_eids[test_size:]], neg_v[neg_eids[test_size:]]
 
 
 ######################################################################
diff --git a/tutorials/large/L2_large_link_prediction.py b/tutorials/large/L2_large_link_prediction.py
--- a/tutorials/large/L2_large_link_prediction.py
+++ b/tutorials/large/L2_large_link_prediction.py
@@ -395,7 +395,7 @@ def closure():
 # You also need to label the edges, 1 if positive and 0 if negative.
 #
 
-test_src = torch.cat([test_pos_src, test_neg_src])
+test_src = torch.cat([test_pos_src, test_pos_dst])
 test_dst = torch.cat([test_neg_src, test_neg_dst])
 test_graph = dgl.graph((test_src, test_dst), num_nodes=graph.num_nodes())
 test_graph.edata['label'] = torch.cat(


Test patch:
Repository: dmlc/dgl

Issue:
name 'UnfoldingAndAttention' is not defined
## 🐛 Bug

https://docs.dgl.ai/api/python/nn.pytorch.html#twirlsconv
run example error
![image](https://user-images.githubusercontent.com/16085476/144708043-124d4d61-6026-4a43-be21-bbe3a0d38ec0.png)






Discussion:
cc @FFTYYY 

Resolving patch:
diff --git a/python/dgl/nn/pytorch/conv/twirlsconv.py b/python/dgl/nn/pytorch/conv/twirlsconv.py
--- a/python/dgl/nn/pytorch/conv/twirlsconv.py
+++ b/python/dgl/nn/pytorch/conv/twirlsconv.py
@@ -6,7 +6,6 @@
 import torch.nn.functional as F
 from .... import function as fn
 
-
 class TWIRLSConv(nn.Module):
     r"""
 
@@ -74,13 +73,8 @@ class TWIRLSConv(nn.Module):
     >>> feat = th.ones(6, 10)
     >>> conv = TWIRLSConv(10, 2, 128, prop_step = 64)
     >>> res = conv(g , feat)
-    >>> res
-    tensor([[ 0.4556, -2.6692],
-            [ 0.4556, -2.6692],
-            [ 0.4556, -2.6692],
-            [ 1.0112, -5.9241],
-            [ 0.8011, -4.6935],
-            [ 0.8844, -5.1814]], grad_fn=<AddmmBackward>)
+    >>> res.size()
+    torch.Size([6, 2])
     """
 
     def __init__(self,
@@ -148,10 +142,10 @@ def __init__(self,
         self.mlp_bef = MLP(self.input_d, self.hidden_d, self.size_bef_unf, self.num_mlp_before,
                            self.dropout, self.norm, init_activate=False)
 
-        self.unfolding = UnfoldingAndAttention(self.hidden_d, self.alp, self.lam, self.prop_step,
-                                               self.attn_aft, self.tau, self.T, self.p,
-                                               self.use_eta, self.init_att, self.attn_dropout,
-                                               self.precond)
+        self.unfolding = TWIRLSUnfoldingAndAttention(self.hidden_d, self.alp, self.lam,
+                                                     self.prop_step, self.attn_aft, self.tau,
+                                                     self.T, self.p, self.use_eta, self.init_att,
+                                                     self.attn_dropout, self.precond)
 
         # if there are really transformations before unfolding, then do init_activate in mlp_aft
         self.mlp_aft = MLP(self.size_aft_unf, self.hidden_d, self.output_d, self.num_mlp_after,


Test patch:
diff --git a/tests/pytorch/test_nn.py b/tests/pytorch/test_nn.py
--- a/tests/pytorch/test_nn.py
+++ b/tests/pytorch/test_nn.py
@@ -1323,6 +1323,15 @@ def test_edge_predictor(op):
     pred = nn.EdgePredictor(op, in_feats, out_feats, bias=True).to(ctx)
     assert pred(h_src, h_dst).shape == (num_pairs, out_feats)
 
+def test_twirls(): 
+    g = dgl.graph(([0,1,2,3,2,5], [1,2,3,4,0,3]))
+    feat = th.ones(6, 10)
+    conv = nn.TWIRLSConv(10, 2, 128, prop_step = 64)
+    res = conv(g , feat)
+    assert ( res.size() == (6,2) )
+    
+
+
 if __name__ == '__main__':
     test_graph_conv()
     test_graph_conv_e_weight()
@@ -1354,3 +1363,4 @@ def test_edge_predictor(op):
     test_atomic_conv()
     test_cf_conv()
     test_hetero_conv()
+    test_twirls()
\ No newline at end of file
Repository: dmlc/dgl

Issue:
Inconsistent documentation and code of MultiLayerNeighborSampler
## 📚 Documentation

In the docstring of `MultiLayerNeighborSampler`, the default value of parameter `replace` is `True`, while in the code it's `False`:
```python
class MultiLayerNeighborSampler(BlockSampler):
    """
    ...

    replace : bool, default True
        Whether to sample with replacement
    ...
    """
    def __init__(self, fanouts, replace=False, return_eids=False):
        super().__init__(len(fanouts), return_eids)
        self.fanouts = fanouts
        self.replace = replace
        # ...
```

Same issue exists in <https://docs.dgl.ai/api/python/dgl.dataloading.html#dgl.dataloading.neighbor.MultiLayerNeighborSampler>


Discussion:
Hi @ZZy979 , it is indeed a mistake. Would you like to fix it? Thanks in advance.

Resolving patch:
diff --git a/python/dgl/dataloading/neighbor.py b/python/dgl/dataloading/neighbor.py
--- a/python/dgl/dataloading/neighbor.py
+++ b/python/dgl/dataloading/neighbor.py
@@ -71,7 +71,7 @@ class MultiLayerNeighborSampler(NeighborSamplingMixin, BlockSampler):
 
         If -1 is provided for one edge type on one layer, then all inbound edges
         of that edge type will be included.
-    replace : bool, default True
+    replace : bool, default False
         Whether to sample with replacement
     return_eids : bool, default False
         Whether to return the edge IDs involved in message passing in the MFG.


Test patch:
Repository: dmlc/dgl

Issue:
[Bug] FileExistsError when sometimes importing dgl from multiprocess training
## 🐛 Bug
Sometimes, when I launch my Pytorch distributed trainer (which spawns multiple trainer processes, eg once for each GPU for multi-gpu model training), my training job fails with the following error:

```
# pardon the possibly out-of-order stack trace, multiple processes are interleaving the stdout
    import dgl
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/dgl/__init__.py", line 13, in <module>
    from .backend import load_backend, backend_name
  File "/usr/local/lib/python3.7/os.py", line 221, in makedirs
    mkdir(name, mode)
  File "trainer/utils/cli.py", line 137, in <module>
    locals()["run_" + args.which](args, extra)
  File "/usr/local/lib/python3.7/site-packages/dgl/backend/__init__.py", line 107, in <module>
    load_backend(get_preferred_backend())
  File "trainer/utils/cli.py", line 27, in run_local
    trainer_class = locate(args.trainer)
FileExistsError: [Errno 17] File exists: '/root/.dgl'
  File "/usr/local/lib/python3.7/site-packages/dgl/backend/__init__.py", line 103, in get_preferred_backend
    set_default_backend(default_dir, 'pytorch')
FileExistsError: [Errno 17] File exists: '/root/.dgl'
```

I see this occur fairly often, say ~10-20% of the time. Usually, retrying the train command fixes things.

For what it's worth: I am running this within a Docker container, using a DGL nightly build from `2021-10-18`

## To Reproduce

Steps to reproduce the behavior:

I don't have a repro script. But, hopefully this stack trace can point out a diagnosis + fix.

## Expected behavior

Importing dgl shouldn't cause an error.

## Environment

 - DGL Version (e.g., 1.0): >0.7 (Nightly build from 2021-10-18).
 - Backend Library & Version (e.g., PyTorch 0.4.1, MXNet/Gluon 1.3):
 - OS (e.g., Linux): Linux
 - How you installed DGL (`conda`, `pip`, source): From nightly
 - Build command you used (if compiling from source):
 - Python version: 3.7
 - CUDA/cuDNN version (if applicable):
 - GPU models and configuration (e.g. V100):
 - Any other relevant information:

## Additional context



Discussion:
hi,

sorry for the inconvenience. You can try executing import dgl seperately before everything. Such a s `python -c 'import dgl'`. The reason is that it'll create config file. We'll consider adding Filelock for this in the future

Resolving patch:
diff --git a/python/dgl/backend/set_default_backend.py b/python/dgl/backend/set_default_backend.py
--- a/python/dgl/backend/set_default_backend.py
+++ b/python/dgl/backend/set_default_backend.py
@@ -3,8 +3,8 @@
 import json
 
 def set_default_backend(default_dir, backend_name):
-    if not os.path.exists(default_dir):
-        os.makedirs(default_dir)
+    # the exists_ok requires python >= 3.2
+    os.makedirs(default_dir, exists_ok=True)
     config_path = os.path.join(default_dir, 'config.json')
     with open(config_path, "w") as config_file: 
         json.dump({'backend': backend_name.lower()}, config_file)


Test patch:
Repository: dmlc/dgl

Issue:
bugs on docs of compact_graphs
i follow examples in [docs](https://docs.dgl.ai/generated/dgl.compact_graphs.html?highlight=compact_graphs#dgl.compact_graphs) of dgl,but error occurs

```
g = dgl.heterograph({('user', 'plays', 'game'): ([1, 3], [3, 5])},
                     {'user': 20, 'game': 10})
new_g, induced_nodes = dgl.compact_graphs(g)
induced_nodes
```



> 
> ---------------------------------------------------------------------------
> DGLError                                  Traceback (most recent call last)
> <ipython-input-15-c4cdc2d81d98> in <module>
> ----> 1 new_g, induced_nodes = dgl.compact_graphs(g)
>       2 induced_nodes
> 
> ~/software/anaconda3/lib/python3.8/site-packages/dgl/heterograph.py in __getitem__(self, key)
>    2229 
>    2230         if len(etypes) == 0:
> -> 2231             raise DGLError('Invalid key "{}". Must be one of the edge types.'.format(orig_key))
>    2232 
>    2233         if len(etypes) == 1:
> 
> DGLError: Invalid key "0". Must be one of the edge types.




Discussion:

Resolving patch:
diff --git a/python/dgl/transforms/functional.py b/python/dgl/transforms/functional.py
--- a/python/dgl/transforms/functional.py
+++ b/python/dgl/transforms/functional.py
@@ -1994,8 +1994,8 @@ def compact_graphs(graphs, always_preserve=None, copy_ndata=True, copy_edata=Tru
     The following would compact the graph above to another bipartite graph with only
     two users and two games.
 
-    >>> new_g, induced_nodes = dgl.compact_graphs(g)
-    >>> induced_nodes
+    >>> new_g = dgl.compact_graphs(g)
+    >>> new_g.ndata[dgl.NID]
     {'user': tensor([1, 3]), 'game': tensor([3, 5])}
 
     The mapping tells us that only user #1 and #3 as well as game #3 and #5 are kept.
@@ -2013,8 +2013,8 @@ def compact_graphs(graphs, always_preserve=None, copy_ndata=True, copy_edata=Tru
 
     >>> g2 = dgl.heterograph({('user', 'plays', 'game'): ([1, 6], [6, 8])},
     >>>                      {'user': 20, 'game': 10})
-    >>> (new_g, new_g2), induced_nodes = dgl.compact_graphs([g, g2])
-    >>> induced_nodes
+    >>> new_g, new_g2 = dgl.compact_graphs([g, g2])
+    >>> new_g.ndata[dgl.NID]
     {'user': tensor([1, 3, 6]), 'game': tensor([3, 5, 6, 8])}
 
     Then one can see that user #1 from both graphs, users #3 from the first graph, as


Test patch:
Repository: dmlc/dgl

Issue:
Running error on examples/pytorch/gat/train.py 
## 🐛 Bug

When running the cpu version of gat, an error is reported, indicating that cuda is not supported.

## To Reproduce

Steps to reproduce the behavior:

python train.py --dataset=cora --gpu=-1

Traceback (most recent call last):
  File "train.py", line 181, in <module>
    main(args)
  File "train.py", line 110, in main
    torch.cuda.synchronize()
  File "/opt/homebrew/Caskroom/miniconda/base/envs/dgl/lib/python3.8/site-packages/torch/cuda/__init__.py", line 493, in synchronize
    _lazy_init()
  File "/opt/homebrew/Caskroom/miniconda/base/envs/dgl/lib/python3.8/site-packages/torch/cuda/__init__.py", line 210, in _lazy_init
    raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled


## Expected behavior

The example should run successful.

## Environment

 - DGL Version (e.g., 1.0): 0.8.1
 - Backend Library & Version (e.g., PyTorch 0.4.1, MXNet/Gluon 1.3):1.11.0
 - OS (e.g., Linux): macOS 12.3.1 
 - How you installed DGL (`conda`, `pip`, source): conda
 - Build command you used (if compiling from source): 
 - Python version: 3.8.13
 - CUDA/cuDNN version (if applicable):
 - GPU models and configuration (e.g. V100):
 - Any other relevant information:

## Additional context

<!-- Add any other context about the problem here. -->

It seems that the conditional judgment of cuda is missing here (line 110 and 121).

https://github.com/dmlc/dgl/blob/4b87e47f15552cb27ddf8e594769ad0361d06627/examples/pytorch/gat/train.py#L109-L111

https://github.com/dmlc/dgl/blob/4b87e47f15552cb27ddf8e594769ad0361d06627/examples/pytorch/gat/train.py#L120-L122



Discussion:

Resolving patch:
diff --git a/examples/pytorch/gat/train.py b/examples/pytorch/gat/train.py
--- a/examples/pytorch/gat/train.py
+++ b/examples/pytorch/gat/train.py
@@ -107,7 +107,8 @@ def main(args):
     for epoch in range(args.epochs):
         model.train()
         if epoch >= 3:
-            torch.cuda.synchronize()
+            if cuda:
+                torch.cuda.synchronize()
             t0 = time.time()
         # forward
         logits = model(features)
@@ -118,7 +119,8 @@ def main(args):
         optimizer.step()
 
         if epoch >= 3:
-            torch.cuda.synchronize()
+            if cuda:
+                torch.cuda.synchronize()
             dur.append(time.time() - t0)
 
         train_acc = accuracy(logits[train_mask], labels[train_mask])


Test patch: