Repository: dmlc/dgl
Pull request: [Example][Bugfix] Fix arma example
## Description
To fix #4202, the crash due to in-place operations for one variable needed for backward propagation.

## Checklist
Please feel free to remove inapplicable items for your PR.
- [x] The PR title starts with [$CATEGORY] (such as [NN], [Model], [Doc], [Feature]])
- [x] Changes are complete (i.e. I finished coding on this PR)
- [x] All changes have test coverage
- [x] Code is well-documented
- [x] To the best of my knowledge, examples are either not affected by this change,
      or have been fixed to be compatible with this change
- [x] Related issue is referred in this PR
- [x] If the PR is for a new model/paper, I've updated the example index [here](../examples/README.md).

## Changes
At the end of forward function, to compute average output over all stacks, instead of doing `+=` (in-place operations) as:
`output += features`
using `unsqueeze() + cat()` to store all stack outputs and return the mean of them.

## Additional notes

With profiling, no significant performance penalty observed with this fix (less than 10% slow-down)

Problem statement:
[Example][Bug] Running error on the example case: example/pytorch/arma
## 🐛 Bug

Example of using DGL to implement GNN with convolutional ARMA filters failed.

## To Reproduce

`python citation.py --gpu 0`

<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->
```
Traceback (most recent call last):
  File "citation.py", line 142, in <module>
    acc_lists.append(main(args))
  File "citation.py", line 84, in main
    train_loss.backward()
  File "/opt/conda/lib/python3.8/site-packages/torch/_tensor.py", line 396, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/opt/conda/lib/python3.8/site-packages/torch/autograd/__init__.py", line 173, in backward
    Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [2708, 7]], which is output 0 of ReluBackward0, is at version 1; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).
```

`python citation.py --gpu 0 --dataset Citeseer --num-stacks 3`

```
Traceback (most recent call last):
  File "citation.py", line 142, in <module>
    acc_lists.append(main(args))
  File "citation.py", line 84, in main
    train_loss.backward()
  File "/opt/conda/lib/python3.8/site-packages/torch/_tensor.py", line 396, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/opt/conda/lib/python3.8/site-packages/torch/autograd/__init__.py", line 173, in backward
    Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [3327, 6]], which is output 0 of ReluBackward0, is at version 2; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

```

## Expected behavior

The case should run through

## Environment

 - DGL Version (e.g., 1.0): 0.9
 - Backend Library & Version (e.g., PyTorch 0.4.1, MXNet/Gluon 1.3): 1.12
 - OS (e.g., Linux): ubuntu
 - How you installed DGL (`conda`, `pip`, source): source
 - Build command you used (if compiling from source):
 - Python version: 3.8
 - CUDA/cuDNN version (if applicable): 11.7
 - GPU models and configuration (e.g. V100): A100
 - Any other relevant information:

## Additional context

<!-- Add any other context about the problem here. -->

Hints from issue discussion:

Commit message: Fix arma example

Patch to review:
diff --git a/examples/pytorch/arma/model.py b/examples/pytorch/arma/model.py
index b3fd5a04fabd..ef6ab5bf534b 100644
--- a/examples/pytorch/arma/model.py
+++ b/examples/pytorch/arma/model.py
@@ -65,6 +65,7 @@ def forward(self, g, feats):
             degs = g.in_degrees().float().clamp(min=1)
             norm = torch.pow(degs, -0.5).to(feats.device).unsqueeze(1)
             output = None
+            tot_output = None
 
             for k in range(self.K):
                 feats = init_feats
@@ -88,13 +89,14 @@ def forward(self, g, feats):
                     
                     if self.activation is not None:
                         feats = self.activation(feats)
-                    
-                if output is None:
-                    output = feats
+
+                output = feats.unsqueeze(-3)
+                if k == 0:
+                    tot_output = output
                 else:
-                    output += feats
-                
-            return output / self.K 
+                    tot_output = torch.cat((tot_output, output), 0)
+
+            return tot_output.mean(dim=-3) 
 
 class ARMA4NC(nn.Module):
     def __init__(self,


Code review comments:
diff_hunk: @@ -88,13 +89,14 @@ def forward(self, g, feats):
                     
                     if self.activation is not None:
                         feats = self.activation(feats)
-                    
-                if output is None:
-                    output = feats
+
+                output = feats.unsqueeze(-3)
+                if k == 0:
+                    tot_output = output
                 else:
-                    output += feats
-                
-            return output / self.K 
+                    tot_output = torch.cat((tot_output, output), 0)
path: examples/pytorch/arma/model.py
text: @user1:
I'd prefer defining `tot_output` as a list to store `output`s and then calculating their mean by `torch.stack(...).mean()`.

@author:
Okay, I will update accordingly.