Parameter ... has been marked as ready twice.
Author: levuloihust99
Pull request: no

### System Info

* Ubuntu 20.04 Desktop
* Architecture x86_64
* Python 3.8.10
* torch 1.12.1+cu116
* GPU P100 x 3

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I have the following code in file named `debug.py`

```python
import os
import torch
import argparse


class CustomModel(torch.nn.Module):
    def __init__(self):
        super(CustomModel, self).__init__()
        self.w1 = torch.nn.Parameter(torch.tensor(1.5), requires_grad=True)
        self.w2 = torch.nn.Parameter(torch.tensor(1.5), requires_grad=True)
        self.w3 = torch.nn.Parameter(torch.tensor(1.5), requires_grad=True)
    
    def forward(self, x, order):
        if order == 0:
            return self.w1 * x
        elif order == 1:
            return self.w2 * x
        else:
            return self.w3 * x


def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("--local_rank", type=int, default=-1)
    args = parser.parse_args()

    local_rank = args.local_rank
    torch.cuda.set_device(local_rank)
    device = torch.device("cuda", local_rank)
    torch.distributed.init_process_group(backend="nccl")
    
    model = CustomModel()
    model.to(device)

    # setup distributed
    model = torch.nn.parallel.DistributedDataParallel(model, device_ids=[local_rank],
                                                          output_device=local_rank,
                                                          find_unused_parameters=True)

    x = torch.tensor(1.)
    y1 = model(x, 0)
    y2 = model(x, 1)
    y3 = model(x, 2)
    y1.backward()


if __name__ == "__main__":
    main()
```

I run this code with the following command 
```shell
TORCH_DISTRIBUTED_DEBUG=DETAIL python -m torch.distributed.launch --nproc_per_node=3 debug.py
```
Then, I got the error
<pre>
Expected to mark a variable ready only once. This error is caused by one of the following reasons: 1) Use of a module parameter outside the `forward` function. Please make sure model parameters are not shared across multiple concurrent forward-backward passes. or try to use _set_static_graph() as a workaround if this module graph does not change during training loop.2) Reused parameters in multiple reentrant backward passes. For example, if you use multiple `checkpoint` functions to wrap the same part of your model, it would result in the same set of parameters been used by different reentrant backward passes multiple times, and hence marking a variable ready multiple times. DDP does not support such use cases in default. You can try to use _set_static_graph() as a workaround if your module graph does not change over iterations.
Parameter at index 0 with name .w1 has been marked as ready twice. This means that multiple autograd engine  hooks have fired for this particular parameter during this iteration.
</pre>

If I change the line `y1.backward()` to `y2.backward()`, the parameter that `has been marked as ready twice` change to `.w2`

As the error suggested, there might have shared parameters across multiple forward-backward pass or a same set of parameters used by multiple backward passes. However, I find none of these two suggestions match the above provided code.

The error disappeared if I set `find_unused_parameters=False`. However, in my actual code, this setting caused another error, which was `Expected to have finished reduction in the prior iteration before starting a new one. This error indicates that your module has parameters that were not used in producing loss. You can enable unused parameter detection by passing the keyword argument find_unused_parameters=True to torch.nn.parallel.DistributedDataParallel...`.

I am able to change my code to fix the problem. But the focus of my question is why such a simple code above produced the error? Why was there a parameter that `has been marked as ready twice`?


### Expected behavior

The code run without errors

Comments: