Repository: pytorch
Issue: Keys of a `ModuleDict` cannot have the same name as existing `ModuleDict` class attributes.
Labels: module: nn,triaged

### 🐛 Describe the bug

The keys of a `ModuleDict` cannot have the same name as existing `ModuleDict` class attributes:

```python
import torch

torch.nn.ModuleDict({'type': torch.nn.Module()})    # Raises KeyError "attribute 'type' already exists"
torch.nn.ModuleDict({'to': torch.nn.Module()})    # Raises KeyError "attribute 'to' already exists"
```

A full list of similarly illegal keys is enumerated by `dir(torch.nn.ModuleDict())`:
```
['T_destination', '__annotations__', '__call__', '__class__', '__contains__', '__delattr__', '__delitem__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattr__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__len__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setitem__', '__setstate__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_apply', '_backward_hooks', '_buffers', '_call_impl', '_forward_hooks', '_forward_pre_hooks', '_get_backward_hooks', '_get_name', '_is_full_backward_hook', '_load_from_state_dict', '_load_state_dict_pre_hooks', '_maybe_warn_non_full_backward_hook', '_modules', '_named_members', '_non_persistent_buffers_set', '_parameters', '_register_load_state_dict_pre_hook', '_register_state_dict_hook', '_replicate_for_data_parallel', '_save_to_state_dict', '_slow_forward', '_state_dict_hooks', '_version', 'add_module', 'apply', 'bfloat16', 'buffers', 'children', 'clear', 'cpu', 'cuda', 'double', 'dump_patches', 'eval', 'extra_repr', 'float', 'forward', 'get_buffer', 'get_extra_state', 'get_parameter', 'get_submodule', 'half', 'items', 'keys', 'load_state_dict', 'modules', 'named_buffers', 'named_children', 'named_modules', 'named_parameters', 'parameters', 'pop', 'register_backward_hook', 'register_buffer', 'register_forward_hook', 'register_forward_pre_hook', 'register_full_backward_hook', 'register_parameter', 'requires_grad_', 'set_extra_state', 'share_memory', 'state_dict', 'to', 'to_empty', 'train', 'training', 'type', 'update', 'values', 'xpu', 'zero_grad']
```

This poses a problem for applications like [Ludwig](https://github.com/ludwig-ai/ludwig) where the keys of a `ModuleDict` can be user-defined. Forcing users to rename the keys of their modules just to avoid conflicting with torch’s `ModuleDict` class attributes is an awkward restriction.

To mitigate name conflicts, we've created a `ModuleDict` wrapper that appends a suffix to incoming keys (i.e. `____`) before adding to or fetching from a `ModuleDict`.

This is certainly more overhead, but it also nicely guarantees that the `ModuleDict` can be made to work with any user-defined keys.

Our solution, in case it's useful to reference:
```python
import torch


SUFFIX = "____"
SUFFIX_LENGTH = len(SUFFIX)


class RobustModuleDict(torch.nn.Module):
    """Torch ModuleDict wrapper that permits keys with any name.

    Torch's ModuleDict doesn't allow certain keys to be used if they
    conflict with existing class attributes, e.g.

    > torch.nn.ModuleDict({'type': torch.nn.Module()})  # Raises KeyError.

    This class is a simple wrapper around torch's ModuleDict that
    mitigates possible conflicts by using a key-suffixing protocol.
    """

    def __init__(self):
        super().__init__()
        self.module_dict = torch.nn.ModuleDict()

    def __getitem__(self, key) -> torch.nn.Module:
        return self.module_dict[key + SUFFIX]

    def __setitem__(self, key: str, module: torch.nn.Module) -> None:
        self.module_dict[key + SUFFIX] = module

    def __len__(self) -> int:
        return len(self.module_dict)

    def keys(self) -> List[str]:
        return [key[:-SUFFIX_LENGTH] for key in self.module_dict.keys()]

    def values(self) -> List[torch.nn.Module]:
        return [module for _, module in self.module_dict.items()]

    def items(self) -> List[Tuple[str, torch.nn.Module]]:
        return [
            (key[:-SUFFIX_LENGTH], 
             module) for key, module in self.module_dict.items()
        ]

    def update(self, modules: Dict[str, torch.nn.Module]) -> None:
        for key, module in modules.items():
            self[key] = module

    def __next__(self) -> None:
        return next(iter(self))

    def __iter__(self) -> None:
        return iter(self.keys())
```

Example usage:

```python
    robust_dict = RobustModuleDict()

    to_module = torch.nn.Module()
    type_module = torch.nn.Module()

    robust_dict["to"] = to_module
    robust_dict["type"] = type_module

    assert len(robust_dict) == 2
    assert robust_dict.keys() == ["to", "type"]
    assert robust_dict.items() == [("to", to_module), ("type", type_module)]
    assert robust_dict["to"] == to_module

    robust_dict.update({"1": torch.nn.Module()})

    assert len(robust_dict) == 3
    assert [key for key in robust_dict] == ["to", "type", "1"]
```

### Versions

PyTorch version: 1.10.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 12.1 (arm64)
GCC version: Could not collect
Clang version: 13.0.0 (clang-1300.0.29.30)
CMake version: version 3.22.0
Libc version: N/A

Python version: 3.8.12 | packaged by conda-forge | (default, Oct 12 2021, 21:21:17)  [Clang 11.1.0 ] (64-bit runtime)
Python platform: macOS-12.1-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.21.4
[pip3] pytorch-lightning==1.5.6
[pip3] torch==1.10.0
[pip3] torchaudio==0.10.1
[pip3] torchinfo==1.5.4
[pip3] torchmetrics==0.6.1
[pip3] torchvision==0.11.2
[conda] mypy_extensions           0.4.3            py38h10201cd_4    conda-forge
[conda] numpy                     1.21.4           py38hbf7bb01_0    conda-forge
[conda] pytorch                   1.10.0          cpu_py38ha3d347f_0    conda-forge
[conda] pytorch-cpu               1.10.0          cpu_py38hd610c6a_0    conda-forge
[conda] pytorch-lightning         1.5.6                    pypi_0    pypi
[conda] torchaudio                0.10.1                   pypi_0    pypi
[conda] torchinfo                 1.5.4              pyhd8ed1ab_0    conda-forge
[conda] torchmetrics              0.6.1              pyhd8ed1ab_0    conda-forge
[conda] torchvision               0.11.2                   pypi_0    pypi

cc @albanD @mruberry @jbschlosser @walterddr @kshitij12345