Repository: pytorch
Issue: pin_memory *still* destroys custom containers 
Labels: module: dataloader,triaged,Stale

### 🐛 Describe the bug

pin_memory=True on data loading destroys custom container classes inheriting from list. Although addressed in #64779, it still breaks custom data storage.

Example.

```python
from typing import Any
import torch

class Item(list):
    def __init__(self, *args, **kwargs):
        super().__init__(*args)
        self.__dict__.update(**kwargs)
    def __setattr__(self, name: str, value: Any) -> None:
        return super().__setattr__(name, value)

z = torch.zeros(23)
o = torch.ones(23)
I = Item([z,o], issue="bug", metadata=["zeros", "ones"])

out = type(I)(x.pin_memory() for x in I) #this is more or less what #64779 does
print(I.issue)
# ok -> "bug"
print(out.issue)
# AttributeError: 'Item' object has no attribute 'issue'
```

The fix is rather simple - insofar as pin memory goes. On the first conditional pin_memory converts tensors; on the last condition it converts classes with "pin_memory" attr; since tensors have this attr, that could be the first line, and all I would have to do would be to implement a custom pin memory for my class. 

But perhaps stronger code would replace elements instead of casting return (as done on #64779) - below. both modifications.

```python
def pin_memory(data):
    if hasattr(data, "pin_memory"): # remove redundant torch.is_tensor, tensors have attr pin_memory
        return data.pin_memory()
    elif isinstance(data, string_classes):
        return data
    elif isinstance(data, collections.abc.Mapping):
        return {k: pin_memory(sample) for k, sample in data.items()}
    elif isinstance(data, tuple) and hasattr(data, '_fields'):  # namedtuple
        return type(data)(*(pin_memory(sample) for sample in data))
    elif isinstance(data, collections.abc.Sequence): # non destructive, instead of casting, replace
        for i, sample in enumerate(data):
            data[i] = pin_memory(sample) 
    return data
```
default_collate would also require similar treatment, however it can be simply replaced with a custom collate_fn.



### Versions

* this should also fail on latest version, yet I havent checked it. 

Collecting environment information...
PyTorch version: 1.9.0
Is debug build: No
CUDA used to build PyTorch: 11.1

OS: Ubuntu 18.04.5 LTS
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
CMake version: version 3.10.2

Python version: 3.8
Is CUDA available: Yes
CUDA runtime version: Could not collect
GPU models and configuration: GPU 0: GeForce GTX 1070
Nvidia driver version: 460.91.03
cuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5

Versions of relevant libraries:
[pip3] efficientnet-pytorch==0.6.3
[pip3] numpy==1.21.1
[pip3] pytorch-fid==0.2.0
[pip3] pytorch-lightning==1.4.1
[pip3] segmentation-models-pytorch==0.2.0
[pip3] streamer-pytorch==0.0.2
[pip3] torch==1.9.0
[pip3] torch-ema==0.2
[pip3] torch-fidelity==0.3.0
[pip3] torchaudio==0.9.0a0+33b2469
[pip3] torchmetrics==0.4.1
[pip3] torchvision==0.10.0
[conda] blas                      2.109                       mkl    conda-forge
[conda] blas-devel                3.9.0                     9_mkl    conda-forge
[conda] efficientnet-pytorch      0.6.3                    pypi_0    pypi
[conda] ffmpeg                    4.3                  hf484d3e_0    pytorch
[conda] libblas                   3.9.0                     9_mkl    conda-forge
[conda] libcblas                  3.9.0                     9_mkl    conda-forge
[conda] liblapack                 3.9.0                     9_mkl    conda-forge
[conda] liblapacke                3.9.0                     9_mkl    conda-forge
[conda] mkl                       2021.3.0           h726a3e6_557    conda-forge
[conda] mkl-devel                 2021.3.0           ha770c72_558    conda-forge
[conda] mkl-include               2021.3.0           h726a3e6_557    conda-forge
[conda] pytorch                   1.9.0           py3.8_cuda11.1_cudnn8.0.5_0    pytorch
[conda] pytorch-fid               0.2.0                    pypi_0    pypi
[conda] pytorch-lightning         1.4.1                    pypi_0    pypi
[conda] segmentation-models-pytorch 0.2.0                    pypi_0    pypi
[conda] streamer-pytorch          0.0.2                    pypi_0    pypi
[conda] torch-ema                 0.2                      pypi_0    pypi
[conda] torch-fidelity            0.3.0                    pypi_0    pypi
[conda] torchaudio                0.9.0                      py38    pytorch
[conda] torchmetrics              0.4.1                    pypi_0    pypi
[conda] torchvision               0.10.0                   pypi_0    pypi

cc @SsnL @VitalyFedyunin @ejguan @NivekT @jamesr66a @ezyang