Repository: pytorch
Issue: Using operator[] in GenericPackedTensorAccessor impossible in cpu code in .cu files (tensor on cpu)
Labels: module: build,triaged

## 🐛 Bug
I want to use the same `__host__ __device__` function for computation on the cpu and gpu, therefore i put it into a ".cu" file. It accesses data using the accessor objects retrieved via `tensor.packed_accessor32<...>(...)`. I later I create a `__host__ __device__` lambda and dispatch it to either the cpu or the gpu, depending on the tensor's device. This way I can use the host debugger and other tools for development (performance in cpu mode is not critical for me).

The problem is, that
`C10_DEVICE const TensorAccessor<..> operator[](index_t i)` is marked with `C10_DEVICE` and not `C10_HOST_DEVICE`.
`C10_DEVICE` is something like
```
#ifdef __CUDACC__
#define C10_DEVICE __device__
#else
#define C10_DEVICE
#endif 
```
`__CUDACC__` will be defined by nvcc for `.cu` files only. Therefore the operator can be called from cpu code if it is in `.cpp` files, but not if the cpu code is located in `.cu` files.

I understand that this was probably intended as a safeguard, but the effect is not very safe. There is no error or warning, the process simply exits when calling `operator[]`.

One solution would be to replace these `C10_DEVICE` with `C10_HOST_DEVICE`.

## To Reproduce

Steps to reproduce the behavior:

1. Create a CPU tensor in a `.cu` file
2. Get an accessor to it.
3. call `accessor[0]`

## Expected behavior

`accessor[0]` should work the same as in a .cpp file

## Environment
```
Collecting environment information...
PyTorch version: 1.6.0
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.1 LTS (x86_64)
GCC version: (Ubuntu 9.3.0-10ubuntu2) 9.3.0
Clang version: 10.0.0-4ubuntu1 
CMake version: version 3.18.2

Python version: 3.7 (64-bit runtime)
Is CUDA available: True
CUDA runtime version: 10.1.243
GPU models and configuration: GPU 0: GeForce GTX 1080
Nvidia driver version: 450.66
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.18.5
[pip3] numpydoc==1.1.0
[pip3] torch==1.6.0
[pip3] torchvision==0.7.0
[conda] blas                      1.0                         mkl  
[conda] cudatoolkit               10.2.89              hfd86e86_1  
[conda] mkl                       2020.2                      256  
[conda] mkl-service               2.3.0            py37he904b0f_0  
[conda] mkl_fft                   1.1.0            py37h23d657b_0  
[conda] mkl_random                1.1.1            py37h0573a6f_0  
[conda] numpy                     1.18.5           py37ha1c710e_0  
[conda] numpy-base                1.18.5           py37hde5b4d6_0  
[conda] numpy-devel               1.18.5           py37h4ba5984_0  
[conda] numpydoc                  1.1.0                      py_0  
[conda] pytorch                   1.6.0           py3.7_cuda10.2.89_cudnn7.6.5_0    pytorch
[conda] torchvision               0.7.0                py37_cu102    pytorch

```
## Additional information
The relevant header is located in `torch/include/ATen/core/TensorAccessor.h` in my local installation.

It might be a good idea to replace C10_DEVICE by C10_HOST_DEVICE in some other files as well, for instance
`ATen/native/Distributions.h`.

cc @malfet @seemethere @walterddr