Repository: pytorch
Issue: torch.is_grad_enabled() is False when using custom_op decorator
Labels: module: docs,triaged,module: custom-operators,oncall: pt2

### 🐛 Describe the bug

I am trying to make a C++ operation compilable by defining a custom OP for my Python wrapper of the C++ functions.
Inside the custom op, I would like to know if gradients are currently enabled or not - because I have to create temporary data in the forward pass that is needed in the backward pass. 

But when I use torch.library.custom_op decorator, `torch.is_grad_enabled()` always returns False inside the decorated function. 
It also remains false when I define a backward pass with `register_autograd`.

This is at least unexpected behaviour, but in my opinion also likely a bug? Is there a way around this?
Currently using Pytorch 2.5 nightly from 20240731. For a minimal example, you can take the [example code from PyTorch](https://pytorch.org/tutorials/advanced/python_custom_ops.html).

Unrelated problem:
When I return a strided tensor (which is created in the C++ op), the backward pass does not work, because it expects a non-strided gradient. Even when using the same strides in `register_fake`, it does not fix the problem. I always have to make the tensor contiguous before returning.

### Versions

```
Collecting environment information...
PyTorch version: 2.5.0.dev20240731
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.6 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
Clang version: Could not collect
CMake version: version 3.29.3
Libc version: glibc-2.31

Python version: 3.10.0 | packaged by conda-forge | (default, Nov 20 2021, 02:24:10) [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.15.0-1062-aws-x86_64-with-glibc2.31
Is CUDA available: True
CUDA runtime version: 12.1.105
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: 
GPU 0: NVIDIA H100 80GB HBM3
GPU 1: NVIDIA H100 80GB HBM3
GPU 2: NVIDIA H100 80GB HBM3
GPU 3: NVIDIA H100 80GB HBM3
GPU 4: NVIDIA H100 80GB HBM3
GPU 5: NVIDIA H100 80GB HBM3
GPU 6: NVIDIA H100 80GB HBM3
GPU 7: NVIDIA H100 80GB HBM3

Nvidia driver version: 535.183.01

Versions of relevant libraries:
[pip3] numpy==1.26.4
[pip3] torch==2.5.0.dev20240731
[pip3] torchao==0.6.0.dev20240916+cu121
[pip3] torchaudio==2.4.0.dev20240731
[pip3] torcheval==0.0.7
[pip3] torchtune==0.2.1
[pip3] torchvision==0.20.0.dev20240731
[pip3] triton==3.0.0
[conda] blas                      2.116                       mkl    conda-forge
[conda] blas-devel                3.9.0            16_linux64_mkl    conda-forge
[conda] libblas                   3.9.0            16_linux64_mkl    conda-forge
[conda] libcblas                  3.9.0            16_linux64_mkl    conda-forge
[conda] liblapack                 3.9.0            16_linux64_mkl    conda-forge
[conda] liblapacke                3.9.0            16_linux64_mkl    conda-forge
[conda] mkl                       2022.1.0           h84fe81f_915    https://aws-ml-conda.s3.us-west-2.amazonaws.com
[conda] mkl-devel                 2022.1.0           ha770c72_916    conda-forge
[conda] mkl-include               2022.1.0           h84fe81f_915    conda-forge
[conda] numpy                     1.26.4          py310hb13e2d6_0    conda-forge
[conda] pytorch                   2.5.0.dev20240731 py3.10_cuda12.1_cudnn9.1.0_0    pytorch-nightly
[conda] pytorch-cuda              12.1                 ha16c6d3_6    pytorch-nightly
[conda] pytorch-mutex             1.0                        cuda    pytorch-nightly
[conda] torchao                   0.6.0.dev20240916+cu121          pypi_0    pypi
[conda] torchaudio                2.4.0.dev20240731     py310_cu121    pytorch-nightly
[conda] torcheval                 0.0.7                    pypi_0    pypi
[conda] torchtriton               3.0.0+dedb7bdf33           py310    pytorch-nightly
[conda] torchtune                 0.2.1                    pypi_0    pypi
[conda] torchvision               0.20.0.dev20240731     py310_cu121    pytorch-nightly

```

cc @svekars @brycebortree @sekyondaMeta @ezyang @chauhang @penguinwu @zou3519 @bdhirsh @yf225