Repository: pytorch
Issue: torch.compile not compatible with multiprocessing pool
Labels: triaged,bug,oncall: pt2,module: inductor

### 🐛 Describe the bug

Attempting to call `torch.compile()` within a multiprocessing pool results in:

```sh
torch._dynamo.exc.BackendCompilerFailed: debug_wrapper raised AssertionError: daemonic processes are not allowed to have children
```

I am using `transformers` `ViTModel` to encode images and cache them to disk inside my DataLoader transform. To improve performance, I was hoping to use `torch.compile()` since I perform this pre-processing transform on CPU. However, compiling in my transform's `__init__` runs into pickling issues when the DataLoader forks the workers. As for compiling after the workers have forked, I run into the issue above.

### Error logs

```sh
RuntimeError: Caught BackendCompilerFailed in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/jovyan/.conda/envs/videocv/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 670, in call_user_compiler
    compiled_fn = compiler_fn(gm, self.fake_example_inputs())
  File "/home/jovyan/.conda/envs/videocv/lib/python3.10/site-packages/torch/_dynamo/debug_utils.py", line 1055, in debug_wrapper
    compiled_gm = compiler_fn(gm, example_inputs)
  File "/home/jovyan/.conda/envs/videocv/lib/python3.10/site-packages/torch/__init__.py", line 1388, in __call__
    from torch._inductor.compile_fx import compile_fx
  File "/home/jovyan/.conda/envs/videocv/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 21, in <module>
    from . import config, metrics, overrides, pattern_matcher
  File "/home/jovyan/.conda/envs/videocv/lib/python3.10/site-packages/torch/_inductor/pattern_matcher.py", line 19, in <module>
    from .lowering import lowerings as L
  File "/home/jovyan/.conda/envs/videocv/lib/python3.10/site-packages/torch/_inductor/lowering.py", line 3868, in <module>
    import_submodule(kernel)
  File "/home/jovyan/.conda/envs/videocv/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1304, in import_submodule
    importlib.import_module(f"{mod.__name__}.{filename[:-3]}")
  File "/home/jovyan/.conda/envs/videocv/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/home/jovyan/.conda/envs/videocv/lib/python3.10/site-packages/torch/_inductor/kernel/bmm.py", line 4, in <module>
    from ..select_algorithm import (
  File "/home/jovyan/.conda/envs/videocv/lib/python3.10/site-packages/torch/_inductor/select_algorithm.py", line 20, in <module>
    from .codecache import code_hash, DiskCache, PyCodeCache
  File "/home/jovyan/.conda/envs/videocv/lib/python3.10/site-packages/torch/_inductor/codecache.py", line 721, in <module>
    AsyncCompile.warm_pool()
  File "/home/jovyan/.conda/envs/videocv/lib/python3.10/site-packages/torch/_inductor/codecache.py", line 660, in warm_pool
    pool._adjust_process_count()
  File "/home/jovyan/.conda/envs/videocv/lib/python3.10/concurrent/futures/process.py", line 692, in _adjust_process_count
    self._spawn_process()
  File "/home/jovyan/.conda/envs/videocv/lib/python3.10/concurrent/futures/process.py", line 709, in _spawn_process
    p.start()
  File "/home/jovyan/.conda/envs/videocv/lib/python3.10/multiprocessing/process.py", line 118, in start
    assert not _current_process._config.get('daemon'), \
AssertionError: daemonic processes are not allowed to have children
```

### Minified repro

_No response_

### Versions

```sh
PyTorch version: 2.0.0+cu117
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.6 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: Could not collect
CMake version: version 3.26.0
Libc version: glibc-2.31

Python version: 3.10.9 | packaged by conda-forge | (main, Feb  2 2023, 20:20:04) [GCC 11.3.0] (64-bit runtime)
Python platform: Linux-5.4.0-121-generic-x86_64-with-glibc2.31
Is CUDA available: True
CUDA runtime version: 11.6.112
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA A40
Nvidia driver version: 470.57.02
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.3.3
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.3.3
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.3.3
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.3.3
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.3.3
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.3.3
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.3.3
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU: removed

Versions of relevant libraries:
[pip3] mypy==1.0.1
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.24.2
[pip3] pytorch-optimizer==2.5.0
[pip3] pytorch-triton==2.0.0+b8b470bc59
[pip3] torch==2.0.0
[pip3] torchinfo==1.7.2
[pip3] torchvision==0.15.1
[pip3] triton==2.0.0
[conda] numpy                     1.24.2                   pypi_0    pypi
[conda] pytorch-optimizer         2.5.0                    pypi_0    pypi
[conda] pytorch-triton            2.0.0+b8b470bc59          pypi_0    pypi
[conda] torch                     2.0.0                    pypi_0    pypi
[conda] torchinfo                 1.7.2                    pypi_0    pypi
[conda] torchvision               0.15.1                   pypi_0    pypi
[conda] triton                    2.0.0                    pypi_0    pypi
```

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305 @zou3519 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler @soumith @ngimel @Xia-Weiwen @desertfire