Repository: pytorch
Issue: torch.jit.script does not work with DataParallel
Labels: oncall: jit

If you have a question or would like help and support, please ask at our
[forums](https://discuss.pytorch.org/).

If you are submitting a feature request, please preface the title with [feature request].
If you are submitting a bug report, please fill in the following details.

## Issue description

I was trying to generate a ScriptModule for 3DMPPE to deploy those models on torchserve. However, when I tried to run `torch.jit.script(DataParallel(PoseNet()))`, I got the following error:

```
Traceback (most recent call last):
  File "/Users/yeonwoosung/Desktop/musculoskeletal-checker/ai_engine/rootnet_trace.py", line 55, in <module>
    script_module = torch.jit.script(rootnet_pose_model)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/torch/jit/_script.py", line 1286, in script
    return torch.jit._recursive.create_script_module(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/torch/jit/_recursive.py", line 476, in create_script_module
    return create_script_module_impl(nn_module, concrete_type, stubs_fn)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/torch/jit/_recursive.py", line 488, in create_script_module_impl
    method_stubs = stubs_fn(nn_module)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/torch/jit/_recursive.py", line 757, in infer_methods_to_compile
    stubs.append(make_stub_from_method(nn_module, method))
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/torch/jit/_recursive.py", line 69, in make_stub_from_method
    return make_stub(func, method_name)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/torch/jit/_recursive.py", line 54, in make_stub
    ast = get_jit_def(func, name, self_name="RecursiveScriptModule")
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/torch/jit/frontend.py", line 293, in get_jit_def
    return build_def(parsed_def.ctx, fn_def, type_line, def_name, self_name=self_name, pdt_arg_types=pdt_arg_types)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/torch/jit/frontend.py", line 331, in build_def
    param_list = build_param_list(ctx, py_def.args, self_name, pdt_arg_types)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/torch/jit/frontend.py", line 355, in build_param_list
    raise NotSupportedError(ctx_range, _vararg_kwarg_err)
torch.jit.frontend.NotSupportedError: Compiled functions can't take variable number of arguments or use keyword-only arguments with defaults:
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/torch/nn/parallel/data_parallel.py", line 150
    def forward(self, *inputs, **kwargs):
                                ~~~~~~~ <--- HERE
        with torch.autograd.profiler.record_function("DataParallel.forward"):
```

So, it seems like the `torch.jit.script` does not like variable-length arguments such as *args and **kwargs..

What I want to ask is:
1) Do you have any plan to support DataParallel for torchsciprt?
2) Is there any other way that I could convert DataParallel model to TorchScript model?

## System Info

```
Collecting environment information...
PyTorch version: 1.13.1+cu117
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.1 LTS (x86_64)
GCC version: (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35

Python version: 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0] (64-bit runtime)
Python platform: Linux-5.15.0-58-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 12.0
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3070 Ti Laptop GPU
Nvidia driver version: 525.78.01
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.24.1
[pip3] torch==1.13.1
[pip3] torchvision==0.14.1
[conda] Could not collect
```

- PyTorch or Caffe2: PyTorch
- How you installed PyTorch (conda, pip, source): pip
- Build command you used (if compiling from source):
- OS: Ubuntu
- PyTorch version: 1.13.1
- Python version: 3.10.6
- CUDA/cuDNN version: 12.0
- GPU models and configuration: Nvidia GeForce 3070Ti


cc @EikanWang @jgong5 @wenzhe-nrv @sanchitintel