Repository: pytorch
Issue: Torchscript Compiled functions don't support keyword-only arguments with defaults
Labels: oncall: jit

### 🚀 The feature, motivation and pitch

In #69980 (and the other PRs in the stack), the `optim/_functional` API is being refactored to include both foreach and single tensor capabilities. This includes adding a boolean argument `foreach` to the functional form for each optimizer in order to determine which logic is used. 

Ideally, we want to make `foreach` a kwonly argument with a default so that existing users of the API will not be affected by this change. However, doing this yields the following error message as the functional forms are called with `@torch.jit.script` in torch/distributed/optim:

`torch.jit.frontend.NotSupportedError: Compiled functions can't take variable number of arguments or use keyword-only arguments with defaults` (detailed paste below)

```
Traceback (most recent call last):
  File "/fsx/users/mg1998/branches/optim_foreach_cleanup/test/test_functional_optim.py", line 8, in <module>
    from torch.distributed.optim import functional_optim_map
  File "/fsx/users/mg1998/branches/optim_foreach_cleanup/torch/distributed/optim/__init__.py", line 14, in <module>
    from .functional_adadelta import _FunctionalAdadelta
  File "/fsx/users/mg1998/branches/optim_foreach_cleanup/torch/distributed/optim/functional_adadelta.py", line 17, in <module>
    class _FunctionalAdadelta(object):
  File "/fsx/users/mg1998/branches/optim_foreach_cleanup/torch/jit/_script.py", line 1302, in script
    _compile_and_register_class(obj, _rcb, qualified_name)
  File "/fsx/users/mg1998/branches/optim_foreach_cleanup/torch/jit/_recursive.py", line 44, in _compile_and_register_class
    script_class = torch._C._jit_script_class_compile(qualified_name, ast, defaults, rcb)
  File "/fsx/users/mg1998/branches/optim_foreach_cleanup/torch/jit/_recursive.py", line 838, in try_compile_fn
    return torch.jit.script(fn, _rcb=rcb)
  File "/fsx/users/mg1998/branches/optim_foreach_cleanup/torch/jit/_script.py", line 1315, in script
    ast = get_jit_def(obj, obj.__name__)
  File "/fsx/users/mg1998/branches/optim_foreach_cleanup/torch/jit/frontend.py", line 264, in get_jit_def
    return build_def(parsed_def.ctx, fn_def, type_line, def_name, self_name=self_name, pdt_arg_types=pdt_arg_types)
  File "/fsx/users/mg1998/branches/optim_foreach_cleanup/torch/jit/frontend.py", line 302, in build_def
    param_list = build_param_list(ctx, py_def.args, self_name, pdt_arg_types)
  File "/fsx/users/mg1998/branches/optim_foreach_cleanup/torch/jit/frontend.py", line 337, in build_param_list
    raise NotSupportedError(ctx_range, _vararg_kwarg_err)
torch.jit.frontend.NotSupportedError: Compiled functions can't take variable number of arguments or use keyword-only arguments with defaults:
  File "/fsx/users/mg1998/branches/optim_foreach_cleanup/torch/optim/adadelta.py", line 133
             eps: float,
             weight_decay: float,
             foreach: bool = None):
                             ~~~~ <--- HERE
    r"""Functional API that performs Adadelta algorithm computation.
```

A simple way to repro this error message would be

```
import torch

@torch.jit.script
def f(a: float, *, b: bool = False):
    return a

```

### Alternatives

A workaround here would be to add `foreach` as a kwarg (rather than kwonly) as torchscript compilation supports kwargs. However, this might be error prone, especially if more boolean kwargs with defaults are added as users of the API might swap the arguments, hence we would like support for kwonly arguments with a default. 

### Additional context

_No response_

@albanD