Repository: pytorch
Issue: Allow positional arguments to be passed as kwargs for autograd custom Function
Labels: module: autograd,triaged,has workaround,actionable

## 🐛 Bug

According to the [Extending PyTorch](https://pytorch.org/docs/stable/notes/extending.html) doc page, optional arguments are allowed in a `torch.autograd.Function`. This works well when passing the optional argument as a positional argument, but fails when passing the optional argument as a keyword argument.

## To Reproduce

For example, consider the following minimal non-working example, which implements the custom function `y = factor*sin(x)`:

```python
import torch
import numpy as np

class MySin(torch.autograd.Function):

    @staticmethod
    def forward(ctx, input, factor=1):
        args = input.detach().numpy()
        res = factor*np.sin(args)
        ctx.save_for_backward(input)
        ctx.factor = factor
        return torch.from_numpy(res)

    @staticmethod
    def backward(ctx, grad_output):
        args = ctx.saved_tensors[0].detach().numpy()
        jacobian = ctx.factor*np.diag(np.cos(args))
        grad_input = torch.from_numpy(grad_output.numpy().T @ jacobian)
        return grad_input, None

my_sin = MySin.apply
```

Calling it with `factor` passed positional, or not at all, works fine:

```python
>>> x = torch.autograd.Variable(torch.tensor([0.812]), requires_grad=True)
>>> out = my_sin(x, 6)
>>> out.backward()
>>> print(x.grad, 6*np.cos(0.812))
tensor([4.1283]) 4.12829088343
```

However, passing it as a keyword argument results in an error:
```python
>>> x = torch.autograd.Variable(torch.tensor([0.812]), requires_grad=True)
>>> out = my_sin(x, factor=6)
>>> out.backward()
>>> x.grad
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-60-d83054d85504> in <module>()
      1 x = torch.autograd.Variable(torch.tensor([0.812]), requires_grad=True)
----> 2 out = my_sin(x, factor=6)
      3 out.backward()
      4 x.grad

TypeError: apply() takes no keyword arguments
```

## Expected behavior

Passing the optional argument as a keyword argument should match the behaviour when passing as a positional argument:
```python
>>> x = torch.autograd.Variable(torch.tensor([0.812]), requires_grad=True)
>>> out = my_sin(x, factor=6)
>>> out.backward()
>>> print(x.grad, 6*np.cos(0.812))
tensor([4.1283]) 4.12829088343
```

This is useful for two reasons:
1. It is consistent with standard Python use, and helps to avoid unexpected behaviour
2. Sometimes it is useful to use custom functions to wrap 'black-box' NumPy functions which evaluate their value and their gradient, and don't have a torch equivalent. In a lot of cases, these depend on keyword arguments, and it would be nice to be able to specify `**kwargs` in the static `forward` method signature.

## Environment

PyTorch version: 1.0.1.post2
Is debug build: No
CUDA used to build PyTorch: None

OS: Ubuntu 14.04.5 LTS
GCC version: (Ubuntu 5.5.0-12ubuntu1~14.04) 5.5.0 20171010
CMake version: version 3.2.2

Python version: 3.6
Is CUDA available: No
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA

Versions of relevant libraries:
[pip3] numpy==1.13.3
[pip3] torch==1.0.1.post2
[pip3] torchvision==0.2.1
[conda] Could not collect


cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7