Repository: pytorch
Issue: How to export torch.optim.LBFGS using torch.onnx.export
Labels: module: onnx,module: optimizer,triaged

### 🚀 The feature, motivation and pitch

I have a python code that solve linear equations with torch.optim.LBFGS. And I want to make it work in C++. One posible way is to use libtorch. But I wander if I can export it like nn.Module with torch.onnx.export.
Here is my python code:
```
import torch
import torch.nn as nn
import onnxruntime as rt
from torch.autograd import Variable

def test(jac_t, state):
    n_actions = 5
    dt = 0.01

    # target = torch.randn(n_actions, 1)
    target = torch.tensor([[ 0.0754],
                       [ 1.2151],
                       [-1.4920],
                       [ 1.1642],
                       [ 0.2289]])
    mat = torch.matmul(jac_t, target) * dt + state

    init = torch.randn(n_actions, 1)
    init = torch.tensor([[-0.3018],
                       [ 1.1070],
                       [-1.4571],
                       [ 1.0705],
                       [-0.8479]])
    q_dot = Variable(init, requires_grad=True)
    v = [q_dot]
    optimizer = torch.optim.LBFGS(v)#, lr=0.1)
    for i in range(0, 10):
        def cost():
            optimizer.zero_grad()
            next_state = torch.matmul(jac_t, q_dot) * dt + state
            d = torch.pow(next_state - mat, 2).sum()
            d.backward()
            return d
        optimizer.step(cost)
        d = cost()
        if d < 1e-3:
            break
    return init

class Test(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear1 = nn.Linear(5, 1)
        self.linear2 = nn.Linear(1, 1)

    def forward(self, jac_t, state):
        out = self.linear1(jac_t), self.linear2(state)
        c = test(jac_t, state)
        return out, c

if __name__ == '__main__':
    ttt = Test()
    ttt.eval()
    n_actions = 5
    # jac_t = torch.randn(6, n_actions)
    jac_t = torch.tensor([[ 2.0041,  2.2399, -0.0553,  1.4054,  0.2301],
                          [ 1.4019, -2.3094, -1.0461,  0.7753,  1.0787],
                          [-0.6338,  0.1553, -1.1531,  1.0613, -0.2952],
                          [ 0.0541, -0.3652, -0.5361,  2.0200,  0.9431],
                          [ 0.4075,  1.4435, -1.5067, -0.5096,  0.7448],
                          [-0.6440, -0.6492,  0.3728, -2.8277, -1.1983]])
    # state = torch.randn(6, 1)
    state = torch.tensor([[-1.1193],
                          [ 0.2084],
                          [-1.4547],
                          [-1.2416],
                          [ 0.9738],
                          [ 1.6379]])
    torch.onnx.export(ttt, (jac_t, state), 'ttt.onnx')

    a = ttt.forward(jac_t, state)
    print('a', a[-1])

    sess = rt.InferenceSession('ttt.onnx')
    b = sess.run(None, {"jac_t": jac_t.numpy(), "state": state.numpy()})
    print('b', b[-1])
```
The outputs of a and b are the same (both close to the value of target), which meas that the inference with the exported onnx file do some calculation like python code. But if I comment out  `init = torch.tensor([[-0.3018]...` and use `init = torch.randn(n_actions, 1)`, the output of b will be wrong.
So I guess the calculation of the exported onnx module is not dynamic. It records the way to add/multiply to the result, something like Computational Graphs. In fact I have to use `out = self.linear1(jac_t), self.linear2(state)` to put jac_t, state into Computational Graphs.
What's the proper way to export torch.optim.LBFGS?

### Alternatives

_No response_

### Additional context

_No response_

cc @vincentqb @jbschlosser @albanD @janeyx99 @crcrpar