Question for training NNs for simulaitons

username: anchorP34
cooked: <p>Hello there! I believe I’m taking an unconventional approach to training a pytorch NN model, and everything is working except for the model weights updating.</p>
<p>I am working with college football data. Each week, games are played and then there are rankings at the end of the week to determine the best 25 teams. I would like to pass in a NN model as an input to this simulation, and then the output is just the total loss value (that created).</p>
<p>The idea is that I would like to pass in a NN model, get the loss value of the simulation, update the NN model based on the loss value, and then keep looping so the NN adjusts to lower the total loss of the simulation.</p>
<p>So, there is no predefined X, which is almost every single tutorial out there. Here is my example:</p>
<pre><code class="lang-auto">import torch
from Simulation import Simulation

class MLP(nn.Module):
  '''
    Multilayer Perceptron for regression.
  '''
  def __init__(self):
    super().__init__()
    self.layers = nn.Sequential(
      nn.Linear(14, 7),
      nn.ReLU(),
      nn.Linear(7, 2)
    )

  def forward(self, x):
    '''
      Forward pass
    '''
    return self.layers(x)

nn_model = MLP()

def loss_fn(simulation):
    """
    Takes in a simulation and calculates the total loss values
    """
    simulation.run()
    loss_vals = torch.tensor(simulation.loss_values, dtype=torch.float32, requires_grad=True)
    return loss_vals.sum()

# Initialize optimizer
learning_rate = .01
optim = torch.optim.SGD(nn_model.parameters(), lr=learning_rate)

for epoch in range(100):

    # Clear optimizer gradient
    optim.zero_grad()

    simulation = Simulation(nn_model)
    simulation_loss = loss_fn(simulation)

    # Simulation loss back propogation
    simulation_loss.backward()

    # Update gradients for optimizer
    optim.step()

</code></pre>
<p>Currently, when I run this, the nn_model does not get adjusted at all and the loss value stays consistent. I have a feeling I am misusing the gradient wrong since I am recreating the loss values each time I run the simulation. I don’t see any other kind of examples like this online, but I think it can work?</p>
<p>Can anyone help?</p>

username: anchorP34
cooked: <p>For some more details, if I run this, it appears the nn_model gradient is <strong>None</strong></p>
<p>Input:</p>
<pre><code class="lang-auto">for p in nn_model.parameters():
    print(p)
    print(p.grad)
</code></pre>
<p>Output:</p>
<pre><code class="lang-auto">Parameter containing:
tensor([[-0.1978,  0.0469,  0.1164, -0.1257,  0.0034, -0.1209, -0.0137,  0.0971,
          0.0351, -0.0083, -0.0641,  0.2506,  0.2017,  0.0914],
        [-0.1092,  0.1701, -0.1264, -0.0823, -0.1256, -0.0195,  0.1005,  0.0596,
          0.0111,  0.0668,  0.1365,  0.2540, -0.1946,  0.1809],
        [ 0.2438,  0.0185,  0.0482, -0.0455,  0.0429,  0.2448,  0.1374,  0.1691,
         -0.2104,  0.1133,  0.0118,  0.2505, -0.1687,  0.0878],
        [ 0.0927,  0.0283,  0.0867,  0.2292, -0.1618,  0.2453,  0.0285, -0.0992,
         -0.2631, -0.0152,  0.1378, -0.1377, -0.2546,  0.0670],
        [-0.2437, -0.0359,  0.0560,  0.0900,  0.1361, -0.1953, -0.1693,  0.1974,
          0.1044,  0.1183,  0.0585,  0.1207,  0.2146,  0.0622],
        [ 0.1107,  0.0769, -0.0725, -0.0245, -0.0653, -0.1510,  0.0954,  0.1853,
         -0.2302,  0.0065,  0.1409,  0.1542, -0.1193, -0.1056],
        [ 0.0874,  0.0231,  0.0989,  0.1284, -0.1804, -0.0444, -0.1670, -0.1974,
         -0.0350, -0.1522, -0.1807, -0.0870, -0.2510, -0.2306]],
       requires_grad=True)
None
Parameter containing:
tensor([ 0.2406, -0.1373, -0.1369,  0.0979, -0.2350,  0.1192,  0.0918],
       requires_grad=True)
None
Parameter containing:
tensor([[ 0.3746,  0.2180, -0.2989, -0.2731, -0.0852,  0.0394, -0.1043],
        [-0.1727, -0.2764, -0.3176, -0.2372,  0.3607,  0.3645,  0.1437]],
       requires_grad=True)
None
Parameter containing:
tensor([ 0.2851, -0.2031], requires_grad=True)
None
</code></pre>