Network not converging sometimes for a simple XOR network

username: SeaShellAtSeaSide
cooked: <p>Hello I’m new to using the C++ API of pytorch (libtorch) and I implemented a simple XOR net for learning purposes and then I encounter this behavior, I don’t know what I did wrong.</p>
<p>Basically what is happening is sometimes when I run the program there are some instances where the XOR net will not converge and get stuck (the loss is not going down on an acceptable rate anymore), but most of the time it works and converge fast.</p>
<p>My hunch is it might be because of the random initialization of the weights of the network, that sometimes it initializes weights that are not good, but I don’t really know if this is the case.</p>
<p>Here is my implementation:</p>
<pre data-code-wrap="cpp"><code class="lang-cpp">#include &lt;iostream&gt;
#include &lt;memory&gt;
#include &lt;torch/torch.h&gt;

struct XorNet: torch::nn::Module {
  XorNet() {
    linear_stack = register_module("linear_stack",
      torch::nn::Sequential(
        torch::nn::Linear(torch::nn::LinearOptions(2, 2).bias(true)),
        torch::nn::Sigmoid(),
        torch::nn::Linear(torch::nn::LinearOptions(2, 1).bias(true)),
        torch::nn::Sigmoid()
      )
    );
  }

  torch::Tensor forward(torch::Tensor x) {
    auto x1 = linear_stack-&gt;forward(x);
    return x1;
  }

  torch::nn::Sequential linear_stack{nullptr};
};

int main() {
  torch::Device device(torch::cuda::is_available() ? torch::kCUDA : torch::kCPU);

  XorNet model;
  model.to(device);
  
  std::cout &lt;&lt; "Weights BEFORE training:\n";
  for (const auto&amp; pair : model.named_parameters()) {
    std::cout &lt;&lt; pair.key() &lt;&lt; ":\n" &lt;&lt; pair.value() &lt;&lt; "\n\n";
  }

  torch::optim::Adam optimizer(model.parameters(), torch::optim::AdamOptions(0.01f));
  torch::nn::BCELoss loss_fn;
  
  torch::Tensor inputs = torch::tensor({
    {0.f, 0.f}, {0.f, 1.f}, {1.f, 0.f}, {1.f, 1.f}
  });

  torch::Tensor labels = torch::tensor({{0.f}, {1.f}, {1.f}, {0.f}});

  inputs = inputs.to(device).to(torch::kFloat32);
  labels = labels.to(device).to(torch::kFloat32);

  std::cout &lt;&lt; "inputs  : \n" &lt;&lt; inputs &lt;&lt; "\n";
  std::cout &lt;&lt; "labels  : \n" &lt;&lt; labels &lt;&lt; "\n";

  auto initial_output = model.forward(inputs);
  auto initial_loss = loss_fn(initial_output, labels);

  std::cout &lt;&lt; "\nBefore Training:\n";
  std::cout &lt;&lt; "output  : \n" &lt;&lt; initial_output &lt;&lt; "\n";
  std::cout &lt;&lt; "loss    : \n" &lt;&lt; initial_loss   &lt;&lt; "\n";

  model.train();

  const size_t MAX_EPOCH = 1'000'000;
  size_t epoch = 0;

  while (epoch &lt; MAX_EPOCH) {
    optimizer.zero_grad();

    auto training_output = model.forward(inputs);
    auto training_loss = loss_fn(training_output, labels);

    training_loss.backward();
    optimizer.step();

    // log loss every 500 epochs
    if (epoch % 500 == 500 - 1) {
      std::cout &lt;&lt; "training loss (epoch[" &lt;&lt; epoch + 1 &lt;&lt; "]) : " &lt;&lt; training_loss.item().toFloat() &lt;&lt; "\n";
    }

    if (training_loss.item().toFloat() &lt; 0.005f) {
      std::cout &lt;&lt; "TRAINING DONE LOSS BELOW &lt; 0.005f | epochs: " &lt;&lt; epoch + 1 &lt;&lt; "\n";
      break;
    }

    epoch++;
  }

  if (epoch == MAX_EPOCH) {
    std::cout &lt;&lt; "MAX EPOCH ACHIEVED\n";
  }

  model.eval();

  std::cout &lt;&lt; "\n\nWeights AFTER training:\n";
  for (const auto&amp; pair : model.named_parameters()) {
    std::cout &lt;&lt; pair.key() &lt;&lt; ":\n" &lt;&lt; pair.value() &lt;&lt; "\n\n";
  }

  auto trained_output = model.forward(inputs);
  auto trained_loss = loss_fn(trained_output, labels);

  std::cout &lt;&lt; "\nAfter Training:\n";
  std::cout &lt;&lt; "output    : \n" &lt;&lt; trained_output &lt;&lt; "\n";
  std::cout &lt;&lt; "loss start: " &lt;&lt; initial_loss.item().toFloat() &lt;&lt; "\n";
  std::cout &lt;&lt; "loss end  : " &lt;&lt; trained_loss.item().toFloat() &lt;&lt; "\n";

  std::cout &lt;&lt; "Total Training Epochs Done : " &lt;&lt; epoch + 1 &lt;&lt; "\n";

  return 0;
} 
</code></pre>
<p>Example where the loss seems to be stuck</p>
<pre data-code-wrap="bash"><code class="lang-bash">training loss (epoch[604000]) : 0.346574
training loss (epoch[604500]) : 0.346574
...
training loss (epoch[610500]) : 0.346574
training loss (epoch[611000]) : 0.346574
</code></pre>
<p>Example where it converge fast:</p>
<pre data-code-wrap="bash"><code class="lang-bash">Weights BEFORE training:
linear_stack.0.weight:
 0.6285  0.4231
-0.5949 -0.6533
[ CPUFloatType{2,2} ]

linear_stack.0.bias:
-0.1745
-0.5666
[ CPUFloatType{2} ]

linear_stack.2.weight:
 0.4732  0.4593
[ CPUFloatType{1,2} ]

linear_stack.2.bias:
 0.2365
[ CPUFloatType{1} ]

inputs  : 
 0  0
 0  1
 1  0
 1  1
[ CPUFloatType{4,2} ]
labels  : 
 0
 1
 1
 0
[ CPUFloatType{4,1} ]

Before Training:
output  : 
 0.6500
 0.6473
 0.6537
 0.6536
[ CPUFloatType{4,1} ]
loss    : 
0.742499
[ CPUFloatType{} ]
training loss (epoch[500]) : 0.313808
training loss (epoch[1000]) : 0.0393235
training loss (epoch[1500]) : 0.0170335
training loss (epoch[2000]) : 0.00958167
training loss (epoch[2500]) : 0.0060585
TRAINING DONE LOSS BELOW &lt; 0.005f | epochs: 2737


Weights AFTER training:
linear_stack.0.weight:
 9.1454 -9.0918
 9.2082 -9.3059
[ CPUFloatType{2,2} ]

linear_stack.0.bias:
 4.5912
-4.9715
[ CPUFloatType{2} ]

linear_stack.2.weight:
-10.6332  11.3042
[ CPUFloatType{1,2} ]

linear_stack.2.bias:
 5.0559
[ CPUFloatType{1} ]


After Training:
output    : 
 0.0045
 0.9929
 0.9962
 0.0045
[ CPUFloatType{4,1} ]
loss start: 0.742499
loss end  : 0.00499383
Total Training Epochs Done : 2737
</code></pre>