Beginner: Solar & Cloud NN

TimmiB: <p>Hi there,</p>
<p>i am very new to NNs and pyTorch. Is this the right approach to learn how the production evolves over the day taking the percentage of clouds into account?</p>
<pre><code class="lang-auto">import matplotlib.pyplot as plt
class Solar(torch.nn.Module):
    def __init__(self, inputSize, outputSize):
        super(Solar, self).__init__()
        self.linear1a = torch.nn.Linear(inputSize, outputSize)        
        self.linear1b = torch.nn.Linear(inputSize, outputSize)
        self.linear2 = torch.nn.Linear(inputSize, outputSize)    
        self.relu = torch.nn.ReLU()
        self.linear3 = torch.nn.Linear(inputSize, outputSize)        

    def forward(self, hours, clouds):
        out_hours = self.linear1a(hours)        
        out_clouds = self.linear1b(clouds)        
        out_c = self.linear2(out_hours-clouds)
        out_d = self.relu(out_c)
        out_e = self.linear3(out_d)
        
        return out_e
</code></pre>
<p>I am asking because the network learn negative values in the early morging and late night.<br>
Loss is stuck at 0.001</p>
<p>Regards</p>
<p>Tim</p>
<p>Regards</p>

ptrblck: <p>I’m not familiar with your use case, but I guess you want to subtract <code>out_clounds</code> from <code>out_hours</code> here:</p>
<pre><code class="lang-python">out_c = self.linear2(out_hours-clouds)
</code></pre>
<p>instead of the <code>clouds</code> input tensor?</p>

TimmiB: <p>Yes, you are right.</p>
<p>But still i am stuck at 0.00091 loss and the edges are negative.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/a/7ac3712cc4a0935ea023c0c09ab2e8bea716c830.png" data-download-href="https://discuss.pytorch.org/uploads/default/7ac3712cc4a0935ea023c0c09ab2e8bea716c830" title="Bildschirmfoto vom 2023-03-23 10-51-55"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/a/7ac3712cc4a0935ea023c0c09ab2e8bea716c830_2_690x485.png" alt="Bildschirmfoto vom 2023-03-23 10-51-55" data-base62-sha1="hw0Z1288RDwcB5pbPJYqfx2f7nW" width="690" height="485" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/a/7ac3712cc4a0935ea023c0c09ab2e8bea716c830_2_690x485.png, https://discuss.pytorch.org/uploads/default/optimized/3X/7/a/7ac3712cc4a0935ea023c0c09ab2e8bea716c830_2_1035x727.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/7/a/7ac3712cc4a0935ea023c0c09ab2e8bea716c830.png 2x" data-dominant-color="FBFCFC"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Bildschirmfoto vom 2023-03-23 10-51-55</span><span class="informations">1183×833 59.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

TimmiB: <p>Here is the training data.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/7/27a47f33496f7c87ebd215ad1d504a1d58fd6ffe.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/27a47f33496f7c87ebd215ad1d504a1d58fd6ffe" title="solarplot"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/7/27a47f33496f7c87ebd215ad1d504a1d58fd6ffe_2_690x494.jpeg" alt="solarplot" data-base62-sha1="5EH3mEm9r4JgZu9KI0w4ZwdCvM2" width="690" height="494" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/7/27a47f33496f7c87ebd215ad1d504a1d58fd6ffe_2_690x494.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/2/7/27a47f33496f7c87ebd215ad1d504a1d58fd6ffe_2_1035x741.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/2/7/27a47f33496f7c87ebd215ad1d504a1d58fd6ffe.jpeg 2x" data-dominant-color="DAD9D5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">solarplot</span><span class="informations">1111×797 60.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

TimmiB: <p>Initializing helped to get rid of the negative edges.</p>
<pre><code class="lang-auto">        torch.nn.init.normal_(self.linear1a.weight)
        torch.nn.init.constant_(self.linear1b.weight, 0.5)
        torch.nn.init.ones_(self.linear2.weight)
        torch.nn.init.ones_(self.linear3.weight) 
</code></pre>
<p>Still stuck at 0.001 average loss.</p>