The predicted training values are not similar with actual training values

Siva1: <p>I am new to the implementation of the neural network , I am trying to implement a Bayesian neural network where the my training dataset size is of 2k samples of 8D.<br>
Here are the steps I have followed</p>
<ol>
<li>Have normalized the both x_train and y_train using the sklearn standard scaler.</li>
<li>My neural network<br>
<em># Define the Bayesian neural network</em><br>
<em>class BayesianNeuralNet(nn.Module):</em></li>
</ol>
<ul>
<li>
<p>def <strong>init</strong>(self,dropout_rate):*</p>
</li>
<li>
<pre><code>   super(BayesianNeuralNet, self).__init__()*
</code></pre>
</li>
<li>
<pre><code>   self.hidden1 = bnn.BayesLinear(prior_mu=0, prior_sigma=0.1, in_features=8, out_features=256) # Input layer to hidden layer*
</code></pre>
</li>
<li>
<pre><code>   self.act1 = nn.LeakyReLU()*
</code></pre>
</li>
<li>
<pre><code>   self.dropout1 = nn.Dropout(dropout_rate)*
</code></pre>
</li>
<li>
<pre><code>   self.hidden2 = bnn.BayesLinear(prior_mu=0, prior_sigma=0.1, in_features=256, out_features=256) # Hidden layer to hidden layer*
</code></pre>
</li>
<li>
<pre><code>   self.act2 = nn.LeakyReLU()*
</code></pre>
</li>
<li>
<pre><code>   self.dropout2 = nn.Dropout(dropout_rate)*
</code></pre>
</li>
<li>
<pre><code>   self.hidden3 = bnn.BayesLinear(prior_mu=0, prior_sigma=0.1, in_features=256, out_features=1)  # Hidden layer to output layer*
</code></pre>
</li>
<li>
<pre><code>   *
</code></pre>
</li>
<li>
<p>def forward(self, x):*</p>
</li>
<li>
<pre><code>   hidden1 = self.act1(self.hidden1(x))*
</code></pre>
</li>
<li>
<pre><code>   hidden1_dropout = self.dropout1(hidden1)*
</code></pre>
</li>
<li>
<pre><code>   hidden2 = self.act2(self.hidden2(hidden1_dropout))*
</code></pre>
</li>
<li>
<pre><code>   hidden2_dropout = self.dropout2(hidden2)*
</code></pre>
</li>
<li>
<pre><code>   output = self.hidden3(hidden2_dropout)*
</code></pre>
</li>
<li>
<pre><code>   # output = torch.clamp(output,min=0.003)*
</code></pre>
</li>
<li>
<pre><code>   output = torch.log(1 + torch.exp(output))*
</code></pre>
</li>
<li>
<pre><code>   # output = torch.tanh(output)*
</code></pre>
</li>
<li>
<pre><code>   return output*
</code></pre>
</li>
</ul>
<p>3)Here is my model</p>
<h1><a name="define-the-bayesian-neural-network-model-1" class="anchor" href="#define-the-bayesian-neural-network-model-1"></a>Define the Bayesian neural network model</h1>
<p>model = BayesianNeuralNet(dropout_rate=0.9)</p>
<p>loss_fn = nn.MSELoss()<br>
KL_loss_fn = bnn.BKLLoss(reduction=‘mean’,last_layer_only=False)<br>
KL_weight = 0.001</p>
<p>optimizer = torch.optim.Adam(model.parameters(), lr=1e-2)</p>
<h1><a name="train-the-bayesian-neural-network-2" class="anchor" href="#train-the-bayesian-neural-network-2"></a>Train the Bayesian neural network</h1>
<p>num_epochs = 10000</p>
<p>for epoch in range(num_epochs):<br>
for batch_x, batch_y in train_loader:</p>
<pre><code># Forward pass
outputs = model(batch_x)
loss = loss_fn(outputs, batch_y)
KL = KL_loss_fn(model)
cost = loss+KL_weight*KL

optimizer.zero_grad()  # Zero the gradients
cost.backward()        # Backward Propagation
optimizer.step()       # Update the model parameters

if (epoch+1) % 100 == 0:
  print(f'Epoch [{epoch+1}/{num_epochs}], Loss: {loss.item():.4f}')
</code></pre>
<ol start="4">
<li>And my predicted min and max values are<br>
Max_val_train_data: 1.3269883<br>
Min_value_train_data: 0.83512664</li>
</ol>
<p><strong>where as expected is</strong><br>
<strong>Max_val_train_data: 1.3269</strong><br>
<strong>Min_value: 0.0688</strong></p>
<p>Here my predicted min value is not being changed from Min_value_train_data: 0.83512664 .</p>
<p>Thank you for your responses in advance and suggest me where my model is wrong</p>