Is there significance to the weights of the first / last layers?

username: laro
cooked: <p>I am working on a time series classification problem (5 classes) where the input contains 4 features with 90 samples for each feature.</p>
<pre><code class="lang-auto">class GRUModel(nn.Module):
    def __init__(self, feature_size, hidden_size, output_size):
        super(GRUModel, self).__init__()
        self.gru = nn.GRU(feature_size, hidden_size, batch_first=True)
        self.fc1 = nn.Linear(hidden_size, 64)
        self.fc2 = nn.Linear(64, output_size)
    
    def forward(self, x):
        out, _ = self.gru(x)
        out = out[:, -1, :] 
        out = self.fc1(out)
        out = self.fc2(out)
        return out
</code></pre>
<p>I’m looking on the weights of the first layer (the layer which first multiple the features):</p>
<pre><code class="lang-auto">l_feature_names     = ['Feature_1', 'Feature_2', 'Feature_3', 'Feature_4', 'Feature_5', 'Feature_6', 'Feature_7', 'Feature_8']
first_layer_weights = np.abs(model.gru.weight_ih_l0.data.sum(axis=0))

plt.rc('font', size=16)
plt.figure(figsize=(30, 6))
plt.bar(l_feature_names, first_layer_weights)
plt.title('First layer')
plt.ylabel('Weights')
plt.show()
</code></pre>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/d/4d636a2776891889446e79997ced803866430381.png" data-download-href="https://discuss.pytorch.org/uploads/default/4d636a2776891889446e79997ced803866430381" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/4/d/4d636a2776891889446e79997ced803866430381_2_690x158.png" alt="image" data-base62-sha1="b2BL83IyDwhLEpGIgmp6B0FQbpn" width="690" height="158" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/4/d/4d636a2776891889446e79997ced803866430381_2_690x158.png, https://discuss.pytorch.org/uploads/default/optimized/3X/4/d/4d636a2776891889446e79997ced803866430381_2_1035x237.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/4/d/4d636a2776891889446e79997ced803866430381_2_1380x316.png 2x" data-dominant-color="B2CDE0"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1666×382 23.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p><strong>when looking at the graph I created above, is it correct that features with larger weights have a greater impact on the model, or is that not necessarily the case?"</strong></p>
<p>Additionally, by examining the weights of the last layer:</p>
<pre><code class="lang-auto">l_class_names     = ['class_1','class_2', 'class_3', 'class_4', 'class_5']
last_layer_weights = np.abs(model.fc2.weight.data.sum(axis=1))

plt.rc('font', size=16)
plt.figure(figsize=(30, 6))
plt.bar(l_class_names, last_layer_weights)
plt.title('First layer')
plt.ylabel('Weights')
plt.show()
</code></pre>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/4/c49af0fac1db4a52954b848d6025569dfc08f76e.png" data-download-href="https://discuss.pytorch.org/uploads/default/c49af0fac1db4a52954b848d6025569dfc08f76e" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/c/4/c49af0fac1db4a52954b848d6025569dfc08f76e_2_690x152.png" alt="image" data-base62-sha1="s3fxKW5RLheqqNMWQcPyA46Xt82" width="690" height="152" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/c/4/c49af0fac1db4a52954b848d6025569dfc08f76e_2_690x152.png, https://discuss.pytorch.org/uploads/default/optimized/3X/c/4/c49af0fac1db4a52954b848d6025569dfc08f76e_2_1035x228.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/c/4/c49af0fac1db4a52954b848d6025569dfc08f76e_2_1380x304.png 2x" data-dominant-color="97BFDB"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1654×365 18 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p><strong>can we determine for which classes the model has more confidence in its classification?"</strong></p>