How can i ignore thoes layers?

username: logy
cooked: <p>i need to extract features before classifying it … i think i should ignore those layers (avgpool and head)… right? how can i do this<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/6/46ed99f6a7d9224486ad217ef9ac8ee6b0e4f3d9.png" data-download-href="https://discuss.pytorch.org/uploads/default/46ed99f6a7d9224486ad217ef9ac8ee6b0e4f3d9" title="Screenshot from 2021-12-19 15-51-02"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/4/6/46ed99f6a7d9224486ad217ef9ac8ee6b0e4f3d9_2_690x360.png" alt="Screenshot from 2021-12-19 15-51-02" data-base62-sha1="a7stRmVpobvUR9yTQcfInCqZdwJ" width="690" height="360" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/4/6/46ed99f6a7d9224486ad217ef9ac8ee6b0e4f3d9_2_690x360.png, https://discuss.pytorch.org/uploads/default/optimized/3X/4/6/46ed99f6a7d9224486ad217ef9ac8ee6b0e4f3d9_2_1035x540.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/4/6/46ed99f6a7d9224486ad217ef9ac8ee6b0e4f3d9.png 2x" data-dominant-color="0D0D0D"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2021-12-19 15-51-02</span><span class="informations">1054×551 67.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: logy
cooked: <p>is there any help please ?</p>

username: ptrblck
cooked: <p>You could either use forward hooks to get the desired activation as described <a href="https://discuss.pytorch.org/t/how-can-l-load-my-best-model-as-a-feature-extractor-evaluator/17254/6">here</a> or replace the unwanted layers with <code>nn.Identity</code> modules.</p>

username: logy
cooked: <p>thanks a lot for replying but will the results with both methods be the same ? and if I want the features for the images from this model but without a classification layer … should I remove or ignore the last two layers (head and avgpool) … as I tried to ignore them but got the features with large size</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="logy" data-post="4" data-topic="139734">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/logy/48/44859_2.png" class="avatar"> logy:</div>
<blockquote>
<p>thanks a lot for replying but will the results with both methods be the same ?</p>
</blockquote>
</aside>
<p>I’m not sure which both models you mean, so could you explain it a bit more, please?</p>
<aside class="quote no-group" data-username="logy" data-post="4" data-topic="139734">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/logy/48/44859_2.png" class="avatar"> logy:</div>
<blockquote>
<p>and if I want the features for the images from this model but without a classification layer … should I remove or ignore the last two layers (head and avgpool)</p>
</blockquote>
</aside>
<p>As previously said, you could replace these layers with <code>nn.Identity</code> or use a forward hook to “skip” them (i.e. get the activation before it’s passed to these two layers). Alternatively, you could of course also write a custom model and remove these layers.</p>

username: logy
cooked: <p>thanks . excuse me I tried to extract features from transformer from their GitHub profile by the <code>head. identity()</code> and with using <code>Timm</code>  … but I got the different shape for the features from both … using their class I got 768 but using the class where I got 1536 … why this different happen, please ?</p>

username: ptrblck
cooked: <p>I don’t know what you are using exactly, but check if some functional API calls are performed in the <code>forward</code> pass and might differ between both approaches.<br>
If that’s not the case, check if you’ve forgotten to remove/replace a pooling layer, which might change the shape etc.</p>