Differing output for model.eval() vs model.train()

username: 5had3z
cooked: <p>I’m currently implementing a DETR head for my model, to replace a semantic segmentation head. However, I’m getting strange results for the bboxes when in model.eval() where they’re all clustered closely together around the middle of the image, whereas in model.train() they seem reasonably spread out and actually detecting things.</p>
<p>Gradients on/off has no effect. There are no branches in the model code for module.training or anything like that, so I’m unsure why this occuring and where to look to resolve this; I’ve never had this issue before…</p>
<p>Ubuntu20.04 + Pytorch 1.7.1 + RTX3090 (CUDA 11.1)</p>

username: Henry_Chibueze
cooked: <p>If possible I’d like to see your code</p>

username: 5had3z
cooked: <p>Github repo and branch (feature/detr_head) is linked below, the model definition can be found in nnet_training/nnet_models/detr_sfd.py</p><aside class="onebox githubfolder">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="16" height="16">
      <a href="https://github.com/5had3z/stereo-to-all/tree/feature/detr_head" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>
  <article class="onebox-body">
    <img src="https://avatars.githubusercontent.com/u/59598353?s=400&amp;amp;v=4" class="thumbnail onebox-full-image" width="60" height="60">

<h3><a href="https://github.com/5had3z/stereo-to-all/tree/feature/detr_head" target="_blank" rel="noopener nofollow ugc">5had3z/stereo-to-all</a></h3>

<p><a href="https://github.com/5had3z/stereo-to-all/tree/feature/detr_head" target="_blank" rel="noopener nofollow ugc">feature/detr_head</a></p>

  <p><span class="label1">The aim of this project is to generate a pointcloud with motion vectors and semantic segmentation of stereoscopic image sequences. Targeting real-time deployment on an nVIDIA Xavier. - 5had3z/stere...</span></p>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


username: Henry_Chibueze
cooked: <p>Sorry for the late reply.<br>
There’s nothing here tho</p>

username: 5had3z
cooked: <p>Yeah sorry I merged the branch into master. I’m using deeplab’s panotpic architecture rather than DETR, which worked fine straight away, no boundary box schenanigans. And since that’s working now I’m coming up with my own arch which will better unify panoptic segmentation into one singlular pipeline (No results yet, still working on it).</p>
<p>Link to the master branch is below.</p><aside class="onebox githubfolder">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="16" height="16">
      <a href="https://github.com/5had3z/CerberusNet/" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>
  <article class="onebox-body">
    <img src="https://avatars.githubusercontent.com/u/59598353?s=400&amp;amp;v=4" class="thumbnail onebox-full-image" width="60" height="60">

<h3><a href="https://github.com/5had3z/CerberusNet/" target="_blank" rel="noopener nofollow ugc">5had3z/CerberusNet</a></h3>


  <p><span class="label1">The aim of this project is to generate a pointcloud with motion vectors and semantic segmentation of stereoscopic image sequences. Targeting real-time deployment on an nVIDIA Xavier. - 5had3z/Cerbe...</span></p>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>