When I implement DAB-DETR have strange output

ximen0806449: <p>I use PyTorch to implement a more simple vision model of DAB-DETR. When I train 10 images in training mode and batch-size set 2 it looks like not bad, but when I am in the eval mode, my model sometimes outputs other images bounding boxes. So, I did other tests to demonstrate what happens. I change the order of the input image, I find the problem shows as well. I really can’t figure it out.</p>
<p>Because the code is too long, so I post my Github link here.</p>
<p><a href="https://github.com/Zhong-Zi-Zeng/Simple-Version-of-DAB-DETR.git" rel="noopener nofollow ugc">https://github.com/Zhong-Zi-Zeng/Simple-Version-of-DAB-DETR.git</a></p>
<p>Looks like these situations:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/e/fea839d4756ee436f5a40c4bc5a9a1fe3833be48.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/fea839d4756ee436f5a40c4bc5a9a1fe3833be48" title="Figure_1"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/e/fea839d4756ee436f5a40c4bc5a9a1fe3833be48_2_690x431.jpeg" alt="Figure_1" data-base62-sha1="AkNHo6lxcjJdOVvX5auweifjmdO" width="690" height="431" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/e/fea839d4756ee436f5a40c4bc5a9a1fe3833be48_2_690x431.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/f/e/fea839d4756ee436f5a40c4bc5a9a1fe3833be48_2_1035x646.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/f/e/fea839d4756ee436f5a40c4bc5a9a1fe3833be48_2_1380x862.jpeg 2x" data-dominant-color="AAADAF"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Figure_1</span><span class="informations">1600×1000 98.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ximen0806449: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/7/77f0496df3eebeb6c3add16e6ad5176fde3f32e6.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/77f0496df3eebeb6c3add16e6ad5176fde3f32e6" title="Figure_2"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/7/77f0496df3eebeb6c3add16e6ad5176fde3f32e6_2_690x431.jpeg" alt="Figure_2" data-base62-sha1="h71D88X4JYIQ3FD9n1Rq9tFMfNs" width="690" height="431" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/7/77f0496df3eebeb6c3add16e6ad5176fde3f32e6_2_690x431.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/7/7/77f0496df3eebeb6c3add16e6ad5176fde3f32e6_2_1035x646.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/7/7/77f0496df3eebeb6c3add16e6ad5176fde3f32e6_2_1380x862.jpeg 2x" data-dominant-color="C9C3A8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Figure_2</span><span class="informations">1600×1000 68.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ximen0806449: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/1/f156a1746d88ab755343938483ddcfa765d4a4b5.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/f156a1746d88ab755343938483ddcfa765d4a4b5" title="Figure_3"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/1/f156a1746d88ab755343938483ddcfa765d4a4b5_2_690x431.jpeg" alt="Figure_3" data-base62-sha1="yqYFTCyv9iR7pEMFCDsxcBJw9g1" width="690" height="431" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/1/f156a1746d88ab755343938483ddcfa765d4a4b5_2_690x431.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/f/1/f156a1746d88ab755343938483ddcfa765d4a4b5_2_1035x646.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/f/1/f156a1746d88ab755343938483ddcfa765d4a4b5_2_1380x862.jpeg 2x" data-dominant-color="A3A29B"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Figure_3</span><span class="informations">1600×1000 88.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ximen0806449: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/d/fd49c22ab17e507be92dcb25b7d50e3d6d7585fc.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/fd49c22ab17e507be92dcb25b7d50e3d6d7585fc" title="Figure_5"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/d/fd49c22ab17e507be92dcb25b7d50e3d6d7585fc_2_690x431.jpeg" alt="Figure_5" data-base62-sha1="A8GPgeuSADAwY8G46kzWdx1vmXy" width="690" height="431" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/d/fd49c22ab17e507be92dcb25b7d50e3d6d7585fc_2_690x431.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/f/d/fd49c22ab17e507be92dcb25b7d50e3d6d7585fc_2_1035x646.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/f/d/fd49c22ab17e507be92dcb25b7d50e3d6d7585fc_2_1380x862.jpeg 2x" data-dominant-color="AFA395"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Figure_5</span><span class="informations">1600×1000 90.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ximen0806449: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/5/85929edb5d156b4b40ca93bab72f33190ff873b3.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/85929edb5d156b4b40ca93bab72f33190ff873b3" title="Figure_6"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/8/5/85929edb5d156b4b40ca93bab72f33190ff873b3_2_690x431.jpeg" alt="Figure_6" data-base62-sha1="j3DDt9yOwNxj1QdNexUp0Us5x4v" width="690" height="431" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/8/5/85929edb5d156b4b40ca93bab72f33190ff873b3_2_690x431.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/8/5/85929edb5d156b4b40ca93bab72f33190ff873b3_2_1035x646.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/8/5/85929edb5d156b4b40ca93bab72f33190ff873b3_2_1380x862.jpeg 2x" data-dominant-color="C8C3A8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Figure_6</span><span class="informations">1600×1000 71 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>