How to find the right weight decay value in optimizer

lightersss: <p>Hi,every.<br>
I am trying to using weight decay to norm the loss function.I set the weight_decay of Adam(Adam) to 0.01(blue),0.005(gray),0.001(red) and I got the results in the pictures.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/e/4e98b7b4b31e8344069b3ff907fd787dd7fb7c07.png" data-download-href="https://discuss.pytorch.org/uploads/default/4e98b7b4b31e8344069b3ff907fd787dd7fb7c07" title="test loss"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/4/e/4e98b7b4b31e8344069b3ff907fd787dd7fb7c07_2_690x162.png" alt="test loss" data-base62-sha1="bdirc9FgyZ6BSXOj8w4YbXo55Kn" width="690" height="162" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/4/e/4e98b7b4b31e8344069b3ff907fd787dd7fb7c07_2_690x162.png, https://discuss.pytorch.org/uploads/default/optimized/3X/4/e/4e98b7b4b31e8344069b3ff907fd787dd7fb7c07_2_1035x243.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/4/e/4e98b7b4b31e8344069b3ff907fd787dd7fb7c07_2_1380x324.png 2x" data-dominant-color="FBFCFD"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">test loss</span><span class="informations">2097×495 43.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
It seems 0.01 is too big and 0.005 is too small or it’s something wrong with my model and data.<br>
I use the unet as model and the data is DRIVE dataset.I use grayscale pictures as input and increase the dataset by CLAHE,gamma correction and flip,learning rate is 0.001.<br>
And I always found my model get the best reslut in the beginning of the training!!<br>
Thank you</p>

lightersss: <p>the picture above is test loss.And this is training loss<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/1/31d63861d3e33edee9d1a99316bf4e0cd519cbfe.png" data-download-href="https://discuss.pytorch.org/uploads/default/31d63861d3e33edee9d1a99316bf4e0cd519cbfe" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/1/31d63861d3e33edee9d1a99316bf4e0cd519cbfe_2_690x138.png" alt="image" data-base62-sha1="76SmmW0QEytzgHLx2Z8Grzu3OJE" width="690" height="138" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/1/31d63861d3e33edee9d1a99316bf4e0cd519cbfe_2_690x138.png, https://discuss.pytorch.org/uploads/default/optimized/3X/3/1/31d63861d3e33edee9d1a99316bf4e0cd519cbfe_2_1035x207.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/3/1/31d63861d3e33edee9d1a99316bf4e0cd519cbfe_2_1380x276.png 2x" data-dominant-color="FBFCFD"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">2007×404 39.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

lightersss: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/f/6f6ea229c04e2ead18989e9330f797ad52dd11ef.png" data-download-href="https://discuss.pytorch.org/uploads/default/6f6ea229c04e2ead18989e9330f797ad52dd11ef" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/f/6f6ea229c04e2ead18989e9330f797ad52dd11ef_2_690x84.png" alt="image" data-base62-sha1="fTM1SrwmFSrpKy2pt9EcYwuiRwb" width="690" height="84" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/f/6f6ea229c04e2ead18989e9330f797ad52dd11ef_2_690x84.png, https://discuss.pytorch.org/uploads/default/optimized/3X/6/f/6f6ea229c04e2ead18989e9330f797ad52dd11ef_2_1035x126.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/6/f/6f6ea229c04e2ead18989e9330f797ad52dd11ef_2_1380x168.png 2x" data-dominant-color="F4F7F9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">2075×254 56.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div> this is the metrics on test set</p>