Hyperparameter optimization for PyTorch models

lavanya: <p>A <a href="https://github.com/wandb/examples/tree/master/pytorch-cnn-fashion" rel="noopener nofollow ugc">nice intro</a> to doing hyperparameter optimization for pytorch models.</p>
<p><a href="https://github.com/wandb/examples/tree/master/pytorch-cnn-fashion" rel="noopener nofollow ugc">Github repo</a></p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/9/e9403436336214ee059f5d19f86a53c05e435eaa.png" data-download-href="https://discuss.pytorch.org/uploads/default/e9403436336214ee059f5d19f86a53c05e435eaa" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/9/e9403436336214ee059f5d19f86a53c05e435eaa_2_690x311.png" alt="image" data-base62-sha1="xhqO1UaTtPFCyza14u4Xup09He2" width="690" height="311" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/9/e9403436336214ee059f5d19f86a53c05e435eaa_2_690x311.png, https://discuss.pytorch.org/uploads/default/optimized/3X/e/9/e9403436336214ee059f5d19f86a53c05e435eaa_2_1035x466.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/e/9/e9403436336214ee059f5d19f86a53c05e435eaa_2_1380x622.png 2x" data-dominant-color="EDEDEE"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1600×723 208 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>Thanks for sharing! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"><br>
Do you have any comparison between Weights &amp; Biases Sweep and e.g. <a href="https://optuna.org/">Optuna</a>?</p>

lavanya: <p>Hi thanks for your question!</p>
<p>W&amp;B supports all the features Optuna has (defining sophisticated parameter spaces, saving/resuming, distributed optimization) and also has more <a href="https://docs.wandb.com/sweeps/ray-tune" rel="noopener nofollow ugc">advanced search strategies</a> like hyperopt. It also generates visualizations that help see which hyperparameters were the most important and worth digging into more.</p>
<p>Here’s an example <a href="https://app.wandb.ai/wandb/s2c?workspace=user-lavanyashukla" rel="noopener nofollow ugc">sweeps workspace</a> if you’d like to play with them.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/6/865793701c591e9c5e379c35badcc88ab694eb3e.png" data-download-href="https://discuss.pytorch.org/uploads/default/865793701c591e9c5e379c35badcc88ab694eb3e" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/8/6/865793701c591e9c5e379c35badcc88ab694eb3e_2_690x307.png" alt="image" data-base62-sha1="jarBV0gBAXhZdlZkblMQBpnEkQ6" width="690" height="307" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/8/6/865793701c591e9c5e379c35badcc88ab694eb3e_2_690x307.png, https://discuss.pytorch.org/uploads/default/optimized/3X/8/6/865793701c591e9c5e379c35badcc88ab694eb3e_2_1035x460.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/8/6/865793701c591e9c5e379c35badcc88ab694eb3e_2_1380x614.png 2x" data-dominant-color="F7F8F8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1546×690 41.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>