How to add graphs to hparams in tensorboard?

username: aledjackson
cooked: <p>I’m trying to use tensorboard with pytorch and cannot find any clear documentation as to how to add graphs to the hparams dashboard. The image below shows what I want<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/0/50c5b2b9d0c6848f41600e3632f51e87ef78f9d5.png" data-download-href="https://discuss.pytorch.org/uploads/default/50c5b2b9d0c6848f41600e3632f51e87ef78f9d5" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/5/0/50c5b2b9d0c6848f41600e3632f51e87ef78f9d5_2_690x200.png" alt="image" data-base62-sha1="bwxLmG38SiXSoAu2BPALM4hKXOJ" width="690" height="200" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/5/0/50c5b2b9d0c6848f41600e3632f51e87ef78f9d5_2_690x200.png, https://discuss.pytorch.org/uploads/default/optimized/3X/5/0/50c5b2b9d0c6848f41600e3632f51e87ef78f9d5_2_1035x300.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/5/0/50c5b2b9d0c6848f41600e3632f51e87ef78f9d5_2_1380x400.png 2x" data-dominant-color="FAEFE4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1813×526 34.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
however I can only add singular values for the metrics:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/f/bf80a10adee2a65c843386938f9d7f808e53397c.png" data-download-href="https://discuss.pytorch.org/uploads/default/bf80a10adee2a65c843386938f9d7f808e53397c" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/f/bf80a10adee2a65c843386938f9d7f808e53397c_2_690x172.png" alt="image" data-base62-sha1="rk6MzhqWzV2ykwBAJJYoh7t2XH6" width="690" height="172" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/f/bf80a10adee2a65c843386938f9d7f808e53397c_2_690x172.png, https://discuss.pytorch.org/uploads/default/optimized/3X/b/f/bf80a10adee2a65c843386938f9d7f808e53397c_2_1035x258.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/b/f/bf80a10adee2a65c843386938f9d7f808e53397c_2_1380x344.png 2x" data-dominant-color="FCEEE0"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1813×454 20.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
Thanks in advance.</p>

username: aledjackson
cooked: <p>Having dug a little deeper into <code>torch.utils.tensorboard</code>, I can’t seem to find how to use this functionality. However I can achieve it by modifying the <code>SummaryWriter.add_hparams()</code> function as such:</p>
<pre data-code-wrap="python"><code class="lang-python">			def add_hparams(self, hparam_dict, metric_dict, hparam_domain_discrete=None, run_name=None):
				torch._C._log_api_usage_once("tensorboard.logging.add_hparams")
				if type(hparam_dict) is not dict or type(metric_dict) is not dict:
					raise TypeError('hparam_dict and metric_dict should be dictionary.')
				exp, ssi, sei = hparams(hparam_dict, metric_dict, hparam_domain_discrete)

				self.file_writer.add_summary(exp)
				self.file_writer.add_summary(ssi)
				self.file_writer.add_summary(sei)
				for k, v in metric_dict.items():
					if v is not None:
						self.add_scalar(k, v)
</code></pre>
<p>To use this, when you create the <code>SummaryWriter</code> you should set <code>log_dir=log_dir+'/[run_name]'</code>. I have some example code and pictures to show how to use this below.</p>
<pre data-code-wrap="python3"><code class="lang-python3">s = SummaryWriter(log_dir=f'{log_dir}/run1')
h_params = {'This': 1, 'is': 2, 'a': 3, 'test': 4}
metrics = {'accuracy/accuracy': None, 'loss/loss': None}

accuracy_values = [(1, 0.6), (2, 0.8), (3, 0.9), (4, 0.95)]
loss_values = [(1, 3), (2, 1), (3, 0.5), (4, 0.11)]
for step, v in accuracy_values:
	s.add_scalar('accuracy/accuracy', v, step)
for step, v in loss_values:
	s.add_scalar('loss/loss', v, step)

s.add_hparams(h_params, metrics)
s.close()
</code></pre>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/2/c20cbf2a1f0de0ce67df806e608f128bdec188fa.png" data-download-href="https://discuss.pytorch.org/uploads/default/c20cbf2a1f0de0ce67df806e608f128bdec188fa" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/c/2/c20cbf2a1f0de0ce67df806e608f128bdec188fa_2_690x172.png" alt="image" data-base62-sha1="rGDWdBIUhKBOHsF4aiNScTCiDmq" width="690" height="172" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/c/2/c20cbf2a1f0de0ce67df806e608f128bdec188fa_2_690x172.png, https://discuss.pytorch.org/uploads/default/optimized/3X/c/2/c20cbf2a1f0de0ce67df806e608f128bdec188fa_2_1035x258.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/c/2/c20cbf2a1f0de0ce67df806e608f128bdec188fa_2_1380x344.png 2x" data-dominant-color="FBEDDF"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1813×454 30.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Something that I’m not sure if it’s a bug or not is that in the original code, any scalars are logged to the log_dir whilst hyper parameters are logged in their own directory and given a run_name.</p>
<p>(I’ve just posted this as hopefully this is useful to someone)</p>

username: rajathsalegame
cooked: <p>Thanks, this is a massive lifesaver. I think there would definitely be a benefit to rewriting the PyTorch Tensorboard interface to make it more intuitive and increasing the detail of documentation. It shouldn’t be that hard to be able to log metrics on a per-epoch basis with hyperparameters.</p>
<p>“Something that I’m not sure if it’s a bug or not is that in the original code, any scalars are logged to the log_dir whilst hyper parameters are logged in their own directory and given a run_name.” As for this part, this also seems to be a weird default behavior of the add_hyperparameters option to use the log_dir along with the current timestamp (time.time()). See <a href="https://github.com/pytorch/pytorch/blob/bf88a4dad5b3c06f43a9ce07a51ec0ab0dbfe2d3/torch/utils/tensorboard/writer.py#L305-L307" class="inline-onebox" rel="noopener nofollow ugc">pytorch/writer.py at bf88a4dad5b3c06f43a9ce07a51ec0ab0dbfe2d3 · pytorch/pytorch · GitHub</a>. A quick fix is to just delete those lines and just use the original log_dir.</p>

username: kyeongjun1007
cooked: <p>How can I plot multiple graph in one plot?</p>
<p>I tried these below.</p>
<pre><code class="lang-auto">    def add_hparams(self, param_dict, metrics_dict, step=None):
        exp, ssi, sei = hparams(param_dict, metrics_dict)

        self.writer.file_writer.add_summary(exp)
        self.writer.file_writer.add_summary(ssi)
        self.writer.file_writer.add_summary(sei)

        if step is not None:
            for k, v in metrics_dict.item() :
                self.writer.add_scalar(k, v, step)
</code></pre>
<p>but if I set the metrics_dict = {‘train_loss’ : train_loss, ‘valid_loss’ : valid_loss}, than there are two plot in board.</p>
<pre><code class="lang-auto">    def add_hparams(self, param_dict, metrics_dict, step=None):
        exp, ssi, sei = hparams(param_dict, metrics_dict)

        self.writer.file_writer.add_summary(exp)
        self.writer.file_writer.add_summary(ssi)
        self.writer.file_writer.add_summary(sei)

        if step is not None:
            self.writer.add_scalars('loss', metrics_dict, step)
</code></pre>
<p>and if I set the metrics_dict = {‘loss’ : {‘train_loss’ : train_loss, 'valid_loss : valid_loss}} than It not works… (board can show valid loss only)</p>