Repository: Lightning-AI/pytorch-lightning

Issue:
Tensorboard display nothing
<!-- 
### Common bugs:
1. Tensorboard not showing in Jupyter-notebook see [issue 79](https://github.com/PyTorchLightning/pytorch-lightning/issues/79).    
2. PyTorch 1.1.0 vs 1.2.0 support [see FAQ](https://github.com/PyTorchLightning/pytorch-lightning#faq)    
-->

## 🐛 Bug

<!-- A clear and concise description of what the bug is. -->

### To Reproduce

Steps to reproduce the behavior:

I run the code below successfuly, and when i open my tensorboard, scalar display in board, but there's not value on the plot, vert confusing~

<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->


#### Code sample
<!-- Ideally attach a minimal code sample to reproduce the decried issue. 
Minimal means having the shortest code but still preserving the bug. -->
```python
class DNNClassifer(pl.LightningModule):
    def __init__(self):
        super().__init__()

        # network
       
        # others
        self.criterion = MSELoss()
        self.logger_kun = init_logger("dnn", notebook=True)
        

    def forward(self, x):
        x = self.model.forward(x)
        return x
    
    def configure_optimizers(self):
        return torch.optim.Adam(self.parameters())
    
    def training_step(self, batch, batch_idx):
        x, y, data_load_times = batch
        y_pred = self(x)
        
        loss = self.criterion(y_pred.squeeze(), y)
        self.logger.experiment.add_scalar('loss', loss, self.global_step)
        result = pl.TrainResult(minimize=loss)
        result.log("train_loss", loss, on_epoch=True)
        return result
        
    def validation_step(self, batch, batch_idx):
        x, y, data_load_times = batch
        y_pred = self(x)
        loss = self.criterion(y_pred.squeeze(), y)
        
        result = pl.EvalResult(checkpoint_on=loss)
        result.log("valid_loss", loss)
        return result
```
![image](https://user-images.githubusercontent.com/13381684/93210396-1baf9880-f792-11ea-9415-a1ea4ae4d48d.png)

### Expected behavior
<!-- A clear and concise description of what you expected to happen. -->
Display value on tensorboard

### Environment

Please copy and paste the output from our
[environment collection script](https://raw.githubusercontent.com/PyTorchLightning/pytorch-lightning/master/tests/collect_env_details.py)
(or fill out the checklist below manually).

You can get the script and run it with:
```
wget https://raw.githubusercontent.com/PyTorchLightning/pytorch-lightning/master/tests/collect_env_details.py
# For security purposes, please check the contents of collect_env_details.py before running it.
python collect_env_details.py
```

* CUDA:
	- GPU:
	- available:         False
	- version:           10.2
* Packages:
	- numpy:             1.18.1
	- pyTorch_debug:     False
	- pyTorch_version:   1.6.0
	- pytorch-lightning: 0.9.0
	- tensorboard:       2.2.0
	- tqdm:              4.42.1
* System:
	- OS:                Linux
	- architecture:
		- 64bit
		- 
	- processor:         x86_64
	- python:            3.7.6
	- version:           #47~18.04.1-Ubuntu SMP Thu May 7 13:10:50 UTC 2020

### Additional context

<!-- Add any other context about the problem here. -->



Discussion:
how many batches are in one epoch?
try a lower value for row_log_interval Trainer argument
Trainer's row_log_interval parameter is set to 50 by default. If the number of batches is less than row_log_interval, train metrics are never logged.
fyi, there is already an issue here #3466 which requests to base row_log_interval onto global step. It would solve the issue described here.
> batches

em... I think give a warning when row_log_interval < 50 will be a better user experience

Resolving patch:
diff --git a/pytorch_lightning/trainer/connectors/logger_connector.py b/pytorch_lightning/trainer/connectors/logger_connector.py
--- a/pytorch_lightning/trainer/connectors/logger_connector.py
+++ b/pytorch_lightning/trainer/connectors/logger_connector.py
@@ -461,9 +461,11 @@ def __gather_result_across_time_and_optimizers(self, epoch_output):
 
         return gathered_epoch_outputs
 
-    def log_train_step_metrics(self, batch_idx, batch_output):
+    def log_train_step_metrics(self, batch_output):
         # when metrics should be logged
-        should_log_metrics = (batch_idx + 1) % self.trainer.row_log_interval == 0 or self.trainer.should_stop
+        should_log_metrics = (
+            (self.trainer.global_step + 1) % self.trainer.row_log_interval == 0 or self.trainer.should_stop
+        )
         if should_log_metrics or self.trainer.fast_dev_run:
             # logs user requested information to logger
             metrics = batch_output.batch_log_metrics
diff --git a/pytorch_lightning/trainer/training_loop.py b/pytorch_lightning/trainer/training_loop.py
--- a/pytorch_lightning/trainer/training_loop.py
+++ b/pytorch_lightning/trainer/training_loop.py
@@ -443,15 +443,15 @@ def optimizer_zero_grad(self, batch_idx, optimizer, opt_idx):
 
     def on_before_backward(self, batch_idx, optimizer):
         # track gradient norms
-        grad_norm_dic = self._track_gradient_norm(batch_idx)
+        grad_norm_dic = self._track_gradient_norm()
 
         # clip gradients
         self.trainer.accelerator_backend.clip_gradients(optimizer)
         return grad_norm_dic
 
-    def _track_gradient_norm(self, batch_idx):
+    def _track_gradient_norm(self):
         grad_norm_dict = {}
-        if (batch_idx + 1) % self.trainer.row_log_interval == 0:
+        if (self.trainer.global_step + 1) % self.trainer.row_log_interval == 0:
             if float(self.trainer.track_grad_norm) > 0:
                 model = self.trainer.get_model()
                 grad_norm_dict = model.grad_norm(self.trainer.track_grad_norm)
@@ -542,12 +542,12 @@ def run_training_epoch(self):
             # -----------------------------------------
             # SAVE METRICS TO LOGGERS
             # -----------------------------------------
-            self.trainer.logger_connector.log_train_step_metrics(batch_idx, batch_output)
+            self.trainer.logger_connector.log_train_step_metrics(batch_output)
 
             # -----------------------------------------
             # SAVE LOGGERS (ie: Tensorboard, etc...)
             # -----------------------------------------
-            self.save_loggers_on_train_batch_end(batch_idx)
+            self.save_loggers_on_train_batch_end()
 
             # update LR schedulers
             monitor_metrics = deepcopy(self.trainer.logger_connector.callback_metrics)
@@ -786,9 +786,11 @@ def build_train_args(self, batch, batch_idx, opt_idx, hiddens):
 
         return args
 
-    def save_loggers_on_train_batch_end(self, batch_idx):
+    def save_loggers_on_train_batch_end(self):
         # when loggers should save to disk
-        should_save_log = (batch_idx + 1) % self.trainer.log_save_interval == 0 or self.trainer.should_stop
+        should_save_log = (
+            (self.trainer.global_step + 1) % self.trainer.log_save_interval == 0 or self.trainer.should_stop
+        )
         if should_save_log or self.trainer.fast_dev_run:
             if self.trainer.is_global_zero and self.trainer.logger is not None:
                 self.trainer.logger.save()


Test patch:
diff --git a/tests/trainer/test_trainer.py b/tests/trainer/test_trainer.py
--- a/tests/trainer/test_trainer.py
+++ b/tests/trainer/test_trainer.py
@@ -7,7 +7,7 @@
 from argparse import Namespace
 from copy import deepcopy
 from pathlib import Path
-from unittest.mock import patch
+from unittest.mock import patch, call, ANY
 
 import cloudpickle
 import pytest
@@ -1127,3 +1127,24 @@ def setup(self, stage):
     trainer.test(ckpt_path=None)
     assert trainer.stage == 'test'
     assert trainer.get_model().stage == 'test'
+
+
+@pytest.mark.parametrize("train_batches, max_steps, log_interval", [
+    pytest.param(10, 10, 1),
+    pytest.param(3, 10, 1),
+    pytest.param(3, 10, 5),
+])
+@patch("pytorch_lightning.loggers.tensorboard.TensorBoardLogger.log_metrics")
+def test_row_log_interval(log_metrics_mock, tmpdir, train_batches, max_steps, log_interval):
+    model = EvalModelTemplate()
+    trainer = Trainer(
+        default_root_dir=tmpdir,
+        row_log_interval=log_interval,
+        log_save_interval=log_interval,
+        limit_train_batches=train_batches,
+        limit_val_batches=0,
+        max_steps=max_steps,
+    )
+    trainer.fit(model)
+    expected_calls = [call(metrics=ANY, step=s) for s in range(log_interval - 1, max_steps, log_interval)]
+    log_metrics_mock.assert_has_calls(expected_calls)