logging loss bug when accumulate_grad_batches >= 1
🐛 Bug

When  accumulate_grad_batches >= 1, logging loss is divided by accumulate_grad_batches.
For example, when accumulate_grad_batches=1, logging loss is 4; when accumulate_grad_batches=2, loss is 2; accumulate_grad_batches=4, loss is 1
Expected behavior

logging loss should be similar no matter accumulate_grad_batch is any number.
Environment

PyTorch Version (e.g., 1.0): nightly
OS (e.g., Linux): Linux
How you installed PyTorch (conda, pip, source): pip
Build command you used (if compiling from source):
Python version: 3.6
CUDA/cuDNN version: 10.2
GPU models and configuration:
Any other relevant information:

Additional context