Repository: pytorch
Issue: KeyError: 'track_running_stats' in batchnorm.extra_repr
Labels: module: printing,module: serialization,triaged

## 🐛 Bug

<!-- A clear and concise description of what the bug is. -->

## To Reproduce

Steps to reproduce the behavior:



1. `import torch`
2. `state = torch.load("./rot-000003456-020897.pt")   #  checkpoints can be found at here https://github.com/NVlabs/ocrorot`
3. `print(state)`

<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->

## Expected behavior

<!-- A clear and concise description of what you expected to happen. -->

```
Out[2]: ------------------------------------------------------------------------                                                                                             
KeyError                               Traceback (most recent call last)                                                                                                     
~/anaconda3/envs/rnd/lib/python3.6/site-packages/IPython/core/formatters.py in __call__(self, obj)                                                                           
    700                 type_pprinters=self.type_printers,                                                                                                                   
    701                 deferred_pprinters=self.deferred_printers)                                                                                                           
--> 702             printer.pretty(obj)                                                                                                                                      
    703             printer.flush()                                                                                                                                          
    704             return stream.getvalue()                                                                                                                                 
                                                                                                                                                                             
~/anaconda3/envs/rnd/lib/python3.6/site-packages/IPython/lib/pretty.py in pretty(self, obj)                                                                                  
    392                         if cls is not object \                                                                                                                       
    393                                 and callable(cls.__dict__.get('__repr__')):                                                                                          
--> 394                             return _repr_pprint(obj, self, cycle)                                                                                                    
    395                                                                                                                                                                      
    396             return _default_pprint(obj, self, cycle)                                                                                                                 
                                                                                                                                                                             
~/anaconda3/envs/rnd/lib/python3.6/site-packages/IPython/lib/pretty.py in _repr_pprint(obj, p, cycle)                                                                        
    698     """A pprint that just redirects to the normal repr function."""                                                                                                  
    699     # Find newlines and replace them with p.break_()                                                                                                                 
--> 700     output = repr(obj)                                                                                                                                               
    701     lines = output.splitlines()
    702     with p.group():

~/anaconda3/envs/rnd/lib/python3.6/site-packages/torch/nn/modules/module.py in __repr__(self)
   1345         child_lines = []
   1346         for key, module in self._modules.items():
-> 1347             mod_str = repr(module)
   1348             mod_str = _addindent(mod_str, 2)
   1349             child_lines.append('(' + key + '): ' + mod_str)

~/anaconda3/envs/rnd/lib/python3.6/site-packages/torch/nn/modules/module.py in __repr__(self)
   1339         # We treat the extra repr like the sub-module, one item per line
   1340         extra_lines = []
-> 1341         extra_repr = self.extra_repr()
   1342         # empty string will be split into list ['']
   1343         if extra_repr:

~/anaconda3/envs/rnd/lib/python3.6/site-packages/torch/nn/modules/batchnorm.py in extra_repr(self)
     72     def extra_repr(self):
     73         return '{num_features}, eps={eps}, momentum={momentum}, affine={affine}, ' \
---> 74                'track_running_stats={track_running_stats}'.format(**self.__dict__)
     75
     76     def _load_from_state_dict(self, state_dict, prefix, local_metadata, strict,

KeyError: 'track_running_stats'
```
## Environment

Please copy and paste the output from our
[environment collection script](https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py)
(or fill out the checklist below manually).

You can get the script and run it with:
```
wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py
```

(rnd) z@z:~/research/ocrorot$ python collect_env.py
Collecting environment information...
PyTorch version: 1.6.0
Is debug build: No
CUDA used to build PyTorch: 10.2

OS: Ubuntu 18.04.4 LTS
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
CMake version: version 3.10.2

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 10.2.89
GPU models and configuration: GPU 0: GeForce GTX 1050 Ti
Nvidia driver version: 450.51.05
cuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.1.4

Versions of relevant libraries:
[pip3] numpy==1.19.1
[pip3] torch==1.6.0
[pip3] torchvision==0.7.0
[conda] numpy                     1.19.1                    <pip>
[conda] torch                     1.6.0                     <pip>
[conda] torchvision               0.7.0                     <pip>


## Additional context

<!-- Add any other context about the problem here. -->