Repository: Lightning-AI/pytorch-lightning
Pull request: advanced profiler describe + cleaned up tests
# Before submitting

- [x] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [x] Did you read the [contributor guideline](https://github.com/PyTorchLightning/pytorch-lightning/blob/master/.github/CONTRIBUTING.md)?
- [x] Did you make sure to update the docs?   
- [x] Did you write any new necessary tests?  

## What does this PR do?
Fixes #836 

## PR review    
Anyone in the community is free to review the PR once the tests have passed.     
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.


Problem statement:
advanced profiler description fails for python 3.6
## 🐛 Bug

Python 3.6 doesn't have the `pstats.SortKey.CUMULATIVE` enum so the profiler description breaks.

### To Reproduce

Steps to reproduce the behavior:

Use Python 3.6, pass in the AdvancedProfiler, get report at end of a training run. 

```
profiler = AdvancedProfiler(line_count_restriction=10)
trainer = Trainer(profiler=profiler)
trainer.fit(model)
```

Stack trace:
```
    164         for action_name, pr in self.profiled_actions.items():
    165             s = io.StringIO()
--> 166             sortby = pstats.SortKey.CUMULATIVE
    167             ps = pstats.Stats(pr, stream=s).strip_dirs().sort_stats(sortby)
    168             ps.print_stats(self.line_count_restriction)

AttributeError: module 'pstats' has no attribute 'SortKey'
```


#### Code sample

```
from pytorch_lightning import Trainer
from pytorch_lightning.profiler import AdvancedProfiler
from argparse import Namespace
from pl_examples.basic_examples.lightning_module_template import LightningTemplateModel


# define model
hparams = {
    "batch_size": 128,
    "in_features": 784,
    "hidden_dim": 512,
    "drop_prob": 0.0,
    "out_features": 10,
    "learning_rate": 5e-3,
    "data_root": "data"
}
hparams = Namespace(**hparams)
model = LightningTemplateModel(hparams)

# overfit on small batch
profiler = AdvancedProfiler(line_count_restriction=10)
trainer = Trainer(profiler=profiler, overfit_pct=0.05, min_epochs=10)
trainer.fit(model)
```

### Expected behavior

<!-- A clear and concise description of what you expected to happen. -->

### Environment

Collecting environment information...
PyTorch version: 1.4.0
Is debug build: No
CUDA used to build PyTorch: 10.1

OS: Ubuntu 18.04.3 LTS
GCC version: (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
CMake version: version 3.12.0

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 10.0.130
GPU models and configuration: GPU 0: Tesla P100-PCIE-16GB
Nvidia driver version: 418.67
cuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5

Versions of relevant libraries:
[pip3] numpy==1.17.5
[pip3] pytorch-lightning==0.6.1.dev0
[pip3] torch==1.4.0
[pip3] torchsummary==1.5.1
[pip3] torchtext==0.3.1
[pip3] torchvision==0.5.0
[conda] Could not collect

Hints from issue discussion:

Commit message: add test case to capture previous bug

Patch to review:
diff --git a/pytorch_lightning/profiler/profiler.py b/pytorch_lightning/profiler/profiler.py
index 32f220897a9dc..ffecba5dff34b 100644
--- a/pytorch_lightning/profiler/profiler.py
+++ b/pytorch_lightning/profiler/profiler.py
@@ -163,8 +163,7 @@ def describe(self):
         self.recorded_stats = {}
         for action_name, pr in self.profiled_actions.items():
             s = io.StringIO()
-            sortby = pstats.SortKey.CUMULATIVE
-            ps = pstats.Stats(pr, stream=s).strip_dirs().sort_stats(sortby)
+            ps = pstats.Stats(pr, stream=s).strip_dirs().sort_stats('cumulative')
             ps.print_stats(self.line_count_restriction)
             self.recorded_stats[action_name] = s.getvalue()
         if self.output_filename is not None:
diff --git a/tests/test_profiler.py b/tests/test_profiler.py
index 1b26030de8ea4..b773979080fc8 100644
--- a/tests/test_profiler.py
+++ b/tests/test_profiler.py
@@ -26,6 +26,9 @@ def test_simple_profiler():
     np.testing.assert_allclose(p.recorded_durations["b"], [2], rtol=0.2)
     np.testing.assert_allclose(p.recorded_durations["c"], [1], rtol=0.2)
 
+    # ensure this doesn't throw any errors
+    description = p.describe()
+
 
 def test_advanced_profiler():
     def _get_duration(profile):
@@ -53,3 +56,6 @@ def _get_duration(profile):
     np.testing.assert_allclose(b_duration, [2], rtol=0.2)
     c_duration = _get_duration(p.profiled_actions["c"])
     np.testing.assert_allclose(c_duration, [1], rtol=0.2)
+
+    # ensure this doesn't throw any errors
+    description = p.describe()


Code review comments:
diff_hunk: @@ -26,6 +26,9 @@ def test_simple_profiler():
     np.testing.assert_allclose(p.recorded_durations["b"], [2], rtol=0.2)
     np.testing.assert_allclose(p.recorded_durations["c"], [1], rtol=0.2)
 
+    # ensure this doesn't throw any errors
+    description = p.describe()
path: tests/test_profiler.py
text: @user1:
I do not see `description` to be used...

@author:
yes i’m just making sure that it can successfully call the method without throwing an error. might be worth moving that line to a new test case `test_simple_profiler_describe()`

@user1:
you may just drop output `_ = p.describe()` or just `p.describe()`
I would say that single letter variable is not good practice... :]

@author:
agreed, i'll clean the tests up a bit