internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: Lightning-AI/pytorch-lightning

Issue:
Metrics error due to inplace operation, "computation has been modified by an inplace operation". 
Hey, @williamFalcon, I got a new error since I upgraded the library today.
I used the accuracy metric, but got an error.

### Code sample:
```
# in lightning module
def training_step(self, batch, batch_idx):
        x, y = batch
        y_hat = self(x)
        loss = F.cross_entropy(y_hat, y)
        acc = accuracy(y_hat, y)      # from the functional metric classification
        tensorboard_logs = {'train_loss': loss}
        return {'loss': loss, 'log': tensorboard_logs}
```


### Error msg:
```
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.LongTensor [32]] is at version 2; expected version 1 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).
```

### Can be *solved* using `.clone()` method.
However, when I clone the `y` before feeding to the accuracy function, no error was shown.
```
acc = accuracy(y_hat, y.clone())
````
But, it's inconvenience if user has to do it manually, isn't it? 
Actually, I can use the code above without `clone` before I upgrade to the latest. So, it might due to the latest update/rebase causing this error.

> The same error shown for `f1_score` metric.


Discussion:

Resolving patch:
diff --git a/pytorch_lightning/metrics/functional/classification.py b/pytorch_lightning/metrics/functional/classification.py
--- a/pytorch_lightning/metrics/functional/classification.py
+++ b/pytorch_lightning/metrics/functional/classification.py
@@ -181,9 +181,9 @@ def stat_scores_multiple_classes(
     num_classes = get_num_classes(pred=pred, target=target, num_classes=num_classes)
 
     if pred.dtype != torch.bool:
-        pred.clamp_max_(max=num_classes)
+        pred = pred.clamp_max(max=num_classes)
     if target.dtype != torch.bool:
-        target.clamp_max_(max=num_classes)
+        target = target.clamp_max(max=num_classes)
 
     possible_reductions = ('none', 'sum', 'elementwise_mean')
     if reduction not in possible_reductions:


Test patch:
Repository: Lightning-AI/pytorch-lightning

Issue:
Using IterableDatasets without __len__ for Training
Calling `fit(model, trainloader, evalloader)` internally calls `enforce_datamodule_dataloader_override`. This function 
has the if statement `if (train_dataloader or val_dataloaders) and datamodule:`. https://github.com/PyTorchLightning/pytorch-lightning/blob/2c935d048e69a2890889dea768ecceb0252cf321/pytorch_lightning/trainer/configuration_validator.py#L13

This is similar to the PR #1560, the problem is that the `if(dl)` translates to `if(bool(dl))`, but there's no dataloader.__bool__ so bool() uses dataloader.__len__ > 0. But... dataloader.__len__ uses IterableDataset.__len__ for IterableDatasets for which __len__ is undefined.

The fix is also the same, the `if dl` should be replaced by `if dl is not None`.

I will open a PR fixing this.


Discussion:
Hi! thanks for your contribution!, great first issue!

Resolving patch:
diff --git a/pytorch_lightning/trainer/configuration_validator.py b/pytorch_lightning/trainer/configuration_validator.py
--- a/pytorch_lightning/trainer/configuration_validator.py
+++ b/pytorch_lightning/trainer/configuration_validator.py
@@ -10,7 +10,7 @@ def __init__(self, trainer):
 
     def enforce_datamodule_dataloader_override(self, train_dataloader, val_dataloaders, datamodule):
         # If you supply a datamodule you can't supply train_dataloader or val_dataloaders
-        if (train_dataloader or val_dataloaders) and datamodule:
+        if (train_dataloader is not None or val_dataloaders is not None) and datamodule is not None:
             raise MisconfigurationException(
                 'You cannot pass train_dataloader or val_dataloaders to trainer.fit if you supply a datamodule'
             )


Test patch:
Repository: Lightning-AI/pytorch-lightning

Issue:
Document fixes: early_stop_checkpoint is not an valid argument for pl.Trainer
## 📚 Documentation

In `pytorch-lightning/docs/source/new-project.rst`, there is an example demonstrating how to use `Trainer`. However, `early_stop_checkpoint` is not a valid argument for `Trainer`. I also found that `early_stop_checkpoint` only appears here in the whole repo. Therefore, we might need to remove it from the doc.





Discussion:

Resolving patch:
diff --git a/pytorch_lightning/trainer/__init__.py b/pytorch_lightning/trainer/__init__.py
--- a/pytorch_lightning/trainer/__init__.py
+++ b/pytorch_lightning/trainer/__init__.py
@@ -903,15 +903,16 @@ def on_train_end(self, trainer, pl_module):
 amp_backend
 ^^^^^^^^^^^
 
-Define a preferable mixed precision, either NVIDIA Apex ("apex") or PyTorch built-in ("native") AMP which is supported from v1.6.
+Define a preferable mixed precision, either PyTorch built-in ("native") AMP,
+which is supported from v1.6, or NVIDIA Apex ("apex").
 
 .. testcode::
 
+    # using PyTorch built-in AMP, default used by the Trainer
+    trainer = Trainer(amp_backend='native')
+
     # using NVIDIA Apex
     trainer = Trainer(amp_backend='apex')
-    
-    # using PyTorch built-in AMP
-    trainer = Trainer(amp_backend='native')
 
 val_percent_check
 ^^^^^^^^^^^^^^^^^
diff --git a/pytorch_lightning/trainer/trainer.py b/pytorch_lightning/trainer/trainer.py
--- a/pytorch_lightning/trainer/trainer.py
+++ b/pytorch_lightning/trainer/trainer.py
@@ -310,6 +310,7 @@ def __init__(
                     Can be remote file paths such as `s3://mybucket/path` or 'hdfs://path/'
                     Defaults to `default_root_dir`.
 
+            amp_backend: The mixed precision backend to use ("native" or "apex")
 
             amp_level: The optimization level to use (O1, O2, etc...).
 


Test patch:
Repository: Lightning-AI/pytorch-lightning

Issue:
AttributeError: 'NoneType' object has no attribute 'best_model_path' when `checkpoint_callback` = False
## 🐛 Bug

Model does not complete training on TPU and error's out with error `AttributeError: 'NoneType' object has no attribute 'best_model_path'` when `checkpoint_callback=False`.

### To Reproduce

Steps to reproduce the behavior:
```python
trainer=Trainer(tpu_cores=8, max_epochs=5, checkpoint_callback=False, distributed_backend='tpu')
trainer.fit(model)  
```
<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->

```
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-6-6de8b6915beb> in <module>
      1 trainer = Trainer(tpu_cores=8, precision=16, max_epochs=5, checkpoint_callback=False, distributed_backend='tpu')
----> 2 trainer.fit(model)

/opt/conda/lib/python3.7/site-packages/pytorch_lightning/trainer/states.py in wrapped_fn(self, *args, **kwargs)
     32             if entering is not None:
     33                 self.state = entering
---> 34             result = fn(self, *args, **kwargs)
     35 
     36             # The INTERRUPTED state can be set inside the run function. To indicate that run was interrupted

/opt/conda/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py in fit(self, model, train_dataloader, val_dataloaders, datamodule)
   1058             self.accelerator_backend.setup()
   1059             self.accelerator_backend.train(model)
-> 1060             self.accelerator_backend.teardown(model)
   1061 
   1062         else:

/opt/conda/lib/python3.7/site-packages/pytorch_lightning/accelerators/tpu_backend.py in teardown(self, model)
     60 
     61         # transfer back the best path to the trainer
---> 62         self.trainer.checkpoint_callback.best_model_path = best_path
     63         # todo, pass also bets score
     64 

AttributeError: 'NoneType' object has no attribute 'best_model_path'
```

### Expected behavior

Should complete training.

### Additional context

Seems similar to https://github.com/PyTorchLightning/pytorch-lightning/issues/2547



Discussion:

Resolving patch:
diff --git a/pytorch_lightning/accelerators/tpu_backend.py b/pytorch_lightning/accelerators/tpu_backend.py
--- a/pytorch_lightning/accelerators/tpu_backend.py
+++ b/pytorch_lightning/accelerators/tpu_backend.py
@@ -67,7 +67,8 @@ def teardown(self):
         last_path = self.mp_queue.get()
 
         # transfer back the best path to the trainer
-        self.trainer.checkpoint_callback.best_model_path = best_path
+        if self.trainer.checkpoint_callback is not None:
+            self.trainer.checkpoint_callback.best_model_path = best_path
         # todo, pass also bets score
 
         # load last weights


Test patch:
Repository: Lightning-AI/pytorch-lightning

Issue:
ddp_backend in 0.9.0rc12 fails if no CUDA_VISIBLE_DEVICES found
## 🐛 Bug
In ddp_backend, training immediately fails if the environment variable CUDA_VISIBLE_DEVICES isn't set. This line should handle the None case gracefully: https://github.com/PyTorchLightning/pytorch-lightning/blob/master/pytorch_lightning/accelerators/ddp_backend.py#L90

### To Reproduce
Start a run using ddp on CPU. This was discovered using torchelastic to launch


#### Code sample
<!-- Ideally attach a minimal code sample to reproduce the decried issue. 
Minimal means having the shortest code but still preserving the bug. -->

### Expected behavior
This shouldn't crash if the environment variable isn't set. We could default to `num_gpus = 0` in this case. 
Replacing the line above with something like this could work:

`num_gpus = os.environ.get('CUDA_VISIBLE_DEVICES', []).split(',').__len__()
`



Discussion:
This is a bug in our usage of distributed backend, not an issue with lightning
@ananthsub good catch, mind send a PR?
@Borda @awaelchli  reading more about the accelerator changes, I am confused about the split between ddp and ddp_cpu backends. For example, if the user wants to launch training with torchelastic or a non-slurm launcher, then we use the ddp backend. This will fail if the training is on CPUs. The docs aren't updated with the latest changes either: https://pytorch-lightning.readthedocs.io/en/latest/multi_gpu.html#distributed-modes
the `ddp` is GPU based compare to `ddp_cpu` is CPU only...
@Borda in our test case, we launch with these trainer parameters: 
`{'max_epochs': 25, 'gpus': 0, 'num_nodes': 1, 'distributed_backend': 'ddp', 'default_root_dir': '/tmp/'}
`
and this is the stacktrace we run into:
```
    trainer.fit(self)
  File "pytorch_lightning/trainer/states.py", line 34, in wrapped_fn
    result = fn(self, *args, **kwargs)
  File "pytorch_lightning/trainer/trainer.py", line 1030, in fit
    results = self.accelerator_backend.spawn_ddp_children(model)
  File "pytorch_lightning/accelerators/ddp_backend.py", line 90, in spawn_ddp_children
    num_gpus = os.environ['CUDA_VISIBLE_DEVICES'].split(',').__len__()
  File "/usr/lib/python3.7/os.py", line 679, in __getitem__
    raise KeyError(key) from None
KeyError: 'CUDA_VISIBLE_DEVICES'
```
Should this be detected by the config validator earlier? 

Resolving patch:
diff --git a/pytorch_lightning/accelerators/ddp_backend.py b/pytorch_lightning/accelerators/ddp_backend.py
--- a/pytorch_lightning/accelerators/ddp_backend.py
+++ b/pytorch_lightning/accelerators/ddp_backend.py
@@ -87,7 +87,7 @@ def spawn_ddp_children(self, model):
         command = [sys.executable] + command
 
         # since this script sets the visible devices we replace the gpus flag with a number
-        num_gpus = os.environ['CUDA_VISIBLE_DEVICES'].split(',').__len__()
+        num_gpus = os.environ.get('CUDA_VISIBLE_DEVICES', []).split(',').__len__()
 
         if '--gpus' in command:
             gpu_flag_idx = command.index('--gpus')


Test patch:
Repository: Lightning-AI/pytorch-lightning

Issue:
Issue with pl.Trainer.from_argparse_args(...)
<!-- 
### Common bugs:
1. Tensorboard not showing in Jupyter-notebook see [issue 79](https://github.com/PyTorchLightning/pytorch-lightning/issues/79).    
2. PyTorch 1.1.0 vs 1.2.0 support [see FAQ](https://github.com/PyTorchLightning/pytorch-lightning#faq)    
-->

## 🐛 Bug

<!-- A clear and concise description of what the bug is. -->

### To Reproduce

Steps to reproduce the behavior:

1. Use `parser = pl.Trainer.add_argparse_args(parser)`
2. Run `python main.py --overfit_batches 1` 
3. The training runs over the whole dataset instead of running on a single batch
<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->

![Screenshot from 2020-08-13 13-45-55](https://user-images.githubusercontent.com/8587189/90109358-6ab97680-dd6b-11ea-92d9-f1d47aea8435.png)


#### Code sample
<!-- Ideally attach a minimal code sample to reproduce the decried issue. 
Minimal means having the shortest code but still preserving the bug. -->

### Expected behavior
Only one batch should have run.

<!-- A clear and concise description of what you expected to happen. -->

### Environment

* CUDA:
	- GPU:
		- Tesla P100-PCIE-16GB
	- available:         True
	- version:           10.1
* Packages:
	- numpy:             1.18.5
	- pyTorch_debug:     False
	- pyTorch_version:   1.6.0+cu101
	- pytorch-lightning: 0.8.5
	- tensorboard:       2.3.0
	- tqdm:              4.41.1
* System:
	- OS:                Linux
	- architecture:
		- 64bit
		- 
	- processor:         x86_64
	- python:            3.6.9
	- version:           #1 SMP Thu Jul 23 08:00:38 PDT 2020
### Additional context

<!-- Add any other context about the problem here. -->



Discussion:
thanks. looking at it

Resolving patch:
diff --git a/pytorch_lightning/core/datamodule.py b/pytorch_lightning/core/datamodule.py
--- a/pytorch_lightning/core/datamodule.py
+++ b/pytorch_lightning/core/datamodule.py
@@ -317,7 +317,7 @@ def add_argparse_args(cls, parent_parser: ArgumentParser) -> ArgumentParser:
         depr_arg_names = blacklist + added_args
         depr_arg_names = set(depr_arg_names)
 
-        allowed_types = (str, float, int, bool)
+        allowed_types = (str, int, float, bool)
 
         # TODO: get "help" from docstring :)
         for arg, arg_types, arg_default in (
diff --git a/pytorch_lightning/trainer/trainer.py b/pytorch_lightning/trainer/trainer.py
--- a/pytorch_lightning/trainer/trainer.py
+++ b/pytorch_lightning/trainer/trainer.py
@@ -736,7 +736,7 @@ def add_argparse_args(cls, parent_parser: ArgumentParser) -> ArgumentParser:
         blacklist = ['kwargs']
         depr_arg_names = cls.get_deprecated_arg_names() + blacklist
 
-        allowed_types = (str, float, int, bool)
+        allowed_types = (str, int, float, bool)
 
         # TODO: get "help" from docstring :)
         for arg, arg_types, arg_default in (


Test patch:
Repository: Lightning-AI/pytorch-lightning

Issue:
Fixes #2942



Discussion:
# [Codecov](https://codecov.io/gh/PyTorchLightning/pytorch-lightning/pull/2969?src=pr&el=h1) Report
> Merging [#2969](https://codecov.io/gh/PyTorchLightning/pytorch-lightning/pull/2969?src=pr&el=desc) into [master](https://codecov.io/gh/PyTorchLightning/pytorch-lightning/commit/639a4cbd250e74217bf86f1f122db54151dd9652&el=desc) will **increase** coverage by `4%`.
> The diff coverage is `0%`.


```diff
@@           Coverage Diff           @@
##           master   #2969    +/-   ##
=======================================
+ Coverage      86%     90%    +4%     
=======================================
  Files          81      81            
  Lines        7554    7554            
=======================================
+ Hits         6497    6798   +301     
+ Misses       1057     756   -301     
```


Resolving patch:
diff --git a/pytorch_lightning/accelerators/ddp_backend.py b/pytorch_lightning/accelerators/ddp_backend.py
--- a/pytorch_lightning/accelerators/ddp_backend.py
+++ b/pytorch_lightning/accelerators/ddp_backend.py
@@ -87,7 +87,7 @@ def spawn_ddp_children(self, model):
         command = [sys.executable] + command
 
         # since this script sets the visible devices we replace the gpus flag with a number
-        num_gpus = os.environ.get('CUDA_VISIBLE_DEVICES', []).split(',').__len__()
+        num_gpus = torch.cuda.device_count()
 
         if '--gpus' in command:
             gpu_flag_idx = command.index('--gpus')


Test patch:
Repository: Lightning-AI/pytorch-lightning

Issue:
ModelCheckpoint instance to JSON
## 🚀 Feature

Have a `to_json(path)` function that writes the `best_k_models` to a json file in `path`.

### Motivation

Why having a JSON is good? 
Then you don't need to load the checkpoints to know the metrics and things like a posteriori stochastic weight averaging gets  easier. 

It used to be easier in 0.6, but loss values are now tensors, so it's not directly serializable.
This is currently needed: 
```python
best_k = {k: v.item() for k, v in checkpoint.best_k_models.items()}
with open(json_path, "w") as f:
    json.dump(best_k, f, indent=0)
```

Instead, we could have 
```python
checkpoint.to_json(json_path)
```

I'm ready to make a PR for it. Just tell me if this would be useful for you. 


Discussion:

Resolving patch:
diff --git a/pytorch_lightning/callbacks/model_checkpoint.py b/pytorch_lightning/callbacks/model_checkpoint.py
--- a/pytorch_lightning/callbacks/model_checkpoint.py
+++ b/pytorch_lightning/callbacks/model_checkpoint.py
@@ -22,8 +22,10 @@
 
 import os
 import re
+import yaml
 from copy import deepcopy
-from typing import Any, Dict, Optional
+from typing import Any, Dict, Optional, Union
+from pathlib import Path
 
 import numpy as np
 import torch
@@ -531,3 +533,13 @@ def _do_check_save(
             if cur_path != filepath:
                 self._del_model(cur_path)
 
+    def to_yaml(self, filepath: Optional[Union[str, Path]] = None):
+        """
+        Saves the `best_k_models` dict containing the checkpoint
+        paths with the corresponding scores to a YAML file.
+        """
+        best_k = {k: v.item() for k, v in self.best_k_models.items()}
+        if filepath is None:
+            filepath = os.path.join(self.dirpath, "best_k_models.yaml")
+        with open(filepath, "w") as fp:
+            yaml.dump(best_k, fp)


Test patch:
diff --git a/tests/callbacks/test_model_checkpoint.py b/tests/callbacks/test_model_checkpoint.py
--- a/tests/callbacks/test_model_checkpoint.py
+++ b/tests/callbacks/test_model_checkpoint.py
@@ -1,4 +1,5 @@
 import os
+import yaml
 import pickle
 import platform
 import re
@@ -36,6 +37,24 @@ def test_model_checkpoint_with_non_string_input(tmpdir, save_top_k):
     )
 
 
+@pytest.mark.parametrize('save_top_k', [-1, 0, 1, 2])
+def test_model_checkpoint_to_yaml(tmpdir, save_top_k):
+    """ Test that None in checkpoint callback is valid and that chkp_path is set correctly """
+    tutils.reset_seed()
+    model = EvalModelTemplate()
+
+    checkpoint = ModelCheckpoint(filepath=tmpdir, monitor='val_loss', save_top_k=save_top_k)
+
+    trainer = Trainer(default_root_dir=tmpdir, checkpoint_callback=checkpoint, overfit_batches=0.20, max_epochs=2)
+    trainer.fit(model)
+
+    path_yaml = os.path.join(tmpdir, 'best_k_models.yaml')
+    checkpoint.to_yaml(path_yaml)
+    d = yaml.full_load(open(path_yaml, 'r'))
+    best_k = {k: v.item() for k, v in checkpoint.best_k_models.items()}
+    assert d == best_k
+
+
 @pytest.mark.parametrize(
     "logger_version,expected",
     [(None, "version_0"), (1, "version_1"), ("awesome", "awesome")],
Repository: Lightning-AI/pytorch-lightning

Issue:
Save checkpoint on Ctrl+C
I understand that pl handles cli ctrl+c with graceful degradation. I also see the ability to save checkpoints which is triggered by metrics queries. Is there is an idiomatic way to have pl save a checkpoint when I ctrl+c?


Discussion:
Hi! thanks for your contribution!, great first issue!
we should add a callback, on_ctrl_c or something 
no need, it is already the on on_interupr
@nkyriazis mind send a PR with saving in `on_interupr`
Hi I am a new contributor. I would like to work on this issue if no one else is working on it. I could not locate the `on_interupr` utility on the master branch. Am I missing something?
go for it!

why not just support ctrl+s?  isn’t this a more user friendly approach haha?

or do you also mean save a checkpoint on interrupt?
Saving a checkpoint on ctrl+s is also not a bad idea.
Terminate program without saving checkpoint : ctrl+c
Terminate program and save latest checkpoint : ctrl+s
Does this sound good?
i was thinking more like ctrl+s saves a checkpoint at that time. 

then separately if you want to finish training ctrl+c. 

so, the only new change is to add a checkpoint save call on ctrl+s
Wouldn't this be an issue if the program is in middle of a training epoch. Generally we would like to save checkpoints at the end of an epoch right?
@williamFalcon I'm not sure this is possible

actually I don't think so, because python does not raise an Exception on ctrl+s. So there's nothing we can actually hook into, since all inputs, that are not caught by pythons io or raise an exception are postponed until the program is finished afaik.

So probably we need (and imo also should) go with ctrl+c

Resolving patch:
diff --git a/pytorch_lightning/trainer/training_loop.py b/pytorch_lightning/trainer/training_loop.py
--- a/pytorch_lightning/trainer/training_loop.py
+++ b/pytorch_lightning/trainer/training_loop.py
@@ -1132,6 +1132,10 @@ def run_training_teardown(self):
 
         self._teardown_already_run = True
 
+        # Save latest checkpoint
+        log.info('Saving latest checkpoint..')
+        self.check_checkpoint_callback(should_check_val=False)
+
         # Train end events
         with self.profiler.profile('on_train_end'):
             # callbacks


Test patch: