internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: googleapis/python-spanner-django

Issue:
Kokoro build is failing for new PRs with no change in code
Kokoro build is failing for new PRs with no change in code https://github.com/googleapis/python-spanner-django/pull/652


Discussion:

Resolving patch:
diff --git a/version.py b/version.py
--- a/version.py
+++ b/version.py
@@ -4,4 +4,4 @@
 # license that can be found in the LICENSE file or at
 # https://developers.google.com/open-source/licenses/bsd
 
-__version__ = "2.2.1b1"
+__version__ = "2.2.1b2"


Test patch:
diff --git a/django_test_apps.txt b/django_test_apps.txt
--- a/django_test_apps.txt
+++ b/django_test_apps.txt
@@ -1,9 +1,6 @@
-admin_filters
 admin_utils
 custom_lookups
 extra_regress
 expressions_case
-model_regress
-modeladmin
 proxy_models
-serializers
\ No newline at end of file
+serializers
Repository: Lightning-AI/pytorch-lightning

Issue:
Nitpick: `ptl` may be better as `pl`
(Feel free to ignore.)

All the usage examples do `import pytorch_lighting as ptl`. Instead of `ptl`, `pl` may  be better as it doesn't clash with any library I know of, is 2 characters like NumPy's `np`, and is harder to mistype as `plt`, which many researchers probably also have imported. Since the library is in its early days, I don't think it would be that dramatic a change and is a little easier to read for people like me who often mix up letters like that.

On the other hand, it's pretty clear that it's not matplotlib from context, is yet another change, and is an aesthetic choice at its root, so it may not be worth it.


Discussion:
Good point. I forgot about matplotlib (plt) lol. I would for sure mess that up.   

Do you want to submit a PR changing docs and examples from ptl to pl?

Resolving patch:
diff --git a/examples/new_project_templates/lightning_module_template.py b/examples/new_project_templates/lightning_module_template.py
--- a/examples/new_project_templates/lightning_module_template.py
+++ b/examples/new_project_templates/lightning_module_template.py
@@ -13,7 +13,7 @@
 from torch.utils.data import DataLoader
 from torch.utils.data.distributed import DistributedSampler
 
-import pytorch_lightning as ptl
+import pytorch_lightning as pl
 from pytorch_lightning.root_module.root_module import LightningModule
 
 
@@ -195,17 +195,17 @@ def __dataloader(self, train):
 
         return loader
 
-    @ptl.data_loader
+    @pl.data_loader
     def tng_dataloader(self):
         print('tng data loader called')
         return self.__dataloader(train=True)
 
-    @ptl.data_loader
+    @pl.data_loader
     def val_dataloader(self):
         print('val data loader called')
         return self.__dataloader(train=False)
 
-    @ptl.data_loader
+    @pl.data_loader
     def test_dataloader(self):
         print('test data loader called')
         return self.__dataloader(train=False)


Test patch:
diff --git a/tests/debug.py b/tests/debug.py
--- a/tests/debug.py
+++ b/tests/debug.py
@@ -6,14 +6,14 @@
 import os
 import shutil
 
-import pytorch_lightning as ptl
+import pytorch_lightning as pl
 import torch
 from torch.nn import functional as F
 from torch.utils.data import DataLoader
 from torchvision.datasets import MNIST
 
 
-class CoolModel(ptl.LightningModule):
+class CoolModel(pl.LightningModule):
 
     def __init(self):
         super(CoolModel, self).__init__()
@@ -43,15 +43,15 @@ def validation_end(self, outputs):
     def configure_optimizers(self):
         return [torch.optim.Adam(self.parameters(), lr=0.02)]
 
-    @ptl.data_loader
+    @pl.data_loader
     def tng_dataloader(self):
         return DataLoader(MNIST('path/to/save', train=True), batch_size=32)
 
-    @ptl.data_loader
+    @pl.data_loader
     def val_dataloader(self):
         return DataLoader(MNIST('path/to/save', train=False), batch_size=32)
 
-    @ptl.data_loader
+    @pl.data_loader
     def test_dataloader(self):
         return DataLoader(MNIST('path/to/save', train=False), batch_size=32)
Repository: Lightning-AI/pytorch-lightning

Issue:
Incorrect Implementation for Accumulating Batch Gradients in Trainer
Current Behavior:
If accumulate_grad_batches > default of 1, the Trainer will proceed to take the loss from each batch and run loss.backward() for each batch accumulated, running the optimizer.step() once the desired number of batches has undergone backprop.
Loss averaging is only done for batch_loss_value.

Correct Behavior:
The loss from the output needs to be divided by accumulate_grad_batches before loss.backward() is run, otherwise the overall magnitude of the gradient could be up to N times greater for a simulated batch size N times bigger than the actual.


Discussion:

Resolving patch:
diff --git a/pytorch_lightning/models/trainer.py b/pytorch_lightning/models/trainer.py
--- a/pytorch_lightning/models/trainer.py
+++ b/pytorch_lightning/models/trainer.py
@@ -899,6 +899,9 @@ def __run_tng_batch(self, data_batch, batch_nb):
 
         self.__add_tqdm_metrics(model_specific_tqdm_metrics_dic)
 
+        # accumulate loss (if accumulate_grad_batches = 1 no effect)
+        loss = loss / self.accumulate_grad_batches
+
         # backward pass
         if self.use_amp:
             # scale loss when using amp
@@ -918,12 +921,11 @@ def __run_tng_batch(self, data_batch, batch_nb):
             for param in model.parameters():
                 print(param.grad.float().sum())
 
-        # avoid memory leaks
+        # track total loss for logging (avoid mem leaks)
         self.batch_loss_value += loss.item()
 
         # gradient update with accumulated gradients
         if (self.batch_nb + 1) % self.accumulate_grad_batches == 0:
-
             # clip gradients
             if self.gradient_clip > 0:
                 model = self.__get_model()
@@ -941,11 +943,7 @@ def __run_tng_batch(self, data_batch, batch_nb):
                 # clear gradients
                 optimizer.zero_grad()
 
-            # queuing loss across batches blows it up proportionally...
-            #  divide out the number accumulated
-            self.batch_loss_value = self.batch_loss_value / self.accumulate_grad_batches
-
-            # track loss
+            # calculate running loss for display
             self.running_loss.append(self.batch_loss_value)
             self.batch_loss_value = 0
             self.avg_loss = np.mean(self.running_loss[-100:])


Test patch:
Repository: Lightning-AI/pytorch-lightning

Issue:
transfer_to_batch_gpu returns null when input has primitives
**Describe the bug**
when passing a batch such as:   
```python    
batch = list(tensor, tensor, [0,1,2])  
```   
the list of ints won't be returned correctly

**Additional context**
Fix should add a return of the item it no condition matches



Discussion:

Resolving patch:
diff --git a/pytorch_lightning/models/trainer.py b/pytorch_lightning/models/trainer.py
--- a/pytorch_lightning/models/trainer.py
+++ b/pytorch_lightning/models/trainer.py
@@ -921,8 +921,8 @@ def transfer_batch_to_gpu(self, batch, gpu_id):
         if isinstance(batch, torch.Tensor):
             return batch.cuda(gpu_id)
 
-        # when list
-        elif isinstance(batch, list):
+        # when list/tuple
+        elif isinstance(batch, list) or isinstance(batch, tuple):
             for i, x in enumerate(batch):
                 batch[i] = self.transfer_batch_to_gpu(x, gpu_id)
             return batch
@@ -934,6 +934,9 @@ def transfer_batch_to_gpu(self, batch, gpu_id):
 
             return batch
 
+        # nothing matches, return the value as is without transform
+        return batch
+
     def __tng_forward(self, data_batch, batch_nb, opt_idx):
         """
         Handle forward for each training case (distributed, single gpu, etc...)


Test patch:
Repository: Lightning-AI/pytorch-lightning

Issue:
Non plain-tensor batches
Hello!

I have been trying to use Lightning to train a graph neural network built with [torch_geometric](http://github.com/rusty1s/pytorch_geometric/) package using a GPU.

This is the error I get when I try to fit the model:
```python3
~/miniconda3/envs/pyg/lib/python3.7/site-packages/torch_geometric/nn/conv/gcn_conv.py in forward(self, x, edge_index, edge_weight)
     83     def forward(self, x, edge_index, edge_weight=None):
     84         """"""
---> 85         x = torch.matmul(x, self.weight)
     86 
     87         if self.cached and self.cached_result is not None:

RuntimeError: Expected object of backend CPU but got backend CUDA for argument #2 'mat2'
```
I investigated the code of Lightning for a probable cause and found that `transfer_batch_to_gpu` causes this error.

The current behavior of this function considers a batch to be either a 'plain' `torch.Tensor` or some simple collection of such objects (a list, a dict or a tuple). The problem is that torch_geometric uses a custom aggregate type `Data` ([docs](https://pytorch-geometric.readthedocs.io/en/latest/modules/data.html#torch_geometric.data.Data)) which itself implements `to` method (and does not implement `cuda`, I believe).

It would be nice if you could make the code more flexible to process this case (and similar cases in general) correctly. I believe, the best solution is to replace `isinstance(batch, torch.Tensor)` by a condition of `batch` object having either `to` or `cuda` method implemented. This should solve the problem in general.


Discussion:
good suggestion. want to make the change and submit a PR? 
Okay, I will do that.

Resolving patch:
diff --git a/pytorch_lightning/models/trainer.py b/pytorch_lightning/models/trainer.py
--- a/pytorch_lightning/models/trainer.py
+++ b/pytorch_lightning/models/trainer.py
@@ -995,10 +995,13 @@ def __log_vals_blacklist(self):
         return blacklist
 
     def transfer_batch_to_gpu(self, batch, gpu_id):
-        # base case
-        if isinstance(batch, torch.Tensor):
+        # base case: object can be directly moved using `cuda` or `to`
+        if callable(getattr(batch, 'cuda', None)):
             return batch.cuda(gpu_id)
 
+        elif callable(getattr(batch, 'to', None)):
+            return batch.to(torch.device('cuda', gpu_id))
+
         # when list
         elif isinstance(batch, list):
             for i, x in enumerate(batch):


Test patch:
Repository: Lightning-AI/pytorch-lightning

Issue:
Make test_tube optional
**Is your feature request related to a problem? Please describe.**
`test_tube` is only needed when using `TestTubeLogger`, thus it should not be an install requirement like `mlflow`.


Discussion:

Resolving patch:
diff --git a/pytorch_lightning/logging/__init__.py b/pytorch_lightning/logging/__init__.py
--- a/pytorch_lightning/logging/__init__.py
+++ b/pytorch_lightning/logging/__init__.py
@@ -1,6 +1,9 @@
 from .base import LightningLoggerBase, rank_zero_only
-from .test_tube_logger import TestTubeLogger
 
+try:
+    from .test_tube_logger import TestTubeLogger
+except ModuleNotFoundError:
+    pass
 try:
     from .mlflow_logger import MLFlowLogger
 except ModuleNotFoundError:


Test patch:
Repository: Lightning-AI/pytorch-lightning

Issue:
Remove SLURM warning when in interactive mode
No need to show the "we launched processes for you" warning when in interactive SLURM mode.

Suggested by @adefazio 


Discussion:

Resolving patch:
diff --git a/pytorch_lightning/trainer/trainer.py b/pytorch_lightning/trainer/trainer.py
--- a/pytorch_lightning/trainer/trainer.py
+++ b/pytorch_lightning/trainer/trainer.py
@@ -704,14 +704,6 @@ def fit(self, model):
             else:
                 nb_gpus = self.nb_requested_gpus
                 nb_tasks = self.nb_slurm_tasks
-                msg = f"""
-                You requested {nb_gpus}s GPUs but launched {nb_tasks}s slurm tasks.
-                We will launch {nb_gpus}s processes for you.
-                We recommend you let slurm manage the processes by setting:
-                --ntasks-per-node={nb_gpus}s
-                If you're not using SLURM, ignore this message!
-                """
-                warnings.warn(msg)
                 mp.spawn(self.ddp_train, nprocs=self.num_gpus, args=(model, ))
 
         # 1 gpu or dp option triggers training using DP module


Test patch:
Repository: Lightning-AI/pytorch-lightning

Issue:
set MASTER_PORT automatically
when using DDP, use must set a manual MASTER_PORT. However, we should automatically set one.

The problem is that we can't choose a random one as each process might generate a different solution. Instead, I propose to use the SLURM JOB ID as the seed to k possible ports. Then every process can deterministically generate the same sequence of ports. With that list, the root node can init NCCL connection making its way down the list until a port is open.

However, if we choose the length of the job id to use correctly we may potentially not run into collisions and won't need to iterate a list of potential ports.


Discussion:

Resolving patch:
diff --git a/pytorch_lightning/trainer/trainer.py b/pytorch_lightning/trainer/trainer.py
--- a/pytorch_lightning/trainer/trainer.py
+++ b/pytorch_lightning/trainer/trainer.py
@@ -180,6 +180,7 @@ def __init__(self,
         if self.logger is None:
             self.logger = TestTubeLogger(
                 save_dir=self.default_save_path,
+                version=self.slurm_job_id,
                 name='lightning_logs'
             )
 
@@ -240,6 +241,15 @@ def __init__(self,
         self.amp_level = amp_level
         self.__init_amp(use_amp)
 
+    @property
+    def slurm_job_id(self):
+        try:
+            job_id = os.environ['SLURM_JOB_ID']
+            job_id = int(job_id)
+        except Exception as e:
+            job_id = None
+        return job_id
+
     def __configure_weights_path(self, checkpoint_callback, weights_save_path):
         """
         Weight path set in this priority:
@@ -882,12 +892,25 @@ def __init_tcp_connection(self):
         :param tries:
         :return:
         """
-        # sets the appropriate port
+
+        # use slurm job id for the port number
+        # guarantees unique ports across jobs from same grid search
+        try:
+            # use the last 4 numbers in the job id as the id
+            default_port = os.environ['SLURM_JOB_ID']
+            default_port = default_port[-4:]
+
+            # all ports should be in the 10k+ range
+            default_port = int(default_port) + 10000
+
+        except Exception as e:
+            default_port = 12910
+
+        # if user gave a port number, use that one instead
         try:
             port = os.environ['MASTER_PORT']
         except Exception:
-            port = 12910
-            os.environ['MASTER_PORT'] = str(port)
+            os.environ['MASTER_PORT'] = str(default_port)
 
         # figure out the root node addr
         try:


Test patch:
Repository: Lightning-AI/pytorch-lightning

Issue:
Pass experiment tags to MLFlowLogger
**Is your feature request related to a problem? Please describe.**
When using MLFlowLogger, I'm unable to easily set experiment tags, like username or run name.

**Describe the solution you'd like**
Add parameter `tags=None` which is passed to `MLFlowLogger`. Tags will be passed to `create_run` method

**Describe alternatives you've considered**
Manually hack logger, get experiment from it and set tag there

If you don't see any drawbacks, I can make a PR



Discussion:
Sounds like a great idea. Want to take a stab at a PR?

Resolving patch:
diff --git a/pytorch_lightning/logging/mlflow_logger.py b/pytorch_lightning/logging/mlflow_logger.py
--- a/pytorch_lightning/logging/mlflow_logger.py
+++ b/pytorch_lightning/logging/mlflow_logger.py
@@ -9,11 +9,12 @@
 
 
 class MLFlowLogger(LightningLoggerBase):
-    def __init__(self, experiment_name, tracking_uri=None):
+    def __init__(self, experiment_name, tracking_uri=None, tags=None):
         super().__init__()
         self.client = mlflow.tracking.MlflowClient(tracking_uri)
         self.experiment_name = experiment_name
         self._run_id = None
+        self.tags = tags
 
     @property
     def run_id(self):
@@ -28,7 +29,7 @@ def run_id(self):
             self.client.create_experiment(self.experiment_name)
             experiment = self.client.get_experiment_by_name(self.experiment_name)
 
-        run = self.client.create_run(experiment.experiment_id)
+        run = self.client.create_run(experiment.experiment_id, tags=self.tags)
         self._run_id = run.info.run_id
         return self._run_id
 


Test patch:
Repository: Lightning-AI/pytorch-lightning

Issue:
Experiment versions should start with 000 for easier listing
Instead of ```/dir/version_9``` it should be ```/dir/version_0009```   

Suggested by @adefazio


Discussion:

Resolving patch:
diff --git a/pytorch_lightning/trainer/trainer.py b/pytorch_lightning/trainer/trainer.py
--- a/pytorch_lightning/trainer/trainer.py
+++ b/pytorch_lightning/trainer/trainer.py
@@ -260,7 +260,7 @@ def __configure_checkpoint_callback(self):
         if self.checkpoint_callback is None:
             # init a default one
             if isinstance(self.logger, TestTubeLogger):
-                ckpt_path = '{}/{}/version_{}/{}'.format(
+                ckpt_path = '{}/{}/version_{:04d}/{}'.format(
                     self.default_save_path,
                     self.logger.experiment.name,
                     self.logger.experiment.version,


Test patch:
Repository: Lightning-AI/pytorch-lightning

Issue:
Error when model checkpoint and no early stop
**Describe the bug**
When creating a `Trainer`, if we set a `checkpoint_callback=ModelCheckpoint(...)` and `early_stop_callback = False` we get an error at this [line](https://github.com/williamFalcon/pytorch-lightning/blob/master/pytorch_lightning/trainer/trainer_io.py#L157) here. 

**To Reproduce**
Steps to reproduce the behavior:
1. Create a `ckpt = ModelCheckpoint(...)`
2. Create a `Trainer`
3. Setting `checkpoint_callback = ckpt` and `early_stop_callback=False`
4. See the error `AttributeError: 'NoneType' object has no attribute 'wait'`

**Expected behavior**
It should be possible to save the model without setting an EarlyStopping condition. Of course one could set an EarlyStopping with the max integer, but changing the condition from an `or` to an `and` solves the problem. 

**Desktop**
 - OS: Ubuntu 19.04
 - Browser: Firefox Quantum
 - Version: 69.0.2


Discussion:

Resolving patch:
diff --git a/pytorch_lightning/trainer/trainer_io.py b/pytorch_lightning/trainer/trainer_io.py
--- a/pytorch_lightning/trainer/trainer_io.py
+++ b/pytorch_lightning/trainer/trainer_io.py
@@ -151,10 +151,10 @@ def dump_checkpoint(self):
             'global_step': self.global_step
         }
 
-        if self.checkpoint_callback is not None or self.checkpoint_callback is not False:
+        if self.checkpoint_callback is not None and self.checkpoint_callback is not False:
             checkpoint['checkpoint_callback_best'] = self.checkpoint_callback.best
 
-        if self.early_stop_callback is not None or self.checkpoint_callback is not False:
+        if self.early_stop_callback is not None and self.checkpoint_callback is not False:
             checkpoint['early_stop_callback_wait'] = self.early_stop_callback.wait
             checkpoint['early_stop_callback_patience'] = self.early_stop_callback.patience
 
@@ -207,10 +207,10 @@ def restore_training_state(self, checkpoint):
         :param checkpoint:
         :return:
         """
-        if self.checkpoint_callback is not None or self.checkpoint_callback is not False:
+        if self.checkpoint_callback is not None and self.checkpoint_callback is not False:
             self.checkpoint_callback.best = checkpoint['checkpoint_callback_best']
 
-        if self.early_stop_callback is not None or self.early_stop_callback is not False:
+        if self.early_stop_callback is not None and self.early_stop_callback is not False:
             self.early_stop_callback.wait = checkpoint['early_stop_callback_wait']
             self.early_stop_callback.patience = checkpoint['early_stop_callback_patience']
 


Test patch:
Repository: Lightning-AI/pytorch-lightning

Issue:
Fix readme for Pypi
@Borda These lines failed to upload. Commented out for this release:
https://github.com/williamFalcon/pytorch-lightning/blob/master/setup.py#L50-L51

Please run ```twine check dist/*``` to see the issues


Discussion:

Resolving patch:
diff --git a/pytorch_lightning/__init__.py b/pytorch_lightning/__init__.py
--- a/pytorch_lightning/__init__.py
+++ b/pytorch_lightning/__init__.py
@@ -5,10 +5,9 @@
 __author_email__ = 'waf2107@columbia.edu'
 __license__ = 'Apache-2.0'
 __homepage__ = 'https://github.com/williamFalcon/pytorch-lightning'
-__docs__ = """# PyTorch Lightning
-
-The lightweight PyTorch wrapper for ML researchers. Scale your models. Write less boilerplate.
-"""
+# this has to be simple string, see: https://github.com/pypa/twine/issues/522
+__docs__ = "PyTorch Lightning is the lightweight PyTorch wrapper for ML researchers." \
+           " Scale your models. Write less boilerplate."
 
 
 try:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -46,15 +46,18 @@ def load_requirements(path_dir=PATH_ROOT, comment_char='#'):
     url=pytorch_lightning.__homepage__,
     download_url='https://github.com/williamFalcon/pytorch-lightning',
     license=pytorch_lightning.__license__,
-    packages=find_packages(exclude=['examples']),
-    # long_description=open('README.md', encoding='utf-8').read(),
-    # long_description_content_type='text/markdown',
+    packages=find_packages(exclude=['tests']),
+
+    long_description=open('README.md', encoding='utf-8').read(),
+    long_description_content_type='text/markdown',
     include_package_data=True,
     zip_safe=False,
+
     keywords=['deep learning', 'pytorch', 'AI'],
     python_requires='>=3.6',
     setup_requires=[],
     install_requires=load_requirements(PATH_ROOT),
+
     classifiers=[
         'Environment :: Console',
         'Natural Language :: English',


Test patch:
diff --git a/tests/requirements.txt b/tests/requirements.txt
--- a/tests/requirements.txt
+++ b/tests/requirements.txt
@@ -5,6 +5,7 @@ pytest>=3.0.5
 pytest-cov
 flake8
 check-manifest
-test_tube
+# test_tube  # already installed in main req.
 mlflow
 comet_ml
+twine==1.13.0
\ No newline at end of file