internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: lightly-ai/lightly

Issue:
Bug: train_embedding fails when pytorch-lightning version is a release candit
## Bug description, how to reproduce and reason
When running the line
https://github.com/lightly-ai/lightly/blob/6cee517ff73ab8fd3d91cbd9aa379df2ae4eada7/lightly/embedding/_base.py#L88-L92
with a pytorch-lightning version with a release candidate (e.g. "1.2.0rc1"), the line fails with

```
  File "/Users/malteebnerlightly/Documents/GitHub/lightly/venv/lib/python3.8/site-packages/lightly/cli/train_cli.py", line 150, in train_cli
    return _train_cli(cfg)
  File "/Users/malteebnerlightly/Documents/GitHub/lightly/venv/lib/python3.8/site-packages/lightly/cli/train_cli.py", line 119, in _train_cli
    encoder.train_embedding(**cfg['trainer'])
  File "/Users/malteebnerlightly/Documents/GitHub/lightly/venv/lib/python3.8/site-packages/lightly/embedding/_base.py", line 89, in train_embedding
    pl_version = [int(v) for v in pl.__version__.split('.')]
  File "/Users/malteebnerlightly/Documents/GitHub/lightly/venv/lib/python3.8/site-packages/lightly/embedding/_base.py", line 89, in <listcomp>
    pl_version = [int(v) for v in pl.__version__.split('.')]
ValueError: invalid literal for int() with base 10: '0rc1'
```

To reproduce, run the following python code (failing):
```python
version = "1.2.0rc1"
pl_version = [int(v) for v in version.split('.')]
```
## Proposed solutions

### Option A:
Change the version checker to account for "rcX" in the version string.
### Option B:
Increase the minimum requirement for pytorch lightning and drop the version check.


Discussion:
Incrementing the requirements seems sensible to me

Resolving patch:
diff --git a/lightly/embedding/_base.py b/lightly/embedding/_base.py
--- a/lightly/embedding/_base.py
+++ b/lightly/embedding/_base.py
@@ -82,21 +82,8 @@ def train_embedding(self, **kwargs):
             A trained encoder, ready for embedding datasets.
 
         """
-        # backwards compatability for old pytorch-lightning versions:
-        # they changed the way checkpoint callbacks are passed in v1.0.3
-        # -> do a simple version check
-        # TODO: remove when incrementing minimum requirement for pl
-        pl_version = [int(v) for v in pl.__version__.split('.')]
-        ok_version = [1, 0, 4]
-        deprecated_checkpoint_callback = \
-            all([pl_v >= ok_v for pl_v, ok_v in zip(pl_version, ok_version)])
-
-        if deprecated_checkpoint_callback:
-            trainer = pl.Trainer(**kwargs,
-                                 callbacks=[self.checkpoint_callback])
-        else:
-            trainer = pl.Trainer(**kwargs,
-                                 checkpoint_callback=self.checkpoint_callback)
+
+        trainer = pl.Trainer(**kwargs, callbacks=[self.checkpoint_callback])
 
         trainer.fit(self)
 


Test patch:
Repository: lightly-ai/lightly

Issue:
Ignore lightly outputs when creating a dataset
When working on a folder of images, e.g.
```
dataset/
L image_1.jpg
L image_2.jpg
L image_3.jpg
```
the following commands will not work when in the dataset directory:
```
lightly-embed input_dir=./
lightly-train input_dir=./
lightly-magic input_dir=./
```

This is because the command-line tool creates a directory `lightly_outputs` where logs and results are stored. However, when creating the `LightlyDataset`, this directory will be interpreted as a subfolder with images in it which leads to an error. We can handle this by ignoring the `lightly_outputs` directory.


Discussion:

Resolving patch:
diff --git a/lightly/data/_helpers.py b/lightly/data/_helpers.py
--- a/lightly/data/_helpers.py
+++ b/lightly/data/_helpers.py
@@ -37,6 +37,19 @@ def _contains_videos(root: str, extensions: tuple):
     return any(is_video)
 
 
+def _is_lightly_output_dir(dirname: str):
+    """Checks whether the directory is a lightly_output directory.
+
+    Args:
+        dirname: Directory to check.
+
+    Returns:
+        True if dirname is "lightly_outputs" else false.
+
+    """
+    return 'lightly_outputs' in dirname
+
+
 def _contains_subdirs(root: str):
     """Checks whether directory contains subdirectories.
 
@@ -48,6 +61,7 @@ def _contains_subdirs(root: str):
 
     """
     list_dir = os.listdir(root)
+    list_dir = list(filter(lambda x: not _is_lightly_output_dir(x), list_dir))
     is_dir = \
         [os.path.isdir(os.path.join(root, f)) for f in list_dir]
     return any(is_dir)


Test patch:
Repository: lightly-ai/lightly

Issue:
Make transform settable in Lightly Dataset 
Currently if one has created a Lightly Dataset and wants to apply transforms at a later stage, we have to do it as follows:
```
dataset = data.LightlyDataset(root="./", name='CIFAR10', download=True)
...
use dataset to train SimCLR model for example
...
test_transforms = torchvision.transforms.ToTensor()
dataset.dataset.transform = test_transforms
```

We should extend the Dataset wrapper to directly support transforms.


Discussion:
The same problem occurs with `LightlyDataset.from_torch_dataset(dataset, transform, index_to_filename)`: Even if a transform is passed it is never set, as the main init only sets a transform if the dataset is not None.
> 
> 
> The same problem occurs with `LightlyDataset.from_torch_dataset(dataset, transform, index_to_filename)`: Even if a transform is passed it is never set, as the main init only sets a transform if the dataset is not None.

Can you not simply set the transform of the dataset you pass?

One fix that comes to mind is
1. setting the transform correctly in the constructor
2. having properties for all the attributes of the torchvision datasets (including transform), e.g.
```python
class LightlyDataset(...):

    @property
    def transform(self):
        return self.dataset.transform

    @transform.setter
    def transform(self, t):
        self.dataset.transform = t

Resolving patch:
diff --git a/lightly/data/dataset.py b/lightly/data/dataset.py
--- a/lightly/data/dataset.py
+++ b/lightly/data/dataset.py
@@ -248,3 +248,17 @@ def dump(self,
         # dump images
         for i, filename in zip(indices, filenames):
             _dump_image(self.dataset, output_dir, filename, i, fmt=format)
+
+    @property
+    def transform(self):
+        """Getter for the transform of the dataset.
+
+        """
+        return self.dataset.transform
+
+    @transform.setter
+    def transform(self, t):
+        """Setter for the transform of the dataset.
+
+        """
+        self.dataset.transform = t


Test patch:
diff --git a/tests/data/test_LightlyDataset.py b/tests/data/test_LightlyDataset.py
--- a/tests/data/test_LightlyDataset.py
+++ b/tests/data/test_LightlyDataset.py
@@ -199,3 +199,16 @@ def test_video_dataset(self):
         out_dir = tempfile.mkdtemp()
         dataset.dump(out_dir)
         self.assertEqual(len(os.listdir(out_dir)), len(dataset))
+
+    def test_transform_setter(self):
+        
+        tmp_dir, _, _ = self.create_dataset()
+        dataset = LightlyDataset(input_dir=tmp_dir)
+        # the transform of both datasets should be None
+        self.assertIsNone(dataset.transform)
+        self.assertIsNone(dataset.dataset.transform)
+        # use the setter
+        dataset.transform = torchvision.transforms.ToTensor()
+        # assert that the transform is set in the nested dataset
+        self.assertIsNotNone(dataset.transform)
+        self.assertIsNotNone(dataset.dataset.transform)
Repository: lightly-ai/lightly

Issue:
Add solarization augmentation
# Solarization Augmentation

Solarization, as described in e.g. [BYOL](https://arxiv.org/pdf/2006.07733.pdf) can further improve the results of contrastive learning frameworks (the [SimCLR](https://arxiv.org/pdf/2002.05709.pdf) paper also mentions its benefits.

In a first step, an augmentation should be implemented under `lightly.transforms` which applies solarization to an image with a certain probability. The implementation should closely follow the one of `lightly.transforms.GaussianBlur` and use `torchvision.transform.functional.solarize`.


Discussion:

Resolving patch:
diff --git a/lightly/transforms/__init__.py b/lightly/transforms/__init__.py
--- a/lightly/transforms/__init__.py
+++ b/lightly/transforms/__init__.py
@@ -10,3 +10,4 @@
 
 from lightly.transforms.gaussian_blur import GaussianBlur
 from lightly.transforms.rotation import RandomRotate
+from lightly.transforms.solarize import RandomSolarization
diff --git a/lightly/transforms/solarize.py b/lightly/transforms/solarize.py
new file mode 100644
--- /dev/null
+++ b/lightly/transforms/solarize.py
@@ -0,0 +1,44 @@
+""" Solarization """
+
+# Copyright (c) 2021. Lightly AG and its affiliates.
+# All Rights Reserved
+
+import numpy as np
+from PIL import ImageOps
+
+
+class RandomSolarization(object):
+    """Implementation of random image Solarization.
+
+    Utilizes the integrated image operation `solarize` from Pillow. Solarization
+    inverts all pixel values above a threshold (default: 128).
+
+    Attributes:
+        probability:
+            Probability to apply the transformation
+        threshold:
+            Threshold for solarization.
+    """
+
+    def __init__(self, prob: float = 0.5,
+                 threshold: int = 128):
+        self.prob = prob
+        self.threshold = threshold
+
+    def __call__(self, sample):
+        """Solarizes the given input image
+
+        Args:
+            sample:
+                PIL image to which solarize will be applied.
+
+        Returns:
+            Solarized image or original image.
+
+        """
+        prob = np.random.random_sample()
+        if prob < self.prob:
+            # return solarized image
+            return ImageOps.solarize(sample, threshold=self.threshold)
+        # return original image
+        return sample


Test patch:
diff --git a/tests/transforms/test_Solarize.py b/tests/transforms/test_Solarize.py
new file mode 100644
--- /dev/null
+++ b/tests/transforms/test_Solarize.py
@@ -0,0 +1,14 @@
+import unittest
+from PIL import Image
+
+from lightly.transforms.solarize import RandomSolarization
+
+
+class TestRandomSolarization(unittest.TestCase):
+
+    def test_on_pil_image(self):
+        for w in [32,64,128]:
+            for h in [32,64,128]:
+                solarization = RandomSolarization(0.5)
+                sample = Image.new('RGB', (w, h))
+                solarization(sample)
Repository: lightly-ai/lightly

Issue:
Add option to show which lightly version is used by CLI
# Add option to show which lightly version is used by CLI

It would be a good feature to show the version which is used by the CLI so as to help a user debug if something goes wrong.


Discussion:

Resolving patch:
diff --git a/lightly/cli/version_cli.py b/lightly/cli/version_cli.py
new file mode 100644
--- /dev/null
+++ b/lightly/cli/version_cli.py
@@ -0,0 +1,30 @@
+# -*- coding: utf-8 -*-
+"""**Lightly Version:** Show the version of the installed package.
+
+Example:
+    >>> # show the version of the installed package
+    >>> lightly-version
+"""
+
+# Copyright (c) 2021. Lightly AG and its affiliates.
+# All Rights Reserved
+
+import hydra
+import lightly
+
+
+def _version_cli():
+    version = lightly.__version__
+    print(f'lightly version {version}', flush=True)
+
+
+@hydra.main(config_path='config', config_name='config')
+def version_cli(cfg):
+    """Prints the version of the used lightly package to the terminal.
+
+    """
+    _version_cli()
+
+
+def entry():
+    version_cli()
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -58,6 +58,7 @@ def load_requirements(path_dir=PATH_ROOT, filename='base.txt', comment_char='#')
             "lightly-magic = lightly.cli.lightly_cli:entry",
             "lightly-upload = lightly.cli.upload_cli:entry",
             "lightly-download = lightly.cli.download_cli:entry",
+            "lightly-version = lightly.cli.version_cli:entry",
         ]
     }
 


Test patch:
diff --git a/.github/workflows/test_setup.yml b/.github/workflows/test_setup.yml
--- a/.github/workflows/test_setup.yml
+++ b/.github/workflows/test_setup.yml
@@ -35,6 +35,7 @@ jobs:
         lightly-upload --help
         lightly-magic --help
         lightly-download --help
+        lightly-version
     - name: test of CLI on a real dataset
       run: |
         LIGHTLY_SERVER_LOCATION="localhost:-1"
Repository: lightly-ai/lightly

Issue:
CO2 regularizer breaks with PyTorch 1.4.0
# CO2 regularizer breaks with PyTorch 1.4.0

The `KLDivLoss` in PyTorch only received the `log_target` flag after 1.4.0 so we need to handle the case where the flag is not available.


Discussion:

Resolving patch:
diff --git a/lightly/loss/regularizer/co2.py b/lightly/loss/regularizer/co2.py
--- a/lightly/loss/regularizer/co2.py
+++ b/lightly/loss/regularizer/co2.py
@@ -46,7 +46,19 @@ def __init__(self,
                 memory_bank_size: int = 0):
 
         super(CO2Regularizer, self).__init__(size=memory_bank_size)
-        self.kl_div = torch.nn.KLDivLoss(reduction='batchmean', log_target=True)
+        # try-catch the KLDivLoss construction for backwards compatability
+        self.log_target = True
+        try:
+            self.kl_div = torch.nn.KLDivLoss(
+                reduction='batchmean',
+                log_target=True
+            )
+        except TypeError:
+            self.log_target = False
+            self.kl_div = torch.nn.KLDivLoss(
+                reduction='batchmean'
+            )
+
         self.t_consistency = t_consistency
         self.alpha = alpha
 
@@ -97,10 +109,8 @@ def _get_pseudo_labels(self,
         # divide by temperature
         logits = logits / self.t_consistency
 
-        # the input to kl_div is expected to be log(p) and we set the
-        # flag log_target to True, so both probabilities should be passed as log
-        log_probs = torch.nn.functional.log_softmax(logits, dim=-1)
-        return log_probs
+        # the input to kl_div is expected to be log(p) 
+        return torch.nn.functional.log_softmax(logits, dim=-1)
 
 
     def forward(self,
@@ -135,5 +145,11 @@ def forward(self,
         p = self._get_pseudo_labels(out0, out1, negatives)
         q = self._get_pseudo_labels(out1, out0, negatives)
         
-        # calculate kullback leibler divergence from log probabilities
-        return self.alpha * 0.5 * (self.kl_div(p, q) + self.kl_div(q, p))
+        # calculate symmetrized kullback leibler divergence
+        if self.log_target:
+            div = self.kl_div(p, q) + self.kl_div(q, p)
+        else:
+            # can't use log_target because of early torch version
+            div = self.kl_div(p, torch.exp(q)) + self.kl_div(q, torch.exp(p))
+
+        return self.alpha * 0.5 * div


Test patch:
Repository: lightly-ai/lightly

Issue:
Typo in filename of lightly subset
# Typo in filename of lightly subset

See this: 
![grafik](https://user-images.githubusercontent.com/65946090/114988852-16850a00-9e97-11eb-9f49-5f0337e64b5b.png)



Discussion:

Resolving patch:
diff --git a/lightly/data/lighty_subset.py b/lightly/data/lightly_subset.py
similarity index 100%
rename from lightly/data/lighty_subset.py
rename to lightly/data/lightly_subset.py


Test patch:
diff --git a/tests/data/test_LightlySubset.py b/tests/data/test_LightlySubset.py
--- a/tests/data/test_LightlySubset.py
+++ b/tests/data/test_LightlySubset.py
@@ -6,7 +6,7 @@
 import torchvision
 
 from lightly.data.dataset import LightlyDataset
-from lightly.data.lighty_subset import LightlySubset
+from lightly.data.lightly_subset import LightlySubset
 
 from tests.data.test_LightlyDataset import TestLightlyDataset
Repository: lightly-ai/lightly

Issue:
use ActiveLearningScoreType as string, not enum
## Description
- generate new code from the updated openapi specs in https://github.com/lightly-ai/lightly-core/issues/505
This will change the ActiveLearningScoreType to be a string instead of an enum. 
- Rename the `classification_uncertainty_...` score names by removing the `uncertainty_` prefix

## Dependencies
- https://github.com/lightly-ai/lightly-core/issues/505



Discussion:

Resolving patch:
diff --git a/lightly/active_learning/scorers/detection.py b/lightly/active_learning/scorers/detection.py
--- a/lightly/active_learning/scorers/detection.py
+++ b/lightly/active_learning/scorers/detection.py
@@ -262,7 +262,7 @@ def calculate_scores(self) -> Dict[str, np.ndarray]:
         scores_dict_classification = \
             _reduce_classification_scores_over_boxes(model_output=self.model_output)
         for score_name, score in scores_dict_classification.items():
-            scores[f"classification_{score_name}"] = score
+            scores[score_name] = score
 
         return scores
 
diff --git a/lightly/openapi_generated/swagger_client/models/active_learning_score_type.py b/lightly/openapi_generated/swagger_client/models/active_learning_score_type.py
--- a/lightly/openapi_generated/swagger_client/models/active_learning_score_type.py
+++ b/lightly/openapi_generated/swagger_client/models/active_learning_score_type.py
@@ -25,14 +25,6 @@ class ActiveLearningScoreType(object):
     Do not edit the class manually.
     """
 
-    """
-    allowed enum values
-    """
-    PREDICTION_ENTROPY = "prediction-entropy"
-    PREDICTION_MARGIN = "prediction-margin"
-    OBJECT_FREQUENCY = "object-frequency"
-    BALD = "BALD"
-
     """
     Attributes:
       swagger_types (dict): The key is attribute name
diff --git a/lightly/openapi_generated/swagger_client/models/dimensionality_reduction_method.py b/lightly/openapi_generated/swagger_client/models/dimensionality_reduction_method.py
--- a/lightly/openapi_generated/swagger_client/models/dimensionality_reduction_method.py
+++ b/lightly/openapi_generated/swagger_client/models/dimensionality_reduction_method.py
@@ -29,8 +29,8 @@ class DimensionalityReductionMethod(object):
     allowed enum values
     """
     PCA = "PCA"
-    UMAP = "UMAP"
     TSNE = "TSNE"
+    UMAP = "UMAP"
 
     """
     Attributes:


Test patch:
Repository: lightly-ai/lightly

Issue:
MoCo tutorial accuracy fails with pl
# MoCo tutorial accuracy fails with pl

Pytorch-lightning expects probabilities for the accuracy calculation, see

https://github.com/PyTorchLightning/pytorch-lightning/blob/cf5dc04db4a84e31d8c360c350b3896b67b38aeb/pytorch_lightning/metrics/classification/helpers.py#L55

Our MoCo tutorial outputs logits and hence fails during the validation step.


Discussion:

Resolving patch:
diff --git a/docs/source/tutorials_source/package/tutorial_moco_memory_bank.py b/docs/source/tutorials_source/package/tutorial_moco_memory_bank.py
--- a/docs/source/tutorials_source/package/tutorial_moco_memory_bank.py
+++ b/docs/source/tutorials_source/package/tutorial_moco_memory_bank.py
@@ -306,6 +306,7 @@ def training_epoch_end(self, outputs):
     def validation_step(self, batch, batch_idx):
         x, y, _ = batch
         y_hat = self.forward(x)
+        y_hat = torch.nn.functional.softmax(y_hat, dim=1)
         self.accuracy(y_hat, y)
         self.log('val_acc', self.accuracy.compute(),
                  on_epoch=True, prog_bar=True)


Test patch:
Repository: lightly-ai/lightly

Issue:
Lightly download does not work with videos
`lightly-download` does not work when trying to copy frames from an `input_dir` containing videos.

**Feedback:** It copies all the images to an output folder. The file names are correct, but the images are not the right ones.
Does this only work if I work with single images? Then I just convert them



Discussion:

Resolving patch:
diff --git a/lightly/data/dataset.py b/lightly/data/dataset.py
--- a/lightly/data/dataset.py
+++ b/lightly/data/dataset.py
@@ -4,6 +4,7 @@
 # All Rights Reserved
 
 import os
+import bisect
 import shutil
 import tempfile
 
@@ -277,10 +278,14 @@ def dump(self,
             filenames = self.get_filenames()
         else:
             indices = []
+            filenames = sorted(filenames)
             all_filenames = self.get_filenames()
-            for i in range(len(filenames)):
-                if filenames[i] in all_filenames:
-                    indices.append(i)
+            for index, filename in enumerate(all_filenames):
+                filename_index = bisect.bisect_left(filenames, filename)
+                # make sure the filename exists in filenames
+                if filename_index < len(filenames) and \
+                    filenames[filename_index] == filename:
+                    indices.append(index)
 
         # dump images
         for i, filename in zip(indices, filenames):


Test patch:
diff --git a/tests/data/test_LightlyDataset.py b/tests/data/test_LightlyDataset.py
--- a/tests/data/test_LightlyDataset.py
+++ b/tests/data/test_LightlyDataset.py
@@ -219,8 +219,11 @@ def test_video_dataset(self):
         dataset = LightlyDataset(input_dir=self.input_dir)
 
         out_dir = tempfile.mkdtemp()
-        dataset.dump(out_dir)
-        self.assertEqual(len(os.listdir(out_dir)), len(dataset))
+        dataset.dump(out_dir, dataset.get_filenames()[(len(dataset) // 2):])
+        self.assertEqual(len(os.listdir(out_dir)), len(dataset) // 2)
+        for filename in os.listdir(out_dir):
+            self.assertIn(filename, dataset.get_filenames()[(len(dataset) // 2):])  
+
 
     def test_transform_setter(self, dataset: LightlyDataset = None):
Repository: lightly-ai/lightly

Issue:
NNCLR
Hi,thanks for your work,
In NNCLR's model:
 "
# forward pass of first input x0
        f0 = self.backbone(x0).flatten(start_dim=1)
        z0 = self.projection_mlp(f0)
        p0 = self.prediction_mlp(f0)
 "

I think it's wrong, it should be that :
"
  # forward pass of first input x0
        f0 = self.backbone(x0).flatten(start_dim=1)
        z0 = self.projection_mlp(f0)
        p0 = self.prediction_mlp(z0)
" .


Discussion:
Hi @HBU-Lin-Li you are right! Thanks for pointing this out :)

In fact, here's the reference pseudo-code from the paper:
![grafik](https://user-images.githubusercontent.com/65946090/138820391-e08f7cbb-8828-4b99-a50e-7d2a189acbaf.png)


If you have already fixed it locally it would be great if you could do a quick pull request. Please let me know.
No..., 
I don't how to do pull request.
Ok, then I'll quickly do it.

Resolving patch:
diff --git a/lightly/models/nnclr.py b/lightly/models/nnclr.py
--- a/lightly/models/nnclr.py
+++ b/lightly/models/nnclr.py
@@ -148,7 +148,7 @@ def __init__(self,
         )
         
         self.prediction_mlp = NNCLRPredictionHead(
-            num_ftrs,
+            out_dim,
             pred_hidden_dim,
             out_dim,
         )
@@ -202,7 +202,7 @@ def forward(self,
         # forward pass of first input x0
         f0 = self.backbone(x0).flatten(start_dim=1)
         z0 = self.projection_mlp(f0)
-        p0 = self.prediction_mlp(f0)
+        p0 = self.prediction_mlp(z0)
 
         out0 = (z0, p0)
 
@@ -217,7 +217,7 @@ def forward(self,
         # forward pass of second input x1
         f1 = self.backbone(x1).flatten(start_dim=1)
         z1 = self.projection_mlp(f1)
-        p1 = self.prediction_mlp(f1)
+        p1 = self.prediction_mlp(z1)
 
         out1 = (z1, p1)
 


Test patch: