anon-iclr-submission/benchname-bug-localization huggingface dataset:
Repository: continualai/avalanche
Issue: Figures are never closed in CM metrics

Matplotlib complains that figures are left open when creating images of Confusion Matrices.

The exact warning is:

> avalanche/evaluation/metrics.py:292: RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (`matplotlib.pyplot.figure`) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParam `figure.max_open_warning`).

We should consider closing them in a "finally" block.


Changed files:
avalanche/evaluation/metrics.py

diff --git a/avalanche/evaluation/metrics.py b/avalanche/evaluation/metrics.py
index d7a5c6df..fd6c4711 100644
--- a/avalanche/evaluation/metrics.py
+++ b/avalanche/evaluation/metrics.py
@@ -309,7 +309,8 @@ class CM(object):
 
         buf = io.BytesIO()
         fig.savefig(buf, format='jpg', dpi=50)
-        # fig.savefig("temp.jpg", format='jpg')
+        plt.close(fig)
+
         buf.seek(0)
         image = PIL.Image.open(buf)
         image = ToTensor()(image)
Repository: continualai/avalanche
Issue: GDumb memory update

`GDumb` does not remove samples when the number of classes increases.

Changed files:
avalanche/training/plugins/gdumb.py

diff --git a/avalanche/training/plugins/gdumb.py b/avalanche/training/plugins/gdumb.py
index cd41932e..9bb6904f 100644
--- a/avalanche/training/plugins/gdumb.py
+++ b/avalanche/training/plugins/gdumb.py
@@ -1,10 +1,13 @@
 from collections import defaultdict
+from typing import TYPE_CHECKING
 
 import torch
 from torch.utils.data import TensorDataset
 
 from avalanche.benchmarks.utils import AvalancheConcatDataset
 from avalanche.training.plugins.strategy_plugin import StrategyPlugin
+if TYPE_CHECKING:
+    from avalanche.training import BaseStrategy
 
 
 class GDumbPlugin(StrategyPlugin):
@@ -28,7 +31,8 @@ class GDumbPlugin(StrategyPlugin):
         # count occurrences for each class
         self.counter = defaultdict(lambda: defaultdict(int))
 
-    def adapt_train_dataset(self, strategy, **kwargs):
+    def after_train_dataset_adaptation(self, strategy: 'BaseStrategy',
+                                       **kwargs):
         """ Before training we make sure to organize the memory following
             GDumb approach and updating the dataset accordingly.
         """
@@ -52,9 +56,10 @@ class GDumbPlugin(StrategyPlugin):
 
             if target_value not in current_counter or \\
                     current_counter[target_value] < patterns_per_class:
-                # full memory: replace item from most represented class
-                # with current pattern
+                # add new pattern into memory
                 if sum(current_counter.values()) >= self.mem_size:
+                    # full memory: replace item from most represented class
+                    # with current pattern
                     to_remove = max(current_counter, key=current_counter.get)
                     for j in range(len(current_mem.tensors[1])):
                         if current_mem.tensors[1][j].item() == to_remove:
@@ -81,4 +86,4 @@ class GDumbPlugin(StrategyPlugin):
                 current_counter[target_value] += 1
 
         self.ext_mem[strategy.experience.task_label] = current_mem
-        strategy.adapted_dataset = AvalancheConcatDataset(self.ext_mem)
+        strategy.adapted_dataset = AvalancheConcatDataset(self.ext_mem.values())
Repository: continualai/avalanche
Issue: `eval_streams` parameter of `train_exp` can't be null.

🐛 **Describe the bug**
When directly using `train_exp` on a Strategy with `eval_every` >= 0 and `train_streams` set to null, the eval_stream is not set to the given experience as described in the doc, causing an error in `_periodic_eval`.

🐜 **To Reproduce**
```python
from avalanche.benchmarks.classic import SplitMNIST
from avalanche.models import SimpleMLP
from avalanche.training.strategies import Naive

from torch.optim import SGD
from torch.nn import CrossEntropyLoss

scenario = SplitMNIST(n_experiences=5)
model = SimpleMLP(num_classes=scenario.n_classes)
cl_strategy = Naive(
    model, SGD(model.parameters(), lr=0.001, momentum=0.9),
    CrossEntropyLoss(), train_mb_size=500, eval_mb_size=100, train_epochs=1,
    eval_every=0)

results = []
for experience in scenario.train_stream:

    # This line works
    cl_strategy.train_exp(experience, eval_streams=[scenario.train_stream])

    # This line breaks
    cl_strategy.train_exp(experience, eval_streams=None)
```

🐝 **Expected behavior**
I think the most intuitive behavior would be to create an evaluation stream only composed of the given training experience.


Changed files:
avalanche/training/strategies/base_strategy.py

diff --git a/avalanche/training/strategies/base_strategy.py b/avalanche/training/strategies/base_strategy.py
index dbd2a20e..27301291 100644
--- a/avalanche/training/strategies/base_strategy.py
+++ b/avalanche/training/strategies/base_strategy.py
@@ -238,9 +238,6 @@ class BaseStrategy:
             experiences = [experiences]
         if eval_streams is None:
             eval_streams = [experiences]
-        for i, exp in enumerate(eval_streams):
-            if not isinstance(exp, Sequence):
-                eval_streams[i] = [exp]
 
         self.before_training(**kwargs)
         for self.experience in experiences:
@@ -250,19 +247,25 @@ class BaseStrategy:
         res = self.evaluator.get_last_metrics()
         return res
 
-    def train_exp(self, experience: Experience, eval_streams, **kwargs):
+    def train_exp(self, experience: Experience, eval_streams=None, **kwargs):
         """
         Training loop over a single Experience object.
 
         :param experience: CL experience information.
         :param eval_streams: list of streams for evaluation.
-            If None: use training experiences for evaluation.
+            If None: use the training experience for evaluation.
             Use [] if you do not want to evaluate during training.
         :param kwargs: custom arguments.
         """
         self.experience = experience
         self.model.train()
 
+        if eval_streams is None:
+            eval_streams = [experience]
+        for i, exp in enumerate(eval_streams):
+            if not isinstance(exp, Sequence):
+                eval_streams[i] = [exp]
+
         # Data Adaptation (e.g. add new samples/data augmentation)
         self.before_train_dataset_adaptation(**kwargs)
         self.train_dataset_adaptation(**kwargs)
Repository: continualai/avalanche
Issue: in EWC the penalty is 0

🐛 **Describe the bug**
A clear and concise description of what the bug is.
Currently in [EWC ](https://github.com/ContinualAI/avalanche/blob/master/avalanche/training/plugins/ewc.py)plugin the penalty is always 0 
```
n_units = saved_param.shape[0]
cur_param = saved_param[:n_units]
penalty += (imp * (cur_param - saved_param).pow(2)).sum()
```

because `cur_param` is equal to `saved_param` in the penalty from computation `penalty += (imp * (cur_param - saved_param).pow(2)).sum()`


Changed files:
avalanche/benchmarks/classic/cmnist.py, avalanche/training/plugins/ewc.py

diff --git a/avalanche/benchmarks/classic/cmnist.py b/avalanche/benchmarks/classic/cmnist.py
index 8f2866c4..c36319d8 100644
--- a/avalanche/benchmarks/classic/cmnist.py
+++ b/avalanche/benchmarks/classic/cmnist.py
@@ -250,7 +250,7 @@ def PermutedMNIST(
         n_experiences=len(list_train_dataset),
         task_labels=return_task_id,
         shuffle=False,
-        class_ids_from_zero_in_each_exp=False,
+        class_ids_from_zero_in_each_exp=True,
         one_dataset_per_exp=True,
         train_transform=train_transform,
         eval_transform=eval_transform,
@@ -373,7 +373,7 @@ def RotatedMNIST(
         n_experiences=len(list_train_dataset),
         task_labels=return_task_id,
         shuffle=False,
-        class_ids_from_zero_in_each_exp=False,
+        class_ids_from_zero_in_each_exp=True,
         one_dataset_per_exp=True,
         train_transform=train_transform,
         eval_transform=eval_transform,
diff --git a/avalanche/training/plugins/ewc.py b/avalanche/training/plugins/ewc.py
index 09608b43..b184e56e 100644
--- a/avalanche/training/plugins/ewc.py
+++ b/avalanche/training/plugins/ewc.py
@@ -89,7 +89,7 @@ class EWCPlugin(SupervisedPlugin):
                     # dynamic models may add new units
                     # new units are ignored by the regularization
                     n_units = saved_param.shape[0]
-                    cur_param = saved_param[:n_units]
+                    cur_param = cur_param[:n_units]
                     penalty += (imp * (cur_param - saved_param).pow(2)).sum()
         elif self.mode == "online":
             prev_exp = exp_counter - 1
@@ -101,7 +101,7 @@ class EWCPlugin(SupervisedPlugin):
                 # dynamic models may add new units
                 # new units are ignored by the regularization
                 n_units = saved_param.shape[0]
-                cur_param = saved_param[:n_units]
+                cur_param = cur_param[:n_units]
                 penalty += (imp * (cur_param - saved_param).pow(2)).sum()
         else:
             raise ValueError("Wrong EWC mode.")
Repository: continualai/avalanche
Issue: AGEM blocks on Split CIFAR-100

From https://github.com/ContinualAI/reproducible-continual-learning/issues/16

AGEM blocks in the `sample_from_memory` call when retrieving the next element from the `GroupBalancedInfiniteDataLoader`, which represents the AGEM buffer. 
The process does not always block at the same experience and the problem is not there for other benchmarks like Permuted MNIST.

You can reproduce the problem with this script

```python
import avalanche as avl
from avalanche.evaluation import metrics
import torch

device = torch.device('cuda:0')

benchmark = avl.benchmarks.SplitCIFAR100(17, return_task_id=True, fixed_class_order=list(range(85)))
model = avl.models.SimpleCNN(num_classes=100)
criterion = torch.nn.CrossEntropyLoss()

interactive_logger = avl.logging.InteractiveLogger()

evaluation_plugin = avl.training.plugins.EvaluationPlugin(
    metrics.accuracy_metrics(epoch=True, experience=True, stream=True),
    loggers=[interactive_logger], benchmark=benchmark)

cl_strategy = avl.training.AGEM(
    model, torch.optim.SGD(model.parameters(), lr=1e-3, momentum=0.9), criterion,
    patterns_per_exp=65, sample_size=200,
    train_mb_size=10, train_epochs=1, eval_mb_size=128,
    device=device, evaluator=evaluation_plugin)

for experience in benchmark.train_stream:
    cl_strategy.train(experience)
    res = cl_strategy.eval(benchmark.test_stream)
```
I am currently trying to better investigate the problem but I also wanted to add the issue here for reference

Changed files:
avalanche/training/plugins/agem.py

diff --git a/avalanche/training/plugins/agem.py b/avalanche/training/plugins/agem.py
index da352a4d..c6a0b84d 100644
--- a/avalanche/training/plugins/agem.py
+++ b/avalanche/training/plugins/agem.py
@@ -1,3 +1,5 @@
+import warnings
+
 import torch
 from torch.utils.data import random_split
 
@@ -110,10 +112,13 @@ class AGEMPlugin(SupervisedPlugin):
         return next(self.buffer_dliter)
 
     @torch.no_grad()
-    def update_memory(self, dataset, num_workers=4, **kwargs):
+    def update_memory(self, dataset, num_workers=0, **kwargs):
         """
         Update replay memory with patterns from current experience.
         """
+        if num_workers > 0:
+            warnings.warn("Num workers > 0 is known to cause heavy"
+                          "slowdowns in AGEM.")
         removed_els = len(dataset) - self.patterns_per_experience
         if removed_els > 0:
             dataset, _ = random_split(
Repository: continualai/avalanche
Issue: CORe50Dataset return the whole dataset (train + test set) with the attribute train=True

🐛 **Describe the bug**
The CORe50Dataset return the whole dataset (train + test set) with the attribute train=True

🐜 **To Reproduce**
```python
core50_train = CORe50Dataset(
    root="../data/dataset/core50",
    train=True,
    mini=False,
    object_level=True
)
print(len(core50_train)   # should be ~120k, is more than 160k (whole dataset)
```

🐝 **Expected behavior**
Only the training set should be returned.

🐞 **Screenshots**
N/A

🦋 **Additional context**
N/A


Changed files:
avalanche/benchmarks/datasets/core50/core50.py, avalanche/benchmarks/datasets/openloris/openloris.py

diff --git a/avalanche/benchmarks/datasets/core50/core50.py b/avalanche/benchmarks/datasets/core50/core50.py
index 5d3f41fb..76b60e7a 100644
--- a/avalanche/benchmarks/datasets/core50/core50.py
+++ b/avalanche/benchmarks/datasets/core50/core50.py
@@ -169,7 +169,7 @@ class CORe50Dataset(DownloadableDataset):
 
         self.idx_list = []
         if self.train:
-            for i in range(self._nbatch + 1):
+            for i in range(self._nbatch):
                 self.idx_list += self.LUP[self._scen][self._run][i]
         else:
             self.idx_list = self.LUP[self._scen][self._run][-1]
diff --git a/avalanche/benchmarks/datasets/openloris/openloris.py b/avalanche/benchmarks/datasets/openloris/openloris.py
index 913419af..c8daeec8 100644
--- a/avalanche/benchmarks/datasets/openloris/openloris.py
+++ b/avalanche/benchmarks/datasets/openloris/openloris.py
@@ -84,7 +84,7 @@ class OpenLORIS(DownloadableDataset):
         # any scenario and factor is good here since we want just to load the
         # train images and targets with no particular order
         scen = "domain"
-        factor = 0
+        factor = [_ for _ in range(4)]
         ntask = 9
 
         print("Loading paths...")
@@ -95,8 +95,9 @@ class OpenLORIS(DownloadableDataset):
         with open(str(self.root / "Labels.pkl"), "rb") as f:
             self.all_targets = pkl.load(f)
             self.train_test_targets = []
-            for i in range(ntask + 1):
-                self.train_test_targets += self.all_targets[scen][factor][i]
+            for fact in factor:
+                for i in range(ntask + 1):
+                    self.train_test_targets += self.all_targets[scen][fact][i]
 
         print("Loading LUP...")
         with open(str(self.root / "LUP.pkl"), "rb") as f:
@@ -104,10 +105,12 @@ class OpenLORIS(DownloadableDataset):
 
         self.idx_list = []
         if self.train:
-            for i in range(ntask + 1):
-                self.idx_list += self.LUP[scen][factor][i]
+            for fact in factor:
+                for i in range(ntask):
+                    self.idx_list += self.LUP[scen][fact][i]
         else:
-            self.idx_list = self.LUP[scen][factor][-1]
+            for fact in factor:
+                self.idx_list += self.LUP[scen][fact][-1]
 
         self.paths = []
         self.targets = []
Repository: alkaline-ml/pmdarima
Issue: [DOC] Misleading doc in SlidingWindowForecastCV

As mentioned in the comment section of #201, there is a misleading statement in the [`SlidingWindowForecastCV`](https://github.com/tgsmith61591/pmdarima/blob/master/pmdarima/model_selection/_split.py#L200):

```
    step : int, optional (default=1)
        The size of step taken to increase the training sample size when not
        performing using a window.
```

This is an artifact of when both CV strategies were lumped into the same class. This should read:

```
    step : int, optional (default=1)
        The size of step taken between training folds.
```

Good issue for anyone looking to get involved.

Changed files:
pmdarima/model_selection/_split.py

diff --git a/pmdarima/model_selection/_split.py b/pmdarima/model_selection/_split.py
index cb2b3430..6a0f46ce 100644
--- a/pmdarima/model_selection/_split.py
+++ b/pmdarima/model_selection/_split.py
@@ -198,8 +198,7 @@ class SlidingWindowForecastCV(BaseTSCrossValidator):
         the last training sample for the test set.
 
     step : int, optional (default=1)
-        The size of step taken to increase the training sample size when not
-        performing using a window.
+        The size of step taken between training folds.
 
     window_size : int or None, optional (default=None)
         The size of the rolling window to use. If None, a rolling window of
Repository: jupyter-widgets/ipyleaflet
Issue: Choropleth layer with np.integer data

As pointed out by @bill-coman:

`value_min` and `value_max` attributes being Float traits, passing a numpy array to `choro_data` containing integers can throw an exception.

Changed files:
ipyleaflet/leaflet.py

diff --git a/ipyleaflet/leaflet.py b/ipyleaflet/leaflet.py
index 76032bf..2d3d2dd 100644
--- a/ipyleaflet/leaflet.py
+++ b/ipyleaflet/leaflet.py
@@ -14,7 +14,7 @@ from ipywidgets import (
 from ipywidgets.widgets.trait_types import InstanceDict
 
 from traitlets import (
-    Float, Unicode, Int, Tuple, List, Instance, Bool, Dict, Enum,
+    CFloat, Float, Unicode, Int, Tuple, List, Instance, Bool, Dict, Enum,
     link, observe, default, validate, TraitError, Union, Any
 )
 
@@ -608,8 +608,8 @@ class GeoData(GeoJSON):
 class Choropleth(GeoJSON):
     geo_data = Dict()
     choro_data = Dict()
-    value_min = Float(None, allow_none=True)
-    value_max = Float(None, allow_none=True)
+    value_min = CFloat(None, allow_none=True)
+    value_max = CFloat(None, allow_none=True)
     colormap = Instance(ColorMap)
     key_on = Unicode('id')
Repository: harshcasper/rotten-scripts
Issue: (FLK-E741) Do not use variables named 'l', 'O', or 'I'

## Description
Variables named `I`, `O`, and `l` can be very hard to read. This is because the letter `I` and the letter `l` are easily confused, and the letter `O` and the number `0` can be easily confused. Change the names of these variables to something more descriptive.

## Occurrences
There are 2 occurrences of this issue in the repository.

See all occurrences on DeepSource &rarr; [deepsource.io/gh/HarshCasper/Rotten-Scripts/issue/FLK-E741/occurrences/](https://deepsource.io/gh/HarshCasper/Rotten-Scripts/issue/FLK-E741/occurrences/)


Changed files:
Python/Track_Data_Usage/track_data_usage.py

diff --git a/Python/Track_Data_Usage/track_data_usage.py b/Python/Track_Data_Usage/track_data_usage.py
index 107a611..667a7b3 100644
--- a/Python/Track_Data_Usage/track_data_usage.py
+++ b/Python/Track_Data_Usage/track_data_usage.py
@@ -5,36 +5,38 @@ import time
 import threading
 import subprocess as sp
 
+
 class network():
-	def monitor(self, limit, unit):
-		check = "vnstat"
-		proc = sp.Popen(check, shell=True, stdout=sp.PIPE)
-		output = proc.communicate()
-		output = str(output)
-		l = []
-		for t in output.split():
-			try:
-				if t=="MiB" or t=="GiB":
-					l.append(t)
-				else:
-					l.append(float(t))
-			except ValueError:
-				pass
-		if unit==l[5] and limit < l[4]:
-			print("Network usage limit exceeded!")
-			top.mainloop()
-		arg = [limit, unit]
-		threading.Timer(60.0, monitor, arg).start()
+    def monitor(self, limit, unit):
+        check = "vnstat"
+        proc = sp.Popen(check, shell=True, stdout=sp.PIPE)
+        output = proc.communicate()
+        output = str(output)
+        len = []
+        for t in output.split():
+            try:
+                if t == "MiB" or t == "GiB":
+                    len.append(t)
+                else:
+                    len.append(float(t))
+            except ValueError:
+                pass
+        if unit == len[5] and limit < len[4]:
+            print("Network usage limit exceeded!")
+            top.mainloop()
+        arg = [limit, unit]
+        threading.Timer(60.0, monitor, arg).start()
+
+    def main():
+        if len(sys.argv) > 3 or len(sys.argv) < 3:
+            print("Command usage: Python data.py <data usage in MiB or GiB")
+            print("example: python data.py 500MiB")
+            print("or python data.py 2GiB")
+            exit(1)
+        else:
+            limit = float(sys.argv[1])
+            unit = str(sys.argv[2])
+            monitor(limit, unit)
 
-	def main():
-		if len(sys.argv) > 3 or len(sys.argv) < 3:
-			print("Command usage: Python data.py <data usage in MiB or GiB")
-			print("example: python data.py 500MiB")
-			print("or python data.py 2GiB")
-			exit(1)
-		else: 
-			limit = float(sys.argv[1])
-			unit = str(sys.argv[2])
-			monitor(limit, unit)
-	if __name__ == '__main__':
-		main()
\\ No newline at end of file
+    if __name__ == '__main__':
+        main()
Repository: nextstrain/ncov
Issue: Metadata: date_submitted 

Hi,

Within my meta data I have not been including "date_submitted". To date this was not a problem, nextstrain ran to completion and an auspice json was produced.

I just updated and found if date_submitted is not included an error is made in diagnostic.py on the lines
```
to_exclude |= (rare_mutations>args.rare_mutations)&(~recent_sequences)
to_exclude |= (np.abs(clock_deviation+rare_mutations)>args.clock_plus_rare)&(~recent_sequences)
```
I have changed these back to the previous implementation and things look to be working fine. 
If this field is needed I will generate a date_submitted column and fill it was some numbers.

Thanks.

Changed files:
scripts/diagnostic.py

diff --git a/scripts/diagnostic.py b/scripts/diagnostic.py
index 61b0e992..203b96a3 100644
--- a/scripts/diagnostic.py
+++ b/scripts/diagnostic.py
@@ -78,11 +78,15 @@ if __name__ == '__main__':
     to_exclude |= np.abs(clock_deviation)>args.clock_filter_recent
     if check_recency:
         to_exclude |= (np.abs(clock_deviation)>args.clock_filter)&(~recent_sequences)
+        to_exclude |= (rare_mutations>args.rare_mutations)&(~recent_sequences)
+        to_exclude |= (np.abs(clock_deviation+rare_mutations)>args.clock_plus_rare)&(~recent_sequences)
+    else:
+        to_exclude |= (rare_mutations>args.rare_mutations)
+        to_exclude |= (np.abs(clock_deviation+rare_mutations)>args.clock_plus_rare)
     if check_clade_dates:
         to_exclude |= dates<clade_dates
     to_exclude |= snp_clusters>args.snp_clusters
-    to_exclude |= (rare_mutations>args.rare_mutations)&(~recent_sequences)
-    to_exclude |= (np.abs(clock_deviation+rare_mutations)>args.clock_plus_rare)&(~recent_sequences)
+
 
     if "QC_mixed_sites" in metadata.columns:
         to_exclude |= metadata.QC_mixed_sites=='bad'