internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: adamchainz/django-mysql

Issue:
MySQLCache.set_many should return a list of failing keys
This new behaviour in Django 2.0 ( django/django#7520 ) should be implemented.


Discussion:

Resolving patch:
diff --git a/django_mysql/cache.py b/django_mysql/cache.py
--- a/django_mysql/cache.py
+++ b/django_mysql/cache.py
@@ -289,6 +289,7 @@ def set_many(self, data, timeout=DEFAULT_TIMEOUT, version=None):
 
         with connections[db].cursor() as cursor:
             cursor.execute(query, params)
+        return []
 
     def delete(self, key, version=None):
         key = self.make_key(key, version=version)


Test patch:
diff --git a/tests/testapp/test_cache.py b/tests/testapp/test_cache.py
--- a/tests/testapp/test_cache.py
+++ b/tests/testapp/test_cache.py
@@ -846,11 +846,16 @@ def test_set_many(self):
         caches['no_cull'].get('nonexistent')
 
         with self.assertNumQueries(1):
-            caches['no_cull'].set_many({"key1": "spam"})
+            result = caches['no_cull'].set_many({"key1": "spam"})
+        assert result == []
 
         # Multiple keys can be set using set_many
         with self.assertNumQueries(1):
-            caches['no_cull'].set_many({"key1": "spam", "key2": "eggs"})
+            result = caches['no_cull'].set_many({
+                'key1': 'spam',
+                'key2': 'eggs',
+            })
+        assert result == []
         assert cache.get("key1") == "spam"
         assert cache.get("key2") == "eggs"
Repository: adamchainz/django-mysql

Issue:
Add cache.touch() method
Coming in Django 2.1 as per django/django#7610, we should support it in MySQLCache


Discussion:
@normos is working on this :)

Resolving patch:
diff --git a/django_mysql/cache.py b/django_mysql/cache.py
--- a/django_mysql/cache.py
+++ b/django_mysql/cache.py
@@ -394,6 +394,25 @@ def clear(self):
         with connections[db].cursor() as cursor:
             cursor.execute("DELETE FROM {table}".format(table=table))
 
+    def touch(self, key, timeout=DEFAULT_TIMEOUT, version=None):
+        key = self.make_key(key, version=version)
+        self.validate_key(key)
+        exp = self.get_backend_timeout(timeout)
+        db = router.db_for_write(self.cache_model_class)
+        table = connections[db].ops.quote_name(self._table)
+        with connections[db].cursor() as cursor:
+            cursor.execute(
+                self._touch_query.format(table=table),
+                [exp, key, self._now()],
+            )
+
+    _touch_query = collapse_spaces("""
+        UPDATE {table}
+        SET expires = %s
+        WHERE cache_key = %s AND
+              expires >= %s
+    """)
+
     def validate_key(self, key):
         """
         Django normally warns about maximum key length, but we error on it.


Test patch:
diff --git a/tests/testapp/test_cache.py b/tests/testapp/test_cache.py
--- a/tests/testapp/test_cache.py
+++ b/tests/testapp/test_cache.py
@@ -357,6 +357,24 @@ def test_clear(self):
         assert cache.get("key1") is None
         assert cache.get("key2") is None
 
+    def test_touch_without_timeout(self):
+        cache.set("key1", "spam", timeout=0.1)
+        cache.touch("key1", timeout=0.4)
+        time.sleep(0.2)
+        assert "key1" in cache
+
+    def test_touch_with_timeout(self):
+        cache.set("key1", "spam", timeout=0.1)
+        cache.touch("key1")
+        time.sleep(0.2)
+        assert "key1" in cache
+
+    def test_touch_already_expired(self):
+        cache.set("key1", "spam", timeout=0.1)
+        time.sleep(0.2)
+        cache.touch("key1", timeout=0.4)
+        assert "key1" not in cache
+
     def test_long_timeout(self):
         '''
         Using a timeout greater than 30 days makes memcached think
Repository: adamchainz/django-mysql

Issue:
dangling connection created during system check may become unusable
*Summary*: mysql connection created during system check is not closed timely. This connection might (mysteriously) become unusable afterwards (for reason I could not fully understand for now), which blocks the database access in application logic.

*Description*: I'm using django with celery. Today I noticed that any task that accesses database by Django ORM just hangs forever (i.e. they only start but never finish). After some digging I realized it's database connection that blocks forever and never comes back.

After another really painful debugging process, I figured out that, during django's system checking process, django-mysql created a persistent database connection rather than a temporary one, thus get re-used in application code. But somehow its connection is dropped, which causes any code that implicitly depends on it blocks forever.

Specifically, I think the following code in `django_mysql/checks.py` should use a `BaseDatabaseWrapper.temporary_connection()`, therefore ensure connection is closed cleanly afterwards.

```python
def check_variables(app_configs, **kwargs):
    errors = []

    for alias, connection in mysql_connections():
        with connection.cursor() as cursor: # connection.temporary_connection()
            cursor.execute("""SELECT @@sql_mode,
                                     @@innodb_strict_mode,
                                     @@character_set_connection""")
            # ...
```

(Sorry if this poor bug report seems in a hurry, I'm really tired (it's 4 AM my brain isn't working...) )

*Django Version*: Django 2.0.6

*Database and version used*: mysqld  Ver 5.7.22 for Linux on x86_64 (MySQL Community Server (GPL))

*Version*: Django-MySQL 2.2.2



Discussion:
Temporary connection sounds good to me - you want to make the PR and get the GitHub poitns since you've been to all this effort?

Resolving patch:
diff --git a/django_mysql/checks.py b/django_mysql/checks.py
--- a/django_mysql/checks.py
+++ b/django_mysql/checks.py
@@ -17,7 +17,7 @@ def check_variables(app_configs, **kwargs):
     errors = []
 
     for alias, connection in mysql_connections():
-        with connection.cursor() as cursor:
+        with connection.temporary_connection() as cursor:
             cursor.execute("""SELECT @@sql_mode,
                                      @@innodb_strict_mode,
                                      @@character_set_connection""")


Test patch:
Repository: adamchainz/django-mysql

Issue:
Officially support Django 2.1
Testing is passing with 2.1, document and release if any changes required.


Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -50,6 +50,7 @@ def get_version(filename):
         'Framework :: Django :: 1.10',
         'Framework :: Django :: 1.11',
         'Framework :: Django :: 2.0',
+        'Framework :: Django :: 2.1',
         'Intended Audience :: Developers',
         'License :: OSI Approved :: BSD License',
         'Natural Language :: English',


Test patch:
Repository: adamchainz/django-mysql

Issue:
Importing ABC directly from collections was removed in Python 3.9
https://github.com/adamchainz/django-mysql/blob/bc4ff381d3a61af6dc448312205c2c19d9c5f97e/src/django_mysql/models/lookups.py#L126


Discussion:
Thanks, though it sems this will actually be deferred: https://mail.python.org/archives/list/python-dev@python.org/thread/EYLXCGGJOUMZSE5X35ILW3UNTJM3MCRE/

Resolving patch:
diff --git a/src/django_mysql/models/lookups.py b/src/django_mysql/models/lookups.py
--- a/src/django_mysql/models/lookups.py
+++ b/src/django_mysql/models/lookups.py
@@ -1,5 +1,5 @@
-import collections
 import json
+from collections.abc import Sequence
 
 import django
 from django.db.models import CharField, Lookup, Transform
@@ -123,7 +123,7 @@ def as_sql(self, qn, connection):
 
 class JSONSequencesMixin(object):
     def get_prep_lookup(self):
-        if not isinstance(self.rhs, collections.Sequence):
+        if not isinstance(self.rhs, Sequence):
             raise ValueError(
                 "JSONField's '{}' lookup only works with Sequences".format(
                     self.lookup_name


Test patch:
Repository: adamchainz/django-mysql

Issue:
TypeError: iterator() got an unexpected keyword argument 'chunk_size'
Summary: the iterator doesn't support the chunk_size argument of the core Django QuerySet iterator function 

The following code in django_mysql.models.query line 157 - 160 fixes the problem.
    def iterator(self, **kwargs):
        if getattr(self, "_found_rows", 0) is None:
            raise ValueError("sql_calc_found_rows() doesn't work with iterator()")
        return super().iterator(**kwargs)

Django Version: 3.1.5
Database and version used: mariadb 10.14
Version: Django-MySQL 3.8.0



Discussion:

Resolving patch:
diff --git a/src/django_mysql/models/query.py b/src/django_mysql/models/query.py
--- a/src/django_mysql/models/query.py
+++ b/src/django_mysql/models/query.py
@@ -154,10 +154,10 @@ def found_rows(self):
             )
         return self._found_rows
 
-    def iterator(self):
+    def iterator(self, *args, **kwargs):
         if getattr(self, "_found_rows", 0) is None:
             raise ValueError("sql_calc_found_rows() doesn't work with iterator()")
-        return super().iterator()
+        return super().iterator(*args, **kwargs)
 
     @property
     def _result_cache(self):


Test patch:
diff --git a/tests/testapp/test_models.py b/tests/testapp/test_models.py
--- a/tests/testapp/test_models.py
+++ b/tests/testapp/test_models.py
@@ -371,6 +371,10 @@ def test_iterator_without_it_works(self):
         authors = Author.objects.all()[:5]
         authors.iterator()
 
+    def test_iterator_without_it_works_with_chunk_size(self):
+        authors = Author.objects.all()[:5]
+        authors.iterator(chunk_size=100)
+
     def test_it_working(self):
         with self.assertNumQueries(2):
             authors = Author.objects.sql_calc_found_rows()[:5]
Repository: adamchainz/django-mysql

Issue:
Make 'EnumField.choices' affect DB for Django 4.1+
https://github.com/django/django/pull/15561#issuecomment-1090077702


Discussion:

Resolving patch:
diff --git a/src/django_mysql/models/fields/enum.py b/src/django_mysql/models/fields/enum.py
--- a/src/django_mysql/models/fields/enum.py
+++ b/src/django_mysql/models/fields/enum.py
@@ -2,6 +2,7 @@
 
 from typing import Any, cast
 
+import django
 from django.db.backends.base.base import BaseDatabaseWrapper
 from django.db.models import CharField
 from django.utils.encoding import force_str
@@ -13,6 +14,9 @@
 class EnumField(CharField):
     description = _("Enumeration")
 
+    if django.VERSION >= (4, 1):
+        non_db_attrs = tuple(f for f in CharField.non_db_attrs if f != "choices")
+
     def __init__(
         self,
         *args: Any,


Test patch:
diff --git a/tests/testapp/test_enumfield.py b/tests/testapp/test_enumfield.py
--- a/tests/testapp/test_enumfield.py
+++ b/tests/testapp/test_enumfield.py
@@ -3,9 +3,10 @@
 import pytest
 from django.core.exceptions import ValidationError
 from django.core.management import call_command
-from django.db import connection
+from django.db import connection, models
 from django.db.utils import DataError
 from django.test import TestCase, TransactionTestCase, override_settings
+from django.test.utils import isolate_apps
 
 from django_mysql.models import EnumField
 from tests.testapp.models import EnumModel, NullableEnumModel
@@ -152,6 +153,28 @@ def test_adding_field_with_default(self):
         with connection.cursor() as cursor:
             assert table_name not in table_names(cursor)
 
+    @isolate_apps("tests.testapp")
+    def test_alter_field_choices_changes(self):
+        class Temp(models.Model):
+            field = EnumField(choices=["apple"])
+
+        with connection.schema_editor() as editor:
+            editor.create_model(Temp)
+
+        @self.addCleanup
+        def drop_table():
+            with connection.schema_editor() as editor:
+                editor.delete_model(Temp)
+
+        old_field = Temp._meta.get_field("field")
+        new_field = EnumField(choices=["apple", "banana"])
+        new_field.set_attributes_from_name("field")
+
+        with connection.schema_editor() as editor, self.assertNumQueries(1):
+            editor.alter_field(Temp, old_field, new_field, strict=True)
+        with connection.schema_editor() as editor, self.assertNumQueries(1):
+            editor.alter_field(Temp, new_field, old_field, strict=True)
+
 
 class TestFormfield(TestCase):
     def test_formfield(self):
Repository: jdb78/pytorch-forecasting

Issue:
Best Model not provided
The AR example has a bestmodel with an address to a personal computer, this bestmodel is not provided and cause errors when running the notebook.


Discussion:
Very good point. Thanks a lot, @silvavn. Will fix it.

Resolving patch:
diff --git a/pytorch_forecasting/models/nbeats/__init__.py b/pytorch_forecasting/models/nbeats/__init__.py
--- a/pytorch_forecasting/models/nbeats/__init__.py
+++ b/pytorch_forecasting/models/nbeats/__init__.py
@@ -287,7 +287,7 @@ def to_prediction(y):
             else:
                 ax2.plot(time, to_prediction(output[title]), label=title.capitalize())
         ax[1].set_xlabel("Time")
-        ax[1].set_ylabel("Trend")
+        ax[1].set_ylabel("Decomposition")
 
         fig.legend()
         return fig


Test patch:
Repository: jdb78/pytorch-forecasting

Issue:
[NBeats] plot_interpretation with axes list
- PyTorch-Forecasting version: 0.8.2
- PyTorch version: 1.8.0
- Python version: 3.6.9
- Operating System: Ubuntu 18.04

### Expected behavior

plot_interpretation with ax as list of axes gives 'AttributeError: 'list' object has no attribute 'get_figure'.

Docs says ax needs List[matplotlib axes] but the source seems to call get_figure() directly from ax itself:
---
        if ax is None:
            fig, ax = plt.subplots(2, 1, figsize=(6, 8))
        else:
            fig = ax.get_figure()
---


Discussion:

Resolving patch:
diff --git a/pytorch_forecasting/models/nbeats/__init__.py b/pytorch_forecasting/models/nbeats/__init__.py
--- a/pytorch_forecasting/models/nbeats/__init__.py
+++ b/pytorch_forecasting/models/nbeats/__init__.py
@@ -285,7 +285,7 @@ def plot_interpretation(
         if ax is None:
             fig, ax = plt.subplots(2, 1, figsize=(6, 8))
         else:
-            fig = ax.get_figure()
+            fig = ax[0].get_figure()
 
         time = torch.arange(-self.hparams.context_length, self.hparams.prediction_length)
 


Test patch:
Repository: jdb78/pytorch-forecasting

Issue:
Default trainer kwargs in temporal_fusion_transformer/tuning.py cannot be overwritten
```python
[I 2021-01-27 09:42:03,226] A new study created in memory with name: no-name-8f54548f-a847-4450-9a1c-93acfdc5609e
[W 2021-01-27 09:42:03,229] Trial 0 failed because of the following error: TypeError("ABCMeta object got multiple values for keyword argument 'progress_bar_refresh_rate'",)
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/optuna/_optimize.py", line 198, in _run_trial
    value_or_values = func(trial)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_forecasting/models/temporal_fusion_transformer/tuning.py", line 145, in objective
    **trainer_kwargs,
TypeError: ABCMeta object got multiple values for keyword argument 'progress_bar_refresh_rate'
```


Discussion:

Resolving patch:
diff --git a/pytorch_forecasting/models/temporal_fusion_transformer/tuning.py b/pytorch_forecasting/models/temporal_fusion_transformer/tuning.py
--- a/pytorch_forecasting/models/temporal_fusion_transformer/tuning.py
+++ b/pytorch_forecasting/models/temporal_fusion_transformer/tuning.py
@@ -129,8 +129,8 @@ def objective(trial: optuna.Trial) -> float:
         learning_rate_callback = LearningRateMonitor()
         logger = TensorBoardLogger(log_dir, name="optuna", version=trial.number)
         gradient_clip_val = trial.suggest_loguniform("gradient_clip_val", *gradient_clip_val_range)
-        trainer_kwargs.setdefault("gpus", [0] if torch.cuda.is_available() else None)
-        trainer = pl.Trainer(
+        default_trainer_kwargs = dict(
+            gpus=[0] if torch.cuda.is_available() else None,
             max_epochs=max_epochs,
             gradient_clip_val=gradient_clip_val,
             callbacks=[
@@ -142,7 +142,10 @@ def objective(trial: optuna.Trial) -> float:
             logger=logger,
             progress_bar_refresh_rate=[0, 1][optuna_verbose < optuna.logging.INFO],
             weights_summary=[None, "top"][optuna_verbose < optuna.logging.INFO],
-            **trainer_kwargs,
+        )
+        default_trainer_kwargs.update(trainer_kwargs)
+        trainer = pl.Trainer(
+            **default_trainer_kwargs,
         )
 
         # create model


Test patch:
diff --git a/tests/test_models/test_temporal_fusion_transformer.py b/tests/test_models/test_temporal_fusion_transformer.py
--- a/tests/test_models/test_temporal_fusion_transformer.py
+++ b/tests/test_models/test_temporal_fusion_transformer.py
@@ -215,7 +215,12 @@ def test_hyperparameter_optimization_integration(dataloaders_with_covariates, tm
             max_epochs=1,
             n_trials=3,
             log_dir=tmp_path,
-            trainer_kwargs=dict(fast_dev_run=True, limit_train_batches=5),
+            trainer_kwargs=dict(
+                fast_dev_run=True,
+                limit_train_batches=5,
+                # overwrite default trainer kwargs
+                progress_bar_refresh_rate=20,
+            ),
             use_learning_rate_finder=use_learning_rate_finder,
         )
     finally:
Repository: jdb78/pytorch-forecasting

Issue:
MAPE metric issue
- PyTorch-Forecasting version: 0.8.2
- PyTorch version: 1.7.0+cu101
- Python version: 3.6.9
- Operating System: Windows 

Just wanted to notify you that MAPE in the latest version might be calculated incorrectly. 

Also wanted to point out in the docs & code comments, MAPE & MAE are Mean Absolute. There's a typo listing those metrics as Mean Average...
![image](https://user-images.githubusercontent.com/24784133/104771325-1d896880-5740-11eb-88b2-f7a75b0f39ab.png)

Also as you can see the difference of how MAPE looks in previous versions of Pytorch Forecasting (on the right) vs. the most recent (on the left). The screenshot is from different runs logged in Weights & Biases.
![image](https://user-images.githubusercontent.com/24784133/104772540-9806b800-5741-11eb-97fd-42c8417e1f70.png)


Discussion:
Great catch. Noticed as well that pytorch lightning adjusted the naming conventions. Will give it a look.

Resolving patch:
diff --git a/pytorch_forecasting/metrics.py b/pytorch_forecasting/metrics.py
--- a/pytorch_forecasting/metrics.py
+++ b/pytorch_forecasting/metrics.py
@@ -638,7 +638,7 @@ def loss(self, y_pred: torch.Tensor, target: torch.Tensor) -> torch.Tensor:
 
 class SMAPE(MultiHorizonMetric):
     """
-    Symmetric mean average percentage. Assumes ``y >= 0``.
+    Symmetric mean absolute percentage. Assumes ``y >= 0``.
 
     Defined as ``2*(y - y_pred).abs() / (y.abs() + y_pred.abs())``
     """
@@ -651,7 +651,7 @@ def loss(self, y_pred, target):
 
 class MAPE(MultiHorizonMetric):
     """
-    Mean average percentage. Assumes ``y >= 0``.
+    Mean absolute percentage. Assumes ``y >= 0``.
 
     Defined as ``(y - y_pred).abs() / y.abs()``
     """


Test patch:
Repository: jdb78/pytorch-forecasting

Issue:
Predict progress bar only works in notebook
If I use the `predict` method with `show_progress_bar=True` not in a Jupyter notebook, then the progressbar doesn't update. 
It uses `from tqdm.notebook import tqdm`, but with `from tqdm.autonotebook import tqdm` it could be more general.


Discussion:

Resolving patch:
diff --git a/pytorch_forecasting/models/base_model.py b/pytorch_forecasting/models/base_model.py
--- a/pytorch_forecasting/models/base_model.py
+++ b/pytorch_forecasting/models/base_model.py
@@ -16,7 +16,7 @@
 from torch.nn.utils import rnn
 from torch.optim.lr_scheduler import LambdaLR, ReduceLROnPlateau
 from torch.utils.data import DataLoader
-from tqdm.notebook import tqdm
+from tqdm.autonotebook import tqdm
 
 from pytorch_forecasting.data import TimeSeriesDataSet
 from pytorch_forecasting.data.encoders import EncoderNormalizer, GroupNormalizer, MultiNormalizer, NaNLabelEncoder


Test patch:
Repository: jdb78/pytorch-forecasting

Issue:
`optimizer_params` not taken into account
- PyTorch-Forecasting version: 0.9.1

`pytorch-forecasting` ignores all values within `optimizer_params` when the `optimizer` param of the `BaseModel` is set to "adam", "adamw" or "ranger", as can be seen [here](https://github.com/jdb78/pytorch-forecasting/blob/e5af8950698b6c79394a1b7178017fa9192f377e/pytorch_forecasting/models/base_model.py#L877). Weirdly enough, it isn't the case when `optimizer="sgd"`.

I'd expect `optimizer_params` to be consistently taken into account.


Discussion:

Resolving patch:
diff --git a/pytorch_forecasting/models/base_model.py b/pytorch_forecasting/models/base_model.py
--- a/pytorch_forecasting/models/base_model.py
+++ b/pytorch_forecasting/models/base_model.py
@@ -913,11 +913,15 @@ def configure_optimizers(self):
             except TypeError:  # in case there is no weight decay
                 optimizer = self.optimizer(self.parameters(), lr=lr, **optimizer_params)
         elif self.hparams.optimizer == "adam":
-            optimizer = torch.optim.Adam(self.parameters(), lr=lr)
+            optimizer = torch.optim.Adam(
+                self.parameters(), lr=lr, weight_decay=self.hparams.weight_decay, **optimizer_params
+            )
         elif self.hparams.optimizer == "adamw":
-            optimizer = torch.optim.AdamW(self.parameters(), lr=lr, weight_decay=self.hparams.weight_decay)
+            optimizer = torch.optim.AdamW(
+                self.parameters(), lr=lr, weight_decay=self.hparams.weight_decay, **optimizer_params
+            )
         elif self.hparams.optimizer == "ranger":
-            optimizer = Ranger(self.parameters(), lr=lr, weight_decay=self.hparams.weight_decay)
+            optimizer = Ranger(self.parameters(), lr=lr, weight_decay=self.hparams.weight_decay, **optimizer_params)
         elif self.hparams.optimizer == "sgd":
             optimizer = torch.optim.SGD(
                 self.parameters(), lr=lr, weight_decay=self.hparams.weight_decay, **optimizer_params


Test patch: