internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: stephenmcd/mezzanine

Issue:
[BUG] `set_dynamic_settings` always sets the MySQL test database collation to `utf8_general_ci`
### Is there an existing issue for this?

- [X] I have searched the existing Issues

### Current Behavior

For MySQL databases, the `set_dynamic_settings` util always sets the MySQL test database collation to `utf8_general_ci`: https://github.com/stephenmcd/mezzanine/blob/001a16b052578ba8b29e847a1949dcdee5b400f6/mezzanine/utils/conf.py#L250
Unfortunately, the `utf8_general_ci` collation is only compatible with the `utf8` charset, which means that it is not possible to run tests with a test database that uses a different charset, eg. `utf8mb4_*` charsets. When attempting to do so, we see an error like:
```
(1253, "COLLATION 'utf8_general_ci' is not valid for CHARACTER SET 'utf8mb4'")
```

### Expected Behavior

We should be able to set the test database collation in the app settings and have the `set_dynamic_settings` util respect that, rather than overriding it.

### Steps To Reproduce

Here is a repo containing a sample Django app with mezzanine, that demonstrates the issue: https://github.com/bevy/mezzanine-fix-demo
Following the steps in the repo and running `tox`, you can see that the error occurs with mezzanine 5.1.0 and both Django 2.2 and 3.2 (but does *not* occur when using a [version of mezzanine](https://github.com/stephenmcd/mezzanine/commit/06281cf9c030e17f342d2eed1c3b267728c07870) that defers to the test database collation in the app settings).

### Environment

```markdown
- Operating System: macOS Big Sur/Ubuntu 18.04 
- Python version: 3.7
- Django version: 2.2/3.2
- Database engine and version: MySQL 5.7
- Mezzanine version: 5.1.0
```


### Anything else?

This commit exists to fix the issue, but hasn't been merged yet! https://github.com/stephenmcd/mezzanine/commit/06281cf9c030e17f342d2eed1c3b267728c07870


Discussion:

Resolving patch:
diff --git a/mezzanine/utils/conf.py b/mezzanine/utils/conf.py
--- a/mezzanine/utils/conf.py
+++ b/mezzanine/utils/conf.py
@@ -247,7 +247,7 @@ def mezzanine_settings():
                 db["NAME"] = db_path
         elif shortname == "mysql":
             # Required MySQL collation for tests.
-            db.setdefault("TEST", {})["COLLATION"] = "utf8_general_ci"
+            db.setdefault("TEST", {}).setdefault("COLLATION", "utf8_general_ci")
 
 
 def real_project_name(project_name):


Test patch:
Repository: helmholtz-analytics/heat

Issue:
Fix project description on PyPI
PyPI requires both a description and long_description to be set, with the former being used for listing a project among others and the latter for the detailed project page.


Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -1,19 +1,28 @@
 from distutils.core import setup
 
+with open("README.md", "r") as fh:
+    long_description = fh.read()
+
 setup(
     name='heat',
     packages=['heat'],
     version='0.0.1',
     description='A framework for high performance data analytics and machine learning.',
+    long_description=long_description,
+    long_description_content_type='text/markdown',
     author='Helmholtz Association',
     author_email='martin.siggel@dlr.de',
     url='https://github.com/helmholtz-analytics/heat',
-    #  download_url = 'https://github.com/helmholtz-analytics/heat/archive/0.1.tar.gz', # TBD
     keywords=['data', 'analytics', 'tensors', 'distributed', 'gpu'],
-    classifiers=[],
+    classifiers=[
+        'Development Status :: 2 - Pre-Alpha',
+        'Programming Language :: Python :: 3.5',
+        'License :: OSI Approved :: MIT License',
+        'Intended Audience :: Science/Research',
+        'Topic :: Scientific/Engineering'
+    ],
     install_requires=[
         'numpy>=1.13.0',
-        # 'torch>=0.4.0'
     ],
     extras_require={
         'hdf5': ['h5py>=2.8.0']


Test patch:
Repository: helmholtz-analytics/heat

Issue:
ht.randn(): unit test expects pytorch to detect malformed input
The unit test that checks whether `ht.randn` throws a `ValueError` depends on the fact that pytorch
shall detect malformed input. Unfortunately, this is not the case for my pytorch installation (trying to print the resulting tensor, however, results in an infinite loop, such that we can be sure that pytorch does not magically know how to handle negative dimensions ... )

For completeness the pytest output:
```
        with self.assertRaises(ValueError):
>           ht.randn(-1, 3, dtype=ht.float64)
E           AssertionError: ValueError not raised

heat/core/tests/test_tensor.py:278: AssertionError
```

There seems to be already some `all_ints` variable available in `ht.randn`, which could be augmented to also check for positive arguments?


Discussion:

Resolving patch:
diff --git a/heat/core/tensor.py b/heat/core/tensor.py
--- a/heat/core/tensor.py
+++ b/heat/core/tensor.py
@@ -786,8 +786,13 @@ def randn(*args, dtype = torch.float32, split = None):
     """
     num_of_param = len(args)
 
-    # check if all positional arguments are integers
-    all_ints = all([isinstance(_, int) for _ in args])
+    # check if all positional arguments are integers and greater than zero
+    all_ints = all(isinstance(_, int) for _ in args)
+    if not all_ints:
+        raise TypeError("Only integer-valued dimensions as arguments possible")
+    all_positive = all(_ > 0 for _ in args)
+    if not all_positive:
+        raise ValueError("Not all tensor dimensions are positive")
 
     # define shape of tensor according to args
     gshape = (args)


Test patch:
Repository: helmholtz-analytics/heat

Issue:
ht.floor: fix docstring to properly render math
Everytime I edit edit `tensor.py` and run pytest, pytest prints a warning that the docstring of ht.floor contains an unknown escape sequence `\lfloor`. Mysteriously, this warning does not show up in the second run of pytest, which might explain why this was not caught in CI yet.


Discussion:

Resolving patch:
diff --git a/heat/core/tensor.py b/heat/core/tensor.py
--- a/heat/core/tensor.py
+++ b/heat/core/tensor.py
@@ -225,10 +225,10 @@ def exp(self, out=None):
         return operations.exp(self, out)
 
     def floor(self, out=None):
-        """
+        r"""
         Return the floor of the input, element-wise.
 
-        The floor of the scalar x is the largest integer i, such that i <= x. It is often denoted as \lfloor x \rfloor.
+        The floor of the scalar x is the largest integer i, such that i <= x. It is often denoted as :math:`\lfloor x \rfloor`.
 
         Parameters
         ----------


Test patch:
Repository: helmholtz-analytics/heat

Issue:
unrecieved Isend in balance function
there is a bug in the balance function where if it is used before another function which uses isend and recv then the recv in the second function will fail. the solution to this is to move the line with `send_amt = if else...`  outside of the if loop


Discussion:

Resolving patch:
diff --git a/heat/core/dndarray.py b/heat/core/dndarray.py
--- a/heat/core/dndarray.py
+++ b/heat/core/dndarray.py
@@ -553,8 +553,8 @@ def balance_(self):
                 if self.comm.rank == spr:
                     for pr in range(spr):
                         send_amt = abs((chunk_map[pr, self.split] - lshape_map[pr, self.split]).item())
+                        send_amt = send_amt if send_amt < self.lshape[self.split] else self.lshape[self.split]
                         if send_amt:
-                            send_amt = send_amt if send_amt < self.lshape[self.split] else self.lshape[self.split]
                             send_slice[self.split] = slice(0, send_amt)
                             keep_slice[self.split] = slice(send_amt, self.lshape[self.split])
 
@@ -592,8 +592,8 @@ def balance_(self):
             if self.comm.rank == spr:
                 for pr in range(self.comm.size - 1, spr, -1):
                     send_amt = abs((chunk_map[pr, self.split] - lshape_map[pr, self.split]).item())
+                    send_amt = send_amt if send_amt < self.lshape[self.split] else self.lshape[self.split]
                     if send_amt:
-                        send_amt = send_amt if send_amt < self.lshape[self.split] else self.lshape[self.split]
                         send_slice[self.split] = slice(self.lshape[self.split] - send_amt, self.lshape[self.split])
                         keep_slice[self.split] = slice(0, self.lshape[self.split] - send_amt)
 


Test patch:
Repository: helmholtz-analytics/heat

Issue:
Update lasso example
**Description**
The lasso example references heat.ml.regression.lasso

**To Reproduce**
Steps to reproduce the behavior:
1. Which module/class/function is affected?
  examples/lasso/demo.py
2. What are the circumstances under which the bug appears?
  run demo.py
3. What is the exact error-message/errorous behavious?
  `Traceback (most recent call last):
  File "demo.py", line 10, in <module>
    import heat.ml.regression.lasso as lasso
ModuleNotFoundError: No module named 'heat.ml'`

**Expected behavior**
Should import heat.core.regression.lasso

**Version Info**
 Which version are you using?
0.2.1

Update lasso example
**Description**
The lasso example references heat.ml.regression.lasso

**To Reproduce**
Steps to reproduce the behavior:
1. Which module/class/function is affected?
  examples/lasso/demo.py
2. What are the circumstances under which the bug appears?
  run demo.py
3. What is the exact error-message/errorous behavious?
  `Traceback (most recent call last):
  File "demo.py", line 10, in <module>
    import heat.ml.regression.lasso as lasso
ModuleNotFoundError: No module named 'heat.ml'`

**Expected behavior**
Should import heat.core.regression.lasso

**Version Info**
 Which version are you using?
0.2.1



Discussion:

Resolving patch:
diff --git a/examples/lasso/demo.py b/examples/lasso/demo.py
--- a/examples/lasso/demo.py
+++ b/examples/lasso/demo.py
@@ -7,7 +7,7 @@
 import heat as ht
 from matplotlib import pyplot as plt
 from sklearn import datasets
-import heat.ml.regression.lasso as lasso
+import heat.core.regression.lasso as lasso
 import plotfkt
 
 # read scikit diabetes data set


Test patch:
Repository: helmholtz-analytics/heat

Issue:
Heat software development status "Beta"
**Related**
--

**Feature functionality**
The software development status in PyPI is listed as "3 - Alpha". We are currently considering Heat as Beta, so this should be reflected in the status, which I propose to set to "4 - Beta".

**Additional context**
--



Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -23,7 +23,7 @@
     keywords=["data", "analytics", "tensors", "distributed", "gpu"],
     python_requires="~=3.6",
     classifiers=[
-        "Development Status :: 3 - Alpha",
+        "Development Status :: 4 - Beta",
         "Programming Language :: Python :: 3.6",
         "Programming Language :: Python :: 3.7",
         "Programming Language :: Python :: 3.8",


Test patch:
Repository: helmholtz-analytics/heat

Issue:
Clean up Lasso file, make examples work




Discussion:
bump, is this done?

Resolving patch:
diff --git a/examples/classification/demo_knn.py b/examples/classification/demo_knn.py
--- a/examples/classification/demo_knn.py
+++ b/examples/classification/demo_knn.py
@@ -10,7 +10,7 @@
 from heat.classification.kneighborsclassifier import KNeighborsClassifier
 
 # Load dataset from hdf5 file
-X = ht.load_hdf5("../../heat/datasets/data/iris.h5", dataset="data", split=0)
+X = ht.load_hdf5("../../heat/datasets/iris.h5", dataset="data", split=0)
 
 # Generate keys for the iris.h5 dataset
 keys = []
diff --git a/examples/lasso/demo.py b/examples/lasso/demo.py
--- a/examples/lasso/demo.py
+++ b/examples/lasso/demo.py
@@ -1,8 +1,11 @@
 import numpy as np
 import torch
 import sys
+import os
 
-sys.path.append("../../")
+# Fix python path if run from terminal
+curdir = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, os.path.abspath(os.path.join(curdir, "../../")))
 
 import heat as ht
 from matplotlib import pyplot as plt
@@ -14,8 +17,8 @@
 diabetes = datasets.load_diabetes()
 
 # load diabetes dataset from hdf5 file
-X = ht.load_hdf5("../../heat/datasets/data/diabetes.h5", dataset="x", split=0)
-y = ht.load_hdf5("../../heat/datasets/data/diabetes.h5", dataset="y", split=0)
+X = ht.load_hdf5("../../heat/datasets/diabetes.h5", dataset="x", split=0)
+y = ht.load_hdf5("../../heat/datasets/diabetes.h5", dataset="y", split=0)
 
 # normalize dataset #DoTO this goes into the lasso fit routine soon as issue #106 is solved
 X = X / ht.sqrt((ht.mean(X ** 2, axis=0)))
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -43,5 +43,6 @@
         "hdf5": ["h5py>=2.8.0"],
         "netcdf": ["netCDF4>=1.5.6"],
         "dev": ["pre-commit>=1.18.3"],
+        "examples": ["scikit-learn>=0.24.0", "matplotlib>=3.1.0"],
     },
 )


Test patch:
Repository: helmholtz-analytics/heat

Issue:
[Bug]: convolve with distributed kernel on multiple GPUs
### What happened?

convolve does not work if the kernel is distributed when more than one GPU is available.

### Code snippet triggering the error

```python
import heat as ht

dis_signal = ht.arange(0, 16, split=0, device='gpu', dtype=ht.int)
dis_kernel_odd = ht.ones(3, split=0, dtype=ht.int, device='gpu')
conv = ht.convolve(dis_signal, dis_kernel_odd, mode='full')
```


### Error message or erroneous outcome

```shell
$ CUDA_VISIBLE_DEVICES=0,1,2,3 srun --ntasks=2 -l python test.py 
1:Traceback (most recent call last):
1:   File ".../test.py", line 7, in <module>
1:     conv = ht.convolve(dis_signal, dis_kernel_odd, mode='full')
1:   File ".../heat-venv_2023/lib/python3.10/site-packages/heat/core/signal.py", line 161, in convolve
1:     local_signal_filtered = fc.conv1d(signal, t_v1)
1: RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0! (when checking argument for argument weight in method wrapper__cudnn_convolution)
```


### Version

main (development branch)

### Python version

3.10

### PyTorch version

1.12

### MPI version

```shell
OpenMPI 4.1.4
```



Discussion:
https://github.com/helmholtz-analytics/heat/blob/8597417f274af2cdc6cd522f2bcbe1b3e6a21a08/heat/core/signal.py#L159

This is causing the issue. Unlike our own implementation, it keeps the device between processes.
@mtar should I use 'Bcast' then, as mentioned here in #790?
> @mtar should I use 'Bcast' then, as mentioned here in #790?

Yes, that should fix it.

Resolving patch:
diff --git a/heat/core/signal.py b/heat/core/signal.py
--- a/heat/core/signal.py
+++ b/heat/core/signal.py
@@ -156,7 +156,8 @@ def convolve(a: DNDarray, v: DNDarray, mode: str = "full") -> DNDarray:
         size = v.comm.size
 
         for r in range(size):
-            rec_v = v.comm.bcast(t_v, root=r)
+            rec_v = t_v.clone()
+            v.comm.Bcast(rec_v, root=r)
             t_v1 = rec_v.reshape(1, 1, rec_v.shape[0])
             local_signal_filtered = fc.conv1d(signal, t_v1)
             # unpack 3D result into 1D


Test patch:
Repository: helmholtz-analytics/heat

Issue:
Continuous Benchmarking for Preprocessing Module
The preprocessing module currently lacks tests in the continuous benchmarking 
W.r.t. #1184 it could be interesting to track changes in required memory for the in-place-versions of the transformations. 


Discussion:
Branch [features/1246-Continuous_Benchmarking_for_Preprocessing_Module](https://github.com/helmholtz-analytics/heat/tree/features/1246-Continuous_Benchmarking_for_Preprocessing_Module) created!

Resolving patch:
diff --git a/benchmarks/cb/main.py b/benchmarks/cb/main.py
--- a/benchmarks/cb/main.py
+++ b/benchmarks/cb/main.py
@@ -9,7 +9,9 @@
 from linalg import run_linalg_benchmarks
 from cluster import run_cluster_benchmarks
 from manipulations import run_manipulation_benchmarks
+from preprocessing import run_preprocessing_benchmarks
 
 run_linalg_benchmarks()
 run_cluster_benchmarks()
 run_manipulation_benchmarks()
+run_preprocessing_benchmarks()
diff --git a/benchmarks/cb/preprocessing.py b/benchmarks/cb/preprocessing.py
new file mode 100644
--- /dev/null
+++ b/benchmarks/cb/preprocessing.py
@@ -0,0 +1,55 @@
+# flake8: noqa
+import heat as ht
+from mpi4py import MPI
+from perun import monitor
+
+# we benchmark the in-place versions (`copy=False`) of the preprocessing functions
+# for each function, both the forward and the inverse transformation are applied
+
+
+@monitor()
+def apply_inplace_standard_scaler_and_inverse(X):
+    scaler = ht.preprocessing.StandardScaler(copy=False)
+    Y = scaler.fit_transform(X)
+    X = scaler.inverse_transform(Y)
+
+
+@monitor()
+def apply_inplace_min_max_scaler_and_inverse(X):
+    scaler = ht.preprocessing.MinMaxScaler(copy=False)
+    Y = scaler.fit_transform(X)
+    X = scaler.inverse_transform(Y)
+
+
+@monitor()
+def apply_inplace_max_abs_scaler_and_inverse(X):
+    scaler = ht.preprocessing.MaxAbsScaler(copy=False)
+    Y = scaler.fit_transform(X)
+    X = scaler.inverse_transform(Y)
+
+
+@monitor()
+def apply_inplace_robust_scaler_and_inverse(X):
+    scaler = ht.preprocessing.RobustScaler(copy=False)
+    Y = scaler.fit_transform(X)
+    X = scaler.inverse_transform(Y)
+
+
+@monitor()
+def apply_inplace_normalizer(X):
+    scaler = ht.preprocessing.Normalizer(copy=False)
+    scaler.fit_transform(X)
+
+
+def run_preprocessing_benchmarks():
+    n_data_points = 5000
+    n_features = 50
+    X = ht.random.randn(n_data_points, n_features, split=0)
+
+    apply_inplace_standard_scaler_and_inverse(X)
+    apply_inplace_min_max_scaler_and_inverse(X)
+    apply_inplace_max_abs_scaler_and_inverse(X)
+    apply_inplace_robust_scaler_and_inverse(X)
+    apply_inplace_normalizer(X)
+
+    del X


Test patch:
Repository: matrix-org/synapse

Issue:
Python executable hardcoded introduces error inside virtualenv
```
ubuntu@ubuntu-xenial:~/synapse$ /home/ubuntu/synapse/bin/python  bin/synctl start
Starting ...
/usr/bin/python: No module named synapse.app
error starting (exit code: 1); see above for logs
```



Discussion:

Resolving patch:
diff --git a/synapse/app/synctl.py b/synapse/app/synctl.py
--- a/synapse/app/synctl.py
+++ b/synapse/app/synctl.py
@@ -24,7 +24,7 @@
 import sys
 import yaml
 
-SYNAPSE = ["python", "-B", "-m", "synapse.app.homeserver"]
+SYNAPSE = [sys.executable, "-B", "-m", "synapse.app.homeserver"]
 
 GREEN = "\x1b[1;32m"
 RED = "\x1b[1;31m"


Test patch:
Repository: matrix-org/synapse

Issue:
Unable to change the display name in a m.room.member event (SYN-707)
See sytest.

(Imported from https://matrix.org/jira/browse/SYN-707)

(Reported by @NegativeMjark)


Discussion:
Jira watchers: @NegativeMjark

Actually I think this is a far more general issue than relating to sync. It appears that synapse doesn't let you alter displayname / avatar_url in a member event at all. E.g. I just tried it:

    $ matrixtool client -u @telegrambot:localhost:8080 -U @telegram_115630664:localhost:8080 \
      json /_matrix/client/r0/rooms/'!'jCzGkzsQCMTIIlhWAV:localhost:8080/state/m.room.member/@telegram_115630664:localhost:8080 \
      '{"avatar_url":null,"displayname":"My name here","membership":"join"}'
    {
       "event_id" : "$14852784770hbJNl:localhost:8080"
    }

    $ matrixtool client -u @telegrambot:localhost:8080 \
      json /_matrix/client/r0/rooms/'!'jCzGkzsQCMTIIlhWAV:localhost:8080/state/m.room.member/@telegram_115630664:localhost:8080
    {
       "avatar_url" : null,
       "membership" : "join",
       "displayname" : null
    }

Resolving patch:
diff --git a/synapse/handlers/message.py b/synapse/handlers/message.py
--- a/synapse/handlers/message.py
+++ b/synapse/handlers/message.py
@@ -208,8 +208,10 @@ def create_event(self, event_dict, token_id=None, txn_id=None, prev_event_ids=No
                     content = builder.content
 
                     try:
-                        content["displayname"] = yield profile.get_displayname(target)
-                        content["avatar_url"] = yield profile.get_avatar_url(target)
+                        if "displayname" not in content:
+                            content["displayname"] = yield profile.get_displayname(target)
+                        if "avatar_url" not in content:
+                            content["avatar_url"] = yield profile.get_avatar_url(target)
                     except Exception as e:
                         logger.info(
                             "Failed to get profile information for %r: %s",


Test patch:
Repository: matrix-org/synapse

Issue:
'dynamic_thumbnails' ignores aspect when scaling
Turning on the 'dynamic_thumbnails' option on synapse results in images you see in the riot web client being scaled to the exact dimensions given in the URL query rather than the dimensions acting as a bounding box and aspect-scaling. This makes images look very silly.


Discussion:
There is a PR for the above commit: https://github.com/matrix-org/synapse/pull/1816

Resolving patch:
diff --git a/synapse/rest/media/v1/media_repository.py b/synapse/rest/media/v1/media_repository.py
--- a/synapse/rest/media/v1/media_repository.py
+++ b/synapse/rest/media/v1/media_repository.py
@@ -240,6 +240,9 @@ def _generate_thumbnail(self, input_path, t_path, t_width, t_height,
         if t_method == "crop":
             t_len = thumbnailer.crop(t_path, t_width, t_height, t_type)
         elif t_method == "scale":
+            t_width, t_height = thumbnailer.aspect(t_width, t_height)
+            t_width = min(m_width, t_width)
+            t_height = min(m_height, t_height)
             t_len = thumbnailer.scale(t_path, t_width, t_height, t_type)
         else:
             t_len = None


Test patch: