internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: yt-project/yt

Issue:
Upcoming incompatibility with matplotlib 3.6
### Bug report

**Bug summary**

This PR was just merged upstream https://github.com/matplotlib/matplotlib/pull/22298 and breaks yt by removing a bit of the API we were using (`matplotlib.cm.cmap_d`)

This is a critical breakage, as even `import yt` will raise an `AttributeError`.

The changelog indicates that it's a deprecation cycle end but it seems like we never caught the deprecation warning in time.
It seems likely that the _right_ fix would be to move whatever code is using `cmap_d` from yt to cmyt, however I'll try to make a quick, back portable hotfix first.


Discussion:

Resolving patch:
diff --git a/yt/visualization/color_maps.py b/yt/visualization/color_maps.py
--- a/yt/visualization/color_maps.py
+++ b/yt/visualization/color_maps.py
@@ -8,6 +8,7 @@
 from packaging.version import Version
 
 from yt.funcs import get_brewer_cmap
+from yt.utilities.logger import ytLogger as mylog
 
 from . import _colormap_data as _cm
 from ._commons import MPL_VERSION
@@ -33,8 +34,6 @@ def add_colormap(name, cdict):
     """
     # Note: this function modifies the global variable 'yt_colormaps'
     yt_colormaps[name] = cc.LinearSegmentedColormap(name, cdict, 256)
-    mcm.datad[name] = cdict
-    mcm.__dict__[name] = cdict
     mcm.register_cmap(name, yt_colormaps[name])
 
 
@@ -64,8 +63,6 @@ def register_yt_colormaps_from_cmyt():
     """
     from matplotlib.pyplot import get_cmap
 
-    from yt.utilities.logger import ytLogger as mylog
-
     for hist_name, alias in _HISTORICAL_ALIASES.items():
         if MPL_VERSION >= Version("3.4.0"):
             cmap = get_cmap(alias).copy()
@@ -79,7 +76,6 @@ def register_yt_colormaps_from_cmyt():
             # Matplotlib 3.4.0 hard-forbids name collisions, but more recent versions
             # will emit a warning instead, so we emulate this behaviour regardless.
             mylog.warning("cannot register colormap '%s' (naming collision)", hist_name)
-            continue
 
 
 register_yt_colormaps_from_cmyt()
@@ -87,17 +83,18 @@ def register_yt_colormaps_from_cmyt():
 # Add colormaps in _colormap_data.py that weren't defined here
 _vs = np.linspace(0, 1, 256)
 for k, v in list(_cm.color_map_luts.items()):
+    if k in yt_colormaps:
+        continue
+    cdict = {
+        "red": np.transpose([_vs, v[0], v[0]]),
+        "green": np.transpose([_vs, v[1], v[1]]),
+        "blue": np.transpose([_vs, v[2], v[2]]),
+    }
     try:
-        colormaps = mcm._cmap_registry
-    except AttributeError:  # mpl < 3.3.0
-        colormaps = mcm.cmap_d
-    if k not in yt_colormaps and k not in colormaps:
-        cdict = {
-            "red": np.transpose([_vs, v[0], v[0]]),
-            "green": np.transpose([_vs, v[1], v[1]]),
-            "blue": np.transpose([_vs, v[2], v[2]]),
-        }
         add_colormap(k, cdict)
+    except ValueError:
+        # expected if another map with identical name was already registered
+        mylog.warning("cannot register colormap '%s' (naming collision)", k)
 
 
 def get_colormap_lut(cmap_id: Union[Tuple[str, str], str]):


Test patch:
Repository: yt-project/yt

Issue:
DEPR: many deprecation warnings are missing proper stacklevel
### Bug report

**Bug summary**

@rjfarber pointed out that the `stacklevel` argument to `issue_deprecation_warning` was currently unused despite the 27 existing calls. It seems unlikely that the default value is actually correct in every single one of these so it should be inspected more carefully.
 


Discussion:
Nevermind, I checked every calls manually and the default value is actually correct everywhere because all deprecated functions live at the surface of the user interface and none of them are decorated, so no need for tricks.
looked closer and found at least one that actually needs fixing...

Resolving patch:
diff --git a/yt/data_objects/construction_data_containers.py b/yt/data_objects/construction_data_containers.py
--- a/yt/data_objects/construction_data_containers.py
+++ b/yt/data_objects/construction_data_containers.py
@@ -183,6 +183,7 @@ def __init__(
                 "Please use method directly.",
                 since="3.2",
                 removal="4.2",
+                stacklevel=4,
             )
             method = style
         if method == "sum":


Test patch:
Repository: yt-project/yt

Issue:
BUG: yt crashes on startup if the config file is invalid toml
@cphyc originally noted this problem in #3831
```bash
echo "! this is invalid" > yt.toml 
python -c "import yt"
```
fails with the backtrace
```backtrace
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/XXX/Documents/prog/yt/yt/__init__.py", line 13, in <module>
    import yt.utilities.physical_constants as physical_constants
  File "/home/XXX/Documents/prog/yt/yt/utilities/physical_constants.py", line 3, in <module>
    from yt.units.yt_array import YTQuantity
  File "/home/XXX/Documents/prog/yt/yt/units/yt_array.py", line 3, in <module>
    from yt.funcs import array_like_field  # NOQA: F401
  File "/home/XXX/Documents/prog/yt/yt/funcs.py", line 33, in <module>
    from yt.utilities.logger import ytLogger as mylog
  File "/home/XXX/Documents/prog/yt/yt/utilities/logger.py", line 4, in <module>
    from yt.config import ytcfg
  File "/home/XXX/Documents/prog/yt/yt/config.py", line 215, in <module>
    ytcfg.read(_local_config_file)
  File "/home/XXX/Documents/prog/yt/yt/config.py", line 154, in read
    data = tomllib.load(fh)
  File "/home/XXX/anaconda3/envs/py39/lib/python3.9/site-packages/tomli/_parser.py", line 66, in load
    return loads(s, parse_float=parse_float)
  File "/home/XXX/anaconda3/envs/py39/lib/python3.9/site-packages/tomli/_parser.py", line 116, in loads
    raise suffixed_err(src, pos, "Invalid statement")
tomli.TOMLDecodeError: Invalid statement (at line 1, column 1)
```

_Originally posted by @cphyc in https://github.com/yt-project/yt/pull/3831#pullrequestreview-898674860_

We could embed this step in a `try/except` block and raise a warning instead of an error (still keeping the critical information from the original error).


Discussion:

Resolving patch:
diff --git a/yt/utilities/configure.py b/yt/utilities/configure.py
--- a/yt/utilities/configure.py
+++ b/yt/utilities/configure.py
@@ -1,4 +1,5 @@
 import os
+import warnings
 from typing import Callable, List
 
 # TODO: import tomllib from the standard library instead in Python >= 3.11
@@ -83,10 +84,16 @@ def read(self, file_names):
             if not os.path.exists(fname):
                 continue
             metadata = {"source": f"file: {fname}"}
-            with open(fname, "rb") as fh:
-                data = tomllib.load(fh)
-            self.update(data, metadata=metadata)
-            file_names_read.append(fname)
+            try:
+                with open(fname, "rb") as fh:
+                    data = tomllib.load(fh)
+            except tomllib.TOMLDecodeError as exc:
+                warnings.warn(
+                    f"Could not load configuration file {fname} (invalid TOML: {exc})"
+                )
+            else:
+                self.update(data, metadata=metadata)
+                file_names_read.append(fname)
 
         return file_names_read
 


Test patch:
Repository: yt-project/yt

Issue:
ENH: missing def for poloidal/toroidal magnetic fields in polar geometry
### Bug report

**Bug summary**

Poloidal and toroidal magnetic fields definitions in polar geometry are missing, see https://github.com/yt-project/yt/blob/e16331088faf7296672ce9d1987a09d036f458ea/yt/fields/magnetic_field.py#L87-L149



Discussion:

Resolving patch:
diff --git a/yt/fields/magnetic_field.py b/yt/fields/magnetic_field.py
--- a/yt/fields/magnetic_field.py
+++ b/yt/fields/magnetic_field.py
@@ -84,6 +84,9 @@ def _magnetic_pressure(field, data):
         units=unit_system["pressure"],
     )
 
+    _magnetic_field_toroidal = None
+    _magnetic_field_poloidal = None
+
     if registry.ds.geometry == "cartesian":
 
         def _magnetic_field_poloidal(field, data):
@@ -116,7 +119,7 @@ def _magnetic_field_toroidal(field, data):
             phi = data["index", "spherical_phi"]
             return get_sph_phi_component(Bfields, phi, normal)
 
-    elif registry.ds.geometry == "cylindrical":
+    elif registry.ds.geometry in ("cylindrical", "polar"):
 
         def _magnetic_field_poloidal(field, data):
             bm = data.get_field_parameter("bulk_magnetic_field")
@@ -146,34 +149,28 @@ def _magnetic_field_toroidal(field, data):
             bm = data.get_field_parameter("bulk_magnetic_field")
             return data[ftype, "magnetic_field_phi"] - bm[ax]
 
-    else:
-
-        # Unidentified geometry--set to None
-
-        _magnetic_field_toroidal = None
-        _magnetic_field_poloidal = None
-
-    registry.add_field(
-        (ftype, "magnetic_field_poloidal"),
-        sampling_type="local",
-        function=_magnetic_field_poloidal,
-        units=u,
-        validators=[
-            ValidateParameter("normal"),
-            ValidateParameter("bulk_magnetic_field"),
-        ],
-    )
-
-    registry.add_field(
-        (ftype, "magnetic_field_toroidal"),
-        sampling_type="local",
-        function=_magnetic_field_toroidal,
-        units=u,
-        validators=[
-            ValidateParameter("normal"),
-            ValidateParameter("bulk_magnetic_field"),
-        ],
-    )
+    if _magnetic_field_toroidal is not None:
+        registry.add_field(
+            (ftype, "magnetic_field_toroidal"),
+            sampling_type="local",
+            function=_magnetic_field_toroidal,
+            units=u,
+            validators=[
+                ValidateParameter("normal"),
+                ValidateParameter("bulk_magnetic_field"),
+            ],
+        )
+    if _magnetic_field_poloidal is not None:
+        registry.add_field(
+            (ftype, "magnetic_field_poloidal"),
+            sampling_type="local",
+            function=_magnetic_field_poloidal,
+            units=u,
+            validators=[
+                ValidateParameter("normal"),
+                ValidateParameter("bulk_magnetic_field"),
+            ],
+        )
 
     def _alfven_speed(field, data):
         B = data[ftype, "magnetic_field_strength"]


Test patch:
Repository: yt-project/yt

Issue:
DOC: cleanup fields page
The following page will require some attention before the 4.1 release, in particular regarding the new "no ambiguous fields" rules

https://yt-project.org/doc/analyzing/fields.html#field-types-known-to-yt

some notes:

the following note is wrong since #2483 (this is adressed in #3416 as a pretext to trigger docs builds)
>Note
>add_gradient_fields currently only supports Cartesian geometries!
)

the `"nbody"` particle type is not documented. This is a particle union (just like `"all"`) which contains all particle types for which a `"particle_mass"` field is available.


Discussion:

Resolving patch:
diff --git a/yt/data_objects/data_containers.py b/yt/data_objects/data_containers.py
--- a/yt/data_objects/data_containers.py
+++ b/yt/data_objects/data_containers.py
@@ -1431,7 +1431,6 @@ def _determine_fields(self, fields):
                 continue
 
             ftype, fname = self._tupleize_field(field)
-            # print(field, " : ",ftype, fname)
             finfo = self.ds._get_field_info(ftype, fname)
 
             # really ugly check to ensure that this field really does exist somewhere,


Test patch:
Repository: yt-project/yt

Issue:
REG: fail to configure yt from the command line if $HOME/.config/ doesn't exist
### Bug report

**Bug summary**

This was discovered downstream thanks to yt_astro_analysis' CI

**Code for reproduction**
```shell
yt config set --global yt suppress_stream_logging True
```

**Actual outcome**

```python-traceback
Traceback (most recent call last):
  File "/Users/robcleme/dev/yt-project/yt/yt/utilities/configure.py", line 106, in write
    file_handler.write(config_as_str)
AttributeError: 'str' object has no attribute 'write'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/robcleme/.pyenv/versions/38-yt-dev/bin/yt", line 33, in <module>
    sys.exit(load_entry_point('yt', 'console_scripts', 'yt')())
  File "/Users/robcleme/dev/yt-project/yt/yt/utilities/command_line.py", line 1632, in run_main
    args.func(args)
  File "/Users/robcleme/dev/yt-project/yt/yt/utilities/command_line.py", line 224, in run
    self(args)
  File "/Users/robcleme/dev/yt-project/yt/yt/utilities/command_line.py", line 1418, in __call__
    set_config(args.section, args.option, args.value, self.config_file)
  File "/Users/robcleme/dev/yt-project/yt/yt/utilities/configure.py", line 173, in set_config
    write_config(config_file)
  File "/Users/robcleme/dev/yt-project/yt/yt/utilities/configure.py", line 177, in write_config
    CONFIG.write(config_file)
  File "/Users/robcleme/dev/yt-project/yt/yt/utilities/configure.py", line 109, in write
    with open(file_handler, mode="w") as fh:
FileNotFoundError: [Errno 2] No such file or directory: '/Users/robcleme/.config/yt/yt.toml'
```

**Expected outcome**

The configuration dir should be created.
I bet this regression is caused by #3626


Discussion:

Resolving patch:
diff --git a/yt/utilities/configure.py b/yt/utilities/configure.py
--- a/yt/utilities/configure.py
+++ b/yt/utilities/configure.py
@@ -1,6 +1,7 @@
 import os
 import sys
 import warnings
+from pathlib import Path
 from typing import Callable, List
 
 import tomli_w
@@ -106,12 +107,19 @@ def write(self, file_handler):
         config_as_str = tomli_w.dumps(value)
 
         try:
-            # Assuming file_handler has a write attribute
+            file_path = Path(file_handler)
+        except TypeError:
+            if not hasattr(file_handler, "write"):
+                raise TypeError(
+                    f"Expected a path to a file, or a writable object, got {file_handler}"
+                ) from None
             file_handler.write(config_as_str)
-        except AttributeError:
-            # Otherwise we expect a path to a file
-            with open(file_handler, mode="w") as fh:
-                fh.write(config_as_str)
+        else:
+            pdir = file_path.parent
+            if not pdir.exists():
+                warnings.warn(f"{pdir!s} does not exist, creating it (recursively)")
+                os.makedirs(pdir)
+            file_path.write_text(config_as_str)
 
     @staticmethod
     def get_global_config_file():


Test patch:
Repository: yt-project/yt

Issue:
BUG: pending deprecation warnings from MPL 3.6
### Bug report

**Bug summary**

Matplotlib's colormap API is still evolving and new pending deprecation warnings are now emitted, which breaks yt's bleeding edge CI

example failed job: https://github.com/yt-project/yt/runs/7929987453?check_suite_focus=true
origin of the change: https://github.com/matplotlib/matplotlib/pull/23668

I note that `cmyt` is unaffected by these new warnings.


Discussion:
merging #3849 reintroduced some calls to the deprecated API so bleeding-edge CI is failing again. It should be a smaller fix this time.

Resolving patch:
diff --git a/yt/visualization/_handlers.py b/yt/visualization/_handlers.py
--- a/yt/visualization/_handlers.py
+++ b/yt/visualization/_handlers.py
@@ -5,7 +5,6 @@
 
 import numpy as np
 import unyt as un
-from matplotlib.cm import get_cmap
 from matplotlib.colors import Colormap, LogNorm, Normalize, SymLogNorm
 from packaging.version import Version
 
@@ -13,6 +12,7 @@
 from yt.config import ytcfg
 from yt.funcs import get_brewer_cmap, is_sequence, mylog
 from yt.visualization._commons import MPL_VERSION
+from yt.visualization.color_maps import _get_cmap
 
 if sys.version_info >= (3, 8):
     from typing import Literal
@@ -418,14 +418,14 @@ def draw_minorticks(self, newval) -> None:
 
     @property
     def cmap(self) -> Colormap:
-        return self._cmap or get_cmap(ytcfg.get("yt", "default_colormap"))
+        return self._cmap or _get_cmap(ytcfg.get("yt", "default_colormap"))
 
     @cmap.setter
     def cmap(self, newval) -> None:
         if isinstance(newval, Colormap) or newval is None:
             self._cmap = newval
         elif isinstance(newval, str):
-            self._cmap = get_cmap(newval)
+            self._cmap = _get_cmap(newval)
         elif is_sequence(newval):
             # tuple colormaps are from palettable (or brewer2mpl)
             self._cmap = get_brewer_cmap(newval)


Test patch:
Repository: yt-project/yt

Issue:
BUG: incorrect conversion from array to boolean
### Bug report

**Bug summary**
This line
https://github.com/yt-project/yt/blob/2f4766957a2e88461cf1473bb5873335e80b529d/yt/frontends/flash/io.py#L164

triggers a bunch of `DeprecationWarning`s in testing
```
/home/fido/workspace/yt_py38_git/yt/frontends/flash/io.py:164: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.
```

It is not directly obvious to me how it should be fixed, since I am not familiar with this region. I'll assign @matthewturk who last edited this line.


Discussion:

Resolving patch:
diff --git a/yt/frontends/flash/io.py b/yt/frontends/flash/io.py
--- a/yt/frontends/flash/io.py
+++ b/yt/frontends/flash/io.py
@@ -145,7 +145,7 @@ def _read_particle_fields(self, chunks, ptf, selector):
                 if _xyz[0].size > 0:
                     bxyz.append(_xyz)
                 mask = selector.select_points(x, y, z, 0.0)
-                blockless = _xyz[0] > 0
+                blockless = (_xyz[0] > 0).any()
                 # This checks if none of the particles within these blocks are
                 # included in the mask We need to also allow for blockless
                 # particles to be selected.


Test patch:
Repository: yt-project/yt

Issue:
BUG: iterative plot annotations are broken
### Bug report

**Bug summary**

This is a regression in yt 4.1

See this example notebook
https://github.com/neutrinoceros/yt_41_reg_report/blob/main/example.ipynb


Discussion:
bisects to 87eef6771a6c0fb720788b97e3fb0b860d9dbe6a, appropriately labelled "RFC: refactor plot callbacks registration. (...)"

Resolving patch:
diff --git a/yt/visualization/plot_modifications.py b/yt/visualization/plot_modifications.py
--- a/yt/visualization/plot_modifications.py
+++ b/yt/visualization/plot_modifications.py
@@ -30,7 +30,11 @@
 )
 from yt.utilities.math_utils import periodic_ray
 from yt.utilities.on_demand_imports import NotAModule
-from yt.visualization._commons import _swap_arg_pair_order, _swap_axes_extents
+from yt.visualization._commons import (
+    _swap_arg_pair_order,
+    _swap_axes_extents,
+    invalidate_plot,
+)
 from yt.visualization.base_plot_types import CallbackWrapper
 from yt.visualization.image_writer import apply_colormap
 from yt.visualization.plot_window import PWViewerMPL
@@ -83,6 +87,7 @@ def __init_subclass__(cls, *args, **kwargs):
         incompatible_plot_types = cls._incompatible_plot_types
         type_name = cls._type_name
 
+        @invalidate_plot
         def closure(self, *args, **kwargs):
             nonlocal supported_geometries
             nonlocal incompatible_plot_types
@@ -101,6 +106,7 @@ def closure(self, *args, **kwargs):
             if self._plot_type in incompatible_plot_types:
                 raise YTUnsupportedPlotCallback(type_name, self._plot_type)
             self._callbacks.append(cls(*args, **kwargs))
+            return self
 
         update_wrapper(
             wrapper=closure,


Test patch:
Repository: yt-project/yt

Issue:
BUG: eps_writer.multiplot_yt regression in yt 4.1
### Bug report

**Bug summary**

This bug was reported on Slack by @mihirskulkarni, I've  started minimizing the reproducer script

**Code for reproduction**

```python
import yt
import yt.visualization.eps_writer as eps
from yt.testing import fake_amr_ds

ds = fake_amr_ds(
    fields=[("gas", "density"), ("gas", "temperature")],
    units=["g/cm**3", "K"]
)

slc = yt.SlicePlot(
    ds,
    "y",
    [
        ("gas", "density"),
        ("gas", "temperature"),
    ],
)
eps_fig = eps.multiplot_yt(2, 1, slc)
```

**Actual outcome**

```
...
AttributeError: 'WindowPlotMPL' object has no attribute 'zmin'
```


This script worked on yt 4.0.5 and is broken since yt 4.1.0
The exact commit that this regression happened is 379c0826cb1ede4819995f2162d55e8d96cc6eea... This is an oversight from https://github.com/yt-project/yt/pull/3849

I'll work on a patch


Discussion:
Actually it seems that the one test in `yt/visualization/tests/test_eps_writer.py` is broken too, so my guess is that it is currently not run on any CI.

Resolving patch:
diff --git a/yt/visualization/eps_writer.py b/yt/visualization/eps_writer.py
--- a/yt/visualization/eps_writer.py
+++ b/yt/visualization/eps_writer.py
@@ -3,6 +3,7 @@
 import numpy as np
 import pyx
 from matplotlib import pyplot as plt
+from matplotlib.colors import LogNorm, Normalize
 
 from yt.config import ytcfg
 from yt.units.unit_object import Unit  # type: ignore
@@ -885,16 +886,21 @@ def colorbar_yt(self, plot, field=None, cb_labels=None, **kwargs):
                 _, _, z_title = plot._get_field_title(self.field, plot.profile)
                 _zlabel = pyxize_label(z_title)
             _zlabel = _zlabel.replace("_", r"\;")
-            _zlog = plot.get_log(self.field)[self.field]
-            if plot.plots[self.field].zmin is None:
-                zmin = plot.plots[self.field].image._A.min()
-            else:
-                zmin = plot.plots[self.field].zmin
-            if plot.plots[self.field].zmax is None:
-                zmax = plot.plots[self.field].image._A.max()
+
+            _p = plot.plots[self.field]
+            _norm = _p.norm_handler.get_norm(plot.frb[self.field])
+            norm_type = type(_norm)
+            if norm_type is LogNorm:
+                _zlog = True
+            elif norm_type is Normalize:
+                # linear scaling
+                _zlog = False
             else:
-                zmax = plot.plots[self.field].zmax
-            _zrange = (zmin, zmax)
+                raise RuntimeError(
+                    "eps_writer is not compatible with scalings other than linear and log, "
+                    f"received {norm_type}"
+                )
+            _zrange = (_norm.vmin, _norm.vmax)
         else:
             _zlabel = plot._z_label.replace("_", r"\;")
             _zlog = plot._log_z


Test patch: