internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: yt-project/yt

Issue:
BUG: Octree loading with 1 zone not functioning at present
This comes from the `load_octree` docstring:

```python
import yt
import numpy as np
oct_mask = np.zeros(25)
oct_mask[[0,  5,  7, 16]] = 8
octree_mask = np.array(oct_mask, dtype=np.uint8)
quantities = {}
quantities["gas", "density"] = np.random.random((22, 1))
bbox = np.array([[-10.0, 10.0], [-10.0, 10.0], [-10.0, 10.0]])

ds_octree = yt.load_octree(
    octree_mask=octree_mask,
    data=quantities,
    bbox=bbox,
    num_zones=1,
    partial_coverage=0,
)

s = yt.SlicePlot(ds_octree, "x", "density")
s.save()
```

At present it isn't working; I am working on a fix.  I believe it's related to the `nzones != 2` PR, but I'm not certain.


Discussion:

Resolving patch:
diff --git a/yt/loaders.py b/yt/loaders.py
--- a/yt/loaders.py
+++ b/yt/loaders.py
@@ -1096,11 +1096,11 @@ def load_octree(
     -------
 
     >>> import numpy as np
-    >>> oct_mask = np.zeros(25)
+    >>> oct_mask = np.zeros(33) # 5 refined values gives 7 * 4 + 5 octs to mask
     ... oct_mask[[0,  5,  7, 16]] = 8
     >>> octree_mask = np.array(oct_mask, dtype=np.uint8)
     >>> quantities = {}
-    >>> quantities["gas", "density"] = np.random.random((22, 1))
+    >>> quantities["gas", "density"] = np.random.random((29, 1)) # num of false's
     >>> bbox = np.array([[-10.0, 10.0], [-10.0, 10.0], [-10.0, 10.0]])
 
     >>> ds = load_octree(


Test patch:
Repository: yt-project/yt

Issue:
BUG: min_level and max_level are not reliably set
Some of the handlers are not correctly setting `max_level`, which then propagates to things like `ds.r[:]`.  For instance:

```python
import yt
import numpy as np
oct_mask = np.zeros(33)
oct_mask[[0,  5,  7, 16]] = 8
octree_mask = np.array(oct_mask, dtype=np.uint8)
quantities = {}
quantities["gas", "density"] = np.random.random((29, 1))
bbox = np.array([[-10.0, 10.0], [-10.0, 10.0], [-10.0, 10.0]])

ds_octree = yt.load_octree(
        octree_mask=octree_mask,
        data=quantities,
        bbox=bbox,
        num_zones=1,
        partial_coverage=0,
)
#ds_octree.max_level = 10

print(ds_octree.r[:]["ones"])
```

This does not actually return anything!  The array is empty, because the `_selector` that is created has a `max_level` that it inherits from the `ds_octree`, which is set to 0.  Setting it to any value at or greater than the maximum level will work to get all the values.

This is kind of a big deal I think, and probably worth being a blocker.  I don't know when this was introduced.


Discussion:

Resolving patch:
diff --git a/yt/frontends/stream/data_structures.py b/yt/frontends/stream/data_structures.py
--- a/yt/frontends/stream/data_structures.py
+++ b/yt/frontends/stream/data_structures.py
@@ -866,6 +866,8 @@ def _initialize_oct_handler(self):
             "partial_coverage": self.ds.partial_coverage,
         }
         self.oct_handler = OctreeContainer.load_octree(header)
+        # We do now need to get the maximum level set, as well.
+        self.ds.max_level = self.oct_handler.max_level
 
     def _identify_base_chunk(self, dobj):
         if getattr(dobj, "_chunk_info", None) is None:


Test patch:
diff --git a/yt/frontends/stream/tests/test_stream_octree.py b/yt/frontends/stream/tests/test_stream_octree.py
--- a/yt/frontends/stream/tests/test_stream_octree.py
+++ b/yt/frontends/stream/tests/test_stream_octree.py
@@ -1,4 +1,5 @@
 import numpy as np
+from numpy.testing import assert_equal
 
 import yt
 
@@ -36,7 +37,7 @@ def test_octree():
     octree_mask = np.array(OCT_MASK_LIST, dtype=np.uint8)
 
     quantities = {}
-    quantities[("gas", "density")] = np.ones((22, 1), dtype="float64")
+    quantities[("gas", "density")] = np.random.random((22, 1))
 
     bbox = np.array([[-10.0, 10.0], [-10.0, 10.0], [-10.0, 10.0]])
 
@@ -50,3 +51,10 @@ def test_octree():
 
     proj = ds.proj(("gas", "density"), "x")
     proj[("gas", "density")]
+
+    assert_equal(ds.r[:]["ones"].size, 22)
+    rho1 = quantities["gas", "density"].ravel()
+    rho2 = ds.r[:]["density"].copy()
+    rho1.sort()
+    rho2.sort()
+    assert_equal(rho1, rho2)
Repository: yt-project/yt

Issue:
BUG: marker annotations are broken (regression in yt 4.2.1)
### Bug report

**Bug summary**

`annotate_marker` is broken. Originally reported by Sriram Swamynathan on Slack.

**Code for reproduction**

adapted from docs
```python
import yt

yt.set_log_level('info')
ds = yt.load("IsolatedGalaxy/galaxy0030/galaxy0030")
s = yt.SlicePlot(ds, "x", ("gas", "density"))
s.set_axes_unit("kpc")

# Plot marker and text in data coords
s.annotate_marker((0.2, 0.5, 0.9), coord_system="data")
s.annotate_text((0.2, 0.5, 0.9), "data: (0.2, 0.5, 0.9)", coord_system="data")

# Plot marker and text in plot coords
s.annotate_marker((200, -300), coord_system="plot")
s.annotate_text((200, -300), "plot: (200, -300)", coord_system="plot")

# Plot marker and text in axis coords
s.annotate_marker((0.1, 0.2), coord_system="axis")
s.annotate_text((0.1, 0.2), "axis: (0.1, 0.2)", coord_system="axis")

# Plot marker and text in figure coords
# N.B. marker will not render outside of axis bounds
s.annotate_marker((0.1, 0.2), coord_system="figure", color="black")
s.annotate_text(
    (0.1, 0.2),
    "figure: (0.1, 0.2)",
    coord_system="figure",
    text_args={"color": "black"},
)
s.save("/tmp/test_marker.png")
```

**Actual outcome**

![test_marker](https://github.com/yt-project/yt/assets/14075922/4e64822e-22af-40df-a942-c4c499413af3)


**Expected outcome**
![test_marker](https://github.com/yt-project/yt/assets/14075922/837d85f5-aeba-4f7e-a9f1-96037369d871)

(this is the output from yt 4.2.0)


The change bisect to 846d9cc4e949c053a09d39ea73b0fbd89fff4dd4 (#4475)


Discussion:

Resolving patch:
diff --git a/yt/visualization/plot_modifications.py b/yt/visualization/plot_modifications.py
--- a/yt/visualization/plot_modifications.py
+++ b/yt/visualization/plot_modifications.py
@@ -1932,18 +1932,20 @@ def __init__(
         self.plot_args = {
             "color": "white",
             "s": 50,
-            "transform": None,
             **(plot_args or {}),
             **kwargs,
         }
         self.coord_system = coord_system
+        self.transform = None
 
     def __call__(self, plot):
         x, y = self._sanitize_coord_system(
             plot, self.pos, coord_system=self.coord_system
         )
         x, y = self._sanitize_xy_order(plot, x, y)
-        plot._axes.scatter(x, y, marker=self.marker, **self.plot_args)
+        plot._axes.scatter(
+            x, y, marker=self.marker, transform=self.transform, **self.plot_args
+        )
         self._set_plot_limits(plot)
 
 


Test patch:
Repository: yt-project/yt

Issue:
BLD: future-compat issue with numpy (numpy 1.7 deprecated C API)
### Bug report

**Bug summary**

We've had build-time warnings `"Using deprecated NumPy API..."` for ever, and until recently there was nothing to be done about it, because the C code using deprecating numpy API was generated by Cython and there was no way to prevent that. However [it's now possible with Cython 3.0](http://docs.cython.org/en/latest/src/userguide/migrating_to_cy30.html?highlight=numpy%20deprecated%20api#numpy-c-api).


It's not clear exactly *when* Numpy 1.7 deprecated API will be dropped but with numpy 2.0 around the corner, I'd expect it sooner rather than later.



Discussion:

Resolving patch:
diff --git a/setupext.py b/setupext.py
--- a/setupext.py
+++ b/setupext.py
@@ -409,6 +409,12 @@ def finalize_options(self):
             self.include_dirs.append(numpy.get_include())
             self.include_dirs.append(ewah_bool_utils.get_include())
 
+            define_macros = [("NPY_NO_DEPRECATED_API", "NPY_1_7_API_VERSION")]
+            if self.define is None:
+                self.define = define_macros
+            else:
+                self.define.extend(define_macros)
+
         def build_extensions(self):
             self.check_extensions_list(self.extensions)
 


Test patch:
Repository: yt-project/yt

Issue:
BLD: oldest-supported-numpy is now unneeded
Since numpy 1.25, building backward compatible wheels against newer versions of numpy is supported, see:
https://numpy.org/doc/stable/release/1.25.0-notes.html#compiling-against-the-numpy-c-api-is-now-backwards-compatible-by-default

As a result, `oldest-supported-numpy` will likely stop being updated in the near future and we should migrate back to `numpy>=1.25` as a build-time requirement instead.

I currently have no reasons to think this migration is urgent, but depending on how things play out in the coming weeks it might be necessary to do it in order to add compatibility with Python 3.12

It may still be worth trying it out sooner rather than later, if time allows it.


Discussion:
important note: numpy 1.25 doesn't support Python 3.8, so `oldest-supported-numpy` is still needed for that specific version.
Xref https://github.com/scipy/oldest-supported-numpy/issues/76

One unexpected lesson from the discussion upstream is that wheels built with numpy 1.x (e.g., using `oldest-supported-numpy`) will not be runtime-compatible with numpy 2.0 (due early 2024), but the new backward-compatibility mechanism introduced in numpy 1.25 will allow wheels built on 2.0 to run fine with older numpys. With this in mind, migrating away from `oldest-supported-numpy` appear like a necessary step towards compatibility with numpy 2.0, so let's give it a try.

Resolving patch:
diff --git a/setupext.py b/setupext.py
--- a/setupext.py
+++ b/setupext.py
@@ -409,7 +409,15 @@ def finalize_options(self):
             self.include_dirs.append(numpy.get_include())
             self.include_dirs.append(ewah_bool_utils.get_include())
 
-            define_macros = [("NPY_NO_DEPRECATED_API", "NPY_1_7_API_VERSION")]
+            define_macros = [
+                ("NPY_NO_DEPRECATED_API", "NPY_1_7_API_VERSION"),
+            ]
+            if sys.version_info >= (3, 9):
+                # keep in sync with runtime requirements (pyproject.toml)
+                define_macros.append(("NPY_TARGET_VERSION", "NPY_1_18_API_VERSION"))
+            else:
+                pass
+
             if self.define is None:
                 self.define = define_macros
             else:


Test patch:
Repository: yt-project/yt

Issue:
BUG: instantiating a plot window that requires cartopy crash with internal error if cartopy isn't installed 
### Bug report

**Bug summary**

most likely this regression is a side effect of #4621
**Code for reproduction**

```python
import yt
import yt.testing

ds = yt.testing.fake_amr_ds(
    fields=[("gas", "density")],
    units=["g/cm**3"],
    geometry="geographic",
)
yt.SlicePlot(ds, "altitude", "density")
```

**Actual outcome**

```python-traceback
Traceback (most recent call last):
  File "/Users/robcleme/dev/yt-project/yt/yt/utilities/on_demand_imports.py", line 78, in inner
    return func(self)
           ^^^^^^^^^^
  File "/Users/robcleme/dev/yt-project/yt/yt/utilities/on_demand_imports.py", line 200, in crs
    import cartopy.crs as crs
ModuleNotFoundError: No module named 'cartopy'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/robcleme/dev/yt-project/yt/t.py", line 11, in <module>
    yt.SlicePlot(ds, "altitude", "density")
  File "/Users/robcleme/dev/yt-project/yt/yt/visualization/plot_window.py", line 1837, in __init__
    PWViewerMPL.__init__(
  File "/Users/robcleme/dev/yt-project/yt/yt/visualization/plot_window.py", line 864, in __init__
    PlotWindow.__init__(self, *args, **kwargs)
  File "/Users/robcleme/dev/yt-project/yt/yt/visualization/plot_window.py", line 249, in __init__
    self._projection = get_mpl_transform(projection)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/robcleme/dev/yt-project/yt/yt/visualization/geo_plot_utils.py", line 78, in get_mpl_transform
    instantiated_func = valid_transforms[key](*args, **kwargs)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/robcleme/dev/yt-project/yt/yt/visualization/geo_plot_utils.py", line 41, in _func
    return getattr(cartopy.crs, transform_name)(*args, **kwargs)
                   ^^^^^^^^^^^
  File "/Users/robcleme/dev/yt-project/yt/yt/utilities/on_demand_imports.py", line 80, in inner
    return self._default_factory(self._name, exc)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: NotCartopy.__init__() takes 2 positional arguments but 3 were given
```

**Expected outcome**

The error message should state that cartopy is missing


Discussion:
taking that back, this bug is much older and a fix should be backported to yt 4.2

Resolving patch:
diff --git a/yt/utilities/on_demand_imports.py b/yt/utilities/on_demand_imports.py
--- a/yt/utilities/on_demand_imports.py
+++ b/yt/utilities/on_demand_imports.py
@@ -172,8 +172,8 @@ class NotCartopy(NotAModule):
     for cartopy imports.
     """
 
-    def __init__(self, pkg_name):
-        self.pkg_name = pkg_name
+    def __init__(self, pkg_name, exc: Optional[BaseException] = None):
+        super().__init__(pkg_name, exc)
         if any(s in sys.version for s in ("Anaconda", "Continuum")):
             # the conda-based installs of cartopy don't have issues with the
             # GEOS library, so the error message for users with conda can be


Test patch:
Repository: yt-project/yt

Issue:
BUG: calling `yt.load_sample` with no arguments is broken on main
### Bug report

**Bug summary**

This is a regression in yt 4.3 (dev)

I suspect it came from #4646 (cc @chrishavlin). It also shows that this path is currently not tested.

**Code for reproduction**

```shell
$ python -c "import yt; yt.load_sample()`
```

**Actual outcome**

```python-traceback
One can see which sample datasets are available at: https://yt-project.org/data
or alternatively by running: yt.sample_data.api.get_data_registry_table()
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/robcleme/dev/yt-project/yt/yt/loaders.py", line 1666, in load_sample
    loadable_path, load_kwargs = _get_sample_data(
    ^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: cannot unpack non-iterable NoneType object
```

**Expected outcome**
with yt 4.2.2, no error is raised
```
One can see which sample datasets are available at: https://yt-project.org/data
or alternatively by running: yt.sample_data.api.get_data_registry_table()
```

**Version Information**
  * Operating System: any
  * Python Version: any
  * yt version: 4.3.0dev0
  * Other Libraries (if applicable): N/A




Discussion:
Yup, my pr seems the likely culprit. Happy to put in a fix later today if you don't get to it first :) 
Be my guest !

Resolving patch:
diff --git a/yt/loaders.py b/yt/loaders.py
--- a/yt/loaders.py
+++ b/yt/loaders.py
@@ -1487,14 +1487,6 @@ def _get_sample_data(
     # broadcast to other processes during parallel execution).
     import tarfile
 
-    if fn is None:
-        print(
-            "One can see which sample datasets are available at: https://yt-project.org/data\n"
-            "or alternatively by running: yt.sample_data.api.get_data_registry_table()",
-            file=sys.stderr,
-        )
-        return None
-
     from yt.sample_data.api import (
         _download_sample_data_file,
         _get_test_data_dir_path,
@@ -1663,6 +1655,15 @@ def load_sample(
     - Corresponding sample data live at https://yt-project.org/data
 
     """
+
+    if fn is None:
+        print(
+            "One can see which sample datasets are available at: https://yt-project.org/data\n"
+            "or alternatively by running: yt.sample_data.api.get_data_registry_table()",
+            file=sys.stderr,
+        )
+        return None
+
     loadable_path, load_kwargs = _get_sample_data(
         fn, progressbar=progressbar, timeout=timeout, **kwargs
     )


Test patch:
diff --git a/yt/tests/test_load_sample.py b/yt/tests/test_load_sample.py
--- a/yt/tests/test_load_sample.py
+++ b/yt/tests/test_load_sample.py
@@ -196,3 +196,9 @@ def test_data_dir_broken():
     )
     with pytest.warns(UserWarning, match=msg):
         load_sample("ToroShockTube")
+
+
+def test_filename_none(capsys):
+    assert load_sample() is None
+    captured = capsys.readouterr()
+    assert "yt.sample_data.api.get_data_registry_table" in captured.err
Repository: yt-project/yt

Issue:
DEP: (yt 4.3) set upper limit on runtime requirement on numpy (<2.0)
This issue is a reminder that, if yt 4.3.0 is released, as scheduled (sometime about end of September), *before* numpy 2.0 (currently aimed at December 2023), we should update the runtime requirement on the backport branch (basically replaying https://github.com/yt-project/yt/pull/4573).


Discussion:

Resolving patch:
diff --git a/doc/source/conf.py b/doc/source/conf.py
--- a/doc/source/conf.py
+++ b/doc/source/conf.py
@@ -70,7 +70,7 @@
 # The short X.Y version.
 version = "4.3"
 # The full version, including alpha/beta/rc tags.
-release = "4.3-dev"
+release = "4.3.0"
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.
diff --git a/yt/_version.py b/yt/_version.py
--- a/yt/_version.py
+++ b/yt/_version.py
@@ -7,7 +7,7 @@
     "version_info",
 ]
 
-__version__ = "4.3.dev0"  # keep in sync with pyproject.toml
+__version__ = "4.3.0"  # keep in sync with pyproject.toml
 
 
 class VersionTuple(NamedTuple):


Test patch:
Repository: yt-project/yt

Issue:
MNT: CPython 3.12 compatibility (progress tracker)
Opening this to keep track of exactly where we're at and what's left to do:
- #4625
- #4626
- https://github.com/yt-project/yt/pull/4687
- [ ] future: switch tests with full dependencies to 3.12 (and remove the simple job introduced in #4687). This last step should be done when all optional dependencies with significant build time have cp312 wheels on PyPI. In particular, look out for
   - [x] [h5py](https://pypi.org/simple/h5py/)
   - [ ] [cartopy](https://pypi.org/simple/cartopy/)
   - [x] [netCDF4](https://pypi.org/simple/netCDF4/)
   - [x] [aiohttp](https://pypi.org/simple/aiohttp/)


Discussion:

Resolving patch:
diff --git a/yt/loaders.py b/yt/loaders.py
--- a/yt/loaders.py
+++ b/yt/loaders.py
@@ -1588,7 +1588,15 @@ def safe_extract(tar, path=".", members=None, *, numeric_owner=False):
                     if not is_within_directory(path, member_path):
                         raise Exception("Attempted Path Traversal in Tar File")
 
-                tar.extractall(path, members, numeric_owner=numeric_owner)
+                if sys.version_info >= (3, 12):
+                    # the filter argument is new in Python 3.12, but not specifying it
+                    # explicitly raises a deprecation warning on 3.12 and 3.13
+                    extractall_kwargs = {"filter": "data"}
+                else:
+                    extractall_kwargs = {}
+                tar.extractall(
+                    path, members, numeric_owner=numeric_owner, **extractall_kwargs
+                )
 
             safe_extract(fh, save_dir)
         os.remove(tmp_file)


Test patch:
Repository: yt-project/yt

Issue:
ENH: Only open auxilliary files for Tipsy as needed
### Bug report

**Bug summary**

In `yt/frontends/tipsy/io.py` around line 160-ish we set up a dict of filehandles for the auxiliary files.  These aren't actually necessary to set up *unless* we want to read from them.  That determination happens later on, around line 172.

It might be possible to simple change the iteration `for afield in self._aux_fields:` to `for afield in afields:`.



Discussion:

Resolving patch:
diff --git a/yt/frontends/tipsy/io.py b/yt/frontends/tipsy/io.py
--- a/yt/frontends/tipsy/io.py
+++ b/yt/frontends/tipsy/io.py
@@ -157,9 +157,12 @@ def _read_particle_data_file(self, data_file, ptf, selector=None):
         f = open(data_file.filename, "rb")
 
         # we need to open all aux files for chunking to work
-        aux_fh = {}
-        for afield in self._aux_fields:
-            aux_fh[afield] = open(data_file.filename + "." + afield, "rb")
+        _aux_fh = {}
+
+        def aux_fh(afield):
+            if afield not in _aux_fh:
+                _aux_fh[afield] = open(data_file.filename + "." + afield, "rb")
+            return _aux_fh[afield]
 
         for ptype, field_list in sorted(ptf.items(), key=lambda a: poff.get(a[0], -1)):
             if data_file.total_particles[ptype] == 0:
@@ -170,19 +173,19 @@ def _read_particle_data_file(self, data_file, ptf, selector=None):
             p = np.fromfile(f, self._pdtypes[ptype], count=count)
             auxdata = []
             for afield in afields:
-                aux_fh[afield].seek(aux_fields_offsets[afield][ptype])
+                aux_fh(afield).seek(aux_fields_offsets[afield][ptype])
                 if isinstance(self._aux_pdtypes[afield], np.dtype):
                     auxdata.append(
                         np.fromfile(
-                            aux_fh[afield], self._aux_pdtypes[afield], count=count
+                            aux_fh(afield), self._aux_pdtypes[afield], count=count
                         )
                     )
                 else:
-                    aux_fh[afield].seek(0)
+                    aux_fh(afield).seek(0)
                     sh = aux_fields_offsets[afield][ptype]
                     if tp[ptype] > 0:
                         aux = np.genfromtxt(
-                            aux_fh[afield], skip_header=sh, max_rows=count
+                            aux_fh(afield), skip_header=sh, max_rows=count
                         )
                         if aux.ndim < 1:
                             aux = np.array([aux])
@@ -209,7 +212,7 @@ def _read_particle_data_file(self, data_file, ptf, selector=None):
 
         # close all file handles
         f.close()
-        for fh in list(aux_fh.values()):
+        for fh in _aux_fh.values():
             fh.close()
 
         return return_data


Test patch: