Repository: pandas-dev/pandas
Pull request #58647 (2024-05-08 21:26:50)

BLD, CI: Use `cibuildwheel` to build Emscripten/Pyodide wheels, push nightlies to Anaconda.org
- [x] This is a follow-up PR to #57896 as stated in #57891
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

## Description

This PR duplicates the Pyodide wheel builder job that was set up in #57896 into `wheels.yml`, except that in this case the wheel is not tested and the wheel builder uses `cibuildwheel` enabled with Pyodide to build the wheels. Subsequently, the `upload_wheels.sh` script will be used with the appropriate repository secret to upload the Pyodide wheel to https://anaconda.org/scientific-python-nightly-wheels/pandas.

## Additional context

1. This is how this was done for NumPy: https://github.com/numpy/numpy/pull/26134
2. At a later time, these wheels will be used to configure interactive documentation via JupyterLite through `jupyterlite-sphinx` that acts upon doctest-based examples. The blocker in that regard is setting up a method to pre-install these wheels to provide them to the `jupyterlite-pyodide-kernel` and make them available at run time, and progress is being made in this area.

Adding an xref for tracking purposes, the action from `main` will be used until a stable release of `cibuildwheel` is available that can build Pyodide wheels: https://github.com/pypa/cibuildwheel/pull/1456

lithomas1: Looks like cibuildwheel might be adding pyodide here
https://github.com/pypa/cibuildwheel/pull/1456

Maybe we should wait for that PR?
agriyakhetarpal: Hi @lithomas1, I would advise holding off on that PR – it's stalled for many reasons, for example the fact that Pyodide does releases bi-annually with updated Python versions and Emscripten versions which will break the ABI – i.e., while the specification for breaking changes and ABI stability for the platform might exist, it has not been formalised through the means of a PEP yet; and because PyPI has not yet received support through a PEP for formalising the wasm32 tag through `packaging` and `pip`, which means that `micropip` remains the only choice for installing these wheels (which, then, does not come with `resolvelib`, AFAIK) and wheels need to be pushed to alternative indices – Anaconda
org, in this case. However, out-of-tree build support in the ecosystem is improving day by day; it could be a viable candidate sometime soon. In the meantime, this current method of building should be a reasonable alternative since it's tightly integrated into and similar to `pyodide`'s in-tree builds with the `pyodide build` CLI.
agriyakhetarpal: I ended up asking around on that PR and it has been merged now after more than a year of effort; `cibuildwheel` hasn't had a release with it yet, but following these recent developments I have tried it for NumPy and it works nicely: https://github.com/numpy/numpy/pull/26570 – let me replace the configuration here as well and rebase this branch.

I would still recommend using this existing configuration because it's much more stable and it will allow us to get these wheels uploaded to Anaconda.org much quicker, but in any case, that's not a high-priority pressing issue right now, so I would not mind if there is a delay with that and I understand that the review process can take time, running on its own accord.
agriyakhetarpal: Both the wheel build and the tests pass 🎉: https://github.com/pandas-dev/pandas/actions/runs/9307939926/job/25620279093

I'll convert this to a draft for now – I will mark it as ready for review as soon as a stable `cibuildwheel` release hits.
agriyakhetarpal: Based on https://github.com/pandas-dev/pandas/actions/runs/9466987065/job/26080112499, I think there are some tests that are not running in the regular Emscripten job, and I am not sure why yet. I'll fix them as necessary in subsequent commits.
agriyakhetarpal: Ready for review, @mroeschke and @lithomas1.
WillAyd: I am -1 on adding this to CI or claiming any official support; this just makes a more Arrow-centric future even more difficult
agriyakhetarpal: @WillAyd, thanks for your comment. Could a reasonable alternative be to upload these nightly wheels silently without an announcement or a CHANGELOG entry, at least for now? It has been a while since the Pyodide job was added in #57896 and I figure that it has been running smoothly without any outstanding errors – I don't see anything new having turned up in issues or PRs with either "Emscripten" or "Pyodide" being referenced :)

Moreover, https://github.com/apache/arrow/pull/37822 just got merged a few days ago, and though https://github.com/pyodide/pyodide/issues/2933 isn't going to get into Pyodide 0.26.2, it will be going into Pyodide 0.27.X later in the year, marking official Arrow support in/for Pyodide. Though I understand and I am aware of the PDEP and the ramifications by now on bandwidth use and the drastic increase in download sizes, an Arrow-centric future might be reasonably possible and is definitely getting closer. I hope that you can reconsider your stance!
WillAyd: @agriyakhetarpal that's great news on the pyarrow support. With that being the case I have no concerns

Patch:
diff --git a/.github/workflows/wheels.yml b/.github/workflows/wheels.yml
index f61ef550f7..02100648b6 100644
--- a/.github/workflows/wheels.yml
+++ b/.github/workflows/wheels.yml
@@ -100,6 +100,13 @@ jobs:
         - [windows-2022, win_amd64]
         # TODO: support PyPy?
         python: [["cp310", "3.10"], ["cp311", "3.11"], ["cp312", "3.12"]]
+
+        # Build Pyodide wheels and upload them to Anaconda.org
+        # NOTE: this job is similar to the one in unit-tests.yml except for the fact
+        # that it uses cibuildwheel instead of a standard Pyodide xbuildenv setup.
+        include:
+          - buildplat: [ubuntu-22.04, pyodide_wasm32]
+            python: ["cp312", "3.12"]
     env:
       IS_PUSH: ${{ github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v') }}
       IS_SCHEDULE_DISPATCH: ${{ github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' }}
@@ -146,6 +153,7 @@ jobs:
         env:
           CIBW_PRERELEASE_PYTHONS: True
           CIBW_BUILD: ${{ matrix.python[0] }}-${{ matrix.buildplat[1] }}
+          CIBW_PLATFORM: ${{ matrix.buildplat[1] == 'pyodide_wasm32' && 'pyodide' || 'auto' }}
 
       - name: Set up Python
         uses: mamba-org/setup-micromamba@v1
diff --git a/ci/code_checks.sh b/ci/code_checks.sh
index b01866a6d6..c4d91da70a 100755
--- a/ci/code_checks.sh
+++ b/ci/code_checks.sh
@@ -70,10 +70,6 @@ if [[ -z "$CHECK" || "$CHECK" == "docstrings" ]]; then
         --format=actions \
         -i ES01 `# For now it is ok if docstrings are missing the extended summary` \
         -i "pandas.Series.dt PR01" `# Accessors are implemented as classes, but we do not document the Parameters section` \
-        -i "pandas.DataFrame.max RT03" \
-        -i "pandas.DataFrame.mean RT03" \
-        -i "pandas.DataFrame.median RT03" \
-        -i "pandas.DataFrame.min RT03" \
         -i "pandas.DataFrame.plot PR02" \
         -i "pandas.Grouper PR02" \
         -i "pandas.MultiIndex.append PR07,SA01" \
@@ -166,7 +162,7 @@ if [[ -z "$CHECK" || "$CHECK" == "docstrings" ]]; then
         -i "pandas.Series.product RT03" \
         -i "pandas.Series.reorder_levels RT03,SA01" \
         -i "pandas.Series.sem PR01,RT03,SA01" \
-        -i "pandas.Series.skew RT03,SA01" \
+        -i "pandas.Series.skew SA01" \
         -i "pandas.Series.sparse PR01,SA01" \
         -i "pandas.Series.sparse.density SA01" \
         -i "pandas.Series.sparse.fill_value SA01" \
diff --git a/doc/source/getting_started/intro_tutorials/04_plotting.rst b/doc/source/getting_started/intro_tutorials/04_plotting.rst
index 49cf7d32e0..e9f83c602d 100644
--- a/doc/source/getting_started/intro_tutorials/04_plotting.rst
+++ b/doc/source/getting_started/intro_tutorials/04_plotting.rst
@@ -32,8 +32,10 @@ How do I create plots in pandas?
     air_quality.head()
 
 .. note::
-    The usage of the ``index_col`` and ``parse_dates`` parameters of the ``read_csv`` function to define the first (0th) column as
-    index of the resulting ``DataFrame`` and convert the dates in the column to :class:`Timestamp` objects, respectively.
+    The ``index_col=0`` and ``parse_dates=True`` parameters passed to the ``read_csv`` function define
+    the first (0th) column as index of the resulting ``DataFrame`` and convert the dates in the column
+    to :class:`Timestamp`  objects, respectively.
+
 
 .. raw:: html
 
diff --git a/doc/source/whatsnew/v3.0.0.rst b/doc/source/whatsnew/v3.0.0.rst
index cd91792488..639655ab28 100644
--- a/doc/source/whatsnew/v3.0.0.rst
+++ b/doc/source/whatsnew/v3.0.0.rst
@@ -556,6 +556,7 @@ MultiIndex
 - :func:`DataFrame.loc` with ``axis=0``  and :class:`MultiIndex` when setting a value adds extra columns (:issue:`58116`)
 - :meth:`DataFrame.melt` would not accept multiple names in ``var_name`` when the columns were a :class:`MultiIndex` (:issue:`58033`)
 - :meth:`MultiIndex.insert` would not insert NA value correctly at unified location of index -1 (:issue:`59003`)
+- :func:`MultiIndex.get_level_values` accessing a :class:`DatetimeIndex` does not carry the frequency attribute along (:issue:`58327`, :issue:`57949`)
 -
 
 I/O
diff --git a/pandas/core/arrays/datetimelike.py b/pandas/core/arrays/datetimelike.py
index c90ff410b4..ad0bde3abb 100644
--- a/pandas/core/arrays/datetimelike.py
+++ b/pandas/core/arrays/datetimelike.py
@@ -65,6 +65,7 @@ from pandas._typing import (
     ScalarIndexer,
     Self,
     SequenceIndexer,
+    TakeIndexer,
     TimeAmbiguous,
     TimeNonexistent,
     npt,
@@ -2340,6 +2341,27 @@ class TimelikeOps(DatetimeLikeArrayMixin):
             return self
         return type(self)._simple_new(out_data, dtype=self.dtype)
 
+    def take(
+        self,
+        indices: TakeIndexer,
+        *,
+        allow_fill: bool = False,
+        fill_value: Any = None,
+        axis: AxisInt = 0,
+    ) -> Self:
+        result = super().take(
+            indices=indices, allow_fill=allow_fill, fill_value=fill_value, axis=axis
+        )
+
+        indices = np.asarray(indices, dtype=np.intp)
+        maybe_slice = lib.maybe_indices_to_slice(indices, len(self))
+
+        if isinstance(maybe_slice, slice):
+            freq = self._get_getitem_freq(maybe_slice)
+            result._freq = freq
+
+        return result
+
     # --------------------------------------------------------------
     # Unsorted
 
diff --git a/pandas/core/frame.py b/pandas/core/frame.py
index 5ef663564a..ee48f54681 100644
--- a/pandas/core/frame.py
+++ b/pandas/core/frame.py
@@ -4473,7 +4473,7 @@ class DataFrame(NDFrame, OpsMixin):
             or punctuations (besides underscores) or starting with digits must be
             surrounded by backticks. (For example, a column named "Area (cm^2)" would
             be referenced as ```Area (cm^2)```). Column names which are Python keywords
-            (like "list", "for", "import", etc) cannot be used.
+            (like "if", "for", "import", etc) cannot be used.
 
             For example, if one of your columns is called ``a a`` and you want
             to sum it with ``b``, your query should be ```a a` + b``.
@@ -4554,8 +4554,8 @@ class DataFrame(NDFrame, OpsMixin):
         For example, ```it's` > `that's``` will raise an error,
         as it forms a quoted string (``'s > `that'``) with a backtick inside.
 
-        See also the Python documentation about lexical analysis
-        (https://docs.python.org/3/reference/lexical_analysis.html)
+        See also the `Python documentation about lexical analysis
+        <https://docs.python.org/3/reference/lexical_analysis.html>`__
         in combination with the source code in :mod:`pandas.core.computation.parsing`.
 
         Examples
diff --git a/pandas/core/generic.py b/pandas/core/generic.py
index 2a0495dff6..fc9821a657 100644
--- a/pandas/core/generic.py
+++ b/pandas/core/generic.py
@@ -11777,6 +11777,8 @@ numeric_only : bool, default False
 Returns
 -------
 {name1} or scalar\
+
+    Value containing the calculation referenced in the description.\
 {see_also}\
 {examples}
 """
diff --git a/pyproject.toml b/pyproject.toml
index 9156c73efb..47fd540d67 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -177,6 +177,16 @@ test-command = ""
 select = "*-macosx*"
 environment = {CFLAGS="-g0"}
 
+[[tool.cibuildwheel.overrides]]
+select = "*pyodide*"
+test-requires = "pytest>=7.3.2 hypothesis>=6.46.1"
+# Pyodide repairs wheels on its own, using auditwheel-emscripten
+repair-wheel-command = ""
+test-command = """
+  PANDAS_CI='1' python -c 'import pandas as pd; \
+  pd.test(extra_args=["-m not clipboard and not single_cpu and not slow and not network and not db", "--no-strict-data-files"]);' \
+  """
+
 [tool.ruff]
 line-length = 88
 target-version = "py310"


Test patch:
diff --git a/.github/workflows/unit-tests.yml b/.github/workflows/unit-tests.yml
index 982877ee7f..ddb6ecbe83 100644
--- a/.github/workflows/unit-tests.yml
+++ b/.github/workflows/unit-tests.yml
@@ -391,6 +391,7 @@ jobs:
         env:
           PYTHON_GIL: 0
 
+  # NOTE: this job must be kept in sync with the Pyodide build job in wheels.yml
   emscripten:
     # Note: the Python version, Emscripten toolchain version are determined
     # by the Pyodide version. The appropriate versions can be found in the
diff --git a/pandas/tests/indexes/multi/test_get_level_values.py b/pandas/tests/indexes/multi/test_get_level_values.py
index 28c77e7892..4db74a716c 100644
--- a/pandas/tests/indexes/multi/test_get_level_values.py
+++ b/pandas/tests/indexes/multi/test_get_level_values.py
@@ -122,3 +122,12 @@ def test_values_loses_freq_of_underlying_index():
     midx.values
     assert idx.freq is not None
     tm.assert_index_equal(idx, expected)
+
+
+def test_get_level_values_gets_frequency_correctly():
+    # GH#57949 GH#58327
+    datetime_index = date_range(start=pd.to_datetime("1/1/2018"), periods=4, freq="YS")
+    other_index = ["A"]
+    multi_index = MultiIndex.from_product([datetime_index, other_index])
+
+    assert multi_index.get_level_values(0).freq == datetime_index.freq
diff --git a/pandas/tests/test_common.py b/pandas/tests/test_common.py
index 7b93416600..ca97af0d3e 100644
--- a/pandas/tests/test_common.py
+++ b/pandas/tests/test_common.py
@@ -7,6 +7,8 @@ import sys
 import numpy as np
 import pytest
 
+from pandas.compat import WASM
+
 import pandas as pd
 from pandas import Series
 import pandas._testing as tm
@@ -233,6 +235,7 @@ def test_temp_setattr(with_exception):
     assert ser.name == "first"
 
 
+@pytest.mark.skipif(WASM, reason="Can't start subprocesses in WASM")
 @pytest.mark.single_cpu
 def test_str_size():
     # GH#21758