Repository: pandas-dev/pandas (python)

Issue:
PERF: df.unstack() is 500 times slower since pandas>=2.1 
### Pandas version checks

- [X] I have checked that this issue has not already been reported.

- [X] I have confirmed this issue exists on the [latest version](https://pandas.pydata.org/docs/whatsnew/index.html) of pandas.

- [ ] I have confirmed this issue exists on the main branch of pandas.


### Reproducible Example

import pandas as pd
import numpy as np
import time
df = pd.DataFrame(np.random.random(size=(10000, 100)))
st = time.time()
df.unstack()  # this operation takes 500x more in pandas>=2.1 
print(f"time {time.time() -st}")

### Installed Versions

<details>
INSTALLED VERSIONS
------------------
commit                : bdc79c146c2e32f2cab629be240f01658cfb6cc2
python                : 3.11.9.final.0
python-bits           : 64
OS                    : Windows
OS-release            : 10
Version               : 10.0.19045
machine               : AMD64
processor             : Intel64 Family 6 Model 158 Stepping 9, GenuineIntel
byteorder             : little
LC_ALL                : None
LANG                  : None
LOCALE                : English_United Kingdom.1252

pandas                : 2.2.1
numpy                 : 1.26.4
pytz                  : 2023.3.post1
dateutil              : 2.8.2
setuptools            : 68.2.2
pip                   : 23.3.1
Cython                : None
pytest                : None
hypothesis            : None
sphinx                : None
blosc                 : None
feather               : None
xlsxwriter            : None
lxml.etree            : None
html5lib              : None
pymysql               : None
psycopg2              : None
jinja2                : None
IPython               : None
pandas_datareader     : None
adbc-driver-postgresql: None
adbc-driver-sqlite    : None
bs4                   : None
bottleneck            : 1.3.7
dataframe-api-compat  : None
fastparquet           : None
fsspec                : None
gcsfs                 : None
matplotlib            : None
numba                 : None
numexpr               : 2.8.7
odfpy                 : None
openpyxl              : None
pandas_gbq            : None
pyarrow               : None
pyreadstat            : None
python-calamine       : None
pyxlsb                : None
s3fs                  : None
scipy                 : None
sqlalchemy            : None
tables                : None
tabulate              : None
xarray                : None
xlrd                  : None
zstandard             : None
tzdata                : 2023.3
qtpy                  : None
pyqt5                 : None
</details>


### Prior Performance

same code as above is 500x faster for pandas<=2.0.3. 
Issue happens on Windows and Linux, with Python 3.10 and 3.12, with backend numpy and pyarrow.
The slow down seems to be in the stack_v3 function in the initial loop.

Pull request:
PERF: future_stack=True with non-MulitIndex columns
- [x] closes #58391 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


```python
df = pd.DataFrame(np.random.random(size=(10000, 100)))
%timeit df.unstack()
17.2 ms ± 159 μs per loop (mean ± std. dev. of 7 runs, 100 loops each)  <--- PR
205 ms ± 3.37 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)     <--- main
```

Patch:
diff --git a/doc/source/whatsnew/v3.0.0.rst b/doc/source/whatsnew/v3.0.0.rst
index 184ca58190..46af55cedf 100644
--- a/doc/source/whatsnew/v3.0.0.rst
+++ b/doc/source/whatsnew/v3.0.0.rst
@@ -421,6 +421,7 @@ Other Deprecations
 - Deprecated lowercase strings ``w``, ``w-mon``, ``w-tue``, etc. denoting frequencies in :class:`Week` in favour of ``W``, ``W-MON``, ``W-TUE``, etc. (:issue:`58998`)
 - Deprecated parameter ``method`` in :meth:`DataFrame.reindex_like` / :meth:`Series.reindex_like` (:issue:`58667`)
 - Deprecated strings ``w``, ``d``, ``MIN``, ``MS``, ``US`` and ``NS`` denoting units in :class:`Timedelta` in favour of ``W``, ``D``, ``min``, ``ms``, ``us`` and ``ns`` (:issue:`59051`)
+- Deprecated the ``arg`` parameter of ``Series.map``; pass the added ``func`` argument instead. (:issue:`61260`)
 - Deprecated using ``epoch`` date format in :meth:`DataFrame.to_json` and :meth:`Series.to_json`, use ``iso`` instead. (:issue:`57063`)
 
 .. ---------------------------------------------------------------------------
@@ -622,6 +623,7 @@ Performance improvements
 - Performance improvement in :meth:`CategoricalDtype.update_dtype` when ``dtype`` is a :class:`CategoricalDtype` with non ``None`` categories and ordered (:issue:`59647`)
 - Performance improvement in :meth:`DataFrame.__getitem__` when ``key`` is a :class:`DataFrame` with many columns (:issue:`61010`)
 - Performance improvement in :meth:`DataFrame.astype` when converting to extension floating dtypes, e.g. "Float64" (:issue:`60066`)
+- Performance improvement in :meth:`DataFrame.stack` when using ``future_stack=True`` and the DataFrame does not have a :class:`MultiIndex` (:issue:`58391`)
 - Performance improvement in :meth:`DataFrame.where` when ``cond`` is a :class:`DataFrame` with many columns (:issue:`61010`)
 - Performance improvement in :meth:`to_hdf` avoid unnecessary reopenings of the HDF5 file to speedup data addition to files with a very large number of groups . (:issue:`58248`)
 - Performance improvement in ``DataFrameGroupBy.__len__`` and ``SeriesGroupBy.__len__`` (:issue:`57595`)
diff --git a/pandas/__init__.py b/pandas/__init__.py
index c570fb8d70..5dc6a8c3bc 100644
--- a/pandas/__init__.py
+++ b/pandas/__init__.py
@@ -4,19 +4,17 @@ __docformat__ = "restructuredtext"
 
 # Let users know if they're missing any of our hard dependencies
 _hard_dependencies = ("numpy", "dateutil")
-_missing_dependencies = []
 
 for _dependency in _hard_dependencies:
     try:
         __import__(_dependency)
     except ImportError as _e:  # pragma: no cover
-        _missing_dependencies.append(f"{_dependency}: {_e}")
+        raise ImportError(
+            f"Unable to import required dependency {_dependency}. "
+            "Please see the traceback for details."
+        ) from _e
 
-if _missing_dependencies:  # pragma: no cover
-    raise ImportError(
-        "Unable to import required dependencies:\n" + "\n".join(_missing_dependencies)
-    )
-del _hard_dependencies, _dependency, _missing_dependencies
+del _hard_dependencies, _dependency
 
 try:
     # numpy compat
diff --git a/pandas/core/reshape/reshape.py b/pandas/core/reshape/reshape.py
index c60fe71a7f..d2a838b616 100644
--- a/pandas/core/reshape/reshape.py
+++ b/pandas/core/reshape/reshape.py
@@ -936,7 +936,20 @@ def stack_v3(frame: DataFrame, level: list[int]) -> Series | DataFrame:
         [k for k in range(frame.columns.nlevels - 1, -1, -1) if k not in set_levels]
     )
 
-    result = stack_reshape(frame, level, set_levels, stack_cols)
+    result: Series | DataFrame
+    if not isinstance(frame.columns, MultiIndex):
+        # GH#58817 Fast path when we're stacking the columns of a non-MultiIndex.
+        # When columns are homogeneous EAs, we pass through object
+        # dtype but this is still slightly faster than the normal path.
+        if len(frame.columns) > 0 and frame._is_homogeneous_type:
+            dtype = frame._mgr.blocks[0].dtype
+        else:
+            dtype = None
+        result = frame._constructor_sliced(
+            frame._values.reshape(-1, order="F"), dtype=dtype
+        )
+    else:
+        result = stack_reshape(frame, level, set_levels, stack_cols)
 
     # Construct the correct MultiIndex by combining the frame's index and
     # stacked columns.
@@ -1018,6 +1031,8 @@ def stack_reshape(
     -------
     The data of behind the stacked DataFrame.
     """
+    # non-MultIndex takes a fast path.
+    assert isinstance(frame.columns, MultiIndex)
     # If we need to drop `level` from columns, it needs to be in descending order
     drop_levnums = sorted(level, reverse=True)
 
@@ -1027,18 +1042,14 @@ def stack_reshape(
         if len(frame.columns) == 1:
             data = frame.copy(deep=False)
         else:
-            if not isinstance(frame.columns, MultiIndex) and not isinstance(idx, tuple):
-                # GH#57750 - if the frame is an Index with tuples, .loc below will fail
-                column_indexer = idx
-            else:
-                # Take the data from frame corresponding to this idx value
-                if len(level) == 1:
-                    idx = (idx,)
-                gen = iter(idx)
-                column_indexer = tuple(
-                    next(gen) if k in set_levels else slice(None)
-                    for k in range(frame.columns.nlevels)
-                )
+            # Take the data from frame corresponding to this idx value
+            if len(level) == 1:
+                idx = (idx,)
+            gen = iter(idx)
+            column_indexer = tuple(
+                next(gen) if k in set_levels else slice(None)
+                for k in range(frame.columns.nlevels)
+            )
             data = frame.loc[:, column_indexer]
 
         if len(level) < frame.columns.nlevels:
diff --git a/pandas/core/series.py b/pandas/core/series.py
index 03a2ce85a0..d6a982c65e 100644
--- a/pandas/core/series.py
+++ b/pandas/core/series.py
@@ -52,6 +52,9 @@ from pandas.util._decorators import (
     doc,
     set_module,
 )
+from pandas.util._exceptions import (
+    find_stack_level,
+)
 from pandas.util._validators import (
     validate_ascending,
     validate_bool_kwarg,
@@ -4320,7 +4323,7 @@ class Series(base.IndexOpsMixin, NDFrame):  # type: ignore[misc]
 
     def map(
         self,
-        arg: Callable | Mapping | Series,
+        func: Callable | Mapping | Series | None = None,
         na_action: Literal["ignore"] | None = None,
         **kwargs,
     ) -> Series:
@@ -4333,8 +4336,8 @@ class Series(base.IndexOpsMixin, NDFrame):  # type: ignore[misc]
 
         Parameters
         ----------
-        arg : function, collections.abc.Mapping subclass or Series
-            Mapping correspondence.
+        func : function, collections.abc.Mapping subclass or Series
+            Function or mapping correspondence.
         na_action : {None, 'ignore'}, default None
             If 'ignore', propagate NaN values, without passing them to the
             mapping correspondence.
@@ -4404,9 +4407,22 @@ class Series(base.IndexOpsMixin, NDFrame):  # type: ignore[misc]
         3  I am a rabbit
         dtype: object
         """
-        if callable(arg):
-            arg = functools.partial(arg, **kwargs)
-        new_values = self._map_values(arg, na_action=na_action)
+        if func is None:
+            if "arg" in kwargs:
+                # `.map(arg=my_func)`
+                func = kwargs.pop("arg")
+                warnings.warn(
+                    "The parameter `arg` has been renamed to `func`, and it "
+                    "will stop being supported in a future version of pandas.",
+                    FutureWarning,
+                    stacklevel=find_stack_level(),
+                )
+            else:
+                raise ValueError("The `func` parameter is required")
+
+        if callable(func):
+            func = functools.partial(func, **kwargs)
+        new_values = self._map_values(func, na_action=na_action)
         return self._constructor(new_values, index=self.index, copy=False).__finalize__(
             self, method="map"
         )
diff --git a/web/pandas/community/benchmarks.md b/web/pandas/community/benchmarks.md
index 1e63832a5a..5a8198a979 100644
--- a/web/pandas/community/benchmarks.md
+++ b/web/pandas/community/benchmarks.md
@@ -36,9 +36,8 @@ available at the [pandas sponsors]({{ base_url }}about/sponsors.html) page.
 
 Results of the benchmarks are available at:
 
-- Original server: [asv](https://asv-runner.github.io/asv-collection/pandas/)
-- OVH server: [asv](https://pandas.pydata.org/benchmarks/asv/) (benchmarks results can
-  also be visualized in this [Conbench PoC](http://57.128.112.95:5000/)
+- GitHub Actions results: [asv](https://pandas-dev.github.io/asv-runner/)
+- OVH server: [asv](https://pandas.pydata.org/benchmarks/asv/)
 
 ### Original server configuration
 


Test patch:
diff --git a/pandas/tests/extension/base/reshaping.py b/pandas/tests/extension/base/reshaping.py
index 2915c0585f..a760cbc399 100644
--- a/pandas/tests/extension/base/reshaping.py
+++ b/pandas/tests/extension/base/reshaping.py
@@ -3,6 +3,8 @@ import itertools
 import numpy as np
 import pytest
 
+from pandas.core.dtypes.dtypes import NumpyEADtype
+
 import pandas as pd
 import pandas._testing as tm
 from pandas.api.extensions import ExtensionArray
@@ -266,7 +268,13 @@ class BaseReshapingTests:
         expected = expected.astype(object)
 
         if isinstance(expected, pd.Series):
-            assert result.dtype == df.iloc[:, 0].dtype
+            if future_stack and isinstance(data.dtype, NumpyEADtype):
+                # GH#58817 future_stack=True constructs the result specifying the dtype
+                # using the dtype of the input; we thus get the underlying
+                # NumPy dtype as the result instead of the NumpyExtensionArray
+                assert result.dtype == df.iloc[:, 0].to_numpy().dtype
+            else:
+                assert result.dtype == df.iloc[:, 0].dtype
         else:
             assert all(result.dtypes == df.iloc[:, 0].dtype)
 
diff --git a/pandas/tests/series/methods/test_map.py b/pandas/tests/series/methods/test_map.py
index 84b60a2afe..384b7ce3dc 100644
--- a/pandas/tests/series/methods/test_map.py
+++ b/pandas/tests/series/methods/test_map.py
@@ -604,3 +604,27 @@ def test_map_kwargs():
     result = Series([2, 4, 5]).map(lambda x, y: x + y, y=2)
     expected = Series([4, 6, 7])
     tm.assert_series_equal(result, expected)
+
+
+def test_map_arg_as_kwarg():
+    with tm.assert_produces_warning(
+        FutureWarning, match="`arg` has been renamed to `func`"
+    ):
+        Series([1, 2]).map(arg={})
+
+
+def test_map_func_and_arg():
+    # `arg`is considered a normal kwarg that should be passed to the function
+    result = Series([1, 2]).map(lambda _, arg: arg, arg=3)
+    expected = Series([3, 3])
+    tm.assert_series_equal(result, expected)
+
+
+def test_map_no_func_or_arg():
+    with pytest.raises(ValueError, match="The `func` parameter is required"):
+        Series([1, 2]).map()
+
+
+def test_map_func_is_none():
+    with pytest.raises(ValueError, match="The `func` parameter is required"):
+        Series([1, 2]).map(func=None)
diff --git a/pandas/tests/test_downstream.py b/pandas/tests/test_downstream.py
index 76fad35304..6282aecdfe 100644
--- a/pandas/tests/test_downstream.py
+++ b/pandas/tests/test_downstream.py
@@ -4,6 +4,7 @@ Testing that we work in the downstream packages
 
 import array
 from functools import partial
+import importlib
 import subprocess
 import sys
 
@@ -186,41 +187,21 @@ def test_yaml_dump(df):
     tm.assert_frame_equal(df, loaded2)
 
 
-@pytest.mark.single_cpu
-def test_missing_required_dependency():
-    # GH 23868
-    # To ensure proper isolation, we pass these flags
-    # -S : disable site-packages
-    # -s : disable user site-packages
-    # -E : disable PYTHON* env vars, especially PYTHONPATH
-    # https://github.com/MacPython/pandas-wheels/pull/50
-
-    pyexe = sys.executable.replace("\\", "/")
-
-    # We skip this test if pandas is installed as a site package. We first
-    # import the package normally and check the path to the module before
-    # executing the test which imports pandas with site packages disabled.
-    call = [pyexe, "-c", "import pandas;print(pandas.__file__)"]
-    output = subprocess.check_output(call).decode()
-    if "site-packages" in output:
-        pytest.skip("pandas installed as site package")
-
-    # This test will fail if pandas is installed as a site package. The flags
-    # prevent pandas being imported and the test will report Failed: DID NOT
-    # RAISE <class 'subprocess.CalledProcessError'>
-    call = [pyexe, "-sSE", "-c", "import pandas"]
-
-    msg = (
-        rf"Command '\['{pyexe}', '-sSE', '-c', 'import pandas'\]' "
-        "returned non-zero exit status 1."
-    )
+@pytest.mark.parametrize("dependency", ["numpy", "dateutil"])
+def test_missing_required_dependency(monkeypatch, dependency):
+    # GH#61030
+    original_import = __import__
+    mock_error = ImportError(f"Mock error for {dependency}")
+
+    def mock_import(name, *args, **kwargs):
+        if name == dependency:
+            raise mock_error
+        return original_import(name, *args, **kwargs)
 
-    with pytest.raises(subprocess.CalledProcessError, match=msg) as exc:
-        subprocess.check_output(call, stderr=subprocess.STDOUT)
+    monkeypatch.setattr("builtins.__import__", mock_import)
 
-    output = exc.value.stdout.decode()
-    for name in ["numpy", "dateutil"]:
-        assert name in output
+    with pytest.raises(ImportError, match=dependency):
+        importlib.reload(importlib.import_module("pandas"))
 
 
 def test_frame_setitem_dask_array_into_new_col(request):