Repository: pandas-dev/pandas (python)

Issue:
DOC: Adding new columns in a DataFrame of Nullable type might be better explained
### Pandas version checks

- [X] I have checked that the issue still exists on the latest versions of the docs on `main` [here](https://pandas.pydata.org/docs/dev/)


### Location of the documentation

https://pandas.pydata.org/pandas-docs/stable/user_guide/integer_na.html

### Documentation problem

When adding a new column to an exisitng dataframe using pd.NA is tricky.
```python
>>> df = pd.DataFrame({'A': [1, 2]})
>>> df.dtypes
A    int64
dtype: object
>>> df
   A
0  1
1  2
```
Adding a new column of type `datetime64[ns]` works as expected using `pd.NaT`:
```python
>>> df['T'] = pd.NaT
>>> df.dtypes
A             int64
T    datetime64[ns]
dtype: object
>>> df
   A   T
0  1 NaT
1  2 NaT
```
However, adding a column using `pd.NA` produces a column of type `object`:
```python
>>> df['N'] = pd.NA
>>> df.dtypes
A             int64
T    datetime64[ns]
N            object
dtype: object
>>> df
   A   T     N
0  1 NaT  <NA>
1  2 NaT  <NA>
```
That is _exactly what it is asked for_ but not what I was naively expecting (a column of type `Int64`)
This is logical as the `pd.NA` type is used also for `string` and `boolean` types and maybe others.

The best way I found to create a new, empty column of a desired nullable type is the following:
```python
>>> df['N'] = pd.Series(dtype='Int64')
>>> df['S'] = pd.Series(dtype='string')
>>> df['B'] = pd.Series(dtype='boolean')
>>> df.dtypes
A             int64
T    datetime64[ns]
N             Int64
S            string
B           boolean
dtype: object
>>> df
   A   T     N     S     B
0  1 NaT  <NA>  <NA>  <NA>
1  2 NaT  <NA>  <NA>  <NA>
```


### Suggested fix for documentation

I think it will be enough to mention this point as an additional entry in the page from nullable datatypes, to show that creating a column initialised with `pd.NA` is not different to creating a column with any other, non numeric python object.


Pull request:
DOC: Add notes to nullable types documentation about pd.NA column type
- [X] closes #49201 
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Patch:
diff --git a/ci/code_checks.sh b/ci/code_checks.sh
index 039700f306..ade7173cf0 100755
--- a/ci/code_checks.sh
+++ b/ci/code_checks.sh
@@ -467,14 +467,11 @@ if [[ -z "$CHECK" || "$CHECK" == "docstrings" ]]; then
         -i "pandas.json_normalize RT03,SA01" \
         -i "pandas.merge PR07" \
         -i "pandas.merge_asof PR07,RT03" \
-        -i "pandas.merge_ordered PR07" \
         -i "pandas.period_range RT03,SA01" \
-        -i "pandas.pivot PR07" \
         -i "pandas.plotting.andrews_curves RT03,SA01" \
         -i "pandas.plotting.lag_plot RT03,SA01" \
         -i "pandas.plotting.scatter_matrix PR07,SA01" \
         -i "pandas.qcut PR07,SA01" \
-        -i "pandas.read_spss SA01" \
         -i "pandas.set_eng_float_format RT03,SA01" \
         -i "pandas.testing.assert_extension_array_equal SA01" \
         -i "pandas.tseries.offsets.BDay PR02,SA01" \
diff --git a/doc/source/getting_started/intro_tutorials/09_timeseries.rst b/doc/source/getting_started/intro_tutorials/09_timeseries.rst
index 14db38c382..6ba3c17fac 100644
--- a/doc/source/getting_started/intro_tutorials/09_timeseries.rst
+++ b/doc/source/getting_started/intro_tutorials/09_timeseries.rst
@@ -295,7 +295,7 @@ Aggregate the current hourly time series values to the monthly maximum value in
 
 .. ipython:: python
 
-    monthly_max = no_2.resample("ME").max()
+    monthly_max = no_2.resample("MS").max()
     monthly_max
 
 A very powerful method on time series data with a datetime index, is the
diff --git a/doc/source/user_guide/boolean.rst b/doc/source/user_guide/boolean.rst
index 3c361d4de1..7de0430123 100644
--- a/doc/source/user_guide/boolean.rst
+++ b/doc/source/user_guide/boolean.rst
@@ -37,6 +37,19 @@ If you would prefer to keep the ``NA`` values you can manually fill them with ``
 
    s[mask.fillna(True)]
 
+If you create a column of ``NA`` values (for example to fill them later)
+with ``df['new_col'] = pd.NA``, the ``dtype`` would be set to ``object`` in the
+new column. The performance on this column will be worse than with
+the appropriate type. It's better to use
+``df['new_col'] = pd.Series(pd.NA, dtype="boolean")``
+(or another ``dtype`` that supports ``NA``).
+
+.. ipython:: python
+
+   df = pd.DataFrame()
+   df['objects'] = pd.NA
+   df.dtypes
+
 .. _boolean.kleene:
 
 Kleene logical operations
diff --git a/doc/source/user_guide/integer_na.rst b/doc/source/user_guide/integer_na.rst
index 1a727cd78a..76a2f22b79 100644
--- a/doc/source/user_guide/integer_na.rst
+++ b/doc/source/user_guide/integer_na.rst
@@ -84,6 +84,19 @@ with the dtype.
    In the future, we may provide an option for :class:`Series` to infer a
    nullable-integer dtype.
 
+If you create a column of ``NA`` values (for example to fill them later)
+with ``df['new_col'] = pd.NA``, the ``dtype`` would be set to ``object`` in the
+new column. The performance on this column will be worse than with
+the appropriate type. It's better to use
+``df['new_col'] = pd.Series(pd.NA, dtype="Int64")``
+(or another ``dtype`` that supports ``NA``).
+
+.. ipython:: python
+
+   df = pd.DataFrame()
+   df['objects'] = pd.NA
+   df.dtypes
+
 Operations
 ----------
 
diff --git a/doc/source/user_guide/timeseries.rst b/doc/source/user_guide/timeseries.rst
index ab3f5b314e..d5137baa95 100644
--- a/doc/source/user_guide/timeseries.rst
+++ b/doc/source/user_guide/timeseries.rst
@@ -1864,7 +1864,7 @@ to resample based on datetimelike column in the frame, it can passed to the
        ),
    )
    df
-   df.resample("ME", on="date")[["a"]].sum()
+   df.resample("MS", on="date")[["a"]].sum()
 
 Similarly, if you instead want to resample by a datetimelike
 level of ``MultiIndex``, its name or location can be passed to the
@@ -1872,7 +1872,7 @@ level of ``MultiIndex``, its name or location can be passed to the
 
 .. ipython:: python
 
-   df.resample("ME", level="d")[["a"]].sum()
+   df.resample("MS", level="d")[["a"]].sum()
 
 .. _timeseries.iterating-label:
 
diff --git a/doc/source/whatsnew/v3.0.0.rst b/doc/source/whatsnew/v3.0.0.rst
index 7f5c879c0d..802a9fd7e2 100644
--- a/doc/source/whatsnew/v3.0.0.rst
+++ b/doc/source/whatsnew/v3.0.0.rst
@@ -456,6 +456,7 @@ Datetimelike
 - Bug in :func:`date_range` where the last valid timestamp would sometimes not be produced (:issue:`56134`)
 - Bug in :func:`date_range` where using a negative frequency value would not include all points between the start and end values (:issue:`56382`)
 - Bug in :func:`tseries.api.guess_datetime_format` would fail to infer time format when "%Y" == "%H%M" (:issue:`57452`)
+- Bug in :meth:`Dataframe.agg` with df with missing values resulting in IndexError (:issue:`58810`)
 - Bug in :meth:`DatetimeIndex.is_year_start` and :meth:`DatetimeIndex.is_quarter_start` does not raise on Custom business days frequencies bigger then "1C" (:issue:`58664`)
 - Bug in :meth:`DatetimeIndex.is_year_start` and :meth:`DatetimeIndex.is_quarter_start` returning ``False`` on double-digit frequencies (:issue:`58523`)
 - Bug in setting scalar values with mismatched resolution into arrays with non-nanosecond ``datetime64``, ``timedelta64`` or :class:`DatetimeTZDtype` incorrectly truncating those scalars (:issue:`56410`)
diff --git a/pandas/core/apply.py b/pandas/core/apply.py
index 33f5062358..2039386c47 100644
--- a/pandas/core/apply.py
+++ b/pandas/core/apply.py
@@ -1850,11 +1850,13 @@ def relabel_result(
                 com.get_callable_name(f) if not isinstance(f, str) else f for f in fun
             ]
             col_idx_order = Index(s.index).get_indexer(fun)
-            s = s.iloc[col_idx_order]
-
+            valid_idx = col_idx_order != -1
+            if valid_idx.any():
+                s = s.iloc[col_idx_order[valid_idx]]
         # assign the new user-provided "named aggregation" as index names, and reindex
         # it based on the whole user-provided names.
-        s.index = reordered_indexes[idx : idx + len(fun)]
+        if not s.empty:
+            s.index = reordered_indexes[idx : idx + len(fun)]
         reordered_result_in_dict[col] = s.reindex(columns)
         idx = idx + len(fun)
     return reordered_result_in_dict
diff --git a/pandas/core/internals/managers.py b/pandas/core/internals/managers.py
index 82b88d090f..64109f5c16 100644
--- a/pandas/core/internals/managers.py
+++ b/pandas/core/internals/managers.py
@@ -821,11 +821,13 @@ class BaseBlockManager(PandasObject):
             raise IndexError("Requested axis not found in manager")
 
         if axis == 0:
-            new_blocks = self._slice_take_blocks_ax0(
-                indexer,
-                fill_value=fill_value,
-                only_slice=only_slice,
-                use_na_proxy=use_na_proxy,
+            new_blocks = list(
+                self._slice_take_blocks_ax0(
+                    indexer,
+                    fill_value=fill_value,
+                    only_slice=only_slice,
+                    use_na_proxy=use_na_proxy,
+                )
             )
         else:
             new_blocks = [
@@ -857,7 +859,7 @@ class BaseBlockManager(PandasObject):
         *,
         use_na_proxy: bool = False,
         ref_inplace_op: bool = False,
-    ) -> list[Block]:
+    ) -> Generator[Block, None, None]:
         """
         Slice/take blocks along axis=0.
 
@@ -875,9 +877,9 @@ class BaseBlockManager(PandasObject):
         ref_inplace_op: bool, default False
             Don't track refs if True because we operate inplace
 
-        Returns
-        -------
-        new_blocks : list of Block
+        Yields
+        ------
+        Block : New Block
         """
         allow_fill = fill_value is not lib.no_default
 
@@ -892,9 +894,10 @@ class BaseBlockManager(PandasObject):
                 # GH#32959 EABlock would fail since we can't make 0-width
                 # TODO(EA2D): special casing unnecessary with 2D EAs
                 if sllen == 0:
-                    return []
+                    return
                 bp = BlockPlacement(slice(0, sllen))
-                return [blk.getitem_block_columns(slobj, new_mgr_locs=bp)]
+                yield blk.getitem_block_columns(slobj, new_mgr_locs=bp)
+                return
             elif not allow_fill or self.ndim == 1:
                 if allow_fill and fill_value is None:
                     fill_value = blk.fill_value
@@ -902,25 +905,21 @@ class BaseBlockManager(PandasObject):
                 if not allow_fill and only_slice:
                     # GH#33597 slice instead of take, so we get
                     #  views instead of copies
-                    blocks = [
-                        blk.getitem_block_columns(
+                    for i, ml in enumerate(slobj):
+                        yield blk.getitem_block_columns(
                             slice(ml, ml + 1),
                             new_mgr_locs=BlockPlacement(i),
                             ref_inplace_op=ref_inplace_op,
                         )
-                        for i, ml in enumerate(slobj)
-                    ]
-                    return blocks
                 else:
                     bp = BlockPlacement(slice(0, sllen))
-                    return [
-                        blk.take_nd(
-                            slobj,
-                            axis=0,
-                            new_mgr_locs=bp,
-                            fill_value=fill_value,
-                        )
-                    ]
+                    yield blk.take_nd(
+                        slobj,
+                        axis=0,
+                        new_mgr_locs=bp,
+                        fill_value=fill_value,
+                    )
+                return
 
         if sl_type == "slice":
             blknos = self.blknos[slobj]
@@ -935,18 +934,15 @@ class BaseBlockManager(PandasObject):
 
         # When filling blknos, make sure blknos is updated before appending to
         # blocks list, that way new blkno is exactly len(blocks).
-        blocks = []
         group = not only_slice
         for blkno, mgr_locs in libinternals.get_blkno_placements(blknos, group=group):
             if blkno == -1:
                 # If we've got here, fill_value was not lib.no_default
 
-                blocks.append(
-                    self._make_na_block(
-                        placement=mgr_locs,
-                        fill_value=fill_value,
-                        use_na_proxy=use_na_proxy,
-                    )
+                yield self._make_na_block(
+                    placement=mgr_locs,
+                    fill_value=fill_value,
+                    use_na_proxy=use_na_proxy,
                 )
             else:
                 blk = self.blocks[blkno]
@@ -961,7 +957,7 @@ class BaseBlockManager(PandasObject):
                     for mgr_loc in mgr_locs:
                         newblk = blk.copy(deep=deep)
                         newblk.mgr_locs = BlockPlacement(slice(mgr_loc, mgr_loc + 1))
-                        blocks.append(newblk)
+                        yield newblk
 
                 else:
                     # GH#32779 to avoid the performance penalty of copying,
@@ -972,7 +968,7 @@ class BaseBlockManager(PandasObject):
 
                     if isinstance(taker, slice):
                         nb = blk.getitem_block_columns(taker, new_mgr_locs=mgr_locs)
-                        blocks.append(nb)
+                        yield nb
                     elif only_slice:
                         # GH#33597 slice instead of take, so we get
                         #  views instead of copies
@@ -981,12 +977,10 @@ class BaseBlockManager(PandasObject):
                             bp = BlockPlacement(ml)
                             nb = blk.getitem_block_columns(slc, new_mgr_locs=bp)
                             # We have np.shares_memory(nb.values, blk.values)
-                            blocks.append(nb)
+                            yield nb
                     else:
                         nb = blk.take_nd(taker, axis=0, new_mgr_locs=mgr_locs)
-                        blocks.append(nb)
-
-        return blocks
+                        yield nb
 
     def _make_na_block(
         self, placement: BlockPlacement, fill_value=None, use_na_proxy: bool = False
diff --git a/pandas/core/reshape/merge.py b/pandas/core/reshape/merge.py
index e6e84c2135..ddf6bd3c70 100644
--- a/pandas/core/reshape/merge.py
+++ b/pandas/core/reshape/merge.py
@@ -316,7 +316,9 @@ def merge_ordered(
     Parameters
     ----------
     left : DataFrame or named Series
+        First pandas object to merge.
     right : DataFrame or named Series
+        Second pandas object to merge.
     on : label or list
         Field names to join on. Must be found in both DataFrames.
     left_on : label or list, or array-like
diff --git a/pandas/core/reshape/pivot.py b/pandas/core/reshape/pivot.py
index 86da19f13b..ff993c039b 100644
--- a/pandas/core/reshape/pivot.py
+++ b/pandas/core/reshape/pivot.py
@@ -11,10 +11,6 @@ from typing import (
 import numpy as np
 
 from pandas._libs import lib
-from pandas.util._decorators import (
-    Appender,
-    Substitution,
-)
 
 from pandas.core.dtypes.cast import maybe_downcast_to_dtype
 from pandas.core.dtypes.common import (
@@ -29,7 +25,6 @@ from pandas.core.dtypes.generic import (
 )
 
 import pandas.core.common as com
-from pandas.core.frame import _shared_docs
 from pandas.core.groupby import Grouper
 from pandas.core.indexes.api import (
     Index,
@@ -656,8 +651,6 @@ def _convert_by(by):
     return by
 
 
-@Substitution("\ndata : DataFrame")
-@Appender(_shared_docs["pivot"], indents=1)
 def pivot(
     data: DataFrame,
     *,
@@ -665,6 +658,152 @@ def pivot(
     index: IndexLabel | lib.NoDefault = lib.no_default,
     values: IndexLabel | lib.NoDefault = lib.no_default,
 ) -> DataFrame:
+    """
+    Return reshaped DataFrame organized by given index / column values.
+
+    Reshape data (produce a "pivot" table) based on column values. Uses
+    unique values from specified `index` / `columns` to form axes of the
+    resulting DataFrame. This function does not support data
+    aggregation, multiple values will result in a MultiIndex in the
+    columns. See the :ref:`User Guide <reshaping>` for more on reshaping.
+
+    Parameters
+    ----------
+    data : DataFrame
+        Input pandas DataFrame object.
+    columns : str or object or a list of str
+        Column to use to make new frame's columns.
+    index : str or object or a list of str, optional
+        Column to use to make new frame's index. If not given, uses existing index.
+    values : str, object or a list of the previous, optional
+        Column(s) to use for populating new frame's values. If not
+        specified, all remaining columns will be used and the result will
+        have hierarchically indexed columns.
+
+    Returns
+    -------
+    DataFrame
+        Returns reshaped DataFrame.
+
+    Raises
+    ------
+    ValueError:
+        When there are any `index`, `columns` combinations with multiple
+        values. `DataFrame.pivot_table` when you need to aggregate.
+
+    See Also
+    --------
+    DataFrame.pivot_table : Generalization of pivot that can handle
+        duplicate values for one index/column pair.
+    DataFrame.unstack : Pivot based on the index values instead of a
+        column.
+    wide_to_long : Wide panel to long format. Less flexible but more
+        user-friendly than melt.
+
+    Notes
+    -----
+    For finer-tuned control, see hierarchical indexing documentation along
+    with the related stack/unstack methods.
+
+    Reference :ref:`the user guide <reshaping.pivot>` for more examples.
+
+    Examples
+    --------
+    >>> df = pd.DataFrame(
+    ...     {
+    ...         "foo": ["one", "one", "one", "two", "two", "two"],
+    ...         "bar": ["A", "B", "C", "A", "B", "C"],
+    ...         "baz": [1, 2, 3, 4, 5, 6],
+    ...         "zoo": ["x", "y", "z", "q", "w", "t"],
+    ...     }
+    ... )
+    >>> df
+        foo   bar  baz  zoo
+    0   one   A    1    x
+    1   one   B    2    y
+    2   one   C    3    z
+    3   two   A    4    q
+    4   two   B    5    w
+    5   two   C    6    t
+
+    >>> df.pivot(index="foo", columns="bar", values="baz")
+    bar  A   B   C
+    foo
+    one  1   2   3
+    two  4   5   6
+
+    >>> df.pivot(index="foo", columns="bar")["baz"]
+    bar  A   B   C
+    foo
+    one  1   2   3
+    two  4   5   6
+
+    >>> df.pivot(index="foo", columns="bar", values=["baz", "zoo"])
+          baz       zoo
+    bar   A  B  C   A  B  C
+    foo
+    one   1  2  3   x  y  z
+    two   4  5  6   q  w  t
+
+    You could also assign a list of column names or a list of index names.
+
+    >>> df = pd.DataFrame(
+    ...     {
+    ...         "lev1": [1, 1, 1, 2, 2, 2],
+    ...         "lev2": [1, 1, 2, 1, 1, 2],
+    ...         "lev3": [1, 2, 1, 2, 1, 2],
+    ...         "lev4": [1, 2, 3, 4, 5, 6],
+    ...         "values": [0, 1, 2, 3, 4, 5],
+    ...     }
+    ... )
+    >>> df
+        lev1 lev2 lev3 lev4 values
+    0   1    1    1    1    0
+    1   1    1    2    2    1
+    2   1    2    1    3    2
+    3   2    1    2    4    3
+    4   2    1    1    5    4
+    5   2    2    2    6    5
+
+    >>> df.pivot(index="lev1", columns=["lev2", "lev3"], values="values")
+    lev2    1         2
+    lev3    1    2    1    2
+    lev1
+    1     0.0  1.0  2.0  NaN
+    2     4.0  3.0  NaN  5.0
+
+    >>> df.pivot(index=["lev1", "lev2"], columns=["lev3"], values="values")
+          lev3    1    2
+    lev1  lev2
+       1     1  0.0  1.0
+             2  2.0  NaN
+       2     1  4.0  3.0
+             2  NaN  5.0
+
+    A ValueError is raised if there are any duplicates.
+
+    >>> df = pd.DataFrame(
+    ...     {
+    ...         "foo": ["one", "one", "two", "two"],
+    ...         "bar": ["A", "A", "B", "C"],
+    ...         "baz": [1, 2, 3, 4],
+    ...     }
+    ... )
+    >>> df
+       foo bar  baz
+    0  one   A    1
+    1  one   A    2
+    2  two   B    3
+    3  two   C    4
+
+    Notice that the first two rows are the same for our `index`
+    and `columns` arguments.
+
+    >>> df.pivot(index="foo", columns="bar", values="baz")
+    Traceback (most recent call last):
+       ...
+    ValueError: Index contains duplicate entries, cannot reshape
+    """
     columns_listlike = com.convert_to_list_like(columns)
 
     # If columns is None we will create a MultiIndex level with None as name
diff --git a/pandas/io/spss.py b/pandas/io/spss.py
index 2c464cc7e9..313ffa79cb 100644
--- a/pandas/io/spss.py
+++ b/pandas/io/spss.py
@@ -52,6 +52,14 @@ def read_spss(
     DataFrame
         DataFrame based on the SPSS file.
 
+    See Also
+    --------
+    read_csv : Read a comma-separated values (csv) file into a pandas DataFrame.
+    read_excel : Read an Excel file into a pandas DataFrame.
+    read_sas : Read an SAS file into a pandas DataFrame.
+    read_orc : Load an ORC object into a pandas DataFrame.
+    read_feather : Load a feather-format object into a pandas DataFrame.
+
     Examples
     --------
     >>> df = pd.read_spss("spss_data.sav")  # doctest: +SKIP


Test patch:
diff --git a/pandas/tests/groupby/aggregate/test_aggregate.py b/pandas/tests/groupby/aggregate/test_aggregate.py
index 1f140063fd..26602baedb 100644
--- a/pandas/tests/groupby/aggregate/test_aggregate.py
+++ b/pandas/tests/groupby/aggregate/test_aggregate.py
@@ -62,6 +62,32 @@ def test_agg_ser_multi_key(df):
     tm.assert_series_equal(results, expected)
 
 
+def test_agg_with_missing_values():
+    # GH#58810
+    missing_df = DataFrame(
+        {
+            "nan": [np.nan, np.nan, np.nan, np.nan],
+            "na": [pd.NA, pd.NA, pd.NA, pd.NA],
+            "nat": [pd.NaT, pd.NaT, pd.NaT, pd.NaT],
+            "none": [None, None, None, None],
+            "values": [1, 2, 3, 4],
+        }
+    )
+
+    result = missing_df.agg(x=("nan", "min"), y=("na", "min"), z=("values", "sum"))
+
+    expected = DataFrame(
+        {
+            "nan": [np.nan, np.nan, np.nan],
+            "na": [np.nan, np.nan, np.nan],
+            "values": [np.nan, np.nan, 10.0],
+        },
+        index=["x", "y", "z"],
+    )
+
+    tm.assert_frame_equal(result, expected)
+
+
 def test_groupby_aggregation_mixed_dtype():
     # GH 6212
     expected = DataFrame(