DEPR: Deprecate returning a DataFrame in SeriesApply.apply_standard
State: closed

- [x] closes #52116
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.



Labels:
name: Deprecate
description: Functionality to remove in pandas
name: Apply
description: Apply, Aggregate, Transform, Map
name: Series
description: Series data structure

Comments:
> It's not clear to me what the output in this case will be after the deprecation. Can you give an example?

This pattern should not be used with `Series.apply`, but should the user find other ways to archive the same result. So the result after the change will generally look strange using `apply`.

**IF** they use the same method after the change, they will get a `Series` with Series as its elements:

```python
>>> import pandas as pd
>>> ser = pd.Series(range(3))
>>> func = lambda x: pd.Series({"x": x, "x**2": x**2})
>>> ser.apply(func)  # now
   x  x**2
0  1     1
1  2     4
2  3     9
>>> ser.apply(func)  # after change
0    x       0
x**2    0
dtype: int64
1    x       1
x**2    1
dtype: int64
2    x       2
x**2    4
dtype: int64
dtype: object
>>> ser.apply(func)[0]  # single element after change
x       0
x**2    0
dtype: int64
```

This pattern will be not useful generally after the change is implemented, so they should use better patterns, e.g. 

```python
>>> func = lambda x: pd.DataFrame({"x": x, "x**2": x**2})
>>> ser.pipe(func)  # now  & after this deprecation, it's faster
   x  x**2
0  0     0
1  1     1
2  2     4
>>> pd.DataFrame({"x": ser, "x**2": ser**2})  # or this...
   x  x**2
0  0     0
1  1     1
2  2     4
```

Just find something better/faster than that old pattern...
I've addressed the comments by @rhshadrach. 
Thanks @topper-123
@topper-123  This tripped up a test we do on nightly builds in pandas-stubs, which we can fix, but I think the error message could be improved.  If the recommendation is to use `Series.pipe()` instead to get the same result, then I think that should go in the whatsnew document, i.e., instead of saying "This pattern was very slow and it's recommended to use alternative methods to archive the same goal", say "This pattern was very slow and it's recommended to use`Series.pipe()` to achieve the same goal".  In addition, including that text about `Series.pipe()` should be in the error message (assuming that's the preferred workaround).

Also, the error message doesn't look right due to spacing:
```python
>>> import pandas as pd
>>> pd.__version__
'2.1.0.dev0+463.ge9e034badb'
>>> s = pd.Series([-10, 2, 2, 3.4, 10, 10])
>>> def makeseries(x: float) -> pd.Series:
...     return pd.Series([x, 2 * x])
...
>>> s.apply(makeseries)
<stdin>:1: FutureWarning: Returning a DataFrame from Series.apply when the supplied functionreturns a Series is deprecated and will be removed in a future version.
      0     1
0 -10.0 -20.0
1   2.0   4.0
2   2.0   4.0
3   3.4   6.8
4  10.0  20.0
5  10.0  20.0
```
Note the "functionreturns" misses a space.

Let me know if I should create a separate issue for this.

Hi @Dr-Irv.

The ability to have series put into a `DataFrame` can be misused in many different manners. That pipe example was a way to be able to do that particular operation vectorized, i.e. fast. and it can't be generalized.

A general pattern you could exchange the deprecated patterns with would be to do it like `ser.pipe(lambda x: pd.DataFrame(x.map(value_func_that_returns_series).to_list()))` though that would be dog slow also. To me that's  a quite ugly expression and I guess we could inform them about it in the whatsnew section, with a clear warning about performance and that users should find other patterns to do the operation. IMO that code pattern should not be in the error message, it could encourage slow code.,

You're right about the error message, there's a missing space there.

Tell me what you think about the above.

> Tell me what you think about the above.

I think you'll need to expand on this topic in the `whatsnew` document.  Especially since you removed this from the userguide:
```
``apply`` on a Series can operate on a returned value from the applied function
that is itself a series, and possibly upcast the result to a DataFrame:

.. ipython:: python
    def f(x):
        return pd.Series([x, x ** 2], index=["x", "x^2"])
    s = pd.Series(np.random.rand(5))
    s
    s.apply(f)
```

This is a useful construct - you have a `Series`, and you want to get a `DataFrame` that has multiple columns that are functions of the `Series`.  Now maybe the recommended way of changing the above code would be:
```python
pd.DataFrame(s, columns=['s']).assign(x=lambda df: df.s, x2=lambda df: df.s ** 2).drop(columns=["s"])
```
There are clearly multiple alternatives, but with this deprecation, I think you might want to demonstrate those alternatives in the `whatsnew` document, since we had it in the user guide before as a pattern that people most likely copied.

Agreed with @Dr-Irv 

This needs more explanation somewhere about what to do now
There isa DeprecationWarning missing:

```
df = pd.DataFrame({"x": [1, 2, 3, 4], "y": [10, 20, 30, 40]})


def return_df(x):
    # will create new DataFrame which columns is ['sum', 'mean']
    return pd.Series([x.sum(), x.mean()], index=["sum", "mean"])

df.apply(return_df, axis=1)
```

Looks like axis takes another path?