BUG: Fix #25959 - Don't call .array in DatetimeLikeArrayMixin's map
State: closed

- [x] closes #25959
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

In Series.apply we call .array on returning the result of DatetimeLikeArrayMixin map, which causes an error.


Labels:
name: Bug
name: Timezones
description: Timezone data dtype

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25964?src=pr&el=h1) Report
> Merging [#25964](https://codecov.io/gh/pandas-dev/pandas/pull/25964?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/e5d15b20e828c66182a4e175dd5a5b9b951d9950?src=pr&el=desc) will **increase** coverage by `0.03%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/25964/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/25964?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #25964      +/-   ##
==========================================
+ Coverage   91.69%   91.72%   +0.03%     
==========================================
  Files         174      174              
  Lines       50749    50754       +5     
==========================================
+ Hits        46534    46556      +22     
+ Misses       4215     4198      -17
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.23% <100%> (+0.04%)` | :arrow_up: |
| #single | `41.69% <50%> (+0.38%)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/25964?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/series.py](https://codecov.io/gh/pandas-dev/pandas/pull/25964/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc2VyaWVzLnB5) | `93.67% <100%> (ø)` | :arrow_up: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/25964/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `78.94% <0%> (-10.53%)` | :arrow_down: |
| [pandas/compat/\_\_init\_\_.py](https://codecov.io/gh/pandas-dev/pandas/pull/25964/diff?src=pr&el=tree#diff-cGFuZGFzL2NvbXBhdC9fX2luaXRfXy5weQ==) | `93.54% <0%> (-0.4%)` | :arrow_down: |
| [pandas/plotting/\_style.py](https://codecov.io/gh/pandas-dev/pandas/pull/25964/diff?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19zdHlsZS5weQ==) | `76.92% <0%> (-0.26%)` | :arrow_down: |
| [pandas/plotting/\_misc.py](https://codecov.io/gh/pandas-dev/pandas/pull/25964/diff?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19taXNjLnB5) | `38.23% <0%> (-0.23%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/25964/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.02% <0%> (-0.12%)` | :arrow_down: |
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/25964/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `90.6% <0%> (-0.11%)` | :arrow_down: |
| [pandas/plotting/\_converter.py](https://codecov.io/gh/pandas-dev/pandas/pull/25964/diff?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb252ZXJ0ZXIucHk=) | `63.66% <0%> (-0.06%)` | :arrow_down: |
| [pandas/core/groupby/generic.py](https://codecov.io/gh/pandas-dev/pandas/pull/25964/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ3JvdXBieS9nZW5lcmljLnB5) | `88.96% <0%> (-0.05%)` | :arrow_down: |
| [pandas/core/dtypes/dtypes.py](https://codecov.io/gh/pandas-dev/pandas/pull/25964/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2R0eXBlcy5weQ==) | `96.65% <0%> (-0.02%)` | :arrow_down: |
| ... and [21 more](https://codecov.io/gh/pandas-dev/pandas/pull/25964/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25964?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25964?src=pr&el=footer). Last update [e5d15b2...36429a0](https://codecov.io/gh/pandas-dev/pandas/pull/25964?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

> Some comments and also needs a whatsnew entry in 0.25

Added!
Hello @ThomasKluiters! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2019-05-18 19:09:18 UTC
I don't know why the tests are failing - locally all seems fine

Edit - updating from master seemed to have resolved the issue.
@jreback @mroeschke I resolved my issues, could you have a look please?
Can you investigate what happens if we don’t cal tolist? What tests fail?

> On Apr 9, 2019, at 06:00, Thomas Kluiters <notifications@github.com> wrote:
> 
> @ThomasKluiters commented on this pull request.
> 
> In pandas/core/series.py:
> 
> > @@ -3688,7 +3688,11 @@ def f(x):
>  
>          if len(mapped) and isinstance(mapped[0], Series):
>              from pandas.core.frame import DataFrame
> -            return DataFrame(mapped.tolist(), index=self.index)
> +            # GH 25959 we don't need to call tolist
> +            # if we've been given an ExtensionArray
> +            if not isinstance(mapped, ExtensionArray):
> It was initially there - I do not know, I resolved your problem though.
> 
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub, or mute the thread.

> Can you investigate what happens if we don’t cal tolist? What tests fail?
> […](#)
> On Apr 9, 2019, at 06:00, Thomas Kluiters ***@***.***> wrote: @ThomasKluiters commented on this pull request. In pandas/core/series.py: > @@ -3688,7 +3688,11 @@ def f(x): if len(mapped) and isinstance(mapped[0], Series): from pandas.core.frame import DataFrame - return DataFrame(mapped.tolist(), index=self.index) + # GH 25959 we don't need to call tolist + # if we've been given an ExtensionArray + if not isinstance(mapped, ExtensionArray): It was initially there - I do not know, I resolved your problem though. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

Removing the tolist will make this test fail:
Relate to issue #2316
`____________________________________________________________________________ TestSeriesAggregate.test_with_nested_series ____________________________________________________________________________

self = <pandas.tests.series.test_apply.TestSeriesAggregate object at 0x12ebc31d0>
datetime_series = 2000-01-03   -1.347822
2000-01-04    0.739531
2000-01-05    0.230190
2000-01-06   -0.397356
2000-01-07   -0.128090
200...2-08   -1.054427
2000-02-09    0.360638
2000-02-10   -0.639306
2000-02-11   -0.517083
Freq: B, Name: ts, dtype: float64

    def test_with_nested_series(self, datetime_series):
        # GH 2316
        # .agg with a reducer and a transform, what to do
        result = datetime_series.apply(lambda x: Series(
            [x, x ** 2], index=['x', 'x^2']))
        expected = DataFrame({'x': datetime_series,
                              'x^2': datetime_series ** 2})
>       tm.assert_frame_equal(result, expected)
E       AssertionError: DataFrame are different
E       
E       DataFrame shape mismatch
E       [left]:  (30, 1)
E       [right]: (30, 2)
`
So are we using tolist to get an array from a series? Maybe call pd.array on the object instead of tolist?
> So are we using tolist to get an array from a series? Maybe call pd.array on the object instead of tolist?

pd.array seems to be the best solution, it removes the need to check for an EA. Any suggestions on how I can fix the conflict best in the whatsnew?
Actually I think we have a better function than pd.array. It’s something like extract_array in internals. 

> On Apr 10, 2019, at 02:42, Thomas Kluiters <notifications@github.com> wrote:
> 
> So are we using tolist to get an array from a series? Maybe call pd.array on the object instead of tolist?
> 
> pd.array seems to be the best solution, it removes the need to check for an EA. Any suggestions on how I can fix the conflict best in the whatsnew?
> 
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub, or mute the thread.

> extract_array

extract_array does not work I'm afraid, it will fail the same test 
can you merge master
> can you merge master

Done!
can you merge master and ping on green.
you have an isort error; can you merge master and fix up. ping on green.
> you have an isort error; can you merge master and fix up. ping on green.

Sorry, but I can't seem to find what's wrong with the build - it seems to be failing at /doc/source/reference/frame.rst - "DataFrame" has no attribute "to_panel".
Should be fixed after #26308
can you merge master
linting error:

```
./pandas/core/series.py(3708,80): error E501: line too long (82 > 79 characters)
Bash exited with code '1'.
```
ping on green.
@jreblack ping
thanks @ThomasKluiters