PERF: Fixed regression in Series(index=idx) constructor
State: closed

From https://github.com/pandas-dev/pandas/pull/18496/

Special cases empty series construction, since the reindex is not necessary.

xref https://github.com/pandas-dev/pandas/issues/18532#issuecomment-385190350

Setup

```python
data = None
idx = date_range(start=datetime(2015, 10, 26),
                              end=datetime(2016, 1, 1),
                              freq='50s')
dict_data = dict(zip(idx, range(len(idx))))
data = None if data is None else dict_data
```

benchmark

```
%timeit out = Series(data, index=idx)
```

HEAD:

```
113 µs ± 6.22 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)
```

0.21.0

```
96.5 µs ± 6.05 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)
```

Master

```
320 ms ± 11.3 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
```

Labels:
name: Performance
description: Memory or execution speed performance

Comments:
cc @toobaz
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/20865?src=pr&el=h1) Report
> Merging [#20865](https://codecov.io/gh/pandas-dev/pandas/pull/20865?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/563a6ad108a79b0192a25d51d743812a18e21b15?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/20865/graphs/tree.svg?token=eZ4WkYLtcO&width=650&src=pr&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/20865?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #20865      +/-   ##
==========================================
+ Coverage   91.77%   91.78%   +<.01%     
==========================================
  Files         153      153              
  Lines       49313    49327      +14     
==========================================
+ Hits        45259    45275      +16     
+ Misses       4054     4052       -2
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.17% <100%> (ø)` | :arrow_up: |
| #single | `41.92% <100%> (+0.03%)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/20865?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/series.py](https://codecov.io/gh/pandas-dev/pandas/pull/20865/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc2VyaWVzLnB5) | `94.03% <100%> (+0.03%)` | :arrow_up: |
| [pandas/io/formats/latex.py](https://codecov.io/gh/pandas-dev/pandas/pull/20865/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvbGF0ZXgucHk=) | `100% <0%> (ø)` | :arrow_up: |
| [pandas/core/arrays/categorical.py](https://codecov.io/gh/pandas-dev/pandas/pull/20865/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL2NhdGVnb3JpY2FsLnB5) | `95.61% <0%> (+0.04%)` | :arrow_up: |
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/20865/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `84.59% <0%> (+0.2%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/20865?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/20865?src=pr&el=footer). Last update [563a6ad...d8b1312](https://codecov.io/gh/pandas-dev/pandas/pull/20865?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Good call on the init_dict. For some reason I thought it had to be in the main Series.__init__, since _init_dict returns a BlockManager & index, but this seems to work.

```
In [4]: %timeit out = Series(data, index=idx)
140 µs ± 1.24 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)
```

Shouldn't need a whatsnew since https://github.com/pandas-dev/pandas/pull/18496 was in 0.23. I think that was the original slowdown.

And this was caught by the `series_methods.SeriesConstructor.time_constructor(None)` asv, so we're good there (that's the benchmark I've been running).
Does d8b1312 change your opinion on putting this stuff in `_init_dict`? It's getting borderline too messy now.

`Series._init_dict` sorts the index (keys) (well on non-PY36). This would break

```python
Series(index=['b', 'a'])
```

Since the user input isn't actually dict-like, it's just going down _init_dict.
I might be wrong (can't test now, sorry), but a simpler solution could maybe be to replace

https://github.com/pandas-dev/pandas/blob/4afc7563895830d224ac949f571edef2f069c314/pandas/core/series.py#L180

with

``` python
if data in [None, {}]:
    data = []
```
?
That won't quiet work since `data` may not be hashable, but I like the
thought. Will try a variant.

On Mon, Apr 30, 2018 at 8:34 AM, Pietro Battiston <notifications@github.com>
wrote:

> I might be wrong (can't test now, sorry), but a simpler solution could be
> to replace
>
> https://github.com/pandas-dev/pandas/blob/4afc7563895830d224ac949f571ede
> f2f069c314/pandas/core/series.py#L180
>
> with
>
> if data in [None, {}]:
>     data = []
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/20865#issuecomment-385399351>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABQHIk_2hzXwI065z48-lX8D9FfUQJWXks5ttxLQgaJpZM4Trxgq>
> .
>

Ah, no, doesn't quite work since the list isn't broadcast to the length of
index. We raise on Series([], index=[0]) since they're different lengths.

On Mon, Apr 30, 2018 at 8:39 AM, Tom Augspurger <tom.augspurger88@gmail.com>
wrote:

> That won't quiet work since `data` may not be hashable, but I like the
> thought. Will try a variant.
>
> On Mon, Apr 30, 2018 at 8:34 AM, Pietro Battiston <
> notifications@github.com> wrote:
>
>> I might be wrong (can't test now, sorry), but a simpler solution could be
>> to replace
>>
>> https://github.com/pandas-dev/pandas/blob/4afc7563895830d224
>> ac949f571edef2f069c314/pandas/core/series.py#L180
>>
>> with
>>
>> if data in [None, {}]:
>>     data = []
>>
>> —
>> You are receiving this because you authored the thread.
>> Reply to this email directly, view it on GitHub
>> <https://github.com/pandas-dev/pandas/pull/20865#issuecomment-385399351>,
>> or mute the thread
>> <https://github.com/notifications/unsubscribe-auth/ABQHIk_2hzXwI065z48-lX8D9FfUQJWXks5ttxLQgaJpZM4Trxgq>
>> .
>>
>
>

> Ah, no, doesn't quite work since the list isn't broadcast to the length of index.

Right!

I thought at least replacing ``data = {}`` with ``data = na_value_for_dtype(dtype)`` would alleviate the problem for the ``data=None`` case, but I'm getting some infinite recursion when collecting tests which I'm unable to explain.
Merging in 2 hours if there are no further comments. I think we're just choosing between the two least-bad outcomes here.
let me look again
lgtm. thanks (I believe you said this was after 0.22 so no whatsnew needed)
Right, slowdown was only on master.