REF: StringArray._from_sequence, use less memory
State: closed

- [x] closes #35499
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

@ldacey, can you try if this fixes your problem?

CC @simonjayhawkins  

Labels:
name: Performance
description: Memory or execution speed performance
name: Strings
description: String extension data type and string data

Comments:
I looked at how the array in ``pd.Series(data, str)`` is created. There is a similar function that ensures a string array, but it's slower.

``pd.Series(data, str)`` ends up calling ``dtypes.cast.construct_1d_ndarray_preserving_na``, where curently all scalars are first converted to "<U" dtype, then they're converted back to ``object`` dtype (and ``str`` type) and finally it adds back the nan-likes, where appropriate. That's a lot of steps and in generally the same problem as I'm looking into for ``StringArray``.

By using the new method in ``construct_1d_ndarray_preserving_na`` also, we avoid creating a lot of new objects, by doing a single iteration and reusing exisiting str objects and adding nan-likes in the same iteration rather than in a seperate step. This change means faster performance and lower memory usage:

```python
>>> data = ["aaaaaa" for _ in range(201368)]
>>> %timeit pd.Series(data, dtype=str)
66.5 ms ± 656 µs per loop  # master
36.4 ms ± 330 µs per loop  # this PR
>>> %timeit pd.Series(data, dtype="string")
63.9 ms ± 832 µs per loop  # master
37.2 ms ± 459 µs per loop  # this PR
```

So a bit less than a doubling in performance.

This PR might in the new form now not only be a bug fix for #35499, but also a more fundamental change. Maybe the change to ``construct_1d_ndarray_preserving_na`` should go in a seperate PR that goes into v1.2?
> Maybe the change to `construct_1d_ndarray_preserving_na` should go in a seperate PR that goes into v1.2?

I think now that we are using semver, I'm happy including all the bugfixes in patch versions. I think that'll keep minor releases more focused on enhancements but would also mean more work maintaining the backport branch. let's see what others think.
can u add a memory asv for this case
I'm not familiar with memory ASVS, only timing ones. Can you point to a file where I can one find? (I can then use that as a template for mine)
> I'm not familiar with memory ASVS, only timing ones. Can you point to a file where I can one find? (I can then use that as a template for mine)

https://github.com/pandas-dev/pandas/blob/master/asv_bench/benchmarks/rolling.py#L24
I've added ASV's. I'm however not able to run them on my machine (windows 10), so not what they show what I think they should show...Maybe someone could check it?
> I've added ASV's. I'm however not able to run them on my machine (windows 10), so not what they show what I think they should show...Maybe someone could check it?

you can just run memits to assert that they are workign, no?
@topper-123 can you rebase and show the `%memit` on before / after
Ok, I've used memit, didn't know about that, it's nice.

Results:

```python
>>> data = ["aaaaaa" for _ in range(201368)]
>>>
>>> %timeit pd.Series(data, dtype=str)
75.2 ms ± 725 µs per loop  # master
40.6 ms ± 389 µs per loop  # this PR
>>> %memit pd.Series(data, dtype=str)
peak memory: 100.96 MiB, increment: 18.87 MiB  # master
peak memory: 84.96 MiB, increment: 1.63 MiB  # this PR
>>>
>>> %timeit pd.Series(data, dtype="string")
70 ms ± 928 µs per loop  # master
41.9 ms ± 910 µs per loop  # this PR
>>> %memit pd.Series(data, dtype="string")
peak memory: 96.68 MiB, increment: 13.47 MiB  # master
peak memory: 85.25 MiB, increment: 1.55 MiB  # this PR
```

so a significant decrease in both time and memory usage.

the time and memory improvements are of course related, coming from avoiding the (simplified) pattern  ``np.array(data, dtype=str).astype(object)``, that is used in master.
thanks @topper-123 
@meeseeksdev backport 1.1.x