API: allow nan-likes in StringArray constructor
State: closed

- [x] closes #40839
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

Looking at OP #30980 this was done to ensure that a double pass is not made over data, but I don't think I'm doing that here. 
This is a precursor for #40687 
Marking as needs discussion since it might be controversial which nan-likes to allow in the constructor.

Labels:
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Strings
description: String extension data type and string data
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
> question for about should we accept NaT / Decimal(NaN) here though. I am kind of -0 on this.

I am also -1 on this. I certainly understand the reason to allow `np.nan` since that's what we generally use now (although I am personally not sure why `StringArray(..)` needs to accept this, if the more flexible `pd.array(..)` already does this), but I would not start accepting NaT/Decimal(NaN).
I am happy to remove ``NaT`` and ``Decimal('NaN')``, as I really only want ``None`` and ``np.nan`` to be accepted in addition to ``pd.NA``.

The problem with pd.array is that it will cast non-string elements to strings, whereas the StringArray constructor will validate the inputs to make sure they are strings or pd.NA. This difference is important for #40687, where we don't want to cast the elements to strings if they are not strings(We want an error to be raised that we can catch).
moving off 1.3 for now, can re-add if fixed before release
Finally got some benchmarks in. (I ran it 3 times). Results are kinda flaky but the perf seems to be equivalent in general. 
<Details>
(pandas-dev) thomasli@Jufangs-MBP asv_bench % asv continuous upstream/master HEAD -b ^strings.StringArrayConstruction
· Creating environments
· Discovering benchmarks
·· Uninstalling from conda-py3.8-Cython0.29.21-jinja2-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt.
·· Installing caa57050 <stringarray-nan> into conda-py3.8-Cython0.29.21-jinja2-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt.
· Running 6 total benchmarks (2 commits * 1 environments * 3 benchmarks)
[  0.00%] · For pandas commit a811c969 <master> (round 1/2):
[  0.00%] ·· Building for conda-py3.8-Cython0.29.21-jinja2-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt..
[  0.00%] ·· Benchmarking conda-py3.8-Cython0.29.21-jinja2-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 16.67%] ··· Running (strings.StringArrayConstruction.time_string_array_construction--)..
[ 25.00%] · For pandas commit caa57050 <stringarray-nan> (round 1/2):
[ 25.00%] ·· Building for conda-py3.8-Cython0.29.21-jinja2-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt..
[ 25.00%] ·· Benchmarking conda-py3.8-Cython0.29.21-jinja2-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 41.67%] ··· Running (strings.StringArrayConstruction.time_string_array_construction--)..
[ 50.00%] · For pandas commit caa57050 <stringarray-nan> (round 2/2):
[ 50.00%] ·· Benchmarking conda-py3.8-Cython0.29.21-jinja2-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 58.33%] ··· strings.StringArrayConstruction.peakmem_stringarray_construction                                                                                                 80.1M
[ 66.67%] ··· strings.StringArrayConstruction.time_string_array_construction                                                                                             1.32±0.08ms
[ 75.00%] ··· strings.StringArrayConstruction.time_string_array_with_nan_construction                                                                                    1.54±0.07ms
[ 75.00%] · For pandas commit a811c969 <master> (round 2/2):
[ 75.00%] ·· Building for conda-py3.8-Cython0.29.21-jinja2-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt..
[ 75.00%] ·· Benchmarking conda-py3.8-Cython0.29.21-jinja2-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 83.33%] ··· strings.StringArrayConstruction.peakmem_stringarray_construction                                                                                                 75.8M
[ 91.67%] ··· strings.StringArrayConstruction.time_string_array_construction                                                                                             1.26±0.07ms
[100.00%] ··· strings.StringArrayConstruction.time_string_array_with_nan_construction                                                                                     1.45±0.2ms

BENCHMARKS NOT SIGNIFICANTLY CHANGED.
(pandas-dev) thomasli@Jufangs-MBP asv_bench % asv continuous upstream/master HEAD -b ^strings.StringArrayConstruction
· Creating environments
· Discovering benchmarks
·· Uninstalling from conda-py3.8-Cython0.29.21-jinja2-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt.
·· Installing caa57050 <stringarray-nan> into conda-py3.8-Cython0.29.21-jinja2-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt.
· Running 6 total benchmarks (2 commits * 1 environments * 3 benchmarks)
[  0.00%] · For pandas commit a811c969 <master> (round 1/2):
[  0.00%] ·· Building for conda-py3.8-Cython0.29.21-jinja2-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt..
[  0.00%] ·· Benchmarking conda-py3.8-Cython0.29.21-jinja2-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 16.67%] ··· Running (strings.StringArrayConstruction.time_string_array_construction--)..
[ 25.00%] · For pandas commit caa57050 <stringarray-nan> (round 1/2):
[ 25.00%] ·· Building for conda-py3.8-Cython0.29.21-jinja2-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt..
[ 25.00%] ·· Benchmarking conda-py3.8-Cython0.29.21-jinja2-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 41.67%] ··· Running (strings.StringArrayConstruction.time_string_array_construction--)..
[ 50.00%] · For pandas commit caa57050 <stringarray-nan> (round 2/2):
[ 50.00%] ·· Benchmarking conda-py3.8-Cython0.29.21-jinja2-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 58.33%] ··· strings.StringArrayConstruction.peakmem_stringarray_construction                                                                                                 79.9M
[ 66.67%] ··· strings.StringArrayConstruction.time_string_array_construction                                                                                              1.16±0.2ms
[ 75.00%] ··· strings.StringArrayConstruction.time_string_array_with_nan_construction                                                                                     1.46±0.1ms
[ 75.00%] · For pandas commit a811c969 <master> (round 2/2):
[ 75.00%] ·· Building for conda-py3.8-Cython0.29.21-jinja2-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt..
[ 75.00%] ·· Benchmarking conda-py3.8-Cython0.29.21-jinja2-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 83.33%] ··· strings.StringArrayConstruction.peakmem_stringarray_construction                                                                                                   76M
[ 91.67%] ··· strings.StringArrayConstruction.time_string_array_construction                                                                                             1.35±0.08ms
[100.00%] ··· strings.StringArrayConstruction.time_string_array_with_nan_construction                                                                                     1.46±0.3ms

BENCHMARKS NOT SIGNIFICANTLY CHANGED.
(pandas-dev) thomasli@Jufangs-MBP asv_bench % asv continuous upstream/master HEAD -b ^strings.StringArrayConstruction
· Creating environments
· Discovering benchmarks
·· Uninstalling from conda-py3.8-Cython0.29.21-jinja2-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt.
·· Installing caa57050 <stringarray-nan> into conda-py3.8-Cython0.29.21-jinja2-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt.
· Running 6 total benchmarks (2 commits * 1 environments * 3 benchmarks)
[  0.00%] · For pandas commit a811c969 <master> (round 1/2):
[  0.00%] ·· Building for conda-py3.8-Cython0.29.21-jinja2-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt..
[  0.00%] ·· Benchmarking conda-py3.8-Cython0.29.21-jinja2-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 16.67%] ··· Running (strings.StringArrayConstruction.time_string_array_construction--)..
[ 25.00%] · For pandas commit caa57050 <stringarray-nan> (round 1/2):
[ 25.00%] ·· Building for conda-py3.8-Cython0.29.21-jinja2-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt..
[ 25.00%] ·· Benchmarking conda-py3.8-Cython0.29.21-jinja2-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 41.67%] ··· Running (strings.StringArrayConstruction.time_string_array_construction--)..
[ 50.00%] · For pandas commit caa57050 <stringarray-nan> (round 2/2):
[ 50.00%] ·· Benchmarking conda-py3.8-Cython0.29.21-jinja2-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 58.33%] ··· strings.StringArrayConstruction.peakmem_stringarray_construction                                                                                                 79.9M
[ 66.67%] ··· strings.StringArrayConstruction.time_string_array_construction                                                                                              1.23±0.1ms
[ 75.00%] ··· strings.StringArrayConstruction.time_string_array_with_nan_construction                                                                                    1.55±0.03ms
[ 75.00%] · For pandas commit a811c969 <master> (round 2/2):
[ 75.00%] ·· Building for conda-py3.8-Cython0.29.21-jinja2-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt..
[ 75.00%] ·· Benchmarking conda-py3.8-Cython0.29.21-jinja2-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 83.33%] ··· strings.StringArrayConstruction.peakmem_stringarray_construction                                                                                                 75.9M
[ 91.67%] ··· strings.StringArrayConstruction.time_string_array_construction                                                                                              1.42±0.1ms
[100.00%] ··· strings.StringArrayConstruction.time_string_array_with_nan_construction                                                                                    1.76±0.05ms
       before           after         ratio
     [a811c969]       [caa57050]
     <master>         <stringarray-nan>
-      1.42±0.1ms       1.23±0.1ms     0.87  strings.StringArrayConstruction.time_string_array_construction

SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.
PERFORMANCE INCREASED.
</Details>
For some reason, the Construction benchmarks are showing up as n/a so I had to put my new benchmarks in a separate class.
Hello @lithomas1! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:















There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-12-27 20:14:09 UTC
@lithomas1 merge conflicts
@simonjayhawkins gentle ping.
@simonjayhawkins gentle ping, this is blocking #40687
status here?
Gonna circle back to this one soon in the next few weeks. Unfortunately, this is probably going to be a pain because I need to deal with 2D StringArrays :(.
@jreback @jorisvandenbossche @jbrockmendel @simonjayhawkins 
This is ready for another look now. The major change now is that we also accept float("nan") now in addition to pd.NA, np.nan, and None now. Originally, only pd.NA, np.nan, and None were accepted, but np.nan wasn't catching all the numpy nans. We are not using np.isnan because np.datetime64("nat") would also be accepted and that is not desirable.
I'll try to benchmark again soon.
Big picture, is there any way to avoid adding a keyword to _from_sequence?  Maybe a) use pd.array?  b) make the less-strict behavior the only behavior?  c) implement _from_sequence_not_strict like DTA/TDA, which is code smell but at least it'll be a matching code smell?
> Big picture, is there any way to avoid adding a keyword to _from_sequence? Maybe a) use pd.array? b) make the less-strict behavior the only behavior? c) implement _from_sequence_not_strict like DTA/TDA, which is code smell but at least it'll be a matching code smell?

I'm thinking about a variant of c. The use case for this was in https://github.com/pandas-dev/pandas/pull/40687#discussion_r637390830. 

pd.array doesn't work b/c it always casts stuff to string, even nans. The less-strict(casting non-strings) behavior is already the default behavior.
e.g.
```
>>> pd.array(np.array(["a", np.nan, "b"]))
<StringArray>
['a', 'nan', 'b']
Length: 3, dtype: string
```
@lithomas1 you've made some progress here via other PRs, is this still active?
@lithomas1 this looked ok, can you merge master
Thanks for the pull request, but it appears to have gone stale. Feel free to reopen when you have time to merge the main branch and address the review comments.

If anyone is looking for something to do I'd just like to say I'm one user who would very much appreciate being able to use the new string dtype with arbitrary null dtypes. I'd pick this up if I had the skills!