Repository: streamlit/streamlit

Issue:
st.dataframe does not handle StringDtype in Pandas >= 1.0.0
# Summary

Displaying a DataFrame with a 'string' or StringArray dtype column fails with a NotImplementedError. This dtype is new in Pandas 1.0.0.

# Steps to reproduce

Run this app:

```python
import pandas as pd
import streamlit as st

st.header('DataFrame with StringDtype')
df = pd.DataFrame({
    'a': pd.Series('we are the knights who say ni'.split(), dtype='string'),
})
st.dataframe(df)
```

# Debug info

- Streamlit version: 0.54.0
- Python version: 3.7.6
- Pandas version: 1.0.0
- Using Pipenv
- OS version: MacOS 10.15.3

st.dataframe does not handle StringDtype in Pandas >= 1.0.0
# Summary

Displaying a DataFrame with a 'string' or StringArray dtype column fails with a NotImplementedError. This dtype is new in Pandas 1.0.0.

# Steps to reproduce

Run this app:

```python
import pandas as pd
import streamlit as st

st.header('DataFrame with StringDtype')
df = pd.DataFrame({
    'a': pd.Series('we are the knights who say ni'.split(), dtype='string'),
})
st.dataframe(df)
```

# Debug info

- Streamlit version: 0.54.0
- Python version: 3.7.6
- Pandas version: 1.0.0
- Using Pipenv
- OS version: MacOS 10.15.3



Discussion:
Thanks for the issue @Stigjb ! There are number of `Dtypes` that we don't have support for yet.

Added this to the board for triage.
> Thanks for the issue @Stigjb ! There are number of `Dtypes` that we don't have support for yet.
> 
> Added this to the board for triage.

This is a problem of pandas not streamlit, if u run 
```
df = pd.DataFrame({'a': pd.Series('we are the knights who say ni'.split(), dtype='string'),})
```
in notebook, the same error will raise ,just do not use dtype='string' and set it to 'object' or 'str' will fix this issue, python do not have a  type name 'string'😢
@NickYi1990 , the code runs perfectly fine in a notebook. `dtype='string'` only works if your Pandas version is >= 1.0.0, which was only recently released.
> @NickYi1990 , the code runs perfectly fine in a notebook. `dtype='string'` only works if your Pandas version is >= 1.0.0, which was only recently released.

ops, I missed the version, yes! my pandas version is 0.24.1, thanks for pointing out this.
Closing as it is not a bug with Streamlit.  Thanks for filing it though, it's useful to get some info about pandas changes that may affect us soon.
Not sure @nthmost should have closed this. The example code above gives me an error in the `st.dataframe` call. As @Stigjb says the construction of the pandas dataframe is fine with pandas>=1. Converting the new pandas dtypes back to objects is not straightforward.

I'm on
```
streamlit: 0.57.2

python: 3.7.6 | packaged by conda-forge | (default, Mar 5 2020, 15:03:29) [Clang 9.0.1 ]

platform: Darwin-18.7.0-x86_64-i386-64bit

pandas: 1.0.2
```
Not sure my previous comment was as clear as it could have been. What I meant to say is that the pandas 1.0 code in the @Stigjb 's example code is perfectly valid and generates a valid pandas dataframe with a `pd.StringDtype` column. However when this is passed to `st.dataframe`, streamlit raises an exception. This looks like a bug in streamlit to me.
There are multiple new dtypes that are not supported in `st.dataframe`. In addition to `pd.StringDtype`, there is a `pd.CategoricalDtype` (also new in Pandas v1), which Streamlit cannot display with `st.dataframe` yet.
Thanks for the issue @Stigjb ! There are number of `Dtypes` that we don't have support for yet.

Added this to the board for triage.
> Thanks for the issue @Stigjb ! There are number of `Dtypes` that we don't have support for yet.
> 
> Added this to the board for triage.

This is a problem of pandas not streamlit, if u run 
```
df = pd.DataFrame({'a': pd.Series('we are the knights who say ni'.split(), dtype='string'),})
```
in notebook, the same error will raise ,just do not use dtype='string' and set it to 'object' or 'str' will fix this issue, python do not have a  type name 'string'😢
@NickYi1990 , the code runs perfectly fine in a notebook. `dtype='string'` only works if your Pandas version is >= 1.0.0, which was only recently released.
> @NickYi1990 , the code runs perfectly fine in a notebook. `dtype='string'` only works if your Pandas version is >= 1.0.0, which was only recently released.

ops, I missed the version, yes! my pandas version is 0.24.1, thanks for pointing out this.
Closing as it is not a bug with Streamlit.  Thanks for filing it though, it's useful to get some info about pandas changes that may affect us soon.
Not sure @nthmost should have closed this. The example code above gives me an error in the `st.dataframe` call. As @Stigjb says the construction of the pandas dataframe is fine with pandas>=1. Converting the new pandas dtypes back to objects is not straightforward.

I'm on
```
streamlit: 0.57.2

python: 3.7.6 | packaged by conda-forge | (default, Mar 5 2020, 15:03:29) [Clang 9.0.1 ]

platform: Darwin-18.7.0-x86_64-i386-64bit

pandas: 1.0.2
```
Not sure my previous comment was as clear as it could have been. What I meant to say is that the pandas 1.0 code in the @Stigjb 's example code is perfectly valid and generates a valid pandas dataframe with a `pd.StringDtype` column. However when this is passed to `st.dataframe`, streamlit raises an exception. This looks like a bug in streamlit to me.
There are multiple new dtypes that are not supported in `st.dataframe`. In addition to `pd.StringDtype`, there is a `pd.CategoricalDtype` (also new in Pandas v1), which Streamlit cannot display with `st.dataframe` yet.

Resolving patch:
diff --git a/lib/streamlit/elements/data_frame_proto.py b/lib/streamlit/elements/data_frame_proto.py
--- a/lib/streamlit/elements/data_frame_proto.py
+++ b/lib/streamlit/elements/data_frame_proto.py
@@ -388,6 +388,11 @@ def _marshall_any_array(pandas_array, proto_array):
         proto_array.int64s.data.extend(pandas_array)
     elif pandas_array.dtype == np.object:
         proto_array.strings.data.extend(map(str, pandas_array))
+    # dtype='string', <class 'pandas.core.arrays.string_.StringDtype'>
+    # NOTE: StringDtype is considered experimental.
+    # The implementation and parts of the API may change without warning.
+    elif pandas_array.dtype.name == "string":
+        proto_array.strings.data.extend(map(str, pandas_array))
     # Setting a timezone changes (dtype, dtype.type) from
     #   'datetime64[ns]', <class 'numpy.datetime64'>
     # to


Test patch:
diff --git a/lib/tests/streamlit/data_frame_proto_test.py b/lib/tests/streamlit/data_frame_proto_test.py
--- a/lib/tests/streamlit/data_frame_proto_test.py
+++ b/lib/tests/streamlit/data_frame_proto_test.py
@@ -229,6 +229,12 @@ def test_marshall_any_array(self):
         data_frame_proto._marshall_any_array(obj_data, obj_proto)
         self.assertEqual(obj_proto.strings.data, truth)
 
+        # StringDtype
+        string_data = pd.Series(["foo", "bar", "baz"], dtype="string")
+        string_proto = AnyArray()
+        data_frame_proto._marshall_any_array(string_data, string_proto)
+        self.assertEqual(string_proto.strings.data, string_data.tolist())
+
         # No timezone
         dt_data = pd.Series([np.datetime64("2019-04-09T12:34:56")])
         dt_proto = AnyArray()