Repository: pandas-dev/pandas
Pull request: TST: add test to read empty array
- [ ] closes #41241
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Problem statement:
BUG: New param [use_nullable_dtypes] of pd.read_parquet() can't handle empty parquet file
- [x] I have checked that this issue has not already been reported.

- [x] I have confirmed this bug exists on the latest version of pandas.

- [ ] (optional) I have confirmed this bug exists on the master branch of pandas.

---

#### Code Sample, a copy-pastable example

```python

df_pq = pd.read_parquet(x, use_nullable_dtypes = True)

```

#### Problem description

Get error when add the new parameter **use_nullable_dtypes** to pd.read_parquet().
If remove it , everything go back to normal.
OS: Ubuntu 16 
Python: 3.8

A empty parquet file from spark causes the problem. Its schema is:

Authors,AuthorId,int64
Authors,Rank,int32
Authors,NormalizedName,string
Authors,DisplayName,string
Authors,LastKnownAffiliationId,int64
Authors,PaperCount,int64
Authors,PaperFamilyCount,int64
Authors,CitationCount,int64
Authors,CreatedDate,date32[day]

error msg:

    df_pq = pd.read_parquet(x,use_nullable_dtypes = True)
  File "/vjan/lib/python3.8/site-packages/pandas/io/parquet.py", line 459, in read_parquet
    return impl.read(
  File "/vjan/lib/python3.8/site-packages/pandas/io/parquet.py", line 221, in read
    return self.api.parquet.read_table(
  File "pyarrow/array.pxi", line 751, in pyarrow.lib._PandasConvertible.to_pandas
  File "pyarrow/table.pxi", line 1668, in pyarrow.lib.Table._to_pandas
  File "/vjan/lib/python3.8/site-packages/pyarrow/pandas_compat.py", line 792, in table_to_blockmanager
    blocks = _table_to_blocks(options, table, categories, ext_columns_dtypes)
  File "/vjan/lib/python3.8/site-packages/pyarrow/pandas_compat.py", line 1133, in _table_to_blocks
    return [_reconstruct_block(item, columns, extension_columns)
  File "/vjan/lib/python3.8/site-packages/pyarrow/pandas_compat.py", line 1133, in <listcomp>
    return [_reconstruct_block(item, columns, extension_columns)
  File "/vjan/lib/python3.8/site-packages/pyarrow/pandas_compat.py", line 751, in _reconstruct_block
    pd_ext_arr = pandas_dtype.__from_arrow__(arr)
  File "/vjan/lib/python3.8/site-packages/pandas/core/arrays/integer.py", line 121, in __from_arrow__
    return IntegerArray._concat_same_type(results)
  File "/vjan/lib/python3.8/site-packages/pandas/core/arrays/masked.py", line 271, in _concat_same_type
    data = np.concatenate([x._data for x in to_concat])
  File "<__array_function__ internals>", line 5, in concatenate
ValueError: need at least one array to concatenate

#### Expected Output

read the empty parquet file and generate an empty df

#### Output of ``pd.show_versions()``

<details>

1.2.4

</details>

Hints from issue discussion:
I've run into this as well on pandas 1.2.4 and pyarrow 3.0.0. a simple repro is:
```
import pandas as pd
df = pd.DataFrame({"value": pd.array([], dtype=pd.Int64Dtype()})
df.to_parquet("/path")
df2 = pd.read_parquet("/path")
...
ValueError: need at least one array to concatenate
```

seems just the presence of the nullable dtype is enough to trigger the error
hmm. cc @jorisvandenbossche 
> I've run into this as well on pandas 1.2.4 and pyarrow 3.0.0. a simple repro is:

fixed in commit: [1fb626d0252e8c68e2ad9f7b863c709c67498b1d] BUG: Handle zero-chunked pyarrow.ChunkedArray in StringArray (#41052)

@bob-zhao-work can you try on master
@simonjayhawkins 
Hello, I would like to work on this issue. This will be my first contribution to oss.
I understand that I can test [this commit](https://github.com/simonjayhawkins/pandas/commit/c426ece31e26863653bd6fb84d5b9539e518baa4) in master branch, is that correct?
Thanks @nakatomotoi. pandas has a test suite that is run on ci when a PR is opened. This issue requires a test to be added to the test suite so that we can close the issue knowing that future similar regressions should be less likely.

see https://github.com/pandas-dev/pandas/issues?q=is%3Aissue+is%3Aclosed+label%3A%22Needs+Tests%22 for issues like this that have been closed and check out the associated PRs for insipiration.

The developer guide is https://pandas.pydata.org/pandas-docs/dev/development/index.html
@simonjayhawkins 
Thank you very much for your kind reply.
Now I am going to try to add a test case.
take

Commit message: TST: add test to read zero-chunked array

Patch to review:
diff --git a/pandas/tests/io/test_parquet.py b/pandas/tests/io/test_parquet.py
index 01715ee133e96..dc6014bce815a 100644
--- a/pandas/tests/io/test_parquet.py
+++ b/pandas/tests/io/test_parquet.py
@@ -931,6 +931,14 @@ def test_read_parquet_manager(self, pa, using_array_manager):
         else:
             assert isinstance(result._mgr, pd.core.internals.BlockManager)
 
+    def test_read_zero_chunked_array(self, pa):
+        df = pd.DataFrame({"value": pd.array([], dtype=pd.Int64Dtype())})
+        print(pa)
+        with tm.ensure_clean() as path:
+            df.to_parquet(path, pa)
+            result = read_parquet(path, pa)
+        check_round_trip(df, pa, expected=result)
+
 
 class TestParquetFastParquet(Base):
     def test_basic(self, fp, df_full):


Code review comments:
diff_hunk: @@ -931,6 +931,14 @@ def test_read_parquet_manager(self, pa, using_array_manager):
         else:
             assert isinstance(result._mgr, pd.core.internals.BlockManager)
 
+    def test_read_zero_chunked_array(self, pa):
+        df = pd.DataFrame({"value": pd.array([], dtype=pd.Int64Dtype())})
+        print(pa)
+        with tm.ensure_clean() as path:
path: pandas/tests/io/test_parquet.py
text: @user1:
check_round_trip does this

@author:
I change check_roud_trip to tm.assert_frame_equal

@user1:
no my point is you *can* use check_round_tirp and you don't need L936-939

diff_hunk: @@ -931,6 +931,14 @@ def test_read_parquet_manager(self, pa, using_array_manager):
         else:
             assert isinstance(result._mgr, pd.core.internals.BlockManager)
 
+    def test_read_zero_chunked_array(self, pa):
+        df = pd.DataFrame({"value": pd.array([], dtype=pd.Int64Dtype())})
+        print(pa)
path: pandas/tests/io/test_parquet.py
text: @user1:
remove the print

@author:
I made a mistake. removed it.