Repository: pandas-dev/pandas
Pull request: Fix reading SAS7BDAT files with zero rows
- [x] closes #18198 (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Problem statement:
Reading a SAS file with 0 rows gives None
#### Code Sample, a copy-pastable example if possible

```python
In [2]: import pandas as pd

In [3]: pd.read_csv("zero_observations.csv")
Out[3]: 
Empty DataFrame
Columns: [a, b, c, d]
Index: []

In [4]: pd.read_sas("zero_observations.sas7bdat")

In [5]: 
```

This is `a.csv`:
```
$ cat zero_observations.csv
a,b,c,d
```

#### Problem description

When reading a SAS file with 0 records, it gives `None`. But when reading a CSV file with 0 records it gives an empty dataframe.

In SAS, we can atleast identify the datatypes correctly and make an empty dataframe with this.

#### Output of ``pd.show_versions()``

<details>
INSTALLED VERSIONS
------------------
commit: None
python: 3.5.2.final.0
python-bits: 64
OS: Darwin
OS-release: 16.7.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: en_US.UTF-8
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.21.0
pytest: 3.2.1
pip: 9.0.1
setuptools: 23.0.0
Cython: 0.27.3
numpy: 1.13.3
scipy: 0.18.1
pyarrow: None
xarray: None
IPython: 6.1.0
sphinx: 1.4.9
patsy: None
dateutil: 2.6.1
pytz: 2017.3
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: 2.0.0
openpyxl: None
xlrd: 1.1.0
xlwt: None
xlsxwriter: 0.9.8
lxml: None
bs4: None
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.8.1
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None


</details>

Hints from issue discussion:
is this just an issue for #18184 ?

normally we do open issues first but since u opened the pr we actually don’t need an issue (fyi for the future)
This is actually an issue for 0 rows and not the 0 variables case.
The PR #18184 handles the case for 0 variables. Im not very sure how to fix the one for 0 rows hence created a issue for that
When going through this, I realized there may be some confusion in expected behavior.

In `read_csv()` if I give a file with 0 rows and iterator, it gives:
```python
In [30]: for i, idf in enumerate(pd.read_csv("/tmp/a.csv", iterator=True)):
    ...:     print("IterCount:", i)
    ...:     print(idf)
    ...:     
IterCount: 0
Empty DataFrame
Columns: [a, b, c, d]
Index: []
```

This seems non-intuitive because even though there are 0 rows, the iterator runs atleast once. While in `read_sas()` the equivalent iterator runs 0 times.
In `read_csv()` when I give a file with 2 records, and chunksize=2 it gives an iterator with 1 item. Not two items (First one with 2 rows, Second with 0 rows)
```python
In [32]: for i, idf in enumerate(pd.read_csv("/tmp/b.csv", iterator=True, chunksize=2)):
    ...:     print("IterCount:", i)
    ...:     print(idf)
    ...:     
IterCount: 0
   a  b  c  d
0  1  2  3  4
1  5  6  7  8
```

This is definitely a bit of a corner case but I'd like to know what is the expected behavior.

I believe the following makes sense:
 - If it is an iterator, give `None` so that the iterator can stop correctly
 - If it it not an iterator and the whole dataframe is being read, identify EmptyDataFrames with header names and column types and return the empty data frame
Any thoughts on this one ?
Is this issue still open? Can I pick it up?
@AbdealiJK this shouldn't be too hard to fix, but in order to test will need a zero-observation sas7bdat file.  can you provide one?
Hi

``` SAS
data one_row (compress=no);
	char_field = "abc";
	num_field = 123.4;
run;
data zero_rows_no_compress (compress=no);
	set one_row (obs=0);
run;
```
The above SAS code created the attached SAS file with 0 rows.  Hope this helps @jbrockmendel @mukesh5

[zero_rows_no_compress.zip](https://github.com/pandas-dev/pandas/files/5150911/zero_rows_no_compress.zip)


Hi Sorry @jbrockmendel - seems like I missed your ping earlier.
I don't have access to a SAS environment any more for the past year, so haven't been able to follow up on this, nor provide an example file
just in case pyreadstat can read this file OK (if this is what you are expecting):

```
>>> import pyreadstat
>>> df, meta = pyreadstat.read_sas7bdat("zero_rows_no_compress.sas7bdat")
>>> df
Empty DataFrame
Columns: [char_field, num_field]
Index: []
```

So one option could be to add pyreadstat as backend for read_sas (it is already used in read_spss). It would also help solving other issues: https://github.com/pandas-dev/pandas/issues/37088, https://github.com/pandas-dev/pandas/issues/35545, https://github.com/pandas-dev/pandas/issues/22720
take

Commit message: Add missing file

Patch to review:
diff --git a/doc/source/whatsnew/v1.5.0.rst b/doc/source/whatsnew/v1.5.0.rst
index 4a0b9a97a9d11..af68ffb3b3c66 100644
--- a/doc/source/whatsnew/v1.5.0.rst
+++ b/doc/source/whatsnew/v1.5.0.rst
@@ -814,6 +814,7 @@ I/O
 - Bug in :func:`read_excel` when reading a ``.ods`` file with newlines between xml elements (:issue:`45598`)
 - Bug in :func:`read_parquet` when ``engine="fastparquet"`` where the file was not closed on error (:issue:`46555`)
 - :meth:`to_html` now excludes the ``border`` attribute from ``<table>`` elements when ``border`` keyword is set to ``False``.
+- Bug in :func:`read_sas` returned ``None`` rather than an empty DataFrame for SAS7BDAT files with zero rows (:issue:`18198`)
 -
 
 Period
diff --git a/pandas/io/sas/sas7bdat.py b/pandas/io/sas/sas7bdat.py
index debd686475432..1e071690d35fb 100644
--- a/pandas/io/sas/sas7bdat.py
+++ b/pandas/io/sas/sas7bdat.py
@@ -737,7 +737,7 @@ def read(self, nrows: int | None = None) -> DataFrame | None:
             self.close()
             raise EmptyDataError("No columns to parse from file")
 
-        if self._current_row_in_file_index >= self.row_count:
+        if nrows > 0 and self._current_row_in_file_index >= self.row_count:
             return None
 
         m = self.row_count - self._current_row_in_file_index
diff --git a/pandas/tests/io/sas/data/zero_rows.sas7bdat b/pandas/tests/io/sas/data/zero_rows.sas7bdat
new file mode 100644
index 0000000000000..a5ba95b639507
Binary files /dev/null and b/pandas/tests/io/sas/data/zero_rows.sas7bdat differ
diff --git a/pandas/tests/io/sas/test_sas7bdat.py b/pandas/tests/io/sas/test_sas7bdat.py
index 0dd1fa175fa3f..92d13c8878159 100644
--- a/pandas/tests/io/sas/test_sas7bdat.py
+++ b/pandas/tests/io/sas/test_sas7bdat.py
@@ -216,6 +216,15 @@ def test_zero_variables(datapath):
         pd.read_sas(fname)
 
 
+def test_zero_rows(datapath):
+    # GH 18198
+    fname = datapath("io", "sas", "data", "zero_rows.sas7bdat")
+    df = pd.read_sas(fname)
+    tm.assert_frame_equal(
+        df, pd.DataFrame([{"char_field": "a", "num_field": 1.0}]).iloc[:0]
+    )
+
+
 def test_corrupt_read(datapath):
     # We don't really care about the exact failure, the important thing is
     # that the resource should be cleaned up afterwards (BUG #35566)


Code review comments:
diff_hunk: @@ -216,6 +216,15 @@ def test_zero_variables(datapath):
         pd.read_sas(fname)
 
 
+def test_zero_rows(datapath):
+    # GH 18198
+    fname = datapath("io", "sas", "data", "zero_rows.sas7bdat")
+    df = pd.read_sas(fname)
+    tm.assert_frame_equal(
+        df, pd.DataFrame([{"char_field": "a", "num_field": 1.0}]).iloc[:0]
path: pandas/tests/io/sas/test_sas7bdat.py
text: @user1:
Nit: Could you set these to `result` and `expected` and then call `tm.assert_frame_equal(result, expected)`?