Repository: pandas-dev/pandas

Issue:
to_excel cannot handle large sheet
#### Problem description

For **extremely** large sheets (row_num > 1048576, col_num > 16384), see [the code of XlsxWriter Engine](https://github.com/jmcnamara/XlsxWriter/blob/master/xlsxwriter/worksheet.py#L183), area which exceeds the limit won't be printed.
**_However, no error or warning is raised._**

#### Expected Output
```This sheet is too large! Area off (1048576, 1638) will not be printed.```

#### Output of ``pd.show_versions()``
latest (v0.24.2)


Discussion:
A little strange that nothing happens on the xlsxwriter side, but yes would definitely take a PR that guards and raises on our end
I think this makes more sense as an enhancement request for xlsxwriter that would just pass through here rather than doing it on our end
I'm new to pandas-dev. Does the closed tag on the feature request I see above indicate the work is already done and this entire issue should be closed?
I think we could handle on the pandas side - Xlsxwriter's interface is essentially cell-based - there isn't a way to know in advance if those limits are going to be broken.  Because pandas know the total table size in advance, I think it makes sense to check and raise.
I agree @chris-b1. I assume this hasn't been addressed, but I'm uncertain due to the red "Closed" indicator I see beside the feature request above my original comment.
That closed is from the linked issue in the xlsxwriter repo
Ah, makes sense. Thank you!
Going to raise a ValueError unless anyone has a better suggestion.
I believe xlsx writer hangs when passed a sheet that is too large. I test whether calling df.to_excel raises a ValueError when given a sheet that is too large. In that case, the test passes. However, if too large of a sheet is passed, and the code to check input size is not there, the to_excel call will hang resulting in the test stalling. Any ideas on how to best proceed? I believe the best course would be to ignore testing the new checking logic, because adding the test I previously described would just add unhelpful code.
Solved.

Resolving patch:
diff --git a/pandas/io/formats/excel.py b/pandas/io/formats/excel.py
--- a/pandas/io/formats/excel.py
+++ b/pandas/io/formats/excel.py
@@ -341,6 +341,9 @@ class ExcelFormatter:
         This is only called for body cells.
     """
 
+    max_rows = 2**20
+    max_cols = 2**14
+
     def __init__(self, df, na_rep='', float_format=None, cols=None,
                  header=True, index=True, index_label=None, merge_cells=False,
                  inf_rep='inf', style_converter=None):
@@ -648,6 +651,13 @@ def write(self, writer, sheet_name='Sheet1', startrow=0,
         from pandas.io.excel import ExcelWriter
         from pandas.io.common import _stringify_path
 
+        num_rows, num_cols = self.df.shape
+        if num_rows > self.max_rows or num_cols > self.max_cols:
+            raise ValueError("This sheet is too large! Your sheet size is: " +
+                             "{}, {} ".format(num_rows, num_cols) +
+                             "Max sheet size is: {}, {}".
+                             format(self.max_rows, self.max_cols))
+
         if isinstance(writer, ExcelWriter):
             need_save = False
         else:


Test patch:
diff --git a/pandas/tests/io/test_excel.py b/pandas/tests/io/test_excel.py
--- a/pandas/tests/io/test_excel.py
+++ b/pandas/tests/io/test_excel.py
@@ -1185,6 +1185,24 @@ class and any subclasses, on account of the `autouse=True`
 class TestExcelWriter(_WriterBase):
     # Base class for test cases to run with different Excel writers.
 
+    def test_excel_sheet_size(self):
+
+        # GH 26080
+        breaking_row_count = 2**20 + 1
+        breaking_col_count = 2**14 + 1
+        # purposely using two arrays to prevent memory issues while testing
+        row_arr = np.zeros(shape=(breaking_row_count, 1))
+        col_arr = np.zeros(shape=(1, breaking_col_count))
+        row_df = pd.DataFrame(row_arr)
+        col_df = pd.DataFrame(col_arr)
+
+        msg = "sheet is too large"
+        with pytest.raises(ValueError, match=msg):
+            row_df.to_excel(self.path)
+
+        with pytest.raises(ValueError, match=msg):
+            col_df.to_excel(self.path)
+
     def test_excel_sheet_by_name_raise(self, *_):
         import xlrd