Repository: modin-project/modin
Pull request: FIX-#2320: Raise exceptions in read_csv in some cases with `skipfooter!=0`
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>

<!--
Thank you for your contribution!
Please review the contributing docs: https://modin.readthedocs.io/en/latest/development/contributing.html
if you have questions about contributing.
-->

## What do these changes do?

<!-- Please give a short brief about these changes. -->

- [x] first commit message and PR title follow format outlined [here](https://modin.readthedocs.io/en/latest/development/contributing.html#commit-message-formatting)
  > **_NOTE:_**  If you edit the PR title to match this format, you need to add another commit (even if it's empty) or amend your last commit for the CI job that checks the PR title to pick up the new PR title.
- [x] passes `flake8 modin/ asv_bench/benchmarks scripts/doc_checker.py`
- [x] passes `black --check modin/ asv_bench/benchmarks scripts/doc_checker.py`
- [x] signed commit with `git commit -s` <!-- you can amend your commit with a signature via `git commit -amend -s` -->
- [x] Resolves #2320 <!-- issue must be created for each patch -->
- [x] tests added and passing
- [x] module layout described at `docs/development/architecture.rst` is up-to-date <!-- if you have added, renamed or removed files or directories please update the documentation accordingly -->


Problem statement:
read_csv with Ray engine doesn't raise exceptions while Pandas raises
### System information
- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: Ubuntu 18.04
- **Modin version** (`modin.__version__`): 0.8.1.1+34.ga571e10
- **Python version**: 3.8.6
- **Code we can use to reproduce**:
```
import os

os.environ["MODIN_ENGINE"] = "ray"

import pandas
import modin.pandas as pd
from modin.pandas.test.utils import df_equals

test_filename = "test.csv"
kwargs = {
    "filepath_or_buffer": test_filename,
    "skipfooter": 1,
    "nrows": 1,
}
str_two_cols = """col1,col2
one,two
0,1
2,3
"""

try:
    with open(test_filename, "w") as f:
        f.write(str_two_cols)

    try:
        df_pandas = pandas.read_csv(**kwargs)
    except Exception as exc:
        print("pandas.read_csv raised exception:\n", exc)
    df_pd = pd.read_csv(**kwargs)
    print("pd.read_csv output:\n", df_pd)
finally:
    os.remove(test_filename)
```
<!--
You can obtain the Modin version with

python -c "import modin; print(modin.__version__)"
-->

### Describe the problem
<!-- Describe the problem clearly here. -->

### Source code / logs
<!-- Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached. Try to provide a reproducible test case that is the bare minimum necessary to generate the problem. -->
```
pandas.read_csv raised exception:
 'skipfooter' not supported with 'nrows'
pd.read_csv output:
 Empty DataFrame
Columns: [col1, col2]
Index: []
```

Hints from issue discussion:
Another case with Ray engine:
```
import os

os.environ["MODIN_ENGINE"] = "ray"

import pandas
import modin.pandas as pd

test_filename = "test.csv"

kwargs = {
    "filepath_or_buffer": test_filename,
    "delimiter": " ",
}

test_data = "col1 col2  col3   col4\n" "5 6   7  8\n" "9  10    11 12\n"

try:
    with open(test_filename, "w") as f:
        f.write(test_data)

    try:
        df_pandas = pandas.read_csv(**kwargs)
    except Exception as exc:
        print("pandas.read_csv raised exception:\n", exc)
    df_pd = pd.read_csv(**kwargs)
    print("pd.read_csv output:\n", df_pd)
finally:
    os.remove(test_filename)
```
Output:
```
pandas.read_csv raised exception:
 Error tokenizing data. C error: Expected 7 fields in line 3, saw 8

pd.read_csv output:
    col1  col2  Unnamed: 2  col3  Unnamed: 4  Unnamed: 5  col4
0     5   6.0         NaN   NaN         7.0         NaN     8
1     9   NaN        10.0   NaN         NaN         NaN    11
```
I am still able to reproduce this mismatch on the latest master.

Commit message: FIX-#2320: raise exceptions in read_csv in some cases with skipfooter

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>

Patch to review:
diff --git a/modin/core/io/text/text_file_dispatcher.py b/modin/core/io/text/text_file_dispatcher.py
index 9aeec1c5166..926f99b72c8 100644
--- a/modin/core/io/text/text_file_dispatcher.py
+++ b/modin/core/io/text/text_file_dispatcher.py
@@ -661,6 +661,10 @@ def check_parameters_support(
         if read_kwargs["lineterminator"] is not None:
             return (False, "`lineterminator` parameter is not supported")
 
+        if read_kwargs.get("skipfooter"):
+            if read_kwargs.get("nrows") or read_kwargs.get("engine") == "c":
+                return (False, "raise exception by pandas itself")
+
         skiprows_supported = True
         if is_list_like(skiprows_md) and skiprows_md[0] < header_size:
             skiprows_supported = False
diff --git a/modin/experimental/core/execution/native/implementations/hdk_on_native/io/io.py b/modin/experimental/core/execution/native/implementations/hdk_on_native/io/io.py
index da5f9c390d2..dfb6039a48d 100644
--- a/modin/experimental/core/execution/native/implementations/hdk_on_native/io/io.py
+++ b/modin/experimental/core/execution/native/implementations/hdk_on_native/io/io.py
@@ -454,6 +454,9 @@ def _read_csv_check_support(
                     f"Invalid file path or buffer object type: {type(filepath_or_buffer)}"
                 )
 
+        if read_csv_kwargs.get("skipfooter") and read_csv_kwargs.get("nrows"):
+            return (False, "raise exception by pandas itself")
+
         for arg, def_value in cls.read_csv_unsup_defaults.items():
             if read_csv_kwargs[arg] != def_value:
                 return (
diff --git a/modin/pandas/test/test_io.py b/modin/pandas/test/test_io.py
index 3c857a9786a..d3d618801d5 100644
--- a/modin/pandas/test/test_io.py
+++ b/modin/pandas/test/test_io.py
@@ -335,7 +335,6 @@ def test_read_csv_parsing_1(
 
         eval_io(
             fn_name="read_csv",
-            check_exception_type=None,  # issue #2320
             raising_exceptions=None,
             check_kwargs_callable=not callable(converters),
             # read_csv kwargs
@@ -421,7 +420,6 @@ def test_read_csv_parsing_2(
                     )
             eval_io(
                 fn_name="read_csv",
-                check_exception_type=None,  # issue #2320
                 raising_exceptions=None,
                 check_kwargs_callable=not callable(skiprows),
                 # read_csv kwargs
@@ -449,7 +447,6 @@ def test_read_csv_parsing_3(
 
         eval_io(
             fn_name="read_csv",
-            check_exception_type=None,  # issue #2320
             raising_exceptions=None,
             # read_csv kwargs
             filepath_or_buffer=pytest.csvs_names["test_read_csv_yes_no"],
@@ -744,7 +741,6 @@ def test_read_csv_file_format(
                 )
 
             eval_io(
-                check_exception_type=None,  # issue #2320
                 raising_exceptions=None,
                 fn_name="read_csv",
                 # read_csv kwargs


Code review comments:
diff_hunk: @@ -454,6 +454,9 @@ def _read_csv_check_support(
                     f"Invalid file path or buffer object type: {type(filepath_or_buffer)}"
                 )
 
+        if read_csv_kwargs.get("skipfooter") and read_csv_kwargs.get("nrows"):
+            return (False, "raise exception by pandas itself")
path: modin/experimental/core/execution/native/implementations/hdk_on_native/io/io.py
text: @user1:
```suggestion
            return (False, "Exception is raised by pandas itself")
```

diff_hunk: @@ -661,6 +661,10 @@ def check_parameters_support(
         if read_kwargs["lineterminator"] is not None:
             return (False, "`lineterminator` parameter is not supported")
 
+        if read_kwargs.get("skipfooter"):
+            if read_kwargs.get("nrows") or read_kwargs.get("engine") == "c":
+                return (False, "raise exception by pandas itself")
path: modin/core/io/text/text_file_dispatcher.py
text: @user1:
```suggestion
                return (False, "Exception is raised by pandas itself")
```