Repository: unionai-oss/pandera

Issue:
No row index provided in failure_cases["index"] after type check failure
**Describe the bug**
If a column fails the initial type check, no index is provided for that row in the failure_cases["index"], so I am unable to drop the row from the original dataframe.

- [X] I have checked that this issue has not already been reported.
- [x] I have confirmed this bug exists on the latest version of pandera.
- [ ] (optional) I have confirmed this bug exists on the master branch of pandera.

**Note**: Please read [this guide](https://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports) detailing how to provide the necessary information for us to reproduce your bug.

#### Code Sample, a copy-pastable example

```python
schema: pa.DataFrameSchema = pa.DataFrameSchema(columns={
  'Col1': pa.Column(str),
})

df: pd.DataFrame = pd.DataFrame({
    "Col1": ["1", "2", 3]
})

try:
    schema(df, lazy=True)
except pa.errors.SchemaErrors as exc:
    filtered_df = df[~df.index.isin(exc.failure_cases["index"])]

```

#### Expected behavior
I would expect the failure cases index to include the index for the 3rd row that contained an integer for Col1 instead of a string. This way, the failing row could be dropped if my desire is to ignore the failing rows and continue processing rather than halt the application. However there is no index present. Printing out `exc.failure_cases["index"]` reads:
```
Out[1]: 
0    None
Name: index, dtype: object
```

#### Desktop (please complete the following information):

 - OS: Windows



Discussion:
Hello, any update on this bug? 
Hi @shanetorres which version of pandera are you using?

I'm seeing a different issue, which is that validation is passing, which is a bug in the way `str` types are validated. Can you try doing a [development installation](https://pandera.readthedocs.io/en/stable/CONTRIBUTING.html#environment-setup) of the `main` branch and re-running your code?

Resolving patch:
diff --git a/pandera/engines/pandas_engine.py b/pandera/engines/pandas_engine.py
--- a/pandera/engines/pandas_engine.py
+++ b/pandera/engines/pandas_engine.py
@@ -685,7 +685,18 @@ def check(
         pandera_dtype: dtypes.DataType,
         data_container: Optional[PandasObject] = None,
     ) -> Union[bool, Iterable[bool]]:
-        return isinstance(pandera_dtype, (numpy_engine.Object, type(self)))
+        if data_container is None:
+            return isinstance(pandera_dtype, (numpy_engine.Object, type(self)))
+
+        # NOTE: this is a hack to handle the following case:
+        # pyspark.pandas doesn't support types with a Series of type object
+        if type(data_container).__module__.startswith("pyspark.pandas"):
+            is_python_string = data_container.map(lambda x: str(type(x))).isin(  # type: ignore[operator]
+                ["<class 'str'>", "<class 'numpy.str_'>"]
+            )
+        else:
+            is_python_string = data_container.map(type).isin([str, np.str_])  # type: ignore[operator]
+        return is_python_string | data_container.isna()
 
 
 Engine.register_dtype(


Test patch:
diff --git a/tests/core/test_dtypes.py b/tests/core/test_dtypes.py
--- a/tests/core/test_dtypes.py
+++ b/tests/core/test_dtypes.py
@@ -518,6 +518,23 @@ def test_numpy_dtypes(alias, np_dtype):
         )
 
 
+def test_numpy_string():
+    """
+    Test that numpy string dtype check identifies the correct failure case.
+    """
+    schema: pa.DataFrameSchema = pa.DataFrameSchema(
+        columns={
+            "col1": pa.Column(str),
+        }
+    )
+    df: pd.DataFrame = pd.DataFrame({"col1": ["1", "2", 3]})
+    try:
+        print(schema(df, lazy=True))
+    except pa.errors.SchemaErrors as exc:
+        assert exc.failure_cases["failure_case"].item() == 3
+        assert exc.failure_cases["index"].item() == 2
+
+
 @pytest.mark.parametrize(
     "examples",
     [
diff --git a/tests/strategies/test_strategies.py b/tests/strategies/test_strategies.py
--- a/tests/strategies/test_strategies.py
+++ b/tests/strategies/test_strategies.py
@@ -558,9 +558,8 @@ def test_dataframe_strategy_with_indexes(data_type, data):
     )
 
     dataframe_schema_index(data.draw(dataframe_schema_index.strategy(size=10)))
-    dataframe_schema_multiindex(
-        data.draw(dataframe_schema_multiindex.strategy(size=10))
-    )
+    example = data.draw(dataframe_schema_multiindex.strategy(size=10))
+    dataframe_schema_multiindex(example)
 
 
 @hypothesis.given(st.data())