Repository: unionai-oss/pandera

Issue:
Schema with MultiIndex's subindex dtype declared as something other than 'object' fails to validate an empty dataframe
**Describe the bug**
On an empty dataframe, with an empty multi index (ie empty sub indexes), validating the dataframe using the schema (declaring each subindex with non 'object' dtype) silently converts subindexes dtypes to 'object' types. Thereby failing the schema validation when the indexes in the schema declare for example 'Int64' dtype for each subindex.

- [x] I have checked that this issue has not already been reported.
- [x] I have confirmed this bug exists on the latest version of pandera.
- [x] (optional) I have confirmed this bug exists on the master branch of pandera.

**Note**: Please read [this guide](https://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports) detailing how to provide the necessary information for us to reproduce your bug.

#### Code Sample, a copy-pastable example

```python
class TestSchema(pandera.SchemaModel)
       level0: pandera.typing.Index[pd.Int64Dtype] = pandera.Field(coerce=True)
       level1: pandera.typing.Index[pd.Int64Dtype] = pandera.Field(coerce=True)


data = pd.DataFrame(index=pd.MultiIndex.from_arrays([[]] * 2))
schema = TestSchema.to_schema()
schema.validate(data, {'lazy': false, 'inplace':True})
# Throws SchemaError: expected series '0' to have type Int64, got object

```

#### Expected behavior
An **empty** dataframe's multiindex whose schema indicate a specific dtype (eg: Int64) must be converted to said dtype and pass the schema validation.

#### Desktop (please complete the following information):

 - OS: macOS 12.4
 - Browser: NA
 - Version: pandera v0.12
 - 
#### Screenshots
None

#### Additional context
None
Schema with MultiIndex's subindex dtype declared as something other than 'object' fails to validate an empty dataframe
**Describe the bug**
On an empty dataframe, with an empty multi index (ie empty sub indexes), validating the dataframe using the schema (declaring each subindex with non 'object' dtype) silently converts subindexes dtypes to 'object' types. Thereby failing the schema validation when the indexes in the schema declare for example 'Int64' dtype for each subindex.

- [x] I have checked that this issue has not already been reported.
- [x] I have confirmed this bug exists on the latest version of pandera.
- [x] (optional) I have confirmed this bug exists on the master branch of pandera.

**Note**: Please read [this guide](https://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports) detailing how to provide the necessary information for us to reproduce your bug.

#### Code Sample, a copy-pastable example

```python
class TestSchema(pandera.SchemaModel)
       level0: pandera.typing.Index[pd.Int64Dtype] = pandera.Field(coerce=True)
       level1: pandera.typing.Index[pd.Int64Dtype] = pandera.Field(coerce=True)


data = pd.DataFrame(index=pd.MultiIndex.from_arrays([[]] * 2))
schema = TestSchema.to_schema()
schema.validate(data, {'lazy': false, 'inplace':True})
# Throws SchemaError: expected series '0' to have type Int64, got object

```

#### Expected behavior
An **empty** dataframe's multiindex whose schema indicate a specific dtype (eg: Int64) must be converted to said dtype and pass the schema validation.

#### Desktop (please complete the following information):

 - OS: macOS 12.4
 - Browser: NA
 - Version: pandera v0.12
 - 
#### Screenshots
None

#### Additional context
None


Discussion:

Resolving patch:
diff --git a/noxfile.py b/noxfile.py
--- a/noxfile.py
+++ b/noxfile.py
@@ -289,14 +289,6 @@ def requirements(session: Session) -> None:  # pylint:disable=unused-argument
         sys.exit(1)
 
 
-@nox.session(python=PYTHON_VERSIONS)
-def mypy(session: Session) -> None:
-    """Type-check using mypy."""
-    install_extras(session, extra="all", pandas_stubs=False)
-    args = session.posargs or SOURCE_PATHS
-    session.run("mypy", "--follow-imports=silent", *args, silent=True)
-
-
 EXTRA_NAMES = [
     extra
     for extra in REQUIRES
diff --git a/pandera/schema_components.py b/pandera/schema_components.py
--- a/pandera/schema_components.py
+++ b/pandera/schema_components.py
@@ -585,7 +585,17 @@ def coerce_dtype(self, obj: pd.MultiIndex) -> pd.MultiIndex:
             multiindex_cls = ps.MultiIndex
         return multiindex_cls.from_arrays(
             [
+                # v.dtype may be different than 'object'.
+                # - Reuse the original index array to keep the specialized dtype:
+                #   v.to_numpy()  converts the array dtype to array of 'object' dtype.
+                #   Thus removing the specialized index dtype required to pass a schema's
+                #   index specialized dtype : eg: pandera.typing.Index(pandas.Int64Dtype)
+                # - For Pyspark only, use to_numpy(), with the effect of keeping the
+                #   bug open on this execution environment: At the time of writing, pyspark
+                #   v3.3.0 does not provide a working implementation of v.array
                 v.to_numpy()
+                if type(v).__module__.startswith("pyspark.pandas")
+                else v.array
                 for k, v in sorted(
                     coerced_multi_index.items(), key=lambda x: x[0]
                 )


Test patch:
diff --git a/tests/core/test_schema_components.py b/tests/core/test_schema_components.py
--- a/tests/core/test_schema_components.py
+++ b/tests/core/test_schema_components.py
@@ -251,6 +251,35 @@ def tests_multi_index_subindex_coerce() -> None:
         schema(data, lazy=True)
 
 
+@pytest.mark.parametrize("coerce", [True, False])
+def tests_multi_index_subindex_coerce_with_empty_subindex(coerce) -> None:
+    """MultIndex component should override each sub indexes dtype,
+    even if the sub indexes are empty (ie do not rely on
+    numpy to infer the subindex dtype.
+    """
+    indexes = [
+        Index(pd.Int64Dtype, coerce=coerce),
+        Index(pd.StringDtype, coerce=coerce),
+    ]
+
+    data = pd.DataFrame(index=pd.MultiIndex.from_arrays([[]] * len(indexes)))
+
+    schema_override = DataFrameSchema(index=MultiIndex(indexes))
+
+    if coerce:
+        validated_df_override = schema_override(data)
+        for level_i in range(validated_df_override.index.nlevels):
+            assert isinstance(
+                validated_df_override.index.get_level_values(level_i).dtype,
+                type(indexes[level_i].dtype.type),  # type: ignore[attr-defined]
+            )
+    else:
+        with pytest.raises(
+            errors.SchemaErrors, match="A total of 2 schema errors were found"
+        ):
+            schema_override(data, lazy=True)
+
+
 def test_schema_component_equality_operators():
     """Test the usage of == for Column, Index and MultiIndex."""
     column = Column(Int, Check(lambda s: s >= 0))