Repository: modin-project/modin

Issue:
BUG: `df.describe` fails for DataFrame created with `add_prefix`
### System information
- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: macOS 12.2.1
- **Modin version** (`modin.__version__`): 0.15.0+75.g3f985ed6
- **Python version**: 3.9.12
- **Code we can use to reproduce**:

```python
import modin.pandas as pd
import numpy as np

df = pd.DataFrame(np.random.randn(20, 16)).add_prefix('col')

# This fails
df.describe()
```

### Describe the problem
`df.describe()` fails on a DataFrame that has its columns set with `add_prefix`. 

### Source code / logs
<!-- Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached. Try to provide a reproducible test case that is the bare minimum necessary to generate the problem. -->

<details>
<summary>Stack trace</summary>

```
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Input In [19], in <cell line: 1>()
----> 1 df.describe()

File ~/Documents/modin/modin/logging/logger_decorator.py:128, in enable_logging.<locals>.decorator.<locals>.run_and_log(*args, **kwargs)
    113 """
    114 Compute function with logging if Modin logging is enabled.
    115 
   (...)
    125 Any
    126 """
    127 if LogMode.get() == "disable":
--> 128     return obj(*args, **kwargs)
    130 logger = get_logger()
    131 logger_level = getattr(logger, log_level)

File ~/Documents/modin/modin/pandas/base.py:1172, in BasePandasDataset.describe(self, percentiles, include, exclude, datetime_is_numeric)
   1169 else:
   1170     percentiles = np.array([0.25, 0.5, 0.75])
   1171 return self.__constructor__(
-> 1172     query_compiler=self._query_compiler.describe(
   1173         percentiles=percentiles,
   1174         include=include,
   1175         exclude=exclude,
   1176         datetime_is_numeric=datetime_is_numeric,
   1177     )
   1178 )

File ~/Documents/modin/modin/logging/logger_decorator.py:128, in enable_logging.<locals>.decorator.<locals>.run_and_log(*args, **kwargs)
    113 """
    114 Compute function with logging if Modin logging is enabled.
    115 
   (...)
    125 Any
    126 """
    127 if LogMode.get() == "disable":
--> 128     return obj(*args, **kwargs)
    130 logger = get_logger()
    131 logger_level = getattr(logger, log_level)

File ~/Documents/modin/modin/core/storage_formats/pandas/query_compiler.py:1563, in PandasQueryCompiler.describe(self, **kwargs)
   1560 def describe(self, **kwargs):
   1561     # Use pandas to calculate the correct columns
   1562     empty_df = (
-> 1563         pandas.DataFrame(columns=self.columns)
   1564         .astype(self.dtypes)
   1565         .describe(**kwargs)
   1566     )
   1567     new_index = empty_df.index
   1569     # Note: `describe` convert timestamp type to object type
   1570     # which results in the loss of two values in index: `first` and `last`
   1571     # for empty DataFrame.

File ~/opt/anaconda3/envs/modin/lib/python3.9/site-packages/pandas/core/generic.py:5879, in NDFrame.astype(self, dtype, copy, errors)
   5877 for col_name in dtype.keys():
   5878     if col_name not in self:
-> 5879         raise KeyError(
   5880             "Only a column name can be used for the "
   5881             "key in a dtype mappings argument. "
   5882             f"'{col_name}' not found in columns."
   5883         )
   5885 # GH#44417 cast to Series so we can use .iat below, which will be
   5886 #  robust in case we
   5887 from pandas import Series

```

</details>


Discussion:
I get the same error on my macbook at Modin 3f985ed6864cc1b5b587094d75ca5b2695e4139f.
It looks like the error here was caused by the fact that when we call `add_prefix`, we call `rename` and we do not update the labels for the DataFrame dtypes which results in this error. I suspect that we may have other issues with dtypes since we don't check for them in `df_equals` which is used a lot in our tests. See: https://github.com/modin-project/modin/issues/3804. We should make the `df_equals` fix a priority and fix it soon. 

Resolving patch:
diff --git a/modin/core/dataframe/pandas/dataframe/dataframe.py b/modin/core/dataframe/pandas/dataframe/dataframe.py
--- a/modin/core/dataframe/pandas/dataframe/dataframe.py
+++ b/modin/core/dataframe/pandas/dataframe/dataframe.py
@@ -1754,13 +1754,14 @@ def map_fn(df):
             return df.rename(index=new_row_labels, columns=new_col_labels, level=level)
 
         new_parts = self._partition_mgr_cls.map_partitions(self._partitions, map_fn)
+        new_dtypes = None if self._dtypes is None else self._dtypes.set_axis(new_cols)
         return self.__constructor__(
             new_parts,
             new_index,
             new_cols,
             self._row_lengths,
             self._column_widths,
-            self._dtypes,
+            new_dtypes,
         )
 
     def sort_by(


Test patch:
diff --git a/modin/pandas/test/dataframe/test_map_metadata.py b/modin/pandas/test/dataframe/test_map_metadata.py
--- a/modin/pandas/test/dataframe/test_map_metadata.py
+++ b/modin/pandas/test/dataframe/test_map_metadata.py
@@ -205,6 +205,9 @@ def test_add_prefix(data):
     new_modin_df = modin_df.add_prefix(test_prefix)
     new_pandas_df = pandas_df.add_prefix(test_prefix)
     df_equals(new_modin_df.columns, new_pandas_df.columns)
+    # TODO(https://github.com/modin-project/modin/issues/3804):
+    # make df_equals always check dtypes.
+    df_equals(new_modin_df.dtypes, new_pandas_df.dtypes)
 
 
 @pytest.mark.parametrize("data", test_data_values, ids=test_data_keys)