Repository: modin-project/modin

Issue:
[PERF] compute `dtypes` when performing binary operations
It should be possible and beneficial to compute `dtypes` when doing binary operations without remote calls so other subsequent operations, which require `dypes` to be known, do not perform the remote calls to compute it.

Here are some examples when we can compute/pass known dtypes to the core dataframe.

1. `left_df + right_df`, leftleft_df.dtypes and right_df.dtypes are known and equal to each other
2. `left_df + 1`, left_df.dtypes is known, dtype of 1 is int.


Discussion:
https://github.com/modin-project/modin/issues/4868
As resultant dtypes for each function  are dependent on the operation. Looks like this has to be done specific to each function. 

eg For add(df1+d2)   for each column the resultant dtype would be determined by  any of the 2 below cases



**Case 1**)Check whether dtype of column in df1/df2 can be casted to dtype  of column in df2/df1.  eg If np.int8  can be casted to np.int16, then np.int16 becomes resultant dtype  

eg 

```
df1 = pd.DataFrame(np.ones((10, 10)),dtype=np.int8)
df2 = pd.DataFrame(np.ones((10, 10)),dtype=np.int16)
df3=df1+df2
print(df3.dtypes)  # resultant would be of dtype int16 
```

**Case 2**)In case if dtypes cant be casted. A dtype which is just higher than both of the dtypes is assigned

eg:

```
df1 = pd.DataFrame(np.ones((10, 10)),dtype=np.float16)
df2 = pd.DataFrame(np.ones((10, 10)),dtype=np.int16)
df3=df1+df2
print(df3.dtypes)  # prints float32 which is not in any of the input dtypes
```

Note: With pandas supporting 20 dtypes there can be 190 ways to combine them. With 163 of them falling in first case and 27 in second case. The first case is easily implementable. Looking into ways to implement case 2 without giving say 27 rules to handle each set of dtypes.
We could use the following method from pandas. It seems to me it is used in pandas to determine resultant dtypes.

```python
from pandas.core.dtypes.cast import find_common_type

find_common_type([np.float16, np.int16])
dtype('float32')
```

Resolving patch:
diff --git a/modin/core/dataframe/algebra/binary.py b/modin/core/dataframe/algebra/binary.py
--- a/modin/core/dataframe/algebra/binary.py
+++ b/modin/core/dataframe/algebra/binary.py
@@ -19,11 +19,126 @@
 from .operator import Operator
 
 
+def coerce_int_to_float64(dtype: np.dtype) -> np.dtype:
+    """
+    Coerce dtype to float64 if it is a variant of integer.
+
+    If dtype is integer, function returns float64 datatype.
+    If not, returns the datatype argument itself.
+
+    Parameters
+    ----------
+    dtype : np.dtype
+        NumPy datatype.
+
+    Returns
+    -------
+    dtype : np.dtype
+        Returns float64 for all int datatypes or returns the datatype itself
+        for other types.
+
+    Notes
+    -----
+    Used to precompute datatype in case of division in pandas.
+    """
+    if dtype in np.sctypes["int"] + np.sctypes["uint"]:
+        return np.dtype(np.float64)
+    else:
+        return dtype
+
+
+def compute_dtypes_common_cast(first, second) -> np.dtype:
+    """
+    Precompute data types for binary operations by finding common type between operands.
+
+    Parameters
+    ----------
+    first : PandasQueryCompiler
+        First operand for which the binary operation would be performed later.
+    second : PandasQueryCompiler
+        Second operand for which the binary operation would be performed later.
+
+    Returns
+    -------
+    dtypes
+        The pandas series with precomputed dtypes.
+
+    Notes
+    -----
+    The dtypes of the operands are supposed to be known.
+    """
+    dtypes_first = first._modin_frame._dtypes.to_dict()
+    dtypes_second = second._modin_frame._dtypes.to_dict()
+    columns_first = set(first.columns)
+    columns_second = set(second.columns)
+    common_columns = columns_first.intersection(columns_second)
+    mismatch_columns = columns_first.union(columns_second) - common_columns
+    # If at least one column doesn't match, the result of the non matching column would be nan.
+    nan_dtype = np.dtype(type(np.nan))
+    dtypes = pandas.Series(
+        [
+            pandas.core.dtypes.cast.find_common_type(
+                [
+                    dtypes_first[x],
+                    dtypes_second[x],
+                ]
+            )
+            for x in common_columns
+        ],
+        index=common_columns,
+    )
+    dtypes = pandas.concat(
+        [
+            dtypes,
+            pandas.Series(
+                [nan_dtype] * (len(mismatch_columns)),
+                index=mismatch_columns,
+            ),
+        ]
+    )
+    dtypes = dtypes.sort_index()
+    return dtypes
+
+
+def compute_dtypes_boolean(first, second) -> np.dtype:
+    """
+    Precompute data types for boolean operations.
+
+    Parameters
+    ----------
+    first : PandasQueryCompiler
+        First operand for which the binary operation would be performed later.
+    second : PandasQueryCompiler
+        Second operand for which the binary operation would be performed later.
+
+    Returns
+    -------
+    dtypes
+        The pandas series with precomputed dtypes.
+
+    Notes
+    -----
+    Finds a union of columns and finds dtypes for all these columns.
+    """
+    columns_first = set(first.columns)
+    columns_second = set(second.columns)
+    columns_union = columns_first.union(columns_second)
+    dtypes = pandas.Series([np.dtype(bool)] * len(columns_union), index=columns_union)
+    dtypes = dtypes.sort_index()
+    return dtypes
+
+
 class Binary(Operator):
     """Builder class for Binary operator."""
 
     @classmethod
-    def register(cls, func, join_type="outer", labels="replace"):
+    def register(
+        cls,
+        func,
+        join_type="outer",
+        labels="replace",
+        infer_dtypes=None,
+    ):
         """
         Build template binary operator.
 
@@ -36,6 +151,14 @@ def register(cls, func, join_type="outer", labels="replace"):
         labels : {"keep", "replace", "drop"}, default: "replace"
             Whether keep labels from left Modin DataFrame, replace them with labels
             from joined DataFrame or drop altogether to make them be computed lazily later.
+        infer_dtypes : {"common_cast", "float", "bool", None}, default: None
+            How dtypes should be inferred.
+                * If "common_cast", casts to common dtype of operand columns.
+                * If "float", performs type casting by finding common dtype.
+                  If the common dtype is any of the integer types, perform type casting to float.
+                  Used in case of truediv.
+                * If "bool", dtypes would be a boolean series with same size as that of operands.
+                * If ``None``, do not infer new dtypes (they will be computed manually once accessed).
 
         Returns
         -------
@@ -96,11 +219,23 @@ def caller(
                         )
                     )
                 else:
+                    if (
+                        other._modin_frame._dtypes is not None
+                        and query_compiler._modin_frame._dtypes is not None
+                    ):
+                        if infer_dtypes == "bool":
+                            dtypes = compute_dtypes_boolean(query_compiler, other)
+                        if infer_dtypes == "common_cast":
+                            dtypes = compute_dtypes_common_cast(query_compiler, other)
+                        elif infer_dtypes == "float":
+                            dtypes = compute_dtypes_common_cast(query_compiler, other)
+                            dtypes = dtypes.apply(coerce_int_to_float64)
                     return query_compiler.__constructor__(
                         query_compiler._modin_frame.n_ary_op(
                             lambda x, y: func(x, y, *args, **kwargs),
                             [other._modin_frame],
                             join_type=join_type,
+                            dtypes=dtypes,
                         )
                     )
             else:
diff --git a/modin/core/dataframe/pandas/dataframe/dataframe.py b/modin/core/dataframe/pandas/dataframe/dataframe.py
--- a/modin/core/dataframe/pandas/dataframe/dataframe.py
+++ b/modin/core/dataframe/pandas/dataframe/dataframe.py
@@ -2815,7 +2815,12 @@ def _copartition(self, axis, other, how, sort, force_repartition=False):
 
     @lazy_metadata_decorator(apply_axis="both")
     def n_ary_op(
-        self, op, right_frames: list, join_type="outer", copartition_along_columns=True
+        self,
+        op,
+        right_frames: list,
+        join_type="outer",
+        copartition_along_columns=True,
+        dtypes=None,
     ):
         """
         Perform an n-opary operation by joining with other Modin DataFrame(s).
@@ -2831,6 +2836,9 @@ def n_ary_op(
         copartition_along_columns : bool, default: True
             Whether to perform copartitioning along columns or not.
             For some ops this isn't needed (e.g., `fillna`).
+        dtypes : series, default: None
+            Dtypes of the resultant dataframe, this argument will be
+            received if the resultant dtypes of n-opary operation is precomputed.
 
         Returns
         -------
@@ -2885,6 +2893,7 @@ def n_ary_op(
             joined_columns,
             row_lengths,
             column_widths,
+            dtypes,
         )
 
     @lazy_metadata_decorator(apply_axis="both")
diff --git a/modin/core/storage_formats/pandas/query_compiler.py b/modin/core/storage_formats/pandas/query_compiler.py
--- a/modin/core/storage_formats/pandas/query_compiler.py
+++ b/modin/core/storage_formats/pandas/query_compiler.py
@@ -375,34 +375,34 @@ def to_numpy(self, **kwargs):
     # such that columns/rows that don't have an index on the other DataFrame
     # result in NaN values.
 
-    add = Binary.register(pandas.DataFrame.add)
-    combine = Binary.register(pandas.DataFrame.combine)
-    combine_first = Binary.register(pandas.DataFrame.combine_first)
-    eq = Binary.register(pandas.DataFrame.eq)
-    floordiv = Binary.register(pandas.DataFrame.floordiv)
-    ge = Binary.register(pandas.DataFrame.ge)
-    gt = Binary.register(pandas.DataFrame.gt)
-    le = Binary.register(pandas.DataFrame.le)
-    lt = Binary.register(pandas.DataFrame.lt)
-    mod = Binary.register(pandas.DataFrame.mod)
-    mul = Binary.register(pandas.DataFrame.mul)
-    rmul = Binary.register(pandas.DataFrame.rmul)
-    ne = Binary.register(pandas.DataFrame.ne)
-    pow = Binary.register(pandas.DataFrame.pow)
-    radd = Binary.register(pandas.DataFrame.radd)
-    rfloordiv = Binary.register(pandas.DataFrame.rfloordiv)
-    rmod = Binary.register(pandas.DataFrame.rmod)
-    rpow = Binary.register(pandas.DataFrame.rpow)
-    rsub = Binary.register(pandas.DataFrame.rsub)
-    rtruediv = Binary.register(pandas.DataFrame.rtruediv)
-    sub = Binary.register(pandas.DataFrame.sub)
-    truediv = Binary.register(pandas.DataFrame.truediv)
-    __and__ = Binary.register(pandas.DataFrame.__and__)
-    __or__ = Binary.register(pandas.DataFrame.__or__)
-    __rand__ = Binary.register(pandas.DataFrame.__rand__)
-    __ror__ = Binary.register(pandas.DataFrame.__ror__)
-    __rxor__ = Binary.register(pandas.DataFrame.__rxor__)
-    __xor__ = Binary.register(pandas.DataFrame.__xor__)
+    add = Binary.register(pandas.DataFrame.add, infer_dtypes="common_cast")
+    combine = Binary.register(pandas.DataFrame.combine, infer_dtypes="common_cast")
+    combine_first = Binary.register(pandas.DataFrame.combine_first, infer_dtypes="bool")
+    eq = Binary.register(pandas.DataFrame.eq, infer_dtypes="bool")
+    floordiv = Binary.register(pandas.DataFrame.floordiv, infer_dtypes="common_cast")
+    ge = Binary.register(pandas.DataFrame.ge, infer_dtypes="bool")
+    gt = Binary.register(pandas.DataFrame.gt, infer_dtypes="bool")
+    le = Binary.register(pandas.DataFrame.le, infer_dtypes="bool")
+    lt = Binary.register(pandas.DataFrame.lt, infer_dtypes="bool")
+    mod = Binary.register(pandas.DataFrame.mod, infer_dtypes="common_cast")
+    mul = Binary.register(pandas.DataFrame.mul, infer_dtypes="common_cast")
+    rmul = Binary.register(pandas.DataFrame.rmul, infer_dtypes="common_cast")
+    ne = Binary.register(pandas.DataFrame.ne, infer_dtypes="bool")
+    pow = Binary.register(pandas.DataFrame.pow, infer_dtypes="common_cast")
+    radd = Binary.register(pandas.DataFrame.radd, infer_dtypes="common_cast")
+    rfloordiv = Binary.register(pandas.DataFrame.rfloordiv, infer_dtypes="common_cast")
+    rmod = Binary.register(pandas.DataFrame.rmod, infer_dtypes="common_cast")
+    rpow = Binary.register(pandas.DataFrame.rpow, infer_dtypes="common_cast")
+    rsub = Binary.register(pandas.DataFrame.rsub, infer_dtypes="common_cast")
+    rtruediv = Binary.register(pandas.DataFrame.rtruediv, infer_dtypes="float")
+    sub = Binary.register(pandas.DataFrame.sub, infer_dtypes="common_cast")
+    truediv = Binary.register(pandas.DataFrame.truediv, infer_dtypes="float")
+    __and__ = Binary.register(pandas.DataFrame.__and__, infer_dtypes="bool")
+    __or__ = Binary.register(pandas.DataFrame.__or__, infer_dtypes="bool")
+    __rand__ = Binary.register(pandas.DataFrame.__rand__, infer_dtypes="bool")
+    __ror__ = Binary.register(pandas.DataFrame.__ror__, infer_dtypes="bool")
+    __rxor__ = Binary.register(pandas.DataFrame.__rxor__, infer_dtypes="bool")
+    __xor__ = Binary.register(pandas.DataFrame.__xor__, infer_dtypes="bool")
     df_update = Binary.register(
         copy_df_for_func(pandas.DataFrame.update, display_name="update"),
         join_type="left",


Test patch: