Repository: databricks/koalas

Issue:
Operations on Columns of Differing Types
I just noticed this difference between `koalas` and `pandas` when it comes to performing operations on columns of different types e.g. `int` and `str`. Pandas will throw a `TypeError` whereas `koalas` won't and will actually return something. I'm sure this isn't a bug but I wondered if there's anywhere where this behaviour is documented that I can read about? The return types seem to be a little unpredictable. I have only looked at `str` and `int` so far but it would be great if there was more information on all the different types and why they return what they return. Also, is there a reason, we don't throw a `TypeError`? This would be a very quick check just based on the `dtype` of the column

## Example
### Koalas
```python
df = ks.DataFrame({'a':[1, 2, 3], 'b':['a', '2', 'c'], 'c':['d', 'e', 'f']})
df['add_ab'] = df['a'] + df['b']
df['sub_ab'] = df['a'] - df['b']
df['div_ab'] = df['a'] / df['b']
df['mul_ab'] = df['a'] * df['b']
df['add_bc'] = df['b'] + df['c']
df['sub_bc'] = df['b'] - df['c']
df['div_bc'] = df['b'] / df['c']
df['mul_bc'] = df['b'] * df['c']
print(df)
#    a  b  c  add_ab  sub_ab  div_ab  mul_ab add_bc  sub_bc    div_bc  mul_bc
# 0  1  a  d     NaN     NaN     inf     NaN     ad     NaN      None     NaN
# 1  2  2  e     4.0     0.0     1.0     4.0     2e     NaN  Infinity     NaN
# 2  3  c  f     NaN     NaN     inf     NaN     cf     NaN      None     NaN
```

- Interestingly, '2' as a string in column 'b' behaves as a number.
- div_ab results in `inf` whereas the other ab operations result in `NaN`
- div_bc this time results in `None` where the other operations (ignoring add) result in `NaN` again
    - except for row 2 which this time results in `Infinity`

Essentially, I'm not sure why the string version of 2 gets typecast to an integer. And why division results in behaviour that is different to the other operations. And why this is sometimes `None` and sometimes `inf`. And what the difference is between `inf` and `Infinity`.

### Pandas
```python
df = pd.DataFrame({'a':[1, 2, 3], 'b':['a', '2', 'c'], 'c':['d', 'e', 'f']})
df['mul_ab'] = df['a'] * df['b']
df['add_bc'] = df['b'] + df['c']
print(df)
#    a  b  c mul_ab add_bc
# 0  1  a  d      a     ad
# 1  2  2  e     22     2e
# 2  3  c  f    ccc     cf
```

Comparing to the `pandas` version. The two operations included above are the only two that do not result in a `TypeError`. Here the 2 remains as a string. And the allowed operations are as expected.

## Sidenote
```python
df = ks.DataFrame({'a':[1, 2, 3], 'b':['a', 2, 'c'], 'c':['d','e', 'f']})
```
throws an `ArrowTypeError` (the 2 in col 'b' is now an int rather than a string). Pandas however allows this Dataframe to be created. Furthermore, I'm not sure how to make sense of this given that the 2 gets effectively turned into an `int` anyway if given as a `str`.

## Summary

To summarise, I'm not sure what of the above are bugs (if any). But a detailed breakdown of how functions on columns of different types behave compared to their pandas counterparts would be much appreciated. 

koalas==1.2.0
pandas==1.0.5


Discussion:
Some of the string operations seem bugs. Let us investigate it and submit a PR.

Regarding the sidenote, PySpark (and PyArrow) doesn't allow to have different data type values in a single column, so neither does Koalas. That's one of strict restrictions of Koalas.

Resolving patch:
diff --git a/databricks/koalas/base.py b/databricks/koalas/base.py
--- a/databricks/koalas/base.py
+++ b/databricks/koalas/base.py
@@ -34,6 +34,7 @@
     DateType,
     DoubleType,
     FloatType,
+    IntegralType,
     LongType,
     StringType,
     TimestampType,
@@ -46,6 +47,7 @@
     NATURAL_ORDER_COLUMN_NAME,
     SPARK_DEFAULT_INDEX_NAME,
 )
+from databricks.koalas.spark import functions as SF
 from databricks.koalas.spark.accessors import SparkIndexOpsMethods
 from databricks.koalas.typedef import as_spark_type, spark_type_to_pandas_dtype
 from databricks.koalas.utils import align_diff_series, same_anchor, scol_for, validate_axis
@@ -171,6 +173,11 @@ def spark_column(self):
     __neg__ = column_op(Column.__neg__)
 
     def __add__(self, other):
+        if not isinstance(self.spark.data_type, StringType) and (
+            (isinstance(other, IndexOpsMixin) and isinstance(other.spark.data_type, StringType))
+            or isinstance(other, str)
+        ):
+            raise TypeError("string addition can only be applied to string series or literals.")
         if isinstance(self.spark.data_type, StringType):
             # Concatenate string columns
             if isinstance(other, IndexOpsMixin) and isinstance(other.spark.data_type, StringType):
@@ -184,6 +191,13 @@ def __add__(self, other):
             return column_op(Column.__add__)(self, other)
 
     def __sub__(self, other):
+        if (
+            isinstance(self.spark.data_type, StringType)
+            or (isinstance(other, IndexOpsMixin) and isinstance(other.spark.data_type, StringType))
+            or isinstance(other, str)
+        ):
+            raise TypeError("substraction can not be applied to string series or literals.")
+
         if isinstance(self.spark.data_type, TimestampType):
             # Note that timestamp subtraction casts arguments to integer. This is to mimic pandas's
             # behaviors. pandas returns 'timedelta64[ns]' from 'datetime64[ns]'s subtraction.
@@ -220,7 +234,24 @@ def __sub__(self, other):
                 raise TypeError("date subtraction can only be applied to date series.")
         return column_op(Column.__sub__)(self, other)
 
-    __mul__ = column_op(Column.__mul__)
+    def __mul__(self, other):
+        if isinstance(other, str):
+            raise TypeError("multiplication can not be applied to a string literal.")
+
+        if (
+            isinstance(self.spark.data_type, IntegralType)
+            and isinstance(other, IndexOpsMixin)
+            and isinstance(other.spark.data_type, StringType)
+        ):
+            return column_op(SF.repeat)(other, self)
+
+        if isinstance(self.spark.data_type, StringType) and (
+            (isinstance(other, IndexOpsMixin) and isinstance(other.spark.data_type, IntegralType))
+            or isinstance(other, int)
+        ):
+            return column_op(SF.repeat)(self, other)
+
+        return column_op(Column.__mul__)(self, other)
 
     def __truediv__(self, other):
         """
@@ -240,6 +271,13 @@ def __truediv__(self, other):
         +-----------------------|---------|---------+
         """
 
+        if (
+            isinstance(self.spark.data_type, StringType)
+            or (isinstance(other, IndexOpsMixin) and isinstance(other.spark.data_type, StringType))
+            or isinstance(other, str)
+        ):
+            raise TypeError("division can not be applied on string series or literals.")
+
         def truediv(left, right):
             return F.when(F.lit(right != 0) | F.lit(right).isNull(), left.__div__(right)).otherwise(
                 F.when(F.lit(left == np.inf) | F.lit(left == -np.inf), left).otherwise(
@@ -250,6 +288,13 @@ def truediv(left, right):
         return numpy_column_op(truediv)(self, other)
 
     def __mod__(self, other):
+        if (
+            isinstance(self.spark.data_type, StringType)
+            or (isinstance(other, IndexOpsMixin) and isinstance(other.spark.data_type, StringType))
+            or isinstance(other, str)
+        ):
+            raise TypeError("modulo can not be applied on string series or literals.")
+
         def mod(left, right):
             return ((left % right) + right) % right
 
@@ -257,12 +302,21 @@ def mod(left, right):
 
     def __radd__(self, other):
         # Handle 'literal' + df['col']
-        if isinstance(self.spark.data_type, StringType) and isinstance(other, str):
-            return self._with_new_scol(F.concat(F.lit(other), self.spark.column))
+        if not isinstance(self.spark.data_type, StringType) and isinstance(other, str):
+            raise TypeError("string addition can only be applied to string series or literals.")
+
+        if isinstance(self.spark.data_type, StringType):
+            if isinstance(other, str):
+                return self._with_new_scol(F.concat(F.lit(other), self.spark.column))
+            else:
+                raise TypeError("string addition can only be applied to string series or literals.")
         else:
             return column_op(Column.__radd__)(self, other)
 
     def __rsub__(self, other):
+        if isinstance(self.spark.data_type, StringType) or isinstance(other, str):
+            raise TypeError("substraction can not be applied to string series or literals.")
+
         if isinstance(self.spark.data_type, TimestampType):
             # Note that timestamp subtraction casts arguments to integer. This is to mimic pandas's
             # behaviors. pandas returns 'timedelta64[ns]' from 'datetime64[ns]'s subtraction.
@@ -291,9 +345,19 @@ def __rsub__(self, other):
                 raise TypeError("date subtraction can only be applied to date series.")
         return column_op(Column.__rsub__)(self, other)
 
-    __rmul__ = column_op(Column.__rmul__)
+    def __rmul__(self, other):
+        if isinstance(other, str):
+            raise TypeError("multiplication can not be applied to a string literal.")
+
+        if isinstance(self.spark.data_type, StringType) and isinstance(other, int):
+            return column_op(SF.repeat)(self, other)
+
+        return column_op(Column.__rmul__)(self, other)
 
     def __rtruediv__(self, other):
+        if isinstance(self.spark.data_type, StringType) or isinstance(other, str):
+            raise TypeError("division can not be applied on string series or literals.")
+
         def rtruediv(left, right):
             return F.when(left == 0, F.lit(np.inf).__div__(right)).otherwise(
                 F.lit(right).__truediv__(left)
@@ -318,6 +382,12 @@ def __floordiv__(self, other):
         |          -10          |   null  | -np.inf |
         +-----------------------|---------|---------+
         """
+        if (
+            isinstance(self.spark.data_type, StringType)
+            or (isinstance(other, IndexOpsMixin) and isinstance(other.spark.data_type, StringType))
+            or isinstance(other, str)
+        ):
+            raise TypeError("division can not be applied on string series or literals.")
 
         def floordiv(left, right):
             return F.when(F.lit(right is np.nan), np.nan).otherwise(
@@ -333,6 +403,9 @@ def floordiv(left, right):
         return numpy_column_op(floordiv)(self, other)
 
     def __rfloordiv__(self, other):
+        if isinstance(self.spark.data_type, StringType) or isinstance(other, str):
+            raise TypeError("division can not be applied on string series or literals.")
+
         def rfloordiv(left, right):
             return F.when(F.lit(left == 0), F.lit(np.inf).__div__(right)).otherwise(
                 F.when(F.lit(left) == np.nan, np.nan).otherwise(F.floor(F.lit(right).__div__(left)))
@@ -341,6 +414,9 @@ def rfloordiv(left, right):
         return numpy_column_op(rfloordiv)(self, other)
 
     def __rmod__(self, other):
+        if isinstance(self.spark.data_type, StringType) or isinstance(other, str):
+            raise TypeError("modulo can not be applied on string series or literals.")
+
         def rmod(left, right):
             return ((right % left) + left) % left
 
diff --git a/databricks/koalas/spark/functions.py b/databricks/koalas/spark/functions.py
--- a/databricks/koalas/spark/functions.py
+++ b/databricks/koalas/spark/functions.py
@@ -77,6 +77,15 @@ def array_repeat(col, count):
     )
 
 
+def repeat(col, n):
+    """
+    Repeats a string column n times, and returns it as a new string column.
+    """
+    sc = SparkContext._active_spark_context
+    n = _to_java_column(n) if isinstance(n, Column) else _create_column_from_literal(n)
+    return _call_udf(sc, "repeat", _to_java_column(col), n)
+
+
 def _call_udf(sc, name, *cols):
     return Column(sc._jvm.functions.callUDF(name, _make_arguments(sc, *cols)))
 


Test patch:
diff --git a/databricks/koalas/tests/test_dataframe.py b/databricks/koalas/tests/test_dataframe.py
--- a/databricks/koalas/tests/test_dataframe.py
+++ b/databricks/koalas/tests/test_dataframe.py
@@ -1750,6 +1750,110 @@ def test_binary_operators(self):
             lambda: ks.range(10).add(ks.range(10).id),
         )
 
+    def test_binary_operator_add(self):
+        # Positive
+        pdf = pd.DataFrame({"a": ["x"], "b": ["y"], "c": [1], "d": [2]})
+        kdf = ks.from_pandas(pdf)
+
+        self.assert_eq(kdf["a"] + kdf["b"], pdf["a"] + pdf["b"])
+        self.assert_eq(kdf["c"] + kdf["d"], pdf["c"] + pdf["d"])
+
+        # Negative
+        ks_err_msg = "string addition can only be applied to string series or literals"
+
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: kdf["a"] + kdf["c"])
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: kdf["c"] + kdf["a"])
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: kdf["c"] + "literal")
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: "literal" + kdf["c"])
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: 1 + kdf["a"])
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: kdf["a"] + 1)
+
+    def test_binary_operator_sub(self):
+        # Positive
+        pdf = pd.DataFrame({"a": [2], "b": [1]})
+        kdf = ks.from_pandas(pdf)
+
+        self.assert_eq(kdf["a"] - kdf["b"], pdf["a"] - pdf["b"])
+
+        # Negative
+        kdf = ks.DataFrame({"a": ["x"], "b": [1]})
+        ks_err_msg = "substraction can not be applied to string series or literals"
+
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: kdf["a"] - kdf["b"])
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: kdf["b"] - kdf["a"])
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: kdf["b"] - "literal")
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: "literal" - kdf["b"])
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: 1 - kdf["a"])
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: kdf["a"] - 1)
+
+        kdf = ks.DataFrame({"a": ["x"], "b": ["y"]})
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: kdf["a"] - kdf["b"])
+
+    def test_binary_operator_truediv(self):
+        # Positive
+        pdf = pd.DataFrame({"a": [3], "b": [2]})
+        kdf = ks.from_pandas(pdf)
+
+        self.assert_eq(kdf["a"] / kdf["b"], pdf["a"] / pdf["b"])
+
+        # Negative
+        kdf = ks.DataFrame({"a": ["x"], "b": [1]})
+        ks_err_msg = "division can not be applied on string series or literals"
+
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: kdf["a"] / kdf["b"])
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: kdf["b"] / kdf["a"])
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: kdf["b"] / "literal")
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: "literal" / kdf["b"])
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: 1 / kdf["a"])
+
+    def test_binary_operator_floordiv(self):
+        kdf = ks.DataFrame({"a": ["x"], "b": [1]})
+        ks_err_msg = "division can not be applied on string series or literals"
+
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: kdf["a"] // kdf["b"])
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: kdf["b"] // kdf["a"])
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: kdf["b"] // "literal")
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: "literal" // kdf["b"])
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: 1 // kdf["a"])
+
+    def test_binary_operator_mod(self):
+        # Positive
+        pdf = pd.DataFrame({"a": [3], "b": [2]})
+        kdf = ks.from_pandas(pdf)
+
+        self.assert_eq(kdf["a"] % kdf["b"], pdf["a"] % pdf["b"])
+
+        # Negative
+        kdf = ks.DataFrame({"a": ["x"], "b": [1]})
+        ks_err_msg = "modulo can not be applied on string series or literals"
+
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: kdf["a"] % kdf["b"])
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: kdf["b"] % kdf["a"])
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: kdf["b"] % "literal")
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: 1 % kdf["a"])
+
+    def test_binary_operator_multiply(self):
+        # Positive
+        pdf = pd.DataFrame({"a": ["x", "y"], "b": [1, 2], "c": [3, 4]})
+        kdf = ks.from_pandas(pdf)
+
+        self.assert_eq(kdf["b"] * kdf["c"], pdf["b"] * pdf["c"])
+        self.assert_eq(kdf["c"] * kdf["b"], pdf["c"] * pdf["b"])
+        self.assert_eq(kdf["a"] * kdf["b"], pdf["a"] * pdf["b"])
+        self.assert_eq(kdf["b"] * kdf["a"], pdf["b"] * pdf["a"])
+        self.assert_eq(kdf["a"] * 2, pdf["a"] * 2)
+        self.assert_eq(kdf["b"] * 2, pdf["b"] * 2)
+        self.assert_eq(2 * kdf["a"], 2 * pdf["a"])
+        self.assert_eq(2 * kdf["b"], 2 * pdf["b"])
+
+        # Negative
+        kdf = ks.DataFrame({"a": ["x"], "b": [2]})
+        ks_err_msg = "multiplication can not be applied to a string literal"
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: kdf["b"] * "literal")
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: "literal" * kdf["b"])
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: kdf["a"] * "literal")
+        self.assertRaisesRegex(TypeError, ks_err_msg, lambda: "literal" * kdf["a"])
+
     def test_sample(self):
         pdf = pd.DataFrame({"A": [0, 2, 4]})
         kdf = ks.from_pandas(pdf)