Repository: pandas-dev/pandas
Pull request: BUG: clip doesn't preserve dtype by column
- [x] closes #24162
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

```python3
>>> import pandas as pd
>>> data = pd.DataFrame([[1, 2], [3, 4]], columns=['int1', 'int2'])
>>> data.clip_upper(1)
__main__:1: FutureWarning: clip_upper(threshold) is deprecated, use clip(upper=threshold) instead
   int1  int2
0     1     1
1     1     1
```
**Before**
```python3
>>> data['float'] = data['int1'].astype(float)
>>> data.clip_upper(1)
   int1  int2  float
0   1.0   1.0    1.0
1   1.0   1.0    1.0
```
**After**
```python3
>>> data['float'] = data['int1'].astype(float)
>>> data.clip_upper(1)
   int1  int2  float
0     1     1    1.0
1     1     1    1.0
```

Problem statement:
DataFrame.clip_upper does not preserve dtype per column
#### Code Sample

```python
import pandas as pd
data = pd.DataFrame({'INT': [-1, 0, 10, 9],
              'FLOAT': [-0.148, 0.2347, 38.237, 12.2233]},
             index=pd.date_range("20180101 00:00", periods=4))

print('Original data:')
print(data.head())

print('\nThis is probably not a bug but my misunderstanding:')
print('(So how would I apply "clip_upper" inplace on parts of the dataframe?)')
data.loc[[True, True, True, False], ['INT']].clip_upper(8, inplace=True)
print(data.head()) 
# I used then:
# data.loc[[True, True, True, False], ['INT']] = data.loc[[True, True, True, False], ['INT']].clip_upper(8)     

print('\nIt seems that clip_upper does not preserve the dtypes:')
print(data.clip_upper(8).head())

print('\nSame for inplace:')
data.clip_upper(8, inplace=True)
print(data.head())
```
##### Output of this code:

<details>

```
Original data:
            INT    FLOAT
2018-01-01   -1  -0.1480
2018-01-02    0   0.2347
2018-01-03   10  38.2370
2018-01-04    9  12.2233

(A) This is probably not a bug but my misunderstanding:
(So how would I apply "clip_upper" inplace on parts of the dataframe?)
            INT    FLOAT
2018-01-01   -1  -0.1480
2018-01-02    0   0.2347
2018-01-03   10  38.2370
2018-01-04    9  12.2233

(B) It seems that clip_upper does not preserve the dtypes:
            INT   FLOAT
2018-01-01 -1.0 -0.1480
2018-01-02  0.0  0.2347
2018-01-03  8.0  8.0000
2018-01-04  8.0  8.0000

(C) Same for inplace:
            INT   FLOAT
2018-01-01 -1.0 -0.1480
2018-01-02  0.0  0.2347
2018-01-03  8.0  8.0000
2018-01-04  8.0  8.0000
```

</details>

#### Problem description

`clip_upper` with int- and float- columns convert int-column to float.

Calling `data.clip_upper(10)` with an integer, I would expect that it leaves the int-column as integers and the float-column as float. However, it converts everything to float. (see (B) and (C))

Moreover, `clip_upper` with `inplace=True` does not work with `.loc` but this might as well be me understanding the concept wrong... (see (A))

Same for `clip_lower`.

#### Expected Output
For (A):
```
            INT    FLOAT
2018-01-01   -1  -0.1480
2018-01-02    0   0.2347
2018-01-03    8  38.2370
2018-01-04    9  12.2233
```

For (B) and (C):
```
            INT   FLOAT
2018-01-01 -1 -0.1480
2018-01-02  0  0.2347
2018-01-03  8  8.0000
2018-01-04  8  8.0000
```
#### Output of ``pd.show_versions()``

<details>

INSTALLED VERSIONS
------------------
commit: None

pandas: 0.23.4
pytest: 4.0.1
pip: 18.1
setuptools: 40.6.2
Cython: 0.29
numpy: 1.15.4
scipy: 1.1.0
pyarrow: None
xarray: None
IPython: 7.2.0
sphinx: 1.8.2
patsy: None
dateutil: 2.7.5
pytz: 2018.7
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: 3.0.1
openpyxl: 2.5.11
xlrd: 1.1.0
xlwt: 1.3.0
xlsxwriter: 1.1.2
lxml: 4.2.5
bs4: 4.6.3
html5lib: 1.0.1
sqlalchemy: 1.2.14
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
</details>

Hints from issue discussion:
The example is very confusing and pulls in a lot of unnecessary elements. Please keep reports *minimal* in the future.

This is much easier to produce with a small sample:

```python
In [66]: data = pd.DataFrame([[1, 2], [3, 4]], columns=['int1', 'int2']) 
In [66]: data.clip_upper(1)                                                     
Out[66]: 
   int1  int2
0     1     1
1     1     1

In [67]: data['float'] = data['int1'].astype(float)                             

In [68]: data.clip_upper(1)                                                     
Out[68]: 
   int1  int2  float
0   1.0   1.0    1.0
1   1.0   1.0    1.0
```

dtype should probably be preserved by column though it appears the mere presence of a float casts the entire frame.

Investigation and PRs are always welcome
Hi @WillAyd 👋 ,

Happy to look at this issue 🐞 and will revert with a PR 🚀 . 

Thanks,

Ming
```python

In [16]: data
Out[16]:
   int  float
0    1    2.0
1    3    4.0
In [17]: axes_dict = data._construct_axes_dict()
['index', 'columns'] None {'index': RangeIndex(start=0, stop=2, step=1), 'columns': Index(['int', 'float'], dtype='object')}

In [18]: result = data._constructor(data.values, **axes_dict).__finalize__(data)

In [19]: result
Out[19]:
   int  float
0  1.0    2.0
1  3.0    4.0

```

Underlying problem is constructor method which is casting dtype of int column to float.
I will take a look at working of property decorator and create a PR

Commit message: modify expected case

Patch to review:
diff --git a/pandas/core/generic.py b/pandas/core/generic.py
index 65b219cc57f3a..6794d0b37653a 100644
--- a/pandas/core/generic.py
+++ b/pandas/core/generic.py
@@ -7132,28 +7132,21 @@ def _clip_with_scalar(self, lower, upper, inplace=False):
                 (upper is not None and np.any(isna(upper)))):
             raise ValueError("Cannot use an NA value as a clip threshold")
 
-        result = self.values
-        mask = isna(result)
-
-        with np.errstate(all='ignore'):
-            if upper is not None:
-                result = np.where(result >= upper, upper, result)
-            if lower is not None:
-                result = np.where(result <= lower, lower, result)
-        if np.any(mask):
-            result[mask] = np.nan
+        result = self
 
-        axes_dict = self._construct_axes_dict()
-        result = self._constructor(result, **axes_dict).__finalize__(self)
+        if upper is not None:
+            subset = self.le(upper, axis=None) | isna(result)
+            result = result.where(subset, upper, axis=None, inplace=inplace)
+        if lower is not None:
+            if inplace:
+                result = self
+            subset = self.ge(lower, axis=None) | isna(result)
+            result = result.where(subset, lower, axis=None, inplace=inplace)
 
-        if inplace:
-            self._update_inplace(result)
-        else:
-            return result
+        return result
 
     def _clip_with_one_bound(self, threshold, method, axis, inplace):
 
-        inplace = validate_bool_kwarg(inplace, 'inplace')
         if axis is not None:
             axis = self._get_axis_number(axis)
 
diff --git a/pandas/tests/frame/test_analytics.py b/pandas/tests/frame/test_analytics.py
index 88262220015c7..12cbcff4ffc79 100644
--- a/pandas/tests/frame/test_analytics.py
+++ b/pandas/tests/frame/test_analytics.py
@@ -1895,7 +1895,8 @@ def test_clip_mixed_numeric(self):
         df = DataFrame({'A': [1, 2, 3],
                         'B': [1., np.nan, 3.]})
         result = df.clip(1, 2)
-        expected = DataFrame({'A': [1, 2, 2.],
+        # GH 24162, clipping now preserves types
+        expected = DataFrame({'A': [1, 2, 2],
                               'B': [1., np.nan, 2.]})
         tm.assert_frame_equal(result, expected, check_like=True)
 


Code review comments:
diff_hunk: @@ -7132,28 +7132,21 @@ def _clip_with_scalar(self, lower, upper, inplace=False):
                 (upper is not None and np.any(isna(upper)))):
             raise ValueError("Cannot use an NA value as a clip threshold")
 
-        result = self.values
-        mask = isna(result)
-
-        with np.errstate(all='ignore'):
-            if upper is not None:
-                result = np.where(result >= upper, upper, result)
-            if lower is not None:
-                result = np.where(result <= lower, lower, result)
-        if np.any(mask):
-            result[mask] = np.nan
+        result = self
 
-        axes_dict = self._construct_axes_dict()
-        result = self._constructor(result, **axes_dict).__finalize__(self)
+        if upper is not None:
+            subset = self.le(upper, axis=None) | isna(result)
+            result = result.where(subset, upper, axis=None, inplace=inplace)
+        if lower is not None:
+            if inplace:
path: pandas/core/generic.py
text: @user1:
Why is this `inplace` check only when `lower is not None`? It looks like you also assign `result = self` before the upper and lower check.

@author:
Hi @user1, thanks for reviewing!

`result = self` is to ensure there is a return value in case both `lower` and `upper` are None, which they can. when inplace=True, result is None

@user1:
Got it. Thanks!

@user2:
I think its just better to use the existing idiom

```
if inplace:
     self._update_inplace(result0
else:
    return self
```


@author:
actioned. thanks.