Repository: pandas-dev/pandas

Issue:
pd.to_numeric silently not converting when errors='coerce'
#### Code Sample, a copy-pastable example if possible

```python
import pandas as pd
pd.to_numeric([-47393996303418497800, 'string'], errors='coerce')

```

Output:

array([-47393996303418497800, 'string'], dtype=object)

#### Problem description

to_numeric is silently not coercing 'string' to nan.

Seems similar to #17125 and #17007

pandas version 0.23.4


Discussion:
Yeah, the specific issue here seems to be that your number is outside of int64 space

```python
-47393996303418497800 < np.iinfo('int64').min
# Out[18]: True
```
Some discussion in #17007 - I suppose the right thing do is convert to a float, even though lossy

```python
-47393996303418497800 - int(float(-47393996303418497800))
# Out[24]: 1272
```

cc @gfyoung 
Yeah...this is a tough one.  When I initially worked on this, I really didn't want to coerce to `float` given how lossy that could be in cases like this.  That being said, if you're passing in `coerce`, does that mean you're willing to accept lossiness in return for coercing every entry to some kind of number?

@fakio : What would you personally prefer?

cc @jreback 
Some more examples I find strange:

```python
pd.to_numeric([100000000000000000000])
array([1.e+20])

pd.to_numeric([-100000000000000000000])
OverflowError: int too big to convert

pd.to_numeric(100000000000000000000)
100000000000000000000

pd.to_numeric(-100000000000000000000)
-100000000000000000000

pd.to_numeric('100000000000000000000')
pd.to_numeric('-100000000000000000000')
pd.to_numeric(['100000000000000000000'])
pd.to_numeric(['-100000000000000000000'])
ValueError: Integer out of range. at position 0

pd.to_numeric(['10000000000000000000000000', 'string'], errors='coerce')
array([nan, nan])

pd.to_numeric([10000000000000000000000000, 'string'], errors='coerce')
array([1.e+25,    nan])

pd.to_numeric(['-100000000000000000000', 'string'], errors='coerce')
array([nan, nan])

pd.to_numeric([-100000000000000000000, 'string'], errors='coerce')
array([-100000000000000000000, 'string'], dtype=object)
```

@gfyoung the problem is converting one specific entry or having to convert everything to float because of nan?
> having to convert everything to float because of nan?

@fakio : If one element is `nan`, all elements must become float.  That's just how the internals of the arrays work from `numpy`.  Thus, that's why you get the lossy nature as I alluded to [above](https://github.com/pandas-dev/pandas/issues/24910#issuecomment-457339811).  I was asking you whether you would be okay with that as an end user.

Another limitation from the `numpy` world is that there are numeric limits to the numbers you can pass in.  Those limits are `np.iinfo(np.int64).min` to `np.iinfo(np.uint64).max`.  This would address why numbers like `+/- 100000000000000000000` can cause issues.

That being said, your examples do point to some issues:

* We should be documenting that we can accept scalars for `pd.to_numeric`, which is not in the docs but is accepted in the implementation.
* The consistency of behavior between `100000000000000000000` and `[100000000000000000000]`.  That looks buggy to me...
> I was asking you whether you would be okay with that as an end user.

It's tough, I don't usually deal with big numbers, but I think converting to float is ok, and it is consistent with pd.to_numeric([100000000000000000000]) already doing it.

> The consistency of behavior between 100000000000000000000 and [100000000000000000000]. That looks buggy to me...

and '10000000000000000000000000', they all give three different results

Also strange:
```python
pd.to_numeric([10000000000000000000000000, '10000000000000000000000000'])
array([1.e+25, 1.e+25])

pd.to_numeric(['10000000000000000000000000'])
ValueError: Integer out of range. at position 0
`````
This is not an API question in fact.  It's a bug because we actually decided that we should force to float, but the test that we have for this behavior only applied to large positive numbers.  For some reason, it isn't happening for large negatives...

Resolving patch:
diff --git a/pandas/core/tools/numeric.py b/pandas/core/tools/numeric.py
--- a/pandas/core/tools/numeric.py
+++ b/pandas/core/tools/numeric.py
@@ -19,6 +19,14 @@ def to_numeric(arg, errors='raise', downcast=None):
     depending on the data supplied. Use the `downcast` parameter
     to obtain other dtypes.
 
+    Please note that precision loss may occur if really large numbers
+    are passed in. Due to the internal limitations of `ndarray`, if
+    numbers smaller than `-9223372036854775808` (np.iinfo(np.int64).min)
+    or larger than `18446744073709551615` (np.iinfo(np.uint64).max) are
+    passed in, it is very likely they will be converted to float so that
+    they can stored in an `ndarray`. These warnings apply similarly to
+    `Series` since it internally leverages `ndarray`.
+
     Parameters
     ----------
     arg : scalar, list, tuple, 1-d array, or Series


Test patch:
diff --git a/pandas/tests/tools/test_numeric.py b/pandas/tests/tools/test_numeric.py
--- a/pandas/tests/tools/test_numeric.py
+++ b/pandas/tests/tools/test_numeric.py
@@ -4,11 +4,50 @@
 from numpy import iinfo
 import pytest
 
+import pandas.compat as compat
+
 import pandas as pd
 from pandas import DataFrame, Index, Series, to_numeric
 from pandas.util import testing as tm
 
 
+@pytest.fixture(params=[None, "ignore", "raise", "coerce"])
+def errors(request):
+    return request.param
+
+
+@pytest.fixture(params=[True, False])
+def signed(request):
+    return request.param
+
+
+@pytest.fixture(params=[lambda x: x, str], ids=["identity", "str"])
+def transform(request):
+    return request.param
+
+
+@pytest.fixture(params=[
+    47393996303418497800,
+    100000000000000000000
+])
+def large_val(request):
+    return request.param
+
+
+@pytest.fixture(params=[True, False])
+def multiple_elts(request):
+    return request.param
+
+
+@pytest.fixture(params=[
+    (lambda x: Index(x, name="idx"), tm.assert_index_equal),
+    (lambda x: Series(x, name="ser"), tm.assert_series_equal),
+    (lambda x: np.array(Index(x).values), tm.assert_numpy_array_equal)
+])
+def transform_assert_equal(request):
+    return request.param
+
+
 @pytest.mark.parametrize("input_kwargs,result_kwargs", [
     (dict(), dict(dtype=np.int64)),
     (dict(errors="coerce", downcast="integer"), dict(dtype=np.int8))
@@ -172,7 +211,6 @@ def test_all_nan():
     tm.assert_series_equal(result, expected)
 
 
-@pytest.mark.parametrize("errors", [None, "ignore", "raise", "coerce"])
 def test_type_check(errors):
     # see gh-11776
     df = DataFrame({"a": [1, -3.14, 7], "b": ["4", "5", "6"]})
@@ -183,11 +221,100 @@ def test_type_check(errors):
         to_numeric(df, **kwargs)
 
 
-@pytest.mark.parametrize("val", [
-    1, 1.1, "1", "1.1", -1.5, "-1.5"
-])
-def test_scalar(val):
-    assert to_numeric(val) == float(val)
+@pytest.mark.parametrize("val", [1, 1.1, 20001])
+def test_scalar(val, signed, transform):
+    val = -val if signed else val
+    assert to_numeric(transform(val)) == float(val)
+
+
+def test_really_large_scalar(large_val, signed, transform, errors):
+    # see gh-24910
+    kwargs = dict(errors=errors) if errors is not None else dict()
+    val = -large_val if signed else large_val
+
+    val = transform(val)
+    val_is_string = isinstance(val, str)
+
+    if val_is_string and errors in (None, "raise"):
+        msg = "Integer out of range. at position 0"
+        with pytest.raises(ValueError, match=msg):
+            to_numeric(val, **kwargs)
+    else:
+        expected = float(val) if (errors == "coerce" and
+                                  val_is_string) else val
+        assert tm.assert_almost_equal(to_numeric(val, **kwargs), expected)
+
+
+def test_really_large_in_arr(large_val, signed, transform,
+                             multiple_elts, errors):
+    # see gh-24910
+    kwargs = dict(errors=errors) if errors is not None else dict()
+    val = -large_val if signed else large_val
+    val = transform(val)
+
+    extra_elt = "string"
+    arr = [val] + multiple_elts * [extra_elt]
+
+    val_is_string = isinstance(val, str)
+    coercing = errors == "coerce"
+
+    if errors in (None, "raise") and (val_is_string or multiple_elts):
+        if val_is_string:
+            msg = "Integer out of range. at position 0"
+        else:
+            msg = 'Unable to parse string "string" at position 1'
+
+        with pytest.raises(ValueError, match=msg):
+            to_numeric(arr, **kwargs)
+    else:
+        result = to_numeric(arr, **kwargs)
+
+        exp_val = float(val) if (coercing and val_is_string) else val
+        expected = [exp_val]
+
+        if multiple_elts:
+            if coercing:
+                expected.append(np.nan)
+                exp_dtype = float
+            else:
+                expected.append(extra_elt)
+                exp_dtype = object
+        else:
+            exp_dtype = float if isinstance(exp_val, (
+                int, compat.long, float)) else object
+
+        tm.assert_almost_equal(result, np.array(expected, dtype=exp_dtype))
+
+
+def test_really_large_in_arr_consistent(large_val, signed,
+                                        multiple_elts, errors):
+    # see gh-24910
+    #
+    # Even if we discover that we have to hold float, does not mean
+    # we should be lenient on subsequent elements that fail to be integer.
+    kwargs = dict(errors=errors) if errors is not None else dict()
+    arr = [str(-large_val if signed else large_val)]
+
+    if multiple_elts:
+        arr.insert(0, large_val)
+
+    if errors in (None, "raise"):
+        index = int(multiple_elts)
+        msg = "Integer out of range. at position {index}".format(index=index)
+
+        with pytest.raises(ValueError, match=msg):
+            to_numeric(arr, **kwargs)
+    else:
+        result = to_numeric(arr, **kwargs)
+
+        if errors == "coerce":
+            expected = [float(i) for i in arr]
+            exp_dtype = float
+        else:
+            expected = arr
+            exp_dtype = object
+
+        tm.assert_almost_equal(result, np.array(expected, dtype=exp_dtype))
 
 
 @pytest.mark.parametrize("errors,checker", [
@@ -205,15 +332,6 @@ def test_scalar_fail(errors, checker):
         assert checker(to_numeric(scalar, errors=errors))
 
 
-@pytest.fixture(params=[
-    (lambda x: Index(x, name="idx"), tm.assert_index_equal),
-    (lambda x: Series(x, name="ser"), tm.assert_series_equal),
-    (lambda x: np.array(Index(x).values), tm.assert_numpy_array_equal)
-])
-def transform_assert_equal(request):
-    return request.param
-
-
 @pytest.mark.parametrize("data", [
     [1, 2, 3],
     [1., np.nan, 3, np.nan]