BUG: using dtype='int64' argument of Series causes ValueError: values cannot be losslessly cast to int64 for integer strings
State: closed

- [x] closes #44923
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry

Added extra check before returning ValueError.

Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
As dtype will be an integer dtype there is no need to explicitly check it we can check all integer dtypes.
@jbrockmendel @jreback any updates?
@jreback @jbrockmendel any update on this?
> @jreback @jbrockmendel any update on this?

@shubham11941140 there is not need to ping. you need to answer @jbrockmendel  last question. I guess you tests is not specific enough (if it already passed)
Actually I have added the discussion point to another test case, which might indicate this. I was asking to add another testcase which gives a lossless conversion.
> Actually I have added the discussion point to another test case, which might indicate this. I was asking to add another testcase which gives a lossless conversion.

great
Do I add it?
The last testcase should be correct as the conversion is lossless including this one.
> Do I add it? The last testcase should be correct as the conversion is lossless including this one.

does that test fail *before* your change? that's the key, does this replicate the original issue.
It was currently failing the test.
So I will add it

@jbrockmendel any update on this? I think I have covered everything.
@jreback any update?
> @jreback any update?

@shubham11941140 you are touching some very particular code, i don't know how to proceed here. this PR keeps expanding in scope & code changes w/o test cases.
I will explain it here in detail. It is the same test case in the ```test_constructors.py``` file

```result = Series(["0", "1", "2"], dtype=any_int_dtype)```
```exp = Series([0, 1, 2], dtype=any_int_dtype)```

```any_int_dtype``` covers ```uint8``` and every other unsigned integer dtype

So, taking ```uint8```:

```result = Series(["0", "1", "2"], dtype=uint8)```
```exp = Series([0, 1, 2], dtype=uint8)```

I have written these 2 cases and they must be equal i.e. ```tm.assert_frame_equal``` must hold.

The issue is that when dtype=uint8 or for the matter of fact any dtype where ```is_unsigned_integer_dtype(dtype)``` holds, we reach the condition of ```(arr < 0).any()```.

If you look very carefully, ```arr = ["0", "1", "2"]```, our ```np.ndarray```, that contains strings. The function ```(arr < 0).any()``` is a numpy function that DOES NOT ALLOW the comparison of strings to integers (As "0" is a ```str``` and 0 is ```int```) and leads to a ```TypeError```.

However, there is no error in this case as strings can be implicitly cast to int by this function.

This is done within the casted variable which is ```casted = [0, 1, 2]``` which is then casted into ```uint``` dtype. If we apply the function ```(casted < 0).any()```, there should not be any error for the above specified testcase.

For this testcase as specified, I have created a try except block that prevents other tests from breaking and allows type casting of strings to unsigned integers without running into a ```TypeError``` of the NumPy function.

In this manner, I am not running into an issue with the ```unsigned_integer_dtype``` and can perform the implicit cast.

I hope this can now explain the code change I have done associated with the test case. If you are unable to understand anything further. I can schedule a call to explain it. 

@jreback 



@jreback do I need to give a more elaborate explanation?
will take another look today.
@jbrockmendel @jreback any update?
> any update?

The check on L2108 still needs to be changed to something meaningful.  The check is about the casting not being lossy.
How do I check whether it is a lossless change? Is there any construct?
> How do I check whether it is a lossless change? Is there any construct?

you need to construct a test explicitly for this
@jreback @jbrockmendel . I have removed the obvious statement used for the return condition and added to the above condition.

This is obviously checking the lossless condition and it is combined with the Overflow condition as previously written as if the user gives very large strings to be casted into small integers such as ```int8```, we will not run into a similar issue again.

I think this will satisfy the requisite conditions now.
```lib.infer_dtype(casted) < lib.infer_dtype(arr)``` it is the same like the extension of ```casted.dtype < arr.dtype``` to inferred dtypes as they are not properly initialized in the input. However they check the same as the condition of the lossless casting as the warning message means.
> lib.infer_dtype(casted) < lib.infer_dtype(arr) it is the same like the extension of casted.dtype < arr.dtype to inferred dtypes as they are not properly initialized in the input. However they check the same as the condition of the lossless casting as the warning message means.

lib.infer_dtype returns a string.  Why is comparing two strings useful here?
if the dtype is a string it is a situation of lossless cast, so that can be specified directly.
@shubham11941140 if you want to merge master and update to comments
Branch is updated.
I did not understand what do you want me to do?
On running the query, it mentions a FutureWarning when comparing ```return bool(asarray(a1 == a2).all())```.

This kind of error is not coming from the changes I have made.
The issue is that string comparison with integer is going past the limit of the NumPy comparison because of which it is failing.
![image](https://user-images.githubusercontent.com/63910248/157061096-ff83adba-7163-40da-8d31-f7d0f480a0d7.png)

The strange part is that this should be raising a Warning with the ```np.array_equal``` comparison.

Very strange is that numpy dev is raising an error which is not happening on my machine.
I have tested the build on my local machine and it is not failing, but 2 errors come here.