BUG: str dtype ignored for column with dot
State: closed

- [x] closes #50270
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

This is a direct attempt to fix the problem with 'dot' in strings. Maybe it would be better to use for this purpose attribute `_no_thousands_columns`.

Labels:
name: IO CSV
description: read_csv, to_csv

Comments:
Could you, please, @MarcoGorelli, review my PR?
Hey - I'm not overly familiar with this part of the code, but I'll take a look

could you start by adding a test please?
> Hey - I'm not overly familiar with this part of the code, but I'll take a look

Thank you.

> could you start by adding a test please?

Yes, of course, I’ll add the test.
Hi @natmokval , I think maybe update `_no_thousands_columns` is a better idea. 
could you check this commit?  https://github.com/pandas-dev/pandas/compare/main...yanxiaole:pandas:issue-50270
I like the idea of updating no_convert_columns. Could you check if this works?
> I like the idea of updating no_convert_columns. Could you check if this works?

I tried to use `_no_thousands_columns` to exclude non-numeric columns but got test failures.
Some of them are related to the problem with dtype:
```
Attribute "dtype" are different
E       [left]:  int32
E       [right]: int64
```
[pandas/tests/io/parser/common/test_common_basic.py](https://github.com/pandas-dev/pandas/actions/runs/3847586332/jobs/6554227543#step:3:2073)
Locally the same tests pass.

I run tests locally `pytest pandas -n 4 -m "not slow and not network and not db and not single_cpu" -r sxXq`:
`======= 189138 passed, 1418 skipped, 3340 xfailed, 14 warnings in 537.93s (0:08:57) =======`

Advice on how to deal with this is much appreciated.

> I think maybe update _no_thousands_columns is a better idea.
could you check this commit? https://github.com/pandas-dev/pandas/compare/main...yanxiaole:pandas:issue-50270

Thank you @yanxiaole, for your suggestion. I tried to use it as it is and got some test failures. Then I made slight corrections and still have some failures. Still trying to find out what are causes of these failures.
Thanks for working on this, not an easy one

Looks like this correctly parses column `a`, but now `b` and `c` have changed
```python
In [1]: data = """a;b;c\n0000.7995;16.000;0\n3.03.001.00514;0;4.000\n4923.600.041;23.000;131"""

In [2]: df1 = pd.read_csv(io.StringIO(data), sep=';', dtype={'a': str}, thousands='.', engine='c')
   ...: df2 = pd.read_csv(io.StringIO(data), sep=';', dtype={'a': str}, thousands='.', engine='python')

In [3]: df1
Out[3]: 
                a      b     c
0       0000.7995  16000     0
1  3.03.001.00514      0  4000
2    4923.600.041  23000   131

In [4]: df2
Out[4]: 
                a     b      c
0       0000.7995  16.0    0.0
1  3.03.001.00514   0.0    4.0
2    4923.600.041  23.0  131.0
```
@natmokval sorry I was on holiday last week.
the failed case seems due to 32 bit env, I'm not familiar with this part, @phofl could you take a look and give some advice?
While running tests locally I don't have failures like [these](https://github.com/pandas-dev/pandas/actions/runs/4096799204/jobs/7064705493#step:5:222). 
```
E       Attribute "dtype" are different
E       [left]:  int32
E       [right]: int64
```
I can't reproduce the problems locally to resolve them.
I am out of ideas on how to debug the failures. Some help is appreciated.

Thank you for the comments.
I agree, after adding parametrization to `test_no_thousand_with_dot_convert_for_non_numeric_cols `
the function `test__search_replace_num_columns` becomes duplicative. I deleted it.

@phofl, could you please take a look at this pr? I made the changes that you suggested.
Thanks @natmokval , well done on persevering with this one!