opened
MahendraCherukupalli
username_0
<!-- Please describe the issue in detail here, and fill in the fields below -->

### Reproducing code example:

<!-- A short code example that reproduces the problem/missing feature. It should be
self-contained, i.e., possible to run as-is via 'python myproblem.py' -->

```python
import numpy as np
<< your code here >>
import numpy as np
import pandas as pd
df=pd.read_csv('link')
df.info() and df.describe() gives error as "TypeError: Cannot interpret '<attribute 'dtype' of 'numpy.generic' objects>' as a data type" and also plotting(df.plot()) gives same error.
```

### Error message:

<!-- If you are reporting a segfault please include a GDB traceback, which you
can generate by following
https://github.com/numpy/numpy/blob/master/doc/source/dev/development_environment.rst#debugging -->

<!-- Full error message, if any (starting from line Traceback: ...) -->

### NumPy/Python version information:

<!-- Output from 'import sys, numpy; print(numpy.__version__, sys.version)' -->
TypeError: Cannot interpret '<attribute 'dtype' of 'numpy.generic' objects>' as a data type
issue
created
seberg
username_1
There is a unfortuate incompatibility with old pandas and 1.20. Updating pandas to a newer version should fix it, see also https://github.com/pandas-dev/pandas/issues/39520#issuecomment-772630011

Updating to `pandas>=1.0.5` should solve it. Supposedly also `pandas==0.25.3` works. If you are stuck with pandas 0.24.x you may not be able to usse numpy 1.20.x though, unfortunaly.
comment
created
jessdwitch
username_2
@username_1 Any idea why someone might be suddenly seeing this error when calling `df.to_csv` with `pandas==0.24.2` and `numpy==1.16.5`? Based on what you're saying above, it seems like those should be in the "OK" range.
comment
created
seberg
username_1
@username_2 I am not sure if there are other (likely) ways to get this error, and I doubt it is possible with the above example (or a similar one, such as creating an empty dataframe IIRC).

Can you double check `np.__version__` and `pd.__version__` in your shell to make sure you are running the expected versions?  Sometimes you might get multiple versions or so, which can quickly become very confusing.
comment
created
jessdwitch
username_2
Thank you for the quick reply! I did check those already, since there _are_ multiple versions installed (`numpy==1.20.0` and `pandas==0.25.3` when conda is deactivated, and the versions noted above when the conda environment the script is running in is activated). I double checked the logs, and while I don't have the specific versions printing to the log, I do see the env activating and confirmed `0.24.2` and `1.16.5` were the ones installed to that env.
comment
created
seberg
username_1
Hmm, can you post an example and the full traceback?  I am wondering if I mixed up the issues and this one was not related to 1.20.x at all.

To be honest, if you are not running 1.20.x, then searching/asking on pandas is more likely to be successfull.
comment
created
jessdwitch
username_2
Gotcha. Tbh asking around pandas was my first thought, but this is literally the only google result with this error (well, this and an SO thread Mahendra created). Here's the traceback starting with the `to_csv` call. If that doesn't spark anything, I'll leave you be and pop in over there, but thank you regardless for taking a look.

```
  File "/root/miniconda/lib/python3.7/site-packages/annotation/summarize_annotation.py", line 159, in create_cluster_table
    pd.DataFrame(cluster_dict).to_csv(os.path.join(directory, 'report.csv'), index=False)
  File "/root/miniconda/lib/python3.7/site-packages/pandas/core/frame.py", line 411, in __init__
    mgr = init_dict(data, index, columns, dtype=dtype)
  File "/root/miniconda/lib/python3.7/site-packages/pandas/core/internals/construction.py", line 257, in init_dict
    return arrays_to_mgr(arrays, data_names, index, columns, dtype=dtype)
  File "/root/miniconda/lib/python3.7/site-packages/pandas/core/internals/construction.py", line 82, in arrays_to_mgr
    arrays = _homogenize(arrays, index, dtype)
  File "/root/miniconda/lib/python3.7/site-packages/pandas/core/internals/construction.py", line 323, in _homogenize
    val, index, dtype=dtype, copy=False, raise_cast_failure=False
  File "/root/miniconda/lib/python3.7/site-packages/pandas/core/internals/construction.py", line 712, in sanitize_array
    subarr = construct_1d_arraylike_from_scalar(value, len(index), dtype)
  File "/root/miniconda/lib/python3.7/site-packages/pandas/core/dtypes/cast.py", line 1233, in construct_1d_arraylike_from_scalar
    subarr = np.empty(length, dtype=dtype)
TypeError: Cannot interpret '<attribute 'dtype' of 'numpy.generic' objects>' as a data type
```
comment
created
seberg
username_1
Sorry, do you have a minimal reproducer, preferably printing out the versions?  It does look like this issue, but in that case the pandas code in `contruct_1d_arraylike_from_scalar` would include `isinstance(dtype, (np.dtype, type(np.dtype)))` or so (which is incorrect, because `type(np.dtype)` changed and was always weird).

Long story short, it looks a lot like this issue but I if you are really not on NumPy 1.20 `type(np.dtype) is type` and I am not aware of what might be wrong.
comment
created
jessdwitch
username_2
Hey sorry for disappearing, but you were totally right. For some reason the Conda environment was using the pandas from within the env, but the numpy from outside, causing the conflict. We ended up just downgrading the numpy from outside the env to match the one within the env and everything is happy now. Thank you so much!
comment
created
mattip
username_3
Closing. Please reopen if needed.
comment
closed
mattip
username_3
issue
created
aarthim123
username_4
Hi I tried updating pandas to 1.0.5 and I still get the same error message. TypeError: Cannot interpret '<attribute 'dtype' of 'numpy.generic' objects>' as a data type . I have a numpy version of 1.20
comment
created
mattip
username_3
@username_4 this issue is specifically about the error in the title. Please open an error on an appropriate issue tracker for problems with installing pandas on conda please use a more appropriate forum. You might have better luck searching for similar error messages. If you do open an issue, you probably should report the output of `conda list` in order to untangle what you have installed and what you might need.
comment
created
MahendraCherukupalli
username_0
@username_4 use ..pip install numpy==1.16.5
comment
created
RichardScottOZ
username_5
I just saw this:-

<class 'numpy.ndarray'> <class 'numpy.ndarray'>
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-15-14380fd16e5f> in <module>
     49         print(type(v_list[i]), type(face_list[i]) )
     50         #dataset = pd.DataFrame({'Group':'Neoprot-Ordovician','Surface': s, 'X': v_list[i][:, 0], 'Y': v_list[i][:, 1], 'Z': v_list[i][:, 2], 'SG': -1   })
---> 51         dataset = pd.DataFrame({'Group':'Neoprot-Ordovician','Surface': s, 'X': v_list[i][:, 0].astype(float)  })
     52         dataset["Name"] = metadata_list[i]["NAME"]
     53         dataset["CRS"] = str(metadata_list[i]["CRS"])

~\AppData\Local\Continuum\anaconda3\envs\gemgis\lib\site-packages\pandas\core\frame.py in __init__(self, data, index, columns, dtype, copy)
    433             )
    434         elif isinstance(data, dict):
--> 435             mgr = init_dict(data, index, columns, dtype=dtype)
    436         elif isinstance(data, ma.MaskedArray):
    437             import numpy.ma.mrecords as mrecords

~\AppData\Local\Continuum\anaconda3\envs\gemgis\lib\site-packages\pandas\core\internals\construction.py in init_dict(data, index, columns, dtype)
    252             arr if not is_datetime64tz_dtype(arr) else arr.copy() for arr in arrays
    253         ]
--> 254     return arrays_to_mgr(arrays, data_names, index, columns, dtype=dtype)
    255 
    256 

~\AppData\Local\Continuum\anaconda3\envs\gemgis\lib\site-packages\pandas\core\internals\construction.py in arrays_to_mgr(arrays, arr_names, index, columns, dtype)
     67 
     68     # don't force copy because getting jammed in an ndarray anyway
---> 69     arrays = _homogenize(arrays, index, dtype)
     70 
     71     # from BlockManager perspective

~\AppData\Local\Continuum\anaconda3\envs\gemgis\lib\site-packages\pandas\core\internals\construction.py in _homogenize(data, index, dtype)
    320                     val = dict(val)
    321                 val = lib.fast_multiget(val, oindex.values, default=np.nan)
--> 322             val = sanitize_array(
    323                 val, index, dtype=dtype, copy=False, raise_cast_failure=False
    324             )

~\AppData\Local\Continuum\anaconda3\envs\gemgis\lib\site-packages\pandas\core\construction.py in sanitize_array(data, index, dtype, copy, raise_cast_failure)
    463                 value = maybe_cast_to_datetime(value, dtype)
    464 
--> 465             subarr = construct_1d_arraylike_from_scalar(value, len(index), dtype)
    466 
    467         else:

~\AppData\Local\Continuum\anaconda3\envs\gemgis\lib\site-packages\pandas\core\dtypes\cast.py in construct_1d_arraylike_from_scalar(value, length, dtype)
   1459                 value = ensure_str(value)
   1460 
-> 1461         subarr = np.empty(length, dtype=dtype)
   1462         subarr.fill(value)
   1463 

TypeError: Cannot interpret '<attribute 'dtype' of 'numpy.generic' objects>' as a data type
comment
created
RichardScottOZ
username_5
pandas                    1.0.1            py38he350917_0    conda-forge
numpy                     1.20.2           py38h09042cb_0    conda-forge
comment
created
seberg
username_1
Please just update your pandas. If you want to stay on `1.0.x` that is fine.  `1.0.5` is just a bug-fix release that was reported to also include a fix for this.  If 1.0.5 is not sufficient to fix it, let us know so others can find a solution easier.
The pandas version you list are clearly mentioned as _not_ compatible in the pandas issue.

If you don't want to upgrade pandas for whatever reason, you may have to stick to NumPy 1.19.x as well.
comment
created
RichardScottOZ
username_5
For reference if someone else comes across it - it happened due to installing something else downgrading the pandas version.
comment
created
MigueL71994
username_6
pip install --upgrade numpy
pip install --upgrade pandas
comment
created
dinhtrang24
username_7
Hi, I've already upgrade both pandas (0.24.2) and numpy (1.21.5). When I tried data.info(), it still doesn't work. Any thoughts?

`---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-12-6208d269f320> in <module>
----> 1 data.info()

/opt/apps/apps/binapps/anaconda3/2019.07/lib/python3.7/site-packages/pandas/core/frame.py in info(self, verbose, buf, max_cols, memory_usage, null_counts)
   2503                         self.index._is_memory_usage_qualified()):
   2504                     size_qualifier = '+'
-> 2505             mem_usage = self.memory_usage(index=True, deep=deep).sum()
   2506             lines.append("memory usage: {mem}\n".format(
   2507                 mem=_sizeof_fmt(mem_usage, size_qualifier)))

/opt/apps/apps/binapps/anaconda3/2019.07/lib/python3.7/site-packages/pandas/core/frame.py in memory_usage(self, index, deep)
   2597         if index:
   2598             result = Series(self.index.memory_usage(deep=deep),
-> 2599                             index=['Index']).append(result)
   2600         return result
   2601 

/opt/apps/apps/binapps/anaconda3/2019.07/lib/python3.7/site-packages/pandas/core/series.py in __init__(self, data, index, dtype, name, copy, fastpath)
    260             else:
    261                 data = sanitize_array(data, index, dtype, copy,
--> 262                                       raise_cast_failure=True)
    263 
    264                 data = SingleBlockManager(data, index, fastpath=True)

/opt/apps/apps/binapps/anaconda3/2019.07/lib/python3.7/site-packages/pandas/core/internals/construction.py in sanitize_array(data, index, dtype, copy, raise_cast_failure)
    640 
    641             subarr = construct_1d_arraylike_from_scalar(
--> 642                 value, len(index), dtype)
    643 
    644         else:

/opt/apps/apps/binapps/anaconda3/2019.07/lib/python3.7/site-packages/pandas/core/dtypes/cast.py in construct_1d_arraylike_from_scalar(value, length, dtype)
   1185                 value = to_str(value)
   1186 
-> 1187         subarr = np.empty(length, dtype=dtype)
   1188         subarr.fill(value)
   1189 

TypeError: Cannot interpret '<attribute 'dtype' of 'numpy.generic' objects>' as a data type
`
comment
created
jtlz2
username_8
Similarly - this is still an issue for me
comment
created
seberg
username_1
The last time I saw someone who updated their pandas version and then still had the problem, had an environment issue that made them pick up the wrong pandas version after all.  Please double check `pandas.__version__` and `numpy.__version__` in whatever you are running (e.g. the script) itself?
comment
created
levalencia
username_9
I am having the same issue.
On a new compute instance in Azure ML, I am using Python 3.8 Kernel.

I checked the versions:
pandas 0.25.3
numpy 1.22.2


`.describe `gives the same error;
`TypeError: Cannot interpret '<attribute 'dtype' of 'numpy.generic' objects>' as a data type
`
I also tried:
`scaled_df.select_dtypes(include=[np.number])`

same error:
`TypeError: Cannot interpret '<attribute 'dtype' of 'numpy.generic' objects>' as a data type`


What should I do?
comment
created
dinhtrang24
username_7
Yes. I have checked both script and terminal again. 

pandas == 0.24.2 
numpy == 1.21.5

The error doesn't not disappear
comment
created
seberg
username_1
@username_7 that pandas version is known to be too old.  I had asked Luis, because I though 0.24.3 may be new enough (I am not quite sure).  You have to either upgrade pandas, since it is an old version, or if you are stuck with such an old pandas version downgrade NumPy.
(Or I suppose apply a patch to pandas, but unless you have very concrete reasons for using that pandas version, you should upgrade.)
comment
created
dinhtrang24
username_7
@username_1 Hi, thanks for the reply. I figured out the problem is that I have installed different versions of pandas and numpy using pip and pip3. After upgrading and matching their versions, the problem was solved.
comment