Cannot import datasets - ValueError: pyarrow.lib.IpcWriteOptions size changed, may indicate binary incompatibility

### Describe the bug

When trying to import datasets, I get a pyarrow ValueError:

Traceback (most recent call last):
  File "/Users/edward/test/test.py", line 1, in <module>
    import datasets
  File "/Users/edward/opt/anaconda3/envs/cs235/lib/python3.9/site-packages/datasets/__init__.py", line 43, in <module>
    from .arrow_dataset import Dataset
  File "/Users/edward/opt/anaconda3/envs/cs235/lib/python3.9/site-packages/datasets/arrow_dataset.py", line 65, in <module>
    from .arrow_reader import ArrowReader
  File "/Users/edward/opt/anaconda3/envs/cs235/lib/python3.9/site-packages/datasets/arrow_reader.py", line 28, in <module>
    import pyarrow.parquet as pq
  File "/Users/edward/opt/anaconda3/envs/cs235/lib/python3.9/site-packages/pyarrow/parquet/__init__.py", line 20, in <module>
    from .core import *
  File "/Users/edward/opt/anaconda3/envs/cs235/lib/python3.9/site-packages/pyarrow/parquet/core.py", line 45, in <module>
    from pyarrow.fs import (LocalFileSystem, FileSystem, FileType,
  File "/Users/edward/opt/anaconda3/envs/cs235/lib/python3.9/site-packages/pyarrow/fs.py", line 49, in <module>
    from pyarrow._gcsfs import GcsFileSystem  # noqa
  File "pyarrow/_gcsfs.pyx", line 1, in init pyarrow._gcsfs
ValueError: pyarrow.lib.IpcWriteOptions size changed, may indicate binary incompatibility. Expected 88 from C header, got 72 from PyObject

### Steps to reproduce the bug

`import datasets`

### Expected behavior

Successful import

### Environment info

Conda environment, MacOS
python 3.9.12
datasets 2.12.0



Based on https://github.com/rapidsai/cudf/issues/10187, this probably means your `pyarrow` installation is not compatible with `datasets`.

Can you please execute the following commands in the terminal and paste the output here?
```
conda list | grep arrow
``` 
```
python -c "import pyarrow; print(pyarrow.__file__)"
```




> Based on [rapidsai/cudf#10187](https://github.com/rapidsai/cudf/issues/10187), this probably means your `pyarrow` installation is not compatible with `datasets`.
> 
> Can you please execute the following commands in the terminal and paste the output here?
> 
> ```
> conda list | grep arrow
> ```
> 
> ```
> python -c "import pyarrow; print(pyarrow.__file__)"
> ```


Here is the output to the first command:
```
arrow-cpp                 11.0.0           py39h7f74497_0  
pyarrow                   12.0.0                   pypi_0    pypi
```
and the second:
```
/Users/edward/opt/anaconda3/envs/cs235/lib/python3.9/site-packages/pyarrow/__init__.py
```
Thanks!





after installing pytesseract 0.3.10, I got the above error. FYI 

RuntimeError: Failed to import transformers.trainer because of the following error (look up to see its traceback):
pyarrow.lib.IpcWriteOptions size changed, may indicate binary incompatibility. Expected 88 from C header, got 72 from PyObject

I got the same error, pyarrow 12.0.0 released May/2023 (https://pypi.org/project/pyarrow/) is not compatible, running `pip install pyarrow==11.0.0` to force install the previous version solved the problem.

Do we need to update dependencies? 

Please note that our CI properly passes all tests with `pyarrow-12.0.0`, for Python 3.7 and Python 3.10, for Ubuntu and Windows: see for example https://github.com/huggingface/datasets/actions/runs/5157324334/jobs/9289582291

For conda with python3.8.16 this solved my problem! thanks!

> I got the same error, pyarrow 12.0.0 released May/2023 (https://pypi.org/project/pyarrow/) is not compatible, running `pip install pyarrow==11.0.0` to force install the previous version solved the problem.
> 
> Do we need to update dependencies? I can work on that if no one else is working on it.



Thanks for replying. I am not sure about those environments but it seems like pyarrow-12.0.0 does not work for conda with python 3.8.16. 

> Please note that our CI properly passes all tests with `pyarrow-12.0.0`, for Python 3.7 and Python 3.10, for Ubuntu and Windows: see for example https://github.com/huggingface/datasets/actions/runs/5157324334/jobs/9289582291



Got the same error with:

```
arrow-cpp                 11.0.0          py310h7516544_0  
pyarrow                   12.0.0                   pypi_0    pypi

python                    3.10.11              h7a1cb2a_2  

datasets                  2.13.0             pyhd8ed1ab_0    conda-forge
```

> I got the same error, pyarrow 12.0.0 released May/2023 (https://pypi.org/project/pyarrow/) is not compatible, running `pip install pyarrow==11.0.0` to force install the previous version solved the problem.
> 
> Do we need to update dependencies?

This solved the issue for me as well.

> I got the same error, pyarrow 12.0.0 released May/2023 (https://pypi.org/project/pyarrow/) is not compatible, running `pip install pyarrow==11.0.0` to force install the previous version solved the problem.
> 
> Do we need to update dependencies?

Solved it for me also

> 基于 [rapidsai/cudf#10187](https://github.com/rapidsai/cudf/issues/10187)，这可能意味着您的安装与 不兼容。`pyarrow``datasets`
> 
> 您能否在终端中执行以下命令并将输出粘贴到此处？
> 
> ```
> conda list | grep arrow
> ```
> 
> ```
> python -c "import pyarrow; print(pyarrow.__file__)"
> ```

arrow-cpp                 11.0.0          py310h7516544_0  
pyarrow                   12.0.1                   pypi_0    pypi

/root/miniconda3/lib/python3.10/site-packages/pyarrow/__init__.py

Got the same problem with

arrow-cpp                 11.0.0          py310h1fc3239_0  
pyarrow                   12.0.1                   pypi_0    pypi

miniforge3/envs/mlp/lib/python3.10/site-packages/pyarrow/__init__.py

Reverting back to pyarrow 11 solved the problem.


Solved with `pip install pyarrow==11.0.0`

I got different. Solved with
pip install pyarrow==12.0.1
pip install cchardet

env:
Python 3.9.16
transformers  4.32.1

> I got the same error, pyarrow 12.0.0 released May/2023 (https://pypi.org/project/pyarrow/) is not compatible, running `pip install pyarrow==11.0.0` to force install the previous version solved the problem.
> 
> Do we need to update dependencies?

This works for me as well

> I got different. Solved with pip install pyarrow==12.0.1 pip install cchardet
> 
> env: Python 3.9.16 transformers 4.32.1

I guess it also depends on the Python version. I got Python 3.11.5 and pyarrow==12.0.0. 
It works! 

Hi, if this helps anyone, pip install pyarrow==11.0.0 did not work for me (I'm using Colab) but this worked: 
!pip install --extra-index-url=https://pypi.nvidia.com cudf-cu11