Python interpreter version (3.13) is newer than PyO3's maximum supported version (3.12)
Author: aniketnarvekar
Pull request: no
Labels: name: bug

### System Info

Hi, I was trying to install transformers using pip in python virtual environment created using venv. I'm getting following error:

`(.env) -> python3 -m pip install transformers
Collecting transformers
  Using cached transformers-4.45.2-py3-none-any.whl.metadata (44 kB)
Collecting filelock (from transformers)
  Using cached filelock-3.16.1-py3-none-any.whl.metadata (2.9 kB)
Collecting huggingface-hub<1.0,>=0.23.2 (from transformers)
  Using cached huggingface_hub-0.25.2-py3-none-any.whl.metadata (13 kB)
Collecting numpy>=1.17 (from transformers)
  Using cached numpy-2.1.2-cp313-cp313-macosx_14_0_x86_64.whl.metadata (60 kB)
Collecting packaging>=20.0 (from transformers)
  Using cached packaging-24.1-py3-none-any.whl.metadata (3.2 kB)
Collecting pyyaml>=5.1 (from transformers)
  Using cached PyYAML-6.0.2-cp313-cp313-macosx_10_13_x86_64.whl.metadata (2.1 kB)
Collecting regex!=2019.12.17 (from transformers)
  Using cached regex-2024.9.11-cp313-cp313-macosx_10_13_x86_64.whl.metadata (40 kB)
Collecting requests (from transformers)
  Using cached requests-2.32.3-py3-none-any.whl.metadata (4.6 kB)
Collecting safetensors>=0.4.1 (from transformers)
  Using cached safetensors-0.4.5-cp313-cp313-macosx_10_12_x86_64.whl.metadata (3.8 kB)
Collecting tokenizers<0.21,>=0.20 (from transformers)
  Using cached tokenizers-0.20.1.tar.gz (339 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting tqdm>=4.27 (from transformers)
  Using cached tqdm-4.66.5-py3-none-any.whl.metadata (57 kB)
Collecting fsspec>=2023.5.0 (from huggingface-hub<1.0,>=0.23.2->transformers)
  Using cached fsspec-2024.9.0-py3-none-any.whl.metadata (11 kB)
Collecting typing-extensions>=3.7.4.3 (from huggingface-hub<1.0,>=0.23.2->transformers)
  Using cached typing_extensions-4.12.2-py3-none-any.whl.metadata (3.0 kB)
Collecting charset-normalizer<4,>=2 (from requests->transformers)
  Using cached charset_normalizer-3.4.0-cp313-cp313-macosx_10_13_x86_64.whl.metadata (34 kB)
Collecting idna<4,>=2.5 (from requests->transformers)
  Using cached idna-3.10-py3-none-any.whl.metadata (10 kB)
Collecting urllib3<3,>=1.21.1 (from requests->transformers)
  Using cached urllib3-2.2.3-py3-none-any.whl.metadata (6.5 kB)
Collecting certifi>=2017.4.17 (from requests->transformers)
  Using cached certifi-2024.8.30-py3-none-any.whl.metadata (2.2 kB)
Using cached transformers-4.45.2-py3-none-any.whl (9.9 MB)
Using cached huggingface_hub-0.25.2-py3-none-any.whl (436 kB)
Using cached numpy-2.1.2-cp313-cp313-macosx_14_0_x86_64.whl (6.6 MB)
Using cached packaging-24.1-py3-none-any.whl (53 kB)
Using cached PyYAML-6.0.2-cp313-cp313-macosx_10_13_x86_64.whl (181 kB)
Using cached regex-2024.9.11-cp313-cp313-macosx_10_13_x86_64.whl (288 kB)
Using cached safetensors-0.4.5-cp313-cp313-macosx_10_12_x86_64.whl (392 kB)
Using cached tqdm-4.66.5-py3-none-any.whl (78 kB)
Using cached filelock-3.16.1-py3-none-any.whl (16 kB)
Using cached requests-2.32.3-py3-none-any.whl (64 kB)
Using cached certifi-2024.8.30-py3-none-any.whl (167 kB)
Using cached charset_normalizer-3.4.0-cp313-cp313-macosx_10_13_x86_64.whl (125 kB)
Using cached fsspec-2024.9.0-py3-none-any.whl (179 kB)
Using cached idna-3.10-py3-none-any.whl (70 kB)
Using cached typing_extensions-4.12.2-py3-none-any.whl (37 kB)
Using cached urllib3-2.2.3-py3-none-any.whl (126 kB)
Building wheels for collected packages: tokenizers
  Building wheel for tokenizers (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for tokenizers (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [71 lines of output]
      Running `maturin pep517 build-wheel -i /Users/prashantazad/Aniket/Language/Programming/Hosted/Python/CODER/.env/bin/python3 --compatibility off`
      🍹 Building a mixed python/rust project
      🔗 Found pyo3 bindings
      🐍 Found CPython 3.13 at /Users/prashantazad/Aniket/Language/Programming/Hosted/Python/CODER/.env/bin/python3
      📡 Using build options features, bindings from pyproject.toml
      💻 Using `MACOSX_DEPLOYMENT_TARGET=10.12` for x86_64-apple-darwin by default
         Compiling proc-macro2 v1.0.86
         Compiling unicode-ident v1.0.13
         Compiling autocfg v1.4.0
         Compiling libc v0.2.159
         Compiling target-lexicon v0.12.16
         Compiling once_cell v1.20.1
         Compiling cfg-if v1.0.0
         Compiling memchr v2.7.4
         Compiling shlex v1.3.0
         Compiling cc v1.1.22
         Compiling crossbeam-utils v0.8.20
         Compiling fnv v1.0.7
         Compiling ident_case v1.0.1
         Compiling strsim v0.11.1
         Compiling num-traits v0.2.19
         Compiling portable-atomic v1.9.0
         Compiling serde v1.0.210
         Compiling smallvec v1.13.2
         Compiling quote v1.0.37
         Compiling syn v2.0.79
         Compiling aho-corasick v1.1.3
         Compiling crossbeam-epoch v0.9.18
         Compiling pyo3-build-config v0.21.2
         Compiling lock_api v0.4.12
         Compiling rayon-core v1.12.1
         Compiling pkg-config v0.3.31
         Compiling either v1.13.0
         Compiling regex-syntax v0.8.5
         Compiling byteorder v1.5.0
         Compiling parking_lot_core v0.9.10
         Compiling onig_sys v69.8.1
         Compiling crossbeam-deque v0.8.5
         Compiling getrandom v0.2.15
         Compiling matrixmultiply v0.3.9
         Compiling memoffset v0.9.1
         Compiling scopeguard v1.2.0
         Compiling paste v1.0.15
         Compiling heck v0.4.1
         Compiling rand_core v0.6.4
         Compiling regex-automata v0.4.8
         Compiling darling_core v0.20.10
         Compiling pyo3-ffi v0.21.2
         Compiling pyo3 v0.21.2
         Compiling pyo3-macros-backend v0.21.2
      error: failed to run custom build command for `pyo3-ffi v0.21.2`

      Caused by:
        process didn't exit successfully: `/private/var/folders/09/m5p1h_bj10g2fknj_cxqg2sc0000gn/T/pip-install-7546wwlz/tokenizers_b61cc212a5ec417d9257adad0f7d1e1d/bindings/python/target/release/build/pyo3-ffi-fe49fca49ddc7885/build-script-build` (exit status: 1)
        --- stdout
        cargo:rerun-if-env-changed=PYO3_CROSS
        cargo:rerun-if-env-changed=PYO3_CROSS_LIB_DIR
        cargo:rerun-if-env-changed=PYO3_CROSS_PYTHON_VERSION
        cargo:rerun-if-env-changed=PYO3_CROSS_PYTHON_IMPLEMENTATION
        cargo:rerun-if-env-changed=PYO3_PRINT_CONFIG
        cargo:rerun-if-env-changed=PYO3_USE_ABI3_FORWARD_COMPATIBILITY

        --- stderr
        error: the configured Python interpreter version (3.13) is newer than PyO3's maximum supported version (3.12)
        = help: please check if an updated version of PyO3 is available. Current version: 0.21.2
        = help: set PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 to suppress this check and build anyway using the stable ABI
      warning: build failed, waiting for other jobs to finish...
      💥 maturin failed
        Caused by: Failed to build a native library through cargo
        Caused by: Cargo build finished with "exit status: 101": `env -u CARGO MACOSX_DEPLOYMENT_TARGET="10.12" PYO3_ENVIRONMENT_SIGNATURE="cpython-3.13-64bit" PYO3_PYTHON="/Users/prashantazad/Aniket/Language/Programming/Hosted/Python/CODER/.env/bin/python3" PYTHON_SYS_EXECUTABLE="/Users/prashantazad/Aniket/Language/Programming/Hosted/Python/CODER/.env/bin/python3" "cargo" "rustc" "--features" "pyo3/extension-module" "--message-format" "json-render-diagnostics" "--manifest-path" "/private/var/folders/09/m5p1h_bj10g2fknj_cxqg2sc0000gn/T/pip-install-7546wwlz/tokenizers_b61cc212a5ec417d9257adad0f7d1e1d/bindings/python/Cargo.toml" "--release" "--lib" "--" "-C" "link-arg=-undefined" "-C" "link-arg=dynamic_lookup" "-C" "link-args=-Wl,-install_name,@rpath/tokenizers.tokenizers.cpython-313-darwin.so"`
      Error: command ['maturin', 'pep517', 'build-wheel', '-i', '/Users/prashantazad/Aniket/Language/Programming/Hosted/Python/CODER/.env/bin/python3', '--compatibility', 'off'] returned non-zero exit status 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for tokenizers
Failed to build tokenizers
ERROR: ERROR: Failed to build installable wheels for some `pyproject.toml` based projects (tokenizers)`

I'm using 
MacOS 14.6.1, Python 3.13.

 I know it's version issue as stated in line `error: the configured Python interpreter version (3.13) is newer than PyO3's maximum supported version (3.12)`

But I'm not sure where is PyO3 package declaration is? When an error happens, the Activity Indicator shows `rustc` using high CPU and `pyproject.toml` file used by rust. So its rust package create error, but where is the root file that PYO3 version is spcified?

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Install Python 3.13
2. Create Python virtual Environment
3. `python3 -m pip install transformers`

### Expected behavior

The transformers should be installed successfully.

Comments:
cc @ArthurZucker I believe we're working on that in tokenizers?

Yes we are! https://github.com/huggingface/tokenizers/pull/1646 ! 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.