pip install "transformers[sentencepiece]" doesn't work!
Author: vpapaioannou
Pull request: no
Labels: name: bug

### System Info

MacOS Sequoia 15.0.1, python 3.13.0

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. I created a python 13 virtual environment using conda on MacOS Sequoia 15.0.1.
2. I activated the environment w/ 'conda activate my_python_venv`.
2. Given the instruction here, https://huggingface.co/learn/nlp-course/chapter0/1, I ran the command `pip install "transformers[sentencepiece]"` and this is the error I get.
```
Building wheels for collected packages: sentencepiece
  Building wheel for sentencepiece (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [99 lines of output]
      /Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/_distutils/dist.py:261: UserWarning: Unknown distribution option: 'test_suite'
        warnings.warn(msg)
      running bdist_wheel
      running build
      running build_py
      creating build/lib.macosx-10.15-x86_64-cpython-313/sentencepiece
      copying src/sentencepiece/__init__.py -> build/lib.macosx-10.15-x86_64-cpython-313/sentencepiece
      copying src/sentencepiece/_version.py -> build/lib.macosx-10.15-x86_64-cpython-313/sentencepiece
      copying src/sentencepiece/sentencepiece_model_pb2.py -> build/lib.macosx-10.15-x86_64-cpython-313/sentencepiece
      copying src/sentencepiece/sentencepiece_pb2.py -> build/lib.macosx-10.15-x86_64-cpython-313/sentencepiece
      running build_ext
      Package sentencepiece was not found in the pkg-config search path.
      Perhaps you should add the directory containing `sentencepiece.pc'
      to the PKG_CONFIG_PATH environment variable
      No package 'sentencepiece' found
      ./build_bundled.sh: line 21: cmake: command not found
      ./build_bundled.sh: line 22: nproc: command not found
      ./build_bundled.sh: line 22: cmake: command not found
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
          exec(compile('''
          ~~~~^^^^^^^^^^^^
          # This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          ...<31 lines>...
          exec(compile(setup_py_code, filename, "exec"))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          ''' % ('/private/var/folders/pf/m0zg4n_x68b83v_2xzrpj1380000gn/T/pip-install-_7dg5qos/sentencepiece_2f9832006d114e8cb0a1723f23c3b92d/setup.py',), "<pip-setuptools-caller>", "exec"))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/private/var/folders/pf/m0zg4n_x68b83v_2xzrpj1380000gn/T/pip-install-_7dg5qos/sentencepiece_2f9832006d114e8cb0a1723f23c3b92d/setup.py", line 169, in <module>
          setup(
          ~~~~~^
              name='sentencepiece',
              ^^^^^^^^^^^^^^^^^^^^^
          ...<29 lines>...
              test_suite='sentencepiece_test.suite',
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          )
          ^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/__init__.py", line 117, in setup
          return distutils.core.setup(**attrs)
                 ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/_distutils/core.py", line 183, in setup
          return run_commands(dist)
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/_distutils/core.py", line 199, in run_commands
          dist.run_commands()
          ~~~~~~~~~~~~~~~~~^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/_distutils/dist.py", line 954, in run_commands
          self.run_command(cmd)
          ~~~~~~~~~~~~~~~~^^^^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/dist.py", line 950, in run_command
          super().run_command(command)
          ~~~~~~~~~~~~~~~~~~~^^^^^^^^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
          cmd_obj.run()
          ~~~~~~~~~~~^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/command/bdist_wheel.py", line 398, in run
          self.run_command("build")
          ~~~~~~~~~~~~~~~~^^^^^^^^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
          self.distribution.run_command(command)
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/dist.py", line 950, in run_command
          super().run_command(command)
          ~~~~~~~~~~~~~~~~~~~^^^^^^^^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
          cmd_obj.run()
          ~~~~~~~~~~~^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/_distutils/command/build.py", line 135, in run
          self.run_command(cmd_name)
          ~~~~~~~~~~~~~~~~^^^^^^^^^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
          self.distribution.run_command(command)
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/dist.py", line 950, in run_command
          super().run_command(command)
          ~~~~~~~~~~~~~~~~~~~^^^^^^^^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
          cmd_obj.run()
          ~~~~~~~~~~~^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/command/build_ext.py", line 98, in run
          _build_ext.run(self)
          ~~~~~~~~~~~~~~^^^^^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/_distutils/command/build_ext.py", line 359, in run
          self.build_extensions()
          ~~~~~~~~~~~~~~~~~~~~~^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/_distutils/command/build_ext.py", line 476, in build_extensions
          self._build_extensions_serial()
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/_distutils/command/build_ext.py", line 502, in _build_extensions_serial
          self.build_extension(ext)
          ~~~~~~~~~~~~~~~~~~~~^^^^^
        File "/private/var/folders/pf/m0zg4n_x68b83v_2xzrpj1380000gn/T/pip-install-_7dg5qos/sentencepiece_2f9832006d114e8cb0a1723f23c3b92d/setup.py", line 87, in build_extension
          subprocess.check_call(['./build_bundled.sh', __version__])
          ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/subprocess.py", line 419, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['./build_bundled.sh', '0.2.0']' returned non-zero exit status 127.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for sentencepiece
  Running setup.py clean for sentencepiece
Failed to build sentencepiece
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (sentencepiece)
```

### Expected behavior

To be installed w/o issues.

Comments:
@vpapaioannou does `pip install sentencepiece` work for you, or do you get the same error?

There is a problem again. I am using Bash shell, .sh and not .zsh or anything else. Closing and opening the terminal doesn't work either. Running `pip install "transformers[sentencepiece]"` fails as well.

```
Collecting sentencepiece
  Using cached sentencepiece-0.2.0.tar.gz (2.6 MB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: sentencepiece
  Building wheel for sentencepiece (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [99 lines of output]
      /Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/_distutils/dist.py:261: UserWarning: Unknown distribution option: 'test_suite'
        warnings.warn(msg)
      running bdist_wheel
      running build
      running build_py
      creating build/lib.macosx-10.15-x86_64-cpython-313/sentencepiece
      copying src/sentencepiece/__init__.py -> build/lib.macosx-10.15-x86_64-cpython-313/sentencepiece
      copying src/sentencepiece/_version.py -> build/lib.macosx-10.15-x86_64-cpython-313/sentencepiece
      copying src/sentencepiece/sentencepiece_model_pb2.py -> build/lib.macosx-10.15-x86_64-cpython-313/sentencepiece
      copying src/sentencepiece/sentencepiece_pb2.py -> build/lib.macosx-10.15-x86_64-cpython-313/sentencepiece
      running build_ext
      Package sentencepiece was not found in the pkg-config search path.
      Perhaps you should add the directory containing `sentencepiece.pc'
      to the PKG_CONFIG_PATH environment variable
      No package 'sentencepiece' found
      ./build_bundled.sh: line 21: cmake: command not found
      ./build_bundled.sh: line 22: nproc: command not found
      ./build_bundled.sh: line 22: cmake: command not found
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
          exec(compile('''
          ~~~~^^^^^^^^^^^^
          # This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          ...<31 lines>...
          exec(compile(setup_py_code, filename, "exec"))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          ''' % ('/private/var/folders/pf/m0zg4n_x68b83v_2xzrpj1380000gn/T/pip-install-f7qe6dcb/sentencepiece_498a4cd13f9d4e92a8cb07fc4c4cd888/setup.py',), "<pip-setuptools-caller>", "exec"))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/private/var/folders/pf/m0zg4n_x68b83v_2xzrpj1380000gn/T/pip-install-f7qe6dcb/sentencepiece_498a4cd13f9d4e92a8cb07fc4c4cd888/setup.py", line 169, in <module>
          setup(
          ~~~~~^
              name='sentencepiece',
              ^^^^^^^^^^^^^^^^^^^^^
          ...<29 lines>...
              test_suite='sentencepiece_test.suite',
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          )
          ^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/__init__.py", line 117, in setup
          return distutils.core.setup(**attrs)
                 ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/_distutils/core.py", line 183, in setup
          return run_commands(dist)
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/_distutils/core.py", line 199, in run_commands
          dist.run_commands()
          ~~~~~~~~~~~~~~~~~^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/_distutils/dist.py", line 954, in run_commands
          self.run_command(cmd)
          ~~~~~~~~~~~~~~~~^^^^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/dist.py", line 950, in run_command
          super().run_command(command)
          ~~~~~~~~~~~~~~~~~~~^^^^^^^^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
          cmd_obj.run()
          ~~~~~~~~~~~^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/command/bdist_wheel.py", line 398, in run
          self.run_command("build")
          ~~~~~~~~~~~~~~~~^^^^^^^^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
          self.distribution.run_command(command)
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/dist.py", line 950, in run_command
          super().run_command(command)
          ~~~~~~~~~~~~~~~~~~~^^^^^^^^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
          cmd_obj.run()
          ~~~~~~~~~~~^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/_distutils/command/build.py", line 135, in run
          self.run_command(cmd_name)
          ~~~~~~~~~~~~~~~~^^^^^^^^^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
          self.distribution.run_command(command)
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/dist.py", line 950, in run_command
          super().run_command(command)
          ~~~~~~~~~~~~~~~~~~~^^^^^^^^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
          cmd_obj.run()
          ~~~~~~~~~~~^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/command/build_ext.py", line 98, in run
          _build_ext.run(self)
          ~~~~~~~~~~~~~~^^^^^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/_distutils/command/build_ext.py", line 359, in run
          self.build_extensions()
          ~~~~~~~~~~~~~~~~~~~~~^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/_distutils/command/build_ext.py", line 476, in build_extensions
          self._build_extensions_serial()
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/site-packages/setuptools/_distutils/command/build_ext.py", line 502, in _build_extensions_serial
          self.build_extension(ext)
          ~~~~~~~~~~~~~~~~~~~~^^^^^
        File "/private/var/folders/pf/m0zg4n_x68b83v_2xzrpj1380000gn/T/pip-install-f7qe6dcb/sentencepiece_498a4cd13f9d4e92a8cb07fc4c4cd888/setup.py", line 87, in build_extension
          subprocess.check_call(['./build_bundled.sh', __version__])
          ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/bill/anaconda3/envs/pyhugface/lib/python3.13/subprocess.py", line 419, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['./build_bundled.sh', '0.2.0']' returned non-zero exit status 127.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for sentencepiece
  Running setup.py clean for sentencepiece
Failed to build sentencepiece
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (sentencepiece)
```

@vpapaioannou what version of pip are you using?

Closing this for now because it seems to be an issue with installing `sentencepiece`, which is out of the control of the `transformers` maintainers! However, feel free to keep discussing the problem here if you want - I'm just closing the issue to indicate that the `transformers` team doesn't need to triage it.

pip version is 24.2 for python 13.

Issue solved by using Python 3.10. sentencepiece doesn't have the necessary wheel files for newer versions of Python. I suggest to have a check for this if possible or at least a warning.

@vpapaioannou I just checked and `pip install sentencepiece` works on Python 3.13 - what OS are you using?

MacOS Sequoia 15.0.1

Hmmn - given that it installs for me and I successfully get a 3.13 wheel, I'm not sure if we should raise a warning. Maybe raise the install failure issue on the [sentencepiece](https://github.com/google/sentencepiece) repo instead?

What steps did you follow e.g. did you create a virtual environment, did you use conda? I can try them as well and let you know.

I just ran `pip install sentencepiece` in a miniconda environment with Python 3.13.0 on Ubuntu 24.04.

Thank you. Are you able to try on Mac as well?

Ran into this same issue with `uv` on MacOS.

After a `uv init`, my `.python-version` was set to `3.13`.

When I changed it to `3.12`, the sentencepiece error went away, and I was able to install, without the errors.

Seems downgrading Python to 3.12 or lower fixes this for me.

installing cmake worked for me

After trying everything above, executing `export CMAKE_POLICY_VERSION_MINIMUM=3.5` worked for me (macOS Sequoia 15.3.2)

MacOS Sequoia 15.5, switch to python 3.11 worked for me (firstly use python 3.13)