FileNotFoundError when python runs setup.py for sentencepiece
Author: DaveXanatos
Pull request: no

# 🐛 Bug

FileNotFoundError when python runs setup.py for sentencepiece
I am Running Python 3.7, Tensorflow 2.1, Buster

Model I am using (Bert, XLNet ...):  Would be using gpt-2 if I can install it...

Language I am using the model on is English

The problem arises when installing using

    pip install transformers

## To reproduce

Steps to reproduce the behavior:

Run on a Raspberry Pi 4B (4Gig) running Python 3.7, Tensorflow 2.1 and Buster

1.  pip install transformers

Wait for other downloads and installing to complete and this will eventually arise:

```
Collecting sentencepiece (from transformers)
  Downloading https://files.pythonhosted.org/packages/1b/87/c3c2fa8cbec61fffe031ca9f0da512747520bec9be7f886f748457daac31/sentencepiece-0.1.83.tar.gz (497kB)
    100% |████████████████████████████████| 501kB 225kB/s 
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-xohx1aio/sentencepiece/setup.py", line 29, in <module>
        with codecs.open(os.path.join('..', 'VERSION'), 'r', 'utf-8') as f:
      File "/usr/lib/python3.7/codecs.py", line 898, in open
        file = builtins.open(filename, mode, buffering)
    FileNotFoundError: [Errno 2] No such file or directory: '../VERSION'
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-xohx1aio/sentencepiece/

```

SO I TRIED download the wheel file from https://github.com/google/sentencepiece/releases for my python version and installing it with pip install sentencepiece-xxx-cpxx-xx.whl

However I see only Mac, x86, and "manylinux" wheels but the manylinux wheels specifically reference iOS or x86, nothing I can see for Arm Core 71 (linux_armv7l).

Also tried a straight install on sentencepiece with identical failure::

```
$ pip install sentencepiece
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting sentencepiece
  Using cached https://files.pythonhosted.org/packages/1b/87/c3c2fa8cbec61fffe031ca9f0da512747520bec9be7f886f748457daac31/sentencepiece-0.1.83.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-6tdniw95/sentencepiece/setup.py", line 29, in <module>
        with codecs.open(os.path.join('..', 'VERSION'), 'r', 'utf-8') as f:
      File "/usr/lib/python3.7/codecs.py", line 898, in open
        file = builtins.open(filename, mode, buffering)
    FileNotFoundError: [Errno 2] No such file or directory: '../VERSION'
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-6tdniw95/sentencepiece/

```


## Expected behavior

I was expecting installation to complete successfully.

## Environment info

$ python transformers-cli env
python: can't open file 'transformers-cli': [Errno 2] No such file or directory

This makes sense since transformers never finishes installing
     
- `transformers` version: Newest available with pip install
- Platform:  Raspbian Buster
- Python version:  3.7.3
- PyTorch version (GPU?):
- Tensorflow version (GPU?):  2.1 cpu
- Using GPU in script?:
- Using distributed or parallel set-up in script?:


Thanks for your assistance.



Comments:
Sounds like a [sentencepiece](https://github.com/google/sentencepiece) issue?

I have an inquiry there as well since just a straight install of sentencepiece gives same results - I was just hoping there might be a way of using the HuggingFace Transformers without sentencepiece (although the name sentencepiece suggests to me that it may perform some critical functions...)  something like "pip install transformers --skip sentencepiece" ? :)


@DaveXanatos Even I faced same issues. Issue seems to be with pip package of sentencepiece. I have opened an issue with them.

As a word around, I installed sentence piece from conda and it worked. After installing this you can install transformer.

`conda install -c powerai sentencepiece`


Be wary of using conda and pip at the same time, if you don't know _exactly_ what you are doing, this will lead to unexpected complications.

SentencePiece is required for most recent models, so it is a hard dependency. I advise you to just wait until this is solved in the sentencepiece library, or download and install an earlier version of their library, e.g. https://github.com/google/sentencepiece/releases/tag/v0.1.84

@tkhan3 Thanks for the conda possibility, I will look into that in the interim.

@BramVanroy I have heard this before and a couple of years ago I completely hosed my build doing just this :)  Where would you suggest, as the most direct route to understanding exactly the differences between pip installs and conda installs in terms of paths, dependencies, etc., such that I could conda install with confidence a package in an otherwise pip installed environment?

I'm afraid I cannot help with that. I stay away from conda as much as I can. Pipenv is my main driver, falling back to an environment's pip where necessary.

@BramVanroy I'm with you on that from my experience, although I know some folks swear by it...  thanks for the warning.  I'll probably flash a backup image and then go and play with the possibilities and see if I can get it to work... I can always reflash back to the backup if I break everything again.  If I have success I'll let you know what I found.

Closing this. I propose that the discussion is moved to the sentencepiece library. https://github.com/google/sentencepiece/issues/452

I got a similar issue.
When I install sentence-transformers on Linux by python, I got an error message: 

ERROR: Could not find a version that satisfies the requirement transformers>=3.0.2 (from sentence-transformers) (fr
om versions: 0.1, 2.0.0, 2.1.0, 2.1.1, 2.2.0, 2.2.1, 2.2.2, 2.3.0, 2.4.0, 2.4.1, 2.5.0, 2.5.1)
ERROR: No matching distribution found for transformers>=3.0.2 (from sentence-transformers)

system: Linux 4.9.0-12-amd64 #1 SMP Debian 4.9.210-1+deb9u1 (2020-06-07) x86_64 on GCP VM instance.
Is there any suggestion?


I had the same issue but on Unbuntu 20.04.1.
My problem was that i used a pip version to old to install sentencepiece, as it requires pip>=19.3.  (https://github.com/google/sentencepiece/issues/572#issuecomment-716890916)
So my solution was to upgrade my pip installation to 20.2.4
$ pip install --upgrade pip

Similar issues has been discussed here https://github.com/google/sentencepiece/issues/572

Hope it helps