Error installing transformers 3.1.0
Author: bluteaur
Pull request: no
Labels: name: wontfix

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 3.1.0
- Platform: Darwin-19.6.0-x86_64-i386-64bit
- Python version: 3.7.7
- PyTorch version (GPU?): 1.6.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.
 
 albert, bert, GPT2, XLM: @LysandreJik 
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao 
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 blenderbot: @mariamabarham
 Bart: @sshleifer
 Marian: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao 
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

## Information

Model I am using (Bert, XLNet ...):
pipeline("zero-shot-classification")

The problem arises when using:
pip3 install transformers=="3.1.0"

The tasks I am working on is:
Just installing the package to use zero-shot-classification

## To reproduce

Steps to reproduce the behavior:

1. pip3 install transformers=="3.1.0"

Alternatively:
1. pip3 install tokenizers=="0.8.1.rc2"


Notes:
It seems that the tokenizers version '0.8.1.rc2' is the issue. I can install the system just fine on different systems by changing the version to '0.8.0' in transformers/setup.py. Alternatively pip3 install transformers=="3.1.0" tokenizers=="0.8.0" seems to be a working method of installation, but tokenizers version "0.8.1.rc2" still has the error. 

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

  error: build failed
  /tmp/pip-build-env-mey29riz/overlay/lib/python3.6/site-packages/setuptools/dist.py:452: UserWarning: Normalizing '0.8.1.rc2' to '0.8.1rc2'
    warnings.warn(tmpl.format(**locals()))
  cargo rustc --lib --manifest-path Cargo.toml --features pyo3/extension-module --release --verbose -- --crate-type cdylib
  error: cargo failed with code: 101
  ERROR: Failed building wheel for tokenizers
  Running setup.py clean for tokenizers
Failed to build tokenizers
ERROR: Could not build wheels for tokenizers which use PEP 517 and cannot be installed directly

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

The expected behaviour would be to install properly without error.

Comments:
Yeah the latest release of tokenizers is 0.8.1 but the current code still references the rc2:
https://github.com/huggingface/transformers/blob/master/setup.py#L113

I was able to get this working by installing tokenizers 0.8.1. I then
installed transformers 3.1.0 without dependencies using --no-dependencies
flag (had to install a few other dependencies manually).

Your mileage may vary.

On Fri, Sep 11, 2020 at 23:21 Fabrizio Milo <notifications@github.com>
wrote:

>
>
> Yeah the latest release of tokenizers is 0.8.1 but the current code still
> references the rc2:
>
>
> https://github.com/huggingface/transformers/blob/master/setup.py#L113
>
>
>
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/6970#issuecomment-691394983>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ACMFA4WL57SIWJT5U2LIBHDSFLSLZANCNFSM4Q3OYZ6Q>
> .
>
>
>


I simply installed the transformer 3.0.0 version until they fix this problem. 
`python3 -m pip install transformers==3.0.0`

> I simply installed the transformer 3.0.0 version until they fix this problem.
> `python3 -m pip install transformers==3.0.0`

I need version 3.1.0 for the latest 0-shot pipeline. But the following fixed the problem that @alexuadler mentioned:

pip3 install tokenizers=="0.8.1"
pip3 install transformers=="3.1.0" --no-dependencies

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.