noamaanMulla-03/transformers-issues huggingface dataset:
Support for private models from huggingface.co
Author: julien-c
Pull request: yes

Add a `use_auth_token` flag (or string) to all `from_pretrained` entry points, to specify token to use as Bearer authorization for remote files. 

- if it's a string, use it
- If it's true, will get token from `~/.huggingface/token` (will throw if no token there)

You can test this with:

```python
model = AutoModelForMaskedLM.from_pretrained("pierric/hf-private", use_auth_token=True)
```

We'll add unit tests down the line but need to think about which environment those tests are going to hit.

⚠️ For now, I decided against adding token by default to all calls if user is logged in. Let's discuss though!

Comments:
@patrickvonplaten I don't follow here. In this PR we want to have a way to either pass a token directly, or to opt in to use the one that's store in `~/`. Don't see how I can do that with just an optional string?

> @patrickvonplaten I don't follow here. In this PR we want to have a way to either pass a token directly, or to opt in to use the one that's store in `~/`. Don't see how I can do that with just an optional string?

I might have misunderstood a bit what constrains there are on the functionality. I thought, the following logic is possible and makes sense here:

- If user passes a string `use_auth_token`, then use this as the token
- Else look for token in `~/.huggingface`:
          - if there is no token and model is private -> throw error
          - if there is no token and model is **not** private -> load the model as usual
          - if there is a token -> use this one

Not sure if there is something I am completely overlooking here in the logic though, *e.g.* if we cannot know before hand whether the model is private or not


> > @patrickvonplaten I don't follow here. In this PR we want to have a way to either pass a token directly, or to opt in to use the one that's store in `~/`. Don't see how I can do that with just an optional string?
> 
> I might have misunderstood a bit what constrains there are on the functionality. I thought, the following logic is possible and makes sense here:
> 
> * If user passes a string `use_auth_token`, then use this as the token
> * Else look for token in `~/.huggingface`:
>   - if there is no token and model is private -> throw error
>   - if there is no token and model is **not** private -> load the model as usual
>   - if there is a token -> use this one
> 
> Not sure if there is something I am completely overlooking here in the logic though, _e.g._ if we cannot know before hand whether the model is private or not

Ok never mind - as discussed offline this would require more features to add which is out-of-scope for this PR -> so LGTM!

Also cc'ing @borisdayma as this PR adds a `exist_ok` param to `HfApi.create_repo()`
Image rendering not working in example notebook
Author: darigovresearch
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: n/a
- Platform: n/a
- Python version: n/a
- PyTorch version (GPU?): n/a
- Tensorflow version (GPU?): n/a
- Using GPU in script?: n/a 
- Using distributed or parallel set-up in script?: n/a

### Who can help
As advised looking at the git blame @mfuntowicz @n1t0 could you advise?

<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.


 albert, bert, GPT2, XLM: @LysandreJik 
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 TextGeneration: @TevenLeScao 
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten @TevenLeScao
 Blenderbot: @patrickvonplaten
 Bart: @patrickvonplaten
 Marian: @patrickvonplaten
 Pegasus: @patrickvonplaten
 mBART: @patrickvonplaten
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 RAG: @patrickvonplaten, @lhoestq
 FSMT: @stas00
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Go to https://github.com/huggingface/transformers/blob/master/notebooks/02-transformers.ipynb
2. Go to section `Want it lighter? Faster? Let's talk distillation!`
3. You should see that there is an image which is not rendering like the below
![image](https://user-images.githubusercontent.com/30328618/101995456-cff18a00-3cc1-11eb-81c2-3685e96c5f52.png)


<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
Should be an image to explain something

<!-- A clear and concise description of what you would expect to happen. -->


Comments:
Update notebook table and transformers intro notebook
Author: sgugger
Pull request: yes

# What does this PR do?

Update the examples table and the notebooks table to include all recent examples. Also fix the intro notebook to the transformers library, in particular, the image that was missing.

Fixes #9083


Comments:
Discussion on pinning/testing notebooks needs to be global on all notebooks (not just one) so merging this for now. We can think of a strategy and implement it in a follow-up PR.
Add message to documentation that longformer doesn't support token_type_ids
Author: HHousen
Pull request: yes

# What does this PR do?

Fixes #9111. This pull request adds a notice to the Longformer model documentation that it does not have `token_type_ids`, similarly to the message on the [RoBERTa documentation](https://huggingface.co/transformers/model_doc/roberta.html).


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

@sgugger (documentation)
@patrickvonplaten (longformer)


Comments:
Thanks for your PR! It looks like you did not run the `make style` command to format properly your changes.

@sgugger Sorry about that. It should be good now.

Thanks a lot!
Fix local_files_only for TF
Author: LysandreJik
Pull request: yes

The `local_files_only` was not working for TF. The flag is to be used as such:

```py
from transformers import TFBertModel
model = TFBertModel.from_pretrained('bert-base-cased', local_files_only=True)
```
Setting it to `True` for any TF model would result in the following error:
```
Traceback (most recent call last):
  File "/Users/jik/Library/Application Support/JetBrains/PyCharm2020.1/scratches/scratch_1.py", line 7, in <module>
    model = TFBertModel.from_pretrained('bert-base-cased', local_files_only=True)
  File "/Users/jik/Workspaces/python/transformers/src/transformers/modeling_tf_utils.py", line 578, in from_pretrained
    local_files_only=local_files_only,
  File "/Users/jik/Workspaces/python/transformers/src/transformers/file_utils.py", line 663, in cached_path
    local_files_only=local_files_only,
  File "/Users/jik/Workspaces/python/transformers/src/transformers/file_utils.py", line 801, in get_from_cache
    "Cannot find the requested files in the cached path and outgoing traffic has been"
ValueError: Cannot find the requested files in the cached path and outgoing traffic has been disabled. To enable model look-ups and downloads online, set 'local_files_only' to False.
```

This is because the `file_utils.get_from_cache` method obtains the filename from the `url_to_filename` method:

https://github.com/huggingface/transformers/blob/1246b20f6d81bcd949078d26cf5ab3d0f3acccc6/src/transformers/file_utils.py#L777

The issue is that this method adds `.h5` to TF models:

https://github.com/huggingface/transformers/blob/1246b20f6d81bcd949078d26cf5ab3d0f3acccc6/src/transformers/file_utils.py#L585-L586

This works when the file is already built using `{filename}.{etag}`, resulting in `{filename}.{etag}.h5`. However, since the etag is `None` with `local_files_only=True`, this results in `{filename}.h5`. 

The method tries to find the saved files using the filename followed by `.*`:

https://github.com/huggingface/transformers/blob/1246b20f6d81bcd949078d26cf5ab3d0f3acccc6/src/transformers/file_utils.py#L788-L792

This doesn't work since it's looking for `{filename}.h5.*` which doesn't exist. It should instead be looking for `{filename}.*`, which is what it's doing now.

Fix https://github.com/huggingface/transformers/issues/5016

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Thanks, @LysandreJik .  I manually applied this fix to a local installation of `transformers==3.3.0` and can confirm that this fix resolves #5016 .  Can this please be merged?

This still doesn't work in 3.3.0. I am getting the same issue on running without internet with transformers==3.3.0

You're perfectly correct! This was merged the 1st of October, and 3.3.0 was released in September. Please install a more recent version to get that fix (version v3.4.0 being the first release which contained that fix).

Thanks, I'll check.
BertForSequenceClassification and DistilBertForSequenceClassification use pooler output in different ways
Author: AndreaSottana
Pull request: no

Hi,
The `BertForSequenceClassification` includes a forward pass of the BertModel, and it takes the second element (index 1) from its output before moving forward, as shown [here ](https://github.com/huggingface/transformers/blob/master/src/transformers/models/bert/modeling_bert.py#L1378 )
This is the return of `BertModel`
```
    return BaseModelOutputWithPoolingAndCrossAttentions(
            last_hidden_state=sequence_output,
            pooler_output=pooled_output,
            hidden_states=encoder_outputs.hidden_states,
            attentions=encoder_outputs.attentions,
            cross_attentions=encoder_outputs.cross_attentions,
        )
```
hence `output[1]` is taking the pooler_output.

However, in `DistilBertForSequenceClassification`, it takes the first element (index 0) of the `DistilBertModel`'s forward pass, i.e. `distilbert_output[0]`, as shown [here](https://github.com/huggingface/transformers/blob/master/src/transformers/models/distilbert/modeling_distilbert.py#L625)

This is the last hidden state for all tokens.
Why is there this discrepancy between the two models? The behaviour of `DistilBertForSequenceClassification` makes more intuitive sense to me.
Why is `BertForSequenceClassification` using only the pooler_output from `BertModel` (i.e. the hidden state of the first token)? Why are all other hidden states of other tokens not needed here, but needed in the distilled version?

Thanks for the help!
    

Comments:
BERT and DistilBERT are different models. DistilBERT isn't simply a BERT model with fewer layers, but a BERT model without the pooling layer as you have seen, and with no token type embeddings.

We try to stay as close to the original implementations as possible, hence why BERT is done this way, and why DistilBERT was done differently. I invite you to read the paper or study the original BERT codebase to see how it was done, it should be very similar (or the same) as it is done here.

Thanks a lot for clarifying @LysandreJik I'll close the issue then!
T5 checkpoint contains weights missing on current model.
Author: dscarmo
Pull request: no

## Environment info
Colab (16 december 2020), transformers 4.0.1

### Who can help
 T5: @patrickvonplaten

## Information
When loading the pre-trained T5 weights, directy with .from_pretrained, with the newest version, it returns the following warning:

>  Some weights of the model checkpoint at t5-small were not used when initializing T5ForConditionalGeneration: ['decoder.block.0.layer.1.EncDecAttention.relative_attention_bias.weight']
> - This IS expected if you are initializing T5ForConditionalGeneration from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
> - This IS NOT expected if you are initializing T5ForConditionalGeneration from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).

This also caused old checkpoints of mine to not load due to the missing weight. 

## To reproduce
Reproduced in Colab:
https://colab.research.google.com/drive/158OiSKHz80b0PQYaWB6c-AI9xKCzE1ns?usp=sharing

## Expected behavior

Loading T5 should not return warnings if i am loading the pre-trained weights from the library. 


Comments:
Yes we should fix the warning. It's no problem that those weights are missing i.e.:
#8933

Thank you!
BERT and SpanBERT for Coreference Resolution
Author: techscribr
Pull request: no
Labels: name: New model

# 🌟 New model addition

## Model description

This is a recent approach for co-reference resolution using BERT, implemented from the papers [BERT for Coreference Resolution: Baselines and Analysis](https://arxiv.org/abs/1908.09091) and [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://arxiv.org/abs/1907.10529), which is the current state of the art on OntoNotes (79.6 F1). It uses tensorflow 1.14.0.

Reason why this is interesting is it achieves strong improvements on the OntoNotes (+3.9 F1) and GAP (+11.5 F1) benchmarks. Also, I think it would be a nice addition to huggingface library, as it has only the neuralcoref as the coreference resolution module.

## Open source status

* [x] the model implementation is available: (https://github.com/mandarjoshi90/coref)
* [x] the model weights are available: (https://github.com/facebookresearch/SpanBERT)
* [x] who are the authors: (@mandarjoshi90, @jkkummerfeld, @wenyudu)


Comments:
Commenting for visibility - is this available now ? can't seem to find it, would love to use this for a question-answering project i'm working on! 

I'd also like to see this model incorporated into the core list of supported models. I did note that you can download it from the community models here though: https://huggingface.co/SpanBERT/spanbert-base-cased

Are there any translations of the above repository (https://github.com/mandarjoshi90/coref) into the awesome HuggingFace API ? That would be very cool to test :D !

I would like to work on this...but will need some guidance
Put all models in the constants
Author: sgugger
Pull request: yes

# What does this PR do?

It was impossible to use all pretrained checkpoints in the tapas tokenzier file because they were not in the constants of the file. This PR fixes that.

Comments:
An alternative is to remove all content, but the previous state where one variable as some keys but the others not leads to failures.

Yes, we really need to remove all of this @LysandreJik @thomwolf 

We do, but right now we still use them in the tests

It was a bit confusing to me which to add in the constants and which not.. 😅 

@sgugger one more small fix you can add is in tapas.rst under "Usage: inference", there should be one more space:

![docs](https://user-images.githubusercontent.com/48327001/102513749-e6b22b00-408b-11eb-9adb-060108f1b2e9.PNG)

(Sorry, if I see more things I make a new PR myself :p)

Pushing your fix on master directly Niels
Fix link to old NER fine-tuning script
Author: mrm8488
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->


Comments:
distilbert-base-uncased
Author: anitaj-j
Pull request: no
Labels: name: wontfix

# 🐛 Bug/Question

## Information
 am trying to execute :

**import ktrain

from ktrain import text

MODEL_NAME='distilbert-base-uncased'

t=text.Transformer(MODEL_NAME, maxlen=500, classes=np.unique(y_train))**

I get the following error:

OSError: Model name 'distilbert-base-uncased' was not found in tokenizers model name list (distilbert-base-uncased, distilbert-base-uncased-distilled-squad, distilbert-base-cased, distilbert-base-cased-distilled-squad, distilbert-base-german-cased, distilbert-base-multilingual-cased). We assumed 'distilbert-base-uncased' was a path, a model identifier, or url to a directory containing vocabulary files named ['vocab.txt'] but couldn't find such vocabulary files at this path or url.

ktrain version 0.14.6 transformers version 2.8.0

Libraries was installed using pip install. I am following the tutorial : https://towardsdatascience.com/text-classification-with-hugging-face-transformers-in-tensorflow-2-without-tears-ee50e4f3e7ed 


Any help would be appreciated.

Comments:
hi! did you find a solution? I got the same problem!

I have the same problem with a freshly installed version. Any ideas?

This can happen if there is a network or firewall issue that prevents the `transformers` library from downloading the required files.   To replicate on a laptop, you can delete the cache (e.g., `~/.cache/torch/transformers`), turn off wifi, and run this:
```python
from transformers import *
tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased')
```
If following the above steps, you will see the exact same error as posted above (i.e., `OSError: Model name 'distilbert-base-uncased' was not found in tokenizers model name list `).

As a workaround for machines with download issues, you can download the tokenizer files (e.g., `vocab.txt`) and model files manually using machine with access, copy folder to machine with firewall/network issues, and then point `transformers` to the folder containing the downloaded files (e.g., `DistilBertTokenizer.from_pretrained('path/to/folder)`).

See [this ktrain FAQ entry](https://github.com/amaiya/ktrain/blob/master/FAQ.md#how-do-i-use-ktrain-without-an-internet-connection) for more information on workarounds/solutions on using `transformers` without internet or behind firewall.  While the entry focuses on using `transformers` from within the  *ktrain* library, most information applies generally to `transformers`.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


This is resolved if you ensure you use transformers==3.4.0