noamaanMulla-03/transformers-issues huggingface dataset:
Fix broken link to distill models in docs
Author: Randl
Pull request: yes

# What does this PR do?
Fixes broken link to distill models in docs
<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).


## Who can review?

@sgugger @VictorSanh

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
Done

Thanks a lot!
[examples/pytorch/image-classification] why does it require `torch>=1.9.0`
Author: stas00
Pull request: no

Why do we have these particular requirements at:

https://github.com/huggingface/transformers/blob/b90096fe1424e1927e549b73b5db67341a0d27d0/examples/pytorch/image-classification/requirements.txt#L1-L2

I tested it and it works with pt-1.8.2 just fine.
```
RUN_SLOW=1 pyt examples/pytorch/test_examples.py -k test_run_image_classification
```

Let's not set a higher version requirement than it actually is needed.

I propose to remove both entries altogether unless they are actually needed and if so tested with the lowest supported version.

It's important in this particular case since HF Deepspeed integration tests are now included in the Deepspeeed CI which runs on pt-1.8 so I can't test HF image models on the Deepspeed side then.

Thank you!

@nateraw, @LysandreJik 

Comments:
Feel free to PR if ya want if it's a blocker for you! Otherwise I'll get to it when I fix the config issue (on mobile can't link to issue number)

I can wait no problem, @nateraw - I just didn't know which lower version is actually required.

Thank you!
Unigram frequencies in GPT-2 or XLnet?
Author: jhlau
Pull request: no

Question: does the GPT2 or XLnet tokenizer contain unigram frequencies? From the discussion here (https://github.com/huggingface/pytorch-transformers/issues/477), it looks like the tokenizer from TransformerXL has it, but I'm not sure if the same applies for GPT2 and XLnet. If they do contain unigram frequencies, can you point me to the objects in the GPT2/XLnet's tokenizer that have this frequency information?

Comments:
XLnet tokenizer utilizes SentencePiece and you can use it's score for unigram as relative frequency with something like `math.exp(XLNetTokenizer.sp_model.GetScore(token_id))`.

Ah perfect. So the raw score gives the unigram log probability, and the exp of it gives the normalised frequency.

How about GPT-2? Is this information saved in the tokenizer?

As far as I see, GPT-2 tokenizer does not contain frequency information. What's more tokens in dictionary are not ordered according to frequency (so it is not possible to estimate frequency assuming Zipf's distribution), but according to length.

Many thanks for looking into this. I suppose I have no choice but to mine unigram frequencies myself... (at this point I am looking to use a similar web corpus, such as https://skylion007.github.io/OpenWebTextCorpus/ to do this; if you have any pointers please chime in).

Hello @jhlau
I was looking for a similar unigram frequency for GPT-2. Would you happen to have acquired (or created) such a list and be willing to share?

There you go: https://drive.google.com/file/d/1FhObTkvhT46Xy-Vyqi-gku8Ho2c1XvHx/view?usp=sharing

Python3 pickle file. Unigram mined based on the openwebtextcorpus linked above.

Hi @jhlau 
Can you please explain how you mined the unigrams?
I looked at your file (thank you for sharing!) and it seems to me as if the keys are case-sensitive and that there was some method like SentencePieces or BPE used (I noticed keys like "ing" while very common verbs are not included). I'd like to be able to tell the frequency of a word, how should I go about it?
Thank you!

GPT-2 uses BPE, so the openwebtextcorpus is tokenised with BPE, and then unigram frequencies are collected.

> There you go: https://drive.google.com/file/d/1FhObTkvhT46Xy-Vyqi-gku8Ho2c1XvHx/view?usp=sharing
> 
> Python3 pickle file. Unigram mined based on the openwebtextcorpus linked above.

Hi @jhlau, it seems the link here has expired. It will be quite helpful if you can provide it again. Many thanks!

@GeassTaiga I no longer have it anymore unfortunately =/
Update no_* argument (HfArgumentParser)
Author: BramVanroy
Pull request: yes

Changes the order so that the no_* argument is created after the original argument AND sets the default for this no_* argument to False.

Functionality-wise this should not change anything and there should not be any breaking changes. What does change, is usability, as it is now less ambiguous to users what the arguments' real default is. E.g.:

**Before**
```
  --no_dataloader_pin_memory
                        Whether or not to pin memory for DataLoader. (default: True)
  --dataloader_pin_memory [DATALOADER_PIN_MEMORY]
                        Whether or not to pin memory for DataLoader. (default: True)
```

**After**
```
  --dataloader_pin_memory [DATALOADER_PIN_MEMORY]
                        Whether or not to pin memory for DataLoader. (default: True)
  --no_dataloader_pin_memory
                        Whether or not to pin memory for DataLoader. (default: False)
```

Fixes #13847

@sgugger



Comments:
Update Tatoeba conversion
Author: Traubert
Pull request: yes

# What does this PR do?

The Helsinki-NLP / Tatoeba NMT models have gone through various
architectural changes, and the old conversion code fails on them. This
commit is something of a rewrite to remedy this, in particular parsing
supplied yaml files rather than README.md files.

This was previously reviewed by @patrickvonplaten [here](https://github.com/huggingface/transformers/pull/12192), but we didn't get around to merging it because of an unclean commit history, so I made a fresh branch now to be extra sure.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

@patrickvonplaten 

Comments:
Great, thanks, and thanks @patil-suraj, I'll try to come back to those at some point.
Fixing empty prompts for text-generation when BOS exists.
Author: Narsil
Pull request: yes

# What does this PR do?

Fixes #13774

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->



## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@LysandreJik

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

Comments:
It seems there is an issue with the TensorFlow pipeline tests
Empty prompts failing in dev sources
Author: xloem
Pull request: no

It looks like #13308, which otherwise is really inspiring around engaging organization of the code, also introduced a bug around completing empty prompts:
```
transformers.pipeline('text-generation')('')
```
```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/.local/lib/python3.8/site-packages/transformers/pipelines/text_generation.py", line 150, in __call__
    return super().__call__(text_inputs, **kwargs)
  File "/home/user/.local/lib/python3.8/site-packages/transformers/pipelines/base.py", line 915, in __call__
    return self.run_single(inputs, preprocess_params, forward_params, postprocess_params)
  File "/home/user/.local/lib/python3.8/site-packages/transformers/pipelines/base.py", line 922, in run_single
    model_outputs = self.forward(model_inputs, **forward_params)
  File "/home/user/.local/lib/python3.8/site-packages/transformers/pipelines/base.py", line 871, in forward
    model_outputs = self._forward(model_inputs, **forward_params)
  File "/home/user/.local/lib/python3.8/site-packages/transformers/pipelines/text_generation.py", line 162, in _forward
    generated_sequence = self.model.generate(input_ids=input_ids, **generate_kwargs)  # BS x SL
  File "/home/user/.local/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context
    return func(*args, **kwargs)
  File "/home/user/.local/lib/python3.8/site-packages/transformers/generation_utils.py", line 1016, in generate
    return self.sample(
  File "/home/user/.local/lib/python3.8/site-packages/transformers/generation_utils.py", line 1529, in sample
    outputs = self(
  File "/home/user/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/user/.local/lib/python3.8/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 949, in forward
    transformer_outputs = self.transformer(
  File "/home/user/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/user/.local/lib/python3.8/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 673, in forward
    input_ids = input_ids.view(-1, input_shape[-1])
RuntimeError: cannot reshape tensor of 0 elements into shape [-1, 0] because the unspecified dimension size -1 can be any value and is ambiguous
```

Comments:
cc @Narsil 

Added the fix (forgot to readd existing code took the opportunity to add the test)
Improve error message when loading models from Hub
Author: aphedges
Pull request: yes

# What does this PR do?

Improves the error message when loading a model from the Hub that is the same name as a local directory.

I encountered an issue where I couldn't load `gpt2` from the Hub, and it turned out to be because there was a local directory named `gpt2` that was not the saved model. I do not seem to be the only person who's discovered this issue, as can be seen by <https://github.com/huggingface/transformers/issues/8661>, <https://github.com/huggingface/transformers/issues/10918#issuecomment-877807753>, and <https://github.com/huggingface/transformers/issues/13073>. I therefore decided to improve the error message to make the problem easier to catch. I made the change everywhere in the codebase I could find.

I'm not entirely satisfied with the new wording, but trying to completely describe the name collision problem in a short message is difficult. I'd appreciate any suggestions. One thought I had was to swap the order of the first two bullets to make it more clear that local files and directories have priorities, but I wasn't sure whether that would really help.

## Who can review?

- tokenizers: @n1t0, @LysandreJik

Comments:
@sgugger, I wasn't fond of the message either, so I applied your new wording to all the files. Thanks for the suggestion!
Initial support for symbolic tracing with torch.fx allowing dynamic axes
Author: michaelbenayoun
Pull request: yes

# What does this PR do?

This PR enables to symbolic trace models without having to specify a fixed batch size and / or sequence length. 
To specify that an axis should not have a fixed shape, the value should be set to -1:

```python
traced = symbolic_trace(
    model,
    input_names=["input_ids", "attention_mask", "token_type_ids"],
    batch_size=-1, # means that batch size should be dynamic
    sequence_length=-1 # means that the sequence length should be dynamic
)
```

Currently, only the following models support dynamic axes:

- Albert
- Bert
- DistilBert
- MobileBert
- MegatronBert
- Electra

Traced models with dynamic axes cannot be retraced out of the box, to solve this 2 functions are provided:

- `prepare_for_retracing`: takes a traced model as input and outputs a model that can be retraced and some information that is needed to get back to dynamic axes
- `restore_after_retracing`: takes a retraced model and the information returned by `prepare_for_retracing` and set the dynamic axes back

Example:
```python
prepared, attributes = prepare_for_retracing(traced)
retraced = some_retracing_func(prepared)
final_model = restore_after_retracing(retraced, attributes)
``` 

To make things less tedious, `retrace_graph_with` is provided: it takes a traced model, and either a Tracer or a tracing function and performs the retracing:

```python
final_model = retrace_graph_with(traced, func=some_tracing_func)
```

Being able to retrace a model is important because that is how quantization can be done for instance.

Finally, this PR also provides sanity checks that validate that symbolic tracing is available for the model provided to `symbolic_trace`, and that it can be traced with dynamic axes if the user asks for it.

Comments:
@sgugger What do you think of the way I've changed things regarding the config classes imports?
(Distributed)LengthGroupedSampler: allow only providing lengths but not a dataset
Author: ZhaofengWu
Pull request: no

# 🚀 Feature request

Currently `LengthGroupedSampler` & `DistributedLengthGroupedSampler` require a `dataset` and optionally `lengths`. However, the sole purpose of `dataset` here is to get the lengths, so these classes should allow providing `lengths` only but not a `dataset`. This is easily doable for `LengthGroupedSampler` by giving `None` as a `dataset` (though not very elegant). However, `DistributedLengthGroupedSampler` actually calls `len(dataset)` even when `lengths` is provided (this can be easily circumvented by calling `len(lengths)` instead if provided). I think it'd be great if both classes make `dataset` an optional parameter but check that at least one of `dataset` and `lengths` is provided.

## Motivation

Sometimes people do not use a HuggingFace Dataset instance, but find the functionality provided by `LengthGroupedSampler` convenient to use. So they can manually provided `lengths`.

## Your contribution

I can submit a PR, if needed.


Comments:
cc @sgugger 

This would be a nice addition indeed. If you want to work on this, don't hesitate to submit a PR!

Submitted: #13820
Errors when fine-tuning RAG on cloud env
Author: agi-templar
Pull request: no

Hi the team,

I'm trying to fine-tune RAG with [the scripts you provided](https://github.com/huggingface/transformers/tree/9ec0f01b6c3aff4636869aee735859fb6f89aa98/examples/research_projects/rag). My env is cloud servers (4 V100 with 48G GRAM), and I always have these errors when do the fine-tuning:

> RuntimeError: Error in faiss::Index* faiss::read_index(faiss::IOReader*, int) at /__w/faiss-wheels/faiss-wheels/faiss/faiss/impl/index_read.cpp:480: Error: 'ret == (size)' failed: read error in <cache path>: 6907889358 != 16160765700 (Success)

It seems like errors are from faiss (and I don't know how to interpret it. Sizes do not macth?). I used this command to do the fine-tuning:

```bash
- python run_rag_ft.py
        --data_dir /msmarco
        --output_dir ./msmarco_rag
        --model_name_or_path facebook/rag-sequence-nq
        --model_type rag_sequence
        --fp16
        --gpus 4
        --distributed_retriever pytorch
        --num_retrieval_workers 4
        --fp16
        --profile
        --do_train
        --do_predict
        --n_val -1
        --train_batch_size 8
        --eval_batch_size 1
        --max_source_length 128
        --max_target_length 40
        --val_max_target_length 40
        --test_max_target_length 40
        --label_smoothing 0.1
        --dropout 0.1
        --attention_dropout 0.1
        --weight_decay 0.001
        --adam_epsilon 1e-08
        --max_grad_norm 0.1
        --lr_scheduler polynomial
        --learning_rate 3e-05
        --num_train_epochs 2
        --warmup_steps 500
        --gradient_accumulation_steps 1
```

Nothing special but just use my own data (MSMARCO). I sticked to the pytorch for the distributed retriever, and have not yet tested the ray version. Is that the problem?

I cannot run this on my local machines because of OOM errors (two 24GRAM GPUs).


I think @patrickvonplaten could help me on this. Thanks!


Comments:
Hey @DapangLiu,

We sadly don't actively maintain the `research_projects` folder except for Wav2Vec2. Could you try to use the forum: https://discuss.huggingface.co/ instead? 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
How to use the pretraining task of ProphetNet
Author: StevenTang1998
Pull request: no

I want to use the pretraining task of ProphetNet, that recovers the mask span of the input sentence.
I follow the instruction of Figure 1 in the paper.

For example, the input is `But I [MASK][MASK] my life for some lovin\' and some gold` and I only recover the first `[MASK]`. (the sentence is from the pretraining corpus BookCorpus)
I use the following code:
```python
from transformers import ProphetNetTokenizer, ProphetNetForConditionalGeneration
tokenizer = ProphetNetTokenizer.from_pretrained('prophetnet')
model = ProphetNetForConditionalGeneration.from_pretrained('prophetnet')

# the sentence is from the pretraining corpus BookCorpus
input_ids = tokenizer('But I traded all my life for some lovin\' and some gold', return_tensors="pt")['input_ids']
mask_id = input_ids[0][2]
input_ids[0][2:4] = tokenizer.pad_token_id

decoder_input_ids = tokenizer('[MASK][MASK] I', return_tensors="pt")['input_ids']
# the way of MASS: decoder_input_ids = tokenizer('[MASK][MASK][MASK]', return_tensors="pt")['input_ids']

output = model(input_ids=input_ids, decoder_input_ids=decoder_input_ids)
probs = output.logits[0][2]
# the rank of the target word in the vocabulary
print((probs[mask_id]<probs).sum())
```

However, the rank of `traded` is 15182 among 30522 words.
And I also tried different masked words and masked spans, but the results are all unexpected.

So, I want to ask if my way to recover the mask has some errors? @patrickvonplaten 

Comments:
cc @qiweizhen

@StevenTang1998 - could you maybe try to use the forum: https://discuss.huggingface.co/ for such questions. I haven't played around with the model enough to give a qualified answer here sadly :-/ 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.