noamaanMulla-03/transformers-issues huggingface dataset:
Add Flax Examples and Cloud TPU README
Author: avital
Pull request: yes

# What does this PR do?

Adds a Flax examples README. Pretty bare for now, but will include a link to Cloud TPU instructions once they are up.

I hope my use of relative links works well, but looking for feedback. The main goal here is to have a canonical link we can point to. Perhaps later this should live on the proper docs page but I thought a README is a fine first step.

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).


Comments:
cc @patrickvonplaten
Fix a bug in summarization example which did not load model from config properly
Author: tomy0000000
Pull request: yes

# What does this PR do?

Current example script does not load model when config is supplied, just a small bug fix.

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger

Comments:
FlaxGPT2
Author: patil-suraj
Pull request: yes

# What does this PR do?

[From Patrick:]

This PR adds GPT2 for Flax and lays the groundwork for `generation` in Flax. 
The model is a straightforward translation from PyTorch with two exceptions:

1.) the `past_key_values/caching` is more complex since `generation` has to compile to be usable on TPU with Flax/Jax. This means that the cache cannot grow dynamically, but has to be a static, padded tensor that is filled during generation. This means that the cache can now only be used as follows:
i) decide on a `max_length` to which the model should generate at most and call `model.init_cache(batch_size, max_length)`. This will then add a `"cache"` variable to the expected variables by the model and returns a 0 padded tensor of shape `(batch_size, max_length)`. 
ii) This cache is then subsequently passed & retrieved similar to how it's done in PyTorch:  `logits, past_key_values = model(input_ids, past_key_values=cache)`. When doing so, internally, the `cache` is marked as `mutable` so that the Flax model can step-by-step fill the padded tensor. We also need to pass a `_use_cache` boolean internally to tell the model to initialize the cache in the first place.
iii) The implemented caching system differs slightly from the original flax caching system [here](https://github.com/google/flax/blob/6111a4b2c5cb56520549c6dfc911135fbd8dbb7f/flax/linen/attention.py#L252) since we want to allow the very first decoding pass to take an `input_prompt` that is longer than 1 (*e.g.* for GPT3-like few-shot prompting) - see: https://github.com/google/flax/issues/1317.
iv) A test verifies that compiled generation works as expected
v) This also means that when using the cache, the `attention_mask` has to have the same static shape as the cache, I guess we could have better error messages here to, but I think this should mostly be handled by the generation method which will be added in a follow-up PR: https://github.com/huggingface/transformers/pull/11685.

2.) Since in Flax we rely on `flax.linen.dot_product_attention`, we are adding the attention mask once whereas in PyTorch we are setting certain values to the padding value via `torch.where(...)` [here](https://github.com/huggingface/transformers/blob/c73e35323d5805bff27c5dbd9a5691008be1316a/src/transformers/models/gpt2/modeling_gpt2.py#L187) and are adding large negative numbers [here](https://github.com/huggingface/transformers/blob/c73e35323d5805bff27c5dbd9a5691008be1316a/src/transformers/models/gpt2/modeling_gpt2.py#L191). This is unnecessarily costly in Flax and only matters for unrealistic cases where tokens in the middle of the sequence are padded. For all realistic cases PT and Flax behaves exactly the same, which is shown in the overwritten comparison tests.

Comments:
@LysandreJik @sgugger - this PR is blocking a couple of other things. Let me know if you guys aren't really ok with some of those things
Fix usage of head masks by PT encoder-decoder models' `generate()` function
Author: stancld
Pull request: yes

This PR adds missing arguments `head_mask`, `decoder_head_mask` and `cross_attn_head_mask` into `prepare_inputs_for_generation` function of **PyTorch encoder-decoder models** so that these args will be used during the generation when `generate()` function is called.

EDIT: Need to fix the new test for ProphetNet

<hr>

### Example

```python
out = bart.generate(input_ids, ...)
tokenizer.decode(out[0], ...)
```
```
>>> 'The Eiffel Tower in Paris has been officially opened to the public.'
```


**Behaviour before the PR:**

```python
out = bart.generate(input_ids, decoder_head_mask=decoder_head_mask, ...)
tokenizer.decode(out[0], ...)
```

```diff
- >>> 'The Eiffel Tower in Paris has been officially opened to the public.'
```

**Behaviour after the PR:**

```python
out = bart.generate(input_ids, decoder_head_mask=decoder_head_mask, ...)
tokenizer.decode(out[0], ...)
```
```diff
+ >>> 'The Eiffel Tower in Paris has been officially opened to the public for the first time since it was completed in 1903.'
```


<hr>

**Reviewers:** @patrickvonplaten

Comments:
Hey @stancld,

Thanks a lot for this contribution! Could we add one test to verify that generation works with head_mask for all encoder-decoder models? 

I think it could be added to `test_generation_utils.py` 

Hey @patrickvonplaten, I've added one test. At this moment, there are two little issues I'm gonna handle later today so that all encoder-decoder models will pass this new test.

Hi @patrickvonplaten, sorry for being silent for a while as I've been a bit too busy. As you suggest, I skip the test for `ProphetNetForConditionalGeneration` model and now all the tests pass :)
Fixed: Better names for nlp variables in pipelines' tests and docs.
Author: 01-vyom
Pull request: yes

# What does this PR do?

Fixes #9455 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@Narsil @sgugger 
<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
Could you take care of the merge conflicts and we should be good to merge? Thanks!

Thanks a lot for this !
Add batching to pipelines
Author: skurzhanskyi
Pull request: no

# Add batching to pipelines

Are there any plans to add a batching option to existing pipelines? Currently, the model tries to process all the input simultaneously, which sometimes (if the input is considerable) leads to memory errors.

Comments:
Hi! You may find the discussion on this PR useful: https://github.com/huggingface/transformers/pull/11251

Thanks for explaining this
mbart-large-cc25 tokenization_utils_fast.py TypeError 
Author: lysa-n
Pull request: no

## Environment info
Hi, I am trying to fine-tune a dutch summarization algorithm. I used the [following ](https://github.com/huggingface/notebooks/blob/master/examples/summarization.ipynb) example notebook provided by huggingface.co. 

To prepare the targets for the model, we need to tokenize them inside the as_target_tokenizer context manager. This will make sure the tokenizer uses the special tokens corresponding to the targets. This is achieved by running the following code:

```
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
  
tokenizer = AutoTokenizer.from_pretrained("facebook/mbart-large-cc25")

with tokenizer.as_target_tokenizer():
    print(tokenizer(["Hello, this one sentence", "This is another sentence."]))
```

However, I get the following error:

```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-24-0fc6af9091da> in <module>()
----> 1 with tokenizer.as_target_tokenizer():
      2     print(tokenizer(["Hello, this one sentence", "This is another sentence."]))
      3 

3 frames
/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_fast.py in convert_ids_to_tokens(self, ids, skip_special_tokens)
    293         tokens = []
    294         for index in ids:
--> 295             index = int(index)
    296             if skip_special_tokens and index in self.all_special_ids:
    297                 continue

TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'
```
How can I work around this TypeError? I am not a professional and this is actually the first time submitting any question at all. 
Thanks in advance :)

Comments:
Hi @lysa-n,
For multilingual models you must define input language(src_lang) and target language(tgt_lang). Since you are using it for summarization for the Dutch language the src_lang and tgt_lang will be the same.
This should work:
```python
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
  
tokenizer = AutoTokenizer.from_pretrained("facebook/mbart-large-cc25", src_lang='nl_XX', tgt_lang='nl_XX')

with tokenizer.as_target_tokenizer():
    print(tokenizer(["Hello, this one sentence", "This is another sentence."]))
```
Note: Please cross-check the Dutch language code

> Hi @lysa-n,
> For multilingual models you must define input language(src_lang) and target language(tgt_lang). Since you are using it for summarization for the Dutch language the src_lang and tgt_lang will be the same.
> This should work:
> 
> ```python
> from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
>   
> tokenizer = AutoTokenizer.from_pretrained("facebook/mbart-large-cc25", src_lang='nl_XX', tgt_lang='nl_XX')
> 
> with tokenizer.as_target_tokenizer():
>     print(tokenizer(["Hello, this one sentence", "This is another sentence."]))
> ```
> 
> Note: Please cross-check the Dutch language code

Hi @vishal-burman, 

This seems to work. Thank you so much!
How to kill bad starts when pre-training from scratch
Author: StellaVerkijk
Pull request: no

### Environment info
transformers version: 4.4.3
Platform: linux
Python version: 3.8.5
PyTorch version (GPU?): -
Tensorflow version (GPU?): 2.4.1
Using GPU in script?: yes
Using distributed or parallel set-up in script?: parallel

### Information
Hi!
I am pre-training a RoBERTa model from scratch and was wondering about the possibility of killing bad starts. Because the model will be initiated with random weights when pre-training from scratch, and these initial weights might influence the performance of the final model, I want to do my best to at least not get the worst weight initialization. I have heard that it might be a possibility to calculate perplexity and let that score be decisive of whether to kill the training process or not. Does anyone have experience with how to do this, or does someone have a better idea to review weight initialization and kill bad starts?


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
TypeError: expected str, bytes or os.PathLike object, not NoneType
Author: Decem-Y
Pull request: no


Comments:
from transformers import LongformerModel, LongformerTokenizer, RobertaTokenizer, AutoTokenizer
 
pretrain_model_path = 'schen/longformer-chinese-base-4096'
tokenizer = LongformerTokenizer.from_pretrained(pretrain_model_path)


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
MemoryError: when we run_language_model.py to train an English Adapter
Author: JackyXiangcheng
Pull request: no

run run_language_model.py
https://github.com/Adapter-Hub/adapter-transformers/blob/master/examples/contrib/legacy/run_language_modeling.py
in such codes:

python3 run_language_modeling.py \
--output_dir=/mnt/localdata/cao/output_language_adapter_en/ \
--model_type=bert \
--model_name_or_path=bert-base-multilingual-cased \
--do_train \
--train_data_file=/mnt/localdata/cao/data_for_model/EN_train_updated.txt \
--do_eval \
--eval_data_file=/mnt/localdata/cao/data_for_model/EN_valid.txt \
--mlm \
--language en \
--train_adapter \
--adapter_config pfeiffer \
--per_gpu_train_batch_size 4 \
--per_gpu_eval_batch_size 4 \
--learning_rate 5e-5 \
--dataloader_num_workers 32 \
--cache_dir /mnt/localdata/cao/en_cache_dir/ 

get error:

with open(file_path, encoding="utf-8") as f:
text = f.read()
MemoryError

the train_data_file is around 6GB, I think it is the loading problem.
So how can we load large files?

Comments:
You could try the another one  [run_language_modeling.py](https://github.com/Adapter-Hub/adapter-transformers/blob/master/examples/contrib/legacy/run_language_modeling.py) with `line-by-line` and add a `batch_size` along with `batched=True` to eliminate the memory error:

```
        tokenized_datasets = datasets.map(
            ...
            batched=True,
            batch_size=200,
            ...
        )
```

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Get following error with EncoderDecoder model: TypeError: forward() got an unexpected keyword argument 'use_cache'
Author: mandareln
Pull request: no

Hi

I am trying to create an EncoderDecoder model where i want to use a pre-trained encoder model and initialise decoder from scratch.  Following the code snippet.

----------------------------------------------------------
encoder = AutoModel.from_pretrained('bert-base-uncased')
decoder_config = BertConfig(vocab_size = vocabsize,
                    max_position_embeddings = max_length, 
                    num_attention_heads = num_attention_heads,
                    num_hidden_layers = num_hidden_layers,
                    hidden_size = hidden_size,
                    type_vocab_size = 1,
                    is_decoder=True)    

decoder = BertForMaskedLM(config=decoder_config)
model = EncoderDecoderModel(encoder=encoder, decoder=decoder)
----------------------------------------------------------
Model gets built without any errors but when i try to make a forward pass, i get the error:
TypeError: forward() got an unexpected keyword argument 'use_cache'.

Following is the dummy forward pass function

outputs = model(input_ids=input_ids, decoder_input_ids=input_ids, labels=input_ids)



Comments:
Hi @mandareln 

You should use `BertLMHeadModel` class if you want to use bert as decoder, here you are using `BertForMaskedLM` which is the reason for this error as it does not have the `use_cache` argument.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Deprecate commands from the transformers-cli that are in the hf-cli
Author: LysandreJik
Pull request: yes

Commands that are both in the `transformers-cli` and in the `huggingface-cli` are deprecated here and will be quickly removed.

I'm voting for deprecating them and not removing them even though better ways exist as I suspect some users to use the `transformers-cli` in bash scripts to automatically upload models to the hub.

Context from @julien-c:

> my thoughts is that we should deprecate the subset of transformers-cli command that are in huggingface-cli, as the commands are identical and having both is confusing.
>
> Transformers-specific commands (model conversion, new model templating) can stay in transformers-cli.
>
> What do you think?

Comments:
my reason for removing (still keeping a descriptive error obviously) rather than deprecating is that I'd love to know if people actually use those (and if they do, do they use them in scripts or manually)

But I will defer to the great transformers-maintainers as the final decision 💖

sounds good to me!
Wrong results for GLUE task STS-B
Author: vanessea
Pull request: no

## Environment info
- `transformers` version: 4.6.0.dev0
- Platform: Linux-4.13.0-26-generic-x86_64-with-glibc2.10
- Python version: 3.8.8
- PyTorch version (GPU?): 1.8.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: True
- Using distributed or parallel set-up in script?: False

### Who can help
 @sgugger, @patil-suraj

## Information

Model I am using (Bert, XLNet ...): bert-base-cased

The problem arises when using:
* [ ] the official example scripts:  examples/pytorch/text-classification/run_glue.py

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: STS-B

## To reproduce

export TASK_NAME=stsb

python run_glue.py \
  --model_name_or_path bert-base-cased \
  --task_name $TASK_NAME \
  --do_train \
  --do_eval \
  --max_seq_length 128 \
  --per_device_train_batch_size 32 \
  --learning_rate 2e-5 \
  --num_train_epochs 3 \
  --output_dir /tmp/$TASK_NAME/

## Expected behavior
According to README, Person/Spearman corr. should be close to 88.64/88.48
However, I get Person/Spearman corr. equal to 28.36/27.70

The following warning occurs:
python3.8/site-packages/torch/nn/modules/loss.py:528: UserWarning: Using a target size (torch.Size([32])) that is different to the input size (torch.Size([32, 1])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.
  return F.mse_loss(input, target, reduction=self.reduction)


Comments:
Are you sure you don't have any bad checkpoints in where your `output_dir` points to? Just executed the command and it works perfectly fine on my end. You can add `--overwrite_output_dir` to your command to make sure to overwrite what is there.

> Are you sure you don't have any bad checkpoints in where your `output_dir` points to? Just executed the command and it works perfectly fine on my end. You can add `--overwrite_output_dir` to your command to make sure to overwrite what is there.

I'm sure the problem is not `output_dir`, it points to an empty directory in the beginning. I've rerun the code adding `--overwrite_output_dir` just to make sure, still getting the same results. The results for other GLUE tasks are all correct, only problem with STS-B.
wav2vec 2.0 doesn't appear to do vector quantization
Author: noahtren
Pull request: no

In the [paper](https://arxiv.org/abs/2006.11477) from FAIR, they describe wav2vec 2.0 as using a vector quantization module to learn discrete vectors of speech units (section 2.) As far as I know, this should be happening between `Wav2Vec2FeatureExtractor` and `Wav2Vec2FeatureProjection`.

The HuggingFace implementation doesn't seem to do any vector quantization. Is this a correct implementation?

Comments:
Hi, I think VQ only works for Pretraining, it doesn't look like Transformers currently support Pretrain

Vector quantization is only required for pretraining which is currently not supported. It should be added soon: https://github.com/huggingface/transformers/issues/10873.

Thanks! Just to clarify, was the [base model](https://huggingface.co/facebook/wav2vec2-base) pretrained with quantization, and it's just that the port to HF doesn't include the quantization module?

The port didn't include the quantization module - we should re-port the model :-) 

It was trained with quantization if I remember correctly

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
correct the input_ids value and batch_sentences value.
Author: weiruichen01
Pull request: yes

I found two places with minor typo in transformers/docs/source/preprocessing.rst. 

First, the example output does not match the input in Base use section. "Hello, I'm a single sentence!" should be mapped to [101, 8667, 117, 146, 112, 182, 170, 1423, 5650, 106, 102] rather than [101, 138, 18696, 155, 1942, 3190, 1144, 1572, 13745, 1104, 159, 9664, 2107, 102] which is the original edition. (A quick check is that they do not match in length) 

Second, the example input does not match the output in the same section. For consistency, I changed the sentence to be identical to the example given above it by adding a comma.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.