noamaanMulla-03/transformers-issues huggingface dataset:
Add attention-mask support for ViTModel
Author: Ryan-Qiyu-Jiang
Pull request: yes

# What does this PR do?

Transformers currently only support head-masks, however recent works
in vision language modeling have used the ViTModel with attention-masks.
Adding attention-masks to ViTSelfAttention and ViTModel allow
transformers modeling_vit to be used in implementing these models.

This commit adds attention-mask implementations from BertSelfAttention
as the forward param attention_mask. This change preserves the existing
behavior by default.

A specific example of how this change will be useful is being able to
implement ViLT https://arxiv.org/abs/2102.03334, using ViTModel as the main trunk.

Unit-tests seemed to pass, though there wasn't a unit-test for head_masks,
so I just added a check for attention_mask in the signature.
Thanks!


src/transformers/models/vit/modeling_vit.py @LysandreJik, @sgugger 


Comments:
@NielsRogge reordered params!

Thanks. I think this should also be added for the Flax implementation of ViT, for consistency.

It could perhaps also be added for BEiT.

@NielsRogge added to flax_vit, beit, and flax_beit.

Thanks. Can you also edit the corresponding tests? In `test_modeling_vit.py` (and similarly, in test_modeling_deit.py, test_modeling_beit.py and the flax models), we can add the `attention_mask` to the `prepare_config_and_inputs` method, such that the `attention_mask` is also used in all tests.

@Ryan-Qiyu-Jiang the shape of the `attention_mask` provided to these models is equal to (batch_size, seq_len), with the sequence length equal to the number of patches + 1 for the [CLS] token (and + 2 for DeiT, as this model also adds a special distillation token to the sequence).

This should also be added to the docstrings of the respective models. 

Hi @Ryan-Qiyu-Jiang, after working on another PR (#13874), I now wonder what the use is of an `attention_mask` being defined for vision models. Is this for the case where images in a batch are padded, and one doesn't want to take into account the `attention_scores` for padding pixels?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
How to use transformers pipeline with multi-gpu?
Author: Deep-sea-boy
Pull request: no

`ner_model = pipeline('ner', model=model, tokenizer=tokenizer, device=0, grouped_entities=True)`

the device indicated pipeline to use no_gpu=0(only using GPU), please show me how to use multi-gpu.

Comments:
same question...

Currently this information is not supported.

You could to `n pipelines` on the `n` devices as a workaround, would that work ?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
[M2M100Tokenizer] fix _build_translation_inputs
Author: patil-suraj
Pull request: yes

# What does this PR do?

The `_build_translation_inputs` method of `M2M100Tokenizer` hardcodes the `return_tensors` argument, but it's also passed as a `extra_kwargs` in translation pipeline. So the pipeline fails with the error

```TypeError: M2M100Tokenizer object got multiple values for keyword argument 'return_tensors'```

This PR adds `return_tensors` parameter to `_build_translation_inputs`. 


Comments:
Raise exceptions instead of using asserts  in modeling_openai #12789
Author: nbertagnolli
Pull request: yes

# What does this PR do?
Replaces control flow assertions in modeling_openai.py to address issue #12789.

Contributes to #12789

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. [link](https://github.com/huggingface/transformers/issues/12789)
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
Tagging @sgugger I saw you commenting on the issue and some other PRs for this issue : ).

Comments:
Thanks for your help on this!
Getting out of vocabulary tokens for pretrained models.
Author: armintabari
Pull request: no

Is there a way to get the out of vocabulary tokens? For example to add them to the tokenizer later on?

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
How to export to ONNX distilbert with question-answering head
Author: ankandrew
Pull request: no

I am having a little bit of trouble exporting to ONNX: `distilbert` with `question-answering` head.

### Code

I convert to ONNX according to [docs](https://huggingface.co/transformers/serialization.html#converting-an-onnx-model-using-the-transformers-onnx-package):

```shell
# Convert to ONNX
!python -m transformers.onnx --model='mrm8488/distill-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es' --feature question-answering 'onnx_exported'
```

But `--feature question-answering ` seems like is not an acceptable feature.

Is this expected?

### Versions

```
transformers	4.11.2
Python		3.7.12
```

### Inquiry

The reason I am doing this is to run pipeline with `onnx` and get extra juice out of it (performance). Is there a better way than this manual export approach?

Comments:
I don't think the `question-answering` head is implemented, cc @michaelbenayoun 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
[Feature] Add an option to use custom pad token_id in `tokenizer.pad()`
Author: qqaatw
Pull request: no

# 🚀 Feature request

Adds an option to use custom pad token_id in `tokenizer.pad()`. If this option is not specified, using tokenizer-level `pad_token`.

Here is an example below::
```
input = {"input_ids": [[5, 2], [3]]}
padded = tokenizer.pad(input, pad_id=-100)

#Output: {'input_ids': [[5, 2], [3, -100]], 'attention_mask': [[1, 1], [1, 0]]}
```

## Motivation

Since `tokenizer.pad()` is a public function that can be used not only for regular tokenization but padding existing token ids, adding this option can enable additional use cases. For example, the `labels` of `BertForMaskedLM` accepts `-100` as the masked loss position, and we can thereby use the custom token_id to pad positions we don't want to calculate.

## Your contribution

I can open a PR for it.

cc @LysandreJik 


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
[doc] performance and parallelism updates
Author: stas00
Pull request: yes

This PR:

- updates the performance doc to break down all the memory used by the model (+ mentioned `bitsandbytes` which saves 3/4 optim memory)
- updates the parallelism doc to introduce Varuna and expand on Sagemaker model parallelism solutions - both published a paper just recently

@sgugger 

Comments:
Please don't remove `prepare_seq2seq_batch` in future versions
Author: zartdinov
Pull request: no

# 🚀 Feature request

I think we can just add some extra attribute like `pad_token_id`=-100 to ignore padding in the loss

## Motivation
It can be easily and cleanly used with `datasets` library now:

```python
encoded_train_dataset = train_dataset.map(
    lambda batch: tokenizer.prepare_seq2seq_batch(
        batch['text'], batch['summary'], padding='max_length', truncation=True, max_length=256, max_target_length=64
    ),
    batched=True,
    remove_columns=train_dataset.column_names,
)
```
Instead of writing every time custom preprocess function like here:
https://github.com/huggingface/transformers/blob/9adff7a0f49f88a6cc718a1d30088988dc78bb6a/examples/pytorch/translation/run_translation.py#L407

@sgugger @patrickvonplaten @patil-suraj 

Comments:
Add support for WMT21 tokenizer in M2M100Tokenizer
Author: patil-suraj
Pull request: yes

# What does this PR do?

The tokenizer for WMT21 translation models is similar to `M2M100Tokenizer` with the only difference being it uses different language codes. 

This PR adds support for wmt21 tokenizers in `M2M100Tokenizer` by adding the `language_codes` attribute which specifies what language codes to use.

Comments:
Support OpenNMT models
Author: jordimas
Pull request: no
Labels: name: New model

It will be great if OpenNMT (https://opennmt.net/) and CTranslate2 (https://github.com/OpenNMT/CTranslate2) model support is provided out of the box. 

Comments:
Hi @jordimas !

Would you be interested in adding this model to transformers? I briefly looked at the code and it looks similar to [mbart](https://github.com/huggingface/transformers/blob/master/src/transformers/models/mbart/modeling_mbart.py)/[m2m](https://github.com/huggingface/transformers/blob/master/src/transformers/models/m2m_100/modeling_m2m_100.py)/[Marian](https://github.com/huggingface/transformers/blob/master/src/transformers/models/marian/modeling_marian.py) style models in transformers. So it should be fairly straightforward to port this by looking at the design of these models.

As you might already know, each model in the library requires 3 files, which will look something like this:
- `configuration_open_nmt.py`
- `modeling_open_nmt.py`
- `tokenization_open_nmt.py`


We provide a template using CookieCutter which lets you set up these files for you, even filling in the names, as explained [here](https://github.com/huggingface/transformers/tree/master/templates/adding_a_new_model). It also creates documentation pages, test files, and so on.

I would be happy to help with this, so feel free to ping me if you have any issues. Thank you :)
run_summarization.py - num_update_steps_per_epoch calculation
Author: bpraveenk
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.11.0.dev0
- Platform: Linux-5.4.0-1055-azure-x86_64-with-glibc2.10
- Python version: 3.8.1
- PyTorch version (GPU?): not installed (NA)
- Tensorflow version (GPU?): 2.5.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Disributed
### Who can help
 @Rocketknight1, @patrickvonplaten, @patil-suraj

**This is not a bug/issue specific to a model**  

Could some one help me understand why the number of update steps is 

        `num_update_steps_per_epoch = len(train_dataset) // training_args.per_device_train_batch_size`

and not:

        `num_update_steps_per_epoch = len(train_dataset) //  total_train_batch_size`

Since the batch_size is: total_train_batch_size =  training_args.per_device_train_batch_size * num_replicas


The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: run_summarization.py

## To reproduce



Comments:
You are referring to this line here no? https://github.com/huggingface/transformers/blob/29dfb2dbb10cdba6327ff287db56b182c1db29b1/examples/tensorflow/summarization/run_summarization.py#L580 

So it's the tensorflow examples I assume. 

@Rocketknight1 - could you take a look here. I agree with @bpraveenk that  `num_update_steps_per_epoch = len(train_dataset) //  total_train_batch_size` would make more sense

@bpraveenk You are correct that this is a bug. I'll submit a PR to fix it soon, but in general this example is a little out-of-date and due to be partially rewritten. In the mean time if you want a more up-to-date TensorFlow summarization example, take a look at [this notebook](https://github.com/huggingface/notebooks/blob/master/examples/summarization-tf.ipynb).

Thank you @Rocketknight1
Allow per-version configurations
Author: LysandreJik
Pull request: yes

Similarly to https://github.com/huggingface/transformers/pull/12713, this allows per-version configurations. This is necessary for LayoutXLM, which up to now was using the configuration-defined `XLMRobertaTokenizer`, but which should now use the `LayoutXLMTokenizer`.

Updating the configuration would mean breaking all previous versions of `transformers` that were using LayoutXLM. Not updating this parameter means that LayoutXLM will never benefit from `LayoutXLMTokenizer` through the `AutoTokenizer` API.

Resolves https://github.com/huggingface/transformers/issues/14275

This implements similar tests to the tokenizer, but instead of using `bert-base-cased`, it uses the actual model that is at issue (`microsoft/layoutxlm-base`). This model should continue using the `XLMRobertaTokenizer` until a new minor version is released, as the configuration I uploaded is named `config.4.13.0.json`: https://huggingface.co/microsoft/layoutxlm-base/blob/main/config.4.13.0.json

Comments:
Get multiple results from Hugging face pipeline library
Author: BatMrE
Pull request: no

I am using simple transformers library to get auto suggested text for my question based on a context .
It give me a single suggestion, is there any possible way to get multiple results for the same.
    
    text2text_generator = pipeline("text2text-generation")
    text2text_generator("question: What is ? context: 42 is the answer to life, the universe and everything")

The "text2text-generation" model uses T5 model on its backend.

I tried something like:

```
from transformers import T5Tokenizer, T5Model
tokenizer = T5Tokenizer.from_pretrained('t5-small')

model = T5ForConditionalGeneration.from_pretrained('t5-small')
input_ids = tokenizer("question: What is ? context:42 is the answer to life, the universe and everything", return_tensors="pt").input_ids  # Batch size 1

outputs = model.generate(input_ids,do_sample=True, top_k=5)
tokenizer.decode(outputs[0])
```

nothing works.!

Comments:
Hello! The `Text2TextGenerationPipeline` accepts any keyword arguments to be handled by the `generate` method that does the generation under the hood. You can check the input signature of that method here to see what arguments it accepts: [generate](https://huggingface.co/transformers/main_classes/model.html#transformers.generation_utils.GenerationMixin.generate)

Namely, it accepts the `do_sample` argument. You can try it out:

```py
>>> from transformers import pipeline
>>> text2text_generator = pipeline("text2text-generation")

>>> text2text_generator("question: What is ? context: 42 is the answer to life, the universe and everything", do_sample=True, min_length=15)
[{'generated_text': '42 also is the answer to life, the universe and everything but the universe'}]

>>> text2text_generator("question: What is ? context: 42 is the answer to life, the universe and everything", do_sample=True, min_length=15)
[{'generated_text': '42 is the answer to life, the universe and everything but the universe'}]

>>> text2text_generator("question: What is ? context: 42 is the answer to life, the universe and everything", do_sample=True, min_length=15)
[{'generated_text': 'The answer to life, the universe and everything to everybody. 42 is the answer'}]
```

Great, that adds a lot of value to output. Thanks @LysandreJik 

Can I get multiple results in a single generation?
I think it may be a bug for the  num_return_sequences just take effect in 

`
model_outputs = self.forward(model_inputs, **forward_params)
`

but in 

`
outputs = self.postprocess(model_outputs, **postprocess_params)
`

the decode always return one result

`
    record = {
        f"{self.return_name}_text": self.tokenizer.decode(
            model_outputs["output_ids"][0],
            skip_special_tokens=True,
            clean_up_tokenization_spaces=clean_up_tokenization_spaces,
        )
    }
`
Fixed a bug for num_return_sequences don't take effect in Text2TextGenerationPipeline
Author: nursery42
Pull request: yes

# Fixed a bug for num_return_sequences don't take effect in Text2TextGenerationPipeline.
The previous postprocess function always returns one result. This leads to num_return_sequences doesn't take effect in Text2TextGenerationPipeline and only can get one text result, no matter how we change the parameters.
We can get multiple text results in the postprocess method and get multiple results from Text2TextGenerationPipeline.



<!-- Remove if not applicable -->

Fixes https://github.com/huggingface/transformers/issues/13027#issuecomment-969899988


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.




Comments:
FX helper functions and quick fixes
Author: michaelbenayoun
Pull request: yes

# What does this PR do?

This PR adds small helper functions and fixes small issues regarding `torch.fx` symbolic tracing for models from the library.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Should this be merged @michaelbenayoun?

No, this is not up to date anymore.
Closing this!
how to config gpu for run_text_classification?
Author: reneix
Pull request: no

I'm using example script for a text classfication，it works well on my laptop without gpu, but not using gpu in a docker environment.
would you please help on it?

nvidia-smi returns: 
No running processes found

example script:
https://github.com/huggingface/transformers/tree/master/examples/tensorflow/text-classification
version:
tf 2.4.1
cudnn 10.0
transformers 4.12.3


Comments:
Hi,

As stated in the [README](https://github.com/huggingface/transformers/tree/master/examples/tensorflow/text-classification#multi-gpu-and-tpu-usage):

> By default, the script uses a MirroredStrategy and will use multiple GPUs effectively if they are available. TPUs can also be used by passing the name of the TPU resource with the --tpu argument.

So maybe you can first check if Tensorflow [recognizes the GPU](https://stackoverflow.com/questions/38009682/how-to-tell-if-tensorflow-is-using-gpu-acceleration-from-inside-python-shell).

thanks, seem tf2.4 need cuda11, I will check it first
Experimenting with adding proper get_config() and from_config() methods
Author: Rocketknight1
Pull request: yes

Should fix issues with saving/loading Keras models containing Transformers models as layers, among other problems.

Comments:
It would be ideal to add tests to ensure that the model's serialization works correctly.

Just a small comment from my side - I think I stumbled across the problem that the returned get_config was not fully JSON serializable (as it contained a reference to a tf DType class) - I circumvented this by adding a custom JSONEncoder for it; but it may be worthwhile testing for json.dumps(config) as well?

@Zahlii That's a great idea! Would you be willing to write a test for it, or modify the existing test, and tag me for review? 

See https://github.com/huggingface/transformers/pull/14415