noamaanMulla-03/transformers-issues huggingface dataset:
Embedding layer Pruning implementation
Author: PhilipMay
Pull request: no

I want to use a multilingual model (in my case MT5) to train it for two languages (English and German).
These models have a very high vocab size because they are multilingual.
This large embedding needs a lot of RAM on GPU.

My idea is: Tokenize a large English and German corpus and find out which tokens are needed.
Then remove all tokens that are not needed. I call this pruning...
This removal needs to be done on the model embedding layer
and on the tokenizer (in my case a sentencepiece tokenizer).

Did anybody ever implement something like that? Is there maybe some example code?
@sgugger @LysandreJik 

If I manage to implement I might provide the functionality as a PR.


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Add EncodedInput as an alternative input type of _batch_encode_plus in tokenization_utils_fast
Author: sz128
Pull request: no

# 🚀 Feature request

The [_batch_encode_plus](https://github.com/huggingface/transformers/blob/6989264963d1f8871404889243d2f15de198ee42/src/transformers/tokenization_utils.py#L483) function in [tokenization_utils](https://github.com/huggingface/transformers/blob/master/src/transformers/tokenization_utils.py) supports to use EncodedInput and EncodedInputPair as inputs. I hope the fast tokenizer can also support that in [tokenization_utils_fast](https://github.com/huggingface/transformers/blob/master/src/transformers/tokenization_utils_fast.py).

## Motivation

It would be more flexible to add EncodedInput and EncodedInputPair as alternative input types for the default fast tokenizer, meanwhile this feature has been implemented in the original tokenizer.


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
How to finetune mT5
Author: laibamehnaz
Pull request: no

I am using mT5 for the task of summarization on a language other than English. But even after training for 30 epochs, the generations are very bad with rouge 1 as 31.5, whereas mBART gives a rouge 1 of 43.1 after training only for 11 epochs. 
I wanted to know if mT5's performance is expected to be like this compared to mBART, or am I doing something wrong. 
Appreciate any help. Thank you :) 


Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discuss.huggingface.co) instead?

Thanks!

> Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
> Could you ask your question on the [forum](https://discuss.huggingface.co) instead?
> 
> Thanks!

Alright. Thank you so much!!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
DistilBERT - Operation reordering for compatibility with TensorRT
Author: novatig
Pull request: yes

This change avoids an `expand_as` operation on a boolean tensor. Improves compatibility with TensorRT 8.0.

@LysandreJik
@mfuntowicz



Comments:
Many thanks @novatig 🙏🏻 💪🏻 

As part of Infinity, this PR enables exporting DistilBERT based models to TensorRT plans for efficient inference as explained by @novatig 

Test failure unrelated and fixed on `master`.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Causal Mask in BertGeneration
Author: JamesHujy
Pull request: no

I cannot find the implementation of CausalMask in BertGeneration. Can you help me locate it?

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

It's here:
https://github.com/huggingface/transformers/blob/cabcc75171650f9131a4cf31c62e1f102589014e/src/transformers/modeling_utils.py#L243
:-)
Note that `is_decoder` has to be set to True 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Fix load_tf_weights alias.
Author: qqaatw
Pull request: yes

# What does this PR do?

1. Address #13154 
2. I'm checking except for Albert, whether other models have the same problem or not.

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@LysandreJik

Comments:
Models having `load_tf_weights` are checked, no bug found!

This looks correct to me!
Improve documentation of pooler_output in ModelOutput
Author: navjotts
Pull request: yes

# What does this PR do?
Improves the doc string for pooler_output in modeling_outputs.py – making it more clear, and opening its availability to a more generic use-case than just BERT-family of models.

**Motivation**:
I was writing a `cls_pooler` for a sentence embeddings usage, and initially thought this is the CLS token output from the last layer – which is not the case, that would just be `last_hidden_state[0]`

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?
@sgugger 

Comments:
Thanks, you just need to run `make style` on your branch to fix the styling issue.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

> Thanks, you just need to run `make style` on your branch to fix the styling issue.

Oh I missed this message @sgugger – just noticed. 
Created a fresh PR – https://github.com/huggingface/transformers/pull/13228
Allow local_files_only for fast pretrained tokenizers
Author: BramVanroy
Pull request: yes

# What does this PR do?

There seems to have been a legacy issue where `local_files_only` did not work for fast tokenizers. I understand that priority focus is given to the environment variable `TRANSFORMERS_OFFLINE` (which did work) but I'd argue that it is best to have such file-related arguments work in the same manner across models, tokenizers, configs. 

This change is quite small. The argument `local_files_only` already existed in `PretrainedTokenizerBase.from_pretrained` (but was not present in the docstring, I added it now) - but it was never passed to `get_fast_tokenizer_file`. This latter function ultimately only skipped online look-up if `is_offline_mode()`. But as discussed above it might be better to include a local argument to control this behaviour in addition to an absolute (environmental) one. This PR makes sure that `local_files_only` has the same effect when loading a slow or fast tokenizer.

Fixes #12571 


## Before submitting
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).


## Who can review?

@n1t0, @LysandreJik


Comments:
BatchFeature should cast to `np.float32` by default
Author: patrickvonplaten
Pull request: no
Labels: name: WIP

Currently the default dtype for Speech Feature Extractors is `numpy.float64` which leads to two problems:

1) It makes the data processing extremely expensive for the RAM. Many sound formats are stored in int16 (such as `.wav`) and are then transformed to float64 which unnecessarly increases RAM by a factor of 4. We should at least stick to `float32`
2) Currently we have added some hacks to the Wav2Vec2 and Speech2TextTransformer feature extractors to prevent Double vs. Float dtype mismatches: https://github.com/huggingface/transformers/blob/f6e254474cb4f90f8a168a599b9aaf3544c37890/src/transformers/models/wav2vec2/feature_extraction_wav2vec2.py#L87

The main problem is that `np.asarray([....])` by default creates a np.float64 array and that we just pass that format along.
=> We should either always cast to float32 in BatchFeature (see here: https://github.com/huggingface/transformers/blob/f6e254474cb4f90f8a168a599b9aaf3544c37890/src/transformers/feature_extraction_utils.py#L151) or add a flag `dtype` to BatchFeature.

@patrickvonplaten 

Comments:
Bump notebook from 6.1.5 to 6.4.1 in /examples/research_projects/lxmert
Author: dependabot[bot]
Pull request: yes
Labels: name: dependencies

Bumps [notebook](http://jupyter.org) from 6.1.5 to 6.4.1.


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=notebook&package-manager=pip&previous-version=6.1.5&new-version=6.4.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/huggingface/transformers/network/alerts).

</details>

Comments:
Disentangle auto modules from other modeling files
Author: sgugger
Pull request: yes

# What does this PR do?

This PR cleans up the auto modules to have them rely on string mappings and dynamically import the model when they are needed, instead of having a hard dependency on every modeling file.

There is no breaking changes are all the MAPPING classes are still present and will behave like regular dictionaries, just loading the objects as needed. On the internal tooling side, this allows us to remove the script that was extracting the names of the auto-mapping (since we have them now) and the file that stored them.

Comments:
Cannot use RemBert
Author: KappalaSaikumar
Pull request: no

![Screenshot (1508)](https://user-images.githubusercontent.com/65230225/130734536-0d498812-2ee4-45fe-96a0-6c33fc007916.png)

When I am trying to use the AutoTokenizer for the newly added RemBert model it's giving me this error

![Screenshot (1509)](https://user-images.githubusercontent.com/65230225/130736371-869e52d9-d4cb-4dc1-a480-8234d658420c.png)

When I try importing RemBertTokenizer then it's giving me this error.

Comments:
Make sure to install Transformers from master: `pip install git+https://github.com/huggingface/transformers.git`

Thanks @NielsRogge It's working fine now
Printing weights of a pre-trained model 
Author: nivi1501
Pull request: no

During the generation of the key matrix, query matrix, and value matrix, what are the weights used? How do I print these weights?

Comments:
In PyTorch, you can easily print out the weights of any model, like so (let's take BERT as an example):

```
from transformers import BertModel

model = BertModel.from_pretrained("bert-base-uncased")

for name, param in model.named_parameters():
      print(name, param.shape)
```
This prints a long list of all parameter names, together with their shape. The keys, values and queries are parameters of each layer of BERT (BERT-base has 12 layers, so there are 12 key, value and query matrices). One of them is the following:

```
encoder.layer.0.attention.self.query.weight torch.Size([768, 768])
encoder.layer.0.attention.self.query.bias torch.Size([768])
encoder.layer.0.attention.self.key.weight torch.Size([768, 768])
encoder.layer.0.attention.self.key.bias torch.Size([768])
encoder.layer.0.attention.self.value.weight torch.Size([768, 768])
encoder.layer.0.attention.self.value.bias torch.Size([768])
```

Yes, we can print the shape and parameter names by using this code. However, I wish to print the whole matrix [768,768] /[768]: (the value of the matrix). There must be some value assigned to this after the training is done.

Just replace `print(name, param.shape)` by `print(name, param)` in the code above.

Yea, that works. Thanks a lot
Replace assert statement with if condition and raise ValueError
Author: nishprabhu
Pull request: yes

# What does this PR do?

The goal of this PR is to replace assert statements with if statements and raise appropriate exceptions (see issue #12789)

Replaces 
```
assert lr_init > lr_end, f"lr_end ({lr_end}) must be be smaller than initial lr ({lr_init})"
```
with 
```
if not (lr_init > lr_end):
    raise ValueError(f"lr_end ({lr_end}) must be be smaller than initial lr ({lr_init})")
```
in optimization.py 


Contributes towards fixing issue #12789


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger 


Comments:
Add config option to skip 1-D position embeddings in LayoutLM
Author: nishprabhu
Pull request: no

# 🚀 Feature request
Add an option in LayoutLM config to not use 1-D position embeddings. The config currently allows us to choose between "absolute", "relative_key", and "relative_key_query". Can we add another option like "none" to not use 1-D positional embeddings?

## Motivation
The input to LayoutLM consists of tokens of text and their corresponding bounding boxes from document images. This is typically obtained by passing the document image through an OCR. 

LayoutLM uses 1-D as well as 2-D position embeddings. While OCRs provide reliable 2-D positions for each word in the document image, the order of words (1-D positions) are not always correct. For example, if we OCR a two column document, or a document containing a table, or any other visually rich document, the order of words in the OCR output is very unreliable. This unreliable position information harms accuracy in several downstream tasks. I have personally seen improvements in some tasks when I manually disable 1-D position embeddings in LayoutLM, and force the model to only look at the 2-D positions. Can we provide an easy way to do this by adding an option in the LayoutLM config to make 1-D position embeddings optional?

## Your contribution
I am willing to work on this and submit a PR, but this is the first time I am contributing to the library and might require some help.


Comments:
I'm not sure whether we should add such an option, because models like BERT, RoBERTa, ... basically all Transformer models within this repository don't have this option. Either we add that option to all, either we don't in my opinion. 

Absolute position embeddings are almost always beneficial, so not sure if adding this will have value, perhaps we could strive for simplicity. cc @sgugger @LysandreJik 

Agreed. Unless there are pretrained checkpoints available that require this kind of change, you should just tweak the code of `modeling_layoutlm` to your needs for this.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
请问一下，情感分析的模型只是针对英文的吗
Author: ArlanCooper
Pull request: no

classifier = pipeline('sentiment-analysis')
这里面封装的模型是只针对英文的吗？我使用中文发现完全不对，比如：
words = '这是一个不错的服务'
judge1 = classifier(words)
words = '这是一个很好的服务'
judge2 = classifier(words)
print(judge1,judge2)

结果展示:
[{'label': 'NEGATIVE', 'score': 0.9500061273574829}] [{'label': 'NEGATIVE', 'score': 0.92015540599823}]

Comments:
This is a bit intimidating to me haha. I'll use deep learning to translate your issue (😮 ):

DeepL says:

classifier = pipeline('sentiment-analysis')
Is the model encapsulated in this only for English? I found it completely wrong using Chinese, e.g.
words = 'this is a good service'
judge1 = classifier(words)
words = 'This is a good service'
judge2 = classifier(words)
print(judge1,judge2)

=> answer: yes, the default sentiment analysis pipeline is English-only, as it uses a `DistilBertForSequenceClassification` model fine-tuned on English data (I'm not sure, Ii's a bummer that it's difficult to know what the default model is that is used for each pipeline, see #12845). You can indeed, as Patrick mentions below, use a custom model from the hub.

You could use the model hub to find sentiment analysis models in Chinese as follows:

https://huggingface.co/models?language=zh&pipeline_tag=text-classification&sort=downloads

And then do:

```python
classifier = pipeline('sentiment-analysis', model="uer/roberta-base-finetuned-chinanews-chinese")
```

> And then do:
> 
> ```python
> classifier = pipeline('sentiment-analysis', model="uer/roberta-base-finetuned-chinanews-chinese")
> ```

thank you  for your answer,that's help a lot

> This is a bit intimidating to me haha. I'll use deep learning to translate your issue (😮 ):
> 
> DeepL says:
> 
> classifier = pipeline('sentiment-analysis')
> Is the model encapsulated in this only for English? I found it completely wrong using Chinese, e.g.
> words = 'this is a good service'
> judge1 = classifier(words)
> words = 'This is a good service'
> judge2 = classifier(words)
> print(judge1,judge2)
> 
> => answer: yes, the default sentiment analysis pipeline is English-only, as it uses a `DistilBertForSequenceClassification` model fine-tuned on English data (I'm not sure, Ii's a bummer that it's difficult to know what the default model is that is used for each pipeline, see #12845). You can indeed, as Patrick mentions below, use a custom model from the hub.

sorry i use chinese, and thank you  for your answer,that's help a lot
Some `model_type`s cannot be in the mapping
Author: LysandreJik
Pull request: yes

Some `model_type`s cannot be in the mapping. This PR offers a fallback for these cases.

The following had stopped working (tested by `test_bert2bert_summarization`):

```
tokenizer = BertTokenizer.from_pretrained("patrickvonplaten/bert2bert-cnn_dailymail-fp16"
``` 

Comments:
Add error message concerning revision
Author: BramVanroy
Pull request: yes

# What does this PR do?

Adds one more item to the error message when a model, config, or tokenizer cannot be loaded. If (and only if) the user provided a revision, the error message will also tell them that they need to check on the model page whether the revision number actually exists.

This does _not_ close issue https://github.com/huggingface/transformers/issues/13264 as using the short commit hash does not work yet. This PR only slightly improves user-friendliness in terms of possible user errors.


## Before submitting
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? https://github.com/huggingface/transformers/issues/13264#issuecomment-905614404


## Who can review?

@LysandreJik

Comments: