noamaanMulla-03/transformers-issues huggingface dataset:
Fix incorrect comments about atten mask for pytorch backend
Author: lygztq
Pull request: yes

# What does this PR do?

I found these comments do not match the actual behavior. In these comments we use `-10000.0` to mask out elements before `softmax` but the actual value used is `torch.finfo(dtype).min`.


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.


Comments:
_The documentation is not available anymore as the PR was closed or merged._

@lygztq Great catch. This inconsistency was introduced in (my) PR #17306, where the docstrings were not updated - my bad.

I think it's more clear to change from `... the smallest value ...` to `... the dtype's smallest value ...`. WDYT?

> I think it's more clear to change from `... the smallest value ...` to `... the dtype's smallest value ...`. WDYT?

I agree, this is better.
T5base loss function 
Author: KevinZhoutianyi
Pull request: no

Hi!

I am now using the T5-base pretrained model for translation. For the loss function, it seems that you did not get rid of the loss
for the <padding> token. (ie. you did not use ignore index in cross entropy loss). 

I used model(x_id,x_attn,y_id,y_attn).loss to get the loss from the pre-trained model. It seems that the value is the same as the crossentropy loss without ignore padding as followed.

```
#model loss
logits = temp.logits
model_loss = temp.loss
print('model_loss',model_loss)

#my loss 
_criterion= torch.nn.CrossEntropyLoss( reduction='none')# **without ignoring padding**
loss_seq = self._criterion(logits.view(-1,logits.shape[-1]), target_ids.view(-1)).view(batch_size,-1)
loss_vec = torch.mean(loss_seq ,-1).squeeze()
print('mylossvecmean',loss_vec.mean())
```
Btw, is there any example for fine-tuning T5 for translation? I failed to achieve the BLEU they showed in the paper.

Thanks!

Comments:
Hey @KevinZhoutianyi,

Sorry I don't fully here exactly what the problem is. Could you add a reproducible code snippet that shows the problem? 

Regarding fine-tuning T5 for translation, there are some examples on the forum: https://discuss.huggingface.co/ I believe.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

I have also noticed that, and still no idea why the implemented loss function of T5 (and BART) does not set "ignore_index" ( [nn.CrossEntropyLoss](https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html)). I think PAD token should be ignored when computing loss.

Update: I guess I have found the reason.   The implementation uses a default setting of "nn.CrossEntropyLoss" function, where the "ignore_index" is set to -100.  So, when computing the loss, you need to convert the labels pad index, shown as below:

![Screen Shot 2022-09-25 at 20 07 03](https://user-images.githubusercontent.com/27895653/192140447-d3b2994c-fa9d-441a-a3da-6f8f36689648.png). 

See: [https://huggingface.co/docs/transformers/model_doc/t5#transformers.T5Model](https://huggingface.co/docs/transformers/model_doc/t5#transformers.T5Model
)
Error using inputs_embeds argument in TFXLNetModel
Author: rojagtap
Pull request: no
Labels: name: wontfix

While using the TFXLNetModel:
`xlnet = TFXLNetModel.from_pretrained('xlnet-base-cased')`
according to the docs, `input_ids` and `inputs_embeds` can be optionally used. However, when I tried using:

`xlnet(inputs_embeds=embeddings, attention_mask=attn_masks)[0]`
it throws: `ValueError: The first argument to Layer.call must always be passed.`

which I thought is an issue with the `inputs` argument which must be a positional one:
`xlnet(inputs=None, inputs_embeds=embeddings, attention_mask=attn_masks)[0]` using this gave me:
`RuntimeError: Attempting to capture an EagerTensor without building a function.`

And finally passing both `inputs` and `inputs_embeds` gave: `ValueError: You cannot specify both input_ids and inputs_embeds at the same time`

Can someone suggest a workaround on this?
P.S. the `embeddings` variable is the `last_hidden_state` from another bert which matches the config for the `inputs_embeds` shape.
Note that `input_ids` parameter won't count as it gave the same error if I didn't use the `inputs` argument.

Comments:
Hey, @patrickvonplaten  I observed the same with TFBertModel. So pretty much evident that the issue must be with the parent (if at all that helps)

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


> 
> 
> While using the TFXLNetModel:
> `xlnet = TFXLNetModel.from_pretrained('xlnet-base-cased')`
> according to the docs, `input_ids` and `inputs_embeds` can be optionally used. However, when I tried using:
> 
> `xlnet(inputs_embeds=embeddings, attention_mask=attn_masks)[0]`
> it throws: `ValueError: The first argument to Layer.call must always be passed.`

try something like this:
xlnet({'attention_mask':attention_mask, 'token_type_ids':token_type_ids},inputs_embeds=embeddings, training=training)

worked for me when getting the same error for a bert model


```python
model_outputs = self.transformer(
    input_ids=None,  # add this line
    inputs_embeds=dense_feature,
    attention_mask=attention_mask
)
```

Is this still relevant? Gently pinging @gante @Rocketknight1 here 

I got around the issue by changing the code from this
```
config = MobileBertConfig()
mbert = TFMobileBertModel(config)
mbert(inputs={"input_ids":input_ids, "attention_mask":attention_mask})
```
to 
```
config = MobileBertConfig()
mbert = TFMobileBertModel(config)
mbert(input_ids=input_ids, attention_mask=attention_mask)
```

transformers version : 4.22.1
Separate Push CI images from Scheduled CI
Author: ydshieh
Pull request: yes

# What does this PR do?

⚠️ **Before merge, I need to build the new push CI images that have the new tags.** 

Currently, if `setup.py` is changed, Push CI will re-build the CI images before running tests.
https://github.com/huggingface/transformers/blob/7e84723fe4e9a232e5e27dc38aed373c0c7ab94a/.github/workflows/self-push-caller.yml#L39-L43

However, this may cause different jobs in a scheduled CI workflow run to use images with different versions.

Recently, when `tokenizers` is changed to `0.13`, some jobs failed in the scheduled CI due to the new image (with `tokenizers 0.13`) but the `transformers` code in those runs still required `tokenizers < 0.13`.

**This PR separates the push CI images from scheduled CI.**


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Added in `Transformers testing design, internal document` on Notion.


<img width="545" alt="Screenshot 2022-09-23 191436" src="https://user-images.githubusercontent.com/2521628/192018244-918525d6-f9a8-4077-a8a1-e1ccc2b47a3b.png">

### Text version

The CI for a push event (to main branch) will check if setup.py is changed. If yes, it will launch the docker image build CI before launching the actual tests. This is to make sure the tests will run against the specified package versions in setup.py. In order to avoid the conflict with the daily schedule CI, which should use the same image version for all jobs during a workflow run, we separate the CI images used for push events and schedule CI. The docker images used for push events start with the tag of the images used in the corresponding jobs in scheduled CI, but with a postfix push-ci. For example, transformers-all-latest-gpu in schedule CI will be transformers-all-latest-gpu-push-ci in push CI.
OPT vocab size of model and tokenizer does not match
Author: dhansmair
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.19.2
- Platform: Linux-5.4.0-72-generic-x86_64-with-glibc2.29
- Python version: 3.8.10
- Huggingface_hub version: 0.7.0
- PyTorch version (GPU?): 1.11.0+cu113 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: no


### Who can help?

@LysandreJik

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('facebook/opt-350m')
tok = AutoTokenizer.from_pretrained('facebook/opt-350m', use_fast=False)
    
print(model.config.vocab_size)        # 50272
print(tok.vocab_size)                      # 50265
```


### Expected behavior

Hello,
I'm not sure whether this is a bug or if I am missing something.
In the reproduction script above, the model has a bigger vocabulary than the tokenizer. In my project, the LM produces the token `50272`, which the tokenizer doesn't know and thus the decode() function fails. 
(I use my own text generation script, so is it by any chance that the model is not supposed to output the last 7 tokens that the tokenizer doesn't know?)

Best, David

Comments:
cc @ArthurZucker 

Hey, thanks for noticing this! I am gonna add @younesbelkada  to the loop. 
It seems that the original tokenizer [vocabulary](https://github.com/facebookresearch/metaseq/blob/main/projects/OPT/assets/gpt2-vocab.json) has 50264 words, with some "madeupwords". Let us have a look! 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Duplicate of https://github.com/huggingface/transformers/issues/17431#issuecomment-1224231170

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
PerceiverAudioPreprocessor: forward() missing 1 required positional argument: 'pos'
Author: basilevh
Pull request: no
Labels: name: Good First Issue

The [`forward()` method implementation of `PerceiverAudioPreprocessor`](https://github.com/huggingface/transformers/blob/38cc35069c10d153e872162265288263bb7394b7/src/transformers/models/perceiver/modeling_perceiver.py#L3267) seems to be problematic because it has an extra required `pos` argument, whereas all other preprocessors do not (or it is optional). In fact, while it is passed to `_build_network_inputs()`, `pos` is not actually used in that method. Hence, I am not sure why it exists. Moreover, it breaks the default `PerceiverModel` operation, because that class assumes that there only one positional argument called `inputs` [when calling the preprocessor](https://github.com/huggingface/transformers/blob/38cc35069c10d153e872162265288263bb7394b7/src/transformers/models/perceiver/modeling_perceiver.py#L865). This seems to be a serious issue in my opinion because it crashes any instantiation of `PerceiverModel` with a `PerceiverAudioPreprocessor` as `input_preprocessor`.

Comments:
Hi,

Thanks for raising this issue. This was a leftover from porting Deepmind's original repo. However, as I'm implementing the position embeddings differently in PyTorch (vs. the original implementation which was in Haiku), this can probably be deleted. I'll work on a PR to remove it everywhere.

Reopening this and marking it as a good first issue.

Is this issue open? I'd love to take it on.
Remove pos arg from Perceiver's Pre/Postprocessors
Author: aielawady
Pull request: yes

Fix #15971
@NielsRogge 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Use `assertAlmostEqual` in `BloomEmbeddingTest.test_logits`
Author: ydshieh
Pull request: yes

# What does this PR do?

`BloomEmbeddingTest.test_logits` currently uses `assertEqual` to compare 2 floats. We should instead use `assertAlmostEqual`.

Currently we don't see this test failing. However, with different PyTorch versions, we might get test failure - this happens for Past CI (PyTroch 1.10), where we got
```bash
AssertionError: 1.9311904907226562e-05 != 1.9431114196777344e-05
```





Comments:
_The documentation is not available anymore as the PR was closed or merged._
Benchmarking text generations
Author: hadifar
Pull request: no

### Feature request

Currently, we can benchmark Bert or Roberta with `PyTorchBenchmark`. I am just wondering, can we do the same thing for text-generation models, e.g., T5?

```
from transformers import PyTorchBenchmark, PyTorchBenchmarkArguments
config = AutoConfig.from_pretrained('t5-base')
benchmark = PyTorchBenchmark(args, configs=[config], batch_sizes=[1,2,4, 8], sequence_lengths=[128,256, 512])

```

The above code throws the following exception:
```

ValueError                                Traceback (most recent call last)
<ipython-input-6-df0caab4d791> in <module>
----> 1 results = benchmark.run()
      2 print(results)

/usr/local/lib/python3.7/dist-packages/transformers/benchmark/benchmark_utils.py in run(self)
    708                     if self.args.inference:
    709                         if self.args.memory:
--> 710                             memory, inference_summary = self.inference_memory(model_name, batch_size, sequence_length)
    711                             inference_result_memory[model_name]["result"][batch_size][sequence_length] = memory
    712                         if self.args.speed:

ValueError: too many values to unpack (expected 2)
```

### Motivation

It would complete the benchmark API.

### Your contribution

I can submit an initial PR for benchmarking the text generation models.

Comments:
Already discussed here: https://github.com/huggingface/transformers/issues/15512
Move the model type check in DocumentQuestionAnswering to support Donut
Author: ankrgyl
Pull request: yes

# What does this PR do?

Prior to this change, you'd see an error while instantiating a pipeline with Donut:

```
In [3]: pipe = pipeline(task="document-question-answering", model='naver-clova-ix/donut-base-finetuned-docvqa')
The model 'VisionEncoderDecoderModel' is not supported for document-question-answering. Supported models are ['LayoutLMForQuestionAnswering', 'LayoutLMv2ForQuestionAnswering', 'LayoutLMv3ForQuestionAnswering'].
```

because it's not part of `AutoModelForDocumentQuestionAnswering`. I've moved around that check so that it does not apply to the Donut case.

Fixes #18926


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@NielsRogge @Narsil

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Use repo_type instead of deprecated datasets repo IDs
Author: sgugger
Pull request: yes

# What does this PR do?

In the next release of `huggingface_hub` trying to access any repo ID with the `"datasets/"` prefix will fail. `repo_type="dataset"` must be used instead.
This PR fixes all instances except for research projects.

Comments:
Research projects are examples we do not actively maintain and pin all dependencies to a specific version.

_The documentation is not available anymore as the PR was closed or merged._
TAPAS model usage issue
Author: deep-mining-swang
Pull request: no
Labels: name: bug

### System Info

Hi, 

I have some issues when I am using TAPAS. 
`from transformers import TapasConfig, TapasForQuestionAnswering  `
report following errors:
RuntimeError: Failed to import transformers.models.tapas.modeling_tapas because of the following error (look up to see its traceback):
module 'distutils' has no attribute 'version'

After I do `pip install setuptools==59.5.0`
it just returned following error:
Segmentation fault (core dumped)

Here is my env
[transformers](https://pypi.python.org/pypi/transformers)==4.20.1
[evaluate](https://pypi.python.org/pypi/evaluate)==0.1.2
[bert-score](https://pypi.python.org/pypi/bert-score)==0.3.11
[datasets](https://pypi.python.org/pypi/datasets)==2.3.2
[accelerate](https://pypi.python.org/pypi/accelerate)
[deepspeed](https://pypi.python.org/pypi/deepspeed)==0.6.5
[wordninja](https://pypi.python.org/pypi/wordninja)==2.0.0
[sacrebleu](https://pypi.python.org/pypi/sacrebleu)==2.1.0
[fasttext](https://pypi.python.org/pypi/fasttext)==0.9.2
[nltk](https://pypi.python.org/pypi/nltk)==3.7
[scikit-learn](https://pypi.python.org/pypi/scikit-learn)==1.0.2

### Who can help?

@NielsRogge

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

from transformers import TapasConfig, TapasForQuestionAnswering

### Expected behavior

Segmentation fault (core dumped)



Comments:
This is likely an error due to a mismatch of versions between your Torch and TorchScatter installations

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Closing this as we've answered the question, feel free to re-open if you still have this issue
`MCTCTFeatureExtractor` requires `torchaudio >= 0.10`
Author: ydshieh
Pull request: yes

# What does this PR do?

In past CI with PyTorch 1.9, we got for `MCTCTFeatureExtractor` 
```bash
AttributeError: module 'torchaudio.functional' has no attribute 'melscale_fbanks'
```

This PR only adds a warning in `feature_extraction_mctct`, so we have this information tracked. Let's discuss if we should skip the corresponding tests.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Unable to instantiate tokenizer with `TRANSFORMERS_OFFLINE=1`
Author: SeanNaren
Pull request: no
Labels: name: bug

Just some context, we use `TRANSFORMERS_OFFLINE=1` in the NeMo CI to ensure we load from the local cache. With the latest transformers version we noticed this bug in our CI!

### System Info

- `transformers` version: 4.22.1
- Platform: Linux-5.15.0-41-generic-x86_64-with-glibc2.31
- Python version: 3.9.12
- Huggingface_hub version: 0.9.1
- PyTorch version (GPU?): 1.12.1+cu113 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help?

@SaulLu 


### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Create this script `reprod.py`:

```python
from transformers import AutoTokenizer

AutoTokenizer.from_pretrained(pretrained_model_name_or_path='gpt2')
```

run:

```
python reprod.py
TRANSFORMERS_OFFLINE=1 python reprod.py
```

First one runs successfully, second one fails:

```
Traceback (most recent call last):
  File "/home/snarenthiran/NeMo/reprod.py", line 3, in <module>
    AutoTokenizer.from_pretrained(pretrained_model_name_or_path='gpt2')
  File "/home/snarenthiran/anaconda3/lib/python3.9/site-packages/transformers/models/auto/tokenization_auto.py", line 549, in from_pretrained
    tokenizer_config = get_tokenizer_config(pretrained_model_name_or_path, **kwargs)
  File "/home/snarenthiran/anaconda3/lib/python3.9/site-packages/transformers/models/auto/tokenization_auto.py", line 418, in get_tokenizer_config
    commit_hash = extract_commit_hash(resolved_config_file, commit_hash)
  File "/home/snarenthiran/anaconda3/lib/python3.9/site-packages/transformers/utils/hub.py", line 225, in extract_commit_hash
    search = re.search(r"snapshots/([^/]+)/", resolved_file)
  File "/home/snarenthiran/anaconda3/lib/python3.9/re.py", line 201, in search
    return _compile(pattern, flags).search(string)
TypeError: expected string or bytes-like object
```

### Expected behavior

To create the tokenizer from the local files.

Comments:
Fix cached_file in offline mode for cached non-existing files
Author: sgugger
Pull request: yes

# What does this PR do?

In offline mode, we sometimes returned the `_CACHED_NO_EXIST` constant instead of `None` for non-existing objects, since we now cache that non-existence. This PR fixes it.

Fixes #19186 

Comments:
Thank you for such a quick fix!

_The documentation is not available anymore as the PR was closed or merged._