noamaanMulla-03/transformers-issues huggingface dataset:
Documentation: Dataset to Model interface examples
Author: iandanforth-alation
Pull request: no

# 🚀 Feature request

Add explicit examples to the custom dataset docs which use fully realized key-value pairs and link to the forward methods of the relevant models to emphasize the contract between Dataset `__getitem__()` output and model `forward()`.


## Motivation

This issue arises when you consider the interaction of Datasets and Models. They may be independently well documented but when trying to use them together there are gaps.

Example:

You want to fine tune T5 using `Trainer` using a custom `Dataset`.

The minimal API for T5 a user needs to be aware of in this scenario is small but not well documented in the context of custom datasets. The `Dataset` must return an object in this format from it's `__getitem__()` method.

```
        return {
            'input_ids': input_ids, 
            'attention_mask': attention_mask, 
            'labels': labels,
        }
```

Those three keys are then passed by `Trainer` to T5 via its forward method. Arguably those parameters are the most important first interface you need to know about to train T5 (inside or outside of `Trainer`). 

This *is* documented in the T5 docs, however the key disconnect is that in the custom dataset docs none of the examples use explicit, fully realized set of key-value pairs. Also the docs don't emphasize that those return values *must match* the expected inputs to a model's `forward()` method.

## Your contribution

I can add these if this get's a few thumbs up.


Comments:
The custom dataset doc page is outdated and will be rewritten soon, you should use the [examples scripts](https://github.com/huggingface/transformers/tree/master/examples) or [example notebooks](https://huggingface.co/transformers/notebooks.html) as a base to fine-tune models.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Validation and Evaluation not cumputed in run_qa.py
Author: mamoon115
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.10.0.dev0
- Platform: ubuntu
- Python version:3.8

Models:

- roberta

Library:
- trainer: @sgugger
- pipelines: @LysandreJik


## Information

Model I am using  is Roberta.

* while running run_qa.py on squad 2.0 data there is missing training metrics, validation, and evaluation loss. 
* There is training loss and evaluation metric display after finetuning.

The tasks I am working on is:
* an official QuestionAnswer/Squad task



Comments:
What command did you use to run the script? Setting `--eval_strategy epoch` for instance will give you the evaluation every epoch.

@sgugger Yes. 
At the time of evaluation [loss](https://github.com/huggingface/transformers/blob/75b8990d9068a2c6ef448c190f2595c17fbcb993/src/transformers/trainer.py#L2206) is empty

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Bart Generation
Author: Hannibal046
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.9.0
- Platform:Linux
- Python version:3.7
- PyTorch version (GPU?):1.9.0
- Using GPU in script?:true
- Using distributed or parallel set-up in script?:false

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.-->

@patrickvonplaten 
@patil-suraj 
@sgugger 
@patil-suraj



## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [x] the official example scripts:run_summarization.py
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: summarization
* [ ] my own task or dataset: (give details below)

### Question 1
In `src/examples/pytorch/summarizations/run_summarization.py`. I choose Bart as my model.

since it uses `BartTokenzier` and `DataCollatorForSeq2Seq`. So the labels to the datacollator is `<bos> summarization <eos>` and the automatically generated `decoder_input_ids` is `<eos> <bos> summarization`, because the `decoder_start_token_id` in `bart_config` is the same as that of `<eos>` , is there any special reasons to do it ? I think the `labels` should be `summarization <eos>` and `decoder_input_ids` should be `<bos> summarization`.


### Question 2
Why `decoder_start_token_id` is the same as `<eos>` , which means all bart will use `<eos>` as its first token to start generating, isn't this against the way the bart was trained?

Comments:
Bart was trained to have EOS as it's start_token_id and we've noticed that forcing the first token to be BOS gives better results, see: https://github.com/huggingface/transformers/issues/3668

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
[GPT-Neo] Simplify local attention
Author: patil-suraj
Pull request: yes

# What does this PR do?

Co-authored-by: finetuneanon <finetuneanon@users.noreply.github.com>

This PR is a continuation of #11630 which simplifies GPT Neo's local attention implementation. All credit to @finetuneanon for finding this issue, providing the fix, and a detailed explanation. Thanks a lot for working on this :) 

The issue is described in #11320 and  performance evaluation results are available here:
[#12106 (comment)](https://github.com/huggingface/transformers/pull/12106#discussion_r650008259)

This PR does some cleanup and updates tests on top of @finetuneanon changes.

Fixes #11320, Fixes #11787, Fixes #12964, Fixes #11096

Comments:
I had error in running:
```
  File "src\transformers\models\gpt_neo\configuration_gpt_neo.py", line 218, in __init__
    from .modeling_gpt_neo import GPTNeoAttentionMixin
ImportError: cannot import name 'GPTNeoAttentionMixin'
```
Need a patch for this PR.
Unexpected result when tokenizing a single token rather than a sentence with multilingual tokenizer
Author: Doragd
Pull request: no

I want to check how a word is divided into sub-words, but I found an unexpected result.

e.g. mbart:
```python
mbart_tokenizer = AutoTokenizer.from_pretrained('facebook/mbart-large-cc25', use_fast=True)
mbart_tokenizer.tokenize('.')
>>> ['▁', '.']
mbart_tokenizer.tokenize('a dog.')
>>> ['▁a', '▁dog', '.']
```
Why the `'.'` is tokenized to two sub-words `['▁', '.']` when it is a single token but to `'.'` when it is a part of a sentence?

e.g. mT5
```python
t5_tokenizer = T5Tokenizer.from_pretrained("google/mt5-small")
t5_tokenizer.tokenize('.')
>>> ['▁', '.']
t5_tokenizer.tokenize('a')
>>> ['▁', 'a']
mbart_tokenizer.tokenize('a dog.')
>>> ['▁', 'a', '▁dog', '.']
```


Comments:
CodeT5
Author: flozi00
Pull request: no
Labels: name: New model

# 🌟 New model addition

## Model description

<!-- Important information -->

This model is an T5 version by Salesforce trained for coding. Some articles says it should beat openai codex, so it could be interesting to add it to the hub

## Open source status

* [X] the model implementation is available: (give details) https://github.com/salesforce/CodeT5
* [X] the model weights are available: (give details) https://github.com/salesforce/CodeT5
* [ ] who are the authors: (mention them, if possible by @gh-username)


Comments:
https://huggingface.co/Salesforce/codet5-base/tree/main

Thanks @NielsRogge
[Flax][WIP] added Flax Pegasus Models
Author: bhadreshpsavani
Pull request: yes
Labels: name: WIP

# What does this PR do?

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?
@patil-suraj @patrickvonplaten 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Hi @bhadreshpsavani thanks for the PR, let us know if you want to continue working on this :) 

Sure @patil-suraj,
I want to continue on it but I will need some help from you I think!
Shall I contact you slack If I need any guidance from you?
I will update this PR today with latest changes!

Sounds good, happy to help :)

Given that we have a cookie-cutter example now, it might be worth actually starting from scratch using the cookie-cutter that is based on BART - will probably be more efficient

Hi @patrickvonplaten 
I will start from scratch! That would be better.
I create another PR, that will be fine right?

This one is being handled by above mentioned PR
The transferred onnx model is much bigger than the origin pytorch model
Author: junleiz
Pull request: no

python -m transformers.onnx --model=facebook/bart-large /home/sysadmin/downlaod/onnx_models/bart-large

Pytorch version: 1.9.0
transformers version: 4.9.1
platform: centos 7
python version: 3.7

The original bart model is aroud 2GB, But the transferred bart-large model is more than 3gb. This could because some shared weights are are duplicated in the onnx model

Comments:
Hi @leoozy, 

Thanks for bringing this to our attention.

I don't have all the details about the machinery ONNX/ORT are using to export the weights, but you're certainly right about some shared buffers being copied over at multiple places.

May be @tianleiwu from ORT would have some more insights about this specific behaviour? 

@mfuntowicz @tianleiwu I checked the converted bart.onnx model. There is a huge weight called shared.weight, which is the weights of embeding layers (size: seq_length x vocabulary_size)。The encoding and decoding process shared the weights but the shape used in the two processes are different. When encoding, the shape is (vocabulary_size X seq_length). When decoding, the shape is (seq_length X vocabulary size). So, the onnx file saved it duplicatedly because of the different shapes. Models excess 2GB have a lot of limits while being optimized using onnxruntime. 

@leoozy, please try do_constant_folding=False in torch.onnx.export to see whether it could reduce the onnx model size. 

Updated: I tried in my machine, onnx model file size is 1.5 GB, pytorch model is about 1.0 GB. I also verified that removing duplicated weights in onnx model won't help (result is still around 1.5GB).

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Display error message for pipeline loading failures
Author: xloem
Pull request: yes

# What does this PR do?

Displays an error message when a model class fails to load a model for a pipeline.

This helped me understand what was going on when pipelines failed to load.

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@Narsil @LysandreJik 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

I think this should be left to the following error message, that will let users know what classes were tried (all of them) and that they failed. The exact code should be left as follow-up.

This warning added is just noise in regular usage as it is expected that some classes won't work (so always issueing a warning when nothing is wrong).

This is not a super strong opinion though.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Ignore `past_key_values` during GPT-Neo inference
Author: aphedges
Pull request: yes

# What does this PR do?

Applies #8633 to GPT-Neo.

I was getting errors like `RuntimeError: Sizes of tensors must match except in dimension 2. Got 20 and 19 (The offending index is 0)` during the evaluation step of `Trainer.train()` with GPT-Neo.

This was fixed in an internal implementation of `GPTNeoForSequenceClassification` by @manuelciosici before an official version of the class was released for `transformers`. I converted our code to use the `transformers` version of the class and realized the solution should be upstreamed. I'm just mentioning this here because I don't get all the credit for this patch.

This might be needed for other models as well, but I don't know of a good way to figure that out without testing them all.

## Who can review?

trainer: @sgugger

Comments:
Fix attention mask size checking for CLIP
Author: Renovamen
Pull request: yes

# What does this PR do?

Fix a small error in attention mask size checking of CLIP.


## Before submitting

- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests), Pull Request section?


## Who can review?

@patil-suraj

Comments:
Fixing backward compatiblity for non prefixed tokens (B-, I-).
Author: Narsil
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #13325

@LysandreJik


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

Comments:
[Speech2Text2] Skip newly added tokenizer test
Author: patrickvonplaten
Pull request: yes

The PR: https://github.com/huggingface/transformers/commit/3dd538c4d37248961d4cf99f4c07e8a5fe54984c added a new tokenizer test that should have been skipped for Speech2Text2 tokenizer. The Speech2Text2 tokenizers are yet to be added as explained in https://github.com/huggingface/transformers/pull/13186

Comments:
Clearer indication for overridden method in generation
Author: ktangri
Pull request: no
Labels: name: WIP

The expectation for the `prepare_inputs_for_generation` function to be overridden can be made clearer by changing https://github.com/huggingface/transformers/blob/700cee344691afc41f68aa18fedea463b22f95f1/src/transformers/generation_utils.py#L369-L374

to raise a `NotImplementedError` that provides the information mentioned in the function's comment.

@patrickvonplaten 

Comments:
Also putting this in the "Fix generation docs" task basket

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
TypeError: __init__() got an unexpected keyword argument 'save_strategy'
Author: jmasom
Pull request: no

### Environment
- `transformers` version: 4.2.2
- Platform: Linux-4.15.0-151-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.4.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Not explicitly (but probably)
- Using distributed or parallel set-up in script?: Not explicitly (but probably)

### Who can help
@sgugger

### Details
I am using RoBERTa for seq classification, but that is not where my issue is coming from. My issue is coming from the Trainer API. Specifically, when I try to specify save_strategy=epoch in TrainingArguments, I get the error message in the issue title. I tried updating to a more recent version of Transformers as per another issue, but that didn't work. I'm not sure what to do about it.

Comments:
You probably did not properly install it. The environment above shows 4.2.2 and `save_strategy` was introduced later on.
You can check the version of Transformers executed by your script by adding
```
import transformers
print(transformers.__version__)
```
at the top of it.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Better documentation for generation parameter defaults
Author: patrickvonplaten
Pull request: no
Labels: name: WIP

# Generation default params documentation

It's very hard to follow how the generation parameters are set when running generation. When looking at the official function: https://github.com/huggingface/transformers/blob/700cee344691afc41f68aa18fedea463b22f95f1/src/transformers/generation_utils.py#L644 all parameters default to `None`, but are then later overwritten by the config's default parameters, *e.g.* here: https://github.com/huggingface/transformers/blob/700cee344691afc41f68aa18fedea463b22f95f1/src/transformers/generation_utils.py#L878 . This is very hard to trace or follow. We should at least put a warning or note that clearly states that all generation parameters (and actually all forward) parameters **always** default to the config.

What do you think @LysandreJik @patil-suraj @sgugger ? 

If you agree, I'll open a PR for it :-) 

Comments:
Fine by me! I think it can just be stated at the beginning before each arg is documented.

Putting this is the "Improve generation task basket" so that this is handled once the generation docs are improved as well

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.