noamaanMulla-03/transformers-issues huggingface dataset:
Word limit with mBART-50 translation
Author: phayat
Pull request: no

I'm using facebook/mbart-large-50-many-to-many-mmt model to translate french texts to english, but it seems the translation is limited to the first 110 words of the input text. 

Do you confirm? and is there a way to fix this?

Thanks in advance.

Comments:
Hey @phayat ! Could you please post a code-snippet so we could reproduce the issue ? Thanks!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

hey @patil-suraj @phayat have you found the solution
"Is next sentence" pre-training task availability for Language Modeling scripts
Author: shabie
Pull request: no

# 🚀 Feature request

BERT was trained on 2 pretraining tasks. The scripts [here ](https://github.com/huggingface/transformers/tree/master/examples/pytorch/language-modeling) in the repo for language modeling only provide scripts for masked language modeling.

Is there any specific reason for this?

I was hoping if those scripts could be extended to include "next sentence" pretraining task to remain faithful to the pretraining methodology used by BERT if I choose to further pretrain on some corpus.

## Motivation

BERT uses 2 pre-training tasks and the scripts provide only of them.

## Your contribution

I am not sure if I can extend the script myself. I will gladly look into them if I know the there aren't any good reasons for them not being provided by HF in the first place.

Thanks a lot!

Comments:
Related issues: https://github.com/huggingface/transformers/issues/1622, https://github.com/huggingface/transformers/issues/2898 and https://github.com/huggingface/transformers/issues/2166

However, example scripts are made to be very understandable and very easy to tweak, so modifying them to include the next sentence prediction objective for BERT shouldn't be complicated! 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Same issue! Anybody done it before?
Have been trying using Hugging face library for more than a month now. Still running into issues
Fix sentinel token IDs in data collator for Flax T5 pretraining script
Author: rahuln
Pull request: yes

# What does this PR do?

Modifies the sentinel token IDs used in the data collator for the Flax T5 pretraining script so that they go in decreasing order starting at `len(tokenizer) - 1`, which matches the [original T5 code](https://github.com/google-research/text-to-text-transfer-transformer/blob/main/t5/data/preprocessors.py#L2895).

Fixes #14282 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
Hey @rahuln! I'm pinging @patrickvonplaten for review as you have worked with him until now, but please note that he's off until next week so he'll review your PR when he's back! Thanks for your understanding.

Great! Thanks a lot for digging into this issue and fixing it
[`BLIP`] fix doctest
Author: younesbelkada
Pull request: yes

# What does this PR do?

This PR fixes `BLIP` doctest. 

Link to failing job: https://github.com/huggingface/transformers/actions/runs/3964164193 

The docstring of the `forward` method of `BlipForQuestionAnswering` has been corrected to educate users on how to correctly use this module after https://github.com/huggingface/transformers/pull/21021 being merged. 
The logic now of the `forward` method is pretty much similar to [`T5`](https://github.com/huggingface/transformers/blob/main/src/transformers/models/t5/modeling_t5.py#L1592-L1610).  

cc @ydshieh 💯 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[DETR and friends] Use AutoBackbone as alternative to timm
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR makes it possible to leverage our own backbone classes, like ResNet or Swin Transformer, instead of relying on timm for the following models:
- DETR
- Conditional DETR
- Deformable DETR
- Table Transformer

 This allows people to use these frameworks without having to rely on the timm dependency.

I've added an attribute to the config "use_timm_backbone" which is set to True by default, but can be set to False.

To do:

- [x] fix copies, once design gets approved

Comments:
_The documentation is not available anymore as the PR was closed or merged._

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

@sgugger feel free to approve :)

Can you make all tests pass before asking for a final review?

The remaining tests which are failing are due to `make fix-copies`, however I'll only start updating the other models once this design is approved.

@sgugger thanks for the review, addressed all comments!
Update TF doc test template
Author: amyeroberts
Pull request: yes

# What does this PR do?

The PR #21106 introduced failures in some doctests: 
* src/transformers/models/deit/modeling_tf_deit.py::transformers.models.deit.modeling_tf_deit.TFDeiTForImageClassificationWithTeacher.call
* src/transformers/models/resnet/modeling_tf_resnet.py::transformers.models.resnet.modeling_tf_resnet.TFResNetModel.call
* src/transformers/models/segformer/modeling_tf_segformer.py::transformers.models.segformer.modeling_tf_segformer.TFSegformerForImageClassification.call
* src/transformers/models/vit/modeling_tf_vit.py::transformers.models.vit.modeling_tf_vit.TFViTModel.call

This was due to `processor_class` no longer being passed to `add_code_sample_docstrings` e.g. the changes to [modeling_tf_deit.py](https://github.com/huggingface/transformers/pull/21106/files#diff-d8a9a4a182509f1903e7dbcd751d605285c8b62d0c8213a1a9ae1ba15e9fcc77). 

Whilst `processor_class` could be removed for the PyTorch models doctests, `{processor_class}` hadn't been removed for the equivalent TensorFlow doctest templates. This updates the test templates to match the equivalent PyTorch ones and resolve failing tests. 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Closed as changes are a subset of changes introduced in #21225
[GIT] Convert more checkpoints
Author: NielsRogge
Pull request: yes

# What does this PR do?

Microsoft open-sourced some more GIT checkpoints (see https://github.com/microsoft/GenerativeImage2Text/issues/34#issuecomment-1374378625), hence I've converted them by extending the conversion script.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Optimize by not computing gradients for parameters set to requires_grad=False
Author: raghavanone
Pull request: yes

Fix  #21182 

@sgugger 



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_21236). All of your documentation changes will be reflected on that endpoint.

@sgugger  Need to retrigger this build .
Exclude the parameters with `requires_grad=False` in the `Trainer` optimizer.
Author: avsolatorio
Pull request: no

### Feature request

Attempt to optimize the training for models with weights/parameters that are set to `requires_grad=False`. This is done by excluding these parameters in the optimizer.

### Motivation

I am building a Seq2Seq model where I use a pre-trained model for the encoder. I freeze all the parameters of the encoder by setting `requires_grad=False`. I expected the training to speed up compared to a model where both the encoder and decoder weights are trainable. However, I found that there's no difference in speed and also memory.

I investigated a bit and found that all the model parameters, regardless of whether gradients are required to be computed, are included in the optimizer https://github.com/huggingface/transformers/blob/00ba7cadd812437708b380ab078a3cfe8cfaff31/src/transformers/trainer.py#L1021-L1030

I tested an idea and subclassed the `Seq2SeqTrainer`. So, I updated the above snippet with this:

```Python
            optimizer_grouped_parameters = [
                {
                    # Add here the `p.requires_grad` condition
                    "params": [p for n, p in opt_model.named_parameters() if (n in decay_parameters and p.requires_grad)],
                    "weight_decay": self.args.weight_decay,
                },
                {
                    # Add here the `p.requires_grad` condition
                    "params": [p for n, p in opt_model.named_parameters() if (n not in decay_parameters and p.requires_grad)],
                    "weight_decay": 0.0,
                },
            ]
```

Doing this actually improved both the speed and the memory during the training.

I was wondering if this is something we can add to the codebase. If not, I am curious as to why we shouldn't exclude the parameters that are intended not to be trainable in the optimizer.

### Your contribution

I can make the PR if this is an acceptable change. 🤗

Comments:
Sounds like a welcome change!
DistilBertModel to sequence classification
Author: Rane90
Pull request: no

### System Info

Hi,

I'm trying to create a ```DistilBertModel ``` model for sequence classification, such that ```max_position_embeddings=1024``` (otherwise I would have used ```DistilBertForSequenceClassification``` which is defult to ```max_position_embeddings=512``` )

I define the model in the following way:

```
 configuration = DistilBertConfig(max_position_embeddings=1024)
 model = DistilBertModel(configuration)
```

When forwarding an input to the model in the following way:
```
output = model(ids, attention_mask = mask, return_dict=False)[0]
```
such that ```ids.shape = (batch_size, 1024)``` and ```mask.shape = (batch_size, 1024)``` the shape of the output is ```(batch_size, 1024, 768)``` .

My question is: What is the best practice to convert this output into a probability vector over the number of labels such the modified output shape would be ``(batch_size, num_labels)```?

I thought of a few options including flattening the current output + an additional FC layer, but I'm not sure this is the best practice.

would it be possible to add a config parameter for ```DistilBertConfig``` to automatically enable this behavior?

Thank you in advance :)
@ArthurZucker , @younesbelkada

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction


```
from transformers import DistilBertConfig, DistilBertModel

configuration = DistilBertConfig(max_position_embeddings=1024)
model = transformers.BertModel.from_pretrained('bert-base-uncased')

output = self.l1(ids, mask, return_dict=False)[0]

print(output.shape)
# (batch_size, 1024, 768)
````

### Expected behavior

My question is: What is the best practice to convert this output into a probability vector over the number of labels such the modified output shape would be ``(batch_size, num_labels)```?

Comments:
Hey! These kind of question have more sense if you ask in the [forum](https://discuss.huggingface.co/), as it is not exactly an issue nor a bug. Nevertheless, IMO you should use `DistilBertForSequenceClassification` and just modify the `max_position_embeddings`. 

Closing as it is not an issue
Add scikit-learn dependency to train langage-modeling
Author: mostafaelhoushi
Pull request: yes

# What does this PR do?

In order to run the language modeling training script, we need `scikit-learn` to be installed, so this PR adds it to the requirements.txt


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@ArthurZucker



Comments:
_The documentation is not available anymore as the PR was closed or merged._
[Tokenizers] Missmatch between fast and slow 
Author: ArthurZucker
Pull request: no
Labels: name: Core: Tokenization

name: bug

When I worked on the implementation of Whisper I realised that two different behaviors appear when you use a `fast` or `slow` tokenizer and have a OOV.
Simple snippet : 
```python
>>> from transformers import GPT2Tokenizer, GPT2TokenizerFast
>>> fast = GPT2TokenizerFast.from_pretrained("gpt2")
>>> slow = GPT2Tokenizer.from_pretrained("gpt2")
>>> # the vocab size is 50257
>>> fast.decode(50258)
''
```
```python 
>>> slow.decode(50258)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/arthur_huggingface_co/transformers/src/transformers/tokenization_utils_base.py", line 3468, in decode
    return self._decode(
  File "/home/arthur_huggingface_co/transformers/src/transformers/tokenization_utils.py", line 938, in _decode
    for token in filtered_tokens:
TypeError: 'NoneType' object is not iterable
```

My question I guess is : which one is the expected one?
Here is my take : 
- It should work, but output a warning saying that an OOV was encountered and was ignored. 

WDYT @sgugger @LysandreJik @Narsil  

Comments:
Thanks for reporting.

- `tokenizers` CAN do warnings (although I was under the impression there was an effort to reduce warnings.).
- In my personal opinion, raising an Exception is better when OOV than silently ignoring. That being said, it would be a massive breaking change, so I'm hesitant to "fix" that way.

Yes, an exception should be raised and it's more of a bug fix than a breaking change IMO. Users will be surprised, but they should be surprised when there is an out-of-vocab index.

@Narsil @sgugger  I can take this up. Will raise an exception in PreTrainedTokenizer when OOV is encountered . 

(This might break the tokenizer used by Whisper, as all of the timestamp tokens are not `in` the vocabulary, but are still used and need to be decoded as `''`)

Couldn't we put them in the vocab as `Timestamp <n>` (being the seconds offset ?) (Even as special tokens ?)

we can, but we would also have to help the openAi team with their tokenizer that is based on pour GPT2TokenizerFast 😅

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
How to reset dataloader or global_step for continued training?
Author: ajesujoba
Pull request: no

I have finetuned MT5-base model for machine translation task on a corpus `A` and I would want to swap the dataset and continue training on corpus `B` (this is not finetuning but a form of continued training), and force the dataloader to start afresh instead of continuing from the point where it stopped previously on corpus `A`. Currently, to do continue training, I used `resume_from_checkpoint` to provide the checkpoint of the MT model for corpus `A` and provide the path to the new data.
Is there an argument or parameter to reset data loader? I noticed there is a  `ignore_data_skip` parameter but this does not solve the issue.



Comments:
if the continued training stops on `B` due to GPU limit, how do I ensure that when I try to continue training on `B`, it has the right data index to start from?


Please use the [forums](https://discuss.huggingface.co/) for questions like this, as we keep issues for bugs and feature requests only. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
trainer.save_model   load error 
Author: oosij
Pull request: no

reference code : https://github.com/bhadreshpsavani/UnderstandingNLP/blob/master/go_emotion_of_transformers_multilabel_text_classification_v2.ipynb

----
----> 6 save_model ='./model/emotion'
----> 7 trainer.save_model(save_model)
----> 8 load_model = AutoModel.from_pretrained(save_model)

4 frames
[/usr/local/lib/python3.8/dist-packages/transformers/configuration_utils.py](https://localhost:8080/#) in __init__(self, **kwargs)
    315                     f"{self.id2label}. The number of labels wil be overwritten to {self.num_labels}."
    316                 )
--> 317             print(id2label)
    318             self.id2label = dict((int(key), value) for key, value in self.id2label.items())
    319             # Keys are always strings in JSON so convert ids to int here.

AttributeError: 'list' object has no attribute 'items'


---

After fine-tuning the emotion classification model with 28 classes, I learned and saved it through trainer() , but the model is not loaded. can someone help?


Comments:
Please use the [forums](https://discuss.huggingface.co/) to help debug your code. Happy to help here when you have a short reproducer, but otherwise we keep issues for bugs and feature requests only.

Hi @oosij, this is a very annoying issue from the HF implementation of the id2label management. The model will load correctly if id2label is provided as a dict when you specify the model before fine-tuning. Providing id2label in a form of a list causes no trouble during training / inference until you try to restore the fine-tuned model.

Consider something like this:
```python
id2label = {i: e for i, e in enumerate(tags)}
label2id = {e: i for i, e in enumerate(tags)}

log.info('Loading pre-trained checkpoint of a model...')
tokenizer = AutoTokenizer.from_pretrained(config.model.checkpoint, add_prefix_space=True)
model = AutoModelForTokenClassification.from_pretrained(
    config.model.checkpoint,
    id2label=id2label,
    label2id=label2id,
)

# ... fine-tune the model

trainer.save_model(config.training.save_path)
debug = AutoModelForTokenClassification.from_pretrained(config.training.save_path) # This will work now
```

@IINemo Would you like to make a PR adding a sanity check that immediately raises an error if the user tries to provide label2id/id2label that are lists instead of dictionaries?

@sgugger Ok, give me some time.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Add missing checkpoint for doctest
Author: amyeroberts
Pull request: yes

# What does this PR do?

The checkpoint for mobilenetv2 was accidentally removed in #21106 (see file change [here](https://github.com/huggingface/transformers/pull/21106/files#diff-f224d96e46d68f58f9632184f915210b3217a61253c42ff6354763e1b0f34050)), resulting in failing doctests. This adds it back. 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Generate: save generation config with the models' `.save_pretrained()`
Author: gante
Pull request: yes

# What does this PR do?

As originally discussed in #20388, this PR makes `model.save_pretrained()` also call `model.generation_config.save_pretrained()` if it is a generation-capable model (on all 3 frameworks).

It also adds a bunch of tests, namely:
- tests whether the generation config can be pushed to the hub
- tests whether `model.save_pretrained()` actually saves `generation_config.json` if it is a model that can generate (on all 3 frameworks)

Comments:
_The documentation is not available anymore as the PR was closed or merged._