noamaanMulla-03/transformers-issues huggingface dataset:
Decode input ids back to string in LayoutLMV3 Processor
Author: purnasai-cyient
Pull request: no

Hi, I am using Layoutlmv3. I Have used LMV3 processor. At train time I have labels as well along with image,text,boxes and labels. But at Inference time, I do not have labels. So the code for inference goes like below.

```
processor = AutoProcessor.from_pretrained("microsfotlmv3_repo", apply_ocr=False)
encoding = processor(images = resize_image, 
                    text = tokens,
                    boxes= boxes,
                    return_offsets_mapping=True, 
                    return_tensors="pt",
                    padding  = "max_length",
                    truncation = True,
                    max_length = 512
                    )
offset_mapping = encoding.pop('offset_mapping')
outputs = test_model1(**encoding)
predictions = outputs.logits.argmax(-1).squeeze().tolist()
is_subword = np.array(offset_mapping.squeeze().tolist())[:,0] != 0
true_predictions = [id2label[pred] for idx, pred in enumerate(predictions) if not is_subword[idx]]
```
Currently, I am decoding text like:
```
cleaned_input_ids = encoding['input_ids'][encoding['attention_mask']>0]
text = processor.tokenizer.decode(cleaned_input_ids.squeeze().tolist())
text = text[4:-4]
tokens = text.split(" ")
```
But the count of tokens/text and count of true_predictions/labels are not matching.

I am expecting the result to be:
“sun”:label,
“rises”:label,
“in”:label,
“the”:label,
“east”:label.

Currently i am not able to map them, as their counts/lengths are not matching. How to resolve this.

Tagging @NielsRogge  , and Others

Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discuss.huggingface.co) instead?

Thanks!

HI @LysandreJik, I have raised this in the forum as well. I added it here for more support. Thanks

Hi, I've answered this question on the [forum](https://discuss.huggingface.co/t/how-to-decode-inputids-back-to-string-in-layoutlmv3/24609/2), let's continue the discussion there.
[Doctest] Add `configuration_clip.py`
Author: daspartho
Pull request: yes

Add `configuration_clip.py` to `utils/documentation_tests.txt` for doctest.

Based on issue #19487

Noticed the model initialization and config initialization lines were switched so made some additional changes to correct it.

@ydshieh could you please check if it's okay?
Thank you =)

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@daspartho Very cool!

Would you like to take the challenge to add doc example to `CLIPConfig`, which will use `from_text_vision_configs`. This will help a lot of the library users 🔥 Let me know :-)

@ydshieh Sure! I'd like to take on the task =)

@ydshieh made some changes; could you please check if it looks good?

@ydshieh added an example using the `from_text_vision_configs` method; could you please review the changes to see if they're okay? 
Thanks :)

Just a final comment and we are ready to merge!

@ydshieh made the suggested changes; good to go =)

Thank you for the PR and your patience :-) @daspartho
Fix accelerate tests
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Fixes the accelerate test based on new accelerate update.
Some of the test are fixed, but not all yet, so WIP

`for i in range(len(max_memory)-2):` 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_19746). All of your documentation changes will be reflected on that endpoint.
[Doctest] Add `wav2vec2_conformer.py`
Author: juancopi81
Pull request: yes

Add `configuration_wav2vec2_conformer.py` to `utils/documentation_tests.txt` for doctest.

Based on issue: [19487](https://github.com/huggingface/transformers/issues/19487)

@ydshieh could you please check it? If it is ok for you I can keep working on the "wav2vec2" ones!
Thank you very much for your support :)

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hi @juancopi81 Wav2Vec2 is already done.
See [documentation_tests.txt](https://github.com/huggingface/transformers/blob/main/utils/documentation_tests.txt) on the main branch. You can work on other model configs if you would like to 🤗 

Great! Thanks a lot @ydshieh 🤗
[Doctest] Add `configuration_wavlm.py`
Author: juancopi81
Pull request: yes

Add configuration_wav2vec2_conformer.py to utils/documentation_tests.txt for doctest.

Based on issue: [19487](https://github.com/huggingface/transformers/issues/19487)


@ydshieh Could you please take a look? Thank you very much for your kind support!

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Newly created cache directories does not include the version.txt file
Author: dmiller1-godaddy
Pull request: no

### System Info

- `transformers` version: 4.22.2
- Platform: macOS-11.7-x86_64-i386-64bit
- Python version: 3.9.13
- Huggingface_hub version: 0.10.0
- PyTorch version (GPU?): 1.12.1 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help?

@sgugger

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Steps to reproduce this behaviour:
1. Start with a clean machine with no cache directories at all
2. Download a pre-trained model (in our case, a clip model) to create the cache directory and download model files
```
from transformers import CLIPModel
from transformers import CLIPProcessor

model = CLIPModel.from_pretrained("openai/clip-vit-base-patch32")
processor CLIPProcessor.from_pretrained("openai/clip-vit-base-patch32")
```
3. Notice there is no `version.txt` file there under the `hub` directory causing transformers library to assume it's an older version of the cache

### Expected behavior

Upon creation of the cache directory a `version.txt` should be created to indicate it is the correct version of the cache system.

For context, we are taking a copy of the cache folder and saving that for later use in production under a read only system for faster startup times. As we don't want the box to startup and download a pretrained model every time. However this issue has caused the transformers library to always want to write/migrate the cache as it thinks it is on an older version.

Comments:
Thanks for reporting! I can reproduce and the PR above fixes the problem normally.
add a python module called "loss.py" for custom losses published in papers
Author: maryam9494
Pull request: no

### Feature request

I'm currently working on a text generation project at work which made me encounter a new loss name "unlikelihood loss".
Thankfully there was an implementation already existed on the internet.
But I was wondering that would be neat to create a separate module just for loss functions specially custom ones published in papers.
I think that would help people out a lot.
I would also like to be the first contribute to this new feature if you think that would be helpful.

[link to the paper](https://arxiv.org/pdf/1908.04319.pdf)

### Motivation

I always run into problem when I  can't understand the mathematics in paper deep enough to be able to implement it myself.
And Hugging Face has already helped a lot.
I think that would be a neat feature to create a separate loss module or script, located in utils or models directory.
That would help people like me that struggle with implementations.

### Your contribution

yes of curse.
I would be glad to contribute to this new feature.


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Allow custom head size for self attention in BERT
Author: xinyangz
Pull request: no

### Feature request

Right now the `attention_head_size` of BERT self attention is set to hidden_size / num_attention_heads:
https://github.com/huggingface/transformers/blob/855dcae8bb743c3f8f0781742d7fa2fa3aaa3e22/src/transformers/models/bert/modeling_bert.py#L260-L262

However, the `all_head_size` of self attention layer doesn't have to match the `hidden_size` of the model. For example, we may train a deeper model with narrower layers. In fact [this paper](https://arxiv.org/pdf/2106.09650.pdf) found that doing so will increase the model performance with minimal hit on the training and inference speed.

My proposal is to add an option `attention_head_size` to `BertConfig` to allow more flexible model architectures.

### Motivation

Currently, there is no easy way to change the `attention_head_size` without changing the `hidden_size` of the whole model. As a result, it's hard to set up a narrower and deeper or a wider and shallower model.

### Your contribution

I can submit a PR to add an option to `BertConfig` and update the code where needed.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[WIP] Add TF support for OWL-ViT
Author: alaradirik
Pull request: yes

Adds TensorFlow support for the [OWL-ViT](https://huggingface.co/docs/transformers/model_doc/owlvit) model.

- Creates transformers/models/owlvit/modeling_tf_owlvit.py
- Creates tests/models/owlvit/test_modeling_tf_owlvit.py


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_18450). All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Image transforms add center crop
Author: amyeroberts
Pull request: yes

# What does this PR do?

Adds `center_crop` to the image transforms library, to be used in the future image processors. Performs equivalent processing as done in the `ImageFeatureExtractionMixin`

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Custom preprocessing of text
Author: datistiquo
Pull request: no
Labels: name: wontfix

I feel like this a silly question. But I just thought in using BERT, wait, before working with fasttext for example I had to do preprocessing like word stemming/lemmatization and stopwords. How is the advise for using BERT models?

Does it makes problems if I do stemming or lemmatization before feeding to BERT tokenizer?

questions over questions...

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


It shouldn't be necessary as it performs byte pair encoding (BPE) when a word isn't in it's vocabulary. For example "Gallbladder palpaple". Palpaple isn't in my vocabulary so it breaks the word into many partial words that are in the vocabulary as: ['p', '##al', '##pa', '##ple']. This would match variations that would otherwise need to be stemmed or converted to it's lemma.

This will however be an issue if you are using the model to perform cosine similarly. The results are terrible when you have many words out of vocabulary.
[Doctest] Add `configuration_decision_transformer.py`
Author: xlahuerta
Pull request: yes

Add `configuration_decision_transformer.py` to `utils/documentation_tests.txt` for doctest.

Based on issue https://github.com/huggingface/transformers/issues/19487

@ydshieh could you please check it?
Thank you :)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[Doctests] Add `configuration_detr.py`
Author: xlahuerta
Pull request: yes

Add `configuration_detr.py` to `utils/documentation_tests.txt` for doctest.

Based on issue https://github.com/huggingface/transformers/issues/19487

@ydshieh could you please check it?
Thank you :)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
docs: All broken links were fixed in CONTRIBUTING.md file.
Author: mdfaizanahmed786
Pull request: yes

# What does this PR do?
### This PR fixes the broken links in contributing file.

- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
**Yes, this PR does fix broken links in contributing file.**

- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
**Yes, I've read the contributor guideline.**

- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
**No, it was not discussed in any GitHub issue or in the form.**

- [x] Did you write any new necessary tests?
**Yes, before creating a PR, I've checked the broken links in the contributing file.**


Comments:
_The documentation is not available anymore as the PR was closed or merged._

> Thanks for the fixes!

You're welcome!
Fix issue #19300
Author: raghavanone
Pull request: yes

# What does this PR do?

This PR fixing issue #19300 

<!-- Remove if not applicable -->

Fixes # (issue)
#19300

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@sgugger 



Comments:
@sgugger  How do I get output_dir in on_train_end callback ? 

@sgugger  Never mind, Let me fix the failing tests. 

Let me know if you need any help!

@sgugger  The tests pass now. There was bug in my change and it is great that our tests caught it. 

Thanks again for all your work on this!
[Doctest] Add `configuration_speech_to_text_2.py`
Author: daspartho
Pull request: yes

Add `configuration_speech_to_text_2.py` to `utils/documentation_tests.txt` for doctest.

Based on issue #19487

@ydshieh could you please take a look at it?
Thanks :)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Lite Transformer with Long-Short Range Attention
Author: astariul
Pull request: no
Labels: name: New model

### Model description

Abstract : 

> Transformer has become ubiquitous in natural language processing (e.g., machine translation, question answering); however, it requires enormous amount of computations to achieve high performance, which makes it not suitable for mobile applications that are tightly constrained by the hardware resources and battery. In this paper, we present an efficient mobile NLP architecture, Lite Transformer to facilitate deploying mobile NLP applications on edge devices. The key primitive is the Long-Short Range Attention (LSRA), where one group of heads specializes in the local context modeling (by convolution) while another group specializes in the long-distance relationship modeling (by attention). Such specialization brings consistent improvement over the vanilla transformer on three well-established language tasks: machine translation, abstractive summarization, and language modeling. Under constrained resources (500M/100M MACs), Lite Transformer outperforms transformer on WMT’14 English-French by 1.2/1.7 BLEU, respectively. Lite Transformer reduces the computation of transformer base model by 2.5× with 0.3 BLEU score degradation. Combining with pruning and quantization, we further compressed the model size of Lite Transformer by 18.2×. For language modeling, Lite Transformer achieves 1.8 lower perplexity than the transformer at around 500M MACs. Notably, Lite Transformer outperforms the AutoML-based Evolved Transformer by 0.5 higher BLEU for the mobile NLP setting without the costly architecture search that requires more than 250 GPU years.

### Open source status

- [x] The model implementation is available
- [x] The model weights are available

### Provide useful links for the implementation

[Paper](https://arxiv.org/pdf/2004.11886.pdf)
[Code](https://github.com/mit-han-lab/lite-transformer)
[Old version of the paper, when the model was called `Mobile Transformer (MBT)`](https://openreview.net/attachment?id=ByeMPlHKPH&name=original_pdf)

Comments:
Maybe of interest to @hollance :)

Yeah looks interesting!

Related to
https://github.com/mit-han-lab/lite-transformer
https://arxiv.org/pdf/2004.11886.pdf

It looks like the paper I linked is just a previous, unpublished version of Lite Transformer paper (linked by @atturaioe) 

I'll edit the issue accordingly. Thanks @atturaioen !

@hollance  @LysandreJik  Can I pick up this if it is it is not in WIP ?
Trainer with save_total_limit=1 keeps 2 checkpoints when EarlyStoppingCallback is active
Author: DavidNemeskey
Pull request: no
Labels: name: bug

### System Info

Transformers: 4.21.3
Python: 3.10.4
OS: Linux  5.4.0-91-generic #102-Ubuntu SMP x86_64
Torch: 1.12.1+cu113
GPU: A100 40G


### Who can help?

@sgugger 

### Information

As the title says. See below for details.

### Tasks

I trained BERT-based sequence classifiers.

### Reproduction

1. Run a training () with `EarlyStoppingCallback` active, `save_total_limit` set to 1 and `early_stopping` to something (e.g. 3)
2. Run it for a number of iterations greater than `early_stopping`
3. Check the number of checkpoints.

What happens is that if the best performing checkpoint is not the last, both the last and the best checkpoints are kept.

### Expected behavior

With `save_total_limit=1`, I would expect that only the best checkpoint is kept. 

Comments:
Thanks for the writeup. That's not a bug, but very much intended. This is the only instance in which the `save_total_limit` argument is not fully respected, because:
- we need to keep the best model
- we also need to keep the last model to be able to resume training if a crash happens

@sgugger "If a crash happens". But what about when the training has ended? The checkpoint is not really needed anymore then, is it?

In any case, I could not find anything about this exception in the documentation. If it is not because of my inaptitude at looking up information (which it well may be), it might be worth adding a sentence to the description of the `save_total_limit`.

Sure, do you want to make a PR for this?

Just to clarify: a PR for what? :smile: 

To add the exception to the documentation and/or delete the checkpoint at the end of training.

@sgugger  I would like to fix this issue, can help on where this fix needs to go ? 

@sgugger  I think we can close this ticket

Thank you guys for taking care of this!
RuntimeError from fine-tuning the automodel
Author: mali726
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.22.0.dev0
- Platform: Linux-5.4.0-110-generic-x86_64-with-glibc2.31
- Python version: 3.9.12
- Huggingface_hub version: 0.8.1
- PyTorch version (GPU?): 1.12.1+cu102 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: (True)
- Using distributed or parallel set-up in script?: (True)


### Who can help?

@LysandreJik

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

python run_mlm.py --model_name_or_path digitalepidemiologylab/covid-twitter-bert-v2 --train_file path_to_train_file --per_device_train_batch_size 8 --do_train --fp16 True --num_train_epochs 10 --save_steps 50000 --output_dir path_to_output

RuntimeError: The size of tensor a (1024) must match the size of tensor b (512) at non-singleton dimension 1

### Expected behavior

I am fine-tuning the language model on my own dataset (run_mlm.py without modification). It worked on bert-large-uncased model, but for model digitalepidemiologylab/covid-twitter-bert-v2, I received the following error: RuntimeError: The size of tensor a (1024) must match the size of tensor b (512) at non-singleton dimension 1.

Thank you for your help!

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
wiki_dpr model which features are using (i,e there are so many models are there)
Author: NareshSandrugu
Pull request: no

### Feature request

i want to test my own data for getting same embeddings for which are used for the wiki_dpr model.

### Motivation

i'm interest to explore more on own data

### Your contribution

till now not there

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.