noamaanMulla-03/transformers-issues huggingface dataset:
Whether instantiation is premature?
Author: Fei-Wang
Pull request: no

https://github.com/huggingface/transformers/blob/515ed3ad2a11a6b0cd9800b2ad4d3b313fdaea8c/src/transformers/feature_extraction_utils.py#L429

For example my code is:
```python3 
processor = LayoutXLMProcessor.from_pretrained('microsoft/layoutxlm-base', apply_ocr=False)
```
I don't need to use OCR, but pytesseract still has to be installed. 
Would it be a good solution to this kind of problem if the parameters were updated with kwargs before instantiation？

Comments:
@NielsRogge any suggestions?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Hi,

Thanks for the issue and apologies for the late reply. I'll look into this.

cc'ing @sgugger. 

The problem is here is that, 'microsoft/layoutxlm-base' on the hub has a `preprocessor_config.json` which has `apply_ocr = True`, hence the `from_pretrained()` method will raise the error that PyTesseract should be installed. However, the additional kwarg `apply_ocr=False` is only taken into account after this.

Yes, that is the `FeatureExtractionMixin` is coded. I suggest moving the `requires_backends(self, "pytesseract")` to the call method instead of the init to solve this issue.
[LayoutLMv2] Update requires_backends of feature extractor
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR moves the `requires_backends` to the call method instead of the init of `LayoutLMv2FeatureExtractor`.

Fixes #15269

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_15941). All of your documentation changes will be reflected on that endpoint.
Made MaskFormerModelTest faster
Author: FrancescoSaverioZuppichini
Pull request: yes

# What does this PR do?
This PR changes the configuration inside `tests/maskformer/test_modeling_maskformer.py::MaskFormerModelTest` creating a smaller model to make the test run faster

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_15942). All of your documentation changes will be reflected on that endpoint.
Unable to run run_glue.py offline
Author: Nithanaroy
Pull request: no

## Environment info
An host without internet access but with ~/.cache/huggingface pre-populated

### Who can help
@sgugger @LysandreJik @SaulLu 

## Information

Model I am using (Bert, XLNet ...): 
```
--model_name_or_path bert-base-uncased --task_name mrpc --do_train --output_dir /home/jobuser/output_bart_test --per_device_train_batch_size 16 --do_eval --do_predict --per_device_eval_batch_size 8 --overwrite_output_dir --cache_dir /home/jobuser/hf_cache
```

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce
1. Run https://github.com/huggingface/transformers/blob/master/examples/tensorflow/text-classification/run_glue.py with internet using
```
--model_name_or_path bert-base-uncased --task_name mrpc --do_train --output_dir /home/jobuser/output_bart_test --per_device_train_batch_size 16 --do_eval --do_predict --per_device_eval_batch_size 8 --overwrite_output_dir --cache_dir /home/jobuser/hf_cache
```
2. Run the same command without internet (literally disconnect the wire / wifi)
3. It fails at https://github.com/huggingface/transformers/blob/master/examples/tensorflow/text-classification/run_glue.py#L282

## Expected behavior
It should leverage the existing caches and train without breaking

Comments:
You need to set the env variable `HF_DATASETS_OFFLINE=1 TRANSFORMERS_OFFLINE=1` to be able to use the libraries in offline mode, as pointed out in the [documentation](https://huggingface.co/docs/transformers/installation#offline-mode)

@sgugger thanks for the blazing fast response! I forgot to add that I set `HF_DATASETS_OFFLINE=1`. I didnt know about `TRANSFORMERS_OFFLINE=1`. It works now.
MarianTokenizer: get vocab length in `as_target_tokenizer` mode
Author: AmitMY
Pull request: yes

# What does this PR do?

When using the MarianTokenizer with a different source and target SPMs, it is expected to get the length of the target vocabulary when performing:
```python
with tokenizer.as_target_tokenizer():
    print(len(tokenizer))
```

This PR introduces a minimal change to get the correct size depending on the selected SPM.


## Before submitting
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Did you write any new necessary tests?


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_15943). All of your documentation changes will be reflected on that endpoint.
TypeError: meshgrid() got an unexpected keyword argument 'indexing'
Author: F-Yuan303
Pull request: no

when i tried to run the example given in ViLTModel below. I encountered problem"TypeError: meshgrid() got an unexpected keyword argument 'indexing'"


`from transformers import ViltProcessor, ViltModel
from PIL import Image
import requests

url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)
text = "hello world"

processor = ViltProcessor.from_pretrained("dandelin/vilt-b32-mlm")
model = ViltModel.from_pretrained("dandelin/vilt-b32-mlm")

inputs = processor(image, text, return_tensors="pt")
outputs = model(**inputs)
last_hidden_states = outputs.last_hidden_state`





- `transformers` version: 4.16.2
- Platform: Linux-4.18.0-193.el8.x86_64-x86_64-with-centos-8.2.2004-Core
- Python version: 3.7.11
- PyTorch version (GPU?): 1.9.1+cu111 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: NA
- Using distributed or parallel set-up in script?: NA







Comments:
Hi,

You may need to upgrade to PyTorch 1.10 for this to work.

Sorry i'm new in nlp, I changed the pytorch to 1.10 and it comes another problem.

Some weights of the model checkpoint at dandelin/vilt-b32-mlm were not used when initializing ViltModel: ['mlm_score.transform.LayerNorm.bias', 'mlm_score.transform.dense.weight', 'mlm_score.bias', 'mlm_score.decoder.weight', 'mlm_score.transform.LayerNorm.weight', 'mlm_score.transform.dense.bias']
- This IS expected if you are initializing ViltModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing ViltModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).

I have found a solution from the internet, thanks a lot.
Add Camembert to models exportable with ONNX
Author: chainyo
Pull request: yes

# What does this PR do?

I added lines to make Camembert models available for Onnx conversion.

# Issue

This is linked to this conversation #13952

@LysandreJik 


Comments:
I don't understand why `run_tests_torch` are failing because it was ok before. It seems that it doesn't come from my PR.
random word masking index shouble be great than 0
Author: Dicer-Zz
Pull request: no

## Information

https://github.com/huggingface/transformers/blob/05fa1a7ac17bb7aa07b9e0c1e138ecb31a28bbfe/src/transformers/data/data_collator.py#L771

This line should be:
```python
random_words = torch.randint(1, len(self.tokenizer), labels.shape, dtype=torch.long)
```
because random word masking index should be great than 0 (0 means [PAD] in bert's vocabulary).

I found the bug when I want to change masked input_ids to tokens using tokenizer.convert_ids_to_tokens. If replace a masked id by 0, and then convert_ids_to_tokens, all tokens after 0 will be discarded.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.15.0
- Platform: Linux-5.4.0-62-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.2
- PyTorch version (GPU?): 1.10.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <False>
- Using distributed or parallel set-up in script?: <False>


Comments:
cc @sgugger 

I don't think there is anything in the research article or the original pretraining code that excludes the pad token from the random words, so I would leave this as is. You can use your own data collator with any modification you want :-)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Perplexity VERY high but generated text coherent
Author: aqred1
Pull request: no

I have trained GPT2 from scratch on a minority language which I tokenized with BPE. 

When I finished training, the perplexity on the eval set was `678.80`, so I thought the model was really bad.

I however tried to sample from the model using very different sampling procedures, and finally got very coherent texts with the following parameters:

`model.generate(input_ids, do_sample=True, top_k=950,  repetition_penalty=1.2, eos_token_id=0)`

- Is the model generating good texts because my top_k is larger than the perplexity?
- Is the perplexity perhaps so high because my vocab size is 50256?

Is there perhaps a bug in how the perplexity is computed after training?  This is the final value I got from the last epoch and also stored in `eval_results.json`.


Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discuss.huggingface.co) instead?

Thanks!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
modeling_visual_bert in case of self.bypass_transformer=True
Author: osamanatouf2
Pull request: no

https://github.com/huggingface/transformers/blob/05fa1a7ac17bb7aa07b9e0c1e138ecb31a28bbfe/src/transformers/models/visual_bert/modeling_visual_bert.py#L830 The previous line should be changed from encoded_output --> encoder_outputs so that the return statement on line 861 works. Also, the following line https://github.com/huggingface/transformers/blob/05fa1a7ac17bb7aa07b9e0c1e138ecb31a28bbfe/src/transformers/models/visual_bert/modeling_visual_bert.py#L828  has an issue with indexing in dimension 2 (this will require more testing).




Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
can I use different pre-trained model for autoTokenizer and autoModel?
Author: SharonJin422
Pull request: no
Labels: name: Migration

In the train_config.json file, I set:
"model_name_or_path": "ckiplab/albert-tiny-chinese",
 "tokenizer_name": "bert-base-chinese",

this also make sense right?

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Sorry just seeing this now - you can use different tokenizer/models, but be aware that if they were not trained together then the embeddings of the model will not correspond to the IDs output by the tokenizer. The result will be gibberish, unfortunately.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
It seems that `RagSequenceForGeneration.generate` is computing inaccurate loss value
Author: repun
Pull request: no

Hello,

I think `generate` function of `RagSequenceForGeneration` does not seem to compute loss value properly.
As mentioned in the original [RAG paper](https://arxiv.org/pdf/2005.11401.pdf), thorough decoding of Rag-Sequence model needs 2 forward passes.

1. Find candidate sequences through beam search for each document
2. Compute loss for each candidate sequence with another forward pass

Then, return the sequence with the highest likelihood.

I found that `n_docs` argument used in beam search does not passed when `forward` function called for loss computation.

From `RagSequenceForGeneration.generate`,
```
    @torch.no_grad()
    def generate(
        self,
        input_ids: Optional[torch.LongTensor] = None,
        attention_mask: Optional[torch.LongTensor] = None,
        context_input_ids=None,
        context_attention_mask=None,
        doc_scores=None,
        do_deduplication=None,  # defaults to True
        num_return_sequences=None,  # defaults to 1
        num_beams=None,  # defaults to 1
        n_docs=None,
        **model_kwargs
    ):
            ...

            # then, run model forwards to get nll scores:
            if input_ids is not None:
                new_input_ids = input_ids[index : index + 1].repeat(num_candidates, 1)
                outputs = self(new_input_ids, labels=output_sequences, exclude_bos_score=True)
            ...
```

I think same documents used for beam search, also need to be used for loss computation.
If `n_docs` other than default value is passed to the `generate` function, it will end up computing loss somewhat inaccurate, possibly can change the final generated sequence.
If this is the case, I think `generate` function should be fixed, which can be simply done by just passing `n_docs` argument to `forward` inside `generate`.

Please correct me if there are any mistakes.
Thanks

@patrickvonplaten 

Comments:
Maybe of interest to @lhoestq too

Looking into it next week - sorry for being so late here

Hey @repun,

I'm not 100% following everything here. Could you maybe open a PR so that we can look into it with some code changes? 
In general, I think it's totally fine if the number of documents to retrieve during inference in `generate()` differs from the number of documents to retrieve during training. Could you explain in a bit more detail why the loss value is not accurate and how the loss value has to do with `generate()` or are we just talking about inference (and not training at all here?)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Fix initialization of mutable types
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR fixes the initialization of mutable types for Swin Transformer, SegFormer and BEiT (as pointed out by @FrancescoSaverioZuppichini in #15277).

It also fixes a small docs issue for Swin Transformer.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_15365). All of your documentation changes will be reflected on that endpoint.

This is present in many more configurations, so if we decide to switch it should be for all of them and not only for three configs.

Having a default for a mutable parameter is dangerous when the function mutates the parameter, which is not the case for our config inits, it's merely stored. I don't think we need to switch to `None`, which would result in the signatures in the doc being less complete.

I'm also not convinced it's super important as these are lookup values, but if you feel strongly a simpler way of doing this is to switch everything tu tuples, which are immutable. It keeps the arguments in the docstrings as well and can be converted to lists quite easily.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
fix bug in run_clm.py
Author: dataaug
Pull request: yes

Sorry I don't have time to follow contributor guideline. 

group_texts function in transformers/examples/pytorch/language-modeling/run_clm.py will lead to training failure.

This is because the possibility that the length of a batch is too small is not considered.

I hope this can help some people. 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_15363). All of your documentation changes will be reflected on that endpoint.

May cause errors  "ValueError: expected sequence of length 1024 at dim 1 (got 79)".  This pull request can solve this problem

This is not a fix that will actually work since most models trained on causal language modeling objective do not have a padding token. 0 thus does not work as an index, and it shouldn't be the index picked for the labels either since PyTorch only ignore the index -100 in the loss.

It's also seem like a very edge case since it implies that you get the total length of 1000 texts in your dataset be shorter than the block size.

@sgugger  Thank you for your reply～ 

> This is not a fix that will actually work since most models trained on causal language modeling objective do not have a padding token. 0 thus does not work as an index, and it shouldn't be the index picked for the labels either since PyTorch only ignore the index -100 in the loss.

Agree. Padding should be set more appropriately, Or discard this part directly.

> It's also seem like a very edge case since it implies that you get the total length of 1000 texts in your dataset be shorter than the block size.

Disagree.  This often happens in the case of short text( I tested on 7.4 million short texts with an average length of 15). Function group_texts （"map processes 1,000 texts together"）probably process data incorrectly in the last "1,000 texts", because it can actually be any number between 0 and 1000.  This causes total_length  less than block_size which leads to dataloader error.

I think this should be fixed. Do I need to resubmit my pull request?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
How to get T5 encoder&decoder's hidden states and keep requires_grad=True
Author: scarydemon2
Pull request: no

transformers version:
4.15.0

Hi! I am making a custom T5 generation model, and I have some problems. In my model, I want to take out the decoder's last hidden states and encoder's last hidden states to do some other operations, and these operations will be used to fine-tune the encoder and decoder. So I need last_hidden_state's requires_grad=True and is_leaf=False.
I get the encoder_last_hidden_states and decoder_last_hidden_states in the following code:

        outputs1 = self.T5model1(
        input_ids=batch["source_ids"],
        attention_mask=batch["source_mask"],
        labels=lm_labels,
        decoder_attention_mask=batch['target_mask'],
        output_hidden_states=True,
        )
        decoder_last_hidden_states=outputs1['decoder_hidden_states'][-1]
        encoder_last_hidden_states=outputs1['encoder_last_hidden_state']
But I found that the requires_grad of decoder_last_hidden_states and encoder_last_hidden_states are False and is_leaf of them is True. It looks like the gradient doesn't propagate. 
How do I get these vectors so that I can finetune the full six-layer encoder and six-layer decoder.


Comments:
Documentation in CTRL linksto 404
Author: osanseviero
Pull request: no

In https://huggingface.co/docs/transformers/master/en/model_doc/ctrl#overview, when clicking "See reusing the past in generative models", the user is redirected to a 404 (https://huggingface.co/docs/transformers/master/en/quickstart#using-the-past)

Comments:
Nice catch! @stevhliu, would you like to take a look at that?

Looks like a [throwback](https://huggingface.co/transformers/v2.2.0/quickstart.html) to `transformers 2.2.0`! Should we point the user to [`CTRLModel.forward()`](https://huggingface.co/docs/transformers/master/en/model_doc/ctrl#transformers.CTRLModel.forward.past_key_values) where they can get more information about the parameter? It looks like I can also update it to say PyTorch accepts `past_key_values` and TF accepts `past`.

Another option could be to also add a code example in [`CTRLModel`](https://huggingface.co/docs/transformers/master/en/model_doc/ctrl#transformers.CTRLModel).

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

What you propose sounds good to me @stevhliu!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

This is now fixed after #15615