noamaanMulla-03/transformers-issues huggingface dataset:
`seed_generator` would casue tensorflow gpu memory growth immediately
Author: Atakey
Pull request: no

https://github.com/huggingface/transformers/blob/7cced021fa8ddc59f0f77384300760d34545394e/src/transformers/generation_tf_utils.py#L349

In transformers version >=4.18.0, import something from `generation_tf_utils.py` would casue gpu memory growth immediately if I don't set env `TF_FORCE_GPU_ALLOW_GROWTH=true` before importing modules from this file.

And I locate the problem at this line. 

Making attr `seed_generator` lazy loaded could solve this.

```
# ...
class TFGenerationMixin:
    """
    A class containing all of the functions supporting generation, to be used as a mixin in [`TFPreTrainedModel`].
    """

    # seed_generator = tf.random.Generator.from_non_deterministic_state()

    _seed_generator = None
    
    @property
    def seed_generator(self):
        if self._seed_generator is None:
            self._seed_generator = tf.random.Generator.from_non_deterministic_state()
        return self._seed_generator
# ...
```


Comments:
Hey @Atakey 👋  Great finding! Would you be able to open a PR? The suggestion you gave looks great!
Result of T5 tokenizer doesn't match
Author: JueXiuHuang
Pull request: no
Labels: name: bug

### System Info

transformers: 4.16.2
python: 3.7.11
pytorch: 1.10.0

### Who can help?

@patrickvonplaten

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

```
from transformers import T5Tokenizer, T5ForConditionalGeneration

text = '(CNN)Then-President Donald Trump angrily demanded to go to the US Capitol on January 6, 2021, and berated his protective detail when he didnt get his way.'

tokenizer = T5Tokenizer.from_pretrained('t5-base')

tokenize_result = tokenizer.tokenize(text)

ids_1 = tokenizer(text, max_length=128, truncation=True,
      padding='max_length', return_tensors='np')['input_ids']
ids_2 = tokenizer(tokenize_result, is_split_into_words=True, 
      max_length=128, truncation=True, padding='max_length', return_tensors='np')['input_ids']
ids_3 = tokenizer.convert_tokens_to_ids(tokenize_result)

print(tokenizer.batch_decode(ids_1, skip_special_tokens=True)[0])
# >>> (CNN)Then-President Donald Trump angrily demanded to go to the US Capitol on January 6, 2021, and berated his protective detail when he didnt get his way.

print(tokenizer.batch_decode(ids_2, skip_special_tokens=True)[0])
# >>> ( C NN ) The n - P resident Donald Trump an gri ly demande d to go to the US Capitol on January 6, 20 21, and be rated his protective detail when he didn t get his way.

print(tokenizer.decode(ids_3, skip_special_tokens=True))
# >>> (CNN)Then-President Donald Trump angrily demanded to go to the US Capitol on January 6, 2021, and berated his protective detail when he didnt get his way.
```

### Expected behavior

Three print results should be the same, but when we pre-tokenize with same tokenizer and convert to ids with flag is_split_into_words=True, the tokenizer gives different result.

Comments:
I think your issue was already resolved here:
https://github.com/huggingface/transformers/issues/8217

In short:

`is_split_into_words=True`

assumes that the input is a list of words, not tokens.
When you input `tokenize_result` to tokenizer with `is_split_into_words=True`, the tokenizer assumes that the tokens are distinct words (very short words). During decoding these tokens are interpeted as words and that's why there are so many additional spaces in the second result.

try replacing
```
ids_2 = tokenizer(tokenize_result, is_split_into_words=True, 
      max_length=128, truncation=True, padding='max_length', return_tensors='np')['input_ids']
```
with
```
ids_2 = tokenizer(text.split(' '), is_split_into_words=True, 
      max_length=128, truncation=True, padding='max_length', return_tensors='np')['input_ids']
```
You'll see that all results will be the same.
Place inputs on device when include_inputs_for_metrics is True
Author: sgugger
Pull request: yes

# What does this PR do?

As pointed out in #18038, the inputs accumulated in the Trainer when `include_inputs_for_metrics=True` are not on the proper device, which then fails in distributed setups. This PR fixes that.

Fixes #18038

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Sort doc toc
Author: sgugger
Pull request: yes

# What does this PR do?

This PR adds a script to automatically sort the model doc part of the documentation ToC, so we never need things like #18011 ever again.

The script runs in the traditional `make style/fixup/quality` setup meaning that:
- `make style` or `make fixup` fix the ToC by auto-sorting it and removing duplicates
- `make quality` just checks whether or not the ToC is properly sorted.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
ValueError: AlbertForMaskedLM does not support gradient checkpointing.
Author: warm-ice0x00
Pull request: no

ValueError: AlbertForMaskedLM does not support gradient checkpointing.
`model.gradient_checkpointing_enable()` doesn't help.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Drop columns after loading samples in prepare_tf_dataset
Author: Rocketknight1
Pull request: yes

Another super-small fix to `prepare_tf_dataset()` - this time we apply the same fix we applied to `to_tf_dataset()`, and keep columns until after samples have been loaded from the dataset. This ensures that columns that are needed to compute the transform aren't dropped.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

I think this PR is ready to go, but it's waiting on https://github.com/huggingface/datasets/pull/4553 to be merged and a release in Datasets. Tests will fail until that makes it through, so I won't merge until then!
Trainer.train always saving model every 500 steps, regardless of input training args [fixed, user error]
Author: daniel-furman
Pull request: no
Labels: name: bug

### System Info

Google Colab, GPU enabled

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Use the Trainer.train with either the epochs saving strategy or adjusting the number of steps for saving, either way, the model is saved every 500 steps, which is a huge problem for training situations with a large number of steps! 

### Expected behavior

The training args should change how often the model is saved to disk, but it is not working. 

Comments:
Nevermind, I was using the wrong training args - oops! All working now. 

Closing this issue then :-)
End2End RAG training hangs
Author: YovaKem
Pull request: no
Labels: name: bug

### System Info

```shell
Python 3.9.0 (default, Nov 15 2020, 14:28:56) 
...
>>> transformers.__version__
'4.17.0'
>>> torch.__version__
'1.7.1'
>>> ray.__version__
'1.11.0'
```


### Who can help?

@shamanez, thanks for a great example! I'm having trouble training a model end2end and wonder if you could help out. I have two GPUs available so I've set the arguments relevant for the end2end training to:
```
    --gpus 1  \
    --end2end \
    --distributed_retriever ray \
    --num_retrieval_workers 4  \
    --index_gpus 1 \
    --gpu_order [3,6]
```

This configuration doesn't seem to work since it results in `len(self.retrieval_workers)=0` which means that the call to `re_load()` in `RagRayDistributedRetriever` just hangs eternally. 

I also tried setting `--gpu 2` above, but that breaks with error 

```*** ValueError: Failed to look up actor with name 'retrieval_worker_0'. This could because 1. You are trying to look up a named actor you didn't create. 2. The named actor died. 3. You did not use a namespace matching the namespace of the actor.```

I don't quite know how ray works so I don't know if ranging `re_load()' to this would break it:

```
        if len(self.retrieval_workers) > 0:
          ray.get([worker.clear_object.remote() for worker in self.retrieval_workers])

          # build the index object again
          index = self._build_index(self.config)

          ray.get(
              [
                  worker.create_rag_retriever.remote(
                      self.config, self.question_encoder_tokenizer, self.generator_tokenizer, index
                  )
                  for worker in self.retrieval_workers
              ]
          )
        else:
          self.index = self._build_index(self.config)
```

What are your thoughts? Thanks!

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Set end2end arguments as shown above.

### Expected behavior

```shell
Continuous training without hanging at function re_load()
```


Comments:
Hi this bug is caused by the laters RAY version. Actually, I fixed it in the original RAG repository :). 

Please change this [line](https://github.com/huggingface/transformers/blob/main/examples/research_projects/rag-end2end-retriever/finetune_rag.py#L692)  similar to [this](line). 

In other words, just add a name pace into the RAY init. :) 

Thanks a lot!

Hi @shamanez , Change which line similar to which line? the link ([this](https://github.com/huggingface/transformers/issues/line)) is broken.

I've updated the entire codebase. please check the latest version.
Bert output last hidden state
Author: ehsan-soe
Pull request: no
Labels: name: wontfix

## ❓ Questions & Help

Hi,

Suppose we have an utterance of length 24 (considering special tokens) and we right-pad it with 0 to max length of 64.
If we use Bert pertained model to get the last hidden states, the output would be of size [1, 64, 768]. 
Can we use just the first 24 as the hidden states of the utterance? I mean is it right to say that the output[0, :24, :] has all the required information?
I realized that from index 24:64, the outputs has float values as well.

Comments:
Hello! I believe that you are currently computing values for your padding indices, resulting in your confusion. There is a parameter `attention_mask` to be passed to the `forward`/`__call__` method which will prevent the values to be computed for the padded indices!

@LysandreJik thanks for replying.
Consider the example given in the modeling_bert.py script:

```
        tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
        model = BertModel.from_pretrained('bert-base-uncased')
        input_ids = torch.tensor(tokenizer.encode("Hello, my dog is cute")).unsqueeze(0)  # Batch size 1
        padding = [0] * ( 128 - len(input_ids))
        input_ids += padding

        attn_mask = input_ids.ne(0) # I added this to create a mask for padded indices
        outputs = model(input_ids, attention_mask=attn_mask)
        last_hidden_states = outputs[0]  # The last hidden-state is the first element of the output tuple
```
even with passing attention_mask parameter, it still compute values for the padded indices.
Am I doing something wrong?

> Can we use just the first 24 as the hidden states of the utterance? I mean is it right to say that the output[0, :24, :] has all the required information?
> I realized that from index 24:64, the outputs has float values as well.

yes, the remaining indices are values of padding embeddings, you can try/prove it out by different length of padding 

take a look at that post #1013 (XLNet) and #278 (Bert)

@cherepanovic Thanks for your reply. 
Oh See, I tried padding w/wo passing attention mask and I realized the output would be completely different for all indices.
So I understand that when we use padding we must pass the attention mask for sure, this way the output (on non padded indices) would be equal (not exactly, but almost) to when we don't use padding at all, right?


>  would be equal (not exactly, but almost)

right

@cherepanovic Just my very main question is are the output values in the padded indices, create noise or in other word misleading? or can we just make use of the whole output without being worried that for example the last 20 indices in the output is for padded tokens.

@ehsan-soe can you describe your intent more precisely 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


**would be equal (not exactly, but almost)**

Why are they not exactly the same? (assuming all random seeds are set the same)
How to change config parameters when loading the model with `from_pretrained`
Author: danyaljj
Pull request: no

I have created a model by extending `PreTrainedBertModel`: 

```python 
class BertForMultiLabelClassification(PreTrainedBertModel):

    def __init__(self, config, num_labels=2):
        super(BertForMultiLabelClassification, self).__init__(config)
        self.num_labels = num_labels
        self.bert = BertModel(config)
        self.dropout = nn.Dropout(config.attention_probs_dropout_prob)
        self.classifier = nn.Linear(config.hidden_size, num_labels)
        self.apply(self.init_bert_weights)

    # some code here ... 
```

I am creating an instance of this model: 

```python 
model = BertForMultiLabelClassification.from_pretrained(args.bert_model,
                                                            cache_dir=PYTORCH_PRETRAINED_BERT_CACHE / 'distributed_{}'.format(
                                                                args.local_rank),
                                                            num_labels=num_labels)
```
what is an effective way to modify parameters of the default config, when creating an instance of `BertForMultiLabelClassification`? (say, setting a different value for `config.hidden_dropout_prob`). 
Any thoughts on what is an effective way to do this?


Comments:
I have the same question. I need to change the `hidden_dropout_prob`. How is that possible?

Same question. Any best practice? 

Oh, I find this code works:
```python
hidden_droput_prob = 0.3
config = BertConfig.from_pretrained("bert-base-uncased", num_labels=num_labels, hidden_dropout_prob=hidden_dropout_prob)
model = BertForMultiLabelClassification.from_pretrained("bert-base-uncased", config=config)
```
And `print(model)` could see that drop_out changes:
```sh
  (output): BertOutput(
    (dense): Linear(in_features=3072, out_features=768, bias=True)
    (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
    (dropout): Dropout(p=0.3, inplace=False)
```


Hi @Opdoop - this code overrides the config parameters of the pertained BERT model. Did I understand correctly? 

Also, how can we ensure that the `num_labels` parameter is also updated? I don't see it in the output of `print(model)`.

@kaankork 
* For override:
Yes. Your understand is correct.
* For `num_labels`:
It's been a long time. I didn't sure. But you should see `num_labels` by the last layer shape in `print(model)` .

Any update on this?

I need to pass some values to config as well, will save me a lot of time....

Just use the `update` method. 
For example, if you want to change the number of hidden layers, simply use `config.update({'num_hidden_layers': 1})`.
MarianMTModel no longer has postprocess_next_token_scores function
Author: dsvilarkovic
Pull request: no

Hello I just saw that in file https://github.com/huggingface/transformers/blob/v4.19.2/src/transformers/data/test_generation_utils.py you mention calling the postprocess_next_token_scores() method from MarianMTModel, which doesn't seem to exist anymore in it's source file anymore: https://github.com/huggingface/transformers/blob/v4.19.2/src/transformers/models/marian/modeling_marian.py and neither exist as a PreTrainedModel method anymore, and neither in GenerationMixin model (in GenerationMixin it used to be most recently in https://huggingface.co/transformers/v3.3.1/_modules/transformers/generation_utils.html).

I would be really happy if you could tell what is the alternative to this function in the most recent library version!

Comments:
cc @patil-suraj 

Hi @dsvilarkovic !

This functionality still exists, but the `generate` method is refactored to support this more cleanly. This is now implemented as `LogitsProcessor`. cf #6949

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix slow CI by pinning resampy
Author: sgugger
Pull request: yes

# What does this PR do?

The recent release of resampy (0.3.1) seems to suddenly make a lot of things (even unrelated to speech) very slow and the CI has several jobs timing out. This PR fixes that by pinning resampy to any previous version.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Update localized READMES when template is filled.
Author: sgugger
Pull request: yes

# What does this PR do?

When adding a new model, `make fix-copies` will automatically create a new entry for the model in the README if the user forgot, with some `<FILL XXX>` to fill. This is also added in the localized READMES.

Once this has been done, the localized READMES are not updated if the user has filled the template on the main README, which is a bit annoying. We can see an example in #17821 

This PR fixes that.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Model  LayoutLMv3  - error feature_extractor not find pytesseract
Author: DimIsaev
Pull request: no
Labels: name: bug

### System Info

transformers 4.20.1
tesseract 5.1.0-32-gf36c0
pytesseract 0.3.9 




### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Colab 

Install librires

```
!apt -qq update
!add-apt-repository ppa:alex-p/tesseract-ocr-devel -y
!apt -qq install -y tesseract-ocr

!tesseract --version | grep tesseract
!tesseract --list-langs
```

```
>>> tesseract 5.1.0-32-gf36c0
>>> List of available languages in "/usr/share/tesseract-ocr/5/tessdata/" (3):
eng
osd
```


```
from transformers import LayoutLMv3FeatureExtractor
from PIL import Image

image = Image.open("/content/3875.jpg").convert("RGB")
```

error code
```
feature_extractor = LayoutLMv3FeatureExtractor()
encoding = feature_extractor(image, return_tensors="pt")
print(encoding.keys())
dict_keys(['pixel_values', 'words', 'boxes'])
```

Error message
```
NameError                                 Traceback (most recent call last)
[<ipython-input-29-5693feae799a>](https://localhost:8080/#) in <module>()
      1 # option 1: with apply_ocr=True (default)
      2 feature_extractor = LayoutLMv3FeatureExtractor()
----> 3 encoding = feature_extractor(image, return_tensors="pt")
      4 print(encoding.keys())
      5 # dict_keys(['pixel_values', 'words', 'boxes'])

1 frames
[/usr/local/lib/python3.7/dist-packages/transformers/models/layoutlmv3/feature_extraction_layoutlmv3.py](https://localhost:8080/#) in apply_tesseract(image, lang)
     51 
     52     # apply OCR
---> 53     data = pytesseract.image_to_data(image, lang=lang, output_type="dict")
     54     words, left, top, width, height = data["text"], data["left"], data["top"], data["width"], data["height"]
     55 

NameError: name 'pytesseract' is not defined
```






### Expected behavior

not error, OCR  boxes

Comments:
Hi,

Can you verify that you can do `import pytesseract`? It may be that you need to restart the runtime in Colab.

> Hi,
> 
> Can you verify that you can do `import pytesseract`? It may be that you need to restart the runtime in Colab.

restart solved the problem
Trainer.predict multiple progress bars
Author: neverix
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.21.0.dev0
- Platform: Linux-5.4.188+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.13
- Huggingface_hub version: 0.8.1
- PyTorch version (GPU?): 1.11.0+cu113 (False)
- Tensorflow version (GPU?): 2.8.2 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help?

@sgugger 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```
model = AutoModelForSequenceClassification.from_pretrained("bert-base-uncased")
trainer = Trainer(model)
trainer.predict([{"input_ids": torch.zeros(20).long()} for _ in range(48)])  # [6/6  00:03]
trainer.predict([{"input_ids": torch.zeros(20).long()} for _ in range(96)])  # [6/6  00:12] despite having more examples
```

### Expected behavior

Two progress bars, one of length 6 and the other of length 12.


Comments:
![screenshot demonstrating issue](https://user-images.githubusercontent.com/46641404/176514686-1e47aa34-0e81-4110-b8d3-f886c19b7dfe.png)

The cause of the issue is that `Trainer.predict()` calls `on_prediction_step` but not `on_evaluate` for `predict()`, so every prediction run after the first one will reuse the progress bar object because `on_evaluate` is the callback responsible for destroying it.

A simple fix would be to add an `on_predict` method to the `ProgressCallback`.

Alternatively, `Trainer.predict` could just call `on_evaluate` in the end.

There is no `on_predict` event, but I guess we can reuse `on_evaluate` here. Do you want to make a PR?

I wrote a draft, but it breaks in Jupyter because `NotebookProgressBar` adds [custom logic](https://github.com/huggingface/transformers/blob/3f936df66287f557c6528912a9a68d7850913b9b/src/transformers/utils/notebook.py#L318) to `on_evaluate`. Creating `on_predict` might be necessary.

Feel free to create it then!
Make predict() close progress bars after finishing (#17952)
Author: neverix
Pull request: yes

Fixes #17952 by adding an `on_predict` callback. 

## Who can review?

@sgugger


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Not sure about the tests, should they be there for notebooks too? I'll go with this for now

I just triggered all them by pushing a copy of your branch in the main fork of the repo, circleCI is very finnicky. Let's check everything is green!

Failure is flaky, so this is good to merge, thanks again!