noamaanMulla-03/transformers-issues huggingface dataset:
Big TF test cleanup
Author: Rocketknight1
Pull request: yes

Now we've done a big overhaul of the TF model internals, a lot of tests can be fixed. Several tests were disabled for being buggy or too slow - these are almost all performant now, so I re-enabled them. Runtime for the re-enabled tests was 15-20 seconds on my local machine.

Also, we had a number of TF test failures in the daily CI. I think this PR should fix all of them, except for two cases:

Firstly, some models have issues with `resize_token_embeddings`. These failures are caused by the transition to `TFSharedEmbedding` that @gante is currently working on, and I didn't want to interfere! The usual cause is that `resize_token_embeddings` replaces the new-style `TFSharedEmbedding` with an old `tf.Variable`.

Secondly, there are a couple of failures in generate tests. I'm also leaving this to @gante because he knows much more about that code than me :sweat_smile: 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

(@Rocketknight1 ping me if the gen tests are not sorted after the latest push)

I think everything has been addressed now, but I'm not going to merge this one today because there's another PR affecting our tests (#24301) and ideally I'd like to be able to separately view their impact on the CI!

> I think everything has been addressed now, but I'm not going to merge this one today

Nice 👍 .

I never merge PRs on Firday evening or early afternoon. I don't want to get a ☎️ ⚡ !

Wait, you merged ...!? (but you said you are not going to merge 🤔 )
Document what layerdrop does
Author: XekriRedmane
Pull request: no

### System Info

- `transformers` version: 4.28.1
- Platform: Windows-10-10.0.22621-SP0
- Python version: 3.11.2
- Huggingface_hub version: 0.14.1
- Safetensors version: not installed
- PyTorch version (GPU?): 2.0.1+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: NO
- Using distributed or parallel set-up in script?: NO

### Who can help?

@sgugger 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

In `transformers/models/wav2vec2/configuration_wav2vec2.py` there is a parameter `layerdrop` in `__init__` which is not documented. This parameter is set (and overrides the default `0.1`) in the examples at `examples/pytorch/speech-recognition/README.md`, so it seems to be important.

### Expected behavior

Document `layerdrop`.

Comments:
cc @sanchit-gandhi 

Hey @RobertBaruch - good catch! It's indeed missing from the Wav2Vec2 config docstring. Would you like to open a PR to add this info? Easiest would be to copy the details from one of the existing configs where the info is present, e.g. OPT:
https://github.com/huggingface/transformers/blob/130e15429116689c9d747be2cdd8c4be7bb7e2bd/src/transformers/models/opt/configuration_opt.py#L70

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Closed via https://github.com/huggingface/transformers/pull/23691
Problems while Running ImageGPT
Author: chinge55
Pull request: no

### System Info

- `transformers` version: 4.28.1
- Platform: Linux-4.15.0-197-generic-x86_64-with-glibc2.27
- Python version: 3.11.3
- Huggingface_hub version: 0.15.1
- Safetensors version: not installed
- PyTorch version (GPU?): 2.0.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I tried to run the code from the official example: 
https://huggingface.co/docs/transformers/model_doc/imagegpt#transformers.ImageGPTForCausalImageModeling
```python
from transformers import AutoImageProcessor, ImageGPTForCausalImageModeling
import torch
import matplotlib.pyplot as plt
import numpy as np

image_processor = AutoImageProcessor.from_pretrained("openai/imagegpt-small")
model = ImageGPTForCausalImageModeling.from_pretrained("openai/imagegpt-small")
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)

# unconditional generation of 8 images
batch_size = 8
context = torch.full((batch_size, 1), model.config.vocab_size - 1)  # initialize with SOS token
context = torch.tensor(context).to(device)
output = model.generate(
    input_ids=context, max_length=model.config.n_positions + 1, temperature=1.0, do_sample=True, top_k=40
)

clusters = image_processor.clusters
height = image_processor.size["height"]
width = image_processor.size["width"]

samples = output[:, 1:].cpu().detach().numpy()
#Error line below   
samples_img = [
    np.reshape(np.rint(127.5 * (clusters[s] + 1.0)), [height, width, 3]).astype(np.uint8) for s in samples
]  # convert color cluster tokens back to pixels
f, axes = plt.subplots(1, batch_size, dpi=300)

for img, ax in zip(samples_img, axes):
    ax.axis("off")
    ax.imshow(img)
```

Error on line: ```samples_img = [...]``` on the part ```clusters[s]```. 

This is because: 
```samples.shape  = (8,1024)```
At that point, ```s.shape = (1024,)```. 
So, ```Clusters[s]``` cannot index properly. 

**Error Message**:  TypeError: only integer scalar arrays can be converted to a scalar index


### Expected behavior

It should plot 8 different predictions from the ImageGPT Model. 
![image](https://github.com/huggingface/transformers/assets/33897366/504978be-f708-4b67-b28a-5a82a01f0c50)


Comments:
@chinge55 Thanks for reporting! Indeed, the example was assuming the clusters were a numpy array, but were stored as a list of lists in the image processor. I've opened a PR to resolve
Fix ImageGPT doc example
Author: amyeroberts
Pull request: yes

# What does this PR do?

There was a bug in the example, where the clusters in the image processor were stored as lists, but the example assumed they were numpy arrays. 

At the moment, clusters are stored as a list of lists, but [converted to a numpy array during processing](https://github.com/huggingface/transformers/blob/0b7b4429c78de68acaf72224eb6dae43616d820c/src/transformers/models/imagegpt/image_processing_imagegpt.py#L230). This PR converts to a numpy array when setting the class attribute and if new `cluster` are passed into `preprocess`. 

This: 
* Maintains backwards compatibility with old configurations
* Saved configs aren't changed (`clusters` is still converted to a list of lists when serializing) see this [dummy image processor](https://huggingface.co/amyeroberts/dummy_imagegpt_image_processor_np_clusters) created from this branch. 
* Is more efficient - we're not converting the same list of lists every batch. 

A potential breaking change is if users were accessing the `clusters` attribute and using it as a list of lists. As this was caught because users were using the clusters as a numpy array (according to the example) I expect this impact to be low. 

Fixes #24189

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add test for proper TF input signatures
Author: Rocketknight1
Pull request: yes

This is a relatively simple PR intended to check that all TF models have proper input signatures that match their inputs. I was calling the function `self._prune_signature` in a few places to verify this, which felt a bit hacky. This test should let us get rid of `self._prune_signature` by enforcing valid signatures for all models.

Edit: I'm also slipping in a typo fix (fine-tine -> fine-tune) I saw while I was running the tests

Double-edit: I'm also slipping in a fix to an incorrect indentation in the `test_dataset_conversion` test that was causing some unnecessary repetition

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Adding ddp_broadcast_buffers argument to Trainer
Author: TevenLeScao
Pull request: yes

# What does this PR do?

In #22482, using the Trainer with `gpt2` and other similar models failed in naive distributed mode. Passing `ddp_broadcast_buffers=False` to Pytorch's DDP wrapper fixes the issue. This PR surfaces that argument to the Trainer user.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Llama: retrocompatibility support for inner layers
Author: gante
Pull request: yes

# What does this PR do?


Comments:
_The documentation is not available anymore as the PR was closed or merged._

(It seems like it is no longer needed, closing)

@gante hi, i also got the error "None type is not subscribable " in apply_rotary_pos_emb. Why is it no longer need? Why not merge it?

Hi @Adam1679 -- have you followed the conversation in [this issue](https://github.com/huggingface/transformers/issues/22407)? It explains the rationale behind the change 🤗 

@gante yes. but i encountered the same issee without using GPTQ-for-LLaMA package.

@Adam1679 then one of the two things is happening:
1. You are using exclusively hugging face libraries -- ensure you have the latest versions and, if the error persists, please an issue with a short snippet that reproduces the issue
2. You are using external libraries -- request their owners to update the transformers version and/or update their internal code
Decoding with skip_special_tokens=True doesn't remove pad token
Author: Praful932
Pull request: no

### System Info

- `transformers` version: 4.31.0.dev0
- Platform: Linux-5.15.107+-x86_64-with-glibc2.31
- Python version: 3.10.12
- Huggingface_hub version: 0.15.1
- Safetensors version: 0.3.1
- PyTorch version (GPU?): 2.0.1+cu118 (True)
- Tensorflow version (GPU?): 2.12.0 (True)
- Flax version (CPU?/GPU?/TPU?): 0.6.9 (gpu)
- Jax version: 0.4.10
- JaxLib version: 0.4.10
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: -

### Who can help?

@younesbelkada

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

collab nb - https://colab.research.google.com/drive/1VH-ZVITqJ5k6umvMXvQc2Suxzf6KEXyX?usp=sharing

### Expected behavior

I expected the <pad> token to not be present in the output, just the processed text

Comments:
cc @ArthurZucker 

@ArthurZucker can confirm, but I suspect the pad token of the tokenizer is probably not defined correctly, in the notebook you have:

```python
print(tokenizer.pad_token,tokenizer.pad_token_id)
>>> [PAD] 32100
```

and in the generated text: 
```python
print(decoded_outputs)
>>> ['<pad> bhubaneswar
', '<pad> Hippos are large animals
<pad>']
```

You probably need to update the tokenizer with the correct pad token and eos tokens, in your case respectively `<pad>` and `
`

As mentioned by Younes, the padding token is not correct, `tokenizer.additional_special_tokens` does not have the `<pad>`  token, and the `tokenizier.pad_token` is not set to `<pad>`. Thus the token is an `AddedToken` which is why you can see it in the `tokenizer.added_tokens_encoder` for example, but it is not a special token, this it is skipped. 

Thanks for the prompt reply, cool So this is an issue with the model's tokenizer not having the right initial configuration? - and fixing it either by the [author on the model repo](https://huggingface.co/lmsys/fastchat-t5-3b-v1.0) or doing the following step by the user is the immediate remediation for this:
`tokenizer.add_special_tokens({'pad_token' : "<pad>"})`

I can confirm that this fixes the default behaviour
Check for Bool instead of Optionals
Author: seboslaw
Pull request: no

Hey guys, 

forgive me if this note is naive (I'm not a Python professional) but during debugging the code I got thrown off by this line:

```
            if return_dict_in_generate and output_scores:
                beam_indices = tuple((beam_indices[beam_idx[i]] + (beam_idx[i],) for i in range(len(beam_indices))))
```

https://github.com/huggingface/transformers/blame/3335724376319a0c453049d0cd883504f530ff52/src/transformers/generation/utils.py#L2976

It seems like you're simply checking whether `return_dict_in_generate` and `output_scores` are not `None` instead of checking the underlying `Bool`s. I assume this is intended, correct? I'm asking because I passed `False` for both values and was wondering why it would still run the `beam_indices = tuple...` line.

Comments:
Hi @seboslaw, thanks for raising this issue. 

I don't believe the logic there is checking for `None` values. In [L2850](https://github.com/huggingface/transformers/blame/3335724376319a0c453049d0cd883504f530ff52/src/transformers/generation/utils.py#L2850), `return_dict_in_generate` is set to either the bool value passed in, or defaults to the bool value in the config if unset / is None. The same happens to `output_scores` in [L2843](https://github.com/huggingface/transformers/blame/3335724376319a0c453049d0cd883504f530ff52/src/transformers/generation/utils.py#LL2843C18-L2843C18). 

However, even if it's less clear than an explicit `None` check e.g. `if x is None`, the `beam_indices` logic should only ever be executed if both `return_dict_in_generate` and `output_scores` both evaluate to `True` e.g. the following will only print out `True, True`. 

```
for a, b in (
    (None, None), (None, False), (False, None), (True, None), (None, True), (True, False), (False, True), (True, True)
):
    if a and b:
        print(a, b)
```

i.e. if the `beam_indices` line is still executing when both values are being set to `False` there's a probably a bug somewhere. Could you follow the issue template and provide details such that we can help debug this? Specifically:

* The running environment: run `transformers-cli env` in the terminal and copy-paste the output
* A minimal reproducible code snippet we can copy and run to replicate the issue? 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Make FlaxPreTrainedModel a Flax Module
Author: cgarciae
Pull request: yes

# What does this PR do?

WIP. Makes `FlaxPreTrainedModel` a `nn.Module` so Flax users can easily integrate it into other Flax networks or systems that expect Flax Modules. 

See for #22499 some discussion of the approach.


Comments:
@sanchit-gandhi's previous comment:

"""


This looks super clean @cgarciae - really like this way of making the FlaxPreTrainedModel into an nn.Module through dataclasses.

Had some questsions about the PR (mainly for my understanding), but think the general design philosophy looks good here.

From my testing it all seems to work as expected - think though we should add some very visible warning messages though when a user passes _do_init=False to advise them to use the returned model as a Flax nn.Module (rather than falling-back on the __call__ method of FlaxPreTrainedModel). This is the only breaking change I see from this PR, but one that is unavoidable (since it's the exact thing we're trying to change).

=> perhaps as a start we first make a PR that triggers this warning (advising users that the functinality is going to change in N months / releases time), and then have this PR as a follow-up that makes the changes?

For Flax BERT though this approach gives equivalence between the FlaxPreTrainedModel and the nn.Module -> for models that do extra pre-processing we can just modify the __call__ to do all the data pre-processing?

"""

Thanks for the review @sanchit-gandhi !

> => perhaps as a start we first make a PR that triggers this warning (advising users that the functinality is going to change in N months / releases time), and then have this PR as a follow-up that makes the changes?

I was planning on entirely deleting the `params` argument from `__call__` 😅. If we want to make the change a bit more gradual then maybe we could do something like this:

```diff
- if self._do_init:
+ if self.scope is None:
```
This would condition on whether the module being called inside `apply` or not.

@sanchit-gandhi cleaned the PR a little, these are the minimal changes required.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_22627). All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Leaving closed in favour of #22866

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
AutoModelFromCausalLLM of Bloom not releasing GPU memory after each inference batch
Author: chuckhope
Pull request: no

### System Info

Hi there, I have set torch.no_grad() and torch.cuda.empty_cache(), but the GPU still encounters out-of-memory (OOM) errors after a few inferences. My torch version is 1.13.1, deepspeed version is 0.9, and transformer version is 4.28.

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I have set torch.no_grad() and torch.cuda.empty_cache() for AutoModelFromCausalLLM

### Expected behavior

auto release the memory usage 

Comments:
Hi @chuckhope, thanks for raising this issue. 

So that we can best help, could you provide the following information: 
* a minimal code snippet to reproduce the error
* Information about the hardware - are you running on a single GPU? 
* Information about the model - are you using a single checkpoint? If so, which one? If not, is this observed for all checkpoints? 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Device control characters lead to an error in average NER aggregation
Author: edloginova
Pull request: no

### System Info

Python 3.8.10, transformers versions tried: 4.30.1 and 4.30.2. Tried in a Docker container with Linux Ubuntu 20.04 and on Google Colab.

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

1. Load a pretrained language model as NER pipeline with average aggregation
2. Run it on a sequence `"\x11\x11"`

Colab: https://colab.research.google.com/drive/1Xm2kFAIsb1vt8R8JvdiLitDZ2m6WszcP?usp=sharing

The error happens on line 336 of `transformers\pipelines\token_classification.py`, in `aggregate_word()` function: `word = self.tokenizer.convert_tokens_to_string([entity["word"] for entity in entities])`. There, entities remains `None` while it should be `[]`; it seems to happen because in `aggregate()` function (same file), when `aggregation_strategy` is set to average, we call `aggregate_words()` and there `entities` is `[]`, so we end up with `word_group` as still `None`, and that gets passed to `aggregate_word`.

### Expected behavior

The aggregations runs without any errors and produces an empty list.

Comments:
cc @Narsil @ArthurZucker 

Created a fix PR.
Skip `test_disk_offload` for `WhisperModelTest`
Author: ydshieh
Pull request: yes

# What does this PR do?

Since #22486, `WhisperModelTest.test_disk_offload` to fail. I just blindly skip this test and guess it is ok.....?


Comments:
_The documentation is not available anymore as the PR was closed or merged._

I am not able to use `openai/whisper-base` as I get GPU OOM. When I tried to use `openai/whisper-tiny.en`, I have a hard time to change the parameters to get a working input dict for the model.

If I just changed the values in model tester to get a larger model (but random), I get different kinds of error like `IndexError: list index out of range` in `dispatch_model` or `RuntimeError: Tensor on device meta is not on the expected device cuda:0!` in model forward. 

But even if I revert the change in #22486, the above attempts to use larger (fake) models still have the same issue. I guess we will have to look into this.

Convert to draft to avoid being merged.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.