noamaanMulla-03/transformers-issues huggingface dataset:
Add `init_include_buffers` kwargs to `modeling_utils.from_pretrained`
Author: shingjan
Pull request: no

### Feature request

There is a parameter for `init_empty_weights` context manager from `accelerate`, `include_buffers` which defines if buffers of nn.Module should be initialized as meta tensors during loading. I propose that we expose this param as a kwarg in `modeling_utils`.

### Motivation

The motivation is that now with https://github.com/huggingface/accelerate/pull/1826, user can use `torch.device` as context in `accelerate` with `init_empty_weights`, it would be great if this argument can be exposed to `model.from_pretrained` to give user more control over `init_empty_weights`.

### Your contribution

~Will send out a PR for this~ PR #25449 is drafted for this feature proposal.

cc: @sgugger 

Comments:
Sadly the arguments we had to the main method of Transformers are limited to big use cases as we don;t want this method to get to 100 kwargs. For this it would be better to define an env variable in accelerate that would switch the default of `include_buffers` which would allow you to do this change without adding a new kwarg, all-the-while preserving the existing default.

@sgugger I see. That makes sense. Thanks!
Difference between AutoTokenizer.from_pretrained and BertTokenizer.from_pretrained
Author: rxlian
Pull request: no

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

While loading pretrained BERT model, what's the difference between AutoTokenizer.from_pretrained and BertTokenizer.from_pretrained? I'm very new to transformers and still confused about some basic things.
Thanks. 
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:


Comments:
Hi! The [documentation](https://huggingface.co/transformers/model_doc/auto.html) mentions:

> In many cases, the architecture you want to use can be guessed from the name or the path of the pretrained model you are supplying to the from_pretrained method.
> 
> AutoClasses are here to do this job for you so that you automatically retrieve the relevant model given the name/path to the pretrained weights/config/vocabulary:
> 
> Instantiating one of AutoModel, AutoConfig and AutoTokenizer will directly create a class of the relevant architecture (ex: model = AutoModel.from_pretrained('bert-base-cased') will create a instance of BertModel).

So if the string with which you're calling `from_pretrained` is a BERT checkpoint (like `bert-base-uncased`), then this:

```py
AutoTokenizer.from_pretrained("bert-base-uncased")
```
is the same as this:
```py
BertTokenizer.from_pretrained("bert-base-uncased")
```

However, `Auto*` are more flexible as you can specify any checkpoint and the correct model will be loaded, e.g.:

```py
AutoTokenizer.from_pretrained("gpt2")  # works and returns the correct GPT2Tokenizer instance
BertTokenizer.from_pretrained("gpt2")  # fails
```

Closing this for now, let me know if you have other questions.

@LysandreJik 

I'm trying to recreate the  
```train_new_from_iterator``` 
method from the 
```class PreTrainedTokenizerFast(PreTrainedTokenizerBase)```
class

But the class that we have for training is different

```
auto = transformers.AutoTokenizer.from_pretrained("bert-base-cased")
print(type(auto))
<class 'transformers.models.bert.tokenization_bert_fast.BertTokenizerFast'>
```

What is the correct way to do so?

very nice Infromation AutoTokenizer

tanks a lot

Thanks!
AttributeError: 'TFBertForQuestionAnswering' object has no attribute 'prepare_tf_dataset'
Author: daniau23
Pull request: no

### System Info

transformers → 4.18.0
datasets- → 2.14.4
tensorflow -->2.10.0
python --> 3.8.13

### Who can help?

text models: @ArthurZucker and @younesbelkada

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

So I am following the Huggingface course for Question answering and I am getting this error
`AttributeError : ‘TFBertForQuestionAnswering’ object has no attribute ‘prepare_tf_dataset’`

from the code given in the course
`tf_train_dataset = model.prepare_tf_dataset(
train_dataset,
collate_fn=data_collator,
shuffle=True,
batch_size=16,
)
tf_eval_dataset = model.prepare_tf_dataset(
validation_dataset,
collate_fn=data_collator,
shuffle=False,
batch_size=16,
)`

Course link:[Question answering - Hugging Face NLP Course](https://huggingface.co/learn/nlp-course/chapter7/7?fw=tf#post-processing)

### Expected behavior

Should have returned the needed training and validation datasets.

Comments:
free(): invalid pointer
Author: Geremia
Pull request: no

### System Info


Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 4.32.0.dev0
- Platform: Linux-6.1.42-x86_64-AMD_Ryzen_Threadripper_2990WX_32-Core_Processor-with-glibc2.37
- Python version: 3.9.17
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.2
- Accelerate version: 0.21.0
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.1.0.dev20230812+cu121 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no



### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```bash
$ ct2-transformers-converter 
free(): invalid pointer
Annullato
```

### Expected behavior

I expect it not to encounter this error.

Comments:
There is nothing linked to the Transformers library in the reproducer you shared. Are you sure you are opening the issue in the right repo?

@sgugger No, I'm not sure. Is `ct2-transformers-converter` not part of Hugging Face's Transformers? [OpenNMT documentation](https://opennmt.net/CTranslate2/guides/transformers.html#transformers) seemed to imply it was:

```bash
pip install transformers[torch]
ct2-transformers-converter --model facebook/m2m100_418M --output_dir ct2_model
```

Update: [`ct2-transformers-converter` is part of CTranslate2.](https://github.com/OpenNMT/CTranslate2/blob/61d34502325bfa3c5ef8a11cd2e391d0efed1bf9/python/setup.py#L122)
Error loading models in 8-bit
Author: RonanKMcGovern
Pull request: no

### System Info

There is an error stating that bitsandbytes and accelerate are required (even though the latest versions were already installed.

```
!pip install -q -U git+https://github.com/huggingface/accelerate.git
!pip install -q -U bitsandbytes
!pip install -q -U git+https://github.com/huggingface/transformers.git

model_id = 'meta-llama/Llama-2-7b-chat-hf'

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

model = AutoModelForCausalLM.from_pretrained(model_id, device_map="cpu", load_in_8bit=True)
```

I then tried with `!pip install transformers==4.31`

and that resolved the issue. @gante likely related to the prior [issue](https://github.com/huggingface/transformers/pull/25411#event-10057101239)


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

See above

### Expected behavior

Model should load.

Comments:
I tried to run this with colab notebook CPU and got same error but once I run it on T4 GPU no errors appeared and worked so I assume you can not use **load_in_8bit=True** on cpu device actually. you can still keep **device_map="cpu"** but  you should have GPU environment I guess. 

Yes `load_in_8bit` requires using GPUs, not CPUs.

Correct, sorry for the false flag @gante
Potential security issue
Author: psmoros
Pull request: no

Hello 👋

I run a security community that finds and fixes vulnerabilities in OSS. A researcher (@b3ef) has found a potential issue, which I would be eager to share with you.

Could you add a `SECURITY.md` file with an e-mail address for me to send further details to? GitHub [recommends](https://docs.github.com/en/code-security/getting-started/adding-a-security-policy-to-your-repository) a security policy to ensure issues are responsibly disclosed, and it would help direct researchers in the future.

Looking forward to hearing from you 👍

(cc @huntr-helper)

Comments:
You can report the security issue on huntr.dev, we monitor this website (cc @coyotte508 ).

Hey @sgugger I'm actually from huntr.dev; sorry for reaching we already have your email :) See you!

Ah ah, no problem!

Send it to security@huggingface :)
Set can_generate for SpeechT5ForTextToSpeech
Author: ylacombe
Pull request: yes

# What does this PR do?

Following the [discussion](https://github.com/huggingface/transformers/pull/24952#discussion_r1293224895) on checking whether the generate or forward method will be used in the TTS pipeline, it makes sense to set `can_generate=True` for `SpeechT5ForTextToSpeech`, so that it's easier to check if it can generate.



<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->



## Before submitting
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? 

## Who can review?

Hey @sanchit-gandhi and @sgugger , what do you think of this??
Thanks!


Comments:
_The documentation is not available anymore as the PR was closed or merged._
decoder_kwargs are not passed over to AutomaticSpeechRecognitionPipeline.tokenizer.decode 
Author: devxpy
Pull request: no

### Feature request

The `postprocess()` function here

https://github.com/huggingface/transformers/blob/fb78769b9c053876ed7ae152ee995b0439a4462a/src/transformers/pipelines/automatic_speech_recognition.py#L492-L494

should pass down the `decoder_kwargs` it receives down to the decoder

https://github.com/huggingface/transformers/blob/fb78769b9c053876ed7ae152ee995b0439a4462a/src/transformers/pipelines/automatic_speech_recognition.py#L563



### Motivation

Sometimes the decoder will output special tokens - https://github.com/huggingface/transformers/issues/15275 - and there's no way to pass `skip_special_tokens=True` to the decoder 

https://github.com/huggingface/transformers/blob/fb78769b9c053876ed7ae152ee995b0439a4462a/src/transformers/models/wav2vec2/tokenization_wav2vec2.py#L407-L416

### Your contribution

I can submit a PR

Comments:
From https://github.com/huggingface/transformers/blob/fb78769b9c053876ed7ae152ee995b0439a4462a/src/transformers/pipelines/automatic_speech_recognition.py#L549-L552

That `decoder_kwargs` is for an instance of `BeamSearchDecoderCTC` and not a tokenizer.

But cc @ArthurZucker to see if he has more words to say.


I guess we can have `tokenizer_decoder_kwargs` too :)

Yep, pipeline does not support `**tokenizer_kwargs` yet. This has been talked about in #22995 and #12039. 
I am in for `tokenizer_kwargs`, not `tokenizer_decoder_kwargs`. We need to as less specific as possible. 

A con is that having lots of kwargs is hard to maintain and we are trying to get away from this. The tokenizer class can save the `_init_kwargs` which contains the last parameters with which the tokenizer was called. You can set them and pass the tokenizer to the pipeline 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
OSError: Error no file named pytorch_model.bin
Author: wangzff
Pull request: no

### System Info



transformers==4.30.2

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

(base) /mnt/workspace/lawyer-llama/demo> python demo_web.py --port 7863 --checkpoint /mnt/workspace/lawyer-llama/lawyer-llama-13b-beta1.0
Loading model...
Traceback (most recent call last):
File "/mnt/workspace/lawyer-llama/demo/demo_web.py", line 52, in
model = LlamaForCausalLM.from_pretrained(checkpoint, device_map="auto", torch_dtype=torch.float16)
File "/home/pai/lib/python3.9/site-packages/transformers/modeling_utils.py", line 2449, in from_pretrained
raise EnvironmentError(
OSError: Error no file named pytorch_model.bin, tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory /mnt/workspace/lawyer-llama/lawyer-llama-13b-beta1.0.

transformers==4.30.2

### Expected behavior

help me！

(base) /mnt/workspace/lawyer-llama/demo> python demo_web.py --port 7863 --checkpoint /mnt/workspace/lawyer-llama/lawyer-llama-13b-beta1.0
Loading model...
Traceback (most recent call last):
File "/mnt/workspace/lawyer-llama/demo/demo_web.py", line 52, in
model = LlamaForCausalLM.from_pretrained(checkpoint, device_map="auto", torch_dtype=torch.float16)
File "/home/pai/lib/python3.9/site-packages/transformers/modeling_utils.py", line 2449, in from_pretrained
raise EnvironmentError(
OSError: Error no file named pytorch_model.bin, tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory /mnt/workspace/lawyer-llama/lawyer-llama-13b-beta1.0.

transformers==4.30.2

Comments:
Hi @wangzff, thanks for raising this issue. 

If you look at the contents of the checkpoint passed into the script, do you see the model weights? i.e. what is the output of `ls -al mnt/workspace/lawyer-llama/lawyer-llama-13b-beta1.0`?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
self_attention_mask
Author: oroojlooy
Pull request: no

### System Info


- `transformers` version: 4.29.2
- Platform: Linux-5.4.17-2136.318.7.1.el7uek.x86_64-x86_64-with-glibc2.17
- Python version: 3.8.16
- Huggingface_hub version: 0.15.1
- Safetensors version: not installed
- PyTorch version (GPU?): 2.0.1+cu118 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>


### Who can help?

@sg

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

When I call the `bigcode/starcoderbase` model, the output comes `nan`. I debugged the code and found that it happens in the `modeling_gpt_bigcode.py` where it gets the attention output in `GPTBigCodeAttention._attn()`. 

Moving back to find the root of the issue I find that the `attention-mask` is edited in `GPTBigCodeModel.forward()` function as shown below. I could not understand the reason behind this modification of the `attention_mask` so though better I ask it here to see if it is intended or if there is any bug in that.
 

```
        # Self-attention mask.
        query_length = input_shape[-1]
        key_length = past_length + query_length
        self_attention_mask = self.bias[None, key_length - query_length : key_length, :key_length]

        if attention_mask is not None:
            self_attention_mask = self_attention_mask * attention_mask.view(batch_size, 1, -1).to(
                dtype=torch.bool, device=self_attention_mask.device
            )

        # MQA models: (batch_size, query_length, n_heads, key_length)
        # MHA models: (batch_size, n_heads, query_length, key_length)
        attention_mask = self_attention_mask.unsqueeze(2 if self.multi_query else 1)
```

Note that this modification results in attention mask like `[False, False, ...., False]` which using it in the `_attn()` functions results in:
```
attn_weights = torch.where(attention_mask, attn_weights, mask_value) 
	attn_weights[1,1,1,:].max() = -\infty
attn_weights = softmax(attn_weights, dim=-1)
	attn_weights[1,1,1,:].max() = [nan, ..., nan] (size of 2048)
```



### Expected behavior

As above. 

Comments:
cc @ArthurZucker @younesbelkada 

Hey @oroojlooy thanks for opening an issue. Sorry didn't really have time to dive into this, could you try with the latest version of `transformers`. Could you also share you script on how you called the model! 🤗 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
🚨🚨🚨 Remove softmax for EfficientNetForImageClassification 🚨🚨🚨
Author: amyeroberts
Pull request: yes

# What does this PR do?

 **\/!\ This is a breaking change /!\\** 

EfficientNet implementation erroneously added a softmax to the models logits in the classification head. This breaks with convention and [documentation](https://github.com/huggingface/transformers/blob/87c9d8a10f3935a46fc7b254d3bea97bca4bfcce/src/transformers/modeling_outputs.py#L1215) for the [model outputs](https://github.com/huggingface/transformers/blob/87c9d8a10f3935a46fc7b254d3bea97bca4bfcce/src/transformers/models/efficientnet/modeling_efficientnet.py#L623). 

This results in the very small probabilities seen in the hosted widgets on the checkpoints pages, as the logits effectively have softmax applied twice. 

<img width="1568" alt="image" src="https://github.com/huggingface/transformers/assets/22614925/111c8346-f679-4c2e-8800-b36cf5b385ff">


Most of the efficientnet checkpoints were downloaded < 100 times in the past month. However, [efficientnet-b7](https://huggingface.co/google/efficientnet-b7) is more popular with a few thousand downloads. 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
cc @rwightman Managed to track down the issue with the efficient net predictions on the hub 

_The documentation is not available anymore as the PR was closed or merged._
Remove logging code in TF Longformer that fails to compile
Author: Rocketknight1
Pull request: yes

This PR fixes an issue where one of the logging blocks in `TFLongformer` was hidden behind a conditional that didn't play nicely with TF compilation. The issue is only triggered in certain compilation paths, which is why our tests didn't pick this up.

Fixes #25418 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

There are 6 occurrences of `if (...) padding_len > 0` in the source file [modeling_tf_longformer.py](https://github.com/huggingface/transformers/pull/25496/files#diff-782b222e9d393fe6750cf8e4cd870bcf3748a92ade5086e518b4d716a80080f8). I recommend fixing them all.

@rdisipio In most cases, TF's `autograph` should correctly recognize and compile those. I believe (though I'm not certain) that the issue arises here because the effect of the conditional is purely logging, without any TF code, and so autograph does not correctly convert that to TF graph operations.

If you encounter issues with the other lines, though, then I'm wrong - please reopen the issue and I'll fix it!

it works now! also thanks for the explanation, I'm not really familiar with what `autograph` does behind the scenes.

Cheers,
R.

No-one is familiar with what `autograph` does behind the scenes, lol. If anyone is reading this and can give me a full explanation of why the conditional failed to compile here but the others succeed, let me know!