noamaanMulla-03/transformers-issues huggingface dataset:
Models with no max_pos_embeddings  will fail on run_clm.py
Author: AdamLouly
Pull request: no

### System Info

In run_clm.py we set up block_size to config.max_position_embeddings, assuming that all configs will have max_position_embeddings.
some models does not have max_position_embeddings as a part of their definition so they will fail when trying to train using run_clm.py

bloom is one example.

where you get this error if you try to run it:
AttributeError: 'BloomConfig' object has no attribute 'max_position_embeddings'

we can solve this by setting a check on whether the value exist otherwise put a default value on it.
I will create a PR  to address this.


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

run bloom on nightly version of transformers

### Expected behavior

AttributeError: 'BloomConfig' object has no attribute 'max_position_embeddings'

Comments:
Hi @AdamLouly - thanks for reporting an opening a PR to fix!
Fix offload disk for loading derivated model checkpoint into base model
Author: SunMarc
Pull request: yes

# What does this PR
This PR fixes #27199. It was not possible load weights from a derived model into the base model when we are offloading parameters to the disk +  safetensors. The issue was that we were not able to create the right `offload_index` because the weights didn't match. 

Example : 

```py
from transformers import AutoModel
device_map = {'embed_tokens': 0, 'layers.0': "disk", 'layers.1': 0, 'layers.2': 0, 'layers.3': 0, 'layers.4': 0, 'layers.5': 0, 'layers.6': 0, 'layers.7': 1, 'layers.8': 1, 'layers.9': 1, 'layers.10': 1, 'layers.11': 1, 'layers.12': 1, 'layers.13': 1, 'layers.14': 1, 'layers.15': 1, 'layers.16': 2, 'layers.17': 2, 'layers.18': 2, 'layers.19': 2, 'layers.20': 2, 'layers.21': 2, 'layers.22': 2, 'layers.23': 2, 'layers.24': 2, 'layers.25': 3, 'layers.26': 3, 'layers.27': 3, 'layers.28': 3, 'layers.29': 3, 'layers.30': 3, 'layers.31': 3, 'norm': 3}
model = AutoModel.from_pretrained("meta-llama/Llama-2-7b-hf", device_map = device_map, use_safetensors=True)
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks for the review @amyeroberts ! Any idea where I should add this test ? I don't think we want to test this for all our models but maybe just for one ? 

@SunMarc Agreed - just for one model should be enough. I'd suggest putting them in [test_modeling_utils.py](https://github.com/huggingface/transformers/blob/88832c01c8a962b653874c4ce4ed8df5783ac5cd/tests/test_modeling_utils.py#L4)
Simplify getting the number of embedding tokens.
Author: kwonmha
Pull request: yes

Simplifies the way to get the number of embedding tokens.

Following additional discussions on PR #26024 


## Who can review?

@ArthurZucker


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26333). All of your documentation changes will be reflected on that endpoint.

@pacman100 
hello, please have a look at this PR.
It's been a while since I created this PR.

From #26112, I understood that `weight.shape[0]` always has accurate info and `num_embedding` doesn't.
But I can say that in this case, right after embedding parameter has been created, it looks OK.

Anyway, it's up to you.
Feel free to close this PR.

Hello @kwonmha , yes, as per that comment, this PR doesn't look like a good fix as you are changing the code which get the embedding size post resizing of the embedding layer. Closing this PR, thank you!
[`pytest`] Avoid flash attn test marker warning
Author: ArthurZucker
Pull request: yes

# What does this PR do?

Adds the `flash_attn_test` marker to the `pyproject.toml` to remove the following warning:
```python 
tests/test_modeling_common.py:3131
  /home/arthur/transformers/tests/test_modeling_common.py:3131: PytestUnknownMarkWarning: Unknown pytest.mark.flash_attn_test - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @mark.flash_attn_test

tests/test_modeling_common.py:3174
  /home/arthur/transformers/tests/test_modeling_common.py:3174: PytestUnknownMarkWarning: Unknown pytest.mark.flash_attn_test - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @mark.flash_attn_test
```
Allows to de-select them with:
```python 
pytest ... -m "not flash_attn_test"
```

Comments:
Adding a model
Author: Ranitbag007
Pull request: no
Labels: name: New model

### Model description

I have built a llm and can you tell me how to register my own tokenizer with huggingface hub?


### Open source status

- [x] The model implementation is available
- [ ] The model weights are available

### Provide useful links for the implementation

_No response_

Comments:
Hi @Ranitbag007, 

If you're wanting general instructions, you can find the information in our documentation. [This section details](https://huggingface.co/docs/transformers/v4.15.0/model_sharing#add-new-files-to-your-model-repo) adding a tokenizer to the hub. [This page](https://huggingface.co/docs/transformers/v4.35.2/en/custom_models#writing-a-custom-configuration) details how to add a model to be loaded through the `AutoModel` API. The equivalent for tokenizers is having the tokenizer inherit from `PretrainedTokenizer`.  

If you've tried this and there's an issue, you'll need to open an issue, following the template with full details about what you've tried and the error messages occurred. 

I am using sentencepiece tokenizer model  and unable to register the tokenizer 

As per [my comment here](https://github.com/huggingface/transformers/issues/27427#issuecomment-1805735559), you need to follow the issue template, providing full details of your running environment, a reproducible code snippet and details of the error encountered.
docs: add docs for map, and add num procs to load_dataset
Author: pphuc25
Pull request: yes

Hi,
In the .map, this do both for train and eval dataset, so I add description of .map train and eval to make them clear and logs in terminal look clear
I also add num_proc in load_dataset with the purpose of faster

I would like to cc @stevhliu review my PR.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_27520). All of your documentation changes will be reflected on that endpoint.
Update the TF pin for 2.15
Author: Rocketknight1
Pull request: yes

This PR updates the TF pin to allow TF 2.15. Leaving it as a draft for now while I make sure this doesn't break anything!

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Seems to work locally! Marking this as ready for review.

@ydshieh I didn't know that! It seems to be caused by one of the dependencies being slightly behind - 2.14 is the most recent version, and 2.15 is in RC.

I'm not sure what to do about it, though. Since local tests pass, though, we can probably assume that 2.15 is okay?

> we can probably assume that 2.15 is okay?

As the CI won't run with 2.15 (with and after this PR), **it's not a blocking factor to merge this PR.**

My comment above is just to say we probably need to check what happens and decide if  to make effort to have 2.15 installed for CI. If TF 2.15 works will be relevant only at that point.



Understood - I suspect the issue is that one of the dependencies (I think maybe `tf2onnx`?) has a version limit for TF. Whenever they update that cap, our CI will probably suddenly move to 2.14. Either way, I'll merge this PR for now, since local tests for me suggest 2.14 and 2.15 are working fine.
Revert "add attention_mask and position_ids in assisted model"
Author: patrickvonplaten
Pull request: yes

Reverts huggingface/transformers#26892 as it breaks speculative decoding of Whisper

Comments:
Hi @patrickvonplaten . Sorry for making this mistake, would you please wait for the new [PR](https://github.com/huggingface/transformers/pull/27503) merge, it should have fixed the problem.

@patrickvonplaten -- @jiqing-feng's PR should fix it :) The PR being reverted here did break speculative decoding for encoder-decoder architectures. CI did not caught it since the non-slow test is based on sample and it had a lucky run.

Merging so that distil whisper works again
ERROR | index 4 is out of range
Author: hdnh2006
Pull request: no

### System Info

Hello everyone!

I am trying to deploy [this](https://huggingface.co/facebook/musicgen-stereo-small) model for testing purposes and I am facing several issues to run it in both, locally and using huggingface endpoints.

I have followed exactly [this](https://huggingface.co/blog/run-musicgen-as-an-api) and all the steps. However, in both, locally and in inference endpoints, I am facing the same issue and I get this error:
```
IndexError: index 4 is out of range
```
I just was able to run the model on Google Colab and I realized the transformers version was `4.36.0.dev0` so I tried to put on my `requirements.txt` file something like `transformers>=4.36.0` but it seems this version is not available on inference endpoints docker container.

So, in my particular case, do you have any idea how to solve this issue?

Thanks in advance

### Who can help?

I don't know who I must tag, sorry if I'm wrong:
@Narsil @ArthurZucker  

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```
# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-to-audio", model="facebook/musicgen-stereo-small")
```

### Expected behavior

The token or audio provided by the model.

Comments:
Hi @hdnh2006, thanks for raising this issue! 

Version `v4.26.0.dev0` indicates that the installed transformers was from the `main` branch on git. We haven't release 4.26 yet, so you can't install directly using pip and the library name.

To install from source (dev version):
```
pip install git+https://github.com/huggingface/transformers
```

Thanks for the information, I did it and it worked!
Set `usedforsecurity=False` in hashlib methods (FIPS compliance)
Author: Wauplin
Pull request: yes

Solves https://github.com/huggingface/transformers/issues/27034 (cc @DueViktor).

This PR makes `transformers` FIPS-compliant regarding hashlib usage by setting `usedforsecurity=False` in every hashlib method (used for file checking, not cryptography purposes). It's based on utilities added in https://github.com/huggingface/huggingface_hub/pull/1782 and released in `huggingface_hub==v0.19.0`. 

**Note:** before merging this we need to release a new tokenizers version that would allow new `huggingface_hub` version (see https://github.com/huggingface/tokenizers/pull/1385). Tests are currently failing because of this which is expected. cc @ArthurZucker what's the status on the next release?

Comments:
> Thanks! Not 100% sure we need to rely on the utility as it's a pretty small piece of code for something pretty code + not very intuitive to me that we need to import from hf hub. If others are fine good for me as well

Agree it's not big piece of code. But having it from an explicitly named `insecure_hashlib` module makes it more easily distinguishable IMO => no need to check each hashlib call to verify that the parameter is correctly set. Also you would have to duplicate the logic that checks whether you are running on python 3.9 or not (2 lines, but still not best to copy-paste around). 
Also the dependency on `huggingface_hub` already exists so no big deal IMO (and updating to latest version  forces to get the latest fixes we shipped -that's more of a cool side-effect)

You are right! 🤗 

Thanks for merging https://github.com/huggingface/transformers/pull/27494 @ArthurZucker ! I merged `main` into this PR and once CI is green I think we'll be good to merge :)

Still some issue with the dependencies... :confused: 
[I opened a PR](https://github.com/huggingface/huggingface_hub/pull/1828) in huggingface_hub and will make another patch release later.

I converted this PR to draft because of the dependency issues we're having. I'll push a new fix in `huggingface_hub` to  definitely settle this (see [slack thread](https://huggingface.slack.com/archives/C014N4749J9/p1700069962026839) -internal- for explanations).

**Please do not merge even if CI is green.**

@Wauplin I can see the PR was marked as ready for review again. Does this mean it's now mergeable? 

Would be good to get a final 👍 from @ydshieh to make sure all relevant files for our CI have been updated. 

Look good!

Confirmed with @Wauplin over slack that the PR is good to go - merging!
[`Styling`] stylify using ruff
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Removes our dependency on `black` in favor of rust:
- faster `make style` I'll try to get numbers but that one is pretty obvious 
- not need to use doc-builder's black styling as the doc is styled in ruff 
- faster overall CI to check the repo quality 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
use isinstance instead of type comparison
Author: kashif
Pull request: yes

# What does this PR do?

Fix for E721 errors which are annoying when doing `make style` with a newer `ruff`

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26748). All of your documentation changes will be reflected on that endpoint.

@amyeroberts thanks! so I relaxed the `ruff` version as with these changes the latest `ruff` runs without raising any issues... but yes i un-did that as per your request

fixed by #27144
New Model Architecture
Author: LegallyCoder
Pull request: no
Labels: name: New model

### Model description

I am writing a new model architecture. Its implementation is almost complete. I'm wondering how to integrate it into the hub. I forked the transformers lib and tried copying a model architecture, but it didn't work and it gave too many errors. How are things going here?

### Open source status

- [ ] The model implementation is available
- [ ] The model weights are available

### Provide useful links for the implementation

_No response_

Comments:
And i completed the architecture . But i must edit the code. And i have few questions. 
Do we write the task classes or does someone else?
What features should we pay attention to in the files?
Sorry, this will be my first PR.

Hi @LegallyCoder, thanks for raising an issue! 

This is a question best placed in our [forums](https://discuss.huggingface.co/). We try to reserve the github issues for feature requests and bug reports.

For instructions on how to add a model and use a model on the hub, please refer to our documentation: 
* https://huggingface.co/docs/transformers/model_sharing
* https://huggingface.co/docs/transformers/custom_models

If you've followed these instructions and there is an error, please raise an issue and follow the [issue template](https://github.com/huggingface/transformers/blob/main/.github/ISSUE_TEMPLATE/bug-report.yml). 

Sorry, i didnt know this. You are right. I closed this isue.
Generate: improve assisted generation tests
Author: gante
Pull request: yes

# What does this PR do?

Strengthens the test suite for assisted generation. With these modifications, previously found API problems will be properly caught in advance.

## Post mortem
### Why weren't API problems caught before?

Assisted generation has two loops: the loop to obtain the candidate tokens from the assistant model (inner loop), and the loop to generate the final tokens from the main model (outer loop). Both loops are slightly different depending on whether the main model accepts the matches or not -- there are different code paths depending on whether `n_matches > 0` or not.

The following cases were being tested and had no API issues:
1. `n_matches == 0`
2. `n_matches > 0`, but we only run 1 iteration of the outer loop

👉 We weren't explicitly testing the case where `n_matches > 0` AND we ran more than 1 outer loop iteration.

### If we weren't testing that case, why was the CI randomly red?

Each individual test had a ~97% chance of being green. The (random) assistant model was building the candidate sequence from the most likely tokens from its vocabulary (size = 99), and the main model was comparing the candidate sequence against sampling from its logits. Most of the times, `n_matches == 0`, so the test passed. However, sometimes we had `n_matches > 0`, but not to the point where it was enough to complete assisted generation in 1 outer loop.

👉 There was a low chance (per test) of hitting the failing case, resulting in inconsistent CI failures

Comments:
@amyeroberts `is_decoder` is a poorly named flag 😅 contrarily to `is_encoder_decoder`, which controls many aspects in generation, `is_decoder` only controls one thing AFAIK -- whether to enable `use_cache` ([example](https://github.com/huggingface/transformers/blob/651408a077f842e76e75bfc7d02b8ac38eeb6480/src/transformers/models/bert/modeling_bert.py#L953)) and pipe the cache around in encoders with a LM Head.

It is also not mutually exclusive with `is_encoder_decoder` (it should be IMO 👀)

All tests that require caching, such as the assisted generation ones, have to set `model.config.is_decoder = True`. Otherwise, the tests will fail in the encoder with LM Heads (see image below)
![Screenshot 2023-11-16 at 18 07 27](https://github.com/huggingface/transformers/assets/12240844/b57ab460-32b1-41c4-b7a6-94853db0a9ea)



@gante Thanks for explaining! I thought they were mutually exclusive
[WIP] Bounding box transformations
Author: rafaelpadilla
Pull request: yes

# What does this PR do?

Facilitates bounding box transformations among different formats:
`XYXY` <-> `XYWH` <-> `XCYCWH` <-> `RELATIVE_XYXY` <-> `RELATIVE_XYWH` <-> `RELATIVE_XCYCWH`


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

**Still WIP**

Comments:
_The documentation is not available anymore as the PR was closed or merged._

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_27163). All of your documentation changes will be reflected on that endpoint.
Fixing RotaryEmbedding.forward to return float16 values in float16 precision mode.
Author: kikutakou
Pull request: yes

# What does this PR do?

RotaryEmbedding.forward() returns values with float32 precision even in float16 precision mode.
This affects to the subsequent calculation and takes extra GPU memory usage.
This PR fixes that problem.


Fixes #24261


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?




Comments:
I will investigate whether or not this is the source of instabilities in Llama2! If so, will adresse it

No time to deep dive into this at the moment! If someone wants to check this feel free to do so! 😉 

@ArthurZucker

Thanks for the comment.

> The initialisation with torch.float16 as an argument of from_pretrained is not really doing it's job.

I've investigated and changed the patch to fix this issue.
Could you have a look at this patch?

`from_pretrained` changes torch default_dtype to the specified dtype, then initialize all weights.
[`GPTNeoXRotaryEmbedding.__init__()` calls `float()`](https://github.com/huggingface/transformers/blob/57943630e24651e6d954b912e7fcdb2b4c719cc4/src/transformers/models/gpt_neox/modeling_gpt_neox.py#L300C1-L301C1) which always returns float32 even when default dtype is float16.
This was the reason.


This was actually fixed by #25830 ! 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix AMD CI not showing GPU
Author: ydshieh
Pull request: yes

# What does this PR do?

For AMD CI jobs, like [this one](https://github.com/huggingface/transformers/actions/runs/6879141724/job/18711232327), in the `Environment` section, sometimes it shows `Number of GPUs available: 0`.

After investigation from the infra team (thanks to Guillaume), they told me the changes in this PR fix the issue: removing the environment variable `HIP_VISIBLE_DEVICES`.

Comments:
After merging are we able to reenable the tests from #27541 ? 

> After merging are we able to reenable the tests from #27541 ?

No, these 2 are irrelevant. #27541  is about `AMD docker image can't be built` while this one is `some issue at testing time (even if we build the image manually)`
Generate: fix flaky tests
Author: gante
Pull request: yes

# What does this PR do?

Fixes two flaky tests:
1. `tests/models/clvp/test_modeling_clvp.py::ClvpDecoderTest::test_constrained_beam_search_generate` was failing at a ~3% rate because the range for which we were sampling tokens to force in constrained beam search was based on the random input_ids and sometimes invalid. 👉 fix = hard code the valid range based on `config.vocab_size`
2. `tests/models/marian/test_modeling_marian.py::MarianStandaloneDecoderModelTest::test_sample_generate` was failing at a ~0.1% rate because of a combination of poor logits processor setup + random input_ids 👉 fix = don't set up the logits processor with a combination that may lead to no valid continuation being possible

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add visualbert in visual question answering model mapping
Author: fxmarty
Pull request: yes

Following the doc https://huggingface.co/docs/transformers/v4.25.1/en/model_doc/visual_bert#transformers.VisualBertForQuestionAnswering I reckon this one was missing.

## Who can review?
@ydshieh do I need to write any additional test for this?


Comments:
No, but we should check if the pipeline testing (in the PR CI job page) runs against this newly added `visual_bert`.


_The documentation is not available anymore as the PR was closed or merged._

VisualBERT isn't supported by the VQA pipeline, the pipeline is currently very specifically implemented to work with ViLT.