noamaanMulla-03/transformers-issues huggingface dataset:
gguf conversion add_prefix_space=None for llama3
Author: itazap
Pull request: yes

fixes #31630 

gguf conversion forces add_prefix_space=False for llama3, this is not required and forces from_slow, which fails. changing to None + test

@ArthurZucker 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_31937). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Revert "Incorrect Whisper long-form decoding timestamps "
Author: sanchit-gandhi
Pull request: yes

Reverts huggingface/transformers#32003

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32148). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix typing to be compatible with later py versions
Author: amyeroberts
Pull request: yes

c.f. https://github.com/huggingface/transformers/pull/30650#discussion_r1687824363

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32155). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix mask creations of `GPTNeoX` and `GPT2`
Author: vasqu
Pull request: yes

# What does this PR do?

Fixes #31943

SDPA implementations done by me have introduced a careless mistake in the mask creation, e.g. SDPA has to create a mask even when the attention mask is `None`, for example when cached input is passed as demonstrated in the issue mentioned above.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@ArthurZucker @amyeroberts

Comments:
The timeouts are back :cry: 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_31944). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@ArthurZucker I've added the tests to GPTNeoX and GPT2. Not sure if the reference to the issue is a good idea or not, just went with it.
static cache implementation is not compatible with attn_implementation==flash_attention_2
Author: faaany
Pull request: no
Labels: name: Feature request

name: bug

name: Cache

### System Info

- `transformers` version: 4.43.0.dev0
- Platform: Linux-4.18.0-425.3.1.el8.x86_64-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 0.23.5
- Safetensors version: 0.4.3
- Accelerate version: 0.33.0.dev0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.3.0+cu121 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA A100 80GB PCIe

### Who can help?

@ArthurZucker 

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```bash
pytest -rA tests/test_cache_utils.py::CacheIntegrationTest -k "test_static_cache_greedy_decoding_pad_left and flash_attention"
```
fails with 
```bash
    def forward(
        self,
        hidden_states: torch.Tensor,
        attention_mask: Optional[torch.LongTensor] = None,
        position_ids: Optional[torch.LongTensor] = None,
        past_key_value: Optional[Cache] = None,
        output_attentions: bool = False,
        use_cache: bool = False,
        cache_position: Optional[torch.LongTensor] = None,
    ) -> Tuple[torch.Tensor, Optional[torch.Tensor], Optional[Tuple[torch.Tensor]]]:
        if isinstance(past_key_value, StaticCache):
>           raise ValueError(
                "`static` cache implementation is not compatible with `attn_implementation==flash_attention_2` "
                "make sure to use `sdpa` in the mean time, and open an issue at https://github.com/huggingface/transformers"
            )
E           ValueError: `static` cache implementation is not compatible with `attn_implementation==flash_attention_2` make sure to use `sdpa` in the mean time, and open an issue at https://github.com/huggingface/transformers

src/transformers/models/llama/modeling_llama.py:388: ValueError
```

And the right padding test case also fails:
```bash
pytest -rA tests/test_cache_utils.py::CacheIntegrationTest -k "test_static_cache_greedy_decoding_pad_right and flash_attention"
```



### Expected behavior

Either we don't test `flash_attention` in this case, or we should add a if check to skip setting `cache_implementation` to `static`. 

Comments:
I made a possible fix suggestion in this PR draft: https://github.com/huggingface/transformers/pull/32039. But I am not sure whether this is correct. So I also filed this issue.  

cc @gante too 

Incompatibility also affecting Gemma2 with flash-attn, as it doesn't support dynamic cache
cannot import name 'Module' from '_pytest.doctest'
Author: jingyanwangms
Pull request: no

### System Info

transformers                 4.32.0.dev0
torch                        2.1.0.dev20230523+cu117

Error:
Traceback (most recent call last):
  File "/workspace/transformers/examples/pytorch/language-modeling/run_clm.py", line 52, in <module>
Traceback (most recent call last):
  File "/workspace/transformers/examples/pytorch/language-modeling/run_clm.py", line 52, in <module>
    from transformers.testing_utils import CaptureLogger
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/transformers-4.32.0.dev0-py3.8.egg/transformers/testing_utils.py", line 111, in <module>
    from transformers.testing_utils import CaptureLogger
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/transformers-4.32.0.dev0-py3.8.egg/transformers/testing_utils.py", line 111, in <module>
    from _pytest.doctest import (
ImportError: cannot import name 'Module' from '_pytest.doctest' (/opt/conda/envs/ptca/lib/python3.8/site-packages/_pytest/doctest.py)
    from _pytest.doctest import (

### Who can help?

@sgugger 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

python -m torch.distributed.launch --nproc_per_node=8 --use-env /workspace/transformers/examples/pytorch/language-modeling/run_clm.py --model_name_or_path xlnet-base-cased --dataset_name wikitext --dataset_config_name wikitext-2-raw-v1 --label_smoothing 0.1 --do_train --output_dir /dev/shm --overwrite_output_dir --max_steps 200 --logging_steps 20 --per_device_train_batch_size 8 --fp16

### Expected behavior

example runs without error

Comments:
You might need a `pip install --upgrade pytest`.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

I still have this problem and it hasn't been resolved
fix: Fixed an if condition that is always evaluating to true
Author: Sai-Suraj-27
Pull request: yes

# What does this PR do?
This `if condition` will always evaluate to true.
https://github.com/huggingface/transformers/blob/9cf4f2aa9a9cecbb22e813931ef3bb72fc773540/src/transformers/models/dpt/convert_dpt_hybrid_to_pytorch.py#L46

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@amyeroberts 

Comments:
[docs] change temperature to a positive value
Author: faaany
Pull request: yes

## What does this PR do?
When running the tutorial example code, I got the following error:
```bash
Traceback (most recent call last):
  File "/workspace/DOC/tutorials.py", line 25, in <module>
    outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.)
  File "/root/miniforge3/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/workspace/transformers/src/transformers/generation/utils.py", line 1955, in generate
    self._get_logits_warper(generation_config, device=input_ids.device)
  File "/workspace/transformers/src/transformers/generation/utils.py", line 765, in _get_logits_warper
    warpers.append(TemperatureLogitsWarper(generation_config.temperature))
  File "/workspace/transformers/src/transformers/generation/logits_process.py", line 292, in __init__
    raise ValueError(except_msg)
ValueError: `temperature` (=0.0) has to be a strictly positive float, otherwise your next token scores will be invalid. If you're looking for greedy decoding strategies, set `do_sample=False`.
```
Let's change it to a positive value. 

Comments:
@faaany If you mark ready for review (not longer a draft) we can merge 

Hi @amyeroberts , done! 

Thanks!
Fix/phi3 rope impl
Author: ryan-minato
Pull request: yes

# What does this PR do?

Improving the Implementation of Phi3SuScaledRotaryEmbedding to Reduce Unnecessary Computation

Fixes #31339 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@ArthurZucker @younesbelkada 



Comments:
cc @ArthurZucker 

I have modified the implementation to maintain compatibility with previous implementations. 

Because I discovered that the past experimental code directly modified the long_factor and short_factor parameters.

It seems that the CI failure is unrelated to my modifications.

```
tests/test_tokenization_common.py:4123: in test_clean_up_tokenization_spaces
    tokenizer = BertTokenizer.from_pretrained("google-bert/bert-base-uncased")
/usr/local/lib/python3.10/site-packages/transformers/tokenization_utils_base.py:2110: in from_pretrained
    return cls._from_pretrained(
/usr/local/lib/python3.10/site-packages/transformers/tokenization_utils_base.py:2324: in _from_pretrained
    with open(tokenizer_file, encoding="utf-8") as tokenizer_file_handle:
E   FileNotFoundError: [Errno 2] No such file or directory: '/root/.cache/huggingface/hub/models--google-bert--bert-base-uncased/snapshots/86b5e0934494bd15c9632b12f734a8a67f723594/tokenizer.json'
```

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
SafetensorError: Error while deserializing header: HeaderTooLarge
Author: KyrieCui
Pull request: no

### System Info

transformers version: 4.35.0
Platform: Linux-4.18.0-477.27.1.el8_8.x86_64.x86_64-x86_64-with-glibc2.28
Python version: 3.9.16
Huggingface_hub version:0.16.4
Accelerate version: 0.21.0
Safetensors version: 0.3.1
PyTorch version (GPU?): 22.0.1+cu117 (True)


### Who can help?

 @ArthurZucker  @SunMarc   @younesbelkada

### Information

- [X] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction
```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
base_model = '/llm/llama2-2-70b-chat-hf'
model = AutoModelForCausalLM( base_model, load_in_8bit=True, device_map={"",0},use_safetensors=True)


in load_state_dict (checkpoint_file)
      462 """
      463 Reads a Pytorch checkpoint file, returning properly formatted errors if they arise.   
      464 """
      465 if checkpoint_file.endswith(".safetensors") and is_safetensors_available():
-->466     with safe_open(checkpoint_file,framework="pt") as f:
      467     metadata=f.metadate()
SafetensorError: Error while deserializing header: HeaderTooLarge    
```
### Expected behavior

Expected tao load the model successful

Comments:
Hey, could you try to use the latest release of safetensors? 
Otherwise not sure I can help you with a checkpoint that is private ! 

Hello, Sir. 
I met this error when loading the 6th checkpoint, so I re-downloaded the sixth safetensors file. But still the same error. Should I try to re- download all of the safetensors files?

If you download them from the hub anyway, you should just use from_pretrained, they will automatically be cache in your .cache and re-used accordingly. 

@KyrieCui this might be a storage issue as well, can you make sure you have enough Disk space on your device?

thanks for all of your support! I fixed this issue by re-downloading all of the safetensors files. 

i am having the same but i have manually merge the weights going  layer to layer then merged them cause i am applying the Fedml approach can anyone help me through this ?

I got this error and the problem turned out to be that I didn't actually have the model file: I just had the `git lfs` smudge file, a pointer to the actual 1.4 GB file.  The fix was to go to where I'd checked out the model, do `git lfs install`, then `git lfs fetch`, then find the blob under `.git` that was the model. (I don't have experience with `git lfs`; there's probably a simpler way to get the real file.)

So double-check that the file under `models/` is actually a model.

A better error message that shows information about the file it failed to open would be helpful; anything that's under 200 bytes is incredibly unlikely to be a model.

@gpacix Great tip! Just to add on, if you install git lfs first, you can `git clone` the repo directly which should now include the model in its entirety rather than the pointers.
phi-3's LlamaTokenizer ignores newline character.
Author: kaitolucifer
Pull request: no
Labels: name: Core: Tokenization

name: bug

### System Info

python==3.10.14
transformers==4.42.3

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```
from transformers import AutoTokenizer


tokenizer = AutoTokenizer.from_pretrained(
    "microsoft/Phi-3-mini-4k-instruct",
    use_fast=False,
    padding_side="right",
    model_max_length=language_model.config.max_position_embeddings,
    add_bos_token=True,
)
tokenizer.add_tokens(["
"])
input_text = "
\nYou are a helpful language and vision assistant. " +\
             "You are able to understand the visual content that the user provides, " +\
             "and assist the user with a variety of tasks using natural language.
\n" +\
             "
\n" +\
             "
\n" +\
             "What is this?
\n" +\
             "
\n"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids
print(tokenizer.decode(input_ids[0], skip_special_tokens=True))
```

output:

```

 You are a helpful language and vision assistant. You are able to understand the visual content that the user provides, and assist the user with a variety of tasks using natural language. 
 
 
 
What is this? 
 

```

It seems like tokenizer ignores newline character .

### Expected behavior

output should be identical to `input_text`
```


You are a helpful language and vision assistant. You are able to understand the visual content that the user provides, and assist the user with a variety of tasks using natural language.





What is this?




```

Comments:
cc @ArthurZucker @itazap 

Hello! For this model, the added tokens (`
, 
`, etc) have a `rstrip=True` parameter that strips whitespace to the right of the token, which is set this way in Phi-3. You can overwrite this by re-adding the tokens with `rstrip=False`

```python
from transformers import AutoTokenizer, AddedToken

tokenizer = AutoTokenizer.from_pretrained(
    "microsoft/Phi-3-mini-4k-instruct",
    use_fast=False,
    padding_side="right",
    add_bos_token=True,
)

# For all the tokens you need to update, you can check in tokenizer.get_added_vocab()
for token_str in ["
", "
", "
", "
", "
"]:
    token = AddedToken(token_str, rstrip=False)
    tokenizer.add_tokens(token)
```
Replace Decord with PyAV
Author: ji-huazhong
Pull request: yes

# What does this PR do?

follow up https://github.com/huggingface/transformers/pull/21572

same as https://github.com/huggingface/transformers/pull/29747, closed this PR


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @pacman100

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc and @younesbelkada

Documentation: @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->

@amyeroberts 


Comments: