noamaanMulla-03/transformers-issues huggingface dataset:
Add test when downloading from gated repo
Author: Wauplin
Pull request: yes

Following PR after https://github.com/huggingface/transformers/pull/25034.
Goal is to add a test to check that accessing a gated repo raises a custom message for the user. Cannot be merged yet as the server returns "RepoNotFound" when user is not authenticated, even when it's a public gated repo. Once server implementation is fixed ([see internal PR](https://github.com/huggingface/moon-landing/pull/7106)), this PR should pass.

Until then, no need to review/try to fix it.

**Note:** a workaround would be to have a token for a user in the CI but that would require a secret in GH which would not work in PRs. Let's keep it simple and have a test only for unauthenticated users.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sgugger The fix is merged and deployed server-side so the test can now be added in `transformers` :)
override .cuda() to check if model is already quantized
Author: ranchlai
Pull request: yes

# What does this PR do?
This PR is a quick fix by adding .cuda() to prevent device casting after 8-bit quantization. Same spirit as #20409. @younesbelkada  @sgugger Would you please have a quick look .

It fixes the following unexpected error: 

### For reberta-large, output `nan` without raising an error
```python
from transformers import AutoTokenizer, AutoModelForMaskedLM
from transformers import AutoConfig
from transformers import pipeline


model_name = "roberta-large"  # or any other models

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
config = AutoConfig.from_pretrained(
    model_name,
)

model = AutoModelForMaskedLM.from_pretrained(
    model_name, trust_remote_code=True, load_in_8bit=True, device_map="auto"
)
model.cuda()
unmasker = pipeline("fill-mask", model=model, tokenizer=tokenizer)
print(unmasker("Hello I'm a <mask> model."))

>>> [{'score': nan, 'token': 3, 'token_str': '<unk>', 'sequence': "Hello I'm a model."}, {'score': nan, 'token': 4, 'token_str': '.', 'sequence': "Hello I'm a. model."}, {'score': nan, 'token': 1, 'token_str': '<pad>', 'sequence': "Hello I'm a model."}, {'score': nan, 'token': 0, 'token_str': '
', 'sequence': "Hello I'm a model."}, {'score': nan, 'token': 2, 'token_str': '
', 'sequence': "Hello I'm a model."}] 
```
### for mpt, RuntimeError as follows
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from transformers import AutoConfig


model_name = "mosaicml/mpt-7b"

tokenizer = AutoTokenizer.from_pretrained(model_name)
config = AutoConfig.from_pretrained(model_name)

model = AutoModelForCausalLM.from_pretrained(
    model_name, load_in_4bit=True, device_map="auto"
)
model.cuda()
text = "Here is a recipe for vegan banana bread:\n"
input_ids = tokenizer.encode(text, return_tensors="pt").to("cuda:0")
output = model.generate(input_ids, max_length=100, do_sample=True)
response = tokenizer.decode(output[0])
print(response)
>>> output = torch.nn.functional.linear(A, F.dequantize_4bit(B, state).to(A.dtype).t(), bias)
>>> RuntimeError: mat1 and mat2 shapes cannot be multiplied (10x4096 and 1x25165824)
```


Comments:
_The documentation is not available anymore as the PR was closed or merged._
added compiled model support for inference
Author: markovalexander
Pull request: yes

# What does this PR do?

Adds support for torch.compile-ed models in pipelines.

Basically, you cannot run torch.compile-ed model under `torch.inference_mode()` context and should use `torch.no_grad` instead

Examples:

<img width="1004" alt="image" src="https://github.com/huggingface/transformers/assets/22663468/04f0bde9-c7d0-4776-9fc4-bd224fadebce">

Here you see start of the super long traceback that ends with:
```
RuntimeError: Inference tensors do not track version counter.

While executing %getitem : [#users=1] = call_function[target=operator.getitem](args = (%attention_mask, 
(slice(None, None, None), None, None, slice(None, None, None))), kwargs = {})
Original traceback:
  File "/home/alexander/jupyter_server/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py", line 890,
in get_extended_attention_mask
    extended_attention_mask = attention_mask[:, None, None, :]
 |   File 
"/home/alexander/jupyter_server/.venv/lib/python3.8/site-packages/transformers/models/bert/modeling_bert.py", line 
993, in forward
    extended_attention_mask: torch.Tensor = self.get_extended_attention_mask(attention_mask, input_shape)
 |   File 
"/home/alexander/jupyter_server/.venv/lib/python3.8/site-packages/transformers/models/bert/modeling_bert.py", line 
1758, in forward
    outputs = self.bert(
```

and 

```
BackendCompilerFailed: debug_wrapper raised RuntimeError: Inference tensors do not track version counter.

While executing %getitem : [#users=1] = call_function[target=operator.getitem](args = (%attention_mask, 
(slice(None, None, None), None, None, slice(None, None, None))), kwargs = {})
Original traceback:
  File "/home/alexander/jupyter_server/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py", line 890,
in get_extended_attention_mask
    extended_attention_mask = attention_mask[:, None, None, :]
 |   File 
"/home/alexander/jupyter_server/.venv/lib/python3.8/site-packages/transformers/models/bert/modeling_bert.py", line 
993, in forward
    extended_attention_mask: torch.Tensor = self.get_extended_attention_mask(attention_mask, input_shape)
 |   File 
"/home/alexander/jupyter_server/.venv/lib/python3.8/site-packages/transformers/models/bert/modeling_bert.py", line 
1758, in forward
    outputs = self.bert(

```

Possible solution right now:

<img width="692" alt="image" src="https://github.com/huggingface/transformers/assets/22663468/f6b2c599-78b9-4e0c-89b7-31ded7532a52">

## Who can review?
@Narsil


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Update `use_auth_token` -> `token` in example scripts
Author: ydshieh
Pull request: yes

# What does this PR do?

Update example scripts to use `token`.

We have `datasets!=2.5.0` in transformers, and I see `datasets=2.7.0` still only uses `use_auth_token`, so I don't touch the usage in `load_dataset`. Let me know if we should change to `token` + pin a higher minimum `datasets` version.

The files under `examples/research_projects` are not touched.

Comments:
`use_auth_token` has been deprecated in favor of `token` in the latest release of `datasets` 🙂. 

> `use_auth_token` has been deprecated in favor of `token` in the latest release of `datasets` 🙂.

Yes, I got to check it and update my comment :-)

But thanks a lot for the information. It's very nice.

_The documentation is not available anymore as the PR was closed or merged._
[`Mpt`] Fix mpt slow test
Author: younesbelkada
Pull request: yes

# What does this PR do?

As per title, failure report here: https://github.com/huggingface/transformers/actions/runs/5675853423/job/15381788713

Probably an issue with libs I had when designing the tests (had torch +cu117 instead of cu118) - ran the tests on the latest docker image and they all pass

cc @sgugger @ydshieh   


Comments:
_The documentation is not available anymore as the PR was closed or merged._
[`InstructBlip`] Fix instructblip slow test
Author: younesbelkada
Pull request: yes

# What does this PR do?

Fixes the current failing daily CI test: https://github.com/huggingface/transformers/actions/runs/5675853423/job/15381807774 let's make the daily CI happy!

Ran the test on the latest docker image and the test now pass with these values.

cc @sgugger @ydshieh 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_25171). All of your documentation changes will be reflected on that endpoint.
🌐 [i18n-KO] Translated `transformers_agents.md` to Korean
Author: sim-so
Pull request: yes

# What does this PR do?

Translated the `transformers_agents.md` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

May you please review this PR? 
@sronger, @TaeYupNoh, @kj021, @HanNayeoniee, @eenzeenee, @sim-so

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

@sgugger, @ArthurZucker, @eunseojo 
May you please review this PR?

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@eenzeenee @sronger님 리뷰 감사합니다! 
제안해주신 수정사항을 모두 반영했습니다 ☺️

Could you review this PR? 😃
@sgugger, @ArthurZucker, @eunseojo
[MusicGen] Fix integration tests
Author: sanchit-gandhi
Pull request: yes

# What does this PR do?

Fixes the integration tests for MusicGen:
1. Places all input tensors on the correct device
2. Updates expected values with those obtained on cuda
3. Fixes for fp16 generation

cc @ydshieh 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Future compatibility with LangChain
Author: LucasMartinCalderon
Pull request: no

Is there  a specific timeline for future LLM agents compatibility with LangChain?
What other current compatibility solutions with LangChain are there? 


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
RuntimeError: Expected to mark a variable ready only once. 
Author: lucasjinreal
Pull request: no

```
RuntimeError: Expected to mark a variable ready only once. This error is caused by one of the following reasons: 1) Use of a module parameter outside the `forward` function. Please make sure model parameters are not shared across 
multiple concurrent forward-backward passes. or try to use _set_static_graph() as a workaround if this module graph does not change during training loop.2) Reused parameters in multiple reentrant backward passes. For example, if 
you use multiple `checkpoint` functions to wrap the same part of your model, it would result in the same set of parameters been used by different reentrant backward passes multiple times, and hence marking a variable ready 
multiple times. DDP does not support such use cases in default. You can try to use _set_static_graph() as a workaround if your module graph does not change over iterations.
Parameter at index 129 with name base_model.model.model.layers.31.self_attn.v_proj.lora_B.default.weight has been marked as ready twice. This means that multiple autograd engine  hooks have fired for this particular parameter 
```

After upgrade transformers, train Lora models got above error.



Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
RWKV split CPU & GPU results in high perplexity
Author: 3outeille
Pull request: no

### System Info

Using https://github.com/huggingface/transformers/pull/22797#event-9203076880 PR, I tried to evaluate perplexity on wikitext2 using HuggingFace RWKV but found a weird behavior (gist to reproduce the bug: https://gist.github.com/3outeille/e74ec833ec2800a94325f8dad8e0da3d).

- When model is fully loaded on CPU or GPU, perlexity is fine
- When some block of RWKV are loaded in CPU and GPU, perplexity is high

Any idea ?

### Who can help?

@sgugger, @younesbelkada

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

https://gist.github.com/3outeille/e74ec833ec2800a94325f8dad8e0da3d

### Expected behavior

- Full CPU ✔️ : 
  - `nlls: tensor([2.0129, 2.3220, 2.3500])`
  - `Perplexity: 9.284077644348145`
- Full GPU ✔️ :
  - `nlls: tensor([2.0137, 2.3223, 2.3496], device='cuda:0', dtype=torch.float16)`
  - `Perplexity: 9.2890625`
 - Split 🔴 :
    - `nlls: tensor([15.6641, 15.9141, 16.5469], device='cuda:0', dtype=torch.float16)`
    - `Perplexity: 9312564.0`

Comments:
@younesbelkada  Any update ?

Hi @3outeille 
Sadly I didn't had time to check that out, are you still facing the issue with the latest main branch of transformers & accelerate?

Hi @younesbelkada, I update transformers & accelerate to the latest release version as shown here: https://github.com/3outeille/hf_rwkv_bug/blob/master/requirements.txt and the bug is still here

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Review added
Author: repo-reviews
Pull request: no

### Thank you for building **transformers**!

@AuroraCoboAguilera created a review titled:

*Hugging face, the accelerator to develop your own NLP*

on [repo-reviews.github.io](https://repo-reviews.github.io) to share their experience using **transformers**.

[link to review](https://repo-reviews.github.io//reviews/2023-06-21_AuroraCoboAguilera_huggingface_transformers)

If you would like to help your super-users share their experiences using your repo, add a [badge](https://github.com/repo-reviews/repo-reviews.github.io#add-badges) to your README.md.

We hope that sharing these experiences helps your users **increase their productivity**.

--
Please be kind,
I’m a human!


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[WIP] Add SPTSv2
Author: JesseSilverberg
Pull request: yes

# What does this PR do?

This PR adds SPTSv2. Per the docs, I am opening this PR immediately after generating the boilerplate.

Fixes #24235



Comments:
cc @alaradirik for information. Please let us know when your model is ready for review or if you need any help :-)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
How to unwrap after auto_wrap in FSDP?
Author: ZN1010
Pull request: no

I am currently fine-tuning a LLM (LLaMA) and would like to retrieve the gradients of each weight (parameter) after every gradient update. However, I notice that weights are (auto) wrapped into stuff like “_fsdp_wrapped_module._flat_param” during training. I need to map these wrapped weights to the original LLaMA architecture such as “self_attn.v_proj”. Any code examples?

I guess “summon_full_params()” might be the function that I look for, but I am not sure if that is correct. I also have difficulty using this function. Thanks a lot for any help!

Comments:
This is more a question for the PyTorch forums as it's purely related to FSDP. Still cc-ing @pacman100 in case he has any idea.

Hello, as Sylvain mentioned this question is for the PyTorch forums. `summon_full_params` usage can be found in these tests: https://github.com/pytorch/pytorch/blob/main/test/distributed/checkpoint/test_fsdp_optim_state.py#L56-L59

I am not sure if it contains the information related to the gradients of a given parameter. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
TF LLaMA Port
Author: Rocketknight1
Pull request: yes

This is an autoconversion of the LLaMA code to TF by GPT-4. As always, expect things to be broken until I finish debugging it!

TODO list:
- [ ] Get tests to pass
- [ ] No `MainLayer` -  we shouldn't need it! Make sure weight naming can still be controlled.
- [ ] Explore full `float16` weights
- [ ] Explore passing `DTensor` layouts

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24375). All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Musicgen: CFG is manually added 
Author: gante
Pull request: yes

# What does this PR do?

This PR exists to keep `musicgen`'s current functionalities considering the changes in #24654.

In a nutshell, the #24654 has a more flexible version of CFG (allows negative prompting, is compatible with existing generation methods, doesn't need to expand the batch size by 2 before the fwd pass, lower memory requirements), but would mean an execution time regression on `musicgen` (because it needs 2x forward passes).

Comments:
@sanchit-gandhi there is a potential change due to order of operations, depending on the processors commonly used. Would you be able to confirm whether it would be okay like this?

_The documentation is not available anymore as the PR was closed or merged._
Can BlipForImageTextRetrieval be used to generate captions?
Author: Vibhu04
Pull request: no

### System Info

`transformers` version: 4.31.0.dev0
Platform: Linux-5.15.0-76-generic-x86_64-with-debian-bullseye-sid
Python version: 3.7.15
Huggingface_hub version: 0.15.1
Safetensors version: 0.3.1
PyTorch version (GPU?): 1.13.1+cu117 (True)
Tensorflow version (GPU?): 2.11.0 (False)
Flax version (CPU?/GPU?/TPU?): not installed (NA)
Jax version: not installed
JaxLib version: not installed
Using GPU in script?: no
Using distributed or parallel set-up in script?: no

### Who can help?

@younesbelkada @ArthurZucker @amyeroberts

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

I am seeking a Blip model that can serve two purposes: predicting the similarity between an input image and text and generating a caption for an input image. I am aware that `BlipForImageTextRetrieval` is suitable for predicting the similarity between an image and text, while `BlipForConditionalGeneration` can generate captions for images. However, I was wondering whether either of these models can be employed to perform the alternate task as well.

A bit more context: I have a fine-tuned `BlipForImageTextRetrieval` model that I would like to use for generating captions.

### Expected behavior

Any guidance on obtaining a Blip model that can do both the tasks mentioned above would be extremely helpful. Thanks.

Comments:
Hey! Since you are not reporting a bug, could you open a discussion on [the forum](https://discuss.huggingface.co/) for this kind of questions?

Sure, apologies. 

No worries! 🤗 Feel free to ping @younesbelkada there
Fix `main_input_name` in `src/transformers/keras_callbacks.py`
Author: ydshieh
Pull request: yes

# What does this PR do?

Fix #24872

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@Rocketknight1 Just for you to have a final comment when you are back 🙏 

Also LGTM, thanks for catching the bug!
Better error message in `_prepare_output_docstrings`
Author: ydshieh
Pull request: yes

# What does this PR do?

Currently, if an output type has no docstring, or its docstring doesn't have `Args` or `Parameters`, we get an error 
```bash
  File "/transformers/src/transformers/utils/doc.py", line 137, in _prepare_output_docstrings
    full_output_type = f"{output_type.__module__}.{output_type.__name__}"
UnboundLocalError: local variable 'params_docstring' referenced before assignment
```
when `_prepare_output_docstrings` is called (for example, running a script that uses the relevant model).

This is not super informative what's wrong and what to fix.

This PR adds an error message to explain what's going on.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Incorrect information on BlipImageProcessor documentation
Author: LukeBailey181
Pull request: no

**Is your feature request related to a problem? Please describe.**
There is slightly incorrect information on the [BlipImageProcessor documentation](https://huggingface.co/docs/transformers/main/en/model_doc/blip#transformers.BlipImageProcessor). 

**Describe the solution you'd like**
The documentation currently says that the `image_mean` and `image_std` arguments default to `IMAGENET_STANDARD_MEAN` and `IMAGENET_STANDARD_STD` respectively. This seems to be incorrect looking at the source code however, in the init of `BlipImageProcessor` found [here](https://github.com/huggingface/transformers/blob/main/src/transformers/models/blip/image_processing_blip.py#L43) the values default to `OPENAI_CLIP_MEAN` and `OPENAI_CLIP_STD` respectively as defined [here](https://github.com/huggingface/transformers/blob/main/src/transformers/utils/constants.py). Means are pretty similar but the standard deviations are quite different, so may be worth updating the documentation here :)  


Comments:
Transferring to transformers

Hi @LukeBailey181, thanks for pointing out! 

Would you like to open a PR to update the documentation? This way you would get the git contribution. 

Wonderful yes I will create the PR!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.