noamaanMulla-03/transformers-issues huggingface dataset:
Generate: torch.compile-ready generation config preparation
Author: gante
Pull request: yes

# What does this PR do?

Pulled from https://github.com/huggingface/transformers/pull/29374

This PR changes the code block that prepared the generation config instance so as to be `torch.compile(..., fullgraph=True)` ready. 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29443). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Add Microsoft's Code Reviewer model as a dedicated model
Author: fasterinnerlooper
Pull request: no
Labels: name: New model

### Model description

The model has three goals:
1. Provide feedback on the quality of the code change.
2. Provide feedback on the code change itself
3. When give a code change and a review comment, perform the required action to rectify the issue identified in the comment

The biggest reason for adding this to the HuggingFace library as a dedicated model is so the model size can be expanded and retrained, and potentially provide more accurate and meaningful output.

The model has some files in HuggingFace at the moment, but the implementation is incomplete. As seen in the GitHub repo, the T5 model is augmented and the state_dict is loaded on top of this modified model. Ideally, everything should live in HuggingFace.

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

Current weights and model implementation: https://huggingface.co/microsoft/codereviewer
Paper: https://arxiv.org/abs/2203.09095
GitHub repo: https://github.com/microsoft/CodeBERT/tree/master/CodeReviewer

Comments:
Fix `TextGenerationPipeline.__call__` docstring
Author: alvarobartt
Pull request: yes

# What does this PR do?

This PR fixes some issues with the formatting of the `TextGenerationPipeline.__call__` docstring, see the image below.

<img width="821" alt="image" src="https://github.com/huggingface/transformers/assets/36760800/e6166eb6-db61-4543-a808-b7cef16b4490">

## Before submitting

- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Documentation: @stevhliu and @MKhalusova

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29491). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix TrainingArguments regression with torch <2.0.0 for dataloader_prefetch_factor
Author: ringohoffman
Pull request: yes

# What does this PR do?

`dataloader_prefetch_factor` was added to `TrainingArguments` in #28498 with the default value `None`, but  versions of `torch`<2.0.0 do not accept `None` and will raise an error if `num_workers == 0 and prefetch_factor != 2`

Fixes https://discuss.huggingface.co/t/prefetch-factor-issue/74367/4

~~Also remove [a superfluous check](https://github.com/huggingface/transformers/blame/e9476832942a19cf99354776ef112babc83c139a/src/transformers/training_args.py#L1810-L1814) that duplicates [the <2.0.0 check](https://github.com/pytorch/pytorch/blob/49444c3e546bf240bed24a101e747422d1f8a0ee/torch/utils/data/dataloader.py#L240-L242) and [the >=2.0.0 check](https://github.com/pytorch/pytorch/blob/c263bd43e8e8502d4726643bc6fd046f0130ac0e/torch/utils/data/dataloader.py#L244-L246).~~

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

@muellerz @amyeroberts 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29447). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

🤔 these failures seem unrelated?
[`Core generation`] Let's be less restrictive on the arguments passed to the generation calls. 
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Updates generate calls 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29123). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
combining feature from two pre-train model in transformer then passing them into a classifer 
Author: Arwa491
Pull request: no

I'm trying to extract the feature using two different per-train models combine these features in one vector and then pass this vector into a classifier for a final classification 
is that possible using hugging face pre-train models , because i already trained  the model on my data and uploaded them into hugging face 

Comments:
Hi @Arwa491, thanks for raising an issue! 

This is a question best placed in our [forums](https://discuss.huggingface.co/). We try to reserve the github issues for feature requests and bug reports.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Support FlashAttention2 on torch2.2 
Author: luohao123
Pull request: no

Flasshattn2 didn't support V100, and it just throws error on V100 gpus.

Since torch2.2 has built in flashatten2 support ,consider suppor t it?

Comments:
You should already be able to use this with `attn_implementation="sdpa"`

@ArthurZucker waoo, so high efficiency, is 3.37 support now? BTW, what will it fallback when enable it but on V100?

I don't know the exact details but it should still used fused operations ! 
transformers 4.35 already supported this I believe (not for all models of course) through sdpa

@ArthurZucker thanks. Does llama and llava support?

Yes

@luohao123 for using FA2 through the SDPA interface of pytorch you should wrap your model's forward or generate with the following context manager:
```diff
+ with torch.backends.cuda.sdp_kernel(enable_flash=True, enable_math=False, enable_mem_efficient=False):
    model.generate(xxx)
```

@younesbelkada  Does it has simpler config configs in transformers?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Closing as it is supported!
Load transformer models with shared memory 
Author: ahmedkotb98
Pull request: no
Labels: name: Feature request

### System Info

.

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

.

### Expected behavior

I want to load transformer models with share memory for example when open multi workers I want to load the model once and all workers use it and not load the model for every worker

Comments:
ReduceLROnPlateau version of get_constant_schedule_with_warmup
Author: AngledLuffa
Pull request: no
Labels: name: Feature request

### Feature request

There is a very useful scheduler for people too lazy to code up the sequential scheduler pieces from pytorch:

https://huggingface.co/docs/transformers/en/main_classes/optimizer_schedules#transformers.get_constant_schedule_with_warmup

It would be convenient to have a version of this where the long flat section is actually a ReduceLROnPlateau:

https://pytorch.org/docs/stable/generated/torch.optim.lr_scheduler.ReduceLROnPlateau.html



### Motivation

I'm really lazy and would love to have this piece of work done for me, plus there would probably be other people who find it useful

### Your contribution

Could do, if it's a feature other people would like to see but no one is eager to pick up the gauntlet

Comments:
FYI @muellerzr
HELP: ZoeDetector.from_pretrained() got an unexpected keyword argument 'cache_dir'
Author: rdtdy
Pull request: no

### System Info

Intel(R) Core(TM) i7-14700KF   3.40 GHz
Windows 11 

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Error occurred when executing AIO_Preprocessor:

ZoeDetector.from_pretrained() got an unexpected keyword argument 'cache_dir'

File "D:\Comfyui\ComfyUI\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Comfyui\ComfyUI\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Comfyui\ComfyUI\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Comfyui\ComfyUI\custom_nodes\comfyui_controlnet_aux-main\__init__.py", line 109, in execute
return getattr(aux_class(), aux_class.FUNCTION)(**params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Comfyui\ComfyUI\custom_nodes\comfyui_controlnet_aux-main\node_wrappers\zoe.py", line 17, in execute
model = ZoeDetector.from_pretrained(HF_MODEL_NAME, cache_dir=annotator_ckpts_path).to(model_management.get_torch_device())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

### Expected behavior

Solve this problem

Comments:
Sorry this issue should be opened on the ` ComfyUI` repo!
Exllama v2 Quantization support
Author: ghost
Pull request: no
Labels: name: Feature request

### Feature request

Support for exl2 quantized models based on https://github.com/turboderp/exllamav2.

### Motivation

The recent absence of GPTQ and AWQ quants by [TheBloke](https://huggingface.co/TheBloke), as well as more recent activity by [LoneStriker](https://huggingface.co/LoneStriker) and [Bartowski](https://huggingface.co/bartowski) in uploading exl2 (and GGUF) quants for new LLMs.

### Your contribution

This would be far outside my circle of competence.

Comments:
It is already supported see #28634
Make model compatible with `torch.func`
Author: pietrolesci
Pull request: no
Labels: name: Feature request

Hi there,

I am trying to use DebertaV3 as an alternative to BERT for an experiment that requires `torch.func` transformations. However, since Deberta implementation relies on a [`torch.autograd.Function`](https://pytorch.org/docs/master/autograd.html#torch.autograd.Function), `torch.func` is not able to transform it. I get the following error

```bash
RuntimeError: In order to use an autograd.Function with functorch transforms 
  (vmap, grad, jvp, jacrev, ...), it must override the setup_context staticmethod. 
  For more details, please see https://pytorch.org/docs/master/notes/extending.func.html
```

The responsible lines are the classes implementing a `torch.autograd.Function`, for example

https://github.com/huggingface/transformers/blob/a69cbf4e64c7bc054d814d64f6877180f7cd3a25/src/transformers/models/deberta_v2/modeling_deberta_v2.py#L79

which according to the PyTorch documentation ([https://pytorch.org/docs/master/notes/extending.func.html](https://pytorch.org/docs/master/notes/extending.func.html)) must implement a

```python
@staticmethod
    def setup_context(ctx, inputs, output):
        ...
```

method. 

Would it be possible to update this implementation to make it compatible with function transformations?

Comments:
Related issues: #27734
Google Gemma don't know what 1+1 is equal to？
Author: zhaoyun0071
Pull request: no

### System Info

[v4.38.1](https://github.com/huggingface/transformers/releases/tag/v4.38.1)

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction
```

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("./gemma_2B")
model = AutoModelForCausalLM.from_pretrained("./gemma_2B", device_map="auto", torch_dtype=torch.float32)

input_text = "1+1=？"
input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")

outputs = model.generate(**input_ids,max_length=50)
# print(outputs)
print(tokenizer.decode(outputs[0]))


```

### Expected behavior

output is bellow

```
<bos>1+1=？

1+1=？

1+1=？

1+1=？

1+1=？

1+1=？

1+1=？

1+1=？

1
```

Comments:
it's a base model, try talking to the [instruct model](https://huggingface.co/google/gemma-2b-it) or this

```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("./gemma_2B")
model = AutoModelForCausalLM.from_pretrained("./gemma_2B", device_map="auto", torch_dtype=torch.float32)

input_text = "Question: 1+1=\nAnswer:"
input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")

outputs = model.generate(**input_ids,max_length=50)
# print(outputs)
print(tokenizer.decode(outputs[0]))
```

There is nothing we can do about how good / bad the mode is! Closing as unrelated!
DeepSpeed Support Stage 3 
Author: bdytx5
Pull request: no
Labels: name: Feature request

### System Info

Does the trainer support stage 3? 

According to https://huggingface.co/transformers/v4.3.0/main_classes/trainer.html - it does not. 

Thanks,
Brett 

### Who can help?

na

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

use trainer with stage 3

### Expected behavior

Parameter partitioning 

Comments:
CC @pacman100 

Hello, Trainer supports DeepSpeed Stage 3. please take a look at the latest documentation at https://huggingface.co/docs/transformers/deepspeed
AutoModel.from_config() should attempt to load model relative to where the config was loaded.
Author: rl337
Pull request: no
Labels: name: Feature request

### System Info


Currently if you do an AutoConfig.from_pretrained() and then use that config to try to AutoModel.from_config() it loses the context from where the config.json was loaded.  This wouldn't be a problem if AutoModel.from_config() also took a path_or_model_id style param but it doesn't.

This becomes problematic mainly for situations where you want to write single code for both training and fine tuning models.  In this workflow, the notion of training and fine tuning is largely the same code so creating the model to then train looks something like this:

```
try:
    model = AutoModel.form_pretrained(..., config=conf, ...)
except OSError, ValueError:
    model = AutoModel.from_config(conf)
```

If you're loading models from file like i am, not being able to easily AutoModel.from_config() with a specified path is problematic.  I would like all of my intermediate training steps to be done locally to potentially unrelated directory structures to the source. 

I think that we have two easy options. 
1. Add a field to the PretrainedConfig which is only populated when it's loaded from file which is the path to the config file the config was loaded from.  We can call it something like _loaded_from_file.  It would have to be removed prior to save()
2. allow AutoModel.from_config() to take a path_or_model_id in addition to the config.  so the call would look something like:

```
model = AutoModel.from_config(conf, path_or_model_id='/path/to/dir/containing/model')
```

cc:
@Rocketknight1 

Trying out recommended workflow of issuing bugs before PRs. 


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Load a model config specifying custom code in the AutoMap from a path that is not also valid as `$CWD/model-id/model`
2. try to instantiate a version of your model from AutoModel.from_config() using that config.

### Expected behavior

I'd like to have an instance of the model that's of the same type as the model that'd be returned if from_pretrained() were used. 

Comments:
Revert "Automatic safetensors conversion when lacking these files (#2…
Author: LysandreJik
Pull request: yes

Temporary revert in order to cleanly fix the issue reported in https://github.com/huggingface/transformers/issues/29500

Comments:
Failing test is flaky

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29507). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
prophetnet_multi
Author: wyl-willing
Pull request: no
Labels: name: New model

### Model description

Multilingual text generation model based on prophetnet.

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

_No response_

Comments:
Could you add a link to the originl github repo and the weights / the paper as well? 🤗
fix: Avoid error when fsdp_config is missing xla_fsdp_v2
Author: ashokponkumar
Pull request: yes

# What does this PR do?

Prevent Keyerror when fsdp_config is missing xla_fsdp_v2 key.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @pacman100

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc and @younesbelkada

Documentation: @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
Flava multimodal add attention mask
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Fixes #29408 . Multimodal part of FlavaModel was generating attentions masks with full ones on the fly, ignoring attentions masks passed by the user. Fixed by generating masks as a concatenation of the two modality's masks.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@ArthurZucker @gante 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29446). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@ArthurZucker Oh yeah, added a check that the mask is not `None` and ran all `Flava` tests, including slow. 

I will work on the test for padding vs. unpadding soon. It will be a general modeling test, so I guess we do not need to write a separate padding test for `Flava` right now.

@gante this can be merged