noamaanMulla-03/transformers-issues huggingface dataset:
Error: The selected decoder is not prepared for the encoder hidden states to be passed.
Author: nikhilajoshy
Pull request: no

### System Info

- `transformers` version: 4.37.2
- Platform: Windows-10-10.0.19045-SP0
- Python version: 3.11.0
- Huggingface_hub version: 0.20.3
- Safetensors version: 0.4.2
- Accelerate version: 0.26.1
- Accelerate config:    not found
- PyTorch version (GPU?): 2.2.0+cpu (False)

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

def init_enc_dec(enc_model_name: str = "google/mt5-small", dec_model_name: str = "facebook/opt-350m"):
    config_encoder = AutoConfig.from_pretrained(enc_model_name)
    config_encoder.is_encoder_decoder = False
    config_encoder.add_cross_attention = False
    config_encoder.is_decoder = False
    config_encoder.num_decoder_layers = 0

    config_decoder = AutoConfig.from_pretrained(dec_model_name)
    config_decoder.add_cross_attention = True
    config_decoder.is_decoder = True

    encoder = AutoModel.from_pretrained(enc_model_name, config=config_encoder)
    decoder = AutoModel.from_pretrained(dec_model_name, config=config_decoder)
    model = EncoderDecoderModel(encoder=encoder, decoder=decoder)

### Expected behavior

Not able to load any EncoderDecoderModel using any hugging face transformer models

Comments:
Hi @nikhilajoshy, thanks for raising an issue! 

Could you please provide a full traceback of the error encountered? 

Please note that the not all encoder-decoder pairs are compatible - we don't guarantee all possible pairings can be loaded and run. 

> Not able to load any EncoderDecoderModel using any hugging face transformer models

Could you clarify this comment. Are you unable to load any encoder-decoder pairing, or just `"google/mt5-small"` and `"facebook/opt-350m"`? Note, this is likely happening because `mt5` is a encoder-decoder and loading with `AutoModel` is loading both the encoder and decoder. I'd suggest using `MT5EncoderModel` to load the model instead. 

@amyeroberts  it was a problem with the decoder only models. It works with gpt2
test_encoder_decoder_model_generate for vision_encoder_deocder is flaky
Author: amyeroberts
Pull request: no

### System Info

transformers 4.38.0dev 


### Who can help?

@gante 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Hard to reproduce, unfortunately.

Running the single test enough times will trigger a failure:

```
python -m pytest -v tests/models/vision_encoder_decoder/test_modeling_vision_encoder_decoder.py::ViT2TrOCR::test_encoder_decoder_model_generate
```

Fails with: 
```
FAILED tests/models/vision_encoder_decoder/test_modeling_vision_encoder_decoder.py::ViT2TrOCR::test_encoder_decoder_model_generate - AssertionError: torch.Size([13, 8]) != (13, 20)
```

Reference CI run: https://app.circleci.com/pipelines/github/huggingface/transformers/83611/workflows/666b01c9-1be8-4daa-b85d-189e670fc168/jobs/1078635/tests#failed-test-0

### Expected behavior

Non-flaky behaviour for the tests.

Comments:
cc @zucchini-nlp
Flax: Flax examples without pytorch dependencies
Author: gante
Pull request: yes

# What does this PR do?

WIP

Comments:
Nope, the speech recognition example relies on torch's Dataloader :(

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29021). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Mask Generation Task Guide
Author: merveenoyan
Pull request: yes

Added mask generation task guide.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_28897). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@MKhalusova thanks, I addressed your comments I think

@ArthurZucker please take a look. the guide LGTM

I hope feedback from the community is welcome. Nice work! 🤗

@ArthurZucker can you merge this if it's ok? 

@merveenoyan I can merge. Thanks for adding this!
Add tie_weights() to LM heads and set bias in set_output_embeddings()
Author: hackyon
Pull request: yes

# What does this PR do?

This fixes a bug from the wrong bias in prediction heads in some situations. The `predictions.bias` needs to be tied to `predictions.decoder.bias` inside `tie_weights()`. 

Repro Steps:
1. Sync to HEAD in main
2. Add the following test case to `test_modeling_bert.py` and run it:
```
    def test_save_load_bert_prediction_head(self):
        with tempfile.TemporaryDirectory() as tmpdirname:
            model_to_save = BertForMaskedLM.from_pretrained("bert-base-uncased")

            model_to_save.save_pretrained(tmpdirname)

            model = BertForMaskedLM.from_pretrained(
                tmpdirname,
                low_cpu_mem_usage=True,
            )
            model.to(torch_device)
```
3. Error is thrown:
```
FAILED tests/models/bert/test_modeling_bert.py::BertModelTest::test_save_load_bert_prediction_head - NotImplementedError: Cannot copy out of meta tensor; no data!
```
```
    def convert(t):
        if convert_to_format is not None and t.dim() in (4, 5):
            return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None,
                        non_blocking, memory_format=convert_to_format)
>       return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
E       NotImplementedError: Cannot copy out of meta tensor; no data!

../venv/lib/python3.9/site-packages/torch/nn/modules/module.py:1158: NotImplementedError
```

The issue was uncovered in https://github.com/huggingface/transformers/pull/28802.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@ArthurZucker @amyeroberts


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_28948). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
How to improve accuracy (Model on TrOCR)
Author: C108152117
Pull request: no

 I use TrOCR model to generate the math expression LATEX sequence according to the handwritten math expression image .
but the consequences are unsatisfactory ;below are eval/loss 、train loss and the repository :https://github.com/win5923/TrOCR-Handwritten-Mathematical-Expression-Recognition
![image](https://user-images.githubusercontent.com/101984630/160461914-30724c88-5984-4a76-bc3d-782499867406.png)
![image](https://user-images.githubusercontent.com/101984630/160461972-6c8c7161-7e51-4996-8412-c593a55b1612.png)
![image](https://user-images.githubusercontent.com/101984630/160463995-cde3ef6a-5912-4b2d-b1c5-01728efe009a.png)

![image](https://user-images.githubusercontent.com/101984630/160463408-93be2dd2-29c9-4e2e-911b-707fb0d9f3c0.png)
I use CROHME 2014 to test this model but worse than above image's models



Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Not exactly, but i think your pretrained_model's tokenizer is not optimized your math latex. so 

you should use trained latex tokenizer.
find it or collect latex data and create a new tokenizer using sentencepiece



does training a new tokenizer not mess with the pretrained weights? since they use a specific tokenizer.
Ray Release 2.7.0 breaks Trainer.hyperparameter_search 
Author: AphinityAT
Pull request: no

### System Info

When performing hyperparameter search with the ray backend, the following warning is raised and hyperparameter optimization is not performed:

```
File /ray/tune/trainable/util.py:315, in with_parameters(trainable, **kwargs)
    310 if _detect_checkpoint_function(trainable, partial=True):
    311     from ray.tune.trainable.function_trainable import (
    312         _CHECKPOINT_DIR_ARG_DEPRECATION_MSG,
    313     )
--> 315     raise DeprecationWarning(_CHECKPOINT_DIR_ARG_DEPRECATION_MSG)
    317 def inner(config):
    318     fn_kwargs = {}

DeprecationWarning: Accepting a `checkpoint_dir` argument in your training function is deprecated.
Please use `ray.train.get_checkpoint()` to access your checkpoint as a
`ray.train.Checkpoint` object instead. See below for an example:

Before
------

from ray import tune

def train_fn(config, checkpoint_dir=None):
    if checkpoint_dir:
        torch.load(os.path.join(checkpoint_dir, "checkpoint.pt"))
    ...

tuner = tune.Tuner(train_fn)
tuner.fit()

After
-----

from ray import train, tune

def train_fn(config):
    checkpoint: train.Checkpoint = train.get_checkpoint()
    if checkpoint:
        with checkpoint.as_directory() as checkpoint_dir:
            torch.load(os.path.join(checkpoint_dir, "checkpoint.pt"))
    ...

tuner = tune.Tuner(train_fn)
tuner.fit()
```

Python: 3.9
Transformers: 4.33.2
Ray: 2.7.0

**Possible Reason:**
The new Ray Release 2.7.0 changes the behavior of Checkpoints
https://github.com/ray-project/ray/releases/tag/ray-2.7.0

**Temporary Solution:**
Manually downgrade Ray to 2.6.3 

### Who can help?

@richardliaw @amogkam 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Steps to reproduce the behavior:

1. Install Ray 2.7.0
2. Run any hyperparameter code: e.g. https://huggingface.co/docs/transformers/hpo_train or https://github.com/huggingface/notebooks/blob/main/examples/text_classification.ipynb

### Expected behavior

Expected behiour is the execution of the hyperparameter search as described in the documentation or the text classification example

Comments:
Hello,

I am encountering the same error. 


Thanks



The team told me that they are working on a foward fix, did not check yet but feel free to confirm! 

@ArthurZucker just FYI this is linked to some failing `Trainer` tests currently as well, so once resolved those should be fixed too

Hello! Just wondering if there have been any updates to this. 

This problem still exists in Ray-2.8.0. Are there any solutions to fix this?

Some one please fix it!!!

Still facing this issue.

cc @justinvyu 

@jaanli Are you running with `transformers>=4.36.0` where https://github.com/huggingface/transformers/pull/26499 is included?
[`CI` / `core`] Fix CI with GC + pytorch 2.2
Author: younesbelkada
Pull request: yes

# What does this PR do?

Fixes the current failing CI for Mistral, Mixtral and Qwen2 for gradient checkpointing. For some reason, since pytorch 2.2, gradient checkpointing raises an error when going through in-place operations such as `tensor.mul_(xxx)` which was not the case in earlier versions. 

Simply replacing `causal_mask.mul_(~torch.eq(causal_mask, causal_mask.min()).all(dim=-1)[..., None])` by `causal_mask = causal_mask * (~torch.eq(causal_mask, causal_mask.min()).all(dim=-1)[..., None])`

This makes me think we should maybe have a job that runs the CI on torch nightly to catch these early bugs, do we have that already? If not, happy to have a look

cc @ArthurZucker @amyeroberts 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29026). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

It appears the rootcause was slightlly different, see: https://github.com/huggingface/transformers/pull/29027
[`CLeanup`] Revert SDPA attention changes that got in the static kv cache PR
Author: ArthurZucker
Pull request: yes

# What does this PR do?
cc @younesbelkada

#27931 removed `copied from` statements for Persimmon, Qwen2 and Mixstral / mistral which introduced unwanted changes for SDPA

Superseed #29026 
Closes: https://github.com/huggingface/transformers/pull/29026

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29027). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
FIX [`Trainer` / tags]: Fix trainer + tags when users do not pass `"tags"` to `trainer.push_to_hub()`
Author: younesbelkada
Pull request: yes

# What does this PR do?

As per title - and fixes: https://github.com/hiyouga/LLaMA-Factory/pull/2474#issuecomment-1941603142 raised by @hiyouga 

Indeed, we should always push tags if there are any that are saved on the model. Currently the logic is wrong, as it pushes the tags only if `"tags"` is passed to `trainer.push_to_hub()`, in fact we should always push the tag if `model.add_model_tags` has been called, regardless if `tags` is passed or not in `push_to_hub`

cc @amyeroberts 


Comments:
model.generate() produces different results with paddings
Author: zhentaocc
Pull request: no

### System Info

I found this issue when I try to reproduce `https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard` results, specifically gsm8k on `mncai/Llama2-7B-guanaco-dolphin-500`. My result is 13.12 while the one reported was 5.99.   
Also I found paddings make the outputs different (not sure if they are correct). For example, with batch size = 1, which means no padding, it's 13.12; with batch size =8, it is 5.69.   
And I traced the root cause, findings are the same input token ids can get different outputs with or without padding tokens.  
And the `pad_token_id `was 2, which is the same as `eos_token_id`. So if I set `pad_token_id `as 0, the results seems same as that without paddings.

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I tested following runs on RTX8000.
1. `python main.py --model=hf-causal-experimental --model_args="pretrained=mncai/Llama2-7B-guanaco-dolphin-500," --tasks=gsm8k --num_fewshot=5 --batch_size=1 `
2.  `python main.py --model=hf-causal-experimental --model_args="pretrained=mncai/Llama2-7B-guanaco-dolphin-500," --tasks=gsm8k --num_fewshot=5 --batch_size=8 `

### Expected behavior

1. accuracy should be ~13.12
2. accuracy should be ~5.69

Comments:
Feel free to check https://github.com/huggingface/transformers/issues/25420#issuecomment-1775317535 for a better explanation 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Add TinyViT
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR adds TinyViT.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Lag llama
Author: kashif
Pull request: yes

# What does this PR do?

Implementation of general time series forecaster and classifier using only the target values.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[i18n-<languageCode>] Translating docs to <languageName>
Author: goalend
Pull request: no
Labels: name: WIP

<!--
Note: Please search to see if an issue already exists for the language you are trying to translate.
-->

Hi!

Let's bring the documentation to all the <languageName>-speaking community 🌐 (currently 0 out of 267 complete)

Who would want to translate? Please follow the 🤗 [TRANSLATING guide](https://github.com/huggingface/transformers/blob/main/docs/TRANSLATING.md). Here is a list of the files ready for translation. Let us know in this issue if you'd like to translate any, and we'll add your name to the list.

Some notes:

* Please translate using an informal tone (imagine you are talking with a friend about transformers 🤗).
* Please translate in a gender-neutral way.
* Add your translations to the folder called `<languageCode>` inside the [source folder](https://github.com/huggingface/transformers/tree/main/docs/source).
* Register your translation in `<languageCode>/_toctree.yml`; please follow the order of the [English version](https://github.com/huggingface/transformers/blob/main/docs/source/en/_toctree.yml).
* Once you're finished, open a pull request and tag this issue by including #issue-number in the description, where issue-number is the number of this issue. Please ping @stevhliu and @MKhalusova for review.
* 🙋 If you'd like others to help you with the translation, you can also post in the 🤗 [forums](https://discuss.huggingface.co/).

## Get Started section

- [x] [index.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/index.md) https://github.com/huggingface/transformers/pull/20180
- [ ] [quicktour.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/quicktour.md) (waiting for initial PR to go through)
- [ ] [installation.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/installation.md).

## Tutorial section
- [ ] [pipeline_tutorial.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/pipeline_tutorial.md)
- [ ]  [autoclass_tutorial.md](https://github.com/huggingface/transformers/blob/master/docs/source/autoclass_tutorial.md)
- [ ]  [preprocessing.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/preprocessing.md)
- [ ]  [training.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/training.md)
- [ ]  [accelerate.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/accelerate.md)
- [ ]  [model_sharing.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/model_sharing.md)
- [ ]  [multilingual.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/multilingual.md)

<!--
Keep on adding more as you go 🔥
-->

Comments:
DeformableDetrModel support fp16
Author: DonggeunYu
Pull request: yes

# What does this PR do?

This PR for DeformableDetrModel support fp16.

https://github.com/huggingface/transformers/issues/29011

## Who can review?

@amyeroberts



Comments:
Actually, one thing we'll need to add is a test e.g. [like here for MT5](https://github.com/huggingface/transformers/blob/1ecf5f7c982d761b4daaa96719d162c324187c64/tests/models/mt5/test_modeling_mt5.py#L424). 

For the quality checks, running `make fix-copies` and pushing the changes should resolve the issues. You make need to make some additional adjustments to other modeling files to properly reflect the changes.

@amyeroberts
I took all of your feedback on board.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29013). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Marian models broken with latest transformers
Author: orendar
Pull request: no

### System Info

Transformers >= 4.31(and maybe also older), including master.

### Who can help?

@joaoga I noticed that you merged an update to Marian models.

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Thank you for your great work :)

I can confirm that pretrained Marian models are broken on transformers 4.31 and on master, and that they are working on 4.28.

For reproduction see for example the following snippet, taken from Helsinki-NLP/opus-mt-tc-big-he-en model card - on 4.28 I get the expected output, on 4.31 and master I get random gibberish. I don't know if this also happens with other models and languages but I suspect so.

``` 
from transformers import MarianMTModel, MarianTokenizer
import torch

src_text = [
    "היא שכחה לכתוב לו.",
    "אני רוצה לדעת מיד כשמשהו יקרה."
]

model_name = "Helsinki-NLP/opus-mt-tc-big-he-en"
tokenizer = MarianTokenizer.from_pretrained(model_name)
model = MarianMTModel.from_pretrained(model_name, torch_dtype=torch.bfloat16).to('cuda')
translated = model.generate(**tokenizer(src_text, return_tensors="pt", padding=True).to('cuda'))

for t in translated:
    print(tokenizer.decode(t, skip_special_tokens=True))

# expected output:
#     She forgot to write to him.
#     I want to know as soon as something happens.
```

### Expected behavior

See above

Comments:
Hey, this is related to https://github.com/Helsinki-NLP/Tatoeba-Challenge/issues/35 and a duplicate of #26216. The conversion script was a bit faulty and models need a re-upload. I can try to open PRs for relevant models 

That would be greatly appreciated, thank you! Right now I am running on transformers 4.28 but I am sure many others are interested in using the 1k+ translation models on latest version.

I pushed a lot of PR on the hub, if you are still seeing this feel free to ping me. I think some of the big models were not updated yet, we'll see

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hello, we have Marian-converted models that are experiencing this issue ([1] and [2]).

Could anyone recommend a solution please?

These models are known to work with transformers versions greater than 4.26.1 up to 4.30.2, after which garbage is returned by both using the model's generate() method and the transformers.pipeline.

Many thanks!

[1]: https://huggingface.co/techiaith/mt-dspec-health-en-cy
[2]: https://huggingface.co/techiaith/mt-dspec-health-en-cy

Edit: Sorry, just found #26216 which seems to be the correct place for this!

no worries, I think @LysandreJik is planning to finish updating the models that were missed in the great refactor!