noamaanMulla-03/transformers-issues huggingface dataset:
Informer - Transformer For Time-Series Forecasting 
Author: elisim
Pull request: no
Labels: name: New model

# Model description

Following the new support for Time Series Transformers in the [API](https://huggingface.co/docs/transformers/model_doc/time_series_transformer) (and the great blog by @NielsRogge and @kashif [here](https://huggingface.co/blog/time-series-transformers)), I propose adding "Informer" - AAAI 2021 Best Paper model. 

* Paper: [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting
](https://arxiv.org/abs/2012.07436)
* Model implementation: https://github.com/zhouhaoyi/Informer2020

## Why this model?

Compared to other forcasting transformers (see below), Informer seems to be the most "code-stable" one, with the most starts & forks in github.

Popular forcasting Transformes, with link to the repository:

[LogTrans](https://github.com/mlpotter/Transformer_Time_Series) - NIPS 2019 
[Informer](https://github.com/zhouhaoyi/Informer2020) - AAAI 2021 (Best Paper)
[Autoformer](https://github.com/thuml/Autoformer) - NIPS 2021 
[Pyraformer](https://github.com/alipay/Pyraformer) - ICLR 2022
[FEDformer](https://github.com/MAZiqing/FEDformer) - ICML 2022

This list based on the paper: [Are Transformers Effective for Time Series Forecasting?](https://arxiv.org/abs/2205.13504) (AAAI-23)

I would like to implement the model :)

Thank you,
Eli

### Open source status

- [X] The model implementation is available
- [ ] The model weights are available

### Provide useful links for the implementation

@zhouhaoyi - repository creator 

Comments:
thanks @elisim for the issue... indeed i have informer in my list of models to port over. I have the initial implementation of informer and other done: https://github.com/kashif/pytorch-transformer-ts and will move them over to the transformers API

Wow! saw your repo and it looks great! Maybe I might help? :) I sent you an email. 

Thanks, 
Eli 

merged in https://github.com/huggingface/transformers/pull/21099
Include decoder_attention_mask in T5 model inputs
Author: aashiqmuhamed
Pull request: yes

# What does this PR do?

This PR includes decoder_attention_mask as an argument in the prepare_inputs_for_generation function, helping enable the use of custom attention masks in the decoder.

Duplicate of https://github.com/huggingface/transformers/pull/22819

@gante @amyeroberts 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@amyeroberts Could you merge this PR please?
Fix weight tying in TF-ESM
Author: Rocketknight1
Pull request: yes

TF ESM cloned weights instead of tying, which worked when loading from PT but broke when loading from safetensors. This resolves the issue by correctly tying weights when this is enabled in the config. Fixes an ongoing CI error raised by @ydshieh 

Comments:
Also cc @gante in case he hates how I handled weight tying here, I don't want to break TF convention too much!

_The documentation is not available anymore as the PR was closed or merged._

@Rocketknight1 I'm cool with this :D
error when using from indobenchmark
Author: fendiirfan
Pull request: no

### System Info

```
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("indobenchmark/indobart-v2")
```
the error
`ValueError: Tokenizer class IndoNLGTokenizer does not exist or is not currently imported.`

any help guys?


### Who can help?

@ArthurZucker

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Run
```
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("indobenchmark/indobart-v2")
```

### Expected behavior

Can used

Comments:
Hey! The `tokenizer_class` that was set in the configuration.json is wrong as the `IndoNLGTokenizer` does not exist in transformers. You should try to ask the other of the model on the community tab how to use it, or try to use: 
```python 
from transformers import MBartTokenizer
tokenizer = MBartTokenizer.from_pretrained("indobenchmark/indobart-v2")
```
as it appears that the model is an  MBartModel.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
moved labels to the same device as logits for OTP, CODEGEN ,gptj and pixel2struct model
Author: sushmanthreddy
Pull request: yes

# What does this PR do?

As suggested in the [#22561](https://github.com/huggingface/transformers/issues/22561), moved labels to the same device as logits for the OTP model and codegen model

@sgugger  can u pls review and merge this pr??


@sgugger I am really sorry for the mess of this pr, I will not repeat this in the future .... pls once review and merge this...I should have created multiple branches for each pr but sorry this will not be repeated...


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Glad to help!
Pin flax & optax version
Author: amyeroberts
Pull request: yes

# What does this PR do?

Failing on [main](https://app.circleci.com/pipelines/github/huggingface/transformers/62534/workflows/d270e074-306d-4a8f-9434-fcdd979fae1b/jobs/770753) because of a new release of [optax](https://github.com/deepmind/optax/releases/tag/v0.1.5). Pinning until compatible versions with jax resolved. 



Fixes # (issue)


Comments:
_The documentation is not available anymore as the PR was closed or merged._
moved labels to the same device as logits for LILT model
Author: sushmanthreddy
Pull request: yes

# What does this PR do?

As suggested in the [#22561](https://github.com/huggingface/transformers/issues/22561)  moved labels to the same device as logits for the lilt model.

@sgugger pls review and merge it in the main branch.


Comments:
_The documentation is not available anymore as the PR was closed or merged._
[WIP] 🌐 [i18n-KO] Translated `tasks/translation.mdx` to Korean
Author: wonhyeongseo
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.mdx` to Korean" 으로 부탁드립니당 -->
# What does this PR do?

Translated the `tasks/translation.mdx` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179
<!-- 메인 이슈에 기록이 남아요! 가짜연구소 리포를 사용해 연습하실때는 제거해주시면 감사하겠습니다! :smile: -->

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

<!-- 제출 전 체크리스트로, 가짜연구소만의 체크리스트도 <details>로 감싸서 만들어두면 더 좋을 것 같아요. -->

## Who can review?
<!-- 가짜연구소 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
<!-- @sgugger, @ArthurZucker, @eunseojo May you please review this PR? -->

Team PseudoLab, may you please review this PR?

@0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd

Comments:
_The documentation is not available anymore as the PR was closed or merged._

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_22678). All of your documentation changes will be reflected on that endpoint.

Closed in favor of https://github.com/huggingface/transformers/pull/22805
Skip a failing test on main for now
Author: ydshieh
Pull request: yes

# What does this PR do?

On CircleCI, we have a failure on `main`:
```
FAILED tests/models/roberta/test_modeling_roberta.py::RobertaModelTest::test_assisted_greedy_search_matches_greedy_search
```

(BTW, it works on daily CI GPU runners)

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Merge as it just skip the failing test.
Add X-Decoder Model 
Author: ChanBong
Pull request: no
Labels: name: New model

### Model description

X-Decoder is a generalized decoding pipeline that can predict pixel-level segmentation and language tokens seamlessly. X-Decoder is the first work that provides a unified way to support all types of image segmentation and a variety of vision-language (VL) tasks. 

The model exhibits strong transferability to a wide range of downstream tasks in both zero-shot and fine-tuning settings, achieving state-of-the-art open-vocabulary segmentation and referring segmentation on 10 settings of 7 datasets and should be a valuable addition to transformers library

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

Paper: https://arxiv.org/pdf/2212.11270.pdf
Code: https://github.com/microsoft/X-Decoder
Weights: https://huggingface.co/spaces/xdecoder/Demo/blob/main/xdecoder_focalt_last.pt

Author: @eltociear 
Cc: @NielsRogge @alaradirik 

Comments:
Hi @ChanBong, thanks for opening the issue!

You can expect to see an X-Decoder PR is the next two weeks :)

Hi @alaradirik, can we please collaborate in adding this model?

Hi @atharvakavitkar, the PR is almost done but won't include the _referring image editing_ task, which require integration with Stable Diffusion inpainting. Perhaps you could create a tutorial or demo for this task?

Hi @alaradirik, thank you for reaching out to me. I must admit that I have not yet added a model to HuggingFace. But I really want to learn how to do it. Would creating this tutorial be the right step? Or should I search for a simpler model to implement from scratch?
Add `automatic-mask-generation` pipeline for Segment Anything Model (SAM)
Author: ArthurZucker
Pull request: yes

# What does this PR do?
This need the SAM model + rebasing once merged 

```python 
from transformers import pipeline
import matplotlib.pyplot as plt
from PIL import Image
import numpy as np
import time

generator = pipeline("automatic-mask-generation", device = 0)
image_url = "https://huggingface.co/ybelkada/segment-anything/resolve/main/assets/car.png"

dog_url = "/home/arthur_huggingface_co/transformers/Arthur/dog.jpg"
raw_image = Image.open(dog_url).convert("RGB")

start = time.time()
outputs = generator(raw_image, points_per_batch = 256, pred_iou_thresh=1)
print(f"point_batch_size : {256}, {time.time() - start}")

def show_mask(mask, ax, random_color=False):
    if random_color:
        color = np.concatenate([np.random.random(3), np.array([0.6])], axis=0)
    else:
        color = np.array([30 / 255, 144 / 255, 255 / 255, 0.6])
    h, w = mask.shape[-2:]
    mask_image = mask.reshape(h, w, 1) * color.reshape(1, 1, -1)
    ax.imshow(mask_image)
    

plt.imshow(np.array(raw_image))
ax = plt.gca()
for mask in outputs["masks"]:
    show_mask(mask, ax=ax, random_color=True)
plt.axis("off")
plt.show()

plt.savefig("dog_results_2.png")
```
![image](https://user-images.githubusercontent.com/48595927/232851728-936eb6bc-0765-48db-bda6-787cb79205f7.png)


![image](https://user-images.githubusercontent.com/48595927/232851676-d025beb2-07cd-4f0d-9e53-d4ff23456c93.png)

<img width="621" alt="image" src="https://user-images.githubusercontent.com/48595927/232853562-9858cdc5-dc1c-41b3-b067-1ea013c63e0f.png">


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thank you all for your reviews! 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_22840). All of your documentation changes will be reflected on that endpoint.
Fix counting in Slack report for some jobs
Author: ydshieh
Pull request: yes

# What does this PR do?

Fix counting in Slack report for some jobs.

### Context
For the additional jobs (i.e. not model testing jobs), the number of failed tests are being summed across over all machine types (single/multi - gpu). We have strange things like single gpu deepspeed CI has only 1 failure but 86 was shown on the report, see:


<img width="812" alt="image" src="https://user-images.githubusercontent.com/2521628/233582320-81c3b61d-add4-4ad5-b007-2522ad0f44b3.png">





Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix Slack report for Nightly CI and Past CI
Author: ydshieh
Pull request: yes

# What does this PR do?

For these 2 CI, so for we get 

```bash
Single |  Multi | Category
     0 |      0 | [Errored out] Examples directory
     0 |      0 | [Errored out] PyTorch pipelines
     0 |      0 | [Errored out] TensorFlow pipelines
     ...
```
But they don't have these 3 jobs in their workflow. We just need to update the notification script.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
fix CLAP integration tests
Author: hollance
Pull request: yes

# What does this PR do?

I noticed that the CLAP feature extractor tests were not being run, and that once enabled, they fail.

This PR fixes these tests.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: HF Trainer: @stas00, Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._

I don't have merge rights, so if all is good, feel free to merge. :-)
Moved labels to enable parallelism pipeline in Luke model
Author: sushmanthreddy
Pull request: yes

# What does this PR do?

As suggested in the [#22561](https://github.com/huggingface/transformers/issues/22561),moved labels to the same device as logits for the luke model.

@sgugger can u pls review this pr ,there is mistake in the [#22907](https://github.com/huggingface/transformers/pull/22907),I have made necessary changes..

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add inputs_embeds functionality when generating with GPT-Neox 
Author: TobiasLee
Pull request: yes

This PR extends https://github.com/huggingface/transformers/pull/21405  and #21889  by @gante  to GPTNeox models (which also includes the recent Pythia Suite models), , making it accept inputs_embeds when generating.

## Who can Review?
@gante @sgugger 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Update Swin MIM output class
Author: alaradirik
Pull request: yes

# What does this PR do?
- Replaces incorrectly named `logits` output of `SwinMaskedImageModelingOutput` and `SwinV2MaskedImageModelingOutput` classes with the `reconstruction` attribute
- Sets `logits` as a property for backward compatibility and adds a deprecation warning

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [X ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [X ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?



Comments:
_The documentation is not available anymore as the PR was closed or merged._
Make sam ONNX exportable
Author: fxmarty
Pull request: yes

As per title, would be great to have this PR on the next release so that we can support the ONNX export (see https://github.com/huggingface/optimum/pull/995). This piece is the only blocking one.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@ArthurZucker The PR is here: https://github.com/huggingface/optimum/pull/995
Fix: Seq2SeqTrainingArgs overriding to_dict for GenerationConfig json support
Author: Natooz
Pull request: yes

# What does this PR do?

`Seq2SeqTrainingArguments` override the `to_dict()` method from `TrainingArguments`.

This is a fix to #22831 (solution 2), solving an error that happened when saving to json a `Seq2SeqTrainingArguments` object with a `GenerationConfig` attribute.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? #22831
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@sgugger @gante 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Just updated

Thanks for adding this @Natooz 💛
Expose AutoModelForMaskGeneration
Author: fxmarty
Pull request: yes

As per title, with this PR `from transformers import AutoModelForMaskGeneration` works.

An alternative could be to remove `AutoModelForMaskGeneration` (as `AutoModel` already does the job), but currently for sam Hub metadata it is `AutoModelForMaskGeneration` that is used and not `AutoModel`: https://huggingface.co/datasets/huggingface/transformers-metadata/blob/main/pipeline_tags.json#L576

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Either way is fine for me - it's just to have consistency between the transformers Hub metadata and transformers. An either fix would be to just change the metadata, and remove the `AutoModelForMaskGeneration`.

So which one do you want @ArthurZucker ? I'm fine either way.

Let's go with `AutoModelForMaskGeneration` 😉
Seq2SeqTrainingArguments.generation_config not json serializable
Author: Natooz
Pull request: no

### System Info

👋

Following #22323, the `Seq2SeqTrainingArguments` `generation_config` attribute can be a `GenerationConfig` object.
When saving a `Seq2SeqTrainingArguments` object as json (as done during training / with tensorboard), it is first converted to a dictionary. But a `GenerationConfig` is not serializable --> error

### Who can help?

cc @gante @sgugger 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```Python
generation_config = GenerationConfig(
    max_new_tokens=64,
    top_k=20,
    top_p=0.9,
)
gen_training_args = Seq2SeqTrainingArguments(generation_config=generation_config)

as_dict = gen_finetune_training_args.to_dict()
as_json_str = gen_finetune_training_args.to_json_string()  # error here
```

### Expected behavior

To fix this, two possible solutions are :

1. Modify the expected types of `generation_config` to `[str, Path, dict]`, possibly converting arguments passed as `GenerationConfig` to dictionaries in `__post_init__`, and modify the behavior of `Seq2SeqTrainer.load_generation_config` to handle dictionaries;
2. Make `Seq2SeqTrainingArguments` override [`to_dict()`](https://github.com/huggingface/transformers/blob/03462875cc2d6506eb66a74de7d19b93ce968596/src/transformers/training_args.py#L1833) (or directly modify in `TrainingArguments`), to recursively convert non-json-serializable attributes to dictionaries.

Which sounds better do you think ? Or maybe you have a better one.
In any case I can handle it, as I feel responsible for this error.

Comments:
Hey @Natooz 👋 

We're more responsible than you, since we are supposed to catch that sort of issue in advance during the review process ;) But that's okay! It's normal to create bugs while trying to move forward at a good pace 🚀 

I believe that option 2 (recursively converting attributes to dictionaries) would be preferable. @sgugger, WDYT?

Option 2 is fine, but just on `Seq2SeqTrainingArguments`, to replace the generation config by `generation_config.to_dict()`.
Add perf_train_gpu_one.mdx italian translation
Author: Baelish03
Pull request: yes

See issue #17459 
Good evening. 
I didn't translate technical terms and I preferred to keep them in english. So I hope it's all ok. 
Good bye.

Comments:
@Baelish03 Thanks for adding this! To resolve the failing quality checks, you'll need to run `make fixup` locally and push any changes.