noamaanMulla-03/transformers-issues huggingface dataset:
[Doctest] Add new checks
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Checks to make sure that the examples in any `doc/en/...` are tested
Checks to make sure that any model or config examples are also nightly tested

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Will add all the files to the IGNORE_DOC_NON_TESTED

Mostly the intent is to make sure new models that are added to the library are doctested (while currently the reviewer has to make sure it is added, but we usually forget)

Looking at the new `check_pr_documentation_tests` that only checks files that are modified + in the `documentation_test.txt` , my initial goal of making sure that new models are tested is not attained. This PR will adress that, by adding a check to make sure that new model addition are adding the files to test them. cc @ydshieh 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Sure! Closing this.
Support Transformer Engine and FP8 training
Author: zhuzilin
Pull request: no

### Feature request

NVIDIA has proposed the FP8 tensor core along with a [Transformer Engine](https://github.com/NVIDIA/TransformerEngine) library that implements the corresponding kernels and mix precision strategies (i.e. the delay scaling strategy). I wonder if we have plan on supporting transformer engine here? This could make better use of the newest hardware.

### Motivation

Make better use of the newest hardware, especially H100.

Comments:
There is work ongoing to add support to it in [Accelerate](https://github.com/huggingface/accelerate/tree/fp8_integration) first. Once this is tested and merged, we will also port it to the `Trainer`. For now we are hit by a regression problem we are trying to fix with the team at Nvidia.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

The fp8 is now supported by Ada, and it is included in Accelerate. Will work continue on including fp8 in Trainer?
@sgugger

The Trainer will soon use Accelerate, so this will come for free.

Pls update here once the trainer has supported it, thanks!

Looking forward to updates!

When all is function supported in transformers.TrainingArguments?
pin `apex` to a speicifc commit (for DeepSpeed CI docker image)
Author: ydshieh
Pull request: yes

# What does this PR do?

The docker image build for DeepSpeed job in CI fails since ~ one week due to this [apex issue](https://github.com/NVIDIA/apex/issues/1679).

Let's pin to the previous commit until the above mentioned issue is resolved on `apex` side.

Currently, the DeepSpeed job fails as the above failure prevents to use newer images that include some fixes on  `accelerate` side.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Clean up disk sapce during docker image build for `transformers-pytorch-gpu`
Author: ydshieh
Pull request: yes

# What does this PR do?

PyTorch pipeline CI job start to fail due to 
```bash
 ImportError: accelerate>=0.20.3 is required for a normal functioning of this module, but found accelerate==0.20.2.
Try: pip install transformers -U or pip install -e '.[dev]' if you're working with git main
```

**The root cause is: docker image for this job failed to build due to disk space issue**
```bash
RROR: Could not install packages due to an OSError: [Errno 28] No space left on device
```

As usual, let's us save Space!

Comments:
_The documentation is not available anymore as the PR was closed or merged._
🌐 [i18n-KO] Display English version of untranslated pages as a default option for improved UX 
Author: wonhyeongseo
Pull request: no

Part of https://github.com/huggingface/transformers/issues/20179
### Summary

We've been collecting feedback from our users, and a common theme we've identified is concerning the user experience when encountering untranslated pages. Users have expressed a preference to see the English version of a page directly if the localized translation is not yet available. This allows them to utilize tools like Google Chrome's translation feature to understand the content. 

### Detail

#### Current Behavior:

Presently, when a user navigates to a page that hasn't been translated into their selected language, they're presented with a placeholder message indicating that the page is unavailable in their language. This leaves them without immediate access to the content they were seeking.

#### Desired Behavior:

Instead of showing a placeholder message for untranslated pages, users prefer to be automatically shown the English version of the page. This way, they can use automated translation tools if necessary. In addition, users can also choose to read the page in English if they're comfortable doing so.

### Additional Benefits

This approach provides several additional benefits:

- **Conflict mitigation:** This will likely reduce the number of merge conflicts that can occur due to manual reference edits in multiple language versions.
- **Easier Table of Contents (TOC) control:** By defaulting to English versions, we ensure that the TOC structure remains consistent across the site, providing a more predictable and streamlined user experience.

### Next Steps

We suggest implementing this feature to enhance the user experience and streamline the site's operations. This would involve modifying the behavior of the site when a page is unavailable in the selected language. We look forward to feedback and any additional perspectives on this proposed enhancement.

cc: [HF] @sgugger @ArthurZucker [PseudoLab] @0525hhgus @gabrielwithappy @sim-so @HanNayeoniee @jungnerd @KIHOON71


Comments:
This should already be the case without having to do anything special. cc @mishig25
Fix `KerasMetricCallback`: pass `generate_kwargs` even if `use_xla_generation` is False
Author: Kripner
Pull request: yes

# What does this PR do?

Currently, `KerasMetricCallback` ignores the `generate_kwargs` argument if `use_xla_generation` is set to `False` (which is the default). This means that when not using XLA, the user can't pass arguments like `max_new_tokens` to the `generate` method being called in `on_epoch_end`. It's also in contradiction with the docstring for `generate_kwargs`, which states:

> Keyword arguments to pass to `model.generate()` when generating. Has no effect if `predict_with_generate` is `False`.

This PR fixes the issue by passing `generate_kwargs` to `model.generate()` in the branch of execution where `use_xla_generation` is `False`.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@Rocketknight1 
@gante

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix device issue in `SwitchTransformers`
Author: ydshieh
Pull request: yes

# What does this PR do?

Need a tiny fix after #24300.

Currently, we have a failure


```bash

self = <tests.models.switch_transformers.test_modeling_switch_transformers.SwitchTransformersEncoderOnlyModelTest testMethod=test_multi_gpu_data_parallel_forward>

    @staticmethod
    def forward(ctx, target_device, dim, *inputs):
>       assert all(i.device.type != 'cpu' for i in inputs), (
            'Gather function not implemented for CPU tensors'
        )
E       AssertionError: Gather function not implemented for CPU tensors

/usr/local/lib/python3.8/dist-packages/torch/nn/parallel/_functions.py:56: AssertionError
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Merge now. Don't hesitate to leave comments if any @ArthurZucker .
Make `AutoFormer` work with previous torch version
Author: ydshieh
Pull request: yes

# What does this PR do?

Without `import torch.utils.checkpoint` (which we have in other files, like `Bart`), with torch 1.13, we got an error

(running `RUN_SLOW=1 python3 -m pytest -v tests/models/autoformer/test_modeling_autoformer.py::AutoformerModelTest::test_training_gradient_checkpointing`)

```bash
>                   layer_outputs = torch.utils.checkpoint.checkpoint(
                        create_custom_forward(encoder_layer),
                        hidden_states,
                        attention_mask,
                        (head_mask[idx] if head_mask is not None else None),
                    )
E                   AttributeError: module 'torch.utils' has no attribute 'checkpoint'
```

Let's make it work with previous torch version(s) ❤️ .

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix ImageGPT doctest
Author: amyeroberts
Pull request: yes

# What does this PR do?

#24317 Resolved the ImageGPT doc test failing issue, as `clusters` in the image processor were not stored as numpy arrays as expected. This was tested by running the code directly, but I didn't run using  
` pytest --doctest-modules src/transformers/models/imagegpt/modeling_imagegpt.py ` 🙃 

The tests were failing because some code produces an output e.g. model architecture when caling `model.to`, but no "expected" output is provided. We don't want to check these outputs, so this PR adds controls to ignore. 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
_The documentation is not available anymore as the PR was closed or merged._

@ydshieh Running the doctests (properly this time :) ), the tests pass with the ignore statement on the for loop, and fail without (in the same way as on the CI).
xlm-roberta-xlarge doesn't exist
Author: Jack000
Pull request: no

### System Info

from https://huggingface.co/docs/transformers/model_doc/xlm-roberta-xl

`model = XLMRobertaXLForSequenceClassification.from_pretrained("xlm-roberta-xlarge")
`
I can't find xlm-roberta-xlarge on https://huggingface.co/models

there's facebook/xlm-roberta-xl - but this is the raw masked token model and doesn't seem to work with XLMRobertaXLForSequenceClassification, it's missing the classifier head.

model = XLMRobertaForSequenceClassification.from_pretrained("xlm-roberta-large")
works just fine for me

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

from transformers import AutoTokenizer, XLMRobertaXLForSequenceClassification
model = XLMRobertaXLForSequenceClassification.from_pretrained("xlm-roberta-xlarge")

### Expected behavior

I guess xlm-roberta-xlarge should be available, or the docs should be amended..

Comments:
Hi @Jack000 , thanks for reporting this issue. 

This is indeed odd - the checkpoint being reference doesn't seem to exist. In the original PR adding the model to the library - it seems the [checkpoints were added under the facebook org](https://github.com/huggingface/transformers/pull/13727#pullrequestreview-866391378). It's OK if the checkpoint used in the example doesn't have the weights for the specific head e.g. for [multiple choice for bert we use `bert-base-uncased`](https://huggingface.co/docs/transformers/v4.29.1/en/model_doc/bert#transformers.BertForMultipleChoice). Would you like to open a PR to update the checkpoint? 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[Good first issue] LXMERT PyTorch Integration tests
Author: LysandreJik
Pull request: no
Labels: name: PyTorch

name: Tests

name: Good First Issue

The PyTorch implementation of the LXMERT model currently has no integration tests. This is problematic as the behavior can diverge without being noticed.

The [test_modeling_lxmert.py](https://github.com/huggingface/transformers/blob/master/tests/test_modeling_lxmert.py) file should be updated to include integration testing.

An example of a good modeling integration test is visible in the [test_modeling_bert.py#L552-L565](https://github.com/huggingface/transformers/blob/master/tests/test_modeling_bert.py#L552-L565) file:

https://github.com/huggingface/transformers/blob/1809de5165804666ba6c6a02a9d177f6683869cc/tests/test_modeling_bert.py#L552-L565

Some additional tips:
- The test must be marked as slow using the `@slow` decorator, so as to be run *daily*, and not on every commit of every branch/pull request of this repository.
- The test must be decorated with the `@require_torch` decorator so as to only be run in environments using PyTorch.
- A single test is necessary. If you feel like implementing multiple of these, then sharing the same checkpoint would be ideal so as to reduce download time.

Comments:
@LysandreJik I'll tackle this one if that's cool?

Hi @jmwoloso! @sadakmed has already contributed a proposal in #9989 that unfortunately slipped through the net - could you give it a look and let me know if that's what you had in mind?

Hi @LysandreJik! I originally saw #9954 and was going to make mine based upon that, but I see there have been some updates via #9989 so I'll check that out and adjust if needed, but yes, was going to essentially modify the TF integration test to be PT compatible. @sadakmed does mention adding a context manager to deal with the random seed in #10143 so not sure if that is of interest, but the idea is that I'll use the TF implementation and make it PT compatible.

Hi @jmwoloso, DO you mean TF implementation of lxmert integration test, I committed for both tf [#10052](https://github.com/huggingface/transformers/pull/10052) and pt [#9989](https://github.com/huggingface/transformers/pull/9989) (since Feb nd I still have this open tabs in my mind of something is unfinished, really a pain). Both has the same issue of the inability to hardcode input coz it's too large, other details u already know,  How it's implemented know, I dont think that fixing seeds locally will impact others in different classes, not to mention in other tests, a Context manager is a safe way to get around it, constant input without affecting anything else, nd I saw somewhere in a torch library that they use this technique (so it's not a crazy idea).

Seems like this issue might be ready to be closed based on @sadakmed previously [merged PR](https://github.com/huggingface/transformers/pull/9989) in July of 21'. 

cc @LysandreJik 

Indeed! Thanks!
Training loss suddenly increases and stays the same
Author: gungor2
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     -->

## Details
<!-- Description of your issue -->

I am trying to develop a language model. The code is modified version of the example given [here](https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_language_modeling.py).

The problem is that at some point during the training, the training loss spikes up and stays the same. Eval loss follows this trend as well. I am attaching tensorboard plots for reference.

I am using transformers=3.2.0. I tested with transformers=3.3.0 and observed the same issue.

[TensorBoard.pdf](https://github.com/huggingface/transformers/files/5314434/TensorBoard.pdf)


Comments:
Hello, you would probably have more answers if you asked this question on the forums: https://discuss.huggingface.co

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


> # ❓ 问题与帮助
> ## 细节
> 我正在尝试开发语言模型。该代码是[此处](https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_language_modeling.py)给出的示例的修改版本。
> 
> 问题在于，在训练过程中的某个时刻，训练损失激增并保持不变。Eval loss 也遵循这一趋势。我附上张量板图以供参考。
> 
> 我正在使用变压器 = 3.2.0。我使用transformers=3.3.0 进行了测试并观察到了同样的问题。
> 
> [TensorBoard.pdf](https://github.com/huggingface/transformers/files/5314434/TensorBoard.pdf)

I also encountered this problem, is it solved?

Hi, I encountered similar issue when pretraining BERT. Have you solved this problem? Could you please share some insights?
MeZo Forward Pass Implementation
Author: thistleknot
Pull request: no
Labels: name: Feature request

### Feature request

https://github.com/princeton-nlp/MeZO/blob/main/large_models/trainer.py

### Motivation

Faster training

### Your contribution

Just a user atm.

Comments:
cc @sgugger and @pacman100 who know more about `Trainer` and integrations 

Should this be integrated with PEFT instead? https://github.com/huggingface/peft

Anw the motivation is not faster training; in fact it ought to be slower as far as I understand. Rather, it is lower memory requirement.

You are correct.  I misread/transcribed that.  I read x12 memory saved as
x12 more context available which leads to faster inference.

On Sun, Jun 18, 2023 at 4:44 AM jon-chuang ***@***.***> wrote:

> Anw the motivation is not faster training; in fact it ought to be slower
> as far as I understand. Rather, it is lower memory requirement.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/24264#issuecomment-1596114214>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ABHKKOSM7S3TL3DWEYH67UTXL3S3JANCNFSM6AAAAAAZFUOVYQ>
> .
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>


Created an issue in peft.  Wasn't aware hf managed both.
Format of the documentation (文档格式和可读性的问题）
Author: KKIverson
Pull request: no

### System Info

huggingface的文档格式真的看起来很乱，读起来很不流畅，强烈建议官方优化一下：

例如：https://huggingface.co/docs/transformers/installation


1. 段落之间几乎没有分隔符，没有标题，字体大小、行间距之类的也很差；
2. 超链和普通文字的格式几乎一样，容易误触；
3. 代码段也不够美观，看起来就像是从interactive 的IDE写的，不方便初学者复制。


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

希望能改进吧

### Expected behavior

希望能改进吧

Comments:
sorry，我的浏览器兼容性问题。
My bad！！！！
[pipeline] Explicitly set framework is ignored
Author: denis-ismailaj
Pull request: no

### System Info

- `transformers` version: 4.30.2

Omitting the rest because they aren't really relevant. Can submit later if needed.

### Who can help?

@Narsil

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
from transformers import pipeline
from transformers import WhisperProcessor, WhisperForConditionalGeneration

processor = WhisperProcessor.from_pretrained("openai/whisper-tiny")
model = WhisperForConditionalGeneration.from_pretrained("openai/whisper-tiny")

# Using this just to make sure an error if thrown if pipeline tries to check using module instead of specified framework.
class FakeWhisper:
    def __getattr__(self, item):
        return model.__getattr__(item)

pipe = pipeline(
    "automatic-speech-recognition",
    model=FakeWhisper(),
    tokenizer=processor.tokenizer,
    feature_extractor=processor.feature_extractor,
    chunk_length_s=30,
    device=model.device,
    framework="pt",
)
```

The above code raises this error:
```
TypeError: Could not infer framework from class <class '__main__.FakeWhisper'>.
```

### Expected behavior

When specifying the framework explicitely, there is no need to infer it from the module of the model class, as mentioned here:
https://github.com/huggingface/transformers/blob/main/src/transformers/pipelines/__init__.py#L784-L796

But then, inside `infer_framework_load_model`, `infer_framework` is called regardless of the value of the `framework` parameter:
https://github.com/huggingface/transformers/blob/main/src/transformers/pipelines/base.py#L280

Comments: