noamaanMulla-03/transformers-issues huggingface dataset:
`logging_dir` is not being generated.
Author: vanbasten23
Pull request: no

### System Info

Hi. I'm using huggingface model `bert base` on cloud TPU but it couldn't generate the `--logging_dir` that I expected.





### Who can help?

@sgugger 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

The script is at [here](https://github.com/GoogleCloudPlatform/ml-testing-accelerators/blob/master/tests/pytorch/nightly/hf-lm.libsonnet).

To reproduce, run these commands in a docker container on a TPU VM:
```
          $ cd
          $ git clone https://github.com/huggingface/transformers.git
          $ cd transformers && pip install .
          $ pip install datasets evaluate scikit-learn
          $ python3 examples/pytorch/xla_spawn.py \
            --num_cores 8 \
            examples/pytorch/language-modeling/run_mlm.py \
            --logging_dir ./tensorboard-metrics \
            --cache_dir ./cache_dir \
            --dataset_name wikitext \
            --dataset_config_name wikitext-2-raw-v1 \
            --do_train \
            --do_eval \
            --overwrite_output_dir \
            --output_dir language-modeling \
            --logging_steps 30 \
            --save_steps 3000 \
            --overwrite_cache \
            --debug tpu_metrics_debug \
           --model_type=bert \
          --model_name_or_path bert-base-cased \
          --num_train_epochs 1 \
          --per_device_train_batch_size 16 \
          --per_device_eval_batch_size 16
```


### Expected behavior

I expect a folder `tensorboard-metrics` to be generated in `~/transformers` but I couldn't find it.

Comments:
Hi @vanbasten23, 

Do you have tensorboard installed in your environment?

Could you share the the running environment: run `transformers-cli env` in the terminal and copy-paste the output?

> Do you have tensorboard installed in your environment?

Thanks. It turns out once I installed tensorboard, the folder was generated. Btw, do you know how transformer writes to the tensorboard? I searched things like "from torch.utils.tensorboard import SummaryWriter" in the transformer codebase but I couldn't find any reference.


Writing logic is defined in the [TensorBoardCallback](https://github.com/huggingface/transformers/blob/7edc33ac7a2572698045fed3b5115bca23f40805/src/transformers/integrations.py#L573). 

This is added by default as a reporting callback if tensorboard is in your environment [here](https://github.com/huggingface/transformers/blob/7edc33ac7a2572698045fed3b5115bca23f40805/src/transformers/trainer.py#L539). 

Thanks a lot @amyeroberts !
Question in load datasets  of train seq2seq model 
Author: xyx361100238
Pull request: no

### System Info


- `transformers` version: 4.28.0.dev0
- Platform: Linux-5.4.0-149-generic-x86_64-with-debian-bullseye-sid
- Python version: 3.7.16
- Huggingface_hub version: 0.13.2
- PyTorch version (GPU?): 1.13.1+cu116 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@sanchit-gandhi @sgugger 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

I Was change official example scripts data load to my own code:
`    if data_args.data_path is not None:
        print(data_args.data_path)
        raw_datasets = load_dataset("audiofolder", data_dir=data_args.data_path, cache_dir=model_args.cache_dir)
        raw_datasets = raw_datasets.cast_column("audio", Audio(sampling_rate=16000))
        raw_datasets = raw_datasets["train"].train_test_split(test_size=0.005, shuffle=True)`

The processing process is as follows: 
1、Resolving data files
2、Downloading data files
3、Computing checksums
4、Downloading data files
5、Extracting data files
6、Generating train split
What caused the significant difference in step six？
![image](https://github.com/huggingface/transformers/assets/19569322/557c99cd-21fa-4805-a1a7-83fe8ecd8530)


### Expected behavior

load fast，need at least 1000+

> Generating train split: 388773 examples [32:24:45, 1574.04 examples/s]

Comments:
I think this is just CPU workers spinning up, you can verify by asking on the datasets repo where you can get dedicated datasets help: https://github.com/huggingface/datasets

Yes，I have already asking on the datasets repo but no response yet

Feel free to gently ping `mariosasko` (he's usually very responsive)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Removing unnecessary `device=device` in modeling_llama.py
Author: Liyang90
Pull request: yes

Removing unnecessary `device=device`

# What does this PR do?

Removing unnecessary `device=device` in the second argument to `torch.full`.

`torch.full` expects a scaler for the second argument: https://pytorch.org/docs/stable/generated/torch.full.html
So if a device tensor is passed to it, the tensor needs to be synced and sent to CPU first. On TPU, this blocks the tracing of the current iteration that should be overlapped with graph execution of the previous iteration.

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._
[fix] Change the condition of ValueError in "convert_checkpoint_from_transformers_to_megatron"
Author: SeongBeomLEE
Pull request: yes

The "target_tensor_model_parallel_size" is related to "num_attention_heads", and the "target_pipeline_model_parallel_size" is related to "num_hidden_layers". 

However, the old code had "target_tensor_model_parallel_size" related to "num_hidden_layers". 

So we modified the code and added the part about "target_tensor_model_parallel_size".

Thanks!

Comments:
cc @pacman100 

_The documentation is not available anymore as the PR was closed or merged._
Link with accelerate
Author: muellerzr
Pull request: yes

# What does this PR do?

Replicates the logic in https://github.com/huggingface/accelerate/pull/1718 here on the trainer, to reduce the sync overhead as `get_scale` is a full-sync operation, meaning both GPU and CPU need to fully stop before continuing. This PR reduces it by half when not using the Accelerator.

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Support RefinedWebModel as a model_type for Falcon
Author: Rocketknight1
Pull request: yes

This PR allows us to temporarily revert the model_type for Falcon repos to fix some issues.

cc @sgugger @LysandreJik @Narsil @OlivierDehaene @slippylolo 

Comments:
No this is not enough. If we go down that road we need to erase the `RefinedWeb` model type on loading to replace it with `falcon`, so that a new saved version does not keep it.

_The documentation is not available anymore as the PR was closed or merged._

This already happens! The model type that is saved in `config.json` is set in `configuration_falcon.py`, with the line `model_type = "falcon"`. If you load a model with model_type `RefinedWebModel` and save it, the output `config.json` has model_type `falcon`.

Closing because just changing the `model_name` field won't be enough anyway - we also need to revert all the `config.json` parameters.
Fix MobileVitV2 doctest checkpoint
Author: amyeroberts
Pull request: yes

# What does this PR do?

Doctests CI currently fails on MobileVitV2 tests. Doctest was copied from the V1 mobilevit, and checkpoints didn't match. Remove the copied from as there's just some very simple and standard model head logic coped. 

Interestingly, the [example for MobileVit V1](https://github.com/huggingface/transformers/blob/21946a8cf4a273f35ac2f3a53edafc398699f527/src/transformers/models/mobilevit/modeling_mobilevit.py#L1027) I'd expect to fail as it doesn't import torch. [It is included in the doc tests](https://github.com/huggingface/transformers/blob/21946a8cf4a273f35ac2f3a53edafc398699f527/utils/documentation_tests.txt#L300).

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._

> So in MobileVitV2 doc example, the torch has to be imported?

No, not to have the test pass. I added it because it wouldn't run if you copy-pasted the snippet from the docs.

I've added it to MobileVit v1 too now for the same reason :)
Skip torchscript tests for `MusicgenForConditionalGeneration`
Author: ydshieh
Pull request: yes

# What does this PR do?

This model class requires the model tester to prepare `input_values` and `padding_mask` for torchscript tests.
So far I think it is fine to skip it until we have high usage.


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Unexpect behaviour
Author: lucasjinreal
Pull request: no

I have a base model and a lora model trained. And a new model which save with merge_and_unload()

But the saved model loaded always got OOM, is that normal? This should't happen in my opiiopn.

the saved model is same size aas 7B

![image](https://github.com/huggingface/transformers/assets/21303438/a588f3fb-00fc-4d2c-8dd6-8f7d138d42ca)


and the script I load model are same.

Comments:
cc @younesbelkada and @pacman100 
But they won't be able to help you without seeing a reproducer of the issue.

this is very weired, I wish I could post a reproducible code here but this could only be done by opensource all my code which is limited due to policy. However, I think merged lora and the basemodel should be likely the same, why they could even get OOM, could be any possible reason?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
MaskFormerSwin shows as unsupported the index
Author: joaocmd
Pull request: no

Hello, is there any reason why the MaskFormerSwin shows as unsupported on the `index` page?

https://github.com/huggingface/transformers/blob/main/docs/source/en/index.mdx?plain=1#L364
```
|          MaskFormer           |       ❌       |       ❌       |       ✅        |         ❌         |      ❌      |
|        MaskFormerSwin         |       ❌       |       ❌       |       ❌        |         ❌         |      ❌      |
|             mBART             |       ✅       |       ✅       |       ✅        |         ✅         |      ✅      |
```

I think it is implemented in [this file](https://github.com/huggingface/transformers/blob/main/src/transformers/models/maskformer/modeling_maskformer_swin.py).

I also found this PR https://github.com/huggingface/transformers/pull/20344 which seemed like it added the model. The model is also missing from the "Supported models" subsection, but I didn't find its paper so is that part of the reason?


Comments:
@joaocmd Huh, that's odd. Thanks for reporting. Following the shared link, on `main` I see that MaskFormer is shown as a supported model. Perhaps you caught it in a weird moment before a patch was applied? 

<img width="1025" alt="image" src="https://user-images.githubusercontent.com/22614925/233962005-0e4714bc-3b3f-4548-a964-c0fcfe774b2a.png">



^sorry, I just realised that the link went to maskformer but it's `MaskFormerSwin` you're referring to. 


MaskFormersSwin is listed as a "private model" [here](https://github.com/huggingface/transformers/blob/3d3204c025b6b5de013e07dd364208e28b4d9589/utils/check_repo.py#L50). 

I suspect this is because MaskFormerSwin was added in order to be used as a backbone. @NielsRogge - is this correct? 

Yes ideally it shouldn't be in that public list, it's just there to be used for MaskFormer for backwards compatibility purposes.

Users can just use our regular Swin in case they want to use it as backbone for MaskFormer.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hi @amyeroberts and @NielsRogge, should anything be changed so that the model doesn't appear on that public list or should we close this issue?

@joaocmd If you want to open a PR to fix, I'd be very happy to review :) I don't think it's critical to address however.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Implement QFormer for pretrain
Author: dinhanhx
Pull request: no

### Feature request

In [BLIP-2](https://arxiv.org/pdf/2301.12597.pdf), there is a pretraining stage (or stage 1) of QFormer.
![image](https://user-images.githubusercontent.com/38489776/230536840-5b466474-0e29-4029-976b-68c966b2b499.png)

Implementation of QFormer in this stage is requested.

### Motivation

In [HuggingFace's source code of BLIP-2](https://github.com/huggingface/transformers/blob/v4.27.2/src/transformers/models/blip_2/modeling_blip_2.py#L1019), I see no implementations for text inputs, Image-text contrastive loss, Image-grounded text generation loss, Image-text matching loss for pretraining. Currently, The source code only provides for vision-language generative learning (stage 2). Therefore, it will be very helpful for people who are interested in stage 1 of QFormer (like me).

### Your contribution

Unfortunately, I don't think there is a way that I could help.

Comments:
@NielsRogge Gentle ping because I saw your name in the docs

cc @younesbelkada 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

as the issue is reopened, is there any plan to impl the loss for qformer?

Hi @jianantian , I didn't had time to have a look unfortunately, if you want to try your hands on it, feel free to open a PR and we'll guide you frm there!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Deprecate models
Author: sgugger
Pull request: yes

# What does this PR do?

This PR creates the precedent of deprecating models in the library. By deprecating we indicate that we will stop maintaining such models, but there is no intention of actually removing those models and breaking support for them (they might one day move into a separate repo/on the Hub but we would still add the necessary imports to make sure backward compatibility stays). The main point is that we stop testing those models (to ease a bit the burden on our CI).

Deprecated models are moved in models/deprecated so direct import of objects from their modeling files will break (though that's easily fixed by adding the `.deprecated` in the path). They are removed from the `tests` folder and a mention is added in the doc page of the model.

The heuristic to pick the deprecated models in this PR is: models older than a year that got less than a cumulated 1,000 downloads (over all checkpoints) in the last 30 days (counting deduplicated downloads).

Comments:
_The documentation is not available anymore as the PR was closed or merged._

FMI: (M=my)

I probably need to do something with daily CI to take this PR's change into account!
Add accelerate version in transformers-cli env
Author: amyeroberts
Pull request: yes

# What does this PR do?

What is says on the tin. Now Trainer uses accelerate, me & others are having to ask for the accelerate version often in issues.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sgugger I've added the config too, matching the accelerate logic: a specific config file can be passed in, or the default config is read if it exists. LMKWYT :)