noamaanMulla-03/transformers-issues huggingface dataset:
Inconsistent dropout implementation across models
Author: rvashurin
Pull request: no

### Feature request

Dropout implementations vary across different models - for example, T5 implements dropout as a layer https://github.com/huggingface/transformers/blob/e893b1efbbce80b7eaaf24f9e0134450820782b5/src/transformers/models/t5/modeling_t5.py#L285 which is visible in the model printout, while Bart uses functional form of dropout https://github.com/huggingface/transformers/blob/e893b1efbbce80b7eaaf24f9e0134450820782b5/src/transformers/models/bart/modeling_bart.py#L434 which is essentially hidden from the user. It would be useful if either all models had the same (preferably, layer-like) implementation of dropout, or models had flag indicating how dropout is implemented in a model.

### Motivation

I am currently working with some Monte-Carlo sampling from parameter distributions of models, which requires a re-implementation of dropout to essentially repeat masks across batch and beam dimensions, instead of sampling fully random mask for all tensor dimensions. This is a trivial task if the model has separate layer for dropout operation, but not if it uses functional form. The latter case requires one to patch `torch.nn.functional.dropout` during runtime which is as dirty a hack as they come. I understand that this is a very niche problem, yet this would be beneficial from implementation consistency perspective if nothing else.

### Your contribution

I could work on solution if necessary and if HF staff signals that this can possibly be merged.

Comments:
I don't really see a reason why not to standardize the dropout to improve consistency ad readability, but that would probably require us to change quite a lot of models (380~ lines use `nn.functional.dropout` and 758 use `self.dropout = nn.Dropout` )  
Wondering rather if there are any specific known reason to use one rather than the other in the first place. 

FYI @LysandreJik 

Not an issue for me. We did something similar for activation functions ~3 years ago. I'd just make sure it's backwards compatible and that it's the most versatile (in this case, the one that can be patched)

I will try to assess what the best approach is and conjure up some draft PR soon-ish.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Issue with Installing black in Python - needs to install seperately
Author: rajveer43
Pull request: no

### System Info

I am encountering an issue when trying to install "black" after running the following commands:

```bash
pip install -e ".[docs]"
pip install git+https://github.com/huggingface/doc-builder
```

`black` should be installed with the above commands only, every time I need to install it separately.

### Who can help?

![image](https://github.com/huggingface/transformers/assets/64583161/a7480b6f-eca5-4240-9fb4-de5c9d528336)


### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Run 
```bash
pip install -e ".[docs]"
```
2. Run 
```bash
install git+https://github.com/huggingface/doc-builder
```
3. Attempt to install "black" by running 
```bash
pip install black
```

### Expected behavior

I expect` black` to be installed without any issues when above commands are executed.

Comments:
Would run `pip install -e ".[quality]"`, I think you need the styling 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix sliding_window hasattr in Mistral
Author: IlyaGusev
Pull request: yes

# What does this PR do?

It fixes a bug with checking `hasattr` result with `is not None`. `hasattr` returns `True` of `False`, so the condition is always  True without this fix.


## Before submitting
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

@ArthurZucker @younesbelkada or anyone else.

Comments:
One more possible way to fix it is to use `getattr(self.config, "sliding_window", None)` instead.

Just to better understand, how can `sliding_window` be set to `None`? It always defaults to a "non-None" value here: https://github.com/huggingface/transformers/blob/9da451713d5feac451505c09337ca07ff6d0dba0/src/transformers/models/mistral/configuration_mistral.py#L121

@patrickvonplaten correct, it should never be `None` indeed - in any case the condition `hasattr(config, "sliding_window")` can be removed I think

So this way if someone wants to disable sliding windows, they should either delete the attribute or set it to None.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

I think it is still relevant and should be merged.

Done, I've rebased.

Thanks for the fix!
Resume training with deepspeed resets learning rate
Author: BiEchi
Pull request: no

### System Info

- `transformers` version: 4.32.0.dev0
- Platform: Linux-3.10.0-1160.31.1.el7.x86_64-x86_64-with-glibc2.17
- Python version: 3.9.0
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.1
- Accelerate version: 0.21.0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.0.1+cu117 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@pacman100 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Launch script uses `run_mlm.py`:
```
python transformers/examples/pytorch/language-modeling/run_mlm.py \
        --resume_from_checkpoint $CKPT_TO_RESUME \
        --config_name $MODEL \
        --tokenizer_name $TOKENIZER \
        --dataset_name $PT_DATASET \
        --max_steps $MAX_STEPS \
        --preprocessing_num_workers 32 \
        --logging_steps $LOG_STEPS \
        --cache_dir $CACHE_DIR \
        --warmup_steps $WARMUP_STEPS \
        --learning_rate $PT_PEAK_LR \
        --lr_scheduler_type $PT_LR_DECAY \
        --output_dir $OUTPUT_DIR \
        --overwrite_output_dir \
        --deepspeed $DS_CONFIG
```
Deepspeed config:
```
{
    "zero_optimization": {
        "stage": 2,
        "allgather_partitions": true,
        "allgather_bucket_size": 2e8,
        "overlap_comm": true,
        "reduce_scatter": true,
        "reduce_bucket_size": 2e8,
        "contiguous_gradients": true
    },
    "fp16": {
        "enabled": false
    },
    "bf16": {
        "enabled": true
    },  
    "train_micro_batch_size_per_gpu": "auto",
    "gradient_accumulation_steps": "auto",
    "train_batch_size": "auto"
}
```
When resuming from deepspeed checkpoint, learning rate resets:
<img width="655" alt="image" src="https://github.com/huggingface/transformers/assets/60613238/6d9b3043-1b46-4505-bfeb-81eec0736aed">
If I comment out the deepspeed config (which means that Trainer gets rid of DS), the problem resolves:
<img width="654" alt="image" src="https://github.com/huggingface/transformers/assets/60613238/1ca4d189-f58c-4237-9246-6d6d0bfc8f6b">

Is there anything wrong with my deepspeed config?

### Expected behavior

The learning rate shoudn't reset.

Comments:
Solved, we should add the specification for scheduler in the config json for deepspeed as well.
Bug in the code of owlv2 algorithm
Author: zhongwenkun886
Pull request: no

### System Info

python, linux

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

The preprocessing of the image is padding the image to a square image（padding to the right and bottom region）,but it is not considered in post processing，so when the input image is not a square image, the location of the result is wrong.

### Expected behavior

The preprocessing of the image is padding the image to a square image（padding to the right and bottom region）,but it is not considered in post processing，so when the input image is not a square image, the location of the result is wrong.

Comments:
See #27205, which will be fixed by #27698. See also my [demo notebook](https://github.com/NielsRogge/Transformers-Tutorials/blob/master/OWLv2/Zero_and_one_shot_object_detection_with_OWLv2.ipynb), you need to provide the `target_sizes` of the padded image rather than the original one for visualization.

thanks
Add Readme file in Ukrainian language
Author: AnastasiyaKukharska
Pull request: yes

# What does this PR do?

Adding docs in Ukrainian language

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

 @stevhliu 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26947). All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Add function that can get models by prefix
Author: Ujcgjlm
Pull request: yes

A useful feature has been added:
A function can retrieve models with a matching prefix in a particular module

Reduced cyclomatic complexity of the existing function: get_all_auto_configured_models

Comments:
Hey! Feel free to ping me or @ydshieh for a review when this is ready! 

> Hey! Feel free to ping me or @ydshieh for a review when this is ready!

I apologize for not responding. It's done

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Added Typing
Author: Siddhesh-Agarwal
Pull request: yes

# What does this PR do?

- Added Typing
- Fixes #26745 

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@LysandreJik 


Comments:
Thanks for the review. I'll do the necessary changes.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Don't override manual -100 labels with DataCollatorForLanguageModeling
Author: kabachuha
Pull request: yes

# What does this PR do?

The goal of this PR is to prevent the overwriting of all the labels including -100s by input_ids in the process of collating the batches with Transformers's DataCollatorForLanguageModeling.

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #26357

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. https://github.com/huggingface/transformers/issues/26357 also https://github.com/oobabooga/text-generation-webui/issues/4031 and https://github.com/oobabooga/text-generation-webui/pull/4032
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests? (will add to https://github.com/huggingface/transformers/blob/main/tests/trainer/test_data_collator.py if it gains more traction)


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.


Library:

- trainer: @muellerzr and @pacman100


Comments:
Hey! feel free to ping me once this is ready for a review! 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Adding a new model AugViT
Author: ushareng
Pull request: yes
Labels: name: Model on the Hub

### Model description

I would like to contribute AugViT Tensorflow implementation to the Transformers library. You can find the implementation in TensorFlow here https://github.com/ushareng/AugViT

I have created Model card here https://huggingface.co/tensorgirl/TFaugvit/tree/main

Kindly let me know if the above is correct.

### Open source status

- [X] The model implementation is available
- [x] The model weights are available

### Provide useful links for the implementation

TensorFlow implementation of AugViT https://github.com/ushareng/AugViT

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

.

Hey! I think you should share this model [on the hub](https://huggingface.co/docs/transformers/custom_models)! Will be a lot easier to share and a good way to see if the community is riled up for this new model! 

> Hey! I think you should share this model [on the hub](https://huggingface.co/docs/transformers/custom_models)! Will be a lot easier to share and a good way to see if the community is riled up for this new model!

@ArthurZucker It's uploaded to the hub already.  Link: https://huggingface.co/tensorgirl/TFaugvit

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
ValueError: Unexpected result of `train_function` (Empty logs). Please use `Model.compile(..., run_eagerly=True)`, or `tf.config.run_functions_eagerly(True)` for more information of where went wrong, or file a issue/bug to `tf.keras`.
Author: ErHimani
Pull request: no

### System Info

I am trying to train the CLIP model with my custom dataset But I am facing the above issue
my current version of tensorflow ==2.12.0

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

![image](https://github.com/huggingface/transformers/assets/62006705/a8463187-0a43-4caa-9f0f-d90222434ff4)


### Expected behavior

Please provide me the guidance how I can use the code t train on custom dataset

Comments:
Hi @ErHimani, thanks for raising an issue. 

Could you follow the issue template and provide: 
* The running environment: run `transformers-cli env` in the terminal and copy-paste the output
* A minimal code snippet so that was can reproduce the error

Without these, we're unable to help.

hii @amyeroberts I am using the following command:
`python examples\tensorflow\contrastive-image-text\run_clip.py --output_dir  .\clip-roberta-finetuned --vision_model_name_or_path openai/clip-vit-base-patch32 --text_model_name_or_path roberta-base --train_file descriptions.json  --image_column image_path --caption_column text --remove_unused_columns=False --do_train  --per_device_train_batch_size="64" --per_device_eval_batch_size="64"  --learning_rate="5e-5" --warmup_steps="0" --weight_decay 0.1`

@ErHimani Thanks for providing this. In order for us to be able to reproduce, we'll need `descriptions.json`, or an example sample from the dataset to be able to reproduce. We also require the running environment information, as noted above. 

@amyeroberts Please find Link of the custom dataset
description.json:[https://drive.google.com/file/d/14FGJwXRsxns679-ILGlLcBRpqe8UUmGu/view?usp=sharing](url)
Images:[https://drive.google.com/drive/folders/1yr8zapcCPdxlN-5ZSczOIiIeIyS3K_Vt?usp=sharing](url)

@ErHimani Apologies for the delay in getting back to this issue. The links to the datasets are no longer working. Is the issue still persisting with the latest transformers release? If so, would you be willing to re-share the example dataset? 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix Past CI
Author: ydshieh
Pull request: yes

# What does this PR do?

Since mid-November, we have hundreds of following failures in each past CI

> (line 1108)  NotImplementedError: Loading a dataset cached in a LocalFileSystem is not supported.

It is caused by `fsspec==2023.10.0` with an old `datasets`.

This PR just updates `datasets` (at CI runtime) to avoid, and fix thousand failures in total in past CI 🚀 🤣 

Comments:
Update forward signature test for vision models
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR makes sure that vision-only models don't need to overwrite `test_forward_signature`. Instead, `model.main_input_name` is leveraged.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

`main_input_name` is defined in `PreTrainedModel` (and the corresponding TF/Flax counterpart), so

> all (NLP) models have this properly set

is True