noamaanMulla-03/transformers-issues huggingface dataset:
Fix error in mixed precision training of `TFCvtModel`
Author: gcuder
Pull request: yes

# What does this PR do?

This PR fixes the issue that the `TFCvtModel` cannot be trained with `keras.fit` using `mixed-precision`. 
The issue was in this [line](https://github.com/huggingface/transformers/blob/c4bf6f38bda1de3798095515875a119298bf0611/src/transformers/models/cvt/modeling_tf_cvt.py#L96) when a random tensor is initialized without specifying the correct `dtype`.  


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.
@amyeroberts

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Time to Say Goodbye, torch 1.7 and 1.8
Author: ydshieh
Pull request: yes

# What does this PR do?

We have been together for more than 2 years ❤️ 

(see this [discussion](https://github.com/huggingface/transformers/issues/18817))

Comments:
Thank you @sgugger . I will try to make a clean breakup

_The documentation is not available anymore as the PR was closed or merged._

Hope I don't miss anything 

ok, the deepspeed CI is running pt-1.8 - how do we solve that then?

I have passed this change to the Deepspeed team let's see what they say.

edit: they followed suit https://github.com/microsoft/DeepSpeed/pull/3082
[gptj] support older pytorch version
Author: stas00
Pull request: yes

Unbreak 2 issues introduced by https://github.com/huggingface/transformers/pull/22069 . I validated that this version works even with pt-1.9, which is the new lowest version supported by transformers since https://github.com/huggingface/transformers/pull/22291

Fixes:



```
E             File "/mnt/nvme0/code/huggingface/transformers-master/examples/pytorch/language-modeling/run_clm.py", line 412, in main
E               model = AutoModelForCausalLM.from_pretrained(
E             File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/models/auto/auto_factory.py", line 470, in from_pretrained
E               model_class = _get_model_class(config, cls._model_mapping)
E             File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/models/auto/auto_factory.py", line 360, in _get_model_class
E               supported_models = model_mapping[type(config)]
E             File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/models/auto/auto_factory.py", line 602, in __getitem__
E               return self._load_attr_from_module(model_type, model_name)
E             File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/models/auto/auto_factory.py", line 616, in _load_attr_from_module
E               return getattribute_from_module(self._modules[module_name], attr)
E             File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/models/auto/auto_factory.py", line 561, in getattribute_from_module
E               if hasattr(module, attr):
E             File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/utils/import_utils.py", line 1109, in __getattr__
E               module = self._get_module(self._class_to_module[name])
E             File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/utils/import_utils.py", line 1121, in _get_module
E               raise RuntimeError(
E           RuntimeError: Failed to import transformers.models.gptj.modeling_gptj because of the following error (look up to see its traceback):
E           module 'torch' has no attribute 'fx'

```

credits for the fix: @mrwyattii

and:

```
E             File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/models/gptj/modeling_gptj.py", line 61, in create_sinusoidal_positions
E               return torch.concat((torch.sin(sinusoid_inp), torch.cos(sinusoid_inp)), dim=1)
E           AttributeError: module 'torch' has no attribute 'concat'
```

credits for the fix: @njhill

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thank you for the quick review, Sylvain.

Thank you @stas00 @sgugger and sorry again for the breakage.

You haven't done anything wrong, Nick. It's just very difficult to instantly test all the different variations. We have more indepth multi-version CI running on a daily basis, so usually any missed problems get detected on the next day.

And thank you for your contribution!
[`MBart`] Add `accelerate` support for MBart
Author: younesbelkada
Pull request: yes

# What does this PR do?

Partially fixes: #22305 

This PR adds  `accelerate` support for `MBart` models.
To run the `accelerate` tests: 

```bash
RUN_SLOW=1 pytest -m accelerate_tests tests/models/mbart/test_modeling_mbart.py
```
A fix similar to https://github.com/huggingface/transformers/pull/19927 needs to be applied in order for the tests to pass

cc @amyeroberts 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[`bnb`] Fix bnb slow test
Author: younesbelkada
Pull request: yes

# What does this PR do?

Fixes a failing daily CI test. https://github.com/huggingface/transformers/actions/runs/4485738346/jobs/7887641218

Which can be simplified as:
```python
from transformers import AutoModelForCausalLM

model_name = "bigscience/bloom-560m"

memory_mapping = {0: "1GB", 1: "2GB"}
model_parallel = AutoModelForCausalLM.from_pretrained(
    model_name, load_in_8bit=True, max_memory=memory_mapping, device_map="auto"
)

# Check correct device map
print(set(model_parallel.hf_device_map.values()))
>>> EXPECTED={0, 1} / GOT={0}
```

I think this also fixes a bug (which is not negligeable)

It seems that we need to add a check `max_memory is None` otherwise `max_memory` will get overriden right after by
```python
max_memory = get_balanced_memory(
                    model,
                    dtype=torch_dtype,
                    low_zero=(device_map == "balanced_low_0"),
                    **kwargs,
                )
```
Hence `max_memory` seems to be ignored now on the `main` branch in some cases

cc @sgugger @ydshieh 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Seems to be fixed on `main`, probably thanks to https://github.com/huggingface/transformers/pull/22311
Generate: Export TF generate with a TF tokenizer
Author: gante
Pull request: yes

# What does this PR do?

See #22254 

As the title says, this PR adds the possibility to export TF generate with a TF-native tokenizer -- the full thing in a single TF graph 🤯 

The missing piece was removing a redundant `if` before `tf.while_cond` -- `tf.while_cond` checks the condition before running the body, so the existing `if` before it was redundant. It was also the root cause behind the error in #22254, so removing it was a double win 🎉 

A test was added to ensure we don't regress.

Comments:
@Rocketknight1 no need to review, but FYI -- we can now compile the whole thing into a graph :D

_The documentation is not available anymore as the PR was closed or merged._

This is amazing!
Fixed gradient checkpoint bug for TimeSeriesTransformer
Author: pmollerus23
Pull request: yes

# What does this PR do?

Moved gradient checkpointing clause to above the decoder layer implementation. This should fix the bug this issue addresses.

Fixes #21737 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
[GitHub Issue](https://github.com/huggingface/transformers/pull/21733)
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@younesbelkada 



Comments:
_The documentation is not available anymore as the PR was closed or merged._

Strange, I will see if I can install Python for my environment then. Thank you for all your help, and thanks for running make for me!
Mention why one needs to specify max_steps in Trainer
Author: lhoestq
Pull request: yes

Just a minor change following https://discuss.huggingface.co/t/streaming-dataset-into-trainer-does-not-implement-len-max-steps-has-to-be-specified/32893 about why max_steps is needed for iterable datasets

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Huge JSON file causes error in run_summarization.py
Author: AtheerAlgherairy
Pull request: no

I tried to run transformers\examples\pytorch\summarization\run_summarization.py with my own data files (in JSON Lines format).  The problem arises only when using huge JSON file (> 20 GB). With smaller size files it works normally. 

Any suggestions how to use the code with large size JSON files?

**Framework:**
 Transformers 4.20.1
 Pytorch 1.11.0+cu113
 Datasets 2.3.2
 Tokenizers 0.12.1

Comments:
Hi @AtheerAlgherairy This is not a `transformers` issue, and it's very likely some memory issue due to the huge file size. [Hugging Face Forum](https://discuss.huggingface.co/) is the place for such kind of questions.

Without going into the specific detail, for large dataset like this, you should try to use iterator to avoid loading the whole file into the memory from the beginning.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Region error
Author: Aniketsingh12
Pull request: yes

Endregion was not causing error as region was not recognizing.


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_21593). All of your documentation changes will be reflected on that endpoint.

Thanks for suggestion 

Yes, if you could replace `# Metrics` with `# region Metrics` instead that would be great!

Alternatively I could just ditch all of these region tags, since I think I'm the only person who likes them.

@Rocketknight1 I will definitely replace all the metrics with region metrics but any reasons of doing that ?? I am a Beginner so just wanted to understand .

@Aniketsingh12 Using region tags allows IDEs like PyCharm and VS Code ([with an addon](https://marketplace.visualstudio.com/items?itemName=maptz.regionfolder)) to fold the areas of code inside the `# region` and `# endregion`, which can make the code easier to read in a long script. They don't serve any function other than that!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Minor typo in pipeline FillMaskPipeline's documentation.
Author: SamuelLarkin
Pull request: yes

# What does this PR do?

Fixes a minor typo in the documentation for FillMaskPipeline.__call__().

## Before submitting
- [X ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?
@Narsil. @sgugger, @stevhliu and @MKhalusova

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thank you for spotting and fixing this!
a bug in transformers.GenerationMixin
Author: lixinliu1995
Pull request: no

### System Info

transformers 4.27.0. In the greedy_search method of the GenerationMixin class, the unfinished_sequences variable at line 2177 should be placed inside the body of the 'while' loop so that it gets longer and longer with the decoding, Otherwise it is always a value rather than a sequence.
@gante @gante 

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

while True:
     unfinished_sequences = input_ids.new(input_ids.shape[0]).fill_(1)
     if synced_gpus:
         pass 
         ba la ba la ...
   


### Expected behavior

Fix as soon as possible

Comments:
Hey @lixinliu1995 👋  I'm not sure I follow, `unfinished_sequences` holds a boolean for each sentence and is updated each iteration on L2247 (if a sentence contains any of the eos tokens -> set to false)

How would you expect it to behave?

yeah, you are right. It is my mistake.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Added type hints to TFDeiTModel
Author: Batese2001
Pull request: yes

# What does this PR do?

This pull request adds type hints for modeling_tf_deit.py as outlined in Issue #16059

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@Rocketknight1 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Generate: add test for left-padding support
Author: gante
Pull request: yes

# What does this PR do?

This PR adds a test to check whether a decoder-only model supports left padding. The test was somewhat tricky to design -- the reasoning is all commented in the code, for future reference.

Let me know if you agree with the decisions made in the test!

Hopefully we will be able to detect whether a model supports left-padding before merging 🤞 

Comments:
Current failures -- including Llama and GPTNeoX, the two I had my eyes on:
![Screenshot 2023-03-22 at 18 42 57](https://user-images.githubusercontent.com/12240844/227005881-868067e5-a083-4e87-9858-020694869843.png)


_The documentation is not available anymore as the PR was closed or merged._

@sgugger reduced to 10 runs, merging when CI gets green :)
Link for Absent Longformer Task Documentation
Author: mert-kurttutan
Pull request: no

Hi,

I tried to look at the example tasks for Longformer. It lead to a page that does not exist and non-ideal response page.


How to reproduce:

1) Go to https://huggingface.co/docs/transformers/model_doc/longformer#documentation-resources
2) Click on any of the documentation resources, and arrive at the next page.

Maybe better notification saying these documentations dont exist yet would be better 

I guess this is more like a matter of how you present absent web pages.
![error_page](https://user-images.githubusercontent.com/88637659/227191372-4ce460bd-27f4-42de-9556-7b0b660f0d5b.png)
![error_page2](https://user-images.githubusercontent.com/88637659/227191418-b528ba58-b07b-4356-8f31-a15bb289f7b8.png)


Comments:
Transferred to transformers. As far as I can see, this is fixed in main and will work as intended in the next release. https://huggingface.co/docs/transformers/main/en/model_doc/longformer

Oh. Ok. Thanks.
It still seems weird that resources of this page are about generic task demonstration, not particularly related to Longformer
I guess this is a matter of design choice.
Feel free to close  this issue
[New GitHub Action Job] Automatically create/update tiny models
Author: ydshieh
Pull request: yes

# What does this PR do?

[New GitHub Action Job] Automatically create/update tiny models

## Goal

**A scheduled job that create/update tiny models periodically**
  - so **we will have tiny versions for newly added models in `transformers` as soon as possible**

### Some properties
  - For a new model type: The Hub repo. is created
  - For a new framework implementation of an existing model type: A Hub repo. PR is opened
  - We keep track of the commit hash information for tiny models on the Hub
    - The pipeline tests will use the commit hash information stored in `tiny_model_summary.json` file
      - To avoid sudden CI failures due to new commits
    - The CI job will produce a file `updated_tiny_model_summary.json`
      - We should open a PR in `transformers` to update `tiny_model_summary.json`
        - If all pipeline tests pass, we are good to merge and use the new/updated tiny models on the Hub.  

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Just a minor update, including the revision number. Will merge once the CI being green. Thank you for the reviews 🚀
[HFTracer] Make embeddings ops take on the dtype of the weight
Author: jamesr66a
Pull request: yes

# What does this PR do?

Previously, `HFTracer` would assume a dtype of `torch.float32` as the output for `embedding` operators. This would cause issues downstream if you're tracing out a model that is initialized as e.g. `torch.bfloat16`. This makes it so that the embeddings ops outputs take on the dtype of the weight tensor

Comments:
cc @michaelbenayoun 

_The documentation is not available anymore as the PR was closed or merged._
Typo in Greedy Search Description
Author: awinml
Pull request: no

### System Info

- `transformers` version: 4.28.0.dev0
- Platform: Linux-5.15.0-58-generic-x86_64-with-glibc2.29
- Python version: 3.8.10
- Huggingface_hub version: 0.11.1
- PyTorch version (GPU?): 1.13.1+cu117 (False)
- Tensorflow version (GPU?): 2.11.0 (False)
- Flax version (CPU?/GPU?/TPU?): 0.5.0 (cpu)
- Jax version: 0.3.6
- JaxLib version: 0.3.5
- Using GPU in script?: False
- Using distributed or parallel set-up in script?: False

### Who can help?

@sgugger

### Reproduction

There is a small typographical error in the documentation for Greedy Search.

https://github.com/huggingface/transformers/blob/ff20f9cf3615a8638023bc82925573cb9d0f3560/docs/source/en/generation_strategies.mdx?plain=1#L149-L152


### Proposed Solution

This could be fixed by just rewriting this to:
```python
[`generate`] uses greedy search decoding by default so you don't have to pass any parameters to enable it. This means the parameters `num_beams` is set to 1 and `do_sample=False`.
```

Comments:
Sure, would you like to suggest a PR?

Yeah, I can open a PR to fix this.
Fix typo in Greedy Search Description
Author: awinml
Pull request: yes

# What does this PR do?
There is a small typographical error in the documentation for Greedy Search.
Fixes #22335 


- [x] This PR fixes a typo.



## Who can review?

@sgugger 




Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sgugger I rebased with main. All the CI checks pass now.

Thanks!
Update docker files to use official torch 2.0.0
Author: ydshieh
Pull request: yes

# What does this PR do?

(basically just revert to what we have before #22135, except the torch and cuda version numbers) 

We used
```python
--index-url https://download.pytorch.org/whl/test/cu117
```
to run CI before `torch 2.0.0` release. Now since the official release it out, let's use
```python
--index-url https://download.pytorch.org/whl/cu117
```


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Re enable Nightly CI for upcoming PyTorch 1.13
Author: ydshieh
Pull request: yes

# What does this PR do?

Re enable Nightly CI for upcoming PyTorch 1.13.

This is the minimal changes. We might need to check if the docker image could be built with these versions.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Sorry, I forgot one docker image. Change PR to draft.

@LysandreJik I think we don't need to merge this PR. I could just build images and run the tests.
[WHISPER] Unreliable timestamp with whisper for videos under 30 seconds
Author: altryne
Pull request: no

### System Info

Hey, I noticed that there's an unreliable timestamp thing happening which whisper through transformers that doesn't show up in original whisper. 

In this example: 
https://targum.video/v/47160791e0e305ff7f22e84203f1b196

The "people on streches.." was said at the end of the video, but the timestamp was placed around 8 seconds in. 

Here's the same video translated with large-2 whisper from OpenAI

https://targum.video/v/8c5e21ff6da8947c02cdb40097eadf50

### Who can help?

@ArthurZucker 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Can send the exact link where this is happens over DM, but basically the video from this tweet: 

https://twitter.com/wxkaitlin/status/1625326828264071168

Is getting the wrong subtitles (some are missing) + a wrong timestamp

### Expected behavior

Expected the transformers whisper to behave exactly like (but faster) the openAI whisper

Comments:
Can you provide a reproduction script to make sure we are running with the same parameters? 😉 
Also this might ring some bels to @Narsil. 
I know we interacted before, but just want to make sure which transformers version you are using and which calls. Then I'll be able to dig ! Thanks for the issue 😉 


@altryne @ArthurZucker .

While deep diving into whisper, I've notived `openai/whisper` uses timestamp ALL the time, while `transformers` doesn't (you have to ask for timestamps for us to use them).

I have seen BIG discrepancies on some examples, I am guessing because training was somehow biased with timestamps for whisper. 

Could that be it ?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.