noamaanMulla-03/transformers-issues huggingface dataset:
Problem of computing entropy in `run_bertology.py`
Author: CaffreyR
Pull request: no

### System Info

- `transformers` version: 4.20.1
- Platform: macOS-12.4-arm64-arm-64bit
- Python version: 3.9.10
- Huggingface_hub version: 0.8.1
- PyTorch version (GPU?): 1.13.0.dev20220709 (False)
- Tensorflow version (GPU?): 2.8.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@sgugger @thomwolf @stas00 @patrickvonplaten @LysandreJik   Many thanks!

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

In [`run_bertology.py` ](https://github.com/huggingface/transformers/blob/main/examples/research_projects/bertology/run_bertology.py#L108), we use `def entropy` to calculate the `entropy`  of attention matrix. But what if the matrix has negative elements, the `plogp = p * torch.log(p)` will be `nan`

### Expected behavior

`plogp = p * torch.log(p)` will be `nan`

Comments:
Note that this is a research project, so an example provided as is not really maintained :-)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
HFArgumentParser using a mix of json file and command line?
Author: sgunasekar
Pull request: no

Is there a way to make HFArgumentParser to load first from a json/dict and then update any command line arguments on top of it? e.g., if i want to keep a custom defaults file for TrainingArguments but also update some arguments from command line. TIA

Comments:
Please use the [forums](https://discuss.huggingface.co/) to ask questions like this as we keep issues for bugs and feature requests only.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
About the evaluation_loop function of trainer
Author: macheng6
Pull request: no

### Feature request

It is recommended to feed the logits of each batch into the compute_metrics function, and then aggregate the results of each batch.

### Motivation

When I use the evaluate function of the trainer, the evaluation_loop function concatenates all the logits and labels on the validation set and sends them to the compute_metrics function for evaluation. The preds_host and labels_host are of torch.tensor type, so it is easy to exceed the gpu memory.

### Your contribution

pass

Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discuss.huggingface.co) instead?

Thanks!

You can set `eval_accumulation_steps=100`(or even a smaller number) in TraningArgs to avoid GPU memory exceeding.

However, I also think it is a bad design that the default `eval_accumulation_steps`  is infinity actually, also.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Install `tensorflow_probability` for TF pipeline CI
Author: ydshieh
Pull request: yes

# What does this PR do?

So tests like `TQAPipelineTests.test_integration_sqa_tf` or `TQAPipelineTests.test_slow_tokenizer_sqa_tf` could run.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20586). All of your documentation changes will be reflected on that endpoint.
Add `require_torch` to 2 pipeline tests
Author: ydshieh
Pull request: yes

# What does this PR do?

The 2 tests are for `pytorch`, but in TF pipeline test CI job (where `torch` is not available), it runs with TF models.
This is not expected.

Before #20149, these 2 tests are decorated with `require_torch_scatter`. After that PR, the tests try to run with TF, but failed with `TFTapasMainLayer requires the tensorflow_probability library but it was not found in your environment.`
(this is another thing to fix in docker file)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Ci-whisper-asr
Author: ArthurZucker
Pull request: yes

# What does this PR do?
In a recent update, we followed the original code which changed some of the suppress tokens for better performances. This lead to a small change in the output of on particular case. Tested with the original code and we have the correct output now!

Related to #20493 and #20512
See [here](https://huggingface.co/openai/whisper-large/commit/ed97120f929257fb801f99587ada69be0daf5b0a) for the particular commit

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Cannot load TensorFlow model from PyTorch weights split to multiple files
Author: stancld
Pull request: no
Labels: name: WIP

### System Info

- `transformers` version: 4.24.0.dev0
- Platform: Linux-5.15.0-46-generic-x86_64-with-glibc2.29
- Python version: 3.8.10
- Huggingface_hub version: 0.10.1
- PyTorch version (GPU?): 1.13.0+cu117
- Tensorflow version (GPU?): 2.9.2

### Who can help?

@LysandreJik @patrickvonplaten 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```bash
$ git clone https://github.com/stancld/transformers.git -b tf_longt5
$ cd transformers
$ pip install -e .
$ python
```

```python
>>> from transformers import TFLongT5ForConditionalGeneration
>>> m = TFLongT5ForConditionalGeneration.from_pretrained("google/long-t5-tglobal-xl", from_pt=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/transformers/src/transformers/modeling_tf_utils.py", line 2613, in from_pretrained
    raise EnvironmentError(
OSError: google/long-t5-tglobal-xl does not appear to have a file named tf_model.h5 or pytorch_model.bin.
>>> m = TFLongT5ForConditionalGeneration.from_pretrained(MODEL_NAME, from_flax=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/transformers/src/transformers/modeling_tf_utils.py", line 2613, in from_pretrained
    raise EnvironmentError(
OSError: google/long-t5-tglobal-xl does not appear to have a file named tf_model.h5 or pytorch_model.bin.
```

### Expected behavior

Being able to load TensorFlow model from PyTorch checkpoint when split to multiple files due to a large size.

Comments:
No this is not supported yet, we'll work on adding support for this later on :-)

@sgugger Great to hear! 🔝 Feel free then to close this issue if redundant, thanks! :] 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Haven't forgotten, I plan to look into this in December :-)
cross platform from_pretrained
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Allows loading sharded checkpoints in TF models. Should fix #19965
- [x] `from_pt=True`
- [ ] `from_flax=True`

cc @sgugger just FYI 

Comments:
Works great for sharded `pytorch` since a utility was already implemented. Though we are not gonna push for `Flax`, would still help to have the support already! 
```python 
from transformers import TFT5ForConditionalGeneration
MODEL_NAME = "google/flan-t5-xl"
m = TFT5ForConditionalGeneration.from_pretrained(MODEL_NAME, from_pt=True)
```


_The documentation is not available anymore as the PR was closed or merged._

Just need to remove the `# TODOs`
Try replacing tf.int32 with tf.int64 across all tests
Author: Rocketknight1
Pull request: yes

This is a draft PR where I just search-replaced `tf.int32` with `tf.int64` in our tests to check what breaks. I'll probably also need to cast our dummy inputs correctly to make this work, at least!

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_19489). All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Bumping this to keep it open - a great int dtype purge is still on my list, but I got sidetracked with some other high-priority stuff!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Closing this because I think (I hope) that our dtype issues are mostly resolved by now
ESM openfold_utils type hints
Author: ringohoffman
Pull request: yes

# What does this PR do?

This PR generally adds type hints for the files located at `src/transformers/models/esm/openfold_utils/`.

0. add function/method parameter type hints where missing; add type info on collections
1. export `dict_multimap`, `flatten_final_dims`, `permute_final_dims` in `__init__.py` since these functions are currently duplicated in [src/transformers/models/esm/modeling_esmfold.py](https://github.com/huggingface/transformers/blob/2e17db8a8626baeea7efd6f2700be863f026699c/src/transformers/models/esm/modeling_esmfold.py#L218-L238); exporting these from `openfold_utils` should allow us to remove these duplicates
2. refactor `type(x) is y` to use the builtin `isinstance(x, y)`
3. refactor to avoid reassignment to the same variable with a different type (this is frowned upon by type checkers) by using multiple variables / combining expressions to avoid reassignment
4. add `assert` statements to to narrow types
5. add a `FIXME` statement at an apparent bug in [`protein.py`](https://github.com/huggingface/transformers/pull/20544/files#diff-b7388405b8a9b1877a3eeb6b6941091f68e321717beec3abb7727cd3114115bfR84) in which string mutation is attempted
6. various minor refactors

<!-- Remove if not applicable -->

Related: https://github.com/huggingface/transformers/issues/16059

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?

@Rocketknight1 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Wow, this is really comprehensive! All of your edits seem good, and thanks for catching those duplicate functions!

The code is failing some of our code style checks, but I believe I can fix that for you, hang on!

I think the other issues are just old issues with our repo - they'll be fixed if you pull from upstream on your fork's `main` branch in the GitHub UI and then rebase your branch onto that, followed by a force push

@ringohoffman Looks good to me now! Are you okay with me merging it?

> @ringohoffman Looks good to me now! Are you okay with me merging it?

I'm good if you are!
Remove unused `classifier_dropout` in configs
Author: ydshieh
Pull request: yes

# What does this PR do?

Similar to #20554, but this time for `classifier_dropout`.
The existing checkpoints with this attribute in their config files could still be loaded via the `**kwargs` --> so won't fail.


@sgugger If you would prefer me to cleanup multiple different unused config attributes in a single PR, let me know 😉 



Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix whisper and speech to text doc 
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Previously the documentation was badly indented for both models and indicated that
> If `decoder_input_ids` and `decoder_inputs_embeds` are both unset, `decoder_inputs_embeds` takes the value of `inputs_embeds`.`
Which is on valid for the forward pass of the `ForConditionnalGeneration` not for the model alone. 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Replace `set-output` by `$GITHUB_OUTPUT`
Author: ydshieh
Pull request: yes

# What does this PR do?

Apply the suggestion in [GitHub Actions: Deprecating save-state and set-output commands](https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/) to avoid deprecated actions.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[Vision] `.to` function for ImageProcessors
Author: younesbelkada
Pull request: yes

# What does this PR do?

PoC for adding `.to` support on ImageProcessors

related #20453


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Here is a quick v1, but I am afraid it's a bit too much in the sense that I am literally testing every possible combination 
Also regarding tests, we can remove them or put them as slow. I checked with `deit`, `vit` & `vilt` (for multimodal setup) and the tests are green (the failing test for LayoutLM can be easily fixed)
May I ask you to have a quick look @sgugger @ydshieh ? Thanks!

You're looking at something too complicated: `to()` does all that work for you already. You can pass it a string, a device or a dtype.

Yes I was thinking of something very complicated where someone could set `.to(device, dtype)` let's maybe keep it even simpler and force the user to put only a single argument in `.to` ?

EDIT: it seems that there is a workaround for that

Thanks everyone for the feedback! Let me know if you think it's relevant to add the `test_cast_dtype` for all ImageProcessors as it may slow down our CI testing suite

Ahaha no worries! thanks for all the iterations 💪
Add-whisper-conversion
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Add the conversion script from whisper which was deleted during the sprint. See this [commit](https://github.com/huggingface/transformers/pull/19166/commits/f92b9a8181f9a84114becd31a5a4210723cdf1ad).
This will help for the Whisper Event! 

Comments:
See the new checkpoints : https://huggingface.co/openai/whisper-large-v2 

_The documentation is not available anymore as the PR was closed or merged._

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20600). All of your documentation changes will be reflected on that endpoint.
fix plbart doctest
Author: ydshieh
Pull request: yes

# What does this PR do?

We need to update the expect output in doc example after PR #19980

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@ydshieh @sgugger I just want to know how this PR works and why was the doctests failing earlier? Thanks in advance!

As the PR description mentioned, PR #19980 changed `PLBartTokenizer`, and some expected outputs in the tests have to be updated.

@ydshieh got it. Thanks!
fail to import  'microsoft/swin-tiny-patch4-window7-224' in AutoConfig
Author: XZhang97666
Pull request: no

### System Info

OSError: microsoft/swin-tiny-patch4-window7-224 does not appear to have a file named config.json. Checkout 'https://huggingface.co/microsoft/swin-tiny-patch4-window7-224/None' for available files

### Who can help?

@LysandreJik

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

NA

### Expected behavior

import 'microsoft/swin-tiny-patch4-window7-224' successful

Comments:
Hi, @XZhang97666. It seems that config.json file is in the right place(https://huggingface.co/microsoft/swin-tiny-patch4-window7-224/blob/main/config.json). 
Could you please tell what version of `transformers` you're using?
Also have you tried to run the snippet code that at https://huggingface.co/microsoft/swin-tiny-patch4-window7-224?

> Hi, @XZhang97666. It seems that config.json file is in the right place(https://huggingface.co/microsoft/swin-tiny-patch4-window7-224/blob/main/config.json). Could you please tell what version of `transformers` you're using? Also have you tried to run the snippet code that at https://huggingface.co/microsoft/swin-tiny-patch4-window7-224?

Yes. I tried https://huggingface.co/microsoft/swin-tiny-patch4-window7-224, which works. However, another task does not even with the same transformers version (4.22.2).

> Hi, @XZhang97666. It seems that config.json file is in the right place(https://huggingface.co/microsoft/swin-tiny-patch4-window7-224/blob/main/config.json). Could you please tell what version of `transformers` you're using? Also have you tried to run the snippet code that at https://huggingface.co/microsoft/swin-tiny-patch4-window7-224?

I found the issue. My task generate a local folder called "microsoft/..."

Closing this issue as it seems resolved, feel free to reopen.
Documentation fixes
Author: samuelzxu
Pull request: yes

# What does this PR do?

This PR just fixes some typos in the documentation.

Please note: Apart from the typos in the *paragraphs*, the other changes were because of significantly differing results I got from running the examples. For instance, "aweful" didn't result in a high negative score, but "awful" did.

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@sgugger 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Clean up hub
Author: sgugger
Pull request: yes

# What does this PR do?

This PR removes all uses of the old utils in Transformers to migrate to the new version using `cached_file`, and remove said utils from the Transformers library.
It is slightly breaking since we remove objects (in particular `cached_path` is in the main init albeit not documented), but those are all internal tools, so it's okay in my opinion. Only the research examples are left as before.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

I still had an error mentioning `ImportError: cannot import name 'cached_path' from 'transformers.utils' `
@ `tf.__version__ == '2.11.0'`. Is this related? What should i do?

You should stop using that function, as it has been removed from `transformers.utils` in the most recent versions.
AttributeError: 'NoneType' object has no attribute 'format'
Author: xxyzz
Pull request: no

### System Info

transformers version: 4.21.3
OS: Windows 10
Python version: 3.10 

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

-  Install one of spaCy's transformer model.
```
$ python -m pip install spacy[cuda-autodetect]
$ python -m spacy download en_core_web_trf
```

- Set `PYTHONOPTIMIZE` to 2 or use `-OO` option.

-  Load spaCy model:

```python
import spacy

spacy.load("en_core_web_trf")
```

- Get error similar to this:

```
File "C:\x\spacy\__init__.py", line 54, in load
  File "C:\x\spacy\util.py", line 432, in load_model
  File "C:\x\spacy\util.py", line 468, in load_model_from_package
  File "C:\x\en_core_web_lg\__init__.py", line 10, in load
  File "C:\x\spacy\util.py", line 649, in load_model_from_init_py
  File "C:\x\spacy\util.py", line 506, in load_model_from_path
  File "C:\x\spacy\util.py", line 554, in load_model_from_config
  File "C:\x\spacy\language.py", line 1788, in from_config
  File "C:\x\spacy\language.py", line 163, in __init__
  File "C:\x\catalogue\__init__.py", line 119, in get_all
  File "C:\x\catalogue\__init__.py", line 134, in get_entry_points
  File "importlib\metadata\__init__.py", line 162, in load
  File "importlib\__init__.py", line 126, in import_module
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "C:\x\spacy_transformers\__init__.py", line 1, in <module>
  File "C:\x\spacy_transformers\architectures.py", line 6, in <module>
  File "C:\x\spacy_transformers\layers\__init__.py", line 1, in <module>
  File "C:\x\spacy_transformers\layers\listener.py", line 4, in <module>
  File "C:\x\spacy_transformers\data_classes.py", line 5, in <module>
  File "C:\x\transformers\tokenization_utils.py", line 26, in <module>
  File "C:\x\transformers\tokenization_utils_base.py", line 3646, in <module>
AttributeError: 'NoneType' object has no attribute 'format'
```

This could happen when running the code in a Python interpreter which has optimize level set to 2 and it can't be changed, for exmaple: [calibre](https://calibre-ebook.com).

### Expected behavior

No error.

Comments:
updating T5 and BART models to support Prefix Tuning
Author: pacman100
Pull request: yes

# What does this PR do?
1. updating T5 and BART models to support Prefix Tuning. Currently, passing `past_key_value` fails. This PR fixes it. Doesn't impact any current functionality. 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hey, just for reference could you provide a link to an issue or something explaining what `prefix tuning` is?