noamaanMulla-03/transformers-issues huggingface dataset:
Tutorial on token classification throws casting error in Tensorflow 2.11
Author: gcelano
Pull request: no

### System Info

- `transformers` version: 4.25.1
- Platform: Linux-5.15.0-56-generic-x86_64-with-glibc2.35
- Python version: 3.9.0
- Huggingface_hub version: 0.11.1
- PyTorch version (GPU?): not installed (NA)
- Tensorflow version (GPU?): 2.11.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@ArthurZucker @younesbelkada, the tutorial at `https://huggingface.co/docs/transformers/tasks/token_classification` throws the following  error in Tensorflow 2.11 but not in Tensorflow 2.9: 

`(0) UNIMPLEMENTED:  Cast string to float is not supported
	 [[{{node Cast_1}}]]
  (1) CANCELLED:  Function was cancelled before it was started
0 successful operations.
`

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

The tutorial at `https://huggingface.co/docs/transformers/tasks/token_classification` for Tensorflow

### Expected behavior

Training should start, but it does not.

Comments:
Can you please specify where is the error happen? Which step?

The error is thrown after `model.fit`

Okay I'll have a look

Also cc @Rocketknight1 

Yeah, I should probably take this one. Investigating!

Managed to reproduce it.

This is actually a problem with our `AdamWeightDecay`, likely caused by the change in Keras optimizers in 2.11. Figuring out a fix now.
Summarization Pipeline not outputting both text and token_ids
Author: mpeadtt
Pull request: no

### System Info

- Python version: 3.9.15
- Huggingface_hub version: 0.10.1
- PyTorch version (GPU?): 1.12.1 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help?

@Narsil

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```python
from transformers import pipeline

summariser = pipeline("summarization", model="sshleifer/distilbart-cnn-12-6")
output = summariser("this is a test input", return_text=True, return_tensors=True)
print(output)
```
Returns (ignoring length warning message):
```
[{'summary_token_ids': tensor([   2,    0,   42,   16,   10, 1296, 8135, 8135,   31,    5, 2730,    9,
            42, 1566,  479,  152,   16,   45,    5,   78,   86,   52,  348,  450,
            42, 1905,   11,   10, 1296,  422,   30,    5, 2730,    4,   42,   16,
             5,   78,    9,   63,  761,    4,  152,   16,    5,  200,    9,   10,
           651,    9, 1296, 1237,   30,    5, 7601,    9,    5, 1040,  479,    2])}]
```


### Expected behavior

According to the `SummarizationPipeline` [documentation](https://huggingface.co/docs/transformers/main_classes/pipelines#transformers.SummarizationPipeline.__call__) I would expect a list of dictionaries where each dictionary has both `summary_text` and `summary_token_ids` elements. Instead, only one is returned, even though both `return_text=True` and  `return_tensors=True` in the call to the summariser.

I have done a little digging and think it's coming from the way these arguments are handled. In the `_sanitize_parameters` method [here](https://github.com/huggingface/transformers/blob/30d8919ab13dcc212cb00fbb4d3e969aee6c3fc5/src/transformers/pipelines/text2text_generation.py#L73), the `postprocess_params["return_type"]` gets set to either ` ReturnType.TENSORS` or ` ReturnType.TEXT` (or a different `return_type` specified as an input arg).

 The `postprocess` method of the `Text2TextGenerationPipeline` [here](https://github.com/huggingface/transformers/blob/30d8919ab13dcc212cb00fbb4d3e969aee6c3fc5/src/transformers/pipelines/text2text_generation.py#L195) then takes the `return_type` arg, but can only ouput either the `summary_text` or `summary_token_ids`, and not both.

I could have a go at raising a PR for this if you'd like.

Many thanks.




Comments:
Realised that this has been addressed in another issue, apologies.
Add docs xlm roberta
Author: hazrulakmal
Pull request: yes

# What does this PR do?

Fixes  [20055](https://github.com/huggingface/transformers/issues/20055)

I have opened a fresh PR to add resources for XLM-RoBERTa. 
1. Added a link to classification task  fine-tuning using Habana Gaudi blog 
2. Fix typos
3. Remove TF and Flax XLMRobertaForCausalLM 
4. Update my branch to reflect the latest main branch

Thanks for your help reviewing the changes! @stevhliu 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Uninstall `torch_tensorrt` in `DeepSpeed` CI image for now
Author: ydshieh
Pull request: yes

# What does this PR do?

Since we update CI to use PyTorch 1.13, it uses a base image `nvcr.io/nvidia/pytorch:22.04-py3` which contains `torch_tensorrt`.
This causes the CI failing from the test collection - i.e. the whole test suite fails from the beginning.

This PR uninstalls `torch_tensorrt` for now (previously, this is not installed for DeepSpeed CI - the stable release version), so at least the CI could run (and to see if any test would fail with PyTorch 1.13).

We will have to work on the `torch_tensorrt` issue though.

#### Current error message

```bash
==================================== ERRORS ====================================
______________ ERROR collecting tests/deepspeed/test_deepspeed.py ______________
ImportError while importing test module '/workspace/transformers/tests/deepspeed/test_deepspeed.py'.
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[Tests] Improve test_attention_outputs
Author: NielsRogge
Pull request: yes

# What does this PR do?

I noticed several vision models overwrite `test_attention_outputs`, but we have a `has_attentions` flag exactly for this purpose.

Hence, I've adapted the test in `test_modeling_common.py` and `test_modeling_tf_common.p`y to improve this.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix attribute error problem 
Author: fjklqq
Pull request: yes

When the `use_legacy_prediction_loop` parameter is used , `Trainer.predict` will raise an error `AttributeError: 'PredictionOutput' object has no attribute 'num_samples'`

# What does this PR do?
1. Use `EvalLoopOutput` instead of original `PredictionOutput` of `self.prediction_loop`, this makes the return formats of `self.prediction_loop` same as `self.evaluation_loop`.



## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts and @NielsRogge
- speech models: @sanchit-gandhi

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger and @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Remove image_transforms functions from init
Author: amyeroberts
Pull request: yes

# What does this PR do?

Removes functions in the `image_transforms` library from `src/transformers/_init__.py`. 

The functionality this module provides are utils for processing images and are not the primary goal of the library.  

Address an issue raised in #20627 - which highlighted some functions were in the init and others weren't. 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[CI-Test] Fixes but also skips the mT5 tests
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Silences the `MT5` tests. They are irrelevant as the model behind is T5, which function well. 
I want to delete everything but prefer to ask if there is a reason why we have these? 
cc @sgugger 

Comments:
My opinion is that we should have test file(s) for each model type we have. MT5 has its own modeling file and model type, so we should keep it.


_The documentation is not available anymore as the PR was closed or merged._

mT5 does not really have its own modeling file since it's just T5. I'm happy if the tests for that model are contained to a slow integration test to not bloat the CI, like what is done for similar models (Camembert for instance).

I am OK with the decision, but just want to point out that we will lose these models for tiny model creation, which is required for ONNX testing and pipeline testing (in the future). I just want every party involved agree this.

cc @LysandreJik, @Narsil  and @lewtun 

Ok with the decision as well!
layoutxlm model can not convert to onnx
Author: githublsk
Pull request: no

### Model description

I use layoutxlm to training my data for downstream, when I convert the model which I trained to onnx by huggingface code layoutlmv2-to-onnx, it occurs below problem, can you give me some advice? it seems concat two different types causing this problem, but I do not modify any code just using run xfun for token classification, which confused me so much, I hope you can help us


### Open source status

- [ ] The model implementation is available
- [ ] The model weights are available

### Provide useful links for the implementation

I use layoutxlm to training my data for downstream, when I convert the model which I trained to onnx by huggingface code layoutlmv2-to-onnx, it occurs below problem, can you give me some advice? it seems concat two different types causing this problem, but I do not modify any code just using run xfun for token classification, which confused me so much, I hope you can help us
<img width="943" alt="企业微信截图_1656080658142" src="https://user-images.githubusercontent.com/77612906/175556272-af5e91d0-c76e-483f-ad24-95fa7690146e.png">


Comments:
cc @lewtun

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

@lewtun  friendly ping!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Add SpA-Former
Author: kanpuriyanawab
Pull request: yes

# What does this PR do?

This PR adds the SpA-Former model to the 🤗 repository.

I also opened an Issue for adding the model https://github.com/huggingface/transformers/issues/19971

# Who can review?

@NielsRogge

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hi @shivance do you want to proceed with this?
Install torch-tensorrt 1.3.0
Author: ydshieh
Pull request: yes

# What does this PR do?

It turns out that the issue mentioned in #20758 could be fixed by installing newer version of `torch-tensorrt (1.3.0)` (the pre-installed one in the base image was `1.1.0a0`).

Notice that before our CI using torch 1.13.0, we didn't have `torch-tensorrt` installed in the docker image (for daily CI with stable release of torch/deepspeed). But guess we can have it anyway.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Run 'GPT-J' failure due to download dataset fail  (' ConnectionError: Couldn't reach http://eaidata.bmk.sh/data/enron_emails.jsonl.zst ' )
Author: shaoyuta
Pull request: no

### System Info

Copy-and-paste the text below in your GitHub issue.

- huggingface_hub version: 0.11.1
- Platform: Linux-5.15.0-52-generic-x86_64-with-glibc2.35
- Python version: 3.9.12
- Running in iPython ?: No
- Running in notebook ?: No
- Running in Google Colab ?: No
- Token path ?: /home/taosy/.huggingface/token
- Has saved token ?: False
- Configured git credential helpers:
- FastAI: N/A
- Tensorflow: N/A
- Torch: N/A
- Jinja2: N/A
- Graphviz: N/A
- Pydot: N/A


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Steps to reproduce this issue:
1. git clone https://github.com/huggingface/transformers
2. cd transformers
3. python examples/pytorch/language-modeling/run_clm.py --model_name_or_path EleutherAI/gpt-j-6B --dataset_name the_pile --dataset_config_name enron_emails  --do_eval --output_dir /tmp/output --overwrite_output_dir



### Expected behavior

1. This issue looks like due to "http://eaidata.bmk.sh/data/enron_emails.jsonl.zst " couldn't be reached.
2. Is there another way to download the dataset "the_pile" ? 
3. Is there another way to cache the dataset "the_pile" but not let the hg to download it when runtime ?

Comments:
This is an issue with the dataset, so you should probably open this in the Datasets repo :-)

> This is an issue with the dataset, so you should probably open this in the Datasets repo :-)

Thanks for your help
Patch for FlanT5-XXL 8bit support
Author: larsmennen
Pull request: yes

# What does this PR do?

Fixes #20287 .

In #20287 , 3 patches were proposed here: https://github.com/huggingface/transformers/issues/20287#issuecomment-1342219429

* Patch 3 is already covered by https://github.com/huggingface/transformers/pull/20683
* I found patch 2 is actually unnecessary, because there's already a cast to float16 here: https://github.com/younesbelkada/transformers/blob/68a894a5875bfd958b8254afd3bbb23db9c2e813/src/transformers/models/t5/modeling_t5.py#L258-L260 which also applies in this case as we keep `self.wo` in `float32`.
* This PR contains the patch 1, adjusted so it only applies a cast if the `hidden_states` actually has a different `dtype` from the `wo` weights.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [n/a] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

@younesbelkada @sgugger 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

> Thanks so much for the fix @larsmennen ! I would personally advocate to focus only on `T5`, and we can add these patches later on if we figure out that the same issue occur for all subsidiary models! Can you revert the changes for longt5/perceiver & switch (ideally also keep the copy mechanism, so maybe add the `# Copied from` statements but use another model as t5 as reference (for e.g. for perceiver `# Copied from transformers.src.models.longt5. ...`) Also don't forget to run the styling changes ;) (`make fixup`) Thanks again!

That makes sense! done
[Pipeline] fix failing bloom `pipeline` test
Author: younesbelkada
Pull request: yes

# What does this PR do?

This PR fixes the `pipeline` test: `tests/pipelines/test_pipelines_text_generation.py::TextGenerationPipelineTests::test_small_model_pt_bloom_accelerate`

- Link to failing job: https://github.com/huggingface/transformers/actions/runs/3691174891/jobs/6248989365 
- Why this fix is relevant? Before https://github.com/huggingface/transformers/pull/20602 there was an inconsistency between models loaded with `accelerate` with no `device_map` set and with `device_map` set (i.e. without `accelerate`). Before the the aforementioned PR, if you load a model as follows:
```
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("hf-internal-testing/tiny-random-bloom", device_map="auto")
print(model.lm_head.weight.dtype)

model = AutoModelForCausalLM.from_pretrained("hf-internal-testing/tiny-random-bloom")
print(model.lm_head.weight.dtype)
```
You get:
```
torch.bfloat16
torch.float32
```
Which is inconsistent. Since that, to load a model with its native dtype, you need to provide `torch_dtype="auto"`.
This PR fixes the failing test by setting `torch.float32` for the expected `dtype`.

cc @ydshieh @sgugger 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20778). All of your documentation changes will be reflected on that endpoint.
OPT model sizes mismatch between code and webpage
Author: chenho74
Pull request: no

### System Info

N/A

### Who can help?

 @ArthurZucker, @sgugger and @stevhliu

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I was trying to figure out the architecture of OPT models, specifically the pre-layernorm vs post-layernorm setting. I came across these two lines that specified the setting for OPT of different model sizes. 
https://github.com/huggingface/transformers/blob/67acb07e9ef40e6ea08997261e1d14a02530cf8b/src/transformers/models/opt/modeling_opt.py#L322
https://github.com/huggingface/transformers/blob/67acb07e9ef40e6ea08997261e1d14a02530cf8b/src/transformers/models/opt/modeling_opt.py#L346

However it doesn't match with the model sizes available here (if you expand the model list and search `opt-`). I didn't see 1.7B and 175B. 
https://huggingface.co/facebook?sort_models=alphabetical#models

Which one is more accurate? 


### Expected behavior

N/A

Comments:
Hey! Basically the `1.7B` is now the `1.3b` and the `175B` is private and was not released by the META AI team. 
If you want you can open a PR to just correct the model size 😉 

Thanks for the clarification!
Move convert_to_rgb to image_transforms module
Author: amyeroberts
Pull request: yes

# What does this PR do?

Moves the `convert_to_rgb` function to `image_transforms` so all image processors can easily import it. 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Recompile `apex` in `DeepSpeed` CI image
Author: ydshieh
Pull request: yes

# What does this PR do?

The based image ships with a version of `apex`. We need to recompile the new version for torch 1.13 though.

This should fix some CI failures, but not all - we will check again the CI report in the next run, if this is ok for you @stas00. Otherwise I can run the full suite locally, and discuss with you how to fix all of them before merge.

Comments:
_The documentation is not available anymore as the PR was closed or merged._