noamaanMulla-03/transformers-issues huggingface dataset:
The messy code generated by opt125m.
Author: 920232796
Pull request: no

### System Info

```shell
transformers version is 4.19.3
```


### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

import torch
from transformers import AutoModelForCausalLM
from transformers import TextGenerationPipeline, AutoTokenizer, AutoModelWithLMHead

tokenizer = AutoTokenizer.from_pretrained("facebook/opt-125m")
transformers_opt = AutoModelForCausalLM.from_pretrained("facebook/opt-125m")
transformers_opt.eval()

text = "The trophy doesn’t fit in the suitcase because "
text_generator = TextGenerationPipeline(transformers_opt, tokenizer)
out = text_generator(text, max_length=300, do_sample=True, top_p=0.9)
print(f"transformers model out is {out}")



### Expected behavior

```shell
I want to get normal output.
```


Comments:
Hi @920232796 👋  `facebook/opt-125m` is a relatively small model, so it's normal that its outputs are not great (especially with `do_sample=True`). I'd suggest trying `facebook/opt-350m`.

As per our [issues guidelines](https://github.com/huggingface/transformers/blob/main/ISSUES.md), we reserve GitHub issues for bugs in the repository and/or feature requests. For any other requests, we'd like to invite you to use our [forum](https://discuss.huggingface.co/) 🤗

Thank you for your patient reply.

@920232796 actually we have found a problem in the OPT files -- https://github.com/huggingface/transformers/pull/17785

It may improve the quality of the generation, but the comment above remains true :)

Thank you very much!
assertEqual of non-frozen parameters in test_resume_training_with_frozen_params
Author: regisss
Pull request: no
Labels: name: bug

### System Info

```shell
- `transformers` version: 4.20.0
- Platform: Linux-5.13.0-1031-aws-x86_64-with-glibc2.29
- Python version: 3.8.10
- Huggingface_hub version: 0.7.0
- PyTorch version (GPU?): 1.11.0a0+gita4c10ee (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no
- Specific hardware: Habana HPU
```


### Who can help?

@sgugger 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Steps to reproduce the behaviour:

1. Setup an AWS DL1 instance
2. Clone [optimum-habana](https://github.com/huggingface/optimum-habana)
3. Install the package with `pip install optimum-habana/[tests]`
4. Uncomment `test_resume_training_with_frozen_params` in `optimum-habana/tests/test_trainer.py`
5. Run `pytest tests/test_trainer.py -k "frozen"`

### Expected behavior

`test_resume_training_with_frozen_params` in `tests/trainer/test_trainer.py` should not assert if `b` and `b1` are equal [here](https://github.com/huggingface/transformers/blob/eb16be415a74328e5ab62e050330a43054f6bd11/tests/trainer/test_trainer.py#L1429). While `a` is frozen, `b` is not so why should `b` and `b1` be equal?

I understand I use a very specific hardware and the test passes on a GPU, but I actually think it should not pass on the latter and would like to understand why it does :)


Comments:
Please use the forums for questions like this as we keep issues for bugs and feature requests only.

b and b1 should be equal because they are the results of the same trainings. One started from scratch and one resumed from an intermediate checkpoint. If that test fails, you have a problem in reproducibility on HPUs.

Sure, sorry for the inconvenience!

Yep I thought `checkpoint-5` comes from a completely different run, closing this as this issue does not come from the test definition.
Number of channels in the ViTMAE model
Author: kacwin
Pull request: no
Labels: name: Good First Issue

### Feature request

Dear huggingface community,
I am experimenting with the ViTMAE model from the transformers library. The ViTMAEConfig class has the option "num_channels" to specify the number of input (color) channels belonging to an image. If I modify this, say, to 1 (for processing grayscale images), the model throws an error, due to the number "3" being hard-coded into the functions "patchify" and "unpatchify" in the file "modeling_vit_mae.py". 

### Motivation

I would like to request to change this such that any number of input channels is possible. 

### Your contribution

As noted above, one only has to change the functions "patchify" and "depatchify" to either scan for the input color channels, or provide the input color channels as a class variable such that both functions can use it (instead of the hard-coded value "3"). I checked this and on my system it worked out just fine.

Comments:
Hi,

Yes the "patchify" and "unpatchify" methods would need to be updated to supported 1 channel.

Are you interested in opening a PR to add support for this? 

Hey, 
thanks for the fast reply. I am a complete beginner to github (I only used it to manage my personal projects so far) and I will not be getting into the PR-procedure anytime soon. I did not know how to reach out to someone managing the vitmae project and tried out this issue just to point things out. If noone else is having the same problem or noone has time to care for this then this is just how it is I guess.

@NielsRogge updating the hard-coded part to the config value will work? 

I can open a PR to help on this
[ViTMAE] Fix docstrings and variable names
Author: NielsRogge
Pull request: yes

# What does this PR do?

Fixes #17473
Fixes #17665

This PR improves the docstrings and variable names of the patchify, unpatchify and forward_loss methods of ViTMAE. This way, also the number of channels isn't hardcoded anymore.

cc @sayakpaul 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add OPT model parallelize
Author: lkm2835
Pull request: yes

# Model Parallelism for OPT

Added ```parallelize``` and ```deparallelize``` methods on ```OPTDecoder```, ```OPTModel``` and ```OPTForCausalLM```.
Referred to ```gpt2``` model parallelize (https://github.com/huggingface/transformers/pull/8696).

Fixes #17240 

## Who can review?
Let me know if you need any modifications, @patrickvonplaten

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_17245). All of your documentation changes will be reflected on that endpoint.

Hey @lkm2835,

Thanks for your PR - however this way of parallelizing the model is a bit outdated. The recommended way of using the model in parallel is to use `accelerate` see: https://twitter.com/huggingface/status/1524783489593360385

We'll soon have this natively supported in `transformers` as well cc @sgugger 

Then, is it better to close this PR?

> natively

I'm afraid so! There are lots of other "Good first issues" or "Good second issues" though if you'd like to give it a try :-)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Distributed Support for OPT models in transformers
Author: Mrs-Hudson
Pull request: no

### Feature request

Hi
Thanks a lot for adding OPT models to transformers. As of now the 55GB 30B parameter model needs to be loaded into a single GPU, otherwise it t[hrows a CUDA memory error](https://discuss.huggingface.co/t/running-inference-on-opt-30m-on-gpu/17895/2).

It would be great if we could have distributed support for these[ models similar to gpt2](https://github.com/huggingface/transformers/pull/7772) so we can leverage multiple GPUs to run them

### Motivation

Make it easier to run OPT models on available infra

### Your contribution

NA, 
can test changes on my system

Comments:
@patrickvonplaten  rasising a feature request for model parallelism for  the newly added OPT models. Please triage/comment as you see fit

Same answer as in https://github.com/huggingface/transformers/pull/17245#issuecomment-1128064880 here. Think we'll soon have something that works out of the box cc @sgugger 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[WIP] Use word_ids to determine if a pre-entity is a subword in TokenClassificationPipeline
Author: barar-primer
Pull request: yes

# What does this PR do?
Currently, `TokenClassificationPipeline` checks an attribute called `continuing_subword_prefix` in order to determine whether it can use the "correct" token aggregation strategy. Otherwise, it uses a backup heuristic which doesn't work well usually. This check works for BERT, but not for XLNet and RoBERTa:

```
>>> from transformers import AutoTokenizer
>>> bert_tk = AutoTokenizer.from_pretrained("bert-base-cased")
>>> print(getattr(tk._tokenizer.model, "continuing_subword_prefix", None))
##
>>> xlnet_tk = AutoTokenizer.from_pretrained("xlnet-base-cased")
>>> print(getattr(xlnet_tk._tokenizer.model, "continuing_subword_prefix", None))
None
```

However, there is a better way. The fast tokenizers for XLNet and RoBERTa are word-aware and provide a `word_ids` method. This PR updates the pipeline to pass around the `word_ids` list until it is used in `gather_pre_entities`. 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@LysandreJik @Narsil I'd appreciate a draft review and then I'll update tests if this looks good.

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_17238). All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Shard checkpoint for `tf` and `flax`
Author: ArthurZucker
Pull request: no
Labels: name: TensorFlow

name: Core: Modeling

name: Flax

### Feature request

The same sharding capabilities as pytorch should be available to `flax` and `tf`. This is required in order to push the OPT30B model. 

### Motivation

Pushing $>45GB$ models (and have the same behaviour as in pytorch ). 

### Your contribution

Could start working on that when I will be back from holidays! 

Comments:
Use 5e-5 For BigBird PT/Flax equivalence tests
Author: ydshieh
Pull request: yes

# What does this PR do?

Use `5e-5` For BigBird PT/Flax equivalence tests to avoid flaky test failure.

Also change the name `check_outputs` to `check_pt_flax_outputs` (similar to `check_pt_tf_outputs`) and update its logic similar to `check_pt_tf_outputs`. 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
TF Sharded
Author: ArthurZucker
Pull request: yes
Labels: name: TensorFlow

name: Core: Modeling

# What does this PR do?

Introduces the sharding of TF models following the pytroch implementation. 

A simple working example is the following : 

```python 
from transformers import TFOPTModel
save_directory = "opt-350m"
model = TFOPTModel.from_pretrained("facebook/opt-350m")
model.save_pretrained(save_directory, max_shard_size = "1GB")
tf_model = TFOPTModel.from_pretrained(save_directory)
```


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Okay so the tfopt_for_causal_lm/tfopt_model 
prefix  from the tfopt_for_causal_lm/model/decoder/embed_positions/weight:0 in the index json comes from the actual name of the layer (so tf side). This also creates the hack that we sometime need when some layer is shared : for OPT we have the following : 'decoder.embed_tokens/model.decoder.embed_tokens/weight:0' which thus becomes model.decoder.embed_tokens/weight:0  . Most interesting part is that the ‘decoder.embed_tokens’ comes from https://github.com/ArthurZucker/transformers/blob/e950ff48a91840e30966abaf86bdb02dc16fcdab/src/transformers/models/opt/modeling_tf_opt.py#L499-L511 (the load weight prefix hack using load_weight_prefix) I am sure that there is something to do about that so I will detail that and dig a bit further

Looks very nice to me!

Only did a very high-level review. Defering to @gante and @sgugger here :-)
Fix forward reference imports in DeBERTa configs
Author: sgugger
Pull request: yes

# What does this PR do?

#17617 introduced some imports that will create cyclical references, just for type hinting. This PR makes them only imported in a type checking block.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

FYI @JingyaHuang @michaelbenayoun I'll take a look at the slow test tomorrow :)
Fix duplicate code at T5Model
Author: lkm2835
Pull request: yes

# What does this PR do?

Unlike `T5ForConditionalGeneration`, it doesn't seem to be necessary at `T5Model`.

@patrickvonplaten


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hey @lkm2835,

could you give a bit more feedback on why this is not necessary? I don't exactly see why this is necessary at the moment.

Oh, sorry @patrickvonplaten 

1411-1412 and 1414-1415 are same code.
```
1411        if self.model_parallel:
1412            torch.cuda.set_device(self.decoder.first_device)
1413        # Set device for model parallelism
1414        if self.model_parallel:
1415            torch.cuda.set_device(self.decoder.first_device)
1416            hidden_states = hidden_states.to(self.decoder.first_device)
                   ...
``` 

https://github.com/huggingface/transformers/blob/main/src/transformers/models/t5/modeling_t5.py#L1411


In T5ForConditionalGeneration,

```
1619        if self.model_parallel:
1620            torch.cuda.set_device(self.decoder.first_device)

1622        if labels is not None and decoder_input_ids is None and decoder_inputs_embeds is None:
1623            # get decoder inputs from shifting lm labels to the right
1624           decoder_input_ids = self._shift_right(labels)

1626        # Set device for model parallelism
1627        if self.model_parallel:
1628           torch.cuda.set_device(self.decoder.first_device)
1629           hidden_states = hidden_states.to(self.decoder.first_device)
```

1619-1620 and 1627-1628 are same code. But If 1622-1624 need `set_device`, 1619-1620 is necessary.

https://github.com/huggingface/transformers/blob/main/src/transformers/models/t5/modeling_t5.py#L1619
Fix `top_k_top_p_filtering` having unintended behavior
Author: unifyh
Pull request: yes

# What does this PR do?

 - Fix `top_k_top_p_filtering` not passing `filter_value` to
   `TopPLogitsWarper` causing any top-p filtered logits to be -inf
   instead of specified value
 - Add corresponding test

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

@patrickvonplaten 


<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks a lot for the fix @unifyh!
Flax sharded
Author: ArthurZucker
Pull request: yes
Labels: name: Core: Modeling

# What does this PR do?
Adds support for FLAX sharding checkpoints 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

The sharding tests are run for every model which should be avoided

Will merge `flax` before `tf` as the TF one still needs a few modification (mostly cleaning the documentation)
Cannot import name 'load_offloaded_weights' from 'accelerate.utils'
Author: 99991
Pull request: no
Labels: name: bug

### System Info

```shell
>>> import transformers
>>> transformers.__version__
'4.20.1'
>>> import accelerate
>>> accelerate.__version__
'0.8.0'
>>> import sys
>>> sys.platform
'linux'
>>> sys.version
'3.8.10 (default, Mar 15 2022, 12:22:08) \n[GCC 9.4.0]'
>>> import os
>>> os.system("lsb_release -a")
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.4 LTS
Release:        20.04
Codename:       focal
```


### Who can help?

@99991

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Execute the example from "Getting started" https://github.com/UKPLab/sentence-transformers#getting-started
2. Observe crash.

Error message:

```
Failed to import transformers.models.bert.modeling_bert because of the following error (look up to see its traceback):
cannot import name 'load_offloaded_weights' from 'accelerate.utils' (/home/username/.local/lib/python3.8/site-packages/accelerate/utils/__init__.py)

The above exception was the direct cause of the following exception:

  File "/home/username/Desktop/computing_embeddings.py", line 2, in main
    model = SentenceTransformer('all-MiniLM-L6-v2')
```

### Expected behavior

It should not crash.


Comments:
The solution is to upgrade the `accelerate` library. I had version `0.8.0` and upgraded to `0.10.0`.

```
pip install --upgrade accelerate
```

Version `0.10.0` has the missing function `load_offloaded_weights`:

https://github.com/huggingface/accelerate/commit/8b8c5345cd84ba96cca810b677601204e06853ba#diff-331ffa5527e400ee60607a11481feb0197abd8492c000255c337d0bf4312c0c0R43
Add logits_processor parameter, used by `generate`, to `Seq2SeqTrainer` methods `evaluate` and `predict`
Author: eranhirs
Pull request: yes

# What does this PR do?

Following the discussion in #17748, this PR adds the `logits_processor` param to Seq2SeqTrainer `predict` and `evaluate` methods, for easy extensibility.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@gante @sgugger

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sgugger What are your thoughts about adding to the `test_finetune_bert2bert` a call to `trainer.evaluate()` and `trainer.predict(eval_dataset)`?

Sure, we can add this. But is it to test any of this functionality? If not, it should go in its own PR.

It somewhat tests this functionality by calling the methods I changed, but not directly. I can add it later in its own PR.

Merging this one then, thanks again for your contribution!
Clean modeling utils, linked to  #17760 and #17713
Author: ArthurZucker
Pull request: yes

# What does this PR do?
The recently introduced `TF` and `FLAX` sharding scripts #17760 and #17713 both use ` convert_file_size_to_int, get_checkpoint_shard_files`, which were moved to `transformers.utils.hub`. This PR just removed the definition of these two function and imports them form `transformers.utils.hub` 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Use the new hub ci
Author: XciD
Pull request: yes

Uses the new CI environment for tests, which uses Gitaly and is automatically updated to the latest version of the Hub.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_17652). All of your documentation changes will be reflected on that endpoint.

Yay!

> Yay!

Blocked by https://github.com/huggingface/huggingface_hub/pull/898 :)

Done in #17716, @XciD lost a contribution to the codebase =) =)  cc @sgugger @LysandreJik 

If you don't ping us for reviews, your PR doesn't exist :-p 

> If you don't ping us for reviews, your PR doesn't exist :-p

Yes, my bad, I was waiting for another one before pinging 

No worries, was just joking around why neither Lysandre nor I had seen it :)