noamaanMulla-03/transformers-issues huggingface dataset:
MaskFormer post_process_instance_segmentation bug fix convert out side of loop
Author: amyeroberts
Pull request: yes

# What does this PR do?

Resolves a bug in post processing logic in MaskFormer and Mask2Former. 

If `return_coco_annotation` is set to `True` the generated segmentation map was converted to RLE before finishing iterating over all of the queries. This breaks the assumption that segmentation is an array of the same shape as `pred_masks` [here](https://github.com/huggingface/transformers/blob/e97deca9a3f4ddf2a6a44405ed928067d7b729f3/src/transformers/models/maskformer/image_processing_maskformer.py#L1073) in the for loop.

Fixes #25486 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hi,
I am using transformers Version: 4.32.0.dev0, 
do you have an estimated time to release a version with this fix?
thanks

@MayChi22 We release a new version roughly every month, so there will likely be a new release in a few weeks.
Generate: fix default max length warning
Author: gante
Pull request: yes

# What does this PR do?

Fixes the generate max length warning -- it should be emitted when it is `== 20`, corresponding to the model-agnostic default value.

A test is also added to ensure we don't regress.

Comments:
@ArthurZucker Good catch! Added there as well 👍 

_The documentation is not available anymore as the PR was closed or merged._
Loading pretrained RobertaModel,size missmatch error
Author: aipursuing
Pull request: no

My code is as follows:

`config = RobertaConfig.from_pretrained("roberta-base", max_position_embeddings=2048)`
`model = RobertaModel.from_pretrained('roberta-base',config = config)`
then I get the following error:
`size mismatch for roberta.embeddings.position_embeddings.weight: copying a param with shape torch.Size([514, 768]) from checkpoint, the shape in current model is torch.Size([2048, 768]).`

how can I solve this problem? If I want to expand the length of input sentences, what should I do?



Comments:
@aixiaobaikyh Please follow the issue template and fill out all the requesting information such as the transformers version being run. 

If running on a version of transformers released in the past year, the error message shared here is not the full error message that is printed out. The final part instructs on how to resolve: 
```
        You may consider adding `ignore_mismatched_sizes=True` in the model `from_pretrained` method.
```

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Generate function 
Author: Dongximing
Pull request: no

### System Info

The model is llama and the tokenizer is also llama (decapoda-research/llama-7b-hf)
tokenizer = LlamaTokenizer.from_pretrained('decapoda-research/llama-7b-hf',add_special_tokens=False,add_bos_token = False)

I have a question about the model generation. 
```python
prompt = """Tell me some things about NBA"""
input_tokenized_info = tokenizer(prompt, return_tensors="pt")
input_ids, attention_mask = input_tokenized_info['input_ids'], input_tokenized_info['attention_mask']
input_ids = input_ids.to('cuda') 
attention_mask = attention_mask.to('cuda')
outputs = model.generate(input_ids=input_ids, attention_mask=attention_mask,num_beams = 10,no_repeat_ngram_size=1,max_length=200,\
                        return_dict_in_generate=True,output_scores=True,length_penalty=0.9)
print(len(outputs[0][0]))
18
print(len(outputs.scores))
194
print(outputs[0][0])
tensor([24948,   592,   777,  2712,  1048, 21517, 29871, 29906, 29968, 29896,
        29929,   341, 29911,  3189,  1144, 29889,     2,     1],
       device='cuda:0')
print(tokenizer.decode(outputs[0][0], skip_special_tokens=True))
'Tell me some things about NBA 2K19 MT Coins.'
```
I think the score size should be the same as the (output-input) size  

### Who can help?

@ArthurZucker 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I think the output-input size is the same as the score size, also why the end of the token is 1 and then stop it. 

### Expected behavior

How can I generate a good output for example I set a max_length =100, it should have the stop and some "," , not print 100 tokens or stop at BOS token. 


Thanks you 

Comments:
Hi @Dongximing, 

The `decapoda-research/llama-7b-hf` checkpoint shouldn't be used. The tokenizer and weights were released before the Llama PR was merged and are not compatible with the Llama implementation in transformers. There are other checkpoints (e.g. [this one](https://huggingface.co/huggyllama/llama-7b)) which are compatible. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Amazon Sagemaker - huggingface-textgeneration1-gpt-j-6b-fp16
Author: Mrin7
Pull request: no

### System Info

Running on Amazon sagemaker notebook.(https://us-west-2.console.aws.amazon.com/sagemaker/playground?region=us-west-2#/foundation-models/playground/prod-000000021)
ml.m5.xlarge | 4 vcpu | 16 GiB memory




### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

(https://huggingface.co/docs/transformers/main/model_doc/gptj) The hugging face repo claims that the fp16 model should be able to run on 16 GB GPU memory for inference. I am using ml.m5.xlarge | 4 vcpu | 16 GiB memory in Amazon Sagemaker to deploy the model.
Why am I getting data load error in Amazon sagemeaker?

Can someone please share steps to deploy a model in sagemaker. Moreover how do we requisition ASW to allocate us larger resource, if models won't work on these instances.? It requires an organization for instance allocation, are individuals not able to test models on AWS sagemaker.

### Expected behavior

Error hosting endpoint jumpstart-example-huggingface-textgener-2023-07-07-13-55-56-291: Failed. Reason: Failed to extract model data archive from URL "s3://jumpstart-cache-prod-us-west-2/huggingface-infer/prepack/v1.1.2/infer-prepack-huggingface-textgeneration1-gpt-j-6b-fp16.tar.gz". The model data archive is too large. Please reduce the size of the model data archive or move to an instance type with more memory.

Comments:
Hi @Mrin7, thanks for raising an issue! 

When creating issues on github, please create a separate issue for each individual question. 

With regards to your first question, the docs state that the model can fit into 16 GB of RAM for interference, however there may be other processes or objects which have memory requirements resulting in the total amount of GPU RAM needed being  above 16 GB. Without knowing exactly what you're running it's not possible to know. 

For questions on how to deploy on sagemaker, please refer to the docs: https://huggingface.co/docs/sagemaker/inference. If you still have questions, then it's best asked in our [forums](https://discuss.huggingface.co/). We try to reserve the github issues for feature requests and bug reports.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
fix vit hybrid test
Author: SunMarc
Pull request: yes

# What does this PR do ? 
This PR fixes a [test](https://github.com/huggingface/transformers/actions/runs/5503465492/job/14897632810). This bug was introduced by this [PR](https://github.com/huggingface/accelerate/pull/1648) on accelerate library. Basically, for single gpu setup, when we use device_map = 'auto', we don't add hooks to the model anymore. Hence, the test has been failing because we need to move the input to the right device. 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix `MaskFormerModelIntegrationTest` OOM
Author: ydshieh
Pull request: yes

# What does this PR do?

The change in #25297 has some effect of increased memory usage, and `tests/models/maskformer/test_modeling_maskformer.py::MaskFormerModelIntegrationTest::test_with_segmentation_maps_and_loss` starts to fail when `MaskFormerModelIntegrationTest` is run as a whole.

This PR changes the image size in `MaskFormerModelIntegrationTest` to avoid OOM.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
More frozen args
Author: muellerzr
Pull request: yes

# What does this PR do?

Noticed some failures in the Accelerate nightly related to deepspeed (see [here](https://github.com/huggingface/accelerate/actions/runs/5877296024/job/15937618182)), the hyperparameter search needs to be able to modify the `training_arguments` (as that makes sense, it's doing HPS). As a result, manually sets/unsets `_frozen` during HPS. Not too worried about the pattern since the function is already "private"


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger @pacman100 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
More utils doc
Author: sgugger
Pull request: yes

# What does this PR do?

This continues the work of cleaning up the scripts used for quality checks, which comes with a couple of fixes mainly:
- the instructions for releases in the `setup.py` weren't necessarily up to date, so fixing those.
- the script that auto-generates the transformers metadata used by the Hub didn't give the right processing class for `ImageProcessor`s.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
ModuleNotFoundError: No module named 'transformers'
Author: JojoTacoTheCat77
Pull request: no

Hi all, 
I have installed the transformer package into my virtual env and launched jn from that same virtual env. 
Also, I can see the package installed from running pip freeze. Somehow, I keep getting the ModuleNotFoundError. 
![Screenshot 2023-08-16 at 9 21 21 PM](https://github.com/huggingface/transformers/assets/97198215/7e25f5eb-6d72-49ad-ab95-a23624a5ccca)
Here are all the packages in my virtual env. 
`certifi==2023.7.22
charset-normalizer==3.2.0
filelock==3.12.2
fsspec==2023.6.0
huggingface-hub==0.16.4
idna==3.4
Jinja2==3.1.2
MarkupSafe==2.1.3
mpmath==1.3.0
networkx==3.1
numpy==1.25.2
packaging==23.1
PyYAML==6.0.1
regex==2023.8.8
requests==2.31.0
safetensors==0.3.2
sympy==1.12
tokenizers==0.13.3
torch==2.0.1
tqdm==4.66.1
transformers==4.31.0
typing_extensions==4.7.1
urllib3==2.0.4
`
Help please!

Comments:
It is a GPT-4 answerable question and it is related to the issue of using Jupyter Notebook instead of transformers library. (Please check your Python interpreter)

You most probably have a discrepency between the python kernel used by the notebook. Nothing we can do on our end
Fix `torch.fx` tests on nightly CI
Author: ydshieh
Pull request: yes

# What does this PR do?

There is a new `_CodeOnlyModule` type introduced in nightly torch (I have no idea about this). Need an update on our code to make `torch.fx` work for nightly (and future) torch.

Current error:

```
tests/models/layoutlm/test_modeling_layoutlm.py::LayoutLMModelTest::test_torch_fx
(line 753)  AssertionError: Couldn't serialize / deserialize the traced model: Could not generate input named bbox for because root is not a transformers.PreTrainedModel.
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._

> Good for me! Do you know for which models we had this triggered?

All of them, you can see it on

[internal slack channel](https://huggingface.slack.com/archives/C02E34229JA/p1691944652716439)
🚨🚨🚨 Vivit update default rescale_factor value
Author: amyeroberts
Pull request: yes

# What does this PR do?

Updates the default value for `rescale_factor` in line with the updates that happened in #25174 and in the model's checkpoints on the hub [1](https://huggingface.co/google/vivit-b-16x2-kinetics400/commit/8a7171a57f79b9aaa58bc8d977c002a0ea0f0d42), [2](https://huggingface.co/google/vivit-b-16x2/commit/fc341053d36b42d446b3ffccdbd52452712a23f3)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._
YOLOS - reset default return_pixel_mask value
Author: amyeroberts
Pull request: yes

# What does this PR do?

Removes the copied from statement and the changing of the default value of `return_pixel_mask` that got set back in the merging of #25464 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@ydshieh Because #25464 was actually an older branch, and it's such large PR it's hard to spot the change. I broke up some of the work in #25464 to make it less unwieldy e.g. adding copied froms etc, and the changes these introduced where resolved later in separate. I think I then must have messed up a rebase along the line 😅.
Skip `test_onnx_runtime_optimize` for now
Author: ydshieh
Pull request: yes

# What does this PR do?

`tf2onnx` has issue with `TF 2.13`, so let's skip them for now.

Note we have delegated the (pytorch) ONNX tests to Optimum, see [here](https://github.com/huggingface/transformers/pull/24800#issuecomment-1634822781), and it's like we will do this for the TF ONNX tests too (once @michaelbenayoun confirms this).

Comments:
_The documentation is not available anymore as the PR was closed or merged._
change version
Author: SunMarc
Pull request: yes

# What does this PR do ? 
This PR bumps the required version of bnb for training because a major bug was [fixed](https://twitter.com/Tim_Dettmers/status/1687458541643390976?s=20) in 8-bit optimizers. 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add PEFT support directly in transformers pipeline
Author: younesbelkada
Pull request: yes

# What does this PR do?

Replaces https://github.com/huggingface/peft/pull/585 

After discussing with @LysandreJik I made this PoC PR to see whether it is simpler to add PEFT support directly in transformers and centralize all sort of pipelines in transformers pipeline. In the future, we can concentrate the efforts on diffusers side to add PEFT support there as well.

Do not merge before the next PEFT release

Currently the API looks as follows:

```python
from transformers import pipeline 

peft_model_id = "ybelkada/opt-350m-lora"
pipe = pipeline("text-generation", peft_model_id)
print(pipe("hello"))

pipe = pipeline("text-generation", peft_model_id, peft_model_kwargs={"adapter_name": "default"})
print(pipe("hello"))

local_peft_pipeline_path = "./test_lora_pipeline"
pipe.model.save_pretrained(local_peft_pipeline_path)

pipe = pipeline("text-generation", local_peft_pipeline_path)
print(pipe("hello"))
```

## TODOS:

- [ ] seq2seq generation
- [ ] seq classification
- [ ] Add check with task_type
- [ ] add clean tests
- [ ] update docs

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24750). All of your documentation changes will be reflected on that endpoint.

Thanks ! If the PoC gets validated by @Narsil I can extend it to more tasks (seq2seq generation, seq-cls) and add nice tests in the current testing suite

So far `self.check_model_type` is removed, and I know @Narsil suggests it's ok.

I think it could be simply skipped by checking if the model is an instance of PEFT model, and we don't really need to remove it. Leave @sgugger to make the final call though.

Sounds good to me!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Closing as https://github.com/huggingface/transformers/pull/25077 should allow peft models to work out of the box for transformers pipeline
Images in "Efficient Inference on a Single GPU" don't load
Author: osanseviero
Pull request: no

### System Info

NA

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

In https://huggingface.co/docs/transformers/perf_infer_gpu_one there are some unredered images/gifs

### Expected behavior

-

Comments:
cc @stevhliu and @MKhalusova 

I see that the images were not stored in a HF dataset and are likely taken from a third-party blog(?) https://s3.amazonaws.com/moonup/production/uploads/1659861207959-62441d1d9fdefb55a0b7d12c.png
https://s3.amazonaws.com/moonup/production/uploads/1660567469965-62441d1d9fdefb55a0b7d12c.gif
Added initially by @younesbelkada, do you remember what the images were supposed to be? 

Hi,
The fix is simply to replace `https://s3.amazonaws.com/moonup` by `https://cdn-uploads.huggingface.co` (originally pointed out by @coyotte508 internally), I made #25561 that should fix it for images of transformers repo.
[`Docs`] Fix un-rendered images
Author: younesbelkada
Pull request: yes

# What does this PR do?

Fixes https://github.com/huggingface/transformers/issues/25428 and other issues with images on the documentation and this repo that are not rendered properly. 

The fix is to simply replace `https://s3.amazonaws.com/moonup` by `https://cdn-uploads.huggingface.co`

cc @MKhalusova @stevhliu 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Cfg support
Author: lucasjinreal
Pull request: no

### Feature request

Will add cfg support for LLMs?

### Motivation

for better aligned with prompt by make negative mask

### Your contribution

not 

Comments:
This has already been added, check the main branch.
cc @gante

This has already been added, check the main branch.
cc @gante

Awesome, may I ask when would we see it in pypi?

@lucasjinreal this will be part of the next release, which should happen within a month or so. Meanwhile, feel free to install from `main` to access it :) (`pip install --upgrade git+https://github.com/huggingface/transformers.git`)

Check [the PR](https://github.com/huggingface/transformers/pull/24654) for more information about CFG. Example [here](https://github.com/huggingface/transformers/pull/24654/files#diff-d23b812af8462833ad280d968f3e6e2ee7558bacfc2716cdde44a07bead5e065R1267)



thank u all so much