noamaanMulla-03/transformers-issues huggingface dataset:
Symbol Zeros is already exposed as (). Error Help pls 
Author: ZwChiew
Pull request: no

### Major Bug while importing Huggingface locally model in VS code

Was trying to implement a simple paraphrasing tool 
but unable to import / download the library Vamsi/T5_Paraphrase_Paws due to this error: 
Symbol Zeros is already exposed as (). 
Was working fine on google colab / anaconda but not on vs code 
All required libraries and dependencies installed 

Boiler code: 
from transformers import AutoModelForSeq2SeqLM

model = AutoModelForSeq2SeqLM.from_pretrained(
    "Vamsi/T5_Paraphrase_Paws").to('cuda')

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

from transformers import AutoModelForSeq2SeqLM

model = AutoModelForSeq2SeqLM.from_pretrained(
    "Vamsi/T5_Paraphrase_Paws").to('cuda')

### Expected behavior

Please help me solve this issue, this is a core module of a very important system 

Comments:
Okay, did you install `safetensors` and `sentencepiece`? I cannot reproduce your issue

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Added max_length warning to pretrained tokenizer initialization
Author: grantslewis
Pull request: yes

# What does this PR do?

A truncation warning, "Asking to truncate to max_length but no maximum length is provided and the model has no predefined maximum length. Default to no truncation.", while not incorrect (max_length is a parameter for running the tokenize function), could also be interpreted as suggesting 'max_length' is a parameter used across the tokenizer class. However, 'max_length' silently does nothing when it is passed into AutoTokenizer.from_pretrained(), for example, and the previously mentioned warning will still occur. The proposed addition provides a simple warning when the 'model_max_length' is None and 'max_length' is passed in as an argument. The warning suggests the correct parameter name instead. This is meant to help provide a warning to direct developers to the correct variable name, helping prevent any unnecessary troubleshooting. This change should not require any additional tests or dependencies.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

CC @ArthurZucker

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Llama2-13b-chat: Output contains part of prompt (including 
 tag)
Author: mukundt
Pull request: no

### System Info

SageMaker + Llama2-13b-chat-hf (finetuned)

### Who can help?

@ArthurZucker @younesbelkada @gante

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

If I copy the agent response back to the prompt (as a user input), it regurgitates the previous output including the 
 tag, which is very strange.

**Prompt**
```


 

{{ system prompt }}



xxx 
 yyy 


 xxx 
 yyy 


 xxx 
 yyy 


 {{ this is where I copy the agent's response and pass it in as user input, without any special tags }} 

```

**Generated Response verbatim**
```
No, we didn't receive your payment... 
 I'm sorry to hear that. Would you like to make a payment now?
```

### Expected behavior

Output should not contain 


Comments:
Can you provided a reproducer? And share the transformers version you are using? 

@ArthurZucker unfortunately I can't provide a specific reproducer because this model was fine-tuned on a customer's data set, but here is how I produced the problematic output:
```
User: {msg 1}
Assistant: {msg 2}
User: {msg 3}
Assistant: {msg 4}
User: {msg 2}
Assistant: {outputs part of msg 3, including 
}
```

I used [this script for training](https://github.com/philschmid/huggingface-llama-2-samples/blob/master/training/sagemaker-notebook.ipynb), which installs transformers==4.31.0.

@mukundt looking at your example, it may be as simple as the model knowing that it can copy-paste parts of `msg 3`, as it has the history as context and it has seen `msg 3` following `msg 2`.

If you are using `.generate()`, you can set this token as impossible to generate through `bad_word_ids` ([reference](https://huggingface.co/docs/transformers/v4.32.0/en/internal/generation_utils#transformers.NoBadWordsLogitsProcessor), which has an example)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Tree add-model-idefics deleted and Google Colab stoped working
Author: dvelho
Pull request: no

### System Info

Google Colab: https://github.com/huggingface/notebooks/blob/main/examples/idefics/finetune_image_captioning_peft.ipynb

### Who can help?

_No response_

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Run the first cell and you will get this error:

Collecting git+https://github.com/huggingface/transformers.git@add-model-idefics
Cloning https://github.com/huggingface/transformers.git (to revision add-model-idefics) to /tmp/pip-req-build-lgss5sp_
Running command git clone --filter=blob:none --quiet https://github.com/huggingface/transformers.git /tmp/pip-req-build-lgss5sp_
WARNING: Did not find branch or tag 'add-model-idefics', assuming revision or ref.
Running command git checkout -q add-model-idefics
error: pathspec 'add-model-idefics' did not match any file(s) known to git
error: subprocess-exited-with-error

× git checkout -q add-model-idefics did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× git checkout -q add-model-idefics did not run successfully.
│ exit code: 1
╰─> See above for output.

Also it's not listed here:
https://github.com/huggingface/transformers/branches/active?page=1
https://github.com/huggingface/transformers/tree/add-model-idefics
maybe it was deleted?

### Expected behavior

No error when installing

Comments:
Hi @dvelho 
Thanks for your issue,
please install transformers from source
```bash
pip install git+https://github.com/huggingface/transformers.git
```
And the issue should be resolved

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Closing as IDEFICS is now part of transformers latest release
[TokenizerSlow] `replace_additional_special_tokens` is not doing much
Author: ArthurZucker
Pull request: no
Labels: name: Core: Tokenization

Just flagging this as the `add_special_tokens` method got pretty complicated, adding a kwargs, `replace_additional_special_tokens`, that supposedly can prevent replacing the `self._additional_special_tokens` attribute. 
For any tokenizer, this will remove it from the list, but will not update the internal `trie` and thus has no effect at all: 
```python
>>> from transformers import XLMRobertaTokenizer
>>> tokenizer_a = XLMRobertaTokenizer.from_pretrained('xlm-roberta-base')
>>> tokenizer_a.add_special_tokens({"additional_special_tokens":["<//s>"]})
>>> tokenizer_a.additional_special_tokens
['<//s>']
>>> print(tokenizer_a.tokenize("This is a <//s>"))
['▁This', '▁is', '▁a', '<//s>']
>>> tokenizer_a.add_special_tokens({"additional_special_tokens":["<///s>"]}, replace_additional_special_tokens= True)
>>> print(tokenizer_a.tokenize("This is a <//s>"))
['▁This', '▁is', '▁a', '<//s>']
```
This will be addressed in #23909 

Comments:
cc @ydshieh since you added the feature

~~I don't *fully* understand what the code snippet above try to demonstrate.~~

But the origin of `self._additional_special_tokens` is from this issue #20418, where `added_tokens_encoder` will include all the added tokens, but `additional_special_tokens` is being replaced, which is really confusing behavior.

If you look the description in #20418, your code snippet does its job (although yes confusing).

The `replace_additional_special_tokens` with its default value `True` is just to make the behavior not **too** surprising, but keep the backward compatibility.



> It was confusingly for me that the added tokens encoder is not updated.

yeah I know, but that's what it has been for years. (and I agree that the name of this introduced argument itself might be confusing too.) 

> That’s because maybe we should have a separate function just to say that’s we don’t want this token to be special anymore

If you have good idea to address the issue #20418 while reducing the (naming) confusion added in #20424, go ahead :-)

(sorry, I accidentally modified your message 😭 )

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Closing as this is deprecated and changing the list of additional special tokens is a lot more involved than this
Eval Accumulation Steps Silently Fails w/ Accelerate >= 0.20.3?
Author: sam-scale
Pull request: no

### System Info

```
Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 4.31.0
- Platform: Linux-5.15.0-1026-aws-x86_64-with-glibc2.10
- Python version: 3.8.17
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.1
- Accelerate version: 0.21.0
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.0.1+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: yes
```

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

* saw that nothing was getting offloaded to GPU here: https://github.com/huggingface/transformers/blob/main/src/transformers/trainer.py#L3196
* Needed `self.accelerator.sync_gradients` to be `True`
* It's set at this line: https://github.com/huggingface/transformers/blob/main/src/transformers/trainer.py#L1863-L1866
Requires accelerate <= 0.20.3

### Expected behavior

Allow `eval_accumulation_steps` to actually do something even if `accelerate > 0.20.3`

Comments:
@muellerzr just so I know, is there any estimate on this issue?

@muellerzr sent out a PR to fix here: https://github.com/huggingface/transformers/pull/26060

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

I believe this was closed by https://github.com/huggingface/transformers/pull/26060

Closing this!
Update docs to explain disabling callbacks using report_to
Author: bgub
Pull request: yes

# What does this PR do?

Fixes #26036. I added clarification to the Callback documentation page explaining how to "turn off" default callbacks using `TrainerArguments.report_to`.


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
    - https://github.com/huggingface/transformers/issues/26036#issuecomment-1717603016
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@muellerzr @pacman100 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26155). All of your documentation changes will be reflected on that endpoint.

@muellerzr sure! Just pushed that change.
Option to disable CodeCarbon
Author: bgub
Pull request: no

### Feature request

- Allow disabling of CodeCarbon reporting, even when it's installed

### Motivation

I'm running code using the Trainer class across several nodes in an HPC environment using SLURM. It seems to automatically [use the CodeCarbon callback](https://huggingface.co/docs/transformers/v4.33.0/en/main_classes/callback#transformers.integrations.CodeCarbonCallback), but unfortunately doesn't work correctly in my multi-node environment and floods my log files with hundreds of distracting messages (mentioned earlier in https://github.com/mlco2/codecarbon/issues/252).

![image](https://github.com/huggingface/transformers/assets/25597854/430dcc9e-702b-4a42-9877-ea515a777421)

### Your contribution

I'd be happy to contribute and/or open a PR, given some guidance about where to start.

Comments:
Hi @nebrelbug, thanks for raising this issue! 

Sounds like a good idea to me. @muellerzr will know more about it than me and can advise. 

That sounds fine to me as well @nebrelbug. 

We have a few options here:
1. Globally disable `codecarbon` via an environmental variable (similar to `DISABLE_WANDB`, or something along those lines)
2. Have it as a `TrainingArgument` (don't quite like that solution)
3. Or: don't add the codecarbon callback if we know we are in a multi-node scenario. 

I think I'd prefer 3 if possible, otherwise 1 :) WDYT?

@muellerzr I don't think that CodeCarbon is necessarily broken for everyone using multiple nodes, so my personal preference would be option 1. 

Would it be alright if I opened a PR to disable the callback when `DISABLE_CODECARBON` is set?

@nebrelbug actually looking at this more, does using `--report_to "none"` give the desired result? 

If so, we should expand the doc on the `report_to` param in `TrainingArguments` to mention codecarbon

@muellerzr it does, actually! Setting `report_to` to either `"none"` or `"wandb"` results in CodeCarbon not running.

Thanks for verifying! Lets go ahead and add it to the report_to doc string @nebrelbug 

@muellerzr sweet! I actually think the doc string for `report_to` may be complete (though a little hard to find). Instead, I opened up a PR at #26155 to update the Callback doc page. Let me know if that sounds good!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
🎉 Add prompt tuning for bigcode models
Author: mayank31398
Pull request: yes

# What does this PR do?

Enables prompt tuning for BigCode models.
Currently, prompt tuning for bigcode models fail due to this error AttributeError (no GPTBigCodeForCausalLM has no attribute named `word_embeddings`).

@ArthurZucker, @younesbelkada, @pacman100 

Comments:
Hmmm I'm not sure we have the expectation that all models have a `word_embeddings` property, at least we really don't enforce it right now.

Does PEFT make that assumption?

Hi, I just realized PEFT doesn't make any such assumptions.


![Screenshot 2023-10-06 at 2 45 07 PM](https://github.com/huggingface/transformers/assets/32954280/df9b6371-2762-44c7-bc88-a7a74637a0b4)
^^ this is the logic used by PEFT to get the model's vocab.

Why it doesn't work for me:
```python
    def _setup_model_for_peft(self, args: Union[TrainingArgs, InferenceArgs], model_kwargs: dict) -> None:
        self.deepspeed_config = HfDeepSpeedConfig(get_deepspeed_config(args))

        self.peft_config = PromptTuningConfig(
            task_type=TaskType.SEQ_2_SEQ_LM if self.is_encoder_decoder else TaskType.CAUSAL_LM,
            prompt_tuning_init=args.prompt_tuning_init,
            num_virtual_tokens=args.num_virtual_tokens,
            prompt_tuning_init_text=args.prompt_tuning_init_text,
            tokenizer_name_or_path=args.model_name,
        )
        self.model = args.model_class.from_pretrained(**model_kwargs, torch_dtype=self.dtype)
```

I am initializing the model like this. this creates a problem due to `HfDeepSpeedConfig` being called.
DeepSpeed converts all the tensors to dummy device tensors with shape 0 as follows for faster initialization of the model.

```shell
lm_head torch.Size([0])
```

I am not sure what a good solution for this is.
But this PR is not needed to resolve the issue.
@younesbelkada do you have any suggestions?

Turns out that the issue can be resolved using this fix in PEFT:
```python
if value.shape[0] == self.base_model.config.vocab_size or (hasattr(value, "ds_shape") and value.ds_shape[0] == self.base_model.config.vocab_size):
```

this has been fixed with a PR to peft
no changes are needed to this repo.
[Assistant Generation] Improve Encoder Decoder
Author: patrickvonplaten
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

This PR speeds up assistant generation / speculative decoding for encoder-decoder models such as Distill-Whisper by ~20-30%.

Improvements:
- If assistant and model share same encoder, let's allow the user to pass `assistant_encoder_outputs` so that the inputs are not encoded twice (gives ~20% speed-up)
- In the small loop I don't think we have to allocate tensors for the attention mask all the time. This is done automatically by the model if necessary (gives ~3,4% speed-up)
- The heuristic to increase / decrease the number of "look-ahead" tokens doesn't work well for whisper, can we maybe allow the user to somehow disable it? Maybe via a config attribute?

Comments:
_The documentation is not available anymore as the PR was closed or merged._

The failing Hub [test](https://app.circleci.com/pipelines/github/huggingface/transformers/75063/workflows/31a94fbe-b7f7-456a-bff9-5f149743c3fd/jobs/951835) seems to be flaky.

This PR is ready for a final review.
`Copied from` for test files
Author: ydshieh
Pull request: yes

# What does this PR do?

`Copied from` for test files.

Run `make fix-copies` will show 

```bash
python utils/check_copies.py --fix_and_overwrite
Detected changes, rewriting tests/models\longformer\test_tokenization_longformer.py.
```

I will need to update `test_tokenization_longformer.py` before merge.

Comments:
well, I need to check why 

```
ci/circleci: tests_repo_utils
```
fails.

_The documentation is not available anymore as the PR was closed or merged._

Will have to perform some fixes about copied statements before merge.

If there is a single place, say `__init__`, being a non-copy, then so far we can't put `# Copied from` at the class level. We can rework the script a bit to have `# Ignore copied` in a method, but would be better to do this in a follow up PR.

Ok!