noamaanMulla-03/transformers-issues huggingface dataset:
fix inplace modification error by adding clone()
Author: bzantium
Pull request: yes

# What does this PR do?
When using `DistributedDataParallel`, `token_type_embedding` may occur RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation:

<img width="1791" alt="image" src="https://user-images.githubusercontent.com/19511788/177119562-1c341b15-da7b-474e-a1dd-f5f68f843142.png">

This is because of inplace operation at `buffered_token_type_ids = self.embeddings.token_type_ids[:, :seq_length]` so I just added `.clone()` to solve it as follows.

from (original)
```python
if token_type_ids is None:
    if hasattr(self.embeddings, "token_type_ids"):
        buffered_token_type_ids = self.embeddings.token_type_ids[:, :seq_length]
        buffered_token_type_ids_expanded = buffered_token_type_ids.expand(batch_size, seq_length)
        token_type_ids = buffered_token_type_ids_expanded
```

to
```python
if token_type_ids is None:
    if hasattr(self.embeddings, "token_type_ids"):
        buffered_token_type_ids = self.embeddings.token_type_ids[:, :seq_length]
        buffered_token_type_ids_expanded = buffered_token_type_ids.expand(batch_size, seq_length)
        token_type_ids = buffered_token_type_ids_expanded.clone()
```

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@LysandreJik

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Do you have a reproducible code example that raises this error? cc @sgugger @patrickvonplaten 

@LysandreJik I found the problem is in DistributedDataParallel not transformers. I fixed the problem. Sorry to take your time!

prefect solution!
Allow Loading Custom Model Names by Searching for Extension
Author: VitorHugoOli
Pull request: no

### Feature request

I propose that in the `from_pretrained`, instead of relying on a set of specific filenames, the loading function should search for the model by its file extension. By doing so, the model names can be more flexible, allowing users to name their custom models as they see fit without the need to conform to a predefined naming scheme.

**Example:**
```txt
Directory: ./input/model/clip/vit-large-patch14
Files: config.json, preprocessor_config.json, vit-large-patch14.safetensors
```
In this case, the model could be loaded by searching for the .safetensors extension rather than expecting a specific filename like model.safetensors.

### Motivation

Currently, in the `from_pretrained` if a folder contains a custom model with a name that doesn't match one of the predefined specified names, it cannot be simply passed to the loading function. Instead, the file name of the model must be manually changed to match one of the allowed names. This behavior can be restrictive, especially when dealing with multiple custom models with varying names.

**Benefit:**
This change would enhance the usability of the library, making it more adaptable to various use-cases and workflows involving custom models. It would also streamline the process of loading custom models, eliminating the need to rename files to match a predetermined pattern.

### Your contribution

I will make changes to the files `utils/modeling_utils.py` and `utils/__init__.py` to enhance the way models are loaded.

1. In utils/__init__.py: I will define a set of predefined extensions that the loading function can recognize. This list of extensions will be used to identify the model files within a given directory.

2. In utils/modeling_utils.py: I will modify the function _add_variant to accept a variant parameter, which can either be the name of the file or a directive to search for the first file with one of the predefined extensions.

Comments:
In general `from_pretrained` is only intended to work with models saved with `save_pretrained`, not with other models, so this is not a feature we plan to add.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Remove `utils/documentation_tests.txt`
Author: ydshieh
Pull request: yes

# What does this PR do?

This file is already removed in #25680 (we decided to use `no_doctested.txt` to explicitly exclude some files - all others should be doctested). 

However, #24085 added it back (which was a mistake of not having on top of the latest main at that time). This PR just removes this file again.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix ConversationalPipeline tests
Author: Rocketknight1
Pull request: yes

This PR fixes the failing conversation pipeline tests in the CI. The causes were:

1) `BlenderBotSmall` was missing a `default_chat_template`
2) The old `Conversation` object distinguished between processed and unprocessed user inputs. The new object doesn't! This caused a couple of tests that were expecting the old behaviour to fail. `ConversationalPipeline` has already been updated to handle the new behaviour, so only a few test values needed to be changed.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
refactor: add min to block size
Author: pphuc25
Pull request: yes

Hi,
Referring to #26069 and incorporating the recommendation of @sanchit-gandhi, I am submitting this pull request (PR) to include a minimum value constraint on the block_size parameter. This change aims to prevent potential errors from occurring.
I would like to cc @sanchit-gandhi to review my PR, and thank you so much for your suggestion.

Comments:
This is just a mistake when I make a pull request, the main pull request of this is in #26069 

Reviewed directly on the PR linked!
[FSMT] Fix non-shared weights
Author: LysandreJik
Pull request: yes

FSMT shares weights across three layers.

The `tie_weights` method wasn't having the intended effect on all three (only on the input and output embeddings), leading to errors when converting the model to safetensors.

See https://huggingface.co/facebook/wmt19-en-de/discussions/7

Comments:
_The documentation is not available anymore as the PR was closed or merged._
allow easier user customizable `decay_parameters` in trainer
Author: shijie-wu
Pull request: no
Labels: name: Feature request

### Feature request

allow easier user customizable `decay_parameters` in trainer

### Motivation

https://github.com/huggingface/transformers/blob/32ec7345f2d752c294ddf5aff495b657c9cd9d3b/src/transformers/trainer.py#L964-L965

Currently to modify `decay_parameters` for weight decay, user need to inherit and implement the entire `create_optimizer`. By refactor this two line into a function, user can modify `decay_parameters` much easier.

### Your contribution

if this sounds good, i am happy to submit a PR

Comments:
cc @muellerzr if you think this should be done? 

This *feels* like it's okay to me, but I'd need to see what the full implementation will look like @shijie-wu if you want to take a stab at it :)
refactor decay_parameters production into its own function
Author: shijie-wu
Pull request: yes

# What does this PR do?

Fixes #26145


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? https://github.com/huggingface/transformers/issues/26145
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@muellerzr @ArthurZucker

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26152). All of your documentation changes will be reflected on that endpoint.

Hey sorry for the late reply, I'll have look
NEW Model GPT-JX
Author: alignment-ai
Pull request: no
Labels: name: New model

### Model description

request to add a new model gpt-jx-3b to the transformer library as well as request to create a custom model class, class names an be found in the below .txt files as well as at the below of the description, weights are present in hugging face model hub but made public for now, ["alien-ai/gpt-jx-3b"](https://huggingface.co/alien-ai/gpt-jx-3b) --id_repo. Just a little summary of the model has been provided in the repo.Currently the repo only contains pytorch_model.bin file and no files other than that.

we provide you the code for modelling.py ,configuration.py and tokenization.py in .txt format
[modelling_gptjx.txt](https://github.com/huggingface/transformers/files/12428050/modelling_gptjx.txt)

[config.txt](https://github.com/huggingface/transformers/files/12428124/config.txt)

[tokenization.txt]
(https://github.com/huggingface/transformers/files/12428249/tokenization.txt)

please check the files before proceeding.

Make Sure to Set class names: GPTJXForCausalLM, GPTJXModel, GPTJXConfig, GPTJXTokenizer.

Sorry, we are too busy right now so, i was not able to write the description properly. Please Check it Out Yourself. Please create the new model class as soon as possible

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

Authors will be revealed later. repo-- ["alien-ai/gpt-jx-3b"](https://huggingface.co/alien-ai/gpt-jx-3b) 

Comments:
This model has been removed from the hub. It looks like an interesting model, why remove it? 

> This model has been removed from the hub. It looks like an interesting model, why remove it?

It has been withdrawn by us and will be released along with 7b and 15b models
refactor: change default block_size in block size > max position embeddings
Author: pphuc25
Pull request: yes

Hi,
In the original code, this appears to function correctly when the default block_size is set to 1024. However, I believe that this setting might potentially hinder the training performance. Therefore, I have adjusted the default to be max_position_embeddings when it doesn't match case.
I would like to cc @sanchit-gandhi to review my PR, thank you so much

Comments:
Thank you for your helpful information. 

No worries @pphuc25! Would you like to open a PR to set the block size to the minimum of `(1024, config.max_position_embeddings)`? This will prevent the error when we set the block size > max position embeddings

That's the cool idea, I will do it, thank you @sanchit-gandhi.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26069). All of your documentation changes will be reflected on that endpoint.
[Wav2Vec2-Conf / LLaMA] Style fix 
Author: sanchit-gandhi
Pull request: yes

# What does this PR do?

Small style fix to stay consistent with the rest of the modelling code: `torch.nn -> nn`


Comments:
_The documentation is not available anymore as the PR was closed or merged._

CI went red on `main`: the permisson model wasn't checked as part of the PR. Needed to rebase to include it: #26228
`AddedToken` 's argument are ignored when called in `add_tokens` 's method of slow tokenizers
Author: SaulLu
Pull request: no
Labels: name: Core: Tokenization

### System Info

- `transformers` version: 4.25.1
- Platform: Linux-5.10.133+-x86_64-with-glibc2.27
- Python version: 3.8.16
- Huggingface_hub version: 0.11.1
- PyTorch version (GPU?): 1.13.0+cu116 (False)
- Tensorflow version (GPU?): 2.9.2 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help?

@ArthurZucker 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

The explanations of the bug and its reproduction are contained in the following google colab:  https://colab.research.google.com/drive/19SS6Tzlgo0vntFtM6ZsCYq8BNZ5Dy1cS?usp=sharing

### Expected behavior

I would expect the fast and slow tokenizers to treat the `AddedToken`'s arguments in the same way.

I think the loss of information for the slow tokenizer occurs at this line:  https://github.com/huggingface/transformers/blob/a413c725d40027134f28f92974ad7d61751f5640/src/transformers/tokenization_utils.py#L411

Comments:
I have not dropped this, its still on my TODO list. There are a lot of linked issues!

(The update is gonna take longer as I am refactoring the tokenizers)

I had the same issue. It seems that `AddedToken` is implemented [here](https://github.com/huggingface/transformers/blob/e42587f596181396e1c4b63660abf0c736b10dae/src/transformers/tokenization_utils_base.py#L80), if the package `tokenizers` is not available. However, the python implementation using dataclasses does not behave the same way as the rust implementation in `tokenizers`.
Two tokenizer initialization methods result in inconsistent segmentation results for special words
Author: KinvenW
Pull request: no
Labels: name: Core: Tokenization

### System Info

transformers==4.17.0
torch==1.10.0
python==3.7.3

### Who can help?

@ArthurZucker

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction
```
# xlm-roberta-base directory: git clone https://huggingface.co/xlm-roberta-base
from transformers import XLMRobertaTokenizer
tokenizer_a = XLMRobertaTokenizer.from_pretrained('xlm-roberta-base/')
tokenizer_b = XLMRobertaTokenizer('xlm-roberta-base/sentencepiece.bpe.model')

t = 'texta
textb'
print(tokenizer_a.tokenize(t))
print(tokenizer_b.tokenize(t))
```

### Expected behavior

```
# what I expect is that both outputs: 
['▁text', 'a', '
', '▁text', 'b']
['▁text', 'a', '
', '▁text', 'b']

# However, in reality, their outputs are as follows:
['▁text', 'a', '
', '▁text', 'b']
['▁text', 'a', '<', 's', '>', 'text', 'b']
```


Why these two tokenizers have different segmentation results for special words?

Comments:
Hey! Which version of transformers are you using? I tried to reproduce this but it did not work. Download the sentencepiece model from  the official repo, and it worked on main~

> Hey! Which version of transformers are you using? I tried to reproduce this but it did not work. Download the sentencepiece model from the official repo, and it worked on main~

my transformers version is 4.17.0.  

Ok, I can confirm that this is not working for me either! Thanks for reporting. This is most probably a problem related to `add_tokens` in the init: 
```python 
>>> tokenizer_a.unique_no_split_tokens
['
', '<mask>', '<pad>', '
', '<unk>']
>>> tokenizer_b.unique_no_split_tokens
```


Linking this with #23909 as the core problem is similar: 
- Adding a special token does not recreate the Trie (when the special tokens are initialized)
- `from_pretrained`  calls `added_tokens = tokenizer.sanitize_special_tokens()`  which is when the special tokens are added to `unique_no_split`
- Initialising a model from a sentencepiece vocab file does not initialize the Trie (used to split the tokens), since the special tokens are not sanitized. 

I've had to take on a few model addition, the fix PR will be updated soon! It requires a bit more refactoring than initially thought! 

Once `transformers` is released, I'll merge the fix!
ByT5Tokenizer ignores spaces around added tokens
Author: djstrong
Pull request: no

### System Info

transformers 4.23.1

### Who can help?

@patrickvonplaten @SaulLu

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained('google/byt5-base')
tokenizer.add_tokens('<x>', special_tokens=True)
print(tokenizer('<x> <x> <x><x>'))
{'input_ids': [384, 384, 384, 384, 1], 'attention_mask': [1, 1, 1, 1, 1]}
```
in comparison to:
```python
print(tokenizer('a a aa'))
{'input_ids': [100, 35, 100, 35, 100, 100, 1], 'attention_mask': [1, 1, 1, 1, 1, 1, 1]}
```

### Expected behavior

In my task presence of spaces around added tokens are important. Despite that, I think byT5 tokenizer should not ignore any characters (bytes).

Comments:
May also be of interest to @ArthurZucker 

Also cc @Narsil - any ideas here? 

> Also cc @Narsil - any ideas here?

Yes, by default added tokens always use `lstrip/rstrip=True` which swallows prefix/suffix spaces (it's a convenience for <special> so you don't have to worry how to add in within some text.)
Since ByT5  is pure bytes, it doesn't have `tokenizers` support (doesn't make sense speedwise). and using the "slow" class (it's not slow though).


```python
tokenizer = AutoTokenizer.from_pretrained("google/byt5-base")
# tokenizer.add_tokens("<x>", special_tokens=True)
new_token = AddedToken("<x>", lstrip=False, rstrip=False)
tokenizer.add_tokens(new_token, special_tokens=True)
tokenizer._additional_special_tokens.append(new_token)
```

This change will fix it, however it require changing internals which is not great. Definitely  looks like a bug.

Pinging @ydshieh which was looking at this recently and trying to figure out some tokenizer stuff.

I "think" this qualifies as a bug. (Well the original shared code is not OK, the defaults are to strip left and right, but if you do `add_tokens(AddedToken(.., lstrip=False, rstrip=False))` then it should honor that. And the workaround I had to look at a few different internal variables to set it appropriately so that the `Trie` class could do it's job correctly (otherwise it just couldn't see the `AddedToken` values.

Sorry for being late here. So as @Narsil pointed out, 

```python
tokenizer = AutoTokenizer.from_pretrained("google/byt5-base")
new_token = AddedToken("<x>", lstrip=False, rstrip=False)
tokenizer.add_tokens(new_token, special_tokens=True)
```
should work (which is not the case for now) without the need of `tokenizer._additional_special_tokens.append(new_token)`.
And the goal is to make the above code snippet do it job correctly. Is this right?

Hey! I'll take this one on as part of #23909, since it is an issue with `rstrip` and `lstrip` being ignored (as the default behaviour if a token is not special is to always stip)

As mentioned, this will take a bit more time, a big refactoring is coming! 🔥 

Should be merged this week!
[Tokenizer] `skip_special_tokens` not working as expected 
Author: ArthurZucker
Pull request: no
Labels: name: Core: Tokenization

# Reporting a failing API design
This is mostly to help me record some of the biggest issues with the current API for adding tokens. 

This is linked to #23909. Here is a simple snippet: 

```python 
>>> from transformers import AutoTokenizer
>>> tokenizer = AutoTokenizer.from_pretrained("t5-base", use_fast = False)
>>> tokenizer = [
                    AddedToken("[ABC]", normalized=False), 
                    AddedToken("[DEF]", normalized=False),
                    AddedToken("GHI IHG", normalized=False),
]
>>> tokenizer.add_tokens(new_toks)
>>> tokenizer.add_tokens([AddedToken("[SAMPLE]", normalized=True)], special_tokens = True)
>>> print(tokenizer.added_tokens_encoder)
>>> print( tokenizer.all_special_ids)
```
This will show that the newly added token (`[SAMPLE]`) is not part of the `all_special_ids`. However,  `all_special_ids` is used when decoding, to check whether the token should be skipped or not: 
```python 
        for token in filtered_tokens:
            if skip_special_tokens and token in self.all_special_ids:
                continue
            if token in self.added_tokens_encoder:
                if current_sub_text:
                    sub_texts.append(self.convert_tokens_to_string(current_sub_text))
                    current_sub_text = []
                sub_texts.append(token)
            else:
                current_sub_text.append(token)
```

Thus 
```python
>>> encoded = tokenizer.encode("[ABC] [DEF][SAMPLE]", add_special_tokens=False)
>>> tokenizer.decode(encoded, skip_special_tokens = True)
"[ABC] [DEF][SAMPLE]"
```
However, the token is in `added_tokens_encoder` but not in `additional_special_tokens`. 

Now imagine you want `spaces_between_special_tokens` ? This will add spaces between all added tokens, and thus checks if a token is part of `tokenzier.added_tokens_encoder`. 

```python
>>> encoded = tokenizer.encode("[ABC] [DEF][SAMPLE]", add_special_tokens=False)
>>> tokenizer.decode(encoded, spaces_between_special_tokens = True)
"[ABC] [DEF] [SAMPLE]"
>>> tokenizer.decode(encoded, spaces_between_special_tokens = False)
"[ABC][DEF][SAMPLE]"
```


Comments:
Okay! the fix PR should be merged soon. Note that the behaviour is gonna be deprecated!