noamaanMulla-03/transformers-issues huggingface dataset:
Allow passing kwargs through to TFBertTokenizer
Author: Rocketknight1
Pull request: yes

There are some kwargs like `preserve_unused_tokens` in the underlying TF tokenizer layers that might be useful to expose to users. This PR exposes them by passing through any unrecognized `kwargs` in the model `__init__` to the TF tokenizer layer.

Fixes #23798 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Ping @amyeroberts  for core maintainer review now that the extra functionality is working fine (see issue #23798)

I was a bit wary about the kwargs thing too - `FastBertTokenizer` and `BertTokenizerLayer` actually have wildly different arguments, so depending on which one you're using the kwargs you need will be totally different. Still, I think for an advanced use case it's fine - we're just trying to enable some power user behaviours without forcing them to edit the library source, and I'd prefer something general like this over specifically exposing the options I think people need (because I didn't even realize in advance that the `preserve_unused` arg would be valuable!)

Anyway, merging!
Colab Translation notebook link not found
Author: SoyGema
Pull request: no

### System Info

Hello There!
First and foremost, congrats for Transformers Translation [tutorial](https://huggingface.co/docs/transformers/tasks/translation). 👍
It serves as a Spark for building english-to-many translation languages models!
I´m following it along with TF mostly reproducing it in a jupyter Notebook with TF for mac with GPU enabled

At the end of the [Train](https://huggingface.co/docs/transformers/tasks/translation) section , it is showed

_For a more in-depth example of how to finetune a model for translation, take a look at the corresponding PyTorch notebook or [TensorFlow notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/translation-tf.ipynb)._

Inside the notebook, at cell [4] , there shows a message

_**You** can find a script version of this notebook to fine-tune your model in a distributed fashion using multiple GPUs or TPUs [here](https://github.com/huggingface/transformers/tree/master/examples/seq2seq)._

The link is broken .


## Potential fix.
Maybe it could point to Transformer [performance docs](https://huggingface.co/docs/transformers/performance) if you want to go for a more general overview or some specific part of [run_translation.py](https://github.com/huggingface/transformers/blob/main/examples/tensorflow/translation/run_translation.py) script facilitated by team member [here](https://github.com/huggingface/transformers/issues/24254#issuecomment-1594830054) during #24254 help? Please , don't hesitate to share the link as there could be a benefit in implementing it

Thanks so much for the time dedicated to this 
Keep up the amazing work in the Open!

### Who can help?

@Rocket

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Follow tutorial in docs . Go to Notebook at the end of Train Section 
2. Go to Tensorflow Notebook
3. Click link in cell [4] . It seems to go to /seq2seq examples

### Expected behavior

The link should point at a fine-tune script version of the notebook, or at least to docs

Comments:
cc @Rocketknight1 

I opened a PR to the notebooks repo here to fix this: https://github.com/huggingface/notebooks/pull/398

Thanks for warning us about the issue - we appreciate the help to keep our docs up to date!
Tied weights load
Author: sgugger
Pull request: yes

# What does this PR do?

This continue cleaning up a bit the model loading by:
1. Using the new `_tied_weight_keys` class variable when deleting weights without warning for safetensors serialization
2. Fix the logic that deletes tied params in missing keys and add a test (which fails on main)
3. As discussed internally, use a logger.info for the unexepected keys warning when the class used to load the model does not match the class in the config.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[Whisper] Make tests faster
Author: sanchit-gandhi
Pull request: yes

# What does this PR do?

Reduces the input seq length of the Whisper tests from 1500 -> 60 frames. This in turn should speed up the tests quite considerably.


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24105). All of your documentation changes will be reflected on that endpoint.

Sorry, I am asking not because I see a test being slow, but just I saw some more Whisper test failures on daily CI, which is `tests/models/whisper/test_modeling_whisper.py::WhisperModelTest::test_cpu_offload`.

But yes, in general, it's best to use low number. I will take a look.

Note that the Whisper tests have already been flagged as being slow (#23736) so this should help combat this issue!

It's not because it's slow test that we use large value without really valid reason :-). Always better to make them use low values is the goal, unless it's absolute necessary.

I still have questions on why we don't need to pass `input_shape` in the corresponding flax test file.

OK, in flax test file, I see

```
        self.all_model_classes = (
            make_partial_class(model_class, input_shape=self.init_shape) for model_class in self.all_model_classes
        )
```
probably it's the reason.

Yep agreed - the seq len was unnecessarily high here :) You're spot on regarding the init shape: we have to change this based on the sequence length since Flax Whisper initialises the positional embeddings based on the context window, so if we change the seq len (= context window) we need to init the weights with the new shape
Add a check in `ImageToTextPipeline._forward`
Author: ydshieh
Pull request: yes

# What does this PR do?

Inside `ImageToTextPipeline.preprocess`, we have 
```python
        if self.model.config.model_type == "git" and prompt is None:
            model_inputs["input_ids"] = None
```
So it may  happen with a list of `None` (for Git model), and the `_forward` fail.

This PR add a check and change the above case to a single `None` value to avoid failure.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[Tokenizer doc] Clarification about `add_prefix_space`
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Adresses #17391,  updates the documentation that suggested to use `add_prefix_space` when calling the tokenizer

Comments:
_The documentation is not available anymore as the PR was closed or merged._

This blog post on constrained decoding also uses `add_prefix_space` in `__call__` https://huggingface.co/blog/constrained-beam-search


The blog is not hosted on `transformers` but on `blog`, will open a PR for that too later on, thanks for the catch 😉
Better test name and enable pipeline test for `pix2struct`
Author: ydshieh
Pull request: yes

# What does this PR do?

In #24364, `pix2struct` test file didn't get `pipeline_model_mapping` being updated, due to the `heuristic` of finding a test class not working well (try to find the shortest test class name).

Let's try to give the test class a better/short/clear name, (despite we don't really have a base model), `Pix2StructModelTest` instead of `Pix2StructTextImageModelTest`.

This enables the script `add_pipeline_model_mapping_to_test.py` works for `pix2struct`, and then we get the pipeline test being run.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Skip a tapas (tokenization) test in past CI
Author: ydshieh
Pull request: yes

# What does this PR do?

Same as in #24251 where 1 test (from the tokenization test file) is missed.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24378). All of your documentation changes will be reflected on that endpoint.
GPTNeoXAttention does not deal with odd numbers of attention heads
Author: peter-sk
Pull request: no

### System Info

transformers 4.29, HEAD
Linux (not relevant, reproducible on e.g. Mac OS)
python 3.10.11 (not relevant, reproducible on e.g. python 3.9.13)

### Who can help?

@ArthurZucker
@younesbelkada 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

When the hidden size is not evenly divisible by the number of attention heads, GPTNeoXAttention throws an exception while trying to reshape its state.

Here is a minimal example to reproduce:

```
from transformers import pipeline
p=pipeline(model="Isotonic/gpt_neox_225M",task="text-generation")
p("I like to eat ")
```

The exception is the following:

```
/home/jps/anaconda3/envs/transformers/lib/python3.10/site-packages/transformers/models/gpt_neox/modeling_gpt_neox.py:133 in forward                                                              â
RuntimeError: shape '[1, 5, 12, 255]' is invalid for input of size 15360
```

### Expected behavior

What happens here is that the model has 12 attention heads and a hidden size of 1024. Thus, head_size is calculated as 1024 // 12 == 85. Then, 3 * head-size is not 256 but 255, giving the issue here.

I am not quite sure how this is supposed to work. In https://github.com/EleutherAI/gpt-neox, the code checks that the hidden size is evenly divisible by the number of heads. This would not enable the use of this model, but it might give a better error message.

Is there any chance to run such a model?

Comments:
Hey! Thanks for reporting!Feel free to open a PR that raises and error when the hidden size is not divisible by the number of heads. This indeed should not happen.  (Can probably be checked in the config)
[Whisper Docs] Nits
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Adresses #24342, where it is mentioned that the documentation is conter intuitive. Indeed, after a lot of changes, the default value for the `bos_token` that we use is different, thus no official models (hosted on the hub) use `bos_token = "<startoftranscript>"`

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[Wav2Vec2 - MMS] Correct directly loading adapters weights
Author: patrickvonplaten
Pull request: yes

# What does this PR do?

This PR corrects incorrect behavior when loading MMS with non-default adapter weights via `from_pretrained(...)`. The issue is explained well [here](https://github.com/huggingface/transformers/issues/24223#issuecomment-1595856093).

In a nutshell, we cannot load specific weights in the init because these loaded weights are later overwritten again in `from_pretrained`. To solve this I propose to add a new generic
```py
load_adaptive_weights()
```

call to `from_pretrained` that can be overridden by models that inherit from `PretrainedModel`. This both solves the issue #24223 
and is also cleaner IMO since weights shouldn't be loaded when calling the `__init__` method of a model anyways really. It was weird before that:

```py
model = Wav2Vec2ForCTC(config, target_lang="fra")
```
would try to load weights into the model.

cc @sgugger @sanchit-gandhi @amyeroberts wdyt about the design? Happy to add some more tests if ok for you

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Sorry, I accidentally submitted the review without a saved comment. I realised in the `from_pretrained` call why you were using `pass`. I still think raising an exception would be good, as otherwise we can get silent behaviour. Would it be possible to reliably check if `load_adaptive_weights` should be implemented for a model? 

p.s. ignoring the wandb diffs, as they're just from being out-of-date from main
fixing layer indexing error when pipeline parallel > 1
Author: xshaun
Pull request: yes

when applying pipeline parallel, the index of layers converted from transformers to megatron is wrong since there is not offset applied successfully. 
For example, if having 4 layers, and pipeline parallel is 2, we want to the result looks like `layers.0 + layers.1` and `layers.2 + layers.3`, but now the result is `layers.0 + layers.1` and `layers.0 + layers.1`, since user should use `pp_layer_id` that is calculated with `layer + offset` , instead of `layer` that is only a index of range loop.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24381). All of your documentation changes will be reflected on that endpoint.
Code formatting issue with Korean translation of quicktour.mdx
Author: GarrettDaniel
Pull request: no

### System Info

N/A

### Reproduction

In the Korean translation of quicktour.mdx found [here](https://github.com/huggingface/transformers/blob/main/docs/source/ko/quicktour.mdx), there is a small formatting issue in the bash commands to install pytorch and tensorflow.
![image](https://github.com/huggingface/transformers/assets/36463300/3f750ef3-ba13-4977-b8e5-52ec7d8d396e)

### Expected behavior

- The install commands should be rendered as code blocks in the markdown file.
- The formatting can be fixed by adding a newline/return between the two commands like this:
```bash pip install torch```
```bash pip install tensorflow```

Furthermore, these commands can be simplified to one line by using the following syntax, which will install both PyTorch and Tensorflow:
```!pip install torch tensorflow```

I'd be happy to make these changes and help with some of the other Korean documentation.

Comments:
Hey! Thanks for reporting, feel free to open a PR and ping me 😉 

It looks like this issue was resolved on the docs website https://huggingface.co/docs/transformers/v4.30.0/ko/quicktour.  Closing this issue
How BERT 512 limit works?
Author: Bisht9887
Pull request: no

### System Info

I passed a long text of 3000 tokens and it did not give me any error. Does BERT not have 512 token limit? Why it is not giving any error? This is the code I used. You can pass any input with tokens more than 512

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

        from transformers import BertTokenizer
        tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
        
        # Long input text with more than 512 tokens
        text = "This is a very long text with more than 512 tokens..." 
        
        tokens = tokenizer.tokenize(text) 
        print(len(tokens))


### Expected behavior

error when processing more than 512

Comments:
Please use the [forums](https://discuss.huggingface.co/) for such questions. You did not pass your input the model, just the tokenizer.

sure! I will take care of that. I am little new to this. Thanks for info
Migrate doc files to Markdown.
Author: sgugger
Pull request: yes

# What does this PR do?

The new UI in GitHub makes MDX pretty hard to read for diffs, so this PR migrates the doc files from mdx to md. This shouldn't break anything in the doc-builder.

Comments:
I'd add a :warning: emoji as a prefix for the disclaimer, but other than that it looks good to me!

_The documentation is not available anymore as the PR was closed or merged._
Update deprecated torch.ger
Author: kit1980
Pull request: yes

` torch.ger` was deprecated long time ago and `torch.outer` is a direct replacement: https://pytorch.org/docs/stable/generated/torch.ger.html

Comments:
@sgugger please take a look.

_The documentation is not available anymore as the PR was closed or merged._

A test (I believe unrelated to the change) timed out https://app.circleci.com/pipelines/github/huggingface/transformers/66846/workflows/ae63fb76-7260-490e-8301-7c6cf986e693/jobs/833163

Yes it's unrelated, merging :-)
Add `ffmpeg` for `doc_test_job` on CircleCI
Author: ydshieh
Pull request: yes

# What does this PR do?

Need this at least for `docs/source/en/task_summary.md`.

Otherwise, [job](https://app.circleci.com/pipelines/github/huggingface/transformers/66845/workflows/ae6bcd25-5071-4f48-a9ba-d446ae6e060f/jobs/833148) fails

Comments:
_The documentation is not available anymore as the PR was closed or merged._
byebye Hub connection timeout - Recast
Author: ydshieh
Pull request: yes

# What does this PR do?

It's a bit hard to break up with timeout failure, but @Wauplin was working on it to have 60 seconds instead

https://github.com/huggingface/huggingface_hub/pull/1523

We need to change the commit hash to that one in our CircleCI job though.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Incorrect typing of `debug` field in `TrainingArguments`
Author: augray
Pull request: no

### System Info

https://github.com/huggingface/transformers/blob/v4.29.2/src/transformers/training_args.py#L862



### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

If you use `TrainingArguments` and try to pass in a list of strings to `debug`, type checkers will complain.

### Expected behavior

The `debug` field should be typed as `Union[str, List[str]]` so type checkers don't complain.

For my particular use case, I am using a library that serializes dataclasses based on their type annotations, and it fails to serialize if `debug` is `[]` since the type hint says it is `str`. But the `__post_init__` turns `debug=""` into `debug=[]`, so there's no way I can get instances of this class to serialize without passing in non-empty debug options. I would gladly submit a PR myself, but I am not sure if it would have implications for the command line parsers.

Comments:
The `debug` field does not accept a list of string, but a list of `DebugOption`, so the type annotation should be more like `Union[str, List[DebugOption]]`.

Happy to have a look at a PR if you want to fix it.
fix type annotation for debug arg
Author: Bearnardd
Pull request: yes

# What does this PR do?

Fix type annotation for `debug` argument in `training_args.py`

Fixes https://github.com/huggingface/transformers/issues/23958

## Who can review?



Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hi @amyeroberts, I have made a fix to the code. I'm not certain if there is a more concise way to address this, but I added an additional check for `self.debug` being `None`. The reason for this is that when using `Union[str, List[DebugOption]]`, even if `default=""` is specified, it is still evaluated as `None`.

@Bearnardd Thanks for updating. Could you share a snippet to reproduce showing the evaluation of `debug` as `None` if left as default? 

If I create the training arguments directly, when working from `main`, debug defaults to `[]` with the type changes e.g. 

```python
In [1]: from transformers import TrainingArguments

In [2]: args = TrainingArguments("dummy_dir")

In [3]: args.debug
Out[3]: []
```

So it might be an environment or how it's being used in a script thing? 


 

Hi @amyeroberts! It was one od the failing test cases. I will be back at home tomorrow so I will check that to confirm :)

Hi @amyeroberts! I have done some quick debugging. I was able to obtain the same results as you while running your snippet. However the problem appears when you try to run things from CLI. One of the test cases that were failing is `test_run_seq2seq_no_dist` from `transformers/tests/extended/test_trainer_ext.py` which uses command line arguments. In a result of running this test case there is a chain of internal function calls as follows: 

```
parser = HfArgumentParser((ModelArguments, DataTrainingArguments, Seq2SeqTrainingArguments))
model_args, data_args, training_args = parser.parse_args_into_dataclasses()
```

`parse_args_into_dataclasses` underneath calls `self.parse_known_args(args=args)` which is a method derived from `ArgumentParser`.

```
namespace, remaining_args = self.parse_known_args(args=args) # hf_argparser.py:339
```
 In the `argparse` itself there is a default action `--debug` which is initialize as `None`. And here is a trick: if `debug` argument is of type str then argparse is able to internally cast it into empty string however it leaves if as `None` if it is of type `Union[str, List[DebugOption]`. Thats why this test fails if we change type annotation of `debug` argument.


Is this explanation understandable for you or do you need some additional context/information :) ?

@Bearnardd Thanks for such a detailed investigation and write up! In this case, resolving this with the `--debug` flag in `argparse` would be very involved and this `None` check works well :)
Some links in NLLB page are broken.
Author: ranggihwang
Pull request: no

**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

**Describe the solution you'd like**
A clear and concise description of what you want to happen.

**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.

**Additional context**
Add any other context or screenshots about the feature request here.


Hi, this is just a simple notification for crashed links.

https://huggingface.co/docs/transformers/v4.30.0/model_doc/nllb-moe
On this page, two links in the "documentation resources" are broken.

Thank you.


Comments:
cc @stevhliu 

Doc in version 4.30.0 still has problems, not only the doc section but other links :( @stevhliu 

The fix is on main, so it will only be reflected on the main version of the documentation. And if you have found other links to fix, please do tell us or open directly PRs to fix them :-)