noamaanMulla-03/transformers-issues huggingface dataset:
Adds LILT to models exportable with ONNX
Author: mariababich
Pull request: yes

# What does this PR do?

Adds LILT to models exportable with ONNX

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
Hi @mariababich , the ONNX export is now supported in Optimum, I merged your PR there: https://github.com/huggingface/optimum/pull/1098
deepcopy added in pipeline, breaks anything that worked before that used Rlock etc. like streaming generation
Author: pseudotensor
Pull request: no

### System Info

transformers==4.29.2
python 3.10


### Who can help?

@gante @Narsil 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Pass streamer instance of TextIteratorStreamer(), which includes thread, to TextGenerationPipeline as generator kwargs.
2. See pickle error due to new copy.deepcopy() added.

Due to this change by @gante:

https://github.com/huggingface/transformers/commit/b369e507aaa78103baf5d3f3563952b44a0408a1

This is a fully blocking change for me.  I cannot upgrade to new transformers since this change because this breaks streaming scenario.

### Expected behavior

I don't think copy.deepcopy() is appropriate.  The dictionary itself is passed in as **generate_kwargs and any mutation to the dictionary itself has no effect on the parent dictionary or items passed in.

Additionally, the modifications made to the dictionary in that changed code only involve entries *within* the dictionary, not to mutable items inside the dictionary, so none of those changes would have any effect to any other block of code.

A simple shallow copy is sufficient.

But additionally, I can't see any reason for any copy at all.  Changes to the dictionary locally have no effect anywhere else.

Comments:
@pseudotensor you closed this without a comment. Is this issue still relevant ?

Github trying to be too smart with separate issue in another repo

@pseudotensor 👋 

TIL "The dictionary itself is passed in as **generate_kwargs and any mutation to the dictionary itself has no effect on the parent dictionary or items passed in."

In that case you're right, no copy is needed at all!

@pseudotensor should be fixed now (closing the issue, but feel free to reopen if you find further related issues)
Update `(TF)SamModelIntegrationTest`
Author: ydshieh
Pull request: yes

# What does this PR do?

- Add `require_tf`: otherwise, past CI's torch job  (with only torch installed) will fail for this test class
- Add `TF` prefix: as usual convention

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Good catch, and sorry! I'll tell GPT-4 to watch that one in future, lol
Safely import pytest in testing_utils.py
Author: amyeroberts
Pull request: yes

# What does this PR do?

After merging in #23271, hitting `TAB` for autocompleting: `from transformers.` in an ipython session results in a runtime error. 

It appears that this is because `_pytest` and `pytest` are imported in `testing_utils.py`. Although there's no direct imports of `transformers.testing_utils.py`, it seems this module read when using autocomplete. 

Fixes #24227 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add (M)Luke model training for Token Classification in the examples
Author: jplu
Pull request: yes

# What does this PR do?

This PR adds the possibility to train the (M)Luke model for a Token Classification task with the accelerate package. It also adds a tiny update to give the possibility to train over multiple datasets configuration, for example being able to concatenate multiple languages of the XTREME PAN-X dataset and train the model over it.

One can easily test with the command:
```
accelerate launch run_ner_no_trainer.py --model_name_or_path studio-ousia/mluke-base --dataset_name xtreme --dataset_config_name PAN-X.fr,PAN-X.en --output_dir /data/mluke/ --task_name ner --return_entity_level_metrics
```

/cc @sgugger @LysandreJik 


Comments:
Hi there, thanks a lot for you PR! It will be great to be able to fully use LUKE and mLUKE for token classification!

Now the issue is that we try to keep each example pretty simple so that users can easily tweak and customize them. Adding this in the `run_ner_no_trainer` example adds a lot of complexity so how about we make this into its own example instead? It could go into a research project of its own.
Same for the data collator. It's very specific to LUKE so it should be added in a module file in the same folder as the example instead of adding this to the main lib directly, I think.

Sure! I will rethink how to properly reorder the things and will let you know once pushed!

Awesome, thanks a lot for adding this! Also cc'ing the original authors, @ikuyamada @Ryou0634.

I completely agree with Sylvain here, adding it to the existing `run_ner.py` script would make it confusing for people that want to leverage BERT/RoBERTa/etc. 

@sgugger I moved everything into a dedicated folder that is only focus on Luke.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Hi, let's finish this PR and merge it!

@jplu are you able to rebase with master?

Hey @jplu, thanks for your PR! 

I'd just move it to `examples/research_projects` to have a specific LUKE example, vs having a `luke` folder in the `examples` folder :)

Sorry for the late reply. I will update accordingly to what you asked ASAP.

Ok, done on my side!

Done! Let me know if something is missing.
Add MacOS TF version
Author: jplu
Pull request: yes

# What does this PR do?

This PR adds the MacOS TensorFlow version mostly for the M1 Apple laptop that is the recommended version to use.

Comments:
As far as I have been tested until now, yes looks working quite well! I would even say that the work done by the Apple team on it is very impressive!!

I will push new PRs if I encounter new issues with it :)

Great, thanks a lot @jplu :)
Fix squad processor for TF
Author: jplu
Pull request: yes

# What does this PR do?

This PR fixes the Squad processor that prepares and creates a `tf.data.dataset` to be able to be used in the `TFTrainer` through the `run_tf_squad.py` example script.

There were two issues:

1. The `token_type_ids` was forced to be `True` in the tokenizer output even if this argument was not part of the tokenizer's property `model_input_names`. Now it always belongs to the created dataset.
2. The `input_processing` method that parses the inputs of a TF model doesn't allow arguments that are not part of the signature, then the extra features have been removed from the created dataset.

# Fixes issue

#10246 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Fix check copies
Author: jplu
Pull request: yes

# What does this PR do?

The target `make quality` fails when generating the new model table when at least one of the optional packages TF, PT or Flax is not installed. We should not force to have everything installed to do a simple quality check, this can be added to an extra target such as `make full-quality` or something like this.

The fix does a condition checking and replace the raised error by a simple warning.

Comments:
Replacing the error by a warning defeats the purpose of the check as it will make the CI pass when it should fail. We can see if we want to move it in another command, I'm just afraid it will make the failures in the CI (and what the corresponding fixes are) less understandable to the user.

Let me think about this over the weekend, I'll try to find a solution by Monday :-)

Awesome! I let you handle this, so I'm closing the PR.
Tied params cleanup
Author: sgugger
Pull request: yes

# What does this PR do?

This PR is the first in a series that aims to clean up the content of the variables `_keys_to_ignore_xxx` and the logic of warnings sent to the user resulting from them. In particular unless the model implement a hack like RoBERTa, a trained version with no shared version might not save the decoder...

One use of these variables is to detect "normal" shared weights and have a way to remove all but one of those shared weights for safetensors serialization, but the variable used (`_keys_to_ignore_unexepected`) is very noisy (it contains names of weights that are not shared). This PR introduces a variable `_tied_weights_keys` which will be used for that purpose (and in other places of the cleanup later on). It contains the named of the weights that are tied to other weights and that are safe to dismiss when saving (we won't dismiss them on the torch.save side since `torch.save` won't duplicate memory but will use it for `safetensors`). This PR also introduces a test that checks the content of this variable is correct.

Due to the very large number of models having potential shared weights, this PR limits itself at the introduction of this variable, proper fill for all existing models and test. The rest will follow in subsequent PRs.

cc @Narsil for info

Comments:
_The documentation is not available anymore as the PR was closed or merged._
TF: standardize `test_model_common_attributes` for language models
Author: gante
Pull request: yes

# What does this PR do?

`test_model_common_attributes` was overridden in most TF LMs because:
1. it was not able to handle legacy classes with LM heads (there was not a single set of autoclasses that would catch them)
2. many modern decoder-only LMs do not have a bias in the LM head

This PR adapts the test to account for these 2 cases, and removes a large number of overridden tests.


Comments:
_The documentation is not available anymore as the PR was closed or merged._

@amyeroberts now with deprecation messages (as opposed to deletions) and pytest asserts 🤗 
@Rocketknight1 ping 🙌
Generate: GenerationConfig can overwrite attributes at from_pretrained time
Author: gante
Pull request: yes

# What does this PR do?

Fixes #24104

As with other configuration files, we should allow overwriting attributes at `from_pretrained` time. The latest change to the `GenerationConfig` loading code disallowed it -- this PR fixes it and adds a test.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add `torch >=1.12` requirement for `Tapas`
Author: ydshieh
Pull request: yes

# What does this PR do?

Tapas files are changed in #20149 to use torch's `scatter`. The torch tensor's method `scatter_reduce` accept the argument `src` only for torch >= 1.12.

This PR add some warnings/requirements in tapas modeling/test files to avoid test failures in past CI with torch <= 1.11.

(one previous similar PR is #19851)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix how we detect the TF package
Author: Rocketknight1
Pull request: yes

Our framework detection code calls `_is_package_available()` for TensorFlow, but this code fails when only the `tensorflow-cpu` package is present. The failure occurs because `importlib_metadata.version("tensorflow")` throws an error in the version detection branch of `_is_package_available` unless the core `tensorflow` package is installed.

I solved this by just calling `importlib.util.find_spec("tensorflow")` instead of `_is_package_available()`. However, we could also resolve this issue by rewriting `_is_package_available()` so that it only takes the version check branch when `return_version` is `True`. The `importlib_metadata.version()` call is only used to get the package version, but it causes the entire `_is_package_available()` call to fail if it can't find a version, even if the `importlib.util.find_spec()` call was a success.

ccing @sgugger because there's a `TODO` above that function [requesting his attention](https://github.com/huggingface/transformers/blob/main/src/transformers/utils/import_utils.py#L40), so I'd like his input on the right approach here!

Fixes #24253 

Comments:
cc @apbard as well since I think the issue was introduced in #23163, but thankfully it's a relatively easy fix!

_The documentation is not available anymore as the PR was closed or merged._
Skip `GPT-J` fx tests for torch < 1.12
Author: ydshieh
Pull request: yes

# What does this PR do?

After #22069, the fx tests for gpt-j with torch < 1.12 gives an error
```bash
(line 941)  AssertionError: Couldn't trace module: 'len' is not supported in symbolic tracing by default. If you want this call to be recorded, please call torch.fx.wrap('len') at module scope
```
Guess it's best to skip in this case, as it works with recent torch versions.



Comments:
_The documentation is not available anymore as the PR was closed or merged._
add use_orig_params to fsdp with torch compile
Author: ouhenio
Pull request: yes

# What does this PR do?

The FSDP wrapper inside `trainer.py` needs to be initialized with `use_orig_params=True` for FSDP + `torch.compile` to work well together. Therefore, I added some code to check inside the relevant FSDP section if `torch_compile` is set to `True` to then add `use_orig_params` with the corresponding value to FSDP.

Fixes #23341

## Before submitting

- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@sgugger
@pacman100

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_23481). All of your documentation changes will be reflected on that endpoint.

Hello, as the Accelerate now Powers Trainer, please use the accelerate launcher with Trainer for FSDP. That provides support for `use_orig_params` and hence this PR is no longer required.

Please do pip install git+https://github.com/huggingface/transformers and pip install git+https://github.com/huggingface/accelerate

Use Accelerate launcher with Trainer. More info here: [Using Accelerate Launcher with Trainer](https://huggingface.co/docs/transformers/main/en/main_classes/trainer#using-accelerate-launcher-with-trainer). 

Therefore, closing this PR. Thank you for all the effort!
getting AssertionError when using Trainer with `fsdp` and `torch_compile`
Author: ouhenio
Pull request: no

### System Info

While trying to train a `GPT2` model using the `Trainer` with `torch_compile` and `fsdp` flags I get the following error:

```bash
AssertionError: Dynamo only supports FSDP with use_orig_params=True
```

I'm using `python==3.10.9`, and initially used `transformers==4.27.X` but after stumbling upon #22279 I updated to `transformers==4.28.1` but the problem persisted.

### Who can help?

I'm guessing this is @sgugger territory and maybe @ani300 could help too. 😊 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

My training file looks like:

```python
def train(cfg: DictConfig):

    # Setup data and dataloader
    data_module = prepare_data_module(**cfg.data)

    train_dataloader = data_module.train_dataloader()
    val_dataloader = data_module.val_dataloader()

    # Extract tokenizer from datamodule
    tokenizer = data_module.tokenizer

    # Setup model and optimizer
    model = GPT(**cfg.model)

    # Setup data collator
    data_collator = DataCollatorForLanguageModeling(tokenizer, mlm=False)

    # Setup and running training
    train_args = TrainingArguments(**cfg.train_args)

    trainer = Trainer(
        model=model,
        tokenizer=tokenizer,
        train_args=args,
        data_collator=data_collator,
        train_dataset=train_dataloader,
        eval_dataset=val_dataloader,
    )

    trainer.train()
```

Here's my full `TrainingArguments` configuration in `.yaml` format:

```yaml
per_device_train_batch_size: 8
per_device_eval_batch_size: 8
evaluation_strategy: "steps"
eval_steps: 2000
logging_steps: 5000
gradient_accumulation_steps: 8
num_train_epochs: 300
weight_decay: 0.1
warmup_steps: 1_000
lr_scheduler_type: "cosine"
learning_rate: 5e-4
save_steps: 25000
bf16: True
torch_compile: True
tf32: True
fsdp: "full_shard auto_wrap"
fsdp_transformer_layer_cls_to_wrap: 'GPT2Block'
```

And I'm running the training using `torchrun --nproc_per_node=8 train.py` with 8 NVIDIA A40.

### Expected behavior

It should run the training process without problems.

Comments:
meet same issue....

cc @pacman100 

The FSDP wrapper inside `trainer.py` needs to be initialized with `use_orig_params=True` for FSDP + compile to work well together. As of now, that is not the case and there is no flag in the Trainer to make it do so. I can probably find some time later in the week to make a PR, but in any case, that's the issue.

Thanks @ani300. I will attempt doing a PR. :blush: 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Flax LMHeadModel for common models like Bert and Albert
Author: gianlucadetommaso
Pull request: no

### Feature request

A LM Head Model for common models such as Bert and Albert is available in both PyTorch and TensorFlow, but it appears to be missing in Flax. 

### Motivation

We are developing a library in JAX and Flax for uncertainty quantification, and we rely on Hugging Face transformers written in Flax.

### Your contribution

I would love to contribute. However, unfortunately I might not have much time to submit a PR myself.

Comments:
Alright, I found out that LMHeadModel is an old naming choice that was kept not to break compatibility. In Flax, I should be able to do what I want with `FlaxBertForCausalLM`.
QA doc: import torch before it is used
Author: ByronHsu
Pull request: yes

# What does this PR do?

import torch before it is used in qa doc. Otherwise, it complains import error.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@stevhliu


Comments:
_The documentation is not available anymore as the PR was closed or merged._

@amyeroberts @stevhliu mind checking again? thanks!

@ByronHsu Thanks again for fixing! All looks good 👍