noamaanMulla-03/transformers-issues huggingface dataset:
support ONNX export of XDropout in deberta{,_v2} and sew_d
Author: garymm
Pull request: yes

# What does this PR do?

Enables `torch.onnx.export` of the `StableDropout` module in training mode.
In training mode, the `XDropout` torch.autograd.Function is included. This change
adds a `symbolic` function to `XDropout` which produces an ONNX graph that
is equivalent to the `forward` function.


## Before submitting

- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
No. LMK if you want me to open an issue.

- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).

No, LMK if there's some doc that I should update.

- [x] Did you write any new necessary tests?

## Who can review?

@LysandreJik


Comments:
_The documentation is not available anymore as the PR was closed or merged._

@michaelbenayoun thanks for taking a look. I added a test.
Unfortunately I couldn't use the existing testing style because none of the affected models have an ONNX config (that's tracked by https://github.com/huggingface/transformers/issues/16308).

@michaelbenayoun bump, PTAL

@michaelbenayoun bump

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

@garymm any update on this? 

@michaelbenayoun @lewtun do you have any updates on this PR?

Thank you for your contribution
Can't load tokenizer for longt5-xl
Author: whiteRa2bit
Pull request: no
Labels: name: bug

### System Info

transformers version: 4.20.0
Platform: Linux-4.15.0-135-generic
Python version: 3.8.13
PyTorch version (GPU?): torch==1.10.2+cu113
Using GPU in script?: no
Using distributed or parallel set-up in script?: no


### Who can help?

@patrickvonplate


### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I'm using the example provided on this page: https://huggingface.co/google/long-t5-tglobal-xl:
```

from transformers import AutoTokenizer, LongT5Model

tokenizer = AutoTokenizer.from_pretrained("google/long-t5-tglobal-xl")
```



### Expected behavior

A tokenizer for a longt5-xl works

Comments:
I have the same issue: When I try to load the model with its tokenizer I get the following error message:
```
OSError: Can't load tokenizer for 'google/long-t5-tglobal-xl'.
If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name.
Otherwise, make sure 'google/long-t5-tglobal-xl' is the correct path to a directory containing all relevant files for a T5TokenizerFast tokenizer.
```

Indeed, it seems the tokenizer files were not uploaded to that repository. Pinging @stancld, could you mention which tokenizer files should be used here? I'm happy to add these to `google`'s repositories. 

@LysandreJik AFAIK, the `LongT5` models use the same tokenizer as the `T5` model. I'd, therefore, just copy the `tokenizer.json` config e.g. from `long-t5-tglobal-large` to the XL repo, and it should work as expected.

Sounds good, I'll take care of that. Thanks!

Should work now, this was the only repository that needed to be updated (was lacking the tokenizer files). Feel free to close this issue if your problem is solved!
fix ONNX support for bloom
Author: nmntz
Pull request: yes

Merged on https://github.com/huggingface/transformers/pull/18344

This PR aims to fix ONNX export of bloom. All the following tests are passing:
```
RUN_SLOW=1 pytest tests/onnx/test_onnx_v2.py -k "bloom"
RUN_SLOW=1 pytest tests/models/bloom/test_modeling_bloom.py
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._
generate with tf.function (xla) not working for tf model export
Author: nlpcat
Pull request: no
Labels: name: bug

### System Info

using the latest version with tf.function for generate still doesn't work for tensorflow model export which would be useful for serving.

### Who can help?

@gante @patrickvonplaten 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
import tensorflow as tf
from transformers import TFAutoModelForSeq2SeqLM

class MyOwnModel(tf.Module):
    def __init__(self, model_path="t5-small"):
        super(MyOwnModel, self).__init__()
        self.model = TFAutoModelForSeq2SeqLM.from_pretrained(model_path)

    @tf.function(input_signature=(tf.TensorSpec((None, 32), tf.int32, name="input_ids"),
                                  tf.TensorSpec((None, 32), tf.int32, name="attention_mask")), jit_compile=True)
    def serving(self, input_ids, attention_mask):
        return self.model.generate(input_ids=input_ids, attention_mask=attention_mask, max_new_tokens=32)

model = MyOwnModel()
export_dir = "./"
tf.saved_model.save(
    model,
    export_dir,
    signatures={
        "serving_default":
            model.serving
    })
```

error:
```
File "../python3.8/site-packages/transformers/generation_tf_utils.py", line 1561, in _generate  *
        input_ids = self._prepare_decoder_input_ids_for_generation(
    File "../python3.8/site-packages/transformers/generation_tf_utils.py", line 1758, in _prepare_decoder_input_ids_for_generation  *
        return tf.ones((batch_size, 1), dtype=tf.int32) * decoder_start_token_id

    TypeError: Expected int32, but got None of type 'NoneType'.
```

### Expected behavior

generate can be exported in tf model for serving

Comments:
HFTracer.trace can now take callables and torch.nn.Module
Author: michaelbenayoun
Pull request: yes

# What does this PR do?

This PR enables to use the `HFTracer` "meta-tracing" features to trace any Python callable / `torch.nn.Module`.

For `transformers.PreTrainedModel`s, the method `HFTracer._generate_dummy_inputs` already takes care of creating the original dummy inputs needed to handle data-dependent control-flow in the forward pass. 

Now, the user can specify `dummy_inputs` directly to the `HFTracer.trace` method in order to be able to trace other things than `transformers.PreTrainedModel`s. This is useful for pattern matching for instance. 

This becomes possible:

```python
def f(x, y, z=None):
    temp =  x * y
    if z is not None:
        temp += z
    return temp

traced_f = HFTracer().trace(f, dummy_inputs={"x": torch.rand(1, 2), "y": torch.rand(1, 2)})
```

By default, if `dummy_inputs` is specified, every argument to `root` that is not in `dummy_inputs` will be considered a concrete arg (and thus added to `concrete_args`). You can disable that by setting `infer_concrete_args_from_dummy_inputs` to `False`. This is useful if want to provide custom dummy inputs for some inputs, while still keeping the `HFTracer._generate_dummy_inputs`  doing the work for other inputs (provided that `root` is a `transformers.PreTrainedModel` since only this case is supported for automatic dummy inputs generation).

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix load of model checkpoints in the Trainer
Author: sgugger
Pull request: yes

# What does this PR do?

#18221 broke the model reload when the contributor removed the `strict_load` variable (as requested in the review) without setting it to its proper value in the calls to `load_state_dict` after. This PR addresses that.

Fixes #18373

Comments:
_The documentation is not available anymore as the PR was closed or merged._
CodeGen Fix causal mask for half precision
Author: younesbelkada
Pull request: yes

# What does this PR do? 

This PR forces the causal mask to stay in `torch.uint8`. An error occurs when loading a model in half precision since `torch_dtype=torch.float16` casts also the buffers in fp16. Here is a minimal script to reproduce the error:
```
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Salesforce/codegen-2B-mono")
model = AutoModelForCausalLM.from_pretrained("Salesforce/codegen-2B-mono", device_map="auto", torch_dtype=torch.float16)

text = "def quicksort(l):"

encoded_input = tokenizer(text, return_tensors='pt')
output_sequences = model.generate(input_ids=encoded_input['input_ids'], attention_mask=encoded_input['attention_mask'])
print(tokenizer.decode(output_sequences[0], skip_special_tokens=True))
```

In a future PR we could address non-casting the buffers (aka keeping them in their native `dtype`)

Can also confirm the slow tests pass!

cc @ydshieh 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Yeah let's move the discussion to: https://github.com/huggingface/transformers/pull/18471
Add FX support for torch.baddbmm  and torch.Tensor.baddbmm
Author: thomasw21
Pull request: yes

This doesn't quite fix the issue where the model isn't torch scriptable on torch 1.10, but this will help fx support on torch 1.10 at the very least.

I'll merge this after #18344 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Replicating RoBERTa-base GLUE results
Author: markblee
Pull request: no

Hello! I had originally posted this on the [forums](https://discuss.huggingface.co/t/replicating-roberta-base-glue-results/19328) but it seems like there's not much foot traffic there, so hoping to get more visibility here.

I'm trying to replicate RoBERTa-base GLUE results as reported in the [model card](https://huggingface.co/roberta-base#evaluation-results). The numbers in the model card look like they were copied from the paper. Has anyone made an attempt to actually match these numbers with `run_glue.py`? If so, what configuration was used for the trainer? 

If I follow the original configs from [fairseq](https://github.com/facebookresearch/fairseq/tree/fcca32258c8e8bcc9f9890bf4714fa2f96b6b3e1/examples/roberta/config/finetuning), I am unable to match the reported numbers for RTE, CoLA, STS-B, and MRPC.

Any pointers would be much appreciated, thanks!

Comments:
Another option would be to open a discussion in the community tab in https://huggingface.co/roberta-base/discussions and tag the model authors there

Thanks for the suggestion, that's a neat feature! I opened a discussion [here](https://huggingface.co/roberta-base/discussions/1) (although, it's not quite clear how to discover the model authors by handle).

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Attention gradients for models
Author: Rachneet
Pull request: no

### Feature request

Hi, I was wondering if there is a way to get the gradients of the attention weights. In PyTorch, we can do this via hooks and it works perfectly for getting embeddings gradients. But, I had an issue doing this for the transformer attention weights. Is there any way we can make this possible?

### Motivation

This can help with model interpretability with the scaled attention method.

### Your contribution

I can attach my current codebase if you guys would be interested. It works partly but sometimes the gradients are zeroed. I am not sure if this is the correct behavior though.

Comments:
Hi @Rachneet ,

Model interpretability is indeed interesting and useful!

However, I think currently we don't have a plan to integrate the mechanism of getting gradients into `transformers`. There is an library [Captum](https://captum.ai/) which might be useful in this area though.

cc @LysandreJik 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Add VideoMAE
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR adds [VideoMAE](https://github.com/MCG-NJU/VideoMAE), which extends [ViTMAE](https://huggingface.co/docs/transformers/model_doc/vit_mae) to videos. 

The only difference between VideoMAE and ViT is that you need to replace `nn.Conv2d` by `nn.Conv3d` in the patch embedding class. 😂 

To do:

- [ ] Decide on a name for `VideoMAEFeatureExtractor` (should we keep it, or rename to `VideoMAEProcessor`, `VideoMAEPreprocessor`?)
- [ ] Decide on the input format for video models; currently I've chosen `pixel_values` of shape (batch_size, num_frames, num_channels, height, width). The original implementation uses (B, C, T, H, W)
- [ ] Doc examples + tests
- [x] Incorporate changes of #17731 
- [ ] Make VideoMAEFeatureExtractor robust with return_tensors="np" by default, better tests

Comments:
@NielsRogge do you have any ETA on this feature? I am developing a video classification fine-tuning framework, would love to use this model if it gets merged into main!

Currently only video model is PerceiverIO, right?

There seems to remain an issue with the docs:

```
Traceback (most recent call last):
  File "/usr/local/bin/doc-builder", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/site-packages/doc_builder/commands/doc_builder_cli.py", line 47, in main
    args.func(args)
  File "/usr/local/lib/python3.8/site-packages/doc_builder/commands/build.py", line 96, in build_command
    build_doc(
  File "/usr/local/lib/python3.8/site-packages/doc_builder/build_doc.py", line 405, in build_doc
    sphinx_refs = check_toc_integrity(doc_folder, output_dir)
  File "/usr/local/lib/python3.8/site-packages/doc_builder/build_doc.py", line 460, in check_toc_integrity
    raise RuntimeError(
RuntimeError: The following files are not present in the table of contents:
- model_doc/videomae
Add them to ../transformers/docs/source/en/_toctree.yml.
```

@LysandreJik yes I was aware of that, should be fixed now.

Don't merge already please, I'm transferring checkpoints and updating the conversion script.

_The documentation is not available anymore as the PR was closed or merged._
Add `TF_MODEL_FOR_SEMANTIC_SEGMENTATION_MAPPING`
Author: ydshieh
Pull request: yes

# What does this PR do?

The original goal is to fix `TFSegformerModelTest.test_keras_fit`, but it ends up the following

- Add `TF_MODEL_FOR_SEMANTIC_SEGMENTATION_MAPPING` to some `__init__` files.
- Add `training` arguments in a few layers for `TFSegformerModel`
- Update `_prepare_for_class` to deal with 2 more image tasks
- Fix `TFData2VecVisionForSemanticSegmentation` loss: we need batch dimension (without this, `test_dataset_conversion` failed - this was previously skipped due to the lack of labels)



Comments:
_The documentation is not available anymore as the PR was closed or merged._

Test failures are `ValueError: Connection error` - irrelevant.

Thank you, @ydshieh for this. I appreciate the help.
How can I use DDP mode without `torch.distributed.launch` command when I use transformers.Trainer
Author: EeyoreLee
Pull request: no

I checked the historical issues and readed a part of source code. But I didn't find the way out to use DDP without `torch.distributed.launch`. Is there a way to use deepspeed or ddp without modifying the command? Like a field named `strategy: str` to select the way weather I use DDP.

Comments:
cc @sgugger

No, you need to launch your Python script with the torch/DeepSpeed launcher.

> 

Thank you. I got it.
Remove py.typed
Author: sgugger
Pull request: yes

The experiment was fun (not).

As we are getting a rise in issues of users complaining typecheckers are not happy with our annotations, which we have chosen to keep simple for the sake of documentation, this PR removes the `py.typed` file indicating to type-checkers that Transformers is properly typed.

It is not, and it won't be in the near future because static type-checking things in Python requires sacrificing too much in term of clarity of code (at least in my opinion). This PR just makes it "official".

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix pipeline tests
Author: sgugger
Pull request: yes

# What does this PR do?

Changes in #18392 broke tests in the pipelines, this PR fixes them.

Comments:
> Why weren't these ran on the original PR ?

Cause the common test files used not only have the tester for each pipeline, not tests of its own, so is not triggered by the `tests_fetcher`. Will fix in this PR as well.

_The documentation is not available anymore as the PR was closed or merged._
Fix typo in serialization.mdx
Author: eltociear
Pull request: yes


# What does this PR do?
overriden -> overridden





## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_17914). All of your documentation changes will be reflected on that endpoint.

Hey @eltociear! The code quality error comes from a new release from `black`. Rebasing on `main` should solve the issue as you'll benefit from https://github.com/huggingface/transformers/pull/17918.

Let us know if we can help!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Will data be loaded into multiple GPUs automatically?
Author: Leli1024
Pull request: no

Will models that use TensorFlow as a backend be loaded into all available GPU memory or not?

I'm interested in training the OPT model's 1.3b and 30b variants. However there is no cloud computing GPU i can use which has more than 16gb at a time. The problem is that each individual sample in the batch is at least 24gb, when making use of tensorflow, will the memory be distributed among GPUs or will I still get an allocation error?

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Maybe of interest for @Rocketknight1 @gante

Hi @Leli1024 👋 By default, TensorFlow will allocate memory in all available GPUs, but will only execute in one GPU -- meaning that what you want to do won't happen by default. At the moment, we have no multiple GPU examples (@Rocketknight1 correct me if I'm wrong), so you will have to build a custom solution for yourself.

Here is a TensorFlow guide for it: https://www.tensorflow.org/guide/distributed_training

We're co-ordinating with the TensorFlow team to make some examples of exactly this process available using the new `DTensor` API introduced in TensorFlow 2.9. I suspect it'll be another month or so before they're available, though!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.